Data Provenance

Corpus makes the path from source material to API response explicit. For SEC-derived data, Corpus collects directly from EDGAR and, as each accession is processed, archives source submissions and serves normalized records with filing accessions and acceptance timestamps. For market, document, news, and court datasets outside EDGAR, Corpus identifies the public or licensed source and preserves the strongest source identity available.

Primary source is a specific claim. Corpus uses it for data collected directly from SEC EDGAR. Stooq, Accord Fintech, publisher RSS feeds, curated annual-report libraries, and AWS Open Data distributions are identified as third-party, licensed, or public-source collections instead.

Why provenance matters

Research and production systems need to explain an input after the fact. A price, line item, filing section, or judgment should be traceable to a named source, stable identifier, observation date, and the time it became available to the system. Corpus separates those concerns:

Source identity tells you where the record originated.
Event or report time tells you when the underlying event occurred or period ended.
Knowledge time tells you when Corpus could have known the value.
Parser and normalization versions identify the transformation applied to source material.
Availability state distinguishes a known document from one whose bytes or derived text are ready to serve.

United States sources

Dataset	Origin	How Corpus handles it
Companies and filing metadata	SEC EDGAR submissions and XBRL APIs, ticker, exchange, and quarterly index files	Preserves CIK, ticker where available, accession number, form, report date, filing date, acceptance time, registrants, and SEC URLs.
Filing documents and items	SEC EDGAR Archives	For processed accessions, stores replayable complete submissions and available components by content hash, extracts normalized text and supported filing items, and records source offsets and parser versions.
Financial statements and fact projection	SEC Companyfacts and filing XBRL	Maps supported concepts into versioned income, balance-sheet, and cash-flow fields and exposes an ingested concept projection for unmapped fields. The serving projection is not a lossless context-and-unit copy of EDGAR facts and may coalesce same-day duplicates.
Insider transactions	SEC Form 4 and 4/A ownership XML	Parses transaction rows with the filing accession, issuer or reporting-owner identity, transaction date, and EDGAR acceptance time.
Institutional holdings	SEC Form 13F-HR information tables	Parses reported long holdings with manager CIK, report period, accession, and acceptance time. Form 13F is not a complete ownership or short-position dataset.
Corporate events	SEC 8-K, 8-K/A, 6-K, and 6-K/A filings	Extracts supported filing items with accession-level evidence and source offsets.
Daily US prices	Stooq bulk data plus a frozen 2017 Stooq-derived delisted-symbol supplement	Produces a curated split-adjusted historical EOD panel. This is a third-party research source, not an official exchange feed, and it is not a complete survivorship-bias-free US universe.

SEC JSON endpoints are collected from data.sec.gov; filing bytes and accession components come from sec.gov/Archives. Corpus follows the SEC fair-access policy and archives immutable source bytes before derived records are considered complete.

India sources

Dataset	Origin	How Corpus handles it
Securities	ISIN-keyed licensed market files, enriched with the public NSE equity master and BSE scrip master	Reconciles ISIN, NSE symbol, and BSE code without fabricating a mapping when sources disagree or omit it.
Historical prices	Licensed Accord Fintech NSE and BSE historical OHLCV files	Normalizes daily OHLCV as supplied by the source files and keeps observation and knowledge dates separate. This is a historical source, not an intraday or real-time feed.
Annual reports	Corpus-held curated BSE/NSE Annual Reports Library	Uses the collection manifest and filename metadata, stores PDFs as immutable artifacts, extracts text, and reports PDF, text, OCR, and scan availability per artifact. This is not represented as direct issuer or exchange ingestion.
Financial news and disclosures	Allow-listed publisher, regulator, NSE, and BSE RSS or Atom feeds	Stores the fields syndicated by each publisher: headline, canonical link, publish time, category, and available summary. Corpus does not fetch or resell full article bodies for this dataset.
High Court judgments	eCourts-derived Indian High Court Judgments on AWS Open Data, plus DAKSH-curated public case data	Loads court and case metadata and serves a PDF or extracted text only when the corresponding artifact is available.
District-court cases	Development Data Lab’s judicial dataset, an ODbL-licensed, anonymized eCourts panel	Serves structured case metadata for the loaded panel. It does not contain party names and is not a complete national district-court archive.

China sources

Dataset	Origin	How Corpus handles it
Securities and identifiers	Baostock securities master, covering the Shanghai and Shenzhen exchanges	Keys identity on the exchange-qualified code and refuses to resolve a bare six-digit code. Retains delisted instruments with their listing and delisting dates, and models the listed window as point-in-time validity so an as-of read before listing or after delisting returns no identity.
Daily China prices	Baostock free A-share interface, redistributing Shanghai and Shenzhen exchange data	Collects unadjusted (不复权) daily OHLCV, with adjustment factors kept separate, so a later re-adjustment becomes a new knowledge-time revision instead of rewriting history. Records halted sessions explicitly and discards the stale forward-filled prices the source returns for them. Takes the trading calendar from the source rather than the PRC public-holiday schedule, because designated make-up Saturdays are generally not exchange trading days. This is a third-party redistributor, not an official exchange feed.

Raw Baostock responses are content-addressed into immutable storage with a manifest before any curated record is derived, so the China panel is re-derivable from raw plus code without refetching from the upstream service. China price data is classified scraped_prohibited — Baostock conveys no exchange redistribution licence — which makes it internal-only and excluded from every resale surface. The authoritative feeds are licensed to non-Mainland entities through China Investment Information Services for Shanghai and Shenzhen Securities Information Co. for Shenzhen; reclassification requires a signed licence.

From source to response

Register the source - transport, legal basis, trust level, cadence, and expected record contract are declared before ingestion.
Archive the artifact - replayable source bytes are stored with a content hash when the source supplies a document or submission.
Parse and normalize - source-specific parsers produce typed records; parser and taxonomy versions identify the transformation.
Record time correctly - report, event, publication, filing, and knowledge timestamps are kept distinct where the source provides them.
Serve through a governed contract - jurisdiction and capability scopes, processing state, and rights decisions determine what a key may read or export.

How to audit a response

Depending on the resource, look for:

cik, accession_number, filing_type, filing_datetime, url, and registrants for SEC filings
content_hash, parser_version, and source_offsets for filing content and items
normalization_version in financial-statement metadata
isin, nse_symbol, bse_code, and artifact_id for Indian instruments and annual reports
source_id, url, and published_at for news
judgment_id, cnr, court, decision_date, has_pdf, and has_text for judgments
as_of and record-level knowledge_time for point-in-time analysis

Not every source publishes every field. Corpus returns missing authoritative values as null and availability as an explicit boolean or processing status.

Provenance guarantees and limits

Direct EDGAR lineage - SEC-derived records retain an accession-level path to the source filing.
Immutable replay - document pipelines use content-addressed artifacts so a parser result can be reproduced and hash-checked.
Filing amendments stay visible - amended filings retain separate accession metadata. The serving financial-fact projection is not a lossless accession-and-context ledger and may coalesce same-day concept duplicates.
No-look-ahead reads - as_of and knowledge_time support reconstructing the information set available at a historical cutoff.
Named non-SEC sources - Corpus does not describe third-party market or document collections as direct exchange or regulator feeds.
Rights-aware access - an endpoint in the contract does not grant a user redistribution rights. Access and export permissions remain subject to the provisioned key and source rights.
Processing is not hidden - metadata availability does not imply that a PDF, normalized text, filing section, or normalized statement has completed processing.

For dataset scope and dated coverage observations, see Coverage.

Get started

Shared API

Account and credits

Guides

Data Provenance

Why provenance matters

United States sources

India sources

China sources

From source to response

How to audit a response

Provenance guarantees and limits

​Why provenance matters

​United States sources

​India sources

​China sources

​From source to response

​How to audit a response

​Provenance guarantees and limits

Why provenance matters

United States sources

India sources

China sources

From source to response

How to audit a response

Provenance guarantees and limits