> ## Documentation Index > Fetch the complete documentation index at: https://docs.onera.app/llms.txt > Use this file to discover all available pages before exploring further. # Data Provenance > Understand where Corpus data originates and how to trace an API record back to its source. Corpus makes the path from source material to API response explicit. For SEC-derived data, Corpus collects directly from EDGAR and, as each accession is processed, archives source submissions and serves normalized records with filing accessions and acceptance timestamps. For market, document, news, and court datasets outside EDGAR, Corpus identifies the public or licensed source and preserves the strongest source identity available. **Primary source** is a specific claim. Corpus uses it for data collected directly from SEC EDGAR. Stooq, Accord Fintech, publisher RSS feeds, curated annual-report libraries, and AWS Open Data distributions are identified as third-party, licensed, or public-source collections instead. ## Why provenance matters Research and production systems need to explain an input after the fact. A price, line item, filing section, or judgment should be traceable to a named source, stable identifier, observation date, and the time it became available to the system. Corpus separates those concerns: * **Source identity** tells you where the record originated. * **Event or report time** tells you when the underlying event occurred or period ended. * **Knowledge time** tells you when Corpus could have known the value. * **Parser and normalization versions** identify the transformation applied to source material. * **Availability state** distinguishes a known document from one whose bytes or derived text are ready to serve. ## United States sources | Dataset | Origin | How Corpus handles it | | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Companies and filing metadata | [SEC EDGAR submissions and XBRL APIs](https://www.sec.gov/search-filings/edgar-application-programming-interfaces), ticker, exchange, and quarterly index files | Preserves CIK, ticker where available, accession number, form, report date, filing date, acceptance time, registrants, and SEC URLs. | | Filing documents and items | [SEC EDGAR Archives](https://www.sec.gov/Archives/edgar/data/) | For processed accessions, stores replayable complete submissions and available components by content hash, extracts normalized text and supported filing items, and records source offsets and parser versions. | | Financial statements and fact projection | SEC Companyfacts and filing XBRL | Maps supported concepts into versioned income, balance-sheet, and cash-flow fields and exposes an ingested concept projection for unmapped fields. The serving projection is not a lossless context-and-unit copy of EDGAR facts and may coalesce same-day duplicates. | | Insider transactions | SEC Form 4 and 4/A ownership XML | Parses transaction rows with the filing accession, issuer or reporting-owner identity, transaction date, and EDGAR acceptance time. | | Institutional holdings | SEC Form 13F-HR information tables | Parses reported long holdings with manager CIK, report period, accession, and acceptance time. Form 13F is not a complete ownership or short-position dataset. | | Corporate events | SEC 8-K, 8-K/A, 6-K, and 6-K/A filings | Extracts supported filing items with accession-level evidence and source offsets. | | Daily US prices | [Stooq bulk data](https://stooq.com/db/h/) plus a frozen 2017 Stooq-derived delisted-symbol supplement | Produces a curated split-adjusted historical EOD panel. This is a third-party research source, not an official exchange feed, and it is not a complete survivorship-bias-free US universe. | SEC JSON endpoints are collected from `data.sec.gov`; filing bytes and accession components come from `sec.gov/Archives`. Corpus follows the SEC fair-access policy and archives immutable source bytes before derived records are considered complete. ## India sources | Dataset | Origin | How Corpus handles it | | ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Securities | ISIN-keyed licensed market files, enriched with the public [NSE equity master](https://www.nseindia.com/static/market-data/securities-available-for-trading) and BSE scrip master | Reconciles ISIN, NSE symbol, and BSE code without fabricating a mapping when sources disagree or omit it. | | Historical prices | Licensed Accord Fintech NSE and BSE historical OHLCV files | Normalizes daily OHLCV as supplied by the source files and keeps observation and knowledge dates separate. This is a historical source, not an intraday or real-time feed. | | Annual reports | Corpus-held curated BSE/NSE Annual Reports Library | Uses the collection manifest and filename metadata, stores PDFs as immutable artifacts, extracts text, and reports PDF, text, OCR, and scan availability per artifact. This is not represented as direct issuer or exchange ingestion. | | Financial news and disclosures | Allow-listed publisher, regulator, NSE, and BSE RSS or Atom feeds | Stores the fields syndicated by each publisher: headline, canonical link, publish time, category, and available summary. Corpus does not fetch or resell full article bodies for this dataset. | | High Court judgments | eCourts-derived [Indian High Court Judgments on AWS Open Data](https://registry.opendata.aws/indian-high-court-judgments/), plus DAKSH-curated public case data | Loads court and case metadata and serves a PDF or extracted text only when the corresponding artifact is available. | | District-court cases | [Development Data Lab's judicial dataset](https://www.devdatalab.org/judicial-data), an ODbL-licensed, anonymized eCourts panel | Serves structured case metadata for the loaded panel. It does not contain party names and is not a complete national district-court archive. | ## China sources | Dataset | Origin | How Corpus handles it | | -------------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Securities and identifiers | [Baostock](http://www.baostock.com/) securities master, covering the Shanghai and Shenzhen exchanges | Keys identity on the exchange-qualified code and refuses to resolve a bare six-digit code. Retains delisted instruments with their listing and delisting dates, and models the listed window as point-in-time validity so an as-of read before listing or after delisting returns no identity. | | Daily China prices | [Baostock](http://www.baostock.com/) free A-share interface, redistributing Shanghai and Shenzhen exchange data | Collects **unadjusted** (不复权) daily OHLCV, with adjustment factors kept separate, so a later re-adjustment becomes a new knowledge-time revision instead of rewriting history. Records halted sessions explicitly and discards the stale forward-filled prices the source returns for them. Takes the trading calendar from the source rather than the PRC public-holiday schedule, because designated make-up Saturdays are generally not exchange trading days. This is a third-party redistributor, not an official exchange feed. | Raw Baostock responses are content-addressed into immutable storage with a manifest before any curated record is derived, so the China panel is re-derivable from raw plus code without refetching from the upstream service. China price data is classified `scraped_prohibited` — Baostock conveys no exchange redistribution licence — which makes it internal-only and excluded from every resale surface. The authoritative feeds are licensed to non-Mainland entities through China Investment Information Services for Shanghai and Shenzhen Securities Information Co. for Shenzhen; reclassification requires a signed licence. ## From source to response 1. **Register the source** - transport, legal basis, trust level, cadence, and expected record contract are declared before ingestion. 2. **Archive the artifact** - replayable source bytes are stored with a content hash when the source supplies a document or submission. 3. **Parse and normalize** - source-specific parsers produce typed records; parser and taxonomy versions identify the transformation. 4. **Record time correctly** - report, event, publication, filing, and knowledge timestamps are kept distinct where the source provides them. 5. **Serve through a governed contract** - jurisdiction and capability scopes, processing state, and rights decisions determine what a key may read or export. ## How to audit a response Depending on the resource, look for: * `cik`, `accession_number`, `filing_type`, `filing_datetime`, `url`, and `registrants` for SEC filings * `content_hash`, `parser_version`, and `source_offsets` for filing content and items * `normalization_version` in financial-statement metadata * `isin`, `nse_symbol`, `bse_code`, and `artifact_id` for Indian instruments and annual reports * `source_id`, `url`, and `published_at` for news * `judgment_id`, `cnr`, `court`, `decision_date`, `has_pdf`, and `has_text` for judgments * `as_of` and record-level `knowledge_time` for point-in-time analysis Not every source publishes every field. Corpus returns missing authoritative values as `null` and availability as an explicit boolean or processing status. ## Provenance guarantees and limits * **Direct EDGAR lineage** - SEC-derived records retain an accession-level path to the source filing. * **Immutable replay** - document pipelines use content-addressed artifacts so a parser result can be reproduced and hash-checked. * **Filing amendments stay visible** - amended filings retain separate accession metadata. The serving financial-fact projection is not a lossless accession-and-context ledger and may coalesce same-day concept duplicates. * **No-look-ahead reads** - `as_of` and `knowledge_time` support reconstructing the information set available at a historical cutoff. * **Named non-SEC sources** - Corpus does not describe third-party market or document collections as direct exchange or regulator feeds. * **Rights-aware access** - an endpoint in the contract does not grant a user redistribution rights. Access and export permissions remain subject to the provisioned key and source rights. * **Processing is not hidden** - metadata availability does not imply that a PDF, normalized text, filing section, or normalized statement has completed processing. For dataset scope and dated coverage observations, see [Coverage](/coverage).