On this page
Identify the file version and metadata boundary
Record a neutral artifact ID, digest, file length, detected ORC version, reader version, and the byte ranges or offsets the reader reports for the postscript, footer, metadata, and stripes. The Apache ORC, (opens in a new tab) page describes more than one released file version and says a library detects the version before choosing a reader. Do not infer the version from a filename or from a reader default.
The source dates in this guide use 22 October 2026 as retrieval evidence for documentation without a verified publication date. They are not publication claims. Where a verified publication date is available, it is recorded in the source ledger.
| Boundary field | Record | Stop if |
|---|---|---|
| Format version | Detected version and detection method | Reader version is the only evidence |
| File length | Observed length and declared footer span | Footer range exceeds file length |
| Postscript | Present, decoded, and linked to footer | Reader guesses a missing length |
| Footer | Schema and stripe references | Footer is read from a repaired copy only |
| Metadata | Stripe and row index references | Metadata is silently skipped |
| Compression | Codec and block boundary evidence | Bytes are interpreted without decompression context |
Keep the raw tail or a bounded structural digest with the normalized report. If a reader repairs a footer or ignores an unknown field, record that as reader behavior. A parsed footer does not prove every row is present or every column value is semantically valid.
Compare schema identity before reading statistics
Record each column path or field ID, declared type, nested structure, nullability policy if available, and the schema reference used by the reader. ORC files are written with a schema that defines the column tree. Keep the source schema and any projected query schema separate. A projection can omit a column without proving that the source did not contain it.
| Schema evidence | Safe observation | Open question |
|---|---|---|
| Column ID | Stable identifier used by metadata | Did a transform remap IDs? |
| Column path | Nested path and parent relationship | Does a flat label hide a nested field? |
| Type | Declared primitive or complex type | Were values coerced during export? |
| Row count | File or stripe count with source field | Is the count from all stripes? |
| Null policy | Null count or not recorded | Are missing and null distinct? |
| Projection | Requested columns and omitted columns | Could pruning hide needed evidence? |
Use a synthetic schema with symbolic fields to test your worksheet. Do not copy real column names, values, or operational labels into a public packet. Record schema drift as a versioned change, not as a cleanup of the latest file.
Keep stripe statistics tied to their stripe and column
The Apache ORC, (opens in a new tab) documentation describes file, stripe, and row levels. For every statistic, record its level, stripe identity, column identity, count, null presence, minimum and maximum when supplied, numeric sum when supplied, and the method or reader that produced it. A file-level minimum cannot answer which stripe contains a value. A stripe-level maximum does not prove that every row is within the range if the statistic is missing or stale.
| Statistic evidence | Decision it can support | Cannot prove |
|---|---|---|
| File-level min or max | A bounded file candidate for a predicate | Which row or stripe owns the value |
| Stripe-level min or max | A bounded stripe candidate | Complete row coverage |
| Value count | Count recorded at the index level | Distinct count or data quality |
| Null presence | A null state was observed | Why a value is null |
| Numeric sum | A recorded aggregate in the statistic | Independent recomputation |
| Bloom filter | Possible membership pruning | No false positives or semantic match |
Synthetic stripe ledger
{"file":"fixture-orc-01","version":"fixture-v1","schema":[{"id":"fixture-1","path":"events.count","type":"int"}],"stripes":[{"id":"stripe-a","rows":"fixture-count","stats":{"count":"fixture-count","has_null":"fixture-boolean","min":"fixture-min","max":"fixture-max"},"decision":"candidate-only"}]}Separate row indexes, seek positions, and pruning decisions
Apache ORC describes row indexes as statistics for sets of rows within a stripe plus a position for seeking to the row group. Record the row group boundary, column, statistics, seek position, and whether the reader used it. A pruning decision is conditional on the predicate, the exact statistic, null behavior, and reader implementation. It is not a proof that an omitted stripe or row group contains no relevant data.
| Review question | Record | Hold when |
|---|---|---|
| Predicate | Exact normalized predicate and type | A string comparison is applied to a numeric field |
| Index level | File, stripe, or row group | The report does not identify level |
| Seek position | Reader position and row group | Position is missing or inconsistent |
| Null handling | How nulls affect the predicate | Nulls are treated as ordinary values |
| Bloom filter | Present, absent, or unavailable | Filter version or column is unknown |
| Read result | Rows scanned and why | A pruned result is treated as complete |
Stop before relying on an incomplete footer
Stop conditions
- Stop if the format version, footer span, compression boundary, or reader version is unknown.
- Stop if schema column IDs or nested paths cannot be matched to statistics.
- Stop if file, stripe, and row index levels are mixed in one summary.
- Stop if a missing statistic is treated as an empty range or a null count is treated as data quality.
- Stop if a bloom filter or pruning result is used without recording its column, predicate, and reader behavior.
- Stop before sampling real rows when the footer or column labels disclose private operational data.
Pair this worksheet with the data provenance log for writer and reader history and the data card template for collection context. Footer evidence can bound a read; it cannot prove completeness, ownership, privacy clearance, or permission.
For a supported non-health collection, after the owner has authority to describe the scope and a controlled metadata packet, the qualified appraisal request path may be a preparation step. It does not read ORC stripes, run a query, certify pruning, create a quote, match a buyer, or grant a license.