Skip to content

Resource review guide

Compressed vs Expanded Size Review: Record Resource Limits Before Reading an Archive

Separate container bytes, declared member sizes, decoded bytes, and observed consumption before deciding whether a compressed export fits a controlled review budget.

A small compressed object can represent a much larger expanded stream, while a declared size can be missing, truncated, modulo-limited, or inconsistent with what a reader observes. Record each size with its source and unit, set limits before expansion, and classify truncation or limit stops as evidence states rather than silently treating them as complete files. This guide is a resource worksheet, not an unpacker, capacity guarantee, or security decision.

Updated

On this page

Name every size and its evidence source

The source dates shown in this guide identify documentation checks on 22 October 2026, not original publication dates. Follow each citation for the container behavior being discussed. Start with a controlled artifact reference and record units explicitly. At minimum, distinguish bytes received, bytes stored in the wrapper, bytes declared for a member, bytes emitted after decoding, bytes actually inspected, and bytes retained in a derivative.

The Python Software Foundation, (opens in a new tab) documentation describes ZipInfo.file_size and compress_size as separate member facts. Python Software Foundation, (opens in a new tab) exposes tar member metadata, but a tar stream can contain padding, multiple members, and special types. A gzip wrapper has a different contract: RFC Editor, (opens in a new tab) defines ISIZE as the original input size modulo 2^32. Do not present one field as a universal expanded-size truth.

Name every size and its evidence source (table)
MeasurementSource of observationQuestion it can answer
Container bytesObject metadata or controlled transfer countHow much wrapper data arrived?
Compressed member bytesArchive member metadata or stream countHow much encoded member data is declared?
Declared expanded bytesMember header or gzip trailer fieldWhat does the format claim?
Observed expanded bytesBounded decoder counterHow much output was actually produced?
Inspected bytesReview loop and window recordHow much evidence did this check cover?
Retained derivative bytesSeparate artifact manifestWhat did the review write or keep?

Attach a unit, source, and confidence to each value. “Unavailable” is different from zero, and a stopped counter is different from a complete count. Keep compressed and expanded artifact identifiers separate so a report never implies that a derivative is the original source.

Compare declarations with bounded observations

Compare declared and observed values only after confirming that they describe the same member and representation. A mismatch can indicate truncation, a stale header, a concatenated stream, a reader policy, or an incorrect assumption about which layer was measured. It does not by itself identify cause or prove malicious behavior.

Synthetic size ledger

Compare declarations with bounded observations code example, Plain text
artifact_ref       layer                 declared_bytes  observed_bytes  state
synthetic-archive   wrapper               18432            18432           complete
synthetic-member   compressed member      5120             5120           complete
synthetic-member   expanded declaration  12000            12000           matched
synthetic-member   inspected window      unavailable      4096            bounded
synthetic-gzip     ISIZE trailer          modulo field     4294967295      not a full-size proof

The values are invented for this worksheet. The final row illustrates why a gzip ISIZE value must be labeled with its modulo semantics, not copied into a “total expanded bytes” column. The observation state explains whether a reader reached the end, stopped at a configured limit, or could not determine completion.

Compare declarations with bounded observations (table)
ComparisonInterpretationDisposition
Declared and observed matchThe selected layer agrees for the observed runKeep the layer, counter, and completion evidence
Observed stops below declarationThe run is partial, limited, or truncatedHold as incomplete and preserve stop reason
Observed exceeds declarationMetadata or stream interpretation disagreesRecheck member identity, wrapper, and reader behavior
Declaration unavailableNo format field can support the comparisonUse an observed bounded count and mark declaration unavailable
Modulo-limited fieldThe field has a format-specific rangeDo not convert it to an exact total without another source

Set resource limits before expansion

A useful budget is a policy record, not just a ratio. Define maximum container bytes, maximum member count, maximum expanded bytes, maximum nesting or wrapper depth, maximum retained bytes, and an external CPU or wall-clock budget. The application or execution environment must enforce those limits; a worksheet cannot make a decoder bounded after the fact.

  1. Identify the archive and member before allocating a destination or output buffer.
  2. Check available declarations and reject impossible totals before reading expanded data.
  3. Count output incrementally and stop at the configured expanded-byte limit.
  4. Drain or close the bounded stream according to the receiving runtime so the next member is not misread.
  5. Record the stop class, counters, and cleanup result without storing raw expanded content.
  6. Use a separate policy for nested archives and multiple members; a per-member limit is not a whole-job limit.

Use a synthetic fixture and issue table

Start with text fixtures that exercise matched sizes, missing declarations, a member that reaches the expansion limit, a truncated stream, a concatenated wrapper, and a nested archive marker. Use synthetic references and labels. Do not copy a private archive name, path, object tag, or content sample into a public report.

Use a synthetic fixture and issue table (table)
Fixture findingEvidence to retainSafe state
Wrapper count completeContainer identifier and received bytesComplete at wrapper layer
Member count stops at limitMember index, counter, limit idResource limit reached
Expanded count below declarationDeclared field, observed count, stop classIncomplete or truncated
Multiple wrapper membersMember order and per-member countersReconcile independently
Nested archive markerParent member and child referenceApply nested policy before reading
Cleanup failsStable cleanup issue class onlyHold the job and preserve evidence

When a limit is reached, keep the partial artifact out of any workflow that assumes completeness. If a bounded sample is useful, name it as a derivative, record its limit, and retain the source reference separately. A successful sample review is not evidence that the full expansion was observed.

Keep resource evidence separate from downstream decisions

Size and limit evidence can answer whether a selected run completed under a named policy. It cannot establish archive ownership, permission to retain content, privacy clearance, semantic accuracy, or commercial demand. Put those decisions in their own provenance, privacy, rights, and owner-approved records.

Pair this guide with the data provenance log for artifact identity and export guidance for producer behavior. After the owner has a supported non-health scope, authority to describe the collection, and a controlled size ledger or synthetic fixture, the qualified appraisal request path may be a next preparation step. It does not expand archives, certify resource limits, create a quote, match a buyer, issue a legal opinion, or grant a license.

Takeaways

  • Record container, compressed, declared expanded, observed expanded, inspected, and retained sizes separately.
  • Label units, source layers, completion state, and unavailable values explicitly.
  • Treat gzip ISIZE as a modulo field rather than a universal expanded-size proof.
  • Set per-member, whole-job, nesting, retention, and external runtime limits before expansion.
  • Keep a limit stop or partial sample out of workflows that require complete evidence.

Sources

  1. RFC Editor: GZIP file format specification (opens in a new tab)
  2. Python Software Foundation: zipfile - Work with ZIP archives (opens in a new tab)
  3. Python Software Foundation: tarfile - Read and write tar archive files (opens in a new tab)
  4. Python Software Foundation: gzip - Support for gzip files (opens in a new tab)

Use the appraisal path after the size ledger is organized.

It does not expand archives, certify resource limits, create a quote, match a buyer, or grant a license.