On this page
Preserve the byte boundary and header
Keep an immutable artifact id, byte length, capture boundary, and the exact offset range under review. A decoded preview is derived evidence and should never replace the raw byte boundary. Record magic bytes, version markers, declared record count if present, and whether the export was copied or transformed. If the first bytes are missing, state that the header and layout cannot be verified.
A binary file can contain a header, records, padding, variable-length regions, and a trailer. Do not assume every byte belongs to a fixed-width row. A file extension, a successful library open, or a plausible first value does not establish the record boundary. Keep unknown offsets as unknown until a format document or owner note supports them.
| Layout field | Evidence | Stop when |
|---|---|---|
| byte order | declared endian marker or format rule | Order inferred from a value |
| field width | bytes per field | Width differs across versions |
| signedness | signed, unsigned, float, or unknown | Negative interpretation is guessed |
| alignment | padding and alignment rule | Native alignment is assumed |
| record stride | bytes from one record to next | Variable-length area is ignored |
| version | header version and source | Version is inferred from filename |
Separate byte order, width, and alignment
Byte order controls how multiple bytes form a scalar. Width controls how many bytes are consumed. Alignment and padding control where the next field starts. These are independent hypotheses: a four-byte little-endian integer followed by an eight-bit flag is not the same layout as a native-aligned struct with three padding bytes. Write field offsets and end offsets rather than listing only field names.
The Python Software Foundation, (opens in a new tab) documentation warns that native mode follows the platform and compiler and can include pad bytes, while external data exchange requires explicit ordering and padding. Use standard, explicitly documented layout rules for a review worksheet and record any implementation-specific assumption as a limitation.
| Hypothesis | Example record | Review question |
|---|---|---|
| little-endian uint32 | offset 0, width 4 | Does the format specify little endian? |
| big-endian uint32 | offset 0, width 4 | Would the header identify the order? |
| signed int8 | offset 4, width 1 | Is the sign part of the field type? |
| padding | offset 5 to 7 | Is alignment documented or assumed? |
| fixed stride | next record at offset 8 | Do all records have equal width? |
| variable payload | length plus bytes | Which field defines the extent? |
Keep decoded values tied to offsets
A review record should contain raw offset, byte slice reference, proposed field, type, order, width, decoded value if verified, and confidence or unresolved status. This lets a second reviewer challenge the layout without receiving private payloads. A value should not be copied into a summary without its byte-range evidence.
Link to the text encoding and row guide when a binary field contains text, but do not collapse encoding into byte order. UTF-8 bytes, integer endianness, and record stride answer different questions. Keep the source format version and conversion tool or owner note alongside the layout.
Offset discipline
- Record offsets from the start of the artifact, not from a rendered preview.
- Keep field width and padding explicit for every record member.
- Preserve a redacted byte-slice id instead of copying private bytes into prose.
- Mark an inferred value as unverified until the layout source is identified.
- Re-check the stride when a version, architecture, or optional field changes.
Use a short synthetic hex fixture
The fixture below makes byte order and padding visible with harmless values. It is a worksheet, not an executable parser. Write the expected offsets and decoded values beside the bytes, then change one assumption at a time. If two layout hypotheses both fit the fixture, add a discriminating byte pattern rather than choosing the more convenient result.
Do not use customer identifiers or real binary payloads in a public example. A synthetic header, one record, and a second record with a different sign or byte order are enough to test the contract. Keep raw values in a controlled artifact only when the owner-approved boundary permits it.
| Synthetic finding | Disposition | Next evidence |
|---|---|---|
| Declared order and offsets fit both records | continue conditionally | Check header and version evidence |
| Value fits only under native alignment | hold | Find external format layout |
| Stride skips an observed byte | hold | Reconcile padding or optional field |
| Header is absent | layout unverified | Obtain format metadata |
| Decoded value lacks source offset | incomplete | Attach byte-range evidence |
Synthetic fixed-record layout
bytes: 01 00 00 00 7f 00 | 02 00 00 00 80 00
layout: uint32le@0..3, int8@4, pad@5; stride=6
record0: count=1, flag=127 | record1: count=2, flag=-128
check: big-endian interpretation would produce different countsReport layout uncertainty narrowly
Use language such as “little-endian hypothesis fits the synthetic fixture” or “record stride unavailable.” Do not say that a decoder succeeded, therefore every record is valid. Keep corruption, truncation, wrong version, wrong architecture, and wrong semantic field meaning as separate possible causes.
A binary layout can be mechanically reproducible while its values remain semantically unknown. Record units, scaling, coordinate systems, or code lists separately. If the source has no authoritative layout description, preserve the decoded result as a hypothesis and do not promote it to a measurement.
| Finding | Disposition | Do not infer |
|---|---|---|
| Explicit layout fits header and fixture | bounded pass | All records have the same meaning |
| Endianness marker missing | hold | Platform native order is correct |
| Padding differs by version | versioned review | Values are corrupt |
| Byte range truncated | incomplete | The last record is zero |
| Units or semantics unknown | structural only | Decoded number is a metric |
Close with a qualified preparation step
After byte boundary, layout hypothesis, version, offsets, and synthetic evidence are documented, keep the packet structural. Do not upload private binary payloads merely to test an endian question. Use the data provenance log for copies and transformations and a data card for scope.
For a supported non-health collection, with authority to describe the artifact and a controlled handling boundary, the qualified appraisal path may be a next preparation step. It does not decode bytes, certify a layout, create a quote, match a buyer, issue a legal opinion, or grant a license. A reproducible binary interpretation is not proof of rights or accuracy.
Takeaways
- Preserve raw offsets and header evidence before decoding values.
- Specify byte order, width, signedness, alignment, padding, and stride separately.
- Tie every decoded field to a byte range and format version.
- Use synthetic bytes to discriminate layout hypotheses.
- Keep mechanical decoding separate from semantic, rights, and accuracy decisions.