On this page
Preserve the writer schema with the data
The source date shown in this guide identifies a documentation check on 22 October 2026, not an original publication date. Follow the citation for the Avro specification edition being discussed.
The Apache Avro, (opens in a new tab) says binary encoding omits type information, field names, and record separators. A reader therefore needs the writer schema, plus any reader schema used for resolution. Preserve both with the data. Do not store only the latest schema and assume it can reconstruct an older writer.
| Evidence field | What to preserve | Why it matters |
|---|---|---|
| Data artifact | Exact file, message batch, or controlled reference and digest | Identifies the bytes under review |
| Writer schema | Schema used to encode the artifact, not only the latest schema | Allows the reader to interpret the bytes |
| Reader schema | Schema the target application will use | Makes resolution direction explicit |
| Direction | For example, old writer to new reader | Prevents a vague compatibility label |
| Schema identifier | Registry ID, version label, or explicit unknown | Connects the pair to its source |
| Canonical-form reference | Representation or fingerprint with method | Makes comparison reproducible |
| Resolution result | Pass, fail, not tested, or application review | Keeps uncertainty visible |
| Semantic questions | Units, code meanings, time rules, null meaning, defaults | Separates format result from application meaning |
If the writer schema is missing, mark the read blocked or unknown. A successful parse with a guessed schema does not prove that the original values were interpreted correctly.
Apply Avro resolution rules in the right direction
The specification calls the encoding schema the writer schema and the expected schema the reader schema. Record fields match by name, regardless of order. Writer-only fields can be ignored. Reader-only fields need a default or resolution errors. A default fills missing data while reading; it does not make an encoding field optional Apache Avro, (opens in a new tab).
| Writer type | Reader type | Specification-level result |
|---|---|---|
| int | long, float, or double | Promotable |
| long | float or double | Promotable |
| float | double | Promotable |
| string | bytes | Promotable |
| bytes | string | Promotable |
| Any other unlisted pair | Selected reader branch | Requires a matching rule or fails |
These rows describe schema resolution, not application safety. An integer-to-floating promotion can change validation or precision assumptions. String-to-bytes can change display or indexing behavior. Record those consequences separately from the Avro result.
| Schema change | Resolution observation | Application question |
|---|---|---|
| Reader adds a field with a default | Default can supply a missing writer field | Is the default semantically safe for every older record? |
| Reader adds a field without a default | Resolution signals an error | Should the release be held? |
| Writer adds a field the reader lacks | Writer value can be ignored | Can the application tolerate dropped evidence? |
| Reader renames a field with an alias | Writer name may map to reader name | Did the codebook meaning stay the same? |
| Writer sends an unknown enum symbol | Reader enum default may handle it | Does the fallback conceal a new state? |
| Both sides use unions | Selected writer branch resolves to first matching reader branch | Which branch is selected, and is it intended? |
| Only documentation changes | Documentation is ignored for resolution | What human documentation still needs review? |
Aliases do not prove that meaning is unchanged. Compare a renamed field’s codebook and record the application decision separately. A default can preserve the read while hiding a new producer state.
Compare a safe synthetic schema pair
The following pair is synthetic and contains no real event, customer, location, or operational data. It widens a count, renames a field with an alias, and adds a defaulted field. The synthetic default is a fixture value only. It is not a truthful production default and must be replaced or held for application review before use.
Synthetic old writer schema
{
"type": "record",
"name": "SyntheticEvent",
"namespace": "example.synthetic",
"fields": [
{"name": "event_id", "type": "string"},
{"name": "count", "type": "int"},
{"name": "label", "type": ["null", "string"], "default": null}
]
}Synthetic new reader schema
{
"type": "record",
"name": "SyntheticEvent",
"namespace": "example.synthetic",
"fields": [
{"name": "event_id", "type": "string"},
{"name": "count", "type": "long"},
{"name": "status", "type": ["null", "string"], "aliases": ["label"], "default": null},
{"name": "source", "type": "string", "default": "synthetic"}
]
}| Field | Observation | Open question |
|---|---|---|
| event_id | Same name and primitive type | Is the identifier meaning unchanged? |
| count | int to long promotion | Can consumers preserve range and units? |
| label to status | Reader alias maps the old name | Does the codebook use the same meanings? |
| source | Reader-only field with a fixture default | What honest value applies to older records? |
This pair is not proof that an implementation accepts every edge case. A real test needs the actual writer schema, reader, logical types, union branches, and representative synthetic values. Treat the fixture default as unresolved until the application owner supplies a truthful rule.
Record compatibility as a matrix, not a badge
Test each direction that the release process permits. Name which schema wrote the bytes and which schema read them before calling a result backward compatible.
| Test case | Writer schema | Reader schema | Result field |
|---|---|---|---|
| Read older data | Previous release | Candidate release | Decode result, defaults, aliases, and semantic review |
| Read new data with an older consumer | Candidate release | Previous release | Whether ignored fields can be tolerated |
| Replay a stored message | Exact stored writer schema | Current consumer schema | Resolution plus semantic transformations |
| Change a union | Previous branch set | Candidate branch set | Selected branch and default legality |
| Change a logical type | Previous annotation | Candidate annotation | Format result plus unit and precision review |
Use states such as pass, fail, not tested, and requires application review. Parsing Canonical Form can distinguish schema changes that matter to readers from attributes such as documentation that do not. A fingerprint can support stable identification, but the specification warns that fingerprints are not security guarantees. Keep its algorithm and scope beside the identifier, not as proof of provenance, integrity, or permission Apache Avro, (opens in a new tab).
Keep schema compatibility separate from rights and the next step
A reader can resolve two schemas while the data still has personal or confidential content, incorrect labels, or out-of-scope records. Defaults can hide missingness, aliases semantic changes, and ignored fields evidence. Preserve the artifact, schema pair, direction, implementation version, and unresolved questions in a controlled worksheet.
Pair this review with the dataset version and change log for release lineage and the data provenance log for source, rights, processing, and exclusion evidence.
This article is not a serializer, registry, validator, or deployment guide. It is a static schema-resolution worksheet that keeps format observations separate from application and ownership decisions.
After the owner has a supported non-health scope, authority to describe the collection, and a schema-evolution evidence packet, the qualified appraisal request path may be a next preparation step. It does not test Avro schemas, approve compatibility, create a quote, match a buyer, issue a legal opinion, or grant a license. Keep raw payloads and sensitive field values within the owner-approved handling boundary.
Takeaways
- Preserve the writer schema with every Avro artifact and name the reader schema used for the test.
- Test direction explicitly; schema resolution is not one universal compatibility badge.
- Record promotions, defaults, aliases, union branches, enum defaults, and ignored writer fields separately.
- Use synthetic schemas and values before exposing real payloads, and never treat a fixture default as production truth.
- Treat successful resolution as a format result, not proof of semantic correctness, rights, privacy, or legal clearance.