On this page
Give each missing state a meaning
Start with a field-level missingness dictionary. A single null marker may be unavoidable in a source system, but the review record should say what it means when the meaning is known. Record the source marker, interpreted state, interpretation rule, and whether the rule is confirmed or provisional.
| State | Meaning to document |
|---|---|
not_collected | The source process did not request or capture the value |
not_applicable | The field does not apply to this row under the stated rule |
withheld | A value exists but is intentionally not included in the reviewed artifact |
unknown | The owner cannot determine whether a value exists or what it means |
not_yet_processed | The source workflow expects a later stage to populate the field |
error | A processing or validation failure prevented a usable value |
empty_string | A representation that needs interpretation, not an automatic semantic state |
Do not silently map a blank to zero, false, “unknown,” or “not applicable.” Those values can change analysis and downstream decisions. The UK Government, (opens in a new tab) distinguishes completeness from accuracy and treats completeness, uniqueness, consistency, timeliness, validity, and accuracy as separate dimensions.
Create a label contract
A label dictionary should make a classification reproducible and disagreement visible. Use one row for each label field or taxonomy version. A field called consent_status is not proof of consent, and a field called approved is not a legal conclusion. Record which role or process set a workflow label and when, without turning the label into a legal determination.
| Label contract field | What to record |
|---|---|
label_name | Field name and human-readable purpose |
definition | The condition that makes the label apply |
allowed_values | Codes, spelling, case, and display text |
hierarchy | Parent, child, or flat category relationship |
exclusive | Whether more than one value may apply |
abstain_state | Unknown, unresolved, or not reviewed state |
source | Rule, source field, annotation process, or document |
version | Codebook version and effective scope |
conflict_rule | What to do when labels disagree |
change_note | How a later definition differs from an earlier one |
The W3C, (opens in a new tab) supports explicit annotations for tables, rows, columns, and cells. That helps a codebook carry structure and scope, but it does not supply the correct taxonomy for a particular collection.
General information, not legal advice.
Test consistency at several levels
Run checks where the same meaning can be contradicted. A field-level check looks at allowed values and missingness rules. A row-level check compares related fields. A cross-table check compares the same entity or key across distributions. A time check compares codebook versions and effective periods.
| Check | Example rule | Finding to record |
|---|---|---|
| Representation | Blank, null, and sentinel values map to documented states | Unknown mapping or confirmed mapping |
| Allowed value | Every non-missing label appears in the active codebook | Unrecognized code and source reference |
| Exclusivity | Mutually exclusive labels do not occur together | Conflict pair and decision state |
| Hierarchy | A child label has the expected parent | Orphan label or outdated codebook |
| Cross-field | A dependent field is present when its label requires it | Missing dependency or valid exception |
| Cross-table | The same key has compatible labels across exports | Conflict, version difference, or unknown relation |
| Temporal | A label uses the codebook active at its recorded time | Version drift or missing effective date |
Do not fix a conflict by choosing the most common label. Keep the source value, rule version, conflict state, and decision owner. If the codebook changed, a mismatch may be a valid version difference rather than an error. The UK Government, (opens in a new tab) recommends describing inconsistencies, cleaning performed, and reasons unresolved inconsistencies remain.
Use aggregate checks before record review
Run the first pass locally or in a controlled environment. Produce field-level summaries such as distinct missingness states, unrecognized labels, conflict categories, codebook versions, and source partitions. Do not publish small-group counts or rare combinations when they could reveal membership or identity. If a reviewer needs an example, use a synthetic fixture or an owner-approved redacted row whose purpose is documented.
- State which question the check answered.
- State which questions remain unavailable because the source, codebook, or owner evidence was not supplied.
- Keep a field-level evidence reference instead of a raw record.
- Preserve the source label and version when a normalization or correction is proposed.
A field can be fully populated and still have wrong labels. A label can be valid under the current codebook while the source process is incomplete. A missing value can be intentional and therefore not an error. State the scope of the check instead of presenting a universal score.
Decide what happens next
| Finding | Disposition | Required follow-up |
|---|---|---|
| Missingness states and codebook are documented | Structural review ready | Test cross-field and temporal rules |
| Blank values have multiple possible meanings | Hold | Confirm source semantics before recoding |
| Labels are valid but codebook version is absent | Conditional | Add version and effective scope |
| Labels conflict across tables | Hold or preserve both | Resolve source, time, and entity relationship |
| Field is sensitive or rare combinations are present | Do not share a row | Use metadata-only or controlled review |
| All checks are unknown because the source is unavailable | Unknown | Record the evidence gap, not a pass |
General information, not legal advice.
Close with a qualified next step
If the scope is a supported non-health collection and the owner has authority to describe it, the qualified next step may be the appraisal route. That route is a request for review, not automatic acceptance, a quote, a buyer match, a legal opinion, or a license. Do not upload raw sensitive records to demonstrate that a label check exists.
Takeaways
- Give blanks and nulls explicit meanings before measuring completeness.
- Version the label dictionary and document conflict and abstain states.
- Check representation, allowed values, dependencies, relationships, and time.
- Preserve uncertainty and use synthetic or metadata-only artifacts first.
- Treat a consistency result as evidence for the named checks, not a universal quality grade.