On this page
Define the question before the sample
Sampling is safer when each artifact has a narrow purpose. Do not begin with representative rows as a default. Begin with the decision a reviewer needs to make, then choose the smallest artifact that can answer it.
| Review question | Least revealing first artifact | What remains unknown |
|---|---|---|
| Can the files be located and opened? | File manifest with format, encoding, size, and digest | Record meaning and sensitivity |
| Does the export match the declared structure? | Headers, schema, codebook, and synthetic fixture | Whether real values follow the rules |
| Are relationships preserved? | Key inventory and relationship report | The content of linked records |
| Are sensitive fields present? | Field classification and redaction plan | Residual re-identification risk |
| Are labels documented? | Label dictionary and version history | Label accuracy on real records |
| Does a workflow render correctly? | Synthetic or owner-approved redacted fixture | Real-world coverage and edge cases |
The W3C, (opens in a new tab) treats structural metadata, provenance, and data-quality information as useful context for data publication and reuse. Use that boundary to explain what an artifact answers, not to turn a metadata check into a rights or privacy conclusion.
Choose the least revealing artifact
| Current sensitivity and question | Artifact | Gate |
|---|---|---|
| Unknown sensitivity or unclear authority | No record sample | Identify owner, scope, and handling decision |
| File and schema behavior only | Metadata-only manifest | Keep values out of the artifact |
| Shape, coverage, or missingness pattern | Aggregate or field-level report | Suppress small or revealing groups where needed |
| Parser, join, or label behavior | Synthetic fixture | Make the synthetic origin obvious |
| Concrete semantics that metadata cannot show | Redacted artifact | Obtain owner approval and document what was removed |
| Sensitive content remains necessary | Escalated review | Use the appropriate privacy and legal process |
An aggregate can still disclose information when a group is small or unusual. A redacted text can still retain a distinctive phrase. A synthetic row can accidentally copy a real value if it was generated by an unsafe transformation. The artifact type is not a risk verdict. Record the threat, intended audience, transformation, and residual uncertainty.
General information, not legal advice.
Run structural checks in a fixed order
Start with an inventory that can be reviewed without opening records. Useful fields include a neutral file identifier, format, encoding, declared schema version, byte size, digest, source reference, redaction state, approval state, and retention or deletion instruction. Do not include credentials, direct contacts, or raw text in the manifest.
- Check the manifest and confirm that the expected files, formats, encodings, and schema versions are present.
- Check shape: headers, unique column names, compatible row widths, documented date-time rules, and intended join identifiers.
- Exercise validation rules with a synthetic fixture that is clearly marked and contains no copied private values.
- Consider an approved redacted artifact only when metadata and synthetic checks cannot answer the named question.
The W3C, (opens in a new tab) provides a vocabulary for tables, rows, columns, cells, and metadata. It does not decide whether fields are lawful to share or whether labels are accurate. Keep those questions separate from parser and shape checks.
| Synthetic fixture case | Purpose |
|---|---|
| Empty value and explicit unavailable value | Exercise missingness states without a real row |
| Allowed and disallowed category | Exercise the codebook boundary |
| Date boundary | Exercise format and time-zone handling |
| Quoted delimiter | Exercise parser behavior for embedded punctuation |
| Linked record | Exercise join behavior with neutral identifiers |
Record findings without exposing the sample
| Field | Example value style |
|---|---|
check | schema_column_order |
expected | Declared order and type contract |
observed | Pass, review, out of scope, or unknown |
artifact_ref | Manifest or fixture identifier, not a raw path |
privacy_state | Metadata-only, synthetic, redacted, or restricted |
owner_approval | Confirmed, pending, or not applicable with reason |
open_question | The next evidence request |
Do not turn “pass” into a composite quality grade unless a separate, reviewed method defines that grade. A manifest can pass an integrity check while the collection remains incomplete. A synthetic fixture can pass a parser test while real labels remain inconsistent. The UK Government, (opens in a new tab) treats completeness, uniqueness, consistency, validity, timeliness, and accuracy as distinct dimensions. Keep that distinction in the review log.
Set a clear privacy stop
Stop the sampling flow when intended use is unclear, authority is missing, or the sample may contain health information, voice or likeness data, employee records, credentials, client-confidential material, or a rare combination of attributes. Do not ask an owner to upload raw records just to discover whether the export is parseable. Ask for a metadata-only manifest, schema, synthetic fixture, or documented exclusion list first.
NIST’s National Institute of Standards and Technology, (opens in a new tab) discusses choices such as synthetic data, protected environments, and controlled access. Do not call a sample anonymous merely because obvious names were removed. Record the handling boundary and residual uncertainty.
General information, not legal advice.
Close with a qualified next step
If the scope is a supported non-health collection and the owner has authority to describe it, the next qualified step may be the appraisal route. That route is a request for review, not automatic acceptance, a quote, a buyer match, a legal opinion, or a license. Keep any sensitive sample outside this guide and inside the owner-approved handling boundary.
Takeaways
- Define the decision before defining the sample.
- Begin with metadata, structure, aggregates, or synthetic fixtures.
- Treat redaction as a documented transformation with residual risk.
- Keep privacy, rights, quality, and demand as separate questions.
- Stop when authority or handling scope is unknown.
Sources
- W3C: Data on the Web Best Practices (opens in a new tab)
- National Institute of Standards and Technology: De-Identifying Government Datasets: Techniques and Governance (opens in a new tab)
- W3C: Model for Tabular Data and Metadata on the Web (opens in a new tab)
- UK Government: The Government Data Quality Framework (opens in a new tab)