On this page
Separate absence, delete, tombstone, and purge
Use distinct review states rather than inferring a deletion from a row count. A dataset version and change log can explain that a release changed; this guide records what deletion evidence exists for affected keys and what remains unknown. Sources checked on 22 October 2026. Dates in the Sources list are retrieval dates, not claimed publication dates.
| State | What it can mean | What it does not prove |
|---|---|---|
absent_from_export | The row was not in this package’s declared scope or result | That the source deleted it |
delete_event | The source emitted a deletion operation for a key | That every prior copy has been removed |
tombstone | A deletion marker used by an event stream or target system | The reason, authority, or legal effect of deletion |
redacted | Some values were replaced or withheld | That the underlying record no longer exists |
purged | A source or retention process says data was removed | That every downstream system complied |
unknown | Evidence is insufficient | Any conclusion about the row state |
Write the state into the package manifest or evidence ledger. Do not label a row deleted merely because it is absent from one scope, one snapshot, or one file.
Preserve the minimum deletion evidence
| Field | Why it matters |
|---|---|
source_system | Identifies which source made the assertion |
table_or_collection | Identifies the affected scope |
record_key | Identifies the row or entity under the source key policy |
operation | delete, tombstone, truncate, redaction, or a documented source value |
source_position | Log position, sequence, version, or UNAVAILABLE |
source_event_time | Time assigned by the source, separate from processing time |
before_state_available | Whether prior values or only the key are available |
package_action | Included, excluded, replaced, or held |
evidence_ref | Controlled reference to the source record or report |
Do not copy a deleted row into a public worksheet merely to prove deletion. If a key is personal or sensitive, use a restricted reference or a keyed digest under the owner policy. A hash helps correlate evidence; it does not prove that a record is gone or that disclosure is permitted.
Understand what the source actually emits
The PostgreSQL Global Development Group, (opens in a new tab) documentation says a publication can select insert, update, delete, and truncate. Those settings apply to DML changes, not the initial synchronization copy, so label snapshot and later-change phases separately. The same publication documentation also requires a replica identity for a published table to identify updates and deletes. If no usable identity exists, record that limitation instead of inventing a key.
The Debezium project, (opens in a new tab) documentation provides a concrete event-stream example. It describes a delete operation with source metadata and an optional before state controlled by replica identity. A delete can be followed by a tombstone with the same key and a null value so the cited log system can remove older values for that key Apache Software Foundation, (opens in a new tab). This is behavior specific to the cited connector and log system, not a universal tombstone definition.
The same connector documentation distinguishes a table truncate from a row delete and notes that truncate events have no message key. Treat a truncate as a scope-wide operation; do not invent row-level deletion records without a documented mapping. The Apache Software Foundation, (opens in a new tab) documentation is context for partitioned log behavior, not proof that every export uses the cited log semantics.
Build a synthetic deletion fixture first
Fictional keys and state transitions
# Synthetic fixture only. No production records are used.
account-A: create -> update -> delete -> tombstone
account-B: create -> excluded-by-scope
account-C: create -> redacted
account-D: no prior row -> delete event
table-X: truncate event with no row key- Distinguish a delete event with metadata from a null-valued tombstone.
- Mark a missing key or missing before state as unknown, not as a guessed row.
- Recognize a replayed delete as the same source event when the source identity supports it.
- Give a truncate a table-level disposition when no row mapping exists.
- Show which rows were excluded by scope rather than deleted.
| Finding | Disposition |
|---|---|
| Delete has stable key and source position | Record deletion evidence and package action |
| Delete has key but no prior row in this package | Hold as source event; do not infer prior completeness |
| Tombstone has null value but undocumented semantics | Mark marker semantics unknown |
| Row absent after scope or filter change | Record exclusion; do not label deleted |
| Delete lacks usable identity | Hold for source-owner explanation |
| Truncate has no row-level mapping | Preserve table-level event and scope |
Reconcile deletion evidence with package boundaries
Compare deletion records with the package lower and upper bounds, source snapshot reference, table scope, and schema version. A delete just outside the selected window may explain a difference between releases without belonging to either package. A late event may require an overlap or correction package. Keep source event time, processing time, and package publication time in separate fields; do not sort by one and call it source order.
The dataset version and change-log guide owns release boundaries. The data provenance log owns source and transformation history. The file manifest and checksum guide owns package-byte identity. This guide owns deletion-state evidence and the distinction between absence, delete, tombstone, and purge.
- Record the source position or write
UNAVAILABLEwhen the source gives none. - Keep source event time distinct from processing and package times.
- Hold late events or window gaps instead of silently labeling them complete.
- Retain a controlled evidence reference without publishing raw values.
Keep retention separate and choose a qualified next step
A deletion marker can contain a direct identifier, transaction reference, or timestamp that reveals operational activity. Limit access to the evidence ledger, retain only what the owner policy allows, and do not publish raw delete events, keys, or before-images from a real collection. Deletion evidence does not answer whether an export may be licensed. Ownership, authority, contractual restrictions, privacy, retention, and downstream obligations remain separate.
After the owner has documented the source, scope, exclusions, and handling boundary, the appraisal route can be a qualified preparation reference for an eligible, supported non-health collection. It is a request for review, not automatic acceptance, a quote, a buyer match, a legal opinion, or a license. Do not upload sensitive deletion logs or raw records merely to demonstrate a tombstone workflow.
General information, not legal advice.
Takeaways
- Treat absence, delete, tombstone, redaction, and purge as different states.
- Preserve the source key and position when available, plus explicit unknowns when they are not.
- Test snapshots, row deletes, tombstones, truncates, scope exclusions, and replays with synthetic fixtures.
- Keep deletion evidence separate from package identity, rights, privacy, retention, and licensing decisions.