Skip to content

Evidence methods guide

Reference Data Version Join Review: Show Which Label or Rate Was Joined

Trace a reference-data join through key, effective interval, version, overlap, and unmatched-row evidence.

A join can be syntactically successful while selecting the wrong reference version. Review the business key, effective interval, update timestamp, tie rule, and unmatched or multiply matched rows. dbt documents unique keys and updated timestamps as snapshot configuration evidence. W3C PROV-O provides a vocabulary for describing entities, activities, and derivations. These sources help make the join traceable, but they do not decide which reference period an owner intended.

Updated

On this page

Name the reference version question

The source dates shown in this guide identify documentation checks on 22 October 2026, not original publication dates. Start by naming the fact being attached: a category label, exchange rate, region code, threshold, or other reference value. Record the event key and its time basis separately. “Current reference table” is not a reproducible version unless current means a named snapshot or an explicit retrieval point.

Name the reference version question (table)
Join fieldRecordStop if
Business keyNormalized key and source spellingWhitespace or case changes meaning
Event timeTimestamp and timezone contextLocal time is compared to UTC
Effective intervalStart, end, inclusivityOpen end has no stated rule
Reference versionSnapshot id and retrieval timeOnly a file name is retained
Tie ruleNewest, highest priority, or holdMultiple rows can win silently
Unmatched rowsCount and examples using synthetic idsNo report exists

A versioned join needs an explicit answer to “which row was eligible at this event time?” Keep a source key and a join key if they differ. Do not overwrite the raw value with a normalized label before preserving the evidence used to choose it.

Check effective intervals and overlapping versions

Represent each reference row with an effective start, effective end, snapshot identifier, and update observation. Choose whether the start is inclusive and whether an end is exclusive. Adjacent intervals should meet without overlap if that is the owner rule. An overlap is not resolved by sorting unless the rule says which row wins and why.

Check effective intervals and overlapping versions (table)
CaseSynthetic rowsFinding
Adjacentv1 ends at T, v2 starts at TBoundary rule required
Overlapv1 and v2 both cover TAmbiguous unless tie rule exists
GapNo row covers TReference unavailable at event time
Late updateSnapshot changes after eventRetrieval time differs from event time
Duplicate keyTwo rows share key and intervalUnique-key evidence fails

The dbt Developer Hub, (opens in a new tab) documentation distinguishes a unique key from a changed-at signal. Neither field alone proves that an effective interval is complete. Retain the owner’s interval policy and report rows that violate it rather than selecting a convenient answer.

Record the derivation from snapshot to joined output

Treat the reference snapshot, join activity, and output table as separate artifacts. A useful record includes snapshot id, source URI or identifier, retrieval timestamp, query or filter version, key normalization rule, interval policy, and row-level match status. If a manual override exists, record it as an activity and keep the automatic result.

  1. Identify the input event artifact and its time basis.
  2. Identify the reference snapshot and its update evidence.
  3. Record the key normalization and interval predicate.
  4. Count zero, one, many, and held matches.
  5. Store the output value with the selected reference id.
  6. Link corrections or overrides to a separate activity.

Run a synthetic join matrix

Synthetic versioned reference fixture

Run a synthetic join matrix code example, Plain text
event_key | event_time | reference_key | result
fixture-a | 2026-01-10T12:00Z | R1 | v1
fixture-a | 2026-02-10T12:00Z | R1 | hold:overlap
fixture-b | 2026-02-10T12:00Z | R2 | hold:gap
fixture-c | 2026-02-10T12:00Z | R3 | hold:unmatched

reference: R1 v1 [2026-01-01,2026-02-15)
reference: R1 v2 [2026-02-01,2026-03-01)
reference: R2 no row for February
reference: R3 duplicate key withheld
Run a synthetic join matrix (table)
AssertionExpected evidenceDo not do
One matchReference id and interval recordedKeep only display value
OverlapBoth candidate ids and tie rule statusChoose newest by habit
GapNo reference row and owner questionFill from adjacent version
UnmatchedSynthetic key and countPublish a null as if resolved
Re-runSame snapshot and predicate reproduce resultUse moving current data

In this fixture, R1 v1 covers 1 February through 15 February and R1 v2 starts on 1 February, so the 10 February event is covered by both half-open intervals. hold:overlap is therefore the expected result. If v2 started on 15 February instead, the same event would select v1 under this interval convention.

The fixture is deliberately invented. Add a boundary event exactly at an interval end, a key with case and whitespace differences, and a reference update observed after the event. The expected result should be a status plus evidence, not a guessed label.

Keep reference joins separate from business conclusions

A versioned join can show which reference row was eligible and where evidence is missing. It cannot establish that a reference value is correct, that an event is authorized, or that a downstream decision is valid. Keep reference stewardship and domain interpretation outside the join report.

Pair this guide with the dataset version change log for release history and the data provenance log for artifact lineage. After a supported non-health scope, authority to describe the collection, and a controlled reference packet are established, the qualified appraisal request path may be a preparation step. It does not run joins, choose reference values, create a quote, match a buyer, or grant a license.

Takeaways

  • Define the event time, reference key, effective interval, and version together.
  • Report overlap, gaps, duplicates, and unmatched keys instead of hiding them.
  • Retain snapshot, predicate, normalization, and selected reference id.
  • Model corrections as separate provenance activities.
  • Do not treat a successful join as proof of reference correctness.

Sources

  1. dbt Developer Hub: Add snapshots to your DAG (opens in a new tab)
  2. World Wide Web Consortium: PROV-O: The PROV Ontology (opens in a new tab)

Use the appraisal path after the reference join evidence is organized.

It does not run joins, choose reference values, create a quote, match a buyer, or grant a license.