Skip to content

Annotation quality guide

Annotation Label Ontology Versioning: Keep Meaning, Mappings, and History Together

Version label definitions as carefully as label values. Record concepts, aliases, hierarchy changes, mappings, and the evidence for every annotation release.

A label codebook is not an ontology merely because it has names. Review what each concept means, how concepts relate, which labels are allowed together, and which version governed each annotation. Keep deprecated labels resolvable without rewriting source history, and attach a change record to every mapping. This guide is a static worksheet, not a classifier, ontology host, taxonomy approval, privacy decision, or rights conclusion.

Updated

On this page

Separate a label list from an ontology contract

The source dates shown in this guide identify documentation checks on 22 October 2026, not original publication dates. Follow each citation for the vocabulary being discussed. Start by naming the annotation unit: a document, span, image region, audio interval, row, or event. Then define whether a label is a class, a property value, a relation, a workflow state, or a temporary review status.

The W3C, (opens in a new tab) describes a concept scheme with preferred and alternative labels, notes, and semantic relations. Borrowing those distinctions helps a review record say whether two strings are synonyms, translations, deprecated labels, or genuinely different concepts. It does not supply a domain taxonomy. The owner must define scope and examples.

Separate a label list from an ontology contract (table)
Ontology itemRecordQuestion
Concept identityStable id and human labelCan old annotations still name this concept?
Preferred labelDisplay string and languageWhich label is shown to annotators?
Alternative labelAlias, abbreviation, or synonymIs it equivalent or merely related?
DefinitionPositive and negative examplesWhat boundary makes the label apply?
RelationBroader, narrower, related, or mappedWhat does the relation mean?
Scope noteExclusions and uncertaintyWhen should an annotator abstain?
VersionCodebook id and effective rangeWhich contract governed a record?

Do not treat a display label as evidence of a concept definition. “Other,” “unknown,” and “not reviewed” can be workflow states rather than domain categories. Keep them separate so a later reader does not confuse missing annotation with a substantive class.

Make ontology versions reproducible

A version should identify the codebook artifact, effective interval, change summary, approver role if the owner uses one, and the annotation batches to which it applies. A date alone is not a version. Preserve the prior contract and publish an explicit migration note when a concept is renamed, split, merged, or retired.

Make ontology versions reproducible (table)
Change typeMapping questionSafe default
RenameDid meaning remain stable or only the display text?Keep the stable concept id and record the label change
Alias addedIs the new string equivalent in this language?Add an alternative label with a scope note
SplitWhich old records map to each new concept?Map only with evidence; preserve ambiguous state
MergeWhat distinctions are lost?Retain old ids and record information loss
RetireCan old data still be interpreted?Keep a deprecated id and a replacement note
Hierarchy changedDid broader or narrower meaning move?Recheck dependent rules and examples
Definition changedWhich annotations need re-review?Create a new version and identify affected batches

Use World Wide Web Consortium, (opens in a new tab) as a vocabulary for the record itself: a source annotation set is an entity, a migration is an activity, and the resulting release is another entity. A provenance edge can show derivation without pretending that the migration preserved every meaning. Record the agent or role responsible when the owner has that evidence.

Synthetic ontology change ledger

Make ontology versions reproducible code example, Plain text
concept_id | v1_label | v2_state | mapping | evidence
synthetic:parcel | parcel | renamed:package | direct | definition unchanged
synthetic:status | active | split:active,paused | ambiguous | source examples missing
synthetic:other | other | deprecated | none | preserve historical rows
synthetic:region | region | narrower:region.local | review | hierarchy changed

Keep mappings and abstentions visible

A migration table needs a one-to-one, one-to-many, many-to-one, or no-mapping state. A blank mapping hides the difference between a deleted concept and an unresolved one. Keep the source id, destination ids, mapping type, confidence or evidence note, and whether a human decision is still required.

  1. Compare definitions and examples before comparing strings.
  2. Test hierarchy and exclusivity rules after each mapping.
  3. Route split and merge cases to an explicit review state instead of forcing a label.
  4. Keep deprecated identifiers readable in historical annotations.
  5. Record which rows, spans, or regions were migrated and which remain on the old version.
Keep mappings and abstentions visible (table)
Mapping stateMeaningReport wording
directOne concept keeps its meaningMapped under the stated definition
aliasOnly the display or alternate label changedSame concept, label history retained
splitOne source concept has several possible targetsAmbiguous without additional evidence
mergeSeveral source concepts collapse to one targetInformation distinction may be lost
retiredNo replacement is assertedHistorical id preserved; current use unavailable
unmappedNo defensible destination was suppliedNeeds owner or annotator decision

Use a synthetic fixture before touching source annotations

Exercise the migration with invented ids and short examples. Include a stable rename, a split, a merge, a deprecated label, a changed hierarchy, and an unknown mapping. Do not use private text, customer names, locations, or raw annotation payloads to prove that the codebook changed.

Use a synthetic fixture before touching source annotations (table)
CheckEvidence fieldHold condition
Version identityCodebook id and digestVersion cannot be reproduced
Effective scopeBatch, time, or source partitionAffected annotations are unknown
Definition diffOld and new boundary notesOnly strings were compared
Mapping typeDirect, split, merge, retired, or unmappedOne state is hidden in a blank
Example coveragePositive, negative, and abstain examplesExamples missing for a changed concept
Migration resultNew id, old id, and unresolved stateOriginal id overwritten

Keep ontology evidence separate from downstream decisions

Ontology versioning can show which definitions and mappings were used. It cannot establish that labels are accurate, that an annotation was authorized, that a dataset is private, or that a collection is suitable for a use. Keep provenance, privacy, rights, and owner decisions in separate records.

Pair this guide with missing-value and label consistency review for state semantics and the data provenance log for artifact history. After supported non-health scope, authority to describe the collection, and a controlled codebook packet are established, the qualified appraisal request path may be a next preparation step. It does not host an ontology, relabel data, create a quote, match a buyer, issue a legal opinion, or grant a license.

Takeaways

  • Give concepts stable identities and keep preferred, alternative, and deprecated labels distinct.
  • Version definitions, hierarchy, examples, effective scope, and migration evidence together.
  • Represent split, merge, retired, and unresolved mappings explicitly.
  • Use synthetic fixtures before exposing source annotations or private text.
  • Keep ontology evidence separate from accuracy, privacy, rights, and downstream-use decisions.

Sources

  1. W3C: SKOS Simple Knowledge Organization System Primer (opens in a new tab)
  2. World Wide Web Consortium: PROV-O: The PROV Ontology (opens in a new tab)

Use the appraisal path after the ontology packet is organized.

It does not host an ontology, relabel data, create a quote, match a buyer, or grant a license.