A data cleaning budget needs a delivery specification

Consider a buyer who wants to follow an engineering issue through review to the implementation. A collection of issues, reviews and commits will not serve that task if the relevant links stop resolving. Rightmark's company data for agent training explains this use of connected records: ticket identifiers and commit references join the request, the review and the result. The preparation requirement follows from the particular task. Another buyer may want individual code files and have no need for the review history.

That distinction belongs in the work specification before an engineer chooses what to transform. “Preserve useful context” leaves the engineer guessing. “For each included issue in this specimen, the retained review must resolve to the intended implementation in the delivered repository” gives the engineer something to test. It also gives the buyer a way to explain why a broken link matters.

The exclusions need equal precision. In a November 13 owner preparation discussion, a Rightmark team member asked: “Were any channels shared with clients? It changes what we take out before delivery.” The question precedes the cleaning decision. A channel's presence in an export does not settle whether its contents belong in the proposed delivery.

Here is an invented specimen. All records, identifiers, versions, requirements and results below are fictional; the identifiers are labels, not real commit hashes. Assume the owner has excluded client thread C9. The agreed task concerns issue I17, its review R42 and the intended implementation. No finding about actual licensing rights is implied.

The exclusion check passes: C9 is absent. The relationship check fails: starting at I17 gets the reader to R42, then to an identifier missing from the delivered repository. Both results can be true. Removing the excluded thread has not proved that the retained task still works, and repairing the task would not prove that every exclusion was respected.

The correction in this example is specific. Use the recorded map to update the reference in R42, confirm that new-B is the intended retained implementation, then rerun the relationship check. Do not select a similar-looking commit and call it the replacement. Do not put C9 back to make the collection feel more complete. The exclusion remains a requirement while the broken eligible relationship is repaired.

The git-filter-repo manual makes a useful distinction here: its commit map records changed and unchanged commits, and can also record that a commit was removed entirely. A replacement identifier is therefore something to establish from the map, not assume for every old identifier. If the required implementation was removed, the specimen has a gap to resolve with the person defining the task. It does not have an automatic substitute.

This small failure is worth finding before the full run because it changes the work being commissioned. Someone must update the retained export and demonstrate the result. Merely supplying the cleaned repository would leave that work elsewhere. The buyer can now decide whether the proposed task survives the agreed exclusions, while the owner can decide whether preparing the remaining material is worth the effort.

Write the check against the delivery version. A review that opens correctly in the owner's original system says little about a copy whose identifiers have changed. Keep the source, the output, the map and the check result distinguishable. Otherwise a passing demonstration can be repeated against the wrong files while the package meant for delivery remains broken.

The specimen proves only the checks applied to that specimen. It does not establish that the rest of the archive is ready, that the data is anonymous or that a buyer has accepted a license. Its value is narrower and more useful at this stage: it exposes a necessary piece of preparation before that piece is repeated across the full scope.

“Help with cleaning” can describe very different commitments. In Rightmark's unsigned Owner Engagement Agreement v4, clause 2 distinguishes advice, tools and coordination from hands-on technical work. Specific deliverables and external costs require a separate written scope and pricing before the work begins. That is a draft distinction worth making explicit in a preparation discussion, rather than treating a broad assistance line as the finished work order.

The public service choices also put different people at the controls. Rightmark describes its archive Clean Room as owner-run early access, with Archive Check as the first job and the later jobs a pilot build. Its managed de-identification service describes a specialist team performing the work inside the owner's systems. Those descriptions help identify who would do the work. Neither, on its own, specifies responsibility for repairing the Git references in our example.

Before comparing preparation offers, put their deliverables beside each other. These are two invented scope lines, not vendor quotations:

The first line may be enough for a capable internal team. There is no reason to buy the second task twice if an owner engineer already has responsibility for it. Equally, a team without that capacity has not bought a usable delivery simply because it has obtained a tool. The important comparison is the work that remains after each offer, including work on the owner's side.

For an AI dataset buyer, the same table exposes a handoff risk. If the buyer expects a review export that resolves against the delivered repository, that expectation must appear in the scope. Leaving it until receipt turns an unstated requirement into a disagreement about whether delivery is complete. The buyer should specify the required relationship without prescribing unnecessary implementation work the owner may already have solved.

Then separate three situations that can otherwise become one argument over a bill. First, there is work already assigned: updating the eligible export in the second scope line. Second, there is a failure of an agreed check: the supplied export still points to old-A. Third, there is a new request: adding another source system or preserving a relationship the original work order did not include.

Each situation needs a decision about effort and payment. Who corrects a failed agreed check, who verifies the correction and whether it carries an additional charge depend on the actual agreement. A new request needs its own scope decision. Calling everything “rework” hides these differences and makes it harder for either party to judge a proposed charge.

I would settle four roles before approving the full run: the person doing the preparation, the person verifying the result, the party funding it and the person authorized to approve a change. One organization can hold all four roles. Naming them still matters because an engineer discovering a missing export needs a definite next step, not an invitation to keep working until someone objects to the hours.

If nobody yet knows how much work the archive needs, a fixed full-run budget may be premature. Commission a bounded inspection and specimen instead, with its own output and effort limit. The result should identify the unresolved work closely enough to make the next spending decision. “We looked at the data” is a weak deliverable; a failed relationship check, the proposed correction and the work still unestimated give the next decision substance.

There is a strong objection to requiring a specification before cleaning: sometimes you need to inspect and prepare a little data to discover what the specification should say. The reasonable objection is to a fully priced, final scope being demanded before anyone understands the source. It is not a reason to commission an unlimited full run.

Mainstream cleaning guidance already recognizes assessment. In IBM Think's explanation of data cleaning, Julie Rogers, Staff Writer, and Alexandra Jonker, Staff Editor, write: “Data cleaning typically begins with data assessment.” That is compatible with funding an initial investigation. A licensing work order adds a commercial decision about which discovered problems the proposed delivery actually needs solved.

There are also jobs whose justification survives the disappearance of the licensing opportunity. An exposed credential is the clearest example. GitHub's guidance says to revoke or rotate a secret first; doing so stops its use for access and may resolve the problem without a history rewrite. That security response should not wait for a buyer to define a delivery. It does not, by itself, establish that the repository is ready to license.

The same reasoning can apply to ordinary internal work. Take another invented example: a team needs a reporting copy in which timestamps use UTC. Assume the original records are retained, the source offsets are known and the team's existing checks confirm that the conversion preserves the same instants and required records. The team can judge that work against its reporting requirement now. A future buyer may need something different, but that does not erase the internal reason for doing it.

Those assumptions carry the recommendation. If an offset is missing, guessing it would not meet the stated requirement. If the proposed cleanup deletes records the internal report needs, the work no longer has the independent justification described. “It will probably be useful later” is much weaker than an existing purpose with known inputs and checks.

Ask whether you would fund the work if the licensing discussion ended tomorrow. If the answer is yes, name the independent requirement and evaluate the work against it. If the answer is no, make the buyer-dependent requirement explicit before commissioning the full transformation. A shared project can contain both kinds of work, but separating their reasons prevents a possible license from becoming the justification for every backlog item.

This also explains when a short specification is enough. A stable internal process may already define the source, transformation, tests and accountable team. Rewriting that knowledge into a long new document adds little. Refer to the existing version and record what the licensing delivery changes. The amount of documentation should follow the uncertainty; the need for an accountable decision remains.

A technical specification and a budget approval are most useful when they describe the same job. Rightmark's working engineering standard v0.1 connects a frozen source copy, remapped context, tests and the delivered package. It is a design document, not evidence of an implemented code-cleaning service. The useful planning principle is to carry the source version and the required checks through to the decision about the finished copy.

Return to our invented issue-and-review specimen. Suppose the reference has now been corrected, the specimen passes its two agreed checks and an owner engineer judges that the full run fits the allocation below. Source V1 remains unchanged. These are invented assumptions, not observed delivery results.

The following completed work order, W1, uses an arbitrary allowance of 12 engineer hours and 2 review hours. Those figures are not a price, a typical duration or an estimate for anyone's archive. The roles, authorization and decisions are fictional, and the work is assigned to the owner's own team.

This is an authorization to do one defined internal job. It identifies what the engineer can start, who can stop or expand the work and what evidence the technical lead needs at the end. It leaves no need to pretend that a specimen pass has already proved the whole delivery.

The effort limit is equally important when things go badly. Suppose the engineer reaches the allowance with a failed relationship check unresolved. The limit prevents more work being silently authorized; it does not convert the failed output into an acceptable one. The operations lead now has a concrete choice. More time, a narrower requirement and stopping are different decisions, and a narrower requirement must be recorded rather than disguised as a pass.

If an outside provider performs the work, replace the internal allocation with the actual agreed scope, price and correction terms. Do not copy the invented inclusion of correction time as a Rightmark service promise. The owner and provider need to settle those terms for their job; the buyer needs clarity about the delivery it is evaluating.

A useful final check is to hand the work order and proposed package to the verifier. Can they tell which source it came from, what must be absent, which relationships must work and what would cause them to withhold completion? Can the person approving spend tell which remaining work is already funded? If either answer depends on an unwritten conversation, resolve that dependency before the full run.

Start with the collections and gaps you can actually name. Rightmark's archive inventory worksheet provides a place to record them. Use that inventory as input to a delivery specification, then approve the work against the result you need to receive.