Off-the-shelf vs custom training data: a 2026 head-to-head comparison

Compare the same required observations, permitted use and accepted units. The table condenses the decisions developed below.

Re-verify prices, terms, availability and source dates before relying on this comparison.

Start with whether the data will be usable, then what obtaining it commits each party to. Rights, control, coverage and delivery all matter before a supplier's track record can tell you much about fit.

Defined.ai offers existing datasets and custom collection. Shaip distinguishes existing audio licensing, custom collection and transcription or annotation. Compare the work in the proposal, even when both proposals carry the same logo.

1. Will the delivered data meet your acceptance criteria?

A recording can sound clear and still be wrong for your task. If the people, conditions or events you need are absent, consistent labels will not fix the missing observations. Conversely, if the observations are suitable and only the labels need changing, commissioning another recording project may solve the wrong problem.

A useful acceptance test has two layers. First, does the material contain the observation you need? Then, does its condition and labeling meet the agreed checks? Keep those decisions visible in the delivery record. Otherwise, a disagreement about missing coverage can turn into repeated relabeling of files that will never answer the task.

Ask the owner to describe known gaps alongside the sample. The data card template gives that context a place beside dates, formats and provenance. This also protects the scope of the owner's work: preparing a defined subset is a different commitment from promising that the buyer's model will improve.

Bottom line: Choose existing data when the offered material passes the task and quality checks. Choose collection when an essential observation is missing and the pilot demonstrates a workable way to obtain it. Keep annotation on the shortlist when the observation is already there.

Ask both: “Which checks turn a delivered record, hour or episode into an accepted unit, and what happens when a batch misses them?” A sample invitation without a full-delivery acceptance rule leaves the acceptance decision unresolved until delivery.

2. What will the usable data cost, and what are both parties committing to?

The cheapest quoted hour need not be the cheapest usable hour. The comparison changes if one proposal includes transcription, another excludes rejected recordings, or the licenses allow different uses. Owners have a separate calculation: the buyer's bill does not show what the owner keeps after completing the work attached to the offer.

The cited Shaip catalog page and Appen services archive do not supply comparable numeric quotes for a specified dataset, collection and license. A class-wide price range would conceal the very differences you need to price. Take one specification to both discussions and leave any unpriced work visibly unknown, rather than entering zero in the comparison.

For an existing archive, the owner should separate payment for granting rights from payment for any preparation they undertake. A collection proposal adds a different question: what must be produced before an invoice becomes payable? Recording a session, delivering files and having the buyer accept them can be separate events. Decide which event the agreement uses, including the treatment of corrections.

Rightmark's published fee wording says the buyer pays its fee on standard licenses; its share on premium and bespoke deals is agreed in writing. That explains one fee, not the owner's total costs or payment date. It cannot establish that a Rightmark offer leaves an owner more than a collection contract would.

Bottom line: Existing data avoids paying to commission observations you already have access to license. Custom collection can justify its production cost when those observations are missing. Choose on the complete, comparable proposal; owners should judge the agreed payment against their own duties, costs and retained use.

Ask both: “Give me the total for this usable quantity and license, what is excluded, what I owe if we stop, and what the owner or collector must do to be paid.”

3. Who can authorize the data and its proposed use?

Having a file does not settle who can license it. An existing archive may contain material created for clients or supplied under earlier restrictions; a new recording may involve several parties whose permissions need to be arranged. The buyer needs authority for the intended use, and the owner needs a clear account of what the agreement lets others do.

For owners, the difficult question can be narrower than whether the archive is theirs: which part can be licensed for this use? A permission record may cover some recordings and leave others unresolved. Narrowing the offer to the supported subset can make the next discussion concrete without representing the whole archive as cleared.

New collection gives you the opportunity to arrange suitable permissions before the work begins. It still requires agreement about participants, recipients and future use. Have the relevant owner and counsel resolve those details for the actual material and jurisdictions; a provider's general consent statement cannot answer them for you.

Rightmark's buyer NDA restricts portal material to deciding whether to license it. Before sending a sample into a model or another service, obtain an answer about that specific test. An NDA and a sample download are not enough to infer permission for embeddings, training or benchmarking.

Bottom line: Favor the existing collection when its rights evidence supports your use. Favor new collection when the required observations can be gathered with suitable permissions that the available material lacks. Hold either proposal while authority remains disputed.

Ask both: “Who can grant each use we need, what evidence supports that authority, and what may each party retain or reuse when the agreement ends?”

4. Which requirements can you change, and who approves the change?

“Custom” is an incomplete description of the work. Selecting different records changes the subset; changing a camera position changes the observation being captured. Before agreeing to a change, the buyer and owner need to know which kind it is and what previously completed work it affects.

Write changes against the original requirement. If a buyer needs a different label definition, the owner should be able to tell whether the requested work applies to the same files. If the buyer needs an event recorded from another viewpoint, those files may no longer be enough. A new column in the delivery schema cannot supply a sensor measurement that was never captured.

The same discipline helps after a pilot. Record what changed, why it changed and whether earlier captures still qualify. Otherwise, a revised instruction can leave one party counting completed work while the other counts only work made under the new rule.

Bottom line: Choose existing material when selection and preparation can satisfy the requirement. Choose collection when an important variable must be controlled before capture. Put price, rights and acceptance consequences beside every approved change, so neither party silently acquires a larger job.

Ask both: “Which requirements can you meet by selecting or preparing existing records, which require new capture, and who approves the resulting changes to price, rights and delivery?”

5. Does the coverage match your people, conditions and time period?

A large dataset can still be missing the combination that matters. Language, speaker, device and environment should be considered together, not as separate checkmarks. The owner also needs to know whether the buyer wants a subset already held or a commitment to fill gaps with new material.

Historical coverage is a real advantage for existing data. Shaip's text catalog includes a historical news archive; a buyer still needs the dates and rights of the offered subset. New recording is useful for observing a new device or present-day behavior, but it cannot recover the original circumstances of an event nobody recorded.

For current coverage, turn the requirement into combinations that can be counted. If a task needs a particular interaction in a particular environment, knowing that each appears somewhere in a large collection does not show that they appear together. Ask for that intersection in the sample and proposed delivery. The same check belongs in a collection protocol before recruiting more participants.

Bottom line: Custom collection has the clear advantage when required people, equipment or conditions are missing and can be captured to an agreed protocol. Existing data wins when it already contains the needed observations, especially original historical material. An owner should commit either to a documented subset or to defined new capture work, with the distinction visible in the offer.

Ask both: “Show the coverage for every required combination, which gaps remain, and whether filling each gap means finding existing records or producing new ones.”

6. Who does the work before and after delivery?

Buying data does not finish the work of making it usable. Existing material can remove the need to organize capture, while managed collection can supply people and equipment your team lacks. Both proposals still need a clear account of preparation, review, corrections and integration.

Distinguish who does a task from who pays for it. An owner may need to explain its source systems or find permission records even where someone else funds technical preparation. A buyer may commission collection yet still need its own reviewer to approve the pilot. Those responsibilities belong beside the proposed deliverables, so neither party plans around work the other has not agreed to do.

The handoff should also have a test. Ask the receiving team to check a sample export against its expected structure before the full package arrives. Agree how malformed records, missing metadata and version mismatches will be reported. Successful file transfer proves arrival; acceptance requires the checks you agreed in the first dimension.

Bottom line: Favor existing data when it removes unnecessary capture and the remaining preparation is manageable. Favor managed collection when missing observations require a workforce or setup you cannot provide. If preparation is the whole gap, buy that work against the available material.

Ask both: “Name who owns each step from source or capture through preparation, acceptance and integration, and which steps remain with our team or the data owner.”

7. What evidence shows that the route fits this task?

A supplier's case study is useful when you can identify the job it demonstrates. A collection project, a labeling project and a catalog license answer different questions. Owners should be equally precise about the difference between an inquiry, completed work, accepted delivery and payment.

Defined.ai's robotics account describes synchronized recordings for an unnamed customer, with calibration and collection guidelines and weekly delivery reports. The useful evidence is the method for keeping sensor captures consistent. It gives a buyer specific work to examine in a pilot; it does not establish performance for a different model or a standard delivery timetable.

Appen's Cohere case concerns PANDA Plus: human preference judgments, editing preferred responses and feedback. That is relevant when the job requires structured human judgments about model output. It is not evidence that a catalog dataset was licensed or a physical scene recorded, even though all three activities may be described broadly as acquiring training data.

Shaip's ASR account reports 8,000 audio hours collected and 800 hours transcribed. It also describes batch rejection and replacement rules for that project. The different quantities make the deliverable question concrete: a collection total is not a transcript total, and those project rules are not default terms for a new order.

Rightmark's published speech note provides a different kind of evidence. It describes an open buyer request with detailed speech requirements and, separately, a submitted archive still needing permission evidence and sample review. These are two situations, not a matched transaction. They show why owners need to connect an archive's contents and permissions to a specific request before treating interest as an offer; neither establishes an accepted delivery or payment.

Bottom line: Existing data earns the choice through the offered material and a permitted test. Collection earns it through a relevant capture method and a pilot against your requirement. Read each example for the work it demonstrates, then ask for the evidence missing from your own proposal.

Ask both: “Show a comparable job and distinguish what was licensed, collected, labeled, accepted and paid for; which parts of that evidence apply to this requirement?”

An observation can already exist without being available in a public catalog. Rightmark's role in this decision is sourcing eligible private owner data to a written requirement. Its published process gives buyers and owners several points at which to establish what is offered before a full license and delivery.

• A written scope for existing private data. Rightmark's buyer process begins with the requirement and a response about what it can source, then records families, filters, formats, volume and cadence in writing. The buyer can assess actual proposed coverage, while the owner can identify relevant material and the preparation it needs; a request does not establish availability or an offer.

• An ownership pack based on the owner's evidence. The pack is built from owner documents and reports, follows the data route and is shared with the buyer's counsel under NDA. It gives the buyer a basis for reviewing the proposed rights and the owner a way to show exclusions and limits; the pack itself is not legal clearance.

• A controlled sample before the full license. The sample-pack checklist starts with a review question, defines scope, masks material, adds a data card, records approval of the exact release and documents transfer. This lets the buyer inspect format and known gaps while the owner approves the sample, recipient, purpose and path; permission for model-based testing and full-batch acceptance still need their own answers.

• Use and delivery terms before licensing. The published signing step sets use, scope, term, exclusivity, delivery and deletion, with the owner signing before anything is licensed and a licensing partner chosen per deal on code routes. Buyers should inspect the actual permitted-use and derived-data clauses, while owners weigh retained use, preparation duties and payment conditions together; signing is not a promise of approval before every later code license.

• Delivery through approved systems and receipt checks. Rightmark describes admin-approved, revocable connections or uploaded exports, with route-specific code-partner delivery, metadata and an ownership pack, followed by the buyer's scan on receipt. The buyer has a defined package to check and the owner has an access or export path to prepare, while acceptance criteria, corrections and the payment event remain separate agreed matters.

Choose an existing dataset when the actual material covers the people, conditions and period you need, the license permits your use and the remaining preparation is acceptable. A representative sample should connect to the proposed delivery. You should be able to say what you will receive after exclusions, not merely how much the catalog advertises.

There are two particularly strong reasons to take this route. One is that the observations are right and the remaining job is bounded: conversion, a different label scheme or a defined amount of transcription. Price that work before commissioning the observations again. The other is that you need genuine records from a past period. New collection cannot recreate the original event or system state, even if it can produce a useful simulation of similar conditions.

This choice still needs a delivery date and a complete quote. Existing material does not mean immediate access, training permission or no preparation. But when you can inspect the relevant data and define the work left to do, you have a concrete basis for deciding whether a collection proposal adds anything you need.

For an owner, licensing an archive makes sense when the rights, documentation and preparation obligations fit the offer. You do not need to become a recording operator simply because a buyer uses the word custom. Explain what already exists, what you can authorize and what work you will undertake. Keep the amount retained and payment event separate from any appraisal: Rightmark's appraisal path provides an indicative estimate, not an offer.

Choose collection when the missing observations matter and cannot be obtained by selecting, cleaning or labeling the material you have reviewed. The case is strongest when you need people or recording conditions absent from the available data, a new device setup, or a particular combination of events and metadata. Being able to decide what gets observed is a substantial benefit in those situations.

Make the production commitment specific enough to test. Define the protocol, permissions, coverage and acceptance checks, and give someone on your team responsibility for approving the pilot and resolving changes. A collection that follows an unsuitable instruction faithfully can still miss the task. Pilot review is where you find that mismatch before paying to repeat it at volume.

Specify the outputs separately. If you need recordings, transcripts and speaker labels, make each a deliverable with its own checks. Decide what happens if the raw recording is accepted but its transcript needs correction, or if a missing cohort prevents the batch from meeting the agreed coverage. The value of collection is getting the observations you need, not a promise that every recorded unit will qualify.

For an owner or production supplier, this route fits when the work being offered is work you can and want to perform. Price the capture and permission requirements, approvals, corrections and permitted reuse against the payment terms. An attractive project budget does not tell you how much of that work falls to you or when you will be paid.

Consider a third option if

• The observations exist privately, but the catalogs you reviewed do not cover them. Add private-asset sourcing to the search before assuming you need to create the data. Rightmark's written scope, ownership-pack and sample process can support that discussion; availability, authority and an acceptable license still need to be established.
• The observations exist and preparation is the whole job. Scope cleaning, conversion, transcription or annotation against those records, with a test for the finished deliverable. For owners, granting rights to the source and doing that preparation are separate contributions to price and agree.
• Existing data covers only part of the requirement. Consider licensing that portion and commissioning the missing observations. Separate the permissions, accepted units and costs of each piece so the buyer and contributing owners can tell which obligations belong where; this does not assume that every provider offers a combined package.

Keep either transaction on hold if authority or intended use is unresolved. A different supplier or a new recording does not settle permission for the material and use actually proposed.

Relevant reading: Rightmark vs Defined.ai: compare data licensing routes addresses the provider decision once the required data and work are clear.

License existing data when its observations, permitted use and remaining work fit the task. Commission collection when missing observations justify a production plan you can specify and test. Keep private sourcing and preparation in the discussion when those are the jobs you actually need.

Take the Ask both questions into the next conversation with one specification, intended use and acceptance test. Buyers need a complete proposal for usable data. Owners need to identify what they would contribute, the rights they can grant, the preparation they would undertake and the conditions for payment. Those answers make the choice possible without assuming a price, quality or income winner.

Start with the data you need.

Describe the existing private data your team needs, its intended use and the volume you want. Rightmark replies with what it can source, volumes and timing.

Request data