On this page
Evidence matrix
General information, not legal advice.
| Evidence | What to preserve | Review question |
|---|---|---|
| Episode | Start and end, task, scene, robot configuration, operator or policy version | Can another reviewer understand what happened? |
| Action | Command, target, timestamp, controller state, and response | Is the action linked to the observation and outcome? |
| Force and touch | Sensor name, frame, calibration, range, sample rate, and saturation | Are readings comparable across devices and sessions? |
| Vision and motion | Camera calibration, depth or pose source, frame timing, and occlusion notes | Can a model align the visual signal with the action? |
| Failure case | Failure label, trigger, recovery, safety boundary, and reviewer | Is the negative example defined rather than guessed? |
| Industrial sensor | Machine, sensor, unit, cadence, maintenance and fault context | Does a customer or equipment contract limit use? |
| Rights and access | Owner, connected-product terms, operator consent, vendor terms, and export scope | Who can authorize the licensed use? |
NIST’s robotic-hand performance work National Institute of Standards and Technology, (opens in a new tab) emphasizes calibration and comparison against known external loads. That is a technical measurement principle, not a guarantee that a sensor log is accurate. Keep calibration files, test conditions, and known missing intervals beside the data. If the collection mixes devices, write the device and firmware version into the manifest instead of treating all readings as one series.
Failure cases are first-class evidence
General information, not legal advice.
A successful pick or a normal machine cycle can be easy to describe. A failure needs more context. Record the intended action, observed deviation, safety response, operator intervention, and whether the episode is suitable for training, evaluation, or research only. Do not label an episode “safe” merely because no injury appears in the file. Safety conclusions need a qualified review and the relevant operational records.
For industrial data, a public dataset may document faults without granting rights in a private plant. The Data Act Official Journal of the European Union, (opens in a new tab), dated December 22, 2023, recognizes access to certain data generated by connected products and services while preserving restrictions for security and personal-data law. Read customer, equipment, service, and worker agreements before exporting a field collection. A platform user, equipment maker, operator, and employer can have different interests.
Fictional scenario
Imagine a warehouse robot collection with synchronized camera video, joint commands, wrist force, and operator corrections. Some episodes use a loaned gripper, some have missing calibration files, and the operator contract limits secondary use. The safe review splits the episodes by device and rights status, marks missing calibration as a limitation, and keeps a failure taxonomy with example IDs. It can offer a metadata manifest and a redacted sample for technical discussion. It cannot claim that every episode is ready for training or that combining all channels creates automatic added value.
Owner worksheet
- Define the episode and unit. State whether a unit is an episode, trajectory, hour, machine-year, or another source-defined unit.
- Map every signal to a timestamp and device identifier.
- Attach calibration, sensor range, missingness, and failure notes.
- Separate normal, recovery, and safety-critical episodes.
- List customer, equipment, operator, and vendor rights.
- Choose a metadata-only manifest, redacted sample, or administrator-approved export.
- Cite the public paper or regulation as context, not as a claim about your collection.
Sources
- National Institute of Standards and Technology: Performance Metrics and Test Methods for Robotic Hands, Draft NIST Special Publication 1227 (opens in a new tab)
- arXiv: FactoryNet: A Large-Scale Dataset toward Industrial Time-Series Foundation Models (opens in a new tab)
- Official Journal of the European Union: Regulation (EU) 2023/2854 of the European Parliament and of the Council of 13 December 2023 on harmonised rules on fair access to and use of data and amending Regulation (EU) 2017/2394 and Directive (EU) 2020/1828 (Data Act) (opens in a new tab)
- National Institute of Standards and Technology: Standard Metrics and Test Methods for Assessing the Performance of Grasping-Type Robot End-Effectors, standards study group slides (opens in a new tab)
- Scale AI: Expanding Our Data Engine for Physical AI (opens in a new tab)