On this page
Walk the hierarchy and preserve exact dataset paths
Start with a neutral artifact ID, digest, superblock or reader context, and a list of groups, datasets, named datatypes, attributes, and links. An HDF5 path is part of the data contract. Record whether a path is a hard link, soft link, external link, or unresolved link. A friendly display name is not a substitute for the exact path and link target.
| Hierarchy field | Record | Stop if |
|---|---|---|
| Object path | Full path and object kind | Name is normalized without the source path |
| Link type | Hard, soft, external, or unresolved | Target cannot be resolved in the declared boundary |
| Dataset identity | Object reference or neutral ID | Two paths are treated as two datasets without checking links |
| Group attributes | Names, types, and redacted values | Attributes contain unbounded private text |
| External file | Referenced location and availability state | Reader follows it outside the review scope |
The The HDF Group, (opens in a new tab) documentation gives the hierarchy and object model context. Keep path inventory separate from any decision about whether an object should be shared. Metadata can disclose project, person, location, or processing details even when no array values are read.
Compare dataspace, rank, dimensions, and datatype
For each dataset, record rank, current dimensions, maximum dimensions, dimension scales if present, datatype class and size, byte order, signedness, string representation, compound members, and references. Keep the declared shape and any observed read shape as separate columns. A rank of zero, an empty dimension, and a missing dataset are different states.
| Dataset fact | Safe observation | Open question |
|---|---|---|
| Rank | Integer rank or scalar | Does the consumer expect an array? |
| Current dimensions | Ordered dimension sizes | Which axis is time, space, or category? |
| Maximum dimensions | Fixed or extendible declarations | Can later writes change coverage? |
| Datatype | Class, size, members, and byte order | Does the reader preserve all members? |
| String fields | Fixed or variable length, encoding state | Are empty and missing distinct? |
| Dimension scales | Attached scale paths or absent | Are scale values aligned and documented? |
Synthetic dataset record
{
"path": "/fixture/values",
"rank": 2,
"shape": ["fixture_rows", "fixture_columns"],
"max_shape": ["unlimited", "fixture_columns"],
"dtype": "fixture-float",
"state": "metadata only"
}This record intentionally uses symbolic dimensions. It is enough to test whether a worksheet preserves rank and extensibility without copying a private array. If a datatype contains compound members or references, list each member and the representation boundary rather than flattening it into a generic string.
Review layout, chunk shape, and filter pipeline
The The HDF Group, (opens in a new tab) documentation lists compact, contiguous, and chunked layouts. The The HDF Group, (opens in a new tab) documentation explains that chunked storage divides a dataset into chunks and supports compression and other filters. Record layout and chunk dimensions before any performance statement. A chunk shape can affect I/O behavior, but it does not prove that a consumer will be fast or that values are complete.
| Storage field | Record | Stop if |
|---|---|---|
| Layout | Compact, contiguous, chunked, or unknown | Reader reports a fallback layout |
| Chunk shape | Dimensions per chunk | Chunk rank differs from dataset rank |
| Filters | Order, IDs, options, and availability | Filter is unknown or unavailable |
| Fill allocation | Fill state and value reference | Fill semantics are reader-specific and unrecorded |
| External storage | File list, offsets, and lengths | External target is outside scope |
| Allocation | Allocated, sparse, or unknown | Physical state is mistaken for logical row count |
Do not claim lossless or complete data merely because a filter name is present. Record filter metadata and the reader’s ability to decode a synthetic fixture. If a filter cannot be decoded, hold the dataset rather than silently skipping it.
Separate attributes, fill rules, and coverage
Attributes can describe units, labels, coordinates, fill values, scale, offset, provenance, or application-specific fields. Record name, datatype, shape, and a bounded redacted value summary. Preserve the exact value when a controlled reviewer needs it, but keep public worksheets at field level. An attribute named units is evidence of a metadata field, not proof that the values use that unit correctly.
| Review question | Record | Interpretation boundary |
|---|---|---|
| Units and labels | Exact attribute names and normalized labels | Do not infer semantics from a label alone |
| Fill or missing | Typed value and scope | Do not turn fill into zero |
| Scale and offset | Stored and intended domains | Do not discard packed evidence |
| Provenance | Source reference and transform note | Do not treat a note as rights evidence |
| Coverage | Dimension scope, exclusions, and version | Shape is not completeness |
Stop conditions
- Stop if the dataset path or link target is unresolved, especially for external links.
- Stop if rank, dimensions, datatype, or compound members conflict between metadata sources.
- Stop if chunk dimensions, filters, or external storage cannot be decoded in the controlled boundary.
- Stop if attributes hide the fill, packing, unit, or coordinate rule needed for interpretation.
- Stop before reading private arrays when the handling boundary or owner authority is unknown.
Close with a metadata packet and a qualified next step
Keep the path inventory, dataset worksheet, filter evidence, reader version, synthetic fixture result, and unresolved questions together. Pair this guide with the data provenance log for source and transformation history and the data card template for collection context. A readable HDF5 hierarchy does not establish accuracy, completeness, privacy clearance, or rights.
For a supported non-health collection, after the owner has authority to describe the scope and a controlled metadata packet, the qualified appraisal request path may be a preparation step. It does not read HDF5 arrays, decode filters, assess data quality, create a quote, match a buyer, issue a legal opinion, or grant a license.
Takeaways
- Inventory exact paths and link targets before reading datasets.
- Keep rank, dimensions, maximum dimensions, datatype members, and dimension scales distinct.
- Record layout, chunk shape, filters, fill state, and external storage without making performance claims.
- Use symbolic synthetic fixtures and hold unresolved links or filter boundaries.