Explanation

WarningContent review status

This page contains unreviewed content created with large language model (LLM) tools.

These chapters explain the architecture implemented by bitsio, the interfaces it receives from tensorbids, and the operational trade-offs of the storage and resolver design.

• Architecture and library responsibilities: the BITS processing model and the boundaries between tensorbids, bitsio, and bitsfuse.
• Storage backends and array layout: tensor, auxiliary, and table storage; layout profiles; content hashes; and partial read cost.
• Persistent BIDS index and query semantics: indexed fields, entity filters, sidecar inheritance, and index freshness.
• Record assembly and batched loading: joins, lazy region reads, deterministic sampling, and PyTorch workers.
• Copy-on-write workspaces: overlay reads, conflict detection, commit, rebase, discard, and retained audit records.
• Data lineage and provenance: navigators, lineage journals, fidelity contracts, and W3C PROV export.