bitsio
High-throughput I/O and cloud-native data storage through BITS
This page contains unreviewed content created with large language model (LLM) tools.
bitsio is a library that combines BIDS dataset logic with performant tensor storage. It operationalises a dataset surface that aims to be simultaneously:
- interoperable with the BIDS ecosystem and
- suitable for direct consumption by high-throughput use cases like foundation model training.
The Brain Imaging Tensor Structure (BITS) achieves this by separating the scientific semantics of a dataset from its physical storage layout on disk: content remains BIDS-addressable while storage is tensor-first and cloud-native. The bridge linking the logical layer to physical substrates is a program called the resolver, which interprets DAG-shaped recipes called dataset navigators to serve virtual BIDS files from a high-performance, modern storage backend.
bitsio is a concrete implementation of the storage and resolver interfaces defined by the Brain Imaging Tensor Structure (BITS). It stores numerical data as chunked Zarr arrays, auxiliary content (like headers and metadata) in content-addressed storage, and relations and tabular data as Parquet. Its reference resolver reconstructs BIDS files, image objects, or numerical arrays from those components.
tensorbids defines the BITS data model and the interfaces that storage backends, table engines, and resolvers implement. bitsio implements those interfaces. Integrations such as interoperability layers served by filesystem mounts and MONAI adapters belong to bitsfuse.
Use bitsio when a library needs BIDS-compatible dataset semantics together with efficient array reads for analysis, training, or inference. Start with the tutorial.
bitsio is part of the hypercoil federation.