Import tabular data

WarningContent review status

This page contains unreviewed content created with large language model (LLM) tools.

write_table(name, data, *, dictionary=None, roundtrip=...) registers a typed relation from a CSV file, TSV file, or in-memory data frame. tensorbids defines relation semantics; bitsio persists the relation with its Parquet table engine. Open the dataset in writable mode first:

from bitsio import BITSZarrDataset

ds = BITSZarrDataset.open("ds000114.bits", read_only=False)
print(ds.phenotype_instruments(), ds.splits())
[] ['folds']

1 Name the table

Name Table Keyed by
"participants" participants.tsv subject
"phenotype/<instrument>" phenotype/<instrument>.tsv subject (and session when present)
"split/<name>" a named subject partition, such as cross-validation folds subject

During BIDS table normalization, participant_id and session_id become subject and session; their sub- and ses- prefixes are removed.

2 From a CSV or TSV

For a delimited text file, the writer infers column types from cell values. Empty cells and n/a are treated as missing:

from pathlib import Path

Path("scores.csv").write_text(
    "participant_id,score,site\nsub-01,101.5,001\nsub-02,,002\nsub-03,97,001\n"
)
written = ds.write_table("phenotype/scores", "scores.csv")
print(written.write.dtypes)
print(ds.phenotype("scores"))
{'subject': 'string', 'score': 'float64', 'site': 'string'}
shape: (3, 3)
┌─────────┬───────┬──────┐
│ subject ┆ score ┆ site │
│ ---     ┆ ---   ┆ ---  │
│ str     ┆ f64   ┆ str  │
╞═════════╪═══════╪══════╡
│ 01      ┆ 101.5 ┆ 001  │
│ 02      ┆ null  ┆ 002  │
│ 03      ┆ 97.0  ┆ 001  │
└─────────┴───────┴──────┘

Values with significant leading zeros, such as 001, remain strings. To set a type explicitly, supply Dtype in the data dictionary or construct an in-memory frame with the required schema.

3 From a data frame

A Polars or pandas data frame, or an Arrow table, retains its column types. The writer records those types as Dtype entries in the stored data dictionary:

import polars as pl

frame = pl.DataFrame({"subject": ["01", "02", "03"], "score": [101.5, None, 97.0], "site": ["001", "002", "001"]})
print(ds.write_table("phenotype/scores", frame).write.dtypes)
{'subject': 'string', 'score': 'float64', 'site': 'string'}

Writing the same registered relation again replaces its contents and increments its version. When called through a workspace view, the write is isolated in that workspace.

4 Describe the columns

Use dictionary to provide BIDS data-dictionary entries. Supplied fields override inferred metadata for the named columns:

ds.write_table(
    "phenotype/scores",
    "scores.csv",
    dictionary={
        "score": {"Description": "Task score", "Units": "points"},
        "site": {"Description": "Scanner site"},
    },
)
print(ds.phenotype("scores").columns)
['subject', 'score', 'site']

The dictionary is stored as the relation sidecar and is emitted as JSON by export_bids(). Declare categorical levels explicitly; they are not inferred.

5 Use the tables

As fields of a record set:

from tensorbids.query import BIDSReference

records = ds.records(BIDSReference(suffix="T1w"), "phenotype/scores.score", "split/folds.fold")
print(len(records), records.table.columns)
print(len(records.where(fold=0)))
3 ['subject', 'T1w', 'phenotype/scores.score', 'split/folds.fold']
2

With missing="drop" (the default), the record set retains only subjects that have every requested component. A bare column name is accepted by where() when it is unambiguous. Use query_relation() to query a relation without joining it to images.

6 Current limitations

  • export_bids() does not export split relations.
  • The table write path does not currently enforce the dataset’s read-only setting. Open the dataset with read_only=False before writing and do not rely on the current permissive behavior.

7 See also