Control stored numerical precision

WarningContent review status

This page contains unreviewed content created with large language model (LLM) tools.

Use exact narrowing during ingestion when the source values admit a smaller representation. Use explicit affine quantization when a bounded numerical error is acceptable and should be represented in the file’s fidelity contract.

1 Narrow exactly at ingest

auto_create and write_source take narrowing=:

narrowing= Looks at Stores
"header" (default) the header only the source dtype, with the header’s scaling as stored
"range" every value the smallest integer dtype that holds them, if all are finite whole numbers
"step" every value int codes on a uniform step (at most 65,536 distinct values), if they rebuild every value bit for bit; else as "range"

All three modes preserve numerical values exactly. If the requested analysis cannot find a smaller exact representation, ingestion retains the original representation.

from bitsio import BITSZarrDataset
from bitsio.example_data import fetch_ds000114

for narrowing in ("header", "range", "step"):
    _, report = BITSZarrDataset.auto_create(fetch_ds000114(), f"{narrowing}.bits", narrowing=narrowing)
    print(narrowing, [(s.suffix, s.dtype) for s in report.strata])
header [('T1w', 'float32'), ('bold', 'int16')]
range [('T1w', 'float32'), ('bold', 'int16')]
step [('T1w', 'float32'), ('bold', 'int16')]

This dataset does not narrow further: its T1-weighted values are continuous floating-point data, and its BOLD values are already int16. Label maps, masks, and integer-valued data stored in a floating-point container are more likely to benefit.

2 Quantize values to int8

import numpy as np
from tensorbids.authoring import AffineQuantization


class Volume:
    def __init__(self, data, affine):
        self.data, self.affine, self.shape = data, affine, data.shape


ds = BITSZarrDataset.open("ds000114.bits", read_only=False)
(image,) = ds.read(subject="01", suffix="T1w", form="object")
t1w = image.get_fdata(dtype="float32")
volume = Volume(t1w, image.affine)
rung = AffineQuantization.spanning(t1w.min(), t1w.max(), "int8")
target = "sub-01/ses-test/anat/sub-01_ses-test_desc-int8_T1w.nii"
ds.write_volume(
    volume,
    target_path=target,
    geometry=ds.default_geometry(volume, target, dtype="int8"),
    precision=rung,
)
ds.index()
(values,) = ds.read(target_path=target, form="tensor")
print(values.dtype, float(np.abs(values - t1w).max()) <= rung.scale / 2)
(row,) = ds.query(target_path=target).table.select("contract_level").to_dicts()
key = ds.lineage_table().filter(ds.lineage_table()["target_path"] == target)["store_key"][0]
print(row["contract_level"], ds.lineage(key).lossy)
float32 True
lossy True

AffineQuantization.spanning(low, high, dtype) maps the minimum and maximum integer codes to low and high. Values are rounded to the nearest code, so the maximum quantization error is half of one scale step. For a four-dimensional volume, pass values in (time, i, j, k) order.

3 Read values or codes

Normal reads return dequantized values. To retrieve the stored codes together with their scale and offset, select the stored port and attach the dequantization parameters:

from tensorbids.query import BIDSReference

codes = ds.records(BIDSReference(suffix="T1w", desc="int8", port="stored", dequantize="attach"))
(c,) = codes.read([0])["T1w"]
print(c.q.dtype, c.q.shape, c.dtype, np.allclose(c.q * c.scale + c.offset, values, rtol=1e-5))
int8 (256, 156, 256) float32 True

In records(...).loader(), a Codes value is collated into three fields: T1w for the int8 codes, T1w.scale, and T1w.offset. Lazy windows retain the scale and offset of their source image.

4 Constraints

  • A reconstructed NIfTI contains dequantized float32 values, not int8 values encoded with scl_slope and scl_inter.
  • write_array() stores the supplied data type without creating an image fidelity contract. Do not cast to a lossy data type before calling it. Use a container writer such as write_volume() with an explicit precision rung.

5 See also