Inspect and purge discarded workspaces

WarningContent review status

This page contains unreviewed content created with large language model (LLM) tools.

By default, discard removes a workspace’s arrays and tables but archives its lineage journal, producer metadata, and workspace record under bits/overlays/_discarded/<id>-<attempt>/. Each discard of a workspace name creates a numbered attempt. Pass purge=True to discard() to retain no archive.

Create and discard two attempts that write the same target path with different thresholds:

import numpy as np
from bitsio import BITSZarrDataset

ds = BITSZarrDataset.open("ds000114.bits", read_only=False)
target = "sub-01/ses-test/anat/sub-01_ses-test_desc-thresholded_mask.nii"
keys = []
for threshold in (0.25, 0.5):
    ds.open_workspace("threshold")
    exp = ds.at("threshold", read_only=False)
    (t1w,) = exp.read(subject="01", suffix="T1w", form="tensor")
    mask = (t1w > threshold * t1w.max()).astype("uint8")
    keys.append(exp.write_array(target, mask))
    exp.discard()
print(len(keys), keys[0] != keys[1])
2 True

1 List them

for record in ds.discarded_workspaces():
    print(record.name, record.attempt, record.outcome, record.discarded_at is not None)
threshold 1 discarded True
threshold 2 discarded True

Each record includes the workspace name and overlay identifier, attempt number, open and discard times, and the content and history digests of its base dataset.

2 Read one

ds.discarded(name) opens the latest archived attempt for lineage reads. Use attempt to select an earlier archive:

latest = ds.discarded("threshold")
first = ds.discarded("threshold", attempt=1)
print(latest.record.attempt, first.record.attempt)
print([entry.kind.value for entry in first.history(keys[0]).entries])
print(first.lineage(keys[0]).target_path == target, latest.lineage(keys[0]))
print(len(first.events()), first.scopes())
2 1
['ingest']
True None
1 ['01']

The returned archive view exposes history(), lineage(), events(), scopes(), and lineage_table() over the archived journal and the base dataset. Discarded arrays are not available for numerical reads.

3 Compare attempts

def written(attempt):
    table = attempt.lineage_table()
    return table.filter(table["source_tree"] == "overlay").select("target_path", "content_hash")

first_rows, latest_rows = written(first), written(latest)
print(first_rows["target_path"].to_list() == latest_rows["target_path"].to_list())
print(first_rows["content_hash"].to_list() == latest_rows["content_hash"].to_list())
True
False

Both attempts wrote the same target path, but their content hashes differ.

4 Delete one for good

removed = first.purge()
print(len(removed) > 0, first.purge())
print([(r.name, r.attempt) for r in ds.discarded_workspaces()])
True ()
[('threshold', 2)]

purge() permanently deletes the archive. It requires a writable dataset and does not append a provenance event. A second call returns an empty tuple.

5 Constraints

  • On a store without deletion support, discard is logical: bytes and metadata remain at the workspace root, and its name cannot be reused until a deletion-capable store purges them.
  • The archive supports provenance inspection, not reconstruction of deleted array values.

6 See also