pytesdaqx 0.2.3__tar.gz → 0.2.4__tar.gz

{pytesdaqx-0.2.3 → pytesdaqx-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pytesdaqx
-Version: 0.2.3
+Version: 0.2.4
 Summary: TES detector acquisition, instrument control, and raw-data recording
 Author: TESSERACT DAQ developers
 Classifier: Development Status :: 3 - Alpha
@@ -12,7 +12,7 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: Scientific/Engineering :: Physics
 Requires-Python: <3.13,>=3.11
 Description-Content-Type: text/markdown
-Requires-Dist: numpy<2.0,>=1.26
+Requires-Dist: numpy>=1.26
 Requires-Dist: PyYAML
 Requires-Dist: h5py>=3.10
 Requires-Dist: matplotlib

{pytesdaqx-0.2.3 → pytesdaqx-0.2.4}/docs/user/cli/data_inspection.md

@@ -142,3 +142,52 @@ pytesdaqx-build-manifest --acquisition-dir acquisition_I2_D20260727_T120000 --dr
 Do not routinely rewrite the manifest of an active acquisition. Work on a completed copy unless a documented recovery procedure explicitly says otherwise.
 
 The command summary reports the singular acquisition `storage_format` (`zarr` or `hdf5`) and acquisition timing when available, including `fridge_run`, `fridge_run_start`, `acquisition_start`, and the first/last `stream_start`. When rebuilding a Zarr manifest, missing acquisition-wide timing fields in individual stream stores are filled from the top-level acquisition `zarr.json`; each stream's own `stream_start` remains resource-specific. Older pytesdaqx/pytesdaq metadata using `group_*`/`series_*` and `group_start`/`series_start` is accepted on input and converted to the canonical acquisition/stream schema in the rebuilt manifest.
+
+## `pytesdaqx-repair-readout`
+
+Readout constants come from `run_setup.yaml` and are stored with every stream.
+A channel recorded without them cannot be converted to physical units
+afterwards, which stops processing with an error such as
+`Cannot convert rigolTTL to amps: missing/invalid close_loop_norm=nan`.
+This happens for a channel configured with an ADC assignment but no
+`readout_circuit`, typically a TTL or other monitor line that has no SQUID
+path.
+
+Use this command to write the missing constants into the stored channel map of
+an existing acquisition. Sample data are never modified.
+
+```bash
+# Report what would change (default)
+pytesdaqx-repair-readout /data/raw/calibration_I2_D20260924_T152603.zarr \
+    --channel rigolTTL --set close_loop_norm=1
+
+# Write the change
+pytesdaqx-repair-readout /data/raw/calibration_I2_D20260924_T152603.zarr \
+    --channel rigolTTL --set close_loop_norm=1 --apply
+```
+
+Only streams that recorded the channel are updated. In a mixed acquisition the
+IV, dIdV, and background streams usually do not contain an auxiliary channel,
+and are left untouched; the summary reports how many streams were skipped for
+that reason. Repeat `--set` for several constants. `--set` accepts the readout
+keys listed in `pytesdaqx.io.metadata_compat.READOUT_SETTING_KEYS`; other keys
+are rejected, because readers would ignore them.
+
+For a monitor line digitized without a SQUID readout, `close_loop_norm=1` makes
+the "amps" conversion return the measured volts, which keeps threshold settings
+in the units the trigger was tuned against.
+
+This command repairs data already on disk. Add the same constants to the
+channel's `readout_circuit` in `run_setup.yaml` so future acquisitions record
+them directly:
+
+```yaml
+rigolTTL:
+  adc:
+    device: adc1
+    channel: 4
+  record_for: [calibration]
+  readout_circuit:
+    close_loop_norm: 1
+```
+

{pytesdaqx-0.2.3 → pytesdaqx-0.2.4}/docs/user/cli/index.md

@@ -18,6 +18,7 @@ See [Acquisition and validation](acquisition.md).
 | `pytesdaqx-display-metadata` | Display acquisition, stream, and legacy HDF5 metadata |
 | `pytesdaqx-view-traces` | Plot continuous partitions or finite traces |
 | `pytesdaqx-build-manifest` | Rebuild or inspect acquisition manifest information |
+| `pytesdaqx-repair-readout` | Write missing readout constants into recorded metadata |
 
 See [Data inspection](data_inspection.md).
 

{pytesdaqx-0.2.3 → pytesdaqx-0.2.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pytesdaqx"
-version = "0.2.3"
+version = "0.2.4"
 description = "TES detector acquisition, instrument control, and raw-data recording"
 readme = "README.md"
 requires-python = ">=3.11,<3.13"
@@ -21,7 +21,7 @@ classifiers = [
   "Topic :: Scientific/Engineering :: Physics",
 ]
 dependencies = [
-  "numpy>=1.26,<2.0",
+  "numpy>=1.26",
   "PyYAML",
   "h5py>=3.10",
   "matplotlib",
@@ -82,6 +82,7 @@ pytesdaqx-validate-config = "pytesdaqx.cli.validate_config:main"
 pytesdaqx-display-metadata = "pytesdaqx.cli.display_metadata:main"
 pytesdaqx-view-traces = "pytesdaqx.cli.view_traces:main"
 pytesdaqx-build-manifest = "pytesdaqx.cli.build_manifest:main"
+pytesdaqx-repair-readout = "pytesdaqx.cli.repair_readout:main"
 [tool.pytest.ini_options]
 testpaths = ["tests"]
 python_files = ["test_*.py"]

pytesdaqx-0.2.4/pytesdaqx/cli/repair_readout.py

@@ -0,0 +1,168 @@
+"""Write readout-circuit constants into an already recorded acquisition.
+
+Readout constants come from ``run_setup.yaml`` and are stored with each
+stream. A channel recorded without them -- a TTL or other monitor line with
+no SQUID path, or a detector channel whose ``readout_circuit`` was left out
+of the run setup -- cannot be converted to amps afterwards, which stops
+processing. This command writes the missing constants into the stored
+channel map so existing data can be processed. Sample data are never touched.
+
+Only streams that actually recorded the channel are changed: the IV, dIdV and
+background streams of a mixed acquisition normally do not contain an
+auxiliary channel, and are left alone.
+
+Example, for a TTL monitor line digitized without a SQUID readout, where the
+"amps" conversion should just return the measured volts::
+
+    pytesdaqx-repair-readout /data/raw/calibration_I2_D20260924_T152603.zarr \\
+        --channel rigolTTL --set close_loop_norm=1 --apply
+
+The same constants belong in ``run_setup.yaml`` under the channel's
+``readout_circuit`` so that future acquisitions record them directly.
+"""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+from typing import Any
+
+from pytesdaqx.io.acquisition_catalog import AcquisitionCatalog
+from pytesdaqx.io.metadata_compat import READOUT_SETTING_KEYS
+from pytesdaqx.io.zarr_utils import read_attrs, write_attrs
+
+CHANNEL_MAP_KEY = "channel_map"
+READOUT_KEY = "readout_circuit"
+
+
+def parse_setting(text: str) -> tuple[str, Any]:
+    """Parse one ``key=value`` setting, keeping numeric values numeric."""
+    key, sep, raw = text.partition("=")
+    key, raw = key.strip(), raw.strip()
+    if not sep or not key:
+        raise argparse.ArgumentTypeError(f"--set expects key=value, got {text!r}")
+    if key not in READOUT_SETTING_KEYS:
+        raise argparse.ArgumentTypeError(
+            f"Unknown readout setting {key!r}. Readers ignore unknown keys, so "
+            f"writing one would have no effect. Valid keys: "
+            f"{', '.join(sorted(READOUT_SETTING_KEYS))}"
+        )
+    for cast in (int, float):
+        try:
+            return key, cast(raw)
+        except ValueError:
+            continue
+    return key, raw
+
+
+def stream_stores(acquisition: str | Path) -> list[tuple[str, Path]]:
+    """Return ``(stream id, store path)`` for every stream of the acquisition."""
+    catalog = AcquisitionCatalog(acquisition)
+    storage_format = str(catalog.storage_format or "").lower()
+    if storage_format != "zarr":
+        raise ValueError(
+            f"Only Zarr acquisitions can be repaired, got storage_format="
+            f"{catalog.storage_format!r}"
+        )
+    stores: dict[str, Path] = {}
+    for entry in catalog.entries:
+        store = catalog.resource_path(entry)
+        stores.setdefault(str(entry.get("stream_id") or store.stem), store)
+    return sorted(stores.items())
+
+
+def repair_stream(
+    store: Path, channel: str, settings: dict[str, Any], *, apply: bool
+) -> dict[str, Any] | None:
+    """Update one stream's channel map.
+
+    Returns the before/after values, or ``None`` when the stream does not
+    record ``channel``.
+    """
+    array_path = Path(store) / "raw"
+    attrs = read_attrs(array_path)
+    rows = attrs.get(CHANNEL_MAP_KEY)
+    if not isinstance(rows, list):
+        raise ValueError(f"{Path(store).name}: no stored channel map in {array_path}")
+
+    for row in rows:
+        if not isinstance(row, dict) or row.get("detector_channel") != channel:
+            continue
+        circuit = dict(row.get(READOUT_KEY) or {})
+        before = {key: circuit.get(key) for key in settings}
+        circuit.update(settings)
+        if apply:
+            row[READOUT_KEY] = circuit
+            write_attrs(array_path, attrs)
+        return {"before": before, "after": {key: circuit[key] for key in settings}}
+    return None
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        description=(
+            "Write readout-circuit constants into the stored metadata of a "
+            "recorded acquisition, so a channel recorded without them can be "
+            "converted to physical units. Sample data are never modified."
+        ),
+    )
+    parser.add_argument("acquisition", help="Raw pytesdaqx acquisition path")
+    parser.add_argument(
+        "--channel",
+        required=True,
+        help="Detector channel whose readout_circuit should be updated",
+    )
+    parser.add_argument(
+        "--set",
+        dest="settings",
+        required=True,
+        action="append",
+        type=parse_setting,
+        metavar="KEY=VALUE",
+        help=(
+            "Readout constant to write, for example close_loop_norm=1. "
+            "Repeat the option for several constants."
+        ),
+    )
+    parser.add_argument(
+        "--apply",
+        action="store_true",
+        help="Write the changes. Without it, only report what would change.",
+    )
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    settings = dict(args.settings)
+
+    stores = stream_stores(args.acquisition)
+    changed: list[tuple[str, dict[str, Any]]] = []
+    skipped: list[str] = []
+    for stream_id, store in stores:
+        result = repair_stream(store, args.channel, settings, apply=args.apply)
+        if result is None:
+            skipped.append(stream_id)
+        else:
+            changed.append((stream_id, result))
+
+    print(f"Acquisition: {args.acquisition}")
+    print(f"Channel:     {args.channel}")
+    print(f"Settings:    {settings}")
+    print(
+        f"{'Updated' if args.apply else 'Would update'} {len(changed)} of "
+        f"{len(stores)} stream(s); {len(skipped)} stream(s) do not record "
+        f"this channel."
+    )
+    for stream_id, result in changed:
+        print(f"  {stream_id}: {result['before']} -> {result['after']}")
+    if not changed:
+        print(f"  No stream records channel {args.channel!r}: nothing to do.")
+        return 1
+    if not args.apply:
+        print("\nRe-run with --apply to write these changes.")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())