adduct 0.2.1__tar.gz

adduct-0.2.1/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,71 @@
+name: Bug report
+description: Adduct crashed, failed to read a dataset, or did something clearly wrong.
+title: "[bug] "
+labels: ["bug"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for reporting. The three version fields below are required because without
+        them almost every bug report is unactionable — we cannot tell a fixed bug from a
+        live one, or a format difference from a code difference.
+
+        **If adduct flagged something that is actually fine**, that is a false positive, not
+        a bug — please use the *False positive* template instead. It goes into detector
+        calibration rather than the bug queue.
+
+  - type: textarea
+    id: what-happened
+    attributes:
+      label: What happened?
+      description: What did you run, what did you expect, and what did you get instead?
+      placeholder: |
+        I ran `adduct scan ./my_data` and it exited with a traceback instead of a report.
+    validations:
+      required: true
+
+  - type: input
+    id: dataset
+    attributes:
+      label: Dataset
+      description: >
+        The Hugging Face repo id if it is public (e.g. `lerobot/pusht`) — that lets us
+        reproduce it exactly. If it is private, describe the format and layout instead.
+      placeholder: "lerobot/pusht — or: private LeRobot v2.1, 340 episodes, SO-101, 2 cameras"
+    validations:
+      required: true
+
+  - type: input
+    id: adduct-version
+    attributes:
+      label: Adduct version
+      description: "Output of `adduct --version`"
+      placeholder: "adduct 0.1.0"
+    validations:
+      required: true
+
+  - type: input
+    id: python-version
+    attributes:
+      label: Python version and OS
+      description: "Output of `python --version`, plus your OS"
+      placeholder: "Python 3.11.9, Ubuntu 22.04"
+    validations:
+      required: true
+
+  - type: textarea
+    id: output
+    attributes:
+      label: Full output
+      description: >
+        The complete terminal output, including any traceback. This is automatically
+        formatted as code, so no backticks needed.
+      render: shell
+
+  - type: checkboxes
+    id: checks
+    attributes:
+      label: Before submitting
+      options:
+        - label: I checked that no existing issue already reports this.
+          required: true

adduct-0.2.1/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,8 @@
+blank_issues_enabled: true
+contact_links:
+  - name: Security vulnerability
+    url: https://github.com/prabhu-gopal/adduct/security/advisories/new
+    about: Please report security issues privately, never as a public issue. See SECURITY.md.
+  - name: Unsupported dataset format
+    url: https://github.com/prabhu-gopal/adduct/issues/new?labels=format-request&title=%5Bformat%5D+
+    about: Want adduct to read a format it does not support yet? Tell us which, and how common it is.

adduct-0.2.1/.github/ISSUE_TEMPLATE/false_positive.yml

@@ -0,0 +1,79 @@
+name: False positive
+description: Adduct flagged something on your data that is actually fine.
+title: "[false positive] "
+labels: ["false-positive", "calibration"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        **This is the most valuable report you can send us.**
+
+        Adduct's only real asset is that its findings are trustworthy. A detector that
+        cries wolf is worse than no detector, and we cannot find those on synthetic data —
+        we need yours. Every report here goes directly into threshold calibration.
+
+        You do **not** need to share your dataset. The detector id, the numbers adduct
+        reported, and your explanation of why it is wrong is enough to act on.
+
+  - type: input
+    id: detector-id
+    attributes:
+      label: Detector ID
+      description: >
+        The bracketed id printed with the finding, e.g. `[stats.dead_dimension]`. Run
+        `adduct explain <id>` to see what it thinks it is measuring.
+      placeholder: "stats.jitter_outlier"
+    validations:
+      required: true
+
+  - type: textarea
+    id: the-finding
+    attributes:
+      label: The finding, as adduct printed it
+      description: Copy the whole block, including the measured value and the episode count.
+      render: shell
+    validations:
+      required: true
+
+  - type: textarea
+    id: why-wrong
+    attributes:
+      label: Why is this wrong?
+      description: >
+        What does adduct not know about your setup? This is the part we cannot guess.
+      placeholder: |
+        Dimension 6 is a binary gripper open/close, so its variance is supposed to be near
+        zero between grasps. It is not a dead dimension — it is doing its job.
+    validations:
+      required: true
+
+  - type: textarea
+    id: dataset-context
+    attributes:
+      label: Dataset context
+      description: >
+        Robot/embodiment, control rate, action space (joint positions? EEF deltas?),
+        episode count, and the Hugging Face repo id if it is public.
+      placeholder: |
+        SO-101, 30 Hz, 6-DoF joint positions + binary gripper, 220 episodes, private.
+    validations:
+      required: true
+
+  - type: input
+    id: adduct-version
+    attributes:
+      label: Adduct version
+      description: "Output of `adduct --version`"
+      placeholder: "adduct 0.1.0"
+    validations:
+      required: true
+
+  - type: textarea
+    id: json
+    attributes:
+      label: Report JSON (optional, but very helpful)
+      description: >
+        `adduct scan <path> --json report.json` gives us the exact measured values and
+        thresholds. Attach it or paste the relevant finding. It contains no raw data from
+        your dataset — only statistics.
+      render: json

adduct-0.2.1/.github/pull_request_template.md

@@ -0,0 +1,21 @@
+## What does this change?
+
+<!-- One or two sentences. The "why" matters more than the "what". -->
+
+## Why?
+
+<!-- What problem does this solve? Link the issue if there is one. -->
+
+## Checklist
+
+- [ ] Commits are signed off (`git commit -s`) — see [CONTRIBUTING.md](../CONTRIBUTING.md)
+- [ ] `ruff check .`, `mypy`, and `pytest` pass locally
+- [ ] Added a test that fails before this change and passes after
+- [ ] Updated `CHANGELOG.md` under `## [Unreleased]` if this is user-visible
+
+### If this adds or changes a detector
+
+- [ ] The mechanism sentence says *why* the defect degrades a trained policy
+- [ ] Linked evidence that the defect is real (issue, paper, or training run)
+- [ ] Added a fault-injection scenario to the benchmark
+- [ ] Measured recall/precision are in the PR description

adduct-0.2.1/.github/workflows/ci.yml

@@ -0,0 +1,92 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    name: py${{ matrix.python-version }} · ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: true
+
+      - name: Install the project
+        run: uv sync --extra dev --python ${{ matrix.python-version }}
+
+      - name: Lint
+        run: uv run ruff check .
+
+      - name: Format check
+        run: uv run ruff format --check .
+
+      - name: Type check
+        run: uv run mypy
+
+      - name: Test
+        # The suite is fully synthetic: no network, no GPU, no dataset downloads.
+        run: uv run pytest -q
+
+  # The CLI is what users actually touch, and a working test suite has never once caught a
+  # broken entry point or a missing runtime dependency. This installs the built wheel into a
+  # clean environment and runs the binary, which is the only way to catch that.
+  smoke:
+    name: installed-wheel smoke test
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Build the wheel
+        run: uv build
+
+      - name: Verify the wheel contains the package
+        run: |
+          ls -R dist/
+          python -c "
+          import zipfile, glob, sys
+          wheel = glob.glob('dist/*.whl')[0]
+          names = zipfile.ZipFile(wheel).namelist()
+          assert any(n.startswith('adduct/') for n in names), 'no adduct/ inside the wheel'
+          print(f'{wheel}: {len(names)} files, package present')
+          "
+
+      - name: Install into a clean environment and run it
+        run: |
+          python -m venv /tmp/fresh
+          /tmp/fresh/bin/pip install --quiet dist/*.whl
+          /tmp/fresh/bin/adduct --version
+          /tmp/fresh/bin/adduct --help
+          /tmp/fresh/bin/adduct list-detectors | head -5
+          /tmp/fresh/bin/adduct explain stats.dead_dimension
+
+      - name: Errors are messages, not tracebacks
+        run: |
+          set +e
+          out=$(/tmp/fresh/bin/adduct scan ./definitely-not-a-dataset 2>&1)
+          code=$?
+          set -e
+          echo "$out"
+          [ "$code" -eq 2 ] || { echo "expected exit 2 for a usage error, got $code"; exit 1; }
+          case "$out" in *Traceback*) echo "a traceback reached the user"; exit 1 ;; esac

adduct-0.2.1/.github/workflows/release.yml

@@ -0,0 +1,99 @@
+name: Release
+
+# Tag-triggered publish. Never publish from a laptop: the tag is the only thing that can
+# ship a version, so what is on PyPI always corresponds to a commit anyone can check out.
+on:
+  push:
+    tags: ["v*"]
+  workflow_dispatch:
+    inputs:
+      test_pypi:
+        description: "Publish to TestPyPI instead of PyPI"
+        type: boolean
+        default: true
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    name: Build distributions
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+
+      - name: Build
+        run: uv build
+
+      - name: Show what is in the distributions
+        # If `adduct/` is not inside the wheel, pyproject.toml is wrong and the upload
+        # would silently ship an empty package.
+        run: |
+          ls -R dist/
+          python -m zipfile -l dist/*.whl | head -20
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish-testpypi:
+    name: Publish to TestPyPI
+    needs: build
+    runs-on: ubuntu-latest
+    if: github.event_name == 'workflow_dispatch' && inputs.test_pypi
+    environment:
+      name: testpypi
+      url: https://test.pypi.org/p/adduct
+    permissions:
+      # Required for Trusted Publishing: this is what lets uv mint a short-lived OIDC
+      # token instead of us storing a long-lived PyPI API token in GitHub Secrets.
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+      - name: Publish
+        run: uv publish --trusted-publishing always --publish-url https://test.pypi.org/legacy/
+
+  publish-pypi:
+    name: Publish to PyPI
+    needs: build
+    runs-on: ubuntu-latest
+    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
+    environment:
+      name: pypi
+      url: https://pypi.org/p/adduct
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+      - name: Publish
+        run: uv publish --trusted-publishing always
+
+  github-release:
+    name: Attach distributions to the GitHub release
+    needs: publish-pypi
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: softprops/action-gh-release@v2
+        with:
+          files: dist/*
+          generate_release_notes: true

adduct-0.2.1/.gitignore

@@ -0,0 +1,39 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+*.so
+
+# Environments
+.venv/
+venv/
+.env
+
+# Tooling caches
+.mypy_cache/
+.ruff_cache/
+.pytest_cache/
+.coverage
+htmlcov/
+coverage.xml
+
+# Editors / OS
+.vscode/
+.idea/
+.DS_Store
+
+# Adduct scan artifacts (never commit real reports)
+*_report.html
+*_report.json
+adduct-report.*
+
+# Internal design docs: business strategy, roadmap, licensing/moat reasoning.
+# Kept locally for reference, deliberately never pushed to the public repo.
+/docs/
+
+# The local working playbook (branch rules, CI checks, release process). Kept out of the
+# public repo by choice, not because it's sensitive — see the decision in chat history.
+/CLAUDE.md

adduct-0.2.1/CHANGELOG.md

@@ -0,0 +1,179 @@
+# Changelog
+
+All notable changes to this project are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+The **report schema** is versioned separately from the package — see `schema_version` in
+any `--json` output. It changes only when the serialized report shape changes.
+
+## [Unreleased]
+
+## [0.2.1] — 2026-09-01
+
+The first release published under the name `adduct`.
+
+### Added
+
+- **A default scan now says that the policy↔data checks were skipped**, and names the flags
+  that unlock them (`--target bc|act|diffusion|openvla|pi0|octo`, or `--policy <checkpoint>`).
+  Those five checks need a model to compare against, so staying silent without one is
+  correct — but saying nothing made the flags undiscoverable: the only way to learn the
+  checks existed was to read `--help`. The line appears exactly when none of them ran,
+  including on a clean dataset, which is the most misleading moment to stay quiet about
+  checks that never ran.
+
+## [0.2.0] — 2026-08-27
+
+Published measured evidence on real public data, and acted on what it showed. Every
+detector-facing change below is driven by that measurement rather than by taste.
+
+**Upgrading from 0.1.0.** A default scan now returns fewer findings, and that is the point.
+`dynamics.inverse_residual` no longer runs by default because it fired on 100% of a
+20-dataset public corpus; across that corpus the change takes 23 HIGH findings down to 13.
+If you gate CI on `--fail-on HIGH`, expect fewer failures, not more. Nothing is deleted:
+`--all` runs every detector, and `--json`, `--sarif` and `--html` are unchanged and
+complete. Scans should also be quiet now — the numpy and scikit-learn warnings some users
+saw on stderr are fixed at source.
+
+### Added
+
+- **`benchmarks/` — published measured evidence on real public data.** The first run,
+  `benchmarks/2026-08-26-lerobot-20-v0.1.0/`, scans 20 curated public LeRobot datasets
+  (4,342 episodes, 545,964 frames, 2–40 action dims, 5–50 Hz, sim and real, 14 of them
+  Open X-Embodiment conversions): 20/20 parsed without error in 22.3 s on a laptop CPU,
+  producing 189 findings (23 HIGH, 121 MEDIUM, 45 LOW). Of the 46 default detectors, 27
+  fired at least once, 19 never fired, and only 4 ever reached HIGH. Ships the raw sweep
+  JSON for both the uncapped and the default triage pass, a report, the sweep and figure
+  scripts, and a LaTeX paper. The unit suite measures recall on synthetic fixtures; this
+  measures how often detectors complain about healthy real data, which the suite cannot.
+- **Measured: the default 300-episode triage cap does not change the conclusions** on this
+  corpus. Comparing a capped scan against one that reads every episode, all 23 HIGH
+  findings are identical, as are the counts of detectors that fired and detectors reaching
+  HIGH; 2 MEDIUM findings out of 189 differ, on the 3 datasets that the cap subsamples.
+
+### Fixed
+
+- **Scans no longer leak numerical warnings to stderr.** `LinAlgWarning: Ill-conditioned
+  matrix` and numpy `divide by zero` / `overflow` / `invalid value` warnings from
+  scikit-learn could appear during an ordinary `adduct scan`. The ill-conditioning had a
+  real cause: the dynamics fits stack consecutive states, which are collinear by
+  construction, under a ridge penalty of 1e-6 that left that collinearity essentially
+  unregularized. Features are now standardized per fold and the penalty is 1.0, and the
+  solve **abstains** rather than returning a residual from a fit LAPACK flags as
+  unreliable. The remaining warnings are IEEE flags raised inside BLAS during clustering,
+  on input already validated as finite; they are suppressed at one documented boundary in
+  the engine, with `ADDUCT_SHOW_NUMERIC_WARNINGS=1` to restore them for debugging. Genuine
+  non-finite data is still reported by `integrity.nan_inf`. Measured across the 20-dataset
+  benchmark: 42 warning lines before, 0 after.
+
+### Changed
+
+- **`dynamics.inverse_residual` no longer runs in a default scan.** Measured on the
+  20-dataset benchmark: it fires on **100%** of the corpus (20/20) and reports HIGH on 50%
+  (10/20), and reaches the report's visible top-5 on 60% of datasets, more often than either
+  detector already excluded. Its HIGH rate is lower than theirs; the 100% fire rate is what
+  decides it, because a detector that fires on every curated public dataset cannot
+  discriminate whatever severity it attaches. Excluding it takes the corpus from 23 HIGH
+  findings to 13 and the datasets carrying at least one HIGH from 17 of 20 to 11 of 20.
+  Nothing is deleted: it stays fully implemented and reachable with `--all`. One explanation
+  was tested and rejected first (see Fixed, below); the two that survive are confounded in
+  that corpus, so recalibration waits on a corpus of natively-recorded community datasets.
+  `dynamics.forward_residual` is the next candidate and is deliberately **not** excluded
+  yet: it has the highest top-5 visibility of any detector (75%) but never reports HIGH.
+- **A default `adduct scan` now details the top 5 findings instead of 6**, and names the
+  flag that carries the rest (`--html` or `--json`, never `--all`, which adds the
+  held-back over-reporting detectors). The benchmark measured a median of 9.5 findings per
+  dataset with no dataset ever coming back clean; at that density the terminal is a triage
+  surface, not the full record. Nothing is dropped from any machine-readable output.
+- `smoothness.discontinuity_jump` and `integrity.declared_mismatch` (see "Known
+  limitations" under 0.1.0) no longer run in a default scan. Measured on the same
+  20-dataset sweep: when either fired, the report's own ranking (severity × blast radius)
+  put it in the visible top-6 findings 13 of 16 times, winning the #1 or #2 slot in 6 of
+  those — a first-time user's first impression was disproportionately likely to be one of
+  the two things already known to probably be wrong. Nothing is deleted, degraded, or
+  hidden: both are fully implemented and reachable with the new `--all` flag
+  (`adduct.scan(..., all_detectors=True)` in the Python API), and this default list lives
+  at `adduct.detectors.registry.DEFAULT_EXCLUDED`.
+
+### Known limitations
+
+- **Six detectors are effectively always-on** and their thresholds are not yet trustworthy.
+  Measured across the 20-dataset benchmark: `dynamics.inverse_residual` 100%,
+  `dynamics.forward_residual` 90%, `smoothness.jerk_outlier` 85%, `smoothness.curvature`
+  75%, `smoothness.path_efficiency` 70%, `temporal.non_markovian_pause` 70%. A detector
+  firing on 85% of curated public data carries almost no information whatever severity it
+  attaches, and these six are the bulk of a report's length. Recalibration needs a corpus
+  that represents the intended user, which the current one does not (see below).
+- **`dynamics.inverse_residual`'s HIGH severity is not yet trustworthy** (HIGH on 50% of
+  the benchmark). One hypothesis was tested and rejected: fixing the ill-conditioned ridge
+  solve eliminated the numerical fault but left the fire rate and HIGH rate unchanged. The
+  two surviving explanations, control rate and format-conversion provenance, are perfectly
+  confounded in that corpus and cannot be separated there. See
+  `benchmarks/2026-08-26-lerobot-20-v0.1.0/REPORT.md` §6.
+- **The benchmark corpus is not the population adduct is for.** 14 of its 20 datasets are
+  Open X-Embodiment conversions at 5 Hz, against an intended user recording natively at
+  30 Hz. The HIGH rate is 64% on the former and 17% on the latter, so a share of those
+  findings may reflect format conversion rather than data quality.
+- **`--fpr` is inert out of the box, and public metadata makes it hard to fix.** The
+  conformal gate needs a calibration corpus keyed by embodiment (Mondrian), and none ships
+  with the package. The sweep also found that 18 of 20 public LeRobot datasets declare
+  `robot_type: "unknown"`, so an embodiment-keyed taxonomy built from Hub metadata
+  collapses to the single wildcard bucket and forfeits group-conditional validity.
+
+## [0.1.0] — 2026-08-25
+
+First public release.
+
+### Added
+
+- `adduct scan <path>` — analyze a robot-learning dataset and report the defects that hurt
+  training, with severity, affected episodes, the measured value, the mechanism, and a fix.
+- **Hugging Face Hub support**: `adduct scan lerobot/pusht` fetches `meta/` and `data/`
+  (never video) and scans the local snapshot. This is the only network call in the tool.
+- **Formats**: LeRobot v2.1 and v3.0 (autodetected), RLDS/Open-X, robomimic and raw HDF5,
+  Zarr replay buffers, NumPy directories.
+- **Machine-readable output**: `--json` (versioned `schema_version`), `--sarif` (SARIF
+  2.1.0 for GitHub code scanning), and a self-contained `--html` report.
+- **CI gating**: `--ci --fail-on <severity>` exits non-zero only when you ask it to.
+- **Calibration**: `adduct calibrate` builds a conformal FDR corpus from your own
+  known-good data, after which `--fpr` governs the covered detector gates.
+- `adduct list-detectors`, `adduct explain <detector-id>`, and `adduct init`.
+- Adapters and detectors are plugins discovered through entry points — the same mechanism
+  the built-ins use.
+
+### Fixed
+
+- `dynamics.inverse_residual` reported HIGH unconditionally, whatever the measured extent.
+  Across 20 curated public LeRobot datasets it fired at HIGH on 19 of them, with flagged
+  fractions spanning 0.94% to 80.8% all reported identically. Severity now scales on extent
+  (≥20% of transitions) **or** magnitude (a residual as large as the signal itself, which is
+  physically unexplainable however rare). HIGH rate on real data: 95% → 50%,
+  measured with `scripts/hub_smoke.py`.
+- A clean scan printed "No findings" twice.
+
+### Known limitations
+
+- `smoothness.discontinuity_jump` and `integrity.declared_mismatch` report HIGH on 70% and
+  60% of curated public datasets respectively, which is far more likely to be a threshold
+  problem than a real epidemic. Documented in the README rather than silently shipped as
+  trustworthy; re-tuning them needs data we do not have yet.
+
+### Deliberately not included
+
+- **No 0-100 health score.** An aggregate number implies a calibration against real
+  training outcomes that does not exist yet — nothing here measures how much each defect
+  actually costs a trained policy. The report gives severity counts and ranked findings,
+  every one of which is individually defensible. The scoring function
+  (`adduct.synth.pipeline.quality_score`) is still present and tested; the headline
+  returns when a corpus of training runs can back it.
+
+### Notes
+
+- Python 3.10 through 3.13 are supported and tested.
+- No telemetry. Your data never leaves your machine.
+- `schema_version` is `1.0` — the first published report contract. Nothing consumed an
+  earlier shape, since this was the first released version of adduct.
+
+[Unreleased]: https://github.com/prabhu-gopal/adduct/compare/v0.2.1...HEAD
+[0.2.1]: https://github.com/prabhu-gopal/adduct/releases/tag/v0.2.1