testerkit 0.4.0__py3-none-any.whl

testerkit/_docs/how-to/catalog/datasheet-to-test.md

@@ -0,0 +1,169 @@
+# Datasheet → tests with Claude Code
+
+Walks through the `datasheet-to-test` workflow end-to-end: from a part datasheet PDF to a runnable pytest suite, with operator approval at every step. This flow chains spec extraction, instrument selection, station config, and test scaffolding into one conversation, with an approval gate at each step.
+
+For motivation see [why AI integration](../../concepts/overview/ai-integration.md). For the full inventory of what ships, see the [skills reference](../../reference/overview/skills.md).
+
+## Prerequisites
+
+- TesterKit installed (`pip install testerkit`)
+- A part datasheet PDF on disk
+- Claude Code installed and authenticated, with access to a high-capability model on your plan (Opus, GPT-5 / o-series, Gemini 2.5 Pro, or equivalent — the workflow does heavy PDF extraction)
+- A working catalog of instruments — either real catalog YAMLs in your project, or you can use TesterKit's bundled generics
+
+If you don't have Claude Code, swap to any client that supports MCP — the steps work the same, only the invocation differs (slash command vs conversational). See the [skills reference](../../reference/overview/skills.md#slash-commands) for the matrix.
+
+## One-time setup
+
+Register TesterKit's MCP server with Claude Code:
+
+```bash
+testerkit setup claude-code
+```
+
+This does three things ([full reference](../../reference/overview/skills.md#what-the-setup-commands-install)):
+
+1. Registers TesterKit as an MCP server (`claude mcp add testerkit -- <testerkit-bin> mcp serve`)
+2. Copies `/catalog-from-datasheet` and `/process-catalog` slash command stubs into `./.claude/commands/`
+3. Writes or merges `./CLAUDE.md` with the TesterKit project instructions
+
+Restart Claude Code after `testerkit setup` so it picks up the new server.
+
+Confirm MCP registration:
+
+```bash
+claude mcp list
+```
+
+TesterKit should appear in the list. If it doesn't, re-run `testerkit setup claude-code` and check the printed error.
+
+## Invoke the workflow
+
+Open Claude Code in your project directory. Start a new conversation with this prompt:
+
+```
+Run the datasheet-to-test workflow on this datasheet:
+./datasheets/my_part.pdf
+```
+
+Claude fetches the workflow prompt via MCP (`prompts/get datasheet-to-test`) and begins **Phase 1**.
+
+## The phases
+
+The workflow STOPS at every gate. You approve, edit, or reject before it moves on.
+
+### Phase 1 — Parse datasheet
+
+Claude reads the PDF, extracts:
+
+- Pin definitions (names, types, signal directions)
+- Electrical characteristics (voltage ranges, current draws, accuracy specs)
+- Test conditions (temperature, supply voltage points)
+
+You see a structured summary. Approval gate: "Here's what I extracted, anything missing or wrong?"
+
+Common edits at this gate:
+- Renaming pins to match your team's convention
+- Adding characteristics the datasheet implies but doesn't tabulate (e.g., quiescent current at no-load)
+- Removing characteristics you don't intend to test
+
+### Phase 2 — Save part spec
+
+Claude saves the spec to `parts/<id>.yaml`. The spec uses the same [Capability schema](../../reference/catalog/schema.md) as catalog entries. TesterKit validates the YAML against the catalog schema as it saves; you'll see Claude correct shape errors in-flight if it tries to save something invalid.
+
+Approval gate: review the saved YAML. Edit directly if you want — the agent re-reads on the next step.
+
+### Phase 2b — Recommend instruments
+
+Claude calls `testerkit_match(part_id=<id>)` and proposes instruments that cover the part's specs (see [how capability matching works](../../concepts/overview/ai-integration.md)). Three outcomes:
+
+| Outcome | What Claude proposes |
+|---|---|
+| Catalog has matches | Shortlist of matched catalog entries with capability coverage |
+| Catalog is empty / no matches | Asks what equipment you have available |
+| Mixed | Catalog matches for some, asks about gaps |
+
+If you have a specific model the catalog doesn't know yet, Claude offers three paths:
+
+1. **Fast, approximate:** ask Claude to scaffold a catalog entry from just the model number using its prior knowledge — good for a quick start, not production-accurate.
+2. `generic_dmm` / `generic_psu` / `generic_eload` / `generic_oscilloscope` — bundled with TesterKit, approximate capabilities, fine for mocked development.
+3. **Accurate, for production:** `/catalog-from-datasheet <pdf>` — correct to the datasheet, for catalog entries where spec correctness matters.
+
+Pick whichever fits where you are: generics if you're sketching, Claude's prior knowledge for well-known instruments, full datasheet for the production catalog entry.
+
+### Phase 3 — Create station config
+
+Claude generates `stations/<id>.yaml` wiring the selected instruments to roles (`psu`, `dmm`, `uut_load`, etc.) with realistic mock values. TesterKit validates the wiring when it saves.
+
+Approval gate: review the wiring. Most edits here are around VISA resource strings (the agent has no way to discover your bench's actual `TCPIP::*::INSTR` addresses unless you've already populated them or it can call `testerkit_discover` against a live bench).
+
+### Phase 4 — Generate tests
+
+Two files generated:
+
+- `tests/test_<part>.py` — pytest-native test code using the TesterKit fixtures (`context`, `verify`, `measure`, plus instrument role fixtures like `psu`, `dmm`)
+- `tests/test_<part>.yaml` — sidecar YAML with `sweeps:`, `limits:`, `mocks:` for operator-editable values
+
+The generated tests use:
+- `verify(name, value)` for judgment-bearing measurements — limits resolve from the active part spec / sidecar / profile, raises on out-of-band
+- `measure(name, value)` for record-only setup readouts
+- `context.changed("vin")` in parametrized sweeps to skip expensive reconfig
+- Native `@pytest.mark.parametrize` for code-owned sweeps; sidecar `sweeps:` for operator-edited sweeps
+
+Approval gate: review both files. Edit directly — pytest and the YAML are the source of truth, not Claude's memory of what it generated.
+
+### Phase 5 — Execute and analyze
+
+Claude calls `testerkit_run(test="tests/test_<id>.py", station="<station_id>", serial="<UUT-SERIAL>", project=<root>)`. The test executes against the configured station (with `mocks:` from the sidecar if `--mock-instruments` is implied, otherwise against the real bench).
+
+Claude shows you the results table — rows are measurements, columns are sweep axes, cells colored pass/fail/skip. From here you can ask follow-ups:
+
+- "Why did the load=0.8 row fail at vin=4.5?" → Claude pulls the measurement detail via `testerkit_runs` + `testerkit_steps`
+- "Open this run in the browser" → `testerkit_open(type="run", id=<run-id>)` returns a `testerkit serve` URL
+- "Tighten the output_voltage tolerance to ±1%" → Claude edits the sidecar and re-runs
+
+## What's actually saved to your project
+
+After the workflow completes, your project tree has:
+
+```
+my_project/
+├── testerkit.yaml                          # set by testerkit_project init
+├── parts/
+│   └── <id>.yaml                        # phase 2
+├── stations/
+│   └── <station_id>.yaml                # phase 3
+├── tests/
+│   ├── test_<part>.py                # phase 4
+│   └── test_<part>.yaml              # phase 4 sidecar
+└── catalog/                             # only if Phase 2b added an instrument entry
+    └── <instrument>.yaml
+```
+
+Every file is plain YAML or Python. Git diffs work. Code review works. If you want to delete the AI from the loop, `pytest tests/` still runs the test.
+
+## When things go wrong
+
+| Symptom | Cause | Fix |
+|---|---|---|
+| Claude can't find the workflow | MCP server not registered or Claude not restarted after `testerkit setup` | `claude mcp list` to verify; restart Claude Code |
+| Agent loops trying to save an invalid YAML | The save was rejected because the YAML didn't match the catalog schema | Let it iterate — validation is the feedback signal; or stop it and edit the YAML by hand |
+| Test runs but every row is `SKIP` / `MISSING_LIMIT` | Phase 4 generated `verify()` calls without limits, and no sidecar / part spec covers them | Either add limits to the sidecar `limits:` block or switch the calls to `measure()` for characterization-only |
+| Instrument match returns nothing | Catalog is empty or capability requirements aren't covered | Use generics for development, or run `/catalog-from-datasheet` for the missing instruments |
+| Agent picks the wrong instrument | The capability match is correct but the agent's preference doesn't match yours | Tell it directly: "I want the Keithley 2400 for the load instead of the eload" — it'll update the station and re-validate |
+
+## Variations
+
+Three ways to use this flow short of the full pipeline:
+
+1. **Just part spec.** Stop after Phase 2. You get a part YAML; do station + tests by hand.
+2. **Part + tests, mock instruments.** Stop after Phase 4, use the bundled `generic_*` instruments through Phase 2b, run `pytest --mock-instruments`. No real hardware needed; useful for test-code review before the bench is ready.
+3. **Incremental update.** Re-invoke the workflow on a part that already has files. The agent reads the existing YAMLs and proposes diffs rather than from-scratch generation.
+
+## See also
+
+- [Concepts: why AI integration](../../concepts/overview/ai-integration.md) — motivation
+- [Reference: skills](../../reference/overview/skills.md) — full inventory of workflows, agents, MCP tools
+- [How-to: MCP integration](../overview/mcp-integration.md) — per-client setup detail
+- [Reference: catalog schema](../../reference/catalog/schema.md) — the shape part spec and catalog entries share
+- [Tutorial step 3: pytest-native tests](../../tutorial/03-fixtures.md) — what the generated test code looks like in context

testerkit/_docs/how-to/catalog/index.md

@@ -0,0 +1,11 @@
+# How-To — Catalog
+
+AI-assisted authoring against the capability catalog: produce instrument YAML and part YAML from a datasheet PDF with operator approval at every step.
+
+- [Datasheet → tests with Claude Code](datasheet-to-test.md) — end-to-end walkthrough of the `datasheet-to-test` workflow
+
+## See also
+
+- [Concepts → AI integration](../../concepts/overview/ai-integration.md) — what the MCP surface gives an AI client and where the boundary sits
+- [Reference → Catalog schema](../../reference/catalog/schema.md) and [Catalog cookbook](../../reference/catalog/cookbook.md) — the YAML shape the workflow produces
+- [Reference → Skills](../../reference/overview/skills.md) — full inventory of workflows, sub-agents, slash commands the platform ships

testerkit/_docs/how-to/configuration/configuring-stations.md

@@ -0,0 +1,364 @@
+# Configuring Stations
+
+This guide covers station configuration for different environments.
+
+## Basic Station Config
+
+```yaml
+# stations/bench_1.yaml
+id: bench_1
+name: "Production Bench 1"
+location: "Lab A"
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.100::INSTR"
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "GPIB0::5::INSTR"
+```
+
+## Station Fields
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `id` | Defaults to filename stem | Unique identifier — must match the filename stem if declared explicitly |
+| `name` | Yes | Display name |
+| `location` | No | Physical location |
+| `description` | No | Description |
+| `supported_phases` | No | Documents which phases this bench is set up for — shown in the stations UI |
+
+## Instrument Fields
+
+Per [`StationInstrumentConfig`](../../reference/data/models.md):
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `type` | Yes | Instrument type (dmm, psu, etc.) |
+| `driver` | At least one of driver/resource (unless `mock: true`) | The driver to load, written `package.module.DriverName` (e.g. `pymeasure.instruments.keysight.Keysight34461A`) |
+| `resource` | At least one of driver/resource (unless `mock: true`) | VISA address or connection string |
+| `catalog_ref` | No | ID of a catalog entry that declares this instrument's capabilities |
+| `channels` | No | Named channel aliases (`dict[str, str]`) |
+| `description` | No | Human-readable description |
+| `mock` | No | Always mock this instrument (even without `--mock-instruments`). Default `false` |
+| `mock_config` | No | Return values when running in mock mode |
+
+## Using a station's instruments in a test
+
+Each role under `instruments:` becomes a pytest fixture of the same name. A `dmm:` role gives you a `dmm` fixture; `psu:` gives `psu`. Rename the role, rename the fixture.
+
+```python
+def test_rail_voltage(dmm, psu):
+    psu.voltage = 5.0
+    reading = dmm.measure_dc_voltage()
+    assert 3.2 < reading < 3.4
+```
+
+The fixtures are session-scoped — TesterKit connects each instrument once and shares the driver instance across all tests in the run.
+
+## VISA Addresses
+
+### TCP/IP (LAN)
+
+```yaml
+resource: "TCPIP::192.168.1.100::INSTR"
+resource: "TCPIP::192.168.1.100::5025::SOCKET"  # With port
+resource: "TCPIP::dmm.lab.local::INSTR"  # DNS name
+```
+
+### GPIB
+
+```yaml
+resource: "GPIB0::5::INSTR"   # Board 0, address 5
+resource: "GPIB1::12::INSTR"  # Board 1, address 12
+```
+
+### USB
+
+```yaml
+resource: "USB0::0x2A8D::0x0101::MY12345::INSTR"
+# Format: USB{board}::{vendor}::{part}::{serial}::INSTR
+```
+
+### Serial
+
+```yaml
+resource: "ASRL/dev/ttyUSB0::INSTR"  # Linux
+resource: "ASRLCOM3::INSTR"          # Windows
+```
+
+## Mock Mode Configuration
+
+Configure default values for `--mock-instruments` mode:
+
+```yaml
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.100::INSTR"
+    mock_config:
+      measure_dc_voltage: 3.31
+      measure_current: 0.1
+      measure_resistance: 1000
+```
+
+### Running in Mock Mode
+
+```bash
+pytest tests/ --station=stations/bench_1.yaml --mock-instruments --uut-serial=SIM001
+```
+
+The `--mock-instruments` flag uses mock instruments instead of real hardware. Mock values come from `mock_config` in the station, or can be overridden per-test in the sidecar YAML next to your test file (`tests/test_*.yaml`).
+
+## Station Types
+
+A station type is a template — one YAML file per type under `stations/types/<id>.yaml`. It declares the instrument roles (and their `type:` and `driver:`) every station of that type must provide, plus the catalog capability ids the type advertises:
+
+```yaml
+# stations/types/voltage_tester.yaml
+id: voltage_tester
+description: "Basic voltage testing station"
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+capabilities:
+  - keysight.34461a.dc_voltage
+  - keysight.e36312a.dc_source
+```
+
+```yaml
+# stations/types/full_test.yaml
+id: full_test
+description: "Complete test station"
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+  scope:
+    type: scope
+    driver: drivers.scope.MyScope
+```
+
+There is no `required:` field on an instrument entry — every role listed under `instruments:` is required, and any role not listed is omitted by definition. `capabilities:` is a list of catalog capability ids (strings), not inline capability dicts.
+
+Reference in station instances:
+
+```yaml
+# stations/bench_1.yaml
+id: bench_1
+name: "Production Bench 1"
+station_type: voltage_tester
+location: "Lab A"
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.100::INSTR"
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "GPIB0::5::INSTR"
+```
+
+## Multiple Stations
+
+### Production Lab
+
+```yaml
+# stations/prod_bench_1.yaml
+id: prod_bench_1
+name: "Production Bench 1"
+location: "Production Floor, Bay 1"
+
+supported_phases:
+  - production
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.10.101::INSTR"
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "TCPIP::192.168.10.102::INSTR"
+```
+
+### Development Lab
+
+```yaml
+# stations/dev_bench.yaml
+id: dev_bench
+name: "Development Bench"
+location: "R&D Lab"
+
+supported_phases:
+  - development
+  - debug
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "USB0::0x2A8D::0x0101::MY12345::INSTR"
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "GPIB0::5::INSTR"
+  scope:
+    type: scope
+    driver: drivers.scope.MyScope
+    resource: "TCPIP::192.168.1.200::INSTR"
+```
+
+### CI/CD
+
+```yaml
+# stations/ci_station.yaml
+id: ci_station
+name: "CI Environment"
+description: "For automated testing with --mock-instruments"
+
+instruments:
+  dmm:
+    type: dmm
+    mock: true
+    catalog_ref: generic_dmm
+    mock_config:
+      measure_dc_voltage: 3.31
+      measure_current: 0.1
+  psu:
+    type: psu
+    mock: true
+    catalog_ref: generic_psu
+    mock_config:
+      measure_voltage: 5.0
+```
+
+Run in CI:
+```bash
+pytest tests/ --station=stations/ci_station.yaml --mock-instruments --uut-serial=CI-TEST
+```
+
+## Selecting a fixture at run time
+
+Stations don't pin a fixture themselves. The active fixture is selected by `--fixture=...` on the pytest command line (or by a profile that sets it). The plugin validates that the fixture's `part_id` / `part_family` matches the active part spec before any test runs.
+
+```bash
+pytest tests/ \
+  --station=bench_1 \
+  --fixture=fixtures/power_board_fixture.yaml \
+  --uut-serial=SN001
+```
+
+See [Fixtures](../../concepts/configuration/fixtures.md) for the pin-to-instrument mapping model.
+
+## Capability Declarations
+
+A station instance (`stations/<id>.yaml`) does not carry a `capabilities:` field — capabilities are derived from each instrument's catalog entry when the station loads. See [Capabilities](../../concepts/configuration/capabilities.md) for how capability matching works.
+
+If you want to advertise a fixed capability set as part of a *type* (so concrete stations of that type are guaranteed to provide it), declare it on the **station type**, not on the instance, as a list of catalog capability ids:
+
+```yaml
+# stations/types/voltage_bench.yaml
+id: voltage_bench
+description: "Generic DC + AC voltage bench"
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+capabilities:
+  - keysight.34461a.dc_voltage
+  - keysight.34461a.ac_voltage
+  - keysight.e36312a.dc_source
+```
+
+To customize the capabilities a specific instrument provides, edit that instrument's catalog YAML (`catalog/<vendor>/<model>.yaml`) — not the station file.
+
+## Validation
+
+To validate a station file before a run, point pytest at it — a bad config fails fast:
+
+```bash
+pytest --collect-only --station=stations/bench_1.yaml
+```
+
+A missing required field or a type mismatch raises a `pydantic.ValidationError` before any test is collected.
+
+## Shared Instruments (Multi-UUT)
+
+When a multi-UUT fixture runs sites in parallel and more than one site uses the same instrument role, TesterKit connects that instrument once and shares it across sites — no extra config. See [Multi-UUT testing](../execution/multi-uut-testing.md).
+
+## Best Practices
+
+1. **Use descriptive station IDs** — `prod_bench_1` not `station1`
+2. **Include location** — Helps operators find equipment
+3. **Document supported phases** — `supported_phases:` records which phases this bench is set up for; it is shown in the stations UI
+4. **Create a CI station** — Fully mocked for automated tests; keep one `stations/ci_station.yaml` with `mock: true` on every instrument
+5. **Version control** — Track station YAML changes alongside test code
+
+## Common Configurations
+
+### Single DMM
+
+```yaml
+id: simple_station
+name: "Simple DMM Station"
+
+instruments:
+  dmm:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "USB0::0x2A8D::0x0101::MY12345::INSTR"
+```
+
+### Full Production
+
+```yaml
+id: production_station
+name: "Production Station"
+location: "Production Floor"
+
+instruments:
+  dmm_1:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.101::INSTR"
+  dmm_2:
+    type: dmm
+    driver: pymeasure.instruments.keysight.Keysight34461A
+    resource: "TCPIP::192.168.1.102::INSTR"
+  psu:
+    type: psu
+    driver: pymeasure.instruments.keysight.KeysightE36312A
+    resource: "TCPIP::192.168.1.103::INSTR"
+  eload:
+    type: eload
+    driver: drivers.eload.MyELoad
+    resource: "TCPIP::192.168.1.104::INSTR"
+  scope:
+    type: scope
+    driver: drivers.scope.MyScope
+    resource: "TCPIP::192.168.1.105::INSTR"
+```
+
+## Next Steps
+
+- [Stations Concept](../../concepts/configuration/stations.md) — Understanding stations
+- [Capabilities](../../concepts/configuration/capabilities.md) — Capability matching
+- [Custom drivers](custom-drivers.md) — Build a non-VISA driver