plesty-lib 0.3.5.dev2__tar.gz → 0.3.6__tar.gz

{plesty_lib-0.3.5.dev2 → plesty_lib-0.3.6}/CHANGELOG.md

@@ -1,10 +1,358 @@
 # Changelog
 
+## Unreleased
+
+- **A client holding an allocation can call schema operations again, and those
+  calls are now access-controlled.** The server appends the caller's allocation
+  to every `call` it forwards, but a schema operation is a generated closure
+  that validates its inputs strictly and so rejected the injected key by name:
+  `KeyError: measure_analog() got unexpected parameter(s): ['resources']`. Any
+  device that registers resources and exposes schema operations failed on
+  *every* operation call from an allocated client, which made per-client
+  locking unusable on schema-driven devices — that is, on all of them.
+  `@expose_to_api` methods escaped it only because a `**kwargs` absorbed the
+  key; synthetic operations have no signature to widen, and the strictness is
+  what enforces the schema's types and ranges.
+
+  `resources` is now taken out before validation and carried in the solver
+  request beside `func_meta`, where a solver can see the allocation alongside
+  the channels the arguments name.
+
+  The same gap left operations unguarded: `_user_permission_check` was
+  consulted in `write` and `query` only, so once the key stopped raising, a
+  client holding `ao0` could still run any operation on any line. Operations
+  now consult it too, before validating their arguments — a caller who may not
+  run one learns nothing about its inputs. A denied operation raises
+  `PermissionError` rather than returning `False` as `write` does, because an
+  operation's return value is arbitrary and a `False` denial is
+  indistinguishable from a legitimate result. A `FunctionSystem` used on its
+  own has no such policy and permits everything, as before.
+
+- **`run_server()` picks the server's event loop explicitly, so a global
+  policy no longer has to.** Importing `plesty.lib.service` installs a Windows
+  loop policy as a side effect: that is what lets a bare `asyncio.run` in a
+  generated `__main__` get a selector loop, which pyzmq needs because
+  `ProactorEventLoop` has no `add_reader`. Both `asyncio.set_event_loop_policy`
+  and `WindowsSelectorEventLoopPolicy` are deprecated in Python 3.14 — two
+  warnings on every device server import on a bench — and removed in 3.16,
+  where the import would raise instead.
+
+  `plesty.lib.service.run_server(main())` replaces `asyncio.run(main())` and
+  hands the loop factory to the call site, which is the supported mechanism
+  and works on every Python this package targets. `server_loop_factory()`
+  exposes the same choice for callers that build their own runner.
+  `SelectorEventLoop` itself is not deprecated; only the policy machinery is.
+
+  The policy install stays for now, skipped only where it would raise rather
+  than warn (3.16+), because it is the one thing keeping an unmigrated entry
+  point working on Windows. Nothing on a bench changes behaviour: this adds
+  the supported path and leaves the old one standing until the entry points
+  move (sdk#35).
+
+- **The Analyzer contract is locked, and now says so.** It shipped in 0.3.2
+  and has not changed since, the SDK's gate a1 and `plesty init analyzer`
+  generate against it, and the pilot migration put a real analysis module on
+  it without touching any analysis code. All of that was true and none of it
+  was written down where an analyzer author would look: `docs/analyzer.md`
+  described the interface without saying whether it was safe to depend on,
+  which is the one question a contributor has before writing the first
+  subclass. A new "Contract stability" section states what is fixed — the two
+  schemas, the single `analyze()`, the validating `__call__`, the provenance
+  stamp, the five pipeline gates — what may still be added, and what would
+  take a major version to change. Documentation only; no code changed.
+
+- **Every served request now writes one record to the log**, so a monitor can
+  show what is on the wire without a new transport or a wire-format change.
+  The record carries what was asked, of what, by whom, how it ended and how
+  long it took — and never the payload: a result can be a spectrum or a stack
+  of frames, and a log is the wrong home for measurement data. A failure
+  records the exception *type*, not its message, which can carry a path or an
+  address. A reply that could not be routed is recorded as `delivered: false`
+  with its outcome intact — the client timed out and reconnected, the device
+  did the work anyway, and the caller is about to report a failure that did
+  not happen; that case is the one a monitor most needs to show.
+
+  A `describe` is recorded at `DEBUG` rather than `INFO` — liveness, not work.
+  A supervisor watching a bench polls it on a timer (plesty-bench's console
+  asks every declared device every three seconds, whether or not anyone is
+  looking) and each probe opens a fresh socket, so it arrives under a new
+  client identity every time. At `INFO` that is some 1,200 records per device
+  per hour on an idle bench, and a monitor would show almost nothing but the
+  console asking whether the server is alive, with the calls that moved the
+  instrument buried among them. Recorded, not dropped: raising the logger to
+  `DEBUG` brings them back.
+
+  Written through the `plesty.server.traffic` logger, prefixed `[TRAFFIC]`, so
+  it can be quieted or raised on its own:
+  `logging.getLogger("plesty.server.traffic").setLevel(logging.WARNING)`. One
+  line per request — a stage polled at 5 Hz writes roughly 2 MB an hour, and
+  the history reaches back exactly as far as the log does. Reporting never
+  raises: a malformed request or a throwing handler costs the record, not the
+  request. For plesty-bench#3.
+
+- **A Windows stop now closes the device instead of killing the server.**
+  `CTRL_BREAK_EVENT` is the only console control event a supervisor can aim
+  at one process group, so it is what stops a single device server; Python
+  delivers it as `SIGBREAK`, whose default action is to terminate the process
+  outright (exit `0xC000013A`). Nothing installed a handler — not this
+  library, not the SDK scaffold — so every generated server was killed on
+  every stop: no traceback, no output, and the `finally` that closes the
+  device never ran. Whatever `__aexit__` releases was leaked each time. For a
+  spectrometer that was an orphaned `AddInProcess` still holding the camera,
+  which then refused the next start until an elevated `taskkill` or a power
+  cycle cleared it.
+
+  `DeviceTCPIPServer.run` now serves inside `stop_on_break()`, which hands
+  `SIGBREAK` to the interpreter's own `SIGINT` handler for the duration and
+  restores what it found afterwards. A stop raises `KeyboardInterrupt` in the
+  main thread on every platform, so the existing shutdown path runs and the
+  device is released. Servers get this by upgrading the library — no
+  regeneration from the template. Off Windows, and off the main thread where
+  handlers cannot be installed at all, it does nothing.
+
+- **A missed disconnect acknowledgement and a lost answer no longer read as
+  the same event.** The ROUTER refuses to route a reply whose client has
+  gone, and one warning covered both. A `disconnect` ack that misses the
+  composite's one-second window is routine — the client asked to be released
+  and the server was a moment slow finishing an acquisition — and it happens
+  once per teardown, which is what made the log unreadable. It is now debug.
+  The other case is an answer to work the device has already done: the caller
+  saw a timeout and will report a failure that did not happen. That warns and
+  names what was asked for (`call acquire`, `write CameraSettings…`), because
+  a reply reported as merely lost is not something anyone can act on
+  afterwards; a request that cannot be parsed is assumed to matter rather
+  than dismissed.
+
+- **`VI_ERROR_IN_PROGRESS` at open is no longer reported as a wedged
+  instrument.** The previous entry called it a finished diagnosis — wedged
+  before the session existed, every command will time out, only a power cycle
+  helps — and said so at error level. The logs do not support it. On one PM101R
+  it was the opening state of **ten of twelve** consecutive starts, and connect
+  succeeded on all twelve: identity, sensor query and limit reads all answered
+  normally. The message was telling people to unplug a working instrument,
+  which is the fastest way to spend the credibility the real remedy needs.
+
+  It is now recorded at debug and interpreted later. The diagnosis needs two
+  facts, and neither means anything alone: the clear-on-open found a transfer
+  in flight, **and** nothing on the session has ever been answered. One answer
+  proves the instrument was reachable, so a later failure is an ordinary fault
+  for the recovery ladder; a session that opened onto someone else's transfer
+  and has never been answered is the case worth a paragraph. Found by reading
+  the whole log rather than the starts we happened to trigger — a four-sample
+  read of the same data had suggested an alternating TMO/IN_PROGRESS pattern
+  that twelve samples show is simply IN_PROGRESS 10 of 12.
+
+- **A query now waits for MAV before it reads, so a silent instrument strands
+  nothing.** This is the cause behind the entry below rather than another
+  symptom of it. A USBTMC query is two exchanges — the command out, then
+  `REQUEST_DEV_DEP_MSG_IN` and a bulk-in read — and an instrument is entitled
+  to stay silent: an undefined header is answered with `-113` in the error
+  queue and no response at all. When that happened the read timed out and we
+  walked away while the device still held the request. Nothing retracted it.
+  `viClear` cannot be queued behind a pending bulk-in, which is exactly the
+  `VI_ERROR_IN_PROGRESS` the ladder kept reporting, and reopening the session
+  replaces a host-side handle without touching the device — so the ladder's own
+  diagnosis was right and every rung it had was powerless against it. Five such
+  queries in one session stopped a PM101R and a PM400 servicing USB-TMC at all,
+  on different benches, and only unplugging them helped.
+
+  `_query_when_ready` reads the status byte first and issues the read only once
+  bit 4 says a message is waiting. Measured on the PM101R this bug had already
+  cost a power cycle: `MEAS:ENER?`, one of the five that wedged it, became a
+  four-second non-event — the status byte sat at `0x04` (EAV set, MAV clear: an
+  error queued, no answer coming) across forty polls, and the next three
+  queries answered in single-digit milliseconds. Reading the status byte costs
+  a control transfer and touches no bulk endpoint, which is what makes it safe
+  to ask before committing to a read.
+
+  Silence now raises `NoMessageError`, a `VisaIOError` carrying the timeout
+  code so existing callers are unchanged, and the ladder skips every rung for
+  it: there is no transfer to rescue, and a clear cannot conjure an answer out
+  of an instrument that has already declined to give one. Three timeouts and a
+  session teardown become one clean "it does not answer that".
+
+  Two guards, because a status byte that never sets MAV is ambiguous — the
+  instrument may have nothing to say, or may simply not maintain the bit. The
+  gate does not fire until MAV has been seen once on the session; before that a
+  silent byte falls back to reading anyway, which is precisely the old
+  behaviour and no worse, and an answer switches gating off with a warning. A
+  backend that cannot report a status byte at all falls back the same way.
+
+  **This is prevention, not recovery, and the difference matters.** Nothing
+  here implements the USBTMC abort sequence (`ABORT_BULK_IN` →
+  `CHECK_ABORT_BULK_IN_STATUS` → `INITIATE_CLEAR`), so a transfer that does get
+  stranded still needs the instrument to lose power. Three routes remain: the
+  first query of a session, which is unguarded until MAV has been proven; a
+  resource without a separable status read, which keeps the old path; and
+  `stb_gated_queries` switched off. The first is the one that matters — a
+  session whose *opening* query is speculative, which is exactly the
+  uncharacterised-head probe this fix exists for. Prove MAV with something the
+  instrument certainly answers before asking it anything it may not.
+
+- **An instrument that arrived wedged now says so, on the first line.** The
+  clear-on-open is best effort and its failure was logged as such — but one
+  failure there is a diagnosis, not a symptom. `VI_ERROR_IN_PROGRESS` on a
+  session opened seconds ago cannot describe anything this process did: a
+  fresh session has issued no operation to still be in progress, so the
+  *instrument* was holding a stranded transfer before the session existed. It
+  is now reported at error level, saying what it means and what actually
+  clears it — because the two things anyone tries first both fail. Restarting
+  the server rebuilds a session against the same stuck endpoint, and a
+  driver-level re-enumeration (`Disable-PnpDevice`/`Enable-PnpDevice`,
+  `pnputil /restart-device`) resets the port without ever dropping bus power,
+  so a transfer held in the instrument's firmware rides straight through it.
+  Only power loss clears it, which for a bus-powered USB instrument means
+  unplugging the cable.
+
+  This came out of a PM101R that was diagnosed from thirty lines of identical
+  timeouts while the answer sat above them as a generic warning. Every other
+  clear failure — the timeout that is the documented benign case, an
+  instrument with no clear at all — stays the quiet warning it was, so the
+  new message means exactly one thing when it appears. The clear-on-open also
+  discards the host-side buffers first, as the recovery ladder's clear rung
+  already does.
+
+- **The VISA recovery ladder no longer compounds a wedge.** A PM101R with no
+  sensor head answered `*IDN?` and took writes, but every sensor-dependent
+  query timed out — and the ladder then made it worse until the meter answered
+  nothing at all. Four faults, each fixed:
+  - A timeout can leave the session with a transfer still in flight. Every
+    operation on such a session reports `VI_ERROR_IN_PROGRESS`, the device
+    clear included, so the clear-and-retry rung could never succeed and only
+    spent a second timeout to say so. That error now goes straight to the
+    reopen, which is the only rung that clears the condition.
+  - A wedged session refuses to close, and the refusal was logged and the
+    reference dropped — which frees nothing. The backend kept the handle and
+    the instrument kept the interface, so each reopen added *another* live
+    session to the same device; two rungs left the meter holding three, at
+    which point `*IDN?` stopped answering on all of them. A refused close now
+    falls back to closing the underlying VISA handle, which does not go
+    through the session and so cannot be blocked by its state.
+  - The reopen called `open_resource` directly, bypassing the watchdog that
+    `_open` uses — on the one path that runs precisely when the instrument is
+    not answering, and where opening can therefore hang in native code. It now
+    opens through the same watchdog.
+  - A reopen that failed left the address registered to a session that no
+    longer existed, so `is_open` was false, every later command was refused,
+    and `open()` rejected every retry as "already in use" for the life of the
+    process. The address is now unclaimed before the release is attempted and
+    re-registered on success, so a failed reopen leaves the manager closed but
+    reopenable rather than needing a server restart.
+
+  The clear rung also discards the host-side buffers first: a timed-out
+  query's answer may still be half-arrived, and the clear reaches the
+  instrument but not the buffers, so the remnant became the reply to the next
+  command. `_close` releases through the same path, so a device server shutting
+  down on a wedged instrument no longer leaves the interface claimed.
+
+  None of this makes an absent sensor head measure anything — the underlying
+  query still fails. It stops one hardware fault from taking the session, and
+  then the process, down with it.
+
+## 0.3.5 — 2026-08-24
+
+The final release of the line begun in 0.3.5.dev1: everything in the three
+pre-releases below ships here — the run convention (#34), bounded visible
+connects, the rig preflight and `connect_rig` (#36, #37), default logging,
+operation kinds driving the field test at both tiers (#39), and the pol-pl
+rig findings (#35, #38) — plus the parser change below.
+
+Note for consumers upgrading from 0.3.4: the run sources' `pattern=`
+arguments and `data_dir` attributes are gone and `Experiment(run_root=)`
+now defaults to the mount. Adjust those call sites.
+
+- **A schema can name its response parser** (#40). `parser` was listed among
+  the schema keys but only ever held a `ResponseParser` object, so a name
+  written there was stored unchanged and *called* on the first reply. It is
+  now resolved to a parser when the parameter is registered — a misspelled
+  kind, an invalid pattern or an out-of-range capture group fails while the
+  schema loads, naming the parameter. Two built-in kinds cover the text
+  shapes every instrument has: `regex` (`pattern`, `group`) pulls the value
+  out of a reply that carries text around it, `delimited` (`index`,
+  `separator`) keeps one field of a record; both hand the result to
+  `AutoResponseParser`, so the parameter's `type` still does the conversion,
+  and both decode a `bytes` reply first. `auto` stays the default and
+  unchanged — a reply it cannot cast is still returned raw. A device whose
+  frame the built-ins cannot describe registers its own decoder once with
+  `register_parser_kind(name, factory)` and names it per parameter from the
+  schema; `parser_kinds()` lists what is nameable. Passing a
+  `ResponseParser` instance from Python is untouched. This retires the
+  private `pattern` key `plesty-spectrapro-hrs` had invented for exactly
+  this.
+
+## 0.3.5.dev3 — 2026-08-18
+
+Development pre-release for field-testing operation kinds (#39): operations
+declare what they do and the field test drives a motion out and back, a
+lifecycle op once, a control op at rest; plus the pol-pl rig findings #35
+and #38. Same line as 0.3.5.dev2 — the final release is 0.3.5.
+
+- **Operations declare what they do, and the field test drives them by
+  kind** (#39). `@expose_to_api(kind=…)` and a schema `kind` block record
+  one of `read | motion | acquire | lifecycle | control | configure`
+  (`plesty.lib.device.device_utils.OpKind`; `FuncDoc.kind`, the describe
+  payload and `function_docs()` carry it). `FieldTestPipeline` acts on it
+  instead of asking the developer per operation: a `motion` op is driven
+  one `step` out from what its `position_key` reports and put back (the
+  functions gate checks it came back; the stability gate repeats out and
+  back), a `lifecycle` op runs once when allow-listed and never on repeat,
+  a `control` op once at rest after the motions (skipped when nothing
+  moved), a `configure` op is never called. `op_kinds={…}` overrides the
+  module's declaration. On plesty-k10cr1 the operation every pol-pl row
+  calls could not be tested at all: `move_absolute(position)` was called
+  bare and failed on the missing argument.
+- **The client tier drives operations by kind too.** `ClientFieldTest`
+  takes `op_kinds=` (the generated `field_test_client.py` passes the host
+  tier's): a motion is queried through `position_key`, moved one step out
+  and back over the wire as many pairs as the host tier's call count, a
+  lifecycle op once, a control op once after the motions, a configure op
+  never; a motion without `position_key`/`target`/`step` fails the gate.
+- **A provocation that breaks is a failed gate, not a forced timeout.**
+  The drain and recovery gates called the slow op with the payload as one
+  positional dict (`getattr(device, slow_op)(dict(payload))`), so an
+  `@expose_to_api` slow op raised `TypeError` before any transport
+  traffic — and both gates counted the exception as an abandoned call and
+  passed. The call now goes through the operation's own call form, only a
+  timeout-class error counts as forced, and a motion slow op is a delta
+  from where the part is, moved back afterwards (`motion_restored` in the
+  evidence). The slow-op payload is prepared before the timeout is
+  shortened, so reading the start position cannot itself time out.
+
+- **A serial port that is held says so** (#35). pyserial reports a wrong
+  port name and a port open in another process with the same
+  `SerialException`, and on the K10CR1 rig `Access is denied` on the right
+  `COM4` — Kinesis had the stage loaded — cost the session to driver and
+  VCP checks. `TrafficManager.open` now asks the subclass to explain a
+  failure (`_explain_open_error`, `None` by default) and appends the answer
+  to the raised message. `AptTrafficManager` and `SerialTrafficManager`
+  answer for EACCES/EBUSY: on Windows and for EBUSY, the port exists but is
+  open elsewhere — close Kinesis/APT (a loaded stage locks the FTDI device
+  via D2XX and blocks the VCP), stray interpreters, serial terminals, replug
+  to drop a stale handle; on Linux EACCES, add the user to `dialout`. The
+  port's `list_ports` description and hwid follow, so the user sees at once
+  whether the address is the intended device. Windows sets no `errno` on
+  the exception, so the number is read out of the message text. Other
+  failures (`ENOENT`, timeouts) are reported as before.
+- `ModuleInfo.from_project` and the field-test version lookup no longer
+  answer for whatever repository `GIT_DIR` names: a git hook exports it for
+  the repository being pushed, and every `git -C <root>` in `report_artifact`
+  followed it — a field test run from a pre-push hook reported the hook's
+  remote and version, and the pre-push suite failed on the tmp-dir module
+  test. `GIT_*` is scrubbed from the subprocess environment.
+
+- **Client-tier `functions` gate no longer demands a dict of every
+  operation** (#38). It required `isinstance(answer, dict)` for each
+  allow-listed operation, while the host tier asks that only of
+  schema-declared ones; on lightfield_spectrometer `acquire` (`None`) and
+  `get_recent_file` (`str`) came back 21 times each with no failure and
+  the gate still failed. A call that returns is the pass; the answer's
+  type is recorded in the evidence.
+
 ## 0.3.5.dev2 — 2026-08-17
 
 Development pre-release for field-testing the rig start-up (#36, #37):
 bounded, visible connects, the preflight, and default logging. Same line as
-0.3.5.dev1 — the final release is 0.4.0.
+0.3.5.dev1 — the final release is 0.3.5.
 
 - **A run starts only on a rig that is really there** (pol-pl field
   finding: a sweep went on with the powermeter server absent).
@@ -64,9 +412,10 @@ bounded, visible connects, the preflight, and default logging. Same line as
 ## 0.3.5.dev1 — 2026-08-17
 
 Development pre-release for field-testing the run convention (#34); the
-final release of this line is 0.4.0 — the API diff against 0.3.4 (the
+final release of this line is 0.3.5. The API diff against 0.3.4 — the
 `pattern=` arguments and `data_dir` attributes of the run sources are
-gone, `Experiment(run_root=)` defaults to the mount) is a minor bump.
+gone, `Experiment(run_root=)` defaults to the mount — needs those call
+sites adjusted.
 
 - `plesty.lib.monitor.Viz` (#34): the `monitor` / `render` command line over
   an experiment's panels. The experiment registers one function

{plesty_lib-0.3.5.dev2 → plesty_lib-0.3.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: plesty-lib
-Version: 0.3.5.dev2
+Version: 0.3.6
 Summary: A library for the Plesty ecosystem.
 Author: Plesty Development Team
 Author-email: Yunshuang Yuan <yunshuang.yuan@fkp.uni-hannover.de>

{plesty_lib-0.3.5.dev2 → plesty_lib-0.3.6}/docs/analyzer.md

@@ -54,6 +54,33 @@ Schema entries use the same dtype vocabulary as device schemas
 `array_<numpy_dtype>`, `table2d`, … Arrays may declare a `shape` where `None`
 marks a free dimension (`[None, 2]`); inputs may declare `required: False`.
 
+## Contract stability
+
+**This contract is locked.** It shipped in 0.3.2 and has not changed since.
+Write an analyzer against what is described above and it will keep working.
+
+Locked means the declared surface is fixed: the two class-level schemas, the
+single `analyze()`, the validating `__call__` and its four steps, the
+provenance stamp, and the five `AnalyzerPipeline` gates below. Names and
+shapes here are not going to be rearranged under you.
+
+It is locked on evidence rather than by declaration. The pilot migration
+(`hub/analyzers/positionestimate`) put a real analysis module on the contract
+without changing any of its analysis code — the facade was about 170 lines,
+free-dimension shapes and optional inputs both behaved as specified, and all
+five gates passed. The SDK then built on it: gate a1 and `plesty init
+analyzer` generate and check against exactly this surface. A contract two
+independent consumers already depend on is one that has been tested by use,
+which is the only kind of locking worth stating.
+
+What can still change: additions. A new optional schema key, a new dtype in
+the shared vocabulary, a new gate — none of those invalidate an analyzer
+written today. What cannot change without a major version and a deprecation
+period: the four things in the paragraph above. If you find the contract
+genuinely cannot express your transform, that is an issue on plesty-lib, not
+a local workaround — the value of the stamp and the gates is that every
+analyzer carries the same ones.
+
 ## Design rules
 
 - **Synchronous by design.** Analysis is CPU-bound compute, unlike the I/O

{plesty_lib-0.3.5.dev2 → plesty_lib-0.3.6}/docs/architecture.md

@@ -16,7 +16,7 @@ Everything ships under the `plesty.lib` namespace package:
 |---|---|
 | `plesty.lib.utils` | Foundation utilities: logging setup, error handling, the process-wide `ResourceRegistry`, `EnvSettings`, the module/run configuration loader (`module_config`, `load_yaml`), DLL helpers. |
 | `plesty.lib.data` | Data containers and persistence: `PlestyArray`, tables, units, dtype helpers, and the result store (`save_result` / `ResultDocument`, optional HDF5 export). |
-| `plesty.lib.traffic` | Transport backends (traffic managers): Serial, VISA, USB utilities, raw TCP/IP. Moves bytes; knows nothing about parameters or operations. |
+| `plesty.lib.traffic` | Transport backends (traffic managers): Serial, VISA, USB utilities, raw TCP/IP, and Thorlabs APT (binary framing, `apt_protocol`). Moves bytes; knows nothing about parameters or operations. |
 | `plesty.lib.solver` | Protocol translation: the `CmdSolver` / `OpSolver` ABCs and concrete solvers (`SCPISolver`, `ICEBLOCKSolver`) that turn standardized calls into vendor commands. |
 | `plesty.lib.device` | The core device tier: `BaseDeviceSyncModel` with its three system mixins (`ConfigSystem`, `FunctionSystem`, `TelemetrySystem`), the render-agnostic `doc_model()`, async wrappers, ready-made SCPI base devices, and `CompositeDevice`. |
 | `plesty.lib.service` | Remote access: the ZMQ-based device TCP/IP server and client, plus the per-device `ResourceManager` for multi-client access control. |
@@ -24,8 +24,8 @@ Everything ships under the `plesty.lib` namespace package:
 | `plesty.lib.experiment` | Reproducible orchestration: the async `Experiment` ABC with `Plan`/`Step` schedules and the crash-safe `Journal`. |
 | `plesty.lib.monitor` | Live views: the `Monitor` ABC — schema-declared projections of arriving data onto declared traces — and the sources that feed it (`RunSource`, `TelemetrySource`, `PushSource`). Read-only and toolkit-free. |
 | `plesty.lib.ui` | GUI framework: the `Panel` contract, the dockable `Shell` home window, the `Theme` carrying the plesty.net design tokens, and the PySide6/pyqtgraph backend behind the `gui` extra. |
-| `plesty.lib.sim` | `DataGenerator` — schema-driven synthetic data for mock solvers and tests. |
-| `plesty.lib.test` | Shipped test helpers and the standard contract pipelines (`DevicePipeline`, `ExperimentPipeline`, `AnalyzerPipeline`) that hub modules import. |
+| `plesty.lib.sim` | Stand-ins for hardware: `DataGenerator` (schema-driven synthetic data for mock solvers and tests), `DemoDevice` (the standard demo device tests build on instead of hand-writing one), and `AptMotorSimulator` (a Thorlabs APT motor answering real protocol frames). |
+| `plesty.lib.test` | Shipped test helpers and the standard contract pipelines that hub modules import: `DevicePipeline`, `ExperimentPipeline`, `AnalyzerPipeline` (mock, hardware-free), plus `FieldTestPipeline`, `client_field_test` and `field_test_concurrency` for what only an instrument can prove. |
 
 ### Layering
 
@@ -143,15 +143,19 @@ file is the commit record. `load_document` reads metadata without blob I/O;
 `convert_to_hdf5` optionally packs documents into one archive. See
 [Data & Metadata Schemas](data_schemas.md).
 
-**Contract test pipelines** — `plesty.lib.test`. `DevicePipeline` (ten
-gates, SDK gate d1), `ExperimentPipeline` (five hardware-free gates, SDK
-gate E1), and `AnalyzerPipeline` (five gates on schema-generated synthetic
-inputs, groundwork for SDK gate a1) define what every hub module must prove:
-schema integrity, mock round-trips, lifecycle, resource contention (device
-tier); deterministic plans, resolvable ops, serializability (experiment
-tier); analyze-signature match, round-trip validation, provenance (analyzer
-tier). Hub modules expose one pytest function per gate. See
-[Test helpers](device/test_helper.md).
+**Contract test pipelines** — `plesty.lib.test`. `DevicePipeline` (eight
+mock gates, SDK gate d1), `ExperimentPipeline` (five hardware-free gates,
+SDK gate E1), and `AnalyzerPipeline` (five gates on schema-generated
+synthetic inputs, groundwork for SDK gate a1) define what every hub module
+must prove: schema integrity, mock round-trips, lifecycle (device tier);
+deterministic plans, resolvable ops, serializability (experiment tier);
+analyze-signature match, round-trip validation, provenance (analyzer tier).
+A device module exposes one pytest function calling `run_mock_pipeline()`;
+experiment and analyzer modules expose one function per gate. What a mock
+cannot reach — the command-format contract, transport pathologies, resource
+contention — belongs to `FieldTestPipeline` on the instrument,
+`client_field_test` over the wire, and `field_test_concurrency` for devices
+sharing a host. See [Test helpers](device/test_helper.md).
 
 ## Design Decisions
 

{plesty_lib-0.3.5.dev2 → plesty_lib-0.3.6}/docs/device/base_device.md

@@ -122,6 +122,52 @@ class MyScpiSerialDevice(BaseDeviceSyncModel):
 		return self.traffic_manager is not None and self.traffic_manager.is_open
 ```
 
+## Ready-Made Base Devices
+
+Before subclassing `BaseDeviceSyncModel` directly, check whether one of the
+shipped intermediates already covers your transport. Each wires up a traffic
+manager and implements `connect`, `disconnect`, `_write_`, `_query_` and
+`check_operatability`, so a module only declares its schemas and its
+device-specific operations.
+
+| Class | Location | For |
+|---|---|---|
+| `BaseVisaScpiDevice` | `plesty.lib.device.base_visa_scpi_device` | Any SCPI instrument reachable over VISA (USB-TMC, GPIB, serial-over-VISA). |
+| `BaseTCPScpiDevice` | `plesty.lib.device.base_tcp_scpi_device` | A SCPI instrument that listens on a raw TCP socket rather than through VISA. |
+| `BaseAptDevice` | `plesty.lib.device.base_apt_device` | Single-channel Thorlabs APT motion controllers — binary frames rather than text. |
+
+```python
+from plesty.lib.device.base_visa_scpi_device import BaseVisaScpiDevice
+
+
+class MyPowerMeter(BaseVisaScpiDevice):
+	def __init__(self, address: str):
+		super().__init__(
+			address,
+			timeout=5000,
+			param_schema="assets/param_schema.json",
+			op_schema="assets/op_schema.json",
+		)
+```
+
+Both SCPI bases take `write_termination` / `read_termination` (and an optional
+`baud_rate` for serial-backed resources), pair themselves with
+[`SCPISolver`](cmd_solver.md), and add `query_range`, `identity`,
+`check_errors` and `reset` on top of the standard device API.
+
+`BaseAptDevice` is the odd one out because APT is not a text protocol. It takes
+a serial `port`, an `AptAxis` destination and a `channel`, and exposes the
+exchange as messages rather than strings: `command(...)` sends one frame,
+`request(...)` sends one and waits for its reply, `hw_info()` and `get_status()`
+read the controller. Motion is handled in the controller's own count domain —
+`home()`, `move_absolute_counts()`, `move_relative_counts()`, `stop()`,
+`get_position_counts()`, `get_velocity_params()` / `set_velocity_params()` —
+and the subclass converts counts to physical units in its user-facing
+operations. Parameter keys are registered with `register_apt_param()` so the
+config system can read and write them like any other. Passing `transport=` a
+`plesty.lib.sim.apt.AptMotorSimulator` runs the whole class without a motor;
+see [Simulation](../sim.md).
+
 ## State and Synchronization Helpers
 
 Useful built-in helpers:
@@ -261,7 +307,12 @@ model.standard_methods   # the common device API (connect/write/query/reset/...)
 `functions` unifies the two ways a device gains operations: schema-registered
 operations (`source="schema"`) and `@expose_to_api` methods
 (`source="custom"`); framework plumbing is kept out and surfaced separately
-under `standard_methods` (`source="standard"`). Per-parameter and return
+under `standard_methods` (`source="standard"`). Each `FuncDoc` also carries
+its `kind` — what the operation does to the world (`read`, `motion`,
+`acquire`, `lifecycle`, `control`, `configure`), declared with
+`@expose_to_api(kind=..., position_key=..., step=...)` or a `"kind"` block in
+`schema_func.json` and used by the field test to drive the operation
+(see *test_helper*, "Kinds"). Per-parameter and return
 descriptions are parsed from Google-style docstring `Args:`/`Returns:`
 sections — write those in your device methods and the generated documentation
 inherits them.

{plesty_lib-0.3.5.dev2 → plesty_lib-0.3.6}/docs/device/param_system.md

@@ -16,7 +16,8 @@ Core implementation: `ConfigSystem` in `plesty.lib.device.params`.
 4. Read/write permission flags (`read_only`, `write_only`).
 5. Current value cache per parameter (`value`).
 6. Automatic response casting through `AutoResponseParser`.
-7. Human-readable summaries (`param_summary`).
+7. Response parsers named from the schema (`parser`), including a device's own.
+8. Human-readable summaries (`param_summary`).
 
 ## Main Data Structures
 
@@ -179,7 +180,83 @@ Examples:
 3. `"true"/"1" -> bool` (via basic cast helper)
 4. Delimited strings for typed list/tuple annotations
 
-If needed, provide a custom parser in parameter metadata via `parser`.
+A reply the auto parser cannot cast is returned unchanged. When the value has to
+be pulled out of the reply first, name a parser in the parameter's `parser` key.
+
+### Naming a parser from a schema
+
+`parser` takes a kind name, or a mapping of a `kind` and its configuration:
+
+```json
+{
+	"wavelength": {
+		"type": "float",
+		"unit": "nm",
+		"command": "?NM",
+		"parser": {"kind": "regex", "pattern": "(-?\\d+\\.\\d+) nm"}
+	}
+}
+```
+
+The kind is resolved to a `ResponseParser` when the parameter is registered, so a
+misspelled kind, an invalid pattern or an out-of-range capture group fails while
+the schema is being loaded rather than on the first query.
+
+Built-in kinds:
+
+| kind | configuration | reply → value |
+|---|---|---|
+| `auto` | — | `"3.14"` → `3.14` (the default) |
+| `regex` | `pattern`, `group` (default `1`, a number or a group name), `encoding` | `"605.334 nm  ok"` → `605.334` |
+| `delimited` | `index` (default `0`, negatives count from the end), `separator` (default `,`), `encoding` | `"IDN,ACME,X100,1.2.3"` with `index: 2` → `"X100"` |
+
+Both parsers hand their result to `AutoResponseParser`, so the parameter's `type`
+still performs the conversion — the pattern or the separator only decides which
+characters are the value. A `bytes` reply is decoded with `encoding` (default
+`utf-8`) first.
+
+Use `delimited` when the reply carries several fields and the parameter is one of
+them. When the parameter's dtype is itself a list or tuple, `auto` already splits
+the reply and no parser is needed.
+
+### Registering a device's own parser
+
+Binary frames and replies keyed by a mnemonic are the device's own business. The
+module registers a parser once and every parameter then names it from the schema:
+
+```python
+from plesty.lib.device.device_utils import ResponseParser
+from plesty.lib.device.params import register_parser_kind
+
+
+class AptStatusParser(ResponseParser):
+	def __init__(self, field: str):
+		self.field = field
+
+	def parse(self, response, param=None, **kwargs):
+		return decode_apt_status(response)[self.field]
+
+
+register_parser_kind("apt_status", AptStatusParser)
+```
+
+```json
+{
+	"position": {
+		"type": "float",
+		"command": "MGMSG_MOT_REQ_STATUSUPDATE",
+		"parser": {"kind": "apt_status", "field": "position"}
+	}
+}
+```
+
+Schema keys other than `kind` are passed to the factory as keyword arguments.
+Registering the same name twice with the same factory is fine; a different
+factory under a name already in use raises. `parser_kinds()` lists what is
+currently nameable.
+
+Passing a `ResponseParser` instance directly to `register_config(parser=...)`
+from Python keeps working unchanged.
 
 ## Useful API Methods
 
@@ -193,6 +270,7 @@ If needed, provide a custom parser in parameter metadata via `parser`.
 8. `get_config_value(key)`
 9. `set_config_min_max(key, min_value, max_value)`
 10. `param_summary(constraints=True, description=False)`
+11. `register_parser_kind(name, factory)` / `parser_kinds()` (module-level)
 
 ## Minimal End-to-End Example