plesty-lib 0.3.5.dev1__tar.gz → 0.3.5.dev3__tar.gz

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/CHANGELOG.md

@@ -1,5 +1,134 @@
 # Changelog
 
+## 0.3.5.dev3 — 2026-08-18
+
+Development pre-release for field-testing operation kinds (#39): operations
+declare what they do and the field test drives a motion out and back, a
+lifecycle op once, a control op at rest; plus the pol-pl rig findings #35
+and #38. Same line as 0.3.5.dev2 — the final release is 0.4.0.
+
+- **Operations declare what they do, and the field test drives them by
+  kind** (#39). `@expose_to_api(kind=…)` and a schema `kind` block record
+  one of `read | motion | acquire | lifecycle | control | configure`
+  (`plesty.lib.device.device_utils.OpKind`; `FuncDoc.kind`, the describe
+  payload and `function_docs()` carry it). `FieldTestPipeline` acts on it
+  instead of asking the developer per operation: a `motion` op is driven
+  one `step` out from what its `position_key` reports and put back (the
+  functions gate checks it came back; the stability gate repeats out and
+  back), a `lifecycle` op runs once when allow-listed and never on repeat,
+  a `control` op once at rest after the motions (skipped when nothing
+  moved), a `configure` op is never called. `op_kinds={…}` overrides the
+  module's declaration. On plesty-k10cr1 the operation every pol-pl row
+  calls could not be tested at all: `move_absolute(position)` was called
+  bare and failed on the missing argument.
+- **The client tier drives operations by kind too.** `ClientFieldTest`
+  takes `op_kinds=` (the generated `field_test_client.py` passes the host
+  tier's): a motion is queried through `position_key`, moved one step out
+  and back over the wire as many pairs as the host tier's call count, a
+  lifecycle op once, a control op once after the motions, a configure op
+  never; a motion without `position_key`/`target`/`step` fails the gate.
+- **A provocation that breaks is a failed gate, not a forced timeout.**
+  The drain and recovery gates called the slow op with the payload as one
+  positional dict (`getattr(device, slow_op)(dict(payload))`), so an
+  `@expose_to_api` slow op raised `TypeError` before any transport
+  traffic — and both gates counted the exception as an abandoned call and
+  passed. The call now goes through the operation's own call form, only a
+  timeout-class error counts as forced, and a motion slow op is a delta
+  from where the part is, moved back afterwards (`motion_restored` in the
+  evidence). The slow-op payload is prepared before the timeout is
+  shortened, so reading the start position cannot itself time out.
+
+- **A serial port that is held says so** (#35). pyserial reports a wrong
+  port name and a port open in another process with the same
+  `SerialException`, and on the K10CR1 rig `Access is denied` on the right
+  `COM4` — Kinesis had the stage loaded — cost the session to driver and
+  VCP checks. `TrafficManager.open` now asks the subclass to explain a
+  failure (`_explain_open_error`, `None` by default) and appends the answer
+  to the raised message. `AptTrafficManager` and `SerialTrafficManager`
+  answer for EACCES/EBUSY: on Windows and for EBUSY, the port exists but is
+  open elsewhere — close Kinesis/APT (a loaded stage locks the FTDI device
+  via D2XX and blocks the VCP), stray interpreters, serial terminals, replug
+  to drop a stale handle; on Linux EACCES, add the user to `dialout`. The
+  port's `list_ports` description and hwid follow, so the user sees at once
+  whether the address is the intended device. Windows sets no `errno` on
+  the exception, so the number is read out of the message text. Other
+  failures (`ENOENT`, timeouts) are reported as before.
+- `ModuleInfo.from_project` and the field-test version lookup no longer
+  answer for whatever repository `GIT_DIR` names: a git hook exports it for
+  the repository being pushed, and every `git -C <root>` in `report_artifact`
+  followed it — a field test run from a pre-push hook reported the hook's
+  remote and version, and the pre-push suite failed on the tmp-dir module
+  test. `GIT_*` is scrubbed from the subprocess environment.
+
+- **Client-tier `functions` gate no longer demands a dict of every
+  operation** (#38). It required `isinstance(answer, dict)` for each
+  allow-listed operation, while the host tier asks that only of
+  schema-declared ones; on lightfield_spectrometer `acquire` (`None`) and
+  `get_recent_file` (`str`) came back 21 times each with no failure and
+  the gate still failed. A call that returns is the pass; the answer's
+  type is recorded in the evidence.
+
+## 0.3.5.dev2 — 2026-08-17
+
+Development pre-release for field-testing the rig start-up (#36, #37):
+bounded, visible connects, the preflight, and default logging. Same line as
+0.3.5.dev1 — the final release is 0.4.0.
+
+- **A run starts only on a rig that is really there** (pol-pl field
+  finding: a sweep went on with the powermeter server absent).
+  `CompositeDevice.preflight()` checks that every remote sub-device exposes
+  the methods listed under the new `requires` config key and answers
+  `identity()`; `Experiment.setup()` runs it and raises `RigNotReadyError`
+  on any problem. `plesty.lib.experiment.connect_rig(build, optional=…)`
+  is the start-up for every experiment's `__main__`: build, preflight, and
+  on a failure print which device, at which address, from which variable,
+  then ask `[r]etry / [a]bort / [s]kip` (skip only for optional
+  sub-devices, returned as `skipped`); non-interactive processes get the
+  diagnosis logged and `RigNotReadyError`. Text in
+  `experiment/assets/preflight.yaml`.
+- **A composite that is given up closes its clients.** New
+  `CompositeDevice.release()` closes every remote client on its owning
+  thread (server-side disconnect with a 1 s window, linger 0) and stops the
+  threads; `disconnect_all()` does the same for remote clients, and a first
+  connect that fails closes the clients that did connect. A client left to
+  the garbage collector blocked the interpreter in its ZMQ context's
+  `__del__` — seen mid-run after `connect_rig` rebuilt a rig without the
+  powermeter. `connect_rig` releases a rig it gives up.
+- **A first connect to a dead server now returns.** The client's connect
+  failure path closed its DEALER socket with the default infinite LINGER,
+  so `ctx.term()` blocked forever on the undelivered handshake — a rig built
+  against one absent server hung with no output, and no attempt ladder or
+  deadline could bound it. `_DeviceTCPIPClient` now drops undelivered
+  messages (linger 0) when a connect fails and on `close()`.
+  `CompositeDevice` also resolves every address before starting any connect
+  (a half-configured `.env` raises `ValueError` without touching a server)
+  and stops its worker threads when a first connect fails.
+- **`CompositeDevice` signals sub-device connection state and bounds the first
+  connect** (#36). Each sub-device carries a `ConnectionState`
+  (`CONNECTING → CONNECTED → DEGRADED → DISCONNECTED`), read with
+  `status()` / `status(dev)` and observed with `on_status_change(callback)`;
+  every transition is logged (WARNING for `DEGRADED` / `DISCONNECTED`).
+  New config key `connect_deadline_s` (default 10 s): a server that does not
+  answer within it raises `DeviceUnreachableError(dev, address, elapsed_s)`
+  from the constructor instead of running attempts × timeout × backoff
+  (~40 s per dead server before). Under thread affinity the first connects
+  run in parallel on the sub-devices' own threads. `reconnect()` reuses the
+  sub-device's configured `timeout_ms` / `attempts` / `backoff_s` instead of
+  the 50 s defaults; a call timeout marks the sub-device `DEGRADED` before
+  the rebuild. `_DeviceTCPIPClient.set_default_timeout(ms)` added.
+- **Default logging for experiments and composites** (#37): nothing is
+  silent anymore without module code. `Experiment` logs
+  `Run <id>: N step(s) (k already completed), run dir …`, `Setup: …`,
+  `Step i/N <id> (<op>)` / `Step i/N done in x s`, `Teardown: …`, and
+  `Run … completed (N steps) in x s`. `CompositeDevice` logs
+  `Connecting <dev> @ <addr>` / `Connected <dev> @ <addr> in x s`, each failed
+  attempt with the retry delay, `Reconnecting <dev> @ <addr>`, and the timeout
+  a retried call ran under. New `plesty.lib.utils.logger.ensure_logging()`
+  (root level + one console sink, idempotent) is called by both constructors;
+  `setup_logging(app)` builds on it and adds the per-process log file — the
+  first `app` names the file, so a rig built before its experiment no longer
+  steals the file name.
+
 ## 0.3.5.dev1 — 2026-08-17
 
 Development pre-release for field-testing the run convention (#34); the

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: plesty-lib
-Version: 0.3.5.dev1
+Version: 0.3.5.dev3
 Summary: A library for the Plesty ecosystem.
 Author: Plesty Development Team
 Author-email: Yunshuang Yuan <yunshuang.yuan@fkp.uni-hannover.de>

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/docs/device/base_device.md

@@ -261,7 +261,12 @@ model.standard_methods   # the common device API (connect/write/query/reset/...)
 `functions` unifies the two ways a device gains operations: schema-registered
 operations (`source="schema"`) and `@expose_to_api` methods
 (`source="custom"`); framework plumbing is kept out and surfaced separately
-under `standard_methods` (`source="standard"`). Per-parameter and return
+under `standard_methods` (`source="standard"`). Each `FuncDoc` also carries
+its `kind` — what the operation does to the world (`read`, `motion`,
+`acquire`, `lifecycle`, `control`, `configure`), declared with
+`@expose_to_api(kind=..., position_key=..., step=...)` or a `"kind"` block in
+`schema_func.json` and used by the field test to drive the operation
+(see *test_helper*, "Kinds"). Per-parameter and return
 descriptions are parsed from Google-style docstring `Args:`/`Returns:`
 sections — write those in your device methods and the generated documentation
 inherits them.

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/docs/device/composite_device.md

@@ -60,7 +60,9 @@ composite = CompositeDevice(
 |---|---|
 | `address` | Explicit ZMQ address, skipping the environment |
 | `env` | Environment variable to read, overriding the derived name |
-| `timeout_ms`, `attempts`, `backoff_s` | Forwarded to `connect_client` |
+| `timeout_ms`, `attempts`, `backoff_s` | Forwarded to `connect_client` (first connect and every reconnect) |
+| `connect_deadline_s` | Budget for the *first* connect (default 10 s); past it the constructor raises `DeviceUnreachableError` |
+| `requires` | Remote methods the server must expose; checked by `preflight()` (a wrong address connects fine but is not this device) |
 
 The address comes from `address` if given, else the environment; an unset
 variable raises a `ValueError` naming the variable to set.
@@ -106,10 +108,48 @@ explicitly.
 `connect_client(address)` is the module-level builder used for the rebuilds:
 `build_client` plus a connection-handshake retry with linear backoff, which
 matters when several clients open back-to-back against the same server.
+`reconnect(dev)` rebuilds with the sub-device's own `timeout_ms` /
+`attempts` / `backoff_s`, not the defaults.
 
 Every multi-device composite gets this robustness for free instead of
 re-implementing it per module.
 
+## Connection state and a bounded first connect
+
+The composite is never silent about its servers. Constructing it logs one
+line per sub-device (`Connecting hwp @ tcp://… (timeout 5000 ms) …`,
+`Connected hwp @ tcp://… in 0.4 s`, or `Connect to tcp://… failed, attempt
+1/4: … ; retrying in 2 s`) and a summary (`PolPlRig ready in 1.2 s:
+hwp=tcp://…, spec=tcp://…`) — the console sink is switched on by the
+composite itself (`ensure_logging()`), so this shows even when the composite
+is built before the experiment that would configure logging.
+
+The **first connect is bounded**: each configured sub-device gets
+`connect_deadline_s` (default 10 s); a server that does not answer within it
+raises `DeviceUnreachableError(dev, address, elapsed_s)` from the constructor
+instead of grinding through attempts × timeout × backoff. Under thread
+affinity the first connects run in parallel on the sub-devices' own threads,
+so three dead servers cost one deadline, not three. Reconnects during a run
+keep the attempt ladder — a server busy finishing a long move must be waited
+for, not declared dead.
+
+Each sub-device carries a `ConnectionState` — `CONNECTING → CONNECTED →
+DEGRADED` (a call timed out, the client is being rebuilt) `→ DISCONNECTED`
+(connect/reconnect failed, or `disconnect_all()`):
+
+```python
+composite.status()          # {"hwp": ConnectionState.CONNECTED, "spec": ...}
+composite.status("hwp")     # ConnectionState.CONNECTED
+
+@composite.on_status_change
+def _show(dev, old, new, detail):
+    ui.set_led(dev, new)    # runs on the thread that changed the state
+```
+
+Every transition is logged (INFO, or WARNING for `DEGRADED` /
+`DISCONNECTED`) whether or not anyone subscribes; a subscriber that raises
+is logged and ignored — a broken UI hook must not break the rig.
+
 ## One thread per sub-device
 
 ZMQ sockets are not thread-safe, and even migrating one between threads is
@@ -161,6 +201,16 @@ for a data directory either fails or means nothing:
 composite.set_data_path(frames_dir, devices=["spec"])
 ```
 
+## Preflight
+
+`preflight()` proves each sub-device is the device it is supposed to be and
+answers: a remote sub-device must expose every method in its `requires`
+list and its `identity()` must reply; a local one must have the required
+attributes. It returns `{name: problem}` for the failures (empty when the
+rig is ready) and logs one line per sub-device either way. `Experiment`
+runs it in `setup()`; `plesty.lib.experiment.connect_rig` runs it at
+start-up and asks the operator what to do (see the experiment guide).
+
 ## Interplay with experiments
 
 `Experiment` accepts a `CompositeDevice` at construction; its default

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/docs/device/test_helper.md

@@ -455,7 +455,7 @@ if __name__ == "__main__":
 | 2 | `test_connect_lifecycle` | connect / identity / `check_errors` / disconnect / reconnect, each timed; identity stable across the reconnect |
 | 3 | `test_param_roundtrip` | every writable parameter round-trips; the value read back is written back, so device state is unchanged |
 | 4 | `test_param_constraints` | hardware MIN/MAX and option sets against the schema envelope — a schema wider than the hardware fails |
-| 5 | `test_functions` | every operation on the safe allow-list returns a `dict` |
+| 5 | `test_functions` | every operation on the safe allow-list can be called — by kind: a motion one step out and back, a lifecycle op once, a control op once at rest after the motions |
 | 6 | `test_buffer_drain` | a query abandoned mid-flight must not leave its answer for the next query to consume |
 | 7 | `test_error_recovery` | the device recovers from a forced timeout, and the first command after a reconnect succeeds |
 | 8 | `test_stability` | every queryable parameter and safe op repeated `repetitions` times to collect the statistics |
@@ -464,6 +464,29 @@ if __name__ == "__main__":
 (a `"safe": true` entry in `schema_func.json` is forwarded automatically). Without an allow-list
 the function gate is skipped — no motion or emission operation ever runs unattended.
 
+**Kinds.** *How* an allow-listed operation is called comes from what the module declares it
+does — `@expose_to_api(kind=...)` or a `"kind"` block in `schema_func.json`, read back as
+`OpKind` (`plesty.lib.device.device_utils`):
+
+| kind | the gates |
+|---|---|
+| `read` (default) | called with its declared payload, repeated as it is |
+| `motion` | driven from what `position_key` reports: `target = start ± step` (`relative=True` sends the step itself), then put back; the functions gate fails if it does not come back, the stability gate repeats out-and-back; a motion `slow_op` is a delta from the start and is moved back after the provocation |
+| `acquire` | as `read`, but recorded as acting |
+| `lifecycle` (home, reset) | once, when allow-listed; never repeated |
+| `control` (stop, abort) | once, at rest, after the motions; skipped when nothing moved |
+| `configure` | never called — the round-trip gate covers settings |
+
+```python
+@expose_to_api(kind="motion", position_key="MO.Position", step=1.0)
+def move_absolute(self, position: float) -> float: ...
+```
+
+`op_kinds={"move_absolute": OpKind("motion", position_key="MO.Position", step=1.0)}` on the
+pipeline overrides the declaration wholesale. A motion without `position_key`, `target` (the
+first required argument by default) or `step` fails the gate rather than guessing how far a part
+may move.
+
 **Key parameters:**
 
 | Parameter | Default | Description |
@@ -471,6 +494,7 @@ the function gate is skipped — no motion or emission operation ever runs unatt
 | `repetitions` | `20` | Per-command repeat count in the stability gate |
 | `report_path` | `None` | Base path; `run()` writes `<base>.jsonl` (appended) and `<base>.md` |
 | `safe_ops` / `unsafe_ops` | `None` | Operation allow-list and opt-out list |
+| `op_kinds` | `None` | Per-operation `OpKind` overriding the module's declaration |
 | `op_payloads` | `None` | Explicit payload per operation instead of a generated one |
 | `ignore_keys` | `None` | Config keys excluded from every parameter gate |
 | `drain_probe` | auto | The two distinguishable keys used by the drain and recovery gates |

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/docs/experiment.md

@@ -84,6 +84,53 @@ Step methods may be sync or async; results are awaited automatically. Each
 step is retried (`max_retries`, `retry_sleep` constructor arguments) before
 the run aborts.
 
+## Starting on a rig that is really there
+
+An experiment's `__main__` builds the rig with `connect_rig` — the same
+start for every experiment: connect the servers, run the composite's
+`preflight()` (each sub-device exposes the methods its configuration
+`requires` and answers `identity()`), and if anything is missing, tell the
+operator which device, at which address, from which variable — and ask:
+
+```python
+from plesty.lib.experiment import connect_rig
+
+rig, skipped = connect_rig(
+    lambda skip: PolPlRig(with_powermeter="pm" not in skip),
+    optional={"pm"},                       # the experiment can run without it
+)
+```
+
+```text
+The rig is not ready — 1 device(s) failed the preflight:
+  - pm: no answer from tcp://127.0.0.1:5557: TimeoutError('Server did not respond in time')
+      address tcp://127.0.0.1:5557 (from PM_ADDRESS in .env / the environment)
+Check that the pm server(s) are running and that PM_ADDRESS point at them
+(a wrong port connects fine but is not the expected device).
+[r] retry (after fixing .env or starting the server)  [a] abort  [s] continue without pm
+>
+```
+
+`r` rebuilds the rig (an edited `.env` or a freshly started server is picked
+up), `a` raises `RigNotReadyError`, `s` — offered only when every failing
+sub-device is in `optional` — returns the rig without them and names them in
+`skipped`, so the experiment can drop the matching part of its plan. Without
+a terminal on stdin nothing waits: the diagnosis is logged and
+`RigNotReadyError` raised.
+
+The check also runs inside `Experiment.setup()`: a run never starts on a
+composite whose `preflight()` reports a problem, whether or not `connect_rig`
+was used. Declare what each sub-device must be able to do in its
+configuration — that is what turns "the port answers" into "it is the
+stage":
+
+```yaml
+devices:
+  hwp:  {timeout_ms: 5000, requires: [home_stage, move_absolute, get_position]}
+  spec: {timeout_ms: 5000, requires: [acquire, get_recent_file, write, query]}
+  pm:   {timeout_ms: 5000, requires: [measure_power, write]}
+```
+
 ## Data on a shared disk
 
 Blobs a device writes itself (spectrometer frames, camera images) never

{plesty_lib-0.3.5.dev1 → plesty_lib-0.3.5.dev3}/gl-secret-detection-report.json

@@ -19,19 +19,19 @@
       "version": "8.30.1"
     },
     "type": "secret_detection",
-    "start_time": "2026-08-17T12:45:23",
-    "end_time": "2026-08-17T12:45:24",
+    "start_time": "2026-08-18T13:39:46",
+    "end_time": "2026-08-18T13:39:47",
     "status": "success",
     "observability": {
       "events": [
         {
           "event": "collect_secrets_analyzer_scan_metrics_from_pipeline",
-          "time_s": 0.469797198,
+          "time_s": 0.527730295,
           "exit_code": 0,
           "git_strategy": "FetchShallow",
-          "repo_size_kb": 685,
+          "repo_size_kb": 709,
           "commit_count": 1,
-          "bytes_scanned": 314,
+          "bytes_scanned": 317,
           "pipeline_type": "Tag"
         },
         {