plesty-lib 0.2.8__tar.gz → 0.2.9__tar.gz

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/.gitlab-ci.yml

@@ -17,7 +17,7 @@ stages:
   - release
 
 include:
-  - component: $CI_SERVER_FQDN/plesty/plesty-ci/plesty-standard-ci@exp
+  - component: $CI_SERVER_FQDN/plesty/plesty-ci/plesty-standard-ci@main
     inputs:
       standard: quantum
       extra_branch: exp

plesty_lib-0.2.9/CHANGELOG.md

@@ -0,0 +1,42 @@
+# Changelog
+
+## 0.2.9 — 2026-07-24
+
+- update CI component reference to branch `main`.
+- Pin `[tool.plesty] standard = "quantum"` explicitly: core modules always check at quantum regardless of release version, now that the SDK derives the default standard from the version when unset (sdk#19).
+- Documentation overhaul: the landing page now presents the whole library (devices → data → experiments) instead of devices only; new dedicated guides for the experiment framework (`docs/experiment.md` — plans, journaling, resume, `ExperimentPipeline` contract gates) and `CompositeDevice` (`docs/device/composite_device.md` — resilient remote calls); `doc_model()` documented in the base-device guide and `experiment_pipeline` in the test helpers; the three architecture figures are redrawn as SVGs in the PLESTY palette (the synchronized-device figure now includes the telemetry system); fixed the install instructions (`plesty-lib[dev]` is not an extra) and filled the empty contributing page. The generated API reference now documents `plesty.lib` itself (the deprecated `plestylib` shim is excluded via the SDK's new `api-ignore` setting) and sits as the last item of the navigation; several docstrings were fixed for the zero-warning docs build (RST-safe SCPI `*IDN?`/`*OPC?` mentions, resource-tree examples as literal blocks, experiment-pipeline gate summary as a list).
+- Add a structured, render-agnostic device documentation model (sdk#18): `BaseDeviceSyncModel.doc_model()` returns a `DeviceDocModel` with (1) `parameters` grouped by config group (from the parameter system), (2) `functions` — the device's own operations as unified `FuncDoc` objects, combining schema-registered operations (`source="schema"`) and the device's `@expose_to_api` methods (`source="custom"`), and (3) `standard_methods` — the common device API every device shares (`connect`/`write`/`query`/`reset`/…, `source="standard"`). Backed by new `ConfigSystem.parameter_docs()` and `FunctionSystem.function_docs()`/`standard_method_docs()`; `summary()` is unchanged (renderers can migrate to `doc_model()` later). Framework plumbing is kept out of `functions` and surfaced under `standard_methods`.
+- Enrich `doc_model()` method documentation (sdk#18): `_method_doc` now parses Google-style docstring `Args:`/`Returns:` sections so per-parameter and return descriptions populate `FuncDoc` (previously empty for `@expose_to_api` and standard methods), and `*args`/`**kwargs` are excluded from the documented parameter list. `BaseDeviceSyncModel.write`/`query` gained proper `Args:`/`Returns:` docstrings so the standard methods document their `key`/`value` parameters.
+- Acquire the per-device resource lock on connect, not construction (sdk#18): `ResourceRegistry.acquire()` moved from `BaseDeviceSyncModel.__init__` to `__enter__` (released in `__exit__`, and on a failed connect). Constructing a device is now a side-effect-free operation — two instances of the same device id can coexist and only conflict when both connect — which lets documentation generators and offline tooling instantiate a device without holding hardware locks. `ResourceRegistry.acquire()` now also reclaims a key whose previous holder was garbage-collected (a dead weak reference no longer counts as in-use).
+
+- `CompositeDevice` gains resilient remote calls (lib#15): `call(dev, func, ...)` invokes a sub-device method and, on a `TimeoutError`, rebuilds that client (flushing the stale ZMQ reply left in the socket) and retries before the final timeout propagates. A new `reconnect(dev)` and the module-level `connect_client(address, ...)` (build-with-connection-handshake-retry) back it; `__init__` accepts an optional `addresses` mapping so remote sub-devices are reconnectable. Every multi-device composite (e.g. hub scanners) gets this for free instead of re-implementing it.
+- `FunctionSystem` now logs every registered-operation execution at INFO level, symmetric with the `ConfigSystem` write log: `[<device-id>] Running operation <name>(<param>=<value>, ...)`. Parameter values are rendered with a size-bounded repr so array-valued arguments do not flood the log.
+- Add `plesty.lib.device.telemetry` — the unified operational-telemetry schema (lib#4): `DeviceStatus` (pull-style point-in-time snapshot) and `TelemetryEvent` (push-style timestamped record of kind `reading`/`state_change`/`error`/`warning`), plus crash-tolerant JSONL persistence helpers (`append_events`/`read_events`, same format as the experiment journal). The `TelemetrySystem` mixin composes into `BaseDeviceSyncModel` alongside `ConfigSystem`/`FunctionSystem`: every device gets consumer-registered event hooks, `emit_telemetry()`, and a free `status()` snapshot assembled from the connection check and cached parameter values (no device I/O); `write`/`query` emit `state_change`/`reading` events automatically when a hook is registered. No polling loops, threads, or transports — consumers own the event lifecycle.
+- Formalise the experimental-output metadata schema (lib#4): the JSON document written by `save_result` is now the typed `ResultDocument` model (`plesty.lib.data.io`) with validated result types and an automatic write-time stamp; the on-disk layout is unchanged. New `load_document()` reads a result's metadata without any blob I/O; `load_result` routes through the model.
+- Document all three data/metadata schema domains (device configuration, operational telemetry, experimental output) in `docs/data_schemas.md`.
+
+## 0.2.8 — 2026-07-07
+
+- Add `plesty.lib.test.experiment_pipeline.ExperimentPipeline` (lib#14) — the experiment-tier counterpart of `DevicePipeline`: five hardware-free contract gates (public `Experiment` subclass instantiable without devices; deterministic `build_plan()` by content hash; every `Step.op` resolves via the framework's own plan validation; `Plan.save`/`load` round-trip preserves the hash; `setup`/`teardown` overrides are async) plus a one-shot `run_mock_pipeline()`. Experiment HUB modules expose one pytest function per gate; SDK gate E1 (plesty-sdk#15) verifies their presence and passage, mirroring gate d1.
+
+- `Experiment.run` now validates the plan before anything executes: every `Step.op` must name an existing public callable on the experiment and may not be a reserved lifecycle method (`build_plan`, `run`, `setup`, `teardown`) — violations raise the new `InvalidOperationError` listing all offending steps, before any device is touched or journal written. The class docstring now spells out the override contract (`build_plan` required; `setup`/`teardown` optional extensions).
+
+- Add `plesty.lib.experiment` — async `Experiment` ABC with atomic scheduling, status journaling and checkpoint/resume (first step toward lib#3): `Plan`/`Step` define the frozen measurement schedule (content-hashed so a resumed run refuses a changed schedule); `Journal` is a crash-safe append-only JSONL status log replayed to derive run state and the completed-step set; `Experiment` provides the async `setup()`/`run()`/`teardown()` lifecycle with per-step retries, clean handling of cancel/error, and resume via `run(resume=<run_id>)` that skips journaled-complete steps.
+- Add `plesty.lib.data.io` (first step toward lib#4): `save_result`/`load_result` persist measurement results crash-safely — blobs stay in their original raw format (`.npy` for `PlestyArray`, native encoding for image bytes) described by a JSON metadata document (Plesty meta, provenance, timestamp) with a reference link to the blob; `convert_to_hdf5` optionally packs such documents into a unified HDF5 archive (requires the optional `h5py` dependency).
+
+- Surface `op_schema`-registered functions through the `describe` endpoint (lib#13): `DeviceTCPIPServer._describe` now merges the `FunctionSystem._functions` registry into the method listing. These ops are installed as instance attributes and were therefore invisible to the MRO-based `@expose_to_api` walk, so remote clients could not discover or bind them (though they were already callable via the generic `call` path). `DeviceTCPIPClient` now binds them as local proxy methods.
+- Migrate licensing to the [REUSE](https://reuse.software/) standard (lib#12, follows plesty-sdk#10): Python sources carry inline SPDX headers with a collective `SPDX-FileCopyrightText` (Plesty Development Team) plus a `SPDX-FileContributor` per author; non-code files are covered by a central `REUSE.toml`. The root `LICENSE` is kept for forge (GitLab) license detection; `COPYING`, `spdx.tmpl` and the unused `GPL-3.0-or-later.txt` were removed. `plesty check` Gate 9 now verifies `reuse lint` and authorship currency.
+
+## 0.2.7 — 2026-07-02
+
+- Add `EnvSettings` in `plesty.lib.utils.settings`: a dependency-free helper that loads environment variables (optionally seeded from a `.env` file, process env taking precedence) with typed `get`/`require` access. By default `load()` also registers parsed `.env` values into `os.environ` without overriding existing ones (dotenv-style; pass `export=False` to disable). Generic by design — device modules choose their own variable names and defaults for credential/connection info (lib#11, companion to plesty-sdk#13)
+- Expose `identity`, `check_errors`, `check_operatability`, `reset`, `clear` to the remote API via `@expose_to_api` on `BaseDeviceSyncModel`
+- Fix `TCPIPTrafficManager._send_command`: pass actual `timeout` to `receive_one` instead of the boolean flag `timeout_overrided` (lib#9)
+- Fix `sock_smart_read` JSON framing: exit the read loop when `terminator in msg` (consistent with line-data branch) and return `msg.rstrip()` — the previous `rstrip(terminator)` only stripped `\n`, leaving `\r` and causing the brace check to fail for `\r\n`-terminated responses (e.g. AMC300) (lib#9)
+- Fix `DeviceTCPIPClient._send`: drain stale ZMQ reply after `recv()` timeout (100 ms window) so the client remains usable on the next call (lib#10)
+
+## 2026-06-25
+
+- Add `FuncMeta` and `SCPISolver.solve_func` for schema-driven operation dispatch
+- Fix mypy type annotations across all public modules
+- Migrate CI pipeline to `plesty-standard-ci@exp`

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: plesty-lib
-Version: 0.2.8
+Version: 0.2.9
 Summary: A library for the Plesty ecosystem.
 Author: Plesty Development Team
 Author-email: Yunshuang Yuan <yunshuang.yuan@fkp.uni-hannover.de>

plesty_lib-0.2.9/docs/api_reference.md

@@ -0,0 +1,19 @@
+# API Reference
+Entry page for the [full `plesty.lib` API](reference/plesty/lib/index) documentation.
+
+## Core Packages
+
+All packages live under the `plesty.lib` namespace (import as
+`plesty.lib.<package>`).
+
+| Package | Description | 
+| --- | --- | 
+| [`data`](reference/plesty/lib/data/index) | Data containers, units, type helpers, and result persistence 
+| [`device`](reference/plesty/lib/device/index) | Base device model, parameter/function/telemetry systems, async wrappers, composite devices |
+| [`experiment`](reference/plesty/lib/experiment/index) | Async experiment lifecycle: `Experiment` ABC, `Plan`/`Step` schedules, crash-safe `Journal`, checkpoint/resume |
+| [`traffic`](reference/plesty/lib/traffic/index) | Transport backends (TCP/IP, VISA, Serial) |
+| [`solver`](reference/plesty/lib/solver/index) | Command and operation solvers |
+| [`service`](reference/plesty/lib/service/index) | TCP/IP server/client and service builders |
+| [`sim`](reference/plesty/lib/sim/index) | Synthetic data generation for testing |
+| [`utils`](reference/plesty/lib/utils/index) | Shared utilities (env settings loader, logging, errors, registry, DLL helpers) |
+| [`test`](reference/plesty/lib/test/index) | Test helpers |

plesty_lib-0.2.9/docs/contributing.md

@@ -0,0 +1,28 @@
+# Contributing
+
+PlestyLib is developed under the PLESTY governance standard. If you want to
+contribute — a fix, a feature, or a whole new HUB module built on the
+library — start with the
+[PLESTY Contributor Guide](https://docs.plesty.net/contributor-guide/), which
+covers the full workflow end to end.
+
+The short version:
+
+1. **Branching** — work never lands on `main` directly. Branch from the
+   up-to-date `exp` branch (`git checkout exp && git pull && git checkout -b
+   <type>/<slug>`), implement, and merge back into `exp`.
+2. **Quality gates** — every change must pass `uv run plesty check` (the
+   `quantum` standard for core modules: formatting, typing, docstrings,
+   tests, licensing, and more). The pre-push hook installed by `plesty init`
+   runs it automatically.
+3. **Changelog** — add an entry under `## Unreleased` in `CHANGELOG.md` in
+   the same commit as your change.
+4. **Licensing** — the project follows the [REUSE](https://reuse.software/)
+   standard: Python sources carry SPDX headers with a
+   `SPDX-FileContributor` line per author; add yourself when you touch a
+   file.
+
+Issues and merge requests live on
+[gitlab.com/plesty/core/plesty-lib](https://gitlab.com/plesty/core/plesty-lib).
+For questions about the module ecosystem, browse the searchable
+[module registry](https://hub.plesty.net).

plesty_lib-0.2.9/docs/data_schemas.md

@@ -0,0 +1,89 @@
+<!--
+SPDX-FileCopyrightText: 2026 Plesty Development Team
+SPDX-FileContributor: Yunshuang Yuan
+
+SPDX-License-Identifier: LGPL-3.0-or-later
+-->
+
+# Data & Metadata Schemas
+
+`plesty-lib` defines one schema contract per data domain so that every HUB
+module acquires, stores, and exchanges data in the same shape. Three domains
+are covered (issue lib#4):
+
+| Domain | Schema | Module |
+|---|---|---|
+| Device configuration | `ConfigParameter` / `ConfigGroup` | `plesty.lib.device.params` |
+| Operational telemetry | `DeviceStatus` / `TelemetryEvent` | `plesty.lib.device.telemetry` |
+| Experimental output | `ResultDocument` | `plesty.lib.data.io` |
+
+## Device configuration
+
+Structured representation of device parameters and settings — see
+[Parameter System](device/param_system.md). Every parameter is a
+`ConfigParameter` (name, dtype, unit, constraints, command binding) organised
+into `ConfigGroup`s and managed by the `ConfigSystem` that every device model
+composes. Device modules declare their parameters in a YAML/JSON schema which
+`ConfigSystem` materialises at init.
+
+## Operational telemetry
+
+Real-time status, state, and diagnostic data, defined in
+`plesty.lib.device.telemetry`. Two shapes cover pull and push:
+
+* **`DeviceStatus`** — a point-in-time snapshot. Every device inherits
+  `status()` via the `TelemetrySystem` mixin; the default implementation is
+  assembled from the connection check and the cached configuration values
+  (no device I/O), so it is always cheap to call:
+
+  ```python
+  status = device.status()
+  status.connection   # "connected" | "disconnected" | "connecting" | "error"
+  status.state        # "ready", "busy", ... (override device_state() to refine)
+  status.parameters   # last-known parameter values
+  status.last_error   # most recent error message, if any
+  ```
+
+* **`TelemetryEvent`** — a timestamped record of kind `reading`,
+  `state_change`, `error`, or `warning`. Consumers register a hook; the base
+  device emits `state_change`/`reading` events automatically on every
+  successful `write`/`query`, and devices can emit custom events with
+  `emit_telemetry()`:
+
+  ```python
+  events = []
+  device.register_telemetry_hook(events.append)
+  device.write("wavelength", 850.0)   # emits a state_change event
+
+  from plesty.lib.device.telemetry import append_events
+  append_events(run_dir / "telemetry.jsonl", events)
+  ```
+
+Hooks run synchronously and never disturb device operation (a failing hook is
+logged and skipped). When no hook is registered, no event is constructed.
+`append_events`/`read_events` persist events in the same crash-tolerant
+append-only JSONL format as the experiment journal. Polling loops, threads,
+and transports are deliberately out of scope — they belong to monitors and
+services, not to the schema.
+
+## Experimental output
+
+Acquired data plus its metadata, defined in `plesty.lib.data.io`. The storage
+contract keeps blobs in their **original raw format** (`.npy` for
+`PlestyArray`, native encoding for image bytes) described by a JSON metadata
+document — the typed `ResultDocument` model — holding the Plesty meta (name,
+unit, range, description), timestamps, provenance, and a reference link to
+the blob:
+
+```python
+from plesty.lib.data import save_result, load_result, load_document
+
+path = save_result(array, run_dir / "step_0001", provenance={"step_id": "s1"})
+document = load_document(path)   # metadata only — no blob I/O
+result = load_result(path)       # reconstructed PlestyArray
+```
+
+The JSON document is the commit record: a result exists once its `.json`
+file does, and writes are crash-safe (blob first, document last, both
+atomic). `convert_to_hdf5` optionally packs a set of documents into a single
+HDF5 archive for exchange (requires the optional `h5py` dependency).

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/docs/device/base_device.md

@@ -19,7 +19,7 @@ When your device class inherits from `BaseDeviceSyncModel`, you get:
 3. Built-in high-level methods such as `write`, `query`, `state`, and `summary`.
 4. Context manager support (`with device:`) that calls `init`, `connect`, and `disconnect`.
 5. Resource tracking through `ResourceRegistry` to avoid duplicate ownership.
-6. Per-instance resource locking (`_locked_resources`, `_clients`) for multi-client access control.
+6. Per-device resource management (`register_resources`, `allocate_resource`, …) for multi-client access control.
 
 ## Typical Layering
 
@@ -128,7 +128,8 @@ Useful built-in helpers:
 
 1. `state` / `get_state()`: query all registered parameters and return a dictionary.
 2. `synchronize_param_from_device(keys=None, sync_constraints=False)`: pull current values from hardware into the model.
-3. `summary()`: generate a textual API overview.
+3. `summary(style=...)`: render a textual API overview (`short`, `md`, or `google`).
+4. `doc_model()`: return the structured documentation model (see below).
 
 If your device supports querying dynamic constraints, implement:
 
@@ -160,33 +161,64 @@ asyncio.run(main())
 
 ## Multi-Client Resource Control
 
-`BaseDeviceSyncModel` carries three resource-tracking attributes that support exclusive access
+`BaseDeviceSyncModel` embeds resource tracking that supports exclusive access
 control when a device is shared by multiple clients (for example, over a TCP/IP service):
 
-| Attribute | Type | Purpose |
+| Member | Type | Purpose |
 |---|---|---|
-| `_resources` | `list` | Hardware resources registered for automatic release on disconnect |
-| `_locked_resources` | `set` | Resources that are currently allocated to any client |
-| `_clients` | `dict` | Maps `client_id → [allocated resources]` |
+| `_resources` | `list` | Inventory of physical resources this device owns; passed to the TCP server for per-client access control |
+| `_resource_manager` | `ResourceManager \| None` | Embedded manager for direct (non-TCP) clients, created lazily by `register_resources()` |
 
-These attributes mirror the `ResourceManager` pattern from `plesty.lib.service.resource_manager`.
-You can compose a `ResourceManager` inside your device subclass and delegate to its methods, or
-override `_user_permission_check` to enforce access rules based on the per-instance state.
+Four public methods delegate to the embedded `ResourceManager`
+(`plesty.lib.service.resource_manager`):
+
+1. `register_resources(resources)`: declare the resources the device owns (flat list, set, or nested dict tree).
+2. `allocate_resource(resources, client_id)`: request exclusive access; returns `False` if any requested resource is already held by another client.
+3. `release_resource(client_id)`: free every resource held by that client.
+4. `client_resources(client_id)`: canonical resource paths currently allocated to that client.
 
 The default `_user_permission_check` always returns `True` (no restriction). Override it to
-implement real guard logic:
+implement real guard logic — query current allocations via `client_resources` or the embedded
+manager:
 
 ```python
 def _user_permission_check(self, user_resources: list, key: str) -> bool:
-    if not user_resources:
+    if not user_resources or self._resource_manager is None:
         return True
-    return any(r in self._locked_resources for r in user_resources)
+    locked = self._resource_manager._locked_resources
+    return any(r in locked for r in user_resources)
 ```
 
 Both `write` and `query` pass a `resources` kwarg through to `_user_permission_check` when the
 caller supplies it, so access control integrates transparently with the standard operation
 interface.
 
+## Documentation Model (`doc_model`)
+
+Every device exposes a structured, render-agnostic documentation model —
+the single source of truth for its user-facing documentation:
+
+```python
+model = device.doc_model()
+model.parameters         # {group_name: [ConfigParameter, ...]} from the parameter system
+model.functions          # the device's own operations as unified FuncDoc objects
+model.standard_methods   # the common device API (connect/write/query/reset/...)
+```
+
+`functions` unifies the two ways a device gains operations: schema-registered
+operations (`source="schema"`) and `@expose_to_api` methods
+(`source="custom"`); framework plumbing is kept out and surfaced separately
+under `standard_methods` (`source="standard"`). Per-parameter and return
+descriptions are parsed from Google-style docstring `Args:`/`Returns:`
+sections — write those in your device methods and the generated documentation
+inherits them.
+
+Renderers consume the model instead of re-deriving metadata: the SDK docs
+generator, `summary()`, and the server `describe` endpoint all build on it.
+Because constructing a device is side-effect-free (the per-device resource
+lock is only acquired on `connect`), documentation tooling can instantiate
+your device without touching hardware.
+
 ## Best Practices
 
 1. Keep transport logic in traffic managers, not in device classes.

plesty_lib-0.2.9/docs/device/composite_device.md

@@ -0,0 +1,60 @@
+# Composite Device
+
+`CompositeDevice` groups several devices — local instances and/or remote
+TCP/IP clients — behind one object, so an orchestration (typically an
+[Experiment](../experiment.md)) manages a whole setup as a single unit.
+
+## Building a composite
+
+```python
+from plesty.lib.device.composite_device import CompositeDevice, connect_client
+
+composite = CompositeDevice(
+    devices={
+        "stage": stage_device,                                   # local BaseDeviceSyncModel
+        "cam": connect_client("tcp://192.168.1.20:5555"),        # remote client
+    },
+    addresses={"cam": "tcp://192.168.1.20:5555"},                # remote devices that
+)                                                                # should be reconnectable
+```
+
+Sub-devices are reached by name (`composite.stage`, `composite.cam`).
+`connect_all()` / `disconnect_all()` handle every local sub-device, and the
+composite is a context manager:
+
+```python
+with composite:
+    print(composite.identity())   # {"stage": ..., "cam": ...}
+```
+
+Remote clients manage their own connection; the `addresses` mapping is only
+needed for remote sub-devices that should be *reconnectable* (below).
+
+## Resilient remote calls
+
+Long-running orchestrations must survive transient network hiccups. Instead
+of calling a remote sub-device method directly, route it through `call()`:
+
+```python
+frame = composite.call("cam", "acquire", exposure_s=0.1)
+```
+
+On a `TimeoutError`, `call()` rebuilds the sub-device's client — discarding
+the stale ZMQ reply left in the socket, which would otherwise poison every
+subsequent request — waits briefly, and retries (default 3 attempts) before
+the final timeout propagates. `reconnect(dev)` performs the same rebuild
+explicitly.
+
+`connect_client(address)` is the module-level builder used for the rebuilds:
+`build_client` plus a connection-handshake retry with linear backoff, which
+matters when several clients open back-to-back against the same server.
+
+Every multi-device composite gets this robustness for free instead of
+re-implementing it per module.
+
+## Interplay with experiments
+
+`Experiment` accepts a `CompositeDevice` at construction; its default
+`setup()`/`teardown()` call `connect_all()`/`disconnect_all()`. Step
+operations then drive sub-devices via `self.devices.<name>` or, for remote
+calls that should retry, `self.devices.call(...)`.

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/docs/device/index.md

@@ -11,6 +11,10 @@ If you are building a new device API, read in this order:
 7. [Logging System](logging_system.md)
 8. [Test Helpers](test_helper.md)
 
+To orchestrate several devices as one unit (the shape an
+[Experiment](../experiment.md) consumes), see
+[Composite Device](composite_device.md).
+
 ## Topic Guide
 
 ### Core Architecture
@@ -18,6 +22,7 @@ If you are building a new device API, read in this order:
 1. [Base Device](base_device.md): Defines the synchronized device abstraction and lifecycle.
 2. [Traffic Manager](traffic_manager.md): Implements transport communication (Serial, VISA, TCP/IP).
 3. [Command Solver](cmd_solver.md): Maps standardized operations to protocol commands.
+4. [Composite Device](composite_device.md): Groups local and remote devices with resilient remote calls.
 
 ### Metadata and API Contracts
 

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/docs/device/logging_system.md

@@ -80,18 +80,17 @@ from plesty.lib.device.base_tcp_scpi_device import BaseTCPScpiDevice
 
 
 class DemoScpiDevice(BaseTCPScpiDevice):
-	def __init__(self, host: str, port: int):
-		super().__init__(host=host, port=port)
-		self.register_config("POWER", dtype=float, read_only=True, command="MEAS:POW")
+    def __init__(self, host: str, port: int):
+        super().__init__(host=host, port=port)
+        self.register_config("POWER", dtype=float, read_only=True, command="MEAS:POW")
         logging.info("Initialization finished.")
-    
-    def some_utility_function():
+
+    def some_utility_function(self) -> None:
         try:
             # do something
             logging.debug("xxx done")
         except Exception as e:
             logging.error(f"Error: {e}")
-
 ```
 
 Why this helps developers:

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/docs/device/test_helper.md

@@ -13,6 +13,8 @@ and surface obvious breakage quickly.
 | `grouped_param_keys` | pytest fixtures that verify grouped parameter key resolution |
 | `resource_allocation` | Multiprocessing helpers for testing multi-client resource contention |
 | `schema_params` | Helpers for refreshing a JSON parameter schema from a real device |
+| `device_pipeline` | Device-tier contract gates (SDK gate d1) |
+| `experiment_pipeline` | Experiment-tier contract gates (SDK gate E1), hardware-free |
 
 ---
 
@@ -285,6 +287,93 @@ The function is marked `__test__ = False` so pytest does not auto-collect it.
 
 ---
 
+## 6. `device_pipeline` — Device Contract Gates
+
+**Location:** `plesty.lib.test.device_pipeline`
+
+The standard test pipeline for a PLESTY device API module (SDK gate d1). Construct a
+`DevicePipeline` with the device class, its constructor arguments, and the schema paths,
+then call the individual gate methods from pytest test functions — or `run_mock_pipeline()`
+to run all mock gates in one shot.
+
+```python
+# tests/test_pipeline.py of a device HUB module
+from plesty.lib.test.device_pipeline import DevicePipeline
+
+PIPELINE = DevicePipeline(
+    MyDevice,
+    "device-id",
+    param_schema="assets/param_schema.json",
+    op_schema="assets/op_schema.json",
+)
+
+def test_schema_integrity():
+    PIPELINE.test_schema_integrity()
+
+def test_params_mock():
+    PIPELINE.test_params_mock()
+
+# ... one pytest function per gate
+```
+
+| Gate | Method | Checks |
+|---|---|---|
+| 1 | `test_schema_integrity` | `param_schema.json` / `op_schema.json` are well-formed (valid JSON, no duplicate keys, valid types) |
+| 2 | `test_param_key_resolution` | every key in `get_config_list()` resolves via `get_config(key)` |
+| 3 | `test_params_mock` | all config params round-trip through the mock solver |
+| 4 | `test_funcs_mock` | all registered operations accept generated payloads |
+| 5 | `test_lifecycle` | context manager completes; `is_operatable` is `True` after connect |
+| 6 | `test_identity` | `identity()` returns a non-empty string |
+| 7 | `test_check_errors` | `check_errors()` returns an empty list on a healthy mock device |
+| 8 | `test_state_coverage` | `state` keys are a superset of `get_config_list()` |
+| 9 | `test_resource_allocation` | multi-client resource contention is enforced (optional, spawns server + 3 clients) |
+| 10 | `test_hardware_schema_refresh` | real hardware refreshes schema defaults/types (`@pytest.mark.hardware`, skipped in CI) |
+
+---
+
+## 7. `experiment_pipeline` — Experiment Contract Gates
+
+The experiment-tier counterpart of `DevicePipeline`: five hardware-free gates
+validating the plan/checkpoint contract of an
+[Experiment](../experiment.md) subclass. The pipeline never calls `run()`,
+so no instrument is needed.
+
+```python
+# tests/test_pipeline.py of an experiment HUB module
+from plesty.lib.test.experiment_pipeline import ExperimentPipeline
+
+PIPELINE = ExperimentPipeline(MyExperiment)
+
+def test_experiment_subclass():
+    PIPELINE.test_experiment_subclass()
+
+def test_plan_deterministic():
+    PIPELINE.test_plan_deterministic()
+
+def test_plan_ops_resolve():
+    PIPELINE.test_plan_ops_resolve()
+
+def test_plan_serializable():
+    PIPELINE.test_plan_serializable()
+
+def test_lifecycle_hooks():
+    PIPELINE.test_lifecycle_hooks()
+```
+
+| Gate | Checks |
+|---|---|
+| 1 `test_experiment_subclass` | public `Experiment` subclass, instantiable without devices |
+| 2 `test_plan_deterministic` | `build_plan()` twice yields the same `content_hash()` |
+| 3 `test_plan_ops_resolve` | every `Step.op` resolves via the framework's plan validation |
+| 4 `test_plan_serializable` | `Plan.save`/`load` round-trip preserves the hash |
+| 5 `test_lifecycle_hooks` | `setup`/`teardown` overrides are async |
+
+`run_mock_pipeline()` runs all gates in one shot. Experiment HUB modules
+expose one pytest function per gate — SDK gate E1 verifies their presence
+and passage, mirroring the device-tier gate d1.
+
+---
+
 ## Recommended Workflow
 
 1. Start with `device_param_system.auto_test(use_mock_solver=True)` after registering parameters —

{plesty_lib-0.2.8 → plesty_lib-0.2.9}/docs/device/traffic_manager.md

@@ -42,7 +42,7 @@ Common built-in traffic managers include:
 
 1. `SerialTrafficManager` for serial ports.
 2. `VisaTrafficManager` for VISA resources.
-3. `TcpIpTrafficManager` for TCP/IP connections.
+3. `TCPIPTrafficManager` for TCP/IP connections.
 
 ## Demo 1: Use the Built-in SerialTrafficManager
 

plesty_lib-0.2.9/docs/experiment.md

@@ -0,0 +1,119 @@
+# Experiment Framework
+
+`plesty.lib.experiment` turns a measurement routine into a **reproducible,
+crash-tolerant run**. The mental model is checkpointed training in deep
+learning: the *plan* is the training schedule, the *journal* is the training
+log, and the completed-step set plus persisted step results form the
+checkpoint.
+
+An `Experiment` orchestrates devices through a
+[`CompositeDevice`](device/composite_device.md) — it receives Device
+instances, never raw hardware.
+
+## The pieces
+
+| Piece | Role |
+|---|---|
+| `Step` | The smallest unit of work: a stable `id`, an operation name `op`, and its `params`. A step either completes (result persisted and journaled) or is re-run on resume. |
+| `Plan` | The frozen, ordered schedule of steps plus the run configuration. Written to disk at run start, never mutated; a SHA-256 `content_hash()` identifies the schedule. |
+| `Journal` | A crash-safe, append-only JSONL event log (`run_started`, `step_started`, `step_completed`, `step_failed`, `run_completed`/`_canceled`/`_aborted`). Replaying it derives the run state and the completed-step set. |
+| `Experiment` | The async ABC tying it together: validate → journal → execute with retries → persist results → resume. |
+
+Every run owns a directory:
+
+```text
+<run_root>/<run_id>/
+├── plan.json        # frozen schedule + config + content hash
+├── journal.jsonl    # append-only event log
+└── data/            # per-step results: raw blob + JSON metadata
+    ├── step_0000.json
+    └── step_0000.npy
+```
+
+Step results are persisted with
+[`save_result`](data_schemas.md) — the raw blob stays in its native format,
+described by a typed `ResultDocument` with provenance (`step_id`, `op`,
+`params`).
+
+## Writing an experiment
+
+Subclasses override exactly three lifecycle hooks — `build_plan()`
+(required), `setup()`/`teardown()` (optional, extend the defaults which
+connect/disconnect all devices) — plus one public method per step operation:
+
+```python
+import asyncio
+from plesty.lib.data import PlestyArray
+from plesty.lib.experiment import Experiment, Plan, Step
+
+class LineScan(Experiment):
+    def build_plan(self) -> Plan:
+        steps = [
+            Step(id=f"scan[x={x}]", op="scan_point", params={"x": x})
+            for x in range(5)
+        ]
+        return Plan(steps, config={"points": 5})
+
+    def scan_point(self, x: int) -> PlestyArray:
+        # drive devices via self.devices, return the measurement
+        return PlestyArray([float(x)], name="signal", unit="a.u.")
+
+run_id = asyncio.run(LineScan(run_root="runs").run())
+```
+
+Rules that keep runs reproducible:
+
+- **`build_plan()` must be deterministic** for a given configuration —
+  resume compares the regenerated plan against the persisted one by content
+  hash.
+- **Step ids must be stable** (e.g. `"scan[x=3,y=5]"`) — never derived from
+  run time or randomness; resume matches completed steps by id.
+- **Step operations are validated before anything runs**: every `Step.op`
+  must name an existing public method and may not be a reserved lifecycle
+  method (`build_plan`, `run`, `setup`, `teardown`). Violations raise
+  `InvalidOperationError` listing all offending steps — before any device is
+  touched or journal written.
+
+Step methods may be sync or async; results are awaited automatically. Each
+step is retried (`max_retries`, `retry_sleep` constructor arguments) before
+the run aborts.
+
+## Crash, cancel, resume
+
+`run()` returns the run id. If the run is interrupted — `Ctrl-C`, a crash,
+an aborting step — pass the id back to continue:
+
+```python
+run_id = asyncio.run(LineScan(run_root="runs").run())
+# ... interrupted?
+asyncio.run(LineScan(run_root="runs").run(resume=run_id))
+```
+
+On resume the framework:
+
+1. regenerates the plan and **refuses to continue if its content hash
+   differs** from the stored one (`PlanMismatchError`) — a silently changed
+   schedule or configuration cannot corrupt a run;
+2. replays the journal to find completed steps and skips them;
+3. journals the resume (`run_started` with `resumed: true`) and continues.
+
+`teardown()` is always awaited — on completion, failure, and cancellation —
+so devices are released no matter how the run ends.
+
+## Contract tests without hardware
+
+`plesty.lib.test.experiment_pipeline.ExperimentPipeline` verifies the
+plan/checkpoint contract of an experiment module with **no instrument
+attached** (it never calls `run()`). Five gates:
+
+| Gate | Checks |
+|---|---|
+| 1 `test_experiment_subclass` | public `Experiment` subclass, instantiable without devices |
+| 2 `test_plan_deterministic` | `build_plan()` twice → identical `content_hash()` |
+| 3 `test_plan_ops_resolve` | every `Step.op` resolves via the framework's own plan validation |
+| 4 `test_plan_serializable` | `Plan.save`/`load` round-trip preserves the hash |
+| 5 `test_lifecycle_hooks` | `setup`/`teardown` overrides are async |
+
+Experiment HUB modules expose one pytest function per gate (SDK gate E1
+verifies their presence and passage — the experiment-tier mirror of device
+gate d1); see [Test Helpers](device/test_helper.md) for the pattern.