plesty-lib 0.3.6__tar.gz → 0.4.0.dev1__tar.gz

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/CHANGELOG.md

@@ -2,6 +2,107 @@
 
 ## Unreleased
 
+- **`DevicePanel` builds a device's controls from its schema (#41).** `Panel`'s
+  docstring has named three implementations since it was written; only the live
+  view existed, so every module wanting controls in a window hand-wrote a panel
+  from `plesty.widgets` primitives — six of them, each restating what its
+  device already declares. The duplicate drifts, and it drifts in the direction
+  that matters: the hand-written panel for one spectrometer bounded its
+  exposure at 0.01–60000 ms and its hardware averaging at 1–1000, where the
+  device declares no upper bound on either. Limits that exist nowhere in the
+  instrument, invented by whoever wrote the GUI, silently constraining an
+  operator.
+
+  `controls_for()` derives them instead — `dtype` picks the control,
+  `min_value`/`max_value` bound it, `options` makes it a drop-down, `read_only`
+  a readout, `unit` the suffix, `description` the tooltip — and where the
+  device declares no bound the control has none. It is free of any toolkit and
+  tested without a display, so a drift fails a test rather than reaching a
+  bench. Operations become buttons, with a small form when they take arguments;
+  `lifecycle` ones (homing, reset, a dark reference) are left off unless named,
+  because they should not be one click away.
+
+  A panel starts **read-only**. A monitor holds no device and this one
+  necessarily does, so it may be pointed at an instrument an experiment is
+  driving; writes and operations wait behind *Take control*.
+
+- **`describe` carries the device's documentation, not just its method names.**
+  It answered `{"methods": [...]}`, and `doc_model()` — though exposed to the
+  API — returns a dataclass holding `dtype: type` and a `ResponseParser`, which
+  the server refuses to serialize. So nothing across the wire could learn a
+  parameter's unit, bounds, options or description: a console or a remote
+  client had no way to render a control except by inventing its constraints,
+  which is the bug above with a network in the middle. `DeviceDocModel`,
+  `ConfigParameter`, `FuncDoc`, `FuncParam` and `FuncOutput` now have
+  `to_dict()`, and the reply carries it as `doc` **beside** the existing
+  `methods` key — a client written against the old shape is unaffected, and a
+  server older than the payload degrades to method names, which `DevicePanel`
+  reports rather than showing an empty form. A device whose model fails to
+  build still answers `describe`; it is what a supervisor uses to decide the
+  device is alive.
+
+- **A panel can state its own tone.** `Panel.tone()` returns the state colour of
+  the title-bar dot; the shell hardcoded `running` whenever a panel had a status
+  line at all, so a panel that fails *while still running* — a device that
+  stopped answering — had no way to go red.
+
+- **A long call holds the instrument, not the socket (#48).** The accept loop
+  answered one request at a time, inline, so nothing was even read while a
+  device call ran. A supervisor asks `describe` on a timer — the console does
+  it per device every three seconds — and every probe arriving during a 20 s
+  exposure queued behind it, timed out, reconnected, and had its reply dropped
+  as unroutable: a bench read as unreachable for the whole of every long
+  acquisition, and the log carried a warning per probe for work nobody had
+  lost. Requests are now answered in their own task, so the loop keeps
+  receiving. Device work stays serialized exactly as before — the async
+  wrapper holds a per-device lock around every call — and what overtakes it is
+  `describe`, which is introspection over the class and the op registry and
+  touches no hardware. Replies are serialized on their own lock, because a
+  reply is two frames on one shared ROUTER socket.
+
+  A lost *liveness* reply is also no longer a warning. `disconnect` was
+  already excluded for the same reason: the warning exists for an answer to
+  work the device has already done, and a probe that reads state and moves no
+  instrument loses nothing when its reply misses.
+
+- **`CompositeDevice.identity()` runs on the sub-device's own thread (#49).**
+  It was the one path that called the client directly instead of going through
+  `submit` — `call`, `preflight`, `set_data_path` and `disconnect_all` all hop
+  first. Under `thread_affinity` a sub-device's ZMQ socket belongs to its
+  thread, so asking for the identity from another one put two threads on that
+  socket and the replies crossed: a magneto-PL rig whose power regulator was
+  sampling the meter got the identity string back from `measure_power`. The
+  crash was the lucky case — two calls of the same shape crossing swap two
+  readings and say nothing.
+
+- **The functions gate now issues a control operation into a move, instead of
+  at a stage already parked.** #39 built the `control` kind but drove it at
+  rest: controls were sorted last and the stop went out after every motion had
+  finished and been put back. That proves the command is accepted, not that it
+  stops anything — and the case that matters is the other one. k10cr1#2, motion
+  methods returning before the stage had settled, was found by hand at a bench;
+  an in-flight stop is the gate that would have caught it.
+
+  The move is put in flight the way the drain gate provokes a slow call: the
+  transport's timeout is shortened so the command goes out and its answer is
+  abandoned, leaving the part travelling. Then the control is issued, and three
+  things have to hold — it returns while the move is still in flight,
+  `position_key` reads short of where the move was heading, and the part answers
+  afterwards and goes back where it started. The same shape runs over the wire
+  in `ClientFieldTest`, which takes `slow_op_payload` for it.
+
+  No worker thread drives the device. Servers pin device calls to one thread
+  because drivers turned out to have thread *affinity*, and a gate that called
+  one device from two threads would risk failing for a reason that has nothing
+  to do with the control operation.
+
+  **A module that stops late now fails a gate that passed before**, and a
+  control is skipped — with the reason — where no motion is allow-listed or no
+  `slow_op_payload` delta is declared, rather than being called at rest to no
+  purpose. The gate's evidence is also bound before the loop rather than after
+  it, so a run that fails still reports what it did, including whether a
+  stopped part was put back.
+
 - **A client holding an allocation can call schema operations again, and those
   calls are now access-controlled.** The server appends the caller's allocation
   to every `call` it forwards, but a schema operation is a generated closure

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: plesty-lib
-Version: 0.3.6
+Version: 0.4.0.dev1
 Summary: A library for the Plesty ecosystem.
 Author: Plesty Development Team
 Author-email: Yunshuang Yuan <yunshuang.yuan@fkp.uni-hannover.de>

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/docs/ui.md

@@ -2,8 +2,8 @@
 
 `plesty.lib.ui` is the GUI framework of the platform: **one home window that
 hosts any number of rearrangeable sub-windows**. It is deliberately generic —
-a live view is only the first kind of panel; a device control form or a run
-status board is the same contract in the same window.
+a live view (`MonitorPanel`) and a device's controls (`DevicePanel`) are the
+same contract in the same window, and a run status board would be a third.
 
 The framework is toolkit-free down to the last layer. `Panel`, `Theme`, and
 `MonitorPanel` import without a GUI installed; only the Qt backend needs the
@@ -72,11 +72,98 @@ Placement (`area`, `floating`, `closable`, `min_width`, `min_height`) is
 declared at construction and is only the *initial* arrangement — the operator
 owns it from there.
 
+`status()` is the line in the title bar and `tone()` the colour of its dot —
+`running`, `idle`, `warning`, `error`. The default reads a panel as working
+whenever it has something to say, which is right for a view that only draws; a
+panel that can fail *while still running* overrides it, so a device that
+stopped answering is red from across the room.
+
 `MonitorPanel` is the bridge to the [monitor framework](monitor.md): it
 attaches a `PlotRenderer` to a monitor when it is built, and ticks it
 afterwards. Constructing and describing it needs no GUI, so a shell's
 composition is testable headless.
 
+## Device panels
+
+A device already describes its controls completely, so `DevicePanel` builds
+them instead of asking anyone to write them again:
+
+```python
+from plesty.lib.ui import DevicePanel, LocalHandle, run
+
+run([DevicePanel(LocalHandle(spectrometer), name="cct",
+                 include=("exposure", "hardware_average"))])
+```
+
+Every widget comes from the schema. Nothing is hand-placed, and nothing is
+hand-bounded:
+
+| The device declares | The panel renders |
+|---|---|
+| `dtype` `float`/`int` | spin box, adaptive step |
+| `min_value` / `max_value` | its range — and where the device declares none, none |
+| `options` | drop-down |
+| `dtype` `bool` | check box |
+| `read_only` | a readout, not an editor |
+| `write_only` | an editor that is never polled |
+| `unit` | the editor's suffix |
+| `description` | the tooltip |
+| an operation | a button, with a small form when it takes arguments |
+
+That table is the whole point. A panel written by hand for one spectrometer
+bounded its exposure at `0.01–60000 ms` where the device declares no upper
+bound at all, and its hardware averaging at `1–1000` where the device declares
+no limit — numbers that exist nowhere in the instrument, constraining whoever
+used it. `controls_for()` derives them, is free of any toolkit, and is tested
+without a display, so a drift like that fails a test instead of reaching a
+bench.
+
+**What stays out.** A spectrum, a beam image, a stage map are not derivable
+from a schema — those are monitors, and a `MonitorPanel` docks beside this one.
+Operations declared `lifecycle` (homing, reset, replacing a dark reference) are
+left off unless named in `operations=`: they should not be one click away.
+
+**A panel starts read-only.** A monitor holds no device; this one necessarily
+does, and the instrument may be driven by a running experiment. Editors and
+buttons stay disabled behind the panel's *Take control* toggle
+(`set_writable()`).
+
+### Where the device is
+
+`DevicePanel` never talks to a device directly — it drives a `DeviceHandle`,
+which is four calls: `describe()`, `query(key)`, `write(key, value)` and
+`call(op, **kwargs)`. That is what lets one panel serve three places:
+
+| | The handle |
+|---|---|
+| A module's own GUI | `LocalHandle(device)` |
+| A console beside the device server | `DeviceTCPIPClient` — it satisfies the protocol as written |
+| Another machine | the same four calls, relayed by the bench agent |
+
+`describe` carries the device's serialized `DeviceDocModel` — units, bounds,
+options, descriptions — beside the method list it always returned, so a caller
+that never imports plesty-lib reads what the device declared instead of
+inventing it. A server older than that payload answers with method names only,
+and the panel says so rather than showing an empty form.
+
+`LocalHandle` goes through the same serialized payload the wire uses. That is
+deliberate: a module's own GUI then exercises exactly what the console and a
+remote client will see, so a hole in the payload shows up at the bench and not
+only across a network.
+
+### Why it never reads on the GUI thread
+
+Panels are ticked on the shell's clock, and a device read is a round trip to
+hardware. Every call therefore goes to a `DeviceWorker` — one thread per panel
+— and every result comes back through a queue the panel drains on its next
+tick, so the shell's own clock does the hand-over and no Qt signal is involved.
+
+One worker *per panel*, not per shell: a device server that has died answers
+nothing until its client times out, and a shared worker would let that one
+device stall every panel in the window. Requests are coalesced by tag, so a
+device slower than the refresh interval drops the polls it cannot keep up with
+instead of accumulating a backlog and showing readings from minutes ago.
+
 ## Theme
 
 Colours are data, not code. `Theme` reads `assets/palette.json` and renders
@@ -135,9 +222,14 @@ skipped where the `gui` extra is not installed, which is why CI (and any
 headless lab host) still runs the full suite:
 
 ```bash
-QT_QPA_PLATFORM=offscreen uv run pytest tests/test_ui.py
+QT_QPA_PLATFORM=offscreen uv run pytest tests/test_ui.py tests/test_device_panel.py
 ```
 
+A device panel is tested the same way, and further: `open()` starts its worker
+without building a widget, so the whole device-facing half — the schema
+arriving, polling, a refused write, a device that stops answering — is driven
+with no display and no toolkit at all.
+
 ## See also
 
 - [Monitor framework](monitor.md) — what `MonitorPanel` shows.

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/gl-secret-detection-report.json

@@ -19,19 +19,19 @@
       "version": "8.30.1"
     },
     "type": "secret_detection",
-    "start_time": "2026-08-31T15:10:17",
-    "end_time": "2026-08-31T15:10:17",
+    "start_time": "2026-09-03T11:41:33",
+    "end_time": "2026-09-03T11:41:34",
     "status": "success",
     "observability": {
       "events": [
         {
           "event": "collect_secrets_analyzer_scan_metrics_from_pipeline",
-          "time_s": 0.468042116,
+          "time_s": 0.518565688,
           "exit_code": 0,
           "git_strategy": "FetchShallow",
-          "repo_size_kb": 732,
+          "repo_size_kb": 767,
           "commit_count": 1,
-          "bytes_scanned": 7570,
+          "bytes_scanned": 76550,
           "pipeline_type": "Tag"
         },
         {

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/plesty/lib/device/composite_device.py

@@ -719,12 +719,20 @@ class CompositeDevice:
         self._set_state(dev, ConnectionState.CONNECTED, "reconnected")
 
     def identity(self) -> dict[str, Any]:
-        """Query the identity of each sub-device and return a mapping of results."""
+        """Query the identity of each sub-device and return a mapping of results.
+
+        Routed through :meth:`submit` like every other sub-device call, not
+        called on the client directly. Under ``thread_affinity`` a sub-device's
+        socket belongs to its own thread, and asking here from a foreign one
+        put a second thread on it: a magneto-PL rig whose power regulator was
+        sampling the meter got the identity string back from ``measure_power``
+        (2026-09-01). The crash was the lucky case — two calls of the same
+        shape crossing swap two readings and say nothing.
+        """
         identities = {}
         for device_name in self.devices:
-            device = getattr(self, device_name)
             try:
-                identities[device_name] = device.identity()
+                identities[device_name] = self.submit(device_name, "identity").result()
             except Exception as e:
                 identities[device_name] = f"Error: {e}"
         return identities

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/plesty/lib/device/device_utils.py

@@ -31,6 +31,58 @@ class ResponseParser:
         )
 
 
+def dtype_name(dtype: Any) -> str:
+    """Return the wire name of a declared data type.
+
+    A schema declares a type as a Python type, a typing construct, or already
+    as a string. A caller on the far side of the device protocol has to switch
+    on it, so it arrives as a name and never as a ``<class 'float'>`` repr.
+
+    Args:
+        dtype: The declared type, or ``None``.
+
+    Returns:
+        The type's name; ``"Any"`` when nothing was declared.
+    """
+    if dtype is None:
+        return "Any"
+    if isinstance(dtype, str):
+        return dtype
+    name = getattr(dtype, "__name__", None)
+    return name if name else str(dtype).replace("typing.", "")
+
+
+def jsonable(value: Any) -> Any:
+    """Return *value* as something :func:`json.dumps` accepts.
+
+    Schema defaults, options and bounds are whatever the module declared — a
+    numpy scalar, an enum member, a tuple. They cross the wire as
+    documentation, so a value JSON has no form for is described rather than
+    dropped: a reader learns what the default is even when it cannot be one.
+
+    Args:
+        value: The value to convert.
+
+    Returns:
+        The value unchanged when JSON already holds it, a list or dict when it
+        is a container, else its ``str()``.
+    """
+    if value is None or isinstance(value, (bool, int, float, str)):
+        return value
+    if isinstance(value, (list, tuple, set, frozenset)):
+        return [jsonable(item) for item in value]
+    if isinstance(value, dict):
+        return {str(key): jsonable(item) for key, item in value.items()}
+    # A numpy scalar is not an int or a str but holds one.
+    item = getattr(value, "item", None)
+    if callable(item):
+        try:
+            return jsonable(item())
+        except Exception:
+            pass
+    return str(value)
+
+
 # define decoretor for device methods that require operatability check
 def operatable(func: Callable[..., Any]) -> Callable[..., Any]:
     """Decorator that checks device operatability before executing the wrapped function."""

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/plesty/lib/device/doc.py

@@ -14,6 +14,7 @@ it instead of re-deriving the metadata.
 """
 
 from dataclasses import dataclass
+from typing import Any
 
 from plesty.lib.device.funcs import FuncDoc
 from plesty.lib.device.params import ConfigParameter
@@ -35,3 +36,24 @@ class DeviceDocModel:
     parameters: dict[str, list[ConfigParameter]]
     functions: list[FuncDoc]
     standard_methods: list[FuncDoc]
+
+    def to_dict(self) -> dict[str, Any]:
+        """Return the whole model as JSON primitives.
+
+        This is what leaves the device: the ``describe`` reply carries it, so a
+        caller that never imports plesty-lib — a console on another process, a
+        client on another machine — reads the same units, bounds, options and
+        descriptions the device declared, instead of inventing them.
+
+        Returns:
+            A mapping with ``parameters`` (grouped, as the model holds them),
+            ``functions`` and ``standard_methods``, all JSON primitives.
+        """
+        return {
+            "parameters": {
+                group: [param.to_dict() for param in params]
+                for group, params in self.parameters.items()
+            },
+            "functions": [func.to_dict() for func in self.functions],
+            "standard_methods": [func.to_dict() for func in self.standard_methods],
+        }

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/plesty/lib/device/funcs.py

@@ -14,7 +14,13 @@ import re
 import reprlib
 import numpy as np
 
-from plesty.lib.device.device_utils import OpKind, ResponseParser, op_kind_of
+from plesty.lib.device.device_utils import (
+    OpKind,
+    ResponseParser,
+    dtype_name,
+    jsonable,
+    op_kind_of,
+)
 from plesty.lib.data import (
     PlestyArray,
     PlestyTable2D,
@@ -161,6 +167,26 @@ class FuncParam:
     item_dtype: Any = field(default=None)
     description: str = field(default="")
 
+    def to_dict(self) -> dict[str, Any]:
+        """Return this argument as JSON primitives.
+
+        Returns:
+            Name, type, unit, default and constraints — what a caller needs to
+            offer the argument and check it before sending.
+        """
+        return {
+            "name": self.name,
+            "dtype": dtype_name(self.dtype),
+            "unit": self.unit,
+            "default": jsonable(self.default),
+            "required": self.required,
+            "options": jsonable(self.options),
+            "range": jsonable(self.range),
+            "shape": jsonable(self.shape),
+            "item_dtype": dtype_name(self.item_dtype) if self.item_dtype is not None else None,
+            "description": self.description,
+        }
+
 
 @dataclass
 class FuncOutput:
@@ -177,6 +203,35 @@ class FuncOutput:
     headers: list[TableHeader] | None = field(default=None)
     description: str = field(default="")
 
+    def to_dict(self) -> dict[str, Any]:
+        """Return this output as JSON primitives.
+
+        Returns:
+            Name, type, unit and shape — what a caller needs to read the
+            result, including a table's column headers.
+        """
+        return {
+            "name": self.name,
+            "dtype": dtype_name(self.dtype),
+            "unit": self.unit,
+            "required": self.required,
+            "code_mapping": jsonable(self.code_mapping),
+            "range": jsonable(self.range),
+            "shape": jsonable(self.shape),
+            "item_dtype": dtype_name(self.item_dtype) if self.item_dtype is not None else None,
+            "headers": [
+                {
+                    "name": header.name,
+                    "dtype": dtype_name(header.dtype),
+                    "unit": header.unit,
+                    "description": header.description,
+                }
+                for header in self.headers or []
+            ]
+            or None,
+            "description": self.description,
+        }
+
 
 @dataclass
 class FuncDoc:
@@ -204,6 +259,24 @@ class FuncDoc:
     oparams: list["FuncOutput"]
     kind: OpKind = field(default_factory=OpKind)
 
+    def to_dict(self) -> dict[str, Any]:
+        """Return this operation as JSON primitives.
+
+        Returns:
+            The operation with its arguments, its outputs and its
+            :class:`~plesty.lib.device.device_utils.OpKind` — enough for a
+            caller to offer it, and to decide whether it should be one click
+            away.
+        """
+        return {
+            "name": self.name,
+            "source": self.source,
+            "description": self.description,
+            "iparams": [param.to_dict() for param in self.iparams],
+            "oparams": [output.to_dict() for output in self.oparams],
+            "kind": self.kind.to_dict(),
+        }
+
 
 #: Size-bounded repr for operation log lines, so array-valued parameters do
 #: not flood the log.

{plesty_lib-0.3.6 → plesty_lib-0.4.0.dev1}/plesty/lib/device/params.py

@@ -12,7 +12,7 @@ import logging
 import re
 
 from plesty.lib.utils.error_utils import handle_error
-from plesty.lib.device.device_utils import ResponseParser
+from plesty.lib.device.device_utils import ResponseParser, dtype_name, jsonable
 from plesty.lib.data import resolve_dtype, istype, cast_basic_type
 
 
@@ -271,6 +271,30 @@ class ConfigParameter:
     parser: ResponseParser | None = None
     description: str | None = None
 
+    def to_dict(self) -> dict[str, Any]:
+        """Return the caller-facing description of this parameter.
+
+        Everything needed to render or bound the parameter, and nothing about
+        how it reaches the instrument: ``command`` and ``parser`` are this
+        device's own business, and neither has a JSON form.
+
+        Returns:
+            The parameter as JSON primitives.
+        """
+        return {
+            "name": self.name,
+            "dtype": dtype_name(self.dtype),
+            "unit": self.unit,
+            "default": jsonable(self.default),
+            "value": jsonable(self.value),
+            "read_only": self.read_only,
+            "write_only": self.write_only,
+            "min_value": jsonable(self.min_value),
+            "max_value": jsonable(self.max_value),
+            "options": jsonable(self.options),
+            "description": self.description,
+        }
+
 
 @dataclass
 class ConfigGroup: