dsh-labscript 0.3.1 → 0.3.3

package/CHANGELOG.md

@@ -1,24 +1,46 @@
 # Changelog
 
-## 0.3.1
-
-- The DSH-side bridge budget (`requestTimeoutMs`) now defaults to 180 s instead of 60 s, and the
-  `labs_doctor` probe uses it too. It has to stay above runmanager's own client timeout, which
-  now comes from labconfig (`[timeouts] communication_timeout`): at 60 s both budgets were equal,
-  so a slow RPC would be killed by the host before runmanager could report the real error.
-
-## 0.3.0
+## 0.3.3
+
+Makes the tools report what actually happened instead of hiding it, and removes a backend
+nobody could reach. Net change is negative: about 40 lines of reporting added, a dead
+backend removed.
+
+- `labs_doctor` no longer imports labscript to check the environment. That import has side
+  effects (labscript's double-import denier walks the stack and leaves half-initialised
+  modules behind) and it broke the runmanager probe that runs right after it in the same
+  process, so a working environment was reported as two `[fail]` lines pointing at an h5py
+  path. The check is now `importlib.util.find_spec` (presence only: nothing in this plugin
+  imports labscript), error messages keep the exception type instead of only its text, and
+  doctor prints `python-executable` so "wrong interpreter" is visible at a glance.
+- `labs_status` prints a panel field passed as if it were a global (`keys: ["n_shots"]`) as
+  `<value> (panel field, not a global)` instead of `<missing>`.
+- `finish.panel` and `finish.history` report values they cannot write as `finish.skipped`,
+  with the dotted path and the reason (`missing` / `not a number (str)`); `labs_wait` renders
+  one `finish skipped:` line. These used to be dropped without a word, which reads exactly
+  like "the calibration number did not land".
+- Postprocess receives `scan_globals` (every axis, in plan order) when its signature accepts
+  it; `scan_global` is unchanged. Dispatch uses `inspect.signature` instead of retrying on
+  `TypeError`, which also stops a `TypeError` raised inside the analysis function from running
+  it a second time.
+- Removed the `http-sim` backend: an HTTP client for a local simulator service, added for a
+  one-off benchmark in v0.1 and later renamed to hide the internal service name. It had no
+  tests, no documentation and no entry on the Plugins page. `backend` is `labscript` or
+  `mock-labscript` again.
+- Tests: 38 python + 49 node.
+
+## 0.3.2
 
 Implements the upstream request written from the field (labscript-suite workflow terms and
 `runmanager.remote` limits). No output field is renamed or removed.
 
-- `n_shots` is now documented and rendered as the authoritative shot count. The scan-axes line is
+- `n_shots` is documented and rendered as the authoritative shot count. The scan-axes line is
   labelled a heuristic: runmanager expands list/array globals but passes tuples through, and an
   expansion setting of `none` turns an array into a non-axis; neither is visible through
-  `runmanager.remote`. A mismatch now reads "n_shots is authoritative (a tuple global or an
-  expansion setting of none is not an axis)" instead of printing two numbers with no verdict. The
-  bridge's axis guess ignores tuples and dicts, and the mock backend follows the same rule (it used
-  to multiply tuples in, pinning the wrong semantics into the tests).
+  `runmanager.remote`. A mismatch reads "n_shots is authoritative" instead of printing two numbers
+  with no verdict. The bridge's axis guess ignores tuples and dicts, and the mock backend follows the
+  same rule (it used to multiply tuples in, pinning the wrong semantics into the tests); the mock
+  also keeps Python types, without which the tuple/list distinction was untestable.
 - Skill and tool descriptions no longer tell the agent to "read the runmanager output": compilation
   errors appear only in runmanager's output tab (the compiler subprocess writes its traceback to
   stderr and runmanager redirects it to an in-memory Qt widget), so the instruction is now to say
@@ -28,16 +50,17 @@ Implements the upstream request written from the field (labscript-suite workflow
 - `compiled_not_executed` explains the other official causes: BLACS may keep a shot out of its
   queue (the shot's connection table must be a subset of the lab connection table) or put it back
   at the head after a device programming timeout; all of that is visible only in the BLACS GUI.
-  More shot files than expected now points at a changed panel or BLACS Repeat all / Repeat last,
-  which re-runs completed shots without creating new files.
+  More shot files than expected points at a changed panel or BLACS Repeat all / Repeat last, which
+  re-runs completed shots without creating new files.
 - `labs_status` reports `shuffle` (from `get_shuffle`; "unknown" when that runmanager version has
   no such method). With shuffle on, shot order is not axis order: read each shot's parameters from
   its own h5 globals.
 - runmanager host/port/timeout default to labscript's labconfig when unset, resolved by
   `runmanager.remote.Client` itself (`[servers]/[ports]/[timeouts]`, port 42523, timeout 60 s);
-  configured values still win. The Plugins page now leaves host/port blank for that reason, and
-  clearing them hands the endpoint back to labconfig. `labs_doctor` prints the effective endpoint
-  and whether it came from config or labconfig.
+  configured values still win. The Plugins page leaves host/port blank for that reason, and clearing
+  them hands the endpoint back to labconfig. `labs_doctor` prints the effective endpoint and its
+  source. The DSH-side bridge budget (`requestTimeoutMs`) defaults to 180 s so it stays above
+  runmanager's client timeout instead of racing it.
 - mock backend: `mock.errorInGlobals` makes `error_in_globals` true so the engagement refusal can
   be tested, and `mock.shuffle` sets the reported shuffle flag.
 - Docs: a terminology table maps the plugin's short names to the suite's (shot output folder,
@@ -45,8 +68,10 @@ Implements the upstream request written from the field (labscript-suite workflow
 - Compilation logs stay out of scope: nothing about a compilation failure is written anywhere
   readable (see README), so no `labs_compile_log` was added; a guessed diagnosis would be worse
   than none.
-- Tests: 33 python + 47 node (axis heuristic with tuples/dicts, mock error/shuffle switches,
-  labconfig client defaults, extra shot files, scan/shuffle rendering, endpoint clearing).
+- Tool descriptions stay close to their 0.2.8 size (+147 characters): the explanations live in the
+  runtime skill (loaded on demand) and in the wait/engage diagnostics (shown only when they matter),
+  so they do not sit in context on every turn.
+- Tests: 33 python + 47 node.
 
 ## 0.2.8
 

package/README.md

@@ -9,7 +9,7 @@ files. It does not require any custom BLACS/lyse patch. Everything lab-specific
 
 ## Status
 
-v0.2.0. The mock-labscript backend, the protocol/postprocess flow and the
+v0.3.3. The mock-labscript backend, the protocol/postprocess flow and the
 snapshot/diagnosis behaviour are covered by node and python tests. A client
 configuration page (Plugins page) and a read-only `labs_doctor` are included. The real
 labscript backend is implemented against the stock runmanager remote API but has
@@ -75,6 +75,11 @@ guess panel state).
     labs_protocol run demo_scan      # set + scan + reps + engage, returns immediately
     labs_wait                        # waits, then postprocess -> finish -> restore
 
+`postprocess.entry` is called as `analyze(run_folder, scan_global, reps, name)`; when the
+function accepts it, `scan_globals=[...]` (every axis, in order) is passed as well, so a 2D
+scan is not limited to its first axis. A `finish` value that is missing or not a number is
+reported as `skipped` and printed by `labs_wait` instead of being dropped silently.
+
 ## Configuration
 
 Add one row to your DSH profile patch (see examples/config.example.yml):
@@ -140,8 +145,8 @@ is not a physics simulation.
 ## Development
 
     npm install              # only runtime dependency: yaml
-    npm test                 # node --test, 47 tests (fake bridge, no python needed)
-    python python/tests/run_tests.py   # 33 tests (needs numpy + h5py)
+    npm test                 # node --test, 49 tests (fake bridge, no python needed)
+    python python/tests/run_tests.py   # 38 tests (needs numpy + h5py)
     LABSCRIPT_PYTHON=/path/to/python3 node scripts/demo.mjs   # end-to-end mock demo
 
 A fresh clone must run `npm install` before `npm test`; the CI workflow does this.

package/docs/README.zh.md

@@ -49,6 +49,9 @@ hard limits 全部通过配置接入。
 
 postprocess 是后处理：跑完后调用配置的 Python 函数（entry + pythonPaths），
 返回 dict 供 finish 写回。旧协议里的 analysis: 字段当作 postprocess 别名。
+函数按 `analyze(run_folder, scan_global, reps, name)` 调用；签名接受时还会传
+`scan_globals=[...]`（全部扫描轴、按顺序），二维扫描因此能拿到第二轴。finish 解析到
+缺失或非数字的值不再静默丢弃，会在 labs_wait 输出里以 `finish skipped` 列出。
 
 ## 边界
 

package/examples/postprocess_demo.py

@@ -1,12 +1,13 @@
 """Example postprocess entry: after a run, return the numbers that finish.panel
 and finish.history write back. Replace the fake model with your lab analysis.
 
-Signature: analyze(run_folder=..., scan_global=..., reps=..., name=...)
-Returns a JSON-serializable dict.
+Signature: analyze(run_folder=..., scan_global=..., reps=..., name=..., scan_globals=...)
+scan_global is the first axis; scan_globals is every axis in order (a 2D scan needs both).
+Returns a JSON-serializable dict; finish.panel reads the numbers back out of it.
 """
 
 
-def analyze(run_folder, scan_global=None, reps=None, name=None):
+def analyze(run_folder, scan_global=None, reps=None, name=None, scan_globals=None):
     # Placeholder numbers so the write-back path is demonstrable end to end.
     return {
         "fit": {"center": 2.5, "pi_duration_us": 1.25, "r2": 0.99},

package/lib/runtime.js

@@ -29,7 +29,7 @@ const VERSION = PACKAGE.version
 const DEFAULT_ENGAGE_TIMEOUT_S = 900
 const DEFAULT_APPROVAL = { engage: true, protocolRun: true, protocolNames: true }
 const DEFAULT_BACKEND = 'mock-labscript'
-const BACKENDS = ['labscript', 'mock-labscript', 'http-sim']
+const BACKENDS = ['labscript', 'mock-labscript']
 
 export class LabError extends Error {
   constructor(code, message, detail) {
@@ -44,7 +44,7 @@ function normalizeConfig(raw) {
   const input = raw && typeof raw === 'object' ? raw : {}
   const backend = String(input.backend || DEFAULT_BACKEND).toLowerCase()
   if (!BACKENDS.includes(backend)) {
-    throw new LabError('BAD_CONFIG', 'backend must be "labscript", "mock-labscript" or "http-sim", got: ' + backend)
+    throw new LabError('BAD_CONFIG', 'backend must be "labscript" or "mock-labscript", got: ' + backend)
   }
   const approval = Object.assign({}, DEFAULT_APPROVAL, input.approval || {})
   return {
@@ -67,7 +67,6 @@ function normalizeConfig(raw) {
       protocolNames: approval.protocolNames !== false,
     },
     mock: input.mock || {},
-    simulator: input.simulator || {},
     dshHome: input.dshHome || '',
     // Must stay above runmanager's own client timeout (labconfig [timeouts]
     // communication_timeout, default 60 s): the bridge is killed when this expires, which
@@ -89,7 +88,6 @@ function bridgePayload(config) {
     engage: Object.assign({ timeoutS: DEFAULT_ENGAGE_TIMEOUT_S }, config.engage || {}),
     panelHistoryDir: config.panelHistoryDir,
     mock: config.mock,
-    simulator: config.simulator,
   }
 }
 

package/lib/skill.js

@@ -35,8 +35,6 @@ export const SKILL_CONTENT = [
   '  n_executed (or the diagnosis) says whether shots actually acquired data.',
   '- shuffle true means shot order is not axis order: read each shot parameters from its',
   '  own h5 globals, never from the shot index.',
-  '- Never name a global like a Python builtin or a name from "from labscript import *":',
-  '  runmanager accepts it, labscript fails at compile time, and only the output tab shows it.',
   '- labs_protocol run submits a fixed protocol; labs_wait finalizes it (postprocess,',
   '  finish write-back, panel restore).',
   '- backend defaults to mock-labscript; real hardware is only used when the config says',

package/lib/store.mjs

@@ -25,7 +25,7 @@ export function statePath(home) {
 }
 
 const UI_KEYS = ['backend', 'python', 'runmanager', 'protocolsDir']
-const BACKENDS = ['labscript', 'mock-labscript', 'http-sim']
+const BACKENDS = ['labscript', 'mock-labscript']
 
 export function normalizeUiConfig(input) {
   const source = input && typeof input === 'object' ? input : {}

package/lib/tools.js

@@ -62,7 +62,12 @@ function globalLines(args, value) {
   const out = ['globals:']
   for (const key of keys) {
     const shown = raw ? raw[key] : globals[key]
-    out.push('  ' + key + ' = ' + (shown === undefined ? '<missing>' : globalText(shown)))
+    // A panel field requested as if it were a global (n_shots is the usual one) should
+    // say what it is instead of looking like a global that vanished.
+    let text = '<missing>'
+    if (shown !== undefined) text = globalText(shown)
+    else if (Object.prototype.hasOwnProperty.call(value, key)) text = globalText(value[key]) + '  (panel field, not a global)'
+    out.push('  ' + key + ' = ' + text)
   }
   return out
 }
@@ -255,7 +260,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_status',
-    description: 'Read the labscript runmanager panel: loaded labscript, globals (evaluated; raw expressions with raw=true), n_shots, error_in_globals, run_shots, shuffle, shot output folder (called run folder in this tool: where the NEXT engage writes its shot files; it does not exist until engage), pending run progress, and the panel snapshot directory. Read-only; call this before any write. Pass keys=[...] (leaf global names; UI group names are not globals) to print the selected panel values (240 chars each; raw=true prints the raw expression), missing keys are shown as <missing> (the global is in an inactive group or misspelled); keys=["*"] prints every global. scan axes is a heuristic: it counts array-valued globals and their lengths multiply to n_shots_from_arrays, but runmanager does not expand tuples and an expansion setting of none turns an array into a non-axis, neither of which is visible from the remote API. n_shots is authoritative: when the two differ, report n_shots and say that at least one array-valued global is not an axis. shuffle true means shot order does not follow the axis order. Paths in the output are already mapped for this machine through pathMap; do not map them again.',
+    description: 'Read the labscript runmanager panel: loaded labscript, globals (evaluated; raw with raw=true), n_shots, error_in_globals, run_shots, shuffle, shot output folder (run folder in this tool; created by engage), pending run progress, snapshot directory. Read-only; call this before any write. Pass keys=[...] (leaf global names; UI group names are not globals) to print panel values (240 chars each), missing keys show as <missing>; keys=["*"] prints every global. scan axes is a heuristic (runmanager does not expand tuples and expansion=none is invisible), and n_shots is authoritative: when they differ, report n_shots. shuffle true means shot order is not axis order. Paths are already mapped through pathMap; do not map them again.',
     parameters: {
       type: 'object',
       properties: {
@@ -289,7 +294,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_set',
-    description: 'Write runmanager panel globals. One call can do the whole step: values are JSON scalars (bool/number/string); with raw=true a string value is written as an expression (e.g. np.arange(0.5, 6.7, 0.2)) while bool/number values are still converted normally (true -> True, 2.72 -> 2.72), so scalars, an expression scan axis and reps=N can be combined. reps=N writes the configured repetition global as np.arange(N). A list global becomes a scan axis only if runmanager actually expands it (a tuple is passed through whole, and an expansion setting of none drops the axis). A global name must not shadow a Python builtin or a name from "from labscript import *": runmanager accepts the name, labscript fails at compile time, and that failure is visible only in runmanager output tab. A full panel snapshot is taken before the write. Does not run anything.',
+    description: 'Write runmanager panel globals. One call can do the whole step: values are JSON scalars (bool/number/string); with raw=true a string value is written as an expression (e.g. np.arange(0.5, 6.7, 0.2)) while bool/number values are still converted (true -> True, 2.72 -> 2.72), so scalars, an expression scan axis and reps=N can be combined. reps=N writes the configured repetition global as np.arange(N). A global name must not shadow a Python builtin or a "from labscript import *" name: runmanager accepts it, labscript fails at compile time (visible only in runmanager output tab). A panel snapshot is taken before the write. Does not run anything.',
     parameters: {
       type: 'object',
       properties: {
@@ -319,7 +324,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_engage',
-    description: 'Submit the current panel as one sequence: runmanager writes one shot file per point and hands it to BLACS (this is the "shot output folder", called run_folder in this tool). Returns immediately with run_folder, n_shots_at_submit and expected_n_shots; use labs_wait later to wait for completion. It cannot see compilation errors: runmanager shows those only in its output tab. Refuses when error_in_globals is true or another run is still pending; a pending run that produced no shot at all for config.engage.staleS (default 180 s) is dropped and reported as dropped_stale_pending.',
+    description: 'Submit the current panel as one sequence: one shot file per parameter point, handed to BLACS. Returns immediately with run_folder (the shot output folder), n_shots_at_submit and expected_n_shots; use labs_wait later. It cannot see compilation errors: runmanager shows those only in its output tab. Refuses when error_in_globals is true or another run is pending; a pending run with no shot at all is dropped after config.engage.staleS (default 180 s) and reported as dropped_stale_pending.',
     parameters: {
       type: 'object',
       properties: {
@@ -363,7 +368,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_wait',
-    description: 'Wait for the pending run (sequence) to finish (blocking, timeout in seconds). If it came from labs_protocol run, this call also finalizes it: postprocess, finish write-back of panel and calibration history, and panel restore. Returns expected vs actual shot files and, for protocols, the postprocess/finish/restore results. Returns early with diagnosis no_shots or compile_stalled when no shot file appeared for config.engage.noShotS seconds (default 25): compilation errors are visible only in runmanager output tab, which this tool cannot read, so report exactly that and ask the operator to paste the tab; call labs_wait again only if you expect a slow compile. diagnosis compiled_not_executed means shot files exist but BLACS acquired no data: BLACS may be paused, may have refused the shot because its connection table is not a subset of the lab connection table, or a device group failed to program within 5 minutes (the queue then pauses and puts the shot back at its head) - all visible only in the BLACS GUI. More shot files than expected means the panel n_shots changed after submit, or BLACS Repeat all / Repeat last re-ran shots. A run folder that does not exist and is not the pending run of this session fails immediately instead of waiting.',
+    description: 'Wait for the pending run (sequence) to finish (blocking, timeout in seconds). If it came from labs_protocol run, this call also finalizes it: postprocess, finish write-back, panel restore. Returns expected vs actual shot files. A run folder that does not exist and is not the pending run fails immediately instead of waiting. Returns early with diagnosis no_shots or compile_stalled when no shot file appeared for config.engage.noShotS seconds (default 25): compilation errors are visible only in runmanager output tab, which this tool cannot read, so report that and ask the operator to paste the tab. diagnosis compiled_not_executed means shot files exist but BLACS acquired no data: check the BLACS GUI.',
     parameters: {
       type: 'object',
       properties: {
@@ -382,6 +387,9 @@ export function registerTools(ctx, runtime) {
       value.hint ? 'hint: ' + value.hint : '',
       value.finalize && value.finalize.postprocess ? 'postprocess: ' + JSON.stringify(value.finalize.postprocess).slice(0, 400) : '',
       value.finalize && value.finalize.finish ? 'finish: ' + JSON.stringify(value.finalize.finish).slice(0, 300) : '',
+      value.finalize && value.finalize.finish && Array.isArray(value.finalize.finish.skipped) && value.finalize.finish.skipped.length
+        ? 'finish skipped: ' + value.finalize.finish.skipped.map((item) => item.key + ' <- ' + item.dotted + ' (' + item.reason + ')').join('; ')
+        : '',
       value.finalize && value.finalize.restore ? 'restore: ' + (value.finalize.restore.restored ? 'yes' : 'no') : '',
     ]))),
     async execute(args, exec) {
@@ -490,7 +498,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_data',
-    description: 'Read a run folder (shot output folder) HDF5: first-shot globals (pass keys=[...] to print selected values, ["*"] for all), the results groups with results: true, and with scan: a per-shot axis summary (unique points, min/max, median step) plus the numeric result columns; pass limit to print up to 20 raw rows. Each shot file holds the evaluated globals of that shot, its hardware commands and, after execution, images/ and results/ - a shot with no images/results was not executed. With shuffle on, read each shot\'s parameters from its own file instead of assuming axis order. Read-only. run_folder defaults to the pending run, else the last run submitted through this plugin (persisted across restarts); for a historical run pass its folder explicitly. A missing folder is an error: the shot output folder is the NEXT sequence\'s folder and only exists after engage.',
+    description: 'Read a run folder (the shot output folder) HDF5: first-shot globals (keys=[...], ["*"] for all), results groups with results: true, and with scan: a per-shot axis summary (unique points, min/max, median step) plus numeric columns; limit prints up to 20 raw rows. Read-only. run_folder defaults to the pending run, else the last run submitted through this plugin (persisted across restarts); pass a historical folder explicitly. A missing folder is an error: the shot output folder is created by engage.',
     parameters: {
       type: 'object',
       properties: {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-labscript",
-  "version": "0.3.1",
+  "version": "0.3.3",
   "description": "Control labscript-suite experiments from DeepSeek Harness: runmanager globals and scans, submits and waits, YAML fixed-operation protocols with postprocess write-back, panel snapshots, a built-in mock backend and a read-only doctor.",
   "type": "module",
   "main": "lib/index.js",

package/python/bridge.py

@@ -15,6 +15,8 @@ import sys
 import time
 import datetime
 import importlib
+import importlib.util
+import inspect
 import traceback
 
 
@@ -318,7 +320,7 @@ class LabscriptBackend(object):
         try:
             from runmanager.remote import Client
         except Exception as exc:
-            raise BridgeError("BACKEND_UNAVAILABLE", "labscript python environment not importable: " + str(exc))
+            raise BridgeError("BACKEND_UNAVAILABLE", "labscript python environment not importable: " + type(exc).__name__ + ": " + str(exc))
         # Anything left unset is resolved by runmanager.remote.Client from labscript's
         # labconfig ([servers] runmanager, [ports] runmanager, [timeouts]
         # communication_timeout), the same way the runmanager GUI does it. Explicit
@@ -538,113 +540,10 @@ class MockBackend(object):
         return {"aborted": True, "mode": "mock-labscript"}
 
 
-class HttpSimBackend(object):
-    """HTTP client for an optional local simulator service (development backend).
-
-    The simulator must speak the documented JSON/HTTP contract on a loopback address.
-    It is a development tool, not a physics backend for publication.
-    """
-
-    def __init__(self, config):
-        simulator = config.get("simulator") or {}
-        url = str(simulator.get("url") or "http://127.0.0.1:18761").rstrip("/")
-        if not url.startswith("http://127.0.0.1:"):
-            raise BridgeError("BAD_CONFIG", "simulator.url must be http://127.0.0.1:<port>, got: " + url)
-        self.base = url
-        self.timeout = float(simulator.get("timeoutS") or 10)
-        self._opener = None
-
-    def opener(self):
-        import urllib.request
-        if self._opener is not None:
-            return self._opener
-
-        class NoRedirect(urllib.request.HTTPRedirectHandler):
-            def redirect_request(self, req, fp, code, msg, headers, newurl):
-                return None
-
-        self._opener = urllib.request.build_opener(NoRedirect, urllib.request.ProxyHandler({}))
-        return self._opener
-
-    def request(self, method, path, body=None):
-        import urllib.request
-        import urllib.error
-        data = None
-        headers = {}
-        if body is not None:
-            data = json.dumps(jsonable(body)).encode("utf-8")
-            headers["Content-Type"] = "application/json"
-        req = urllib.request.Request(self.base + path, data=data, headers=headers, method=method)
-        try:
-            with self.opener().open(req, timeout=self.timeout) as response:
-                raw = response.read().decode("utf-8")
-        except urllib.error.HTTPError as exc:
-            try:
-                payload = json.loads(exc.read().decode("utf-8"))
-            except Exception:
-                raise BridgeError("BACKEND_UNAVAILABLE", "http-sim HTTP " + str(exc.code) + " at " + path)
-            error = payload.get("error") or {}
-            raise BridgeError(str(error.get("code") or "BACKEND_UNAVAILABLE"), str(error.get("message") or "http-sim error"))
-        except Exception as exc:
-            raise BridgeError("BACKEND_UNAVAILABLE", "http-sim not reachable at " + self.base + ": " + str(exc))
-        try:
-            envelope = json.loads(raw)
-        except Exception:
-            raise BridgeError("BACKEND_UNAVAILABLE", "http-sim response is not JSON")
-        if envelope.get("ok") is not True:
-            error = envelope.get("error") or {}
-            raise BridgeError(str(error.get("code") or "BACKEND_UNAVAILABLE"), str(error.get("message") or "http-sim error"))
-        return envelope.get("data") or {}
-
-    def snapshot(self):
-        return self.request("GET", "/v1/rm/snapshot")
-
-    def get_globals(self, raw=False):
-        snap = self.snapshot()
-        key = "raw_globals" if raw else "globals"
-        return dict(snap.get(key) or {})
-
-    def set_globals(self, values, raw=False):
-        body = dict((k, v.tolist() if hasattr(v, "tolist") else v) for k, v in values.items())
-        data = self.request("POST", "/v1/rm/set_globals", {"values": body, "raw": bool(raw)})
-        return dict(data.get("written") or {})
-
-    def n_shots(self):
-        return int(self.snapshot().get("n_shots") or 0)
-
-    def error_in_globals(self):
-        return bool(self.snapshot().get("error_in_globals"))
-
-    def run_shots(self):
-        return True
-
-    def shuffle(self):
-        snap = self.snapshot()
-        return bool(snap.get("shuffle")) if "shuffle" in snap else None
-
-    def labscript_file(self):
-        return self.snapshot().get("labscript_file")
-
-    def shot_output_folder(self):
-        return self.snapshot().get("shot_output_folder")
-
-    def engage(self):
-        data = self.request("POST", "/v1/rm/engage", {})
-        if data.get("accepted") is not True:
-            raise BridgeError("ENGAGE_REJECTED", "http-sim did not accept engage: " + str(data.get("detail") or ""))
-        return {"accepted": True, "n_shots": data.get("n_shots"), "run_folder": data.get("run_folder"), "mode": "http-sim"}
-
-    def abort(self):
-        data = self.request("POST", "/v1/blacs/stop", {})
-        return {"aborted": True, "via": "http-sim/blacs/stop", "result": data}
-
-
 def make_backend(config):
     backend = (config.get("backend") or "mock-labscript").lower()
     if backend == "mock-labscript":
         return MockBackend(config)
-    if backend == "http-sim":
-        return HttpSimBackend(config)
     if backend == "labscript":
         return LabscriptBackend(config)
     raise BridgeError("BAD_CONFIG", "unknown backend: " + backend)
@@ -791,7 +690,7 @@ def check_limits(config, set_values, scan_values, expected_n_shots):
 # ---------------------------------------------------------------- postprocess
 
 
-def run_postprocess(config, plan, run_folder, scan_global, reps):
+def run_postprocess(config, plan, run_folder, scan_globals, reps):
     name = plan.get("postprocess")
     if not name:
         return None
@@ -812,38 +711,61 @@ def run_postprocess(config, plan, run_folder, scan_global, reps):
         func = getattr(module, func_name)
     except Exception as exc:
         raise BridgeError("POSTPROCESS_IMPORT_FAILED", "cannot import postprocess " + entry + ": " + str(exc))
+    axes = [str(key) for key in (scan_globals or [])]
+    scan_global = axes[0] if axes else None
+    kwargs = {"run_folder": run_folder, "scan_global": scan_global, "reps": reps, "name": name}
     try:
-        result = func(run_folder=run_folder, scan_global=scan_global, reps=reps, name=name)
-    except TypeError:
-        try:
-            result = func(run_folder, scan_global, reps, name)
-        except Exception as exc:
-            raise BridgeError("POSTPROCESS_FAILED", "postprocess raised: " + str(exc))
+        params = inspect.signature(func).parameters
+    except (TypeError, ValueError):
+        params = {}
+    if "scan_globals" in params or any(param.kind is param.VAR_KEYWORD for param in params.values()):
+        kwargs["scan_globals"] = axes
+    if not all(key in params for key in kwargs if key != "scan_globals"):
+        # Positional-only or differently named parameters: keep the historical call.
+        kwargs = {}
+    try:
+        result = func(**kwargs) if kwargs else func(run_folder, scan_global, reps, name)
     except Exception as exc:
-        raise BridgeError("POSTPROCESS_FAILED", "postprocess raised: " + str(exc))
+        raise BridgeError("POSTPROCESS_FAILED", "postprocess raised: " + type(exc).__name__ + ": " + str(exc))
     if not isinstance(result, dict):
         raise BridgeError("POSTPROCESS_BAD_RESULT", "postprocess must return a dict, got " + type(result).__name__)
     return jsonable(result)
 
 
-def resolve_dotted(obj, dotted):
+_MISSING = object()
+
+
+def resolve_dotted(obj, dotted, default=None):
     current = obj
     for part in str(dotted).split("."):
-        if not isinstance(current, dict):
-            return None
-        current = current.get(part)
+        if not isinstance(current, dict) or part not in current:
+            return default
+        current = current[part]
     return current
 
 
+def finish_value(result, dotted):
+    """(number, None) for a usable finish.* value, else (None, reason)."""
+    resolved = resolve_dotted(result, dotted, _MISSING)
+    if resolved is _MISSING:
+        return None, "missing"
+    value = as_number(resolved)
+    if value is None:
+        return None, "not a number (" + type(resolved).__name__ + ")"
+    return value, None
+
+
 def apply_finish(backend, config, plan, result, run_folder):
     out = {}
+    skipped = []
     finish = plan.get("finish") or {}
-    panel = finish.get("panel") or {}
     writes = {}
-    for key, dotted in panel.items():
-        value = as_number(resolve_dotted(result, dotted))
-        if value is not None:
-            writes[str(key)] = value
+    for key, dotted in (finish.get("panel") or {}).items():
+        value, reason = finish_value(result, dotted)
+        if value is None:
+            skipped.append({"target": "panel", "key": str(key), "dotted": str(dotted), "reason": reason})
+            continue
+        writes[str(key)] = value
     if writes:
         out["panel"] = jsonable(backend.set_globals(writes, raw=False))
     history = finish.get("history") or {}
@@ -854,8 +776,9 @@ def apply_finish(backend, config, plan, result, run_folder):
         os.makedirs(os.path.dirname(log_path), exist_ok=True)
         with open(log_path, "a", encoding="utf-8") as handle:
             for key, dotted in history.items():
-                value = as_number(resolve_dotted(result, dotted))
+                value, reason = finish_value(result, dotted)
                 if value is None:
+                    skipped.append({"target": "history", "key": str(key), "dotted": str(dotted), "reason": reason})
                     continue
                 row = {
                     "ts": now_iso(),
@@ -869,6 +792,8 @@ def apply_finish(backend, config, plan, result, run_folder):
                 rows.append(row)
     if rows:
         out["history"] = rows
+    if skipped:
+        out["skipped"] = skipped
     return out
 
 
@@ -1119,6 +1044,14 @@ def cmd_engage(config, args):
     return out
 
 
+# Same cause list for the stall branch and the timeout branch: BLACS did not run the shot.
+BLACS_NO_DATA_HINT = (
+    "shot files exist but show no acquisition data: BLACS may be paused, may have kept the shot out of its queue "
+    "(a shot's connection table must be a subset of the lab connection table) or re-queued it after a device "
+    "programming timeout. Check the BLACS GUI, then labs_abort if needed."
+)
+
+
 def wait_for_run(config, run_folder, expected, timeout_s):
     if not run_folder:
         raise BridgeError("BAD_ARGS", "run_folder is required")
@@ -1188,13 +1121,7 @@ def wait_for_run(config, run_folder, expected, timeout_s):
                 if compiled_since is None:
                     compiled_since = time.time()
                 elif time.time() - compiled_since >= stall:
-                    return result(
-                        False, False, "compiled_not_executed", time.time() - start,
-                        "shot files exist but show no acquisition data after " + str(int(stall)) + " s: BLACS may be "
-                        "paused, or it kept the shot out of the queue (its connection table must be a subset of the "
-                        "lab connection table) or a device group failed to program within 5 minutes (the queue then "
-                        "pauses and puts that shot back at its head), or the run is still being programmed. Check the "
-                        "BLACS GUI and the panel, then labs_abort if needed.")
+                    return result(False, False, "compiled_not_executed", time.time() - start, BLACS_NO_DATA_HINT)
             else:
                 compiled_since = None
         last_count = count
@@ -1221,9 +1148,7 @@ def wait_for_run(config, run_folder, expected, timeout_s):
                 hint = "only " + str(count) + " of " + str(expected) + " shots appeared before the timeout; the run may still be compiling or blocked."
             elif executed is False:
                 diagnosis = "compiled_not_executed"
-                hint = ("shot files exist but no acquisition data was found: BLACS may be paused, may have refused the "
-                        "shot (connection table not a subset of the lab connection table) or re-queued it after a device "
-                        "programming timeout. Check the BLACS GUI.")
+                hint = BLACS_NO_DATA_HINT
             else:
                 diagnosis = "partial"
                 hint = "shots appeared but the run was not confirmed complete before the timeout."
@@ -1241,8 +1166,6 @@ def cmd_abort(config, args):
     if backend_name == "labscript":
         note = ("labscript abort stops compiling/submitting new shots. Shots that are already compiled and queued "
                 "in BLACS, or currently running, are not stopped by it. This is not a hardware interlock.")
-    elif backend_name == "http-sim":
-        note = "http-sim abort stops the simulated queue (pause + clear)."
     else:
         note = "mock backend abort is a no-op."
     result = backend.abort()
@@ -1527,8 +1450,8 @@ def cmd_protocol_finalize(config, args):
     out = {"run_folder": run_folder}
     result = None
     if plan.get("postprocess"):
-        scan_global = next(iter(plan["scan"]), None) if plan["scan"] else None
-        result = run_postprocess(config, plan, map_path(config, run_folder), scan_global, plan["reps"])
+        scan_globals = [str(key) for key in plan["scan"]]
+        result = run_postprocess(config, plan, map_path(config, run_folder), scan_globals, plan["reps"])
         out["postprocess"] = result
     if plan.get("finish"):
         out["finish"] = apply_finish(backend, config, plan, result or {}, run_folder)
@@ -1560,17 +1483,25 @@ def cmd_doctor(config, args):
 
     backend = (config.get("backend") or "mock-labscript").lower()
     _add_check(checks, "backend", "ok", backend)
+    _add_check(checks, "python-executable", "ok", sys.executable)
     if backend == "labscript":
+        # Presence only: importing labscript in this process has side effects (its
+        # double-import denier walks the stack and leaves half-initialised modules
+        # behind), which used to break the runmanager checks below and turn a working
+        # environment into a false negative. Nothing here imports labscript itself.
         missing = []
         for module in ("labscript", "runmanager", "labscript_utils", "zprocess"):
             try:
-                importlib.import_module(module)
+                present = importlib.util.find_spec(module) is not None
             except Exception as exc:
-                missing.append(module + ": " + str(exc))
+                missing.append(module + ": " + type(exc).__name__ + ": " + str(exc))
+                continue
+            if not present:
+                missing.append(module + ": not installed")
         if missing:
             _add_check(checks, "labscript-suite", "fail", "; ".join(missing), "use the labscript-suite python environment")
         else:
-            _add_check(checks, "labscript-suite", "ok", "labscript / runmanager / labscript_utils / zprocess importable")
+            _add_check(checks, "labscript-suite", "ok", "labscript / runmanager / labscript_utils / zprocess installed")
         try:
             probe = LabscriptBackend(config)
             client = probe.client()

package/python/tests/run_tests.py

@@ -560,6 +560,178 @@ def test_doctor_mock():
         shutil.rmtree(tmp, ignore_errors=True)
 
 
+def test_doctor_checks_labscript_without_importing_it():
+    """doctor must not import labscript: the failed import used to break the runmanager
+    checks that follow it in the same process, reporting a false negative."""
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-doctor-import-")
+    try:
+        cfg = make_config(tmp)
+        cfg["backend"] = "labscript"
+
+        class FakeClient(object):
+            def say_hello(self):
+                return True
+
+            def get_version(self):
+                return "fake-1.0"
+
+        class FakeBackend(object):
+            def __init__(self, config):
+                self.config = config
+
+            def client(self):
+                return FakeClient()
+
+            def endpoint(self):
+                return "127.0.0.1:42523 (config)"
+
+            def n_shots(self):
+                return 3
+
+            def run_shots(self):
+                return True
+
+            def error_in_globals(self):
+                return False
+
+            def shot_output_folder(self):
+                return "/tmp/runs"
+
+        original_find = bridge.importlib.util.find_spec
+        original_backend = bridge.LabscriptBackend
+        saved = sys.modules.pop("labscript", None)
+        bridge.importlib.util.find_spec = lambda name, package=None: object()
+        bridge.LabscriptBackend = FakeBackend
+        try:
+            out = bridge.cmd_doctor(cfg, {})
+            assert "labscript" not in sys.modules, "doctor imported labscript"
+        finally:
+            bridge.importlib.util.find_spec = original_find
+            bridge.LabscriptBackend = original_backend
+            if saved is not None:
+                sys.modules["labscript"] = saved
+        names = dict((check["name"], check["status"]) for check in out["checks"])
+        assert names["labscript-suite"] == "ok", names
+        assert names["runmanager"] == "ok", names
+        assert names["python-executable"] == "ok", names
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_doctor_reports_missing_modules_without_importing():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-doctor-missing-")
+    try:
+        cfg = make_config(tmp)
+        cfg["backend"] = "labscript"
+        original_find = bridge.importlib.util.find_spec
+        bridge.importlib.util.find_spec = lambda name, package=None: None
+        try:
+            out = bridge.cmd_doctor(cfg, {})
+        finally:
+            bridge.importlib.util.find_spec = original_find
+        detail = dict((check["name"], check["detail"]) for check in out["checks"])
+        assert "not installed" in detail["labscript-suite"], detail
+        assert "No module named" not in detail["labscript-suite"], detail
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_finish_reports_values_instead_of_dropping_them():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-finish-")
+    try:
+        cfg = make_config(tmp)
+        written = {}
+
+        class FakeBackend(object):
+            def set_globals(self, values, raw=False):
+                written.update(values)
+                return dict(values)
+
+        plan = {
+            "name": "demo",
+            "finish": {
+                "panel": {"duration_test_us": "fit.center", "amp": "fit.width", "flag": "fit.flag", "none": "fit.none"},
+                "history": {"demo.x": "fit.absent"},
+            },
+        }
+        result = {"fit": {"center": 2.5, "width": "abc", "flag": True, "none": None}}
+        out = bridge.apply_finish(FakeBackend(), cfg, plan, result, "/tmp/run")
+        assert written == {"duration_test_us": 2.5}, written
+        skipped = dict((item["key"], item) for item in out["skipped"])
+        assert skipped["amp"]["reason"] == "not a number (str)", skipped
+        assert skipped["flag"]["reason"] == "not a number (bool)", skipped
+        assert skipped["none"]["reason"] == "not a number (NoneType)", skipped
+        assert skipped["demo.x"]["target"] == "history", skipped
+        assert skipped["demo.x"]["reason"] == "missing", skipped
+        assert out["panel"] == {"duration_test_us": 2.5}, out
+        with open(cfg["calibrationLog"], "r", encoding="utf-8") as handle:
+            assert handle.read().strip() == "", "a skipped history value must not be logged"
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_postprocess_gets_every_scan_axis():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-postprocess-axes-")
+    try:
+        cfg = make_config(tmp)
+        seen = {}
+
+        def analyze(run_folder, scan_global, reps, name, scan_globals=None):
+            seen.update({"run_folder": run_folder, "scan_global": scan_global, "reps": reps, "name": name, "scan_globals": scan_globals})
+            return {"fit": {"center": 1.0}}
+
+        module = types.ModuleType("fake_postprocess")
+        module.analyze = analyze
+        cfg["postprocess"] = {"entry": "fake_postprocess:analyze"}
+        original_import = bridge.importlib.import_module
+
+        def fake_import(name):
+            return module if name == "fake_postprocess" else original_import(name)
+
+        bridge.importlib.import_module = fake_import
+        try:
+            plan = {"name": "demo", "postprocess": "fake", "scan": {"duration_test_us": "x", "freq": "y"}}
+            result = bridge.run_postprocess(cfg, plan, "/tmp/run", ["duration_test_us", "freq"], 3)
+        finally:
+            bridge.importlib.import_module = original_import
+        assert result == {"fit": {"center": 1.0}}, result
+        assert seen["scan_globals"] == ["duration_test_us", "freq"], seen
+        assert seen["scan_global"] == "duration_test_us", seen
+        assert seen["reps"] == 3 and seen["name"] == "fake", seen
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_postprocess_keeps_supporting_a_positional_signature():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-postprocess-legacy-")
+    try:
+        cfg = make_config(tmp)
+        seen = []
+
+        def analyze(a, b, c, d):
+            seen.extend([a, b, c, d])
+            return {"fit": {"center": 1.0}}
+
+        module = types.ModuleType("legacy_postprocess")
+        module.analyze = analyze
+        cfg["postprocess"] = {"entry": "legacy_postprocess:analyze"}
+        original_import = bridge.importlib.import_module
+
+        def fake_import(name):
+            return module if name == "legacy_postprocess" else original_import(name)
+
+        bridge.importlib.import_module = fake_import
+        try:
+            plan = {"name": "legacy", "postprocess": "legacy", "scan": {"a": "x", "b": "y"}}
+            result = bridge.run_postprocess(cfg, plan, "/tmp/run", ["a", "b"], 1)
+        finally:
+            bridge.importlib.import_module = original_import
+        assert result == {"fit": {"center": 1.0}}, result
+        assert seen == ["/tmp/run", "a", 1, "legacy"], seen
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
 def test_status_maps_paths_from_runmanager():
     """labs_status must return paths the agent's machine can open."""
     tmp = tempfile.mkdtemp(prefix="dsh-labscript-py-map-")