dsh-labscript 0.3.2 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/README.md +8 -3
- package/docs/README.zh.md +3 -0
- package/examples/postprocess_demo.py +4 -3
- package/lib/runtime.js +2 -4
- package/lib/store.mjs +1 -1
- package/lib/tools.js +9 -1
- package/package.json +1 -1
- package/python/bridge.py +60 -129
- package/python/tests/run_tests.py +172 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,34 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.3
|
|
4
|
+
|
|
5
|
+
Makes the tools report what actually happened instead of hiding it, and removes a backend
|
|
6
|
+
nobody could reach. Net change is negative: about 40 lines of reporting added, a dead
|
|
7
|
+
backend removed.
|
|
8
|
+
|
|
9
|
+
- `labs_doctor` no longer imports labscript to check the environment. That import has side
|
|
10
|
+
effects (labscript's double-import denier walks the stack and leaves half-initialised
|
|
11
|
+
modules behind) and it broke the runmanager probe that runs right after it in the same
|
|
12
|
+
process, so a working environment was reported as two `[fail]` lines pointing at an h5py
|
|
13
|
+
path. The check is now `importlib.util.find_spec` (presence only: nothing in this plugin
|
|
14
|
+
imports labscript), error messages keep the exception type instead of only its text, and
|
|
15
|
+
doctor prints `python-executable` so "wrong interpreter" is visible at a glance.
|
|
16
|
+
- `labs_status` prints a panel field passed as if it were a global (`keys: ["n_shots"]`) as
|
|
17
|
+
`<value> (panel field, not a global)` instead of `<missing>`.
|
|
18
|
+
- `finish.panel` and `finish.history` report values they cannot write as `finish.skipped`,
|
|
19
|
+
with the dotted path and the reason (`missing` / `not a number (str)`); `labs_wait` renders
|
|
20
|
+
one `finish skipped:` line. These used to be dropped without a word, which reads exactly
|
|
21
|
+
like "the calibration number did not land".
|
|
22
|
+
- Postprocess receives `scan_globals` (every axis, in plan order) when its signature accepts
|
|
23
|
+
it; `scan_global` is unchanged. Dispatch uses `inspect.signature` instead of retrying on
|
|
24
|
+
`TypeError`, which also stops a `TypeError` raised inside the analysis function from running
|
|
25
|
+
it a second time.
|
|
26
|
+
- Removed the `http-sim` backend: an HTTP client for a local simulator service, added for a
|
|
27
|
+
one-off benchmark in v0.1 and later renamed to hide the internal service name. It had no
|
|
28
|
+
tests, no documentation and no entry on the Plugins page. `backend` is `labscript` or
|
|
29
|
+
`mock-labscript` again.
|
|
30
|
+
- Tests: 38 python + 49 node.
|
|
31
|
+
|
|
3
32
|
## 0.3.2
|
|
4
33
|
|
|
5
34
|
Implements the upstream request written from the field (labscript-suite workflow terms and
|
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@ files. It does not require any custom BLACS/lyse patch. Everything lab-specific
|
|
|
9
9
|
|
|
10
10
|
## Status
|
|
11
11
|
|
|
12
|
-
v0.
|
|
12
|
+
v0.3.3. The mock-labscript backend, the protocol/postprocess flow and the
|
|
13
13
|
snapshot/diagnosis behaviour are covered by node and python tests. A client
|
|
14
14
|
configuration page (Plugins page) and a read-only `labs_doctor` are included. The real
|
|
15
15
|
labscript backend is implemented against the stock runmanager remote API but has
|
|
@@ -75,6 +75,11 @@ guess panel state).
|
|
|
75
75
|
labs_protocol run demo_scan # set + scan + reps + engage, returns immediately
|
|
76
76
|
labs_wait # waits, then postprocess -> finish -> restore
|
|
77
77
|
|
|
78
|
+
`postprocess.entry` is called as `analyze(run_folder, scan_global, reps, name)`; when the
|
|
79
|
+
function accepts it, `scan_globals=[...]` (every axis, in order) is passed as well, so a 2D
|
|
80
|
+
scan is not limited to its first axis. A `finish` value that is missing or not a number is
|
|
81
|
+
reported as `skipped` and printed by `labs_wait` instead of being dropped silently.
|
|
82
|
+
|
|
78
83
|
## Configuration
|
|
79
84
|
|
|
80
85
|
Add one row to your DSH profile patch (see examples/config.example.yml):
|
|
@@ -140,8 +145,8 @@ is not a physics simulation.
|
|
|
140
145
|
## Development
|
|
141
146
|
|
|
142
147
|
npm install # only runtime dependency: yaml
|
|
143
|
-
npm test # node --test,
|
|
144
|
-
python python/tests/run_tests.py #
|
|
148
|
+
npm test # node --test, 49 tests (fake bridge, no python needed)
|
|
149
|
+
python python/tests/run_tests.py # 38 tests (needs numpy + h5py)
|
|
145
150
|
LABSCRIPT_PYTHON=/path/to/python3 node scripts/demo.mjs # end-to-end mock demo
|
|
146
151
|
|
|
147
152
|
A fresh clone must run `npm install` before `npm test`; the CI workflow does this.
|
package/docs/README.zh.md
CHANGED
|
@@ -49,6 +49,9 @@ hard limits 全部通过配置接入。
|
|
|
49
49
|
|
|
50
50
|
postprocess 是后处理:跑完后调用配置的 Python 函数(entry + pythonPaths),
|
|
51
51
|
返回 dict 供 finish 写回。旧协议里的 analysis: 字段当作 postprocess 别名。
|
|
52
|
+
函数按 `analyze(run_folder, scan_global, reps, name)` 调用;签名接受时还会传
|
|
53
|
+
`scan_globals=[...]`(全部扫描轴、按顺序),二维扫描因此能拿到第二轴。finish 解析到
|
|
54
|
+
缺失或非数字的值不再静默丢弃,会在 labs_wait 输出里以 `finish skipped` 列出。
|
|
52
55
|
|
|
53
56
|
## 边界
|
|
54
57
|
|
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
"""Example postprocess entry: after a run, return the numbers that finish.panel
|
|
2
2
|
and finish.history write back. Replace the fake model with your lab analysis.
|
|
3
3
|
|
|
4
|
-
Signature: analyze(run_folder=..., scan_global=..., reps=..., name=...)
|
|
5
|
-
|
|
4
|
+
Signature: analyze(run_folder=..., scan_global=..., reps=..., name=..., scan_globals=...)
|
|
5
|
+
scan_global is the first axis; scan_globals is every axis in order (a 2D scan needs both).
|
|
6
|
+
Returns a JSON-serializable dict; finish.panel reads the numbers back out of it.
|
|
6
7
|
"""
|
|
7
8
|
|
|
8
9
|
|
|
9
|
-
def analyze(run_folder, scan_global=None, reps=None, name=None):
|
|
10
|
+
def analyze(run_folder, scan_global=None, reps=None, name=None, scan_globals=None):
|
|
10
11
|
# Placeholder numbers so the write-back path is demonstrable end to end.
|
|
11
12
|
return {
|
|
12
13
|
"fit": {"center": 2.5, "pi_duration_us": 1.25, "r2": 0.99},
|
package/lib/runtime.js
CHANGED
|
@@ -29,7 +29,7 @@ const VERSION = PACKAGE.version
|
|
|
29
29
|
const DEFAULT_ENGAGE_TIMEOUT_S = 900
|
|
30
30
|
const DEFAULT_APPROVAL = { engage: true, protocolRun: true, protocolNames: true }
|
|
31
31
|
const DEFAULT_BACKEND = 'mock-labscript'
|
|
32
|
-
const BACKENDS = ['labscript', 'mock-labscript'
|
|
32
|
+
const BACKENDS = ['labscript', 'mock-labscript']
|
|
33
33
|
|
|
34
34
|
export class LabError extends Error {
|
|
35
35
|
constructor(code, message, detail) {
|
|
@@ -44,7 +44,7 @@ function normalizeConfig(raw) {
|
|
|
44
44
|
const input = raw && typeof raw === 'object' ? raw : {}
|
|
45
45
|
const backend = String(input.backend || DEFAULT_BACKEND).toLowerCase()
|
|
46
46
|
if (!BACKENDS.includes(backend)) {
|
|
47
|
-
throw new LabError('BAD_CONFIG', 'backend must be "labscript"
|
|
47
|
+
throw new LabError('BAD_CONFIG', 'backend must be "labscript" or "mock-labscript", got: ' + backend)
|
|
48
48
|
}
|
|
49
49
|
const approval = Object.assign({}, DEFAULT_APPROVAL, input.approval || {})
|
|
50
50
|
return {
|
|
@@ -67,7 +67,6 @@ function normalizeConfig(raw) {
|
|
|
67
67
|
protocolNames: approval.protocolNames !== false,
|
|
68
68
|
},
|
|
69
69
|
mock: input.mock || {},
|
|
70
|
-
simulator: input.simulator || {},
|
|
71
70
|
dshHome: input.dshHome || '',
|
|
72
71
|
// Must stay above runmanager's own client timeout (labconfig [timeouts]
|
|
73
72
|
// communication_timeout, default 60 s): the bridge is killed when this expires, which
|
|
@@ -89,7 +88,6 @@ function bridgePayload(config) {
|
|
|
89
88
|
engage: Object.assign({ timeoutS: DEFAULT_ENGAGE_TIMEOUT_S }, config.engage || {}),
|
|
90
89
|
panelHistoryDir: config.panelHistoryDir,
|
|
91
90
|
mock: config.mock,
|
|
92
|
-
simulator: config.simulator,
|
|
93
91
|
}
|
|
94
92
|
}
|
|
95
93
|
|
package/lib/store.mjs
CHANGED
|
@@ -25,7 +25,7 @@ export function statePath(home) {
|
|
|
25
25
|
}
|
|
26
26
|
|
|
27
27
|
const UI_KEYS = ['backend', 'python', 'runmanager', 'protocolsDir']
|
|
28
|
-
const BACKENDS = ['labscript', 'mock-labscript'
|
|
28
|
+
const BACKENDS = ['labscript', 'mock-labscript']
|
|
29
29
|
|
|
30
30
|
export function normalizeUiConfig(input) {
|
|
31
31
|
const source = input && typeof input === 'object' ? input : {}
|
package/lib/tools.js
CHANGED
|
@@ -62,7 +62,12 @@ function globalLines(args, value) {
|
|
|
62
62
|
const out = ['globals:']
|
|
63
63
|
for (const key of keys) {
|
|
64
64
|
const shown = raw ? raw[key] : globals[key]
|
|
65
|
-
|
|
65
|
+
// A panel field requested as if it were a global (n_shots is the usual one) should
|
|
66
|
+
// say what it is instead of looking like a global that vanished.
|
|
67
|
+
let text = '<missing>'
|
|
68
|
+
if (shown !== undefined) text = globalText(shown)
|
|
69
|
+
else if (Object.prototype.hasOwnProperty.call(value, key)) text = globalText(value[key]) + ' (panel field, not a global)'
|
|
70
|
+
out.push(' ' + key + ' = ' + text)
|
|
66
71
|
}
|
|
67
72
|
return out
|
|
68
73
|
}
|
|
@@ -382,6 +387,9 @@ export function registerTools(ctx, runtime) {
|
|
|
382
387
|
value.hint ? 'hint: ' + value.hint : '',
|
|
383
388
|
value.finalize && value.finalize.postprocess ? 'postprocess: ' + JSON.stringify(value.finalize.postprocess).slice(0, 400) : '',
|
|
384
389
|
value.finalize && value.finalize.finish ? 'finish: ' + JSON.stringify(value.finalize.finish).slice(0, 300) : '',
|
|
390
|
+
value.finalize && value.finalize.finish && Array.isArray(value.finalize.finish.skipped) && value.finalize.finish.skipped.length
|
|
391
|
+
? 'finish skipped: ' + value.finalize.finish.skipped.map((item) => item.key + ' <- ' + item.dotted + ' (' + item.reason + ')').join('; ')
|
|
392
|
+
: '',
|
|
385
393
|
value.finalize && value.finalize.restore ? 'restore: ' + (value.finalize.restore.restored ? 'yes' : 'no') : '',
|
|
386
394
|
]))),
|
|
387
395
|
async execute(args, exec) {
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-labscript",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.3",
|
|
4
4
|
"description": "Control labscript-suite experiments from DeepSeek Harness: runmanager globals and scans, submits and waits, YAML fixed-operation protocols with postprocess write-back, panel snapshots, a built-in mock backend and a read-only doctor.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
package/python/bridge.py
CHANGED
|
@@ -15,6 +15,8 @@ import sys
|
|
|
15
15
|
import time
|
|
16
16
|
import datetime
|
|
17
17
|
import importlib
|
|
18
|
+
import importlib.util
|
|
19
|
+
import inspect
|
|
18
20
|
import traceback
|
|
19
21
|
|
|
20
22
|
|
|
@@ -318,7 +320,7 @@ class LabscriptBackend(object):
|
|
|
318
320
|
try:
|
|
319
321
|
from runmanager.remote import Client
|
|
320
322
|
except Exception as exc:
|
|
321
|
-
raise BridgeError("BACKEND_UNAVAILABLE", "labscript python environment not importable: " + str(exc))
|
|
323
|
+
raise BridgeError("BACKEND_UNAVAILABLE", "labscript python environment not importable: " + type(exc).__name__ + ": " + str(exc))
|
|
322
324
|
# Anything left unset is resolved by runmanager.remote.Client from labscript's
|
|
323
325
|
# labconfig ([servers] runmanager, [ports] runmanager, [timeouts]
|
|
324
326
|
# communication_timeout), the same way the runmanager GUI does it. Explicit
|
|
@@ -538,113 +540,10 @@ class MockBackend(object):
|
|
|
538
540
|
return {"aborted": True, "mode": "mock-labscript"}
|
|
539
541
|
|
|
540
542
|
|
|
541
|
-
class HttpSimBackend(object):
|
|
542
|
-
"""HTTP client for an optional local simulator service (development backend).
|
|
543
|
-
|
|
544
|
-
The simulator must speak the documented JSON/HTTP contract on a loopback address.
|
|
545
|
-
It is a development tool, not a physics backend for publication.
|
|
546
|
-
"""
|
|
547
|
-
|
|
548
|
-
def __init__(self, config):
|
|
549
|
-
simulator = config.get("simulator") or {}
|
|
550
|
-
url = str(simulator.get("url") or "http://127.0.0.1:18761").rstrip("/")
|
|
551
|
-
if not url.startswith("http://127.0.0.1:"):
|
|
552
|
-
raise BridgeError("BAD_CONFIG", "simulator.url must be http://127.0.0.1:<port>, got: " + url)
|
|
553
|
-
self.base = url
|
|
554
|
-
self.timeout = float(simulator.get("timeoutS") or 10)
|
|
555
|
-
self._opener = None
|
|
556
|
-
|
|
557
|
-
def opener(self):
|
|
558
|
-
import urllib.request
|
|
559
|
-
if self._opener is not None:
|
|
560
|
-
return self._opener
|
|
561
|
-
|
|
562
|
-
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
|
563
|
-
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
|
564
|
-
return None
|
|
565
|
-
|
|
566
|
-
self._opener = urllib.request.build_opener(NoRedirect, urllib.request.ProxyHandler({}))
|
|
567
|
-
return self._opener
|
|
568
|
-
|
|
569
|
-
def request(self, method, path, body=None):
|
|
570
|
-
import urllib.request
|
|
571
|
-
import urllib.error
|
|
572
|
-
data = None
|
|
573
|
-
headers = {}
|
|
574
|
-
if body is not None:
|
|
575
|
-
data = json.dumps(jsonable(body)).encode("utf-8")
|
|
576
|
-
headers["Content-Type"] = "application/json"
|
|
577
|
-
req = urllib.request.Request(self.base + path, data=data, headers=headers, method=method)
|
|
578
|
-
try:
|
|
579
|
-
with self.opener().open(req, timeout=self.timeout) as response:
|
|
580
|
-
raw = response.read().decode("utf-8")
|
|
581
|
-
except urllib.error.HTTPError as exc:
|
|
582
|
-
try:
|
|
583
|
-
payload = json.loads(exc.read().decode("utf-8"))
|
|
584
|
-
except Exception:
|
|
585
|
-
raise BridgeError("BACKEND_UNAVAILABLE", "http-sim HTTP " + str(exc.code) + " at " + path)
|
|
586
|
-
error = payload.get("error") or {}
|
|
587
|
-
raise BridgeError(str(error.get("code") or "BACKEND_UNAVAILABLE"), str(error.get("message") or "http-sim error"))
|
|
588
|
-
except Exception as exc:
|
|
589
|
-
raise BridgeError("BACKEND_UNAVAILABLE", "http-sim not reachable at " + self.base + ": " + str(exc))
|
|
590
|
-
try:
|
|
591
|
-
envelope = json.loads(raw)
|
|
592
|
-
except Exception:
|
|
593
|
-
raise BridgeError("BACKEND_UNAVAILABLE", "http-sim response is not JSON")
|
|
594
|
-
if envelope.get("ok") is not True:
|
|
595
|
-
error = envelope.get("error") or {}
|
|
596
|
-
raise BridgeError(str(error.get("code") or "BACKEND_UNAVAILABLE"), str(error.get("message") or "http-sim error"))
|
|
597
|
-
return envelope.get("data") or {}
|
|
598
|
-
|
|
599
|
-
def snapshot(self):
|
|
600
|
-
return self.request("GET", "/v1/rm/snapshot")
|
|
601
|
-
|
|
602
|
-
def get_globals(self, raw=False):
|
|
603
|
-
snap = self.snapshot()
|
|
604
|
-
key = "raw_globals" if raw else "globals"
|
|
605
|
-
return dict(snap.get(key) or {})
|
|
606
|
-
|
|
607
|
-
def set_globals(self, values, raw=False):
|
|
608
|
-
body = dict((k, v.tolist() if hasattr(v, "tolist") else v) for k, v in values.items())
|
|
609
|
-
data = self.request("POST", "/v1/rm/set_globals", {"values": body, "raw": bool(raw)})
|
|
610
|
-
return dict(data.get("written") or {})
|
|
611
|
-
|
|
612
|
-
def n_shots(self):
|
|
613
|
-
return int(self.snapshot().get("n_shots") or 0)
|
|
614
|
-
|
|
615
|
-
def error_in_globals(self):
|
|
616
|
-
return bool(self.snapshot().get("error_in_globals"))
|
|
617
|
-
|
|
618
|
-
def run_shots(self):
|
|
619
|
-
return True
|
|
620
|
-
|
|
621
|
-
def shuffle(self):
|
|
622
|
-
snap = self.snapshot()
|
|
623
|
-
return bool(snap.get("shuffle")) if "shuffle" in snap else None
|
|
624
|
-
|
|
625
|
-
def labscript_file(self):
|
|
626
|
-
return self.snapshot().get("labscript_file")
|
|
627
|
-
|
|
628
|
-
def shot_output_folder(self):
|
|
629
|
-
return self.snapshot().get("shot_output_folder")
|
|
630
|
-
|
|
631
|
-
def engage(self):
|
|
632
|
-
data = self.request("POST", "/v1/rm/engage", {})
|
|
633
|
-
if data.get("accepted") is not True:
|
|
634
|
-
raise BridgeError("ENGAGE_REJECTED", "http-sim did not accept engage: " + str(data.get("detail") or ""))
|
|
635
|
-
return {"accepted": True, "n_shots": data.get("n_shots"), "run_folder": data.get("run_folder"), "mode": "http-sim"}
|
|
636
|
-
|
|
637
|
-
def abort(self):
|
|
638
|
-
data = self.request("POST", "/v1/blacs/stop", {})
|
|
639
|
-
return {"aborted": True, "via": "http-sim/blacs/stop", "result": data}
|
|
640
|
-
|
|
641
|
-
|
|
642
543
|
def make_backend(config):
|
|
643
544
|
backend = (config.get("backend") or "mock-labscript").lower()
|
|
644
545
|
if backend == "mock-labscript":
|
|
645
546
|
return MockBackend(config)
|
|
646
|
-
if backend == "http-sim":
|
|
647
|
-
return HttpSimBackend(config)
|
|
648
547
|
if backend == "labscript":
|
|
649
548
|
return LabscriptBackend(config)
|
|
650
549
|
raise BridgeError("BAD_CONFIG", "unknown backend: " + backend)
|
|
@@ -791,7 +690,7 @@ def check_limits(config, set_values, scan_values, expected_n_shots):
|
|
|
791
690
|
# ---------------------------------------------------------------- postprocess
|
|
792
691
|
|
|
793
692
|
|
|
794
|
-
def run_postprocess(config, plan, run_folder,
|
|
693
|
+
def run_postprocess(config, plan, run_folder, scan_globals, reps):
|
|
795
694
|
name = plan.get("postprocess")
|
|
796
695
|
if not name:
|
|
797
696
|
return None
|
|
@@ -812,38 +711,61 @@ def run_postprocess(config, plan, run_folder, scan_global, reps):
|
|
|
812
711
|
func = getattr(module, func_name)
|
|
813
712
|
except Exception as exc:
|
|
814
713
|
raise BridgeError("POSTPROCESS_IMPORT_FAILED", "cannot import postprocess " + entry + ": " + str(exc))
|
|
714
|
+
axes = [str(key) for key in (scan_globals or [])]
|
|
715
|
+
scan_global = axes[0] if axes else None
|
|
716
|
+
kwargs = {"run_folder": run_folder, "scan_global": scan_global, "reps": reps, "name": name}
|
|
815
717
|
try:
|
|
816
|
-
|
|
817
|
-
except TypeError:
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
718
|
+
params = inspect.signature(func).parameters
|
|
719
|
+
except (TypeError, ValueError):
|
|
720
|
+
params = {}
|
|
721
|
+
if "scan_globals" in params or any(param.kind is param.VAR_KEYWORD for param in params.values()):
|
|
722
|
+
kwargs["scan_globals"] = axes
|
|
723
|
+
if not all(key in params for key in kwargs if key != "scan_globals"):
|
|
724
|
+
# Positional-only or differently named parameters: keep the historical call.
|
|
725
|
+
kwargs = {}
|
|
726
|
+
try:
|
|
727
|
+
result = func(**kwargs) if kwargs else func(run_folder, scan_global, reps, name)
|
|
822
728
|
except Exception as exc:
|
|
823
|
-
raise BridgeError("POSTPROCESS_FAILED", "postprocess raised: " + str(exc))
|
|
729
|
+
raise BridgeError("POSTPROCESS_FAILED", "postprocess raised: " + type(exc).__name__ + ": " + str(exc))
|
|
824
730
|
if not isinstance(result, dict):
|
|
825
731
|
raise BridgeError("POSTPROCESS_BAD_RESULT", "postprocess must return a dict, got " + type(result).__name__)
|
|
826
732
|
return jsonable(result)
|
|
827
733
|
|
|
828
734
|
|
|
829
|
-
|
|
735
|
+
_MISSING = object()
|
|
736
|
+
|
|
737
|
+
|
|
738
|
+
def resolve_dotted(obj, dotted, default=None):
|
|
830
739
|
current = obj
|
|
831
740
|
for part in str(dotted).split("."):
|
|
832
|
-
if not isinstance(current, dict):
|
|
833
|
-
return
|
|
834
|
-
current = current
|
|
741
|
+
if not isinstance(current, dict) or part not in current:
|
|
742
|
+
return default
|
|
743
|
+
current = current[part]
|
|
835
744
|
return current
|
|
836
745
|
|
|
837
746
|
|
|
747
|
+
def finish_value(result, dotted):
|
|
748
|
+
"""(number, None) for a usable finish.* value, else (None, reason)."""
|
|
749
|
+
resolved = resolve_dotted(result, dotted, _MISSING)
|
|
750
|
+
if resolved is _MISSING:
|
|
751
|
+
return None, "missing"
|
|
752
|
+
value = as_number(resolved)
|
|
753
|
+
if value is None:
|
|
754
|
+
return None, "not a number (" + type(resolved).__name__ + ")"
|
|
755
|
+
return value, None
|
|
756
|
+
|
|
757
|
+
|
|
838
758
|
def apply_finish(backend, config, plan, result, run_folder):
|
|
839
759
|
out = {}
|
|
760
|
+
skipped = []
|
|
840
761
|
finish = plan.get("finish") or {}
|
|
841
|
-
panel = finish.get("panel") or {}
|
|
842
762
|
writes = {}
|
|
843
|
-
for key, dotted in panel.items():
|
|
844
|
-
value =
|
|
845
|
-
if value is
|
|
846
|
-
|
|
763
|
+
for key, dotted in (finish.get("panel") or {}).items():
|
|
764
|
+
value, reason = finish_value(result, dotted)
|
|
765
|
+
if value is None:
|
|
766
|
+
skipped.append({"target": "panel", "key": str(key), "dotted": str(dotted), "reason": reason})
|
|
767
|
+
continue
|
|
768
|
+
writes[str(key)] = value
|
|
847
769
|
if writes:
|
|
848
770
|
out["panel"] = jsonable(backend.set_globals(writes, raw=False))
|
|
849
771
|
history = finish.get("history") or {}
|
|
@@ -854,8 +776,9 @@ def apply_finish(backend, config, plan, result, run_folder):
|
|
|
854
776
|
os.makedirs(os.path.dirname(log_path), exist_ok=True)
|
|
855
777
|
with open(log_path, "a", encoding="utf-8") as handle:
|
|
856
778
|
for key, dotted in history.items():
|
|
857
|
-
value =
|
|
779
|
+
value, reason = finish_value(result, dotted)
|
|
858
780
|
if value is None:
|
|
781
|
+
skipped.append({"target": "history", "key": str(key), "dotted": str(dotted), "reason": reason})
|
|
859
782
|
continue
|
|
860
783
|
row = {
|
|
861
784
|
"ts": now_iso(),
|
|
@@ -869,6 +792,8 @@ def apply_finish(backend, config, plan, result, run_folder):
|
|
|
869
792
|
rows.append(row)
|
|
870
793
|
if rows:
|
|
871
794
|
out["history"] = rows
|
|
795
|
+
if skipped:
|
|
796
|
+
out["skipped"] = skipped
|
|
872
797
|
return out
|
|
873
798
|
|
|
874
799
|
|
|
@@ -1241,8 +1166,6 @@ def cmd_abort(config, args):
|
|
|
1241
1166
|
if backend_name == "labscript":
|
|
1242
1167
|
note = ("labscript abort stops compiling/submitting new shots. Shots that are already compiled and queued "
|
|
1243
1168
|
"in BLACS, or currently running, are not stopped by it. This is not a hardware interlock.")
|
|
1244
|
-
elif backend_name == "http-sim":
|
|
1245
|
-
note = "http-sim abort stops the simulated queue (pause + clear)."
|
|
1246
1169
|
else:
|
|
1247
1170
|
note = "mock backend abort is a no-op."
|
|
1248
1171
|
result = backend.abort()
|
|
@@ -1527,8 +1450,8 @@ def cmd_protocol_finalize(config, args):
|
|
|
1527
1450
|
out = {"run_folder": run_folder}
|
|
1528
1451
|
result = None
|
|
1529
1452
|
if plan.get("postprocess"):
|
|
1530
|
-
|
|
1531
|
-
result = run_postprocess(config, plan, map_path(config, run_folder),
|
|
1453
|
+
scan_globals = [str(key) for key in plan["scan"]]
|
|
1454
|
+
result = run_postprocess(config, plan, map_path(config, run_folder), scan_globals, plan["reps"])
|
|
1532
1455
|
out["postprocess"] = result
|
|
1533
1456
|
if plan.get("finish"):
|
|
1534
1457
|
out["finish"] = apply_finish(backend, config, plan, result or {}, run_folder)
|
|
@@ -1560,17 +1483,25 @@ def cmd_doctor(config, args):
|
|
|
1560
1483
|
|
|
1561
1484
|
backend = (config.get("backend") or "mock-labscript").lower()
|
|
1562
1485
|
_add_check(checks, "backend", "ok", backend)
|
|
1486
|
+
_add_check(checks, "python-executable", "ok", sys.executable)
|
|
1563
1487
|
if backend == "labscript":
|
|
1488
|
+
# Presence only: importing labscript in this process has side effects (its
|
|
1489
|
+
# double-import denier walks the stack and leaves half-initialised modules
|
|
1490
|
+
# behind), which used to break the runmanager checks below and turn a working
|
|
1491
|
+
# environment into a false negative. Nothing here imports labscript itself.
|
|
1564
1492
|
missing = []
|
|
1565
1493
|
for module in ("labscript", "runmanager", "labscript_utils", "zprocess"):
|
|
1566
1494
|
try:
|
|
1567
|
-
importlib.
|
|
1495
|
+
present = importlib.util.find_spec(module) is not None
|
|
1568
1496
|
except Exception as exc:
|
|
1569
|
-
missing.append(module + ": " + str(exc))
|
|
1497
|
+
missing.append(module + ": " + type(exc).__name__ + ": " + str(exc))
|
|
1498
|
+
continue
|
|
1499
|
+
if not present:
|
|
1500
|
+
missing.append(module + ": not installed")
|
|
1570
1501
|
if missing:
|
|
1571
1502
|
_add_check(checks, "labscript-suite", "fail", "; ".join(missing), "use the labscript-suite python environment")
|
|
1572
1503
|
else:
|
|
1573
|
-
_add_check(checks, "labscript-suite", "ok", "labscript / runmanager / labscript_utils / zprocess
|
|
1504
|
+
_add_check(checks, "labscript-suite", "ok", "labscript / runmanager / labscript_utils / zprocess installed")
|
|
1574
1505
|
try:
|
|
1575
1506
|
probe = LabscriptBackend(config)
|
|
1576
1507
|
client = probe.client()
|
|
@@ -560,6 +560,178 @@ def test_doctor_mock():
|
|
|
560
560
|
shutil.rmtree(tmp, ignore_errors=True)
|
|
561
561
|
|
|
562
562
|
|
|
563
|
+
def test_doctor_checks_labscript_without_importing_it():
|
|
564
|
+
"""doctor must not import labscript: the failed import used to break the runmanager
|
|
565
|
+
checks that follow it in the same process, reporting a false negative."""
|
|
566
|
+
tmp = tempfile.mkdtemp(prefix="dsh-labscript-doctor-import-")
|
|
567
|
+
try:
|
|
568
|
+
cfg = make_config(tmp)
|
|
569
|
+
cfg["backend"] = "labscript"
|
|
570
|
+
|
|
571
|
+
class FakeClient(object):
|
|
572
|
+
def say_hello(self):
|
|
573
|
+
return True
|
|
574
|
+
|
|
575
|
+
def get_version(self):
|
|
576
|
+
return "fake-1.0"
|
|
577
|
+
|
|
578
|
+
class FakeBackend(object):
|
|
579
|
+
def __init__(self, config):
|
|
580
|
+
self.config = config
|
|
581
|
+
|
|
582
|
+
def client(self):
|
|
583
|
+
return FakeClient()
|
|
584
|
+
|
|
585
|
+
def endpoint(self):
|
|
586
|
+
return "127.0.0.1:42523 (config)"
|
|
587
|
+
|
|
588
|
+
def n_shots(self):
|
|
589
|
+
return 3
|
|
590
|
+
|
|
591
|
+
def run_shots(self):
|
|
592
|
+
return True
|
|
593
|
+
|
|
594
|
+
def error_in_globals(self):
|
|
595
|
+
return False
|
|
596
|
+
|
|
597
|
+
def shot_output_folder(self):
|
|
598
|
+
return "/tmp/runs"
|
|
599
|
+
|
|
600
|
+
original_find = bridge.importlib.util.find_spec
|
|
601
|
+
original_backend = bridge.LabscriptBackend
|
|
602
|
+
saved = sys.modules.pop("labscript", None)
|
|
603
|
+
bridge.importlib.util.find_spec = lambda name, package=None: object()
|
|
604
|
+
bridge.LabscriptBackend = FakeBackend
|
|
605
|
+
try:
|
|
606
|
+
out = bridge.cmd_doctor(cfg, {})
|
|
607
|
+
assert "labscript" not in sys.modules, "doctor imported labscript"
|
|
608
|
+
finally:
|
|
609
|
+
bridge.importlib.util.find_spec = original_find
|
|
610
|
+
bridge.LabscriptBackend = original_backend
|
|
611
|
+
if saved is not None:
|
|
612
|
+
sys.modules["labscript"] = saved
|
|
613
|
+
names = dict((check["name"], check["status"]) for check in out["checks"])
|
|
614
|
+
assert names["labscript-suite"] == "ok", names
|
|
615
|
+
assert names["runmanager"] == "ok", names
|
|
616
|
+
assert names["python-executable"] == "ok", names
|
|
617
|
+
finally:
|
|
618
|
+
shutil.rmtree(tmp, ignore_errors=True)
|
|
619
|
+
|
|
620
|
+
|
|
621
|
+
def test_doctor_reports_missing_modules_without_importing():
|
|
622
|
+
tmp = tempfile.mkdtemp(prefix="dsh-labscript-doctor-missing-")
|
|
623
|
+
try:
|
|
624
|
+
cfg = make_config(tmp)
|
|
625
|
+
cfg["backend"] = "labscript"
|
|
626
|
+
original_find = bridge.importlib.util.find_spec
|
|
627
|
+
bridge.importlib.util.find_spec = lambda name, package=None: None
|
|
628
|
+
try:
|
|
629
|
+
out = bridge.cmd_doctor(cfg, {})
|
|
630
|
+
finally:
|
|
631
|
+
bridge.importlib.util.find_spec = original_find
|
|
632
|
+
detail = dict((check["name"], check["detail"]) for check in out["checks"])
|
|
633
|
+
assert "not installed" in detail["labscript-suite"], detail
|
|
634
|
+
assert "No module named" not in detail["labscript-suite"], detail
|
|
635
|
+
finally:
|
|
636
|
+
shutil.rmtree(tmp, ignore_errors=True)
|
|
637
|
+
|
|
638
|
+
|
|
639
|
+
def test_finish_reports_values_instead_of_dropping_them():
|
|
640
|
+
tmp = tempfile.mkdtemp(prefix="dsh-labscript-finish-")
|
|
641
|
+
try:
|
|
642
|
+
cfg = make_config(tmp)
|
|
643
|
+
written = {}
|
|
644
|
+
|
|
645
|
+
class FakeBackend(object):
|
|
646
|
+
def set_globals(self, values, raw=False):
|
|
647
|
+
written.update(values)
|
|
648
|
+
return dict(values)
|
|
649
|
+
|
|
650
|
+
plan = {
|
|
651
|
+
"name": "demo",
|
|
652
|
+
"finish": {
|
|
653
|
+
"panel": {"duration_test_us": "fit.center", "amp": "fit.width", "flag": "fit.flag", "none": "fit.none"},
|
|
654
|
+
"history": {"demo.x": "fit.absent"},
|
|
655
|
+
},
|
|
656
|
+
}
|
|
657
|
+
result = {"fit": {"center": 2.5, "width": "abc", "flag": True, "none": None}}
|
|
658
|
+
out = bridge.apply_finish(FakeBackend(), cfg, plan, result, "/tmp/run")
|
|
659
|
+
assert written == {"duration_test_us": 2.5}, written
|
|
660
|
+
skipped = dict((item["key"], item) for item in out["skipped"])
|
|
661
|
+
assert skipped["amp"]["reason"] == "not a number (str)", skipped
|
|
662
|
+
assert skipped["flag"]["reason"] == "not a number (bool)", skipped
|
|
663
|
+
assert skipped["none"]["reason"] == "not a number (NoneType)", skipped
|
|
664
|
+
assert skipped["demo.x"]["target"] == "history", skipped
|
|
665
|
+
assert skipped["demo.x"]["reason"] == "missing", skipped
|
|
666
|
+
assert out["panel"] == {"duration_test_us": 2.5}, out
|
|
667
|
+
with open(cfg["calibrationLog"], "r", encoding="utf-8") as handle:
|
|
668
|
+
assert handle.read().strip() == "", "a skipped history value must not be logged"
|
|
669
|
+
finally:
|
|
670
|
+
shutil.rmtree(tmp, ignore_errors=True)
|
|
671
|
+
|
|
672
|
+
|
|
673
|
+
def test_postprocess_gets_every_scan_axis():
|
|
674
|
+
tmp = tempfile.mkdtemp(prefix="dsh-labscript-postprocess-axes-")
|
|
675
|
+
try:
|
|
676
|
+
cfg = make_config(tmp)
|
|
677
|
+
seen = {}
|
|
678
|
+
|
|
679
|
+
def analyze(run_folder, scan_global, reps, name, scan_globals=None):
|
|
680
|
+
seen.update({"run_folder": run_folder, "scan_global": scan_global, "reps": reps, "name": name, "scan_globals": scan_globals})
|
|
681
|
+
return {"fit": {"center": 1.0}}
|
|
682
|
+
|
|
683
|
+
module = types.ModuleType("fake_postprocess")
|
|
684
|
+
module.analyze = analyze
|
|
685
|
+
cfg["postprocess"] = {"entry": "fake_postprocess:analyze"}
|
|
686
|
+
original_import = bridge.importlib.import_module
|
|
687
|
+
|
|
688
|
+
def fake_import(name):
|
|
689
|
+
return module if name == "fake_postprocess" else original_import(name)
|
|
690
|
+
|
|
691
|
+
bridge.importlib.import_module = fake_import
|
|
692
|
+
try:
|
|
693
|
+
plan = {"name": "demo", "postprocess": "fake", "scan": {"duration_test_us": "x", "freq": "y"}}
|
|
694
|
+
result = bridge.run_postprocess(cfg, plan, "/tmp/run", ["duration_test_us", "freq"], 3)
|
|
695
|
+
finally:
|
|
696
|
+
bridge.importlib.import_module = original_import
|
|
697
|
+
assert result == {"fit": {"center": 1.0}}, result
|
|
698
|
+
assert seen["scan_globals"] == ["duration_test_us", "freq"], seen
|
|
699
|
+
assert seen["scan_global"] == "duration_test_us", seen
|
|
700
|
+
assert seen["reps"] == 3 and seen["name"] == "fake", seen
|
|
701
|
+
finally:
|
|
702
|
+
shutil.rmtree(tmp, ignore_errors=True)
|
|
703
|
+
|
|
704
|
+
|
|
705
|
+
def test_postprocess_keeps_supporting_a_positional_signature():
|
|
706
|
+
tmp = tempfile.mkdtemp(prefix="dsh-labscript-postprocess-legacy-")
|
|
707
|
+
try:
|
|
708
|
+
cfg = make_config(tmp)
|
|
709
|
+
seen = []
|
|
710
|
+
|
|
711
|
+
def analyze(a, b, c, d):
|
|
712
|
+
seen.extend([a, b, c, d])
|
|
713
|
+
return {"fit": {"center": 1.0}}
|
|
714
|
+
|
|
715
|
+
module = types.ModuleType("legacy_postprocess")
|
|
716
|
+
module.analyze = analyze
|
|
717
|
+
cfg["postprocess"] = {"entry": "legacy_postprocess:analyze"}
|
|
718
|
+
original_import = bridge.importlib.import_module
|
|
719
|
+
|
|
720
|
+
def fake_import(name):
|
|
721
|
+
return module if name == "legacy_postprocess" else original_import(name)
|
|
722
|
+
|
|
723
|
+
bridge.importlib.import_module = fake_import
|
|
724
|
+
try:
|
|
725
|
+
plan = {"name": "legacy", "postprocess": "legacy", "scan": {"a": "x", "b": "y"}}
|
|
726
|
+
result = bridge.run_postprocess(cfg, plan, "/tmp/run", ["a", "b"], 1)
|
|
727
|
+
finally:
|
|
728
|
+
bridge.importlib.import_module = original_import
|
|
729
|
+
assert result == {"fit": {"center": 1.0}}, result
|
|
730
|
+
assert seen == ["/tmp/run", "a", 1, "legacy"], seen
|
|
731
|
+
finally:
|
|
732
|
+
shutil.rmtree(tmp, ignore_errors=True)
|
|
733
|
+
|
|
734
|
+
|
|
563
735
|
def test_status_maps_paths_from_runmanager():
|
|
564
736
|
"""labs_status must return paths the agent's machine can open."""
|
|
565
737
|
tmp = tempfile.mkdtemp(prefix="dsh-labscript-py-map-")
|