dsh-labscript 0.2.7 → 0.2.8

package/CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## 0.2.8
+
+Fixes from a real labscript-suite session (2026-11-14, 145 tool calls, no run submitted by the
+plugin). Each defect was reproduced against the stock runmanager source before the fix.
+
+- Fix: labs_set failed with "TypeError: 'NoneType' object is not iterable" after the write had
+  already landed. runmanager's set_globals RPC is a side-effect call whose handler returns None,
+  and the bridge fed that reply to dict(). It now reports the requested values and lets the
+  readback decide success. When a write really fails, the error carries `landed` (what the panel
+  holds now) and the hint says to read the keys back instead of claiming the panel may be half
+  written and pointing at labs_restore.
+- Fix: labs_wait blocked for the whole timeout when compilation aborted. It returns after
+  engage.noShotS seconds (default 25) with diagnosis no_shots (no shot file appeared at all) or
+  compile_stalled (files stopped appearing), so the runmanager compile error can be read instead
+  of waiting 5-15 minutes. The hint says to call labs_wait again if a slow compile is expected.
+- Fix: a pending run that never produced a shot blocked labs_engage forever. A pending submission
+  with zero shots and older than engage.staleS (default 180 s) is dropped and reported as
+  dropped_stale_pending; the RUN_PENDING error now states the run's age and both ways out.
+- Fix: with pathMap configured, labs_wait(run_folder=<the folder labs_engage printed>) failed with
+  RUN_FOLDER_MISSING: the pending run kept the unmapped path and the wait compared strings. Pending
+  state now stores the mapped folder (raw kept as remote_run_folder) and both forms match.
+- Fix: shot h5 files that only hold compiled globals were counted as progress, so h5=1/64 looked
+  like "a shot ran". Progress reports n_h5 (files) and n_executed (shots with acquisition data,
+  located by binary search over the file order), completion requires acquisition data in the last
+  shot, and labs_status renders executed=.
+- labs_protocol show prints the closest key names for plan keys missing from the loaded labscript
+  (previously only the run path did), so stale protocols can be renamed without guessing.
+- labs_set description: one call can mix scalars, an expression scan axis and reps=N.
+- Tests: 28 python + 46 node, adding a client whose set_globals returns None, a stalled
+  compilation, a stale zero-shot pending run, mapped/raw pending folders and the show suggestions.
+
 ## 0.2.7
 
 - Fix: labs_wait no longer fails with "value is not lossless JSON". Tool results must be

package/README.md

@@ -103,6 +103,14 @@ already mapped: `labs_status`, `labs_engage`, `labs_wait`, `labs_data` and
 `labs_protocol` show local paths, while `run_folder` keeps the raw runmanager
 value next to `mapped_run_folder`.
 
+`engage` controls how a submitted run is watched: `timeoutS` (default 900) is the
+total wait budget of one `labs_wait`, `noShotS` (default 25) makes the wait return
+early with a diagnosis when no shot appeared, `staleS` (default 180) drops a pending
+submission that never produced a single shot, `pollS`, `settleS` and `stallS` tune the
+polling and the compiled-not-executed detection.
+
+    engage: { timeoutS: 900, noShotS: 25, staleS: 180, stallS: 20 }
+
 The python interpreter must be able to import numpy and h5py; the real backend
 also needs runmanager (labscript-suite). The plugin never stores credentials:
 zprocess authentication is read by labscript itself from labconfig.
@@ -125,8 +133,8 @@ is not a physics simulation.
 ## Development
 
     npm install              # only runtime dependency: yaml
-    npm test                 # node --test, 40 tests (fake bridge, no python needed)
-    python python/tests/run_tests.py   # 22 tests (needs numpy + h5py)
+    npm test                 # node --test, 46 tests (fake bridge, no python needed)
+    python python/tests/run_tests.py   # 28 tests (needs numpy + h5py)
     LABSCRIPT_PYTHON=/path/to/python3 node scripts/demo.mjs   # end-to-end mock demo
 
 A fresh clone must run `npm install` before `npm test`; the CI workflow does this.
@@ -153,9 +161,19 @@ A fresh clone must run `npm install` before `npm test`; the CI workflow does thi
   explicitly when something must stop; labs_abort also cancels an in-flight labs_wait.
   labscript abort stops compiling/submitting new shots; shots already queued in BLACS
   or running are not stopped.
-- labs_wait reports a diagnosis (complete, no_shots, compiling, compiled_not_executed,
-  partial). compiled_not_executed means shots exist but show no acquisition data yet;
-  BLACS may be paused or not executing the queue.
+- labs_wait reports a diagnosis (complete, no_shots, compile_stalled, compiling,
+  compiled_not_executed, partial). no_shots and compile_stalled come back before the
+  timeout when nothing new appeared for engage.noShotS seconds: runmanager probably
+  aborted the compilation, so read its output. compiled_not_executed means shots were
+  compiled but show no acquisition data yet; BLACS may be paused or not executing the queue.
+- labs_engage only reports that runmanager accepted the submission; compilation errors are
+  invisible to it. A pending submission with no shots at all is dropped after
+  engage.staleS seconds and reported as dropped_stale_pending, so a dead run does not block
+  the next one forever.
+- Shot h5 files exist before they are executed: progress reports n_h5 (files) and
+  n_executed (shots with acquisition data), and completion requires acquisition data.
+- The pending run is stored under the pathMap-mapped folder (the raw runmanager path is kept
+  as remote_run_folder), so labs_wait accepts whichever of the two an agent quotes back.
 - Real runmanager engage does not return a run folder; the bridge captures the
   output folder before engage and reports it as the run folder. That shot output
   folder is the NEXT run's folder and does not exist until engage, so labs_data

package/docs/README.zh.md

@@ -11,7 +11,7 @@ hard limits 全部通过配置接入。
 |---|---|
 | labs_doctor | 只读诊断：python、numpy/h5py、labscript/runmanager、路径、审批、版本 |
 | labs_status | 读面板：labscript、globals、n_shots、error_in_globals、run_shots、pending 进度 |
-| labs_set | 写面板（标量 / raw 表达式 / reps），写前全量快照；不运行 |
+| labs_set | 写面板（标量 / raw 表达式 / reps 可一次混用），写前全量快照；不运行 |
 | labs_engage | 提交当前面板，立即返回 n_shots_at_submit 与 expected_n_shots |
 | labs_wait | 等待 pending run（可超时）；若是协议则收尾：postprocess → finish 写回 → restore |
 | labs_abort | 中止当前提交（不排队，不会被 labs_wait 堵住） |
@@ -28,8 +28,8 @@ hard limits 全部通过配置接入。
   真机用 labscript-suite 环境。详见 docs/INSTALL.zh.md。
 - 默认 backend 是 mock-labscript；真机必须显式配置 backend: labscript。
 - 安装后在 DSH 插件页填 backend、python、runmanager host/port、protocolsDir；
-  其余高级配置（scriptsFile、hard limits、postprocess、approval、pathMap）写在
-  profile 的 cordis.patch.yml。
+  其余高级配置（scriptsFile、hard limits、postprocess、approval、pathMap、
+  engage 的超时/早退参数）写在 profile 的 cordis.patch.yml。
 
 ## 协议 YAML
 
@@ -52,6 +52,14 @@ postprocess 是后处理：跑完后调用配置的 Python 函数（entry + pyth
 - 真机 engage / protocol run 会请求 DSH 审批（一次），abort 和面板写入不请求；
   插件不是硬件联锁。
 - labscript abort 只停止编译/提交新的 shot，已进 BLACS 队列或正在跑的 shot 不会被停。
-- 完成判定基于 HDF5 数量、大小稳定和最后一发可读；BLACS paused 会报
-  compiled_not_executed 诊断。
+- 完成判定基于 HDF5 数量、大小稳定和最后一发可读。shot 的 h5 文件在编译阶段就写了出来，
+  所以 labs_status 分别报 h5（文件数）和 executed（真正采到数据的枪数），完成要求最后一发
+  有采集数据；BLACS paused 会报 compiled_not_executed 诊断。
+- 编译 abort 时 labs_wait 不再等满 timeout：距今没有任何新 shot 超过 engage.noShotS
+  （默认 25 s）就返回 no_shots / compile_stalled 诊断，去 runmanager 看编译错误；如果只是
+  编译慢，再调一次 labs_wait 继续等。
+- 一条一枪都没产生的 pending 提交，超过 engage.staleS（默认 180 s）会被下一次 labs_engage
+  丢弃并报 dropped_stale_pending，不再永久挡住提交；RUN_PENDING 的错误里也会写清年龄和两种出路。
+- 配了 pathMap 时 pending 记录映射后的目录（原始路径保留为 remote_run_folder），所以
+  labs_wait 传映射后目录或原始目录都能对上，不会再说"run folder does not exist"。
 

package/lib/skill.js

@@ -21,9 +21,13 @@ export const SKILL_CONTENT = [
   '## Running',
   '- labs_set writes the panel. It does not run anything. Always pass note: one sentence',
   '  saying what this batch is for (it is written to the event log).',
+  '- labs_engage only reports that runmanager accepted the submission; it cannot see',
+  '  compile errors. If nothing runs, the next labs_wait says so (see Stop rules).',
   '- labs_engage submits a run and returns immediately with n_shots_at_submit and',
   '  expected_n_shots. Do not poll in a loop; when the user wants results, call labs_wait.',
   '  After submitting you may end the turn or do independent work.',
+  '- n_h5 counts shot h5 files, which labscript writes before compiling each shot; only',
+  '  n_executed (or the diagnosis) says whether shots actually acquired data.',
   '- labs_protocol run submits a fixed protocol; labs_wait finalizes it (postprocess,',
   '  finish write-back, panel restore).',
   '- backend defaults to mock-labscript; real hardware is only used when the config says',
@@ -43,8 +47,15 @@ export const SKILL_CONTENT = [
   '',
   '## Stop rules',
   '- error_in_globals true: fix or ask; never engage.',
-  '- wait timed out or diagnosis=compiled_not_executed: the shots may be compiled but BLACS',
-  '  may be paused/not executing. Check BLACS and the panel, then abort or ask; do not resubmit blindly.',
+  '- labs_wait returns before the timeout with diagnosis no_shots or compile_stalled when no',
+  '  shot appeared for config.engage.noShotS seconds (default 25). That usually means runmanager',
+  '  aborted the compilation: read the runmanager output, then labs_abort, or labs_wait again if',
+  '  you expect a slow compile. A submission that never produced a shot is dropped by the next',
+  '  labs_engage once it is config.engage.staleS seconds old (default 180) and reported as',
+  '  dropped_stale_pending.',
+  '- diagnosis=compiled_not_executed, or timed_out with h5 files but no acquisition data: the',
+  '  shots were compiled but BLACS may be paused/not executing. Check BLACS and the panel, then',
+  '  abort or ask; do not resubmit blindly.',
   '- A run is already pending: wait or abort before submitting another.',
 ].join(String.fromCharCode(10))
 

package/lib/tools.js

@@ -30,6 +30,9 @@ function progressLine(run) {
   if (run.kind) parts.push('kind=' + run.kind)
   if (run.n_h5 !== undefined) parts.push('h5=' + run.n_h5 + '/' + (run.expected_n_shots || '?'))
   else if (run.expected_n_shots) parts.push('expected=' + run.expected_n_shots)
+  // h5 counts files; labscript writes each file before compiling it, so a compiled
+  // but not yet executed run shows h5>0 until the first acquisition data appears.
+  if (run.n_executed !== undefined) parts.push('executed=' + run.n_executed)
   if (run.complete !== undefined) parts.push('complete=' + run.complete)
   if (run.elapsed_s !== undefined) parts.push('elapsed=' + run.elapsed_s + 's')
   return parts.join('  ')
@@ -82,6 +85,24 @@ function resultsLine(value) {
   return 'results: ' + names.map((group) => group + ' (' + layout[group].length + ' fields)').join(', ')
 }
 
+// Protocol keys that the loaded labscript does not define, with the closest names it
+// does define: a renamed panel global is the most common reason a protocol is stale.
+function nameCheckDetail(check) {
+  const missing = check && Array.isArray(check.missing) ? check.missing : []
+  return missing.map((key) => {
+    const suggestions = (check.suggestions || {})[key] || []
+    return key + (suggestions.length ? ' (closest: ' + suggestions.join(', ') + ')' : '')
+  }).join(', ')
+}
+
+function nameCheckLine(value) {
+  const check = value.name_check
+  if (!check) return ''
+  if (check.checked === false) return 'script key check: not checked (' + (check.reason || 'no script') + ')'
+  const detail = nameCheckDetail(check)
+  return detail ? 'script key check: MISSING ' + detail : ''
+}
+
 function dataGlobalsLines(args, value) {
   const globals = value.globals
   if (!globals) return []
@@ -166,23 +187,57 @@ async function requireApproval(runtime, exec, toolName, reason, approvalKey) {
   }
 }
 
+// The pending run may be stored under either path form: labs_engage returns the
+// mapped folder, the bridge also reports the raw runmanager folder, and an agent may
+// quote either one back.
+function pendingFolders(pending) {
+  if (!pending) return []
+  return [pending.run_folder, pending.mapped_run_folder, pending.remote_run_folder].filter((folder) => Boolean(folder))
+}
+
+function isPendingRun(pending, folder) {
+  return pendingFolders(pending).includes(folder)
+}
+
+// Returns the dropped run description when a dead submission was cleared.
 async function clearStalePending(runtime, exec) {
   const pending = runtime.state.pending
-  if (!pending) return
+  if (!pending) return null
   if (pending.run_folder) {
+    let progress = null
     try {
-      const progress = await runtime.runBridge('progress', {
+      progress = await runtime.runBridge('progress', {
         run_folder: pending.run_folder,
         expected_n_shots: pending.expected_n_shots,
       }, { signal: exec.signal })
-      if (!progress.complete) {
-        throw new LabError('RUN_PENDING', 'a run is still in progress (' + progress.n_h5 + '/' + progress.expected_n_shots + ' shots): call labs_wait first, or labs_abort')
+    } catch {
+      progress = null
+    }
+    if (progress && progress.complete !== true) {
+      const ageS = pending.submitted_at ? Math.round((Date.now() - Date.parse(pending.submitted_at)) / 1000) : 0
+      const staleS = Number((runtime.config.engage || {}).staleS || 180)
+      const nH5 = Number(progress.n_h5 || 0)
+      if (progress.folder_exists && nH5 === 0 && ageS >= staleS) {
+        // Nothing was ever written for this submission and it is minutes old: the
+        // compilation aborted, so the run folder stays empty forever. Drop it and
+        // report the drop instead of refusing to submit again.
+        const dropped = {
+          run_folder: progress.mapped_run_folder || pending.run_folder,
+          age_s: ageS,
+          expected_n_shots: pending.expected_n_shots,
+        }
+        runtime.clearPending()
+        return dropped
       }
-    } catch (error) {
-      if (error instanceof LabError && error.code === 'RUN_PENDING') throw error
+      const next = nH5 === 0
+        ? ' A submission with no shots is dropped automatically after ' + staleS + ' s, or call labs_abort now.'
+        : ' Call labs_wait to inspect it, or labs_abort to abandon it.'
+      throw new LabError('RUN_PENDING', 'a run is still in progress (' + nH5 + '/' + (pending.expected_n_shots || '?') +
+        ' shots, ' + ageS + ' s old):' + next)
     }
   }
   runtime.clearPending()
+  return null
 }
 
 function mockApprovalSkipped(runtime) {
@@ -227,13 +282,13 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_set',
-    description: 'Write runmanager panel globals. values are JSON scalars (bool/number/string); pass raw=true to write expression strings, or reps=N to write the repetition counter (np.arange(N)). A full panel snapshot is taken before the write. Does not run anything.',
+    description: 'Write runmanager panel globals. One call can do the whole step: values are JSON scalars (bool/number/string); with raw=true a string value is written as an expression (e.g. np.arange(0.5, 6.7, 0.2)) while bool/number values are still converted normally (true -> True, 2.72 -> 2.72), so scalars, an expression scan axis and reps=N can be combined. reps=N writes the configured repetition global as np.arange(N). A full panel snapshot is taken before the write. Does not run anything.',
     parameters: {
       type: 'object',
       properties: {
         values: { type: 'object', additionalProperties: true, description: 'Panel globals to write: key -> scalar' },
-        raw: { type: 'boolean', description: 'values are raw expression strings (e.g. np.arange(0, 8, 0.3))' },
-        reps: { type: 'integer', minimum: 1, description: 'Write the configured repetition global as np.arange(reps)' },
+        raw: { type: 'boolean', description: 'String values are written as expressions; bool/number values are still converted (mix them freely with reps=N)' },
+        reps: { type: 'integer', minimum: 1, description: 'Also write the configured repetition global as np.arange(reps)' },
         note: { type: 'string', description: 'One sentence: what this batch is for (written to the event log)' },
       },
       required: ['values'],
@@ -257,7 +312,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_engage',
-    description: 'Submit the current panel for a run (compiles and sends to BLACS). Returns immediately with run_folder, n_shots_at_submit and expected_n_shots; use labs_wait later to wait for completion. Refuses when error_in_globals is true or another run is still pending.',
+    description: 'Submit the current panel for a run (compiles and sends to BLACS). Returns immediately with run_folder, n_shots_at_submit and expected_n_shots; use labs_wait later to wait for completion. Refuses when error_in_globals is true or another run is still pending; a pending run that produced no shot at all for config.engage.staleS (default 180 s) is dropped and reported as dropped_stale_pending.',
     parameters: {
       type: 'object',
       properties: {
@@ -267,29 +322,33 @@ export function registerTools(ctx, runtime) {
     },
     isConcurrencySafe: () => false,
     output: looseOutput((_args, value) => text(lines([
+      value.dropped_stale_pending ? 'dropped stale pending run: ' + value.dropped_stale_pending.run_folder + ' (' + value.dropped_stale_pending.age_s + ' s old, no shots ever appeared)' : '',
       'submitted run: ' + (value.mapped_run_folder || value.run_folder || '<run folder pending>'),
       'n_shots_at_submit: ' + value.n_shots_at_submit + '  expected: ' + value.expected_n_shots,
       'call labs_wait when results are needed.',
     ]))),
     async execute(args, exec) {
       return runtime.serial(async () => {
-        await clearStalePending(runtime, exec)
+        const dropped = await clearStalePending(runtime, exec)
         const before = await runtime.runBridge('status', {}, { signal: exec.signal })
         await requireApproval(runtime, exec, 'labs_engage', 'Submit a labscript run: ' + (before.labscript || '<script>') + ', n_shots=' + before.n_shots + ', backend=' + before.backend, 'engage')
         const data = await runtime.runBridge('engage', { note: args.note }, { signal: exec.signal, timeoutMs: 180000 })
         runtime.setLastRun({
-          run_folder: data.run_folder,
+          run_folder: data.mapped_run_folder || data.run_folder,
+          remote_run_folder: data.run_folder,
           expected_n_shots: data.expected_n_shots,
           submitted_at: data.submitted_at,
         })
         runtime.setPending({
           kind: 'engage',
-          run_folder: data.run_folder,
+          run_folder: data.mapped_run_folder || data.run_folder,
+          remote_run_folder: data.run_folder,
           expected_n_shots: data.expected_n_shots,
           submitted_at: data.submitted_at,
           note: args.note,
           snapshot: null,
         })
+        if (dropped) data.dropped_stale_pending = dropped
         return data
       })
     },
@@ -297,7 +356,7 @@ export function registerTools(ctx, runtime) {
 
   ctx.tools.register({
     name: 'labs_wait',
-    description: 'Wait for the pending run to finish (blocking, timeout in seconds). If the pending run came from labs_protocol run, this call also finalizes it: postprocess, finish write-back of panel and calibration history, and panel restore. Returns expected vs actual shots and, for protocols, the postprocess/finish/restore results. A run folder that does not exist and is not the pending run of this session fails immediately instead of waiting.',
+    description: 'Wait for the pending run to finish (blocking, timeout in seconds). If the pending run came from labs_protocol run, this call also finalizes it: postprocess, finish write-back of panel and calibration history, and panel restore. Returns expected vs actual shots and, for protocols, the postprocess/finish/restore results. Returns early with diagnosis no_shots or compile_stalled when no shot h5 appears for config.engage.noShotS seconds (default 25): the compilation probably aborted, so check runmanager and call labs_wait again only if you expect a slow compile. A run folder that does not exist and is not the pending run of this session fails immediately instead of waiting.',
     parameters: {
       type: 'object',
       properties: {
@@ -312,7 +371,7 @@ export function registerTools(ctx, runtime) {
       'run: ' + (value.mapped_run_folder || value.run_folder),
       value.aborted ? 'wait cancelled by labs_abort (run aborted)' : '',
       value.folder_exists === false ? 'folder: missing' : '',
-      'shots: ' + value.n_h5 + '/' + (value.expected_n_shots || '?') + '  complete: ' + value.complete + (value.timed_out ? '  TIMED OUT' : '') + (value.diagnosis ? '  diagnosis: ' + value.diagnosis : ''),
+      'shots: ' + value.n_h5 + '/' + (value.expected_n_shots || '?') + ' h5 files' + (value.n_executed !== undefined ? ', ' + value.n_executed + ' with acquisition data' : '') + '  complete: ' + value.complete + (value.timed_out ? '  TIMED OUT' : '') + (value.diagnosis ? '  diagnosis: ' + value.diagnosis : ''),
       value.hint ? 'hint: ' + value.hint : '',
       value.finalize && value.finalize.postprocess ? 'postprocess: ' + JSON.stringify(value.finalize.postprocess).slice(0, 400) : '',
       value.finalize && value.finalize.finish ? 'finish: ' + JSON.stringify(value.finalize.finish).slice(0, 300) : '',
@@ -323,8 +382,9 @@ export function registerTools(ctx, runtime) {
         const pending = runtime.state.pending
         const runFolder = args.run_folder || (pending ? pending.run_folder : '')
         if (!runFolder) throw new LabError('NO_PENDING_RUN', 'no run folder: submit with labs_engage or labs_protocol run first, or pass run_folder')
-        const expected = pending && pending.run_folder === runFolder ? pending.expected_n_shots : undefined
-        if (!(pending && pending.run_folder === runFolder)) {
+        const pendingRun = isPendingRun(pending, runFolder)
+        const expected = pendingRun ? pending.expected_n_shots : undefined
+        if (!pendingRun) {
           let probe = null
           try {
             probe = await runtime.runBridge('progress', {
@@ -368,6 +428,7 @@ export function registerTools(ctx, runtime) {
           run_folder: waited.run_folder,
           expected_n_shots: waited.expected_n_shots,
           n_h5: waited.n_h5,
+          n_executed: waited.n_executed,
           complete: waited.complete,
           timed_out: waited.timed_out,
           elapsed: waited.elapsed,
@@ -378,13 +439,13 @@ export function registerTools(ctx, runtime) {
           folder_exists: waited.folder_exists,
         }
         for (const key of Object.keys(out)) if (out[key] === undefined) delete out[key]
-        if (waited.complete && runtime.state.lastRun && runtime.state.lastRun.run_folder === runFolder) {
+        if (waited.complete && runtime.state.lastRun && isPendingRun(runtime.state.lastRun, runFolder)) {
           runtime.setLastRun(Object.assign({}, runtime.state.lastRun, {
             completed_at: new Date().toISOString(),
             n_h5: waited.n_h5,
           }))
         }
-        if (waited.complete && pending && pending.kind === 'protocol' && pending.run_folder === runFolder) {
+        if (waited.complete && pending && pending.kind === 'protocol' && pendingRun) {
           const finalize = await runtime.runBridge('protocol_finalize', {
             protocol: pending.plan,
             run_folder: runFolder,
@@ -393,7 +454,7 @@ export function registerTools(ctx, runtime) {
           }, { signal: exec.signal, timeoutMs: 180000 })
           out.finalize = finalize
           runtime.clearPending()
-        } else if (waited.complete && pending && pending.run_folder === runFolder) {
+        } else if (waited.complete && pending && pendingRun) {
           runtime.clearPending()
         }
         return out
@@ -479,8 +540,8 @@ export function registerTools(ctx, runtime) {
       value.plan && value.plan.scan ? 'scan: ' + Object.keys(value.plan.scan).join(', ') : '',
       value.hard_limits ? 'hard limits: ' + JSON.stringify(value.hard_limits).slice(0, 240) : '',
       value.warnings && value.warnings.length ? 'warnings: ' + value.warnings.join('; ') : '',
-      value.name_check && value.name_check.checked && value.name_check.missing && value.name_check.missing.length ? 'script key check: MISSING ' + value.name_check.missing.join(', ') : '',
-      value.name_check && value.name_check.checked === false ? 'script key check: not checked (' + (value.name_check.reason || 'no script') + ')' : '',
+      nameCheckLine(value),
+      value.dropped_stale_pending ? 'dropped stale pending run: ' + value.dropped_stale_pending.run_folder + ' (' + value.dropped_stale_pending.age_s + ' s old, no shots ever appeared)' : '',
       value.engage ? 'submitted: ' + (value.engage.mapped_run_folder || value.engage.run_folder || '<run folder pending>') + '  n_shots_at_submit: ' + value.n_shots_at_submit : '',
       value.engage ? 'call labs_wait to finalize this protocol.' : '',
     ]))),
@@ -496,38 +557,39 @@ export function registerTools(ctx, runtime) {
         return { name: loaded.name, file: loaded.file, warnings: loaded.warnings.concat(planned.warnings || []), plan: planned.plan, hard_limits: planned.hard_limits, name_check: planned.name_check }
       }
       return runtime.serial(async () => {
-        await clearStalePending(runtime, exec)
+        const dropped = await clearStalePending(runtime, exec)
         const planned = await runtime.runBridge('protocol_plan', { protocol: loaded.plan }, { signal: exec.signal })
         if (planned.hard_limits && planned.hard_limits.checked) {
           // plan command already fails on violations; nothing to do
         }
         const nameCheck = planned.name_check
-        if (nameCheck && nameCheck.checked && nameCheck.missing && nameCheck.missing.length) {
-          const detail = nameCheck.missing.map((key) => {
-            const suggestions = (nameCheck.suggestions || {})[key] || []
-            return key + (suggestions.length ? ' (closest: ' + suggestions.join(', ') + ')' : '')
-          }).join(', ')
-          await requireApproval(runtime, exec, 'labs_protocol', 'Protocol ' + loaded.name + ' uses key(s) that are not in the currently loaded labscript: ' + detail + '. Continue anyway?', 'protocolNames')
+        const missingDetail = nameCheckDetail(nameCheck)
+        if (missingDetail) {
+          await requireApproval(runtime, exec, 'labs_protocol', 'Protocol ' + loaded.name + ' uses key(s) that are not in the currently loaded labscript: ' + missingDetail + '. Continue anyway?', 'protocolNames')
         }
         await requireApproval(runtime, exec, 'labs_protocol', 'Run protocol ' + loaded.name + ': expected ' + planned.plan.expected_n_shots + ' shots, labscript ' + (loaded.plan.labscript || loaded.plan.joint_script) + ', backend ' + runtime.config.backend, 'protocolRun')
         const started = await runtime.runBridge('protocol_start', { protocol: loaded.plan, note: args.note }, { signal: exec.signal, timeoutMs: 180000 })
         if (started.engage) {
           runtime.setLastRun({
-            run_folder: started.engage.run_folder,
+            run_folder: started.engage.mapped_run_folder || started.engage.run_folder,
+            remote_run_folder: started.engage.run_folder,
             expected_n_shots: started.expected_n_shots,
             submitted_at: started.submitted_at,
           })
           runtime.setPending({
             kind: 'protocol',
             plan: loaded.plan,
-            run_folder: started.engage.run_folder,
+            run_folder: started.engage.mapped_run_folder || started.engage.run_folder,
+            remote_run_folder: started.engage.run_folder,
             expected_n_shots: started.expected_n_shots,
             snapshot: started.snapshot,
             submitted_at: started.submitted_at,
             note: args.note,
           })
         }
-        return Object.assign({ name: loaded.name, file: loaded.file, warnings: loaded.warnings.concat(planned.warnings || []), name_check: nameCheck }, started)
+        const out = Object.assign({ name: loaded.name, file: loaded.file, warnings: loaded.warnings.concat(planned.warnings || []), name_check: nameCheck }, started)
+        if (dropped) out.dropped_stale_pending = dropped
+        return out
       })
     },
   })

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-labscript",
-  "version": "0.2.7",
+  "version": "0.2.8",
   "description": "Control labscript-suite experiments from DeepSeek Harness: runmanager globals and scans, submits and waits, YAML fixed-operation protocols with postprocess write-back, panel snapshots, a built-in mock backend and a read-only doctor.",
   "type": "module",
   "main": "lib/index.js",

package/python/bridge.py

@@ -114,6 +114,23 @@ def shot_executed(path):
     except Exception:
         return None
 
+
+def executed_prefix(shots):
+    """How many shot h5 files already hold acquisition data (binary search).
+
+    labscript compiles and submits shots in file order, so executed shots are a prefix
+    of the sorted file list. Opening a handful of files keeps this cheap even when the
+    run folder is a network share.
+    """
+    low, high = 0, len(shots)
+    while low < high:
+        middle = (low + high) // 2
+        if shot_executed(shots[middle]) is True:
+            low = middle + 1
+        else:
+            high = middle
+    return low
+
 # ---------------------------------------------------------------- scan expr
 
 _ALLOWED_CALLS = {"arange", "linspace", "array", "logspace"}
@@ -302,7 +319,11 @@ class LabscriptBackend(object):
         return dict(self.client().get_globals(raw=raw))
 
     def set_globals(self, values, raw=False):
-        return dict(self.client().set_globals(values, raw=raw))
+        # runmanager's set_globals is a side-effect RPC: its handler returns None, so
+        # never feed the reply to dict(). Report what was requested; the caller reads
+        # the panel back to verify.
+        written = self.client().set_globals(values, raw=raw)
+        return dict(written) if isinstance(written, dict) else dict(values)
 
     def n_shots(self):
         return int(self.client().n_shots())
@@ -988,21 +1009,25 @@ def cmd_set(config, args):
     snap = snapshot_panel(backend, config, "set", {"keys": sorted(values.keys()), "raw": raw, "note": args.get("note")}, touched=values.keys())
     try:
         written = backend.set_globals(values, raw=raw)
-    except BridgeError as exc:
-        detail = dict(exc.detail or {})
-        detail["snapshot"] = snap
-        detail["hint"] = "set_globals failed after the panel snapshot; the panel may be partially written. Use labs_restore to undo."
-        raise BridgeError(exc.code, exc.message, detail)
     except Exception as exc:
-        raise BridgeError("SET_FAILED", "set_globals failed: " + type(exc).__name__ + ": " + str(exc), {
-            "snapshot": snap,
-            "hint": "the panel may be partially written; use labs_restore to undo",
-        })
+        detail = dict(getattr(exc, "detail", None) or {})
+        detail["snapshot"] = snap
+        # A failed write can still have landed (runmanager writes each key in turn),
+        # so report what the panel holds now instead of guessing in the hint.
+        try:
+            now = backend.get_globals(raw=True)
+            detail["landed"] = dict((k, jsonable(now.get(k))) for k in values)
+        except Exception:
+            pass
+        detail["hint"] = "the write may have landed partially or not at all: read the keys back with labs_status before retrying, and only labs_restore if they are wrong"
+        raise BridgeError(getattr(exc, "code", None) or "SET_FAILED", "set_globals failed: " + type(exc).__name__ + ": " + str(exc), detail)
     readback = backend.get_globals(raw=False)
     raw_readback = backend.get_globals(raw=True)
     missing = [k for k in values if k not in readback]
     out = {
-        "written": jsonable(written),
+        # Report what was asked for even if a backend returns something else (runmanager's
+        # RPC returns nothing); the readback below carries the truth from the panel.
+        "written": jsonable(written) if isinstance(written, dict) else dict((k, jsonable(v)) for k, v in values.items()),
         "readback": dict((k, jsonable(readback.get(k))) for k in values),
         "raw_readback": dict((k, jsonable(raw_readback.get(k))) for k in values),
         "missing": missing,
@@ -1050,24 +1075,31 @@ def wait_for_run(config, run_folder, expected, timeout_s):
     poll = float(engage_cfg.get("pollS") or 1.0)
     settle = float(engage_cfg.get("settleS") or 2.0)
     stall = float(engage_cfg.get("stallS") or 20.0)
+    # labscript writes each shot h5 before compiling it, so a healthy compilation keeps
+    # adding files. When nothing has appeared for this long the compile almost certainly
+    # aborted; returning a diagnosis beats blocking for the full timeout_s.
+    no_shot = float(engage_cfg.get("noShotS") or 25.0)
+    want = int(expected or 0)
     start = time.time()
+    last_progress = start
     stable_since = None
     compiled_since = None
     last_count = -1
     last_size = -1
-    last_executed = None
+    shots = []
 
-    def result(complete, timed_out, diagnosis, count, executed, elapsed, hint=None):
+    def result(complete, timed_out, diagnosis, elapsed, hint=None):
         payload = {
             "run_folder": run_folder,
             "mapped_run_folder": folder,
             "folder_exists": os.path.isdir(folder),
             "expected_n_shots": expected,
-            "n_h5": count,
+            "n_h5": len(shots),
+            "n_executed": executed_prefix(shots),
             "complete": complete,
             "timed_out": timed_out,
             "diagnosis": diagnosis,
-            "executed": executed,
+            "executed": shot_executed(shots[-1]) if shots else None,
             "elapsed": round(elapsed, 1),
         }
         if hint:
@@ -1078,8 +1110,10 @@ def wait_for_run(config, run_folder, expected, timeout_s):
         shots = sorted(glob.glob(os.path.join(folder, "**", "*.h5"), recursive=True))
         count = len(shots)
         total = sum(os.path.getsize(p) for p in shots) if shots else 0
+        if count != last_count or total != last_size:
+            last_progress = time.time()
         ready = False
-        if count >= (expected or 0) and count == last_count and total == last_size and shots:
+        if count >= (want or 0) and count == last_count and total == last_size and shots:
             ready = shot_ready(shots[-1])
         executed = None
         if ready:
@@ -1089,7 +1123,7 @@ def wait_for_run(config, run_folder, expected, timeout_s):
             if stable_since is None:
                 stable_since = time.time()
             elif time.time() - stable_since >= settle:
-                return result(True, False, "complete", count, True, time.time() - start)
+                return result(True, False, "complete", time.time() - start)
         else:
             stable_since = None
             if ready and executed is False:
@@ -1097,19 +1131,31 @@ def wait_for_run(config, run_folder, expected, timeout_s):
                     compiled_since = time.time()
                 elif time.time() - compiled_since >= stall:
                     return result(
-                        False, False, "compiled_not_executed", count, False, time.time() - start,
+                        False, False, "compiled_not_executed", time.time() - start,
                         "shots were compiled but show no acquisition data after " + str(int(stall)) +
                         " s: BLACS may be paused / the queue may not be running, or the run is still being programmed. Check BLACS and the panel, then labs_abort if needed.")
             else:
                 compiled_since = None
         last_count = count
         last_size = total
-        last_executed = executed
+        quiet = time.time() - last_progress
+        if want and count < want and quiet >= no_shot:
+            if count == 0:
+                diagnosis = "no_shots"
+                hint = ("no shot h5 appeared in " + str(int(no_shot)) + " s: the compilation probably aborted "
+                        "(runmanager shows the error) or BLACS is not running. Check runmanager, call labs_wait "
+                        "again to keep waiting, or labs_abort.")
+            else:
+                diagnosis = "compile_stalled"
+                hint = ("compilation stopped at " + str(count) + " of " + str(want) + " shot h5 files and "
+                        "nothing changed for " + str(int(no_shot)) + " s: it probably aborted (runmanager shows "
+                        "the error). Check runmanager, call labs_wait again to keep waiting, or labs_abort.")
+            return result(False, False, diagnosis, time.time() - start, hint)
         if time.time() - start >= timeout:
             if count == 0:
                 diagnosis = "no_shots"
                 hint = "no shot h5 appeared: check that the run was accepted, that BLACS is running, and that pathMap points at the run folder."
-            elif count < (expected or 0):
+            elif count < want:
                 diagnosis = "compiling"
                 hint = "only " + str(count) + " of " + str(expected) + " shots appeared before the timeout; the run may still be compiling or blocked."
             elif executed is False:
@@ -1118,7 +1164,7 @@ def wait_for_run(config, run_folder, expected, timeout_s):
             else:
                 diagnosis = "partial"
                 hint = "shots appeared but the run was not confirmed complete before the timeout."
-            return result(False, True, diagnosis, count, executed, time.time() - start, hint)
+            return result(False, True, diagnosis, time.time() - start, hint)
         time.sleep(poll)
 
 
@@ -1146,7 +1192,20 @@ def cmd_progress(config, args):
     expected = int(args.get("expected_n_shots") or 0)
     folder = map_path(config, run_folder)
     shots = sorted(glob.glob(os.path.join(folder, "**", "*.h5"), recursive=True)) if folder else []
-    return {"run_folder": run_folder, "mapped_run_folder": folder, "folder_exists": bool(folder) and os.path.isdir(folder), "n_h5": len(shots), "expected_n_shots": expected, "complete": bool(expected and len(shots) >= expected)}
+    # n_h5 counts files, which labscript writes before compiling each shot; n_executed
+    # counts the shots that actually acquired data. A run is only complete when the
+    # last shot has acquisition data, not merely when the file count matches.
+    executed = executed_prefix(shots)
+    complete = bool(expected and len(shots) >= expected and executed == len(shots))
+    return {
+        "run_folder": run_folder,
+        "mapped_run_folder": folder,
+        "folder_exists": bool(folder) and os.path.isdir(folder),
+        "n_h5": len(shots),
+        "n_executed": executed,
+        "expected_n_shots": expected,
+        "complete": complete,
+    }
 
 
 

package/python/tests/run_tests.py

@@ -8,6 +8,7 @@ import os
 import shutil
 import sys
 import tempfile
+import time
 
 HERE = os.path.dirname(os.path.abspath(__file__))
 sys.path.insert(0, os.path.dirname(HERE))
@@ -277,6 +278,159 @@ def test_wait_diagnosis_compiled_not_executed():
         shutil.rmtree(tmp, ignore_errors=True)
 
 
+def write_shot(path, executed):
+    """Write a shot h5 like labscript does: globals first, acquisition data later."""
+    import h5py
+    with h5py.File(path, "w") as handle:
+        handle.create_group("globals")
+        if executed:
+            label = handle.create_group("images/camera/label")
+            label.create_dataset("frame", data=[1.0, 2.0])
+
+
+def test_wait_reports_stalled_compilation_early():
+    """A compile that dies after one shot h5 must not block for the whole timeout."""
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-stalled-")
+    try:
+        cfg = make_config(tmp)
+        cfg["engage"] = {"timeoutS": 10, "pollS": 0.05, "settleS": 0.1, "noShotS": 0.3}
+        run_dir = os.path.join(tmp, "stalled_run")
+        os.makedirs(run_dir, exist_ok=True)
+        write_shot(os.path.join(run_dir, "shot_0000.h5"), executed=False)
+        started = time.time()
+        result = bridge.wait_for_run(cfg, run_dir, 64, 10)
+        elapsed = time.time() - started
+        assert result["diagnosis"] == "compile_stalled", result
+        assert result["timed_out"] is False, result
+        assert result["n_h5"] == 1, result
+        assert result["n_executed"] == 0, result
+        assert elapsed < 3.0, elapsed
+        assert "aborted" in result["hint"], result
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_wait_reports_no_shots_early():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-noshots-")
+    try:
+        cfg = make_config(tmp)
+        cfg["engage"] = {"timeoutS": 10, "pollS": 0.05, "settleS": 0.1, "noShotS": 0.3}
+        run_dir = os.path.join(tmp, "empty_run")
+        os.makedirs(run_dir, exist_ok=True)
+        started = time.time()
+        result = bridge.wait_for_run(cfg, run_dir, 64, 10)
+        elapsed = time.time() - started
+        assert result["diagnosis"] == "no_shots", result
+        assert result["timed_out"] is False, result
+        assert result["n_h5"] == 0 and result["n_executed"] == 0, result
+        assert elapsed < 3.0, elapsed
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_progress_separates_files_from_executed_shots():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-progress-exec-")
+    try:
+        cfg = make_config(tmp)
+        compiled = os.path.join(tmp, "compiled_run")
+        os.makedirs(compiled, exist_ok=True)
+        write_shot(os.path.join(compiled, "shot_0000.h5"), executed=True)
+        write_shot(os.path.join(compiled, "shot_0001.h5"), executed=False)
+        write_shot(os.path.join(compiled, "shot_0002.h5"), executed=False)
+        out = bridge.cmd_progress(cfg, {"run_folder": compiled, "expected_n_shots": 3})
+        assert out["n_h5"] == 3, out
+        assert out["n_executed"] == 1, out
+        assert out["complete"] is False, out
+        done = os.path.join(tmp, "done_run")
+        os.makedirs(done, exist_ok=True)
+        for index in range(2):
+            write_shot(os.path.join(done, "shot_000%d.h5" % index), executed=True)
+        finished = bridge.cmd_progress(cfg, {"run_folder": done, "expected_n_shots": 2})
+        assert finished["complete"] is True, finished
+        assert finished["n_executed"] == 2, finished
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_real_backend_set_globals_survives_none_reply():
+    """runmanager's set_globals returns None; the write must still be reported."""
+    class FakeClient(object):
+        def __init__(self):
+            self.calls = []
+
+        def set_globals(self, values, raw=False):
+            self.calls.append((dict(values), raw))
+            return None
+
+    client = FakeClient()
+    backend = bridge.LabscriptBackend({})
+    backend._client = client
+    written = backend.set_globals({"do_acquire": True, "amplitude": 0.5})
+    assert written == {"do_acquire": True, "amplitude": 0.5}, written
+    assert client.calls == [({"do_acquire": True, "amplitude": 0.5}, False)], client.calls
+
+
+def test_cmd_set_accepts_backend_that_returns_none():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-set-none-")
+    try:
+        cfg = make_config(tmp)
+        original = bridge.make_backend
+
+        def make(config):
+            backend = bridge.MockBackend(config)
+
+            def set_globals(values, raw=False):
+                bridge.MockBackend.set_globals(backend, values, raw=raw)
+                return None
+
+            backend.set_globals = set_globals
+            return backend
+
+        bridge.make_backend = make
+        try:
+            out = bridge.cmd_set(cfg, {"values": {"amplitude": 0.71}, "note": "None-returning client"})
+        finally:
+            bridge.make_backend = original
+        assert out["missing"] == [], out
+        assert out["readback"]["amplitude"] == 0.71, out
+        assert out["written"] == {"amplitude": 0.71}, out
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def test_cmd_set_failure_reports_what_landed():
+    tmp = tempfile.mkdtemp(prefix="dsh-labscript-set-partial-")
+    try:
+        cfg = make_config(tmp)
+        original = bridge.make_backend
+
+        def make(config):
+            backend = bridge.MockBackend(config)
+
+            def set_globals(values, raw=False):
+                if "amplitude" in values:
+                    bridge.MockBackend.set_globals(backend, {"amplitude": values["amplitude"]}, raw=raw)
+                raise ValueError("Global nope not found in any active group")
+
+            backend.set_globals = set_globals
+            return backend
+
+        bridge.make_backend = make
+        try:
+            bridge.cmd_set(cfg, {"values": {"amplitude": 0.71, "nope": 1}})
+        except bridge.BridgeError as exc:
+            assert exc.code == "SET_FAILED", exc.code
+            assert "not found" in exc.message, exc.message
+            assert exc.detail["snapshot"]["name"].endswith(".json"), exc.detail
+            assert exc.detail["landed"]["amplitude"] == "0.71", exc.detail
+            assert "labs_status" in exc.detail["hint"], exc.detail
+        else:
+            raise AssertionError("expected SET_FAILED")
+    finally:
+        bridge.make_backend = original
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
 def test_doctor_mock():
     tmp = tempfile.mkdtemp(prefix="dsh-labscript-doctor-")
     try: