@sema-agent/server 7.129.0 → 7.129.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/MIGRATION.md CHANGED
@@ -7,6 +7,28 @@
7
7
  > ⚠️ 完整清单在仓库根 `CHANGELOG.md`——它**不随 npm tarball 出包**(本文件随包)。看完整迁移窗的
8
8
  > 权威姿势是源码 tag diff:`git diff v<旧>..v<新>`(每版都推 `v<版本>` tag);npm 包页也镜像 CHANGELOG。
9
9
 
10
+ ## 7.129.1 —— k8s / e2b 车道:带超时的命令不再被静默切断(行为修复)· 新旋钮 `K8S_LIVENESS_MS` · 两只活性旋钮非正值拒启
11
+
12
+ 1. **一条命令的界只来自调用方说出口的那些**(零 DDL):墙钟 `timeout`、流式面自己的 `readTimeoutMs`;两样都没说时才由车道活性界(无输出满 N 毫秒即切,
13
+ 缺省 120 s)兜底。界从调用开始算、覆盖全程(含 e2b 建命令那一段;那一段另受 e2b SDK 自己的建命令握手约束,缺省 60 s,比命令的界早时先由它切)。
14
+ 引擎的 Bash 工具每一发都带超时,所以 Bash 调用从此不再被活性界切断(安静的长构建跑到它的超时)。`timedOut` / `timeoutSec` 仍不全是跑满:e2b 上建命令
15
+ 卡住被那只 60 s 握手切掉、或沙箱中途死掉时,这一发仍以 `timeout` 码回来,Bash 照旧报「Command timed out after <N>s」(独立的码属于引擎的码表)。
16
+ 活性切断的句子三车道一句,点名旋钮;错误码仍是引擎的 `timeout`。
17
+ - **新旋钮** `K8S_LIVENESS_MS`(ms,`E2B_LIVENESS_MS` 的孪生:同一只解析器;缺席 ⇒ 120 s;只在 `REMOTE_EXEC=k8s` 时求值)。两只旋钮都**只管没有显式超时的
18
+ 命令** —— 此前为了让带超时的长构建跑完而调大 `E2B_LIVENESS_MS` 的部署,可以调回缺省(调大只会放宽引擎探针与文件读写的挂死护栏)。
19
+ - 🔴 **两只活性旋钮的值必须是正数**:非数、`0`、负数 ⇒ **拒启**,拒句点名旋钮(`E2B_LIVENESS_MS=0` 此前照收,结果是每条 e2b 命令都建不起来)。
20
+ 升级前删掉这类设置或改成正数。嵌入方直接构造 k8s / e2b 执行环境时传非正的 `livenessMs` ⇒ 构造即抛。
21
+ - **受伤方**:依赖「120 s 无输出即杀」兜底带超时调用的部署 —— 带超时的挂死命令要等到它自己的超时(Bash 不传参数时 120 s,与改前同;显式最长到引擎上限)。
22
+ 出路:给命令更短的超时;不带超时的调用照旧受活性界,可调小两只旋钮。
23
+ - **受伤方(调大过引擎的 `BASH_DEFAULT_TIMEOUT_MS` 的部署)**:每一发不传超时参数的 Bash 都带着这个调大了的超时,从此失去 120 s 的挂死护栏 —— 挂死的命令
24
+ 要等满它,且没有本服务的旋钮能补回。出路:把 `BASH_DEFAULT_TIMEOUT_MS` 调回,或给需要护栏的命令显式的短超时。
25
+ - **读面**:e2b 车道的墙钟切断、活性切断与建命令期限到点此前报 `unknown`,现在报 `timeout`(e2b 上把这类切断当 `unknown` 读的消费方改读 `timeout`);
26
+ k8s 流式 exec 此前忽略 `timeout`,现在照它切。
27
+ - **e2b 建命令被切之后才到的句柄**:一到即被杀、输出丢弃;在**看见**它不在跑之前(供应商答了杀,或 `wait()` 以成功 / 非零退出结算 —— 流先断了、连接期限到点不算看见,进程哪怕已死也继续持有)该环境的挂起(`suspendVM`)照旧以
28
+ `command_in_flight` 拒。杀被供应商拒时记一行 warn `e2b_abandoned_start_kill_failed`(沙箱 id / 进程号 / 拒因类名),继续等看见退出;杀被拒且退出看不见的环境从此不能挂起、VM 计费到任务结束(fail-closed),出路 = 让命令退出或结束任务。建命令一直悬着(供应商既不交回
29
+ 句柄、也不报失败)、或杀被拒而进程不退出的那一种,挂起会一直拒(说不出那条命令没在跑,不快照在跑的命令);任务结束销毁环境不受影响。建命令**以失败
30
+ 告终**(含 SDK 自己的 60 s 握手到点)时在途保护即撤,与 7.129.0 同 —— 供应商若其实已起了那个进程、只是回执丢了,它不受这道保护。
31
+
10
32
  ## 7.129.0 —— 引擎 7.51.2 提货 · 会话规则一种文法与列改名(🔴 先停旧副本再起新)· 卡词换与存量一次性改写 · 读拒三席热生效 · 直连不跟重定向 · 两只告警标签改名
11
33
 
12
34
  1. **引擎 7.51.2(精确钉;7.51.0 的硬破坏 + 7.51.1 / 7.51.2 两补丁)**:本服务随引擎的硬破坏在下列第 2–13 条;引擎自己的收窄直接落到部署上的见第 14 条。DDL 只有第 2 条的一处列改名(SQL 后端,由起服一次性迁移做,不用手跑);第 3 条的存量改写同样由起服一次性迁移做。
package/USAGE.md CHANGED
@@ -2315,6 +2315,31 @@ k8s 车道每条 run 一只新 pod(pod 寿命 = 泄漏护栏,不跨 run 复用)
2315
2315
 
2316
2316
  自检是**读数,不是边界**:起服之后加入的节点在下一次起服读到;射程里的三面(所在节点、宿主 DNAT、CNI 不执行)读不出,不在闭集里。
2317
2317
 
2318
+ ### 9.3f 沙箱车道的命令静默切断:显式超时优先(`K8S_LIVENESS_MS`,7.129.1 起)
2319
+
2320
+ k8s 与 e2b 两条沙箱车道各有一只**活性界**:一条命令无输出满这么久(缺省 120 s)即判挂死、切断。一条命令由它的调用方**说出口**的界约束,活性界只在调用方什么都没说时兜底:
2321
+
2322
+ - **带显式超时 ⇒ 只受这个超时约束**,无输出多久都不切。引擎的 Bash 工具每一发都带超时(`timeout` 参数;不传时引擎给缺省值 120 s,上限 600 s)—— 所以 Bash 调用一律走这一条。
2323
+ 引擎的缺省值可由进程 env `BASH_DEFAULT_TIMEOUT_MS` 调大:调大它就是调大每一发不传参数的 Bash 的挂死护栏(挂死的命令要等满它),本服务没有旋钮能再收紧 —— 要护栏就调回它,或给命令显式的短超时。
2324
+ 安静的长构建(Gradle / R8 一段几分钟不出声)给命令一个够长的 `timeout` 即可;超出引擎上限的工作用 `run_in_background`。
2325
+ - 流式调用自己说了静默界(`readTimeoutMs`)⇒ 照它。
2326
+ - 两样都没说(引擎自己的 git 探针、文件读写这类)⇒ 车道活性界兜底,它是这条命令唯一的挂死护栏。
2327
+
2328
+ 界从调用开始算,覆盖命令全程 —— 含 e2b 供应商建命令那一段(命令还没起跑,自然没有输出):建命令卡住按这条命令最早的那一界切(说了超时按超时,没说界按活性界)。
2329
+ e2b SDK 自己的建命令握手另有一界(缺省 60 s),比命令的界早时先由它切:超时或活性界长于 60 s 的命令(含缺省 120 s 的活性界)建命令卡住在约 60 s 被切 —— 缓冲 exec(Bash 走的这一面)报 `timeout`,流式 exec 报 `connect_failed`。
2330
+
2331
+ | 旋钮 | 车道 | 缺省 | 说明 |
2332
+ |---|---|---|---|
2333
+ | `K8S_LIVENESS_MS` | `REMOTE_EXEC=k8s` | 缺席 ⇒ 120000(ms) | 7.129.1 起;与 `E2B_LIVENESS_MS` 同一只解析器:必须是正数,非数 / `0` / 负数拒启;只在本车道被选中时求值 |
2334
+ | `E2B_LIVENESS_MS` | `REMOTE_EXEC=e2b` | 缺席 ⇒ 120000(ms) | 同上(7.129.1 起 `0` / 负数拒启;此前照收,结果是每条命令建不起来) |
2335
+
2336
+ **有显式超时时这两只旋钮不参与**:调大不会让一条带超时的 Bash 跑得更久,调小也不会更早切它 —— 要更早切就给命令更短的超时。
2337
+
2338
+ 切断时报的是真正触发的那一界:活性切断的句子是 `no output for <N> s — cut off by the exec liveness bound (set K8S_LIVENESS_MS or give the command an explicit timeout)`
2339
+ (e2b 点名 `E2B_LIVENESS_MS`);错误码沿用引擎的 `timeout`(引擎码表把墙钟与活性两种超界归同一词,靠句子区分)。Bash 工具结果里的 `timedOut: true` / `timeoutSec: N`
2340
+ 不再是活性切断的投影,但仍不全是「跑满了 N 秒」:e2b 上建命令卡住被那只 60 s 握手切掉、或沙箱中途死掉时,这一发仍以 `timeout` 码回来,Bash 照旧报
2341
+ 「Command timed out after <N>s」(把它们与超时分开的独立码属于引擎的码表)。host 车道没有活性界(命令只受调用方的超时,缓冲 exec 不带超时时受 `HOST_COMMAND_TIMEOUT_MS`)。
2342
+
2318
2343
  ### 9.4 auto 模式(`permissionMode:"auto"`):武装三态、本地 org deny、自查(与 core 的意图武装式合修)
2319
2344
 
2320
2345
  **背景**:此前引擎只在「组织授予」(web-admin 解出的 per-principal `runtimeCaps.autoMode === true`)时武装
@@ -222,6 +222,7 @@ export function createExecutionEnv(ctx) {
222
222
  ...(config.remoteExec.runtimeClass != null ? { runtimeClass: config.remoteExec.runtimeClass } : {}),
223
223
  ...(config.remoteExec.mountPath ? { mountPath: config.remoteExec.mountPath } : {}),
224
224
  ...(config.remoteExec.timeoutMs != null ? { timeoutMs: config.remoteExec.timeoutMs } : {}),
225
+ ...(config.remoteExec.livenessMs != null ? { livenessMs: config.remoteExec.livenessMs } : {}),
225
226
  dataTimeoutMs: workspaceSnapshotTimeoutOf(config.remoteExec),
226
227
  ...(config.remoteExec.memory ? { memory: config.remoteExec.memory } : {}),
227
228
  ...(config.remoteExec.cpu ? { cpu: config.remoteExec.cpu } : {}),
@@ -5,6 +5,7 @@ import { CENTER_CREDENTIAL_RENEW_LEAD_DEFAULT_MS, CENTER_CREDENTIAL_RENEW_LEAD_M
5
5
  import { SANDBOX_IPV6_DEFAULT, SANDBOX_IPV6_WORDS } from "./sandbox-egress-policy.js";
6
6
  import { WORKSPACE_CACHE_DEFAULT, WORKSPACE_CACHE_MOUNT_DEFAULT, WORKSPACE_CACHE_SIZE_DEFAULT, WORKSPACE_CACHE_WORDS } from "./workspace-cache.js";
7
7
  import { WORKSPACE_PERSIST_DEFAULT, WORKSPACE_PERSIST_WORDS, WORKSPACE_SNAPSHOT_KEEP_DEFAULT, WORKSPACE_SNAPSHOT_KEEP_RANGE, WORKSPACE_SNAPSHOT_TIMEOUT_MS_DEFAULT, WORKSPACE_SNAPSHOT_TIMEOUT_MS_RANGE } from "./workspace-space.js";
8
+ import { LIVENESS_MS_DEFAULT } from "./exec-bounds.js";
8
9
  import { THINKING_FORMATS } from "./model-compat.js";
9
10
  import { AUTO_MODE_CLASSIFIER_TIMEOUT_BOUNDS, CENTER_BASE_URL_RULE, CONFIG_PROVIDER_WORDS, DRAIN_SECOND_SIGNAL_MIN_GAP_BOUNDS, DRAIN_SECOND_SIGNAL_MIN_GAP_DEFAULT_MS, FIRST_BYTE_TIMEOUT_BOUNDS, FLEET_LEASE_OUTAGE_MAX_DEFAULT_MS, RESILIENCE_DEFAULTS, TASK_WRITE_FACE_ESCALATIONS } from "./config-types.js";
10
11
  import { webSearchProviderFromEnv, WEB_SEARCH_PROVIDERS } from "./plugins/web-search.js";
@@ -30,6 +31,8 @@ const RESTART_SLICE_SET = {
30
31
  };
31
32
  export const CATALOG_RESTART_SLICES = Object.keys(RESTART_SLICE_SET);
32
33
  const r = (name, domain, type, semantics, opts = {}) => ({ name, domain, type, semantics, ...opts });
34
+ const LIVENESS_KNOB_SEMANTICS = (lane) => `${lane} 命令静默活性上限(ms):无输出满这么久即判挂死切断,只管没有显式超时的命令 —— 带显式超时的命令只受那个超时约束(安静的长构建给命令一个显式超时即可,不必调大本值)`;
35
+ const LIVENESS_KNOB_DEFAULT_NOTE = `缺席 ⇒ adapter 默认 ${LIVENESS_MS_DEFAULT / 1000}s`;
33
36
  const cLimits = (path, over = {}) => ({
34
37
  domain: "limits",
35
38
  path,
@@ -138,7 +141,7 @@ export const CONFIG_CATALOG = [
138
141
  r("E2B_BG_DEFAULT_TIMEOUT_SEC", "orchestration", "number", "e2b 后台 shell 缺省超时(秒)", { staticDefault: "300" }),
139
142
  r("E2B_BG_MAX_CONCURRENT", "orchestration", "number", "e2b 后台 shell 并发帽", { staticDefault: "8" }),
140
143
  r("E2B_BG_MAX_TIMEOUT_SEC", "orchestration", "number", "e2b 后台 shell 超时上限(秒)", { staticDefault: "1800" }),
141
- r("E2B_LIVENESS_MS", "orchestration", "number", "e2b 单命令静默活性上限(长静默构建请调大;后台+轮询仍是首选)", { derivedDefaultNote: "缺席 ⇒ adapter 默认 120s" }),
144
+ r("E2B_LIVENESS_MS", "orchestration", "number", LIVENESS_KNOB_SEMANTICS("e2b"), { derivedDefaultNote: LIVENESS_KNOB_DEFAULT_NOTE }),
142
145
  r("E2B_MOUNT_PATH", "orchestration", "path", "e2b 工作区根(resume 用当前值,改前先 drain 挂起任务)"),
143
146
  r("E2B_ORPHAN_RECLAIM", "orchestration", "boolean", "崩溃遗留沙盒回收腿(boot 一次+周期;判据保守,拿不准恒不杀)", { staticDefault: "true", danger: BILL }),
144
147
  r("E2B_SANDBOX_ENV", "orchestration", "secret", "转发进 e2b 沙箱的本进程 env 点名表(NAME,NAME;点到的键从 worker env 原样取,值不进 spec/center/事件日志;与 LSP_ENV_ALLOW / DOCKER_SANDBOX_ENV 同一只解析器与「点名放行」谓词:形错 / 点名名单型旋钮拒启且不回显值;本进程环境里没有的名按序号 warn、不转发;只在本车道被选中时求值;原文按凭据类对待:只回在场位、不进任何子进程)", { danger: SEC }),
@@ -191,6 +194,7 @@ export const CONFIG_CATALOG = [
191
194
  r("K8S_CA_CERT_B64", "orchestration", "opaque", "k8s API CA 证书(base64;registry secrets 无换行约束)", { danger: SEC }),
192
195
  r("K8S_CPU", "orchestration", "string", "沙箱 pod CPU 限额(profile 数据)"),
193
196
  r("K8S_INSECURE_TLS", "orchestration", "boolean", "跳过 k8s API TLS 校验", { staticDefault: "false", danger: SEC }),
197
+ r("K8S_LIVENESS_MS", "orchestration", "number", LIVENESS_KNOB_SEMANTICS("k8s"), { derivedDefaultNote: LIVENESS_KNOB_DEFAULT_NOTE }),
194
198
  r("K8S_MEMORY", "orchestration", "string", "沙箱 pod 内存限额(2Gi 默认曾被真 OOM)"),
195
199
  r("K8S_MOUNT_PATH", "orchestration", "path", "k8s 工作区挂载路径"),
196
200
  r("K8S_NAMESPACE", "orchestration", "string", "沙箱 pod namespace"),
@@ -898,6 +898,9 @@ export interface ServiceConfigFlat {
898
898
  runtimeClass?: string;
899
899
  mountPath?: string;
900
900
  timeoutMs?: number;
901
+ /** `K8S_LIVENESS_MS` — the lane's liveness bound (ms; the `E2B_LIVENESS_MS` twin, same parser; absent ⇒ the adapter's
902
+ * default 120 s): cuts a command silent for this long ONLY when its caller stated no timeout (`exec-bounds.ts`). */
903
+ livenessMs?: number;
901
904
  /** Resource-profile knobs: sandbox pod limits (adapter defaults: 2Gi / 1 cpu). */
902
905
  memory?: string;
903
906
  cpu?: string;
package/dist/config.js CHANGED
@@ -27,6 +27,7 @@ import { bakeOperatorPrincipalsRefusal, bakeRunnerCredentialRefusal, centerIdent
27
27
  import { declaredSemaConfigDir } from "./sema-config-dir.js";
28
28
  import { lkgPathOverrideOf } from "./config-lkg.js";
29
29
  import { K8S_WORKSPACE_MOUNT_DEFAULT, WORKSPACE_PERSIST_DEFAULT, WORKSPACE_PERSIST_WORDS, WORKSPACE_SNAPSHOT_KEEP_DEFAULT, WORKSPACE_SNAPSHOT_KEEP_RANGE, WORKSPACE_SNAPSHOT_TIMEOUT_MS_DEFAULT, WORKSPACE_SNAPSHOT_TIMEOUT_MS_RANGE, parseWorkspaceSnapshotExclude } from "./workspace-space.js";
30
+ import { laneLivenessOf } from "./exec-bounds.js";
30
31
  import { SANDBOX_IPV6_DEFAULT, SANDBOX_IPV6_WORDS } from "./sandbox-egress-policy.js";
31
32
  import { WORKSPACE_CACHE_DEFAULT, WORKSPACE_CACHE_MOUNT_DEFAULT, WORKSPACE_CACHE_SIZE_DEFAULT, WORKSPACE_CACHE_WORDS, isK8sQuantity, mountsOverlap } from "./workspace-cache.js";
32
33
  import { REMOTE_EXEC_PROVIDER_WORDS, executionLaneOf, executionLaneRequiredRefusal, isSingleMachineDefaultLane, laneRequiredEnv } from "./execution-lane-caps.js";
@@ -333,6 +334,10 @@ function optNumEnvStrict(name) {
333
334
  return undefined;
334
335
  return parseNumOrFail(name, raw);
335
336
  }
337
+ function livenessMsEnv(name) {
338
+ const n = optNumEnvStrict(name);
339
+ return n === undefined ? undefined : laneLivenessOf(n, name).ms;
340
+ }
336
341
  function parseDeviceEnrollTokenTtlEnv() {
337
342
  const raw = process.env.DEVICE_ENROLL_TOKEN_TTL_SEC;
338
343
  if (raw === undefined || raw === "")
@@ -1836,7 +1841,7 @@ function parseOrchestrationDomain(ctx) {
1836
1841
  apiKey: process.env.E2B_API_KEY,
1837
1842
  ...(process.env.E2B_TEMPLATE ? { template: process.env.E2B_TEMPLATE } : {}),
1838
1843
  ...((n) => (n !== undefined ? { timeoutMs: n } : {}))(optNumEnvStrict("E2B_TIMEOUT_MS")),
1839
- ...((n) => (n !== undefined ? { livenessMs: n } : {}))(optNumEnvStrict("E2B_LIVENESS_MS")),
1844
+ ...((n) => (n !== undefined ? { livenessMs: n } : {}))(livenessMsEnv("E2B_LIVENESS_MS")),
1840
1845
  ...(process.env.E2B_ALLOW_NET ? { allowInternetAccess: boolEnv("E2B_ALLOW_NET", true) } : {}),
1841
1846
  ...((names) => (names !== undefined ? { sandboxEnv: allowlistedEnv(names, process.env) } : {}))(parseEnvNameAllowlist("E2B_SANDBOX_ENV", singleUserTurnkey)),
1842
1847
  ...(process.env.E2B_MOUNT_PATH ? { mountPath: process.env.E2B_MOUNT_PATH } : {}),
@@ -1855,6 +1860,7 @@ function parseOrchestrationDomain(ctx) {
1855
1860
  ...(process.env.K8S_RUNTIME_CLASS != null ? { runtimeClass: process.env.K8S_RUNTIME_CLASS } : {}),
1856
1861
  ...(process.env.K8S_MOUNT_PATH ? { mountPath: process.env.K8S_MOUNT_PATH } : {}),
1857
1862
  ...((n) => (n !== undefined ? { timeoutMs: n } : {}))(optNumEnvStrict("K8S_TIMEOUT_MS")),
1863
+ ...((n) => (n !== undefined ? { livenessMs: n } : {}))(livenessMsEnv("K8S_LIVENESS_MS")),
1858
1864
  ...(process.env.K8S_MEMORY ? { memory: process.env.K8S_MEMORY } : {}),
1859
1865
  ...(process.env.K8S_CPU ? { cpu: process.env.K8S_CPU } : {}),
1860
1866
  sandboxIpv6: enumEnv("K8S_SANDBOX_IPV6", SANDBOX_IPV6_DEFAULT, SANDBOX_IPV6_WORDS),
@@ -0,0 +1,69 @@
1
+ /**
2
+ * 执行车道上「什么约束一条命令」的**唯一一条律**(host / e2b / k8s 三只 adapter 同读这一只;纯叶,零依赖):
3
+ *
4
+ * 命令由调用方**说出口**的界约束 —— 墙钟 `timeout`(buffered 与流式两面)与 / 或它自己的静默界 `readTimeoutMs`
5
+ * (流式面)。调用方**两样都没说**时,才由车道自己的活性界兜底:无输出满这么久即判挂死(k8s `K8S_LIVENESS_MS` /
6
+ * e2b `E2B_LIVENESS_MS`,缺省 {@link LIVENESS_MS_DEFAULT};host 没有这一界)—— 那时它是这条命令唯一的挂死护栏。
7
+ * 切断时报的是**真正触发的那一界**(活性切断的句子只在 {@link livenessCutMessage} 铸)。
8
+ *
9
+ * 为什么「说了墙钟就不再活性切断」:墙钟本身就是挂死护栏(它到点必切),而调用方说出一个墙钟,就是在说「这条命令可以跑
10
+ * 这么久」—— 安静的长构建(Gradle / R8 一段几分钟不出声)正是这种命令。两界并存时,活性界会先于墙钟切掉一条按约正在跑的
11
+ * 命令,而调用方(core 的 Bash 工具)只知道自己说过的墙钟,便把这次切断报成「跑满了 N 秒」。两界互斥之后,一次切断只可能
12
+ * 是它报的那一界。
13
+ *
14
+ * 与 core 远端契约(`ExecStreamOptions.readTimeoutMs`:「缺省也 SHOULD 有活性界,一条命令不得等到环境死」)的对齐:
15
+ * 说了墙钟的命令被墙钟兜住,不会无限等;两界都没说的命令仍有车道活性界。切断码沿用 core 的 `timeout`(core 远端码表
16
+ * 把墙钟与活性两种超界都归这一词);两者靠句子区分 —— 独立的活性码属于 core 的码表。
17
+ *
18
+ * 不在本律里的界(adapter 自己的上限,不是调用方说的话,不关活性界):k8s 文件读写的数据上限 `dataTimeoutMs`、e2b 交给
19
+ * 供应商的沙箱寿命上限。adapter 自己的整段数据搬运(k8s 快照 tar 上 / 下)天生静默,它们**自己说**一个墙钟(数据上限),
20
+ * 落在本律的第一支。
21
+ */
22
+ /** 车道活性界的缺省(ms)—— k8s / e2b 两只 adapter 与配置目录的「缺席 ⇒ 默认」说法同读这一只。 */
23
+ export declare const LIVENESS_MS_DEFAULT = 120000;
24
+ /** 有活性界的车道各自的旋钮名(闭集;切断句点名它,运维据此知道调哪一只)。 */
25
+ export type LivenessKnob = "K8S_LIVENESS_MS" | "E2B_LIVENESS_MS";
26
+ /** 一条车道自己的活性界:k8s / e2b 有,host 没有(传 `undefined`)。只经 {@link laneLivenessOf} 铸(值已判过)。 */
27
+ export interface LaneLiveness {
28
+ readonly ms: number;
29
+ readonly knob: LivenessKnob;
30
+ }
31
+ /**
32
+ * 车道活性界的唯一铸点与唯一判据:值必须是有限正数(ms),否则**拒**(抛,句子点名旋钮)。活性界是不说界的命令唯一的挂死护栏 ——
33
+ * `0` / 负数会把它整个撤掉(那条命令从此无界),非数无从计时。boot 的旋钮解析(`E2B_LIVENESS_MS` / `K8S_LIVENESS_MS`)与 adapter
34
+ * 构造(嵌入方直接传 `livenessMs`)都经这一只判,一个判据、一句拒词。
35
+ */
36
+ export declare function laneLivenessOf(ms: number, knob: LivenessKnob): LaneLiveness;
37
+ /** 本条命令的静默界:调用方说的(`readTimeoutMs`),或车道兜底的。 */
38
+ export type LivenessBound = {
39
+ readonly ms: number;
40
+ readonly setBy: "caller";
41
+ } | {
42
+ readonly ms: number;
43
+ readonly setBy: "lane";
44
+ readonly knob: LivenessKnob;
45
+ };
46
+ /** 一条命令最终受的界:墙钟(调用方说的)与静默界,各自可缺。 */
47
+ export interface CommandBounds {
48
+ readonly wallMs: number | undefined;
49
+ readonly liveness: LivenessBound | undefined;
50
+ }
51
+ /** 调用方说出口的界(ms)。`timeoutMs` = 墙钟;`readTimeoutMs` = 它自己的静默界。 */
52
+ export interface StatedBounds {
53
+ readonly timeoutMs?: number | undefined;
54
+ readonly readTimeoutMs?: number | undefined;
55
+ }
56
+ /** 本律的唯一实现(见文件头注)。 */
57
+ export declare function commandBoundsOf(stated: StatedBounds, lane: LaneLiveness | undefined): CommandBounds;
58
+ /**
59
+ * 一条**还没出过一个字节**的命令最早被哪一界切:墙钟与静默界里较早的那只(都从调用开始算)。供应商建命令那一段(命令还没起跑,
60
+ * 自然也没有输出)就受它约束 —— 说了墙钟的命令不会被车道活性界在建命令时切掉;没说界的命令建命令卡住按活性界切 —— 但供应商自己的建命令期限
61
+ * 先到就先切(e2b SDK 的 60 s 握手:到点报的是供应商那一界,不是本模块的任何一只)。两只都缺 ⇒ 无界(只剩供应商自己的界)。
62
+ */
63
+ export declare function firstBoundOf(bounds: CommandBounds): {
64
+ readonly ms: number;
65
+ readonly cut: "wall" | LivenessBound;
66
+ } | undefined;
67
+ /** 活性切断的句子 —— 三车道唯一铸点(车道兜底的界点名旋钮与出路;调用方自己说的界点名那一参数)。 */
68
+ export declare function livenessCutMessage(bound: LivenessBound): string;
69
+ //# sourceMappingURL=exec-bounds.d.ts.map
@@ -0,0 +1,37 @@
1
+ export const LIVENESS_MS_DEFAULT = 120_000;
2
+ export function laneLivenessOf(ms, knob) {
3
+ if (!(Number.isFinite(ms) && ms > 0)) {
4
+ throw new Error(`${knob}=${ms} must be a positive number of milliseconds — it is the only hang guard of a command whose caller states no timeout, and a zero, negative or non-numeric value would remove it`);
5
+ }
6
+ return { ms, knob };
7
+ }
8
+ const said = (ms) => (ms !== undefined && Number.isFinite(ms) && ms > 0 ? ms : undefined);
9
+ export function commandBoundsOf(stated, lane) {
10
+ const wallMs = said(stated.timeoutMs);
11
+ const idleMs = said(stated.readTimeoutMs);
12
+ if (idleMs !== undefined)
13
+ return { wallMs, liveness: { ms: idleMs, setBy: "caller" } };
14
+ if (wallMs !== undefined || lane === undefined)
15
+ return { wallMs, liveness: undefined };
16
+ return { wallMs, liveness: { ms: lane.ms, setBy: "lane", knob: lane.knob } };
17
+ }
18
+ export function firstBoundOf(bounds) {
19
+ const { wallMs, liveness } = bounds;
20
+ if (liveness !== undefined && (wallMs === undefined || liveness.ms <= wallMs))
21
+ return { ms: liveness.ms, cut: liveness };
22
+ return wallMs === undefined ? undefined : { ms: wallMs, cut: "wall" };
23
+ }
24
+ function spoken(ms) {
25
+ if (ms < 1000)
26
+ return `${ms} ms`;
27
+ return ms % 1000 === 0 ? `${ms / 1000} s` : `${(ms / 1000).toFixed(1)} s`;
28
+ }
29
+ export function livenessCutMessage(bound) {
30
+ switch (bound.setBy) {
31
+ case "lane":
32
+ return `no output for ${spoken(bound.ms)} — cut off by the exec liveness bound (set ${bound.knob} or give the command an explicit timeout)`;
33
+ case "caller":
34
+ return `no output for ${spoken(bound.ms)} — cut off by the command's own idle bound (readTimeoutMs)`;
35
+ }
36
+ }
37
+ //# sourceMappingURL=exec-bounds.js.map
@@ -71,8 +71,9 @@ export interface RemoteContainerEnvConfig extends RemoteEnvObservers {
71
71
  * isRunning/files.*. A provider hang is surfaced as a typed retryable `RemoteExecutionError("timeout")` after
72
72
  * this, instead of pinning the worker for the sandbox lifetime. Default {@link DEFAULT_RPC_TIMEOUT_MS}. */
73
73
  rpcTimeoutMs?: number;
74
- /** 🔴 (hang class-fix) Idle/liveness bound (ms) for a command (exec/execStream): max time with NO output
75
- * and not yet complete = suspected hang. Also bounds the command-creation RPC. Default {@link DEFAULT_LIVENESS_MS}. */
74
+ /** 🔴 (hang class-fix) The lane's idle/liveness bound (ms; env `E2B_LIVENESS_MS`): max time with NO output and not yet
75
+ * complete = suspected hang — cuts a running command ONLY when its caller stated neither a wall-clock (`timeout`) nor its own
76
+ * idle bound (`readTimeoutMs`); the rule: `exec-bounds.ts` (creation included). Positive only (`laneLivenessOf`). Default `LIVENESS_MS_DEFAULT`. */
76
77
  livenessMs?: number;
77
78
  /** 🔴 Wall-clock for data-transfer/provisioning RPCs (large file io, snapshot suspend/resume,
78
79
  * create/connect) — wider than {@link rpcTimeoutMs} so a legit multi-minute transfer is not killed. Default
@@ -105,6 +106,8 @@ export declare class RemoteContainerExecutionEnv implements RemoteExecutionEnv,
105
106
  readonly cdPath: "unknown";
106
107
  private readonly cfg;
107
108
  private readonly sandboxApi;
109
+ /** This lane's own liveness bound — what {@link commandBoundsOf} falls back to when a command's caller stated no bound. */
110
+ private readonly laneLiveness;
108
111
  private sbx?;
109
112
  /** 🔴 BL-31 keep-alive — the E2B sandbox lifetime is hard-capped (≈1h, {@link E2B_SANDBOX_MAX_MS}), but a long
110
113
  * leader/repair run legitimately exceeds it. This interval RE-extends the sandbox (`setTimeout(timeoutMs)` resets
@@ -287,23 +290,42 @@ export declare class RemoteContainerExecutionEnv implements RemoteExecutionEnv,
287
290
  * the adapter NEVER blind-retries). A caller AbortSignal still aborts independently (op surfaces it / "aborted").
288
291
  */
289
292
  private rpc;
293
+ /**
294
+ * A command start that already returned to its caller (its creation deadline fired, or it failed): a handle the provider
295
+ * hands over LATER is killed — the caller was told the command was cut, so it must not run on. The rule for the in-flight
296
+ * count this start holds: it is released **once, and only when the command is confirmed not running** (council #4: never
297
+ * snapshot a running command). Confirmations: the provider answered `kill()` (`true` killed / `false` no such process);
298
+ * the handle's one `wait()` reports the process exited (a result, or `CommandExitError` for a non-zero exit). Not
299
+ * confirmations: a refused `kill()` (logged; the exit is still awaited), a `wait()` that ends without an exit (the
300
+ * connection deadline, a lost stream — the process may still run). With neither the count stays held and `suspendVM`
301
+ * keeps refusing `command_in_flight` — the same arm as a start that never settles. No start promise (the op never ran)
302
+ * ⇒ released now. A failed start is released at once too, as in 7.129.0 — and that one is NOT a confirmation: there is
303
+ * no handle to kill or wait on, and a start whose reply was lost (the SDK's own 60 s handshake) may have spawned the
304
+ * process. Closing it means classifying the start failure (provider refused ⇒ not started; lost reply ⇒ held) — open,
305
+ * not done here.
306
+ */
307
+ private settleAbandonedStart;
290
308
  /**
291
309
  * Per-command wall-clock for the E2B SDK (seconds→ms). 🔴 This is a PHYSICAL CAP, NOT the hang
292
310
  * detector. The old code returned `this.cfg.timeoutMs` (the sandbox lifetime) when no `timeout` was given AND
293
311
  * had no liveness check → a hung command waited the WHOLE lifetime (~30 min, the original hang bug). The fix is
294
312
  * NOT to shorten this cap (that would spuriously kill a long-but-progressing `npm ci`) — it's to add a LIVENESS
295
313
  * bound ({@link cfg.livenessMs}) in exec/execStream that catches a *zero-progress* command in ~livenessMs,
296
- * independent of this cap. So: explicit caller `timeout` wins; else the lifetime is the cap (a command can't
297
- * outlive its VM) but hang detection no longer depends on it.
314
+ * independent of this cap. So: explicit caller `timeout` wins — and is then the command's bound (the liveness bound
315
+ * stays out, `exec-bounds.ts`); else the lifetime is the cap (a command can't outlive its VM) and the liveness bound
316
+ * is the hang detector.
298
317
  */
299
318
  private commandTimeoutMs;
300
319
  /**
301
- * 🔴 Liveness for the buffered exec WAIT phase. `handle.wait()` is a long-poll with no read timeout —
320
+ * 🔴 Bounds for the buffered exec WAIT phase. `handle.wait()` is a long-poll with no read timeout —
302
321
  * a command that hangs DURING execution (not creation) would otherwise wait the E2B command timeout (= the
303
- * lifetime when none is set, the original hang bug). Races wait() against an idle check: if no stdout/stderr for
304
- * `livenessMs`, the command is hung → kill + typed retryable "timeout". `lastProgress()` is bumped by the
305
- * stdout/stderr callbacks, so a slow-but-PROGRESSING command (long `npm ci`) is never killed. (execStream has
306
- * its own chunk-driven idle timer; this is the buffered-exec analogue.)
322
+ * lifetime when none is set, the original hang bug). With a `liveness` bound (the caller stated no `timeout` —
323
+ * `exec-bounds.ts`) it races wait() against an idle check: no stdout/stderr for `liveness.ms` ⇒ hung → kill + typed
324
+ * retryable "timeout" with the shared cut sentence. `lastProgress()` is bumped by the stdout/stderr callbacks, so a
325
+ * slow-but-PROGRESSING command (long `npm ci`) is never killed. Both cuts reject with an `ExecutionError` carrying
326
+ * its real code (exec returns it as-is) — a `RemoteExecutionError` here would reach {@link execError}'s text match
327
+ * and land as `unknown`. The rejection is armed BEFORE the kill: the kill rejects `wait()` too, and the race must
328
+ * see the bound, not the kill. (execStream has its own chunk-driven idle timer; this is the buffered-exec analogue.)
307
329
  */
308
330
  private waitWithLiveness;
309
331
  /** Lazily provision the VM on first use: a task that never touches the hand pays zero cost.
@@ -3,13 +3,13 @@ import { Sandbox, CommandExitError, FileType } from "e2b";
3
3
  import { FileError, ExecutionError, RemoteExecutionError, RollingTailBuffer, markTruncated, } from "@sema-agent/core";
4
4
  import { fileErrorFromExec } from "./remote-env-file-error.js";
5
5
  import { numEnvOr, runSymlinkChain, shellQuote } from "./remote-shell.js";
6
+ import { commandBoundsOf, firstBoundOf, laneLivenessOf, livenessCutMessage, LIVENESS_MS_DEFAULT } from "../exec-bounds.js";
6
7
  import { BackgroundShellManager } from "./background-shell-support.js";
7
8
  const PROVIDER = "e2b";
8
9
  const DEFAULT_MOUNT_PATH = "/home/user";
9
10
  const DEFAULT_TIMEOUT_MS = 5 * 60_000;
10
11
  const E2B_SANDBOX_MAX_MS = numEnvOr("E2B_SANDBOX_MAX_MS", 3_600_000, 60_000);
11
12
  const DEFAULT_RPC_TIMEOUT_MS = 60_000;
12
- const DEFAULT_LIVENESS_MS = 120_000;
13
13
  const DEFAULT_DATA_TIMEOUT_MS = 5 * 60_000;
14
14
  const E2B_BG_MAX_CONCURRENT = numEnvOr("E2B_BG_MAX_CONCURRENT", 8, 1);
15
15
  const E2B_BG_DEFAULT_TIMEOUT_SEC = numEnvOr("E2B_BG_DEFAULT_TIMEOUT_SEC", 300, 1);
@@ -22,6 +22,7 @@ export class RemoteContainerExecutionEnv {
22
22
  cdPath = "unknown";
23
23
  cfg;
24
24
  sandboxApi;
25
+ laneLiveness;
25
26
  sbx;
26
27
  keepAlive;
27
28
  handle;
@@ -36,9 +37,10 @@ export class RemoteContainerExecutionEnv {
36
37
  timeoutMs: Math.min(config.timeoutMs ?? DEFAULT_TIMEOUT_MS, E2B_SANDBOX_MAX_MS),
37
38
  allowInternetAccess: config.allowInternetAccess ?? true,
38
39
  rpcTimeoutMs: config.rpcTimeoutMs ?? DEFAULT_RPC_TIMEOUT_MS,
39
- livenessMs: config.livenessMs ?? DEFAULT_LIVENESS_MS,
40
+ livenessMs: config.livenessMs ?? LIVENESS_MS_DEFAULT,
40
41
  dataTimeoutMs: config.dataTimeoutMs ?? DEFAULT_DATA_TIMEOUT_MS,
41
42
  };
43
+ this.laneLiveness = laneLivenessOf(this.cfg.livenessMs, "E2B_LIVENESS_MS");
42
44
  this.cwd = this.cfg.mountPath;
43
45
  this.homeDir = config.homeDir;
44
46
  this.sandboxApi = deps.sandboxApi ?? Sandbox;
@@ -255,9 +257,14 @@ export class RemoteContainerExecutionEnv {
255
257
  s.total += b.length;
256
258
  };
257
259
  let lastProgress = Date.now();
260
+ const startedAt = lastProgress;
261
+ const bounds = commandBoundsOf({ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined }, this.laneLiveness);
262
+ const first = firstBoundOf(bounds);
258
263
  let handle;
264
+ let starting;
265
+ let startCut = false;
259
266
  try {
260
- handle = await this.rpc("commands.run", () => sbx.commands.run(command, {
267
+ handle = await this.rpc("commands.run", () => (starting = sbx.commands.run(command, {
261
268
  background: true,
262
269
  cwd: this.resolve(options?.cwd ?? this.cwd),
263
270
  ...(options?.env ? { envs: options.env } : {}),
@@ -265,6 +272,8 @@ export class RemoteContainerExecutionEnv {
265
272
  ? Math.max(this.commandTimeoutMs(options?.timeout), (E2B_BG_MAX_TIMEOUT_SEC + E2B_BG_PROVIDER_BACKSTOP_PAD_SEC) * 1000)
266
273
  : this.commandTimeoutMs(options?.timeout),
267
274
  onStdout: (d) => {
275
+ if (startCut)
276
+ return;
268
277
  if (tee)
269
278
  teePush(tee.stdout, d);
270
279
  if (!detached) {
@@ -273,6 +282,8 @@ export class RemoteContainerExecutionEnv {
273
282
  }
274
283
  },
275
284
  onStderr: (d) => {
285
+ if (startCut)
286
+ return;
276
287
  if (tee)
277
288
  teePush(tee.stderr, d);
278
289
  if (!detached) {
@@ -280,11 +291,13 @@ export class RemoteContainerExecutionEnv {
280
291
  options?.onStderr?.(d);
281
292
  }
282
293
  },
283
- }), options?.abortSignal, this.cfg.livenessMs);
294
+ })), options?.abortSignal, first.ms);
284
295
  }
285
296
  catch (e) {
286
- this.activeCommands--;
287
- return { ok: false, error: this.execError(e, options?.abortSignal) };
297
+ startCut = true;
298
+ this.settleAbandonedStart(starting);
299
+ const cut = e instanceof RemoteExecutionError && e.code === "timeout" && !options?.abortSignal?.aborted;
300
+ return { ok: false, error: cut ? new ExecutionError("timeout", boundCutMessage(first)) : this.execError(e, options?.abortSignal) };
288
301
  }
289
302
  const onAbort = () => { if (!detached)
290
303
  void handle.kill().catch(() => { }); };
@@ -333,7 +346,8 @@ export class RemoteContainerExecutionEnv {
333
346
  const fg = this.waitWithLiveness(handle, () => lastProgress, {
334
347
  waitP,
335
348
  disarmed: () => detached,
336
- ...(tee ? { wallMs: this.commandTimeoutMs(options?.timeout) } : {}),
349
+ ...(tee ? { wall: { totalMs: this.commandTimeoutMs(options?.timeout), leftMs: Math.max(0, this.commandTimeoutMs(options?.timeout) - (Date.now() - startedAt)) } } : {}),
350
+ liveness: bounds.liveness,
337
351
  });
338
352
  const winner = await (detachedP ? Promise.race([fg, detachedP]) : fg);
339
353
  if (winner === "detached") {
@@ -385,6 +399,9 @@ export class RemoteContainerExecutionEnv {
385
399
  };
386
400
  let bytes = 0;
387
401
  const cap = options?.maxOutputBytes;
402
+ const bounds = commandBoundsOf({ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined, readTimeoutMs: options?.readTimeoutMs }, this.laneLiveness);
403
+ const liveness = bounds.liveness;
404
+ const first = firstBoundOf(bounds);
388
405
  const push = (type, data) => {
389
406
  if (finished)
390
407
  return;
@@ -394,18 +411,22 @@ export class RemoteContainerExecutionEnv {
394
411
  };
395
412
  this.activeCommands++;
396
413
  let handle;
414
+ let starting;
397
415
  try {
398
- handle = await this.rpc("commands.run", () => sbx.commands.run(command, {
416
+ handle = await this.rpc("commands.run", () => (starting = sbx.commands.run(command, {
399
417
  background: true,
400
418
  cwd: this.resolve(options?.cwd ?? this.cwd),
401
419
  ...(options?.env ? { envs: options.env } : {}),
402
420
  timeoutMs: this.commandTimeoutMs(options?.timeout),
403
421
  onStdout: (d) => push("stdout", d),
404
422
  onStderr: (d) => push("stderr", d),
405
- }), options?.signal, this.cfg.livenessMs);
423
+ })), options?.signal, first.ms);
406
424
  }
407
425
  catch (e) {
408
- this.activeCommands--;
426
+ finished = true;
427
+ this.settleAbandonedStart(starting);
428
+ if (e instanceof RemoteExecutionError && e.code === "timeout" && !options?.signal?.aborted)
429
+ throw new RemoteExecutionError("timeout", boundCutMessage(first), e);
409
430
  throw this.remoteError(e, "connect_failed");
410
431
  }
411
432
  handle
@@ -424,7 +445,7 @@ export class RemoteContainerExecutionEnv {
424
445
  exitCode = e.exitCode;
425
446
  }
426
447
  else {
427
- failure = this.remoteError(e, "unknown");
448
+ failure = this.remoteError(e, providerTimedOut(e) ? "timeout" : "unknown");
428
449
  }
429
450
  finished = true;
430
451
  signalReady();
@@ -465,13 +486,14 @@ export class RemoteContainerExecutionEnv {
465
486
  const next = new Promise((resolve) => {
466
487
  wake = resolve;
467
488
  });
468
- const idleMs = options?.readTimeoutMs ?? this.cfg.livenessMs;
469
489
  let timer;
470
- const idle = new Promise((_, reject) => {
471
- timer = setTimeout(() => reject(new RemoteExecutionError("timeout", `execStream idle > ${idleMs}ms (provider hang / unreachable sandbox)`)), idleMs);
472
- });
490
+ const idle = liveness === undefined
491
+ ? undefined
492
+ : new Promise((_, reject) => {
493
+ timer = setTimeout(() => reject(new RemoteExecutionError("timeout", livenessCutMessage(liveness))), liveness.ms);
494
+ });
473
495
  try {
474
- await Promise.race([next, idle]);
496
+ await (idle === undefined ? next : Promise.race([next, idle]));
475
497
  }
476
498
  catch (e) {
477
499
  finished = true;
@@ -794,36 +816,59 @@ export class RemoteContainerExecutionEnv {
794
816
  combined?.dispose();
795
817
  }
796
818
  }
819
+ settleAbandonedStart(starting) {
820
+ let held = true;
821
+ const release = () => {
822
+ if (!held)
823
+ return;
824
+ held = false;
825
+ this.activeCommands--;
826
+ };
827
+ if (starting === undefined)
828
+ return release();
829
+ void starting.then((late) => {
830
+ late.wait().then(release, (e) => {
831
+ if (e instanceof CommandExitError)
832
+ release();
833
+ });
834
+ late.kill().then(release, (e) => {
835
+ this.cfg.logger?.warn?.("e2b_abandoned_start_kill_failed", { sandboxId: this.sbx?.sandboxId, pid: late.pid, error: e instanceof Error ? e.name : typeof e });
836
+ });
837
+ }, release);
838
+ }
797
839
  commandTimeoutMs(timeoutSec) {
798
840
  return timeoutSec != null ? timeoutSec * 1000 : this.cfg.timeoutMs;
799
841
  }
800
842
  async waitWithLiveness(handle, lastProgress, opts) {
801
843
  let timer;
802
844
  let wallTimer;
845
+ const liveness = opts?.liveness;
803
846
  const idle = new Promise((_, reject) => {
804
- const arm = () => {
847
+ const arm = (bound) => {
805
848
  timer = setTimeout(() => {
806
849
  if (opts?.disarmed?.())
807
850
  return;
808
851
  const idleFor = Date.now() - lastProgress();
809
- if (idleFor >= this.cfg.livenessMs) {
852
+ if (idleFor >= bound.ms) {
853
+ reject(new ExecutionError("timeout", livenessCutMessage(bound)));
810
854
  void handle.kill().catch(() => { });
811
855
  this.cfg.metrics?.inc("remote_env_exec_idle_timeout_total", { provider: PROVIDER });
812
- reject(new RemoteExecutionError("timeout", `exec idle > ${this.cfg.livenessMs}ms (provider hang / unreachable sandbox)`));
813
856
  }
814
857
  else {
815
- arm();
858
+ arm(bound);
816
859
  }
817
- }, Math.max(this.cfg.livenessMs - (Date.now() - lastProgress()), 0));
860
+ }, Math.max(bound.ms - (Date.now() - lastProgress()), 0));
818
861
  };
819
- arm();
820
- if (opts?.wallMs != null) {
862
+ if (liveness !== undefined)
863
+ arm(liveness);
864
+ if (opts?.wall !== undefined) {
865
+ const wall = opts.wall;
821
866
  wallTimer = setTimeout(() => {
822
867
  if (opts.disarmed?.())
823
868
  return;
869
+ reject(new ExecutionError("timeout", boundCutMessage({ ms: wall.totalMs, cut: "wall" })));
824
870
  void handle.kill().catch(() => { });
825
- reject(new RemoteExecutionError("timeout", `exec exceeded ${opts.wallMs}ms`));
826
- }, opts.wallMs);
871
+ }, wall.leftMs);
827
872
  }
828
873
  });
829
874
  try {
@@ -905,7 +950,7 @@ export class RemoteContainerExecutionEnv {
905
950
  const name = e?.name;
906
951
  if (abortSignal?.aborted || name === "AbortError")
907
952
  return new ExecutionError("aborted", msg);
908
- if (name === "TimeoutError" || /timeout|timed out/i.test(msg))
953
+ if (providerTimedOut(e))
909
954
  return new ExecutionError("timeout", msg);
910
955
  return new ExecutionError("unknown", msg, e instanceof Error ? e : undefined);
911
956
  }
@@ -924,6 +969,12 @@ export function e2bExecutionEnvFactory(config, deps) {
924
969
  function errMsg(e) {
925
970
  return e instanceof Error ? e.message : String(e);
926
971
  }
972
+ function boundCutMessage(first) {
973
+ return first.cut === "wall" ? `exec exceeded ${first.ms}ms` : livenessCutMessage(first.cut);
974
+ }
975
+ function providerTimedOut(e) {
976
+ return e?.name === "TimeoutError" || /timeout|timed out/i.test(errMsg(e));
977
+ }
927
978
  function anySignal(signals) {
928
979
  const ctor = AbortSignal;
929
980
  if (typeof ctor.any === "function")
@@ -6,6 +6,7 @@ import { randomBytes } from "node:crypto";
6
6
  import { openSync, closeSync, readSync, statSync, truncateSync, unlinkSync, mkdirSync, rmdirSync, mkdtempSync, existsSync, realpathSync, accessSync, constants as FS_CONST } from "node:fs";
7
7
  import { StringDecoder } from "node:string_decoder";
8
8
  import { armPipeDestroyGrace, numEnvOr, SYMLINK_CHAIN_MAX_HOPS } from "./remote-shell.js";
9
+ import { commandBoundsOf, livenessCutMessage } from "../exec-bounds.js";
9
10
  import { hostBackgroundShellEnabled, hostExecSpoolEnabled } from "../config.js";
10
11
  import { IS_WIN32, resolveHostShell, hostShell, pinHostShell, spawnGroupOptions, killTreeHard, killTreeSoft, hardKillGroupNow, collapseWin32EnvKeys, adoptHostForegroundChild } from "./host-platform.js";
11
12
  import { BackgroundShellManager, seedMemStream, feedMemStream, drainMemStream } from "./background-shell-support.js";
@@ -810,8 +811,9 @@ export class RemoteHostExecutionEnv {
810
811
  if (!ready.ok)
811
812
  throw new RemoteExecutionError("connect_failed", ready.error.message, ready.error);
812
813
  const cwd = this.resolve(options?.cwd ?? this.cwd);
813
- const timeoutMs = options?.timeout != null ? options.timeout * 1000 : undefined;
814
- const idleMs = options?.readTimeoutMs;
814
+ const bounds = commandBoundsOf({ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined, readTimeoutMs: options?.readTimeoutMs }, undefined);
815
+ const timeoutMs = bounds.wallMs;
816
+ const liveness = bounds.liveness;
815
817
  const cap = options?.maxOutputBytes;
816
818
  const queue = [];
817
819
  let finished = false;
@@ -905,7 +907,7 @@ export class RemoteHostExecutionEnv {
905
907
  throw failure;
906
908
  if (finished)
907
909
  break;
908
- const waitMs = idleMs != null ? Math.max(1, idleMs - (Date.now() - lastChunk)) : 30_000;
910
+ const waitMs = liveness !== undefined ? Math.max(1, liveness.ms - (Date.now() - lastChunk)) : 30_000;
909
911
  let waitTimer;
910
912
  try {
911
913
  await new Promise((r) => { wake = r; waitTimer = setTimeout(r, waitMs); });
@@ -914,10 +916,10 @@ export class RemoteHostExecutionEnv {
914
916
  if (waitTimer !== undefined)
915
917
  clearTimeout(waitTimer);
916
918
  }
917
- if (idleMs != null && !finished && Date.now() - lastChunk >= idleMs) {
919
+ if (liveness !== undefined && !finished && Date.now() - lastChunk >= liveness.ms) {
918
920
  finished = true;
919
921
  kill();
920
- throw new RemoteExecutionError("timeout", `execStream idle > ${idleMs}ms (command hang)`);
922
+ throw new RemoteExecutionError("timeout", livenessCutMessage(liveness));
921
923
  }
922
924
  }
923
925
  while (queue.length)
@@ -38,7 +38,9 @@ export interface K8sEnvConfig {
38
38
  readyTimeoutMs?: number;
39
39
  /** Control-plane RPC wall-clock (pod create/get/delete). */
40
40
  rpcTimeoutMs?: number;
41
- /** Zero-progress liveness bound for commands (no output AND not finished ⇒ hang). */
41
+ /** The lane's zero-progress liveness bound (env `K8S_LIVENESS_MS`; default {@link LIVENESS_MS_DEFAULT}): a command that
42
+ * produces no output for this long and has not finished is cut as a hang — ONLY when its caller stated neither a wall-clock
43
+ * (`timeout`) nor its own idle bound (`readTimeoutMs`); a stated bound replaces it (the rule: `exec-bounds.ts`). */
42
44
  livenessMs?: number;
43
45
  /** The ONE data bound (`WORKSPACE_SNAPSHOT_TIMEOUT_MS`; default = `WORKSPACE_SNAPSHOT_TIMEOUT_MS_DEFAULT`, owned by
44
46
  * `workspace-space.ts`): the snapshot upload's and restore's wall-clock, the deadline this replica's commits write into
@@ -257,6 +259,8 @@ export declare class RemoteK8sExecutionEnv implements RemoteExecutionEnv, Backgr
257
259
  /** The known occupant's incarnation (`undefined` = this run has no known occupant: a delegated child, an unread session, a direct construction) — the
258
260
  * ownership mark of every snapshot id this env mints, both lanes ({@link mintSnapshotId}); the space lane's partition is the same reading. */
259
261
  private readonly occupantIncarnation;
262
+ /** This lane's own liveness bound — what {@link commandBoundsOf} falls back to when a command's caller stated no bound. */
263
+ private readonly laneLiveness;
260
264
  /** This env's space lane (resolved once from the policy + the occupant's incarnation; see {@link SpaceLane}). */
261
265
  private readonly spaceLane;
262
266
  /** `workspace_space_unavailable` is said once per env. */
@@ -551,8 +555,10 @@ export declare class RemoteK8sExecutionEnv implements RemoteExecutionEnv, Backgr
551
555
  private execUrl;
552
556
  /**
553
557
  * Run a command over the exec WebSocket. Frames: byte0=channel (1 stdout / 2 stderr / 3 status JSON with the
554
- * REAL exit code). Bounds: caller wall-clock (explicit `timeout`) + zero-progress liveness (`livenessMs`) —
555
- * a socket killed by either does NOT kill the remote process (disposable-pod posture, see header).
558
+ * REAL exit code). Bounds = {@link commandBoundsOf} over what the caller stated (`timeoutMs` wall / `readTimeoutMs` idle)
559
+ * and this lane's liveness — the one rule, decided here and nowhere else on this lane — plus the adapter's own ceiling
560
+ * `capMs` (a data bound, not a statement: it never turns the liveness bound off). A socket killed by any bound does NOT
561
+ * kill the remote process (disposable-pod posture, see header).
556
562
  */
557
563
  private runExec;
558
564
  exec(command: string, options?: ExecOpts): Promise<Result<{
@@ -654,7 +660,9 @@ export declare class RemoteK8sExecutionEnv implements RemoteExecutionEnv, Backgr
654
660
  suffix?: string;
655
661
  abortSignal?: AbortSignal;
656
662
  }): Promise<Result<string, FileError>>;
657
- /** exec with the data-transfer bound (large reads/writes legitimately run longer than a control op). */
663
+ /** exec with the data-transfer ceiling (large reads/writes legitimately run longer than a control op). The engine states no
664
+ * bound for a file op, so the lane liveness bound stays on (a stalled transfer is cut by it); the data bound is this
665
+ * adapter's own ceiling (`capMs`), not a statement. */
658
666
  private execData;
659
667
  /** 本腿只定工作目录(缺省 cwd 规则 + 绝对化);串的形状归 {@link buildCwdEnvCommand} / {@link createCwdGuardedCall}(S-410:cwd 不可用 ⇒ 命令不跑;
660
668
  * 车PL:buffered `exec` 上守卫触发 ⇒ `spawn_error`)。 */
@@ -11,6 +11,7 @@ import { spaceCommitAlive } from "../workspace-space-commit.js";
11
11
  import { incarnationLogPrefix, redactIncarnations } from "../session-incarnation.js";
12
12
  import { WORKSPACE_CACHE_SPACE_LABEL, workspaceCacheClaimOf, workspaceCacheEnv } from "../workspace-cache.js";
13
13
  import { recordFailOpen } from "../observability/fail-open.js";
14
+ import { commandBoundsOf, laneLivenessOf, livenessCutMessage, LIVENESS_MS_DEFAULT } from "../exec-bounds.js";
14
15
  import { SANDBOX_IPV6_DEFAULT, sandboxEgressGaps, sandboxPodSysctls } from "../sandbox-egress-policy.js";
15
16
  import { FileError, ExecutionError, RemoteExecutionError, withRetry, RollingTailBuffer, markTruncated, } from "@sema-agent/core";
16
17
  import { fileErrorFromExec, classifyFsStderr } from "./remote-env-file-error.js";
@@ -141,6 +142,7 @@ export class RemoteK8sExecutionEnv {
141
142
  cfg;
142
143
  deps;
143
144
  occupantIncarnation;
145
+ laneLiveness;
144
146
  spaceLane;
145
147
  spaceUnavailableSaid = false;
146
148
  unreadableOccupantSaid = false;
@@ -161,7 +163,7 @@ export class RemoteK8sExecutionEnv {
161
163
  timeoutMs: config.timeoutMs ?? 30 * 60_000,
162
164
  readyTimeoutMs: config.readyTimeoutMs ?? 120_000,
163
165
  rpcTimeoutMs: config.rpcTimeoutMs ?? 30_000,
164
- livenessMs: config.livenessMs ?? 120_000,
166
+ livenessMs: config.livenessMs ?? LIVENESS_MS_DEFAULT,
165
167
  dataTimeoutMs: dataTimeoutMsOf(config),
166
168
  cpu: config.cpu ?? "1",
167
169
  memory: config.memory ?? "2Gi",
@@ -176,6 +178,7 @@ export class RemoteK8sExecutionEnv {
176
178
  this.cwd = this.cfg.mountPath;
177
179
  this.homeDir = config.homeDir;
178
180
  this.deps = deps ?? {};
181
+ this.laneLiveness = laneLivenessOf(this.cfg.livenessMs, "K8S_LIVENESS_MS");
179
182
  const occupant = occupantKeyOf(config.spaceOccupant);
180
183
  this.occupantIncarnation = "incarnation" in occupant ? occupant.incarnation : undefined;
181
184
  this.spaceLane = spaceLaneOf(config.s3Snapshot?.space, config.spaceOccupant);
@@ -875,7 +878,7 @@ export class RemoteK8sExecutionEnv {
875
878
  const excludes = (this.cfg.snapshotExclude ?? []).map((item) => ` --exclude=${shellQuote(`./${item}`)}`).join("");
876
879
  const cmd = `tar czf ${tmp}${excludes} -C ${shellQuote(this.cfg.mountPath)} . && curl -fsS -X PUT -T ${tmp} ${shellQuote(url)}; ` +
877
880
  `rc=$?; rm -f ${tmp}; exit $rc`;
878
- const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs, livenessMs: 0 });
881
+ const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs });
879
882
  if (!r.ok)
880
883
  return { ok: false, error: new RemoteExecutionError("connect_failed", `workspace snapshot upload failed: ${this.redactUrl(r.error.message, url)}`, r.error) };
881
884
  if (r.value.exitCode !== 0)
@@ -911,7 +914,7 @@ export class RemoteK8sExecutionEnv {
911
914
  const c = await this.provision(abortSignal);
912
915
  if (!c.ok)
913
916
  return c;
914
- const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs, livenessMs: 0 });
917
+ const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs });
915
918
  const failure = !r.ok
916
919
  ? { cls: r.error.code === "timeout" ? "timeout" : "transport", detail: `workspace snapshot restore failed: ${this.redactUrl(r.error.message, url)}`, cause: r.error }
917
920
  : r.value.exitCode !== 0
@@ -1127,20 +1130,22 @@ export class RemoteK8sExecutionEnv {
1127
1130
  }
1128
1131
  resolvePromise(r);
1129
1132
  };
1130
- const wallMs = opts?.timeoutMs;
1131
- const wallTimer = wallMs ? setTimeout(() => finish({ ok: false, error: new ExecutionError("timeout", `command exceeded ${wallMs}ms`) }), wallMs) : undefined;
1133
+ const bounds = commandBoundsOf({ timeoutMs: opts?.timeoutMs, readTimeoutMs: opts?.readTimeoutMs }, this.laneLiveness);
1134
+ const capMs = opts?.capMs;
1135
+ const wallMs = bounds.wallMs === undefined ? capMs : capMs === undefined ? bounds.wallMs : Math.min(bounds.wallMs, capMs);
1136
+ const wallTimer = wallMs !== undefined ? setTimeout(() => finish({ ok: false, error: new ExecutionError("timeout", `command exceeded ${wallMs}ms`) }), wallMs) : undefined;
1132
1137
  let livenessTimer;
1133
- const livenessMs = opts?.livenessMs ?? this.cfg.livenessMs;
1138
+ const liveness = bounds.liveness;
1134
1139
  const armLiveness = () => {
1135
- if (livenessMs <= 0)
1140
+ if (liveness === undefined)
1136
1141
  return;
1137
1142
  clearTimeout(livenessTimer);
1138
1143
  const idle = Date.now() - lastProgress;
1139
- if (idle >= livenessMs) {
1140
- finish({ ok: false, error: new ExecutionError("timeout", `no output for ${idle}ms (exec liveness) — suspected hang`) });
1144
+ if (idle >= liveness.ms) {
1145
+ finish({ ok: false, error: new ExecutionError("timeout", livenessCutMessage(liveness)) });
1141
1146
  return;
1142
1147
  }
1143
- livenessTimer = setTimeout(armLiveness, livenessMs - idle);
1148
+ livenessTimer = setTimeout(armLiveness, liveness.ms - idle);
1144
1149
  };
1145
1150
  armLiveness();
1146
1151
  const onAbort = () => finish({ ok: false, error: new ExecutionError("aborted", "command aborted") });
@@ -1343,22 +1348,20 @@ export class RemoteK8sExecutionEnv {
1343
1348
  wake = undefined;
1344
1349
  w?.();
1345
1350
  };
1346
- let lastChunk = Date.now();
1347
- const idleMs = options?.readTimeoutMs ?? this.cfg.livenessMs;
1348
1351
  const cap = options?.maxOutputBytes;
1349
1352
  let bytes = 0;
1350
1353
  const ac = new AbortController();
1351
1354
  const signal = options?.signal ? AbortSignal.any([options.signal, ac.signal]) : ac.signal;
1352
1355
  const done = this.runExec(buildCwdEnvCommand(command, this.workingDirOf(options?.cwd), options?.env, "raw").command, {
1353
1356
  signal,
1357
+ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined,
1358
+ readTimeoutMs: options?.readTimeoutMs,
1354
1359
  boundOutput: true,
1355
1360
  onStdout: (d) => {
1356
- lastChunk = Date.now();
1357
1361
  queue.push({ type: "stdout", data: d });
1358
1362
  signalReady();
1359
1363
  },
1360
1364
  onStderr: (d) => {
1361
- lastChunk = Date.now();
1362
1365
  queue.push({ type: "stderr", data: d });
1363
1366
  signalReady();
1364
1367
  },
@@ -1395,21 +1398,9 @@ export class RemoteK8sExecutionEnv {
1395
1398
  throw failure;
1396
1399
  if (finished)
1397
1400
  break;
1398
- const waitMs = Math.max(1, idleMs - (Date.now() - lastChunk));
1399
- let waitTimer;
1400
- try {
1401
- await new Promise((r) => {
1402
- wake = r;
1403
- waitTimer = setTimeout(r, Math.min(waitMs, 30_000));
1404
- });
1405
- }
1406
- finally {
1407
- if (waitTimer !== undefined)
1408
- clearTimeout(waitTimer);
1409
- }
1410
- if (!finished && Date.now() - lastChunk >= idleMs) {
1411
- throw new RemoteExecutionError("timeout", `execStream idle > ${idleMs}ms (suspected hang)`);
1412
- }
1401
+ await new Promise((r) => {
1402
+ wake = r;
1403
+ });
1413
1404
  }
1414
1405
  while (queue.length)
1415
1406
  yield queue.shift();
@@ -1428,7 +1419,7 @@ export class RemoteK8sExecutionEnv {
1428
1419
  }
1429
1420
  makeBgDriver() {
1430
1421
  const self = this;
1431
- const bgExec = (script, boundOutput) => self.runExec(script, { timeoutMs: self.cfg.rpcTimeoutMs, livenessMs: self.cfg.livenessMs, boundOutput });
1422
+ const bgExec = (script, boundOutput) => self.runExec(script, { timeoutMs: self.cfg.rpcTimeoutMs, boundOutput });
1432
1423
  return {
1433
1424
  async launch(command, options, ctx) {
1434
1425
  const conn = await self.ensureConnected();
@@ -1628,7 +1619,7 @@ export class RemoteK8sExecutionEnv {
1628
1619
  const c = await this.ensureConnected();
1629
1620
  if (!c.ok)
1630
1621
  return c;
1631
- return this.runExec(command, { signal: abortSignal, timeoutMs: this.cfg.dataTimeoutMs });
1622
+ return this.runExec(command, { signal: abortSignal, capMs: this.cfg.dataTimeoutMs });
1632
1623
  }
1633
1624
  workingDirOf(cwd) {
1634
1625
  const dir = cwd ?? this.cwd;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@sema-agent/server",
3
- "version": "7.129.0",
3
+ "version": "7.129.1",
4
4
  "description": "Sema Server — the server/API implementation layer for Sema, wiring core, registry, model providers, and cloud agent execution. Built on @sema-agent/core.",
5
5
  "type": "module",
6
6
  "license": "BUSL-1.1",