@sema-agent/server 7.129.0 → 7.129.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/MIGRATION.md +22 -0
- package/USAGE.md +25 -0
- package/dist/boot/execution-env.js +1 -0
- package/dist/config-catalog.js +5 -1
- package/dist/config-types.d.ts +3 -0
- package/dist/config.js +7 -1
- package/dist/exec-bounds.d.ts +69 -0
- package/dist/exec-bounds.js +37 -0
- package/dist/plugins/remote-env-e2b.d.ts +31 -9
- package/dist/plugins/remote-env-e2b.js +77 -26
- package/dist/plugins/remote-env-host.js +7 -5
- package/dist/plugins/remote-env-k8s.d.ts +12 -4
- package/dist/plugins/remote-env-k8s.js +22 -31
- package/package.json +1 -1
package/MIGRATION.md
CHANGED
|
@@ -7,6 +7,28 @@
|
|
|
7
7
|
> ⚠️ 完整清单在仓库根 `CHANGELOG.md`——它**不随 npm tarball 出包**(本文件随包)。看完整迁移窗的
|
|
8
8
|
> 权威姿势是源码 tag diff:`git diff v<旧>..v<新>`(每版都推 `v<版本>` tag);npm 包页也镜像 CHANGELOG。
|
|
9
9
|
|
|
10
|
+
## 7.129.1 —— k8s / e2b 车道:带超时的命令不再被静默切断(行为修复)· 新旋钮 `K8S_LIVENESS_MS` · 两只活性旋钮非正值拒启
|
|
11
|
+
|
|
12
|
+
1. **一条命令的界只来自调用方说出口的那些**(零 DDL):墙钟 `timeout`、流式面自己的 `readTimeoutMs`;两样都没说时才由车道活性界(无输出满 N 毫秒即切,
|
|
13
|
+
缺省 120 s)兜底。界从调用开始算、覆盖全程(含 e2b 建命令那一段;那一段另受 e2b SDK 自己的建命令握手约束,缺省 60 s,比命令的界早时先由它切)。
|
|
14
|
+
引擎的 Bash 工具每一发都带超时,所以 Bash 调用从此不再被活性界切断(安静的长构建跑到它的超时)。`timedOut` / `timeoutSec` 仍不全是跑满:e2b 上建命令
|
|
15
|
+
卡住被那只 60 s 握手切掉、或沙箱中途死掉时,这一发仍以 `timeout` 码回来,Bash 照旧报「Command timed out after <N>s」(独立的码属于引擎的码表)。
|
|
16
|
+
活性切断的句子三车道一句,点名旋钮;错误码仍是引擎的 `timeout`。
|
|
17
|
+
- **新旋钮** `K8S_LIVENESS_MS`(ms,`E2B_LIVENESS_MS` 的孪生:同一只解析器;缺席 ⇒ 120 s;只在 `REMOTE_EXEC=k8s` 时求值)。两只旋钮都**只管没有显式超时的
|
|
18
|
+
命令** —— 此前为了让带超时的长构建跑完而调大 `E2B_LIVENESS_MS` 的部署,可以调回缺省(调大只会放宽引擎探针与文件读写的挂死护栏)。
|
|
19
|
+
- 🔴 **两只活性旋钮的值必须是正数**:非数、`0`、负数 ⇒ **拒启**,拒句点名旋钮(`E2B_LIVENESS_MS=0` 此前照收,结果是每条 e2b 命令都建不起来)。
|
|
20
|
+
升级前删掉这类设置或改成正数。嵌入方直接构造 k8s / e2b 执行环境时传非正的 `livenessMs` ⇒ 构造即抛。
|
|
21
|
+
- **受伤方**:依赖「120 s 无输出即杀」兜底带超时调用的部署 —— 带超时的挂死命令要等到它自己的超时(Bash 不传参数时 120 s,与改前同;显式最长到引擎上限)。
|
|
22
|
+
出路:给命令更短的超时;不带超时的调用照旧受活性界,可调小两只旋钮。
|
|
23
|
+
- **受伤方(调大过引擎的 `BASH_DEFAULT_TIMEOUT_MS` 的部署)**:每一发不传超时参数的 Bash 都带着这个调大了的超时,从此失去 120 s 的挂死护栏 —— 挂死的命令
|
|
24
|
+
要等满它,且没有本服务的旋钮能补回。出路:把 `BASH_DEFAULT_TIMEOUT_MS` 调回,或给需要护栏的命令显式的短超时。
|
|
25
|
+
- **读面**:e2b 车道的墙钟切断、活性切断与建命令期限到点此前报 `unknown`,现在报 `timeout`(e2b 上把这类切断当 `unknown` 读的消费方改读 `timeout`);
|
|
26
|
+
k8s 流式 exec 此前忽略 `timeout`,现在照它切。
|
|
27
|
+
- **e2b 建命令被切之后才到的句柄**:一到即被杀、输出丢弃;在**看见**它不在跑之前(供应商答了杀,或 `wait()` 以成功 / 非零退出结算 —— 流先断了、连接期限到点不算看见,进程哪怕已死也继续持有)该环境的挂起(`suspendVM`)照旧以
|
|
28
|
+
`command_in_flight` 拒。杀被供应商拒时记一行 warn `e2b_abandoned_start_kill_failed`(沙箱 id / 进程号 / 拒因类名),继续等看见退出;杀被拒且退出看不见的环境从此不能挂起、VM 计费到任务结束(fail-closed),出路 = 让命令退出或结束任务。建命令一直悬着(供应商既不交回
|
|
29
|
+
句柄、也不报失败)、或杀被拒而进程不退出的那一种,挂起会一直拒(说不出那条命令没在跑,不快照在跑的命令);任务结束销毁环境不受影响。建命令**以失败
|
|
30
|
+
告终**(含 SDK 自己的 60 s 握手到点)时在途保护即撤,与 7.129.0 同 —— 供应商若其实已起了那个进程、只是回执丢了,它不受这道保护。
|
|
31
|
+
|
|
10
32
|
## 7.129.0 —— 引擎 7.51.2 提货 · 会话规则一种文法与列改名(🔴 先停旧副本再起新)· 卡词换与存量一次性改写 · 读拒三席热生效 · 直连不跟重定向 · 两只告警标签改名
|
|
11
33
|
|
|
12
34
|
1. **引擎 7.51.2(精确钉;7.51.0 的硬破坏 + 7.51.1 / 7.51.2 两补丁)**:本服务随引擎的硬破坏在下列第 2–13 条;引擎自己的收窄直接落到部署上的见第 14 条。DDL 只有第 2 条的一处列改名(SQL 后端,由起服一次性迁移做,不用手跑);第 3 条的存量改写同样由起服一次性迁移做。
|
package/USAGE.md
CHANGED
|
@@ -2315,6 +2315,31 @@ k8s 车道每条 run 一只新 pod(pod 寿命 = 泄漏护栏,不跨 run 复用)
|
|
|
2315
2315
|
|
|
2316
2316
|
自检是**读数,不是边界**:起服之后加入的节点在下一次起服读到;射程里的三面(所在节点、宿主 DNAT、CNI 不执行)读不出,不在闭集里。
|
|
2317
2317
|
|
|
2318
|
+
### 9.3f 沙箱车道的命令静默切断:显式超时优先(`K8S_LIVENESS_MS`,7.129.1 起)
|
|
2319
|
+
|
|
2320
|
+
k8s 与 e2b 两条沙箱车道各有一只**活性界**:一条命令无输出满这么久(缺省 120 s)即判挂死、切断。一条命令由它的调用方**说出口**的界约束,活性界只在调用方什么都没说时兜底:
|
|
2321
|
+
|
|
2322
|
+
- **带显式超时 ⇒ 只受这个超时约束**,无输出多久都不切。引擎的 Bash 工具每一发都带超时(`timeout` 参数;不传时引擎给缺省值 120 s,上限 600 s)—— 所以 Bash 调用一律走这一条。
|
|
2323
|
+
引擎的缺省值可由进程 env `BASH_DEFAULT_TIMEOUT_MS` 调大:调大它就是调大每一发不传参数的 Bash 的挂死护栏(挂死的命令要等满它),本服务没有旋钮能再收紧 —— 要护栏就调回它,或给命令显式的短超时。
|
|
2324
|
+
安静的长构建(Gradle / R8 一段几分钟不出声)给命令一个够长的 `timeout` 即可;超出引擎上限的工作用 `run_in_background`。
|
|
2325
|
+
- 流式调用自己说了静默界(`readTimeoutMs`)⇒ 照它。
|
|
2326
|
+
- 两样都没说(引擎自己的 git 探针、文件读写这类)⇒ 车道活性界兜底,它是这条命令唯一的挂死护栏。
|
|
2327
|
+
|
|
2328
|
+
界从调用开始算,覆盖命令全程 —— 含 e2b 供应商建命令那一段(命令还没起跑,自然没有输出):建命令卡住按这条命令最早的那一界切(说了超时按超时,没说界按活性界)。
|
|
2329
|
+
e2b SDK 自己的建命令握手另有一界(缺省 60 s),比命令的界早时先由它切:超时或活性界长于 60 s 的命令(含缺省 120 s 的活性界)建命令卡住在约 60 s 被切 —— 缓冲 exec(Bash 走的这一面)报 `timeout`,流式 exec 报 `connect_failed`。
|
|
2330
|
+
|
|
2331
|
+
| 旋钮 | 车道 | 缺省 | 说明 |
|
|
2332
|
+
|---|---|---|---|
|
|
2333
|
+
| `K8S_LIVENESS_MS` | `REMOTE_EXEC=k8s` | 缺席 ⇒ 120000(ms) | 7.129.1 起;与 `E2B_LIVENESS_MS` 同一只解析器:必须是正数,非数 / `0` / 负数拒启;只在本车道被选中时求值 |
|
|
2334
|
+
| `E2B_LIVENESS_MS` | `REMOTE_EXEC=e2b` | 缺席 ⇒ 120000(ms) | 同上(7.129.1 起 `0` / 负数拒启;此前照收,结果是每条命令建不起来) |
|
|
2335
|
+
|
|
2336
|
+
**有显式超时时这两只旋钮不参与**:调大不会让一条带超时的 Bash 跑得更久,调小也不会更早切它 —— 要更早切就给命令更短的超时。
|
|
2337
|
+
|
|
2338
|
+
切断时报的是真正触发的那一界:活性切断的句子是 `no output for <N> s — cut off by the exec liveness bound (set K8S_LIVENESS_MS or give the command an explicit timeout)`
|
|
2339
|
+
(e2b 点名 `E2B_LIVENESS_MS`);错误码沿用引擎的 `timeout`(引擎码表把墙钟与活性两种超界归同一词,靠句子区分)。Bash 工具结果里的 `timedOut: true` / `timeoutSec: N`
|
|
2340
|
+
不再是活性切断的投影,但仍不全是「跑满了 N 秒」:e2b 上建命令卡住被那只 60 s 握手切掉、或沙箱中途死掉时,这一发仍以 `timeout` 码回来,Bash 照旧报
|
|
2341
|
+
「Command timed out after <N>s」(把它们与超时分开的独立码属于引擎的码表)。host 车道没有活性界(命令只受调用方的超时,缓冲 exec 不带超时时受 `HOST_COMMAND_TIMEOUT_MS`)。
|
|
2342
|
+
|
|
2318
2343
|
### 9.4 auto 模式(`permissionMode:"auto"`):武装三态、本地 org deny、自查(与 core 的意图武装式合修)
|
|
2319
2344
|
|
|
2320
2345
|
**背景**:此前引擎只在「组织授予」(web-admin 解出的 per-principal `runtimeCaps.autoMode === true`)时武装
|
|
@@ -222,6 +222,7 @@ export function createExecutionEnv(ctx) {
|
|
|
222
222
|
...(config.remoteExec.runtimeClass != null ? { runtimeClass: config.remoteExec.runtimeClass } : {}),
|
|
223
223
|
...(config.remoteExec.mountPath ? { mountPath: config.remoteExec.mountPath } : {}),
|
|
224
224
|
...(config.remoteExec.timeoutMs != null ? { timeoutMs: config.remoteExec.timeoutMs } : {}),
|
|
225
|
+
...(config.remoteExec.livenessMs != null ? { livenessMs: config.remoteExec.livenessMs } : {}),
|
|
225
226
|
dataTimeoutMs: workspaceSnapshotTimeoutOf(config.remoteExec),
|
|
226
227
|
...(config.remoteExec.memory ? { memory: config.remoteExec.memory } : {}),
|
|
227
228
|
...(config.remoteExec.cpu ? { cpu: config.remoteExec.cpu } : {}),
|
package/dist/config-catalog.js
CHANGED
|
@@ -5,6 +5,7 @@ import { CENTER_CREDENTIAL_RENEW_LEAD_DEFAULT_MS, CENTER_CREDENTIAL_RENEW_LEAD_M
|
|
|
5
5
|
import { SANDBOX_IPV6_DEFAULT, SANDBOX_IPV6_WORDS } from "./sandbox-egress-policy.js";
|
|
6
6
|
import { WORKSPACE_CACHE_DEFAULT, WORKSPACE_CACHE_MOUNT_DEFAULT, WORKSPACE_CACHE_SIZE_DEFAULT, WORKSPACE_CACHE_WORDS } from "./workspace-cache.js";
|
|
7
7
|
import { WORKSPACE_PERSIST_DEFAULT, WORKSPACE_PERSIST_WORDS, WORKSPACE_SNAPSHOT_KEEP_DEFAULT, WORKSPACE_SNAPSHOT_KEEP_RANGE, WORKSPACE_SNAPSHOT_TIMEOUT_MS_DEFAULT, WORKSPACE_SNAPSHOT_TIMEOUT_MS_RANGE } from "./workspace-space.js";
|
|
8
|
+
import { LIVENESS_MS_DEFAULT } from "./exec-bounds.js";
|
|
8
9
|
import { THINKING_FORMATS } from "./model-compat.js";
|
|
9
10
|
import { AUTO_MODE_CLASSIFIER_TIMEOUT_BOUNDS, CENTER_BASE_URL_RULE, CONFIG_PROVIDER_WORDS, DRAIN_SECOND_SIGNAL_MIN_GAP_BOUNDS, DRAIN_SECOND_SIGNAL_MIN_GAP_DEFAULT_MS, FIRST_BYTE_TIMEOUT_BOUNDS, FLEET_LEASE_OUTAGE_MAX_DEFAULT_MS, RESILIENCE_DEFAULTS, TASK_WRITE_FACE_ESCALATIONS } from "./config-types.js";
|
|
10
11
|
import { webSearchProviderFromEnv, WEB_SEARCH_PROVIDERS } from "./plugins/web-search.js";
|
|
@@ -30,6 +31,8 @@ const RESTART_SLICE_SET = {
|
|
|
30
31
|
};
|
|
31
32
|
export const CATALOG_RESTART_SLICES = Object.keys(RESTART_SLICE_SET);
|
|
32
33
|
const r = (name, domain, type, semantics, opts = {}) => ({ name, domain, type, semantics, ...opts });
|
|
34
|
+
const LIVENESS_KNOB_SEMANTICS = (lane) => `${lane} 命令静默活性上限(ms):无输出满这么久即判挂死切断,只管没有显式超时的命令 —— 带显式超时的命令只受那个超时约束(安静的长构建给命令一个显式超时即可,不必调大本值)`;
|
|
35
|
+
const LIVENESS_KNOB_DEFAULT_NOTE = `缺席 ⇒ adapter 默认 ${LIVENESS_MS_DEFAULT / 1000}s`;
|
|
33
36
|
const cLimits = (path, over = {}) => ({
|
|
34
37
|
domain: "limits",
|
|
35
38
|
path,
|
|
@@ -138,7 +141,7 @@ export const CONFIG_CATALOG = [
|
|
|
138
141
|
r("E2B_BG_DEFAULT_TIMEOUT_SEC", "orchestration", "number", "e2b 后台 shell 缺省超时(秒)", { staticDefault: "300" }),
|
|
139
142
|
r("E2B_BG_MAX_CONCURRENT", "orchestration", "number", "e2b 后台 shell 并发帽", { staticDefault: "8" }),
|
|
140
143
|
r("E2B_BG_MAX_TIMEOUT_SEC", "orchestration", "number", "e2b 后台 shell 超时上限(秒)", { staticDefault: "1800" }),
|
|
141
|
-
r("E2B_LIVENESS_MS", "orchestration", "number", "e2b
|
|
144
|
+
r("E2B_LIVENESS_MS", "orchestration", "number", LIVENESS_KNOB_SEMANTICS("e2b"), { derivedDefaultNote: LIVENESS_KNOB_DEFAULT_NOTE }),
|
|
142
145
|
r("E2B_MOUNT_PATH", "orchestration", "path", "e2b 工作区根(resume 用当前值,改前先 drain 挂起任务)"),
|
|
143
146
|
r("E2B_ORPHAN_RECLAIM", "orchestration", "boolean", "崩溃遗留沙盒回收腿(boot 一次+周期;判据保守,拿不准恒不杀)", { staticDefault: "true", danger: BILL }),
|
|
144
147
|
r("E2B_SANDBOX_ENV", "orchestration", "secret", "转发进 e2b 沙箱的本进程 env 点名表(NAME,NAME;点到的键从 worker env 原样取,值不进 spec/center/事件日志;与 LSP_ENV_ALLOW / DOCKER_SANDBOX_ENV 同一只解析器与「点名放行」谓词:形错 / 点名名单型旋钮拒启且不回显值;本进程环境里没有的名按序号 warn、不转发;只在本车道被选中时求值;原文按凭据类对待:只回在场位、不进任何子进程)", { danger: SEC }),
|
|
@@ -191,6 +194,7 @@ export const CONFIG_CATALOG = [
|
|
|
191
194
|
r("K8S_CA_CERT_B64", "orchestration", "opaque", "k8s API CA 证书(base64;registry secrets 无换行约束)", { danger: SEC }),
|
|
192
195
|
r("K8S_CPU", "orchestration", "string", "沙箱 pod CPU 限额(profile 数据)"),
|
|
193
196
|
r("K8S_INSECURE_TLS", "orchestration", "boolean", "跳过 k8s API TLS 校验", { staticDefault: "false", danger: SEC }),
|
|
197
|
+
r("K8S_LIVENESS_MS", "orchestration", "number", LIVENESS_KNOB_SEMANTICS("k8s"), { derivedDefaultNote: LIVENESS_KNOB_DEFAULT_NOTE }),
|
|
194
198
|
r("K8S_MEMORY", "orchestration", "string", "沙箱 pod 内存限额(2Gi 默认曾被真 OOM)"),
|
|
195
199
|
r("K8S_MOUNT_PATH", "orchestration", "path", "k8s 工作区挂载路径"),
|
|
196
200
|
r("K8S_NAMESPACE", "orchestration", "string", "沙箱 pod namespace"),
|
package/dist/config-types.d.ts
CHANGED
|
@@ -898,6 +898,9 @@ export interface ServiceConfigFlat {
|
|
|
898
898
|
runtimeClass?: string;
|
|
899
899
|
mountPath?: string;
|
|
900
900
|
timeoutMs?: number;
|
|
901
|
+
/** `K8S_LIVENESS_MS` — the lane's liveness bound (ms; the `E2B_LIVENESS_MS` twin, same parser; absent ⇒ the adapter's
|
|
902
|
+
* default 120 s): cuts a command silent for this long ONLY when its caller stated no timeout (`exec-bounds.ts`). */
|
|
903
|
+
livenessMs?: number;
|
|
901
904
|
/** Resource-profile knobs: sandbox pod limits (adapter defaults: 2Gi / 1 cpu). */
|
|
902
905
|
memory?: string;
|
|
903
906
|
cpu?: string;
|
package/dist/config.js
CHANGED
|
@@ -27,6 +27,7 @@ import { bakeOperatorPrincipalsRefusal, bakeRunnerCredentialRefusal, centerIdent
|
|
|
27
27
|
import { declaredSemaConfigDir } from "./sema-config-dir.js";
|
|
28
28
|
import { lkgPathOverrideOf } from "./config-lkg.js";
|
|
29
29
|
import { K8S_WORKSPACE_MOUNT_DEFAULT, WORKSPACE_PERSIST_DEFAULT, WORKSPACE_PERSIST_WORDS, WORKSPACE_SNAPSHOT_KEEP_DEFAULT, WORKSPACE_SNAPSHOT_KEEP_RANGE, WORKSPACE_SNAPSHOT_TIMEOUT_MS_DEFAULT, WORKSPACE_SNAPSHOT_TIMEOUT_MS_RANGE, parseWorkspaceSnapshotExclude } from "./workspace-space.js";
|
|
30
|
+
import { laneLivenessOf } from "./exec-bounds.js";
|
|
30
31
|
import { SANDBOX_IPV6_DEFAULT, SANDBOX_IPV6_WORDS } from "./sandbox-egress-policy.js";
|
|
31
32
|
import { WORKSPACE_CACHE_DEFAULT, WORKSPACE_CACHE_MOUNT_DEFAULT, WORKSPACE_CACHE_SIZE_DEFAULT, WORKSPACE_CACHE_WORDS, isK8sQuantity, mountsOverlap } from "./workspace-cache.js";
|
|
32
33
|
import { REMOTE_EXEC_PROVIDER_WORDS, executionLaneOf, executionLaneRequiredRefusal, isSingleMachineDefaultLane, laneRequiredEnv } from "./execution-lane-caps.js";
|
|
@@ -333,6 +334,10 @@ function optNumEnvStrict(name) {
|
|
|
333
334
|
return undefined;
|
|
334
335
|
return parseNumOrFail(name, raw);
|
|
335
336
|
}
|
|
337
|
+
function livenessMsEnv(name) {
|
|
338
|
+
const n = optNumEnvStrict(name);
|
|
339
|
+
return n === undefined ? undefined : laneLivenessOf(n, name).ms;
|
|
340
|
+
}
|
|
336
341
|
function parseDeviceEnrollTokenTtlEnv() {
|
|
337
342
|
const raw = process.env.DEVICE_ENROLL_TOKEN_TTL_SEC;
|
|
338
343
|
if (raw === undefined || raw === "")
|
|
@@ -1836,7 +1841,7 @@ function parseOrchestrationDomain(ctx) {
|
|
|
1836
1841
|
apiKey: process.env.E2B_API_KEY,
|
|
1837
1842
|
...(process.env.E2B_TEMPLATE ? { template: process.env.E2B_TEMPLATE } : {}),
|
|
1838
1843
|
...((n) => (n !== undefined ? { timeoutMs: n } : {}))(optNumEnvStrict("E2B_TIMEOUT_MS")),
|
|
1839
|
-
...((n) => (n !== undefined ? { livenessMs: n } : {}))(
|
|
1844
|
+
...((n) => (n !== undefined ? { livenessMs: n } : {}))(livenessMsEnv("E2B_LIVENESS_MS")),
|
|
1840
1845
|
...(process.env.E2B_ALLOW_NET ? { allowInternetAccess: boolEnv("E2B_ALLOW_NET", true) } : {}),
|
|
1841
1846
|
...((names) => (names !== undefined ? { sandboxEnv: allowlistedEnv(names, process.env) } : {}))(parseEnvNameAllowlist("E2B_SANDBOX_ENV", singleUserTurnkey)),
|
|
1842
1847
|
...(process.env.E2B_MOUNT_PATH ? { mountPath: process.env.E2B_MOUNT_PATH } : {}),
|
|
@@ -1855,6 +1860,7 @@ function parseOrchestrationDomain(ctx) {
|
|
|
1855
1860
|
...(process.env.K8S_RUNTIME_CLASS != null ? { runtimeClass: process.env.K8S_RUNTIME_CLASS } : {}),
|
|
1856
1861
|
...(process.env.K8S_MOUNT_PATH ? { mountPath: process.env.K8S_MOUNT_PATH } : {}),
|
|
1857
1862
|
...((n) => (n !== undefined ? { timeoutMs: n } : {}))(optNumEnvStrict("K8S_TIMEOUT_MS")),
|
|
1863
|
+
...((n) => (n !== undefined ? { livenessMs: n } : {}))(livenessMsEnv("K8S_LIVENESS_MS")),
|
|
1858
1864
|
...(process.env.K8S_MEMORY ? { memory: process.env.K8S_MEMORY } : {}),
|
|
1859
1865
|
...(process.env.K8S_CPU ? { cpu: process.env.K8S_CPU } : {}),
|
|
1860
1866
|
sandboxIpv6: enumEnv("K8S_SANDBOX_IPV6", SANDBOX_IPV6_DEFAULT, SANDBOX_IPV6_WORDS),
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* 执行车道上「什么约束一条命令」的**唯一一条律**(host / e2b / k8s 三只 adapter 同读这一只;纯叶,零依赖):
|
|
3
|
+
*
|
|
4
|
+
* 命令由调用方**说出口**的界约束 —— 墙钟 `timeout`(buffered 与流式两面)与 / 或它自己的静默界 `readTimeoutMs`
|
|
5
|
+
* (流式面)。调用方**两样都没说**时,才由车道自己的活性界兜底:无输出满这么久即判挂死(k8s `K8S_LIVENESS_MS` /
|
|
6
|
+
* e2b `E2B_LIVENESS_MS`,缺省 {@link LIVENESS_MS_DEFAULT};host 没有这一界)—— 那时它是这条命令唯一的挂死护栏。
|
|
7
|
+
* 切断时报的是**真正触发的那一界**(活性切断的句子只在 {@link livenessCutMessage} 铸)。
|
|
8
|
+
*
|
|
9
|
+
* 为什么「说了墙钟就不再活性切断」:墙钟本身就是挂死护栏(它到点必切),而调用方说出一个墙钟,就是在说「这条命令可以跑
|
|
10
|
+
* 这么久」—— 安静的长构建(Gradle / R8 一段几分钟不出声)正是这种命令。两界并存时,活性界会先于墙钟切掉一条按约正在跑的
|
|
11
|
+
* 命令,而调用方(core 的 Bash 工具)只知道自己说过的墙钟,便把这次切断报成「跑满了 N 秒」。两界互斥之后,一次切断只可能
|
|
12
|
+
* 是它报的那一界。
|
|
13
|
+
*
|
|
14
|
+
* 与 core 远端契约(`ExecStreamOptions.readTimeoutMs`:「缺省也 SHOULD 有活性界,一条命令不得等到环境死」)的对齐:
|
|
15
|
+
* 说了墙钟的命令被墙钟兜住,不会无限等;两界都没说的命令仍有车道活性界。切断码沿用 core 的 `timeout`(core 远端码表
|
|
16
|
+
* 把墙钟与活性两种超界都归这一词);两者靠句子区分 —— 独立的活性码属于 core 的码表。
|
|
17
|
+
*
|
|
18
|
+
* 不在本律里的界(adapter 自己的上限,不是调用方说的话,不关活性界):k8s 文件读写的数据上限 `dataTimeoutMs`、e2b 交给
|
|
19
|
+
* 供应商的沙箱寿命上限。adapter 自己的整段数据搬运(k8s 快照 tar 上 / 下)天生静默,它们**自己说**一个墙钟(数据上限),
|
|
20
|
+
* 落在本律的第一支。
|
|
21
|
+
*/
|
|
22
|
+
/** 车道活性界的缺省(ms)—— k8s / e2b 两只 adapter 与配置目录的「缺席 ⇒ 默认」说法同读这一只。 */
|
|
23
|
+
export declare const LIVENESS_MS_DEFAULT = 120000;
|
|
24
|
+
/** 有活性界的车道各自的旋钮名(闭集;切断句点名它,运维据此知道调哪一只)。 */
|
|
25
|
+
export type LivenessKnob = "K8S_LIVENESS_MS" | "E2B_LIVENESS_MS";
|
|
26
|
+
/** 一条车道自己的活性界:k8s / e2b 有,host 没有(传 `undefined`)。只经 {@link laneLivenessOf} 铸(值已判过)。 */
|
|
27
|
+
export interface LaneLiveness {
|
|
28
|
+
readonly ms: number;
|
|
29
|
+
readonly knob: LivenessKnob;
|
|
30
|
+
}
|
|
31
|
+
/**
|
|
32
|
+
* 车道活性界的唯一铸点与唯一判据:值必须是有限正数(ms),否则**拒**(抛,句子点名旋钮)。活性界是不说界的命令唯一的挂死护栏 ——
|
|
33
|
+
* `0` / 负数会把它整个撤掉(那条命令从此无界),非数无从计时。boot 的旋钮解析(`E2B_LIVENESS_MS` / `K8S_LIVENESS_MS`)与 adapter
|
|
34
|
+
* 构造(嵌入方直接传 `livenessMs`)都经这一只判,一个判据、一句拒词。
|
|
35
|
+
*/
|
|
36
|
+
export declare function laneLivenessOf(ms: number, knob: LivenessKnob): LaneLiveness;
|
|
37
|
+
/** 本条命令的静默界:调用方说的(`readTimeoutMs`),或车道兜底的。 */
|
|
38
|
+
export type LivenessBound = {
|
|
39
|
+
readonly ms: number;
|
|
40
|
+
readonly setBy: "caller";
|
|
41
|
+
} | {
|
|
42
|
+
readonly ms: number;
|
|
43
|
+
readonly setBy: "lane";
|
|
44
|
+
readonly knob: LivenessKnob;
|
|
45
|
+
};
|
|
46
|
+
/** 一条命令最终受的界:墙钟(调用方说的)与静默界,各自可缺。 */
|
|
47
|
+
export interface CommandBounds {
|
|
48
|
+
readonly wallMs: number | undefined;
|
|
49
|
+
readonly liveness: LivenessBound | undefined;
|
|
50
|
+
}
|
|
51
|
+
/** 调用方说出口的界(ms)。`timeoutMs` = 墙钟;`readTimeoutMs` = 它自己的静默界。 */
|
|
52
|
+
export interface StatedBounds {
|
|
53
|
+
readonly timeoutMs?: number | undefined;
|
|
54
|
+
readonly readTimeoutMs?: number | undefined;
|
|
55
|
+
}
|
|
56
|
+
/** 本律的唯一实现(见文件头注)。 */
|
|
57
|
+
export declare function commandBoundsOf(stated: StatedBounds, lane: LaneLiveness | undefined): CommandBounds;
|
|
58
|
+
/**
|
|
59
|
+
* 一条**还没出过一个字节**的命令最早被哪一界切:墙钟与静默界里较早的那只(都从调用开始算)。供应商建命令那一段(命令还没起跑,
|
|
60
|
+
* 自然也没有输出)就受它约束 —— 说了墙钟的命令不会被车道活性界在建命令时切掉;没说界的命令建命令卡住按活性界切 —— 但供应商自己的建命令期限
|
|
61
|
+
* 先到就先切(e2b SDK 的 60 s 握手:到点报的是供应商那一界,不是本模块的任何一只)。两只都缺 ⇒ 无界(只剩供应商自己的界)。
|
|
62
|
+
*/
|
|
63
|
+
export declare function firstBoundOf(bounds: CommandBounds): {
|
|
64
|
+
readonly ms: number;
|
|
65
|
+
readonly cut: "wall" | LivenessBound;
|
|
66
|
+
} | undefined;
|
|
67
|
+
/** 活性切断的句子 —— 三车道唯一铸点(车道兜底的界点名旋钮与出路;调用方自己说的界点名那一参数)。 */
|
|
68
|
+
export declare function livenessCutMessage(bound: LivenessBound): string;
|
|
69
|
+
//# sourceMappingURL=exec-bounds.d.ts.map
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
export const LIVENESS_MS_DEFAULT = 120_000;
|
|
2
|
+
export function laneLivenessOf(ms, knob) {
|
|
3
|
+
if (!(Number.isFinite(ms) && ms > 0)) {
|
|
4
|
+
throw new Error(`${knob}=${ms} must be a positive number of milliseconds — it is the only hang guard of a command whose caller states no timeout, and a zero, negative or non-numeric value would remove it`);
|
|
5
|
+
}
|
|
6
|
+
return { ms, knob };
|
|
7
|
+
}
|
|
8
|
+
const said = (ms) => (ms !== undefined && Number.isFinite(ms) && ms > 0 ? ms : undefined);
|
|
9
|
+
export function commandBoundsOf(stated, lane) {
|
|
10
|
+
const wallMs = said(stated.timeoutMs);
|
|
11
|
+
const idleMs = said(stated.readTimeoutMs);
|
|
12
|
+
if (idleMs !== undefined)
|
|
13
|
+
return { wallMs, liveness: { ms: idleMs, setBy: "caller" } };
|
|
14
|
+
if (wallMs !== undefined || lane === undefined)
|
|
15
|
+
return { wallMs, liveness: undefined };
|
|
16
|
+
return { wallMs, liveness: { ms: lane.ms, setBy: "lane", knob: lane.knob } };
|
|
17
|
+
}
|
|
18
|
+
export function firstBoundOf(bounds) {
|
|
19
|
+
const { wallMs, liveness } = bounds;
|
|
20
|
+
if (liveness !== undefined && (wallMs === undefined || liveness.ms <= wallMs))
|
|
21
|
+
return { ms: liveness.ms, cut: liveness };
|
|
22
|
+
return wallMs === undefined ? undefined : { ms: wallMs, cut: "wall" };
|
|
23
|
+
}
|
|
24
|
+
function spoken(ms) {
|
|
25
|
+
if (ms < 1000)
|
|
26
|
+
return `${ms} ms`;
|
|
27
|
+
return ms % 1000 === 0 ? `${ms / 1000} s` : `${(ms / 1000).toFixed(1)} s`;
|
|
28
|
+
}
|
|
29
|
+
export function livenessCutMessage(bound) {
|
|
30
|
+
switch (bound.setBy) {
|
|
31
|
+
case "lane":
|
|
32
|
+
return `no output for ${spoken(bound.ms)} — cut off by the exec liveness bound (set ${bound.knob} or give the command an explicit timeout)`;
|
|
33
|
+
case "caller":
|
|
34
|
+
return `no output for ${spoken(bound.ms)} — cut off by the command's own idle bound (readTimeoutMs)`;
|
|
35
|
+
}
|
|
36
|
+
}
|
|
37
|
+
//# sourceMappingURL=exec-bounds.js.map
|
|
@@ -71,8 +71,9 @@ export interface RemoteContainerEnvConfig extends RemoteEnvObservers {
|
|
|
71
71
|
* isRunning/files.*. A provider hang is surfaced as a typed retryable `RemoteExecutionError("timeout")` after
|
|
72
72
|
* this, instead of pinning the worker for the sandbox lifetime. Default {@link DEFAULT_RPC_TIMEOUT_MS}. */
|
|
73
73
|
rpcTimeoutMs?: number;
|
|
74
|
-
/** 🔴 (hang class-fix)
|
|
75
|
-
*
|
|
74
|
+
/** 🔴 (hang class-fix) The lane's idle/liveness bound (ms; env `E2B_LIVENESS_MS`): max time with NO output and not yet
|
|
75
|
+
* complete = suspected hang — cuts a running command ONLY when its caller stated neither a wall-clock (`timeout`) nor its own
|
|
76
|
+
* idle bound (`readTimeoutMs`); the rule: `exec-bounds.ts` (creation included). Positive only (`laneLivenessOf`). Default `LIVENESS_MS_DEFAULT`. */
|
|
76
77
|
livenessMs?: number;
|
|
77
78
|
/** 🔴 Wall-clock for data-transfer/provisioning RPCs (large file io, snapshot suspend/resume,
|
|
78
79
|
* create/connect) — wider than {@link rpcTimeoutMs} so a legit multi-minute transfer is not killed. Default
|
|
@@ -105,6 +106,8 @@ export declare class RemoteContainerExecutionEnv implements RemoteExecutionEnv,
|
|
|
105
106
|
readonly cdPath: "unknown";
|
|
106
107
|
private readonly cfg;
|
|
107
108
|
private readonly sandboxApi;
|
|
109
|
+
/** This lane's own liveness bound — what {@link commandBoundsOf} falls back to when a command's caller stated no bound. */
|
|
110
|
+
private readonly laneLiveness;
|
|
108
111
|
private sbx?;
|
|
109
112
|
/** 🔴 BL-31 keep-alive — the E2B sandbox lifetime is hard-capped (≈1h, {@link E2B_SANDBOX_MAX_MS}), but a long
|
|
110
113
|
* leader/repair run legitimately exceeds it. This interval RE-extends the sandbox (`setTimeout(timeoutMs)` resets
|
|
@@ -287,23 +290,42 @@ export declare class RemoteContainerExecutionEnv implements RemoteExecutionEnv,
|
|
|
287
290
|
* the adapter NEVER blind-retries). A caller AbortSignal still aborts independently (op surfaces it / "aborted").
|
|
288
291
|
*/
|
|
289
292
|
private rpc;
|
|
293
|
+
/**
|
|
294
|
+
* A command start that already returned to its caller (its creation deadline fired, or it failed): a handle the provider
|
|
295
|
+
* hands over LATER is killed — the caller was told the command was cut, so it must not run on. The rule for the in-flight
|
|
296
|
+
* count this start holds: it is released **once, and only when the command is confirmed not running** (council #4: never
|
|
297
|
+
* snapshot a running command). Confirmations: the provider answered `kill()` (`true` killed / `false` no such process);
|
|
298
|
+
* the handle's one `wait()` reports the process exited (a result, or `CommandExitError` for a non-zero exit). Not
|
|
299
|
+
* confirmations: a refused `kill()` (logged; the exit is still awaited), a `wait()` that ends without an exit (the
|
|
300
|
+
* connection deadline, a lost stream — the process may still run). With neither the count stays held and `suspendVM`
|
|
301
|
+
* keeps refusing `command_in_flight` — the same arm as a start that never settles. No start promise (the op never ran)
|
|
302
|
+
* ⇒ released now. A failed start is released at once too, as in 7.129.0 — and that one is NOT a confirmation: there is
|
|
303
|
+
* no handle to kill or wait on, and a start whose reply was lost (the SDK's own 60 s handshake) may have spawned the
|
|
304
|
+
* process. Closing it means classifying the start failure (provider refused ⇒ not started; lost reply ⇒ held) — open,
|
|
305
|
+
* not done here.
|
|
306
|
+
*/
|
|
307
|
+
private settleAbandonedStart;
|
|
290
308
|
/**
|
|
291
309
|
* Per-command wall-clock for the E2B SDK (seconds→ms). 🔴 This is a PHYSICAL CAP, NOT the hang
|
|
292
310
|
* detector. The old code returned `this.cfg.timeoutMs` (the sandbox lifetime) when no `timeout` was given AND
|
|
293
311
|
* had no liveness check → a hung command waited the WHOLE lifetime (~30 min, the original hang bug). The fix is
|
|
294
312
|
* NOT to shorten this cap (that would spuriously kill a long-but-progressing `npm ci`) — it's to add a LIVENESS
|
|
295
313
|
* bound ({@link cfg.livenessMs}) in exec/execStream that catches a *zero-progress* command in ~livenessMs,
|
|
296
|
-
* independent of this cap. So: explicit caller `timeout` wins
|
|
297
|
-
*
|
|
314
|
+
* independent of this cap. So: explicit caller `timeout` wins — and is then the command's bound (the liveness bound
|
|
315
|
+
* stays out, `exec-bounds.ts`); else the lifetime is the cap (a command can't outlive its VM) and the liveness bound
|
|
316
|
+
* is the hang detector.
|
|
298
317
|
*/
|
|
299
318
|
private commandTimeoutMs;
|
|
300
319
|
/**
|
|
301
|
-
* 🔴
|
|
320
|
+
* 🔴 Bounds for the buffered exec WAIT phase. `handle.wait()` is a long-poll with no read timeout —
|
|
302
321
|
* a command that hangs DURING execution (not creation) would otherwise wait the E2B command timeout (= the
|
|
303
|
-
* lifetime when none is set, the original hang bug).
|
|
304
|
-
* `
|
|
305
|
-
*
|
|
306
|
-
*
|
|
322
|
+
* lifetime when none is set, the original hang bug). With a `liveness` bound (the caller stated no `timeout` —
|
|
323
|
+
* `exec-bounds.ts`) it races wait() against an idle check: no stdout/stderr for `liveness.ms` ⇒ hung → kill + typed
|
|
324
|
+
* retryable "timeout" with the shared cut sentence. `lastProgress()` is bumped by the stdout/stderr callbacks, so a
|
|
325
|
+
* slow-but-PROGRESSING command (long `npm ci`) is never killed. Both cuts reject with an `ExecutionError` carrying
|
|
326
|
+
* its real code (exec returns it as-is) — a `RemoteExecutionError` here would reach {@link execError}'s text match
|
|
327
|
+
* and land as `unknown`. The rejection is armed BEFORE the kill: the kill rejects `wait()` too, and the race must
|
|
328
|
+
* see the bound, not the kill. (execStream has its own chunk-driven idle timer; this is the buffered-exec analogue.)
|
|
307
329
|
*/
|
|
308
330
|
private waitWithLiveness;
|
|
309
331
|
/** Lazily provision the VM on first use: a task that never touches the hand pays zero cost.
|
|
@@ -3,13 +3,13 @@ import { Sandbox, CommandExitError, FileType } from "e2b";
|
|
|
3
3
|
import { FileError, ExecutionError, RemoteExecutionError, RollingTailBuffer, markTruncated, } from "@sema-agent/core";
|
|
4
4
|
import { fileErrorFromExec } from "./remote-env-file-error.js";
|
|
5
5
|
import { numEnvOr, runSymlinkChain, shellQuote } from "./remote-shell.js";
|
|
6
|
+
import { commandBoundsOf, firstBoundOf, laneLivenessOf, livenessCutMessage, LIVENESS_MS_DEFAULT } from "../exec-bounds.js";
|
|
6
7
|
import { BackgroundShellManager } from "./background-shell-support.js";
|
|
7
8
|
const PROVIDER = "e2b";
|
|
8
9
|
const DEFAULT_MOUNT_PATH = "/home/user";
|
|
9
10
|
const DEFAULT_TIMEOUT_MS = 5 * 60_000;
|
|
10
11
|
const E2B_SANDBOX_MAX_MS = numEnvOr("E2B_SANDBOX_MAX_MS", 3_600_000, 60_000);
|
|
11
12
|
const DEFAULT_RPC_TIMEOUT_MS = 60_000;
|
|
12
|
-
const DEFAULT_LIVENESS_MS = 120_000;
|
|
13
13
|
const DEFAULT_DATA_TIMEOUT_MS = 5 * 60_000;
|
|
14
14
|
const E2B_BG_MAX_CONCURRENT = numEnvOr("E2B_BG_MAX_CONCURRENT", 8, 1);
|
|
15
15
|
const E2B_BG_DEFAULT_TIMEOUT_SEC = numEnvOr("E2B_BG_DEFAULT_TIMEOUT_SEC", 300, 1);
|
|
@@ -22,6 +22,7 @@ export class RemoteContainerExecutionEnv {
|
|
|
22
22
|
cdPath = "unknown";
|
|
23
23
|
cfg;
|
|
24
24
|
sandboxApi;
|
|
25
|
+
laneLiveness;
|
|
25
26
|
sbx;
|
|
26
27
|
keepAlive;
|
|
27
28
|
handle;
|
|
@@ -36,9 +37,10 @@ export class RemoteContainerExecutionEnv {
|
|
|
36
37
|
timeoutMs: Math.min(config.timeoutMs ?? DEFAULT_TIMEOUT_MS, E2B_SANDBOX_MAX_MS),
|
|
37
38
|
allowInternetAccess: config.allowInternetAccess ?? true,
|
|
38
39
|
rpcTimeoutMs: config.rpcTimeoutMs ?? DEFAULT_RPC_TIMEOUT_MS,
|
|
39
|
-
livenessMs: config.livenessMs ??
|
|
40
|
+
livenessMs: config.livenessMs ?? LIVENESS_MS_DEFAULT,
|
|
40
41
|
dataTimeoutMs: config.dataTimeoutMs ?? DEFAULT_DATA_TIMEOUT_MS,
|
|
41
42
|
};
|
|
43
|
+
this.laneLiveness = laneLivenessOf(this.cfg.livenessMs, "E2B_LIVENESS_MS");
|
|
42
44
|
this.cwd = this.cfg.mountPath;
|
|
43
45
|
this.homeDir = config.homeDir;
|
|
44
46
|
this.sandboxApi = deps.sandboxApi ?? Sandbox;
|
|
@@ -255,9 +257,14 @@ export class RemoteContainerExecutionEnv {
|
|
|
255
257
|
s.total += b.length;
|
|
256
258
|
};
|
|
257
259
|
let lastProgress = Date.now();
|
|
260
|
+
const startedAt = lastProgress;
|
|
261
|
+
const bounds = commandBoundsOf({ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined }, this.laneLiveness);
|
|
262
|
+
const first = firstBoundOf(bounds);
|
|
258
263
|
let handle;
|
|
264
|
+
let starting;
|
|
265
|
+
let startCut = false;
|
|
259
266
|
try {
|
|
260
|
-
handle = await this.rpc("commands.run", () => sbx.commands.run(command, {
|
|
267
|
+
handle = await this.rpc("commands.run", () => (starting = sbx.commands.run(command, {
|
|
261
268
|
background: true,
|
|
262
269
|
cwd: this.resolve(options?.cwd ?? this.cwd),
|
|
263
270
|
...(options?.env ? { envs: options.env } : {}),
|
|
@@ -265,6 +272,8 @@ export class RemoteContainerExecutionEnv {
|
|
|
265
272
|
? Math.max(this.commandTimeoutMs(options?.timeout), (E2B_BG_MAX_TIMEOUT_SEC + E2B_BG_PROVIDER_BACKSTOP_PAD_SEC) * 1000)
|
|
266
273
|
: this.commandTimeoutMs(options?.timeout),
|
|
267
274
|
onStdout: (d) => {
|
|
275
|
+
if (startCut)
|
|
276
|
+
return;
|
|
268
277
|
if (tee)
|
|
269
278
|
teePush(tee.stdout, d);
|
|
270
279
|
if (!detached) {
|
|
@@ -273,6 +282,8 @@ export class RemoteContainerExecutionEnv {
|
|
|
273
282
|
}
|
|
274
283
|
},
|
|
275
284
|
onStderr: (d) => {
|
|
285
|
+
if (startCut)
|
|
286
|
+
return;
|
|
276
287
|
if (tee)
|
|
277
288
|
teePush(tee.stderr, d);
|
|
278
289
|
if (!detached) {
|
|
@@ -280,11 +291,13 @@ export class RemoteContainerExecutionEnv {
|
|
|
280
291
|
options?.onStderr?.(d);
|
|
281
292
|
}
|
|
282
293
|
},
|
|
283
|
-
}), options?.abortSignal,
|
|
294
|
+
})), options?.abortSignal, first.ms);
|
|
284
295
|
}
|
|
285
296
|
catch (e) {
|
|
286
|
-
|
|
287
|
-
|
|
297
|
+
startCut = true;
|
|
298
|
+
this.settleAbandonedStart(starting);
|
|
299
|
+
const cut = e instanceof RemoteExecutionError && e.code === "timeout" && !options?.abortSignal?.aborted;
|
|
300
|
+
return { ok: false, error: cut ? new ExecutionError("timeout", boundCutMessage(first)) : this.execError(e, options?.abortSignal) };
|
|
288
301
|
}
|
|
289
302
|
const onAbort = () => { if (!detached)
|
|
290
303
|
void handle.kill().catch(() => { }); };
|
|
@@ -333,7 +346,8 @@ export class RemoteContainerExecutionEnv {
|
|
|
333
346
|
const fg = this.waitWithLiveness(handle, () => lastProgress, {
|
|
334
347
|
waitP,
|
|
335
348
|
disarmed: () => detached,
|
|
336
|
-
...(tee ? {
|
|
349
|
+
...(tee ? { wall: { totalMs: this.commandTimeoutMs(options?.timeout), leftMs: Math.max(0, this.commandTimeoutMs(options?.timeout) - (Date.now() - startedAt)) } } : {}),
|
|
350
|
+
liveness: bounds.liveness,
|
|
337
351
|
});
|
|
338
352
|
const winner = await (detachedP ? Promise.race([fg, detachedP]) : fg);
|
|
339
353
|
if (winner === "detached") {
|
|
@@ -385,6 +399,9 @@ export class RemoteContainerExecutionEnv {
|
|
|
385
399
|
};
|
|
386
400
|
let bytes = 0;
|
|
387
401
|
const cap = options?.maxOutputBytes;
|
|
402
|
+
const bounds = commandBoundsOf({ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined, readTimeoutMs: options?.readTimeoutMs }, this.laneLiveness);
|
|
403
|
+
const liveness = bounds.liveness;
|
|
404
|
+
const first = firstBoundOf(bounds);
|
|
388
405
|
const push = (type, data) => {
|
|
389
406
|
if (finished)
|
|
390
407
|
return;
|
|
@@ -394,18 +411,22 @@ export class RemoteContainerExecutionEnv {
|
|
|
394
411
|
};
|
|
395
412
|
this.activeCommands++;
|
|
396
413
|
let handle;
|
|
414
|
+
let starting;
|
|
397
415
|
try {
|
|
398
|
-
handle = await this.rpc("commands.run", () => sbx.commands.run(command, {
|
|
416
|
+
handle = await this.rpc("commands.run", () => (starting = sbx.commands.run(command, {
|
|
399
417
|
background: true,
|
|
400
418
|
cwd: this.resolve(options?.cwd ?? this.cwd),
|
|
401
419
|
...(options?.env ? { envs: options.env } : {}),
|
|
402
420
|
timeoutMs: this.commandTimeoutMs(options?.timeout),
|
|
403
421
|
onStdout: (d) => push("stdout", d),
|
|
404
422
|
onStderr: (d) => push("stderr", d),
|
|
405
|
-
}), options?.signal,
|
|
423
|
+
})), options?.signal, first.ms);
|
|
406
424
|
}
|
|
407
425
|
catch (e) {
|
|
408
|
-
|
|
426
|
+
finished = true;
|
|
427
|
+
this.settleAbandonedStart(starting);
|
|
428
|
+
if (e instanceof RemoteExecutionError && e.code === "timeout" && !options?.signal?.aborted)
|
|
429
|
+
throw new RemoteExecutionError("timeout", boundCutMessage(first), e);
|
|
409
430
|
throw this.remoteError(e, "connect_failed");
|
|
410
431
|
}
|
|
411
432
|
handle
|
|
@@ -424,7 +445,7 @@ export class RemoteContainerExecutionEnv {
|
|
|
424
445
|
exitCode = e.exitCode;
|
|
425
446
|
}
|
|
426
447
|
else {
|
|
427
|
-
failure = this.remoteError(e, "unknown");
|
|
448
|
+
failure = this.remoteError(e, providerTimedOut(e) ? "timeout" : "unknown");
|
|
428
449
|
}
|
|
429
450
|
finished = true;
|
|
430
451
|
signalReady();
|
|
@@ -465,13 +486,14 @@ export class RemoteContainerExecutionEnv {
|
|
|
465
486
|
const next = new Promise((resolve) => {
|
|
466
487
|
wake = resolve;
|
|
467
488
|
});
|
|
468
|
-
const idleMs = options?.readTimeoutMs ?? this.cfg.livenessMs;
|
|
469
489
|
let timer;
|
|
470
|
-
const idle =
|
|
471
|
-
|
|
472
|
-
|
|
490
|
+
const idle = liveness === undefined
|
|
491
|
+
? undefined
|
|
492
|
+
: new Promise((_, reject) => {
|
|
493
|
+
timer = setTimeout(() => reject(new RemoteExecutionError("timeout", livenessCutMessage(liveness))), liveness.ms);
|
|
494
|
+
});
|
|
473
495
|
try {
|
|
474
|
-
await Promise.race([next, idle]);
|
|
496
|
+
await (idle === undefined ? next : Promise.race([next, idle]));
|
|
475
497
|
}
|
|
476
498
|
catch (e) {
|
|
477
499
|
finished = true;
|
|
@@ -794,36 +816,59 @@ export class RemoteContainerExecutionEnv {
|
|
|
794
816
|
combined?.dispose();
|
|
795
817
|
}
|
|
796
818
|
}
|
|
819
|
+
settleAbandonedStart(starting) {
|
|
820
|
+
let held = true;
|
|
821
|
+
const release = () => {
|
|
822
|
+
if (!held)
|
|
823
|
+
return;
|
|
824
|
+
held = false;
|
|
825
|
+
this.activeCommands--;
|
|
826
|
+
};
|
|
827
|
+
if (starting === undefined)
|
|
828
|
+
return release();
|
|
829
|
+
void starting.then((late) => {
|
|
830
|
+
late.wait().then(release, (e) => {
|
|
831
|
+
if (e instanceof CommandExitError)
|
|
832
|
+
release();
|
|
833
|
+
});
|
|
834
|
+
late.kill().then(release, (e) => {
|
|
835
|
+
this.cfg.logger?.warn?.("e2b_abandoned_start_kill_failed", { sandboxId: this.sbx?.sandboxId, pid: late.pid, error: e instanceof Error ? e.name : typeof e });
|
|
836
|
+
});
|
|
837
|
+
}, release);
|
|
838
|
+
}
|
|
797
839
|
commandTimeoutMs(timeoutSec) {
|
|
798
840
|
return timeoutSec != null ? timeoutSec * 1000 : this.cfg.timeoutMs;
|
|
799
841
|
}
|
|
800
842
|
async waitWithLiveness(handle, lastProgress, opts) {
|
|
801
843
|
let timer;
|
|
802
844
|
let wallTimer;
|
|
845
|
+
const liveness = opts?.liveness;
|
|
803
846
|
const idle = new Promise((_, reject) => {
|
|
804
|
-
const arm = () => {
|
|
847
|
+
const arm = (bound) => {
|
|
805
848
|
timer = setTimeout(() => {
|
|
806
849
|
if (opts?.disarmed?.())
|
|
807
850
|
return;
|
|
808
851
|
const idleFor = Date.now() - lastProgress();
|
|
809
|
-
if (idleFor >=
|
|
852
|
+
if (idleFor >= bound.ms) {
|
|
853
|
+
reject(new ExecutionError("timeout", livenessCutMessage(bound)));
|
|
810
854
|
void handle.kill().catch(() => { });
|
|
811
855
|
this.cfg.metrics?.inc("remote_env_exec_idle_timeout_total", { provider: PROVIDER });
|
|
812
|
-
reject(new RemoteExecutionError("timeout", `exec idle > ${this.cfg.livenessMs}ms (provider hang / unreachable sandbox)`));
|
|
813
856
|
}
|
|
814
857
|
else {
|
|
815
|
-
arm();
|
|
858
|
+
arm(bound);
|
|
816
859
|
}
|
|
817
|
-
}, Math.max(
|
|
860
|
+
}, Math.max(bound.ms - (Date.now() - lastProgress()), 0));
|
|
818
861
|
};
|
|
819
|
-
|
|
820
|
-
|
|
862
|
+
if (liveness !== undefined)
|
|
863
|
+
arm(liveness);
|
|
864
|
+
if (opts?.wall !== undefined) {
|
|
865
|
+
const wall = opts.wall;
|
|
821
866
|
wallTimer = setTimeout(() => {
|
|
822
867
|
if (opts.disarmed?.())
|
|
823
868
|
return;
|
|
869
|
+
reject(new ExecutionError("timeout", boundCutMessage({ ms: wall.totalMs, cut: "wall" })));
|
|
824
870
|
void handle.kill().catch(() => { });
|
|
825
|
-
|
|
826
|
-
}, opts.wallMs);
|
|
871
|
+
}, wall.leftMs);
|
|
827
872
|
}
|
|
828
873
|
});
|
|
829
874
|
try {
|
|
@@ -905,7 +950,7 @@ export class RemoteContainerExecutionEnv {
|
|
|
905
950
|
const name = e?.name;
|
|
906
951
|
if (abortSignal?.aborted || name === "AbortError")
|
|
907
952
|
return new ExecutionError("aborted", msg);
|
|
908
|
-
if (
|
|
953
|
+
if (providerTimedOut(e))
|
|
909
954
|
return new ExecutionError("timeout", msg);
|
|
910
955
|
return new ExecutionError("unknown", msg, e instanceof Error ? e : undefined);
|
|
911
956
|
}
|
|
@@ -924,6 +969,12 @@ export function e2bExecutionEnvFactory(config, deps) {
|
|
|
924
969
|
function errMsg(e) {
|
|
925
970
|
return e instanceof Error ? e.message : String(e);
|
|
926
971
|
}
|
|
972
|
+
function boundCutMessage(first) {
|
|
973
|
+
return first.cut === "wall" ? `exec exceeded ${first.ms}ms` : livenessCutMessage(first.cut);
|
|
974
|
+
}
|
|
975
|
+
function providerTimedOut(e) {
|
|
976
|
+
return e?.name === "TimeoutError" || /timeout|timed out/i.test(errMsg(e));
|
|
977
|
+
}
|
|
927
978
|
function anySignal(signals) {
|
|
928
979
|
const ctor = AbortSignal;
|
|
929
980
|
if (typeof ctor.any === "function")
|
|
@@ -6,6 +6,7 @@ import { randomBytes } from "node:crypto";
|
|
|
6
6
|
import { openSync, closeSync, readSync, statSync, truncateSync, unlinkSync, mkdirSync, rmdirSync, mkdtempSync, existsSync, realpathSync, accessSync, constants as FS_CONST } from "node:fs";
|
|
7
7
|
import { StringDecoder } from "node:string_decoder";
|
|
8
8
|
import { armPipeDestroyGrace, numEnvOr, SYMLINK_CHAIN_MAX_HOPS } from "./remote-shell.js";
|
|
9
|
+
import { commandBoundsOf, livenessCutMessage } from "../exec-bounds.js";
|
|
9
10
|
import { hostBackgroundShellEnabled, hostExecSpoolEnabled } from "../config.js";
|
|
10
11
|
import { IS_WIN32, resolveHostShell, hostShell, pinHostShell, spawnGroupOptions, killTreeHard, killTreeSoft, hardKillGroupNow, collapseWin32EnvKeys, adoptHostForegroundChild } from "./host-platform.js";
|
|
11
12
|
import { BackgroundShellManager, seedMemStream, feedMemStream, drainMemStream } from "./background-shell-support.js";
|
|
@@ -810,8 +811,9 @@ export class RemoteHostExecutionEnv {
|
|
|
810
811
|
if (!ready.ok)
|
|
811
812
|
throw new RemoteExecutionError("connect_failed", ready.error.message, ready.error);
|
|
812
813
|
const cwd = this.resolve(options?.cwd ?? this.cwd);
|
|
813
|
-
const
|
|
814
|
-
const
|
|
814
|
+
const bounds = commandBoundsOf({ timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined, readTimeoutMs: options?.readTimeoutMs }, undefined);
|
|
815
|
+
const timeoutMs = bounds.wallMs;
|
|
816
|
+
const liveness = bounds.liveness;
|
|
815
817
|
const cap = options?.maxOutputBytes;
|
|
816
818
|
const queue = [];
|
|
817
819
|
let finished = false;
|
|
@@ -905,7 +907,7 @@ export class RemoteHostExecutionEnv {
|
|
|
905
907
|
throw failure;
|
|
906
908
|
if (finished)
|
|
907
909
|
break;
|
|
908
|
-
const waitMs =
|
|
910
|
+
const waitMs = liveness !== undefined ? Math.max(1, liveness.ms - (Date.now() - lastChunk)) : 30_000;
|
|
909
911
|
let waitTimer;
|
|
910
912
|
try {
|
|
911
913
|
await new Promise((r) => { wake = r; waitTimer = setTimeout(r, waitMs); });
|
|
@@ -914,10 +916,10 @@ export class RemoteHostExecutionEnv {
|
|
|
914
916
|
if (waitTimer !== undefined)
|
|
915
917
|
clearTimeout(waitTimer);
|
|
916
918
|
}
|
|
917
|
-
if (
|
|
919
|
+
if (liveness !== undefined && !finished && Date.now() - lastChunk >= liveness.ms) {
|
|
918
920
|
finished = true;
|
|
919
921
|
kill();
|
|
920
|
-
throw new RemoteExecutionError("timeout",
|
|
922
|
+
throw new RemoteExecutionError("timeout", livenessCutMessage(liveness));
|
|
921
923
|
}
|
|
922
924
|
}
|
|
923
925
|
while (queue.length)
|
|
@@ -38,7 +38,9 @@ export interface K8sEnvConfig {
|
|
|
38
38
|
readyTimeoutMs?: number;
|
|
39
39
|
/** Control-plane RPC wall-clock (pod create/get/delete). */
|
|
40
40
|
rpcTimeoutMs?: number;
|
|
41
|
-
/**
|
|
41
|
+
/** The lane's zero-progress liveness bound (env `K8S_LIVENESS_MS`; default {@link LIVENESS_MS_DEFAULT}): a command that
|
|
42
|
+
* produces no output for this long and has not finished is cut as a hang — ONLY when its caller stated neither a wall-clock
|
|
43
|
+
* (`timeout`) nor its own idle bound (`readTimeoutMs`); a stated bound replaces it (the rule: `exec-bounds.ts`). */
|
|
42
44
|
livenessMs?: number;
|
|
43
45
|
/** The ONE data bound (`WORKSPACE_SNAPSHOT_TIMEOUT_MS`; default = `WORKSPACE_SNAPSHOT_TIMEOUT_MS_DEFAULT`, owned by
|
|
44
46
|
* `workspace-space.ts`): the snapshot upload's and restore's wall-clock, the deadline this replica's commits write into
|
|
@@ -257,6 +259,8 @@ export declare class RemoteK8sExecutionEnv implements RemoteExecutionEnv, Backgr
|
|
|
257
259
|
/** The known occupant's incarnation (`undefined` = this run has no known occupant: a delegated child, an unread session, a direct construction) — the
|
|
258
260
|
* ownership mark of every snapshot id this env mints, both lanes ({@link mintSnapshotId}); the space lane's partition is the same reading. */
|
|
259
261
|
private readonly occupantIncarnation;
|
|
262
|
+
/** This lane's own liveness bound — what {@link commandBoundsOf} falls back to when a command's caller stated no bound. */
|
|
263
|
+
private readonly laneLiveness;
|
|
260
264
|
/** This env's space lane (resolved once from the policy + the occupant's incarnation; see {@link SpaceLane}). */
|
|
261
265
|
private readonly spaceLane;
|
|
262
266
|
/** `workspace_space_unavailable` is said once per env. */
|
|
@@ -551,8 +555,10 @@ export declare class RemoteK8sExecutionEnv implements RemoteExecutionEnv, Backgr
|
|
|
551
555
|
private execUrl;
|
|
552
556
|
/**
|
|
553
557
|
* Run a command over the exec WebSocket. Frames: byte0=channel (1 stdout / 2 stderr / 3 status JSON with the
|
|
554
|
-
* REAL exit code). Bounds
|
|
555
|
-
*
|
|
558
|
+
* REAL exit code). Bounds = {@link commandBoundsOf} over what the caller stated (`timeoutMs` wall / `readTimeoutMs` idle)
|
|
559
|
+
* and this lane's liveness — the one rule, decided here and nowhere else on this lane — plus the adapter's own ceiling
|
|
560
|
+
* `capMs` (a data bound, not a statement: it never turns the liveness bound off). A socket killed by any bound does NOT
|
|
561
|
+
* kill the remote process (disposable-pod posture, see header).
|
|
556
562
|
*/
|
|
557
563
|
private runExec;
|
|
558
564
|
exec(command: string, options?: ExecOpts): Promise<Result<{
|
|
@@ -654,7 +660,9 @@ export declare class RemoteK8sExecutionEnv implements RemoteExecutionEnv, Backgr
|
|
|
654
660
|
suffix?: string;
|
|
655
661
|
abortSignal?: AbortSignal;
|
|
656
662
|
}): Promise<Result<string, FileError>>;
|
|
657
|
-
/** exec with the data-transfer
|
|
663
|
+
/** exec with the data-transfer ceiling (large reads/writes legitimately run longer than a control op). The engine states no
|
|
664
|
+
* bound for a file op, so the lane liveness bound stays on (a stalled transfer is cut by it); the data bound is this
|
|
665
|
+
* adapter's own ceiling (`capMs`), not a statement. */
|
|
658
666
|
private execData;
|
|
659
667
|
/** 本腿只定工作目录(缺省 cwd 规则 + 绝对化);串的形状归 {@link buildCwdEnvCommand} / {@link createCwdGuardedCall}(S-410:cwd 不可用 ⇒ 命令不跑;
|
|
660
668
|
* 车PL:buffered `exec` 上守卫触发 ⇒ `spawn_error`)。 */
|
|
@@ -11,6 +11,7 @@ import { spaceCommitAlive } from "../workspace-space-commit.js";
|
|
|
11
11
|
import { incarnationLogPrefix, redactIncarnations } from "../session-incarnation.js";
|
|
12
12
|
import { WORKSPACE_CACHE_SPACE_LABEL, workspaceCacheClaimOf, workspaceCacheEnv } from "../workspace-cache.js";
|
|
13
13
|
import { recordFailOpen } from "../observability/fail-open.js";
|
|
14
|
+
import { commandBoundsOf, laneLivenessOf, livenessCutMessage, LIVENESS_MS_DEFAULT } from "../exec-bounds.js";
|
|
14
15
|
import { SANDBOX_IPV6_DEFAULT, sandboxEgressGaps, sandboxPodSysctls } from "../sandbox-egress-policy.js";
|
|
15
16
|
import { FileError, ExecutionError, RemoteExecutionError, withRetry, RollingTailBuffer, markTruncated, } from "@sema-agent/core";
|
|
16
17
|
import { fileErrorFromExec, classifyFsStderr } from "./remote-env-file-error.js";
|
|
@@ -141,6 +142,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
141
142
|
cfg;
|
|
142
143
|
deps;
|
|
143
144
|
occupantIncarnation;
|
|
145
|
+
laneLiveness;
|
|
144
146
|
spaceLane;
|
|
145
147
|
spaceUnavailableSaid = false;
|
|
146
148
|
unreadableOccupantSaid = false;
|
|
@@ -161,7 +163,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
161
163
|
timeoutMs: config.timeoutMs ?? 30 * 60_000,
|
|
162
164
|
readyTimeoutMs: config.readyTimeoutMs ?? 120_000,
|
|
163
165
|
rpcTimeoutMs: config.rpcTimeoutMs ?? 30_000,
|
|
164
|
-
livenessMs: config.livenessMs ??
|
|
166
|
+
livenessMs: config.livenessMs ?? LIVENESS_MS_DEFAULT,
|
|
165
167
|
dataTimeoutMs: dataTimeoutMsOf(config),
|
|
166
168
|
cpu: config.cpu ?? "1",
|
|
167
169
|
memory: config.memory ?? "2Gi",
|
|
@@ -176,6 +178,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
176
178
|
this.cwd = this.cfg.mountPath;
|
|
177
179
|
this.homeDir = config.homeDir;
|
|
178
180
|
this.deps = deps ?? {};
|
|
181
|
+
this.laneLiveness = laneLivenessOf(this.cfg.livenessMs, "K8S_LIVENESS_MS");
|
|
179
182
|
const occupant = occupantKeyOf(config.spaceOccupant);
|
|
180
183
|
this.occupantIncarnation = "incarnation" in occupant ? occupant.incarnation : undefined;
|
|
181
184
|
this.spaceLane = spaceLaneOf(config.s3Snapshot?.space, config.spaceOccupant);
|
|
@@ -875,7 +878,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
875
878
|
const excludes = (this.cfg.snapshotExclude ?? []).map((item) => ` --exclude=${shellQuote(`./${item}`)}`).join("");
|
|
876
879
|
const cmd = `tar czf ${tmp}${excludes} -C ${shellQuote(this.cfg.mountPath)} . && curl -fsS -X PUT -T ${tmp} ${shellQuote(url)}; ` +
|
|
877
880
|
`rc=$?; rm -f ${tmp}; exit $rc`;
|
|
878
|
-
const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs
|
|
881
|
+
const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs });
|
|
879
882
|
if (!r.ok)
|
|
880
883
|
return { ok: false, error: new RemoteExecutionError("connect_failed", `workspace snapshot upload failed: ${this.redactUrl(r.error.message, url)}`, r.error) };
|
|
881
884
|
if (r.value.exitCode !== 0)
|
|
@@ -911,7 +914,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
911
914
|
const c = await this.provision(abortSignal);
|
|
912
915
|
if (!c.ok)
|
|
913
916
|
return c;
|
|
914
|
-
const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs
|
|
917
|
+
const r = await this.runExec(cmd, { timeoutMs: this.cfg.dataTimeoutMs });
|
|
915
918
|
const failure = !r.ok
|
|
916
919
|
? { cls: r.error.code === "timeout" ? "timeout" : "transport", detail: `workspace snapshot restore failed: ${this.redactUrl(r.error.message, url)}`, cause: r.error }
|
|
917
920
|
: r.value.exitCode !== 0
|
|
@@ -1127,20 +1130,22 @@ export class RemoteK8sExecutionEnv {
|
|
|
1127
1130
|
}
|
|
1128
1131
|
resolvePromise(r);
|
|
1129
1132
|
};
|
|
1130
|
-
const
|
|
1131
|
-
const
|
|
1133
|
+
const bounds = commandBoundsOf({ timeoutMs: opts?.timeoutMs, readTimeoutMs: opts?.readTimeoutMs }, this.laneLiveness);
|
|
1134
|
+
const capMs = opts?.capMs;
|
|
1135
|
+
const wallMs = bounds.wallMs === undefined ? capMs : capMs === undefined ? bounds.wallMs : Math.min(bounds.wallMs, capMs);
|
|
1136
|
+
const wallTimer = wallMs !== undefined ? setTimeout(() => finish({ ok: false, error: new ExecutionError("timeout", `command exceeded ${wallMs}ms`) }), wallMs) : undefined;
|
|
1132
1137
|
let livenessTimer;
|
|
1133
|
-
const
|
|
1138
|
+
const liveness = bounds.liveness;
|
|
1134
1139
|
const armLiveness = () => {
|
|
1135
|
-
if (
|
|
1140
|
+
if (liveness === undefined)
|
|
1136
1141
|
return;
|
|
1137
1142
|
clearTimeout(livenessTimer);
|
|
1138
1143
|
const idle = Date.now() - lastProgress;
|
|
1139
|
-
if (idle >=
|
|
1140
|
-
finish({ ok: false, error: new ExecutionError("timeout",
|
|
1144
|
+
if (idle >= liveness.ms) {
|
|
1145
|
+
finish({ ok: false, error: new ExecutionError("timeout", livenessCutMessage(liveness)) });
|
|
1141
1146
|
return;
|
|
1142
1147
|
}
|
|
1143
|
-
livenessTimer = setTimeout(armLiveness,
|
|
1148
|
+
livenessTimer = setTimeout(armLiveness, liveness.ms - idle);
|
|
1144
1149
|
};
|
|
1145
1150
|
armLiveness();
|
|
1146
1151
|
const onAbort = () => finish({ ok: false, error: new ExecutionError("aborted", "command aborted") });
|
|
@@ -1343,22 +1348,20 @@ export class RemoteK8sExecutionEnv {
|
|
|
1343
1348
|
wake = undefined;
|
|
1344
1349
|
w?.();
|
|
1345
1350
|
};
|
|
1346
|
-
let lastChunk = Date.now();
|
|
1347
|
-
const idleMs = options?.readTimeoutMs ?? this.cfg.livenessMs;
|
|
1348
1351
|
const cap = options?.maxOutputBytes;
|
|
1349
1352
|
let bytes = 0;
|
|
1350
1353
|
const ac = new AbortController();
|
|
1351
1354
|
const signal = options?.signal ? AbortSignal.any([options.signal, ac.signal]) : ac.signal;
|
|
1352
1355
|
const done = this.runExec(buildCwdEnvCommand(command, this.workingDirOf(options?.cwd), options?.env, "raw").command, {
|
|
1353
1356
|
signal,
|
|
1357
|
+
timeoutMs: options?.timeout != null ? options.timeout * 1000 : undefined,
|
|
1358
|
+
readTimeoutMs: options?.readTimeoutMs,
|
|
1354
1359
|
boundOutput: true,
|
|
1355
1360
|
onStdout: (d) => {
|
|
1356
|
-
lastChunk = Date.now();
|
|
1357
1361
|
queue.push({ type: "stdout", data: d });
|
|
1358
1362
|
signalReady();
|
|
1359
1363
|
},
|
|
1360
1364
|
onStderr: (d) => {
|
|
1361
|
-
lastChunk = Date.now();
|
|
1362
1365
|
queue.push({ type: "stderr", data: d });
|
|
1363
1366
|
signalReady();
|
|
1364
1367
|
},
|
|
@@ -1395,21 +1398,9 @@ export class RemoteK8sExecutionEnv {
|
|
|
1395
1398
|
throw failure;
|
|
1396
1399
|
if (finished)
|
|
1397
1400
|
break;
|
|
1398
|
-
|
|
1399
|
-
|
|
1400
|
-
|
|
1401
|
-
await new Promise((r) => {
|
|
1402
|
-
wake = r;
|
|
1403
|
-
waitTimer = setTimeout(r, Math.min(waitMs, 30_000));
|
|
1404
|
-
});
|
|
1405
|
-
}
|
|
1406
|
-
finally {
|
|
1407
|
-
if (waitTimer !== undefined)
|
|
1408
|
-
clearTimeout(waitTimer);
|
|
1409
|
-
}
|
|
1410
|
-
if (!finished && Date.now() - lastChunk >= idleMs) {
|
|
1411
|
-
throw new RemoteExecutionError("timeout", `execStream idle > ${idleMs}ms (suspected hang)`);
|
|
1412
|
-
}
|
|
1401
|
+
await new Promise((r) => {
|
|
1402
|
+
wake = r;
|
|
1403
|
+
});
|
|
1413
1404
|
}
|
|
1414
1405
|
while (queue.length)
|
|
1415
1406
|
yield queue.shift();
|
|
@@ -1428,7 +1419,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
1428
1419
|
}
|
|
1429
1420
|
makeBgDriver() {
|
|
1430
1421
|
const self = this;
|
|
1431
|
-
const bgExec = (script, boundOutput) => self.runExec(script, { timeoutMs: self.cfg.rpcTimeoutMs,
|
|
1422
|
+
const bgExec = (script, boundOutput) => self.runExec(script, { timeoutMs: self.cfg.rpcTimeoutMs, boundOutput });
|
|
1432
1423
|
return {
|
|
1433
1424
|
async launch(command, options, ctx) {
|
|
1434
1425
|
const conn = await self.ensureConnected();
|
|
@@ -1628,7 +1619,7 @@ export class RemoteK8sExecutionEnv {
|
|
|
1628
1619
|
const c = await this.ensureConnected();
|
|
1629
1620
|
if (!c.ok)
|
|
1630
1621
|
return c;
|
|
1631
|
-
return this.runExec(command, { signal: abortSignal,
|
|
1622
|
+
return this.runExec(command, { signal: abortSignal, capMs: this.cfg.dataTimeoutMs });
|
|
1632
1623
|
}
|
|
1633
1624
|
workingDirOf(cwd) {
|
|
1634
1625
|
const dir = cwd ?? this.cwd;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sema-agent/server",
|
|
3
|
-
"version": "7.129.
|
|
3
|
+
"version": "7.129.1",
|
|
4
4
|
"description": "Sema Server — the server/API implementation layer for Sema, wiring core, registry, model providers, and cloud agent execution. Built on @sema-agent/core.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "BUSL-1.1",
|