c8ctl-plugin-nano 1.57.0 → 1.58.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -3
- package/acp-transcript-producer.mjs +131 -0
- package/c8ctl-plugin.js +440 -31
- package/package.json +11 -10
package/README.md
CHANGED
|
@@ -892,9 +892,38 @@ quoting isn't honoured by `cmd.exe` — so use a container sandbox
|
|
|
892
892
|
(`--sandbox docker|podman`) or bake the switches into `--command` there.
|
|
893
893
|
|
|
894
894
|
**Disk hygiene.** Host job **workspaces** and container sandboxes both get
|
|
895
|
-
automatic cleanup so leaked artifacts can't fill the disk.
|
|
896
|
-
|
|
897
|
-
|
|
895
|
+
automatic cleanup so leaked artifacts can't fill the disk. Each worker
|
|
896
|
+
**process** gets its own private namespace under
|
|
897
|
+
`<state>/agent-runs/worker-<incarnation>/` (a fresh incarnation id every process
|
|
898
|
+
start, published with an immutable `owner.json` before any child dir appears);
|
|
899
|
+
its `run-*` job workspaces and `res-*` result channels live there and are removed
|
|
900
|
+
after each job and swept at startup + on `--reap-interval` (leftovers older than
|
|
901
|
+
`--reap-age`, in-flight dirs skipped). `--keep-runs` only skips the *per-job*
|
|
902
|
+
deletion (so a finished job's workspace survives for inspection); the age-based
|
|
903
|
+
owner-scoped sweep still applies, so a kept dir is eventually reaped once it ages
|
|
904
|
+
past `--reap-age`. That
|
|
905
|
+
ordinary sweep is **owner-scoped** — a worker only ever reaps *its own*
|
|
906
|
+
namespace, so it can never delete a sibling worker's active checkout or result
|
|
907
|
+
channel out from under an in-flight job (the cross-worker data-loss defect fixed
|
|
908
|
+
in [#205](https://github.com/jwulf/c8ctl-plugin-nano/issues/205); age is **not**
|
|
909
|
+
evidence of completion — editing files inside a checkout does not refresh the
|
|
910
|
+
enclosing dir's mtime). Reclaiming an *abandoned* namespace left by a crashed
|
|
911
|
+
worker is a **separate, cross-process-safe** operation: it deletes only a
|
|
912
|
+
namespace whose owning process is *provably* dead (PID-reuse-safe, via a recorded
|
|
913
|
+
process-start token) **and** has no surviving harness, under an exclusive lock
|
|
914
|
+
with a final recheck. Anything uncertain — a live/unknown owner, a possibly-alive
|
|
915
|
+
harness, missing/malformed ownership, a lock held by another reclaimer — is
|
|
916
|
+
**retained with a diagnostic**, never guessed away.
|
|
917
|
+
|
|
918
|
+
> **Mixed-version rollout.** The `worker-*` namespace is deliberately invisible
|
|
919
|
+
> to the old flat `run-*`/`res-*` sweep, and the new reclaimer never deletes
|
|
920
|
+
> unowned legacy flat `run-*`/`res-*` directories. This makes an upgrade safe
|
|
921
|
+
> while **old** worker processes are still running the pre-#205 code. Updating the
|
|
922
|
+
> package on disk does **not** replace code already loaded by a running worker:
|
|
923
|
+
> every old worker must be **drained/restarted onto the fixed version** before any
|
|
924
|
+
> leftover legacy flat directories can be cleaned up, and legacy flat dirs whose
|
|
925
|
+
> owner cannot be proven dead are never auto-migrated or deleted.
|
|
926
|
+
|
|
898
927
|
For container sandboxes a **label-scoped** reaper runs at worker startup
|
|
899
928
|
and on an interval (`--reap-interval`, **milliseconds**, default `300000` = 5m),
|
|
900
929
|
removing finished/`exited` containers older than `--reap-age` (**milliseconds**,
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
// Producer-side ACP → transcript-chunk mapping that PRESERVES message boundaries
|
|
2
|
+
// (jwulf/c8ctl-plugin-nano#206), built on top of the published shared contract
|
|
3
|
+
// from nanobpm/nano-ide#566 (@nanobpm/agentic >= 0.14.0).
|
|
4
|
+
//
|
|
5
|
+
// The problem: streaming ACP output arrives as arbitrary `agent_message_chunk`
|
|
6
|
+
// deltas whose transport boundaries are NOT message boundaries. The canonical
|
|
7
|
+
// bridge `acpUpdateToTranscriptChunk` (nanobpm/nano-ide#534) folds an ACP
|
|
8
|
+
// `session/update` into the exact transcript-chunk bytes the cockpit decodes, but
|
|
9
|
+
// it drops the ACP `messageId` that the shared classifier already extracts — so a
|
|
10
|
+
// consumer folding those chunks through the shared ordered-display derivation
|
|
11
|
+
// (`deriveDisplay`, #566) cannot tell a continuing delta of ONE message from the
|
|
12
|
+
// first delta of a NEW same-speaker message. Two distinct assistant messages
|
|
13
|
+
// emitted back-to-back would wrongly coalesce into one block; a single message
|
|
14
|
+
// split across chunks reconstructs correctly either way.
|
|
15
|
+
//
|
|
16
|
+
// This module carries the AVAILABLE producer semantics — message identity
|
|
17
|
+
// (`messageId`), role/channel and delta/snapshot mode — into the canonical
|
|
18
|
+
// additive `MessageEvent` fields the shared contract added in #566, using the
|
|
19
|
+
// SHARED classifier (`classifyUpdate`) and the SHARED canonical encoder
|
|
20
|
+
// (`encodeTranscriptEvent`). It does NOT hand-roll a parallel wire grammar,
|
|
21
|
+
// grouping implementation or heuristic sentence splitter: the marker, version,
|
|
22
|
+
// kinds and additive fields all come from the package.
|
|
23
|
+
//
|
|
24
|
+
// Fidelity contract (the documented legacy fallback):
|
|
25
|
+
// - ACP `agent_message_chunk` / `agent_thought_chunk` / `user_message_chunk`
|
|
26
|
+
// text is an incremental DELTA (never a cumulative snapshot for the supported
|
|
27
|
+
// ACP providers), so a message event is tagged `mode: "delta"` — a cumulative
|
|
28
|
+
// snapshot is NEVER emitted as an additive delta (the #566 "never append a
|
|
29
|
+
// snapshot as a delta" rule).
|
|
30
|
+
// - Where the provider exposes a `messageId`, it is carried so the display fold
|
|
31
|
+
// groups a message's fragments and separates two distinct same-speaker
|
|
32
|
+
// messages even when their transport chunks are adjacent.
|
|
33
|
+
// - Where the provider omits `messageId` (a documented ACP fidelity gap), NO
|
|
34
|
+
// identity is fabricated and NO boundary is inferred from delays/punctuation:
|
|
35
|
+
// the chunk is emitted through the canonical bridge UNCHANGED (byte-identical
|
|
36
|
+
// to the pre-#206 wire), and the display fold's adjacent-same-speaker
|
|
37
|
+
// coalescing is the legacy fallback.
|
|
38
|
+
// - Tool-call / tool-result / permission / ignored updates are delegated to the
|
|
39
|
+
// canonical bridge untouched, so tool and permission events stay correctly
|
|
40
|
+
// ordered and paired and raw replay is unchanged.
|
|
41
|
+
|
|
42
|
+
import { sessionAcp as defaultSessionAcp, transcript as defaultTranscript } from './agentic.mjs';
|
|
43
|
+
|
|
44
|
+
// Map the shared classifier's message role to a canonical `TranscriptRole`. ACP's
|
|
45
|
+
// `reasoning` (an `agent_thought_chunk`) has no distinct transcript role, so — like
|
|
46
|
+
// the canonical bridge `acpUpdateToTranscriptChunk` — it folds to `assistant`,
|
|
47
|
+
// which `deriveDisplay` renders as a message block rather than dropping to raw
|
|
48
|
+
// bytes. This mirrors the package bridge exactly so the producer never diverges
|
|
49
|
+
// from the shared role mapping.
|
|
50
|
+
function transcriptRole(acpRole) {
|
|
51
|
+
return acpRole === 'user' ? 'user' : 'assistant';
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
// A non-empty string, else null. `messageId` is optional on the ACP update and the
|
|
55
|
+
// shared classifier already normalises it to `string | null`.
|
|
56
|
+
function nonBlankId(value) {
|
|
57
|
+
return typeof value === 'string' && value !== '' ? value : null;
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
/**
|
|
61
|
+
* Map one raw ACP `session/update` `update` object to the canonical transcript-chunk
|
|
62
|
+
* bytes a producer appends, carrying the available message identity / role / delta
|
|
63
|
+
* semantics into the shared additive `MessageEvent` contract (#566). Returns `null`
|
|
64
|
+
* for an update with no canonical meaning (an `ignored` classification), exactly like
|
|
65
|
+
* the underlying bridge, so a caller skips it.
|
|
66
|
+
*
|
|
67
|
+
* Pure and total: any classifier or encoder throw degrades to the canonical bridge,
|
|
68
|
+
* and a bridge throw is itself caught (yielding `null`), so the producer hot path
|
|
69
|
+
* never crashes on one malformed update.
|
|
70
|
+
*
|
|
71
|
+
* @param {unknown} update The raw ACP `session/update` `params.update` object.
|
|
72
|
+
* @param {object} [deps]
|
|
73
|
+
* @param {object} [deps.sessionAcp] The shared ACP surface (`classifyUpdate` +
|
|
74
|
+
* `acpUpdateToTranscriptChunk`); defaults to the package bridge.
|
|
75
|
+
* @param {object} [deps.transcript] The shared transcript surface
|
|
76
|
+
* (`encodeTranscriptEvent`); defaults to the package transcript module.
|
|
77
|
+
* @returns {string | null} The canonical transcript-chunk bytes, or `null`.
|
|
78
|
+
*/
|
|
79
|
+
export function acpUpdateToDisplayChunk(update, deps = {}) {
|
|
80
|
+
const sessionAcp = deps.sessionAcp || defaultSessionAcp;
|
|
81
|
+
const transcript = deps.transcript || defaultTranscript;
|
|
82
|
+
|
|
83
|
+
const classify = typeof sessionAcp?.classifyUpdate === 'function' ? sessionAcp.classifyUpdate : null;
|
|
84
|
+
const encode = typeof transcript?.encodeTranscriptEvent === 'function' ? transcript.encodeTranscriptEvent : null;
|
|
85
|
+
const bridge = typeof sessionAcp?.acpUpdateToTranscriptChunk === 'function' ? sessionAcp.acpUpdateToTranscriptChunk : null;
|
|
86
|
+
|
|
87
|
+
// Fallback to the canonical bridge output for this update. Never throws.
|
|
88
|
+
const viaBridge = () => {
|
|
89
|
+
if (!bridge) return null;
|
|
90
|
+
try { return bridge(update); }
|
|
91
|
+
catch { return null; }
|
|
92
|
+
};
|
|
93
|
+
|
|
94
|
+
// Without the shared classifier + encoder we cannot enrich the message event, so
|
|
95
|
+
// the byte-identical canonical bridge output is the only correct behaviour.
|
|
96
|
+
if (!classify || !encode) return viaBridge();
|
|
97
|
+
|
|
98
|
+
let classified;
|
|
99
|
+
try { classified = classify(update); }
|
|
100
|
+
catch { return viaBridge(); }
|
|
101
|
+
|
|
102
|
+
// Only message chunks carry identity/boundary semantics worth enriching. Every
|
|
103
|
+
// other classification (tool-call, tool-result, ignored) is delegated to the
|
|
104
|
+
// canonical bridge UNCHANGED — tool/permission ordering and raw replay untouched.
|
|
105
|
+
if (!classified || classified.kind !== 'message') return viaBridge();
|
|
106
|
+
|
|
107
|
+
const messageId = nonBlankId(classified.messageId);
|
|
108
|
+
|
|
109
|
+
// No provider-supplied identity → do NOT fabricate one or infer a boundary.
|
|
110
|
+
// Emit through the canonical bridge unchanged (byte-identical to the pre-#206
|
|
111
|
+
// wire) and let the display fold's adjacent-same-speaker coalescing be the
|
|
112
|
+
// documented legacy fallback.
|
|
113
|
+
if (messageId === null) return viaBridge();
|
|
114
|
+
|
|
115
|
+
// Carry the available semantics into the additive `MessageEvent` fields: the
|
|
116
|
+
// producer identity (`messageId`) so the fold groups this message's fragments and
|
|
117
|
+
// separates distinct same-speaker messages, and `mode: "delta"` because ACP
|
|
118
|
+
// message chunks are incremental deltas — never a cumulative snapshot. No `offset`
|
|
119
|
+
// is supplied here; the real store offset is assigned on append (matching every
|
|
120
|
+
// other `encodeTranscriptEvent` call site).
|
|
121
|
+
const event = {
|
|
122
|
+
kind: 'message',
|
|
123
|
+
role: transcriptRole(classified.role),
|
|
124
|
+
text: classified.text,
|
|
125
|
+
messageId,
|
|
126
|
+
mode: 'delta',
|
|
127
|
+
};
|
|
128
|
+
|
|
129
|
+
try { return encode(event); }
|
|
130
|
+
catch { return viaBridge(); }
|
|
131
|
+
}
|
package/c8ctl-plugin.js
CHANGED
|
@@ -42,6 +42,7 @@ import {
|
|
|
42
42
|
readdirSync,
|
|
43
43
|
chmodSync,
|
|
44
44
|
renameSync,
|
|
45
|
+
linkSync,
|
|
45
46
|
realpathSync,
|
|
46
47
|
statfsSync,
|
|
47
48
|
lstatSync,
|
|
@@ -72,6 +73,13 @@ import { createLogRing, resolveLogMaxBytes } from './supervisor-log-ring.mjs';
|
|
|
72
73
|
// raw ACP `session/update` to the exact transcript-chunk bytes the cockpit decodes,
|
|
73
74
|
// replacing the plugin's former hand-rolled `nwfTranscriptEvent` envelope grammar.
|
|
74
75
|
import { sessionAcp as agenticSessionAcp } from './agentic.mjs';
|
|
76
|
+
// Producer-side message-boundary preservation (jwulf/c8ctl-plugin-nano#206). Wraps
|
|
77
|
+
// the canonical bridge to carry the ACP `messageId` / role / delta semantics into
|
|
78
|
+
// the shared additive `MessageEvent` contract (nanobpm/nano-ide#566), so a consumer
|
|
79
|
+
// folding these chunks through `deriveDisplay` reconstructs transport-fragmented
|
|
80
|
+
// deltas into coherent blocks and keeps distinct same-speaker messages apart —
|
|
81
|
+
// falling back to byte-identical bridge output when the provider omits identity.
|
|
82
|
+
import { acpUpdateToDisplayChunk } from './acp-transcript-producer.mjs';
|
|
75
83
|
// Engine-native AgentInstance / AgentHistory durable-transcript producer (issue
|
|
76
84
|
// #194): mints an AgentInstance for an `external` agent job and appends each ACP
|
|
77
85
|
// turn to the engine's append-only AgentHistory via the host SDK client.
|
|
@@ -4427,9 +4435,16 @@ function finalizeGit({ workspaceDir, gitEnv, startSha, workingBranch, envelope,
|
|
|
4427
4435
|
return out;
|
|
4428
4436
|
}
|
|
4429
4437
|
|
|
4430
|
-
//
|
|
4431
|
-
//
|
|
4432
|
-
//
|
|
4438
|
+
// LEGACY flat reaper (pre-issue-#205). Reaps `run-*`/`res-*` directly under the
|
|
4439
|
+
// runs ROOT — the shared-namespace design that caused the cross-worker data-loss
|
|
4440
|
+
// incident. It is NO LONGER CALLED by the worker (which now allocates a private
|
|
4441
|
+
// `worker-<incarnation>/` namespace and uses `reapOwnedNamespace` +
|
|
4442
|
+
// `reclaimOrphanNamespaces`), and is retained ONLY so that (a) a mixed-version
|
|
4443
|
+
// host with an OLD worker still running this code stays interoperable, and (b)
|
|
4444
|
+
// the mixed-version regression test can assert this flat sweep never descends
|
|
4445
|
+
// into a new `worker-*` namespace (it only matches the `run-`/`res-` prefixes at
|
|
4446
|
+
// the root, so `worker-*` dirs are invisible to it). Do not reintroduce it into
|
|
4447
|
+
// the worker cleanup path.
|
|
4433
4448
|
function reapAgentRunDirs({ maxAgeMs = 0, liveRunDirs = new Set() } = {}) {
|
|
4434
4449
|
let reaped = 0;
|
|
4435
4450
|
const root = agentRunsRoot();
|
|
@@ -4458,13 +4473,321 @@ function reapAgentRunDirs({ maxAgeMs = 0, liveRunDirs = new Set() } = {}) {
|
|
|
4458
4473
|
return { reaped };
|
|
4459
4474
|
}
|
|
4460
4475
|
|
|
4476
|
+
// ---- Cross-process-safe worker run/result namespaces (issue #205) ----------
|
|
4477
|
+
// The old shared flat `agent-runs/{run,res}-*` layout let ANY worker process's
|
|
4478
|
+
// age-gated reaper delete another live worker's active checkout or result
|
|
4479
|
+
// channel: `reapAgentRunDirs` excludes only ITS OWN caller's in-flight dirs, so
|
|
4480
|
+
// a sibling process (empty `liveRunDirs`) treated an aged-but-active dir as
|
|
4481
|
+
// garbage and removed it (data loss — see the incident in issue #205). Age is
|
|
4482
|
+
// not evidence of completion; editing files inside a checkout does not refresh
|
|
4483
|
+
// the enclosing dir's mtime.
|
|
4484
|
+
//
|
|
4485
|
+
// The fix isolates each worker PROCESS INCARNATION under its own
|
|
4486
|
+
// `agent-runs/worker-<incarnation>/` namespace and splits cleanup in two:
|
|
4487
|
+
//
|
|
4488
|
+
// * OWNER-SCOPED ordinary cleanup — a worker's startup/periodic reaper only
|
|
4489
|
+
// ever traverses its OWN namespace, where its `liveRunDirs` set is the sole
|
|
4490
|
+
// authority. It can never see, let alone delete, a sibling's dir.
|
|
4491
|
+
// * ORPHAN RECLAMATION — a separate, cross-process-safe sweep that may reclaim
|
|
4492
|
+
// ANOTHER incarnation's namespace ONLY on positive proof the owner process
|
|
4493
|
+
// is dead (PID-reuse-safe) and no harness it spawned survives, under an
|
|
4494
|
+
// exclusive lock with a final recheck. Unknown ownership/liveness always
|
|
4495
|
+
// RETAINS and logs an actionable diagnostic, never deletes.
|
|
4496
|
+
//
|
|
4497
|
+
// The `worker-` namespace prefix is deliberately invisible to the legacy flat
|
|
4498
|
+
// `run-*`/`res-*` sweep, so an OLD worker still running the pre-fix code cannot
|
|
4499
|
+
// enter a new namespace, and the new reaper never touches unowned legacy flat
|
|
4500
|
+
// dirs. See issue #205 and the mixed-version rollout note in the README.
|
|
4501
|
+
|
|
4502
|
+
const WORKER_NS_PREFIX = 'worker-';
|
|
4503
|
+
const OWNER_RECORD = 'owner.json';
|
|
4504
|
+
const RECLAIM_LOCK = '.reclaiming';
|
|
4505
|
+
const LIVE_MARKER_DIR = 'live';
|
|
4506
|
+
|
|
4507
|
+
// A process-start fingerprint for `pid`, used to defeat PID reuse: a recorded
|
|
4508
|
+
// owner is only "the same process" if the PID is alive AND its start token still
|
|
4509
|
+
// matches. Best-effort + cross-platform: Linux reads field 22 (starttime) from
|
|
4510
|
+
// `/proc/<pid>/stat`; elsewhere (macOS/BSD) it shells out to `ps -o lstart=`.
|
|
4511
|
+
// Returns null when neither source is available — callers treat a null/absent
|
|
4512
|
+
// token conservatively (cannot prove reuse ⇒ never reclaim).
|
|
4513
|
+
function pidStartToken(pid = process.pid) {
|
|
4514
|
+
if (!Number.isInteger(pid) || pid <= 0) return null;
|
|
4515
|
+
try {
|
|
4516
|
+
const stat = readFileSync(`/proc/${pid}/stat`, 'utf-8');
|
|
4517
|
+
// comm (field 2) is parenthesised and may itself contain spaces/parens, so
|
|
4518
|
+
// split AFTER the last ')'. starttime is overall field 22 ⇒ index 19 of the
|
|
4519
|
+
// remaining whitespace-separated fields.
|
|
4520
|
+
const rparen = stat.lastIndexOf(')');
|
|
4521
|
+
if (rparen !== -1) {
|
|
4522
|
+
const rest = stat.slice(rparen + 2).trim().split(/\s+/);
|
|
4523
|
+
const starttime = rest[19];
|
|
4524
|
+
if (starttime && /^\d+$/.test(starttime)) return `lx:${starttime}`;
|
|
4525
|
+
}
|
|
4526
|
+
} catch { /* not Linux / no procfs */ }
|
|
4527
|
+
try {
|
|
4528
|
+
const out = execFileSync('ps', ['-o', 'lstart=', '-p', String(pid)], { encoding: 'utf-8', timeout: 5_000 }).trim();
|
|
4529
|
+
if (out) return `ps:${out}`;
|
|
4530
|
+
} catch { /* best effort */ }
|
|
4531
|
+
return null;
|
|
4532
|
+
}
|
|
4533
|
+
|
|
4534
|
+
// A fresh, unique-per-process incarnation id. NOT the reusable configured worker
|
|
4535
|
+
// name: two processes running the same profile must land in distinct namespaces.
|
|
4536
|
+
function newIncarnationId() {
|
|
4537
|
+
return `${process.pid}-${Date.now().toString(36)}-${randomBytes(6).toString('hex')}`;
|
|
4538
|
+
}
|
|
4539
|
+
|
|
4540
|
+
// Sanitise an incarnation id for safe use as a single path segment.
|
|
4541
|
+
function workerNamespaceDir(incarnation, root = agentRunsRoot()) {
|
|
4542
|
+
return join(root, `${WORKER_NS_PREFIX}${String(incarnation).replace(/[^\w.#-]/g, '_')}`);
|
|
4543
|
+
}
|
|
4544
|
+
|
|
4545
|
+
function readOwnerRecord(nsDir) {
|
|
4546
|
+
try {
|
|
4547
|
+
const rec = JSON.parse(readFileSync(join(nsDir, OWNER_RECORD), 'utf-8'));
|
|
4548
|
+
if (!rec || typeof rec !== 'object' || typeof rec.incarnation !== 'string') return null;
|
|
4549
|
+
return rec;
|
|
4550
|
+
} catch { return null; }
|
|
4551
|
+
}
|
|
4552
|
+
|
|
4553
|
+
// Allocate (idempotently) this incarnation's namespace and publish its immutable
|
|
4554
|
+
// ownership record ATOMICALLY *before* any reapable child dir can exist inside
|
|
4555
|
+
// it. A concurrent reclaimer that catches the dir mid-creation sees no owner
|
|
4556
|
+
// record and RETAINS it (never reclaims an incompletely-registered namespace).
|
|
4557
|
+
function allocateWorkerNamespace({ incarnation, worker = null, pid = process.pid, pidStart = null, version = null, root = agentRunsRoot() } = {}) {
|
|
4558
|
+
const nsDir = workerNamespaceDir(incarnation, root);
|
|
4559
|
+
mkdirSync(nsDir, { recursive: true });
|
|
4560
|
+
const ownerFile = join(nsDir, OWNER_RECORD);
|
|
4561
|
+
if (!existsSync(ownerFile)) {
|
|
4562
|
+
const owner = { schema: 1, incarnation, worker, pid, pidStart, host: hostname(), createdAt: new Date().toISOString(), version };
|
|
4563
|
+
const tmp = `${ownerFile}.${process.pid}.${Date.now()}.tmp`;
|
|
4564
|
+
writeFileSync(tmp, JSON.stringify(owner, null, 2));
|
|
4565
|
+
// Publish EXCLUSIVELY: linkSync is an atomic create-if-absent on POSIX and
|
|
4566
|
+
// Windows, so a racing allocator can never overwrite an already-published
|
|
4567
|
+
// owner record (the immutability guarantee). EEXIST ⇒ a peer won the race,
|
|
4568
|
+
// which is success — the record is immutable, so whoever wrote it is fine.
|
|
4569
|
+
try {
|
|
4570
|
+
linkSync(tmp, ownerFile);
|
|
4571
|
+
} catch (err) {
|
|
4572
|
+
if (err?.code !== 'EEXIST') { try { rmSync(tmp, { force: true }); } catch { /* */ } throw err; }
|
|
4573
|
+
} finally {
|
|
4574
|
+
try { rmSync(tmp, { force: true }); } catch { /* */ }
|
|
4575
|
+
}
|
|
4576
|
+
}
|
|
4577
|
+
mkdirSync(join(nsDir, LIVE_MARKER_DIR), { recursive: true });
|
|
4578
|
+
return { nsDir, owner: readOwnerRecord(nsDir) };
|
|
4579
|
+
}
|
|
4580
|
+
|
|
4581
|
+
// Classify a recorded owner's incarnation as 'alive' | 'dead' | 'unknown'.
|
|
4582
|
+
// PID-reuse-safe: a live PID whose start token no longer matches the record is a
|
|
4583
|
+
// DIFFERENT process ⇒ the recorded incarnation is 'dead'. A missing pid, or a
|
|
4584
|
+
// live pid we cannot re-fingerprint (null token on either side), is 'unknown' —
|
|
4585
|
+
// never 'dead' — so reclamation errs toward preservation.
|
|
4586
|
+
function incarnationLiveness(owner, { isAlive = isPidAlive, startToken = pidStartToken } = {}) {
|
|
4587
|
+
if (!owner || !Number.isInteger(owner.pid) || owner.pid <= 0) return 'unknown';
|
|
4588
|
+
let alive;
|
|
4589
|
+
try { alive = isAlive(owner.pid); } catch { return 'unknown'; }
|
|
4590
|
+
if (!alive) return 'dead';
|
|
4591
|
+
if (owner.pidStart == null) return 'unknown'; // never recorded ⇒ can't disprove reuse
|
|
4592
|
+
let current;
|
|
4593
|
+
try { current = startToken(owner.pid); } catch { current = null; }
|
|
4594
|
+
if (current == null) return 'unknown'; // can't re-fingerprint ⇒ conservative
|
|
4595
|
+
return current === owner.pidStart ? 'alive' : 'dead';
|
|
4596
|
+
}
|
|
4597
|
+
|
|
4598
|
+
function jobMarkerPath(nsDir, jobKey) {
|
|
4599
|
+
return join(nsDir, LIVE_MARKER_DIR, `${String(jobKey).replace(/[^\w.#-]/g, '_')}.json`);
|
|
4600
|
+
}
|
|
4601
|
+
|
|
4602
|
+
// Record that a job is in-flight in this namespace (a harness may be spawned).
|
|
4603
|
+
// Written BEFORE the harness starts so a crash mid-spawn still leaves evidence
|
|
4604
|
+
// that a harness could be orphaned (an empty `harnessPids` ⇒ retain conservatively).
|
|
4605
|
+
function writeJobMarker(nsDir, { jobKey, workerPid = process.pid, incarnation = null }) {
|
|
4606
|
+
try {
|
|
4607
|
+
mkdirSync(join(nsDir, LIVE_MARKER_DIR), { recursive: true });
|
|
4608
|
+
const p = jobMarkerPath(nsDir, jobKey);
|
|
4609
|
+
writeFileSync(p, JSON.stringify({ jobKey: String(jobKey), workerPid, incarnation, harnessPids: [], startedAt: new Date().toISOString() }));
|
|
4610
|
+
return p;
|
|
4611
|
+
} catch { return null; }
|
|
4612
|
+
}
|
|
4613
|
+
|
|
4614
|
+
// Append the spawned harness's PID (its process-group leader) to the in-flight
|
|
4615
|
+
// job marker, so orphan reclamation can probe whether it survived the worker.
|
|
4616
|
+
function recordHarnessPid(nsDir, jobKey, pid) {
|
|
4617
|
+
if (!nsDir || !Number.isInteger(pid) || pid <= 0) return;
|
|
4618
|
+
try {
|
|
4619
|
+
const p = jobMarkerPath(nsDir, jobKey);
|
|
4620
|
+
const rec = JSON.parse(readFileSync(p, 'utf-8'));
|
|
4621
|
+
if (!Array.isArray(rec.harnessPids)) rec.harnessPids = [];
|
|
4622
|
+
if (!rec.harnessPids.includes(pid)) {
|
|
4623
|
+
rec.harnessPids.push(pid);
|
|
4624
|
+
const tmp = `${p}.${process.pid}.tmp`;
|
|
4625
|
+
writeFileSync(tmp, JSON.stringify(rec));
|
|
4626
|
+
renameSync(tmp, p);
|
|
4627
|
+
}
|
|
4628
|
+
} catch { /* best effort */ }
|
|
4629
|
+
}
|
|
4630
|
+
|
|
4631
|
+
function removeJobMarker(nsDir, jobKey) {
|
|
4632
|
+
try { rmSync(jobMarkerPath(nsDir, jobKey), { force: true }); } catch { /* best effort */ }
|
|
4633
|
+
}
|
|
4634
|
+
|
|
4635
|
+
function readJobMarkers(nsDir) {
|
|
4636
|
+
let names;
|
|
4637
|
+
try { names = readdirSync(join(nsDir, LIVE_MARKER_DIR)); }
|
|
4638
|
+
catch (err) {
|
|
4639
|
+
// ENOENT ⇒ the live/ dir never existed: genuinely no in-flight markers.
|
|
4640
|
+
// Any other error (EACCES, transient FS) means marker liveness is UNKNOWN,
|
|
4641
|
+
// so return a synthetic malformed marker to force a conservative retain
|
|
4642
|
+
// (never reclaim a namespace whose harness state we couldn't determine).
|
|
4643
|
+
if (err && err.code === 'ENOENT') return [];
|
|
4644
|
+
return [{ jobKey: '(unreadable)', harnessPids: [], malformed: true }];
|
|
4645
|
+
}
|
|
4646
|
+
const out = [];
|
|
4647
|
+
for (const n of names) {
|
|
4648
|
+
if (!n.endsWith('.json')) continue;
|
|
4649
|
+
try { out.push(JSON.parse(readFileSync(join(nsDir, LIVE_MARKER_DIR, n), 'utf-8'))); }
|
|
4650
|
+
catch { out.push({ jobKey: n, harnessPids: [], malformed: true }); }
|
|
4651
|
+
}
|
|
4652
|
+
return out;
|
|
4653
|
+
}
|
|
4654
|
+
|
|
4655
|
+
// Does a (presumed-dead-owner) namespace still have a possibly-live harness? An
|
|
4656
|
+
// in-flight marker that is malformed or records no harness pid (a registration
|
|
4657
|
+
// race — the job started but the harness pid was not yet recorded) is treated as
|
|
4658
|
+
// possibly-live (conservative). Otherwise probe each recorded pid.
|
|
4659
|
+
function namespaceHasLiveHarness(nsDir, { isAlive = isPidAlive } = {}) {
|
|
4660
|
+
for (const m of readJobMarkers(nsDir)) {
|
|
4661
|
+
const pids = Array.isArray(m.harnessPids) ? m.harnessPids : [];
|
|
4662
|
+
if (m.malformed || pids.length === 0) return true;
|
|
4663
|
+
for (const pid of pids) { try { if (isAlive(pid)) return true; } catch { return true; } }
|
|
4664
|
+
}
|
|
4665
|
+
return false;
|
|
4666
|
+
}
|
|
4667
|
+
|
|
4668
|
+
// Shared eligibility used by BOTH the startup and periodic owner-scoped sweeps
|
|
4669
|
+
// (so they can never drift): reap the `run-*`/`res-*` children of `dir` that are
|
|
4670
|
+
// (a) not in `liveRunDirs`, (b) older than `maxAgeMs`, (c) a real directory (an
|
|
4671
|
+
// lstat rejects a symlink — never followed). `onReap(path)` fires per removal for
|
|
4672
|
+
// evidence logging. Confined to `dir`; unrelated entries (owner.json, live/,
|
|
4673
|
+
// .reclaiming, operator files) are ignored.
|
|
4674
|
+
function reapChildRunDirs(dir, { maxAgeMs = 0, liveRunDirs = new Set(), now = Date.now(), onReap } = {}) {
|
|
4675
|
+
let reaped = 0;
|
|
4676
|
+
try {
|
|
4677
|
+
if (!existsSync(dir)) return { reaped };
|
|
4678
|
+
for (const name of readdirSync(dir)) {
|
|
4679
|
+
if (!name.startsWith('run-') && !name.startsWith('res-')) continue;
|
|
4680
|
+
const p = join(dir, name);
|
|
4681
|
+
if (liveRunDirs.has(p)) continue;
|
|
4682
|
+
try {
|
|
4683
|
+
const st = lstatSync(p);
|
|
4684
|
+
if (!st.isDirectory()) continue;
|
|
4685
|
+
if (maxAgeMs > 0 && now - st.mtimeMs < maxAgeMs) continue;
|
|
4686
|
+
rmSync(p, { recursive: true, force: true });
|
|
4687
|
+
reaped++;
|
|
4688
|
+
if (onReap) { try { onReap(p); } catch { /* */ } }
|
|
4689
|
+
} catch { /* skip */ }
|
|
4690
|
+
}
|
|
4691
|
+
} catch (err) {
|
|
4692
|
+
return { reaped, error: err.message };
|
|
4693
|
+
}
|
|
4694
|
+
return { reaped };
|
|
4695
|
+
}
|
|
4696
|
+
|
|
4697
|
+
// OWNER-SCOPED ordinary cleanup: reap aged, not-in-flight job/result dirs inside THIS
|
|
4698
|
+
// worker's own namespace only. Safe by construction — no sibling shares this
|
|
4699
|
+
// namespace, and `liveRunDirs` authoritatively excludes in-flight dirs.
|
|
4700
|
+
function reapOwnedNamespace(nsDir, { maxAgeMs = 0, liveRunDirs = new Set(), now = Date.now(), logger, incarnation } = {}) {
|
|
4701
|
+
return reapChildRunDirs(nsDir, {
|
|
4702
|
+
maxAgeMs,
|
|
4703
|
+
liveRunDirs,
|
|
4704
|
+
now,
|
|
4705
|
+
onReap: (p) => { if (logger) logger.info(`[reaper] ${new Date(now).toISOString()} incarnation=${incarnation ?? '?'} removed own aged, not-in-flight ${basename(p)} (owner-scoped)`); },
|
|
4706
|
+
});
|
|
4707
|
+
}
|
|
4708
|
+
|
|
4709
|
+
// ORPHAN RECLAMATION (cross-process-safe). Sweep sibling `worker-*` namespaces
|
|
4710
|
+
// and reclaim ONLY those whose owner incarnation is provably dead and which have
|
|
4711
|
+
// no surviving harness, under an exclusive per-namespace lock with a final
|
|
4712
|
+
// recheck. Everything uncertain — a missing/malformed owner record, a live or
|
|
4713
|
+
// unknown owner, a possibly-live harness, a lock held by another reclaimer, a
|
|
4714
|
+
// namespace younger than `minAgeMs`, a symlink, or a path escaping `root` — is
|
|
4715
|
+
// RETAINED with a diagnostic. Legacy flat `run-*`/`res-*` (no `worker-` prefix)
|
|
4716
|
+
// are never considered. Injectable probes (`liveness`, `harnessAlive`, `now`)
|
|
4717
|
+
// make every branch deterministically testable.
|
|
4718
|
+
function reclaimOrphanNamespaces({
|
|
4719
|
+
root = agentRunsRoot(),
|
|
4720
|
+
selfIncarnation = null,
|
|
4721
|
+
now = Date.now(),
|
|
4722
|
+
minAgeMs = 0,
|
|
4723
|
+
liveness = incarnationLiveness,
|
|
4724
|
+
harnessAlive = isPidAlive,
|
|
4725
|
+
logger,
|
|
4726
|
+
} = {}) {
|
|
4727
|
+
const reclaimed = [];
|
|
4728
|
+
const retained = [];
|
|
4729
|
+
const stamp = () => new Date(now).toISOString();
|
|
4730
|
+
const note = (name, reason, owner) => {
|
|
4731
|
+
retained.push({ name, reason });
|
|
4732
|
+
if (logger) logger.info(`[reclaim] ${stamp()} actor=${selfIncarnation ?? '?'} target=${name} owner=${owner?.incarnation ?? 'unknown'}(pid ${owner?.pid ?? '?'}) RETAINED — ${reason}`);
|
|
4733
|
+
};
|
|
4734
|
+
let rootReal;
|
|
4735
|
+
try { rootReal = realpathSync(root); } catch (err) {
|
|
4736
|
+
if (logger) logger.info(`[reclaim] ${stamp()} actor=${selfIncarnation ?? '?'} RETAINED all — could not resolve runs root ${root} (${err.code || err.message})`);
|
|
4737
|
+
return { reclaimed, retained };
|
|
4738
|
+
}
|
|
4739
|
+
let names;
|
|
4740
|
+
try { names = readdirSync(root); } catch (err) {
|
|
4741
|
+
if (logger) logger.info(`[reclaim] ${stamp()} actor=${selfIncarnation ?? '?'} RETAINED all — could not scan runs root ${root} (${err.code || err.message})`);
|
|
4742
|
+
return { reclaimed, retained };
|
|
4743
|
+
}
|
|
4744
|
+
for (const name of names) {
|
|
4745
|
+
if (!name.startsWith(WORKER_NS_PREFIX)) continue;
|
|
4746
|
+
const nsDir = join(root, name);
|
|
4747
|
+
let st;
|
|
4748
|
+
try { st = lstatSync(nsDir); } catch (err) { note(name, `could not lstat (${err.code || err.message}) — retained`, null); continue; }
|
|
4749
|
+
if (!st.isDirectory()) { note(name, 'not a directory (symlink?) — skipped', null); continue; }
|
|
4750
|
+
if (minAgeMs > 0 && now - st.mtimeMs < minAgeMs) { note(name, 'younger than min reclaim age', null); continue; }
|
|
4751
|
+
// Containment: the resolved namespace must sit directly under the resolved root.
|
|
4752
|
+
let nsReal;
|
|
4753
|
+
try { nsReal = realpathSync(nsDir); } catch (err) { note(name, `could not resolve real path (${err.code || err.message}) — retained`, null); continue; }
|
|
4754
|
+
if (dirname(nsReal) !== rootReal) { note(name, 'path escapes runs root — skipped', null); continue; }
|
|
4755
|
+
const owner = readOwnerRecord(nsDir);
|
|
4756
|
+
if (!owner) { note(name, 'missing/malformed owner record — not garbage', null); continue; }
|
|
4757
|
+
if (selfIncarnation && owner.incarnation === selfIncarnation) continue; // never our own
|
|
4758
|
+
const state = liveness(owner);
|
|
4759
|
+
if (state !== 'dead') { note(name, `owner ${state}`, owner); continue; }
|
|
4760
|
+
if (namespaceHasLiveHarness(nsDir, { isAlive: harnessAlive })) { note(name, 'a spawned harness may still be alive', owner); continue; }
|
|
4761
|
+
// Exclusive reclamation ownership: an atomic mkdir lock. A loser retains.
|
|
4762
|
+
const lock = join(nsDir, RECLAIM_LOCK);
|
|
4763
|
+
try { mkdirSync(lock); } catch { note(name, 'another reclaimer holds the lock', owner); continue; }
|
|
4764
|
+
try {
|
|
4765
|
+
// FINAL recheck under the lock: identity + liveness + harness must all hold.
|
|
4766
|
+
const owner2 = readOwnerRecord(nsDir);
|
|
4767
|
+
if (!owner2 || owner2.incarnation !== owner.incarnation) { note(name, 'ownership changed under lock', owner2 || owner); continue; }
|
|
4768
|
+
if (liveness(owner2) !== 'dead') { note(name, 'owner became alive under lock', owner2); continue; }
|
|
4769
|
+
if (namespaceHasLiveHarness(nsDir, { isAlive: harnessAlive })) { note(name, 'harness became live under lock', owner2); continue; }
|
|
4770
|
+
let real2;
|
|
4771
|
+
try { real2 = realpathSync(nsDir); } catch (err) { note(name, `could not resolve real path under lock (${err.code || err.message}) — retained`, owner2); continue; }
|
|
4772
|
+
if (dirname(real2) !== rootReal) { note(name, 'path escaped runs root under lock', owner2); continue; }
|
|
4773
|
+
rmSync(nsDir, { recursive: true, force: true });
|
|
4774
|
+
reclaimed.push({ name, owner: owner2 });
|
|
4775
|
+
if (logger) logger.info(`[reclaim] ${stamp()} actor=${selfIncarnation ?? '?'} target=${name} owner=${owner2.incarnation}(pid ${owner2.pid}) RECLAIMED — owner proven dead, no surviving harness`);
|
|
4776
|
+
} finally {
|
|
4777
|
+
// Drop the lock unless the whole namespace was reclaimed (lock gone with it).
|
|
4778
|
+
try { if (existsSync(nsDir)) rmSync(lock, { recursive: true, force: true }); } catch { /* best effort */ }
|
|
4779
|
+
}
|
|
4780
|
+
}
|
|
4781
|
+
return { reclaimed, retained };
|
|
4782
|
+
}
|
|
4783
|
+
|
|
4461
4784
|
// ---- One-shot capture (shared by host + container executors) ---------------
|
|
4462
4785
|
const MAX_CAPTURE_BYTES = 1_048_576; // 1 MiB per stream
|
|
4463
4786
|
|
|
4464
4787
|
// Spawn a child, pipe `stdinData`, capture byte-capped stdout/stderr, enforce a
|
|
4465
4788
|
// timeout (invoking `onTimeout(child)` to tear the child down), and resolve to a
|
|
4466
4789
|
// uniform result. Used by both the host and container executors.
|
|
4467
|
-
function spawnCaptureOneShot({ command, args = [], shell = false, detached = false, cwd, env, stdinData, timeoutMs, idleTimeoutMs, recoveryWindowMs, onTimeout, stream = false, streamPrefix = '', onStreamOut, onStreamErr, relayTap = null, abortSignal = null }) {
|
|
4790
|
+
function spawnCaptureOneShot({ command, args = [], shell = false, detached = false, cwd, env, stdinData, timeoutMs, idleTimeoutMs, recoveryWindowMs, onTimeout, stream = false, streamPrefix = '', onStreamOut, onStreamErr, relayTap = null, abortSignal = null, onSpawn = null }) {
|
|
4468
4791
|
return new Promise((resolve) => {
|
|
4469
4792
|
let child;
|
|
4470
4793
|
const stdoutChunks = [];
|
|
@@ -4526,6 +4849,7 @@ function spawnCaptureOneShot({ command, args = [], shell = false, detached = fal
|
|
|
4526
4849
|
finish({ ok: false, exitCode: null, stdout: '', stderr: '', error: err.message, truncated: false, stderrTruncated: false });
|
|
4527
4850
|
return;
|
|
4528
4851
|
}
|
|
4852
|
+
if (onSpawn) { try { onSpawn(child.pid); } catch { /* best effort */ } }
|
|
4529
4853
|
|
|
4530
4854
|
// #202: a `stop --force`/abort aborts this signal — kill the harness process
|
|
4531
4855
|
// group (via the same onTimeout kill that the hard-cap/idle paths use) and
|
|
@@ -4654,7 +4978,7 @@ function ptyAvailable(ptyFactory) {
|
|
|
4654
4978
|
// spawnCaptureOneShot. A PTY merges stdout+stderr into one stream, so stderr is
|
|
4655
4979
|
// always '' here; that is expected for a live terminal. `ptyFactory` is
|
|
4656
4980
|
// injectable for tests (defaults to node-pty).
|
|
4657
|
-
function spawnCapturePty({ command, args = [], cwd, env, stdinData, timeoutMs, idleTimeoutMs, recoveryWindowMs, cols = 120, rows = 30, ptyFactory, relayTap = null, stream = false, streamPrefix = '', onStreamOut, abortSignal = null }) {
|
|
4981
|
+
function spawnCapturePty({ command, args = [], cwd, env, stdinData, timeoutMs, idleTimeoutMs, recoveryWindowMs, cols = 120, rows = 30, ptyFactory, relayTap = null, stream = false, streamPrefix = '', onStreamOut, abortSignal = null, onSpawn = null }) {
|
|
4658
4982
|
return new Promise((resolve) => {
|
|
4659
4983
|
const factory = ptyFactory || loadPtyModule();
|
|
4660
4984
|
if (!factory || typeof factory.spawn !== 'function') {
|
|
@@ -4712,6 +5036,7 @@ function spawnCapturePty({ command, args = [], cwd, env, stdinData, timeoutMs, i
|
|
|
4712
5036
|
finish({ ok: false, exitCode: null, stdout: '', stderr: '', error: `pty spawn failed: ${err?.message || err}`, truncated: false, stderrTruncated: false });
|
|
4713
5037
|
return;
|
|
4714
5038
|
}
|
|
5039
|
+
if (onSpawn) { try { onSpawn(term?.pid); } catch { /* best effort */ } }
|
|
4715
5040
|
|
|
4716
5041
|
// #202: abort (stop --force) kills the PTY and settles as aborted so the job
|
|
4717
5042
|
// is failed/yielded for immediate retry rather than left to lock-lapse.
|
|
@@ -4921,7 +5246,7 @@ const ACP_MAX_LINE_BYTES = 8 * 1024 * 1024; // 8 MiB
|
|
|
4921
5246
|
// and every caller work unchanged. Because the raw stream is JSON-RPC (not human
|
|
4922
5247
|
// output), `stdout` here is the accumulated human-readable transcript text (what
|
|
4923
5248
|
// we relay), and `stderr` is the child's real stderr (agent diagnostics).
|
|
4924
|
-
function spawnCaptureAcp({ command, args = [], cwd, env, stdinData, timeoutMs, idleTimeoutMs, recoveryWindowMs, relayTap = null, stream = false, streamPrefix = '', onStreamOut, onStreamErr, permission = 'yolo', shell = false, onAcpUpdate = null, abortSignal = null }) {
|
|
5249
|
+
function spawnCaptureAcp({ command, args = [], cwd, env, stdinData, timeoutMs, idleTimeoutMs, recoveryWindowMs, relayTap = null, stream = false, streamPrefix = '', onStreamOut, onStreamErr, permission = 'yolo', shell = false, onAcpUpdate = null, abortSignal = null, onSpawn = null }) {
|
|
4925
5250
|
return new Promise((resolve) => {
|
|
4926
5251
|
const logger = getLogger();
|
|
4927
5252
|
const humanChunks = [];
|
|
@@ -5137,20 +5462,25 @@ function spawnCaptureAcp({ command, args = [], cwd, env, stdinData, timeoutMs, i
|
|
|
5137
5462
|
}
|
|
5138
5463
|
};
|
|
5139
5464
|
|
|
5140
|
-
// #110 / nanobpm/nano-ide#534: map an ACP
|
|
5141
|
-
// transcript-chunk wire form via the shared
|
|
5142
|
-
// `
|
|
5143
|
-
//
|
|
5144
|
-
// `
|
|
5145
|
-
//
|
|
5465
|
+
// #110 / nanobpm/nano-ide#534 / jwulf/c8ctl-plugin-nano#206: map an ACP
|
|
5466
|
+
// session/update to the CANONICAL transcript-chunk wire form via the shared
|
|
5467
|
+
// `@nanobpm/agentic` seams. `acpUpdateToDisplayChunk` (this plugin's producer)
|
|
5468
|
+
// wraps the canonical bridge (`classifyUpdate` composed with
|
|
5469
|
+
// `encodeTranscriptEvent`) and additionally carries the ACP `messageId` / role /
|
|
5470
|
+
// delta semantics into the shared additive `MessageEvent` contract (#566) so a
|
|
5471
|
+
// consumer folding these chunks through `deriveDisplay` reconstructs
|
|
5472
|
+
// transport-fragmented deltas into coherent blocks and keeps distinct
|
|
5473
|
+
// same-speaker messages apart — degrading to byte-identical bridge output when
|
|
5474
|
+
// the provider omits identity. It returns the exact `{ nwfTranscriptEvent: 1,
|
|
5475
|
+
// kind, … }` bytes the cockpit's `parseTranscriptEvent` decodes, or `null` for an
|
|
5146
5476
|
// update with no canonical meaning (an `ignored` classification: a plan, an
|
|
5147
5477
|
// intermediate tool_call_update, a non-text chunk, or a malformed update). No
|
|
5148
|
-
// envelope grammar or vocab is hand-rolled here
|
|
5149
|
-
//
|
|
5150
|
-
// never diverge on the wire again. `null` (and any
|
|
5151
|
-
//
|
|
5478
|
+
// envelope grammar or vocab is hand-rolled here — the marker, version, kinds and
|
|
5479
|
+
// additive fields all come from the package, so a producer and a consumer can
|
|
5480
|
+
// never diverge on the wire again. `null` (and any throw) falls through to the
|
|
5481
|
+
// minimal human-text path below, so nothing is ever dropped.
|
|
5152
5482
|
const encodeTranscriptChunk = (update) => {
|
|
5153
|
-
try { return
|
|
5483
|
+
try { return acpUpdateToDisplayChunk(update, { sessionAcp: agenticSessionAcp }); }
|
|
5154
5484
|
catch { return null; }
|
|
5155
5485
|
};
|
|
5156
5486
|
|
|
@@ -5293,6 +5623,7 @@ function spawnCaptureAcp({ command, args = [], cwd, env, stdinData, timeoutMs, i
|
|
|
5293
5623
|
finish({ ok: false, exitCode: null, stdout: '', stderr: '', error: err.message, truncated: false, stderrTruncated: false });
|
|
5294
5624
|
return;
|
|
5295
5625
|
}
|
|
5626
|
+
if (onSpawn) { try { onSpawn(child.pid); } catch { /* best effort */ } }
|
|
5296
5627
|
|
|
5297
5628
|
// #202: abort (stop --force) — finish() reaps the still-alive child via
|
|
5298
5629
|
// killTree, so settle as aborted and let it reap the ACP harness group.
|
|
@@ -5603,7 +5934,7 @@ function baseAgentEnv(profile, job) {
|
|
|
5603
5934
|
* Both paths resolve to the same result contract.
|
|
5604
5935
|
*/
|
|
5605
5936
|
function runAgentJob(profile, job, opts = {}) {
|
|
5606
|
-
const { timeoutMs, idleTimeoutMs, recoveryWindowMs, envelope, sandbox = 'none', image, runId, secretEnv = {}, passThroughSecretNames = [], cwd, extraEnv = {}, profileEnv = {}, resultFile, stream = false, streamPrefix = '', onStreamOut, onStreamErr, args: commandArgs, terminal = 'pipe', protocol = 'pipe', permission = 'yolo', relaySession = null, ptyFactory, nudgePayload = null, onAcpUpdate = null, abortSignal = null } = opts;
|
|
5937
|
+
const { timeoutMs, idleTimeoutMs, recoveryWindowMs, envelope, sandbox = 'none', image, runId, secretEnv = {}, passThroughSecretNames = [], cwd, extraEnv = {}, profileEnv = {}, resultFile, stream = false, streamPrefix = '', onStreamOut, onStreamErr, args: commandArgs, terminal = 'pipe', protocol = 'pipe', permission = 'yolo', relaySession = null, ptyFactory, nudgePayload = null, onAcpUpdate = null, abortSignal = null, onSpawn = null } = opts;
|
|
5607
5938
|
// #110: `protocol`/`permission` drive the ACP executor branch below. The
|
|
5608
5939
|
// pipe/PTY paths are unchanged, so `protocol === 'pipe'` behaviour is identical.
|
|
5609
5940
|
// A `nudgePayload` (#678) carries the bespoke "re-emit your result" prompt for a
|
|
@@ -5693,6 +6024,7 @@ function runAgentJob(profile, job, opts = {}) {
|
|
|
5693
6024
|
permission,
|
|
5694
6025
|
onAcpUpdate,
|
|
5695
6026
|
abortSignal,
|
|
6027
|
+
onSpawn,
|
|
5696
6028
|
});
|
|
5697
6029
|
}
|
|
5698
6030
|
|
|
@@ -5716,6 +6048,7 @@ function runAgentJob(profile, job, opts = {}) {
|
|
|
5716
6048
|
streamPrefix,
|
|
5717
6049
|
onStreamOut,
|
|
5718
6050
|
abortSignal,
|
|
6051
|
+
onSpawn,
|
|
5719
6052
|
});
|
|
5720
6053
|
}
|
|
5721
6054
|
|
|
@@ -5740,6 +6073,7 @@ function runAgentJob(profile, job, opts = {}) {
|
|
|
5740
6073
|
onStreamErr,
|
|
5741
6074
|
relayTap,
|
|
5742
6075
|
abortSignal,
|
|
6076
|
+
onSpawn,
|
|
5743
6077
|
});
|
|
5744
6078
|
}
|
|
5745
6079
|
|
|
@@ -5811,6 +6145,11 @@ function runAgentJob(profile, job, opts = {}) {
|
|
|
5811
6145
|
onStreamOut,
|
|
5812
6146
|
onStreamErr,
|
|
5813
6147
|
relayTap,
|
|
6148
|
+
// #205: thread the harness-PID callback through the container path too, so a
|
|
6149
|
+
// crashed worker's in-flight job marker records the spawned client PID and
|
|
6150
|
+
// orphan reclamation can tell a possibly-surviving harness from an abandoned
|
|
6151
|
+
// namespace (an empty `harnessPids` otherwise forces indefinite retention).
|
|
6152
|
+
onSpawn,
|
|
5814
6153
|
onTimeout: (child) => {
|
|
5815
6154
|
try { spawnSync(engine, ['rm', '-f', containerName], { timeout: 15_000 }); } catch { /* best effort */ }
|
|
5816
6155
|
try { killTree(child); } catch { /* best effort */ }
|
|
@@ -7218,15 +7557,50 @@ async function workAgent(req, flags) {
|
|
|
7218
7557
|
let reaperTimer = null;
|
|
7219
7558
|
let runDirTimer = null;
|
|
7220
7559
|
|
|
7560
|
+
// --- Cross-process-safe run/result namespace (issue #205) ----------------
|
|
7561
|
+
// Every worker PROCESS gets a fresh incarnation id and its OWN namespace under
|
|
7562
|
+
// agent-runs/worker-<incarnation>/. All of this process's run-*/res- dirs live
|
|
7563
|
+
// there, and its ordinary reaper only ever traverses that namespace, so it can
|
|
7564
|
+
// never delete a sibling worker's active checkout or result channel (the
|
|
7565
|
+
// data-loss defect this issue fixes). The immutable ownership record is
|
|
7566
|
+
// published atomically before any child dir is created.
|
|
7567
|
+
const workerIncarnation = newIncarnationId();
|
|
7568
|
+
const workerPidStart = pidStartToken(process.pid);
|
|
7569
|
+
let pluginVersion = null;
|
|
7570
|
+
try { pluginVersion = JSON.parse(readFileSync(join(pluginDir, 'package.json'), 'utf-8')).version ?? null; } catch { /* best effort */ }
|
|
7571
|
+
let workerNsDir;
|
|
7572
|
+
try {
|
|
7573
|
+
({ nsDir: workerNsDir } = allocateWorkerNamespace({
|
|
7574
|
+
incarnation: workerIncarnation,
|
|
7575
|
+
worker: profile?.name ?? null,
|
|
7576
|
+
pid: process.pid,
|
|
7577
|
+
pidStart: workerPidStart,
|
|
7578
|
+
version: pluginVersion,
|
|
7579
|
+
}));
|
|
7580
|
+
logger.info(`Worker namespace ${basename(workerNsDir)} (incarnation ${workerIncarnation}, pid ${process.pid}) — run/result dirs are isolated here.`);
|
|
7581
|
+
} catch (err) {
|
|
7582
|
+
logger.error(`Could not allocate the worker run/result namespace under ${agentRunsRoot()}: ${err.message}`);
|
|
7583
|
+
process.exit(1);
|
|
7584
|
+
}
|
|
7585
|
+
|
|
7221
7586
|
// Run-dir hygiene runs regardless of sandbox: any sandbox=none job that carries
|
|
7222
|
-
// a repository clones a throwaway workspace under
|
|
7223
|
-
//
|
|
7587
|
+
// a repository clones a throwaway workspace under this worker's namespace, and a
|
|
7588
|
+
// crashed job handler can leave one behind. OWNER-SCOPED: bounded to our own
|
|
7589
|
+
// namespace, age-gated, and skipping in-flight dirs (liveRunDirs) — never a
|
|
7590
|
+
// sibling's namespace or a legacy flat run-*/res- dir. A SEPARATE,
|
|
7591
|
+
// cross-process-safe reclamation sweep handles other incarnations' abandoned namespaces.
|
|
7224
7592
|
{
|
|
7225
|
-
const initialRuns =
|
|
7226
|
-
if (initialRuns.reaped > 0) logger.info(`Reaped ${initialRuns.reaped} leftover job workspace(s) at startup.`);
|
|
7593
|
+
const initialRuns = reapOwnedNamespace(workerNsDir, { maxAgeMs: reapAgeMs, liveRunDirs, logger, incarnation: workerIncarnation });
|
|
7594
|
+
if (initialRuns.reaped > 0) logger.info(`Reaped ${initialRuns.reaped} leftover job workspace(s) in this worker's namespace at startup.`);
|
|
7595
|
+
if (initialRuns.error) logger.warn(`Startup workspace reap warning: ${initialRuns.error}`);
|
|
7596
|
+
const reclaimStartup = reclaimOrphanNamespaces({ selfIncarnation: workerIncarnation, minAgeMs: reapAgeMs, logger });
|
|
7597
|
+
if (reclaimStartup.reclaimed.length > 0) logger.info(`Reclaimed ${reclaimStartup.reclaimed.length} abandoned worker namespace(s) at startup.`);
|
|
7227
7598
|
runDirTimer = setInterval(() => {
|
|
7228
|
-
const r =
|
|
7229
|
-
if (r.reaped > 0) logger.info(`Reaper removed ${r.reaped}
|
|
7599
|
+
const r = reapOwnedNamespace(workerNsDir, { maxAgeMs: reapAgeMs, liveRunDirs, logger, incarnation: workerIncarnation });
|
|
7600
|
+
if (r.reaped > 0) logger.info(`Reaper removed ${r.reaped} aged, not-in-flight job workspace(s) from this worker's namespace.`);
|
|
7601
|
+
if (r.error) logger.warn(`Workspace reaper warning: ${r.error}`);
|
|
7602
|
+
const rc = reclaimOrphanNamespaces({ selfIncarnation: workerIncarnation, minAgeMs: reapAgeMs, logger });
|
|
7603
|
+
if (rc.reclaimed.length > 0) logger.info(`Reclaimed ${rc.reclaimed.length} abandoned worker namespace(s).`);
|
|
7230
7604
|
}, reapIntervalMs);
|
|
7231
7605
|
if (typeof runDirTimer.unref === 'function') runDirTimer.unref();
|
|
7232
7606
|
}
|
|
@@ -7724,8 +8098,8 @@ async function workAgent(req, flags) {
|
|
|
7724
8098
|
const authRef = envelope.repository.authRef;
|
|
7725
8099
|
repoToken = githubCloneToken({ provider, authRef, secretResolver }); // absent → anonymous clone
|
|
7726
8100
|
try {
|
|
7727
|
-
mkdirSync(
|
|
7728
|
-
runDir = mkdtempSync(join(
|
|
8101
|
+
mkdirSync(workerNsDir, { recursive: true });
|
|
8102
|
+
runDir = mkdtempSync(join(workerNsDir, 'run-'));
|
|
7729
8103
|
liveRunDirs.add(runDir);
|
|
7730
8104
|
provisioned = provisionRepo({ envelope, token: repoToken, runDir, timeoutMs: cloneTimeoutMs });
|
|
7731
8105
|
if (provisioned.baseFetchError) {
|
|
@@ -7774,14 +8148,14 @@ async function workAgent(req, flags) {
|
|
|
7774
8148
|
// `cd` to a known absolute path). True confinement is the container
|
|
7775
8149
|
// increment; a provisioned repository envelope stays the preferred path.
|
|
7776
8150
|
try {
|
|
7777
|
-
mkdirSync(
|
|
7778
|
-
runDir = mkdtempSync(join(
|
|
8151
|
+
mkdirSync(workerNsDir, { recursive: true });
|
|
8152
|
+
runDir = mkdtempSync(join(workerNsDir, 'run-'));
|
|
7779
8153
|
liveRunDirs.add(runDir);
|
|
7780
8154
|
cwd = runDir;
|
|
7781
8155
|
} catch (err) {
|
|
7782
8156
|
if (runDir) { try { rmSync(runDir, { recursive: true, force: true }); } catch { /* best effort */ } liveRunDirs.delete(runDir); runDir = null; }
|
|
7783
8157
|
const retries = Math.max(0, (Number(job.retries) || 1) - 1);
|
|
7784
|
-
const msg = `could not create a temp workspace under the
|
|
8158
|
+
const msg = `could not create a temp workspace under the worker namespace: ${err.message}`;
|
|
7785
8159
|
logger.warn(`[${jobType}] job ${job.jobKey} not provisioned — ${msg}; retries left ${retries}`);
|
|
7786
8160
|
return settle.fail(job.jobKey, { errorMessage: msg.slice(0, 2000), retries, retryBackOff: 15_000 });
|
|
7787
8161
|
}
|
|
@@ -7805,18 +8179,29 @@ async function workAgent(req, flags) {
|
|
|
7805
8179
|
let resultFile = null;
|
|
7806
8180
|
try {
|
|
7807
8181
|
try {
|
|
7808
|
-
mkdirSync(
|
|
7809
|
-
resultDir = mkdtempSync(join(
|
|
8182
|
+
mkdirSync(workerNsDir, { recursive: true });
|
|
8183
|
+
resultDir = mkdtempSync(join(workerNsDir, 'res-'));
|
|
7810
8184
|
resultFile = join(resultDir, 'result.json');
|
|
7811
8185
|
// Track it so the run-dir reaper skips it while in-flight and reaps it
|
|
7812
8186
|
// (as a `res-*` dir) if this worker crashes before the cleanup below.
|
|
7813
8187
|
liveRunDirs.add(resultDir);
|
|
7814
8188
|
} catch { resultDir = null; resultFile = null; }
|
|
7815
8189
|
|
|
8190
|
+
// Record an in-flight job marker in this worker's namespace BEFORE the
|
|
8191
|
+
// harness starts (#205). A later orphan reclamation of a dead owner uses
|
|
8192
|
+
// it to detect a possibly-surviving harness: an empty `harnessPids`
|
|
8193
|
+
// (registration race) forces conservative retention; once `onSpawn`
|
|
8194
|
+
// records the harness PID, reclamation can probe whether it outlived us.
|
|
8195
|
+
writeJobMarker(workerNsDir, { jobKey: job.jobKey, workerPid: process.pid, incarnation: workerIncarnation });
|
|
8196
|
+
|
|
7816
8197
|
const runOpts = {
|
|
7817
8198
|
timeoutMs: effectiveHardCapMs,
|
|
7818
8199
|
idleTimeoutMs: effectiveIdleTimeoutMs,
|
|
7819
8200
|
recoveryWindowMs: effectiveRecoveryWindowMs,
|
|
8201
|
+
// #205: record the spawned harness's PID (its process-group leader) in
|
|
8202
|
+
// the in-flight job marker so cross-process orphan reclamation can tell
|
|
8203
|
+
// a still-running orphaned harness from a genuinely abandoned namespace.
|
|
8204
|
+
onSpawn: (pid) => recordHarnessPid(workerNsDir, job.jobKey, pid),
|
|
7820
8205
|
// #202: the supervisor fiber's interruption AbortSignal (a
|
|
7821
8206
|
// `stop --force`/abort). runAgentJob wires it to killTree the harness
|
|
7822
8207
|
// process group so an abort CANCELS the work instead of orphaning the
|
|
@@ -7931,6 +8316,10 @@ async function workAgent(req, flags) {
|
|
|
7931
8316
|
}
|
|
7932
8317
|
} finally {
|
|
7933
8318
|
if (isContainer) liveRunIds.delete(runId);
|
|
8319
|
+
// #205: clear the in-flight job marker now the harness has stopped — the
|
|
8320
|
+
// owning lifecycle's finally is the authoritative "job finished" signal,
|
|
8321
|
+
// so this namespace no longer holds a surviving harness for this job.
|
|
8322
|
+
removeJobMarker(workerNsDir, job.jobKey);
|
|
7934
8323
|
if (runDir && !keepRuns) { try { rmSync(runDir, { recursive: true, force: true }); } catch { /* best effort */ } }
|
|
7935
8324
|
if (runDir) liveRunDirs.delete(runDir);
|
|
7936
8325
|
// Emit the relay session's `phase:close` lifecycle event and drain its
|
|
@@ -8189,6 +8578,12 @@ async function workAgent(req, flags) {
|
|
|
8189
8578
|
} catch (err) {
|
|
8190
8579
|
logger.warn(`supervisor shutdown error — runtime loop may not have shut down cleanly: ${err?.message || err}`);
|
|
8191
8580
|
}
|
|
8581
|
+
// #205: tear down this worker's own namespace on a clean exit. All in-flight
|
|
8582
|
+
// jobs' finallys have run by now (markers cleared, run/res dirs removed), so
|
|
8583
|
+
// nothing here is in use. Skipped under --keep-runs so kept workspaces stay
|
|
8584
|
+
// for debugging and age out via cross-process reclamation, exactly like the
|
|
8585
|
+
// pre-fix age-gated behavior.
|
|
8586
|
+
if (workerNsDir && !keepRuns) { try { rmSync(workerNsDir, { recursive: true, force: true }); } catch { /* best effort */ } }
|
|
8192
8587
|
resolve();
|
|
8193
8588
|
};
|
|
8194
8589
|
|
|
@@ -13422,6 +13817,20 @@ export {
|
|
|
13422
13817
|
isPlaceholderEmail,
|
|
13423
13818
|
postAgentAttribution,
|
|
13424
13819
|
reapAgentRunDirs,
|
|
13820
|
+
reapChildRunDirs,
|
|
13821
|
+
reapOwnedNamespace,
|
|
13822
|
+
reclaimOrphanNamespaces,
|
|
13823
|
+
allocateWorkerNamespace,
|
|
13824
|
+
workerNamespaceDir,
|
|
13825
|
+
readOwnerRecord,
|
|
13826
|
+
incarnationLiveness,
|
|
13827
|
+
pidStartToken,
|
|
13828
|
+
newIncarnationId,
|
|
13829
|
+
writeJobMarker,
|
|
13830
|
+
recordHarnessPid,
|
|
13831
|
+
removeJobMarker,
|
|
13832
|
+
readJobMarkers,
|
|
13833
|
+
namespaceHasLiveHarness,
|
|
13425
13834
|
authUrl,
|
|
13426
13835
|
githubCloneToken,
|
|
13427
13836
|
ghAuthTokenFromCli,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "c8ctl-plugin-nano",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.58.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "c8ctl plugin to start, inspect, and stop a local Nano BPM (nanobpmn) cluster",
|
|
6
6
|
"main": "c8ctl-plugin.js",
|
|
@@ -22,6 +22,7 @@
|
|
|
22
22
|
"files": [
|
|
23
23
|
"c8ctl-plugin.js",
|
|
24
24
|
"agent-instance.mjs",
|
|
25
|
+
"acp-transcript-producer.mjs",
|
|
25
26
|
"platforms.mjs",
|
|
26
27
|
"agentic.mjs",
|
|
27
28
|
"agentic-loader-hook.mjs",
|
|
@@ -67,17 +68,17 @@
|
|
|
67
68
|
"typescript": "^5.9.3"
|
|
68
69
|
},
|
|
69
70
|
"dependencies": {
|
|
70
|
-
"@nanobpm/agentic": "^0.
|
|
71
|
-
"@nanobpm/urban-agent-client": "^0.1.
|
|
71
|
+
"@nanobpm/agentic": "^0.14.0",
|
|
72
|
+
"@nanobpm/urban-agent-client": "^0.1.14"
|
|
72
73
|
},
|
|
73
74
|
"optionalDependencies": {
|
|
74
75
|
"node-pty": "^1.0.0",
|
|
75
|
-
"@nanobpm/c8ctl-plugin-nano-darwin-arm64": "1.
|
|
76
|
-
"@nanobpm/c8ctl-plugin-nano-darwin-x64": "1.
|
|
77
|
-
"@nanobpm/c8ctl-plugin-nano-linux-x64": "1.
|
|
78
|
-
"@nanobpm/c8ctl-plugin-nano-linux-arm64": "1.
|
|
79
|
-
"@nanobpm/c8ctl-plugin-nano-linux-armv7": "1.
|
|
80
|
-
"@nanobpm/c8ctl-plugin-nano-linux-armv6": "1.
|
|
81
|
-
"@nanobpm/c8ctl-plugin-nano-win32-x64": "1.
|
|
76
|
+
"@nanobpm/c8ctl-plugin-nano-darwin-arm64": "1.58.0",
|
|
77
|
+
"@nanobpm/c8ctl-plugin-nano-darwin-x64": "1.58.0",
|
|
78
|
+
"@nanobpm/c8ctl-plugin-nano-linux-x64": "1.58.0",
|
|
79
|
+
"@nanobpm/c8ctl-plugin-nano-linux-arm64": "1.58.0",
|
|
80
|
+
"@nanobpm/c8ctl-plugin-nano-linux-armv7": "1.58.0",
|
|
81
|
+
"@nanobpm/c8ctl-plugin-nano-linux-armv6": "1.58.0",
|
|
82
|
+
"@nanobpm/c8ctl-plugin-nano-win32-x64": "1.58.0"
|
|
82
83
|
}
|
|
83
84
|
}
|