threadwire 0.1.5 → 0.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/CHANGELOG.md +40 -0
  2. package/README.md +17 -4
  3. package/TELEGRAM-INGRESS.md +8 -2
  4. package/bin/isolated-runtime.js +5 -0
  5. package/bin/model-broker.js +5 -0
  6. package/docs/card-10520-plan.md +45 -0
  7. package/docs/container-runtime.md +3 -2
  8. package/docs/delegated-result-protocol.md +161 -0
  9. package/docs/evidence-artifacts.md +106 -0
  10. package/docs/isolated-provider-runtime.md +137 -0
  11. package/package.json +5 -1
  12. package/scripts/provider-shims/front-door.sh.template +18 -0
  13. package/scripts/verify-package.js +22 -1
  14. package/src/absolute-deadline.js +94 -0
  15. package/src/cli.js +362 -42
  16. package/src/context-budget-metrics.js +180 -0
  17. package/src/delegated-result-admission.js +377 -0
  18. package/src/docker-api.js +131 -0
  19. package/src/evidence-store.js +1472 -0
  20. package/src/isolated-runtime-client.js +149 -0
  21. package/src/isolated-runtime.js +982 -0
  22. package/src/isolated-state.js +409 -0
  23. package/src/isolated-worker.js +123 -0
  24. package/src/model-broker-policy.js +139 -0
  25. package/src/model-broker.js +313 -0
  26. package/src/mount-policy.js +28 -0
  27. package/src/normalized-output.js +68 -0
  28. package/src/notice-queue.js +22 -7
  29. package/src/providers/opencode.js +42 -18
  30. package/src/relay-write.js +44 -0
  31. package/src/relay.js +6 -6
  32. package/src/run-worker.js +158 -40
  33. package/src/telegram-ingress/command.js +16 -0
  34. package/src/telegram-ingress/config.js +96 -11
  35. package/src/telegram-ingress/core.js +201 -89
  36. package/src/telegram-ingress/http.js +17 -1
  37. package/src/telegram-webhook.js +22 -0
  38. package/src/types.js +2 -2
  39. package/src/worker-control.js +294 -0
  40. package/src/hermes-protocol.js +0 -126
package/CHANGELOG.md ADDED
@@ -0,0 +1,40 @@
1
+ # Changelog
2
+
3
+ ## Unreleased
4
+
5
+ ## 0.1.8 - 2026-07-26
6
+
7
+ - Add `threadwire run --transcript <path>` to durably capture the exact normalized
8
+ stdout JSONL protocol in a private, exclusive file while keeping activity logs
9
+ and scoped raw evidence artifacts separate.
10
+ - Add an opt-in, fail-closed isolated Codex write runtime. A root-owned
11
+ supervisor launches one worktree-only worker per task while a separate narrow
12
+ model broker retains upstream credentials. Fresh/resume, secret rotation,
13
+ filesystem/network/socket escapes, configuration overrides and cleanup are
14
+ covered by adversarial Docker E2E.
15
+ - Bind broker grants to dedicated trusted run-network listeners and an exact
16
+ allowlisted model/request schema; add stable worktree ancestry checks,
17
+ task-private persistent resume state, contained nested cwd and structured
18
+ Codex arguments. The Compose deployment is now an explicit opt-in override.
19
+ - Allocate authenticated, uniquely labeled state per fresh Codex lineage;
20
+ reject volume adoption and nested worktree mounts; use Docker's structured
21
+ host-bind API; and require an explicit supervisor-side model allowlist.
22
+ - Bound and eagerly expire preflight capabilities, serialize writable lineage
23
+ resumes, and automatically reconcile and garbage-collect authenticated state
24
+ volumes with restart-safe cleanup retry.
25
+ - Bound active runs and Docker/broker operations through cleanup, reserve state
26
+ before awaited revalidation, autonomously close expired grant listeners, and
27
+ reconcile exactly owned workers and networks across every container state.
28
+ - Abort accepted broker work and caller-disconnected runs at one absolute
29
+ deadline; make crash cleanup authority an fsync'd authenticated full run
30
+ identity rather than labels; and durably fsync registry rename evidence.
31
+ - Carry one launch deadline from preflight through prompt/secret/evidence/
32
+ Telegram setup, worker execution and cleanup; make supervisor and broker
33
+ shutdown abort control uploads and active work; move Docker GC outside the
34
+ registry transaction; and stream-cap broker/runtime responses.
35
+ - Gate worker start on one-shot activation of a verified pending broker grant
36
+ and address its listener by the inspected per-run IP, removing dynamic
37
+ network-alias readiness races.
38
+ - Route relay-profile Telegram Codex jobs exclusively through the isolated
39
+ runtime, keep validate-only launches local, preserve bounded stdout/stderr
40
+ evidence channels, and protect live lineages from concurrent reconciliation.
package/README.md CHANGED
@@ -7,6 +7,13 @@ non-root Codex provider, Docker secrets, loopback TLS proxy, verification, and
7
7
  rollback path, see [Container runtime](docs/container-runtime.md). The legacy
8
8
  host launcher below remains supported and is separate from that deployment.
9
9
 
10
+ Relay-originated Codex writes are explicit and default off. `--relay-write`
11
+ requires a reviewed `--workspace-profile` plus the root-owned isolated runtime
12
+ and credential broker. The worker gets one writable task worktree, no upstream
13
+ or Git credential, and no route except its run-scoped broker. Missing
14
+ prerequisites fail before prompt or credential reads with no native fallback.
15
+ See [Isolated provider runtime](docs/isolated-provider-runtime.md).
16
+
10
17
  ## Run with npx
11
18
 
12
19
  Threadwire requires Node.js 22 or newer. Run the published CLI without a global
@@ -67,19 +74,25 @@ Threadwire directly spawns the structurally separate terminal adapters `/opt/dat
67
74
 
68
75
  The relay contract: the caller supplies `THREADWIRE_TARGET` (`telegram:<chat-id>` or `telegram:<chat-id>:<thread-id>`) and, optionally, `THREADWIRE_PROCESS_NUMBER`. The prompt arrives on a piped stdin; all native provider flags pass through after `--`. A positional prompt is never parsed. A missing `THREADWIRE_TARGET` fails closed (exit code 2) and a malformed one is rejected by the launcher's token-free preflight — the shim never invents a default destination and never silently downgrades a requested relay to a direct run. A shim relays only for a worker mode (`codex exec …`, `opencode-local-fleet run …`, or any non-interactive `claude …`); it drops the worker subcommand before relaying and Threadwire re-adds it around the prompt. Version and help queries, non-worker subcommands, interactive TTY sessions, and any invocation already inside a relayed run (`THREADWIRE_ACTIVE=1`, so nested provider workers stay local/native) run the real CLI directly through the libexec adapter.
69
76
 
70
- Pass `--resume-session <exact-provider-session-id>` to continue a stored Codex, Claude, or OpenCode session. Threadwire preserves the provider's native resume ordering and streaming options and never uses implicit latest-session behavior. To request a local JSONL activity log, pass `--activity-log <path>`. It writes only safe local facts: the selected workspace profile plus immutable workspace provenance, provider-started facts (`provider`, child `pid`), and session-available facts (`provider`, native `sessionId`). Raw events, reasoning, tool data, stderr, credentials, prompts, and Telegram token data are excluded. This activity log is local-only and is never enqueued for Telegram delivery.
77
+ Pass `--resume-session <exact-provider-session-id>` to continue a stored Codex, Claude, or OpenCode session. Threadwire preserves the provider's native resume ordering and streaming options and never uses implicit latest-session behavior.
78
+
79
+ Pass `--transcript <path>` to capture the exact normalized, caller-facing JSONL protocol written to run stdout. The file is created privately (mode `0600`), must not already exist, and contains the same records in the same byte order as stdout. It never contains provider JSONL, reasoning, tool payloads, prompts, stderr, or credentials.
80
+
81
+ `--activity-log <path>` is different: it retains its append contract and writes only minimal safe lifecycle facts—the selected workspace profile and immutable provenance, provider start (`provider`, child `pid`), and available native session ID. It is local-only and never enters Telegram or normalized stdout.
71
82
 
72
83
  Provider stdout is decoded as JSONL with a finite 1 MiB UTF-8 byte limit for each pending unterminated record. Exceeding it rejects the run, removes listeners and signal handlers, and terminates the provider without parsing or rendering the oversized content.
73
84
 
74
- For normal `threadwire run` execution, Threadwire's own stdout is a separate caller-facing JSON Lines protocol intended for an outer Hermes launcher. Each line is one JSON object and has no `version` key. It includes normalized `worker_started`, `worker_completed`/`worker_failed`, `tool_started`, `tool_finished`, and suitable `diagnostic` records, followed by exactly one terminal `{"type":"completed","exitCode":0,"assistantText":"..."}` record. When `--workspace-profile` is used, Threadwire first emits one `{"type":"execution_environment",...}` record containing the resolved profile name, in-container repository path, exact Git revision, and immutable worktree identity. Setup, worker, and relay-close errors use terminal exit code 2 while retaining the stderr diagnostic. The terminal `assistantText` is optional and is assembled from parsed assistant text deltas; deltas themselves are never streamed to stdout. Assistant text is credential-redacted, control-normalized, and capped at 16,384 Unicode code points. Tool names, details, and outputs use the relay's credential redaction and control normalization and are capped at 128, 1,024, and 4,096 code points respectively. Duplicate lifecycle phases and tool transitions are suppressed, and at most 64 nonterminal records are emitted per run; the terminal record is always exempt from that limit.
85
+ For normal `threadwire run` execution, Threadwire stdout is the parent-model admission plane. It emits exactly one versioned terminal `delegated_result` JSON object and no progress records. The envelope contains a bounded, credential-redacted, control-normalized final conclusion when one was produced, structured terminal state and exit code, and the exact bounded provider session handle when available. Its closed schema can also represent concise blocker/decision requests, opaque artifact handles, commit/URL references, and a concise validation summary; current provider adapters do not infer these optional fields from prose. Unknown, nested, wrongly typed, cardinality-invalid, or oversized admission candidates fail closed. See [Delegated-result protocol](docs/delegated-result-protocol.md) for the complete schema, limits, and caller obligations.
86
+
87
+ Scoped raw evidence artifacts are separate from both files above. Set the trusted operator setting `THREADWIRE_EVIDENCE_ROOT` for standalone runs to persist the delegated prompt and raw provider stdout/stderr as a scoped file-backed artifact; Telegram ingress uses its dedicated evidence volume automatically. The parent or owning Telegram topic receives only an opaque bearer handle. Retrieve later with `threadwire evidence read` or the authenticated `/evidence` Telegram command and an explicit bounded selector. See [Evidence artifacts](docs/evidence-artifacts.md) for permissions, quotas, retention, recovery, authorization, redaction, privacy, and cleanup behavior.
75
88
 
76
- This Hermes stdout channel does not replace Telegram: the existing human-readable Telegram relay continues independently. Provider stdout remains private input to Threadwire's JSONL parser and is never copied to Threadwire stdout; raw provider records, raw stdout, stderr, reasoning, prompts, tool inputs, and secrets are excluded. Threadwire's own error diagnostics remain on stderr. Help output is the sole non-run, human-readable stdout mode.
89
+ Telegram is the separate user-visible progress plane. Lifecycle notices, diagnostics, tool activity, and assistant streaming continue there independently; `--tool-messages` affects only Telegram. Execution-environment details, provider records and streams, lifecycle progress, diagnostics, tool names/arguments/details/output, stderr, prompts, narration/reasoning, and full transcripts never enter stdout. Threadwire's bounded error diagnostic remains on stderr. Help output is the sole non-run, human-readable stdout mode. This default is an intentional pre-1.0 protocol break from the former mixed Hermes JSONL stream.
77
90
 
78
91
  The Telegram bot token is used only by Threadwire's notifier and both `TELEGRAM_BOT_TOKEN` and `THREADWIRE_TELEGRAM_BOT_TOKEN` are removed from every provider child environment. The three libexec adapters repeat that scrub as defense in depth. Other environment variables, including the providers' own authentication and normal process configuration, are preserved. Outbound Telegram fetch and response-body parsing share a 30-second deadline by default; set `THREADWIRE_TELEGRAM_REQUEST_TIMEOUT_MS` to a positive safe integer no greater than `2147483647` to override it.
79
92
 
80
93
  A single trusted renderer is the only code allowed to enqueue Telegram notices. By default, parsed assistant text is forwarded in order with every nonblank rendered line labeled `[P<n>] Assistant: `; no CLI flag is required. This preserves Markdown, code-block indentation, blank lines, and whitespace attached to nonblank content while preventing assistant text from imitating trusted activity on a later line. Streamed fenced JavaScript blocks (` ```js ` / ` ```javascript `) are instead emitted as independently valid, escaped Telegram HTML code messages: the `[P<n>] Assistant:` label stays above the code block, and long blocks are Unicode-safe chunked without exposing raw fences. A short blank-only stream is suppressed at normal close. Because retention is bounded, a blank-only prefix longer than the buffer limit is emitted in oldest-first chunks while at most one limit's worth remains deferred; a later visible assistant character therefore still reconstructs the input exactly, but an indefinitely blank stream cannot be suppressed indefinitely. With `--tool-messages`, tool activity uses a single edit-in-place status line (silent `🛠 …` on start, edited to begin `✅` on finish); without it, both the send and completion edit are suppressed. Worker start/completion/failure and warning/error diagnostics continue to use fixed labeled templates in either mode. Non-assistant provider fields—including summaries, tool names, metadata IDs, inputs, commands, stderr content, raw JSON, reasoning/thinking, tokens, and retry internals—are not forwarded. When tool messages are enabled, the sole provider-data exception is normalized Codex command-completion output: it is appended only to its original tool status edit as an escaped, bounded, expandable Telegram HTML blockquote after shared credential redaction.
81
94
 
82
- Assistant deltas from all provider stream IDs share one ordered sentence-boundary buffer, preserving provider-event order. The buffer has a finite 16,384-character default: a continuously active punctuation-free stream emits and clears Unicode-safe fixed-size chunks instead of growing indefinitely. Structured events flush any earlier assistant tail, including retained leading whitespace, before their own fixed notice. Completed boundaries enter delivery without a deliberate batching delay; messages combine only when already naturally queued behind delivery. Notices remain ordered, size-limited, paced, and retried conservatively when Telegram reports throttling or a transient failure.
95
+ Assistant deltas from all provider stream IDs share one ordered sentence-boundary buffer, preserving provider-event order. The buffer has a finite 16,384-character default: a continuously active punctuation-free stream emits and clears Unicode-safe fixed-size chunks instead of growing indefinitely. Structured events flush any earlier assistant tail, including retained leading whitespace, before their own fixed notice. Completed boundaries enter delivery without a deliberate batching delay; messages combine only when already naturally queued behind delivery. When Telegram briefly falls behind, Threadwire emits a fixed safe notice that the notifier saturated and then coalesces only eligible assistant progress until headroom returns; lifecycle, diagnostics, and tool activity remain ordered and non-droppable. Notices remain ordered, size-limited, paced, and retried conservatively when Telegram reports throttling or a transient failure.
83
96
 
84
97
  The delivery queue has finite defaults of 100 pending notices and 256 KiB of pending UTF-8 text. Message length and capacity settings require positive safe integers; pacing settings require nonnegative safe integers. A notice larger than the byte limit or a limit-plus-one enqueue latches a terminal capacity error, discards queued (not already in-flight) notices, and rejects every later enqueue. Unicode chunks never split a surrogate pair, and every labeled continuation retains its complete process label. If the label cannot fit with content under a configured message length, delivery latches a terminal chunking error before sending. These rules make delivery loss visible and prevent slow or rate-limited transports from consuming unbounded memory.
85
98
 
@@ -1,6 +1,9 @@
1
1
  # Telegram Ingress for Threadwire
2
2
 
3
- Standalone Threadwire webhook service that accepts Telegram Bot API updates and launches Threadwire's existing worker path (`createProvider` → Telegram transport/sender → `NoticeQueue` → `Relay` → `runWorker`). Hermes is not involved.
3
+ Standalone Threadwire webhook service that accepts Telegram Bot API updates.
4
+ Native providers use the existing worker path; when isolated-runtime authority
5
+ is configured, Codex uses isolated preflight/run exclusively with no native
6
+ fallback. Hermes is not involved.
4
7
 
5
8
  ## Important bot ownership
6
9
 
@@ -49,7 +52,10 @@ The process binds to **loopback (`127.0.0.1`) by default**. Expose it only throu
49
52
  - Provider child env strips Telegram tokens, webhook secret, allowlists, bind/port, concurrency, tool-message, and update-guard service config. `runWorker` also strips Telegram tokens.
50
53
  - No logging of headers, bodies, prompts, or secrets.
51
54
  - Webhook secret must be at least 32 characters after trim.
52
- - Accepted work returns `202` only after the worker child has actually spawned (`onSpawn`); process completion alone is never treated as dispatch success.
55
+ - Native work returns `202` only after the worker child has actually spawned
56
+ (`onSpawn`). Isolated Codex work returns `202` only after the authenticated
57
+ isolated run completes successfully; preflight alone is never treated as
58
+ dispatch success.
53
59
  - Failure before spawn rejects the request with `500` and is reported once via the operational error reporter.
54
60
  - Failure after spawn is reported once via the worker-failure reporter (not double-reported through both channels in the real entrypoint wiring).
55
61
  - Authenticated irrelevant updates return `200`.
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env node
2
+
3
+ import {startIsolatedRuntime} from "../src/isolated-runtime.js"
4
+
5
+ await startIsolatedRuntime()
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env node
2
+
3
+ import {startModelBroker} from "../src/model-broker.js"
4
+
5
+ await startModelBroker()
@@ -0,0 +1,45 @@
1
+ # Card 10520 scoped implementation plan
2
+
3
+ ## Existing information flow
4
+
5
+ 1. `src/cli.js` reads the prompt, resolves the provider command, and gives both to
6
+ `runWorker`.
7
+ 2. `src/run-worker.js` spawns the provider, incrementally frames stdout JSONL,
8
+ calls `onRecord` with each raw provider record, normalizes it through the
9
+ provider parser, and serially calls `onEvent`. Stderr is currently reduced to
10
+ one diagnostic; its bytes are discarded.
11
+ 3. `src/cli.js` sends normalized events to `WorkerControl`, which renders bounded
12
+ progress to the explicit Telegram target. `ActivityLog` separately records
13
+ only workspace provenance, PID, and resumable session identifiers.
14
+ 4. The same normalized stream reaches `DelegatedResultAdmission`, but only
15
+ assistant text is accumulated. On termination, admission writes exactly one
16
+ bounded `delegated_result` JSON object to stdout. That stdout object is the
17
+ parent-facing result; tool/provider records are not emitted there.
18
+
19
+ The current boundary therefore avoids most raw-event admission, but detailed
20
+ prompt/provider/tool/stderr evidence is discarded or reduced and cannot be
21
+ retrieved through an opaque, scoped contract.
22
+
23
+ ## Ownership and implementation
24
+
25
+ `EvidenceStore` is the sole persistence and retrieval owner. `runWorker`
26
+ remains the streaming/process-lifecycle owner and exposes raw stdout and stderr
27
+ chunks to one evidence sink with its existing serial backpressure. `cli.js` and
28
+ authenticated Telegram `dispatchWorker()` each own one run-scoped artifact and
29
+ finalize it with the worker lifecycle. CLI admission and the owning Telegram
30
+ topic receive only the bearer handle. `ActivityLog` does not duplicate evidence.
31
+
32
+ Work proceeds RED-GREEN-REFACTOR:
33
+
34
+ 1. Add failing store and real CLI-pipeline tests proving the current retrieval
35
+ gap and specifying opaque IDs, permissions, atomic lifecycle, quotas,
36
+ recovery, cleanup, scope isolation, bounded reads, UTF-8/binary behavior, and
37
+ streaming redaction.
38
+ 2. Implement the pinned file-backed store and authenticated CLI/Telegram bearer
39
+ retrieval boundaries, then connect both real provider/worker paths without
40
+ changing existing terminal/error semantics.
41
+ 3. Add deterministic concurrency, expiry, corruption, filesystem-attack, and
42
+ tool-heavy end-to-end coverage; document operation and run every repository
43
+ check before a full diff/security audit and commit.
44
+
45
+ Card 10521 metrics and admission-policy expansion are explicitly out of scope.
@@ -97,8 +97,9 @@ node bin/threadwire.js run \
97
97
  --prompt 'Inspect the container worktree.'
98
98
  ```
99
99
 
100
- Provider activity/state persists at `/var/lib/threadwire/activity` and
101
- `/home/threadwire/.codex`. The Codex adapter reads only
100
+ Provider activity persists at `/var/lib/threadwire/activity`, detailed
101
+ delegated evidence persists privately at `/var/lib/threadwire/evidence`, and
102
+ Codex state persists at `/home/threadwire/.codex`. The Codex adapter reads only
102
103
  `OPENAI_API_KEY_FILE`, exports the provider credential immediately before
103
104
  `exec`, removes its pointer, and removes every ingress value and pointer from
104
105
  the provider environment. Threadwire repeats ingress filtering before spawn.
@@ -0,0 +1,161 @@
1
+ # Delegated-result admission protocol
2
+
3
+ Threadwire has two deliberately separate output planes:
4
+
5
+ - Telegram is the user-visible progress plane. It may receive assistant streaming,
6
+ lifecycle notices, diagnostics, and opt-in tool status.
7
+ - Standard output for a normal `threadwire run` is the parent-model admission
8
+ plane. It contains exactly one newline-terminated terminal JSON envelope.
9
+
10
+ Standard error remains a caller diagnostic channel and is never admission
11
+ content. Provider stdout is private parser input.
12
+
13
+ ## Context-budget metrics
14
+
15
+ Each delegated run emits one concise `threadwire-context-metrics` JSON record
16
+ on standard error. It contains content-free integer counters only. Detailed
17
+ fixed-class counters are appended to the run's existing file-backed evidence
18
+ artifact when evidence is configured; they are never added to the admission
19
+ envelope.
20
+
21
+ Byte counters are UTF-8 byte counts. Event and handle counters are counts.
22
+ `estimatedTokens` is `ceil(admittedBytes / 4)` and is explicitly an estimate,
23
+ not provider tokenization. Every counter saturates at 2,147,483,647. Metric
24
+ classes and envelope-field names are closed, fixed sets, so worker-controlled
25
+ values cannot create labels or cardinality. Counters retain no child text,
26
+ commands, arguments, output, secrets, handles, reference values, or other raw
27
+ values.
28
+
29
+ The raw-child counters separately measure provider stdout/stderr stream chunks
30
+ and bytes at the child stream boundary, and successfully JSON-parsed provider
31
+ records at the parser boundary. A parsed record is not an OS chunk: one chunk
32
+ may contain multiple records and one record may span chunks. Rejected counters
33
+ distinguish provider stream, stderr, and lifecycle, diagnostic, and tool
34
+ progress excluded from admission.
35
+
36
+ Progress has fixed `attempted`, `suppressed`, `coalesced`, and `delivered`
37
+ classes. Attempted sizes are JSON-encoded normalized-event bytes. Suppressed
38
+ counts progress disabled by policy, while coalesced counts rendered assistant
39
+ notices merged under notifier saturation. Delivered alone means a rendered
40
+ message or edit was successfully sent; its size is the exact UTF-8 size of the
41
+ JSON-encoded transport value. Consequently batching, rendering, and escaping
42
+ may make delivered sizes differ from attempted sizes.
43
+
44
+ Artifact counters measure finalized logical payload bytes and admitted
45
+ opaque-handle count. When detailed metrics are the last event in that same
46
+ artifact, `artifacts.bytes` is a fixed-point projection that includes the
47
+ newline-terminated metrics JSON itself. Finalization must equal the projection
48
+ or the run fails closed. This avoids a second metrics artifact while giving the
49
+ stored record exact self-inclusive semantics.
50
+
51
+ `admitted.bytes` and `parentContextDeltaBytes` measure the exact UTF-8 bytes of
52
+ the newline-terminated terminal envelope written to stdout. Admitted field
53
+ sizes are the UTF-8 sizes of each field's JSON-encoded value, including string
54
+ quotes, escapes, and multibyte encoding.
55
+
56
+ The regression invariant is exact: given quiet and tool-heavy runs with the
57
+ same terminal conclusion, validation, references, and artifact handles, every
58
+ admitted semantic field is identical and:
59
+
60
+ ```text
61
+ heavy.parentContextDeltaBytes
62
+ <= quiet.parentContextDeltaBytes + 256
63
+ ```
64
+
65
+ The 256-byte allowance is fixed metadata headroom and is independent of raw
66
+ event volume. The current version adds no per-run metric metadata to parent
67
+ context, so equivalent envelopes normally have equal deltas. Tests deliberately
68
+ inject a provider-transcript-shaped conclusion large enough to exceed the
69
+ allowance; this makes accidental stream admission fail deterministically.
70
+
71
+ ## Version 1 envelope
72
+
73
+ The closed envelope has these required fields:
74
+
75
+ ```json
76
+ {
77
+ "version": 1,
78
+ "type": "delegated_result",
79
+ "state": "completed",
80
+ "exitCode": 0
81
+ }
82
+ ```
83
+
84
+ `state` is exactly one of `completed`, `failed`, `blocked`, or
85
+ `needs_decision`. `exitCode` is a nonnegative safe integer. The CLI currently
86
+ produces `completed` for exit code zero and `failed` otherwise.
87
+
88
+ The only optional fields are:
89
+
90
+ | Field | Shape | Limit |
91
+ | --- | --- | --- |
92
+ | `conclusion` | string | 16,384 Unicode code points |
93
+ | `blocker` | string | 1,024 Unicode code points |
94
+ | `decisionRequest` | string | 1,024 Unicode code points |
95
+ | `continuationHandle` | opaque ASCII provider token | 512 characters |
96
+ | `artifactHandles` | array of opaque ASCII IDs or safe POSIX-style paths | 16 entries; 512 characters each |
97
+ | `references` | array of `{kind,value}` | 16 entries; 2,048 code points per value |
98
+ | `validationSummary` | string | 2,048 Unicode code points |
99
+
100
+ A reference `kind` is exactly `commit` or `url`. Commit values are 7–64
101
+ hexadecimal characters. URL values are non-whitespace HTTP or HTTPS URLs
102
+ without embedded credentials.
103
+ Artifact handles are opaque identifiers only; artifact content is forbidden.
104
+ The complete UTF-8 JSON envelope is limited to 98,304 bytes.
105
+
106
+ Threadwire preserves accepted continuation handles, artifact handles, and
107
+ references exactly. Streamed conclusion collection is credential-redacted,
108
+ control-normalized, and deterministically truncated with a final ellipsis when
109
+ necessary. Code-point accounting never splits a UTF-16 surrogate pair.
110
+
111
+ A continuation handle consists of one or more colon-separated opaque segments.
112
+ Each segment must start with an ASCII letter or digit and may then contain only
113
+ ASCII letters, digits, `_`, `-`, and `.`. This covers provider UUIDs, token IDs
114
+ such as `ses_123`, and namespaced IDs such as `provider:session-123` without
115
+ admitting prose. Empty segments and known URL/transport scheme prefixes are
116
+ forbidden. An artifact handle uses a single segment's alphabet and may
117
+ additionally use `/` separators with one optional leading `/`; empty, `.` and
118
+ `..` path components are forbidden. Both handle types reject whitespace,
119
+ backslashes, controls (including C0, C1, and ESC), URL forms, userinfo, query
120
+ strings, and fragments.
121
+
122
+ ## Fail-closed rules
123
+
124
+ Admission is an explicit allowlist, not a projection of arbitrary provider
125
+ data. A candidate is rejected if it:
126
+
127
+ - has an unknown field or terminal state;
128
+ - uses a wrong or nested type, including an object mislabeled as an allowed
129
+ string or opaque handle;
130
+ - adds fields inside a reference;
131
+ - has an empty, oversized, or over-cardinality optional collection;
132
+ - contains an invalid commit or URL reference; or
133
+ - exceeds a per-field or aggregate limit.
134
+
135
+ Unknown normalized worker event kinds fail the run. Known lifecycle,
136
+ diagnostic, and tool events remain progress-only and cannot affect the
137
+ envelope. Threadwire does not silently remove a forbidden field and admit the
138
+ remainder.
139
+
140
+ ## Excluded material
141
+
142
+ The admission envelope never contains execution-environment progress,
143
+ lifecycle progress, diagnostics, provider records or streams, tool names,
144
+ arguments, details, commands or raw output, stderr, prompts,
145
+ narration/reasoning, full transcripts, or artifact content. Optional structured
146
+ fields are populated only from explicit trusted facts; provider prose is not
147
+ parsed to invent them.
148
+
149
+ ## Caller obligations and compatibility
150
+
151
+ The caller must parse exactly one envelope, require `version: 1` and
152
+ `type: "delegated_result"`, enforce this closed schema, and reject missing,
153
+ duplicate, trailing, unknown-version, unknown-field, or oversized records
154
+ before parent transcript or context persistence. Standard error and Telegram
155
+ delivery must never be merged into parent context.
156
+
157
+ This protocol is the default and is an intentional pre-1.0 breaking change.
158
+ Callers of the former unversioned mixed progress stream must upgrade before
159
+ using this release. Future file-backed evidence may use opaque
160
+ `artifactHandles`; retrieval and artifact storage are outside version 1 and are
161
+ not implied by a handle.
@@ -0,0 +1,106 @@
1
+ # Delegated-worker evidence artifacts
2
+
3
+ Threadwire retains detailed delegated-worker evidence without adding it to the
4
+ parent admission stream. Telegram uses the dedicated
5
+ `/var/lib/threadwire/evidence` volume. A standalone run enables the same store
6
+ with the trusted operator setting `THREADWIRE_EVIDENCE_ROOT`; callers cannot
7
+ select a root through CLI arguments.
8
+
9
+ The store records the prompt and framed provider stdout/stderr events, including
10
+ tool calls, commands, raw results, and intermediate narration the provider
11
+ emits. Threadwire stdout still contains one bounded `delegated_result`; evidence
12
+ adds only an unguessable bearer handle. Storage failure fails the run explicitly
13
+ and never falls back to inlining evidence.
14
+
15
+ ## Security and lifecycle
16
+
17
+ The Linux-only store walks an absolute path from a pinned `/` descriptor,
18
+ opening every existing component with `O_DIRECTORY|O_NOFOLLOW`. It accepts
19
+ root-owned non-writable ancestors, root-owned sticky directories such as
20
+ `/tmp`, and same-user directories that are not group/world-writable. Only a missing final component is
21
+ created, relative to its pinned trusted parent, as an owned `0700` directory.
22
+ The store then performs child operations through `/proc/self/fd`; destructive
23
+ work rechecks device/inode. Payload and metadata
24
+ files are `0600`, regular, single-link files. Reconciliation removes only
25
+ recognized implementation-owned artifact and atomic-temporary names. Any
26
+ unknown entry is preserved and makes store opening fail closed.
27
+
28
+ Only one live deployment can open a shared-volume root. The parent opens the
29
+ lock file through the pinned root descriptor and authenticates `/usr/bin/flock`
30
+ through its own descriptor. A synchronous child executes that inherited
31
+ executable descriptor and applies `flock -n` to an inherited duplicate of the
32
+ lock-file open description. The child emits no output and exits; the parent
33
+ retains the locked open description until close or process death.
34
+
35
+ Each event has a bounded kind and length frame, preserving ordering, binary
36
+ data, and split-record provenance without accumulating a transcript in memory.
37
+ Pending evidence counts against quotas, is unreadable, and is not expiry-cleaned
38
+ while active. Fsync and atomic rename publish payload before ready metadata.
39
+ Startup removes partial/corrupt states and rehydrates only exact-schema,
40
+ unexpired ready artifacts.
41
+
42
+ Defaults are 1 MiB/event; 64 MiB and 100,000 events/artifact; 128 MiB and
43
+ 200,000 events/run; 1 GiB, 1,000,000 events, and 10,000 artifacts/store;
44
+ seven-day retention; and a 64 KiB/1,000-line serialized-response ceiling.
45
+ Durable reservations include metadata publication overlap and directory-entry
46
+ allowance, so zero-event artifacts consume capacity.
47
+
48
+ ## Authorization and retrieval
49
+
50
+ The `evidence_…` handle is a 256-bit bearer capability returned only through the
51
+ owning parent envelope or Telegram topic. Metadata stores its SHA-256 binding,
52
+ not the capability. Artifact redaction policy is AES-GCM sealed under the
53
+ capability with versioned AAD binding every immutable lifecycle/accounting
54
+ field. Ready metadata also contains an HMAC-SHA-256 commitment over the complete
55
+ ordered framed payload, keyed from the bearer without storing the key.
56
+ Ephemeral in-process owner objects are tracked only by weak identity membership;
57
+ completed webhook jobs do not accumulate in an enumerable authorization registry.
58
+
59
+ Telegram retrieval also requires an authenticated webhook update whose
60
+ allowlisted sender and server-derived chat/topic/sender identity match the
61
+ artifact metadata and encrypted-policy AAD. A requester
62
+ cannot submit destination, run, or agent identity. Wrong scope, bad capability,
63
+ missing, corrupt, removed, and expired artifacts all report generic
64
+ unavailability.
65
+
66
+ Parent retrieval syntax:
67
+
68
+ ```console
69
+ threadwire evidence read --handle evidence_<bearer> --bytes 2048:512
70
+ threadwire evidence read --handle evidence_<bearer> --lines 40:5
71
+ threadwire evidence read --handle evidence_<bearer> --query "exact command identity" --context-bytes 80
72
+ ```
73
+
74
+ Authenticated Telegram syntax is
75
+ `/evidence evidence_<bearer> bytes 2048:512` or a bounded `lines` selector in
76
+ the owning chat/topic.
77
+
78
+ Byte offsets are zero-based bytes in the redacted logical stream. Line starts
79
+ are one-based and include terminating newlines. Text boundaries are valid UTF-8;
80
+ binary output is base64. Queries are bounded UTF-8 literal searches—never
81
+ regular expressions. One universal encoded output-byte maximum applies even to
82
+ one huge line. Verification should request the smallest useful excerpt.
83
+
84
+ Retrieval opens the payload once with `O_NOFOLLOW`, verifies the descriptor, and
85
+ in one bounded-memory streaming pass authenticates the exact raw frames while
86
+ parsing, redacting, and retaining only the bounded selection. It drains through
87
+ EOF and releases that selection only after timing-safe commitment verification;
88
+ there is no authenticated-then-reread interval. It does not read whole artifacts
89
+ or build transcript-wide offset arrays. Known runtime secret values, securely read secret-file contents,
90
+ and their pointer values become `[REDACTED]`; responses state whether they were
91
+ redacted or truncated. This finite policy is defense in depth, not
92
+ arbitrary-secret detection.
93
+
94
+ ## Operations
95
+
96
+ Run `cleanup()` from maintenance and use scoped removal for revocation. Cleanup
97
+ is idempotent, skips active descriptor-pinned readers, fsyncs the root after
98
+ unlink, and releases capacity only after that transaction succeeds. Trusted
99
+ structured references already approved through the admission boundary remain
100
+ independent of artifact cleanup. No provider currently supplies authenticated
101
+ structured commit or URL metadata, so production does not infer either from
102
+ stdout or ambient repository state.
103
+
104
+ Treat the volume as sensitive provider-transcript data. Exclude it from general
105
+ logs and broad backups. Any required backup must be encrypted and preserve the
106
+ same authorization and retention. Suppressed raw content is never logged.
@@ -0,0 +1,137 @@
1
+ # Isolated provider runtime
2
+
3
+ Relay-originated Codex writes are disabled unless the caller supplies both
4
+ `--relay-write` and a reviewed `--workspace-profile`. The write path never
5
+ falls back to the native provider launcher.
6
+ When the Telegram service is configured with isolated-runtime authority, Codex
7
+ webhook jobs use the same preflight/run boundary exclusively; an unavailable
8
+ runtime fails the job rather than invoking the native runner. Raw provider
9
+ stdout and stderr are returned as separately identified, bounded chunks for
10
+ the existing evidence callbacks.
11
+
12
+ ## Security boundary
13
+
14
+ Telegram ingress has neither provider secrets nor container-runtime authority.
15
+ The root-owned isolated-runtime supervisor has Docker authority and a read-only
16
+ view of allowlisted worktree roots, but no provider credential. The model
17
+ broker alone mounts the upstream credential. Each grant creates a listener
18
+ bound to the broker's trusted per-run network interface. That listener accepts
19
+ one token, one run/network lineage, one allowlisted model, and a closed
20
+ `/v1/responses` schema; it is not a generic HTTP proxy.
21
+ The listener remains non-authorizing while the supervisor creates and verifies
22
+ the worker. The supervisor then activates it exactly once immediately before
23
+ worker start. Workers receive the inspected per-run broker IP directly, so
24
+ launch does not depend on asynchronous container-DNS alias propagation.
25
+
26
+ For each fresh or explicit-resume run the supervisor creates a unique Docker
27
+ internal network, attaches the broker, and launches an immutable worker. The
28
+ worker has a read-only root, all capabilities dropped, no-new-privileges,
29
+ container-local tmp/run, resource limits, and exactly one writable
30
+ worktree mount. It has no host home, common Git directory, sibling worktree,
31
+ host temp, shared provider state, secret, Docker/SSH socket, or external route. Its
32
+ short-lived broker grant is revoked during cleanup.
33
+ The grant owns one absolute deadline and every accepted socket. Expiry,
34
+ explicit revoke, credential reload, or broker shutdown aborts an incomplete
35
+ upload or upstream request, destroys accepted sockets, and closes the listener;
36
+ authorization is checked again after the bounded request body is acquired.
37
+
38
+ Codex state lives in a unique, random, immutable-label Docker volume for each
39
+ fresh lineage, never a host or common Codex home. An HMAC-authenticated,
40
+ root-private supervisor registry maps returned session IDs to that exact
41
+ volume. Resume revalidates the volume's owner, namespace, task, and lineage
42
+ labels before every mount. A collision, unlabeled or differently labeled
43
+ preexisting volume, sibling,
44
+ missing, expired, or lost state fails before worker creation.
45
+ Only one worker may mount a lineage at a time; the supervisor holds an atomic
46
+ reservation from preflight consumption through worker cleanup. Expired aliases
47
+ are removed by authenticated periodic GC. The registry retains lineage
48
+ ownership until Docker confirms deletion, retries cleanup failures, and
49
+ reconciles exactly labelled orphan volumes on startup while preserving volumes
50
+ referenced by in-process runs. On restart, exactly owned created, running, or
51
+ exited orphan workers and their networks are removed before volume GC retries.
52
+ An fsync'd, HMAC-authenticated run record binds the generated names, pinned
53
+ image digest, entrypoint policy, private network, state volume, and exact
54
+ worktree mount source/options. Reconciliation inspects that full identity and
55
+ network endpoint membership; forgeable labels or any mismatched name, image,
56
+ mount, option, command, or endpoint are never deletion authority.
57
+
58
+ Preflight verifies Docker, the immutable image, broker readiness, the
59
+ allowlisted non-symlink worktree with no nested mounts, stable device/inode ancestry, and mount policy before prompt, Telegram
60
+ token, dotenv, evidence, resume state, executable override, or provider
61
+ credential reads. The ancestry is revalidated immediately before container
62
+ creation and start, and the immutable entrypoint verifies the mounted worktree
63
+ and reviewed nested cwd before Codex. Only `--model` and `--color` are accepted;
64
+ they are transported as a JSON array and passed exactly once without shell
65
+ splitting. All other provider arguments are rejected before launch.
66
+ Authenticated preflight admission has strict endpoint/field size limits,
67
+ global and per-task caps, and an eager timer sweep, so abandoned capabilities
68
+ expire without requiring a matching `/run`.
69
+ Consumed capabilities enter a separate global/per-task active-run admission
70
+ before validation can yield. The lineage reservation is acquired synchronously
71
+ at that boundary, preventing periodic GC from deleting resume state during
72
+ revalidation. Docker and broker operations have bounded deadlines, and each
73
+ worker has a bounded operator-configured lifetime; timeout cleanup revokes the
74
+ grant and removes the exactly labelled worker and network before capacity is
75
+ released.
76
+ The relay creates one absolute launch deadline before preflight. The same
77
+ deadline and cancellation signal span prompt/stdin and file-secret reads,
78
+ evidence and Telegram setup, Docker setup, broker readiness, execution,
79
+ response/log reads, upstream work and exact cleanup. A disconnected caller or
80
+ supervisor shutdown aborts its run. Independent per-step timeouts cannot extend
81
+ the overall budget, and response bodies are stream-limited to 2 MiB.
82
+
83
+ Codex bypasses its nested sandbox only inside this mandatory outer container.
84
+ Its task-private home contains only that lineage's session state; persisted
85
+ config, rules, hooks, plugins, MCP files and instruction files are rejected
86
+ before Codex starts. MCP is forced empty and project instruction loading is disabled.
87
+ Failure to create the outer boundary is fatal.
88
+
89
+ ## Deployment and rotation
90
+
91
+ The default `compose.yaml` contains no provider secret, broker, supervisor, or
92
+ Docker socket. Enable writes only with both files and the explicit profile:
93
+
94
+ ```sh
95
+ docker compose -f compose.yaml -f compose.relay-write.yaml --profile relay-write up
96
+ ```
97
+
98
+ The override fails interpolation unless every required image, control token,
99
+ worktree root, allowlist and credential-file setting is present. Set immutable supervisor, broker, and worker images; a host
100
+ `THREADWIRE_WORKSPACE_ROOT`; JSON `THREADWIRE_ALLOWED_WORKTREE_ROOTS`; separate
101
+ runtime and broker control tokens; `THREADWIRE_ALLOWED_CODEX_MODELS` as a JSON
102
+ server-side model allowlist; a unique `THREADWIRE_STATE_NAMESPACE`; a random
103
+ `THREADWIRE_STATE_AUTH_KEY` of at least 32 characters; and a deployment secret file for the Codex
104
+ credential. The supervisor alone receives `/var/run/docker.sock`. Restrict
105
+ broker egress at the host/firewall to provider endpoints.
106
+
107
+ The supervisor defaults to 16 active runs globally, two per reviewed task, a
108
+ 30-second maximum preflight slice, and a one-hour end-to-end launch deadline. Operators may
109
+ lower these finite limits with `THREADWIRE_PREFLIGHT_TIMEOUT_MS`,
110
+ `THREADWIRE_ACTIVE_RUN_CAPACITY`, `THREADWIRE_ACTIVE_TASK_CAPACITY`, and
111
+ `THREADWIRE_WORKER_TIMEOUT_MS`; invalid, zero, or excessive values fail startup.
112
+
113
+ The authenticated registry persists through file fsync, atomic rename, and
114
+ parent-directory fsync. Startup removes only strictly named abandoned temp
115
+ files whose complete contents authenticate under the deployment key; malformed
116
+ or foreign files are left untouched for operator inspection.
117
+
118
+ `THREADWIRE_WORKTREE_VOLUME` is only for nested-Docker/test deployments. Docker
119
+ volume `Subpath` still mounts only the selected task directory. Production uses
120
+ the exact host bind through Docker's structured Mount API with private
121
+ propagation. The supervisor verifies the resulting two-mount inventory before
122
+ start, and the entrypoint independently rejects nested worktree mounts.
123
+
124
+ Replace the broker secret atomically and restart it, or call authenticated
125
+ `/admin/reload`. Reload increments the generation and revokes existing grants.
126
+ Missing, writable, symlinked, empty, or oversized secret files keep it unready.
127
+
128
+ If Docker, broker, pinned image, worktree, security option, or private network
129
+ is unavailable, relay write fails closed. There is no native fallback.
130
+
131
+ ## Verification
132
+
133
+ Run `npm run all-checks`, `npm run verify:package`, and
134
+ `npm run test:isolated-runtime-e2e` with broker, runtime, hostile-probe,
135
+ production relay-worker, and fake-upstream image variables documented
136
+ by that script. E2E uses host-volume canaries and fake-upstream authorization
137
+ logs; worker self-report is not the sole assertion.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "threadwire",
3
- "version": "0.1.5",
3
+ "version": "0.1.8",
4
4
  "description": "Stream Codex, Claude, and OpenCode worker progress to an explicit Telegram destination",
5
5
  "keywords": [
6
6
  "ai-agent",
@@ -14,6 +14,7 @@
14
14
  "private": false,
15
15
  "type": "module",
16
16
  "files": [
17
+ "CHANGELOG.md",
17
18
  "bin/",
18
19
  "docs/",
19
20
  "scripts/",
@@ -22,6 +23,8 @@
22
23
  "threadwire.workspace-profiles.json"
23
24
  ],
24
25
  "bin": {
26
+ "threadwire-isolated-runtime": "bin/isolated-runtime.js",
27
+ "threadwire-model-broker": "bin/model-broker.js",
25
28
  "threadwire": "bin/threadwire.js",
26
29
  "threadwire-telegram-webhook": "bin/telegram-webhook.js"
27
30
  },
@@ -49,6 +52,7 @@
49
52
  "prepublishOnly": "npm run all-checks && npm run verify:package",
50
53
  "test": "node --test \"test/**/*.test.js\"",
51
54
  "test:container-e2e": "sh docker/test/production-e2e.sh",
55
+ "test:isolated-runtime-e2e": "sh docker/test/isolated-runtime-e2e.sh",
52
56
  "typecheck": "tsc --noEmit",
53
57
  "verify:package": "node scripts/verify-package.js"
54
58
  },