instar 1.3.646 → 1.3.648

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (87) hide show
  1. package/dist/commands/reflect.d.ts.map +1 -1
  2. package/dist/commands/reflect.js +7 -3
  3. package/dist/commands/reflect.js.map +1 -1
  4. package/dist/commands/server.d.ts.map +1 -1
  5. package/dist/commands/server.js +41 -2
  6. package/dist/commands/server.js.map +1 -1
  7. package/dist/commands/setup.d.ts.map +1 -1
  8. package/dist/commands/setup.js +17 -0
  9. package/dist/commands/setup.js.map +1 -1
  10. package/dist/config/ConfigDefaults.d.ts.map +1 -1
  11. package/dist/config/ConfigDefaults.js +14 -0
  12. package/dist/config/ConfigDefaults.js.map +1 -1
  13. package/dist/core/CoherenceGate.d.ts.map +1 -1
  14. package/dist/core/CoherenceGate.js +24 -0
  15. package/dist/core/CoherenceGate.js.map +1 -1
  16. package/dist/core/CoherenceReviewer.d.ts +9 -0
  17. package/dist/core/CoherenceReviewer.d.ts.map +1 -1
  18. package/dist/core/CoherenceReviewer.js +18 -2
  19. package/dist/core/CoherenceReviewer.js.map +1 -1
  20. package/dist/core/InputGuard.d.ts.map +1 -1
  21. package/dist/core/InputGuard.js +17 -3
  22. package/dist/core/InputGuard.js.map +1 -1
  23. package/dist/core/MessageSentinel.d.ts.map +1 -1
  24. package/dist/core/MessageSentinel.js +19 -2
  25. package/dist/core/MessageSentinel.js.map +1 -1
  26. package/dist/core/MessagingToneGate.d.ts +8 -0
  27. package/dist/core/MessagingToneGate.d.ts.map +1 -1
  28. package/dist/core/MessagingToneGate.js +16 -1
  29. package/dist/core/MessagingToneGate.js.map +1 -1
  30. package/dist/core/PostUpdateMigrator.d.ts.map +1 -1
  31. package/dist/core/PostUpdateMigrator.js +10 -0
  32. package/dist/core/PostUpdateMigrator.js.map +1 -1
  33. package/dist/core/SingleInstanceLock.d.ts +103 -0
  34. package/dist/core/SingleInstanceLock.d.ts.map +1 -0
  35. package/dist/core/SingleInstanceLock.js +300 -0
  36. package/dist/core/SingleInstanceLock.js.map +1 -0
  37. package/dist/core/SpawnCapIntelligenceProvider.d.ts +89 -0
  38. package/dist/core/SpawnCapIntelligenceProvider.d.ts.map +1 -0
  39. package/dist/core/SpawnCapIntelligenceProvider.js +164 -0
  40. package/dist/core/SpawnCapIntelligenceProvider.js.map +1 -0
  41. package/dist/core/hostSpawnSemaphore.d.ts +188 -0
  42. package/dist/core/hostSpawnSemaphore.d.ts.map +1 -0
  43. package/dist/core/hostSpawnSemaphore.js +471 -0
  44. package/dist/core/hostSpawnSemaphore.js.map +1 -0
  45. package/dist/core/intelligenceProviderFactory.d.ts +0 -12
  46. package/dist/core/intelligenceProviderFactory.d.ts.map +1 -1
  47. package/dist/core/intelligenceProviderFactory.js +27 -9
  48. package/dist/core/intelligenceProviderFactory.js.map +1 -1
  49. package/dist/core/types.d.ts +18 -0
  50. package/dist/core/types.d.ts.map +1 -1
  51. package/dist/core/types.js.map +1 -1
  52. package/dist/feedback-factory/store/JsonlFeedbackStore.d.ts +2 -0
  53. package/dist/feedback-factory/store/JsonlFeedbackStore.d.ts.map +1 -1
  54. package/dist/feedback-factory/store/JsonlFeedbackStore.js +36 -2
  55. package/dist/feedback-factory/store/JsonlFeedbackStore.js.map +1 -1
  56. package/dist/messaging/TelegramAdapter.d.ts.map +1 -1
  57. package/dist/messaging/TelegramAdapter.js +6 -2
  58. package/dist/messaging/TelegramAdapter.js.map +1 -1
  59. package/dist/monitoring/CoherenceMonitor.d.ts.map +1 -1
  60. package/dist/monitoring/CoherenceMonitor.js +6 -1
  61. package/dist/monitoring/CoherenceMonitor.js.map +1 -1
  62. package/dist/monitoring/CommitmentSentinel.d.ts.map +1 -1
  63. package/dist/monitoring/CommitmentSentinel.js +7 -2
  64. package/dist/monitoring/CommitmentSentinel.js.map +1 -1
  65. package/dist/scaffold/templates.d.ts.map +1 -1
  66. package/dist/scaffold/templates.js +5 -0
  67. package/dist/scaffold/templates.js.map +1 -1
  68. package/dist/server/CapabilityIndex.d.ts.map +1 -1
  69. package/dist/server/CapabilityIndex.js +9 -0
  70. package/dist/server/CapabilityIndex.js.map +1 -1
  71. package/dist/server/routes.d.ts.map +1 -1
  72. package/dist/server/routes.js +28 -0
  73. package/dist/server/routes.js.map +1 -1
  74. package/dist/utils/jsonl-tail.d.ts +50 -0
  75. package/dist/utils/jsonl-tail.d.ts.map +1 -0
  76. package/dist/utils/jsonl-tail.js +96 -0
  77. package/dist/utils/jsonl-tail.js.map +1 -0
  78. package/package.json +4 -2
  79. package/scripts/lint-no-direct-destructive.js +4 -0
  80. package/scripts/lint-no-unbounded-llm-spawn.js +137 -0
  81. package/src/data/builtin-manifest.json +63 -63
  82. package/src/scaffold/templates.ts +5 -0
  83. package/upgrades/1.3.647.md +56 -0
  84. package/upgrades/1.3.648.md +99 -0
  85. package/upgrades/eli16/eventloop-bounded-jsonl-reads.md +47 -0
  86. package/upgrades/side-effects/eventloop-bounded-jsonl-reads.md +96 -0
  87. package/upgrades/side-effects/forkbomb-prevention-simple.md +138 -0
@@ -0,0 +1,99 @@
1
+ # Upgrade Guide — vNEXT
2
+
3
+ <!-- assembled-by: assemble-next-md -->
4
+ <!-- bump: minor -->
5
+
6
+ ## What Changed
7
+
8
+ Fork-bomb prevention — the SIMPLE design (`docs/specs/forkbomb-prevention-simple.md`).
9
+ On 2026-06-20 the Instar server fork-bombed its macOS host into OOM **twice**
10
+ (~230-289 concurrent `claude -p` ≈ 90-115GB): `evaluate()` spawned one LLM
11
+ subprocess per call with **zero** concurrency control, `CoherenceGate` fanned ~10
12
+ reviewers in parallel per message, and up to three server instances each
13
+ re-flooded. This release adds the three structural primitives that make that
14
+ physically impossible:
15
+
16
+ - **P1 — Host-wide concurrent-spawn cap (the PRIMARY control).** A host-local
17
+ counting semaphore (`src/core/hostSpawnSemaphore.ts`) bounds how many LLM
18
+ subprocesses run **at once** across every compliant Instar process on the host
19
+ (default **8**). It uses a holder-SET model (count live holder records under an
20
+ exclusive flock, not decrement/increment counter math) so it is crash-safe by
21
+ construction — a double-release is a no-op, a pid-reuse can't steal a slot, a
22
+ partial write is discarded, a crashed holder is reclaimed (pid-dead AND
23
+ heartbeat-stale, **this host only**; a foreign-hostname holder is NEVER
24
+ reclaimed; a `df -P` host-local-disk check gates reclaim, fail-closed). The cap
25
+ is enforced by a per-`evaluate()` wrapper provider
26
+ (`src/core/SpawnCapIntelligenceProvider.ts`) installed at **every** return arm
27
+ of `buildIntelligenceProvider` (the factory funnel) — so CoherenceGate's
28
+ shared-instance fan-out is bound per call. A lint
29
+ (`scripts/lint-no-unbounded-llm-spawn.js`) forward-guards any new provider
30
+ constructed outside the funnel.
31
+ - **P2 — Single-instance lock (`src/core/SingleInstanceLock.ts`).** Refuses a
32
+ duplicate server instance of the same agent on a host (the 3× launchd/fleet/tmux
33
+ multiplier), with a bounded deploy-handoff grace so a normal restart hands off
34
+ cleanly. Foreign-host locks are never reclaimed; `INSTAR_ALLOW_SECOND_INSTANCE=1`
35
+ overrides for a deliberate second instance.
36
+ - **P3 — Bounded ingress.** A saturated cap makes a spawn request poll-retry up to
37
+ `acquireMs` (default 5000ms, ~100ms interval, no in-memory waiter queue so no
38
+ heap growth), bounded by a concurrent-poller ceiling (`waitersMax`, default 64).
39
+ On genuine exhaustion the call throws a typed `LlmCapacityUnavailableError`. The
40
+ **four safety-gating seams** fail CLOSED on that shed instead of auto-passing:
41
+ `MessageSentinel.classify` (held, not `normal`), `InputGuard.reviewTopicCoherence`
42
+ (flagged `suspicious`, not `coherent`), `MessagingToneGate.review` (held
43
+ `pass:false`), and the outbound `CoherenceGate` reviewer path (block-the-turn
44
+ `pass:false`). The deterministic emergency-stop keyword pre-check runs **before**
45
+ the LLM classifier and is **exempt** from the cap — a "stop everything" halt
46
+ always halts.
47
+
48
+ The cap is a **SAFETY FLOOR**: it ships **ON by default** for every agent (read
49
+ with a plain `?? default`, never a dev gate), tunable via `intelligence.spawnCap`
50
+ (`maxConcurrent`/`acquireMs`/`waitersMax`) or env (`INSTAR_HOST_SPAWN_MAX`,
51
+ `INSTAR_SPAWN_ACQUIRE_MS`, `INSTAR_SPAWN_WAITERS_MAX`). A new `GET /spawn-limiter`
52
+ route reports live holder count, cap, available slots, saturation, and waiters. A
53
+ conservative OS-level `NumberOfProcesses` ceiling (512) is added to the launchd
54
+ plist as a host-global belt under the semaphore. A new constitutional standard —
55
+ **"Bounded Blast Radius"** — lands in `docs/STANDARDS-REGISTRY.md` with its lint +
56
+ burst-invariant ratchet in the same PR.
57
+
58
+ ## What to Tell Your User
59
+
60
+ - **Fork-bomb protection is on**: "I now have a hard ceiling on how much work I can
61
+ run at the same time, so I can never flood your machine into running out of
62
+ memory again — there's a safe limit (8 by default) that's always on, plus a guard
63
+ that stops a duplicate of me from starting up. You can ask me 'are we protected
64
+ against a fork-bomb?' or 'how many tasks am I running right now?' any time."
65
+
66
+ ## Summary of New Capabilities
67
+
68
+ | Capability | How to Use |
69
+ |-----------|-----------|
70
+ | Host-wide concurrent-LLM-spawn cap (SEV-1 OOM fix) | Automatic — on by default (limit 8); tune with `intelligence.spawnCap` or env |
71
+ | Spawn-cap status | `GET /spawn-limiter` |
72
+ | Single-instance lock (no duplicate server) | Automatic at server boot; override with `INSTAR_ALLOW_SECOND_INSTANCE=1` |
73
+ | Fail-closed gating under capacity pressure | Automatic — the four inbound/outbound safety gates hold instead of auto-passing |
74
+
75
+ ## Evidence
76
+
77
+ This release fixes a SEV-1 (full host OOM ×2 on 2026-06-20). Evidence:
78
+
79
+ - **Reproduction → fix mechanism.** The incident driver — one `claude -p` per
80
+ `evaluate()` with no concurrency control, fanned ~10-wide by CoherenceGate — is
81
+ now bound at the factory funnel: every provider `buildIntelligenceProvider`
82
+ returns is wrapped per-call by `SpawnCapIntelligenceProvider`, which acquires a
83
+ host-wide slot before the inner spawn and releases it in `finally`.
84
+ - **Burst-invariant ratchet** (`tests/unit/host-spawn-semaphore-burst-invariant.test.ts`):
85
+ 10,000 churned acquire/release attempts NEVER let live holders exceed the cap;
86
+ a TRUE multi-process test forks 12 OS processes racing for 3 slots and observes
87
+ ≤3 successes (the cross-process flock holds the cap). This is the standing
88
+ Bounded-Accumulation proof that the OOM vector cannot recur.
89
+ - **Four fail-closed seams** (`tests/unit/spawn-cap-fail-closed-gates.test.ts`): a
90
+ capacity shed at each gating seam is asserted to HOLD (not produce
91
+ `normal`/`coherent`/`pass:true`), while a generic LLM error still fails open —
92
+ proving capacity is the only new hold. Emergency-stop is asserted exempt
93
+ (classified by the deterministic fast-path before any LLM call).
94
+ - **E2E lifecycle** (`tests/e2e/spawn-cap-forkbomb-lifecycle.test.ts`):
95
+ `GET /spawn-limiter` is alive (200, cap 8, on by default) on a production-init
96
+ server, and a second single-instance-lock acquisition while a live holder exists
97
+ is refused (the duplicate-flood guard), while a clean release hands off.
98
+ - All three test tiers green; `tsc --noEmit` clean; the new lint + the
99
+ `no-silent-llm-fallback` + `no-silent-fallbacks` ratchets green.
@@ -0,0 +1,47 @@
1
+ # Bounded tail-reads for the 12MB telegram-log + 14MB feedback-store — ELI16
2
+
3
+ ## What this is
4
+
5
+ Your agent's server runs everything on one main thread. If any single operation makes that thread
6
+ wait, *everything* waits — the dashboard, your messages, every background check. When that wait is
7
+ long enough, the dashboard's live connection drops (the "Disconnected" flapping you may have seen).
8
+
9
+ This fixes the THIRD and FOURTH freezes behind that flapping. (The first was a slow shared terminal
10
+ manager; the second was reading the macOS keychain the blocking way; the third was a 13MB job-history
11
+ file re-read whole every minute — all already fixed.) This one is about two more big files.
12
+
13
+ ## What was wrong
14
+
15
+ Your agent keeps a running log of every Telegram message it has ever sent or received, in a file
16
+ called `telegram-messages.jsonl`. That file has grown to about **12 megabytes**. Several background
17
+ checks only ever need to glance at the LAST few messages in that log — the last 50, or the last 100.
18
+ But the way they were written, they read the **entire 12MB file** from disk and chopped it up, just
19
+ to look at the tail. And some of these checks run on a **5-minute timer**, every few minutes, forever.
20
+ Each one froze the server for up to **20 seconds**. (A second large file, a 14MB feedback log, had
21
+ the same problem: it was re-read and re-parsed from scratch every processing pass even when nothing
22
+ had changed.)
23
+
24
+ A frozen main thread uses almost no CPU, so the freeze even looked to the agent like the laptop had
25
+ briefly gone to sleep — a misleading signal on top of a real problem.
26
+
27
+ ## What's new
28
+
29
+ Two simple changes, both behavior-preserving:
30
+
31
+ - The background checks that only want the tail of the message log now read just the **last chunk**
32
+ of the file (a 512-kilobyte window — far more than enough to hold the last few hundred messages),
33
+ not the whole thing. So they're instant no matter how big the file gets.
34
+ - The 14MB feedback log now remembers the file's size and last-modified time. If nothing changed
35
+ since it last read it, it returns the copy it already has, reading nothing from disk.
36
+
37
+ So those per-call whole-file reads are gone from every one of these hot paths, and the freezes they
38
+ caused are eliminated. Everything these checks actually do — what they look for, how they parse,
39
+ how they recover from a corrupt file — is unchanged, and proven by tests. One of the new tests even
40
+ plants a problem message at the very END of a multi-megabyte log to prove the tail-read still catches
41
+ recent entries.
42
+
43
+ ## What you need to decide
44
+
45
+ Nothing. It's self-contained and low-risk, fully covered by tests. One honest note: the log files
46
+ themselves still grow large over time — putting a size cap on them is a separate cleanup I've flagged
47
+ to track, not fixed here. Reading them is no longer a freeze; capping their growth comes next.
@@ -0,0 +1,96 @@
1
+ # Side-Effects Review — bounded tail-reads for the 12MB telegram-log + 14MB feedback-store (event-loop freeze fix)
2
+
3
+ **Slug:** `eventloop-bounded-jsonl-reads` · **Tier:** 1 (focused low-risk bounded-read class fix, no
4
+ spec; live-profiler root-cause). Parent principles: **Structure beats Willpower** — never block the
5
+ event loop (the THIRD + FOURTH distinct event-loop-blocker class behind the dashboard "disconnected"
6
+ flapping, after sync tmux + sync keychain + the JobRunHistory 13MB re-read); and **Bounded
7
+ Accumulation** (the size of an append-only log must never determine the cost of reading it).
8
+
9
+ ## Summary of the change
10
+
11
+ FOUR callers each did `fs.readFileSync(messageLogPath, 'utf-8')` on the ~12MB
12
+ `.instar/telegram-messages.jsonl` just to inspect the last 50–200 lines:
13
+ - `CommitmentSentinel.parseRecentMessages` (`src/monitoring/CommitmentSentinel.ts`) — 5-minute timer,
14
+ only ever looks at the last `maxMessagesPerScan*10` lines.
15
+ - `CoherenceMonitor` output-sanity check (`src/monitoring/CoherenceMonitor.ts`) — 5-minute timer,
16
+ only ever looks at the last 50 agent messages.
17
+ - `checkLogForAgentResponse` (PresenceProxy ack path, `src/commands/server.ts`) — event-driven on
18
+ every PresenceProxy tier verdict, only ever looks at the last 50 lines.
19
+ - `TelegramAdapter.getMessageLog` (`src/messaging/TelegramAdapter.ts`) — analysis route, only ever
20
+ returns the last `limit` (default 100) entries.
21
+
22
+ A 12MB synchronous read + `.split('\n')` on a 5-minute timer froze the event loop for up to ~20s per
23
+ pass — the same ~0-CPU false-sleep signature the JobRunHistory fix removed. PLUS `JsonlFeedbackStore`
24
+ (`src/feedback-factory/store/JsonlFeedbackStore.ts`) re-read + re-`JSON.parse`d the entire ~14MB
25
+ `feedback.jsonl` at the start of every processing pass even when nothing had changed.
26
+
27
+ The fix is two behavior-preserving mechanisms:
28
+
29
+ **(A) A shared bounded tail-read utility** — `src/utils/jsonl-tail.ts` (`readJsonlTailLines` /
30
+ `readJsonlTailLastLines`). It `statSync`s the file (O(1)), `openSync` + `readSync`s only the trailing
31
+ `maxBytes` (default 512KB ≈ ~2,600 recent lines, far more than any caller needs) into a fixed buffer,
32
+ drops the first partial line when the window started mid-file, and returns the trailing lines in file
33
+ order. It NEVER loads the whole file and NEVER throws (a read failure returns an empty result,
34
+ matching the `@silent-fallback-ok` behavior of every former full-file caller). It mirrors
35
+ `CoherenceJournal.readTailTolerant`, generalized off that journal's typed-entry shape. The four
36
+ telegram-log readers above now call it instead of `readFileSync`.
37
+
38
+ **(B) A `(size, mtimeMs)` load cache in `JsonlFeedbackStore.loadJsonl`** — when the on-disk file is
39
+ byte-for-byte the one last folded, it serves a clone of the prior parse with ZERO IO/parse; any
40
+ change (append, shrink, delete) re-reads from disk. Mirrors the JobRunHistory cache pattern. The
41
+ clone-on-serve guarantees a caller mutating the returned Map can never poison the cache.
42
+
43
+ ## 1. Correctness / behavioral equivalence
44
+
45
+ The telegram-log readers each consumed only a fixed tail (last 50 / last 200 / last `limit`) of the
46
+ log, so the 512KB window (≈ 2,600 lines) is strictly a superset of every caller's need — the parsed +
47
+ filtered result is identical to the old full-split-then-slice. The partial-first-line drop guarantees
48
+ a truncated record at the window boundary is never mis-parsed. The feedback store's load semantics
49
+ (per-line parse, dedup-last-wins-by-id, torn-line skip, corrupt-file recovery) are preserved exactly;
50
+ the cache is only consulted when the fingerprint matches and is invalidated on any change. Verified:
51
+ 48/48 across the four touched test files (`jsonl-tail` 9, `CoherenceMonitor-bounded-read` 2,
52
+ `CommitmentSentinel` 26, `jsonl-feedback-store` 11) plus the cross-impact suites; tsc exit 0; lints
53
+ green. The two new regression tests spy on `fs.readFileSync` to FAIL if the whole telegram log is
54
+ read, AND prove a bad pattern in a RECENT agent message at the END of a multi-MB log is still
55
+ detected via the tail.
56
+
57
+ ## 2. Cache-coherence / multi-writer safety
58
+
59
+ The telegram-log tail-read holds no state — every call re-reads the live tail, so a concurrent
60
+ appender is always reflected. The feedback-store `(size, mtimeMs)` key invalidates on any byte/mtime
61
+ change (append, compaction rewrite, delete-then-recreate), so an unchanged file is the only cache-hit
62
+ case and a stale fold can never be served. The cache is per-process in-memory (not shared) — no
63
+ cross-process coherence concern (each process reads its own view, same as before). `loadCache` is
64
+ keyed per absolute path and dropped on `existsSync` failure.
65
+
66
+ ## 3. Fail-safe
67
+
68
+ `jsonl-tail.ts` never throws: missing file, stat failure, open/read failure each return an empty
69
+ result — exactly the prior `@silent-fallback-ok` degradation of the full-file callers (an
70
+ observability/housekeeping read must never endanger the observed operation). The feedback store's
71
+ stat-failure path falls through to a full read (`/* fall through to a full read — never let stat
72
+ failure skip the load */`), so a stat error degrades to the correct (slow) result, never wrong data.
73
+
74
+ ## 4. Blast radius
75
+
76
+ One new utility module + its test; four read-site one-liner swaps (CommitmentSentinel, CoherenceMonitor,
77
+ server.ts PresenceProxy ack, TelegramAdapter.getMessageLog); one read-path cache in JsonlFeedbackStore
78
+ + a test-only cache-clear export. No API/route change, no schema change, no write-path semantics
79
+ change, no new external surface.
80
+
81
+ ## 5. Residual (tracked, NOT fixed here — out of scope)
82
+
83
+ The existing `lint-no-wholefile-sync-read` ratchet only catches LITERAL path basenames in the read
84
+ call, but these four sites read VARIABLE paths (`this.messageLogPath`, a `logPath` param), so the
85
+ lint could never have flagged them — which is why they survived the earlier sweeps. The durable
86
+ structural prevention is an **accessor funnel** (route all append-log reads through a single bounded
87
+ accessor that the lint can enforce), which is **Bounded Accumulation Increment 2** — a tracked
88
+ follow-up named here so it is not silently dropped. Separately, the telegram-messages.jsonl and
89
+ feedback.jsonl files themselves still grow UNBOUNDED — a size/retention cap is the same Bounded
90
+ Accumulation track. This change makes READS cheap regardless of size; it does not cap the files.
91
+
92
+ ## 6. Rollback
93
+
94
+ Revert the source files. The change is purely read-path (a bounded tail-read + a fingerprint cache);
95
+ reverting restores the (slow but correct) full-read behavior. No data migration, no on-disk format
96
+ change to undo.
@@ -0,0 +1,138 @@
1
+ # Side-Effects Review — fork-bomb prevention (SIMPLE): host-wide spawn cap + single-instance lock + bounded ingress
2
+
3
+ **Slug:** `forkbomb-prevention-simple` · **Tier:** 2 (converged + approved spec —
4
+ `docs/specs/forkbomb-prevention-simple.md`, cross-model + lessons-aware convergence at
5
+ `docs/specs/reports/forkbomb-prevention-simple-convergence.md`). Parent principles:
6
+ **Structure beats Willpower** (a hard ceiling enforced in code, not a "don't spawn too much"
7
+ note) and **Bounded Blast Radius** (the new constitutional standard this PR lands — safety
8
+ must be physical/capacity-bounded, not only semantic). This is the SEV-1 fix for the
9
+ 2026-06-20 host-OOM incident: the Instar server fork-bombed its macOS host into OOM **twice**
10
+ (~230–289 concurrent `claude -p`, ≈ 90–115 GB) because `IntelligenceProvider.evaluate()`
11
+ spawned one LLM subprocess per call with **zero** concurrency control, `CoherenceGate` fanned
12
+ ~10 reviewers in parallel per message, and up to three server instances each re-flooded.
13
+
14
+ This is the deliberately SIMPLE redesign — three primitives, ON by default. It SUPERSEDES the
15
+ earlier elaborate predecessor (`docs/specs/forkbomb-prevention.md`): the lanes / reserve-math /
16
+ hysteresis / per-gate-typed-shed of that design are explicitly OUT of scope. The cap caps the
17
+ unbounded LLM-subprocess spawns that directly produced the real crash-loop.
18
+
19
+ ## Summary of the change — the 3 primitives + the 4 fail-closed seams
20
+
21
+ **P1 — Host-wide concurrent-spawn cap (the PRIMARY control).** A host-local counting semaphore
22
+ (`src/core/hostSpawnSemaphore.ts`) bounds how many LLM subprocesses run **at once** across every
23
+ compliant Instar process on the host (default **8**). Holder-SET model: it counts live holder
24
+ records under an exclusive `O_CREAT|O_EXCL` flock (reusing the `ProjectRoundLock` primitive), not
25
+ decrement/increment counter math, so it is crash-safe by construction — a double-release is a
26
+ no-op, pid-reuse can't steal a slot, a partial write is discarded. A crashed holder is reclaimed
27
+ only when pid-dead AND heartbeat-stale **on this host**; a foreign-hostname holder is NEVER
28
+ reclaimed (refuse-loud), and a `df -P` host-local-disk check gates reclaim (fail-closed). The cap
29
+ is enforced by a per-`evaluate()` **wrapper provider** (`src/core/SpawnCapIntelligenceProvider.ts`)
30
+ installed at **every** return arm of `buildIntelligenceProvider`, INSIDE the circuit-breaker wrap —
31
+ so `CoherenceGate`'s shared-instance fan-out (one `intelligence` field fanned ~10-wide via
32
+ `Promise.allSettled`) is bound per call. `reflect.ts`'s former raw-fallback construction is routed
33
+ through the same factory funnel.
34
+
35
+ **P2 — Single-instance lock (`src/core/SingleInstanceLock.ts`).** Refuses a duplicate server
36
+ instance of the same agent on a host (the 3× launchd/fleet/tmux multiplier that made the flood
37
+ worse), via the same O_CREAT|O_EXCL pid+hostname+heartbeat holder record. Foreign-host locks are
38
+ never reclaimed; release runs via `finally` + SIGTERM/SIGINT/exit handlers; a bounded deploy-handoff
39
+ grace lets a normal restart hand off cleanly; `INSTAR_ALLOW_SECOND_INSTANCE=1` overrides for a
40
+ deliberate second instance. A genuine duplicate refuses-loud rather than starting a second flooder.
41
+
42
+ **P3 — Bounded ingress + 4 fail-CLOSED gate seams.** A saturated cap makes a spawn request
43
+ poll-retry every ~100 ms up to `acquireMs` (default 5000 ms) — poll-retry, NOT an in-memory waiter
44
+ queue, so no per-waiter heap growth — bounded by a concurrent-poller ceiling (`waitersMax`, default
45
+ 64). On genuine exhaustion the call throws a typed `LlmCapacityUnavailableError`
46
+ (`isCapacityUnavailable`). The **four safety-gating seams** fail CLOSED on that shed instead of
47
+ auto-passing:
48
+ 1. `MessageSentinel.classify` → held, NOT `category:'normal'`.
49
+ 2. `InputGuard.reviewTopicCoherence` → flagged `suspicious`, NOT `verdict:'coherent'`.
50
+ 3. `MessagingToneGate.review` → records unavailable, holds `pass:false`.
51
+ 4. the OUTBOUND `CoherenceGate` reviewer path → block-the-turn `pass:false`, NOT `pass:true`.
52
+
53
+ A non-gating (background) call sheds to its existing heuristic path — loud + counted, **never
54
+ silent**. The deterministic emergency-stop keyword pre-check runs BEFORE the LLM classifier and is
55
+ EXEMPT from the cap — a "stop everything" halt is never gated on capacity.
56
+
57
+ **Safety-floor posture + observability.** The cap is read with a plain `?? default` (NOT a dev gate)
58
+ — it ships **ON by default for every agent**, tunable via `intelligence.spawnCap`
59
+ (`maxConcurrent`/`acquireMs`/`waitersMax`) or env (`INSTAR_HOST_SPAWN_MAX`, `INSTAR_SPAWN_ACQUIRE_MS`,
60
+ `INSTAR_SPAWN_WAITERS_MAX`), migrated existence-checked. A new `GET /spawn-limiter` route reports
61
+ live holder count, cap, available slots, saturation, and waiters. A conservative OS-level
62
+ `NumberOfProcesses` ceiling (512) is added to the launchd plist as a host-global belt under the
63
+ semaphore. The **"Bounded Blast Radius"** constitutional standard lands in
64
+ `docs/STANDARDS-REGISTRY.md` with its lint + burst-invariant ratchet in the same PR.
65
+
66
+ ## 1. Correctness / behavioral equivalence
67
+
68
+ The wrapper is a pure pass-through except for the acquire/release bracket: `evaluate()` acquires a
69
+ host slot, calls `inner.evaluate()`, releases in `finally`. When a slot is available (the normal,
70
+ unsaturated case — ≤8 concurrent) behavior is byte-identical to before; the cap only changes
71
+ behavior under genuine saturation, where the four gating seams hold (the safe direction) and
72
+ background calls degrade to their existing heuristics. The breaker-INSIDE ordering (breaker →
73
+ spawn-cap → provider) ensures a breaker-open shed never holds a slot. Emergency-stop is unchanged:
74
+ it is classified by the deterministic fast-path before any LLM call. Verified: the four fail-closed
75
+ seams are asserted (a capacity shed does NOT produce `normal`/`coherent`/`pass:true`) while a generic
76
+ LLM error still fails open — proving capacity is the ONLY new hold.
77
+
78
+ ## 2. Concurrency / multi-process safety
79
+
80
+ The semaphore is the load-bearing concurrency primitive. The burst-invariant ratchet
81
+ (`tests/unit/host-spawn-semaphore-burst-invariant.test.ts`) churns 10,000 acquire/release attempts
82
+ and asserts live holders NEVER exceed the cap (the standing Bounded-Accumulation proof that the OOM
83
+ vector cannot recur), and a TRUE multi-process test forks 12 OS processes racing for 3 slots and
84
+ observes ≤3 successes — the cross-process flock holds the cap. Every mutation is under the exclusive
85
+ flock; the holder-SET model makes double-release / pid-reuse / partial-write all safe by
86
+ construction. The single-instance lock uses the same flock primitive with the same foreign-host /
87
+ host-local-disk guards.
88
+
89
+ ## 3. Fail-safe / fail-closed
90
+
91
+ Two deliberately opposite directions, both safe:
92
+ - **Gating calls fail CLOSED** under capacity shed (hold the inbound/outbound turn) — a security/
93
+ coherence gate must never auto-pass when it could not actually evaluate.
94
+ - **Background (non-gating) calls fail OPEN to their heuristic** — loud + counted, never silent
95
+ (each is tagged `@silent-fallback-ok` only where genuinely fail-safe; the `no-silent-llm-fallback`
96
+ + `no-silent-fallbacks` ratchets stay green).
97
+ - Reclaim is fail-closed: a foreign-hostname holder is never reclaimed; a `df -P` non-host-local-disk
98
+ result refuses reclaim. Uncertainty never frees a slot it shouldn't.
99
+
100
+ ## 4. Blast radius
101
+
102
+ New: 3 source modules (`hostSpawnSemaphore`, `SpawnCapIntelligenceProvider`, `SingleInstanceLock`),
103
+ 1 lint, 1 route (`GET /spawn-limiter`). Touched: the factory funnel + `reflect.ts` (route the raw
104
+ fallback), `server.ts` boot (configure semaphore + acquire/release the instance lock), the four gate
105
+ seams, `types.ts` + `ConfigDefaults.ts` (the `spawnCap` config), `routes.ts` + `CapabilityIndex.ts`
106
+ (the route), `setup.ts` (launchd belt), `STANDARDS-REGISTRY.md` (the standard),
107
+ `templates.ts` + `PostUpdateMigrator.ts` (CLAUDE.md awareness), `package.json` + the destructive-lint
108
+ allowlist (wire the new lint). No schema change, no write-path semantics change beyond the
109
+ acquire/release bracket. The cap is ON by default — this is intentional (a SAFETY FLOOR), and the
110
+ unsaturated path is behavior-preserving.
111
+
112
+ ## 5. On-by-default justification
113
+
114
+ This is a safety floor, not a feature behind a dev gate. The 2026-06-20 incident was a full host OOM
115
+ ×2; the failure mode the cap prevents is catastrophic and physical. Reading the cap with `?? default`
116
+ (default 8) means every agent — new and updated — gets the protection without opting in. The default
117
+ of 8 is conservative (well below the ~230+ that OOM'd a 128 GB box) yet ample for normal CoherenceGate
118
+ fan-out; it is fully tunable for a larger host. Migration is existence-checked so an operator's
119
+ explicit override is never clobbered.
120
+
121
+ ## 6. Residual (tracked, NOT fixed here — out of scope)
122
+
123
+ - The elaborate predecessor's lanes / reserve-math / hysteresis / per-gate-typed-shed are
124
+ deliberately deferred — the simple design was chosen precisely because it is implementable
125
+ CORRECTLY in one pass. If real load data later shows the single global cap is too coarse, the lane
126
+ model is the documented follow-up.
127
+ - The launchd `NumberOfProcesses` belt is macOS-only; a Linux/systemd equivalent is a follow-up for
128
+ non-macOS hosts (the in-process semaphore is OS-agnostic and is the primary control regardless).
129
+ - No baseline JSON for the new lint — it ships clean (zero violations), so a baseline would be an
130
+ empty no-op (the sibling `lint-no-unfunneled-headless-launch.js` has none either).
131
+
132
+ ## 7. Rollback
133
+
134
+ Revert the source files. The semaphore/lock/wrapper are additive; reverting the factory-funnel wrap +
135
+ the `server.ts` boot lines restores the prior unbounded-spawn behavior, and reverting the four gate
136
+ seams restores their prior fail-open. No data migration and no on-disk format to undo — the
137
+ host-spawn-holders / single-instance-lock files are host-local runtime state that is simply abandoned
138
+ on revert. The config defaults are additive and harmless if left in place.