switchroom 0.19.4 → 0.19.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/dist/auth-broker/index.js +7 -3
  2. package/dist/cli/autoaccept-poll.js +8 -2
  3. package/dist/cli/switchroom.js +20 -5
  4. package/dist/host-control/main.js +1 -1
  5. package/package.json +1 -1
  6. package/profiles/_base/start.sh.hbs +67 -4
  7. package/telegram-plugin/dist/gateway/gateway.js +585 -302
  8. package/telegram-plugin/flushed-turn-supersede.ts +43 -7
  9. package/telegram-plugin/gateway/command-format.ts +253 -0
  10. package/telegram-plugin/gateway/gateway-heartbeat.ts +72 -0
  11. package/telegram-plugin/gateway/gateway.ts +128 -259
  12. package/telegram-plugin/gateway/hang-restart-decision.ts +189 -0
  13. package/telegram-plugin/gateway/liveness-wiring.ts +35 -1
  14. package/telegram-plugin/gateway/outbound-send-path.ts +51 -11
  15. package/telegram-plugin/gateway/pending-inbound-buffer.ts +27 -0
  16. package/telegram-plugin/gateway/session-model-file.ts +13 -0
  17. package/telegram-plugin/gateway/stream-render.ts +18 -1
  18. package/telegram-plugin/gateway/subagent-handback-marker.ts +42 -0
  19. package/telegram-plugin/gateway/turn-active-marker.ts +29 -17
  20. package/telegram-plugin/gateway/worker-feed-dispatch.ts +139 -0
  21. package/telegram-plugin/hooks/silent-end-interrupt-stop.mjs +87 -36
  22. package/telegram-plugin/hooks/silent-end-scan.mjs +263 -3
  23. package/telegram-plugin/render/line-start-guard.ts +76 -4
  24. package/telegram-plugin/reply-owner-resolve.ts +43 -7
  25. package/telegram-plugin/rich-send.ts +8 -1
  26. package/telegram-plugin/tests/command-format.test.ts +212 -0
  27. package/telegram-plugin/tests/flushed-turn-supersede.test.ts +89 -0
  28. package/telegram-plugin/tests/gateway-heartbeat.test.ts +70 -0
  29. package/telegram-plugin/tests/hang-restart-decision.test.ts +146 -0
  30. package/telegram-plugin/tests/hang-restart-marker-integration.test.ts +98 -0
  31. package/telegram-plugin/tests/narrative-lane-golden.test.ts +2 -1
  32. package/telegram-plugin/tests/render/heading-guard-blockquote-glued-hash.test.ts +86 -0
  33. package/telegram-plugin/tests/render/heading-guard.test.ts +114 -0
  34. package/telegram-plugin/tests/render/rich-corpus-seam-regression.test.ts +76 -0
  35. package/telegram-plugin/tests/reply-owner-resolve.test.ts +74 -0
  36. package/telegram-plugin/tests/send-reply-golden.test.ts +221 -6
  37. package/telegram-plugin/tests/silent-end-interrupt-stop-integration.test.ts +63 -0
  38. package/telegram-plugin/tests/silent-end-interrupt-stop-scan.test.ts +60 -16
  39. package/telegram-plugin/tests/silent-end-single-writer-election.test.ts +193 -0
  40. package/telegram-plugin/tests/silent-end.test.ts +60 -5
  41. package/telegram-plugin/tests/stream-render-golden.test.ts +2 -1
  42. package/telegram-plugin/tests/subagent-handback-marker.test.ts +36 -0
  43. package/telegram-plugin/tests/worker-feed-origin-race-defer.test.ts +321 -0
@@ -0,0 +1,189 @@
1
+ /**
2
+ * hang-restart-decision.ts — the progress-based hang-restart discriminator
3
+ * (Stage B safety net).
4
+ *
5
+ * WHY THIS EXISTS
6
+ *
7
+ * The silence-poke framework fallback (`liveness-wiring.ts onFrameworkFallback`)
8
+ * is the last-resort unwedge, but it only tears down GATEWAY state — it never
9
+ * kills the wedged `claude` child. So a turn that hangs mid-tool-call recovers
10
+ * its conversation slot but leaves the child stuck; the "carrie" incident
11
+ * needed a MANUAL restart. Stage B closes that: when the fallback fires with a
12
+ * tool still mid-call AND no real progress, escalate to an actual restart via
13
+ * the SIGTERM-PID1 path, then let the boot classifier route recovery through
14
+ * the ask-first `resume_watchdog_timeout` inbound (never assertive auto-resume).
15
+ *
16
+ * THE DISCRIMINATOR (the crux — do NOT key on "tool open N seconds")
17
+ *
18
+ * A healthy 20-minute research turn must NOT be killed. The honest progress
19
+ * signal is the turn-active marker's mtime: it is touched on every `tool_use`
20
+ * AND on foreground sub-agent JSONL growth (`subagent-watcher.ts`), so a turn
21
+ * that keeps advancing it is working; a stale mtime means work stopped.
22
+ *
23
+ * markerAgeMs (from readTurnActiveMarkerAgeMs) small → progress → NOT a hang
24
+ * markerAgeMs large, or null (no progress signal at all) → stale → hang
25
+ *
26
+ * THE MARKER-ADVANCEMENT BLIND SPOT + THE TOOL-CLASS GUARD (review Finding A)
27
+ *
28
+ * The marker advances only on `tool_use` and sub-agent JSONL growth. A healthy
29
+ * long SINGLE FOREGROUND tool with no sub-agent and no interim tool_use never
30
+ * advances it — a 15-min foreground `Bash` build/test, a big `WebFetch`/crawl,
31
+ * a multi-minute research call. At the 15-min silence ceiling that turn is
32
+ * mid-tool with a stale marker and would be WRONGLY restarted by marker
33
+ * staleness alone. So we additionally protect known-long-running tool classes:
34
+ * if any in-flight tool is a `background`/long-fetch/research class, we do NOT
35
+ * restart, regardless of marker age.
36
+ *
37
+ * RESIDUAL RISK (documented, not silently accepted): a genuine hang INSIDE a
38
+ * protected long-running tool (a truly-wedged `Bash`, a `WebFetch` that never
39
+ * returns) is indistinguishable from healthy long work by any signal the
40
+ * gateway has — the marker cannot advance mid-single-tool either way. Those
41
+ * are NOT auto-restarted here; recovery for that class relies on the tool's
42
+ * own timeout, an operator `/restart`, or the ordinary state-teardown. We
43
+ * choose to never false-kill a healthy long tool over catching the rare
44
+ * hung-inside-a-long-tool case, because the false-kill is the common,
45
+ * user-visible harm and the discriminator exists precisely to avoid it.
46
+ *
47
+ * Recovery routes through the ask-first resume path so a turn that would hang
48
+ * again the same way is reported to the user, not silently re-run — that is
49
+ * also the storm guard (see § STORM GUARD below).
50
+ *
51
+ * Pure + deterministic: the clock, the marker age, and the in-flight tool names
52
+ * are all injected. The gateway owns the SIGTERM side effect.
53
+ */
54
+
55
+ /**
56
+ * Tool classes ported from Stage A's tool-call-deadline taxonomy. `background`
57
+ * covers sub-agent dispatch + long Bash; `human` covers approval waits;
58
+ * `standard` is everything else. Stage B additionally treats a broader set of
59
+ * single-foreground long tools as hang-restart-protected (see
60
+ * `isHangRestartProtectedTool`).
61
+ */
62
+ export type ToolDeadlineClass = 'standard' | 'human' | 'background'
63
+
64
+ /** Human-in-the-loop waits. */
65
+ const HUMAN_WAIT_TOOLS = new Set<string>(['ask_user'])
66
+ /** Sub-agent dispatch — progress is sub-agent JSONL growth. */
67
+ const BACKGROUND_TOOLS = new Set<string>(['Task', 'Agent'])
68
+
69
+ /**
70
+ * Classify a tool into the Stage A taxonomy. `Bash` with `run_in_background` is
71
+ * `background`; `Task`/`Agent` are `background`; `ask_user` is `human`.
72
+ */
73
+ export function classifyToolClass(
74
+ toolName: string,
75
+ opts: { awaitingApproval?: boolean; backgroundBash?: boolean } = {},
76
+ ): ToolDeadlineClass {
77
+ if (opts.awaitingApproval === true) return 'human'
78
+ if (HUMAN_WAIT_TOOLS.has(toolName)) return 'human'
79
+ if (BACKGROUND_TOOLS.has(toolName)) return 'background'
80
+ if (toolName === 'Bash' && opts.backgroundBash === true) return 'background'
81
+ return 'standard'
82
+ }
83
+
84
+ /**
85
+ * Exact-name tools that legitimately run long as a SINGLE foreground call
86
+ * without touching the marker, so their staleness is not a hang signal.
87
+ */
88
+ const HANG_PROTECTED_EXACT = new Set<string>([
89
+ 'Task', 'Agent', // sub-agent dispatch (background class)
90
+ 'Bash', // foreground builds/tests routinely run many minutes
91
+ 'WebFetch', 'WebSearch',
92
+ ])
93
+
94
+ /**
95
+ * Name substrings marking long-running fetch/crawl/research MCP tools (matched
96
+ * case-insensitively) — perplexity_research, deep research, site crawls, etc.
97
+ */
98
+ const HANG_PROTECTED_SUBSTRINGS = ['research', 'perplexity', 'crawl', 'deep_research', 'webkite']
99
+
100
+ /**
101
+ * True when a tool is a known-long-running / opaque-progress class that must
102
+ * NOT be hang-restarted on marker staleness (review Finding A). Covers the
103
+ * `background` class plus foreground Bash, web fetch/search, and research/crawl
104
+ * MCP tools. `ask_user` (`human`) is also protected — an operator wait is not a
105
+ * hang. See the module header § RESIDUAL RISK for the tradeoff.
106
+ */
107
+ export function isHangRestartProtectedTool(toolName: string): boolean {
108
+ if (HANG_PROTECTED_EXACT.has(toolName)) return true
109
+ if (classifyToolClass(toolName) !== 'standard') return true // background / human
110
+ const lower = toolName.toLowerCase()
111
+ return HANG_PROTECTED_SUBSTRINGS.some((s) => lower.includes(s))
112
+ }
113
+
114
+ export interface HangRestartInput {
115
+ /** Names of the tools in flight when the fallback fired (silence-poke
116
+ * snapshot). Empty ⇒ not a mid-tool fallback (ordinary teardown owns it). */
117
+ inFlightToolNames: string[]
118
+ /** Turn-active marker mtime age in ms (`readTurnActiveMarkerAgeMs`), or null
119
+ * when the marker is absent/unstattable (no progress signal). */
120
+ markerAgeMs: number | null
121
+ /** Staleness ceiling: marker age at/above this counts as "no progress". Keyed
122
+ * to the same TURN_HANG_SECS the boot classifier uses so the live decision
123
+ * and the boot reclassification agree on what "stalled" means. */
124
+ stalenessThresholdMs: number
125
+ }
126
+
127
+ export interface HangRestartDecision {
128
+ restart: boolean
129
+ reason: string
130
+ }
131
+
132
+ /**
133
+ * Decide whether a mid-tool framework-fallback should escalate to a real
134
+ * restart. Pure.
135
+ *
136
+ * - not mid-tool → no restart (ordinary teardown path owns it)
137
+ * - a protected long tool in flight → no restart (healthy long tool / sub-
138
+ * agent / research — the crux false-positive we
139
+ * must never kill; see § RESIDUAL RISK)
140
+ * - marker still advancing → no restart (real progress)
141
+ * - marker stale or absent → RESTART (genuine hang: mid-tool + no progress
142
+ * + no protected long tool that would explain it)
143
+ */
144
+ export function decideHangRestart(input: HangRestartInput): HangRestartDecision {
145
+ if (input.inFlightToolNames.length === 0) return { restart: false, reason: 'not-mid-tool' }
146
+ // Finding A: never restart while a known-long-running tool is in flight — a
147
+ // stale marker is EXPECTED for a single long foreground tool.
148
+ const protectedName = input.inFlightToolNames.find(isHangRestartProtectedTool)
149
+ if (protectedName != null) {
150
+ return { restart: false, reason: `protected-long-tool:${protectedName}` }
151
+ }
152
+ // A working turn keeps touching the marker (tool_use + sub-agent JSONL
153
+ // growth), so a SMALL non-null age means real progress.
154
+ if (input.markerAgeMs != null && input.markerAgeMs < input.stalenessThresholdMs) {
155
+ return { restart: false, reason: 'marker-advancing' }
156
+ }
157
+ return { restart: true, reason: 'mid-tool-marker-stale' }
158
+ }
159
+
160
+ /** Default staleness ceiling (ms) — mirrors the watchdog's TURN_HANG_SECS=300. */
161
+ export const DEFAULT_HANG_STALENESS_MS = 300_000
162
+
163
+ /**
164
+ * Resolve the staleness ceiling from the environment, keyed to the SAME
165
+ * `TURN_HANG_SECS` the boot classifier reads, so a live kill and the boot
166
+ * reclassification agree on "stalled". Blank/garbage/non-positive → default.
167
+ */
168
+ export function hangStalenessMs(
169
+ env: Record<string, string | undefined> = process.env,
170
+ ): number {
171
+ const raw = env.TURN_HANG_SECS
172
+ const n = Number(raw)
173
+ if (!Number.isFinite(n) || n <= 0) return DEFAULT_HANG_STALENESS_MS
174
+ return Math.floor(n * 1000)
175
+ }
176
+
177
+ /**
178
+ * § STORM GUARD (review Finding B)
179
+ *
180
+ * There is deliberately NO restart cooldown here. A cooldown could only engage
181
+ * if two hang-restarts fired close together, but two mid-tool framework
182
+ * fallbacks are >= the 15-min silence-defer ceiling apart (they only fire after
183
+ * that ceiling), so a sub-15-min cooldown can never engage on the organic path
184
+ * — a dead guard giving false confidence. The REAL storm guard is the ask-first
185
+ * `resume_watchdog_timeout` recovery: after a hang-restart the agent does NOT
186
+ * auto-resume the wedged work — it reports to the user and asks — so the same
187
+ * hang cannot immediately recur without fresh human action. That is sufficient
188
+ * and self-documenting; a persisted cooldown file is not needed.
189
+ */
@@ -24,7 +24,8 @@ import { emitRuntimeMetric } from '../runtime-metrics.js'
24
24
  import { logStreamingEvent } from '../streaming-metrics.js'
25
25
  import { clearSilentEndState } from '../silent-end.js'
26
26
  import { purgeStaleTurnsForChat } from './turn-state-purge.js'
27
- import { removeTurnActiveMarker } from './turn-active-marker.js'
27
+ import { removeTurnActiveMarker, readTurnActiveMarkerAgeMs } from './turn-active-marker.js'
28
+ import { decideHangRestart } from './hang-restart-decision.js'
28
29
  import { recordTurnEnd } from '../registry/turns-schema.js'
29
30
  import { hostdGetStatusOnce } from './hostd-dispatch.js'
30
31
  import { formatUpdateStatusLine } from './update-status-line.js'
@@ -60,6 +61,7 @@ export function buildSilencePokeOptions(deps: LivenessWiringDeps): Parameters<ty
60
61
  trackRedeliveredInbound,
61
62
  closeActivityLane,
62
63
  closeProgressLane,
64
+ hangRestart,
63
65
  } = deps
64
66
 
65
67
  return {
@@ -160,6 +162,38 @@ export function buildSilencePokeOptions(deps: LivenessWiringDeps): Parameters<ty
160
162
  return
161
163
  }
162
164
 
165
+ // Stage B: escalate a mid-tool hang to a REAL restart. The fallback fires
166
+ // here with a tool still in flight only after silence crossed the 15-min
167
+ // defer ceiling (isLegitimatelyWorking held it that long). The honest
168
+ // hang/health discriminator is the turn-active marker's mtime age: a healthy
169
+ // turn keeps advancing it (tool_use + sub-agent JSONL growth), a true hang
170
+ // lets it go stale. A known-long-running tool in flight (Task/Agent/Bash/
171
+ // WebFetch/research) is protected regardless of marker age — a single long
172
+ // foreground tool legitimately can't advance the marker (Finding A). Only a
173
+ // stale (or absent) marker with no protected long tool triggers the
174
+ // SIGTERM-PID1 restart; the boot classifier then routes recovery through the
175
+ // ask-first resume_watchdog_timeout inbound. We return BEFORE the state-
176
+ // teardown so the turn-active marker survives for the boot reclassification.
177
+ if (hangRestart != null && ctx.inFlightTools.length > 0) {
178
+ const markerAgeMs = readTurnActiveMarkerAgeMs(STATE_DIR)
179
+ const inFlightToolNames = ctx.inFlightTools.map((t) => t.name)
180
+ const decision = decideHangRestart({
181
+ inFlightToolNames,
182
+ markerAgeMs,
183
+ stalenessThresholdMs: hangRestart.stalenessThresholdMs,
184
+ })
185
+ process.stderr.write(
186
+ `telegram gateway: [hang-watchdog] fallback mid-tool chat=${ctx.chatId} ` +
187
+ `thread=${ctx.threadId ?? '-'} silence_ms=${ctx.silenceMs} ` +
188
+ `tools=${inFlightToolNames.join(',')} marker_age_ms=${markerAgeMs ?? 'null'} ` +
189
+ `restart=${decision.restart} reason=${decision.reason}\n`,
190
+ )
191
+ if (decision.restart) {
192
+ hangRestart.request(decision.reason, markerAgeMs ?? ctx.silenceMs)
193
+ return
194
+ }
195
+ }
196
+
163
197
  // Deterministic in-flight update status (klanker incident). If this
164
198
  // gateway dispatched an update_apply that's still running, the
165
199
  // recurring framework fallback carries hostd's REAL phase + elapsed
@@ -56,9 +56,10 @@ import { decideSilentReplyAnchor } from '../silent-reply-anchor.js'
56
56
  import {
57
57
  decideSupersedeCorrection,
58
58
  flushedAnswerMatchesReply,
59
+ DEFAULT_SUPERSEDE_TTL_MS,
59
60
  type FlushedTurnSupersedeRegistry,
60
61
  } from '../flushed-turn-supersede.js'
61
- import { decideAnswerLatchSuppression } from '../reply-owner-resolve.js'
62
+ import { decideAnswerLatchSuppression, type ReplyOwnerTier } from '../reply-owner-resolve.js'
62
63
  import { deriveTelegraphTitle } from '../telegraph.js'
63
64
  import {
64
65
  mintVoiceOnDemandToken,
@@ -658,7 +659,7 @@ export interface SendReplyGatewayDeps {
658
659
  assertSendable(f: string): void
659
660
  statusKey(chatId: string, threadId?: number | null): string
660
661
  streamKey(chatId: string, threadId?: number | null): string
661
- resolveReplyOwnerTurn(liveTurn: CurrentTurn | null, chatId: string, args: Record<string, unknown>): CurrentTurn | null
662
+ resolveReplyOwnerTurn(liveTurn: CurrentTurn | null, chatId: string, args: Record<string, unknown>): { turn: CurrentTurn | null; tier: ReplyOwnerTier }
662
663
  findTurnByOriginId(originTurnId: string | null | undefined): CurrentTurn | null
663
664
  findTurnByQuotedMessageId(chatId: string, replyTo: unknown): CurrentTurn | null
664
665
  resolveAnswerThreadWithLog(
@@ -671,6 +672,7 @@ export interface SendReplyGatewayDeps {
671
672
  ): number | undefined
672
673
  resolveThreadId(chatId: string, explicit?: string | number | null): number | undefined
673
674
  getLatestInboundMessageId(chatId: string, threadId: number | null): number | null | undefined
675
+ getLastSubagentHandbackAt(chatId: string): number | null
674
676
  recordOutbound(rec: {
675
677
  chat_id: string
676
678
  thread_id: number | null
@@ -735,7 +737,7 @@ export async function sendReply(
735
737
  statusKey, streamKey,
736
738
  resolveReplyOwnerTurn, findTurnByOriginId, findTurnByQuotedMessageId,
737
739
  resolveAnswerThreadWithLog, resolveThreadId,
738
- getLatestInboundMessageId, recordOutbound,
740
+ getLatestInboundMessageId, getLastSubagentHandbackAt, recordOutbound,
739
741
  emissionAuthorityFor, clearActivitySummary,
740
742
  startTypingLoop, stopTypingLoop, logOutbound,
741
743
  closeObligationOnSubstantiveReply, finalizeStatusReaction,
@@ -874,18 +876,56 @@ export async function sendReply(
874
876
  // double-send). The quoted / latest-ended recoveries are precisely what the
875
877
  // router already did for the same reply, so unifying here makes the two
876
878
  // resolvers agree and the late-reply supersede fires by identity.
877
- const ownerTurn = resolveReplyOwnerTurn(turn, chat_id, args)
879
+ const { turn: ownerTurn, tier: ownerTier } = resolveReplyOwnerTurn(turn, chat_id, args)
878
880
  const resolvedTurnId = ownerTurn?.turnId ?? null
879
- // #3429 — pass the (normalized) reply text so the registry can apply the
880
- // new-content gate: identity match + TTL alone also fits an async handback
881
- // that merely resolved this flush-delivered ENDED turn as its owner via
882
- // the latest-ended tier. Editing the flushed message in place with that
883
- // handback's text does not re-notify client-side (Telegram edits never
884
- // push) the observed silent non-surfacing of msgs 10482/10486.
881
+ // #3429 — pass the (normalized) reply text so the registry CAN apply the
882
+ // new-content gate: identity match + TTL alone also fits a background
883
+ // sub-agent handback that merely resolved this flush-delivered ENDED turn as
884
+ // its owner via the latest-ended tier. Editing/deleting the flushed message
885
+ // for that handback's unrelated text is the #3429 silent client-side drop
886
+ // (msgs 10482/10486). But that gate ALSO declines the turn's OWN reworded
887
+ // late reply (model narrated → flushed → fired `reply` with a paraphrase),
888
+ // which is the dominant real duplicate (agent:marko 2026-07-20: 11/11
889
+ // declines were own-replies, turns #1177/#1182/#1201 double-sent). So we
890
+ // BYPASS the content gate ONLY when the reply is confidently the flushed
891
+ // turn's OWN answer. The primary, deterministic signal is the absence of a
892
+ // background sub-agent handback that could own this reply: the gateway
893
+ // records when it synthesizes a `subagent_handback` inbound per chat, and if
894
+ // NONE was enqueued for this chat AFTER the flushed turn ended within the
895
+ // supersede TTL, the reply is that turn's own answer → supersede regardless
896
+ // of a model rewording (closes the DM default-reply duplicate, where every
897
+ // own-reply resolves via the latest-ended tier).
898
+ //
899
+ // MUST-FIX 1 (silent-data-loss): the owner-resolution `quoted` / `origin`
900
+ // tiers are derived from MODEL-SUPPLIED args (`args.reply_to` /
901
+ // `args.origin_turn_id`), so a background handback turn can STEER them —
902
+ // pass `reply_to = <the user's original msg id>` and it resolves the PRIOR
903
+ // flushed turn via `quoted`, which used to force the bypass and silently
904
+ // edit-over that turn's delivered answer (#3429-class, model-steerable). So
905
+ // a positive tier NEVER overrides an in-window handback; only the
906
+ // FRAMEWORK-owned `live` tier (the live `currentTurn`, not model-derived,
907
+ // and structurally unable to collide with an ended turn's flush record —
908
+ // `decideSupersede` requires same turnId) may bypass a handback window.
909
+ //
910
+ // The content gate is therefore kept whenever a handback WAS enqueued in the
911
+ // window (any non-live tier): the late reply might BE that handback, so
912
+ // genuinely-new content sends fresh (two messages, #3429 preserved).
913
+ // Concurrency note: a case-A own reply coinciding with an unrelated
914
+ // background handback in the same ≤TTL window degrades to today's behaviour
915
+ // (two messages) — safe (never a silent drop/edit), just not collapsed.
916
+ const ownerEndedAt = ownerTurn?.endedAt ?? null
917
+ const handbackAt = getLastSubagentHandbackAt(chat_id)
918
+ const now = Date.now()
919
+ const handbackCouldOwnReply =
920
+ handbackAt != null &&
921
+ ownerEndedAt != null &&
922
+ handbackAt > ownerEndedAt &&
923
+ now - handbackAt <= DEFAULT_SUPERSEDE_TTL_MS
924
+ const replyIsOwnAnswer = ownerTier === 'live' || !handbackCouldOwnReply
885
925
  const decision = flushedTurnSupersede.take(
886
926
  chat_id,
887
927
  replyThreadId,
888
- { liveTurnId: resolvedTurnId, replyText: text, now: Date.now() },
928
+ { liveTurnId: resolvedTurnId, replyText: text, positiveAttribution: replyIsOwnAnswer, now },
889
929
  )
890
930
  if (decision.supersede) {
891
931
  process.stderr.write(
@@ -79,6 +79,16 @@ export interface PendingInboundBufferOptions {
79
79
  * never breaks the push hot path.
80
80
  */
81
81
  onEvict?: (agent: string, evicted: InboundMessage) => void
82
+ /**
83
+ * fix/backstop-duplicate-reply MUST-FIX 2 — called on every push of a
84
+ * `subagent_handback` envelope (live synthesis AND boot-replay re-push),
85
+ * carrying the envelope's `chatId` and its own `ts` (ms). The gateway wires
86
+ * this to the per-chat subagent-handback marker so the supersede path can tell
87
+ * a flushed turn's own late reply from a background handback attributed to it
88
+ * — INCLUDING after a restart, where the only handback push is the replay.
89
+ * Best-effort: a throw here never breaks the push hot path.
90
+ */
91
+ onHandbackEnqueue?: (chatId: string, ts: number) => void
82
92
  }
83
93
 
84
94
  /**
@@ -342,6 +352,23 @@ export function createPendingInboundBuffer(
342
352
  }
343
353
  }
344
354
  q.push(msg)
355
+ // fix/backstop-duplicate-reply MUST-FIX 2 — stamp the subagent-handback
356
+ // marker at THIS chokepoint, not at the live onFinish enqueue site alone.
357
+ // Every handback enqueue funnels through here — the live synthesis push
358
+ // AND the boot-replay re-push of un-acked spooled inbounds — so stamping
359
+ // here (rather than only at the live site) means a handback replayed after
360
+ // a restart still populates the marker. Otherwise the Map is empty
361
+ // post-boot and a replayed handback's late reply bypasses the #3429
362
+ // content gate → silent edit-over-answer. Uses the envelope's own `ts`
363
+ // (ms, `Date.now()`-derived at synthesis) so the marker reflects when the
364
+ // handback actually happened, not the replay moment. Best-effort.
365
+ if (msg.meta?.source === 'subagent_handback' && opts.onHandbackEnqueue != null) {
366
+ try {
367
+ opts.onHandbackEnqueue(msg.chatId, msg.ts)
368
+ } catch {
369
+ /* marker stamp is best-effort; never break the push hot path */
370
+ }
371
+ }
345
372
  // Durable record FIRST-class to the in-memory queue: spool BEFORE
346
373
  // returning, regardless of the cap eviction above — an entry the
347
374
  // in-memory cap drops still survives in the spool (boot-replayed /
@@ -104,6 +104,19 @@ export function writeSessionModelFile(
104
104
  if (!isValidModelArg(model)) {
105
105
  throw new Error(`refusing to persist non-canonical session model token: ${JSON.stringify(model)}`)
106
106
  }
107
+ // A FRESH carrier gets a FRESH retry budget: clear any bounded-retry counter
108
+ // left over from a prior carrier's boots. Without this, an orphan counter
109
+ // survives the whole session on a healthy snapshot-apply boot (the apply
110
+ // branch re-writes the live counter AFTER the gateway's healthy-boot consume
111
+ // already deleted it), gets snapshotted alongside the NEW carrier at the next
112
+ // boot, and burns the new override's retry budget from a stale count — three
113
+ // such sessions in a row and a perfectly healthy /model is refused with a
114
+ // false "did not reach a healthy boot after 3 attempts" revert.
115
+ try {
116
+ rmSync(join(agentDir, SESSION_MODEL_BOOT_ATTEMPTS_FILE), { force: true })
117
+ } catch {
118
+ /* best-effort — worst case is a stale count, bounded and non-fatal */
119
+ }
107
120
  atomicWrite(
108
121
  join(agentDir, SESSION_MODEL_FILE),
109
122
  serializeSessionModel({ model, configuredDefaultAtWrite, ts: Date.now() }),
@@ -2018,6 +2018,23 @@ export function handleSessionEvent(deps: StreamRenderDeps, ev: SessionEvent): vo
2018
2018
  // represent. The delivery closes the obligation + records dedup, so
2019
2019
  // the represent and a late reply-tool retry are both suppressed —
2020
2020
  // captured-prose delivery and represent are mutually exclusive.
2021
+ // Capture-divergence corner (duplicate-message fix): when this is a
2022
+ // ZERO-reply silent-end AND the gateway's OWN capture came up empty
2023
+ // (so `decideTurnFlush` had nothing to flush — its 'empty-text'
2024
+ // skip), the captured-prose bridge is the ONLY delivery machine
2025
+ // left. The Stop hook scanned the transcript and DID find the
2026
+ // model's short trailing answer, persisting it as `pendingText`. In
2027
+ // that corner the 200-char substance floor would wrongly drop a
2028
+ // legitimately-short answer that no other machine can deliver, so
2029
+ // lower the floor to 1. This is scoped tightly: only when the model
2030
+ // never called reply (`replyCalled === false`) AND the gateway
2031
+ // captured nothing — the interim-ack case (replyCalled) keeps the
2032
+ // full 200 floor (a short closer after a real reply is not a dropped
2033
+ // answer, and the hook already refuses to persist <200 there).
2034
+ const gatewayCapturedEmpty =
2035
+ turn.capturedText.join('\n\n').trim().length === 0
2036
+ const proseMinChars =
2037
+ !turn.replyCalled && gatewayCapturedEmpty ? 1 : CAPTURED_PROSE_MIN_CHARS
2021
2038
  const proseDecision = CAPTURED_PROSE_DELIVERY_ENABLED
2022
2039
  ? decideCapturedProseDelivery({
2023
2040
  turnKey: tKey,
@@ -2025,7 +2042,7 @@ export function handleSessionEvent(deps: StreamRenderDeps, ev: SessionEvent): vo
2025
2042
  // belong to THIS turn, not a stale carryover from a prior turn
2026
2043
  // on the same chat/thread (tKey is not per-turn unique).
2027
2044
  turnId: turn.turnId,
2028
- minChars: CAPTURED_PROSE_MIN_CHARS,
2045
+ minChars: proseMinChars,
2029
2046
  })
2030
2047
  : { deliver: false as const, reason: 'no-state' as const }
2031
2048
  if (proseDecision.deliver && proseDecision.text != null) {
@@ -0,0 +1,42 @@
1
+ /**
2
+ * Per-chat marker of the most recent gateway-synthesized `subagent_handback`
3
+ * enqueue (fix/backstop-duplicate-reply).
4
+ *
5
+ * A BACKGROUND sub-agent completion is a GATEWAY-SYNTHESIZED event, not model
6
+ * output: when a background worker terminates the gateway wakes the agent with a
7
+ * `subagent_handback` inbound. Recording WHEN one was enqueued, per chat, is the
8
+ * ONE deterministic signal that distinguishes the two late-reply cases that both
9
+ * resolve a flush-delivered ENDED turn via the latest-ended tier — the case the
10
+ * owner-resolution tier alone cannot separate (a DM late reply has no
11
+ * live/origin/quoted attribution, so both land on latest-ended):
12
+ *
13
+ * - CASE A — the flushed turn's OWN reworded reply landing late. NO
14
+ * `subagent_handback` was enqueued for this chat after the turn ended, so the
15
+ * reply is that turn's own answer → the supersede path collapses the
16
+ * provisional flush REGARDLESS of the model's rewording (closes the #3429
17
+ * reworded-duplicate regression: agent:marko 2026-07-20, turns
18
+ * #1177/#1182/#1201 double-sent).
19
+ * - CASE B — a background handback attributed to that ended turn. A
20
+ * `subagent_handback` WAS enqueued after the turn ended and within the
21
+ * supersede TTL, so the late reply might BE it → keep the #3429 content gate
22
+ * and send fresh (two messages), never silently edit/delete the flushed
23
+ * answer.
24
+ *
25
+ * One entry per chat (overwritten on each enqueue), so bounded by chat count. No
26
+ * clock reads beyond the caller-supplied `now`; the gateway wires the actual
27
+ * enqueue site and the supersede-path read. Deterministic — keyed on a
28
+ * gateway-emitted event, never on model discipline (Ken's controls-in-code rule).
29
+ */
30
+ export class SubagentHandbackMarker {
31
+ private readonly lastAtByChat = new Map<string, number>()
32
+
33
+ /** Record that a `subagent_handback` was enqueued for `chatId` at `now` (ms). */
34
+ record(chatId: string, now: number): void {
35
+ this.lastAtByChat.set(chatId, now)
36
+ }
37
+
38
+ /** Wall-clock ms of the most recent handback enqueue for `chatId`, or null. */
39
+ lastAt(chatId: string): number | null {
40
+ return this.lastAtByChat.get(chatId) ?? null
41
+ }
42
+ }
@@ -1,26 +1,38 @@
1
1
  /**
2
2
  * Turn-active liveness marker (#412).
3
3
  *
4
- * Writes `<STATE_DIR>/turn-active.json` on turn_start, touches its mtime
5
- * on every tool_use, removes it on turn_complete. The watchdog
6
- * (bin/bridge-watchdog.sh) reads the mtime: if the file exists AND its
7
- * mtime is older than TURN_HANG_SECS (default 300s = 5min), the agent
8
- * is wedged mid-turn and the watchdog restarts.
4
+ * Writes `<STATE_DIR>/turn-active.json` on turn_start, touches its mtime on
5
+ * every tool_use AND (via the subagent-watcher, #501) on foreground sub-agent
6
+ * JSONL growth, removes it on turn_complete. The mtime is therefore "ms since
7
+ * last observable progress": it advances while a turn even a long one running
8
+ * one big tool or a sub-agent — is genuinely working, and goes stale only when
9
+ * work stops (a wedge).
9
10
  *
10
- * Why this exists: PR #410 raised the journal-silence detector to 4000s
11
- * to kill false positives on chat-cadence agents that legitimately
12
- * idle for hours between turns. That left a gap — Stop-hook deadlocks
13
- * (the original failure mode #116 tracked) are no longer caught under
14
- * default thresholds.
11
+ * Why this exists: PR #410 raised the journal-silence detector to 4000s to kill
12
+ * false positives on chat-cadence agents that legitimately idle for hours
13
+ * between turns. That left a gap — Stop-hook deadlocks (the original failure
14
+ * mode #116 tracked) are no longer caught under default thresholds. The
15
+ * distinguisher is "in-turn-and-silent" vs "between-turns-and-silent": the
16
+ * former is a wedge, the latter is healthy idle. This marker exists exactly
17
+ * during in-turn windows, so its staleness uniquely indicates the wedge.
15
18
  *
16
- * The distinguisher is "in-turn-and-silent" vs "between-turns-and-silent":
17
- * the former is a wedge, the latter is healthy idle. This marker exists
18
- * exactly during in-turn windows, so its staleness uniquely indicates
19
- * the wedge.
19
+ * WHO CONSUMES THE STALENESS (contract corrected — the historical
20
+ * `bin/bridge-watchdog.sh` bash watchdog this header once named was never
21
+ * committed to the repo; do not reintroduce a reference to it):
20
22
  *
21
- * Pure file I/O. The actual hang-detection-and-restart loop lives in the
22
- * bash watchdog, where it composes with the existing
23
- * Restart=on-failure / journal-silence / bridge-disconnect detectors.
23
+ * - The gateway BOOT classifier (`markOrphanedWithTimeoutClassification` in
24
+ * gateway.ts): on restart, a marker whose mtime is older than TURN_HANG_SECS
25
+ * (default 300s) reclassifies the orphaned turn as `timeout`, routing the
26
+ * next session to the ask-first `resume_watchdog_timeout` inbound.
27
+ * - The LIVE Stage B hang-restart (`hang-restart-decision.ts`, consulted from
28
+ * the silence-poke framework fallback): a mid-tool fallback with a stale
29
+ * marker escalates to a real SIGTERM-PID1 restart. `readTurnActiveMarkerAgeMs`
30
+ * below is the shared read.
31
+ * - The obligation / phantom-turn sweeps (`readTurnActiveMarkerAgeMs`,
32
+ * `effectiveTurnAgeMs`): a small age means work is still in flight, so a
33
+ * "did I miss this?" re-send is suppressed.
34
+ *
35
+ * Pure file I/O; the mtime is the honest cross-process progress signal.
24
36
  */
25
37
 
26
38
  import {