tokenmaxxing 1.1.0 → 1.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DESIGN.md +3 -2
- package/package.json +1 -1
- package/src/cli/serve.ts +19 -3
- package/src/lib/slackbridge.ts +291 -50
package/DESIGN.md
CHANGED
|
@@ -97,13 +97,14 @@ A local Socket Mode daemon (no public URL) that turns Slack threads into Claude
|
|
|
97
97
|
- **Serve skills** (`src/serve-plugin/`, ships in the package via `files: ["src"]`): a Claude Code plugin loaded per turn (`plugins: [{type: "local", path}]`; discovered skills are enabled by default, so no `skills` option). `tokenmaxxing:ask-the-user` teaches the decision protocol - when input is needed, tag the requester with the raw Slack mention token `<@U...>`, ask compactly, END the turn (the thread reply is the next turn); `tokenmaxxing:serve-session` documents how the session runs (shared repo checkout, resume across turns, worktree-by-default for mutating tasks, handoff). The one dynamic fact skills cannot carry - who asked - rides in per turn via a `UserPromptSubmit` hook whose `additionalContext` ("Slack relay context: ...", built by `serveTurnContext`) names the triggering message author's mention token. The mention survives the pipeline because streamed `markdown_text` deltas pass verbatim and the post-and-edit fallback's `finalize` only linkifies bare `@U...`, never escaping an already-formed `<@U...>`. Skills are independent of the system prompt choice (probe-verified 2026-07-18 under the custom-string `SLACK_SYSTEM_PROMPT`: the init message lists the plugin + both skills + the Skill tool; the skill listing arrives as a conversation system-reminder, and CLAUDE.md loads via settingSources).
|
|
98
98
|
- **Socket lifecycle**: `bot.initialize()` starts the persistent auto-reconnecting SocketModeClient; the daemon then just stays alive. The leased `startSocketModeListener` API must never be looped: it returns instantly without `waitUntil` and the loop starves the event loop (live incident 2026-07-18 - connected but silent).
|
|
99
99
|
- **Id mapping**: Chat SDK ids are adapter-prefixed (`thread.channelId` = `slack:C0123`, `thread.id` = `slack:C0123:<threadTs>`) while links store bare Slack ids - lookups strip the prefix via `bareChannelId`. Subscriptions live in the daemon's memory state, so every mention re-subscribes its thread; queue-skipped messages (`context.skipped`) fold into the next prompt, with the queue-entry TTL raised to 1h (expiry is silent - no app callback in chat 4.34.0 - so it must outlast a depleted-pool park plus a long turn); per-thread turn serialization is owned by the daemon itself (a promise chain per thread id), because the SDK's queue dispatch lock has a 30s TTL extended only between dispatches and every claude turn outlives it; unlinked-channel traffic logs `serve.unlinked_channel` and stays silent in Slack.
|
|
100
|
-
- **Restart resilience** (0.19.0, from the 2026-07-18 dead-thread incident: a deploy restart cut a turn mid-answer and left the thread deaf to follow-ups): startup re-subscribes every `slack-threads/` record straight on the state adapter (`state.subscribe(threadId)`; message routing checks `stateAdapter.isSubscribed`, verified in chat 4.34.0), so open threads survive restarts without needing a fresh mention. SIGTERM/SIGINT drains instead of dying: new turns are dropped loudly (`serve.drain_dropped` plus a tracked in-thread notice - a log-only drop reads as the bot thinking), tracked in-flight turns get up to 300s to finish (a re-snapshotting wait, so late-added notices still flush), then `Chat.shutdown()`; a second signal forces exit; the drain also aborts any depleted-pool park so a countdown never delays a restart. The claude child spawns detached in its own process group (`detachedClaudeSpawn` via the SDK's `spawnClaudeCodeProcess` hook), because a terminal Ctrl-C signals the whole foreground group and a non-detached child died with the daemon before the drain could save the turn; the signal handlers register before `bot.initialize()` so no turn can start while the process still has default signal disposition, and a rejecting `Chat.shutdown()` is caught so the drain always reaches its exit. When a `thread.post` rejects mid-turn (e.g. Slack finalizes an idle stream: `message_not_in_streaming_state`), relayThread
|
|
100
|
+
- **Restart resilience** (0.19.0, from the 2026-07-18 dead-thread incident: a deploy restart cut a turn mid-answer and left the thread deaf to follow-ups): startup re-subscribes every `slack-threads/` record straight on the state adapter (`state.subscribe(threadId)`; message routing checks `stateAdapter.isSubscribed`, verified in chat 4.34.0), so open threads survive restarts without needing a fresh mention. SIGTERM/SIGINT drains instead of dying: new turns are dropped loudly (`serve.drain_dropped` plus a tracked in-thread notice - a log-only drop reads as the bot thinking), tracked in-flight turns get up to 300s to finish (a re-snapshotting wait, so late-added notices still flush), then `Chat.shutdown()`; a second signal forces exit; the drain also aborts any depleted-pool park so a countdown never delays a restart. The claude child spawns detached in its own process group (`detachedClaudeSpawn` via the SDK's `spawnClaudeCodeProcess` hook), because a terminal Ctrl-C signals the whole foreground group and a non-detached child died with the daemon before the drain could save the turn; the signal handlers register before `bot.initialize()` so no turn can start while the process still has default signal disposition, and a rejecting `Chat.shutdown()` is caught so the drain always reaches its exit. When a `thread.post` rejects mid-turn (e.g. Slack finalizes an idle stream after an undocumented window: `message_not_in_streaming_state`), relayThread salvages the dead segment in TEXT space: proven-delivered text is a mirror StreamingMarkdownRenderer's committable prefix over the confirmed chunks (each pull proves the previous append landed; the adapter's renderer holds back the trailing unterminated line until its post-iteration forced flush, so chunk-granular salvage would miss a final reply with no trailing newline - the live 2026-07-21 incident), and the remainder re-posts as a fresh message split at line boundaries, plus unconfirmed cards, bounded by a futility budget that refills on delivery progress; the textLost diagnostic remains only for exhausted salvage, and ambiguous failures duplicate a tail line rather than lose it.
|
|
101
101
|
- **Interrupted-turn recovery** (from the second 2026-07-18 restart incident: a redeploy killed a ship turn 8 minutes in with zero notice - drain cannot save a long turn, and a group-wide kill can take the claude child even detached): the thread record persists the claude session id the moment the init message assigns it, and every turn is wrapped in a durable `activeTurn` marker (original prompt, start time, resume count) written before the spawn and cleared when the turn returns - except that a turn failing DURING a drain keeps its marker, since that failure is presumed to be the shutdown signal killing the child. On startup, a surviving marker means a restart killed that turn: the daemon posts a notice into the thread (the chat-sdk's documented proactive handle, rebuilt with the thread's newest human message as streaming recipient context so the resumed turn keeps its native task cards, that message's author becoming the turn's requester) and auto-resumes the work - resuming the recorded session with a continuation prompt, or replaying the original prompt fresh when the kill landed before the session opened; resumed turns settle like inbound ones (outcome log + finish_thread GC). Retries cap at 3 (each spends real quota) with a loud give-up notice posted before its marker clears; a per-thread turn lock keeps a startup resume from ever racing an inbound message turn in the same cwd, every recovery branch recomputes its decision from a fresh record reload under that lock (so a resume superseded by a faster inbound turn no-ops instead of double-running), and a blocking singleton flock makes a new daemon generation wait for the previous one - drain included - to fully exit before touching any thread record. Uncatchable deaths cannot leak a working child either: SIGHUP drains like SIGTERM (its default disposition skips the exit hook that kills the detached group), and the marker records the child's group pid plus its C-locale ps start-time token at spawn, so recovery kills only an exactly-identified SIGKILL-orphaned claude (never a recycled pid) before resuming its turn.
|
|
102
102
|
- **Depleted-pool recovery** (harvested from Slaude, reshaped around the pool): relayThread consumes the spawn-boundary switch decision instead of discarding it - a depleted pool with a known recovery inside the message's one 14min parking deadline (shared across chained parks, so the thread's queue slot is never held longer in total) posts a park notice and retries at the reset. A mid-turn limit - detected ONLY on errored results, since `is_error` can ride subtype `"success"` (`Claude AI usage limit reached|<epoch>`) and a successful answer discussing limits must never be re-run - is persisted into usage.json first (`recordObservedLimit`: the serve process has no statusLine tee, and the snapshot TTL would otherwise feed the retry the stale pre-limit state), then retried silently after a short beat, so a pool swap makes the hiccup invisible. Bounded recoveries; every drop the relay performs is announced in-thread.
|
|
103
103
|
- **Usage-limit auto-resume** (2026-07-21, superseding the drop-beats-false-promise rule for the known-wake case): a turn that ends at a usage limit with a KNOWN recovery time defers instead of dropping. relayThread reports `deferUntil`, the thread's durable activeTurn marker keeps a `resumeAt`, and a per-thread daemon timer (cancelled on drain, re-armed from the record at startup: future wakes schedule, past wakes recover immediately) fires the same interrupted-turn recovery path - serialized chain, fresh-reload supersede check, pool-recovery notice, `resumeDecision` capping total resumes. The defer triggers: known recovery past the parking deadline or recovery budget, retries exhausted while a post-burn pool probe reports a wake, and an unclassifiable child failure (no limit phrase in the error) converted by a pool probe reporting exhaustion - the pool state is the evidence the error text did not carry. Only an unknown recovery time still posts the honest re-send drop.
|
|
104
104
|
- **Slack-native output hygiene**: relayed turns run a small standalone `systemPrompt` telling the model replies render as Slack markdown, never HTML (a live turn once answered with a literal `<br>`; the SDK's default system prompt is minimal since 0.1.0, so the string replaces nothing).
|
|
105
105
|
- **Slash commands** (2026-07-18): a mention-stripped thread message that starts with `/` runs as a claude slash command - the SDK delivers a string prompt as one stream-json user message and the CLI routes a leading-slash prompt through its command table, on fresh and resumed sessions alike (live-verified with free `/usage` and `/context` reads). Local command output (`num_turns` 0) arrives as a non-streamed assistant message plus `result.result`, which relayThread's no-text fallback posts; `slackstream` also maps the documented `system/local_command_output` wire subtype in case a claude update flips the emitter. `/goal` (built-in since 2.1.139) works headless and its active goal is restored on resume, so it survives the per-message resume pattern. Slack gotcha: the composer eats ANY message whose first character is `/` client-side (channels and thread replies; registered app commands cannot even dispatch from threads), so the supported forms are `@bot /usage` (mention-first) and ` /usage` (leading space, Slack's own documented workaround) - both normalize to a position-0 command via `stripLeadingMention`.
|
|
106
|
-
- **Agent representation** (`src/lib/slackstream.ts`): turns stream natively (`chat.startStream`, which works in channel threads regardless of the assistant:write scope): thinking and tool calls as task cards ("Thinking"/tool name/"Turn", input summary + truncated output) that Slack groups into ONE collapsible plan block per turn (the serve edge wraps every posted segment in a `StreamingPlan` with `groupTasks: "plan"`, Slack's `task_display_mode: "plan"`; user ask 2026-07-20 "squash them into one dropdown", superseding the 2026-07-18 separate-messages shape), and reply text as native markdown with rendered code fences - so one turn streams as a single Slack message: the reply plus one dropdown of its activity. A later main text block (typically post-tool prose) opens on a `\n\n` paragraph break (`textStreamed`), restoring the separation the per-tool message split used to provide; whitespace-only deltas do not arm it. Notices still post as their own messages, and a rejected append (Slack's message cap, a finalized idle stream)
|
|
106
|
+
- **Agent representation** (`src/lib/slackstream.ts`): turns stream natively (`chat.startStream`, which works in channel threads regardless of the assistant:write scope): thinking and tool calls as task cards ("Thinking"/tool name/"Turn", input summary + truncated output) that Slack groups into ONE collapsible plan block per turn (the serve edge wraps every posted segment in a `StreamingPlan` with `groupTasks: "plan"`, Slack's `task_display_mode: "plan"`; user ask 2026-07-20 "squash them into one dropdown", superseding the 2026-07-18 separate-messages shape), and reply text as native markdown with rendered code fences - so one turn streams as a single Slack message: the reply plus one dropdown of its activity. Reply text also splits at `SEGMENT_TEXT_MAX` (10,000 chars): Slack rejects an over-long message with `msg_too_long` (the streamed message accumulates server-side toward the documented 12,000-char `markdown_text` envelope) and nothing in the chat stack bounds or splits text, so one over-cap append used to reject the whole post and lose the reply (live incident 2026-07-20); a break forced inside a code fence closes and reopens the fence, fence parity is tracked over the segment's accumulated text (SDK deltas do not respect markdown token boundaries), and a cut never slices through a backtick run. A later main text block (typically post-tool prose) opens on a `\n\n` paragraph break (`textStreamed`), restoring the separation the per-tool message split used to provide; whitespace-only deltas do not arm it. Notices still post as their own messages, and a rejected append (Slack's message cap, a finalized idle stream) carries its undelivered chunks into a follow-on message via the dead-segment salvage.
|
|
107
|
+
- **One Slack app per daemon** (live incident 2026-07-20): Slack Socket Mode delivers each event envelope to exactly one of an app's open connections, so two serve daemons sharing one Slack app steal each other's events - mentions usually survive (Slack emits `app_mention` plus `message.channels` for one mention and the Chat SDK dedupes them per-process), but a non-mention thread follow-up rides a single envelope and silently dies when the wrong daemon receives it. No code can route Slack's load-balancing (the hello frame's `num_connections` is discarded by `@slack/socket-mode` before any event fires), so the constraint is documented (docs limitations page) and diagnosed: `serve.unlinked_channel` logs once per channel per run with the shared-app explanation. Every host needs its own Slack app with its own tokens.
|
|
107
108
|
- **Todo checklist card**: TodoWrite is bookkeeping, not a real tool run, so it never opens a generic card. Instead each stream gets one stable-id "Todos" card (id `todos`, subagents `todos-<parent_tool_use_id>`) that updates in place on every TodoWrite: `✅ content` for completed, `🔄 activeForm` for the in-progress item (live narration), `⬜ content` for pending (the Chat SDK Plan object's own iconography); card status goes complete only when every item is completed. The "Todos have been modified successfully" tool_result is suppressed, but a FAILED TodoWrite flips the card to error (the optimistic checklist must not claim a state that never took effect). Card ids do not carry across messages, so a TodoWrite after a dead-segment recovery starts a fresh card in the follow-on message: accepted, the latest state is always in the newest message. claude >= 2.1.142 defaults to the structured Task tools and never emits TodoWrite, so relayThread sets `CLAUDE_CODE_ENABLE_TASKS: "0"` in the spawn env (the documented opt-out; without it the checklist card is inert).
|
|
108
109
|
- **Terminal echo** (2026-07-18): the daemon registers a `log()` echo (`setLogEcho` in log.ts, off by default so hooks and the statusline keep their stdout protocols clean), so every event while it runs - serve.* plus the in-process swap/decision events from `ensureBestAccount()`/`stopHookCheck` - also prints one colored line in the foreground terminal (`formatLogLine`: dim HH:MM:SS, event painted red/yellow/cyan by structural endsWith severity, redacted fields). `serve.turn_done`/`serve.turn_failed` (with seconds) close every relayed turn, so a foreground `xx serve` is observable without tailing `tokenmaxxing.log`.
|
|
109
110
|
- **Live-verified end-to-end** (2026-07-18, #tokenmaxxing-dogfooding): mention opens a session (worktree-per-thread at the time), replies stream, thread follow-ups resume with context, cards + fenced code render, segmentation and queue folding behave. Plus the hermetic suite: schemas/links, stream mapping, fail-fast paths.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "tokenmaxxing",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.2",
|
|
4
4
|
"description": "Automatic Claude Code account switching: pool multiple accounts and hot-swap when quota fills, resuming your session on the fresh account.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
package/src/cli/serve.ts
CHANGED
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
import { existsSync, realpathSync } from "node:fs";
|
|
19
19
|
import { delay, omit, uniq } from "es-toolkit";
|
|
20
20
|
import { z } from "zod";
|
|
21
|
-
import { Chat, StreamingPlan, ThreadImpl, type StreamChunk } from "chat";
|
|
21
|
+
import { Chat, ConsoleLogger, StreamingPlan, ThreadImpl, type StreamChunk } from "chat";
|
|
22
22
|
import { createSlackAdapter } from "@chat-adapter/slack";
|
|
23
23
|
import { createMemoryState } from "@chat-adapter/state-memory";
|
|
24
24
|
import {
|
|
@@ -328,6 +328,8 @@ export function buildServeRuntime(seam: {
|
|
|
328
328
|
// holds the restart hostage.
|
|
329
329
|
const drainAbort = new AbortController();
|
|
330
330
|
let draining = false;
|
|
331
|
+
// channels already diagnosed as unlinked this run (see handleTurn).
|
|
332
|
+
const unlinkedLogged = new Set<string>();
|
|
331
333
|
|
|
332
334
|
/** One relayed turn with the durable activeTurn marker around it: written
|
|
333
335
|
* before the spawn, cleared when the turn returns, so a marker surviving
|
|
@@ -426,8 +428,17 @@ export function buildServeRuntime(seam: {
|
|
|
426
428
|
// checked BEFORE the draining branch: unlinked channels are
|
|
427
429
|
// contractually log-only silent, and a drain-window drop notice posted
|
|
428
430
|
// into one would tell a user to resend a message that will never be
|
|
429
|
-
// served (closing-review catch).
|
|
430
|
-
|
|
431
|
+
// served (closing-review catch). Logged once per channel per daemon run:
|
|
432
|
+
// with several daemons sharing one Slack app this fires on every
|
|
433
|
+
// load-balanced envelope for a sibling's channel (live incident
|
|
434
|
+
// 2026-07-20), and the diagnosis needs one line, not a stream.
|
|
435
|
+
if (!unlinkedLogged.has(thread.channelId)) {
|
|
436
|
+
unlinkedLogged.add(thread.channelId);
|
|
437
|
+
log("serve.unlinked_channel", {
|
|
438
|
+
channel: thread.channelId,
|
|
439
|
+
note: "not linked on this host; if another tokenmaxxing serve shares this Slack app, Slack delivers each socket event to only ONE of them and thread replies get lost - give every daemon its own Slack app",
|
|
440
|
+
});
|
|
441
|
+
}
|
|
431
442
|
return; // not a linked channel - stay silent in Slack
|
|
432
443
|
}
|
|
433
444
|
if (draining) {
|
|
@@ -865,6 +876,11 @@ async function runDaemon(): Promise<number> {
|
|
|
865
876
|
// @slack/web-api's fiveRetriesInFiveMinutes literal (dep not declared,
|
|
866
877
|
// so the values are inlined).
|
|
867
878
|
webClientOptions: { retryConfig: { retries: 5, factor: 3.86 }, timeout: 15_000 },
|
|
879
|
+
// the adapter's default logger is info-level and prints a line per
|
|
880
|
+
// redelivered socket envelope ("Processing socket mode retry") - steady
|
|
881
|
+
// noise during restart catch-up. warn matches the Chat logger below;
|
|
882
|
+
// real degradations (streaming fallback etc.) are warn-level and survive.
|
|
883
|
+
logger: new ConsoleLogger("warn", "chat-sdk").child("slack"),
|
|
868
884
|
});
|
|
869
885
|
// held directly (not only via Chat) so startup can re-subscribe recorded
|
|
870
886
|
// threads: subscriptions live in this in-memory state and die with the
|
package/src/lib/slackbridge.ts
CHANGED
|
@@ -10,7 +10,7 @@ import { join } from "node:path";
|
|
|
10
10
|
import { z } from "zod";
|
|
11
11
|
import { delay } from "es-toolkit";
|
|
12
12
|
import { createSdkMcpServer, query, tool, type SpawnOptions } from "@anthropic-ai/claude-agent-sdk";
|
|
13
|
-
import type
|
|
13
|
+
import { StreamingMarkdownRenderer, type StreamChunk } from "chat";
|
|
14
14
|
import { ensureBestAccount, pooledOptions, stopHookCheck, type SwapDecision } from "../sdk.ts";
|
|
15
15
|
import { POST_SWAP_COOLDOWN_MS } from "./decide.ts";
|
|
16
16
|
import { readOAuthAccount } from "./claudejson.ts";
|
|
@@ -217,23 +217,41 @@ function pushableStream(): {
|
|
|
217
217
|
iterable: AsyncIterable<SegmentChunk>;
|
|
218
218
|
push: (chunk: SegmentChunk) => void;
|
|
219
219
|
end: () => void;
|
|
220
|
+
ledger: () => { chunks: SegmentChunk[]; confirmed: number };
|
|
220
221
|
} {
|
|
221
|
-
const
|
|
222
|
+
const chunks: SegmentChunk[] = [];
|
|
223
|
+
let cursor = 0;
|
|
224
|
+
let confirmed = 0;
|
|
222
225
|
let done = false;
|
|
223
226
|
let notify: (() => void) | null = null;
|
|
224
227
|
return {
|
|
225
228
|
push(chunk) {
|
|
226
|
-
|
|
229
|
+
chunks.push(chunk);
|
|
227
230
|
notify?.();
|
|
228
231
|
},
|
|
229
232
|
end() {
|
|
230
233
|
done = true;
|
|
231
234
|
notify?.();
|
|
232
235
|
},
|
|
236
|
+
/** Everything ever pushed plus how much of it the consumer PROVED
|
|
237
|
+
* delivered. The Slack adapter pulls one chunk, awaits its Slack append,
|
|
238
|
+
* then pulls the next (verified in @chat-adapter/slack 4.34.0 stream()),
|
|
239
|
+
* so each pull confirms the append for the previous chunk landed; a
|
|
240
|
+
* consumer that dies mid-append unwinds the for-await with an implicit
|
|
241
|
+
* return() at the yield, leaving that chunk and everything after it
|
|
242
|
+
* unconfirmed. */
|
|
243
|
+
ledger() {
|
|
244
|
+
return { chunks: [...chunks], confirmed };
|
|
245
|
+
},
|
|
233
246
|
iterable: {
|
|
234
247
|
async *[Symbol.asyncIterator]() {
|
|
235
248
|
while (true) {
|
|
236
|
-
|
|
249
|
+
while (cursor < chunks.length) {
|
|
250
|
+
const next = chunks[cursor]!;
|
|
251
|
+
cursor += 1;
|
|
252
|
+
yield next;
|
|
253
|
+
confirmed = cursor;
|
|
254
|
+
}
|
|
237
255
|
if (done) return;
|
|
238
256
|
await new Promise<void>((resolve) => {
|
|
239
257
|
notify = resolve;
|
|
@@ -245,6 +263,23 @@ function pushableStream(): {
|
|
|
245
263
|
};
|
|
246
264
|
}
|
|
247
265
|
|
|
266
|
+
/** Slack rejects an over-long message with msg_too_long, and NOTHING in chat
|
|
267
|
+
* 4.34.0 or the Slack adapter bounds, truncates, or splits reply text
|
|
268
|
+
* (verified in-source 2026-07-20): a natively streamed message accumulates
|
|
269
|
+
* server-side toward the 12,000-char markdown_text envelope (docs.slack.dev
|
|
270
|
+
* documents that limit on chat.postMessage/update and all three streaming
|
|
271
|
+
* methods), the post-and-edit fallback re-sends the FULL accumulated text as
|
|
272
|
+
* markdown_text on every edit, and once anything rendered natively a failed
|
|
273
|
+
* append REJECTS the whole thread.post - the reply dies (live incident
|
|
274
|
+
* 2026-07-20, three failed turns). relayThread therefore splits reply text
|
|
275
|
+
* across Slack messages BEFORE the cap; the 2,000-char margin absorbs the
|
|
276
|
+
* renderer's markdown normalization and mention-linkification expansion.
|
|
277
|
+
* Tradeoff (accepted): the adapter-internal plain-text fallback edits via
|
|
278
|
+
* chat.update `text` (hard 4,000-char cap), but it only engages when the
|
|
279
|
+
* workspace refused native streaming outright - splitting every normal reply
|
|
280
|
+
* 3x tighter to cover that never-hit path is worse than the residual risk. */
|
|
281
|
+
export const SEGMENT_TEXT_MAX = 10_000;
|
|
282
|
+
|
|
248
283
|
/** Full permission name of the finish_thread tool (mcp__<server>__<tool>):
|
|
249
284
|
* it must be in allowedTools, because no one can answer a permission prompt
|
|
250
285
|
* through Slack. */
|
|
@@ -413,59 +448,262 @@ export async function relayThread(input: {
|
|
|
413
448
|
}): Promise<TurnOutcome> {
|
|
414
449
|
const outcome: TurnOutcome = { sessionId: input.sessionId, failed: false, rateLimited: false, finish: false, announcedDrop: false, resultReceived: false, deferUntil: null };
|
|
415
450
|
let segment: ReturnType<typeof pushableStream> | null = null;
|
|
416
|
-
|
|
451
|
+
// acc mirrors the segment's pushed text (bounded by SEGMENT_TEXT_MAX plus a
|
|
452
|
+
// small overshoot): fence parity must be computed over the ACCUMULATED text,
|
|
453
|
+
// never per chunk - SDK deltas do not respect markdown token boundaries, so
|
|
454
|
+
// a ``` split across two deltas would be invisible to per-chunk counting
|
|
455
|
+
// (pullfrog review catch, PR #42).
|
|
456
|
+
let segmentMeta: { text: boolean; acc: string; reopenFence: boolean } | null = null;
|
|
417
457
|
let lastPost: Promise<unknown> = Promise.resolve();
|
|
418
|
-
// Reply TEXT that
|
|
419
|
-
// by a later text-bearing segment: the
|
|
420
|
-
// card-only segment never sets this
|
|
421
|
-
// Tradeoff (flagged and accepted): a later
|
|
422
|
-
// flag even though it is a continuation,
|
|
423
|
-
//
|
|
424
|
-
//
|
|
425
|
-
//
|
|
426
|
-
//
|
|
458
|
+
// Reply TEXT that died with a rejected segment and was neither salvaged into
|
|
459
|
+
// a follow-on message nor re-delivered by a later text-bearing segment: the
|
|
460
|
+
// user has not seen the answer. A lost card-only segment never sets this
|
|
461
|
+
// (decoration, not the answer). Tradeoff (flagged and accepted): a later
|
|
462
|
+
// delivered text segment clears the flag even though it is a continuation,
|
|
463
|
+
// because a rejection whose text chunks were all consumed pre-append-failure
|
|
464
|
+
// was almost certainly delivered (the adapter appends per chunk) except for
|
|
465
|
+
// an unobservable renderer-held tail; sticky loss would fail every long turn
|
|
466
|
+
// with a spurious diagnostic.
|
|
427
467
|
let textLost = false;
|
|
428
468
|
let textLostDetail: string | null = null;
|
|
429
469
|
let postedText = false;
|
|
470
|
+
// Salvage: a rejected post's undelivered chunks re-post as a fresh message
|
|
471
|
+
// (Slack finalizes an idle stream after an UNDOCUMENTED window - verified
|
|
472
|
+
// absent from the chat.startStream/appendStream docs 2026-07-21 - so
|
|
473
|
+
// recovery is reactive on any append failure, never a keepalive tuned to a
|
|
474
|
+
// guessed constant). The budget bounds FUTILITY, not recovery: a death
|
|
475
|
+
// after the message delivered something is progress and refills it (a long
|
|
476
|
+
// turn can outlive any number of idle finalizations, each losing only the
|
|
477
|
+
// gap tail), while a surface that delivers nothing (revoked channel, hard
|
|
478
|
+
// cap on the very first append) burns a strike per attempt and stops.
|
|
479
|
+
const MAX_SEGMENT_SALVAGES = 5;
|
|
480
|
+
let salvagesLeft = MAX_SEGMENT_SALVAGES;
|
|
481
|
+
const openSegment = () => {
|
|
482
|
+
const seg = pushableStream();
|
|
483
|
+
segment = seg;
|
|
484
|
+
// reopenFence: the delivered prefix of a REJECTED predecessor left a code
|
|
485
|
+
// fence open, so the first TEXT entering this salvage segment must be
|
|
486
|
+
// preceded by a reopen or it renders outside the code block. Pending
|
|
487
|
+
// rather than pushed eagerly (pullfrog catches, PR #42): a card-only
|
|
488
|
+
// salvage would otherwise either skip the reopen (later text joining the
|
|
489
|
+
// segment renders unfenced) or dangle an empty open fence at message end
|
|
490
|
+
// when no text ever follows. Materialized by BOTH text entry points.
|
|
491
|
+
const meta = { text: false, acc: "", reopenFence: false };
|
|
492
|
+
segmentMeta = meta;
|
|
493
|
+
lastPost = input.post(seg.iterable).then(
|
|
494
|
+
() => {
|
|
495
|
+
salvagesLeft = MAX_SEGMENT_SALVAGES;
|
|
496
|
+
// segments settle in order (push awaits lastPost before opening the
|
|
497
|
+
// next), so delivered text supersedes an earlier loss.
|
|
498
|
+
if (meta.text) textLost = false;
|
|
499
|
+
},
|
|
500
|
+
(e: unknown) => {
|
|
501
|
+
const detail = (e instanceof Error ? e.message : String(e)).slice(0, 300);
|
|
502
|
+
log("serve.post_error", { err: detail });
|
|
503
|
+
// the consumer is gone (e.g. Slack finalized an idle stream:
|
|
504
|
+
// message_not_in_streaming_state). Salvage runs in TEXT space, not
|
|
505
|
+
// chunk space, because the adapter's renderer buffers across chunks
|
|
506
|
+
// (it holds back the trailing unterminated line, unconfirmed table
|
|
507
|
+
// headers, and unclosed inline markers until the post-iteration
|
|
508
|
+
// finish() flush - pullfrog catch on PR #45: a reply whose last line
|
|
509
|
+
// has no trailing newline dies entirely in that forced flush, with
|
|
510
|
+
// every chunk already consumed). Proven-delivered text = the mirror
|
|
511
|
+
// renderer's committable prefix over the CONFIRMED chunks (each pull
|
|
512
|
+
// proves the previous append landed; the adapter runs the renderer
|
|
513
|
+
// with wrapTablesForAppend: false, so committable is a raw prefix and
|
|
514
|
+
// the mirror is chunking-invariant). Renderer drift would break the
|
|
515
|
+
// prefix check and degrade to a full re-post: duplication, never
|
|
516
|
+
// loss. Same tradeoff for a stop()-failure after a complete flush:
|
|
517
|
+
// the held tail re-posts once rather than risking silent loss.
|
|
518
|
+
const { chunks, confirmed } = seg.ledger();
|
|
519
|
+
if (segment === seg) segment = null;
|
|
520
|
+
const confirmedRaw = chunks
|
|
521
|
+
.slice(0, confirmed)
|
|
522
|
+
.flatMap((c) => (c instanceof Object ? [] : [c]))
|
|
523
|
+
.join("");
|
|
524
|
+
const fullRaw = chunks.flatMap((c) => (c instanceof Object ? [] : [c])).join("");
|
|
525
|
+
const mirror = new StreamingMarkdownRenderer({ wrapTablesForAppend: false });
|
|
526
|
+
mirror.push(confirmedRaw);
|
|
527
|
+
const committed = mirror.getCommittableText();
|
|
528
|
+
const deliveredLen = confirmedRaw.startsWith(committed) ? committed.length : 0;
|
|
529
|
+
const textRemainder = fullRaw.slice(deliveredLen);
|
|
530
|
+
// A confirmed card's append landed (the adapter sends a card inline
|
|
531
|
+
// in the loop body before the next pull), so unconfirmed cards are
|
|
532
|
+
// the set it never appended.
|
|
533
|
+
const deliveredCard = chunks.slice(0, confirmed).some((c) => c instanceof Object);
|
|
534
|
+
// Delivery progress means this death does not count toward the
|
|
535
|
+
// futility budget: only a message that delivered nothing burns one.
|
|
536
|
+
// The key is ACTUAL delivery (committable text or a landed card),
|
|
537
|
+
// never chunk consumption: a reply whose final line has no trailing
|
|
538
|
+
// newline is consumed whole (confirmed advances) while its only
|
|
539
|
+
// append is the post-iteration forced flush, so a consumption key
|
|
540
|
+
// would refill the budget on every persistently failing flush and
|
|
541
|
+
// salvage the same held-back text forever (vercel review catch,
|
|
542
|
+
// PR #45). deliveredLen stays 0 there, the strike is spent, and the
|
|
543
|
+
// zero-delivery case terminates.
|
|
544
|
+
if (deliveredLen > 0 || deliveredCard) salvagesLeft = MAX_SEGMENT_SALVAGES;
|
|
545
|
+
// The salvage sequence preserves STREAM ORDER (cursor review catch,
|
|
546
|
+
// PR #45: re-posting all remainder text and then all cards showed
|
|
547
|
+
// task cards after prose that originally followed them): walk the
|
|
548
|
+
// ledger in order, keeping each text chunk's undelivered suffix and
|
|
549
|
+
// each unconfirmed card at its original position. Text splits at
|
|
550
|
+
// line boundaries (rendering is unchanged: chunks concatenate) so a
|
|
551
|
+
// salvage message that dies too still confirms per line, keeping
|
|
552
|
+
// progress attribution fine-grained.
|
|
553
|
+
const lost: SegmentChunk[] = [];
|
|
554
|
+
let offset = 0;
|
|
555
|
+
for (const [i, c] of chunks.entries()) {
|
|
556
|
+
if (c instanceof Object) {
|
|
557
|
+
if (i >= confirmed) lost.push(c);
|
|
558
|
+
continue;
|
|
559
|
+
}
|
|
560
|
+
const end = offset + c.length;
|
|
561
|
+
let rest = end > deliveredLen ? c.slice(Math.max(0, deliveredLen - offset)) : "";
|
|
562
|
+
offset = end;
|
|
563
|
+
while (rest !== "") {
|
|
564
|
+
const nl = rest.indexOf("\n");
|
|
565
|
+
if (nl === -1) {
|
|
566
|
+
lost.push(rest);
|
|
567
|
+
break;
|
|
568
|
+
}
|
|
569
|
+
lost.push(rest.slice(0, nl + 1));
|
|
570
|
+
rest = rest.slice(nl + 1);
|
|
571
|
+
}
|
|
572
|
+
}
|
|
573
|
+
// A fence OPENED in the delivered prefix leaves later code fenceless
|
|
574
|
+
// in the fresh salvage message (pullfrog catches, PR #42): arm the
|
|
575
|
+
// segment's pending reopen, materialized right before the FIRST text
|
|
576
|
+
// that enters it - whether a salvaged remainder piece here or a later
|
|
577
|
+
// streamed push joining the segment (a card-only salvage must not
|
|
578
|
+
// skip the reopen, and a text-less segment must not dangle one).
|
|
579
|
+
// Fold this dying segment's OWN still-armed reopen into the parity:
|
|
580
|
+
// an armed-but-never-materialized reopenFence means the segment
|
|
581
|
+
// logically BEGAN inside an open fence (a card-only salvage that
|
|
582
|
+
// inherited one and died before any text materialized it), so that
|
|
583
|
+
// open state must propagate to the next salvage or its later text
|
|
584
|
+
// renders unfenced (vercel + cubic chained-salvage catch, PR #42).
|
|
585
|
+
// Once materialized, reopenFence is false and the reopen chunk is in
|
|
586
|
+
// fullRaw, so the XOR is a no-op; a normal segment's flag is false.
|
|
587
|
+
const deliveredFenceOpen =
|
|
588
|
+
((fullRaw.slice(0, deliveredLen).split("```").length - 1) % 2 === 1) !== meta.reopenFence;
|
|
589
|
+
if (lost.length > 0 && salvagesLeft > 0) {
|
|
590
|
+
salvagesLeft -= 1;
|
|
591
|
+
log("serve.post_salvage", { chunks: lost.length, left: salvagesLeft });
|
|
592
|
+
// this handler runs synchronously as the post settles, so opening
|
|
593
|
+
// the salvage segment here keeps the salvaged content ordered ahead
|
|
594
|
+
// of any push still awaiting lastPost; the salvage segment's own
|
|
595
|
+
// settle then decides whether its text counts as delivered.
|
|
596
|
+
const next = openSegment();
|
|
597
|
+
next.meta.reopenFence = deliveredFenceOpen;
|
|
598
|
+
for (const c of lost) next.pushInto(c);
|
|
599
|
+
} else if (textRemainder !== "") {
|
|
600
|
+
textLost = true;
|
|
601
|
+
textLostDetail = detail;
|
|
602
|
+
}
|
|
603
|
+
},
|
|
604
|
+
);
|
|
605
|
+
const pushInto = (chunk: SegmentChunk) => {
|
|
606
|
+
// meta.text only, NEVER postedText: salvaged text was already counted
|
|
607
|
+
// at its original push, and postedText is attempt-scoped - a salvage
|
|
608
|
+
// landing after a retry reset would otherwise re-arm it and suppress
|
|
609
|
+
// the retry's `!postedText && result` fallback, silently dropping a
|
|
610
|
+
// result-only answer (adversarial-review catch on PR #45). acc still
|
|
611
|
+
// accumulates: it mirrors the segment's FULL text on every entry path,
|
|
612
|
+
// so pushText's room accounting and fence parity see salvaged text too
|
|
613
|
+
// (a salvage segment that continued via pushText could otherwise grow
|
|
614
|
+
// past the msg_too_long cap).
|
|
615
|
+
if (!(chunk instanceof Object)) {
|
|
616
|
+
if (meta.reopenFence) {
|
|
617
|
+
meta.reopenFence = false;
|
|
618
|
+
meta.acc += "```\n";
|
|
619
|
+
seg.push("```\n");
|
|
620
|
+
}
|
|
621
|
+
meta.text = true;
|
|
622
|
+
meta.acc += chunk;
|
|
623
|
+
}
|
|
624
|
+
seg.push(chunk);
|
|
625
|
+
};
|
|
626
|
+
return { seg, meta, pushInto };
|
|
627
|
+
};
|
|
430
628
|
const push = async (chunk: SegmentChunk) => {
|
|
431
|
-
|
|
432
|
-
if (!seg) {
|
|
629
|
+
if (segment === null) {
|
|
433
630
|
await lastPost; // strict message order: previous segment fully posted first
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
const meta = { text: false };
|
|
438
|
-
segmentMeta = meta;
|
|
439
|
-
lastPost = input.post(seg.iterable).then(
|
|
440
|
-
() => {
|
|
441
|
-
// segments settle in order (push awaits lastPost before opening the
|
|
442
|
-
// next), so delivered text supersedes an earlier loss.
|
|
443
|
-
if (meta.text) textLost = false;
|
|
444
|
-
},
|
|
445
|
-
(e: unknown) => {
|
|
446
|
-
const detail = (e instanceof Error ? e.message : String(e)).slice(0, 300);
|
|
447
|
-
log("serve.post_error", { err: detail });
|
|
448
|
-
if (meta.text) {
|
|
449
|
-
textLost = true;
|
|
450
|
-
textLostDetail = detail;
|
|
451
|
-
}
|
|
452
|
-
// the consumer is gone (e.g. Slack finalized an idle stream:
|
|
453
|
-
// message_not_in_streaming_state) - drop the dead segment so the
|
|
454
|
-
// next chunk opens a fresh message instead of vanishing into it.
|
|
455
|
-
if (segment === posted) segment = null;
|
|
456
|
-
},
|
|
457
|
-
);
|
|
631
|
+
// a rejection handler may have opened a salvage segment during the wait;
|
|
632
|
+
// joining it instead of opening another keeps its post from being
|
|
633
|
+
// orphaned un-ended.
|
|
458
634
|
}
|
|
635
|
+
const target = segment ?? openSegment().seg;
|
|
459
636
|
if (!(chunk instanceof Object)) {
|
|
637
|
+
// materialize a salvage segment's pending fence reopen (see
|
|
638
|
+
// openSegment) before the first text from this entry point too.
|
|
639
|
+
if (segmentMeta!.reopenFence) {
|
|
640
|
+
segmentMeta!.reopenFence = false;
|
|
641
|
+
segmentMeta!.acc += "```\n";
|
|
642
|
+
target.push("```\n");
|
|
643
|
+
}
|
|
460
644
|
postedText = true;
|
|
461
645
|
segmentMeta!.text = true;
|
|
646
|
+
segmentMeta!.acc += chunk;
|
|
462
647
|
}
|
|
463
|
-
|
|
648
|
+
target.push(chunk);
|
|
464
649
|
};
|
|
650
|
+
// parity by occurrence count over the segment's accumulated text: an odd
|
|
651
|
+
// number of ``` markers means the segment currently sits inside a fence.
|
|
652
|
+
const fenceOpen = () => segmentMeta !== null && (segmentMeta.acc.split("```").length - 1) % 2 === 1;
|
|
465
653
|
const breakSegment = () => {
|
|
466
654
|
segment?.end();
|
|
467
655
|
segment = null;
|
|
468
656
|
};
|
|
657
|
+
/** Reply text routed through here splits across Slack messages before the
|
|
658
|
+
* msg_too_long cap (see SEGMENT_TEXT_MAX): a break prefers the last newline
|
|
659
|
+
* inside the remaining room, and a break forced inside a code fence closes
|
|
660
|
+
* it and reopens it in the next message so both halves render as code. */
|
|
661
|
+
const pushText = async (text: string) => {
|
|
662
|
+
for (let rest = text; rest !== "";) {
|
|
663
|
+
const room = SEGMENT_TEXT_MAX - (segment === null ? 0 : segmentMeta!.acc.length);
|
|
664
|
+
// a fence-close suffix can nudge a segment a few chars past the cap;
|
|
665
|
+
// a full segment just breaks and the loop re-measures a fresh one.
|
|
666
|
+
if (room <= 0) {
|
|
667
|
+
breakSegment();
|
|
668
|
+
continue;
|
|
669
|
+
}
|
|
670
|
+
if (rest.length <= room) {
|
|
671
|
+
await push(rest);
|
|
672
|
+
return;
|
|
673
|
+
}
|
|
674
|
+
// prefer a newline cut only when it lands in the back half of the room:
|
|
675
|
+
// an early newline followed by one giant unbroken run would otherwise
|
|
676
|
+
// make no progress and (with the fence-reopen prefix) loop forever.
|
|
677
|
+
const nl = rest.lastIndexOf("\n", room - 1);
|
|
678
|
+
let cut = nl >= Math.floor(room / 2) ? nl + 1 : room;
|
|
679
|
+
// never slice through a backtick run: a cut inside ``` would strand a
|
|
680
|
+
// partial delimiter on each side and break both halves' rendering
|
|
681
|
+
// (cubic review catch, PR #42). Walk the cut left past the run; a run
|
|
682
|
+
// reaching position 0 keeps the original cut (progress beats rendering
|
|
683
|
+
// for pathological all-backtick input).
|
|
684
|
+
if (rest[cut - 1] === "`" && rest[cut] === "`") {
|
|
685
|
+
let backedUp = cut;
|
|
686
|
+
while (backedUp > 0 && rest[backedUp - 1] === "`") backedUp -= 1;
|
|
687
|
+
if (backedUp > 0) cut = backedUp;
|
|
688
|
+
}
|
|
689
|
+
const head = rest.slice(0, cut);
|
|
690
|
+
await push(head);
|
|
691
|
+
const reopen = fenceOpen();
|
|
692
|
+
if (reopen) await push("\n```");
|
|
693
|
+
breakSegment();
|
|
694
|
+
rest = (reopen ? "```\n" : "") + rest.slice(head.length);
|
|
695
|
+
}
|
|
696
|
+
};
|
|
697
|
+
// settle the whole post chain: a rejection handler may replace lastPost with
|
|
698
|
+
// a salvage segment's post, which still needs ending and settling.
|
|
699
|
+
const settlePosts = async () => {
|
|
700
|
+
while (true) {
|
|
701
|
+
breakSegment();
|
|
702
|
+
const settled = lastPost;
|
|
703
|
+
await settled;
|
|
704
|
+
if (lastPost === settled) return;
|
|
705
|
+
}
|
|
706
|
+
};
|
|
469
707
|
// a recovery status line reads as its own Slack message, not part of a
|
|
470
708
|
// streamed segment.
|
|
471
709
|
const notify = async (text: string) => {
|
|
@@ -487,7 +725,7 @@ export async function relayThread(input: {
|
|
|
487
725
|
* its own delivery resets textLost. */
|
|
488
726
|
const notifyDelivered = async (text: string) => {
|
|
489
727
|
await notify(text);
|
|
490
|
-
await
|
|
728
|
+
await settlePosts();
|
|
491
729
|
return !textLost;
|
|
492
730
|
};
|
|
493
731
|
// false when the daemon started draining mid-sleep.
|
|
@@ -612,10 +850,13 @@ export async function relayThread(input: {
|
|
|
612
850
|
outcome.resultReceived = true;
|
|
613
851
|
}
|
|
614
852
|
}
|
|
615
|
-
for (const part of agentEventChunks({ state: mapState, message }))
|
|
853
|
+
for (const part of agentEventChunks({ state: mapState, message })) {
|
|
854
|
+
if (part instanceof Object) await push(part);
|
|
855
|
+
else await pushText(part);
|
|
856
|
+
}
|
|
616
857
|
}
|
|
617
858
|
// a turn that produced no streamed text (tool-only turns) still reports.
|
|
618
|
-
if (!postedText && result) await
|
|
859
|
+
if (!postedText && result) await pushText(result);
|
|
619
860
|
if (!postedText && !result && outcome.failed && !outcome.rateLimited) {
|
|
620
861
|
// held back until the depleted-pool probe rules: a "trying again may
|
|
621
862
|
// help" line right before a deferral notice invites a manual re-send
|
|
@@ -768,12 +1009,12 @@ export async function relayThread(input: {
|
|
|
768
1009
|
break;
|
|
769
1010
|
}
|
|
770
1011
|
}
|
|
771
|
-
|
|
772
|
-
|
|
773
|
-
//
|
|
774
|
-
// the answer silently vanished while the outcome would
|
|
775
|
-
// the turn and make one best-effort fresh-message
|
|
776
|
-
//
|
|
1012
|
+
await settlePosts();
|
|
1013
|
+
// Reply text died with a rejected segment, salvage could not re-deliver it
|
|
1014
|
+
// (budget exhausted or the salvage posts died too), and nothing later
|
|
1015
|
+
// re-delivered it: the answer silently vanished while the outcome would
|
|
1016
|
+
// report success. Fail the turn and make one best-effort fresh-message
|
|
1017
|
+
// diagnostic.
|
|
777
1018
|
if (textLost) {
|
|
778
1019
|
outcome.failed = true;
|
|
779
1020
|
const detail = textLostDetail ?? "unknown error";
|