@sayknow-cli/coding-agent 0.7.6 → 0.7.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -50,7 +50,7 @@ export const EMBEDDED_DOCS: Readonly<Record<string, string>> = {
50
50
  "natives-rust-task-cancellation.md": "# Native Rust task execution and cancellation (`pi-natives`)\n\nThis document describes how `crates/pi-natives` schedules native work and how cancellation flows from JS options (`timeoutMs`, `AbortSignal`) into Rust execution.\n\n## Implementation files\n\n- `crates/pi-natives/src/task.rs`\n- `crates/pi-natives/src/grep.rs`\n- `crates/pi-natives/src/glob.rs`\n- `crates/pi-natives/src/fd.rs`\n- `crates/pi-natives/src/ast.rs`\n- `crates/pi-natives/src/shell.rs`\n- `crates/pi-natives/src/pty.rs`\n- `crates/pi-natives/src/html.rs`\n- `crates/pi-natives/src/image.rs`\n- `crates/pi-natives/src/clipboard.rs`\n- `crates/pi-natives/src/text.rs`\n- `crates/pi-natives/src/ps.rs`\n\n## Core primitives (`task.rs`)\n\n`task.rs` defines:\n\n1. `task::blocking(tag, cancel_token, work)`\n - Wraps `napi::AsyncTask` / `Task`.\n - `compute()` runs on libuv worker threads.\n - Returns a JS `Promise<T>` for exported functions.\n - Records a profiling sample through `profile_region(tag)`.\n\n2. `task::future(env, tag, work)`\n - Wraps `env.spawn_future(...)`.\n - Runs async work on Tokio's runtime.\n - Returns `PromiseRaw<'env, T>`.\n - Records a profiling sample through `profile_region(tag)`.\n\n3. `CancelToken` / `AbortToken` / `AbortReason`\n - `CancelToken::new(timeout_ms, signal)` combines an optional deadline and optional JS `AbortSignal` converted from `Unknown`.\n - `CancelToken::heartbeat()` is cooperative cancellation for blocking loops.\n - `CancelToken::wait()` asynchronously waits for signal, timeout, or Ctrl-C.\n - `CancelToken::emplace_abort_token()` creates an abortable flag when a later `Shell.abort()`/internal bridge needs one.\n - `AbortToken::abort(reason)` lets external code request abort.\n\n## `blocking` vs `future`: execution model and selection\n\n### Use `task::blocking`\n\nUse when work is CPU-heavy or fundamentally synchronous/blocking:\n\n- regex/file scanning (`grep`, `glob`, `fuzzyFind`)\n- ast-grep search/edit worker work\n- PTY loop internals through `tokio::task::spawn_blocking`\n- image decode/resize/encode\n- HTML conversion\n- clipboard image read\n\nBehavior:\n\n- Work closure receives a cloned `CancelToken`.\n- Cancellation is only observed where code checks `ct.heartbeat()?`.\n- Closure `Err(...)` rejects the JS promise.\n\n### Use `task::future`\n\nUse when work must `await` async operations:\n\n- shell session orchestration (`Shell.run`, `executeShell`)\n- PTY outer promise (`PtySession.start`) before it enters `spawn_blocking`\n- task racing (`tokio::select!`) between completion and cancellation\n\nBehavior:\n\n- Future code can race normal completion against `ct.wait()`.\n- On cancel path, async implementations typically cancel subordinate machinery and may force-abort after a grace timeout.\n\n## JS API ↔ Rust export mapping (task/cancel relevant)\n\n| JS-facing API | Rust export | Scheduler | Cancellation hookup |\n| --------------------------------------- | ------------------------------------ | -------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |\n| `grep(options, onMatch?)` | `grep` | `task::blocking(\"grep\", ct, ...)` | `CancelToken::new(options.timeoutMs, options.signal)` + heartbeat checks |\n| `glob(options, onMatch?)` | `glob` | `task::blocking(\"glob\", ct, ...)` | `CancelToken::new(...)` + heartbeat checks |\n| `fuzzyFind(options)` | `fuzzy_find` | `task::blocking(\"fuzzy_find\", ct, ...)` | `CancelToken::new(...)` + heartbeat checks |\n| `astGrep(options)` / `astEdit(options)` | ast exports | blocking worker path | timeout/signal fields are accepted by options and checked cooperatively in worker loops |\n| `Shell#run(options, onChunk?)` | `Shell::run` | `task::future(env, \"shell.run\", ...)` | `ct.wait()` raced against run task; bridges to Tokio cancellation token and `AbortToken` |\n| `executeShell(options, onChunk?)` | `execute_shell` | `task::future(env, \"shell.execute\", ...)` | same cancel race and 2s graceful window |\n| `PtySession#start(options, onChunk?)` | `PtySession::start` | `task::future(env, \"pty.start\", ...)` + inner `spawn_blocking` | `CancelToken` checked in sync PTY loop via `heartbeat()` |\n| `htmlToMarkdown(html, options?)` | `html_to_markdown` | `task::blocking(\"html_to_markdown\", (), ...)` | none (`()` token) |\n| `PhotonImage.parse/encode/resize` | `PhotonImage::{parse,encode,resize}` | `task::blocking(...)` | none (`()` token) |\n| `readImageFromClipboard()` | `read_image_from_clipboard` | `task::blocking(\"clipboard.read_image\", (), ...)` | none (`()` token) |\n\n`text.rs`, `tokens.rs`, `keys.rs`, most `ps.rs` functions, and synchronous utility exports do not use `task::blocking`/`task::future` and therefore do not participate in this cancellation path.\n\n## Cancellation lifecycle and state transitions\n\n### `CancelToken` lifecycle\n\n```text\nCreated\n ├─ no signal + no timeout -> passive token\n ├─ signal registered -> AbortSignal callback can set AbortReason::Signal\n └─ deadline set -> timeout check becomes active\n\nRunning\n ├─ heartbeat()/wait() sees signal -> AbortReason::Signal\n ├─ heartbeat()/wait() sees deadline -> AbortReason::Timeout\n ├─ wait() sees Ctrl-C -> AbortReason::User\n └─ no abort -> continue\n\nAborted\n └─ flag stores first observed cause for waiters; heartbeat formats it as \"Aborted: <reason>\"\n```\n\n### Before-start vs mid-execution cancellation\n\n- **Before start / before first cancellation check**:\n - `task::future` users that race on `ct.wait()` can resolve cancellation once they enter `select!`.\n - `task::blocking` users only observe cancellation when closure code reaches `heartbeat()`.\n\n- **Mid-execution**:\n - `blocking`: next `heartbeat()` returns `Err(\"Aborted: ...\")`.\n - `future`: `ct.wait()` branch wins `select!`, then code cancels subordinate async machinery.\n - shell: cancellation triggers a Tokio cancellation token, waits up to 2 seconds, then aborts the task if needed.\n - PTY: heartbeat failure or `kill()` terminates PTY child/process tree and drains output briefly.\n\n## Heartbeat expectations for long-running loops\n\n`heartbeat()` must run at predictable cadence in loops with unbounded or large work sets.\n\nObserved patterns:\n\n- `glob` filtering checks entries during scan/filter work.\n- `fd` scoring checks scanned candidates.\n- `grep` checks before/during expensive search and passes tokens into shared scan/cache helpers.\n- `run_pty_sync` checks every loop tick with a maximum 16ms wait cadence.\n\nPractical rule: no loop over external-size input should exceed a short bounded interval without a heartbeat.\n\n## Failure behavior and error propagation to JS\n\n### Blocking tasks\n\nError path:\n\n1. Closure returns `Err(napi::Error)` (including `heartbeat()` abort).\n2. `Task::compute()` returns `Err`.\n3. `AsyncTask` rejects JS promise.\n\nTypical error strings:\n\n- `Aborted: Timeout`\n- `Aborted: Signal`\n- domain errors (`Failed to decode image: ...`, `Conversion error: ...`, etc.)\n\n### Future tasks\n\nError path:\n\n1. Async body returns `Err(napi::Error)` or join failure is mapped (`... task failed: {err}`).\n2. `task::future`-spawned promise rejects.\n3. Shell and PTY command APIs model cancellation as structured results instead of rejection when the cancellation path wins: `exitCode` omitted, `cancelled` or `timedOut` set.\n\n### Cancellation reporting split\n\n- **Abort as error**: blocking exports using `heartbeat()?`.\n- **Abort as typed result**: shell/PTY command APIs that model cancellation in result structs.\n\nChoose one model per API and document it explicitly.\n\n## Common pitfalls\n\n1. **Missing heartbeat in blocking loops**\n - Symptom: timeout/signal appears ignored until loop ends.\n - Fix: add `ct.heartbeat()?` at loop top and before expensive per-item steps.\n\n2. **Long uncancelable sections**\n - Symptom: cancellation latency spikes during single large call (decode, sort, compression, parser invocation, etc.).\n - Fix: split work into chunks with heartbeat boundaries; if impossible, document latency.\n\n3. **Blocking async executor**\n - Symptom: async API stalls when sync-heavy code runs directly in future.\n - Fix: move CPU/sync blocks to `task::blocking` or `tokio::task::spawn_blocking`.\n\n4. **Inconsistent cancel semantics**\n - Symptom: one API rejects on cancel, another resolves with flags, confusing callers.\n - Fix: standardize per domain and keep docs aligned.\n\n5. **Forgetting cancellation bridge in nested async tasks**\n - Symptom: outer token is cancelled but inner readers/subprocess tasks keep running.\n - Fix: bridge cancellation to inner token/signal and enforce grace timeout + forced abort fallback.\n\n## Checklist for new cancellable exports\n\n1. Classify work correctly:\n - CPU-bound or sync blocking -> `task::blocking`.\n - async I/O / `await` orchestration -> `task::future`.\n\n2. Expose cancel inputs when needed:\n - include `timeoutMs` and `signal` in `#[napi(object)]` options,\n - create `let ct = task::CancelToken::new(timeout_ms, signal);`.\n\n3. Wire cancellation through all layers:\n - blocking loops: `ct.heartbeat()?` at stable intervals,\n - async orchestration: race with `ct.wait()` and cancel sub-tasks/tokens.\n\n4. Decide cancellation contract:\n - reject promise with abort error, or\n - resolve typed `{ cancelled, timedOut, ... }`,\n - keep this contract consistent for the API family.\n\n5. Propagate failures with context:\n - map errors via `Error::from_reason(format!(\"...: {err}\"))`,\n - include stage-specific prefixes (`spawn`, `decode`, `wait`, etc.).\n\n6. Handle before-start and mid-flight cancellation:\n - cancellation check/await must happen before expensive body and during long execution.\n\n7. Validate no executor misuse:\n - no long sync work directly inside async futures without `spawn_blocking`/blocking task wrapper.\n",
51
51
  "natives-shell-pty-process.md": "# Natives Shell, PTY, Process, and Key Internals\n\nThis document covers the execution/process/terminal primitives in `@sayknow-cli/natives`: `shell`, `pty`, `ps`, and `keys`, using the architecture terms from `docs/natives-architecture.md`.\n\n## Implementation files\n\n- `crates/pi-natives/src/shell.rs`\n- `crates/pi-natives/src/shell/windows.rs` (Windows-only PATH enrichment)\n- `crates/pi-natives/src/pty.rs`\n- `crates/pi-natives/src/ps.rs`\n- `crates/pi-natives/src/keys.rs`\n- `crates/pi-natives/src/task.rs`\n- `packages/natives/native/index.d.ts`\n\n## Layer ownership\n\n- **Package entrypoint** (`packages/natives/native/index.js`): loads the `.node` addon and exports generated N-API bindings.\n- **Rust N-API module layer** (`crates/pi-natives/src/*`): shell/PTY process execution, process-tree traversal/termination, and key-sequence parsing.\n- **Consumers** (`packages/coding-agent`, `packages/tui`): higher-level session policy, output artifact/minimizer handling, render policy, and UI key handling.\n\n## Shell subsystem (`shell`)\n\n### API model\n\nTwo execution modes are exposed:\n\n1. **One-shot** via `executeShell(options, onChunk?)`.\n2. **Persistent session** via `new Shell(options?)` then `shell.run(...)` repeatedly.\n\nBoth stream output through a threadsafe callback and return `{ exitCode?, cancelled, timedOut, minimized? }`.\n\n`ShellOptions` supports `sessionEnv`, `snapshotPath`, and optional output `minimizer`. `ShellExecuteOptions` supports command-scoped `env`, session-level `sessionEnv`, `snapshotPath`, timeout/signal, and optional minimizer. `ShellRunOptions` supports command, cwd, command-scoped env, timeout, and signal.\n\n### Session creation and environment model\n\nRust creates `brush_core::Shell` with:\n\n- non-interactive, non-login mode,\n- `no_profile` and `no_rc`,\n- `do_not_inherit_env: true`,\n- bash-mode builtins, with `exec` and `suspend` disabled,\n- explicit environment reconstruction from host env,\n- skip-list for shell-sensitive vars (`PS1`, `PWD`, `SHLVL`, bash function exports, etc.).\n\nSession env behavior:\n\n- `ShellOptions.sessionEnv` / one-shot `sessionEnv` is applied at session creation.\n- `ShellRunOptions.env` / one-shot `env` is command-scoped (`EnvironmentScope::Command`) and popped after the command.\n- `PATH` is merged specially on Windows with case-insensitive dedupe.\n- Windows-only path enrichment (`shell/windows.rs`) appends discovered Git-for-Windows paths when present and not already included.\n- `snapshotPath`, when present, is sourced during session creation with stdout/stderr/stdin wired to null files.\n\n### Runtime lifecycle and state transitions\n\nPersistent shell (`Shell.run`) uses this state machine:\n\n- **Idle/Uninitialized**: `session: None`.\n- **Running**: first `run()` lazily creates a session, stores an abort token, executes command.\n- **Completed + keepalive**: if execution control flow is normal, abort state is cleared and session is reused.\n- **Completed + teardown**: if control flow is loop/script/shell-exit related, session is dropped.\n- **Cancelled/Timed out**: run task is cancelled, grace wait is 2 seconds, task may be force-aborted, session is dropped if lock can be acquired.\n- **Error**: session is dropped.\n\nOne-shot shell (`executeShell`) always creates and drops a fresh session per call.\n\n### Streaming/output and minimizer behavior\n\n- Stdout/stderr are routed into a shared pipe and read concurrently.\n- Reader decodes UTF-8 incrementally; invalid byte sequences emit `U+FFFD` replacement chunks.\n- The command runs in a new process group policy.\n- Optional minimizer configuration can capture and rewrite output. When minimization occurs, the result includes `minimized` with filter name, replacement text, original text, and byte counts.\n- Consumers are responsible for persisting or displaying minimizer artifacts; the native result only carries the data.\n\n### Cancellation, timeout, and abort\n\n- `CancelToken` is constructed from `timeoutMs` and optional `AbortSignal`.\n- On cancellation/timeout, shell cancellation token is triggered, then task gets a 2-second graceful window before forced abort.\n- Structured result flags are used:\n - timeout -> `exitCode` omitted, `timedOut: true`.\n - abort signal / `Shell.abort()` -> `exitCode` omitted, `cancelled: true`.\n\n`Shell.abort()` behavior:\n\n- aborts the current running command for that `Shell` instance through the stored `AbortToken`,\n- resolves successfully even when nothing is running.\n\n### Failure behavior\n\nCommon surfaced errors include:\n\n- session init failures (`Failed to initialize shell`),\n- cwd errors (`Failed to set cwd`),\n- env set/pop failures,\n- snapshot source failures (`Failed to source snapshot`),\n- pipe creation/clone failures,\n- execution failure (`Shell execution failed: ...`),\n- task wrapper failures (`Shell execution task failed: ...`).\n\n## PTY subsystem (`pty`)\n\n### API model\n\n`new PtySession()` exposes:\n\n- `start(options, onChunk?) -> Promise<{ exitCode?, cancelled, timedOut }>`\n- `write(data)`\n- `resize(cols, rows)`\n- `kill()`\n\n`PtyStartOptions` supports `command`, optional `cwd`, optional `env`, `timeoutMs`, `signal`, `cols`, and `rows`.\n\n### Runtime lifecycle and state transitions\n\n`PtySession` state machine:\n\n- **Idle**: `core: None`.\n- **Reserved**: `start()` installs control channel synchronously (`core: Some`) before async work begins, so `write/resize/kill` become immediately valid.\n- **Running**: blocking PTY loop handles child state, reader events, cancellation heartbeat, and control messages.\n- **Terminal closed / drain**: child exit or cancellation starts a short reader drain window.\n- **Finalized**: `core` is always reset to `None` after start task completion (success or error).\n\nConcurrency guard:\n\n- starting while already running returns `PTY session already running`.\n\n### Spawn/attach/write/read/terminate patterns\n\n- PTY opened via `portable_pty::native_pty_system().openpty(...)`.\n- Command currently runs as `sh -lc <command>` with optional `cwd` and env overrides.\n- Default size is `120x40`; dimensions are clamped (`cols 20..400`, `rows 5..200`).\n- `write()` sends raw bytes to PTY stdin.\n- `resize()` sends a control message and clamps dimensions again.\n- `kill()` sends a control message that marks the run cancelled and terminates the child/process tree.\n\nOutput path:\n\n- dedicated reader thread reads master stream,\n- incremental UTF-8 decode emits `U+FFFD` for invalid bytes,\n- chunks forwarded through N-API threadsafe callback.\n\nTermination path:\n\n- Unix: terminate process group when known, terminate child tree, call child kill, then repeat with SIGKILL.\n- Non-Unix: terminate child tree, call child kill, then repeat with SIGKILL-equivalent process-tree helper.\n\n### Cancellation and timeout semantics\n\n- `timeoutMs` and `AbortSignal` feed a `CancelToken`.\n- Loop calls `ct.heartbeat()` periodically with a 16ms maximum wait cadence.\n- Timeout classification is based on the heartbeat error string containing `Timeout`.\n- Cancellation/kill starts a 300ms post-cancel drain window; normal child exit starts a 300ms post-exit drain window.\n\n### Failure behavior\n\nError surfaces include:\n\n- PTY allocation/open failure,\n- PTY spawn failure,\n- writer/reader acquisition failure,\n- child status/wait failures,\n- lock poisoning,\n- control-channel disconnection (`PTY session is no longer available`).\n\nControl call failures when not running:\n\n- `write/resize/kill` return `PTY session is not running`.\n\n## Process-tree subsystem (`ps`)\n\n### API model\n\n- `killTree(pid, signal) -> number`\n- `listDescendants(pid) -> number[]`\n\n### Platform-specific implementation\n\n- **Linux**: recursively reads `/proc/<pid>/task/<pid>/children`.\n- **macOS**: uses `libproc` `proc_listchildpids`.\n- **Windows**: snapshots process table with `CreateToolhelp32Snapshot`, builds parent->children map, terminates with `OpenProcess(PROCESS_TERMINATE)` + `TerminateProcess`.\n\n### Kill-tree behavior\n\n- Descendants are collected recursively.\n- Kill order is bottom-up (deepest descendants first).\n- Root pid is killed last.\n- Return value is count of successful terminations.\n\nSignal behavior:\n\n- POSIX: provided `signal` is passed to `kill`.\n- Windows: `signal` is ignored; termination is unconditional process terminate.\n\n### Failure behavior\n\nThis module is intentionally non-throwing at API surface for ordinary process misses:\n\n- missing/inaccessible process tree branches are skipped,\n- per-pid kill failures are counted as unsuccessful,\n- lookup miss typically yields `[]` from `listDescendants` and `0` from `killTree`.\n\n## Key parsing subsystem (`keys`)\n\n### API model\n\nExposed helpers:\n\n- `parseKey(data, kittyProtocolActive)`\n- `matchesKey(data, keyId, kittyProtocolActive)`\n- `parseKittySequence(data)`\n- `matchesKittySequence(data, expectedCodepoint, expectedModifier)`\n- `matchesLegacySequence(data, keyName)`\n\n### Parsing model\n\nThe parser combines:\n\n- direct single-byte mappings (`enter`, `tab`, `ctrl+<letter>`, printable ASCII),\n- O(1) legacy escape-sequence lookup (PHF map),\n- xterm `modifyOtherKeys` parsing,\n- Kitty protocol parsing (`CSI u`, `CSI ~`, `CSI 1;...<letter>`),\n- normalization to key IDs (`ctrl+c`, `shift+tab`, `pageUp`, `f5`, etc.).\n\nModifier handling:\n\n- only shift/alt/ctrl bits are compared for key matching,\n- lock bits are masked out before comparisons.\n\nLayout behavior:\n\n- base-layout fallback is intentionally constrained so remapped layouts do not create false matches for ASCII letters/symbols.\n\n### Failure behavior\n\n- Unrecognized or invalid sequences produce `null` from parse functions.\n- Match functions return `false` on parse failure or mismatch.\n- No thrown error surface for malformed key input.\n\n## JS API ↔ Rust export mapping\n\n### Shell + PTY + Process\n\n| JS API | Rust N-API export | Notes |\n| --------------------------------- | -------------------------------------- | ----------------------------------------- |\n| `executeShell(options, onChunk?)` | `executeShell` (`execute_shell`) | One-shot shell execution |\n| `new Shell(options?)` | `Shell` class | Persistent shell session |\n| `shell.run(options, onChunk?)` | `Shell::run` | Reuses session on keepalive control flow |\n| `shell.abort()` | `Shell::abort` | Aborts active run for that shell instance |\n| `new PtySession()` | `PtySession` class | Stateful PTY session |\n| `pty.start(options, onChunk?)` | `PtySession::start` | Interactive PTY run |\n| `pty.write(data)` | `PtySession::write` | Raw stdin passthrough |\n| `pty.resize(cols, rows)` | `PtySession::resize` | Clamped terminal dimensions |\n| `pty.kill()` | `PtySession::kill` | Force-kills active PTY child |\n| `killTree(pid, signal)` | `killTree` (`kill_tree`) | Children-first process tree termination |\n| `listDescendants(pid)` | `listDescendants` (`list_descendants`) | Recursive descendants listing |\n\n### Keys\n\n| JS API | Rust N-API export | Notes |\n| ---------------------------------------------- | --------------------------------------------------- | ------------------------------- |\n| `matchesKittySequence(data, cp, mod)` | `matchesKittySequence` (`matches_kitty_sequence`) | Kitty codepoint+modifier match |\n| `parseKey(data, kittyProtocolActive)` | `parseKey` (`parse_key`) | Normalized key-id parser |\n| `matchesLegacySequence(data, keyName)` | `matchesLegacySequence` (`matches_legacy_sequence`) | Exact legacy sequence map check |\n| `parseKittySequence(data)` | `parseKittySequence` (`parse_kitty_sequence`) | Structured Kitty parse result |\n| `matchesKey(data, keyId, kittyProtocolActive)` | `matchesKey` (`matches_key`) | High-level key matcher |\n\n## Abandoned session cleanup and finalization notes\n\n- **Shell persistent session**: if a run is cancelled/timed out/errors/non-keepalive control flow, Rust drops the internal session state. Successful normal runs keep the session for reuse.\n- **PTY session**: `core` is always cleared after `start()` finishes, including failure paths.\n- **No explicit JS finalizer-driven kill contract** is exposed by wrappers; cleanup is primarily tied to run completion/cancellation paths. Callers should use `timeoutMs`, `AbortSignal`, `shell.abort()`, or `pty.kill()` for deterministic teardown.\n",
52
52
  "natives-text-search-pipeline.md": "# Natives Text/Search Pipeline\n\nThis document maps the `@sayknow-cli/natives` text/search/code surface from generated JS/TS exports to Rust N-API modules and back to JS result objects.\n\nTerminology follows `docs/natives-architecture.md`:\n\n- **Generated binding**: public API in `packages/natives/native/index.d.ts`.\n- **Rust module layer**: N-API exports in `crates/pi-natives/src/*`.\n- **Shared scan cache**: `fs_cache`-backed directory-entry cache used by discovery/search flows.\n\n## Implementation files\n\n- `packages/natives/native/index.d.ts`\n- `crates/pi-natives/src/grep.rs`\n- `crates/pi-natives/src/glob.rs`\n- `crates/pi-natives/src/glob_util.rs`\n- `crates/pi-natives/src/fs_cache.rs`\n- `crates/pi-natives/src/fd.rs`\n- `crates/pi-natives/src/ast.rs`\n- `crates/pi-natives/src/text.rs`\n- `crates/pi-natives/src/highlight.rs`\n- `crates/pi-natives/src/tokens.rs`\n\n## JS API ↔ Rust export mapping\n\n| JS API | Rust export (`#[napi]`, snake_case -> camelCase) | Rust module |\n| ------------------------------------------------------------------------------- | ------------------------------------------------ | -------------- |\n| `grep(options, onMatch?)` | `grep` | `grep.rs` |\n| `search(content, options)` | `search` | `grep.rs` |\n| `hasMatch(content, pattern, ignoreCase?, multiline?)` | `hasMatch` | `grep.rs` |\n| `fuzzyFind(options)` | `fuzzyFind` | `fd.rs` |\n| `glob(options, onMatch?)` | `glob` | `glob.rs` |\n| `invalidateFsScanCache(path?)` | `invalidateFsScanCache` | `fs_cache.rs` |\n| `astGrep(options)` | `astGrep` | `ast.rs` |\n| `astEdit(options)` | `astEdit` | `ast.rs` |\n| `wrapTextWithAnsi(text, width, tabWidth)` | `wrapTextWithAnsi` | `text.rs` |\n| `truncateToWidth(text, maxWidth, ellipsis, pad, tabWidth)` | `truncateToWidth` | `text.rs` |\n| `sliceWithWidth(line, startCol, length, strict, tabWidth)` | `sliceWithWidth` | `text.rs` |\n| `extractSegments(line, beforeEnd, afterStart, afterLen, strictAfter, tabWidth)` | `extractSegments` | `text.rs` |\n| `visibleWidth(text, tabWidth)` | `visibleWidth` | `text.rs` |\n| `highlightCode(code, lang, colors)` | `highlightCode` | `highlight.rs` |\n| `supportsLanguage(lang)` | `supportsLanguage` | `highlight.rs` |\n| `getSupportedLanguages()` | `getSupportedLanguages` | `highlight.rs` |\n| `countTokens(input, encoding?)` | `countTokens` | `tokens.rs` |\n\n## Pipeline overview by subsystem\n\n## 1) Regex search (`grep`, `search`, `hasMatch`)\n\n### Input/options flow\n\n1. Callers invoke generated native exports directly; there is no package-local TS wrapper that renames `search` to `searchContent`.\n2. Rust option structs in `grep.rs` deserialize camelCase fields (`ignoreCase`, `maxCount`, `contextBefore`, `contextAfter`, `maxColumns`, `timeoutMs`).\n3. `grep` creates `CancelToken` from `timeoutMs` + `AbortSignal` and runs inside `task::blocking(\"grep\", ...)`.\n4. `search` and `hasMatch` operate on provided string/`Uint8Array` content and do not scan the filesystem.\n\n### Execution branches\n\n- **In-memory branch**\n - `search` -> `search_sync` / search helpers over provided content bytes.\n - `hasMatch` compiles/checks pattern against provided content and returns a boolean.\n - No filesystem scan, no `fs_cache`.\n- **Single-file branch**\n - `grep` resolves path, checks metadata is file, and searches that file.\n- **Directory branch**\n - Optional cache lookup via `fs_cache::get_or_scan` when `cache: true`.\n - Fresh scan via `fs_cache::force_rescan` when `cache: false`.\n - Optional empty-result recheck when cached results are older than the empty-result recheck threshold.\n - Entry filtering: file-only + optional glob filter (`glob_util`) + optional type filter mapping (`js`, `ts`, `rust`, etc.).\n\n### Search/collection semantics\n\n- Regex engine: `grep_regex::RegexMatcherBuilder` with `ignoreCase` and `multiline`.\n- Context resolution:\n - `contextBefore/contextAfter` override legacy `context`.\n - Non-content modes do not collect context.\n- Output modes:\n - `content` -> one `GrepMatch` per hit.\n - `count` and `filesWithMatches` map to count-style entries (`lineNumber=0`, `line=\"\"`, `matchCount` set).\n- Limits:\n - Global `offset` and `maxCount` apply across files.\n - Parallel path is used only when `maxCount` is unset and `offset == 0`; otherwise sequential path preserves deterministic global offset/limit semantics.\n\n### Result shaping back to JS\n\n- Rust `SearchResult`/`GrepResult` fields map to TS interfaces via N-API object conversion.\n- Counters are clamped before crossing N-API where needed.\n- `GrepResult.limitReached` is optional and emitted when true.\n- Streaming callback receives each shaped `GrepMatch` for content or count-style entries.\n\n### Failure behavior\n\n- `search` returns `SearchResult.error` for regex/search failures instead of throwing.\n- `grep` rejects on hard errors such as invalid path, invalid glob/regex, or cancellation timeout/abort.\n- `hasMatch` returns a boolean on success and throws on invalid pattern/UTF-8 conversion errors.\n- File open/search errors in multi-file scans are skipped per-file; scan continues.\n\n### Malformed regex handling\n\n`grep.rs` sanitizes braces before regex compile:\n\n- Invalid repetition-like braces are escaped (`{`/`}` -> `\\{`/`\\}`) when they cannot form `{N}`, `{N,}`, `{N,M}`.\n- This prevents common literal-template fragments (for example `${platform}`) from failing as malformed repetition.\n- Remaining invalid regex syntax still returns a regex error.\n\n## 2) File discovery (`glob`) and fuzzy path search (`fuzzyFind`)\n\n`glob` and `fuzzyFind` share `fs_cache` scans; matching logic differs.\n\n### `glob` flow\n\n1. Caller passes `GlobOptions` directly. `pattern` and `path` are required in the generated type.\n2. Rust resolves the search path and compiles pattern via `glob_util::compile_glob`.\n3. Entry source:\n - `cache=true` -> `get_or_scan` + optional stale-empty `force_rescan`.\n - `cache=false` -> `force_rescan(..., store=false)` (fresh only).\n4. Filtering:\n - skip `.git` always;\n - skip `node_modules` unless requested (`includeNodeModules`) or pattern mentions `node_modules`;\n - apply glob match;\n - apply file-type filter; symlink `file`/`dir` filters resolve target metadata.\n5. Optional sort by mtime descending (`sortByMtime`) before truncating to `maxResults`.\n\n### `fuzzyFind` flow\n\n1. Rust implementation lives in `fd.rs`; generated export is `fuzzyFind`.\n2. Shared scan source from `fs_cache` with the same cache/no-cache split and stale-empty recheck policy.\n3. Scoring:\n - exact / starts-with / contains / subsequence-based fuzzy score;\n - separator/punctuation-normalized scoring path;\n - directory bonus and deterministic tie-break (`score desc`, then `path asc`).\n4. Symlink entries are excluded from fuzzy results.\n\n### Failure behavior\n\n- Invalid glob pattern returns an error from `glob_util::compile_glob`.\n- Search root must resolve to an existing directory for directory discovery flows.\n- Cancellation/timeouts propagate as abort errors via `CancelToken::heartbeat()` checks in loops.\n\n### Malformed glob handling\n\n`glob_util::build_glob_pattern` is tolerant:\n\n- normalizes `\\` to `/`,\n- auto-prefixes simple recursive patterns with `**/` when `recursive=true`,\n- auto-closes unbalanced `{...` alternation groups before compile.\n\n## 3) AST search/edit (`astGrep`, `astEdit`)\n\n`ast.rs` exposes syntax-aware code search and rewrite operations.\n\n- `astGrep(options)` returns matches with byte/line/column coordinates and optional metavariable bindings.\n- `astEdit(options)` returns replacement changes, per-file counts, searched/touched file counts, parse errors, and whether edits were applied.\n- `dryRun` defaults to true for edit options in the generated documentation.\n- Options include language override, path/glob/selector, strictness, limits, parse-error policy, `signal`, and `timeoutMs`.\n\nThese exports are direct native APIs used by tooling; they are not mediated by a TS wrapper in `packages/natives`.\n\n## 4) Shared scan/cache lifecycle (`fs_cache`)\n\n`fs_cache` stores scan results as normalized relative entries (`path`, `fileType`, optional `mtime`) keyed by:\n\n- canonical search root,\n- `include_hidden`,\n- `use_gitignore`.\n\n### Cache state transitions\n\n1. **Miss / disabled**\n - TTL is `0` or key absent/expired -> fresh collection.\n2. **Hit**\n - Entry age is within TTL -> return cached entries + `cache_age_ms`.\n3. **Stale-empty recheck**\n - If query yields zero matches and cache age exceeds the empty-result threshold, force one rescan.\n4. **Invalidation**\n - `invalidateFsScanCache(path?)`:\n - no arg: clear all keys;\n - path arg: remove keys for roots affected by that path.\n\n### Stale-result tradeoff\n\n- Cache favors low-latency repeated scans over immediate consistency.\n- TTL window can return stale positives/negatives.\n- Empty-result recheck reduces stale negatives for older cached scans at the cost of one extra scan.\n- Explicit invalidation is the intended correctness hook after file mutations.\n\n## 5) ANSI text utilities (`text`)\n\nThese are pure, in-memory utilities.\n\n### Boundaries and responsibilities\n\n- `text.rs` owns terminal-cell semantics:\n - ANSI sequence parsing,\n - grapheme-aware width and slicing,\n - wrap/truncate/sanitize behavior,\n - explicit tab-width parameter on width-sensitive APIs.\n- `grep.rs` line truncation (`maxColumns`) is separate:\n - simple character-boundary truncation of matched lines with `...`,\n - not ANSI-state-preserving and not terminal-cell width aware.\n\n### Key behaviors\n\n- `wrapTextWithAnsi`: wraps by visible width, carries active SGR codes across wrapped lines.\n- `truncateToWidth`: visible-cell truncation with ellipsis policy (`Unicode`, `Ascii`, `Omit`), optional right padding.\n- `sliceWithWidth`: column slicing with optional strict width enforcement.\n- `extractSegments`: extracts before/after segments around an overlay while restoring ANSI state for the `after` segment.\n- `sanitizeText` (ANSI/control/surrogate stripping with line-ending normalization) no longer lives in `text.rs`; it moved to `@sayknow-cli/utils` as a pure-JS implementation in `packages/utils/src/sanitize-text.ts`. The native binding was removed in the same change because the JS version was competitive on the benchmarked workloads, and keeping a Rust copy forced every caller (including `pi-utils`) to pull in `@sayknow-cli/natives`.\n- `visibleWidth`: counts visible terminal cells using caller-supplied tab width.\n\n### Failure behavior\n\nText functions generally return deterministic transformed output; errors are limited to N-API argument/string conversion boundaries.\n\n## 6) Syntax highlighting (`highlight`)\n\n`highlight.rs` is pure transformation; it does not use the filesystem scan cache.\n\n### Flow\n\n1. Caller passes `code`, optional `lang`, and ANSI color palette.\n2. Rust resolves syntax by token/name lookup, extension lookup, alias table fallback, then plain-text fallback.\n3. Each line is parsed with syntect `ParseState` and scope stack.\n4. Scopes map to semantic color categories and ANSI color codes are injected/reset.\n\n### Failure behavior\n\n- Per-line parse failure does not fail the call: that line is appended unhighlighted and processing continues.\n- Unknown/unsupported language falls back to plain text syntax.\n\n## 7) Token counting (`tokens`)\n\n`countTokens(input, encoding?)` is an in-memory utility.\n\n- `input` may be a single string or an array of strings.\n- Arrays return one aggregate count and are encoded in parallel in Rust.\n- Default encoding is `O200kBase`; `Cl100kBase` remains available as a compatibility alias routing to `o200k_base` in default builds.\n- The implementation uses ordinary tokenization, not special-token handling.\n\n## Pure utility vs filesystem-dependent flows\n\n| Flow | Filesystem access | Shared cache | Notes |\n| ---------------------------- | ----------------- | -------------------- | --------------------------------------------- |\n| `search` / `hasMatch` | No | No | regex on provided bytes/string only |\n| `text` module functions | No | No | ANSI/width/sanitization only |\n| `highlight` module functions | No | No | syntax + ANSI coloring only |\n| `countTokens` | No | No | tokenization only |\n| `astGrep` / `astEdit` | Yes | No | syntax-aware file search/edit |\n| `glob` | Yes | Optional | directory scans + glob filtering |\n| `fuzzyFind` | Yes | Optional | directory scans + fuzzy scoring |\n| `grep` (file/dir path) | Yes | Optional in dir mode | ripgrep over files, optional filters/callback |\n\n## End-to-end lifecycle summary\n\n1. Caller invokes generated native export with typed options.\n2. Rust validates/normalizes options and builds matcher/search config.\n3. For filesystem flows, entries are scanned (cache hit/miss/rescan where applicable) then filtered/scored/searched.\n4. Worker loops periodically call cancel heartbeat; timeout/abort can terminate execution.\n5. Rust shapes outputs into N-API objects (`lineNumber`, `matchCount`, `limitReached`, etc.).\n6. Generated bindings return typed JS objects and optional per-match callbacks for `grep`/`glob`.\n",
53
- "non-compaction-retry-policy.md": "# Non-compaction auto-retry policy\n\nThis document describes the standard API-error retry path in `AgentSession`.\n\nIt explicitly excludes context-overflow recovery via auto-compaction. Overflow is handled by compaction logic and is documented separately in [`compaction.md`](../docs/compaction.md).\n\n## Implementation files\n\n- [`../src/session/agent-session.ts`](../packages/coding-agent/src/session/agent-session.ts)\n- [`../src/config/settings-schema.ts`](../packages/coding-agent/src/config/settings-schema.ts)\n- [`../src/modes/controllers/event-controller.ts`](../packages/coding-agent/src/modes/controllers/event-controller.ts)\n- [`sdk.md`](./sdk.md) for the external machine interface.\n\n## Scope boundary vs compaction\n\nRetry and compaction are checked from the same `agent_end` path, but they are intentionally separated:\n\n1. `agent_end` inspects the last assistant message.\n2. `#isRetryableError(...)` runs first.\n3. If retry is initiated, compaction checks are skipped for that turn.\n4. Context-overflow errors are hard-excluded from retry classification (`isContextOverflow(...)` short-circuits retry).\n5. Overflow therefore falls through to `#checkCompaction(...)` instead of standard retry.\n\nSo: overload/rate/server/network-style failures use this retry policy; context-window overflow uses compaction recovery.\n\n## Retry classification\n\n`#isRetryableError(...)` requires all of the following:\n\n- assistant `stopReason === \"error\"`\n- `errorMessage` exists\n- message is **not** context overflow\n- `errorMessage` matches transient transport/envelope patterns or `isUsageLimitError(...)`\n\nCurrent retryable inputs are regex/string-classified:\n\n- transient transport/envelope failures, including Anthropic stream-envelope failures before `message_start`\n- overloaded/provider-returned-error wording\n- rate limit / usage limit / too many requests\n- HTTP-like server classes: 429, 500, 502, 503, 504\n- service unavailable / server/internal error\n- provider-suggested retry wording, including OpenAI `retry your request` failures\n- network/connection/socket failures, refused/closed connections, upstream connect/reset-before-headers, socket hang up, timeout/timed out, fetch failed, terminated, retry delay wording, and unexpected socket close messages\n\nManaged fallback uses structured transport facts and typed provider error codes when available. A structured classification of `other` becomes the bounded `unknown` fallback class; error prose cannot promote it to quota or transient. Regex classification is retained only as a legacy fallback.\n\n## Retry lifecycle and state transitions\n\nSession state used by retry:\n\n- `#retryAttempt: number` (`0` means idle)\n- `#retryPromise: Promise<void> | undefined` (tracks in-progress retry lifecycle)\n- `#retryResolve: (() => void) | undefined` (resolves `#retryPromise`)\n- `#retryAbortController: AbortController | undefined` (cancels backoff sleep)\n\nFlow (`#handleRetryableError`):\n\n1. Read `retry` settings group.\n2. If `retry.enabled === false`, stop immediately (`false`, no retry started).\n3. Increment `#retryAttempt`.\n4. Create `#retryPromise` once (first attempt in a chain).\n5. Transient errors retry without an attempt limit; unknown/no-code errors stop after `retry.maxRetries`.\n6. Compute exponential full-jitter delay capped at `retry.maxDelayMs`; legacy parsed provider retry-after values override computed backoff and are capped at `retry.maxDelayMs`, while managed typed Retry-After values are intentionally uncapped.\n7. For usage-limit errors, call auth storage (`markUsageLimitReached(...)`); if credential switching succeeds, force delay to `0`, otherwise use the applicable backoff.\n8. Eligible ordered role-array fallback chains advance on entry-budget exhaustion. A selected fallback entry remains sticky until the head selector's rate-limit cooldown expires, when `retry.fallbackRevertPolicy: cooldown-expiry` probes it again on a new turn.\n9. Emit `auto_retry_start`.\n10. Remove the trailing assistant error message from agent runtime state (kept in persisted session history).\n11. Sleep with abort support.\n12. Schedule `agent.continue()` through the post-prompt task scheduler (`delayMs: 1`) for the same prompt generation.\n\n### What resets retry counters\n\n`#retryAttempt` resets to `0` in these cases:\n\n- first successful non-error, non-aborted assistant message after retries started (emits `auto_retry_end { success: true }`)\n- retry cancellation during backoff sleep\n- max retries exceeded path\n\n`#retryPromise` resolves/clears when retry chain ends (success, cancellation, or max-exceeded), via `#resolveRetry()`.\n\n## Backoff and max-attempt semantics\n\nSettings:\n\n- `retry.enabled` (default `true`)\n- `retry.maxRetries` (default `3`)\n- `retry.baseDelayMs` (default `2000`)\n- `retry.maxDelayMs` (default `300000`)\n- `retry.requestMaxRetries` (default `5`) — provider request retries before a stream is established; counts retries, not the initial request\n- `retry.streamMaxRetries` (default `5`) — provider stream replay retries for replay-safe transient stream failures; counts retries, not the initial stream attempt\n\nAttempt numbering:\n\n- attempt counter is incremented before max-check\n- start events use current attempt (1-based)\n- max-exceeded end event reports `attempt: this.#retryAttempt - 1` (last attempted retry count)\n\nBackoff uses capped exponential full jitter. With default settings the maximum jitter windows are:\n\n- attempt 1: 2000 ms\n- attempt 2: 4000 ms\n- attempt 3: 8000 ms\n\n`retry.maxDelayMs` caps every legacy session retry delay, including provider retry-after hints, which otherwise take precedence over computed backoff. Managed fallback intentionally does not cap typed Retry-After values because it retries within its separate per-entry budget. Legacy transient errors have unbounded attempts; unknown/no-code errors are bounded by `retry.maxRetries`.\n\n## Abort mechanics\n\n### Explicit retry abort\n\n`abortRetry()`:\n\n- aborts `#retryAbortController` (if present)\n- resolves retry promise (`#resolveRetry()`) so awaiters are unblocked\n\nIf abort hits while sleeping, catch path emits:\n\n- `auto_retry_end { success: false, finalError: \"Retry cancelled\" }`\n- resets attempt/controller\n\n### Global operation abort interaction\n\n`abort()` calls `abortRetry()` before aborting the active agent stream. This guarantees retry backoff is cancelled when user issues a general abort.\n\n### TUI interaction\n\nOn `auto_retry_start`, EventController:\n\n- swaps `Esc` handler to `session.abortRetry()`\n- renders loader text: `Retrying (attempt/maxAttempts) in Ns… (esc to cancel)`\n\nOn `auto_retry_end`, it restores prior `Esc` handler and clears loader state.\n\n## Streaming and prompt completion behavior\n\n`prompt()` ultimately waits on `#waitForRetry()` after `agent.prompt(...)` returns.\n\nEffect:\n\n- a prompt call does not fully resolve until any started retry chain finishes (success/failure/cancel)\n- retry lifecycle is part of one logical prompt execution boundary\n\nThis prevents callers from treating a retrying turn as complete too early.\n\n## Controls: settings and SDK actions\n\n### Configuration knobs\n\nThe standard retry controls are defined in the settings schema under `retry`:\n\n- `retry.enabled`\n- `retry.maxRetries`\n- `retry.baseDelayMs`\n- `retry.maxDelayMs`\n\nFallback candidates are configured as ordered selector arrays on preset `model_mapping` roles, top-level `modelRoles`, or `task.agentModelOverrides`; `fallback.maxAttempts` controls the total request-time attempts per concrete entry. Resolution-time unavailable, unauthenticated, and unknown entries advance immediately without consuming that budget. For the session default, `fallback.models` and (with `fallback.auto`) one model per other logged-in provider are appended after the configured chain; see [Fallback chains](./models.md#fallback-chains).\n\nOn settings load, a source-aware one-shot migration still reads legacy `retry.fallbackChains` and combines the effective role chain with its ordered, deduplicated legacy tail into the corresponding role array. The legacy key is ignored after migration; it is not a retry configuration surface.\n\nProgrammatic toggles in session:\n\n- `setAutoRetryEnabled(enabled)` writes `retry.enabled`\n- `autoRetryEnabled` reads `retry.enabled`\n- `isRetrying` reports whether retry lifecycle promise is active\n\n### External control\n\nExternal clients observe retry lifecycle through the [SDK machine interface](./sdk.md). The removed RPC command surface and `RpcClient` helpers are not supported.\n\n## Event emission and failure surfacing\n\nSession-level retry events:\n\n- `auto_retry_start { attempt, maxAttempts, delayMs, errorMessage }`\n- `auto_retry_end { success, attempt, finalError? }`\n- `model_fallback_switched { eventId, from, to, reason, role, scope, activeIndex, chainLength, attemptsUsed }` — emitted once for each real fallback-model switch\n\nPropagation:\n\n- emitted through `AgentSession.subscribe(...)`\n- forwarded to extension runner as extension events\n- exposed to external clients through SDK event subscriptions\n- in the TUI, `model_fallback_switched` updates the fallback-model status/notice and `EventController` consumes retry lifecycle events for loader/error UI\n\nFinal failure surfacing:\n\n- On max-exceeded or cancellation, `auto_retry_end.success === false`\n- TUI shows: `Retry failed after N attempts: <finalError>`\n- Extensions/hooks receive `auto_retry_end` with same fields\n- SDK clients receive the same event stream\n\n## Permanent stop conditions\n\nRetry stops and will not auto-continue when any of these occur:\n\n- `retry.enabled` is false, or legacy retry settings have not been explicitly configured (`legacyRetryConfigured` fail-closed gate)\n- error is not retry-classified\n- error is context overflow (delegated to compaction path)\n- max retries exceeded\n- user cancels retry through the session/SDK action or `Esc` during retry loader\n- global abort (`abort`) cancels retry first\n\nA new retry chain can still start later on a future retryable error after counters reset.\n\n## Operational caveats\n\n- Managed fallback uses typed transport facts and provider error codes; regex text matching is limited to the legacy retry path.\n- Retry strips the failing assistant error from **runtime context** before re-continue, but session history still keeps that error entry.\n- SDK clients observe retry state through session events and state updates.\n- Fallback state is driven by the configured ordered role array and remains on a selected fallback entry across later user prompts. A real model change emits the canonical `model_fallback_switched` event rather than a legacy retry-fallback event.\n- Temporary provider-session scopes retain and restore their own fallback controller and provider state when unwound; an authoritative model selection commits those temporary scopes.\n\n## Provider request/stream retry budgets\n\nThe provider budgets are deliberately separate from session auto-retry:\n\n```yaml\nretry:\n requestMaxRetries: 4\n streamMaxRetries: 100\n```\n\n`requestMaxRetries` maps to provider SDK/fetch retry counts for request setup failures such as retryable 5xx/408/429/network errors. `streamMaxRetries` maps to provider-specific stream replay loops that are safe to repeat without duplicating visible assistant output. Providers that cannot safely replay a stream continue to surface the terminal error so the session-level auto-retry layer can decide whether to retry the turn.\n\nFail-fast cases stay fail-fast: invalid credentials (after any credential-refresh path is exhausted), unsupported model/provider configuration, malformed requests, context overflow, explicit user aborts, and permanent quota failures are not treated as transient provider budget candidates.\n",
53
+ "non-compaction-retry-policy.md": "# Non-compaction auto-retry policy\n\nThis document describes the standard API-error retry path in `AgentSession`.\n\nIt explicitly excludes context-overflow recovery via auto-compaction. Overflow is handled by compaction logic and is documented separately in [`compaction.md`](../docs/compaction.md).\n\n## Implementation files\n\n- [`../src/session/agent-session.ts`](../packages/coding-agent/src/session/agent-session.ts)\n- [`../src/config/settings-schema.ts`](../packages/coding-agent/src/config/settings-schema.ts)\n- [`../src/modes/controllers/event-controller.ts`](../packages/coding-agent/src/modes/controllers/event-controller.ts)\n- [`sdk.md`](./sdk.md) for the external machine interface.\n\n## Scope boundary vs compaction\n\nRetry and compaction are checked from the same `agent_end` path, but they are intentionally separated:\n\n1. `agent_end` inspects the last assistant message.\n2. `#isRetryableError(...)` runs first.\n3. If retry is initiated, compaction checks are skipped for that turn.\n4. Context-overflow errors are hard-excluded from retry classification (`isContextOverflow(...)` short-circuits retry).\n5. Overflow therefore falls through to `#checkCompaction(...)` instead of standard retry.\n\nSo: overload/rate/server/network-style failures use this retry policy; context-window overflow uses compaction recovery.\n\n## Retry classification\n\n`#isRetryableError(...)` requires all of the following:\n\n- assistant `stopReason === \"error\"`\n- `errorMessage` exists\n- message is **not** context overflow\n- `errorMessage` matches transient transport/envelope patterns or `isUsageLimitError(...)`\n\nCurrent retryable inputs are regex/string-classified:\n\n- transient transport/envelope failures, including Anthropic stream-envelope failures before `message_start`\n- overloaded/provider-returned-error wording\n- rate limit / usage limit / too many requests\n- HTTP-like server classes: 429, 500, 502, 503, 504\n- service unavailable / server/internal error\n- provider-suggested retry wording, including OpenAI `retry your request` failures\n- network/connection/socket failures, refused/closed connections, upstream connect/reset-before-headers, socket hang up, timeout/timed out, fetch failed, terminated, retry delay wording, and unexpected socket close messages\n\nManaged fallback uses structured transport facts and typed provider error codes when available. A structured classification of `other` becomes the bounded `unknown` fallback class; error prose cannot promote it to quota or transient. Regex classification is retained only as a legacy fallback.\n\n## Retry lifecycle and state transitions\n\nSession state used by retry:\n\n- `#retryAttempt: number` (`0` means idle)\n- `#retryPromise: Promise<void> | undefined` (tracks in-progress retry lifecycle)\n- `#retryResolve: (() => void) | undefined` (resolves `#retryPromise`)\n- `#retryAbortController: AbortController | undefined` (cancels backoff sleep)\n\nFlow (`#handleRetryableError`):\n\n1. Read `retry` settings group.\n2. If `retry.enabled === false`, stop immediately (`false`, no retry started).\n3. Increment `#retryAttempt`.\n4. Create `#retryPromise` once (first attempt in a chain).\n5. Default transient same-model recovery is bounded by one shared step budget: 7 upstream requests and 15 minutes after the first failure. Every concrete request counts, including hidden SDK resends, credential/token fetches, and websocket frames. An explicit `retry.maxRetries` or `fallback.maxAttempts` keeps its own attempt unit and also stays inside that budget. Unknown/no-code errors stop after `retry.maxRetries`.\n6. Compute exponential full-jitter delay. Default transient recovery uses a 1s base and 30s cap; explicit `retry.baseDelayMs` / `retry.maxDelayMs` win. Legacy parsed provider retry-after values override computed backoff and are capped at `retry.maxDelayMs`, while managed typed Retry-After values are intentionally uncapped.\n7. On a managed fallback chain, quota/rate-limit failures first try another logged-in account of the same provider (`markUsageLimitReached(...)`) and auth failures invalidate the matching credential; if the active credential changes, force delay to `0`, otherwise use the applicable backoff. This credential maintenance (account rotation, restore, usage probes) spends the same step budget as inference: every token/usage HTTP request is admitted as kind `\"token\"`, and maintenance never runs when no step budget is active, so it cannot send unadmitted credential HTTP.\n8. Eligible ordered role-array fallback chains advance on entry-budget exhaustion. A selected fallback entry remains sticky until the head selector's rate-limit cooldown expires, when `retry.fallbackRevertPolicy: cooldown-expiry` probes it again on a new turn. Transient failures never change the model.\n9. Emit `auto_retry_start`.\n10. A failure with no public output removes the trailing assistant error message from agent runtime state (kept in persisted session history). A failure after public text keeps that text exactly as shown and never deletes or re-emits it. When the step is not inside a managed fallback attempt, the model API is `openai-completions` or `anthropic-messages` (not the pi-native transport), and the public tail is plain text, the session checkpoints the text and appends one hidden continuation instruction on the same model. Otherwise (a managed attempt, another API, a tail with tool calls, images, or signed/redacted thinking) the step pauses with its output preserved, and the user resumes it with a new message.\n11. Sleep with abort support.\n12. Schedule `agent.continue()` through the post-prompt task scheduler (`delayMs: 1`) for the same prompt generation.\n13. A process restart with an unconsumed continuation instruction does not send it automatically.\n14. Consecutive first-event or idle timeouts with no output (`retry.maxSilentTimeouts`, default 3, range 1-7) close the step's admission early. The counter resets when an attempt produces output or the model changes; it never raises the 7-request / 15-minute budget.\n\n### Automatic resume of an interrupted step\n\nWith `retry.autoResume` (default `true`) an interrupted step resumes once per user turn without a new user message, on the interrupted model. The resume never switches models or advances a fallback chain. Inside the resumed run, a transient failure under the default shared step budget may still retry on the same model. Any other failure stops with the provider error instead of moving to the next chain entry. This includes quota, auth, and any failure when `fallback.maxAttempts` is set.\n\n- **When it runs:**\n - after a transient failure (or no-output timeout) that follows executed tools or a visible text prefix, when the step's budget can still admit a request;\n - on startup continuation (`--continue` with no new input) when the last recovery checkpoint records an unfinished step.\n- **Not resumed:** rate-limit and quota failures. A stale continuation instruction found at startup is also left for the user.\n- **Budget:**\n - **In-process:** a resume spends what is left of the interrupted step's budget.\n - **Restart:** the old budget, deadline and request outcome are unknown and never reused. The resumed step gets a small fresh envelope of 2 requests within 120 seconds, with an armed deadline. Later steps of the resumed run use the normal budget.\n- **Tools whose outcome was not observed:** each such call receives an error result saying it was interrupted and not re-run. This only arises after a restart. In process, every executed call already has a result, so the resume keeps all tools. While any exist, the resumed turn sees and may run only the built-in `read`, `search`, `find`, `ast_grep` and `locate` tools. A tool qualifies only as the built-in instance with a unique wire name; extension, MCP and name-alike tools never do. Tools behind extension or hook interception wrappers are still identified by their built-in class. The selected tool set is not changed. The restriction ends when the run settles or a user message reaches the model; merely queuing a message does not lift it.\n- **Durability first:** the resume marker, placeholders and hidden instruction are written and flushed before anything is sent. A failed write sends nothing.\n- **It pauses as before** when any of these hold:\n - a resume marker already exists after the last user message;\n - the last answer already finished;\n - unobserved remote work was recorded, including a persisted `uncertain` reason or a Cursor/pi-native step;\n - the tail holds signed/redacted thinking or partial tool calls;\n - a visible prefix that the current serializer cannot replay;\n - the checkpoint's model differs from the current model;\n - the budget is spent;\n - user input is queued;\n - no restart-safe tool is available while unobserved calls exist.\n- **Uncertainty is persisted:** in-step uncertainty is written to the step's `in_flight` checkpoint together with its `selector`, `api` and `transport`. The next admission and every resume decision wait for that write. A failed write closes admission for that step only and is reported; the next user message starts normally.\n\n### What resets retry counters\n\n`#retryAttempt` resets to `0` in these cases:\n\n- first successful non-error, non-aborted assistant message after retries started (emits `auto_retry_end { success: true }`)\n- retry cancellation during backoff sleep\n- max retries exceeded path\n\n`#retryPromise` resolves/clears when retry chain ends (success, cancellation, or max-exceeded), via `#resolveRetry()`.\n\n## Backoff and max-attempt semantics\n\nSettings:\n\n- `retry.enabled` (default `true`)\n- `retry.maxRetries` (default `3`)\n- `retry.baseDelayMs` (default `2000`)\n- `retry.maxDelayMs` (default `300000`)\n- `retry.requestMaxRetries` (default `5`) — provider request retries before a stream is established; counts retries, not the initial request\n- `retry.streamMaxRetries` (default `5`) — provider stream replay retries for replay-safe transient stream failures; counts retries, not the initial stream attempt\n\nAttempt numbering:\n\n- attempt counter is incremented before max-check\n- start events use current attempt (1-based)\n- max-exceeded end event reports `attempt: this.#retryAttempt - 1` (last attempted retry count)\n\nBackoff uses capped exponential full jitter: attempt `k` waits a random delay in `[0, min(base × 2^(k-1), max))`.\n\n- Default shared-budget transient recovery uses a 1000 ms base and a 30 s cap, so the maximum jitter windows are 1000 / 2000 / 4000 ms for attempts 1 / 2 / 3. An explicitly set `retry.baseDelayMs` or `retry.maxDelayMs` replaces the corresponding default.\n- Every other path (explicit `retry.maxRetries`, non-transient classes, managed per-entry budgets) uses `retry.baseDelayMs` (default `2000`) and `retry.maxDelayMs` (default `300000`); with the 2000 ms default the windows are 2000 / 4000 / 8000 ms.\n\n`retry.maxDelayMs` caps every legacy session retry delay, including provider retry-after hints, which otherwise take precedence over computed backoff. Managed fallback intentionally does not cap typed Retry-After values because it retries within its separate per-entry budget. Default transient recovery is bounded by the shared step budget (7 requests, 15 minutes after the first failure); unknown/no-code errors are bounded by `retry.maxRetries`. Setting `retry.enabled` to `false` disables automatic same-model recovery.\n\n## Abort mechanics\n\n### Explicit retry abort\n\n`abortRetry()`:\n\n- aborts `#retryAbortController` (if present)\n- resolves retry promise (`#resolveRetry()`) so awaiters are unblocked\n\nIf abort hits while sleeping, catch path emits:\n\n- `auto_retry_end { success: false, finalError: \"Retry cancelled\" }`\n- resets attempt/controller\n\n### Global operation abort interaction\n\n`abort()` calls `abortRetry()` before aborting the active agent stream. This guarantees retry backoff is cancelled when user issues a general abort.\n\n### TUI interaction\n\nOn `auto_retry_start`, EventController:\n\n- swaps `Esc` handler to `session.abortRetry()`\n- renders loader text: `Retrying (attempt/maxAttempts) in Ns… (esc to cancel)`\n\nOn `auto_retry_end`, it restores prior `Esc` handler and clears loader state.\n\n## Streaming and prompt completion behavior\n\n`prompt()` ultimately waits on `#waitForRetry()` after `agent.prompt(...)` returns.\n\nEffect:\n\n- a prompt call does not fully resolve until any started retry chain finishes (success/failure/cancel)\n- retry lifecycle is part of one logical prompt execution boundary\n\nThis prevents callers from treating a retrying turn as complete too early.\n\n## Controls: settings and SDK actions\n\n### Configuration knobs\n\nThe standard retry controls are defined in the settings schema under `retry`:\n\n- `retry.enabled`\n- `retry.maxRetries`\n- `retry.baseDelayMs`\n- `retry.maxDelayMs`\n- `retry.autoResume` (automatic resume of an interrupted step; default `true`)\n- `retry.maxSilentTimeouts` (consecutive no-output timeouts before a step stops; default 3)\n\nFallback candidates are configured as ordered selector arrays on preset `model_mapping` roles, top-level `modelRoles`, or `task.agentModelOverrides`; `fallback.maxAttempts` controls the total request-time attempts per concrete entry. Resolution-time unavailable, unauthenticated, and unknown entries advance immediately without consuming that budget. For the session default, `fallback.models` and (with `fallback.auto`) one model per other logged-in provider are appended after the configured chain; see [Fallback chains](./models.md#fallback-chains).\n\nOn settings load, a source-aware one-shot migration still reads legacy `retry.fallbackChains` and combines the effective role chain with its ordered, deduplicated legacy tail into the corresponding role array. The legacy key is ignored after migration; it is not a retry configuration surface.\n\nProgrammatic toggles in session:\n\n- `setAutoRetryEnabled(enabled)` writes `retry.enabled`\n- `autoRetryEnabled` reads `retry.enabled`\n- `isRetrying` reports whether retry lifecycle promise is active\n\n### External control\n\nExternal clients observe retry lifecycle through the [SDK machine interface](./sdk.md). The removed RPC command surface and `RpcClient` helpers are not supported.\n\n## Event emission and failure surfacing\n\nSession-level retry events:\n\n- `auto_retry_start { attempt, maxAttempts, delayMs, errorMessage }`\n- `auto_retry_end { success, attempt, finalError? }`\n- `model_fallback_switched { eventId, from, to, reason, role, scope, activeIndex, chainLength, attemptsUsed }` — emitted once for each real fallback-model switch\n\nPropagation:\n\n- emitted through `AgentSession.subscribe(...)`\n- forwarded to extension runner as extension events\n- exposed to external clients through SDK event subscriptions\n- in the TUI, `model_fallback_switched` updates the fallback-model status/notice and `EventController` consumes retry lifecycle events for loader/error UI\n\nFinal failure surfacing:\n\n- On max-exceeded or cancellation, `auto_retry_end.success === false`\n- TUI shows: `Retry failed after N attempts: <finalError>`\n- Extensions/hooks receive `auto_retry_end` with same fields\n- SDK clients receive the same event stream\n\n## Permanent stop conditions\n\nRetry stops and will not auto-continue when any of these occur:\n\n- `retry.enabled` is false, or legacy retry settings have not been explicitly configured (`legacyRetryConfigured` fail-closed gate)\n- error is not retry-classified\n- error is context overflow (delegated to compaction path)\n- max retries exceeded\n- user cancels retry through the session/SDK action or `Esc` during retry loader\n- global abort (`abort`) cancels retry first\n- the step dispatched remote work whose effects cannot be observed or proven unsent: a broker forced credential refresh or broker OAuth refresh, a pi-native gateway dispatch, a Cursor agent run, or a custom OAuth provider refresh. The step stops with the original provider error; no automatic resend, visible-text continuation, credential rotation, or fallback request is admitted for it (every further upstream request of that step is refused as `cancelled`)\n- the shared step budget refuses admission (`request_limit`, `deadline`, or `cancelled`). A refusal is the budget's own terminal answer and is never itself retry-classified; a backoff that would end past the deadline stops locally without contacting upstream\n\nManual retry (`retry()`, `/retry`) is refused while the latest persisted `recovery_checkpoint` on the branch is unresolved (not `completed` with all of its visible messages present), or while the last unfinished step left unobservable remote work unresolved. In both cases only a new user prompt resumes the session, and no upstream request is sent for the interrupted step. Switching sessions, branching, forking, handoff, and rollback re-derive this state from the persisted checkpoints of the resulting transcript and drop in-memory uncertainty that belonged to the previous transcript.\n\nA new retry chain can still start later on a future retryable error after counters reset.\n\n## Operational caveats\n\n- Managed fallback uses typed transport facts and provider error codes; regex text matching is limited to the legacy retry path.\n- Retry strips the failing assistant error from **runtime context** before re-continue, but session history still keeps that error entry. Exception: a visible-text continuation never removes the tail; the public text stays in runtime context and history exactly as shown, and one hidden continuation instruction is appended after it.\n- SDK clients observe retry state through session events and state updates.\n- Fallback state is driven by the configured ordered role array and remains on a selected fallback entry across later user prompts. A real model change emits the canonical `model_fallback_switched` event rather than a legacy retry-fallback event.\n- Temporary provider-session scopes retain and restore their own fallback controller and provider state when unwound; an authoritative model selection commits those temporary scopes.\n\n## Provider request/stream retry budgets\n\n`retry.requestMaxRetries` and `retry.streamMaxRetries` are a separate attempt unit from session auto-retry, but not a separate request envelope. Defaults:\n\n```yaml\nretry:\n requestMaxRetries: 5\n streamMaxRetries: 5\n```\n\n`requestMaxRetries` maps to provider SDK/fetch retry counts for request setup failures such as retryable 5xx/408/429/network errors. `streamMaxRetries` maps to provider-specific stream replay loops that are safe to repeat without duplicating visible assistant output. Inside a session model step the session owns request resends: `requestMaxRetries` is forced to 0 for that step, so a retryable HTTP failure (including its `Retry-After`) reaches the session retry layer instead of waiting inside the SDK. `requestMaxRetries` keeps its meaning for direct SDK callers and other paths without a session step. Any remaining stream-level resend is admitted through the same shared step budget (7 upstream requests, 15 minutes after the first failure) as kind `\"resend\"` just before it is sent, and a resend records a recoverable failure, which starts the 15-minute deadline if it has not started yet. After a step reports unobservable remote work (uncertain upstream), no resend is admitted for it. Once that budget refuses admission, the provider loop ends with the admission error and sends nothing, even if its configured count is not used up. Raising either setting therefore cannot extend recovery beyond the step budget. Providers that cannot safely replay a stream continue to surface the terminal error so the session-level auto-retry layer can decide whether to retry the turn.\n\nFail-fast cases stay fail-fast: invalid credentials (after any credential-refresh path is exhausted), unsupported model/provider configuration, malformed requests, context overflow, explicit user aborts, and permanent quota failures are not treated as transient provider budget candidates.\n",
54
54
  "notebook-tool-runtime.md": "# Notebook tool runtime internals\n\nThis document describes the current `notebook` tool implementation and its relationship to the kernel-backed Python runtime.\n\nThe critical distinction: **`notebook` is a JSON notebook editor, not a notebook executor**. It edits `.ipynb` cell sources directly; it does not start or talk to a Python kernel.\n\n## Implementation files\n\n- [`src/tools/notebook.ts`](../packages/coding-agent/src/tools/notebook.ts)\n- [`src/eval/py/executor.ts`](../packages/coding-agent/src/eval/py/executor.ts)\n- [`src/eval/py/kernel.ts`](../packages/coding-agent/src/eval/py/kernel.ts)\n- [`src/session/streaming-output.ts`](../packages/coding-agent/src/session/streaming-output.ts)\n- [`src/tools/eval.ts`](../packages/coding-agent/src/tools/eval.ts)\n\n## 1) Runtime boundary: editing vs executing\n\n## `notebook` tool (`src/tools/notebook.ts`)\n\n- Supports `action: edit | insert | delete` on a `.ipynb` file.\n- Resolves path relative to session CWD (`resolveToCwd`).\n- Loads notebook JSON, validates `cells` array, validates `cell_index` bounds.\n- Applies source edits in-memory and writes full notebook JSON back with `JSON.stringify(notebook, null, 1)`.\n- Returns textual summary + structured `details` (`action`, `cellIndex`, `cellType`, `totalCells`, `cellSource`).\n\nNo kernel lifecycle exists in this tool:\n\n- no gateway acquisition\n- no kernel session ID\n- no `execute_request`\n- no stream chunks from kernel channels\n- no rich display capture (`image/png`, JSON display, status MIME)\n\n## Notebook-like execution path (`src/tools/eval.ts` + `src/eval/py/*`)\n\nWhen the agent needs to run cell-style Python code (sequential cells, persistent state, rich displays), that goes through the **`eval` tool** with `language: \"python\"`, not `notebook`.\n\nThat path is where kernel modes, restart/cancel behavior, chunk streaming, and output artifact truncation live.\n\n## 2) Notebook cell handling semantics (`notebook` tool)\n\n## Source normalization\n\n`content` is split into `source: string[]` with newline preservation:\n\n- each non-final line keeps trailing `\\n`\n- final line has no forced trailing newline\n\nThis mirrors notebook JSON conventions and avoids accidental line concatenation on later edits.\n\n## Action behavior\n\n- `edit`\n - replaces `cells[cell_index].source`\n - preserves existing `cell_type`\n- `insert`\n - inserts at `[0..cellCount]`\n - `cell_type` defaults to `code`\n - code cells initialize `execution_count: null` and `outputs: []`\n - markdown cells initialize only `metadata` + `source`\n- `delete`\n - removes `cells[cell_index]`\n - returns removed `source` in details for renderer preview\n\n## Error surfaces\n\nHard failures are thrown for:\n\n- missing notebook file\n- invalid JSON\n- missing/non-array `cells`\n- out-of-range index (insert and non-insert have different valid ranges)\n- missing `content` for `edit`/`insert`\n\nThese become `Error:` tool responses upstream; renderer uses notebook path + formatted error text.\n\n## 3) Kernel session semantics (where they actually exist)\n\nKernel semantics are implemented in `executePython` / `PythonKernel` and apply to the Python backend of the `eval` tool.\n\n## Modes\n\n`PythonKernelMode`:\n\n- `session` (default)\n - kernels cached in `kernelSessions` map\n - max 4 sessions; oldest evicted on overflow\n - idle/dead cleanup every 30s, timeout after 5 minutes\n - per-session queue serializes execution (`session.queue`)\n- `per-call`\n - creates kernel for request\n - executes\n - always shuts down kernel in `finally`\n\n## Reset behavior\n\n`eval` passes `reset` only for the first cell in a multi-cell Python call; later cells always run with `reset: false`.\n\n## Kernel death / restart / retry\n\nIn session mode (`withKernelSession`):\n\n- dead kernel detected by heartbeat (`kernel.isAlive()` check every 5s) or execute failure.\n- pre-run dead state triggers `restartKernelSession`.\n- execute-time crash path retries once: restart kernel, rerun handler.\n- `restartCount > 1` in same session throws `Python kernel restarted too many times in this session`.\n\nStartup retry behavior:\n\n- shared gateway kernel creation retries once on `SharedGatewayCreateError` with HTTP 5xx.\n\nResource exhaustion recovery:\n\n- detects `EMFILE`/`ENFILE`/\"Too many open files\" style failures\n- clears tracked sessions\n- calls `shutdownSharedGateway()`\n- retries kernel session creation once\n\n## 4) Environment/session variable injection\n\nKernel startup receives the optional session file path from executor:\n\n- `SKC_SESSION_FILE` (session state file path)\n\n`PythonKernel.#initializeKernelEnvironment(...)` then runs init script inside kernel to:\n\n- `os.chdir(cwd)`\n- inject env entries into `os.environ`\n- prepend cwd to `sys.path` if missing\n\nImplication:\n\n- prelude helpers that read session context rely on this env var in Python process state.\n\n## 5) Streaming/chunk and display handling (kernel-backed path)\n\nThe kernel client processes Jupyter protocol messages per execution:\n\n- `stream` -> text chunk to `onChunk`\n- `execute_result` / `display_data` ->\n - display text chosen by MIME precedence: `text/markdown` > `text/plain` > converted `text/html`\n - structured outputs captured separately:\n - `application/json` -> `{ type: \"json\" }`\n - `image/png` -> `{ type: \"image\" }`\n - `application/x-skc-status` -> `{ type: \"status\" }` (no text emission)\n- `error` -> traceback text pushed to chunk stream + structured error metadata\n- `input_request` -> emits stdin warning text, sends empty `input_reply`, marks stdin requested\n- completion waits for both `execute_reply` and kernel `status=idle`\n\nCancellation/timeout:\n\n- abort signal triggers `interrupt()` (REST `/interrupt` + control-channel `interrupt_request`)\n- result marks `cancelled=true`\n- timeout path annotates output with `Command timed out after <n> seconds`\n\n## 6) Truncation and artifact behavior\n\n`OutputSink` in `src/session/streaming-output.ts` is used by kernel execution paths (`executeWithKernel`):\n\n- sanitizes every chunk (`sanitizeText`)\n- tracks total/output lines and bytes\n- optional artifact spill file (`artifactPath`, `artifactId`)\n- when in-memory buffer exceeds threshold (`DEFAULT_MAX_BYTES` unless overridden):\n - marks truncated\n - keeps tail bytes in memory (UTF-8 safe boundary)\n - can spill full stream to artifact sink\n\n`dump()` returns:\n\n- visible output text (possibly tail-truncated)\n- truncation flag + counts\n- artifact ID (for `artifact://<id>` references)\n\n`eval` converts this metadata into result truncation notices and TUI warnings.\n\n`notebook` tool does **not** use `OutputSink`; it has no stream/artifact truncation pipeline because it does not execute code.\n\n## 7) Renderer assumptions and formatting\n\n## Notebook renderer (`notebookToolRenderer`)\n\n- call view: status line with action + notebook path + cell/type metadata\n- result view:\n - success summary derived from `details`\n - `cellSource` rendered via `renderCodeCell`\n - markdown cells set language hint `markdown`; other cells have no explicit language override\n - collapsed code preview limit is `PREVIEW_LIMITS.COLLAPSED_LINES * 2`\n - supports expanded mode via shared render options\n - uses render cache keyed by width + expanded state\n\nError rendering assumption:\n\n- if first text content starts with `Error:`, renderer formats as notebook error block.\n\n## Python renderer (for actual execution output)\n\nKernel-backed execution rendering expects:\n\n- per-cell status transitions (`pending/running/complete/error`)\n- optional structured status event section\n- optional JSON output trees\n- truncation warnings + optional `artifact://<id>` pointer\n\nThis renderer behavior is unrelated to `notebook` JSON editing results except that both reuse shared TUI primitives.\n\n## 8) Divergence from eval Python backend behavior\n\nIf \"plain Python execution\" means the `eval` tool with `language: \"python\"`:\n\n- `eval` executes code in a kernel, persists state by mode, streams chunks, captures rich displays, handles interrupts/timeouts, and supports output truncation/artifacts.\n- `notebook` performs deterministic notebook JSON mutations only; no execution, no kernel state, no chunk stream, no display outputs, no artifact pipeline.\n\nIf a workflow needs both:\n\n1. edit notebook source with `notebook`\n2. execute code cells via `eval` with `language: \"python\"` (manually passing code), not through `notebook`\n\nCurrent implementation does not provide a single tool that both mutates `.ipynb` and executes notebook cells through kernel context.\n",
55
55
  "notifications-sdk.md": "# Notifications SDK\n\n<p align=\"center\">\n <img src=\"../assets/telegram-mobile-hero.png\" alt=\"Sayknow-CLI mobile answers for coding agents hero illustration\" width=\"100%\" />\n</p>\n\nA small, transport-agnostic way to get **action-needed** signals out of a SKC\nsession and deliver **replies** back — without scraping the terminal and without\nthe depth of the RPC / Coordinator / Bridge surfaces.\n\nThe stable contract is deliberately generic: every running session exposes one\nloopback WebSocket endpoint, and integrations are user-written clients that\nconnect to that endpoint. Telegram, Discord, Slack, mobile apps, and local tools\nall use the same JSON protocol. No upstream Rust, N-API, or wire-protocol change\nis required for a new integration.\n\n> Status: the Rust core (`crates/skc-notifications`) provides the wire protocol,\n> action lifecycle, loopback WebSocket server, and endpoint discovery file. The\n> bundled Telegram daemon is a reference client layered on top of this SDK; it is\n> not the upstream topology.\n\n## Architecture\n\n```\nSKC session (upstream) your client (anywhere)\n┌───────────────────────────────┐ ┌──────────────────────────┐\n│ ask-tool fires / agent idle │ action_needed │ Telegram / Discord / ... │\n│ → notifications core │ ─────────────▶ │ render + collect reply │\n│ ws://127.0.0.1:<port> (+token) │ ◀───────────── │ │\n│ reply → resolve ask gate │ reply │ │\n└───────────────────────────────┘ └──────────────────────────┘\n```\n\n- **One endpoint per session.** Each session runs its own loopback WebSocket\n server. Upstream does not maintain a shared daemon, singleton, or\n chat-to-session registry; multiplexing many sessions into one integration is a\n client-side concern.\n- **Integrations are clients.** A client discovers endpoint files, connects to\n one or more WebSockets, renders `action_needed`, and sends `reply` messages.\n- **Zero upstream change.** New transports do not require changes to\n `crates/skc-notifications` or the JSON protocol.\n- **Off unless configured.** No endpoint exists unless notifications are enabled\n and a token is present.\n- **tmux-agnostic.** The endpoint behaves identically with or without tmux.\n\n### N-API action emission\n\n`NotificationServer#pushFrame(frameJson)` rejects (throws for) every `ActionNeeded`\nframe. Emit asks with `registerAsk(...)` and idle notifications with `noteIdle(...)`\ninstead, so action delivery remains capability-gated per connection.\n\n\n## Endpoint discovery\n\nA running session writes a discovery file at:\n\n```\n<repo>/.skc/state/notifications/<sessionId>.json\n```\n\n(`.skc/state/` is git-ignored.) Shape:\n\n```json\n{\n \"version\": 1,\n \"sessionId\": \"019edd41-...\",\n \"pid\": 12345,\n \"host\": \"127.0.0.1\",\n \"port\": 53124,\n \"url\": \"ws://127.0.0.1:53124\",\n \"token\": \"<per-session token>\",\n \"startedAt\": 1718760000000,\n \"updatedAt\": 1718760000000,\n \"stale\": false\n}\n```\n\n- The file is created `0700`/`0600` (unix) and written atomically.\n- The **token is in the file** because clients need it; never log it raw.\n Stale files (dead PID, past TTL, or explicitly marked) are cleaned up on the\n next start.\n\nConnect with the token as a query parameter:\n\n```\nws://127.0.0.1:<port>/?token=<token>\n```\n\nA wrong/missing token is rejected at the handshake with HTTP `401`.\n\n## Protocol\n\nJSON text frames. Field names are `camelCase`; the `type` discriminator is\n`snake_case`.\n\n### Server → client\n\n`action_needed` — something needs attention:\n\n```json\n{ \"type\": \"action_needed\", \"id\": \"wg_run_stage_1\", \"kind\": \"ask\",\n \"sessionId\": \"sess-1\", \"question\": \"Proceed?\", \"options\": [\"Yes\", \"No\"] }\n```\n\n```json\n{ \"type\": \"action_needed\", \"id\": \"idle-sess-1-7\", \"kind\": \"idle\",\n \"sessionId\": \"sess-1\", \"summary\": \"finished refactor; awaiting next step\" }\n```\n\n- `kind: \"ask\"` is answerable in both interactive/TUI and unattended/RPC modes.\n The `id` is the real workflow-gate id.\n- `kind: \"idle\"` is notify-only and ephemeral (not replayed to clients that\n connect later).\n\n`action_resolved` — a pending action is now terminal and **non-repliable**:\n\n```json\n{ \"type\": \"action_resolved\", \"id\": \"wg_run_stage_1\", \"resolvedBy\": \"local\" }\n```\n\n`resolvedBy` is `local` (answered in the CLI/TUI), `client` (a remote reply won),\nor `timeout`.\n\n`reply_rejected` — sent only to the client whose reply failed:\n\n```json\n{ \"type\": \"reply_rejected\", \"id\": \"wg_run_stage_1\", \"reason\": \"already_answered\" }\n```\n\nReasons: `already_answered`, `unknown_action`, `invalid_answer`,\n`resolver_unavailable`, `idempotency_conflict`, `unauthorized`.\n\n`action_unavailable` — a controlled ask cannot be presented on this connection:\n\n```json\n{ \"type\": \"action_unavailable\", \"id\": \"wg_run_stage_1\", \"sessionId\": \"sess-1\",\n \"reason\": \"missing_capability\", \"requiredCapabilities\": [\"ask_controls_v1\"] }\n```\n\nThis frame is diagnostic and non-actionable: do not render it as an ask or\nattempt a `reply` for it.\n\nThe minimal contract depends on the asks a client handles. A client that handles\nonly ordinary asks (those with empty `controls`) needs `action_needed`,\n`action_resolved`, and `reply_rejected`, and may omit `hello`. A client that\nwants controlled asks **MUST** send a ClientHello on socket open with\n`{ \"type\": \"hello\", \"protocolVersion\": 3, \"capabilities\": [\"ask_controls_v1\"] }`\nand handle `action_unavailable` as a non-actionable diagnostic. Threaded clients\n(like the managed Telegram daemon) may also receive optional server → client\nframes they can render or ignore: `identity_header` (one-time per-session\nrepo/branch/machine header), `context_update` (last message, task, goal, token\nusage, model, diff), `turn_stream` (live/finalized turn output),\n`image_attachment` (agent-produced images), `activity` (busy/idle, drives the\ntyping indicator), `inbound_ack` (delivery state of an injected user message),\n`session_closed` (endpoint teardown; threaded clients may delete/archive the\nremote conversation), `config_update` (current verbosity/redact), `hello`\n(server capability/version), and `pong`.\n\n### Client → server\n\n`reply` — answer a pending `ask`:\n\n```json\n{ \"type\": \"reply\", \"id\": \"wg_run_stage_1\", \"answer\": 0, \"token\": \"<token>\" }\n```\n\n`answer` accepts:\n\n- a number — zero-based option index (`0` = first option);\n- a string — an option label, or free text;\n- an object — `{ \"selected\": [0, \"Maybe\"], \"custom\": \"...\" }` for multi-select;\n- an object — `{ \"controlId\": \"navigation_forward\" }` to commit a controlled\n multi-select only when that navigation control was presented as enabled.\n\nOptional `idempotencyKey` makes retries safe: the same key + same body re-acks;\nthe same key + different body is rejected with `idempotency_conflict`.\n\nClients may also send optional client → server frames: `user_message` (inject/steer\na turn with free text), `config_command` (toggle verbosity/redact in-thread),\n`ping`, and `hello` (capability/version). A ClientHello is optional for ordinary\nempty-controls asks, but is required for controlled asks as specified below.\n\n## Answer semantics\n\nA remote reply answers a pending ask in **both** modes — RPC is not required:\n\n- **Interactive / TUI mode:** the ask tool races the local selector against the\n remote reply (first valid answer wins). If you tap a button in the client, the\n ask resolves with that option; if you answer locally, the client receives\n `action_resolved` (`resolvedBy: \"local\"`) and the action becomes non-repliable.\n- **Unattended / RPC mode:** the reply resolves the real workflow-gate, driving\n the session the same way a local answer would.\n\nIn both modes the first valid reply wins; later replies get `already_answered`.\nIdle pings are notify-only.\n\n### Controlled-ask capability negotiation (protocol v3)\n\nProtocol v3 adds optional `ask_controls_v1` and `ask_selected_ack_v1`\ncapabilities. A controlled ask carries both `options` and non-empty `controls`:\n\n```json\n{ \"type\": \"action_needed\", \"id\": \"wg_run_stage_1\", \"kind\": \"ask\",\n \"sessionId\": \"sess-1\", \"question\": \"Choose all that apply\",\n \"options\": [\"A\", \"B\"],\n \"controls\": [{ \"id\": \"navigation_forward\", \"kind\": \"navigation\", \"label\": \"Done\", \"enabled\": false }] }\n```\n\nControls are typed data. Clients must return\n`{\"controlId\":\"navigation_forward\"}` rather than infer a control from an\noption label.\n\nClients that want controlled asks **MUST** send this ClientHello immediately in\ntheir WebSocket `open` handler:\n\n```json\n{ \"type\": \"hello\", \"protocolVersion\": 3, \"capabilities\": [\"ask_controls_v1\"] }\n```\n\nThe server sends its ServerHello before it processes a ClientHello. A controlled\nask is deferred while the connection awaits that Hello. After the approximately\none-second Hello grace expires, or after a negotiated Hello lacks\n`ask_controls_v1`, the server sends exactly one non-actionable\n`action_unavailable`; it never sends a stripped `action_needed` frame with option\nbuttons. A later ClientHello that adds `ask_controls_v1` may upgrade that\nconnection to the full `action_needed` presentation.\n\nCapabilities are monotonic for one connection: repeated ClientHellos can add\ncapabilities but cannot remove an already-advertised capability or retract a\nfull presentation. Disconnecting and reconnecting starts a fresh negotiation\nwith no capabilities or delivery state. Ordinary asks with empty `controls`\nretain their existing immediate delivery and reply behavior for Hello-less and\nolder clients.\n\n### Protocol v3 semantic acknowledgement\n\nA remote reply is first **claimed** by the native server. The host retains the raw reply JSON, idempotency key, and reply receipt id, then resolves or closes that claim only after ask/gate semantic settlement. Invalid input terminally closes that interaction; a retry is a fresh action id, never a reopened claim.\n\nManaged Telegram clients may receive `ask_selected_ack_request` and return one correlated `ask_selected_ack_result`. Requests are `mode:\"live\"` (the current pending action on the authenticated session connection) or `mode:\"recovery\"` (the persisted session topic only). Recovery neither recreates an ask, keyboard, alias, nor reply route; a missing persisted topic is `failed/route_missing`. `ask_selected_ack_cancel` removes an exact queued request or aborts an in-flight attempt. The request/result/cancel correlation is `requestId` plus `commitKey` and is authorized by the authenticated WebSocket connection and endpoint generation; acknowledgement frames carry no second token.\n\nThe managed daemon sends the literal `Selected!` at most once per generation/commit key, on its `ask` rate-limit lane, with an absolute 8-second deadline and no application retry. A response is `delivered` only with Telegram `{ok:true,result.message_id:number}`; explicit rejection is `failed`, and timeout/network/in-flight abort uncertainty is `unknown`. Callback-query acknowledgement remains independent. The durable workflow answer is exactly-once; visible Telegram delivery is not.\n\nEvery accepted workflow-gate origin, including generic RPC/bridge winners, invokes canonical gate-presentation cleanup before continuation. Generic origins have no Telegram acknowledgement policy. Cancellation, shutdown, and endpoint replacement cancel retained interactions and acknowledgement work. A connection loss before dispatch is definitive `failed/session_closed`; after successful request-frame dispatch but before a correlated result it is `unknown/origin_disconnected`—the host does not infer whether Telegram was called.\n\n## Minimal client example\n\nThis copyable example advertises `ask_controls_v1`, so it can receive and\ncomplete controlled asks. An ordinary client that handles only empty-controls\nasks omits the `open` handler and ClientHello, and keeps only the numeric\nordinary-ask branch below.\n\n```js\nimport { readFileSync } from \"node:fs\";\nimport WebSocket from \"ws\";\n\nconst { url, token } = JSON.parse(\n readFileSync(`.skc/state/notifications/${sessionId}.json`, \"utf8\"),\n);\n\nconst ws = new WebSocket(`${url}/?token=${encodeURIComponent(token)}`);\nlet currentControlledAsk;\n\nfunction sendOptionReply(id, index) {\n ws.send(JSON.stringify({ type: \"reply\", id, answer: index, token }));\n}\n\nfunction commitControlledMultiSelect() {\n const ask = currentControlledAsk;\n const navigation = ask?.controls?.find(\n (control) =>\n control.id === \"navigation_forward\" &&\n control.kind === \"navigation\" &&\n control.enabled,\n );\n if (!navigation) return;\n\n ws.send(JSON.stringify({\n type: \"reply\",\n id: ask.id,\n answer: { controlId: \"navigation_forward\" },\n token,\n }));\n}\n\nws.on(\"open\", () => {\n if (ws.readyState === WebSocket.OPEN) {\n ws.send(JSON.stringify({\n type: \"hello\",\n protocolVersion: 3,\n capabilities: [\"ask_controls_v1\"],\n }));\n }\n});\n\nws.on(\"message\", (data) => {\n const msg = JSON.parse(data.toString());\n if (msg.type === \"action_needed\" && msg.kind === \"ask\") {\n if (msg.controls?.length) {\n currentControlledAsk = msg;\n // Present msg.question / msg.options / msg.controls to the human.\n // An option click calls sendOptionReply(msg.id, index). For a controlled\n // multi-select, that numeric reply only toggles and is non-terminal.\n // Wait for the fresh action_needed frame, then call\n // commitControlledMultiSelect() only from its enabled navigation control.\n return;\n }\n\n // Ordinary ask (`controls: []`): the minimal no-Hello client replies normally.\n sendOptionReply(msg.id, 0);\n } else if (msg.type === \"action_resolved\") {\n if (currentControlledAsk?.id === msg.id) currentControlledAsk = undefined;\n // Mark this action as no longer answerable in your UI.\n } else if (msg.type === \"reply_rejected\") {\n // e.g. reason === \"already_answered\" → the ask was answered elsewhere.\n } else if (msg.type === \"action_unavailable\") {\n // Diagnostic only: do not fabricate an ask or option buttons.\n }\n});\n```\n\n`sendOptionReply(msg.id, index)` has the numeric option/toggle shape\n`{ type:\"reply\", id, answer:index, token }`. The enabled navigation control\nuses the distinct typed commit shape\n`{ type:\"reply\", id, answer:{ controlId:\"navigation_forward\" }, token }`.\nDo not send that commit against the pre-toggle action id: a multi-select option\ntoggle reissues the action, and the enabled navigation control on the latest\nframe commits the selection.\n\nSwap `ws` for a Telegram bot's long-poll loop, a Discord gateway client, or a\nSlack socket-mode app — the contract above is all you implement.\n\n## Managed notification adapters\n\nFor the exact user setup flow (`skc notify setup`, BotFather token, private-chat pairing, status, and troubleshooting), see [Telegram notification onboarding](./telegram-onboarding.md).\n\n## Managed Telegram daemon (bundled reference client)\n\nSKC also ships a managed Telegram reference client for the common phone-notify\nworkflow. It remains a client of the generic SDK: it scans session discovery\nfiles, opens each session WebSocket, and routes Telegram replies back to the\nmatching endpoint.\n\nThe daemon/session engine is shared. Session discovery, WebSocket protocol,\nredaction decisions, rate-limit pooling, reply routing, singleton ownership, and\nlifecycle control are not reimplemented by each chat surface. Telegram, Discord,\nand Slack adapters are thin presentation layers: they render internal notification\nevents into transport payloads and map transport interactions back to `{sessionId,\nactionId,answer}` replies.\n\n### Setup and auto-connect\n\nRun the setup command once:\n\n```sh\nskc notify setup\n```\n\nThe wizard validates the bot token with Telegram, verifies private-chat Threaded\nMode capability via `getMe.has_topics_enabled`, waits for a private DM to the bot,\nand writes canonical global Settings under `config.yml` in the SKC agent\ndirectory. It enables:\n\n- `notifications.enabled`\n- `notifications.telegram.botToken`\n- `notifications.telegram.chatId`\n- `notifications.redact` (optional; default false)\n- `notifications.discord.botToken` / `notifications.discord.channelId` (optional Discord adapter)\n- `notifications.slack.botToken` / `notifications.slack.channelId` (optional Slack adapter)\n\nAfter setup, sessions auto-connect when notifications are enabled. Each session\nstill publishes its own loopback endpoint; the daemon is only the Telegram-side\nmultiplexer.\n\nFor Telegram forum topics, the daemon deletes the per-session topic when the local\nnotification endpoint shuts down, so it disappears from the topic list. A resumed\nsession creates a fresh topic before sending again. The bot must be allowed to\ndelete messages in that chat; without that permission, deletion is best-effort and\ndelivery continues.\n\nPer-session topics are titled `{repo}/{branch} - {title}` by default (falling back\nto `{repo}/{branch}`, then the session title, then `SKC <id>` when identity is\nmissing). Set `notifications.telegram.topics.nameTemplate` to reorder or reshape\nthat title — for example `\"{title} · {repo}/{branch}\"` puts the session title\nfirst so concurrent sessions on the same checkout stay distinguishable in the\nsidebar. The template supports the `{repo}`, `{branch}`, and `{title}`\nplaceholders; unknown placeholders are left verbatim. The template is applied\nonly when every placeholder it references has a value for that session — otherwise\nthe daemon falls back to the built-in composition, so provisional and identity-less\ntopics never render with dangling separators. Leaving the setting unset preserves\nthe default naming exactly.\n\n### Singleton poller and trust model\n\nTelegram `getUpdates` allows only one active long-poll owner per bot token. The\nmanaged daemon enforces **one bot token = one getUpdates poller** with a local\nlock/state file under the agent directory. New sessions attach to the existing\nfresh daemon owner instead of starting another poller, preventing Telegram 409\nconflicts.\n\nThe trust model is intentionally strict:\n\n- setup pairs exactly one private Telegram chat;\n- runtime accepts updates only from that paired chat id;\n- groups, supergroups, channels, and unpaired users never receive session names,\n action ids, pending status, or configuration hints;\n- daemon state stores a token fingerprint, not the raw bot token.\n\n### Routing in private-chat topics\n\nThe paired private chat prefers per-session Telegram topics (Threaded Mode). The\ndaemon tags messages by session, stores compact callback aliases for inline\nbuttons, and routes replies back to the exact session/action. A forum-enabled\nsupergroup is no longer required: when the bot owner enables Threaded Mode in\n@BotFather, the daemon creates one topic per session in the paired private chat.\nSKC cannot enable Threaded Mode through the Bot API; setup only verifies the\ncapability and guides the manual BotFather toggle.\n\nIf BotFather's per-bot **Bot Settings** menu does not show **Threads Settings**\nor **Threaded Mode**, the supported fallback is the normal private-chat pairing.\nSetup can be saved as `threaded=unverified`/`threaded=unknown`, and the daemon\nstill tries topics when Telegram allows them. When `createForumTopic` is refused,\nthe daemon does not drop the send: it routes the notification to the normal\n(flat) paired private chat and posts a one-time nudge: `Flat Telegram private chat\nsupports outbound notifications and inline ask buttons only. Enable Threaded Mode\nin @BotFather > Bot Settings > Threads Settings for free-text replies and session\ncommands.` Pairing is private-only, so flat delivery stays within the user's own\nprivate DM.\n\nSupported reply paths:\n\n- tap an inline button on an ask notification;\n- reply inside the session's thread/topic (replies are thread-native; the\n topic identifies the session, so no session tag is needed).\n\nIn threaded mode the user can also adjust per-session behaviour with in-thread\nconfig commands: `/verbose`, `/lean`, `/verbosity <lean|verbose>`, and\n`/redact <on|off>`. The legacy `/answer <session-tag> <answer>` command is\nremoved — replies are routed by the topic they arrive in.\n\nFlat fallback keeps outbound notifications and inline-button answers working, but\nplain free-text never guesses from the global pending-ask set. Free-text replies\nand `/verbose`/`/lean`/`/verbosity`/`/redact` commands are thread-native and\nrequire Threaded Mode/topic routing. Enable Threaded Mode in @BotFather > Bot\nSettings > Threads Settings when you need free-text replies or session commands.\nDo not pair a group, supergroup, or channel to work around a missing BotFather\nmenu; the bundled setup flow is\nprivate-chat only, and non-private chat ids remain fail-closed to avoid session\ndata leaks.\n\nUnknown, expired, or restart-unvalidated callback aliases fail closed: the daemon\nsends guidance and does not guess a target session or action.\n\n### Discord and Slack setup\n\nDiscord and Slack use the same internal notification events and reply protocol as\nTelegram. Store only runtime credentials in local SKC settings or environment;\nnever paste bot tokens, webhook URLs, transcripts, prompts, host paths, or raw logs\ninto docs, tests, issues, or PR comments.\n\nConfiguration keys:\n\n```yaml\nnotifications:\n enabled: true\n discord:\n botToken: \"<local Discord bot token>\"\n channelId: \"<Discord channel id>\"\n slack:\n botToken: \"<local Slack bot token>\"\n channelId: \"<Slack channel id>\"\n redact: true\n```\n\nThe bundled adapters intentionally render public-safe message bodies and return\nroute metadata only for pending internal actions. They do not own polling,\nsession scans, daemon locks, rate limits, or SDK lifecycle. Production transport\nsenders should consume the adapter payloads and keep all credential-bearing HTTP\nor gateway details outside logged payloads.\n### Redaction\n\n`notifications.redact` strips sensitive content before remote delivery, but\n**asks are exempt**: an ask is an interactive prompt the human must read and\nanswer remotely, so its `question` and `options` are always sent unredacted\n(otherwise it would be unanswerable). When redaction is enabled, `idle`\nsummaries are removed and streamed content frames (`turn_stream`,\n`context_update`, `image_attachment`) are suppressed at their emit sites. When\nredaction is disabled, all content is delivered unchanged.\n\n### Local `/notify`\n\nInside a SKC session, `/notify` controls the current session only:\n\n- `/notify status` reports enabled/disabled state, daemon observation when known,\n and redaction state without printing secrets;\n- `/notify off` disables the current session's notification endpoint and removes\n its discovery record without mutating global Settings;\n- `/notify on` re-enables the current session when global setup is complete and\n `SKC_NOTIFICATIONS=0` is not forcing opt-out.\n\n### Manual Telegram CLI is for debugging\n\n`packages/coding-agent/src/notifications/telegram-cli.ts` remains as a manual\nreference/debug client and template for other integrations. It is not the primary\nTelegram UX.\n\n```sh\nbun run packages/coding-agent/src/notifications/telegram-cli.ts --bot-token \"$BOT_TOKEN\"\n```\n\nBy default it refuses to start when a fresh managed daemon already owns the same\nbot token for the same paired chat, because a second poller will cause Telegram\n409 conflicts. Use `--force` only for deliberate debugging when you have stopped\nor intentionally want to override the daemon guard.\n## Two client surfaces: per-session vs daemon-owned lifecycle control\n\nThe SDK now exposes **two distinct surfaces**. Do not confuse them:\n\n1. **Per-session notification clients (the normal, documented contract above).**\n A client discovers `<repo>/.skc/state/notifications/<sessionId>.json`, connects\n to that session's loopback WebSocket, and handles `action_needed`,\n `action_resolved`, `reply_rejected`, and the optional threaded frames. This is\n all an ordinary integration (Telegram, Discord, Slack, mobile, local tools)\n needs. It requires **zero** upstream changes.\n\n2. **The daemon-owned session *lifecycle* control endpoint (privileged).**\n A separate, **session-independent**, loopback-only, authenticated control\n endpoint that accepts `session_create` / `session_close` / `session_resume`\n frames. It exists because creating a session cannot use a per-session socket\n (none exists before the session does). It is **not** part of the normal\n integration contract: ordinary clients never implement it. Only the bundled,\n trusted daemon (e.g. the managed Telegram daemon) speaks it.\n\n### Lifecycle control endpoint\n\n- **Discovery:** `<agentDir>/notifications/control.json` (daemon-owned, mode\n `0600`), distinct from per-session endpoint files. It carries only non-secret\n endpoint metadata (url/host/port/pid/owner). The control token is held **in\n memory** by the daemon (the sole client) and is **never** written to disk.\n- **Auth:** loopback-only bind (a non-loopback bind is refused). The WebSocket\n upgrade requires `?token=<control-token>` (HTTP `401` otherwise), and every\n lifecycle frame's `token` is re-checked (`unauthorized` on mismatch). The Rust\n ingress authenticates and forwards; it never spawns or applies policy.\n- **Frames:** `session_create` (target `existing_path` | `worktree` |\n `plain_dir`), `session_close` (hard-kill, history preserved, recoverable),\n `session_resume` (reattach if alive, else cold-restart from history); responses\n `session_create_response` / `session_close_response` / `session_resume_response`\n / `session_lifecycle_error`. The protocol also defines a replayable\n `session_ready` per-session frame for readiness-gated creates; the current MVP\n daemon replies once the tmux launch is requested (see the phone guide) rather\n than waiting on it. Inline prompt text (`-- <prompt>`) is rejected in the MVP.\n\n### Trust model and hardening (daemon side)\n\nThe control endpoint trusts the configured paired chat for any path (an accepted\nrisk). It is hardened around that boundary:\n\n- **Strict paired-chat gating** — non-paired chats are rejected *before* any path\n parsing, filesystem, or process action.\n- **Durable idempotency** — a locked, atomic, fsynced ledger keyed by\n `chatId:updateId` + request hash (`telegram-lifecycle-idempotency.json`).\n Duplicate updates never repeat side effects, including across daemon restart; a\n duplicate while in-progress reports pending (never a second spawn); a same id\n with a different body is `duplicate_conflict`; an effect failure is recorded\n `terminal_uncertain` (never auto-respawned).\n- **Per-chat create rate limit.**\n- **Audit log** — append-only `telegram-lifecycle-audit.jsonl` (`0600`) recording\n every accept/reject/duplicate/rate-limit/spawn/success/failure. Raw control\n tokens and raw prompts are never logged (prompt hash + byte length only).\n- **Inline prompts rejected (MVP)** — `session_create` with `-- <prompt>` text is\n rejected with usage; no prompt is ever placed in argv, audit, or responses. (A\n redacted prompt-ref flow is reserved for a future revision.)\n- **SKC-managed-only close** — force-close re-reads the exact `@skc-profile`\n immediately before kill and requires the `@skc-session-id` (and optional\n `@skc-session-state-file`) tag to match; it never touches non-SKC tmux.\n- **Recent-activity picker** — sessions are ranked by history-file mtime and\n enriched with terminal breadcrumbs so the operator picks a recent repo/session\n instead of typing raw paths. Ambiguous resumes fail closed with candidates.\n### Phone test guide (create / close / resume from Telegram)\n\nEnd-to-end manual check once `skc notify setup` has paired your private chat:\n\n1. **Pair + start.** Run `skc notify setup` (BotFather token, DM the bot to pair).\n Start any SKC session with notifications enabled so the daemon owner is\n running (`skc launch` in a repo, or `SKC_NOTIFICATIONS=1`). The owner starts\n the loopback control endpoint and accepts `/session_*` while running; with zero\n active sessions it still idle-exits after the inactivity timeout.\n2. **Create.** From your paired chat, pick `/session_create` from the Telegram\n command menu or send `/session_create path <repo-dir>` (or\n `/session_create worktree <repo> <branch>`, or `/session_create dir <newdir>`).\n `<repo-dir>`, `<repo>`, and `<newdir>` may use `~`/`~/...` for your own home\n directory; named-user forms such as `~alice/repo` are rejected. The bot replies\n once the tmux launch is requested; the session shows up in `/session_recent`\n once it is ready. (Inline prompts via `-- <text>` are rejected for now with\n usage text.)\n3. **List.** `/session_recent` shows recent sessions (most-recent first) to copy\n an id from.\n4. **Close.** `/session_close <sessionId>` hard-kills the SKC-managed session\n (history is preserved); the bot confirms.\n5. **Resume.** `/session_resume <sessionId|prefix>` reattaches if it is still\n alive, otherwise cold-restarts it from saved history. An ambiguous prefix\n replies with the matching candidates instead of guessing.\n\nCommands are accepted **only** from the paired chat; **create** is rate-limited,\nand all lifecycle commands are idempotent per Telegram update id and audited (no\ntokens or prompts are logged).\nFor an automated proof of the wire path without a real bot, see\n`packages/coding-agent/scripts/g011-daemon-path-smoke.ts` (real native control\nendpoint + loopback WebSocket).\n",
56
56
  "onboarding-packet.md": "# Sayknow-CLI Onboarding Packet\n\nThis packet is a docs-only, public-safe context seed for the `sayknow-cli` repository as inspected on 2026-06-01. It is intentionally a no-new-skill artifact: not a new workflow skill, command, agent, configuration surface, issue template, or runtime behavior.\n\n## Purpose in one paragraph\n\nSayknow-CLI is the `skc` coding-agent CLI and supporting monorepo. The product centers on a small public workflow loop: clarify with `deep-interview`, plan with `ralplan`, execute and verify through `ultragoal`, and use `team` only when parallel tmux workers are useful. The main product package is `packages/coding-agent/`; supporting packages provide LLM/provider access, agent runtime, TUI rendering, native helpers, stats, utilities, benchmarks, and SDK machine interfaces.\n\n## Fixed public surface\n\nKeep this invariant front-and-center when onboarding to the repo:\n\n- Default workflow skills: `deep-interview`, `ralplan`, `team`, `ultragoal`.\n- Public role agents: `executor`, `architect`, `planner`, `critic`.\n- Bundled default workflow skill sources live under `packages/coding-agent/src/defaults/skc/skills/`.\n- Bundled role-agent prompt sources live under `packages/coding-agent/src/prompts/agents/`.\n- Runtime state, specs, plans, goals, team state, and local overrides belong under `.skc/` for the product and `.omx/` only for this agent-run orchestration.\n\nDo not add a fifth default skill, fifth public role agent, new command, new config surface, or feature-intake behavior unless that product decision has already been made and the default-surface gates are updated.\n\n## Primary entrypoints\n\n| Area | Repo-relative path | Why it matters |\n| ---------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------- |\n| CLI bootstrap | `packages/coding-agent/src/cli.ts` | Registers top-level CLI commands and routes default launch behavior. |\n| Session launch | `packages/coding-agent/src/main.ts` | Converts CLI/runtime settings into agent-session creation and mode dispatch. |\n| Agent assembly | `packages/coding-agent/src/sdk/session.ts` | Loads settings, default skills, rules, tools, auth/model state, system prompt, and agent runtime. |\n| Built-in tools | `packages/coding-agent/src/tools/index.ts` | Registers file, shell, edit, search, browser, task/subagent, and related public coding-harness tools. Memory backends are private integrations, not public tools. |\n| Default skills | `packages/coding-agent/src/defaults/skc-defaults.ts` | Embeds and installs the four default workflow skills plus deep-interview fragments. |\n| Role agents | `packages/coding-agent/src/task/agents.ts` | Embeds bundled task-agent prompts; tests enforce public role-agent expectations. |\n| Product overview | `README.md` | Explains installation, product story, fixed workflow surface, and development entry commands. |\n| Architecture map | `docs/codebase-overview.md` | Public package map and runtime-flow reference. |\n\n## Package map\n\n- `packages/coding-agent/` — main `skc` CLI, workflows, session runtime, tool registry, discovery, settings, prompts, and tests.\n- `packages/ai/` — provider/model boundary, streaming, auth, model registry, retries, and provider integrations.\n- `packages/agent/` — stateful agent loop and append-only context runtime.\n- `packages/tui/` — terminal UI framework and rendering primitives.\n- `packages/natives/` plus `crates/*` — native helpers, Rust/N-API bindings, shell/PTY, text search, AST, filesystem, and media utilities.\n- `packages/utils/` — shared TypeScript utilities, logging, formatting, process helpers, JSON/frontmatter, and sanitization.\n- `packages/stats/` — local observability dashboard and session/model usage aggregation.\n- `packages/typescript-edit-benchmark/` — TypeScript edit benchmark tooling.\n- External machine clients use the SDK WebSocket interface documented in `docs/sdk.md`; `--mode rpc`, `--mode rpc-ui`, and `--mode bridge` were removed.\n\n## Build, test, and validation commands\n\nPrefer targeted checks first, then broader checks when code changes justify them. For this docs-only packet, lightweight validation is enough.\n\n| Command | Scope | When to use |\n| --------------------------------------------------------------------- | ----------------------------------- | --------------------------------------------------------------------------- |\n| `bun install` | Workspace dependencies | Initial local setup. |\n| `bun run install:defaults` | Local default install | Installs source-bundled default workflow definitions for local development. |\n| `bun packages/coding-agent/src/cli.ts --help` | CLI smoke/discovery | Fast source checkout CLI sanity check. |\n| `bun run check:ts` | Type/lint/default UI checks | Broad TypeScript validation; heavier than docs-only changes. |\n| `bun run test` | Full TS + Rust tests | Broad regression check; use for runtime/product changes. |\n| `bun run ci:test:smoke` | CLI version/help/stats worker smoke | Useful before release/install changes. |\n| `bun scripts/check-visible-definitions.ts` | Default surface gate | Required after workflow-definition changes. |\n| `bun scripts/verify-g002-gates.ts` | Rebrand/default-surface gate | Required after workflow-definition or public-surface changes. |\n| `bun scripts/rebrand-inventory.ts --strict` | Rebrand inventory gate | Required after workflow-definition or public-surface changes. |\n| `bun test packages/coding-agent/test/default-skc-definitions.test.ts` | Four-skills/four-agents contract | Required after default workflow/agent surface changes. |\n\nRepository rule: do not run `tsc` or `npx tsc`; use the Bun scripts above.\n\n## Danger zones\n\n- **Default surface expansion:** `packages/coding-agent/src/defaults/skc/skills/`, `packages/coding-agent/src/defaults/skc-defaults.ts`, `packages/coding-agent/src/prompts/agents/`, and model-assignment tests are contract-heavy. Changes here can accidentally alter the fixed four-skills/four-agents shape.\n- **CLI commands:** `packages/coding-agent/src/cli.ts` and `packages/coding-agent/src/commands/` define visible behavior. Adding commands or aliases is a product-surface change.\n- **Runtime/session assembly:** `packages/coding-agent/src/main.ts`, `packages/coding-agent/src/sdk/session.ts`, discovery, settings, tools, and system-prompt paths can affect every session.\n- **TUI/logging:** Avoid `console.log`, `console.warn`, or `console.error` inside `packages/coding-agent/`; use the centralized logger to avoid corrupting TUI rendering.\n- **Secrets/auth/config:** Keep `docs/secrets.md`, auth broker/gateway code, settings, and environment-variable docs public-safe. Do not expose tokens or private infrastructure.\n- **Native/Rust build:** `packages/natives/` and `crates/*` can require platform-specific toolchains and CI artifact behavior.\n- **Generated model data:** Do not edit `packages/ai/src/models.json` directly; update generators/descriptors/resolvers and regenerate with `bun --cwd=packages/ai run generate-models`.\n\n## Unknowns worth preserving\n\n- Which onboarding packet shape will be most useful for future `skc` context ingestion is still an experiment, not a product contract.\n- Public issue #158 / `sayknow-deep-onboarding` context is summarized only from the user-provided prompt in this run; this packet does not add issue intake or feature workflow behavior.\n- Full CI may depend on runner/system dependencies and native artifacts; docs-only changes usually do not need the full matrix locally.\n- Some packages contain internal or hidden utility prompts/agents beyond the four public role agents. Public-facing docs should keep the four-role contract clear.\n\n## First safe tasks for a new contributor or agent\n\n1. Read `README.md`, `docs/codebase-overview.md`, and this packet.\n2. Run `bun packages/coding-agent/src/cli.ts --help` for a fast CLI surface check after dependencies are installed.\n3. For docs-only edits, run formatting/check commands that do not mutate runtime behavior.\n4. For default-surface edits, run the four required gates listed in the command table before claiming completion.\n5. For package code edits, start with the nearest package test, then escalate to `bun run check:ts` or `bun run test` as risk increases.\n6. Before changing `packages/coding-agent/src/defaults/skc/skills/`, `packages/coding-agent/src/prompts/agents/`, `packages/coding-agent/src/commands/`, or config/settings paths, write down whether the change alters public surface area.\n\n## Context seed checklist\n\nA future agent can use this packet as context if it preserves these constraints:\n\n- Keep changes public-safe and repo-relative.\n- Prefer docs and tests over new runtime abstractions for onboarding experiments.\n- Treat the fixed four-skills/four-agents shape as a product constraint.\n- Verify claims with repo files before summarizing them.\n- Report validation evidence and caveats instead of implying hidden automation.\n",
@@ -8,7 +8,7 @@
8
8
  import { type AssistantMessage, type ImageContent, isContextOverflow } from "@sayknow-cli/ai";
9
9
  import { isKnownSinkPeerClosedError, logger, sanitizeText } from "@sayknow-cli/utils";
10
10
  import type { AgentSession } from "../session/agent-session";
11
- import { isSilentAbort } from "../session/messages";
11
+ import { getVisibleAnswerChain, isSilentAbort } from "../session/messages";
12
12
  import { initializeExtensions } from "./runtime-init";
13
13
 
14
14
  /**
@@ -267,9 +267,19 @@ export async function runPrintMode(session: AgentSession, options: PrintModeOpti
267
267
  }
268
268
 
269
269
  if (printContent) {
270
- for (const content of assistantMsg.content) {
271
- if (content.type === "text") stdout.write(`${sanitizeText(content.text)}\n`);
270
+ // A continued answer prints its preserved prefix too, joined at the seam
271
+ // exactly as it streamed (the continuation resumes mid-line).
272
+ const lines: string[] = [];
273
+ for (const [partIndex, part] of getVisibleAnswerChain(session.state.messages, assistantMsg).entries()) {
274
+ let first = true;
275
+ for (const content of part.content) {
276
+ if (content.type !== "text") continue;
277
+ if (first && partIndex > 0 && lines.length > 0) lines[lines.length - 1] += content.text;
278
+ else lines.push(content.text);
279
+ first = false;
280
+ }
272
281
  }
282
+ for (const line of lines) stdout.write(`${sanitizeText(line)}\n`);
273
283
  }
274
284
  }
275
285
  }
@@ -44,6 +44,7 @@ import {
44
44
  type WorkflowGateTerminalProof,
45
45
  } from "../../modes/shared/agent-wire/workflow-gate-broker";
46
46
  import type { AgentSessionEvent } from "../../session/agent-session";
47
+ import { lastVisibleAnswerText } from "../../session/messages";
47
48
  import { parseThinkingLevel } from "../../thinking";
48
49
  import type {
49
50
  AskAnswerRequest,
@@ -1838,22 +1839,8 @@ function sdkQuerySurface(
1838
1839
  cwd: ctx.cwd,
1839
1840
  kind: ctx.sessionMetadata?.kind ?? "main",
1840
1841
  });
1841
- const lastAssistantText = () => {
1842
- for (const entry of ctx.sessionManager.getBranch().toReversed()) {
1843
- if (entry.type !== "message" || entry.message.role !== "assistant") continue;
1844
- const { content } = entry.message;
1845
- if (typeof content === "string") return content;
1846
- if (Array.isArray(content))
1847
- return content
1848
- .filter(
1849
- (block): block is { type: "text"; text: string } =>
1850
- block.type === "text" && typeof block.text === "string",
1851
- )
1852
- .map(block => block.text)
1853
- .join("");
1854
- }
1855
- return undefined;
1856
- };
1842
+ // A continued answer includes its preserved prefix (the same chain print and /copy use).
1843
+ const lastAssistantText = () => lastVisibleAnswerText(ctx.sessionManager.getBranch());
1857
1844
  const getDiff = async () => {
1858
1845
  try {
1859
1846
  const { stdout } = await execFileAsync("git", ["diff", "--no-ext-diff"], {
@@ -4898,6 +4898,17 @@
4898
4898
  "packages/coding-agent/test/sdk-operation-inventory.test.ts"
4899
4899
  ]
4900
4900
  },
4901
+ {
4902
+ "sourceId": "agent_session:getVisibleAnswerChain",
4903
+ "sourceFile": "packages/coding-agent/src/session/agent-session.ts",
4904
+ "sourceKind": "agent_session",
4905
+ "decision": "exclude",
4906
+ "rationale": "internal accessor/plumbing, not a user-facing control seam",
4907
+ "exclusionMetadata": {
4908
+ "adapterMappings": "not_applicable",
4909
+ "testIds": "not_applicable"
4910
+ }
4911
+ },
4901
4912
  {
4902
4913
  "sourceId": "agent_session:hasCopyCandidateAssistantMessage",
4903
4914
  "sourceFile": "packages/coding-agent/src/session/agent-session.ts",
@@ -111,6 +111,7 @@ import {
111
111
  import { AgentSession, type ForkContextSeed } from "../session/agent-session";
112
112
  import type { AuthStorage } from "../session/auth-storage";
113
113
  import { applyCredentialRankingModeSetting, discoverAuthStorage } from "../session/auth-storage-discovery";
114
+ import { SYNTHESIZED_ON_RESUME_DETAIL } from "../session/auto-resume";
114
115
  import { type CustomMessage, convertToLlm } from "../session/messages";
115
116
  import type { DetectedLanguage } from "../session/response-language";
116
117
  import { createReadonlySessionManager, SessionManager } from "../session/session-manager";
@@ -266,7 +267,7 @@ export function reconcileTrailingToolCalls(messages: AgentMessage[]): AgentMessa
266
267
  text: "Tool result was not persisted before the previous turn ended; synthesized on resume to keep tool_use/tool_result pairing valid.",
267
268
  },
268
269
  ],
269
- details: {},
270
+ details: { [SYNTHESIZED_ON_RESUME_DETAIL]: true },
270
271
  isError: false,
271
272
  timestamp: now,
272
273
  }));
@@ -2534,10 +2535,15 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}
2534
2535
  shouldPause: options.shouldPause,
2535
2536
  preferWebsockets: preferOpenAICodexWebsockets,
2536
2537
  getToolContext: tc => toolContextStore.getContext(tc),
2537
- getApiKey: async provider => {
2538
+ getApiKey: async (provider, requestOptions) => {
2538
2539
  // Read agent.sessionId at call time so credential selection stays aligned
2539
2540
  // with metadataResolver after /new, fork, resume, or branch switches.
2540
- const key = await modelRegistry.getApiKeyForProvider(provider, agent.providerSessionId ?? agent.sessionId);
2541
+ const key = await modelRegistry.getApiKeyForProvider(
2542
+ provider,
2543
+ agent.providerSessionId ?? agent.sessionId,
2544
+ undefined,
2545
+ requestOptions,
2546
+ );
2541
2547
  if (!key) {
2542
2548
  throw new Error(`No API key found for provider "${provider}"`);
2543
2549
  }
@@ -2549,15 +2555,22 @@ export async function createAgentSession(options: CreateAgentSessionOptions = {}
2549
2555
  streamSimple(streamModel, context, {
2550
2556
  ...streamOptions,
2551
2557
  onAuthError: async (provider, oldKey, error) => {
2552
- await modelRegistry.authStorage.invalidateCredentialMatching(provider, oldKey, {
2558
+ // Invalidation (a broker forced refresh) and the re-fetch share the step's admission.
2559
+ const credentialOptions = {
2553
2560
  signal: streamOptions?.signal,
2561
+ fetch: streamOptions?.credentialFetch,
2562
+ onUpstreamRequest: streamOptions?.onUpstreamRequest,
2563
+ onUncertainUpstream: streamOptions?.onUncertainUpstream,
2564
+ };
2565
+ await modelRegistry.authStorage.invalidateCredentialMatching(provider, oldKey, {
2566
+ ...credentialOptions,
2554
2567
  sessionId: agent.sessionId,
2555
2568
  });
2556
2569
  logger.debug("Retrying provider request after credential invalidation", {
2557
2570
  provider,
2558
2571
  error: error instanceof Error ? error.message : String(error),
2559
2572
  });
2560
- return modelRegistry.getApiKeyForProvider(provider, agent.sessionId);
2573
+ return modelRegistry.getApiKeyForProvider(provider, agent.sessionId, undefined, credentialOptions);
2561
2574
  },
2562
2575
  }),
2563
2576
  cursorExecHandlers,