durable-chat 0.0.0-stage → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,795 @@
1
- # Temporary Holding Version
1
+ # durable-chat
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ A small terminal coordinator for durable pi agents.
4
+
5
+ One main conversation stays available to you. It delegates longer work to **background jobs**: separate
6
+ persisted agent conversations with a minimal covering tool set and the Substrate MCP gateway. Jobs keep
7
+ running while you keep chatting, survive process restarts, and can be inspected, continued, compacted,
8
+ reset, or cancelled. A job can hand its remaining work to a successor job or back to the coordinator.
9
+
10
+ Built on `@earendil-works/pi-durable` 1.5.0 (SQLite storage), `pi-ai`, `pi-schema`, and `pi-mcp`.
11
+
12
+ Maintainer overview: [docs/HANDOFF.md](docs/HANDOFF.md).
13
+ Release process: [docs/RELEASING.md](docs/RELEASING.md).
14
+
15
+ ## Install and run
16
+
17
+ Requires Node >= 22.19 (developed on Node 26).
18
+
19
+ Install from [npm](https://www.npmjs.com/package/durable-chat):
20
+
21
+ ```sh
22
+ npm install -g durable-chat
23
+ dchat help
24
+ dchat --provider faux --no-mcp ask "hello" # offline; no model account required
25
+ ```
26
+
27
+ Alternatively, install the reviewed archive from the [GitHub release](https://github.com/thorinside/durable-chat/releases):
28
+
29
+ ```sh
30
+ npm install --global https://github.com/thorinside/durable-chat/releases/download/v1.0.0/durable-chat-1.0.0.tgz
31
+ ```
32
+
33
+ For a source checkout:
34
+
35
+ ```sh
36
+ git clone https://github.com/thorinside/durable-chat.git
37
+ cd durable-chat
38
+ npm ci --ignore-scripts
39
+ npm run build
40
+ npm link --ignore-scripts
41
+ ```
42
+
43
+ Then configure a provider before interactive use:
44
+
45
+ ```sh
46
+ dchat config init # only if ~/.config/durable-chat/config.yaml is missing
47
+ dchat config set baseUrl http://127.0.0.1:1234/v1 # your local inference server
48
+ dchat # full-screen TUI
49
+ dchat --plain # line-oriented chat
50
+ ```
51
+
52
+ Published packages contain built JavaScript, not TypeScript entrypoints. Node refuses native
53
+ TypeScript stripping inside `node_modules`, so a working source checkout or `npm link` alone does
54
+ not verify an npm release. No compiler is required to run an installed release.
55
+ Git is required for managed worktrees; a compatible terminal is required for the interactive TUI.
56
+
57
+ Defaults live in `~/.config/durable-chat/config.yaml` (or `$XDG_CONFIG_HOME/durable-chat/config.yaml`).
58
+ The generated file points the store at `~/.local/share/durable-chat/store.sqlite` and selects local
59
+ inference. One process owns a store; a lock file (`<store>.lock`) enforces that and reclaims a lock left
60
+ by a dead pid.
61
+
62
+ On macOS in Ghostty, an interactive `dchat`, `dchat tui`, or `dchat chat` launch for an already-open
63
+ store focuses its existing terminal (including its tab/split) and exits successfully. The owner is
64
+ matched by PID or TTY; the single-writer lock stays in place. Batch commands and SSH sessions retain
65
+ the normal lock error. If Ghostty cannot focus the terminal, or Automation permission is denied,
66
+ the original error is shown. This uses Ghostty's `macos-applescript` support (enabled by default);
67
+ macOS may ask for Automation permission on the first use.
68
+
69
+ ## Future wake-ups
70
+
71
+ Main can use its `reminder` tool to schedule a one-shot check of a long-running process,
72
+ then reply immediately instead of waiting or polling:
73
+
74
+ ```json
75
+ {"action":"schedule","at":"2030-06-15T14:30:00Z","message":"Check the deployment status and report the next step."}
76
+ ```
77
+
78
+ `at` must be in the future, within 365 days, with seconds and an explicit timezone.
79
+ Messages are limited to 4000 characters; at most 100 reminders may be pending.
80
+ `{"action":"list"}` lists pending IDs; `{"action":"cancel","id":"123"}` suppresses an
81
+ undelivered reminder. Cancellation cannot retract an input already admitted to Main's inbox.
82
+
83
+ Deadlines and delivery receipts persist in SQLite. While the session is running, the
84
+ Harness wakes a sleeping task at its deadline and queues a normal Main input, behind any
85
+ active turn. Restarting recovers pending reminders, including overdue ones, without
86
+ re-admitting already delivered reminders. This does **not** launch dchat or wake a sleeping
87
+ computer: keep an interactive session running, or overdue reminders run when scheduling
88
+ next resumes (for example, opening the TUI). Batch/read-only status commands do not keep a
89
+ reminder service running. Cancellation is durable immediately; its now-inert sleeping task
90
+ settles at the original deadline. Reminders do not check a process themselves or imply it
91
+ has completed; Main decides what to do when the reminder arrives.
92
+
93
+ ## Configuration
94
+
95
+ Configuration is YAML, loaded on every run. Global defaults come from
96
+ `$XDG_CONFIG_HOME/durable-chat/config.yaml`, or `~/.config/durable-chat/config.yaml` when XDG is unset.
97
+ If the launch folder contains `.durable-chat/config.yaml`, its values override the global file.
98
+ Both locations accept `config.yml` as a fallback; `config.yaml` wins when both exist.
99
+ Only the launch folder is checked, not its parents.
100
+
101
+ Precedence is **CLI flag > environment variable > folder-local config > global config > built-in**.
102
+ Local files can contain just the keys you want to change. Nested `mcp` settings merge by key;
103
+ lists replace inherited lists (use `[]` to clear one). Relative paths still resolve against the launch
104
+ folder, not the config file's directory.
105
+
106
+ `--config <path>` or `DURABLE_CHAT_CONFIG` selects a **standalone** file instead of automatic global/local
107
+ layering; CLI `--config` wins over the environment selector. Missing explicit files are errors.
108
+ There is no implicit `.env` loading; `.env.example` lists equivalent environment overrides.
109
+
110
+ ```sh
111
+ durable-chat config path
112
+ durable-chat config show
113
+ durable-chat config keys
114
+ durable-chat config set maxTurns 20
115
+ durable-chat config set mcp.servers '["substrate", "nt_helper"]'
116
+ durable-chat config unset jobMaxTurns
117
+ durable-chat config edit # $VISUAL/$EDITOR
118
+ durable-chat config init --force # rewrite the template
119
+ ```
120
+
121
+ `set`/`unset` edit the YAML document in place, preserving comments and key order, and type-check the whole
122
+ file before writing. Secrets are not accepted in the file; `apiKeyEnv` names an environment variable.
123
+ `config show` prints effective values and loaded layers, never the secret itself.
124
+ `path`, `set`, `unset`, `init`, and `edit` target an existing folder-local file, otherwise the global
125
+ file. Use `--local` to target the folder even before a local file exists.
126
+ Unsetting a local key restores the inherited value. Use `--config <path>` to explicitly edit another
127
+ file. `--local` cannot be combined with `--config` or `DURABLE_CHAT_CONFIG`.
128
+
129
+ ### Folder-local overrides
130
+
131
+ ```sh
132
+ dchat config init --local # creates an empty override file, preserving global defaults
133
+ dchat config set maxTurns 30 # edits .durable-chat/config.yaml now that it exists
134
+ dchat config set store .durable-chat/store.sqlite
135
+ dchat config show # shows the merged settings
136
+ ```
137
+
138
+ For example, a project's `.durable-chat/config.yaml` could contain only:
139
+
140
+ ```yaml
141
+ maxTurns: 30
142
+ store: .durable-chat/store.sqlite
143
+ mcp:
144
+ servers: [substrate, nt_helper]
145
+ ```
146
+
147
+ Model and other unspecified settings still come from the global file. A local config does **not**
148
+ automatically switch databases; set `store` explicitly for folder-local state. Existing database contents
149
+ are not migrated. Ignore `.durable-chat/` in Git when it contains local state, and only load project
150
+ configs you trust. Without a local config, existing shared-store behavior is unchanged.
151
+
152
+ ### Fork the current state into a folder
153
+
154
+ ```sh
155
+ cd /path/to/project
156
+ dchat state fork
157
+ dchat
158
+ ```
159
+
160
+ `state fork` snapshots the currently selected database (normally the global store) into
161
+ `.durable-chat/store.sqlite` and sets `store` in the folder's local config, creating that config if
162
+ needed. Other local overrides and comments are preserved; global configuration and source state are
163
+ unchanged. To choose a different source explicitly:
164
+
165
+ ```sh
166
+ dchat state fork --store ~/.local/share/durable-chat/store.sqlite
167
+ ```
168
+
169
+ This is a consistent SQLite snapshot, including committed WAL data, not a raw file copy. It can read
170
+ a running source, but does not open a Harness, connect MCP, call a model, or resume work while copying.
171
+ Existing local databases or SQLite sidecars are never overwritten, even with `--force`.
172
+ `config show` shows the selected database; persistent `DURABLE_STORE` or `DURABLE_CHAT_CONFIG` overrides
173
+ must be unset to let the new local config take effect.
174
+
175
+ The fork retains conversations, task identities, and pending work. Subsequent changes are independent.
176
+ **Both copies can resume the same unfinished work and repeat external actions**; leave the original
177
+ stopped if you want only the fork to continue. Existing jobs retain their saved working directories:
178
+ forking storage does not move files or rewrite those job locations. Saved worktree assignments are
179
+ copied as-is; main's context-fork tool can create new worktrees for independent code variants.
180
+
181
+ The generated global defaults:
182
+
183
+ ```yaml
184
+ provider: local
185
+ model: nemotron-bolt
186
+ baseUrl: http://dev.local:1234/v1
187
+ contextWindow: 262144
188
+ maxTokens: 32768
189
+ streamTimeoutMs: 600000 # cold local start
190
+ retries: 1
191
+ thinkingLevel: "off"
192
+ maxTurns: 60
193
+ jobMaxTurns: 120
194
+ autoContinue: true
195
+ codemode: true # scripts that call the conversation's tools
196
+ # More than one provider can be configured; provider selects the active entry by name.
197
+ # providers:
198
+ # codex: { kind: openai-codex, model: gpt-6.1-sol }
199
+ # local: { kind: local, model: nemotron-bolt, baseUrl: http://127.0.0.1:1234/v1, contextWindow: 131072 }
200
+ store: ~/.local/share/durable-chat/store.sqlite
201
+ mcp:
202
+ enabled: true
203
+ servers: [substrate]
204
+ tools: []
205
+ config: ~/.pi/agent/mcp.json
206
+ ```
207
+
208
+ ### ChatGPT / Codex subscription
209
+
210
+ Use your ChatGPT/Codex subscription (OAuth) rather than an API key:
211
+
212
+ ```sh
213
+ durable-chat config set provider openai-codex
214
+ durable-chat config set model gpt-6.1-sol
215
+ durable-chat config set thinkingLevel medium
216
+ durable-chat config set contextWindow 500000
217
+ durable-chat config set maxTokens 128000
218
+ # An entry can hold its own settings, as providers.codex.model or providers.local.baseUrl.
219
+ ```
220
+
221
+ It reads the OAuth credential already stored by pi at `~/.pi/agent/auth.json` (override with
222
+ `DURABLE_AUTH_FILE`) and refreshes it in place. If that file has no `openai-codex` credential, sign in
223
+ with `pi` first. This is a subscription quota path, not `OPENAI_API_KEY`; never copy the token into this
224
+ repository.
225
+
226
+ ### OpenAI API key and local servers
227
+
228
+ Switch to the OpenAI API by setting `provider: openai`, `model: <listed id>`, and exporting
229
+ `OPENAI_API_KEY`. Set `provider: faux` for the offline scripted model.
230
+
231
+ The local provider declares `supportsDeveloperRole: false` and a chat-template thinking toggle because
232
+ llama.cpp-compatible servers commonly reject the OpenAI `developer` role. Agents run with
233
+ `thinkingLevel: "off"` to keep coordinator turns fast; change it in the config for reasoning models.
234
+
235
+ The file is read when a process starts; edits apply after restarting the process. Main and every job
236
+ use one session-wide model, initially selected at startup and changeable with `/model`. This includes
237
+ existing conversations, context forks, and handoff successors; there are no per-job model overrides.
238
+ An unfinished request retains its prepared model; it never blocks model selection. `/thinking [level]`
239
+ changes the thinking level for the run — main, existing jobs, and jobs created afterwards — without
240
+ writing the config file, so the next process starts from `thinkingLevel` again. `jobMaxTurns` remains a
241
+ creation-time job setting. Changing `store` selects another database; it does not migrate the old one.
242
+
243
+ ## Repository instructions (AGENTS.md)
244
+
245
+ Main, background jobs, forks and handoff successors automatically receive applicable `AGENTS.md`
246
+ files, from filesystem ancestors through their configured working directory. Files apply only to
247
+ that directory's subtree; more-specific files override ancestor rules within their own scope.
248
+ Sibling directories are not scanned. Paths are canonicalized, including symlinks and new write paths.
249
+ Managed worktrees use their own checked-out instructions, not changes in the source checkout.
250
+
251
+ Repository text is supplied as scoped **user context**, never trusted system policy. Each model
252
+ request gets one fresh, deduplicated snapshot, without appending instruction copies to durable
253
+ history. Forks rediscover instructions for their assigned directory rather than inheriting the
254
+ source's active scopes. Existing jobs are upgraded on startup.
255
+
256
+ When `read`, `write` or `edit` targets another scope, new or changed instructions automatically
257
+ pause the operation **before it executes**. The next model request loads them; the agent can then
258
+ review and retry. This also applies to coding tools called through codemode. Up to 16 recently
259
+ discovered target directories are retained per conversation for the current runtime; changing the
260
+ conversation cwd clears them. Files are refreshed on each request and checked again before coding
261
+ operations. Unreadable, invalid or oversized instruction files fail visibly (128 KiB per file).
262
+
263
+ `bash` accepts an optional `cwd` argument for directory-aware shell work, with the same instruction
264
+ check. It affects that command only. A `cd` inside a shell command does not change the conversation's
265
+ working directory. Arbitrary shell scripts, paths embedded in commands and external/MCP tools cannot
266
+ be statically inspected: agents should use explicit `bash` cwd or a coding-tool path to discover the
267
+ scope first. Instructions are guidance, not a filesystem sandbox. Files may change concurrently;
268
+ this is not a transactional security boundary.
269
+
270
+ ## Context and compaction
271
+
272
+ The context pane and `info`/`task show` show two different numbers:
273
+
274
+ - **context** — estimated size of the model request, using the same estimator Pi Durable uses
275
+ for compaction. It counts the newest assistant usage plus estimated messages after it.
276
+ - **spent** — cumulative recorded usage (`pi.usage`), not the current context.
277
+
278
+ For `gpt-6.1-sol` the window is 500k. Pi Durable’s default compaction policy reserves 16,384 tokens,
279
+ so blocking automatic compaction starts above **~483,616** tokens; background compaction starts
280
+ 32,768 tokens earlier (~450,848) when a selectable old prefix exists. It keeps about 20,000 recent
281
+ tokens, summarizes the older prefix, and places a `pi.compaction` head entry. A summary only counts
282
+ after its placement is stored.
283
+
284
+ Manual compaction: `m` on the selected target in the TUI, or `durable-chat compact [focus]` /
285
+ `durable-chat task compact <id> [focus]`. It runs a compaction task and waits for both the task and
286
+ the summary placement. A short conversation, or one with no selectable old prefix, is a no-op.
287
+
288
+ ## TUI
289
+
290
+ The default interface is a three-pane dashboard: jobs on the left, the selected conversation in the
291
+ middle, and live context on the right. The composer sends to `main` by default; open a job to target it.
292
+
293
+ The mouse wheel, Magic Mouse and trackpad scroll the pane under the pointer, keeping keyboard focus,
294
+ the composer draft and the selected job unchanged. Pane headings stay fixed; keyboard job navigation
295
+ brings the selection back into view. Alt+wheel scrolls faster.
296
+
297
+ Command-click transcript URLs on macOS (Ctrl-click in terminals that use that modifier) to open them
298
+ in the default browser. Markdown links and bare URLs keep their complete destinations when wrapped.
299
+ Clicks routed to the app also open web links; dragging across a link still selects text.
300
+
301
+ The terminal title includes the launch folder, for example `durable-chat — ~/projects/example`.
302
+ The previous title is restored on startup failure or exit using the terminal's title stack.
303
+ Folders outside home remain absolute. Allow program-controlled titles in your terminal; Ghostty's
304
+ `--title` option or a fixed `title` setting overrides title updates from apps.
305
+
306
+ ```text
307
+ ┌ durable-chat ─ local/nemotron-bolt ─ substrate ─ 2 live ─────────────────────┐
308
+ │ JOBS 2 live │ TRANSCRIPT main ● live │ CONTEXT │
309
+ │ ▸ #13 running 12s │ you Read src/ … │ target: main │
310
+ │ #14 sleeping 3s │ ai Started job #13. │ status: busy │
311
+ │ #11 completed │ ↳ ✓ bash · 18ms src/… │ turns: 3 │
312
+ │ #10 failed │ │ tokens: 18.2k │
313
+ ├──────────────────────┴───────────────────────────────┴──────────────────────┤
314
+ │ to main › _ │
315
+ │ ENTER send · CTRL-C ×2 quit · TAB jobs · CTRL-T target · CTRL-N new job │
316
+ └──────────────────────────────────────────────────────────────────────────────┘
317
+ ```
318
+
319
+ | Key | Action |
320
+ | --- | --- |
321
+ | `Tab` / `Shift-Tab` | cycle composer → jobs → transcript |
322
+ | `Ctrl-T` | switch composer target main ↔ selected job |
323
+ | `Ctrl-N` | new job brief |
324
+ | `Alt-S` | steer vs follow-up for the current target (main or job) |
325
+ | `Ctrl-V` | paste an image into the composer (main only) |
326
+ | `Enter` | send, open the selected job, or confirm |
327
+ | `Shift-Enter` | insert a newline in the composer (`Ctrl-J` or `Alt-Enter` fallback) |
328
+ | `j/k`, `g/G`, `PgUp/PgDn` | move selection or scroll |
329
+ | Mouse wheel / trackpad | scroll the pane under the pointer without moving keyboard focus |
330
+ | Cmd-click / Ctrl-click | open transcript links using the terminal's link modifier |
331
+ | `n / c / m / R` | new job · cancel · compact · reset (selected target) |
332
+ | `s / t / f / y` | send-to · expand tools · show finished · copy |
333
+ | `/` | start a slash command in the composer; filter jobs when focus is on jobs/transcript |
334
+ | `[` / `]` | shrink / grow the jobs pane |
335
+ | `{` / `}` | shrink / grow the context pane (shows it on narrow terminals) |
336
+ | `0` | reset pane widths |
337
+ | `i` | inspector (config, MCP notes, usage); `e` edits the config |
338
+ | `a` | abort the main run (jobs survive) |
339
+ | `?` | help |
340
+ | `Ctrl-C` | stop waiting / clear the prompt / close an overlay; arms quitting for 1.5 seconds |
341
+ | `Ctrl-C` twice | quit; small floating confirmation over the dashboard while live work exists |
342
+
343
+ `Ctrl-V` attaches an image to the next message to main. It reads the native clipboard (macOS and
344
+ Windows prebuilds, Linux with X11); `/image <path>` is the fallback where that helper is unavailable.
345
+ Images are prepared with Photon on a worker thread to the model's image limits, and paste is refused
346
+ when the model declares text-only input, because providers drop image blocks for those models.
347
+ Attachments go to main only; jobs keep text input.
348
+
349
+ Transcript prose supports lightweight Markdown: headings, emphasis, lists, links, quotes, tables, and code
350
+ blocks, including streaming replies. Tool output stays literal; code blocks use plain styling without
351
+ syntax highlighting. The plain-chat interface is unchanged.
352
+
353
+ The multiline composer grows from a dark-grey three-row background and scrolls longer drafts while
354
+ keeping the cursor visible. **Shift-Enter inserts a line break; plain Enter sends.** If your terminal
355
+ does not distinguish Shift-Enter from Enter, use **Ctrl-J** (or Alt-Enter) to insert a newline instead.
356
+ Arrow keys navigate the draft; Backspace/Delete can join lines. Bracketed text pastes keep actual
357
+ line breaks (CRLF and CR normalize to LF) without sending. Large pastes appear as folded paste
358
+ markers; their complete text, not the marker, is sent. Visual wrapping adds no newline characters,
359
+ and text copied without line breaks cannot have its original layout reconstructed.
360
+ Pane headings, the target/status line, and individual jobs have spacer rows; the transcript has a solid
361
+ black background. Tab to jobs, then `j/k` or Up/Down to preview the highlighted job's latest transcript
362
+ and live updates without pressing Enter. `g/G` preview the first/last row. Previewing never changes the
363
+ composer or action target: **Enter pins the job** and focuses its transcript; `Ctrl-T` explicitly toggles
364
+ the target. Tab from jobs to transcript keeps the preview for scrolling; returning to the composer
365
+ restores the pinned target's transcript and keeps your draft. Confirmations still name the pinned target.
366
+ While targeting `main`, newly arriving jobs are highlighted automatically unless you are browsing a
367
+ preview. Returning to `main` selects the newest visible job; manual browsing and pinned targets are
368
+ preserved. A single `Ctrl-C` clears the input or stops waiting as before; a second consecutive press
369
+ within 1.5 seconds requests quit. Typing, navigation, or expiry resets that sequence. Live work still
370
+ gets the floating confirmation; `Ctrl-Q` is no longer a quit shortcut.
371
+
372
+ The status line always shows `ctx ~tokens/window`; wide layouts add `context` (estimated active request),
373
+ `compact` (the default automatic-compaction threshold) and `spent` (cumulative recorded usage). Pane
374
+ widths are adjustable from the jobs or transcript focus with `[`/`]`, `{`/`}`, and `0`; resizing is
375
+ per-run.
376
+
377
+ Jobs run in the background and main stays usable. The TUI is the single store writer; closing it pauses
378
+ jobs, and the next run resumes them. `durable-chat --plain` or `durable-chat chat` is the old
379
+ line-oriented interface over the same runtime.
380
+
381
+ Steering applies to both targets: `Alt-S` selects whether the next message joins the current run
382
+ (`steer`) or queues behind it (`followUp`). The CLI equivalent is `task send <id> "<message>" --steer`
383
+ or `/steer <id> <message>` in the chat or TUI composer.
384
+
385
+ Starting a job from the TUI or `task add` writes a native, model-visible `[host]` start notice into
386
+ main's conversation. Every job reports completion, failure, or abort as a native follow-up input by
387
+ default, including jobs started by main's `task` tool (whose tool result already covers the start).
388
+ That means **main automatically continues**: it runs, reviews the result, and tells the user what
389
+ changed. A handoff to main is also an input and starts a run. Main is event-driven: it is told not to
390
+ poll, and `task list`/`task show` are for an explicit status request or a specific detail, not for
391
+ checking whether work is done.
392
+
393
+ Set `autoContinue: false` in the config for passive notices (a write that main sees on its next turn
394
+ without spending tokens immediately). The default is `true`. The setting is captured when a job is
395
+ created, so changing it affects new jobs and their handoff successors.
396
+
397
+ Turn limits are tool rounds per run: `maxTurns: 60` for main and `jobMaxTurns: 120` for jobs. Main’s
398
+ `task` tool also enforces a host-side anti-poll guard: the first `show` on a live job is allowed, but a
399
+ second within 30 seconds returns a “do not poll” result and ends the run; a repeated `list` within 10
400
+ seconds does the same. This protects a model that ignores the prompt.
401
+
402
+ All durable state goes through Pi Durable — `Harness`, `Conversation`, `Task`, `Submission`, native
403
+ documents, `watchTaskGraph`, `conversation.watch`, `entries`, `compact`, `reset`, `abortTask`, and
404
+ native `control.terminate` handoffs. The only host files are configuration, the MCP server config, and
405
+ the store lock.
406
+
407
+ ## Commands
408
+
409
+ | Command | Meaning |
410
+ | --- | --- |
411
+ | `durable-chat` | full-screen TUI dashboard (`--plain` for the line chat) |
412
+ | `durable-chat tui` | force the TUI |
413
+ | `durable-chat chat` | line-oriented chat |
414
+ | `durable-chat ask "<text>" [--wait]` | one-shot main-agent turn; `--wait` joins live jobs afterwards |
415
+ | `durable-chat task add "<brief>" [--detach]` | start a background job agent (waits unless `--detach`) |
416
+ | `durable-chat tasks` | list live and finished jobs |
417
+ | `durable-chat task <id>` | job status, answer, and recent job transcript |
418
+ | `durable-chat task log <id>` | recent job transcript |
419
+ | `durable-chat task send <id> "<message>" [--steer]` | continue the job conversation, or steer its current run |
420
+ | `durable-chat task cancel <id>` | abort the job and its active job conversation |
421
+ | `durable-chat task compact <id> [focus]` | compact the job conversation |
422
+ | `durable-chat task reset <id> [handoff]` | reset the job context |
423
+ | `durable-chat watch [--follow]` | live job board until jobs settle (or until Ctrl-C with `--follow`) |
424
+ | `durable-chat compact [focus]` | compact the main conversation |
425
+ | `durable-chat reset [handoff]` | reset the main context |
426
+ | `durable-chat info` | config, store, model, limits, tools, and MCP status |
427
+ | `durable-chat config ...` | `path`, `show`, `keys`, `init`, `edit`, `get`, `set`, `unset` |
428
+ | `durable-chat state fork [--store <source>]` | snapshot the selected state into this folder and configure the fork |
429
+
430
+ Global flags: `--config <path>`, `--store <path>`, `--provider faux|openai|local`, `--model <id>`,
431
+ `--verbose`, `--plain`, `--no-mcp`, `--force` (for `config init`).
432
+
433
+ Inside the plain chat or TUI composer:
434
+
435
+ ```text
436
+ /tasks list jobs
437
+ /task <id> show a job
438
+ /log <id> recent job transcript
439
+ /new <brief> start a job and let it run
440
+ /send <id> <message> continue a job (follow-up)
441
+ /steer <id> <message> steer the job's current run
442
+ /cancel <id> abort a job
443
+ /compact [focus] compact the main conversation
444
+ /reset [handoff] reset the main context
445
+ /model [id] pick the session-wide model, across configured providers
446
+ /provider [name] list configured providers, or make one active for this session
447
+ /thinking [level] session thinking level for main and jobs; the config file decides the next run
448
+ /image <path> attach an image to the next message to main
449
+ /reboot archive all app state and start a fresh database (confirmation required)
450
+ /rollback [path] select saved state, archive current state, restore a copy, and relaunch
451
+ /wait join live jobs (Ctrl-C stops waiting only)
452
+ /info
453
+ /quit
454
+ ```
455
+
456
+ In the TUI, these are local actions, not model messages: `/help` opens help, `/info` opens the inspector,
457
+ `/tasks` focuses the jobs pane, and `/task`/`/log` open the job transcript. `/new` without a brief opens
458
+ the brief prompt. `/compact` and `/reset` apply to the current target; reset, cancel, and quitting with
459
+ live work retain their confirmations. `/send` and `/steer` address the specified job without changing
460
+ the current target. `/wait` joins a snapshot of current jobs, and Ctrl-C cancels only the wait.
461
+ Use `/filter [query]` in the TUI composer to filter jobs, or press `/` while focused on jobs/transcript.
462
+ Unknown or malformed commands report a local error rather than being sent to the model.
463
+
464
+ The main agent is never blocked by a job: jobs are background tasks in the main conversation and run in
465
+ their own conversations. Closing the process does not delete work; unfinished task phases resume on the
466
+ next open.
467
+
468
+ ### Model picker
469
+
470
+ Use `/model` to browse the models of **every configured provider**, the active entry first. In the
471
+ TUI, type to filter, use Up/Down to select, and press Enter to apply; Esc cancels. Plain chat offers a
472
+ numbered list and accepts a number, exact ID, or `provider/id`. `/model <exact-id>` selects directly.
473
+ When two entries list the same ID, the picker shows the provider-qualified `provider/id` so the choice
474
+ cannot be ambiguous.
475
+
476
+ OpenAI/Codex use their installed chat-model catalog. Local servers are queried at the configured
477
+ OpenAI-compatible `/models` endpoint, using the configured API key when present, with a five-second
478
+ timeout. If discovery fails, the configured model remains available. Browsing sends no generation
479
+ requests or OAuth refreshes.
480
+
481
+ Selection applies to main, existing jobs, future jobs/forks, and handoff successors. It keeps the
482
+ database, conversations, and tags, and restarts the runtime under the continuously held store lock.
483
+ Configured context/output ceilings are retained and capped to the selected model's catalog limits.
484
+ Compact large conversations before moving to a smaller context window.
485
+
486
+ Switching closes the current runtime and leaves prepared requests intact. Resumed requests and deferred
487
+ polls use their checkpoint model; ordinary retries prepare again using the current selection. If a pinned
488
+ model is unavailable, that turn ends with `no_model`; subsequent inputs use the selected model. Built-in
489
+ provider IDs remain available for checkpoints saved before named entries were configured. A failed restart
490
+ recovers the previous runtime.
491
+
492
+ Named OpenAI/Codex entries use the existing Pi credentials for their provider kind. Renaming an entry
493
+ keeps the same login, and refreshed tokens are saved back under the original Pi credential key.
494
+
495
+ Selection is **session-only**: it survives `/reboot` and `/rollback` in the same process, while a normal
496
+ process restart uses your configured default. To resume unfinished work on the selected model after
497
+ quitting, start with `dchat --model <id>`. Use `dchat config set model <id>` to explicitly change the
498
+ configured default.
499
+
500
+ `/provider [name]` lists the configured entries and makes one active, applying that entry's model,
501
+ endpoint, ceilings, timeout, and thinking level through the same reopen path as `/model`. Only the
502
+ active entry is contacted for a dynamic list; the others show their configured model. Like `/model`,
503
+ the switch is session-only: the config file still decides the next run.
504
+
505
+ ### Safe reboot
506
+
507
+ Use `/reboot` in the TUI or plain chat to start fresh at the **same database path**. It always asks for
508
+ confirmation: **Y reboots; Enter, N, or Esc cancels** in the TUI (`y` at the plain-chat prompt).
509
+ This applies to the entire store, even when a job is selected.
510
+
511
+ Reboot stops UI observers, closes the runtime and MCP connections, checkpoints SQLite's committed WAL,
512
+ and moves the database into a private, uniquely named archive directory beside the original:
513
+
514
+ ```text
515
+ <store>.reboot-<UTC timestamp>-<unique suffix>/<database filename>
516
+ ```
517
+
518
+ A fresh database then opens at the original path, with empty conversations, jobs, and tags. The active
519
+ session settings and model remain in use; restart the process to apply config-file edits.
520
+ Project files and Git worktrees stay in place. The archive retains old conversations and unfinished jobs,
521
+ which resume only when that archive is reopened.
522
+
523
+ The archive path appears in the TUI inspector, plain-chat banner, and `dchat info`, and survives
524
+ subsequent restarts. Archives stay available until you explicitly remove them. Use `/rollback` below
525
+ to restore a copy while preserving the archive.
526
+
527
+ The store lock stays held throughout rotation and reopening. Close other SQLite clients before
528
+ rebooting; aliased database files (symlinks or additional hard links) are refused. If fresh startup
529
+ fails, recovery preserves the partial replacement and restores the original when its journals can be
530
+ closed safely. Otherwise the error identifies the preserved archive and replacement for recovery.
531
+
532
+ ### Rollback
533
+
534
+ Use `/rollback` in the TUI or plain chat to select a saved database. It lists up to five recent archives,
535
+ with the last backup first. Enter a number or a file path; an explicit path also works directly:
536
+
537
+ ```text
538
+ /rollback
539
+ /rollback "/path/to/saved state/store.sqlite"
540
+ ```
541
+
542
+ After selecting a file, **Y confirms; Enter cancels**. The current runtime closes, its database is
543
+ archived under another unique name, and a consistent SQLite snapshot of the selected file is restored
544
+ at the **original active database path**. The runtime then relaunches with the active session settings.
545
+ The selected file stays unchanged, including when committed data must be read from its WAL.
546
+
547
+ The inspector and `dchat info` show both the selected source and the newly archived previous state.
548
+ Another `/rollback` can select that previous state to undo the restore. The source must be closed and
549
+ lockable; selecting the active database is refused. Failed replacement/reopening uses the same
550
+ preserving recovery as reboot.
551
+
552
+ Restored unfinished jobs resume automatically. Their saved workspaces, and all project files, retain
553
+ their current filesystem contents; rollback restores app state. Reopening another copy of the same
554
+ pending work can repeat external actions.
555
+
556
+ For advanced use, `dchat --store "/path/to/archive/store.sqlite"` opens an archive **in place** rather
557
+ than copying it to the active path.
558
+
559
+ The TUI design and its implementation notes are in [docs/tui-design.md](docs/tui-design.md).
560
+
561
+ ## What each agent can do
562
+
563
+ **Main agent** — `task` (`create`, `list`, `show`, `tag`, `steer`) and `fork`. It primes context through scouting jobs,
564
+ creates fresh jobs or context-forked variants, redirects running jobs, and waits for native notices instead of polling.
565
+
566
+ ```javascript
567
+ task({ action: "steer", id: "13", message: "Focus on the regression fix; preserve the existing API." })
568
+ ```
569
+
570
+ Steering returns as soon as the message is durably queued. The worker receives it at its next native
571
+ boundary; an executing tool continues until that boundary. The job stays live through the steering
572
+ response and reports the updated result, including when steering arrives during a final answer.
573
+ Queued steering survives a restart. Idle and completed jobs reject steering; create or fork a job for new work.
574
+
575
+ **Job agents** — a minimal covering set:
576
+
577
+ - `read`, `write`, `edit`, `bash` from pi-durable's coding tools
578
+ - `substrate_search`, `substrate_describe`, `substrate_call` when Substrate MCP is connected
579
+ - `handoff` — return the work to main or create a successor job
580
+ - `codemode` — a QuickJS script that calls this conversation's tools; only the script's output comes back
581
+
582
+ `substrate_call` is deliberately a gateway rather than 249 separate tool schemas: search the catalog,
583
+ describe the exact tool, then invoke it with a JSON args object. Jobs cannot start arbitrary jobs; a handoff creates at most one successor.
584
+
585
+ ### Codemode scripts
586
+
587
+ `codemode: true` (default, `DURABLE_CODEMODE`) gives main and every job the `codemode` tool. A script
588
+ runs in QuickJS on a worker thread and reaches only the tools of its own conversation, so main's
589
+ scripts can call `task` and `fork` while job scripts keep the coding tools and the MCP gateway.
590
+
591
+ - Only the script's output reaches the model, so a script can batch calls with `Promise.allSettled()`
592
+ and filter large output first.
593
+ - A script survives a restart: its calls are journaled, and a rerun replays them without running
594
+ finished calls again. Tools marked unsafe on replay (`task`, `fork`, `handoff`, `substrate_call`)
595
+ are not rerun, so scripts should not create jobs.
596
+ - The prompt section lists tool declarations up to a token budget; scripts find the rest with
597
+ `searchTools()`, `describeTool()`, and `describeNamespace()`.
598
+ - `store()`/`load()` keep small JSON per conversation, and a fork sees them as of its fork point.
599
+ - Tool `control` applies only to model-issued calls, so the anti-poll guard in the task tool returns
600
+ its text to a script but cannot end a run started by one.
601
+
602
+ ## Context forks (main-only)
603
+
604
+ Ask main to investigate once, then fork the prepared scout into independently briefed variants. Main's
605
+ tool calls look like:
606
+
607
+ ```text
608
+ fork({ source: "13", brief: "Implement the minimal fix", workspace: "worktree" })
609
+ fork({ source: "13", brief: "Try the alternative design", workspace: "worktree" })
610
+ fork({ source: "13", brief: "Review the risks without editing", workspace: "shared" })
611
+ fork({ source: "main", brief: "Explore another direction", workspace: "shared" })
612
+ ```
613
+
614
+ - `source` is `main`, a job ID as a string, or `tag:reviewer` for a uniquely tagged job. The cutoff is the latest completed assistant turn in
615
+ the source's active context. Prime a source before forking it; an in-flight tool round is excluded.
616
+ - Each fork is a new durable job and a native conversation fork. It inherits visible history and tool
617
+ evidence through the cutoff, with a fresh provider identity and independent subsequent entries.
618
+ Existing running tasks are left with the source. All new jobs report to main.
619
+ - The worker gets the job toolset and current brief, rather than main's coordinator tools. Only main
620
+ is offered `fork`, and the tool also checks the calling conversation. Jobs retain `handoff`.
621
+ - The Jobs pane shows `↳ #13` or `↳ main`. The context pane and `task <id>` details show the source
622
+ entry and assigned workspace/branch. Lineage and workspace plans survive restart.
623
+ - Inherited context is still model input; provider caching determines any processing-cost savings.
624
+
625
+ ### Reusable context tags
626
+
627
+ Main can name prepared contexts by purpose rather than remembering task numbers:
628
+
629
+ ```text
630
+ task({ action: "create", brief: "Learn the review conventions and inspect the architecture", tags: ["reviewer"] })
631
+ task({ action: "tag", id: "13", tags: ["reviewer", "primed"] })
632
+ fork({ source: "tag:reviewer", brief: "Review the new variant; recheck the changed files", tags: ["review-a"] })
633
+ ```
634
+
635
+ Tags are persisted independently of a job's lifecycle and can be changed after it completes.
636
+ `task tag` replaces the tag set; `tags: []` clears it. Tags normalize to lowercase, support letters,
637
+ digits, dots, hyphens and underscores, and are limited to eight tags of at most 48 characters per job.
638
+ Task lists/details and the Jobs pane show tags; the job filter matches them.
639
+
640
+ A tagged fork source must resolve to exactly one job with a completed turn. Missing or ambiguous
641
+ matches return a local tool error with candidate IDs; the app never silently chooses a different
642
+ context. Forks receive only their explicitly supplied tags, keeping the prepared source's label stable.
643
+ A tag identifies reusable context, not proof that its findings are still current.
644
+
645
+ ### Fork workspaces
646
+
647
+ `shared` (the default) uses the source conversation's directory recorded at the cutoff. `worktree` requires Git and a clean
648
+ source checkout, including untracked files, and records an exact base commit. The host creates a unique
649
+ `dchat/<uuid>` branch and worktree under the repository's Git common directory
650
+ (`dchat-worktrees/<uuid>`), preserving a source subdirectory as the worker's `cwd`.
651
+
652
+ The plan is committed with the job before filesystem setup. Setup finishes before the first worker
653
+ request; restart reconciles the same branch/path and reacquires the same brief submission. Setup
654
+ errors become job failures reported to main. Existing dirty worktrees and later variant commits are
655
+ retained; incomplete, locked, or mismatched workspaces are preserved and reported for inspection. Saved
656
+ workspace assignments also guard later tool environments after the job task finishes. A handoff
657
+ successor continues in its predecessor's assigned workspace.
658
+
659
+ Worktrees contain the recorded Git tree: ignored files, dependencies, credentials, and dirty edits are
660
+ not copied. The host runs no project setup hooks or dependency installation. Worktrees and branches
661
+ remain after completion, cancellation, and failure for explicit review/integration/cleanup. Main
662
+ decides which results to combine; the app performs no automatic merge. Worktrees provide separate edit
663
+ directories; jobs still share the host and retain their normal tool access.
664
+
665
+ This context-fork tool is distinct from `dchat state fork`, which copies the entire SQLite store.
666
+ Filesystem snapshots of dirty workspaces remain a separate future option.
667
+
668
+ ## Handoffs
669
+
670
+ A job can call the `handoff` tool instead of finishing:
671
+
672
+ ```text
673
+ handoff({ target: "main", brief: "Finish the release notes", summary: "Tests pass; changelog drafted." })
674
+ handoff({ target: "job", brief: "Publish the release", summary: "Notes reviewed; tag prepared." })
675
+ ```
676
+
677
+ - **Job → main**: the coordinator receives a native input (`[job #N handoff] …`) and starts a run.
678
+ The current job ends `handed_off`; main can inspect it with the task tool.
679
+ - **Job → job**: a successor job is created in main's conversation with the same model, cwd, turn limit,
680
+ and selected extensions. Its brief carries the summary and continuation. The current job ends
681
+ `handed_off`; the successor runs independently. Chains are capped at 8 handoffs.
682
+ - **Awareness**: user-started job chains keep writing `[host]` notices to main. A handoff to main is
683
+ itself the message; no extra completion notice is written.
684
+ - Both directions are native: a conversation document records the intent, the task creates the
685
+ successor or admits the main input, and terminal state is committed through `runtime.commit()`.
686
+
687
+ ## Substrate MCP
688
+
689
+ Job agents read MCP servers from `~/.pi/agent/mcp.json` (and `.pi/mcp.json` for project overrides), the
690
+ same format pi uses. The config keys are:
691
+
692
+ - `mcp.enabled` — `true` by default
693
+ - `mcp.servers` — default `[substrate]`
694
+ - `mcp.tools` — optional extra direct tools to expose (server tool names)
695
+ - `mcp.config` — default `~/.pi/agent/mcp.json`
696
+
697
+ Environment equivalents: `DURABLE_MCP_ENABLED`, `DURABLE_MCP_SERVERS`, `DURABLE_MCP_TOOLS`,
698
+ `DURABLE_MCP_CONFIG`. `--no-mcp` disables MCP for one run.
699
+
700
+ The `substrate` server gets the three-tool gateway. Other servers expose selected tools directly (all
701
+ tools up to 20 if no filter is set). If the server is missing, disabled, or unreachable, job agents get a
702
+ `mcp_substrate_unavailable` tool that explains why; jobs still have the coding tools. `durable-chat info`
703
+ shows the connection state and notes. Tokens are read from the config and never printed.
704
+
705
+ ## Durability notes
706
+
707
+ - Every job is a durable `durable-chat.job` task plus its own conversation. The task submits the brief,
708
+ joins the answer, and records the receipt (answer entry, capped answer text) on the task.
709
+ - Inputs use stable request IDs (`job:<taskId>:run`), so a restart reacquires the admission instead of
710
+ duplicating the brief. A generation cut off mid-request retries; an aborted partial answer is retained
711
+ as an aborted entry next to the retried answer.
712
+ - Cancelling a job marks its task, aborts the job conversation with a bounded cleanup wait, then commits
713
+ an `aborted` terminal outcome.
714
+ - Compaction is joined through the compaction task **and** its summary placement. A no-op (no selectable
715
+ old prefix, declined summary) is reported as a no-op, not as success.
716
+ - Reset is admitted immediately; while the conversation is busy it lands at the next boundary. Raw history
717
+ is retained.
718
+ - Jobs are background tasks, so `Conversation.abort()` on the main conversation does not cancel them;
719
+ `task cancel` targets one job.
720
+ - External effects are not exactly-once. Model/tool retries and process cuts can repeat a bash command;
721
+ `read`/`write`/`edit`/MCP calls are marked replay-unsafe where appropriate. Check the job transcript
722
+ before re-running a job with side effects.
723
+
724
+ ## Limits
725
+
726
+ - The config file selects a user-level store; everything is still one SQLite file.
727
+ - One writer per store. Do not run two `durable-chat` processes on one store (chat + watch included).
728
+ - Jobs only run while some process has the store open and scheduling enabled. Closing the process pauses
729
+ them; the next command resumes them.
730
+ - `task add --detach` admits the job and exits without starting it. Use `watch` (or another command that
731
+ resumes scheduling) to run it.
732
+ - The answer stored on a task is capped at 6000 characters; the full answer remains in the job
733
+ conversation (`task log <id>`).
734
+ - Automatic compaction uses pi-durable defaults. Manual compaction of a short conversation is a no-op by
735
+ design.
736
+
737
+ ## Development
738
+
739
+ ```sh
740
+ npm run check # tsc --noEmit
741
+ npm test # source regression tests
742
+ npm run build # clean dist/ build, rewriting relative .ts imports to .js
743
+ npm run test:package # pack, inspect and install into a temporary node_modules, then offline smoke
744
+ ```
745
+
746
+ `tsconfig.json` sets `skipLibCheck: true` deliberately: `@google/genai`'s published declarations import
747
+ an optional `@modelcontextprotocol/sdk` path that is not installed here. Our own sources are checked with
748
+ `strict`; the limitation is the third-party declaration, not a weakened check of this project.
749
+
750
+ ## Layout
751
+
752
+ ```text
753
+ src/cli.ts argument parsing, interactive REPL, slash commands
754
+ src/models.ts provider model discovery and picker selection
755
+ src/reboot.ts closed-database checkpoint, unique archival, and failed-startup recovery
756
+ src/rollback.ts saved-state discovery, source selection, and read-only SQLite snapshot installation
757
+ src/config.ts YAML defaults, precedence, path resolution, typed editing
758
+ src/config-commands.ts config path/show/keys/init/edit/get/set/unset
759
+ src/harness.ts store lock, providers, registry, root configuration, lifecycle
760
+ src/tasks.ts durable job task (workspace/run/report/abort), job links and handoffs
761
+ src/forks.ts completed-turn source selection for main-only context forks
762
+ src/tags.ts durable job tags and unambiguous prepared-context lookup
763
+ src/workspaces.ts Git worktree planning, persisted assignment, and restart-safe setup
764
+ src/jobs.ts job creation, scanning, answers, logs, compaction, messaging
765
+ src/tool.ts the main agent's `task` tool
766
+ src/mcp.ts MCP config, connections, Substrate gateway
767
+ src/chat.ts streaming and one-shot main turns
768
+ src/watch.ts live job board over the task graph
769
+ src/format.ts job views and text formatting
770
+ src/faux.ts offline scripted model for demos and tests
771
+ src/tui/main.ts run the alt-screen dashboard
772
+ src/tui/app.ts panes, keymap, actions, live updates
773
+ src/tui/state.ts pure job list/selection/target helpers
774
+ src/tui/transcript.ts windowed conversation history + live state
775
+ src/tui/format.ts terminal-friendly entry and tool formatting
776
+ ```
777
+
778
+ ## Safety and privacy
779
+
780
+ Agents can run shell commands and read/write files as your user; repository instructions and
781
+ QuickJS codemode are not an operating-system sandbox for the exposed tools. Use a trusted
782
+ working directory and review model/tool actions. Real providers can send prompts, source files
783
+ and tool output to their services and consume paid quota. MCP servers have their own access
784
+ and credential requirements; Substrate is optional (`--no-mcp`).
785
+
786
+ SQLite stores, archives, transcripts, credentials and project-local configuration can contain
787
+ private data. Keep `.durable-chat/`, `.pi/`, `.env`, auth files and database/log artifacts out of
788
+ Git and release archives. Do not upload a live store when reporting a bug.
789
+
790
+ ## License
791
+
792
+ Original durable-chat code is [MIT licensed](LICENSE). The actual upstream dependency is
793
+ `@earendil-works/pi-durable`, part of [earendil-works/pi](https://github.com/earendil-works/pi),
794
+ not a package named “PyDurable”. Its MIT license and relevant third-party notices are preserved
795
+ in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and `LICENSES/`. Dependencies retain their own licenses.