@gavin-hu/my-pi-agent 0.18.0 → 0.20.0

package/{extensions/todo/README.md → docs/extensions/todo.md}

@@ -1,92 +1,63 @@
 # todo — a TodoWrite-style task list for Pi
 
-A whole-list task tracker for Pi, modelled on Claude Code's `TodoWrite`. The
-model sends the complete list on every call; the previous list is discarded, so
-there are no ids to juggle and no add/toggle bookkeeping. The list follows the
-active session branch and is restored on `/resume` and `/tree`. When a run is
-about to end after mutating work without a todo update, the extension injects
-one short reminder and forces a continuation, so the list does not silently
-drift behind the work.
-
-<img src="../../assets/todo.png" alt="The /todos screen: a Todos title rule, a 3/7 completed summary, and the list with check, spinner, and circle glyphs" width="100%">
-
-```
+A whole-list task tracker for Pi, modelled on Claude Code's `TodoWrite`: the
+model sends the complete list on every call, so there are no ids to juggle and
+no add/toggle bookkeeping. The list follows the active session branch and is
+rebuilt on `/resume` and `/tree`, and the extension keeps it honest by asking for
+one update when work lands without one.
+
+<img src="../../assets/screens/todo.png" alt="The /todos screen: a Todos title rule, a 3/7 completed summary, and the list with check, spinner, and circle glyphs" width="100%">
+
+## Capabilities
+
+Status records what a reader can rely on: **Shipped** is in a tagged release, **Unreleased** is complete in the current tree but not yet tagged, and **Partial** has one released part and one unreleased part. Rows are ordered by user value, highest first.
+
+| Capability | How | What you get | Status |
+|---|---|---|---|
+| Whole-list tracking | `todo` | One call replaces the list; an empty list clears it. | Shipped |
+| Live progress rail | the `todo` widget | Progress and the active item, one line above the editor. | Shipped |
+| Atomic validation | `todo` validation | At most one `in_progress`, at most 50 items, unique non-empty content, and a known status; a violation returns an error with the unchanged list. | Shipped |
+| Terminal-safe normalization | sanitized `content` and `activeForm` | Control characters, newlines, tabs, and repeated spaces collapse to one terminal-safe line before the list is stored. | Shipped |
+| Compact model result | the `todo` result | The model gets progress and the active item, not an echo of the whole list, so every update does not repeat the table. | Shipped |
+| Transcript rendering | the `todo` transcript block | The call line carries progress; the collapsed rows lead with active work, the expanded rows keep submission order, and a cap ends with `… N more`. | Shipped |
+| Full list on demand | `/todos` | A scrollable screen in the order the model submitted. | Shipped |
+| Headless listing | `/todos` outside `tui` | RPC, print, and JSON modes get the same list as a notification, with no terminal screen. | Shipped |
+| Branch-following state | tool-result `details` | The list survives `/resume` and `/tree`, and abandoned branches never leak in. | Shipped |
+| Lag reminder | the `agent_before_settle` nudge | One hidden reminder and one forced continuation when mutating work lands without a todo update. | Shipped |
+| Structured result | `structuredContent` | Codemode scripts read the list as data. | Shipped |
+
+## Quickstart
+
+```bash
 pi --extension ./extensions/todo    # load just this extension
 pi -e .                             # load the whole @gavin-hu/my-pi-agent package
-pi install ./                       # install the package
+pi install ./                       # install the package for a user
 ```
 
-## Tool rendering
-
-How the `todo` call and its result look in the transcript — the call line as a headline and the body indented under it:
-
-<img src="../../assets/tools/todo.png" alt="The todo tool's result: the call line and an indented body with the progress line and the task rows below it, each led by a check, spinner, or circle glyph" width="100%">
-
-## What it does
-
-- Registers one model-callable tool, `todo`, that **replaces** the task list.
-  An empty list clears it. The tool result is a compact summary — progress plus
-  the current item — not an echo of the list the model just sent.
-- Each item has `content`, a `status` (`pending` | `in_progress` |
-  `completed`), and an optional `activeForm` ("Running tests") shown while the
-  item is in progress.
-- Shows a persistent widget above the editor while the list has unfinished
-  work (hidden when empty or fully completed), and a scrollable `/todos`
-  screen on demand. It is the bottom rail, below the [`goal`](../goal/) widget
-  (`Goal / Todos`).
-- Stores the list in tool-result `details`, so it follows the active session
-  branch and survives `/resume` and `/tree` — abandoned branches never leak
-  into the current list, and stored lists are re-sanitized on replay.
-- Keeps the list honest: when a run is about to settle after a successful
-  mutating tool call (`write`/`edit`/`bash`/`powershell`, or a tool declaring
-  `readOnlyHint: false`) without a todo update, it injects one hidden reminder
-  and forces a single continuation. A failed or blocked call changed nothing, so
-  it never triggers one. It fires while the list has unfinished items, and —
-  when no list exists at all — only after three such calls in one turn, at least
-  one of them a file write (`write`/`edit`, or a tool declaring
-  `readOnlyHint: false`), so repeated `bash` inspection is not mistaken for
-  unplanned work. A list the model wrote or cleared during the turn is left
-  alone. At most one reminder per user turn, and it is dropped on the next user
-  turn, so a reader can never trigger it.
-- Declares an `outputSchema` and returns matching `structuredContent`
-  (`{ todos, action, error? }`), so codemode/scripts can read the list as data.
-
-The widget is a one-line summary — progress plus the current item — so the
-active task is always visible without expanding the list:
-
-```
-Todos · 1/3 · Writing the tests
+```text
+/todos                              # show the list, and its progress
 ```
 
-`/todos` is the expanded view and keeps the model's list order (use it to see
-the plan as written).
+The model calls `todo` with the complete list; the widget appears above the
+editor while anything is unfinished.
 
-In the transcript, the call line carries progress only (`todo → 1/4`); the
-result keeps the `✓` / `◒` / `◌` rows, directly under the call line. The item
-text lives on the rows, so the call line never repeats it. Collapsed, it leads
-with active work; expanded, it keeps the submitted order:
+## Tool
 
-```
-todo → 1/4
-  ◒ Writing tests
-  ◌ Ship it
-  ◌ Write docs
-  ✓ Write schema
-```
+<img src="../../assets/tools/todo.png" alt="The todo tool's result: the call line and an indented body with the progress line and the task rows below it, each led by a check, spinner, or circle glyph" width="100%">
 
-Progress lives on the call line, so the rows have no footer; the persistent
-widget and `/todos` still show the count. A capped collapsed rail ends with
-`… N more` and the `app.tools.expand` hint.
+`todo` — `exposure: "direct"` (the default), `executionMode: "sequential"`,
+read-write, closed-world, idempotent. It replaces the list; an empty list clears
+it. The result is a compact summary — progress plus the current item — not an
+echo of the list the model just sent.
 
-## Tool
+| Parameter | Type | Meaning |
+|---|---|---|
+| `todos` | array, required | The complete list. Pass `[]` to clear. |
+| `todos[].content` | string | Imperative description, e.g. `"Write the parser tests"`. |
+| `todos[].status` | `pending` \| `in_progress` \| `completed` | Where the item stands. |
+| `todos[].activeForm` | string | Present-continuous label, required when the item is `in_progress` ("Running tests"). |
 
-| Field | Value |
-|---|---|
-| `name` | `todo` |
-| `todos` | The complete list. Pass `[]` to clear. |
-| `todos[].content` | Imperative description, e.g. `"Write the parser tests"`. |
-| `todos[].status` | `pending`, `in_progress`, or `completed`. |
-| `todos[].activeForm` | Present-continuous label, required when the item is `in_progress`. |
+Result (`details` / `structuredContent`): `{ todos, action, error? }`.
 
 Validation rules (a violation leaves the list unchanged):
 
@@ -108,20 +79,45 @@ unknown `status` or a wrong argument type is rejected by Pi before `execute`
 runs, so the host's validation message is returned instead; the list is still
 unchanged.
 
-```
+```text
 todo({ todos: [{ content: "Write the parser tests", status: "in_progress", activeForm: "Writing the parser tests" }] })
 todo({ todos: [] })   // clear
 ```
 
-### `/todos` command
+The widget is a one-line summary — progress plus the current item — so the
+active task is always visible without expanding the list:
+
+```text
+Todos · 1/3 · Writing the tests
+```
+
+In the transcript, the call line carries progress only (`todo → 1/4`); the
+result keeps the `✓` / `◒` / `◌` rows, directly under the call line. The item
+text lives on the rows, so the call line never repeats it. Collapsed, it leads
+with active work; expanded, it keeps the submitted order:
+
+```text
+todo → 1/4
+  ◒ Writing tests
+  ◌ Ship it
+  ◌ Write docs
+  ✓ Write schema
+```
+
+Progress lives on the call line, so the rows have no footer; the persistent
+widget and `/todos` still show the count. A capped collapsed rail ends with
+`… N more` and the `app.tools.expand` hint.
+
+## Commands
 
 | Command | What it does |
 |---|---|
 | `/todos` | Show the current list. An interactive, scrollable screen in `tui` mode; a notification elsewhere. |
 
-On the `/todos` screen: `↑`/`↓` or `j`/`k` scroll one row, `PgUp`/`PgDn` page,
-`g`/`G` or `Home`/`End` jump to the ends, and `Esc` (or `Ctrl-C`) closes. The
-window grows and shrinks with the terminal height, so the footer hint stays
+`/todos` is the expanded view and keeps the model's list order, so it shows the
+plan as written. On the screen: `↑`/`↓` or `j`/`k` scroll one row, `PgUp`/`PgDn`
+page, `g`/`G` or `Home`/`End` jump to the ends, and `Esc` (or `Ctrl-C`) closes.
+The window grows and shrinks with the terminal height, so the footer hint stays
 visible.
 
 The TUI label is status-dependent: an `in_progress` row shows its `activeForm`
@@ -135,13 +131,14 @@ The tool works in every mode. The persistent widget and the `/todos` screen
 require interactive (`tui`) mode; RPC and non-interactive runs simply get the
 tool result. The lag reminder is mode-independent, because it rides the agent
 boundary and not the terminal UI. While any dock screen is open (`/todos`,
-`/jobs`, `/rewind`, or the `question` questionnaire), the above-editor
-rails are hidden and return on close.
+`/jobs`, `/rewind`, or the `question` questionnaire), the above-editor rails are
+hidden and return on close.
 
 ## Configuration
 
-The widget presentation is configurable from the `todo` section of the shared
-`bytefly.json` (global document, then project document, project wins):
+One key, in the `todo` section of the shared `bytefly.json` (global document,
+then project document, project wins; resolution rules in
+[`../configuration.md`](../configuration.md#todo--the-todo-widget)):
 
 | Key | Default | Meaning |
 |---|---|---|
@@ -201,7 +198,7 @@ depends on config.
   a goal update would sink the list; the list subscribes on `pi.events` and
   re-runs `setWidget` whenever a rail above it changes, keeping the stack
   `Goal / Todos`. It never announces, so the chain cannot ping-pong. See
-  [`lib/rails.ts`](../../lib/rails.ts) and [`goal`](../goal/).
+  [`lib/rails.ts`](../../lib/rails.ts) and [`goal`](./goal.md).
 - **The reminder is a bounded nudge, not a nag.** A prompt guideline cannot
   guarantee a timely update, so the extension backs it with one forced
   continuation. `tool_execution_end` counts a successful call by an explicit
@@ -237,8 +234,7 @@ depends on config.
 | File | Responsibility |
 |---|---|
 | `index.ts` | Factory: wire the runtime, tool, command, reminder events, and session events. |
-| `types.ts` | `Todo`, `TodoStatus`, `TodoDetails`. |
-| `types.ts` | `Todo` and `TodoDetails`; re-exports `TodoStatus` / `TODO_STATUSES` from `lib/todo-state.ts`. |
+| `types.ts` | `Todo`, `TodoDetails`; re-exports `TodoStatus` / `TODO_STATUSES` from `lib/todo-state.ts`. |
 | `schema.ts` | TypeBox parameters; the parser and limits live in `lib/todo-state.ts`. |
 | `state.ts` | Status queries over the replayed list (pure). |
 | `nudge.ts` | Lag detection: mutating and file-write classification, due check, reminder text (pure). |
@@ -248,3 +244,4 @@ depends on config.
 | `runtime.ts` | Session-scoped state, lag tracking, and widget synchronization. |
 | `tools.ts` | `todo` tool registration and rendering. |
 | `commands.ts` | `/todos`. |
+| `tools.test.ts`, `runtime.test.ts`, `ui.test.ts` | The capability suites, declared in `test/suite-manifest.ts`. |

package/{extensions/watch/README.md → docs/extensions/watch.md}

@@ -6,41 +6,56 @@ the TUI, reported to the model at the next turn, or — when a watcher opts into
 `wake` — raise one triggered turn as soon as the agent is idle, including right
 after a running turn settles.
 
-```
+## Capabilities
+
+Status records what a reader can rely on: **Shipped** is in a tagged release, **Unreleased** is complete in the current tree but not yet tagged, and **Partial** has one released part and one unreleased part. Rows are ordered by user value, highest first.
+
+### Watching
+
+| Capability | How | What you get | Status |
+|---|---|---|---|
+| Tail a source | `watch add` | A file is followed from its current end unless `fromStart` is set, and a shrinking file is re-read from the start; a command's merged stdout/stderr is streamed. | Shipped |
+| Line matching | `watch add` | An include pattern with an optional exclude list, case-insensitive unless `caseSensitive` is set. | Shipped |
+| Match delivery | `watch add`, `watch matches` | Pending matches are reported to the model at the start of the next turn. | Shipped |
+| Wake turn | `watch add`, the `wakeOnMatch` key | One triggered turn as soon as the agent is idle, including right after a running turn settles. | Shipped |
+| Desktop alert | `my-pi-agent/watch-match`, `notify.enabled`, `notify.matchFound` | A wake match leaves the terminal as an OS notification. | Shipped |
+| Live status | `watch add`, the `showStatus` key | A `⊚ N` running chip and a `! N` unreported-match chip, plus one alert per unreported burst. | Shipped |
+| Command outcome | `watch status`, `watch list` | A command watcher reports its exit code or error, including the captured signal, instead of only "stopped". | Shipped |
+| Liveness detail | `watch list`, `watch status` | A running watcher names its pid and elapsed time; list rows sort running watchers first, newest next. | Shipped |
+| Transcript rendering | the `watch` transcript block | The outcome line, at most eight rows, and a `(ctrl+o to expand)` hint keep the transcript compact. | Shipped |
+| Sanitized output | `watch matches`, `/watch matches` | Model- and remote-authored output is stripped of control characters before it reaches a terminal or a notification. | Shipped |
+
+### Reading and managing
+
+| Capability | How | What you get | Status |
+|---|---|---|---|
+| Watcher inventory | `watch list`, `watch status`, `/watch list` | Each watcher's source, state, and line and match counts, singly or as a table. | Shipped |
+| Matched lines | `watch matches`, `/watch matches` | The matching lines, sanitized and read from a bounded history. | Shipped |
+| Raw output | `watch logs`, `/watch logs` | The merged output tail, for checking a pattern that matches nothing. | Shipped |
+| Watcher screen | `/watch` | A dock screen with the watcher list, the focused watcher's detail, and its matches pane. | Shipped |
+| Stopping and cleanup | `watch stop`, `watch clear`, `/watch stop`, `/watch clear` | A running watcher stops, stopped ones are removed, and `all` takes the running ones too. | Shipped |
+
+## Quickstart
+
+```bash
 pi --extension ./extensions/watch    # load just this extension
 pi -e .                              # load the whole @gavin-hu/my-pi-agent package
 pi install ./                        # install the package
 ```
 
-## Tool rendering
+```text
+watch add { "path": "server.log", "pattern": "ERROR|FATAL", "label": "server" }
+/watch
+```
 
-How the `watch` call and its result look in the transcript — the call line as a headline and the body indented under it:
+## Tool
 
 <img src="../../assets/tools/watch.png" alt="The watch tool's result: the call line and an indented body with a running file watcher and an exited command watcher" width="100%">
 
-## What it does
-
-- Registers one model-callable tool, `watch`, with actions `add`, `list`,
-  `status`, `matches`, `logs`, `stop`, and `clear`.
-- Tails a text file by polling it, or a shell command by streaming its merged
-  stdout/stderr. A file is followed from its current end unless `fromStart` is
-  set; a shrinking file (rotation/truncation) is re-read from the start.
-- Matches with a regular expression and an optional exclude list,
-  case-insensitive by default and case-sensitive with `caseSensitive: true`.
-- Shows a `⊚ N` running-watcher chip and a `! N` unreported-match chip, raises a
-  `ctx.ui.notify` alert once per unreported burst, and offers a `/watch` screen.
-- Reports pending matches to the model at the start of the next turn, or wakes a
-  turn for a watcher added with `wake: true` as soon as the agent is idle. A wake
-  that arrives mid-turn is raised once that run settles. A wake also leaves the
-  terminal: the
-  match is announced on `my-pi-agent/watch-match`, which bytefly's
-  [notify](../bytefly/README.md#notifications) capability turns into a desktop
-  notification when `notify.enabled` and `notify.matchFound` are on.
-- Watchers are session-only: `shutdown` flushes a partial line, sends SIGTERM to
-  every command's process tree (escalating to SIGKILL after a short grace), and
-  clears the chips.
-
-## Tool
+`watch` — `exposure: "direct"` (the default), `executionMode: "sequential"`,
+read-write, destructive, open-world, not idempotent. One tool with seven
+actions; a watcher outlives the call, so `add` returns an id immediately and the
+rest read it back.
 
 | Field | Value |
 |---|---|
@@ -104,15 +119,10 @@ capped at eight rows, minus the selection-marker column (`listRow`'s
 render the tail of the sanitized lines, with the hidden count noted above and
 `(ctrl+o to expand)`.
 
-### Command: /watch
-
-<img src="../../assets/watch.png" alt="The /watch screen: a Watch title rule, a `2 watchers · 1 running · 3 matches` summary, a selected file watcher and a finished command watcher, the focused watcher's path and pattern with its line and match counts, and the key hints" width="100%">
+## Commands
 
-A bare `/watch` opens a dock screen listing watchers. `j`/`k` (and the wheel)
-move, `Enter`/`l` opens a watcher's matches, `x`/`K` stops a running watcher
-behind a `y/N` confirm, `d` clears stopped ones behind a `y/N` confirm, `Esc`
-closes, and `g`/`G` and PgUp/PgDn scroll the matches pane. Subcommands work in
-every mode:
+`/watch` manages watchers in every mode; a bare `/watch` opens the screen in the
+TUI and prints the list everywhere else.
 
 | Command | Effect |
 |---|---|
@@ -123,6 +133,13 @@ every mode:
 | `/watch stop <id>` | Stop a watcher. |
 | `/watch clear [all]` | Remove stopped watchers, or every watcher with `all`. |
 
+<img src="../../assets/screens/watch.png" alt="The /watch screen: a Watch title rule, a `2 watchers · 1 running · 3 matches` summary, a selected file watcher and a finished command watcher, the focused watcher's path and pattern with its line and match counts, and the key hints" width="100%">
+
+A bare `/watch` opens a dock screen listing watchers. `j`/`k` (and the wheel)
+move, `Enter`/`l` opens a watcher's matches, `x`/`K` stops a running watcher
+behind a `y/N` confirm, `d` clears stopped ones behind a `y/N` confirm, `Esc`
+closes, and `g`/`G` and PgUp/PgDn scroll the matches pane.
+
 ## Behaviour by mode
 
 - **tui** — the status chips, the match alert, and the `/watch` dock screen.
@@ -132,8 +149,9 @@ every mode:
 
 ## Configuration
 
-The `watch` section of the shared `bytefly.json` (global then project). Every
-value is clamped; a missing or malformed document is ignored.
+The `watch` section of the shared `bytefly.json`; resolution rules in
+[`../configuration.md`](../configuration.md#watch--file-and-command-watchers).
+Every value is clamped; a missing or malformed document is ignored.
 
 | Key | Default | Meaning |
 |---|---|---|

package/{extensions/web-access/README.md → docs/extensions/web-access.md}

@@ -3,53 +3,57 @@
 One extension, two tools, keyless and dependency-free (native `fetch`, no
 `curl`): `web_search` runs general web search through a pluggable provider
 (keyless DuckDuckGo by default; optional SearXNG or Brave), and `web_fetch`
-reads a URL as text. Both are `direct` and active by default, and
-read the `web-access` section of the shared `bytefly.json`.
+reads a URL as text. Both are `direct`, active on registration, and read the
+`web-access` section of the shared `bytefly.json`.
 
-```
+## Capabilities
+
+Status records what a reader can rely on: **Shipped** is in a tagged release, **Unreleased** is complete in the current tree but not yet tagged, and **Partial** has one released part and one unreleased part. Rows are ordered by user value, highest first.
+
+| Capability | How | What you get | Status |
+|---|---|---|---|
+| Web search | `web_search` | Titles, URLs, snippets, and a direct answer when the provider has one. | Shipped |
+| Page reading | `web_fetch` | A URL's readable text, with links kept as `[text](url)`. | Shipped |
+| Structured bodies | `web_fetch` | A JSON, CSV, or XML body arrives as an aligned table or an indented tree instead of the raw blob. | Unreleased |
+| Write-style fetch | `web_fetch` (`method`, `headers`, `body`) | POST with custom headers and a bounded body, so form and GraphQL endpoints are reachable; a body implies POST. | Shipped |
+| Paged output | `web_fetch`, `startIndex`, `maxChars` | A long page arrives in budgeted slices and names the offset that continues it. | Shipped |
+| Passage search | `web_fetch`, `find`, `contextChars` | Matching passages instead of the whole page, with a note when a cap hid the rest. | Shipped |
+| Private-address refusal | `web_fetch`, `fetch.allowPrivateHosts` | Loopback, private, link-local, and metadata targets are refused on the first request and on every redirect hop. | Shipped |
+| Charset-aware decoding | `web_fetch` | A GBK/GB18030 or `<meta charset>` page reads as text instead of replacement characters. | Shipped |
+| Page cache | `web_fetch`, `refresh`, `fetch.cacheEnabled` | Paging and `find` reuse one fetch, and `refresh: true` bypasses a stale entry. | Shipped |
+| PDF extraction | `web_fetch`, `fetch.pdfEnabled` | A PDF becomes text once the optional `unpdf` package is installed. | Shipped |
+| Provider choice | `search.provider`, `search.endpoint`, `search.apiKeyEnv` | The keyless default works with no configuration; SearXNG or Brave add full result lists. | Shipped |
+| Structured result | `nextIndex`, `matchesTruncated`, `cached` | A caller can continue paging and tell a capped match set from a complete one without parsing the text. | Shipped |
+| Politeness throttle | `minIntervalMs` | Search requests are spaced instead of hammering a provider on a rapid series of calls. | Shipped |
+
+## Quickstart
+
+```bash
 pi --extension ./extensions/web-access    # load just this extension
 pi -e .                                   # load the whole @gavin-hu/my-pi-agent package
 pi install ./                             # or install the package
 ```
 
-## Tool rendering
+```text
+web_search({ query: "pi coding agent" })  # keyless, no configuration
+web_fetch({ url: "https://pi.dev/" })     # the page as readable text
+```
 
-How the `web_search` and `web_fetch` calls and their results look in the transcript — the call line as a headline and the body indented under it:
+## Tools
 
 <img src="../../assets/tools/web_search.png" alt="The web_search tool's call and result: the call line and an indented body with the query and the ranked results" width="100%">
 
 <img src="../../assets/tools/web_fetch.png" alt="The web_fetch tool's call and result: the call line and an indented body with the URL, title, and fetched text preview" width="100%">
 
-## What it does
-
-- Registers `web_search`, general web search through a pluggable provider:
-  keyless DuckDuckGo by default, or a self-hosted SearXNG instance / keyed Brave
-  search. Returns titles, URLs, snippets, and a direct answer when available,
-  always with no configuration required.
-- Registers `web_fetch`, which fetches a URL (GET or POST) and
-  returns readable text with links kept as `[text](url)`.
-- Decodes by the byte-order mark, the declared `Content-Type` charset, or the
-  page's own `<meta charset>`, so a GBK/GB18030 page reads as text instead of
-  replacement characters.
-- Pages long output and reports the next `startIndex` (`nextIndex` in
-  `structuredContent`); `find` returns matching passages instead of the whole
-  page, and says when the match cap hid further hits.
-- Re-validates every redirect hop against private/internal addresses, and
-  refuses transport headers and oversized bodies.
-- Optionally extracts PDFs (`unpdf`); the package loads lazily and is never
-  required.
-- Caches fetched pages in one in-process cache per registration, so paging and
-  `find` do not refetch; the cache is cleared on `session_start` and
-  `session_shutdown`.
-
-## Tools
+Both tools are `direct` (the default) and active on registration; neither needs
+configuration to run.
 
 | Tool | What it does |
 |---|---|
 | `web_search` | General web search; keyless by default, SearXNG or Brave optional. |
 | `web_fetch` | Fetch a URL and return readable text, pageable and searchable. |
 
-```
+```text
 web_search({ query: "pi coding agent" })
 web_fetch({ url: "https://pi.dev/" })
 web_fetch({ url: "https://pi.dev/", find: ["documentation", "extensions"] })
@@ -68,7 +72,7 @@ flight it draws a railed `Searching… 8s` or `Fetching example.com… 3s` line,
 a finished call appends its duration. A failed page or a host-rejected call
 renders one sanitized `Error:` line in the error role.
 
-```
+```text
 web_search "pi coding agent" (max 8)
 
 via searxng · 8 results · 2.1s
@@ -80,7 +84,7 @@ Answer: Pi is a coding agent harness …
 +3 more (ctrl+o to expand)
 ```
 
-```
+```text
 web_fetch https://pi.dev/
 Pi — a coding-agent component · 200 · 12k chars (cached) · 3.4s
 ```
@@ -90,9 +94,10 @@ which URL supplied the text.
 
 ## Configuration
 
-Merged from the `web-access` section of the global `bytefly.json` and the
-project `<cwd>/.pi/bytefly.json` (project wins). Missing or malformed documents
-and unknown keys are ignored.
+The `web-access` section of the shared `bytefly.json`: `search` configures
+`web_search`, `fetch` configures `web_fetch`. Resolution rules are in
+[`../configuration.md#web-access--search-and-fetch`](../configuration.md#web-access--search-and-fetch),
+which also lists every key with its default and accepted values.
 
 ```jsonc
 // bytefly.json → "web-access"
@@ -183,7 +188,7 @@ whether it came from an explicit choice, the `auto` rule, or the keyless default
 the API-key variable when the provider takes one and no literal `apiKey` is set,
 and whether that variable is present; and, only on an explicit `refresh`, whether
 the configured `endpoint` answers. The key is never reported, only its variable
-name. See [`bytefly`](../bytefly/README.md#what-it-does) for the report.
+name. See [`bytefly`](./bytefly.md#capabilities) for the report.
 
 | Parameter | Type | Notes |
 |---|---|---|
@@ -199,11 +204,16 @@ web_search({ query: "python list comprehension", maxResults: 3 })
 ## `web_fetch`
 
 GETs (or POSTs) an http(s) URL and returns readable text. HTML becomes
-plain text with links kept as `[text](url)`; JSON/text/XML is returned as-is;
-PDFs are text-extracted when the optional `unpdf` package is installed; other
-binary content is reported by type and size. Redirects are followed manually and
-**every hop is re-validated**, so a public URL cannot redirect into a private
-address. Loopback, private, link-local, and cloud-metadata addresses are refused.
+plain text with links kept as `[text](url)`; a JSON, CSV, or XML body becomes a
+table or an indented tree ([Structured bodies](#structured-bodies)); other textual
+content is returned as-is; PDFs are text-extracted when the optional `unpdf`
+package is installed; other binary content is reported by type and size. Bodies
+are decoded by the
+byte-order mark, the declared `Content-Type` charset, or the page's own
+`<meta charset>`, so a GBK/GB18030 page reads as text instead of replacement
+characters. Redirects are followed manually and **every hop is re-validated**, so
+a public URL cannot redirect into a private address. Loopback, private,
+link-local, and cloud-metadata addresses are refused.
 
 | Parameter | Type | Notes |
 |---|---|---|
@@ -237,6 +247,32 @@ web_fetch({ url: "https://pi.dev/", find: ["documentation", "extensions"] })
 web_fetch({ url: "https://api.example/graphql", method: "POST", headers: { "Content-Type": "application/json" }, body: '{"query":"{ viewer { login } }"}' })
 ```
 
+### Structured bodies
+
+A JSON, CSV, or XML body is rendered as text instead of the blob it downloaded: an
+array of records becomes a table with the union of their keys and padded columns,
+an object becomes indented `key: value` lines, comma-separated rows become the same
+table, and XML becomes an indented tree with attributes as `@name`. The declared
+content type decides first; a body that declares nothing usable (`text/plain`) is
+sniffed only when it has a shape — an opening brace, a uniform comma count, a root
+tag — and anything that does not parse keeps its raw form, so a malformed document
+is never made harder to read than it already was.
+
+```text
+URL: https://api.example/users
+Status: 200 (application/json)
+
+id  name   role
+1   Ada    admin
+2   Grace  user
+```
+
+Rendering happens once per fetch, before the result is cached, so `startIndex`,
+`find`, and a second call all read the same string. Two bounds keep it honest: a
+body past `MAX_SOURCE_CHARS` is returned raw, because a paged raw body beats a
+rendered one, and the rendered text stops at `MAX_RENDERED_CHARS` with a note saying
+so — both constants live in `fetch/structured.ts`.
+
 ### Optional dependencies
 
 PDF extraction needs a package that is not installed by default; it is loaded
@@ -259,6 +295,11 @@ the content type and size.
 - **Heuristic extraction**: `web_fetch` strips scripts/styles/nav/footer and
   prefers `<main>`/`<article>`, but is not a full readability engine. PDF
   extraction is optional (`unpdf`) and off unless installed.
+- **A structured body is rendered by a reader, not a parser.** JSON, CSV, and XML
+  are read well enough to show a shape — CSV by RFC 4180 quoting, XML by balancing
+  tags — but a body it cannot parse keeps its raw form rather than a partial one,
+  and the `text/plain` sniff is a heuristic: a page of prose with a uniform comma
+  in every line reads as a one-row table.
 - **`find` output is budgeted.** With `maxMatches: 50` and `contextChars: 2000`
   the passages alone would exceed `maxOutputChars`; the result shows the passages
   that fit and names the knob to change.
@@ -296,11 +337,11 @@ the content type and size.
   `config.ts`, `schema.ts`, `types.ts`, `format.ts`, and `tool.ts`, and do not
   import each other; `index.ts` only registers. Shared text and HTML helpers
   live at the top level (`text.ts`, `html.ts`, `transcript.ts`, `http.ts`).
-- **Shared transport.** [`http.ts`](./http.ts) holds the fetch runner
+- **Shared transport.** [`http.ts`](../../extensions/web-access/http.ts) holds the fetch runner
   (`HttpRunner`, `createFetchRunner`, HTTP error types), used by both halves and
   injectable for tests. It reads the body through a byte-counting stream, so
   `maxBytes` stops a chunked or compressed response before it is buffered whole.
-- **Charset-aware decoding.** [`charset.ts`](./charset.ts) is the one place a
+- **Charset-aware decoding.** [`charset.ts`](../../extensions/web-access/charset.ts) is the one place a
   byte body becomes text: BOM first, then the declared `Content-Type` charset,
   then the page's own `<meta charset>`, then UTF-8. An unknown label falls back
   to UTF-8 instead of failing the fetch.
@@ -311,9 +352,9 @@ the content type and size.
   lazily and falls back to a binary note when it is absent.
 - **Per-hop SSRF validation.** Redirects are followed with `redirect: "manual"`
   under one shared deadline, re-resolving and re-checking each `Location`.
-- **Pluggable search providers.** [`registry.ts`](./search/registry.ts) holds a
+- **Pluggable search providers.** [`registry.ts`](../../extensions/web-access/search/registry.ts) holds a
   value-only provider list (`duckduckgo`, `searxng`, `brave`) and a pure
-  `resolveProvider`; [`search.ts`](./search/search.ts) owns selection, key
+  `resolveProvider`; [`search.ts`](../../extensions/web-access/search/search.ts) owns selection, key
   resolution, a per-instance throttle, and error mapping. A provider with no
   answer and no results reports `provider: "none"`.
 - **Find-in-page.** `exact` is case-sensitive, `insensitive` lowercases both
@@ -328,17 +369,17 @@ the content type and size.
   truncation note against `maxChars`, so a paged page always carries its
   `startIndex` hint inside the limit. `formatMatches` applies the same rule to
   `find`: it includes the passages that fit and names the knob that hid the rest.
-- **Code points without a copy.** [`text.ts`](./text.ts) counts and slices by
+- **Code points without a copy.** [`text.ts`](../../extensions/web-access/text.ts) counts and slices by
   walking the string in place, so a multi-megabyte page is never expanded into an
   array of one-character strings on every call.
 - **One JSON boundary per concern.** `search/json.ts` owns `requestJson` (status
   check and parse error) and `collectResults` (title + http(s) URL + de-dupe), so
   the three providers cannot drift on either.
-- **Page cache.** [`fetch/cache.ts`](./fetch/cache.ts) is a factory
+- **Page cache.** [`fetch/cache.ts`](../../extensions/web-access/fetch/cache.ts) is a factory
   (`createPageCache`) holding an LRU/TTL map keyed by requested URL, so paging and
   find are a single fetch; entries evict by count and total bytes, and one
   instance belongs to one registration.
-- **Transcript sanitization and reuse.** [`transcript.ts`](./transcript.ts)
+- **Transcript sanitization and reuse.** [`transcript.ts`](../../extensions/web-access/transcript.ts)
   holds `oneLine`, shared by both tools to collapse untrusted web text to one
   sanitized, optionally clipped line; the renderers reuse
   `context.lastComponent` instead of allocating each render.
@@ -357,6 +398,7 @@ the content type and size.
 | `search/tool.ts` | `web_search` definition, activation, rendering |
 | `search/provider.ts` | Provider interface (`SearchProvider`, `ProviderContext`) |
 | `search/registry.ts` | Value-only provider list, `resolveProvider`, labels |
+| `search/facts.ts` | What this session resolved to, published for `doctor` (`providerFacts`) |
 | `search/config.ts` | Search config type, defaults, clamping, API-key resolution |
 | `search/schema.ts` | Params/output schema, `resolveRequest()` |
 | `search/search.ts` | Provider resolution, throttle, and outcome mapping |
@@ -368,10 +410,10 @@ the content type and size.
 | `fetch/tool.ts` | `web_fetch` definition, activation, rendering |
 | `fetch/config.ts` | Fetch config type, defaults, clamping |
 | `fetch/schema.ts` | Params/output schema, `resolveRequest()` |
-| `fetch/page.ts` | Fetch + redirect-following + content-type routing |
+| `fetch/page.ts` | Fetch + redirect-following + content-type routing; PDF extraction is injected via `FetchDeps` from the shared `lib/pdf.ts` |
+| `fetch/structured.ts` | JSON/CSV/XML body → readable text, or `undefined` to keep the raw body |
 | `fetch/ssrf.ts` | URL/host validation and private-address blocking |
 | `fetch/extract.ts` | HTML → title + readable text |
-| `fetch/page.ts` | Fetch + redirect-following + content-type routing; PDF extraction is injected via `FetchDeps` from the shared `lib/pdf.ts` |
 | `fetch/find.ts` | Passage search (exact/insensitive/fuzzy) |
 | `fetch/cache.ts` | Per-registration LRU/TTL page cache (`createPageCache`) |
 | `fetch/format.ts`, `fetch/types.ts` | Result text and data types |
@@ -386,7 +428,8 @@ charset decoding (declared, `<meta>`, and BOM precedence), a streamed body over
 the byte cap, redirect re-validation (per-hop SSRF with literal IPs),
 method/header/body validation, the nested config merge and section isolation,
 extraction, formatting/paging through the tool (including a regression test that
-a truncated page keeps its `startIndex` hint), find and its cap/budget notices,
+a truncated page keeps its `startIndex` hint), structured rendering per format and
+its malformed fallbacks, find and its cap/budget notices,
 the per-registration cache, the SSRF guard, the injected PDF dependency, and both
 tools' registration/execution. Tests inject the runner, throttle, and PDF extractor
 through the tools' dependencies, so no network or PDF library is needed; the