@shoalrun/cli 0.0.0-stage → 0.1.0-alpha.11

package/agent-docs/chat/settings.sections/related-reading.md

@@ -0,0 +1,32 @@
+---
+id: reference.chat-config
+title: Retrieval and chat settings
+summary: Look up embedding, chat-model, scoring, and answer-reuse configuration.
+type: reference
+area: discovery
+subarea: discovery-chat
+subjectLifecycle: experimental
+canonical: /chat/settings
+---
+
+# Retrieval and chat settings
+
+> Documentation index: [llms.min.txt](https://www.shoalrun.com/llms.min.txt) · Document manifest: [documents.json](https://www.shoalrun.com/documents.json)
+
+Navigation section: [Docs](https://www.shoalrun.com/.well-known/shoalrun/navigation/docs-50f26fe22670.txt) (ID: docs). [All sections and pages](https://www.shoalrun.com/llms.min.txt)
+
+Breadcrumbs: [Docs](https://www.shoalrun.com/what-is-shoalrun.md) > Search and chat > Reference > Retrieval and chat settings
+
+**Updated: 2026-10-25.**
+
+**Status: Experimental.**
+
+Parent page: [Full Markdown page](/chat/settings.md)
+
+Use this reference to look up fields after choosing a setup. For a working sequence, start with [local chat setup](https://www.shoalrun.com/chat/setup.md).
+
+`retrieval` configures embeddings; `chat` configures answer generation and optional checks. Both sections are optional. A normal site build validates them without calling these providers.
+
+## Related reading
+
+No matching documents.

package/agent-docs/chat/settings.sections/send-extra-request-fields.md

@@ -0,0 +1,42 @@
+---
+id: reference.chat-config
+title: Retrieval and chat settings
+summary: Look up embedding, chat-model, scoring, and answer-reuse configuration.
+type: reference
+area: discovery
+subarea: discovery-chat
+subjectLifecycle: experimental
+canonical: /chat/settings
+---
+
+# Retrieval and chat settings
+
+> Documentation index: [llms.min.txt](https://www.shoalrun.com/llms.min.txt) · Document manifest: [documents.json](https://www.shoalrun.com/documents.json)
+
+Navigation section: [Docs](https://www.shoalrun.com/.well-known/shoalrun/navigation/docs-50f26fe22670.txt) (ID: docs). [All sections and pages](https://www.shoalrun.com/llms.min.txt)
+
+Breadcrumbs: [Docs](https://www.shoalrun.com/what-is-shoalrun.md) > Search and chat > Reference > Retrieval and chat settings
+
+**Updated: 2026-10-25.**
+
+**Status: Experimental.**
+
+Parent page: [Full Markdown page](/chat/settings.md)
+
+Use this reference to look up fields after choosing a setup. For a working sequence, start with [local chat setup](https://www.shoalrun.com/chat/setup.md).
+
+`retrieval` configures embeddings; `chat` configures answer generation and optional checks. Both sections are optional. A normal site build validates them without calling these providers.
+
+## Send extra request fields
+
+Some services accept request fields that Shoalrun doesn't know about. `chat.body` adds them to every request. On OpenRouter, the useful ones are `reasoning`, which controls whether a model thinks before it answers, and `provider`, which picks who serves the model.
+
+```yaml title="docs.config.yaml"
+chat:
+  model: deepseek/deepseek-v4-flash
+  body: { reasoning: { enabled: false } }
+```
+
+The fields can add to a request but can't replace what Shoalrun sets, which is `model`, `messages`, `max_tokens`, `temperature`, and `stream`. Use `chat.model`, `chat.maxTokens`, and `chat.temperature` for those. A field whose name looks like a secret, such as a key, token, or password, fails validation, because the config never holds a key. Each fallback and escalation model inherits `body` and can replace it with its own. An empty `body: {}` clears it. Another service may reject a field it doesn't know, so check what yours accepts.
+
+Service-specific fields can affect latency, cost, and quality. Test them with your own questions and model; support and effects vary by service. Shoalrun does not interpret those extra fields.

package/agent-docs/chat/settings.sections/use-more-than-one-model.md

@@ -0,0 +1,62 @@
+---
+id: reference.chat-config
+title: Retrieval and chat settings
+summary: Look up embedding, chat-model, scoring, and answer-reuse configuration.
+type: reference
+area: discovery
+subarea: discovery-chat
+subjectLifecycle: experimental
+canonical: /chat/settings
+---
+
+# Retrieval and chat settings
+
+> Documentation index: [llms.min.txt](https://www.shoalrun.com/llms.min.txt) · Document manifest: [documents.json](https://www.shoalrun.com/documents.json)
+
+Navigation section: [Docs](https://www.shoalrun.com/.well-known/shoalrun/navigation/docs-50f26fe22670.txt) (ID: docs). [All sections and pages](https://www.shoalrun.com/llms.min.txt)
+
+Breadcrumbs: [Docs](https://www.shoalrun.com/what-is-shoalrun.md) > Search and chat > Reference > Retrieval and chat settings
+
+**Updated: 2026-10-25.**
+
+**Status: Experimental.**
+
+Parent page: [Full Markdown page](/chat/settings.md)
+
+Use this reference to look up fields after choosing a setup. For a working sequence, start with [local chat setup](https://www.shoalrun.com/chat/setup.md).
+
+`retrieval` configures embeddings; `chat` configures answer generation and optional checks. Both sections are optional. A normal site build validates them without calling these providers.
+
+## Use more than one model
+
+Use `fallback` for provider failures and `escalate` for answers flagged by checks. Both are optional:
+
+```yaml title="docs.config.yaml"
+chat:
+  provider: openai
+  model: deepseek/deepseek-v4-flash
+  baseUrl: https://openrouter.ai/api/v1
+  keyEnv: OPENROUTER_API_KEY
+  fallback: [{ model: openai/gpt-6-luna }]
+  escalate: { model: z-ai/glm-5.3-flash, when: [unsupported-claim, uncited] }
+```
+
+Each extra model inherits the provider, address, key variable, and limits from the main settings. Name only what differs. A model on a different service can set its own `baseUrl` and `keyEnv`. The `--model` flag on `ask` replaces the main model and leaves the others alone.
+
+### Fallback
+
+If the main model errors, times out, or returns no text, `ask` tries the next model in the list. Use a model from a different lab, so one outage doesn't stop the chat. The result records which model answered and which failed, with the reason. If every model fails, the error lists all of them.
+
+### Escalation
+
+The main model answers first, and the checks run. If one of the `when` reasons applies, `ask` asks the escalation model the same question with the same passages and runs the same checks. It shows whichever answer has fewer flags, and the first one if they tie. The supported reasons are:
+
+| Reason              | Fires when                                                 |
+| ------------------- | ---------------------------------------------------------- |
+| `unsupported-code`  | The answer has code that isn't in the passages.            |
+| `uncited`           | The answer cites no passage.                               |
+| `unsupported-claim` | A sentence fails the claims check. It needs `chat.verify`. |
+
+Without `when`, the reasons are `unsupported-code` and `uncited`, plus `unsupported-claim` when `verify` is set. The code check is a heuristic that can misjudge a valid example. A second answer from a thinking model adds seconds. So many sites limit `when` to `unsupported-claim` and `uncited`. A declined answer or a direct excerpt is never escalated. If the escalation model fails, the first answer is shown with a note. `--no-escalate` on `ask` turns it off for one run.
+
+Choose an escalation model using reviewed questions that your primary model handles poorly. A larger model is not automatically a better fit.

package/agent-docs/chat/settings.sections/when-the-scorer-is-down.md

@@ -0,0 +1,42 @@
+---
+id: reference.chat-config
+title: Retrieval and chat settings
+summary: Look up embedding, chat-model, scoring, and answer-reuse configuration.
+type: reference
+area: discovery
+subarea: discovery-chat
+subjectLifecycle: experimental
+canonical: /chat/settings
+---
+
+# Retrieval and chat settings
+
+> Documentation index: [llms.min.txt](https://www.shoalrun.com/llms.min.txt) · Document manifest: [documents.json](https://www.shoalrun.com/documents.json)
+
+Navigation section: [Docs](https://www.shoalrun.com/.well-known/shoalrun/navigation/docs-50f26fe22670.txt) (ID: docs). [All sections and pages](https://www.shoalrun.com/llms.min.txt)
+
+Breadcrumbs: [Docs](https://www.shoalrun.com/what-is-shoalrun.md) > Search and chat > Reference > Retrieval and chat settings
+
+**Updated: 2026-10-25.**
+
+**Status: Experimental.**
+
+Parent page: [Full Markdown page](/chat/settings.md)
+
+Use this reference to look up fields after choosing a setup. For a working sequence, start with [local chat setup](https://www.shoalrun.com/chat/setup.md).
+
+`retrieval` configures embeddings; `chat` configures answer generation and optional checks. Both sections are optional. A normal site build validates them without calling these providers.
+
+## When the scorer is down
+
+The scorer is what keeps a question the docs don't cover from reaching the model. If it fails, by timing out, running out of budget, returning an error, or having no key, there are two ways to go.
+
+- **`search` (the default).** The chat doesn't call the model. It shows the closest pages from retrieval and says it can't check answers right now. Nobody can use an outage to get free model output for any question, and readers still get somewhere to go.
+- **`answer`.** The model answers without the gate and the claim check, and the answer carries a note that the checks were skipped. Use this when a reader-facing outage costs more than the risk. On a public site, every question then reaches the model, and only the rate limits and token caps stand in the way.
+
+```yaml title="docs.config.yaml"
+chat:
+  rerank: { provider: jev, onError: answer }
+```
+
+Neither mode caches anything from an outage, so an unchecked answer can't outlive it. This applies only when a scorer is configured. With no `rerank`, the model answers every question, because you chose that.

package/agent-docs/chat/setup.md

@@ -0,0 +1,97 @@
+---
+id: guide.set-up-chat
+title: Set up local documentation chat
+summary: Build a passage index, configure a chat service, and verify cited
+  answers locally.
+type: how-to
+area: discovery
+subarea: discovery-chat
+subjectLifecycle: experimental
+canonical: /chat/setup
+---
+
+# Set up local documentation chat
+
+> Documentation index: [llms.min.txt](https://www.shoalrun.com/llms.min.txt) · Document manifest: [documents.json](https://www.shoalrun.com/documents.json)
+
+Navigation section: [Docs](https://www.shoalrun.com/.well-known/shoalrun/navigation/docs-50f26fe22670.txt) (ID: docs). [All sections and pages](https://www.shoalrun.com/llms.min.txt)
+
+Breadcrumbs: [Docs](https://www.shoalrun.com/what-is-shoalrun.md) > Search and chat > Documentation chat > Set up locally
+
+**Updated: 2026-10-26.**
+
+**Status: Experimental.**
+
+Set up a local chat panel, then check that its answers cite the right documentation. Start in the root of a Shoalrun project that already builds, with the CLI available in your terminal.
+
+You need a chat-service account and its API key. Meaning-based retrieval also needs an embedding provider and key. These commands make provider requests; they do not deploy your site.
+
+For keyword-only retrieval, omit `retrieval` from the example below and omit `--embed` from the indexing command. You still need the passage index and a chat service.
+
+## Set up and try the chat
+
+This example uses Voyage for embeddings and an OpenAI-compatible chat endpoint through OpenRouter. Choose providers and model IDs available to your accounts.
+
+1. ## Configure the providers
+
+   Add these sections to `docs.config.yaml`. Store `VOYAGE_API_KEY` and `OPENROUTER_API_KEY` in your terminal environment, not in the YAML file.
+
+   ```yaml title="docs.config.yaml"
+   retrieval:
+     embeddings:
+       provider: voyage
+       model: voyage-4-large
+       dimension: 1024
+       queryModel: voyage-4-lite
+
+   chat:
+     provider: openai
+     model: deepseek/deepseek-v4-flash
+     baseUrl: https://openrouter.ai/api/v1
+     keyEnv: OPENROUTER_API_KEY
+   ```
+
+2. ## Build and index the documentation
+
+   Build the site, then split its Markdown pages into passages. `--embed` sends those passages to the configured embedding provider.
+
+   ```sh
+   shoalrun build docs.config.yaml site-dist
+   shoalrun index site-dist --embed --config docs.config.yaml
+   ```
+
+3. ## Ask a question from the terminal
+
+   Replace the sample question with one that your own pages answer.
+
+   ```sh
+   shoalrun ask site-dist "how do I reset my password" --config docs.config.yaml
+   ```
+
+   Check the answer's citations and open the source pages. Confirm that the sources support the answer, rather than accepting citations alone as proof.
+
+4. ## Open the local panel
+
+   Start the server and open the address it prints. Ask the same question in the panel and follow its source links. Stop the server with **Ctrl+C** when finished.
+
+   ```sh
+   shoalrun serve site-dist --ask --config docs.config.yaml
+   ```
+
+## If something goes wrong
+
+A missing-key error names the environment variable to set in the same terminal. For missing or stale sources, rebuild the site and run `index` again. Keyword-only retrieval can miss questions that use different wording from the pages.
+
+This minimal setup has no relevance scorer to decline out-of-scope questions. Before broader use, review [answer checks](https://www.shoalrun.com/chat/checks.md) and [hosting limits](https://www.shoalrun.com/chat/host.md).
+
+To record local questions, add `--log` with a path outside the site output. Logging is optional; read [question logging](https://www.shoalrun.com/chat/host.md) before enabling it. The [CLI reference](https://www.shoalrun.com/cli.md) defines the other flags and report fields.
+
+## Related reading
+
+### Reference while working
+
+- [Retrieval and chat settings](https://www.shoalrun.com/chat/settings.md) — Look up provider fields, environment variables, and answer settings. (Status: experimental.)
+
+### Related topics
+
+- [How documentation answers work](https://www.shoalrun.com/chat.md) — Understand the layers behind cited answers before choosing additional services. (Status: experimental.)

package/agent-docs/chat.md

@@ -0,0 +1,102 @@
+---
+id: guide.retrieval-chat
+title: How documentation answers work
+summary: Understand how passages, embeddings, scoring, and a chat model produce
+  cited answers.
+type: explanation
+area: discovery
+subarea: discovery-chat
+subjectLifecycle: experimental
+canonical: /chat
+---
+
+# How documentation answers work
+
+> Documentation index: [llms.min.txt](https://www.shoalrun.com/llms.min.txt) · Document manifest: [documents.json](https://www.shoalrun.com/documents.json)
+
+Navigation section: [Docs](https://www.shoalrun.com/.well-known/shoalrun/navigation/docs-50f26fe22670.txt) (ID: docs). [All sections and pages](https://www.shoalrun.com/llms.min.txt)
+
+Breadcrumbs: [Docs](https://www.shoalrun.com/what-is-shoalrun.md) > Search and chat > Documentation chat > How answers work
+
+**Updated: 2026-10-26.**
+
+**Status: Experimental.**
+
+Documentation chat reads a site you have already built. It is optional: a normal `shoalrun build` does not call the retrieval or chat providers.
+
+Chat first finds relevant passages, then uses them to produce a cited answer. A passage is a short part of a built Markdown page with a link back to its source. Some configurations can return an existing answer or an excerpt instead of generating new text.
+
+For commands, start with [local chat setup](https://www.shoalrun.com/chat/setup.md). [Search and chat](https://www.shoalrun.com/search-and-chat.md) compares chat with the page-search box.
+
+## How the layers fit
+
+Each layer adds a retrieval or answer capability and has its own prerequisites.
+
+| Layer            | What it adds                                                                                                                                                               | It needs                                       |
+| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- |
+| Keyword matching | Finds the most relevant passages by their words. It is what the chat uses when there are no vectors, or the embedding service is unavailable.                              | The passage index. No key.                     |
+| Semantic search  | Finds passages by meaning, so a question matches even when it shares no words with the page.                                                                               | An embedding service and key.                  |
+| Relevance scorer | Reorders the results and declines a question the docs don't cover. It checks each sentence of an answer, and shows a passage word for word when one is clearly the answer. | A scorer service that speaks the Jev protocol. |
+| Chat model       | Writes a cited answer from the best passages.                                                                                                                              | A chat service and key.                        |
+| Hosting          | Runs the chat beside your site with spending caps and an off switch.                                                                                                       | Vercel, and a key-value store.                 |
+
+The site-search box returns links from its own page index. Chat retrieves passages to answer a question. An embedding service helps match different wording; a scorer rates whether the passages support an answer. The [settings reference](https://www.shoalrun.com/chat/settings.md) describes supported providers and the Jev scoring protocol.
+
+Choose services and models in config. Keep their keys in the environment variables named by that config.
+
+## How a question is answered
+
+The diagram shows a chat with optional reuse and scoring enabled. A question stops at an existing answer or direct excerpt when the configured checks accept one.
+
+**How a question is answered**
+
+A question first looks for a cached answer or an answer card. Otherwise passages are found by keyword and meaning, and the relevance scorer decides. A question the docs don't cover gets a list of pages. One clear passage is shown as an excerpt. Everything else goes to the chat model, whose answer is checked before it is shown.
+
+[View How a question is answered diagram](https://www.shoalrun.com/assets/diagrams/mermaid-78c5e9dba3d9234c8040-light.svg)
+
+```mermaid
+flowchart TD
+ question["Question"] -->|"asked before"| ready["Cached answer<br/>or answer card"]
+ question -->|"new question"| find["Find passages by keyword,<br/>and by meaning if set up"]
+ ready --> answer["Answer"]
+ find --> gate{"Relevance scorer decides"}
+ gate -->|"docs do not cover it"| declined["Not covered<br/>List the closest pages"]
+ gate -->|"one clear winner"| excerpt["One clear passage<br/>Show it word for word"]
+ gate -->|"anything else"| model["Chat model<br/>Write a cited answer"]
+ excerpt --> answer
+ model --> checks{"Check citations,<br/>code, and claims"}
+ checks -->|"flagged: ask a second model"| model
+ checks -->|"passes"| answer
+```
+
+The minimal setup uses keyword retrieval, a chat model, and citation and code checks. Caching, answer cards, scoring, and claim verification need extra configuration. Without a scorer, there is no score-based coverage gate.
+
+For an explicit request with up to three action clauses, retrieval can find evidence for each part and include page introductions. This helps keep one topic from filling the passage budget. [Answer checks](https://www.shoalrun.com/chat/checks.md) explains that behavior and its limits.
+
+## Put the model into practice
+
+Add optional layers after testing the minimal setup with questions your readers actually ask. Check source coverage and response quality before choosing scoring thresholds or enabling answer reuse.
+
+Use [set up local documentation chat](https://www.shoalrun.com/chat/setup.md) for commands and verification. The [retrieval and chat settings](https://www.shoalrun.com/chat/settings.md) define provider fields and defaults. [Passage index](https://www.shoalrun.com/chat/passage-index.md) describes the files used by retrieval.
+
+## Related reading
+
+### Next steps
+
+- [Passage index](https://www.shoalrun.com/chat/passage-index.md) — See how a built site is split into passages and embedded. (Status: experimental.)
+
+- [Check and speed up answers](https://www.shoalrun.com/chat/checks.md) — Decline questions the docs don't cover and check what the model writes. (Status: experimental.)
+
+- [Host and limit documentation chat](https://www.shoalrun.com/chat/host.md) — Run the chat beside your site with spending caps and an off switch. (Status: experimental.)
+
+### Reference while working
+
+- [CLI reference](https://www.shoalrun.com/cli.md) — Look up the index, ask, answers, ask-log, chat, and serve commands.
+
+- [Retrieval and chat settings](https://www.shoalrun.com/chat/settings.md) — Look up every retrieval and chat setting. (Status: experimental.)
+
+### Related topics
+
+- [Set up local documentation chat](https://www.shoalrun.com/chat/setup.md) — Understand the layers behind cited answers before choosing additional services. (Status: experimental.)
+
+- [Site search](https://www.shoalrun.com/author/search.md) — Use retrieved passages to produce cited answers instead of page links.