@mercury-fw/cli 0.71.0 → 0.73.0

package/CHANGELOG.md

@@ -1,5 +1,79 @@
 # @mercury-fw/cli
 
+## 0.73.0
+
+### Minor Changes
+
+- a8e2fe9: - the work nobody waits on (titles, memory captures, corrections, the archive's index, the nightly review) waits for the model: it runs only in the requests the turns leave free of the model's `capacity`, one model call at a time, titles, corrections and the archive's index first
+  - `ollamaModel(name, { backgroundSlots })`: how many of the model's requests that work may take at once; 1 when left out, below `capacity` when it's above 1
+  - a turn's corrections, its threshold capture and its message's indexing no longer hold the conversation's next turn
+  - a model of a plugin's own (`ctx.modelFor`) goes through the same queue
+  - `background.job` in the traces: each piece of that work that called a model, its models, model calls, time waited, time taken, and whether it was done, failed or dropped at stop
+  - `mfw create`: `backgroundSlots` read from `OLLAMA_BACKGROUND_SLOTS`, in the env example
+  - **Migration:** to set it from the environment, add `backgroundSlots: envNumber("OLLAMA_BACKGROUND_SLOTS"),` to the turn model's `ollamaModel(...)` in `mercury.config.ts`
+- fb1f30a: - conversation-store contract: `ConversationStorePlugin`, `ConversationStore`, `StoredMessage`, `StoredTitle`, `CONVERSATION_STORE_API_VERSION`
+  - `conversationStore` in `mercury.config.ts`: where every conversation's messages and title are kept
+  - **Breaking:** without `conversationStore` nothing is archived: no reopened or listed conversations, no titles, no `recall_verbatim`
+  - `qdrantConversations()` in `@mercury-fw/store-qdrant`: messages in `conversation_messages` (`QDRANT_MESSAGES_COLLECTION`), titles in `conversation_titles` (`QDRANT_TITLES_COLLECTION`); on start, copies the messages of `verbatim_archive` into `conversation_messages` once, marking the copy done only when all are in
+  - `@mercury-fw/core` no longer depends on `@qdrant/js-client-rest` and no longer reads `QDRANT_URL`
+  - **Breaking:** `GET /health` (HTTP): `conversationStoreReachable` and `vectorIndexReachable` (`null` when not declared) instead of `qdrantReachable`
+  - **Breaking:** channel contract 7: `ChannelHostReads.conversation`, `conversations` and `memoryScroll` optional, given only with a conversation store / a vector index; channels written for contract 6 are refused
+  - `channel-http`, `channel-google-chat`: channel contract 7, framework 0.73 or later
+  - `channel-http`: `/conversation`, `/conversations` (`GET`, `PATCH`) only with a conversation store, `/memory/scroll` only with a vector index; `404` otherwise
+  - `system.start` trace event: `conversationStore` (`declared`, `active`)
+  - `mfw create`: new apps declare `conversationStore: qdrantConversations()`
+  - **Migration:** in `mercury.config.ts`, import `qdrantConversations` from `@mercury-fw/store-qdrant` and add `conversationStore: qdrantConversations(),`; upgrade `@mercury-fw/channel-http` and `@mercury-fw/channel-google-chat` with the framework; a UI reading `qdrantReachable` from `/health` reads the two new fields
+- 4a00b0c: - **Breaking:** new required `MERCURY_DATA_PATH`, the folder for Mercury's own state (account links, profiles, plugins' `dataDir`, traces, `trace-secret`), no longer in the vault's `.mercury/`; the start stops without it
+  - first start with an empty data folder copies the vault's `.mercury/` into it, leaving the original in place; a failed copy is undone and stops the start
+  - `system.migrated` trace event when that copy happens
+  - `mfw identity` reads the links from `MERCURY_DATA_PATH`
+  - `mfw reset wiki` no longer deletes links, profiles and traces
+  - `mfw reset data` deletes the `mercury-data` volume
+  - `mfw create`: new apps get a `mercury-data` volume mounted at `/app/data` and `MERCURY_DATA_PATH=/app/data`
+  - **Migration:** in `Dockerfile`, add `/app/data` to the `mkdir -p` and the `chown mercury:mercury` of the volumes' mount points; in `docker-compose.yml`, add `- mercury-data:/app/data` to the `mercury` service's `volumes` and `mercury-data:` with `name: <app>_mercury-data` under the top-level `volumes`; in `.env`, add `MERCURY_DATA_PATH=/app/data`; then `mfw start`
+- 40fde6b: - `ollamaModel(name, settings?)` and `ollamaEmbedding(name, { capacity }?)` declare a model; `capacity`, how many requests its server answers at once, 1 when left out
+  - `models` in `mercury.config.ts`: `turn` (required), `background` (titles, memory captures, corrections, nightly review; the turn's when left out), `embedding`
+  - a `vectorIndex` without `models.embedding` stops the start
+  - `ctx.modelFor(handle)` in a plugin's `build()`: the model of a handle the plugin was given in its configuration
+  - `system.start` carries `models` (each role's name and capacity) instead of `model` and `modelSettings`
+  - the startup log names each role's model
+  - `mfw create`: the config declares `main` from `OLLAMA_MODEL` and its settings, and `models` with `turn` and `embedding`; `OLLAMA_CAPACITY` in the env example
+  - **Breaking:** `model` is gone from `mercury.config.ts`, and the core no longer reads `OLLAMA_MODEL` or `OLLAMA_EMBEDDING_MODEL`: declare the models in `models`
+  - **Migration:** in `mercury.config.ts`, import `ollamaEmbedding` and `ollamaModel` from `@mercury-fw/core`, add above the config `const main = ollamaModel(process.env.OLLAMA_MODEL, { think: envBoolean("OLLAMA_THINK", true), temperature: envNumber("OLLAMA_TEMPERATURE"), topP: envNumber("OLLAMA_TOP_P"), topK: envNumber("OLLAMA_TOP_K"), capacity: envNumber("OLLAMA_CAPACITY") });`, and replace the `model: { ... }` section with `models: { turn: main, embedding: ollamaEmbedding(process.env.OLLAMA_EMBEDDING_MODEL ?? "nomic-embed-text") },`; `OLLAMA_CAPACITY` is optional
+- b634bd1: - vector-index contract: `VectorIndexPlugin`, `VectorIndex`, `VectorPoint`, `VectorSpaceSpec`, `VectorWhere`, `VECTOR_INDEX_API_VERSION`
+  - `vectorIndex` in `mercury.config.ts`: where episodes, semantic facts, procedural corrections and `recall_verbatim`'s search are kept
+  - **Breaking:** without `vectorIndex` there is no long-term memory: no conversation summaries, no corrections, no `recall_verbatim` (tool and prompt), idle conversations closed without a summary
+  - new package `@mercury-fw/store-qdrant`: `qdrantIndex()`, Qdrant at `QDRANT_URL`, each space the collection of the same name or the one its `QDRANT_*_COLLECTION` names
+  - the memory's vector length comes from the embedding model at startup; `QDRANT_*_VECTOR_SIZE` no longer read
+  - `GET /memory/scroll` (HTTP) takes the index's space name (`episodic_memory`, `semantic_facts`, `verbatim_archive`), not the `QDRANT_*_COLLECTION` name
+  - `system.start` trace event: `vectorIndex` (`declared`, `active`)
+  - `mfw memory` runs the memory CLI of `@mercury-fw/store-qdrant`
+  - `mfw create`: new apps declare `vectorIndex: qdrantIndex()` and depend on `@mercury-fw/store-qdrant`
+  - **Migration:** add `@mercury-fw/store-qdrant` to the app's dependencies, at the framework's version; in `mercury.config.ts`, `import { qdrantIndex } from "@mercury-fw/store-qdrant";` and `vectorIndex: qdrantIndex(),`; `QDRANT_*_VECTOR_SIZE` can go from `.env`
+
+### Patch Changes
+
+- Updated dependencies [a8e2fe9]
+- Updated dependencies [fb1f30a]
+- Updated dependencies [4a00b0c]
+- Updated dependencies [40fde6b]
+- Updated dependencies [b634bd1]
+  - @mercury-fw/core@0.73.0
+  - @mercury-fw/utils@0.73.0
+
+## 0.72.0
+
+### Minor Changes
+
+- 468cb12: - `mfw create`: new apps declare `traces: { sink: jsonlTraces(), level: "meta" }` and depend on `@mercury-fw/trace-jsonl`
+  - **Migration:** to record traces, `bun add @mercury-fw/trace-jsonl` at the framework's version, add `import { jsonlTraces } from "@mercury-fw/trace-jsonl";` and `traces: { sink: jsonlTraces(), level: "meta" },` to `mercury.config.ts`, then in `src/index.ts` pass `tracer: app.tracer` to `loadChannels` and `await app.tracer.flush();` before `process.exit(0)` in `shutdown`
+
+### Patch Changes
+
+- Updated dependencies [468cb12]
+  - @mercury-fw/core@0.72.0
+  - @mercury-fw/utils@0.72.0
+
 ## 0.71.0
 
 ### Patch Changes

package/README.md

@@ -19,7 +19,7 @@ On Mercury's documentation site: [the `mfw` reference](https://lucabro81.github.
   - [`mfw memory list`](#mfw-memory-list)
   - [`mfw memory read <collection> [--limit N]`](#mfw-memory-read-collection---limit-n)
   - [`mfw identity links`, `link`, `unlink`](#mfw-identity-links-link-unlink)
-  - [`mfw reset <memory|wiki>`](#mfw-reset-memorywiki)
+  - [`mfw reset <memory|wiki|data>`](#mfw-reset-memorywikidata)
   - [`mfw credentials setup <plugin>`](#mfw-credentials-setup-plugin)
   - [`mfw credentials check <plugin>`](#mfw-credentials-check-plugin)
   - [`mfw credentials reset <plugin> [--user <key>]`](#mfw-credentials-reset-plugin---user-key)
@@ -53,7 +53,7 @@ Writes a new Mercury app into `<folder>`, which has to be missing or empty; the
 - which channels and which tool plugins to include (none is fine: the REPL always works);
 - the git remote for `origin`, which you can leave empty (most of the time it doesn't exist yet when you scaffold).
 
-It writes the `mercury.config.ts` for that selection, the persona, a `locales/` folder with an `en.json` for the texts people see (empty, or the starting formatter rules' own texts: `app.jira.no-issues` with Jira), the service and REPL entrypoints, a Dockerfile, a compose file with Qdrant, and an env example listing every variable the chosen pieces read. Its `package.json` declares `plugins/*` as workspaces, the place for the app's own plugins (each a package in `plugins/<name>`, which the image copies before installing), and a `test` script (`bun test`, the plugins' tests included); the workspaces make Bun install with its isolated linker, and the Dockerfile's PATH step follows the links it makes. A plugin that hands over lists comes wrapped in the formatter with a starting rule, yours to change.
+It writes the `mercury.config.ts` for that selection, the persona, the traces (`@mercury-fw/trace-jsonl`, at level `meta`, in `traces/` in the data folder), a `locales/` folder with an `en.json` for the texts people see (empty, or the starting formatter rules' own texts: `app.jira.no-issues` with Jira), the service and REPL entrypoints, a Dockerfile, a compose file with Qdrant (the app's vector index and conversation store, `vectorIndex: qdrantIndex()` and `conversationStore: qdrantConversations()`) and a volume for the wiki and one for Mercury's own state, and an env example listing every variable the chosen pieces read. Its `package.json` declares `plugins/*` as workspaces, the place for the app's own plugins (each a package in `plugins/<name>`, which the image copies before installing), and a `test` script (`bun test`, the plugins' tests included); the workspaces make Bun install with its isolated linker, and the Dockerfile's PATH step follows the links it makes. A plugin that hands over lists comes wrapped in the formatter with a starting rule, yours to change.
 
 Then it runs `bun install` in the app and creates a git repository on `main` with a first commit (`bun.lock` included), adding `origin` when you gave one; nothing is pushed. Each step is best effort and never undoes the ones before it: an install that fails still leaves the app committed, without the lockfile, and the closing message says what to run. The repository is skipped when the folder is already inside one (an app created in a monorepo belongs to it). Whether git can commit is git's call: when it refuses (no identity, a hook) or isn't installed, the message reports why and lists the commands to finish by hand. `bun install` prints its own output as it runs.
 
@@ -92,7 +92,7 @@ mfw start --no-cache
 
 ### `mfw stop`
 
-Stops the app and Qdrant and removes their containers (`docker compose down`). The volumes stay, so memory, the wiki and the CLI credentials are all there on the next start.
+Stops the app and Qdrant and removes their containers (`docker compose down`). The volumes stay, so memory, the wiki, Mercury's own state (the data folder) and the CLI credentials are all there on the next start.
 
 ```bash
 mfw stop
@@ -155,23 +155,24 @@ There's no command writing inferred notes on purpose: those are the agent's own,
 
 ### `mfw memory list`
 
-Lists the collections of the memory on Qdrant with how many points each holds, in a one-off container that reaches Qdrant the way the app does. Read-only.
+Lists the collections of the memory on Qdrant with how many points each holds, in a one-off container that reaches Qdrant the way the app does. Read-only. It runs the memory CLI of `@mercury-fw/store-qdrant`, which an app made with `mfw create` has: one without it has no `mfw memory`.
 
 ```bash
 mfw memory list
 ```
 
 ```
+conversation_messages  113 points
 conversation_titles  4 points
 episodic_memory  25 points
 semantic_facts  20 points
 tool_corrections  37 points
-verbatim_archive  0 points
+verbatim_archive  112 points
 ```
 
 ### `mfw memory read <collection> [--limit N]`
 
-Prints a collection's points, each as its id followed by one line per payload field. Newest first where the collection has a timestamp index (episodic memory and the verbatim archive do); in Qdrant's own order otherwise, and it says so. `--limit` defaults to 20. Read-only.
+Prints a collection's points, each as its id followed by one line per payload field. Newest first where the collection has a timestamp index (the episodic memory, the conversation messages and the verbatim archive do; `conversation_messages` also holds one point with no person, marking the copy of an older archive done); in Qdrant's own order otherwise, and it says so. `--limit` defaults to 20. Read-only.
 
 ```bash
 mfw memory read episodic_memory
@@ -188,18 +189,19 @@ mfw identity link google-chat:users/123 oidc:3123
 mfw identity unlink google-chat:users/123
 ```
 
-### `mfw reset <memory|wiki>`
+### `mfw reset <memory|wiki|data>`
 
-Deletes for good what the assistant remembers: `memory` is every collection on Qdrant, `wiki` the whole vault. It reads the volume's real name from the compose file, tells you which one is about to go, and asks you to type the app's name (the `name` in `package.json`); anything else, an empty answer, a `y` or closing the input included, deletes nothing and exits 1. Once confirmed it stops the service using the volume, removes its container and the volume, and starts the service again on an empty one (`docker compose stop`, `rm -f`, `docker volume rm`, `up -d`). After `memory`, a running app is restarted too (`docker compose restart mercury`), since it sets up its collections only when it starts.
+Deletes for good what the assistant remembers: `memory` is every collection on Qdrant, `wiki` the whole vault, `data` Mercury's own state (the linked accounts, the profiles, the plugins' data, the traces), each on a volume of its own. An app whose compose file has no `mercury-data` volume yet has nothing for `data` to delete: the command says so before asking anything. It reads the volume's real name from the compose file, tells you which one is about to go, and asks you to type the app's name (the `name` in `package.json`); anything else, an empty answer, a `y` or closing the input included, deletes nothing and exits 1. Once confirmed it stops the service using the volume, removes its container and the volume, and starts the service again on an empty one (`docker compose stop`, `rm -f`, `docker volume rm`, `up -d`). After `memory`, a running app is restarted too (`docker compose restart mercury`), since it sets up its collections only when it starts.
 
 If a step fails once the service is stopped (the volume still in use by a one-off container, say), it stops there and says the service is down: `mfw start` brings it back.
 
 ```bash
 mfw reset memory
 mfw reset wiki
+mfw reset data
 ```
 
-Useful for clearing out test data; the other layer isn't touched.
+Useful for clearing out test data; the other volumes aren't touched.
 
 ### `mfw credentials setup <plugin>`
 

package/dist/src/app/commands.d.ts

@@ -15,12 +15,13 @@ export type AppDeps = {
     /** The user's home folder, where a CLI keeps its config (`~/.config/<cli>`). */
     home: string;
 };
-/** The core's maintenance CLIs, from the container's working directory (the
- * app's folder, where node_modules/@mercury-fw/core is). A path and not a bin:
- * a monorepo image installs before copying the sources, and Bun doesn't link a
- * bin whose file isn't there yet. The core moves in lockstep with this CLI. */
+/** The framework's maintenance CLIs, from the container's working directory
+ * (the app's folder, where node_modules/@mercury-fw/* is): the core's, and the
+ * memory's in its Qdrant package. A path and not a bin: a monorepo image
+ * installs before copying the sources, and Bun doesn't link a bin whose file
+ * isn't there yet. Both packages move in lockstep with this CLI. */
 export declare const VAULT_CLI = "node_modules/@mercury-fw/core/src/wiki/vault-cli.ts";
-export declare const MEMORY_CLI = "node_modules/@mercury-fw/core/src/memory/memory-cli.ts";
+export declare const MEMORY_CLI = "node_modules/@mercury-fw/store-qdrant/memory-cli.ts";
 export declare const IDENTITY_CLI = "node_modules/@mercury-fw/core/src/identity/links-cli.ts";
 /** What `mfw reset` can wipe: the compose service using the volume, the
  * volume's key in the compose file, and how to say what's lost. */
@@ -35,6 +36,11 @@ export declare const RESET_TARGETS: {
         readonly volume: "wiki-vault";
         readonly what: "the wiki vault (every note)";
     };
+    readonly data: {
+        readonly service: "mercury";
+        readonly volume: "mercury-data";
+        readonly what: "Mercury's own state (account links, profiles, plugins' data, traces)";
+    };
 };
 export type ResetTarget = keyof typeof RESET_TARGETS;
 /** The commands bound to `app`, each returning its exit code. */

package/dist/src/render.d.ts

@@ -13,6 +13,10 @@ export type RenderInput = {
     directory?: string;
     versions: Record<string, string>;
 };
+/** The trace sink every new app declares, writing the traces to files in Mercury's data folder. */
+export declare const TRACE_SINK_PACKAGE = "@mercury-fw/trace-jsonl";
+/** The vector index and the conversation store every new app declares: Qdrant, the compose service. */
+export declare const VECTOR_INDEX_PACKAGE = "@mercury-fw/store-qdrant";
 /** Builds every file of the new app. Throws on an invalid app name, an empty
  * assistant name or role, an id the catalog doesn't have, or a package with no
  * version in `input.versions`. */

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mercury-fw/cli",
-  "version": "0.71.0",
+  "version": "0.73.0",
   "type": "module",
   "license": "MIT",
   "repository": {
@@ -41,23 +41,24 @@
   "comment:shape": "The Mercury CLI (`mfw`). `mfw create <dir>` writes a new Mercury app from the template. The framework packages move in lockstep with it, so a new app gets them at the CLI's own version; plugins and channels at the registry's latest. The plugin and channel packages are devDependencies only for the catalog test. `main(argv)` is exported for create-mercury-agent.",
   "dependencies": {
     "@clack/prompts": "^1.8.1",
-    "@mercury-fw/core": "0.71.0",
-    "@mercury-fw/utils": "0.71.0",
+    "@mercury-fw/core": "0.73.0",
+    "@mercury-fw/utils": "0.73.0",
     "commander": "^15.0.0"
   },
   "devDependencies": {
     "@mercury-fw/auth-oidc": "0.2.1",
     "@mercury-fw/auth-static": "0.2.2",
-    "@mercury-fw/channel-google-chat": "0.12.0",
-    "@mercury-fw/channel-http": "0.16.0",
+    "@mercury-fw/channel-google-chat": "0.13.0",
+    "@mercury-fw/channel-http": "0.17.0",
     "@mercury-fw/directory-static": "0.1.2",
-    "@mercury-fw/formatter": "0.71.0",
+    "@mercury-fw/formatter": "0.73.0",
     "@mercury-fw/plugin-atlassian-admin": "0.5.1",
     "@mercury-fw/plugin-bitbucket": "0.6.1",
     "@mercury-fw/plugin-confluence": "0.1.0",
     "@mercury-fw/plugin-jira": "0.8.1",
-    "@mercury-fw/plugin-web-search": "0.4.1",
+    "@mercury-fw/plugin-web-search": "0.4.2",
     "@mercury-fw/plugin-zitadel": "0.5.1",
+    "@mercury-fw/store-qdrant": "0.73.0",
     "@mercury-fw/typescript-config": "*",
     "@mercury-fw/web-search-searxng": "0.1.1",
     "@types/bun": "^1.4.2",

package/src/app/commands.ts

@@ -38,12 +38,13 @@ const GOOGLE_CHAT_PACKAGE = "@mercury-fw/channel-google-chat";
 /** The app's service, the one the image builds. */
 const SERVICE = "mercury";
 
-/** The core's maintenance CLIs, from the container's working directory (the
- * app's folder, where node_modules/@mercury-fw/core is). A path and not a bin:
- * a monorepo image installs before copying the sources, and Bun doesn't link a
- * bin whose file isn't there yet. The core moves in lockstep with this CLI. */
+/** The framework's maintenance CLIs, from the container's working directory
+ * (the app's folder, where node_modules/@mercury-fw/* is): the core's, and the
+ * memory's in its Qdrant package. A path and not a bin: a monorepo image
+ * installs before copying the sources, and Bun doesn't link a bin whose file
+ * isn't there yet. Both packages move in lockstep with this CLI. */
 export const VAULT_CLI = "node_modules/@mercury-fw/core/src/wiki/vault-cli.ts";
-export const MEMORY_CLI = "node_modules/@mercury-fw/core/src/memory/memory-cli.ts";
+export const MEMORY_CLI = "node_modules/@mercury-fw/store-qdrant/memory-cli.ts";
 export const IDENTITY_CLI = "node_modules/@mercury-fw/core/src/identity/links-cli.ts";
 
 /** What `mfw reset` can wipe: the compose service using the volume, the
@@ -51,6 +52,7 @@ export const IDENTITY_CLI = "node_modules/@mercury-fw/core/src/identity/links-cl
 export const RESET_TARGETS = {
   memory: { service: "qdrant", volume: "qdrant-data", what: "Layer-3 memory (every Qdrant collection)" },
   wiki: { service: SERVICE, volume: "wiki-vault", what: "the wiki vault (every note)" },
+  data: { service: SERVICE, volume: "mercury-data", what: "Mercury's own state (account links, profiles, plugins' data, traces)" },
 } as const;
 export type ResetTarget = keyof typeof RESET_TARGETS;
 

package/src/program.ts

@@ -119,7 +119,7 @@ Examples:
   program
     .command("stop")
     .summary("stops the app")
-    .description("Stops the app and Qdrant and removes their containers. The volumes (memory, wiki, CLI credentials) stay.")
+    .description("Stops the app and Qdrant and removes their containers. The volumes (memory, wiki, Mercury's own state, CLI credentials) stay.")
     .addHelpText("after", INSIDE_AN_APP)
     .action(inApp((app) => app.stop()));
 
@@ -249,9 +249,9 @@ Examples:
 
   program
     .command("reset")
-    .summary("deletes memory or the wiki, after confirmation")
+    .summary("deletes memory, the wiki or Mercury's state, after confirmation")
     .description(
-      "Deletes for good what the assistant remembers: memory is every Qdrant collection, wiki the whole vault. It asks you to type the app's name first (anything else deletes nothing), then brings the service back up empty.",
+      "Deletes for good what the assistant remembers: memory is every Qdrant collection, wiki the whole vault, data Mercury's own state (account links, profiles, plugins' data, traces). It asks you to type the app's name first (anything else deletes nothing), then brings the service back up empty.",
     )
     .addArgument(new Argument("<target>", "what to delete").choices(Object.keys(RESET_TARGETS)))
     .addHelpText("after", INSIDE_AN_APP)

package/src/render.ts

@@ -49,9 +49,19 @@ const CORE_ENV: EnvVar[] = [
   { name: "OLLAMA_TEMPERATURE", comment: "The chat model's temperature, from 0 up (empty: the model's own)" },
   { name: "OLLAMA_TOP_P", comment: "The chat model's top_p, above 0 and at most 1 (empty: the model's own)" },
   { name: "OLLAMA_TOP_K", comment: "The chat model's top_k, a whole number from 1 up (empty: the model's own)" },
+  { name: "OLLAMA_CAPACITY", comment: "How many requests the chat model's runtime serves at once (empty: 1)" },
+  {
+    name: "OLLAMA_BACKGROUND_SLOTS",
+    comment: "How many of those the work nobody waits on may take, below the capacity when it's above 1 (empty: 1)",
+  },
   { name: "OLLAMA_EMBEDDING_MODEL", comment: "Embedding model for the episodic memory", value: "nomic-embed-text" },
   { name: "QDRANT_URL", comment: "Qdrant, the compose service", value: "http://qdrant:6333" },
   { name: "WIKI_VAULT_PATH", comment: "Wiki vault, the named volume's mount point", value: "/app/wiki-vault" },
+  {
+    name: "MERCURY_DATA_PATH",
+    comment: "Mercury's own state (account links, profiles, plugins' data, traces), the named volume's mount point",
+    value: "/app/data",
+  },
   {
     name: "TURN_STALL_TIMEOUT_MS",
     comment: "How long a streaming turn waits for the model's next token before it's stopped with an error (0 turns it off)",
@@ -59,15 +69,32 @@ const CORE_ENV: EnvVar[] = [
   },
 ];
 
-/** The config's `model` section: the main model's settings, each from its env variable (#355). */
-const MODEL_SECTION = [
-  "  // How the main model generates, each value from .env: left empty, the",
-  "  // model's own applies (thinking stays on).",
-  "  model: {",
-  '    think: envBoolean("OLLAMA_THINK", true),',
-  '    temperature: envNumber("OLLAMA_TEMPERATURE"),',
-  '    topP: envNumber("OLLAMA_TOP_P"),',
-  '    topK: envNumber("OLLAMA_TOP_K"),',
+/** Above the config: the turn model, its name, settings and capacity each from its env variable (#355, #383). */
+const MAIN_MODEL = [
+  "/**",
+  " * The model the assistant runs on, from .env: a setting left empty is the",
+  " * model's own (thinking stays on); capacity, how many requests its runtime",
+  " * serves at once, is 1, and so is backgroundSlots, how many of them the work",
+  " * nobody waits on may take.",
+  " */",
+  "const main = ollamaModel(process.env.OLLAMA_MODEL, {",
+  '  think: envBoolean("OLLAMA_THINK", true),',
+  '  temperature: envNumber("OLLAMA_TEMPERATURE"),',
+  '  topP: envNumber("OLLAMA_TOP_P"),',
+  '  topK: envNumber("OLLAMA_TOP_K"),',
+  '  capacity: envNumber("OLLAMA_CAPACITY"),',
+  '  backgroundSlots: envNumber("OLLAMA_BACKGROUND_SLOTS"),',
+  "});",
+].join("\n");
+
+/** The config's `models` section: the model of each of the core's roles (#383). */
+const MODELS_SECTION = [
+  "  // The models it runs on, by role: the turns on `main`, and the work no one",
+  "  // waits on too unless a `background` model is added here; the memory",
+  "  // found by meaning is embedded with `embedding`.",
+  "  models: {",
+  "    turn: main,",
+  '    embedding: ollamaEmbedding(process.env.OLLAMA_EMBEDDING_MODEL ?? "nomic-embed-text"),',
   "  },",
 ].join("\n");
 
@@ -83,12 +110,39 @@ const CONFIG_HEADER = `/**
  * provider when a channel needs one, how the lists its plugins hand over read,
  * the assistant's persona and the languages it speaks (\`i18n\`: a file per
  * language in \`locales/\`, \`en.json\` overriding the English texts) and its version
- * (\`version\`, its \`package.json\`'s, shown on the HTTP channel's \`/version\`). The entrypoints
+ * (\`version\`, its \`package.json\`'s, shown on the HTTP channel's \`/version\`) and where
+ * the traces of what it does go (\`traces\`). The entrypoints
  * (\`src/index.ts\`, \`src/repl.ts\`) hand this config to \`composeMercury\`.
  *
  * Every tool plugin and channel declared here is active.
  */`;
 
+/** The trace sink every new app declares, writing the traces to files in Mercury's data folder. */
+export const TRACE_SINK_PACKAGE = "@mercury-fw/trace-jsonl";
+
+/** The vector index and the conversation store every new app declares: Qdrant, the compose service. */
+export const VECTOR_INDEX_PACKAGE = "@mercury-fw/store-qdrant";
+
+/** The config's `vectorIndex` and `conversationStore` entries, with what each holds. */
+const VECTOR_INDEX_SECTION = [
+  "  // Where the memory is found by meaning (what past conversations were about,",
+  "  // what the agent learned about each person, the corrections to how a tool",
+  "  // is used, and searching past messages): Qdrant, the compose service.",
+  "  // Remove to keep no memory.",
+  "  vectorIndex: qdrantIndex(),",
+  "  // Where what was said is kept: every conversation, to reopen and list it,",
+  "  // and its title. Qdrant, the compose service. Remove to archive nothing.",
+  "  conversationStore: qdrantConversations(),",
+].join("\n");
+
+/** The config's `traces` entry, with what it records and where. */
+const TRACES_SECTION = [
+  "  // What the agent does, as events in traces/ in Mercury's data folder:",
+  "  // a file per day, the last 7 days with activity kept. \"meta\" records names,",
+  "  // outcomes and times; \"args\" also what the tools were given. Remove to record nothing.",
+  '  traces: { sink: jsonlTraces(), level: "meta" },',
+].join("\n");
+
 /** Builds every file of the new app. Throws on an invalid app name, an empty
  * assistant name or role, an id the catalog doesn't have, or a package with no
  * version in `input.versions`. */
@@ -239,7 +293,7 @@ function renderConfig(
 ): string {
   const withRules = tools.filter((t) => t.formatter);
   const imports = [
-    'import { defineMercuryConfig, envBoolean, envNumber } from "@mercury-fw/core";',
+    'import { defineMercuryConfig, envBoolean, envNumber, ollamaEmbedding, ollamaModel } from "@mercury-fw/core";',
     'import en from "./locales/en.json";',
     'import pkg from "./package.json";',
   ];
@@ -254,6 +308,8 @@ function renderConfig(
   for (const c of [...channels, ...(auth ? [auth] : []), ...(directory ? [directory] : [])]) {
     imports.push(`import { ${c.exportName} } from "${c.package}";`);
   }
+  imports.push(`import { jsonlTraces } from "${TRACE_SINK_PACKAGE}";`);
+  imports.push(`import { qdrantConversations, qdrantIndex } from "${VECTOR_INDEX_PACKAGE}";`);
 
   const helpers = withRules.map((t) => `${t.formatter?.helpers}\n\n`).join("");
 
@@ -270,14 +326,18 @@ function renderConfig(
     "/** The assistant's name: what the persona calls it, and every text the model reads. */",
     `const name = ${JSON.stringify(assistantName)};`,
     "",
+    MAIN_MODEL,
+    "",
     `${helpers}export default defineMercuryConfig({`,
     ...renderPersona(role),
     plugins,
     channelList,
     ...(auth ? [`  auth: ${auth.exportName},`] : []),
     ...(directory ? [`  directory: ${directory.exportName},`] : []),
-    MODEL_SECTION,
+    VECTOR_INDEX_SECTION,
+    MODELS_SECTION,
     "  i18n: { locales: { en } },",
+    TRACES_SECTION,
     VERSION_COMMENT,
     "  version: pkg.version,",
     "});",
@@ -318,7 +378,7 @@ function renderPackageJson(
   directory: CatalogEntry | undefined,
 ): string {
   const chosen = [...channels, ...(auth ? [auth] : []), ...tools, ...(directory ? [directory] : [])];
-  const packages = [...new Set(["@mercury-fw/core", ...chosen.flatMap(entryPackages)])];
+  const packages = [...new Set(["@mercury-fw/core", TRACE_SINK_PACKAGE, VECTOR_INDEX_PACKAGE, ...chosen.flatMap(entryPackages)])];
   if (tools.some((t) => t.formatter)) {
     packages.push("@mercury-fw/formatter");
   }
@@ -396,6 +456,7 @@ function renderCompose(name: string, hasHttp: boolean, services: Array<{ name: s
     "        required: false",
     "    volumes:",
     "      - wiki-vault:/app/wiki-vault",
+    "      - mercury-data:/app/data",
     "      # The login of a plugin's CLI, on a volume so what the CLI writes back",
     "      # (refreshed tokens) survives a redeploy: see README.md.",
     "      - cli-credentials:/home/mercury/.config",
@@ -415,6 +476,8 @@ function renderCompose(name: string, hasHttp: boolean, services: Array<{ name: s
     "volumes:",
     "  wiki-vault:",
     `    name: ${name}_wiki-vault`,
+    "  mercury-data:",
+    `    name: ${name}_mercury-data`,
     "  qdrant-data:",
     `    name: ${name}_qdrant-data`,
     "  cli-credentials:",

package/src/versions.ts

@@ -7,7 +7,13 @@
 import pkg from "../package.json";
 
 /** The framework packages a new app can depend on: always at the CLI's version. */
-export const FRAMEWORK_PACKAGES = ["@mercury-fw/cli", "@mercury-fw/core", "@mercury-fw/formatter"];
+export const FRAMEWORK_PACKAGES = [
+  "@mercury-fw/cli",
+  "@mercury-fw/core",
+  "@mercury-fw/formatter",
+  "@mercury-fw/store-qdrant",
+  "@mercury-fw/trace-jsonl",
+];
 
 /** The registry asked when `MFW_REGISTRY` doesn't name another. */
 export const DEFAULT_REGISTRY = "https://registry.npmjs.org";

package/template/Dockerfile.tpl

@@ -32,8 +32,8 @@ COPY --chown=mercury:mercury src ./src
 
 # Mount points of the named volumes, owned by the runtime user before a fresh
 # volume attaches (a new volume takes the ownership it finds here).
-RUN mkdir -p /app/wiki-vault /home/mercury/.config \
-  && chown mercury:mercury /app/wiki-vault \
+RUN mkdir -p /app/wiki-vault /app/data /home/mercury/.config \
+  && chown mercury:mercury /app/wiki-vault /app/data \
   && chown -R mercury:mercury /home/mercury
 
 USER mercury

package/template/src/index.ts.tpl

@@ -12,7 +12,7 @@ import mercuryConfig from "../mercury.config.ts";
 
 const app = await composeMercury(mercuryConfig);
 
-const loadedChannels: LoadedChannel[] = loadChannels(app.channels, { runtime: app.channelRuntime });
+const loadedChannels: LoadedChannel[] = loadChannels(app.channels, { runtime: app.channelRuntime, tracer: app.tracer });
 for (const { name, provider } of loadedChannels) {
   await provider.start(app.handleTurn);
   console.error(`[channel] ${name} started`);
@@ -34,6 +34,8 @@ async function shutdown(signal: string): Promise<void> {
     await provider.stop?.();
     console.error(`[shutdown] channel ${name} stopped`);
   }
+  // The last events written before the signal, stored before exiting.
+  await app.tracer.flush();
   process.exit(0);
 }
 process.on("SIGINT", () => void shutdown("SIGINT"));