mercury-agent 0.16.1 → 0.16.3

package/README.md

@@ -217,6 +217,9 @@ mercury setup --profile https://github.com/user/mercury-stock-agent
 # Create from local directory
 mercury setup --profile ./my-profiles/ml-assistant
 
+# Apply a profile non-interactively (deploy scripts, existing projects)
+mercury profiles apply ./my-profiles/ml-assistant
+
 # List available profiles
 mercury profiles list
 
@@ -224,6 +227,8 @@ mercury profiles list
 mercury profiles export ./my-profile
 ```
 
+Building one for a specific use-case — what goes in `AGENTS.md`, how to keep it accurate, memory, skills, tasks, permissions, testing — is covered in [docs/profile-guide.md](docs/profile-guide.md). A single space inside a general bot gets a **space profile** instead (`scripts/space-profile.ts`, scaffold in [`examples/profiles/_template/`](examples/profiles/_template/)).
+
 ---
 
 ## CLI
@@ -513,6 +518,7 @@ mrctl config set trigger_patterns "@Bot,Bot"
 - [Authentication](docs/auth/overview.md)
 - [Message pipeline](docs/pipeline.md)
 - [Memory system](docs/memory.md)
+- **Building a bot for a use-case:** [Profile guide](docs/profile-guide.md) · [Behaviour layers](docs/behavior-layers.md) · [Skills guide](docs/skills-guide.md) · [Applicative profile contract](docs/authoring-profiles.md)
 - [Scheduled tasks](docs/scheduler.md)
 - [Permissions](docs/permissions.md)
 - [Media handling](docs/media/overview.md)

package/docs/README.md

@@ -0,0 +1,50 @@
+# Mercury — docs
+
+Product docs, planning docs, ops runbooks and workflow scaffolding for
+[mercury](https://github.com/Avishai-Tsabari/mercury). All of it lives in this
+repository, tracked normally.
+
+## Layout
+
+```
+docs/
+  configuration.md, pipeline.md, auth/, security/, ...   product & reference docs
+  runbooks/                     operational runbooks (release steps, OAuth setup)
+  goals/{slug}/                 multi-milestone goals (goal, roadmap, decisions)
+  ideas/{slug}.md               raw ideas, not yet planned
+  backlog/{slug}.md             goal + architecture approved, not started
+  in-progress/{slug}.md         active implementation
+  archive/{slug}.md             shipped or abandoned
+  bugs/{slug}.md                open bug reports
+  debug/{severity}/             bug post-mortems
+  refactor/                     audits, active refactor docs, archive
+  notes/, templates/, html-slides/, pending-updates/
+  ROADMAP.md                    single source of truth for priority
+```
+
+## Content rule
+
+The repo is private, but "private" is not "unreviewed". For everything under
+`docs/`:
+
+- **Name secrets, never paste values.** `MERCURY_ANTHROPIC_API_KEY`, not the key.
+- **Keep third-party identifiers out unless they carry the finding.** A phone
+  number, WhatsApp JID or LID belongs in a doc only when the identifier *is* the
+  evidence — a post-mortem about LID resolution may quote one; a setup guide may
+  not.
+
+## History
+
+Until 2026-08-19 the scaffolding lived in a second private repository,
+`saharw01/mercury-planning`, whose working tree *was* this directory — a
+workaround from when mercury was public and planning docs were kept out of it by
+policy (PR #22, 2026-08-10).
+
+That is over. The repo is private, PRs #46/#47 brought `docs/debug/` and
+`docs/runbooks/` back under version control, and the rest of the scaffolding
+followed. The planning repo's 44 commits were merged in with `-X subtree=docs`,
+so authorship and per-file history survive; `git log --follow` traces through
+both the original subtree split (mercury `d5d2aa6`) and the re-merge.
+
+There is one repo and one `git status` again. Nothing here needs a second remote,
+a nested `.git`, or an allowlist in `.git/info/exclude`.

package/docs/authoring-profiles.md

@@ -1,5 +1,13 @@
 # Authoring Applicative Profiles
 
+> This page is the **contract** — manifest, broker, permissions. For *how to
+> design* a profile that is accurate and stays in character (the layer map,
+> facts inventory, `AGENTS.md` shape, memory, skills, tasks, testing) read
+> [`profile-guide.md`](profile-guide.md) first; for what the model actually
+> receives, [`behavior-layers.md`](behavior-layers.md). Per-space profiles
+> (one space inside a general bot) use `scripts/space-profile.ts` instead of
+> this manifest — see `examples/profiles/_template/`.
+
 The contract for building a **profile** — a package of deterministic business
 logic that wraps raw capabilities (Calendar, email, …) and scopes what members
 can do. Mercury (this repo) provides the schema, loader, permission scoping, and

package/docs/behavior-layers.md

@@ -0,0 +1,204 @@
+# Behaviour layers — where a rule lives, and what the model actually receives
+
+Every agent run is assembled from several layers written by different people
+at different times: the platform prompt baked into the image, an optional
+applicative profile, the owner's global character, the global `AGENTS.md`, the
+space `AGENTS.md`, space preferences, space config, task prompts, and the
+memory files in the workspace. Most "the bot ignores my instruction" reports
+are two of these layers disagreeing. This page is the verified map — what is
+sent, in what order, who owns it, and when a change takes effect.
+
+Verified against `src/agent/container-entry.ts`, `src/core/runtime.ts`,
+`src/storage/db.ts` and pi 0.84.2's `resource-loader.js` / `system-prompt.js`
+(inside the agent image) on 2026-08-21. Re-verify by grepping the image, not
+the repo, after a pi upgrade.
+
+## 1. The system prompt, top to bottom
+
+```
+1. pi preamble ............................ image       container-entry.ts buildSystemPrompt
+     "You are an expert AI assistant…", tool list, "Be concise"
+     (override mode; OVERRIDE_PI_SYSTEM_PROMPT)
+2. Mercury platform block .................. image       container-entry.ts buildMercuryAdditions
+     inbox/outbox · "You are <bot_username>; read /docs/mercury"
+     ## Permissions & Security            ("Permission denied → simply inform the user")
+     ## Moderation                        ("warn first, then mute" — only for a caller holding mutes.mute)
+     ## Environment Variables
+     ## Current model capabilities
+     ## Memory                            (MEMORY.md, <active_episodes>, <members>, member notes)
+     ## Run budget                        (tools-capable models only)
+     ## Destructive Operations — Confirmation Required
+     ## Presenting tool results           ("simple lists", "never show JSON/commands")
+     ## Character                         (mrctl character flow + "standing instruction → set a preference"
+                                           — only for a caller holding prefs.set)
+     reply-anchor sentence                (swipe-replies only)
+3. MERCURY_EXT_SYSTEM_PROMPT ............... host        runtime.ts ~1700-1760
+     a. extension before_container `systemPrompt` fragments
+     b. applicative profile `profile_prompt`           (.mercury/active-profile.json)
+     c. "## Bot Character (set by the owner — applies to all conversations)"
+                                                        (project_config.character;
+                                                         skipped when persona.exclusive=true)
+     d. space `system_prompt`                            (space_config, dashboard-set)
+4. <project_context> ....................... pi          resource-loader.js loadProjectContextFiles
+     <project_instructions path="/home/mercury/.pi/agent/AGENTS.md">   ← .mercury/global/AGENTS.md
+                                                        (not mounted when persona.exclusive=true)
+     <project_instructions path="/spaces/<space>/AGENTS.md">           ← the space AGENTS.md
+     (pi walks from cwd to /; only the space dir is mounted, so only that
+      file and the global one exist. AGENTS.override.md / CLAUDE.md are
+      accepted names too; AGENTS.md is the convention.)
+5. <available_skills> ...................... pi          skills.js formatSkillsForPrompt
+     name + description + SKILL.md path for every skill under
+     /home/mercury/.pi/agent/skills/**   ← .mercury/global/skills/  (extension + built-in skills)
+     /spaces/<space>/.pi/skills/**       ← space-local skills
+6. "Current working directory: /spaces/<space>"
+```
+
+Two consequences worth reading twice:
+
+- **Layers 1–3 are prose the model treats as equally authoritative.** Nothing
+  ranks "space AGENTS.md" above "## Moderation". When two of them speak to the
+  same subject the outcome is a coin flip weighted towards whichever is more
+  specific and later — and the platform text itself says so: *"a per-space
+  rule that is meant to override a global one has to say so in words rather
+  than rely on being more specific."* The only reliable override is an
+  explicit sentence naming what it overrides, plus deleting the competing
+  rule wherever you control it.
+- **Layer 2 is mostly unconditional, and two voices are removable.** Every
+  space receives the security text, the destructive-ops rule and the "simple
+  lists, never raw output" paragraph. Two paragraphs are *capability* text
+  and are emitted only when the caller can act on them: "## Moderation" for a
+  caller holding `mutes.mute`, "## Character" (the preference / character
+  flow) for a caller holding `prefs.set` — a member without `prefs.set` is no
+  longer told that "a standing instruction → set a preference". And a space
+  that owns its standard can set `persona.exclusive=true` (`space_config`,
+  profile-owned): the host then leaves the owner's Bot Character (3c) out of
+  its prompt and does not mount the global `AGENTS.md` (4), so the space file
+  is the only persona left. Everything that still reaches such a space (the
+  platform text, skills, a per-space `system_prompt`) is unchanged. A persona
+  that contradicts what remains (a reporter who must not offer lists) still
+  has to say so in its `AGENTS.md`, in words.
+
+## 2. The user prompt, top to bottom
+
+Built by `buildPrompt()` in `container-entry.ts`:
+
+```
+<caller id name role space/>
+<run_budget>…</run_budget>                  wall clock for this run
+<episodic_memory>…</episodic_memory>        /spaces/<space>/MEMORY.md, verbatim, if present
+<active_episodes>…</active_episodes>        knowledge/episodes/*.md, keyword-scored (see §5)
+<history>…</history>                        sliding window of stored turns (see §4)
+<ambient_messages>…</ambient_messages>      overheard group messages (ambient.enabled)
+<members>…</members>                        space_roles — authoritative id→name→role
+<member_notes>…</member_notes>              knowledge/members/<id>.md, ≤6000 chars
+<preferences>…</preferences>                space_preferences rows
+<attachments>…</attachments>
+<reply_anchor>…</reply_anchor>              swipe-replies only
+<the prompt>                                chat message, or the task prompt, plus any
+                                            before_container promptAppend (e.g. a digest)
+```
+
+A scheduled task uses exactly the same assembly; its prompt row is "the
+prompt", and it runs as the caller who created it.
+
+## 3. Ownership, surface, and when a change lands
+
+| Layer | Owner | Set from | Takes effect |
+|---|---|---|---|
+| pi preamble, Mercury platform block | mercury (code) | `src/agent/container-entry.ts` | **agent image rebuild** (`container/build.sh`), not a restart |
+| `profile_prompt`, `member_permissions` | deploy (applicative profile) | `mercury profiles apply <dir>` → `.mercury/active-profile.json` | **restart** (read at startup) |
+| Bot Character | bot owner (global admin) | `mrctl character set --file`, `PUT /api/character`, dashboard | next run; left out of a `persona.exclusive` space |
+| space `system_prompt` | space admin | dashboard Spaces settings (`space_config.system_prompt`) | next run |
+| global `AGENTS.md` | deploy | `.mercury/global/AGENTS.md` (written by `mercury init` from `resources/templates/AGENTS.md`, overwritten by an applicative profile's `agents_md`) | next run (mounted `:ro`); not mounted for a `persona.exclusive` space |
+| space `AGENTS.md` | space profile / operator | `.mercury/spaces/<space>/AGENTS.md`; `scripts/space-profile.ts apply` | next run (mounted per run) |
+| space preferences | space admin (`prefs.set`) | `mrctl prefs set/delete`; ≤500 chars each, ≤50 per space | next run |
+| space config (`trigger.*`, `context.*`, `ambient.enabled`, `persona.exclusive`, `role.<r>.permissions`, `<ext>.<key>`, `model.active`, `system_prompt`) | space admin / dashboard / profile | `mrctl config set`, dashboard, `space-profile apply` (`extension_config`) | next run (per-run read) |
+| task prompts | whoever owns the `tasks` row | `mrctl tasks create`, dashboard, `space-profile apply` (`tasks:`) | next scheduled run |
+| `MEMORY.md`, `knowledge/**` | the agent (and napkin's jobs) | `write` tool in-container; host edits | next run |
+| extension skills | extension | `mercury.skill("./skill")` → copied to `.mercury/global/skills/<name>/` at startup | **restart** |
+| space-local skills | operator / profile | `.mercury/spaces/<space>/.pi/skills/<name>/SKILL.md` | next run |
+| `mercury.yaml` (`model.chain`, `context.window_size` default, `agent.trace_runs`, `extensions:` defaults) | operator | file + restart; `MERCURY_*` env wins over YAML | restart (some keys, e.g. `model.active`, are per-run) |
+
+"Next run" means the running bot does not need a restart. Two traps:
+
+- A restart never redeploys anything in layer 1 — `container-entry.ts` is
+  baked into the image. Verify by grepping the image
+  (`docker run --rm --entrypoint grep <image> -c "<marker>" /path`), never
+  the repo.
+- `MERCURY_*` env overrides `mercury.yaml`; a YAML edit can silently do
+  nothing.
+
+## 4. The history window
+
+- `context.mode` is a per-space key. `main` and DM auto-spaces are seeded
+  `context`; **any other space defaults to `clear`** (no history at all — only
+  the reply chain when someone swipe-replies). Set it explicitly.
+- `context.window_size` (default 10 — `config.ts contextWindowSize`) counts
+  **user turns**: `getRecentTurns` takes the newest `turnCount*5` non-ambient
+  rows after the compact boundary and cuts at the Nth user row. Assistant rows
+  ride along. Scheduled runs are user rows too, so a space with six
+  scheduled runs a day spends most of its window on procedure text.
+- A swipe-reply halves the window (`floor(window/2)`) and prepends the
+  replied-to chain (`context.reply_chain_depth`).
+- `mrctl compact` moves the boundary permanently; `mrctl clear` clears the
+  next run only; `mrctl recall "<words>"` is a substring search over the
+  whole stored history for anything outside the window.
+- Ambient messages have their own budget (30 rows) and do not compete with
+  turns.
+
+The window is a *recent-conversation* mechanism, not a memory. Anything the
+bot must get right a week later belongs in a file (§5), not in the hope that
+it is still inside the window.
+
+## 5. Memory surfaces (what is injected, when)
+
+| Surface | File | Injected | Condition |
+|---|---|---|---|
+| Episodic memory | `MEMORY.md` in the space dir | every run, verbatim, as `<episodic_memory>` | file exists and is non-empty. No size cap is enforced — the prompt asks for ~1500 tokens |
+| Active episodes | `knowledge/episodes/*.md` | per run, as `<active_episodes>`, ≤800 tokens | frontmatter `status: active|cooling`, `keywords: ["…"]` (JSON array); score = keyword overlap × (active 1.0 / cooling 0.5) × log(mentions+1); body = `summary:` + `## Current State` |
+| Member notes | `knowledge/members/<id>.md` | per run, as `<member_notes>` | `<id>` = platform id with `/`→`_`; ≤6000 chars total |
+| Members | `space_roles` table | per run, as `<members>` | always; authoritative identity |
+| Preferences | `space_preferences` | per run, as `<preferences>` | always |
+| Vault entities | `knowledge/{people,projects,references,daily}/` | **not injected** | the agent must `read`/`napkin search` them; napkin's distillation job writes them |
+| Anything else in the workspace | `knowledge/*.md`, `outbox/`, … | **not injected** | the agent reads it only if a rule tells it to (e.g. the football profile's `knowledge/push-log.md`) |
+
+So: the only *always-on* memories are `MEMORY.md`, preferences and member
+notes. Episodes need keywords that will actually appear in future messages.
+Everything else is opt-in by instruction.
+
+## 6. Which layer for which kind of rule
+
+| Kind of rule | Put it in | Not in |
+|---|---|---|
+| Who the bot is in this space, register, what it covers, sourcing and accuracy standard, formats, when it stays silent | space `AGENTS.md` | preferences (500-char blocks in the user prompt, no structure, easy to contradict) |
+| A procedure: when it runs, what it produces, where it goes | the task prompt (`tasks/*.md` in a space profile) | `AGENTS.md` (a task prompt never repeats an editorial rule; `AGENTS.md` never describes a schedule) |
+| A one-line behavioural setting a space admin may change from chat (`language`, `timezone`) | preference | `AGENTS.md` (would need a deploy to change) |
+| Voice that must hold across all spaces of one deployment | Bot Character | every space's `AGENTS.md` (drifts) |
+| Deterministic business logic, credentials, anything with numbers or dates | a capability / engine / extension, returning structured facts | prompt prose ("compute the day name yourself") |
+| What members may do | `role.member.permissions` / applicative `member_permissions` | prompt prose ("only admins may…") — the model is not the enforcement |
+| Knobs an extension reads (`feed-watch.*`, `longview.*`) | `extension_config` in the space profile (repo-owned, drift-checked) | hand edits on the box |
+| A fact the bot keeps getting wrong | a file the bot reads each turn (`MEMORY.md`, a notebook under `knowledge/`), written with source and date | a longer rule |
+
+One rule in exactly one layer. A rule repeated "for emphasis" is the next
+contradiction — and when you move a rule into `AGENTS.md`, delete the
+preference it came from (`remove_preferences:` in a space profile exists for
+this).
+
+## 7. Reading the live layers
+
+```bash
+# Everything the football space receives, without touching it
+bun scripts/space-profile.ts dump --project ~/whatsapp-bot --space football-friends
+
+# Drift between the repo profile and the live space
+bun scripts/space-profile.ts check --project ~/whatsapp-bot --profile examples/profiles/football-reporter
+
+# The exact prompt a run received (requires agent.trace_runs: true + image rebuild)
+ls .mercury/traces/                 # <container-name>.jsonl, 7-day TTL
+```
+
+`agent.trace_runs` is off by default and needs the image rebuilt when
+toggled. With it off, "what did the model actually see and open?" is
+unanswerable after the fact — turn it on for a bounded window whenever you
+are tuning a profile. A trace holds the full prompt and reply in clear text;
+treat it like debug logging.

package/docs/extensions.md

@@ -121,6 +121,8 @@ The directory must contain a `SKILL.md` file in pi's [skill format](https://agen
 
 Skills can contain multiple files — scripts, references, assets — not just SKILL.md. The agent uses relative paths from SKILL.md to access them.
 
+Only the `name`, `description` and path of each skill reach the system prompt; the agent reads the body on demand when the description matches. How to write a description that fires, where space-local skills must live (`.mercury/spaces/<space>/.pi/skills/`), and how to make a profile actually use its skills: [skills-guide.md](skills-guide.md).
+
 ### `mercury.requires(capabilities)`
 
 Declare that this extension's skill (or CLI workflows) needs certain model capabilities (`tools`, `vision`, etc.). If **no** leg in `MERCURY_MODEL_CHAIN` satisfies **all** listed flags, the extension skill is not copied into the global skills directory and Mercury logs a startup warning.