mercury-agent 0.16.0 → 0.16.2

package/docs/goals/whatsapp-bot-hardening/decisions.md

@@ -0,0 +1,200 @@
+# Decisions: Mercury WhatsApp Bot — Context, Identity & Command Hardening
+
+**Goal**: [whatsapp-bot-hardening](goal.md)
+
+## D-001: Run the live bot from the git checkout, not the npm global install
+- **Category:** architecture
+- **Decided:** `npm link` from `~/mercury-src`, reinstall the systemd user unit
+  via `mercury service install --user`, keep the npm-installed 0.8.12 in place
+  as a <1-minute rollback (`npm unlink` + restore old unit).
+- **Alternatives considered:** keep patching the globally installed package
+  (0.8.12) in place; publish a private build.
+- **Reasoning:** fixes become version-controlled, testing is edit→restart, and
+  `npm i -g` upgrades can no longer silently revert hand-patches (which already
+  happened with the pinchtab fixes).
+- **Revisit if:** the checkout diverges so far from upstream that rebasing costs
+  more than maintaining patches.
+- **Used by:** run-from-source-switchover
+
+## D-002: Message authorship goes in the DB schema (`author_id` / `author_name` columns)
+- **Category:** architecture
+- **Decided:** migrate the `messages` table to carry author id + display name;
+  render `<user name="…" id="…">` in history XML; back the in-memory
+  `pushNames` map with `space_roles.display_name`.
+- **Alternatives considered:** prefix names into the content string (as ambient
+  already does at `runtime.ts:604`); a separate authors join table.
+- **Reasoning:** the identity data already exists clean in
+  `space_roles.display_name`; columns make it queryable and render-safe, and a
+  prefix-in-content hack would be unparseable later. Existing rows backfill as
+  NULL and still render.
+- **Revisit if:** upstream lands its own attribution schema first — then adopt
+  theirs in the rebase.
+- **Used by:** message-author-attribution, ambient-group-context, member-memory
+
+## D-003: Ambient context gets its own bounded query + retention TTL before it is enabled
+- **Category:** trade-off
+- **Decided:** `recordAmbient` ships only together with a dedicated bounded
+  ambient query (not sharing `getRecentTurns`' `turnCount * 5` window) and an
+  ambient TTL in `storage-cleanup.ts`.
+- **Alternatives considered:** enable ambient first, tune the budget later.
+- **Reasoning:** once ambient actually flows, every group message becomes a row;
+  unbounded it crowds real user/assistant turns out of the context window and
+  grows the DB forever. HANDOFF is explicit: do not ship the handler fix
+  without the cap.
+- **Revisit if:** measured ambient volume in `greece2026` is trivially small.
+- **Used by:** ambient-group-context
+
+## D-004: Media cap raised to 64 MB; every dropped attachment produces a promptNote
+- **Category:** scope
+- **Decided:** `media.max_size_mb: 64` in `mercury.yaml` (verified
+  `MERCURY_MEDIA_MAX_SIZE_MB` is unset so YAML takes effect), plus an
+  unconditional system note on any dropped attachment, copying the existing
+  `media-gate.ts` promptNote pattern.
+- **Alternatives considered:** 32 MB; download-extract-discard for oversize
+  documents (deferred as optional follow-up).
+- **Reasoning:** the observed real files were 49–56 MB; the silence was the
+  worse half of the bug — the agent invented a wrong explanation because it was
+  told nothing.
+- **Revisit if:** disk or bandwidth pressure appears in practice.
+- **Used by:** media-size-and-silent-drop
+
+## D-005: Document creation is a new `docs` extension modeled on the `pdf` extension
+- **Category:** tech-stack
+- **Decided:** extension with `install` string adding `pandoc` +
+  `python-docx openpyxl odfpy` to the derived image, a `SKILL.md` teaching
+  docx-to-`outbox/`, `defaultRoles: ["admin", "member"]`.
+- **Alternatives considered:** Google Docs/Workspace integration (real links).
+- **Reasoning:** egress already works (HTML attachments prove it); only
+  production of `.docx` is missing. Workspace OAuth is a separate project.
+- **Revisit if:** users specifically need collaborative Google Docs links.
+- **Used by:** docs-extension
+
+## D-006: `mrctl spaces delete` rejects arguments loudly; the API keeps `/current` only
+- **Category:** architecture
+- **Decided:** error out when `mrctl spaces delete` receives an argument,
+  pointing to the chat command `/spaces delete <id>`; do **not** add a
+  `DELETE /api/spaces/:id` route.
+- **Alternatives considered:** make mrctl honor the id via a new API route.
+- **Reasoning:** the `/current`-only API is a deliberate security boundary
+  (`core/routes/spaces.ts` scopes every mutation to the caller's space).
+  Silently ignoring the argument is what actually destroyed the baduk space.
+- **Revisit if:** never — widening this API is out of scope by design.
+- **Used by:** destructive-command-guards
+
+## D-007: Slash commands work without the trigger word in groups
+- **Category:** pattern
+- **Decided:** honor `/`-prefixed known commands in groups without the trigger
+  word; unknown `/foo` still falls through to ambient.
+- **Alternatives considered:** delete the dead `isCommand` bypass and document
+  that commands require the trigger.
+- **Reasoning:** matches the handler's evident intent (the bypass exists, it
+  just accomplishes nothing today), and matches what users already type — the
+  two live ambient rows are exactly such stranded commands.
+- **Revisit if:** command-name collisions with normal chat starting with `/`.
+- **Used by:** command-routing-consistency
+
+## D-008: Chat stays seeded-admin-only for `/spaces`; `spaces.delete` leaves the blanket admin grant
+- **Category:** trade-off
+- **Decided:** align the divergent gates by tightening the CLI/API side — lift
+  `spaces.delete` out of `getDefaultPermissions`' blanket space-admin grant so
+  it matches the chat surface's seeded-admin gate.
+- **Alternatives considered:** loosen chat to any space admin.
+- **Reasoning:** the tighter gate is the intended one; today a promoted space
+  admin who is denied in chat can destroy the current space via `mrctl` — a
+  bypass, not a feature.
+- **Revisit if:** multi-admin spaces become a real use case.
+- **Used by:** command-routing-consistency
+
+## D-009: `/spaces unlink` is removed from chat; `/model active` is dropped; `/compact` vs `/clear` get honest help text, not a merge
+- **Category:** scope
+- **Decided:** remove the unrecoverable one-way door (6c) from the chat surface
+  entirely (dashboard/API keep the capability); drop the redundant
+  `/model active`; keep both `/compact` and `/clear` but rewrite their help to
+  state the actual `session_boundary` vs `clear_boundary` difference.
+- **Alternatives considered:** make unlink recoverable in chat; merge
+  compact/clear blind.
+- **Reasoning:** unlink's advertised recovery path (`/spaces switch`) is
+  provably unreachable after unlinking; re-link-on-rejoin still ships so the
+  group-removal path self-heals. Merging compact/clear without first
+  documenting the real difference risks destroying a distinction users need.
+- **Revisit if:** a safe recovery UX for unlink is designed later.
+- **Used by:** command-routing-consistency, destructive-command-guards
+
+## D-010: Member memory = hard identity from DB + soft notes as workspace files
+- **Category:** architecture
+- **Decided:** a `<members>` block generated from `space_roles` (id → name,
+  not guessable) plus per-space `knowledge/members/<id>.md` notes written by
+  the agent with its existing `write` tool, injected like `<active_episodes>`.
+- **Alternatives considered:** a new DB table for member profiles.
+- **Reasoning:** reuses existing episodic plumbing, needs no migration for the
+  notes half, and keeps identity facts (who is who) separate from accumulated
+  observations (what they're like).
+- **Revisit if:** notes need querying/aggregation across spaces.
+- **Used by:** member-memory
+
+## D-011: `allowedDomains "*"` stays for this PR, called out as a security trade-off
+- **Category:** trade-off
+- **Decided:** keep the wildcard pinchtab allowlist (the injection guard is
+  verified still active); the PR describes it as a deliberate widening of
+  attack surface, not a plain bugfix. Tightening to a curated list is deferred.
+- **Alternatives considered:** curated domain list now.
+- **Reasoning:** out of scope for this PR; honesty in the PR description is the
+  cheap mitigation available now.
+- **Revisit if:** always — this is queued as a follow-up idea.
+- **Used by:** (PR prep — see roadmap Completion section)
+
+## D-012: One PR at the end, not a PR per story
+- **Category:** scope
+- **Decided:** single PR `fix/whatsapp-context-identity-and-docs` → `main`
+  with a section per change, including the four already-merged pinchtab
+  commits (framed as upstreaming a live hotfix), the uncommitted
+  `.dockerignore` fix, the schema migration, and the explicit list of
+  command-surface behaviour changes.
+- **Alternatives considered:** stacked PRs per milestone.
+- **Reasoning:** the branch already carries the pinchtab commits; the fixes are
+  interdependent (attribution ↔ ambient); and the repo has a single upstream
+  reviewer. The per-section PR body keeps it reviewable.
+- **Revisit if:** upstream asks for it to be split.
+- **Used by:** all stories
+
+## D-013: The D:-clone's `fix/pinchtab-browser-startup` branch is superseded, not pushed
+- **Category:** scope
+- **Decided:** leave the branch in the stale Windows clone
+  (`D:\Projects\whatsapp-bot\mercury`) untouched and unpushed; the four pinchtab
+  commits reach `main` through `fix/whatsapp-context-identity-and-docs` instead.
+- **Alternatives considered:** push it as its own PR and merge it before the
+  hardening PR; delete the branch.
+- **Reasoning:** verified equivalent, not assumed — the combined patch of the
+  four commits is byte-identical in both repos (md5 `83aa486e…` over the
+  index-line-stripped diff, same three files, 56 insertions / 12 deletions).
+  The clone's copy (`3c5d56f..17c92c6`) sits on a base three commits behind
+  `origin/main`; the `mercury-src` copy (`bfe5a33..b12d7c0`) is current with
+  `origin/main` at `70a5ce3`. Pushing both would put the same change on `main`
+  twice. The branch is left in place rather than deleted because it costs
+  nothing and is the only copy of those commit hashes.
+- **Revisit if:** the hardening PR is rejected or split, in which case the
+  pinchtab commits should be re-proposed on their own from `mercury-src`.
+- **Used by:** (PR prep — see roadmap Completion section)
+
+## D-014: One instruction, one layer — never state the same rule twice
+- **Category:** pattern
+- **Decided:** each voice/behaviour rule lives in exactly **one** of the five
+  prompt layers, chosen by scope: global default → `project_config.character`;
+  per-space override → `space_preferences`; per-space long-form structure
+  (role, workflow, honesty rules) → `.mercury/spaces/<id>/AGENTS.md`.
+  `space_config.system_prompt` is treated as unavailable (no setter exists).
+  Quantities are written as **countable** rules, never adjectives.
+- **Alternatives considered:** rely on layer precedence to override — e.g. put
+  the permissive default in the character and a stricter rule in preferences,
+  expecting the more specific layer to win.
+- **Reasoning:** measured, not assumed. On the throwaway `prompttest` space a
+  direct `system_prompt`-vs-`preferences` conflict went 8/10 to `system_prompt`
+  — but a swapped-language control produced the **opposite** winner, and
+  repeating one fixed config gave different winners across runs. Precedence is
+  not reliable, so a conflict is a coin flip and must be designed out rather
+  than resolved by placement. The countable-rule half is from the emoji data:
+  "none at all" → 0%, "minimal" → 33%, "here and there when it fits" → 73%.
+  The layer decides *scope*; the wording decides *compliance*.
+- **Revisit if:** upstream introduces explicit, documented precedence between
+  the layers — then placement becomes a usable tool again.
+- **Used by:** (prompt/voice work — see open-threads.md §A)

package/docs/goals/whatsapp-bot-hardening/goal.md

@@ -0,0 +1,75 @@
+# Mercury WhatsApp Bot — Context, Identity & Command Hardening
+
+**Status**: Active — all 4 milestones shipped; open items in [open-threads.md](open-threads.md)
+**Slug**: whatsapp-bot-hardening
+**Created**: 2026-08-09
+**Last updated**: 2026-08-10
+
+---
+
+## Goal Statement
+
+Fix the six verified defect families documented in the handoff
+(`docs/goals/whatsapp-bot-hardening/archive/handoff-2026-08-09.md`) — ambient group
+context dead code, sender misidentification, silent large-media rejection, no
+document creation, destructive/broken command surface) and add member memory —
+then land everything as one upstream PR from
+`fix/whatsapp-context-identity-and-docs`, with the live bot running the fixed
+code from the git checkout.
+
+## Constraints
+
+- All code work happens in **`~/mercury-src` inside WSL** (branch
+  `fix/whatsapp-context-identity-and-docs`, based on v0.9.0). The Windows clone
+  at `D:\Projects\whatsapp-bot\mercury` is stale reference only — never edit it.
+- The bot is **live** for real family/friends groups. Data-loss bugs (the baduk
+  incident) already happened once; every migration must be preceded by a DB
+  backup, and destructive-command fixes ship before anything that increases
+  message volume is enabled for long.
+- Never read `~/whatsapp-bot/.env` (live API keys). Never run `mercury run`
+  directly; restart only via `systemctl --user restart mercury`.
+- Check gate for every story: `cd ~/mercury-src && bun run check`
+  (typecheck + lint + tests), run inside WSL with the PATH fix
+  (`export PATH=$HOME/.nvm/versions/node/v24.19.0/bin:$HOME/.bun/bin:$PATH`).
+- Debug logging (`log_level: debug`) may be enabled for verification windows
+  only and **must** be reverted — it writes chat-content previews to journald.
+
+## Preferences
+
+- Follow the fix designs already verified in the archived handoff §4 (required) —
+  root causes were confirmed against running code and live data; do not re-derive them.
+- Reuse existing plumbing over new subsystems (required) — e.g. member notes as
+  workspace files like episodes, promptNote pattern from `media-gate.ts`.
+- One PR at the end covering all changes, with explicit sections per change and
+  the behaviour-change list for the command surface (required).
+- Suggested implementation order from the archived handoff §7 (preferred).
+
+## Target Users
+
+- **The owner/admin (Sahar)** — runs the bot, admins the spaces, is the one who
+  typed the command that destroyed the baduk space.
+- **Family/friends group members** — talk in groups the bot is linked to; they
+  expect the bot to know who said what and to read files they send.
+- **Future upstream users of Mercury** — the PR upstreams fixes for bugs that
+  exist verbatim on `origin/main`.
+
+## Success Criteria
+
+- Untagged group messages are recorded: `SELECT space_id, role, COUNT(*) FROM
+  messages GROUP BY space_id, role` shows `ambient` rows for `greece2026`
+  growing, and the bot can answer "what was just said?" after being tagged.
+- Two people talk in a group; a third asks "who said X?" and attribution is
+  correct — including quoted replies after a service restart.
+- `mrctl spaces delete <id>` with an argument exits non-zero with an
+  explanatory error and issues **no** HTTP request; replying "yes" (as a
+  WhatsApp reply) to a `/spaces delete <id>` confirmation deletes the *named*
+  space, not the current one.
+- `/stop` aborts a running agent instead of queueing another model run; a bare
+  `/spaces list` works untagged in a group; an unlinked group can re-link
+  itself from chat.
+- A ~50 MB docx sent to a space is downloaded and readable; an over-limit file
+  produces an explicit "too large" note to the agent instead of silence.
+- "Make me a doc" produces a real `.docx` delivered over WhatsApp.
+- `bun run check` is green, the live service runs from `~/mercury-src`, and one
+  PR containing all changes (including the four pinchtab commits and the
+  `.dockerignore` fix) is open against `main`.