mercury-agent 0.4.5

package/docs/media/whatsapp.md

@@ -0,0 +1,171 @@
+# WhatsApp Media Handling
+
+Downloads media from WhatsApp messages using the Baileys library.
+
+## Supported Media Types
+
+| WhatsApp Type | Detected As | Default MIME | Extensions |
+|---------------|-------------|--------------|------------|
+| `imageMessage` | `image` | `image/jpeg` | jpg, png, gif, webp |
+| `videoMessage` | `video` | `video/mp4` | mp4, 3gp |
+| `audioMessage` (ptt=true) | `voice` | `audio/ogg` | ogg |
+| `audioMessage` (ptt=false) | `audio` | `audio/mpeg` | mp3, m4a, aac, ogg |
+| `documentMessage` | `document` | `application/octet-stream` | pdf, doc, docx, xls, xlsx, txt, etc. |
+| `stickerMessage` | `image` | `image/webp` | webp |
+
+## Data Flow
+
+```
+WAMessage (Baileys)
+       │
+       ▼
+detectWhatsAppMedia(msg.message)
+       │
+       ├─▶ null (no media)
+       │
+       ▼
+WhatsAppMediaInfo { type, mimeType, fileLength?, filename? }
+       │
+       ▼
+downloadWhatsAppMedia(msg, sock, options)
+       │
+       ├─▶ null (too large / download failed)
+       │
+       ▼
+MessageAttachment { path, type, mimeType, sizeBytes?, filename? }
+       │
+       ▼
+Saved to: {workspace}/inbox/{timestamp}-{type}.{ext}
+```
+
+## Implementation
+
+### Detection
+
+```typescript
+// src/adapters/whatsapp-media.ts
+function detectWhatsAppMedia(message: proto.IMessage): WhatsAppMediaInfo | null
+```
+
+Checks message for media fields in order:
+1. `audioMessage?.ptt` → voice note
+2. `audioMessage` → audio
+3. `imageMessage` → image
+4. `videoMessage` → video
+5. `documentMessage` → document
+6. `stickerMessage` → image
+
+### Download
+
+```typescript
+// src/adapters/whatsapp-media.ts
+async function downloadWhatsAppMedia(
+  msg: WAMessage,
+  sock: WASocket,
+  options: MediaDownloadOptions,
+): Promise<MessageAttachment | null>
+```
+
+Uses Baileys' `downloadMediaMessage()`:
+
+```typescript
+const buffer = await downloadMediaMessage(
+  msg,
+  "buffer",
+  {},
+  {
+    logger: silentLogger,
+    reuploadRequest: sock.updateMediaMessage,
+  },
+);
+```
+
+### Adapter Integration
+
+```typescript
+// src/adapters/whatsapp.ts
+createWhatsAppBaileysAdapter({
+  mediaEnabled: true,
+  mediaMaxSizeBytes: 10 * 1024 * 1024,
+  getWorkspace: (spaceId) => ensureSpaceWorkspace(spacesDir, spaceId),
+});
+```
+
+## Size Limits
+
+Files are checked against `mediaMaxSizeBytes` twice:
+
+1. **Before download** — using `fileLength` from message metadata (may be missing)
+2. **After download** — using actual buffer size
+
+Files exceeding the limit are logged and skipped:
+
+```
+[WARN] Skipping large media file messageId=ABC type=video sizeBytes=52428800 maxBytes=10485760
+```
+
+## Reply Context
+
+When replying to a message with media, the reply context includes media info:
+
+```typescript
+// Attributes added to <reply_to>
+media_type="image"
+media_mime="image/jpeg"
+```
+
+Example:
+```xml
+<reply_to name="John" jid="123@wa" message_id="ABC" media_type="image" media_mime="image/jpeg">
+[image]
+</reply_to>
+```
+
+## File Naming
+
+Pattern: `{timestamp}-{type}.{ext}` or `{timestamp}-{filename}` for documents
+
+| Input | Output |
+|-------|--------|
+| image/jpeg | `1709012345-image.jpg` |
+| audio/ogg (voice) | `1709012345-voice.ogg` |
+| application/pdf, "report.pdf" | `1709012345-report.pdf` |
+
+## MIME to Extension Mapping
+
+```typescript
+const MIME_TO_EXT: Record<string, string> = {
+  "image/jpeg": "jpg",
+  "image/png": "png",
+  "image/gif": "gif",
+  "image/webp": "webp",
+  "audio/ogg": "ogg",
+  "audio/mpeg": "mp3",
+  "audio/mp4": "m4a",
+  "video/mp4": "mp4",
+  "video/3gpp": "3gp",
+  "application/pdf": "pdf",
+  // ... etc
+};
+```
+
+Unknown MIME types default to `.bin`.
+
+## Limitations
+
+1. **No re-download** — Media is downloaded once when the message arrives. If the file is deleted, it's gone.
+
+2. **No transcription** — Voice notes are saved as audio files. pi cannot play them. Future: add Whisper transcription.
+
+3. **Reply context doesn't include file** — When replying to a media message, we include metadata but not the actual file path. The original attachment would need to be looked up.
+
+4. **Ephemeral media** — WhatsApp media URLs expire. Download must happen immediately when the message arrives.
+
+## Error Handling
+
+| Error | Behavior |
+|-------|----------|
+| Download fails | Log error, continue without attachment |
+| Buffer empty | Log error, return null |
+| Size exceeded | Log warning, discard buffer |
+| Write fails | Throw error (propagates up) |

package/docs/memory.md

@@ -0,0 +1,137 @@
+# Memory
+
+Mercury stores memory per **space** in the space workspace directory. Memory behavior is extension-driven (for example, via a napkin extension).
+
+## How It Works
+
+Each space gets a workspace at:
+
+```text
+.mercury/spaces/<space-id>/
+```
+
+Core directories Mercury manages:
+
+```text
+.mercury/spaces/<space-id>/
+├── inbox/              # Media received from users
+├── outbox/             # Files produced by the agent
+├── AGENTS.md           # Space instructions
+└── .mercury.session.jsonl
+```
+
+Additional vault structure (for example `.obsidian/`, `knowledge/`, `daily/`, `entities/`) is created by installed extensions.
+
+## Vault Structure
+
+There is no single required memory schema in core Mercury. The exact structure depends on your extension setup.
+
+A common pattern (napkin example) is:
+
+```text
+knowledge/
+├── people/
+├── projects/
+├── references/
+└── daily/
+```
+
+Conversations do not get their own vaults — multiple platform conversations can link into the same space.
+
+## Agent Capabilities
+
+The agent discovers extension commands via installed skills (for example `napkin` skill in `.mercury/extensions/napkin/skill/`).
+
+### Reading
+
+```bash
+napkin search "query"              # Find relevant files
+napkin read "filename"             # Read a specific file
+napkin link back --file "name"     # See what links to a file
+napkin daily read                  # Read today's daily note
+```
+
+### Writing
+
+```bash
+napkin create --name "Liat" --path entities --content "..."
+napkin append --file "Liat" --content "..."
+napkin property set --file "Liat" --name birthday --value "April 15"
+napkin daily append --content "- Discussed vacation plans"
+```
+
+### Wikilinks
+
+The agent uses `[[wikilinks]]` when mentioning entities. This creates a navigable graph:
+
+```markdown
+---
+type: person
+relationship: wife
+---
+
+# Liat
+
+[[Michael]]'s wife. Planning a surprise party at [[Dizengoff Italian Place]].
+```
+
+## User Interaction
+
+Users manage memory through natural chat:
+
+| User says | What happens |
+|-----------|--------------|
+| "Remember that Liat's birthday is April 15" | Agent writes to `entities/Liat.md` |
+| "What do you know about Liat?" | Agent reads and summarizes the entity file |
+| "Forget everything about the project" | Agent deletes the entity file |
+
+## Entity Format
+
+Entities are markdown files with optional YAML frontmatter:
+
+```markdown
+---
+type: person
+birthday: April 15
+---
+
+# Liat
+
+Context and notes about Liat.
+
+_2026-02-20:_ Booked the Italian place for April 12.
+_2026-02-25:_ Changed venue to [[Cafe Nimrod]].
+```
+
+- **Frontmatter** — Structured attributes (replace semantics)
+- **Body** — Accumulated context (append semantics, timestamped)
+- **Wikilinks** — Connections to other entities
+
+## Conditional Context
+
+Mercury can skip loading the full session for standalone prompts (e.g. "what's 2+2?"), reducing token usage. After the run, the prompt and reply are merged back into the session so history stays complete.
+
+See [conditional-context.md](conditional-context.md) for details and configuration.
+
+## Persistence
+
+Memory persists because the agent writes to disk during conversation. When a session compacts or restarts, the vault files remain — the agent reads them fresh on next interaction.
+
+The vault is plain markdown. You can:
+- Open it in Obsidian and browse the graph
+- Edit files directly from the host
+- Back it up like any other directory
+
+## Space preferences (built-in)
+
+Short, structured **per-space** hints (sources, habits, rules) can be stored in SQLite and managed from chat via `mrctl prefs` (see the built-in **preferences** skill). The host injects them into each run as a `<preferences>` block in the user prompt. See [prd-chat-memory.md](prd-chat-memory.md).
+
+## Configuration
+
+Memory behavior is controlled by installed extensions and their config.
+
+To use a shared Obsidian vault across tools, symlink or mount it:
+
+```bash
+ln -s /path/to/your/vault .mercury/spaces/main
+```

package/docs/permissions.md

@@ -0,0 +1,217 @@
+# Permissions
+
+Mercury uses role-based access control (RBAC) per space. Each user has a role, and each role has a set of permissions.
+
+## How It Works
+
+```
+Message arrives
+  │
+  ├─► Resolve caller's role
+  │     • System caller? → role = "system"
+  │     • Seeded admin? → grant admin, store in DB
+  │     • Existing role in DB? → use it
+  │     • Otherwise → "member"
+  │
+  ├─► Load role's permissions
+  │     • Check space_config for override
+  │     • Fall back to built-in defaults
+  │
+  └─► Check permission for action
+        • Has permission → proceed
+        • Denied → return error
+```
+
+## Roles
+
+| Role | Default Permissions | Description |
+|------|---------------------|-------------|
+| `system` | All | Internal system caller (scheduler, etc.) — not assignable |
+| `admin` | All | Full control over the space |
+| `member` | `prompt`, `prefs.get` | Can chat and read space preferences (default for new users) |
+
+Custom roles can be created by assigning permissions to any role name.
+
+## Permissions
+
+| Permission | Description |
+|------------|-------------|
+| `prompt` | Send messages to the assistant |
+| `stop` | Abort running agent and clear queue |
+| `compact` | Reset session boundary (fresh context) |
+| `tasks.list` | View scheduled tasks |
+| `tasks.create` | Create new scheduled tasks |
+| `tasks.pause` | Pause scheduled tasks |
+| `tasks.resume` | Resume paused tasks |
+| `tasks.delete` | Delete scheduled tasks |
+| `config.get` | Read space configuration |
+| `config.set` | Modify space configuration |
+| `prefs.get` | Read space preferences (`mrctl prefs list/get`) |
+| `prefs.set` | Create, update, or delete space preferences (`mrctl prefs set/delete`) |
+| `roles.list` | View roles in the space |
+| `roles.grant` | Assign roles to users |
+| `roles.revoke` | Remove roles from users |
+| `permissions.get` | View role permissions |
+| `permissions.set` | Modify role permissions |
+| `spaces.list` | View all spaces |
+| `spaces.rename` | Rename a space and link/unlink conversations |
+| `spaces.delete` | Delete current space and all related DB data |
+
+## Mutes
+
+Muted users’ messages are dropped for the space until the mute expires or is cleared. The agent can mute via `mrctl mute` (with API confirmation). **Operators** with dashboard access (`MERCURY_API_SECRET`) can list active mutes, unmute, or add a timed mute from **Spaces → (space) → Muted users**.
+
+## Managing Roles
+
+The agent uses `mrctl` to manage roles:
+
+```bash
+# List all roles in the current space
+mrctl roles list
+
+# Grant admin role to a user
+mrctl roles grant 1234567890@s.whatsapp.net --role admin
+
+# Grant a custom role
+mrctl roles grant 1234567890@s.whatsapp.net --role moderator
+
+# Revoke role (user becomes member)
+mrctl roles revoke 1234567890@s.whatsapp.net
+```
+
+## Managing Permissions
+
+Permissions are per-role, per-space:
+
+```bash
+# Show permissions for all roles
+mrctl permissions show
+
+# Show permissions for a specific role
+mrctl permissions show --role member
+
+# Give members ability to stop the agent
+mrctl permissions set member prompt,stop
+
+# Create a moderator role with task management
+mrctl permissions set moderator prompt,stop,tasks.list,tasks.pause,tasks.resume
+
+# Give a role full task control
+mrctl permissions set taskmaster prompt,tasks.list,tasks.create,tasks.pause,tasks.resume,tasks.delete
+```
+
+## Managing Spaces
+
+Spaces can be listed and managed via `mrctl`:
+
+```bash
+# List all spaces with their names
+mrctl spaces list
+
+# Get current space's display name
+mrctl spaces name
+
+# Set current space's display name
+mrctl spaces name "Startup Buddies"
+
+# Delete current space (tasks, messages, roles, config)
+mrctl spaces delete
+
+# List discovered conversations
+mrctl conversations list
+
+# Show only conversations that are not yet linked
+mrctl conversations list --unlinked
+```
+
+Space names are stored in the database and shown in logs/dashboard for easier identification. Conversations remain platform-native and are linked into spaces. Conversation listing uses `spaces.list`; linking and unlinking use `spaces.rename`.
+
+## Seeding Admins
+
+Pre-configure admin users via environment variable. They're granted admin on first interaction with each space:
+
+```bash
+MERCURY_ADMINS=1234567890@s.whatsapp.net,0987654321@s.whatsapp.net
+```
+
+This is useful for bootstrapping — the first admin can then grant roles to others.
+
+## Storage
+
+Roles and permissions are stored in SQLite:
+
+| Table | Purpose |
+|-------|---------|
+| `space_roles` | Maps `(space_id, platform_user_id)` → `role` |
+| `space_config` | Stores `role.<name>.permissions` overrides |
+
+Built-in defaults (`admin` = all, `member` = `prompt` + `prefs.get`) are not stored — they're applied when no override exists.
+
+## System Caller
+
+The `system` role is special:
+
+- Always has all permissions
+- Cannot be modified or assigned
+- Used for internal callers: scheduled tasks, system triggers
+
+```typescript
+if (isSystemCaller(callerId)) return "system";
+```
+
+## Extension Permissions
+
+Extensions register additional permissions at runtime via the extension API:
+
+```typescript
+mercury.permission({ defaultRoles: ["admin", "member"] });
+```
+
+This registers a permission named after the extension (e.g., `napkin`). The behavior:
+
+- **Admin** always gets all permissions (built-in + extension)
+- **Extension `defaultRoles`** are respected — if `["member"]` is specified, members get that permission by default
+- **Per-space overrides** still take precedence over defaults
+- **Built-in permission names** cannot be overridden by extensions
+
+Extension CLIs are called directly by the agent in bash. Permission enforcement is handled by a pi extension that blocks denied CLIs at the bash tool level, based on the caller's role and the `MERCURY_DENIED_CLIS` environment variable set by Mercury's runtime.
+
+Extensions that declare env vars via `mercury.env()` also have those vars gated by permission — they are only injected into containers when the caller has the extension's permission. This prevents credential leakage (e.g., a blocked `gh` CLI's `GH_TOKEN` being used via `curl`).
+
+### API
+
+```typescript
+registerPermission(name, { defaultRoles })   // Register (called by extension loader)
+getAllPermissions()                            // Built-in + extension permissions
+isValidPermission(name)                       // Check if name is valid
+resetPermissions()                            // Clear registered (test isolation)
+```
+
+## Scope
+
+Permissions are **per-space**:
+
+- A user can be `admin` in one space and `member` in another
+- Custom role permissions are space-specific
+- No global roles (except seeded admins, which apply on first interaction per space)
+
+## API
+
+### `resolveRole(db, spaceId, platformUserId, seededAdmins)`
+
+Determines a caller's role:
+1. System caller → `"system"`
+2. Seed admins if needed
+3. Upsert member record
+4. Return stored role or `"member"`
+
+### `getRolePermissions(db, spaceId, role)`
+
+Returns the permission set for a role:
+1. System role → all permissions
+2. Check `space_config` for `role.<name>.permissions`
+3. Fall back to built-in defaults
+
+### `hasPermission(db, spaceId, role, permission)`
+
+Returns `true` if the role has the specified permission.