mercury-agent 0.4.28 → 0.5.1

package/docs/graceful-shutdown.md

@@ -1,62 +1,62 @@
-# Graceful Shutdown
-
-On `SIGTERM` or `SIGINT`, mercury tears down all components in order instead of exiting abruptly.
-
-## Sequence
-
-```
-SIGTERM/SIGINT received
-  │
-  ├─1─► Stop scheduler (clear poll timer)
-  ├─2─► Cancel all pending queue entries
-  ├─3─► Kill running containers (docker kill)
-  ├─4─► Wait for active work to drain (up to 8s)
-  ├─5─► Disconnect adapters (WhatsApp socket, etc.)
-  ├─6─► Stop HTTP server
-  ├─7─► Close SQLite database
-  └─8─► exit(0)
-
-  Second signal → force exit(1)
-  10s timeout  → force exit(1)
-```
-
-## Why this order
-
-1. **Scheduler first** — prevents new work from being created while we're shutting down.
-2. **Cancel pending queue entries** — no point starting queued work we'll just kill.
-3. **Kill containers** — uses `docker kill` for reliable termination. Falls back to `SIGKILL` if the docker command fails. Containers are labeled with `mercury.managed=true` for identification (see [container-lifecycle.md](./container-lifecycle.md)).
-4. **Wait for drain** — gives active container runs a chance to finish cleanly (up to 8s).
-5. **Disconnect adapters** — closes the WhatsApp socket, Slack/Discord connections. Done after containers so in-flight replies can still be posted.
-6. **Stop HTTP server** — stops accepting new webhook/API requests.
-7. **Close DB last** — everything above may still write to the database (message storage, task updates), so the DB connection stays open until the very end.
-
-## Safety mechanisms
-
-| Mechanism | Behavior |
-|-----------|----------|
-| **Double-signal** | Second SIGINT/SIGTERM forces immediate `exit(1)` |
-| **Global timeout** | If cleanup takes longer than 10s, forced `exit(1)` |
-| **Idempotent** | `shutdown()` is guarded by a `shuttingDown` flag — calling it twice is a no-op |
-| **Hook errors** | Individual shutdown hook failures are logged and swallowed — remaining cleanup continues |
-
-## API
-
-### `MercuryCoreRuntime`
-
-```ts
-core.installSignalHandlers()     // trap SIGTERM + SIGINT
-core.onShutdown(async () => {})  // register cleanup callback
-await core.shutdown(timeoutMs?)  // trigger shutdown manually (default 10s)
-core.isShuttingDown              // boolean
-```
-
-### Component methods used during shutdown
-
-| Component | Method | What it does |
-|-----------|--------|-------------|
-| `TaskScheduler` | `stop()` | Clears the poll timer |
-| `SpaceQueue` | `cancelAll()` | Drops all pending entries, returns count |
-| `SpaceQueue` | `waitForActive(ms)` | Resolves when active count hits 0 or timeout |
-| `AgentContainerRunner` | `killAll()` | Kill all running containers via `docker kill` |
-| `Db` | `close()` | Closes SQLite connection |
-| `WhatsAppBaileysAdapter` | `shutdown()` | Ends the Baileys socket |
+# Graceful Shutdown
+
+On `SIGTERM` or `SIGINT`, mercury tears down all components in order instead of exiting abruptly.
+
+## Sequence
+
+```
+SIGTERM/SIGINT received
+  │
+  ├─1─► Stop scheduler (clear poll timer)
+  ├─2─► Cancel all pending queue entries
+  ├─3─► Kill running containers (docker kill)
+  ├─4─► Wait for active work to drain (up to 8s)
+  ├─5─► Disconnect adapters (WhatsApp socket, etc.)
+  ├─6─► Stop HTTP server
+  ├─7─► Close SQLite database
+  └─8─► exit(0)
+
+  Second signal → force exit(1)
+  10s timeout  → force exit(1)
+```
+
+## Why this order
+
+1. **Scheduler first** — prevents new work from being created while we're shutting down.
+2. **Cancel pending queue entries** — no point starting queued work we'll just kill.
+3. **Kill containers** — uses `docker kill` for reliable termination. Falls back to `SIGKILL` if the docker command fails. Containers are labeled with `mercury.managed=true` for identification (see [container-lifecycle.md](./container-lifecycle.md)).
+4. **Wait for drain** — gives active container runs a chance to finish cleanly (up to 8s).
+5. **Disconnect adapters** — closes the WhatsApp socket, Slack/Discord connections. Done after containers so in-flight replies can still be posted.
+6. **Stop HTTP server** — stops accepting new webhook/API requests.
+7. **Close DB last** — everything above may still write to the database (message storage, task updates), so the DB connection stays open until the very end.
+
+## Safety mechanisms
+
+| Mechanism | Behavior |
+|-----------|----------|
+| **Double-signal** | Second SIGINT/SIGTERM forces immediate `exit(1)` |
+| **Global timeout** | If cleanup takes longer than 10s, forced `exit(1)` |
+| **Idempotent** | `shutdown()` is guarded by a `shuttingDown` flag — calling it twice is a no-op |
+| **Hook errors** | Individual shutdown hook failures are logged and swallowed — remaining cleanup continues |
+
+## API
+
+### `MercuryCoreRuntime`
+
+```ts
+core.installSignalHandlers()     // trap SIGTERM + SIGINT
+core.onShutdown(async () => {})  // register cleanup callback
+await core.shutdown(timeoutMs?)  // trigger shutdown manually (default 10s)
+core.isShuttingDown              // boolean
+```
+
+### Component methods used during shutdown
+
+| Component | Method | What it does |
+|-----------|--------|-------------|
+| `TaskScheduler` | `stop()` | Clears the poll timer |
+| `SpaceQueue` | `cancelAll()` | Drops all pending entries, returns count |
+| `SpaceQueue` | `waitForActive(ms)` | Resolves when active count hits 0 or timeout |
+| `AgentContainerRunner` | `killAll()` | Kill all running containers via `docker kill` |
+| `Db` | `close()` | Closes SQLite connection |
+| `WhatsAppBaileysAdapter` | `shutdown()` | Ends the Baileys socket |

package/docs/kb-distillation.md

@@ -1,77 +1,77 @@
-# KB Distillation
-
-KB distillation is **extension-based** in Mercury (not built-in).
-
-As of v0.3.x, Mercury no longer ships a built-in `kb-distill` extension. The recommended approach is to use a user-installed extension (for example, `napkin`) that runs a background job and writes distilled knowledge into each space vault.
-
----
-
-## Recommended Setup
-
-Use the real example extension at:
-
-- `examples/extensions/napkin/`
-
-It demonstrates:
-
-- `workspace_init` hook to scaffold vault structure
-- `before_container` hook to set `NAPKIN_VAULT`
-- background job (`distill`) for periodic extraction
-- dashboard widget + extension store state
-
----
-
-## How Distillation Works (Extension Pattern)
-
-Typical flow:
-
-1. Read messages from `state.db` per `space_id`
-2. Export messages to `.messages/YYYY-MM-DD.jsonl`
-3. Detect changed files (hash compare)
-4. Run `pi` with a distillation prompt against changed files
-5. Update vault files incrementally (append/update, not destructive rewrite)
-
-This keeps runs idempotent and avoids re-processing unchanged days.
-
----
-
-## Data Layout (napkin example)
-
-```text
-.mercury/spaces/<space-id>/
-├── .messages/
-│   └── YYYY-MM-DD.jsonl
-└── knowledge/
-    ├── people/
-    ├── projects/
-    ├── references/
-    ├── daily/
-    └── templates/
-```
-
-Message export format:
-
-```json
-{"ts":1709123456,"role":"ambient","content":"Alice: Great idea!"}
-{"ts":1709123457,"role":"user","content":"What do you think about X?"}
-{"ts":1709123458,"role":"assistant","content":"I think..."}
-```
-
----
-
-## Configuration
-
-With the napkin example extension:
-
-- `MERCURY_KB_DISTILL_INTERVAL_MS=0` → disabled (default)
-- `MERCURY_KB_DISTILL_INTERVAL_MS=3600000` → check every hour
-
-You can also expose interval as extension config keys (see `examples/extensions/napkin/index.ts`).
-
----
-
-## Notes
-
-- Distillation behavior depends on the installed extension implementation
-- Mercury core provides hooks, jobs, DB access, and workspace isolation
-- Distillation logic/prompt lives in the extension, not in core Mercury
+# KB Distillation
+
+KB distillation is **extension-based** in Mercury (not built-in).
+
+As of v0.3.x, Mercury no longer ships a built-in `kb-distill` extension. The recommended approach is to use a user-installed extension (for example, `napkin`) that runs a background job and writes distilled knowledge into each space vault.
+
+---
+
+## Recommended Setup
+
+Use the real example extension at:
+
+- `examples/extensions/napkin/`
+
+It demonstrates:
+
+- `workspace_init` hook to scaffold vault structure
+- `before_container` hook to set `NAPKIN_VAULT`
+- background job (`distill`) for periodic extraction
+- dashboard widget + extension store state
+
+---
+
+## How Distillation Works (Extension Pattern)
+
+Typical flow:
+
+1. Read messages from `state.db` per `space_id`
+2. Export messages to `.messages/YYYY-MM-DD.jsonl`
+3. Detect changed files (hash compare)
+4. Run `pi` with a distillation prompt against changed files
+5. Update vault files incrementally (append/update, not destructive rewrite)
+
+This keeps runs idempotent and avoids re-processing unchanged days.
+
+---
+
+## Data Layout (napkin example)
+
+```text
+.mercury/spaces/<space-id>/
+├── .messages/
+│   └── YYYY-MM-DD.jsonl
+└── knowledge/
+    ├── people/
+    ├── projects/
+    ├── references/
+    ├── daily/
+    └── templates/
+```
+
+Message export format:
+
+```json
+{"ts":1709123456,"role":"ambient","content":"Alice: Great idea!"}
+{"ts":1709123457,"role":"user","content":"What do you think about X?"}
+{"ts":1709123458,"role":"assistant","content":"I think..."}
+```
+
+---
+
+## Configuration
+
+With the napkin example extension:
+
+- `MERCURY_KB_DISTILL_INTERVAL_MS=0` → disabled (default)
+- `MERCURY_KB_DISTILL_INTERVAL_MS=3600000` → check every hour
+
+You can also expose interval as extension config keys (see `examples/extensions/napkin/index.ts`).
+
+---
+
+## Notes
+
+- Distillation behavior depends on the installed extension implementation
+- Mercury core provides hooks, jobs, DB access, and workspace isolation
+- Distillation logic/prompt lives in the extension, not in core Mercury

package/docs/media/overview.md

@@ -1,140 +1,140 @@
-# Media Handling
-
-Mercury downloads and processes media attachments from chat platforms, saving them to space workspaces and passing them to pi for processing. Models can also produce files via the `outbox/` directory.
-
-## Supported Platforms
-
-| Platform | Ingress | Egress | Details |
-|----------|---------|--------|---------|
-| WhatsApp | ✅ Baileys socket | ✅ image/video/audio/document | [whatsapp.md](./whatsapp.md) |
-| Discord | ✅ CDN URL download | ✅ channel.send() with files | Via `DiscordBridge` |
-| Slack | ✅ URL download (auth'd) | ✅ files.uploadV2 | Via `SlackBridge` |
-
-## Architecture
-
-```
-┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌─────────┐
-│   Platform   │────▶│   Bridge     │────▶│   Runtime    │────▶│   pi    │
-│              │     │ (normalize)  │     │ (store/pass) │     │ (view)  │
-└──────────────┘     └──────────────┘     └──────────────┘     └─────────┘
-                            │                    │
-                            ▼                    ▼
-                     ┌──────────────┐     ┌──────────────┐
-                     │  Workspace   │     │  Workspace   │
-                     │   /inbox/    │     │   /outbox/   │
-                     └──────────────┘     └──────────────┘
-```
-
-## Media Types
-
-All platforms map to these generic types defined in `src/types.ts`:
-
-```typescript
-type MediaType = "image" | "video" | "audio" | "voice" | "document";
-
-interface MessageAttachment {
-  path: string;        // Local file path
-  type: MediaType;     // Generic type
-  mimeType: string;    // MIME type (e.g., "image/jpeg")
-  filename?: string;   // Original filename if available
-  sizeBytes?: number;  // File size in bytes
-}
-```
-
-## Configuration
-
-| Env Variable | Default | Description |
-|--------------|---------|-------------|
-| `MERCURY_MEDIA_ENABLED` | `true` | Enable/disable media downloads |
-| `MERCURY_MEDIA_MAX_SIZE_MB` | `10` | Max file size to download (MB) |
-
-## Storage
-
-### Ingress (inbox/)
-
-Incoming media files are saved to the space workspace:
-
-```
-.mercury/spaces/<space_id>/inbox/<timestamp>-<type>.<ext>
-```
-
-Example:
-```
-.mercury/spaces/whatsapp_123456_g_us/inbox/
-├── 1709012345-image.jpg
-├── 1709012400-voice.ogg
-└── 1709012500-document.pdf
-```
-
-### Egress (outbox/)
-
-The model writes files to `outbox/` during a container run. After exit, the runtime scans for files with `mtime >= startTime` and attaches them to the reply:
-
-```
-.mercury/spaces/<space_id>/outbox/
-├── chart.png
-└── summary.pdf
-```
-
-Previous outbox files are not deleted — only new or modified files are sent. See [pipeline.md](../pipeline.md) for details.
-
-## Database Schema
-
-Attachments are stored as JSON in the `messages.attachments` column:
-
-```sql
-ALTER TABLE messages ADD COLUMN attachments TEXT;
-```
-
-```json
-[
-  {
-    "path": "/Users/.../media/1709012345-image.jpg",
-    "type": "image",
-    "mimeType": "image/jpeg",
-    "sizeBytes": 12345
-  }
-]
-```
-
-## Prompt Format
-
-Attachments are passed to pi as XML:
-
-```xml
-<attachments>
-  <attachment type="image" path="/spaces/xxx/media/123-image.jpg" mime="image/jpeg" size="12345" />
-</attachments>
-
-@mercury what's in this image?
-```
-
-Reply context includes media info:
-
-```xml
-<reply_to name="John" jid="123@wa" message_id="ABC" media_type="image" media_mime="image/jpeg">
-Check out this sunset!
-</reply_to>
-```
-
-## pi Capabilities
-
-| Media Type | pi Support |
-|------------|------------|
-| Images (jpg, png, gif, webp) | ✅ Can view via `read` tool |
-| Voice/Audio | ❌ Cannot play — needs transcription |
-| Video | ❌ Cannot play — could extract frames |
-| Documents (txt, code) | ✅ Can read text-based files |
-| Documents (pdf, docx) | ❌ Cannot read binary formats |
-
-## Voice transcription
-
-Voice and audio attachments are not playable inside pi. Install the **`voice-transcribe`** extension (see dashboard catalog or `examples/extensions/voice-transcribe/`) to append a text transcript before the agent runs.
-
-- **Default (`voice-transcribe.provider=local`)**: runs Python on the Mercury host; install deps from the extension’s `requirements.txt`. Set `voice-transcribe.local_engine` to `transformers` (default, e.g. [mike249/whisper-tiny-he-2](https://huggingface.co/mike249/whisper-tiny-he-2)) or `faster_whisper` for [CTranslate2](https://github.com/OpenNMT/CTranslate2) models on the Hub (e.g. [ivrit-ai](https://huggingface.co/ivrit-ai) `*-ct2` repos). See the extension skill for `MERCURY_VOICE_FW_COMPUTE_TYPE` and `MERCURY_VOICE_LANGUAGE`.
-- **API (`voice-transcribe.provider=api`)**: uses the [Hugging Face Inference API](https://huggingface.co/docs/api-inference) with `MERCURY_HF_TOKEN` — choose a model that has a Hub Inference Provider.
-
-## Future Enhancements
-
-- [ ] Video frame extraction
-- [ ] PDF text extraction
+# Media Handling
+
+Mercury downloads and processes media attachments from chat platforms, saving them to space workspaces and passing them to pi for processing. Models can also produce files via the `outbox/` directory.
+
+## Supported Platforms
+
+| Platform | Ingress | Egress | Details |
+|----------|---------|--------|---------|
+| WhatsApp | ✅ Baileys socket | ✅ image/video/audio/document | [whatsapp.md](./whatsapp.md) |
+| Discord | ✅ CDN URL download | ✅ channel.send() with files | Via `DiscordBridge` |
+| Slack | ✅ URL download (auth'd) | ✅ files.uploadV2 | Via `SlackBridge` |
+
+## Architecture
+
+```
+┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌─────────┐
+│   Platform   │────▶│   Bridge     │────▶│   Runtime    │────▶│   pi    │
+│              │     │ (normalize)  │     │ (store/pass) │     │ (view)  │
+└──────────────┘     └──────────────┘     └──────────────┘     └─────────┘
+                            │                    │
+                            ▼                    ▼
+                     ┌──────────────┐     ┌──────────────┐
+                     │  Workspace   │     │  Workspace   │
+                     │   /inbox/    │     │   /outbox/   │
+                     └──────────────┘     └──────────────┘
+```
+
+## Media Types
+
+All platforms map to these generic types defined in `src/types.ts`:
+
+```typescript
+type MediaType = "image" | "video" | "audio" | "voice" | "document";
+
+interface MessageAttachment {
+  path: string;        // Local file path
+  type: MediaType;     // Generic type
+  mimeType: string;    // MIME type (e.g., "image/jpeg")
+  filename?: string;   // Original filename if available
+  sizeBytes?: number;  // File size in bytes
+}
+```
+
+## Configuration
+
+| Env Variable | Default | Description |
+|--------------|---------|-------------|
+| `MERCURY_MEDIA_ENABLED` | `true` | Enable/disable media downloads |
+| `MERCURY_MEDIA_MAX_SIZE_MB` | `10` | Max file size to download (MB) |
+
+## Storage
+
+### Ingress (inbox/)
+
+Incoming media files are saved to the space workspace:
+
+```
+.mercury/spaces/<space_id>/inbox/<timestamp>-<type>.<ext>
+```
+
+Example:
+```
+.mercury/spaces/whatsapp_123456_g_us/inbox/
+├── 1709012345-image.jpg
+├── 1709012400-voice.ogg
+└── 1709012500-document.pdf
+```
+
+### Egress (outbox/)
+
+The model writes files to `outbox/` during a container run. After exit, the runtime scans for files with `mtime >= startTime` and attaches them to the reply:
+
+```
+.mercury/spaces/<space_id>/outbox/
+├── chart.png
+└── summary.pdf
+```
+
+Previous outbox files are not deleted — only new or modified files are sent. See [pipeline.md](../pipeline.md) for details.
+
+## Database Schema
+
+Attachments are stored as JSON in the `messages.attachments` column:
+
+```sql
+ALTER TABLE messages ADD COLUMN attachments TEXT;
+```
+
+```json
+[
+  {
+    "path": "/Users/.../media/1709012345-image.jpg",
+    "type": "image",
+    "mimeType": "image/jpeg",
+    "sizeBytes": 12345
+  }
+]
+```
+
+## Prompt Format
+
+Attachments are passed to pi as XML:
+
+```xml
+<attachments>
+  <attachment type="image" path="/spaces/xxx/media/123-image.jpg" mime="image/jpeg" size="12345" />
+</attachments>
+
+@mercury what's in this image?
+```
+
+Reply context includes media info:
+
+```xml
+<reply_to name="John" jid="123@wa" message_id="ABC" media_type="image" media_mime="image/jpeg">
+Check out this sunset!
+</reply_to>
+```
+
+## pi Capabilities
+
+| Media Type | pi Support |
+|------------|------------|
+| Images (jpg, png, gif, webp) | ✅ Can view via `read` tool |
+| Voice/Audio | ❌ Cannot play — needs transcription |
+| Video | ❌ Cannot play — could extract frames |
+| Documents (txt, code) | ✅ Can read text-based files |
+| Documents (pdf, docx) | ❌ Cannot read binary formats |
+
+## Voice transcription
+
+Voice and audio attachments are not playable inside pi. Install the **`voice-transcribe`** extension (see dashboard catalog or `examples/extensions/voice-transcribe/`) to append a text transcript before the agent runs.
+
+- **Default (`voice-transcribe.provider=local`)**: runs Python on the Mercury host; install deps from the extension’s `requirements.txt`. Set `voice-transcribe.local_engine` to `transformers` (default, e.g. [mike249/whisper-tiny-he-2](https://huggingface.co/mike249/whisper-tiny-he-2)) or `faster_whisper` for [CTranslate2](https://github.com/OpenNMT/CTranslate2) models on the Hub (e.g. [ivrit-ai](https://huggingface.co/ivrit-ai) `*-ct2` repos). See the extension skill for `MERCURY_VOICE_FW_COMPUTE_TYPE` and `MERCURY_VOICE_LANGUAGE`.
+- **API (`voice-transcribe.provider=api`)**: uses the [Hugging Face Inference API](https://huggingface.co/docs/api-inference) with `MERCURY_HF_TOKEN` — choose a model that has a Hub Inference Provider.
+
+## Future Enhancements
+
+- [ ] Video frame extraction
+- [ ] PDF text extraction