mercury-agent 0.4.26 → 0.4.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +24 -1
- package/docs/ROADMAP.md +3 -2
- package/docs/backlog/applicative-profiles.md +135 -149
- package/docs/backlog/caller-bound-capability-token.md +180 -0
- package/docs/debug/major/2026-07-01-strict-yaml-schema-crash.md +85 -0
- package/docs/ideas/business-extensions.md +48 -0
- package/docs/runbooks/gws-oauth-setup.md +211 -0
- package/package.json +1 -1
- package/resources/templates/mercury.example.yaml +84 -84
- package/src/config-file.ts +118 -18
- package/src/config.ts +1 -1
- package/src/core/conversation.ts +2 -2
- package/src/core/handler.ts +1 -1
- package/src/core/routes/console.ts +32 -0
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
# Caller-Bound Capability Token
|
|
2
|
+
|
|
3
|
+
**Status**: Backlog
|
|
4
|
+
**Slug**: caller-bound-capability-token
|
|
5
|
+
**Created**: 2026-07-01
|
|
6
|
+
**Last updated**: 2026-07-01
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## Goal
|
|
11
|
+
|
|
12
|
+
Mercury's in-container CLIs (`mrctl` and extension CLIs) call back to the host control-plane API to perform privileged work. Today that callback authenticates with a **single shared secret** (`API_SECRET`, injected into every container) and the host **trusts the `x-mercury-caller` / `x-mercury-space` request headers verbatim** (`src/core/api.ts:50-72`, `src/cli/mrctl.ts:34-40`). Because `API_SECRET` is readable inside the container, any code running there can call the host API asserting **any** caller or space — i.e. caller identity is spoofable.
|
|
13
|
+
|
|
14
|
+
This is acceptable today because host-brokered capabilities (TradeStation, TTS) are **admin-only** — everyone who can invoke them is already trusted. It becomes a security hole the moment an **untrusted member** gets a host-brokered capability, which is exactly what [[applicative-profiles]] introduces (e.g. a barber-shop customer who must only touch their own appointments). A spoofed caller header would let one customer act as another.
|
|
15
|
+
|
|
16
|
+
This feature adds a **per-turn capability token** minted at container spawn, cryptographically bound to `{callerId, spaceId}`, that the host verifies and uses as the authoritative identity for authorization — instead of trusting request headers.
|
|
17
|
+
|
|
18
|
+
## User Stories
|
|
19
|
+
|
|
20
|
+
- As a Mercury operator, I want host-brokered capabilities to attribute each request to the real caller so an untrusted member cannot impersonate another member.
|
|
21
|
+
- As a profile developer, I want a trustworthy `callerId` on the host side so I can enforce per-user ownership rules safely.
|
|
22
|
+
- As a Mercury maintainer, I want existing admin-only brokers (TradeStation) to stop relying on spoofable headers, closing an inherited weakness.
|
|
23
|
+
|
|
24
|
+
## MVP Scope
|
|
25
|
+
|
|
26
|
+
**In scope:**
|
|
27
|
+
- Mint a per-turn token at container spawn (`src/agent/container-runner.ts`), bound to `{callerId, spaceId}` plus an expiry, injected as an env var (e.g. `MERCURY_CALLER_TOKEN`)
|
|
28
|
+
- Host-side verification in the API auth middleware (`src/core/api.ts`): when the token is present, derive `callerId`/`spaceId` **from the token**, not from `x-mercury-*` headers, for all authorization decisions
|
|
29
|
+
- HMAC signing with a host-only key (no new dependency); token is opaque to the container
|
|
30
|
+
- Migrate the TradeStation caller-bound confirm guard to token-derived identity (`src/core/routes/tradestation.ts:153-158, 245-254`)
|
|
31
|
+
- Backward compatibility: shared-`API_SECRET` path remains for system/admin flows; token is authoritative when present
|
|
32
|
+
|
|
33
|
+
**Out of scope (deferred):**
|
|
34
|
+
- Full JWT/JWKS or key rotation infrastructure — a single host-held HMAC key is sufficient for v1
|
|
35
|
+
- Multi-turn / session-scoped tokens — one token per container turn is enough
|
|
36
|
+
- Revocation lists — tokens are short-lived (expire with the turn)
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Context for Claude
|
|
41
|
+
|
|
42
|
+
- `src/core/api.ts` — Hono control-plane API; auth middleware at lines 35-75 (`safeCompare`, header identity resolution at 50-72). This is where token verification is added
|
|
43
|
+
- `src/cli/mrctl.ts` — in-container CLI; builds request headers at lines 33-41, reads `API_URL`/`API_SECRET`/`CALLER_ID`/`SPACE_ID` from env
|
|
44
|
+
- `src/agent/container-runner.ts` — container spawn; injects `API_URL`/`API_SECRET`/`CALLER_ID`/`SPACE_ID` at ~810-817. This is where the token is minted and injected
|
|
45
|
+
- `src/core/routes/tradestation.ts` — existing caller-bound guard (lines 153-158, 245-254) that currently trusts header identity; the reference consumer to migrate
|
|
46
|
+
- `src/config.ts` — `apiSecret` config (line 212); add the host-only signing key here
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Architecture & Data
|
|
51
|
+
|
|
52
|
+
### Token shape
|
|
53
|
+
|
|
54
|
+
Opaque HMAC-signed token (no new dependency — Node `crypto`):
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
token = base64url(payload) + "." + base64url(HMAC_SHA256(key, base64url(payload)))
|
|
58
|
+
payload = { c: callerId, s: spaceId, exp: <unix-seconds> }
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
- `key` — a host-only secret (`MERCURY_CALLER_TOKEN_KEY`, auto-derived from `API_SECRET` if unset so no new required config). Never injected into containers.
|
|
62
|
+
- `exp` — short TTL (e.g. spawn time + container timeout). The token is useless after the turn.
|
|
63
|
+
|
|
64
|
+
### Flow
|
|
65
|
+
|
|
66
|
+
1. **Spawn** (`container-runner.ts`): mint token from `{callerId, spaceId, exp}`, inject as `MERCURY_CALLER_TOKEN`. Do **not** inject the signing key.
|
|
67
|
+
2. **Callback** (`mrctl.ts`): forward `MERCURY_CALLER_TOKEN` as an `x-mercury-token` header (in addition to the existing `Authorization: Bearer API_SECRET`).
|
|
68
|
+
3. **Verify** (`api.ts` middleware): if `x-mercury-token` is present, verify the HMAC and expiry; on success, set the request's authoritative `callerId`/`spaceId` from the token payload and **ignore** `x-mercury-caller`/`x-mercury-space` for authorization. If absent, fall back to today's behavior (shared secret + headers) for backward compatibility.
|
|
69
|
+
|
|
70
|
+
### Why the token can't be forged from inside the container
|
|
71
|
+
|
|
72
|
+
The signing key never enters the container. The container holds only its **own** turn's token (valid for its own caller/space). It cannot mint a token for a different caller, and the shared `API_SECRET` alone no longer authorizes an identity claim once a route requires the token.
|
|
73
|
+
|
|
74
|
+
### File & Folder Structure
|
|
75
|
+
|
|
76
|
+
| Path | New / Modified | Purpose |
|
|
77
|
+
|------|---------------|---------|
|
|
78
|
+
| `src/core/caller-token.ts` | New | `mintCallerToken()` / `verifyCallerToken()` HMAC helpers |
|
|
79
|
+
| `src/agent/container-runner.ts` | Modified | Mint + inject `MERCURY_CALLER_TOKEN` at spawn; keep key host-side |
|
|
80
|
+
| `src/cli/mrctl.ts` | Modified | Forward token as `x-mercury-token` header |
|
|
81
|
+
| `src/core/api.ts` | Modified | Verify token; prefer token identity over headers |
|
|
82
|
+
| `src/core/routes/tradestation.ts` | Modified | Use token-derived caller for the confirm guard |
|
|
83
|
+
| `src/config.ts` | Modified | Add `callerTokenKey` (derives from `apiSecret` when unset) |
|
|
84
|
+
| `tests/caller-token.test.ts` | New | Mint/verify, tamper, expiry, spoof-rejection tests |
|
|
85
|
+
|
|
86
|
+
### Implementation Sequence
|
|
87
|
+
|
|
88
|
+
1. **Token helpers** — `mintCallerToken({callerId, spaceId, exp}, key)` and `verifyCallerToken(token, key)` in `caller-token.ts`.
|
|
89
|
+
- Read first: `src/config.ts`
|
|
90
|
+
- Verify: unit tests for round-trip + tamper rejection pass
|
|
91
|
+
2. **Mint at spawn** — inject `MERCURY_CALLER_TOKEN` in `container-runner.ts`; ensure the key is NOT injected (add to the env blocklist alongside `MERCURY_API_SECRET`).
|
|
92
|
+
- Read first: `src/agent/container-runner.ts` (~810-817, blocklist ~684)
|
|
93
|
+
- Verify: token present in container env; key absent
|
|
94
|
+
- Blocker: Step 1
|
|
95
|
+
3. **Forward in mrctl** — add `x-mercury-token` header.
|
|
96
|
+
- Read first: `src/cli/mrctl.ts` (33-41)
|
|
97
|
+
- Verify: header present on requests
|
|
98
|
+
- Blocker: Step 2
|
|
99
|
+
4. **Verify host-side** — token-preferred identity in `api.ts` middleware; header fallback retained.
|
|
100
|
+
- Read first: `src/core/api.ts` (35-75)
|
|
101
|
+
- Verify: request with valid token uses token identity; spoofed `x-mercury-caller` is ignored
|
|
102
|
+
- Blocker: Steps 1, 3
|
|
103
|
+
5. **Migrate TradeStation guard** — derive confirm-guard caller from token.
|
|
104
|
+
- Read first: `src/core/routes/tradestation.ts` (153-158, 245-254)
|
|
105
|
+
- Verify: existing TradeStation tests pass with token identity
|
|
106
|
+
- Blocker: Step 4
|
|
107
|
+
6. **Tests** — spoof rejection, tamper, expiry, backward-compat fallback.
|
|
108
|
+
- Verify: `bun test` passes
|
|
109
|
+
- Blocker: Steps 1-5
|
|
110
|
+
|
|
111
|
+
---
|
|
112
|
+
|
|
113
|
+
## Non-Negotiable Rules
|
|
114
|
+
|
|
115
|
+
- **Signing key never enters a container** — it stays host-side; only per-turn tokens are injected.
|
|
116
|
+
- **Token identity wins** — when a token is present and valid, `x-mercury-caller`/`x-mercury-space` are ignored for authorization.
|
|
117
|
+
- **Short-lived** — tokens expire with the turn; no long-lived caller tokens.
|
|
118
|
+
- **Backward compatible** — absent a token, existing shared-secret + header behavior is unchanged, so nothing breaks for current admin-only brokers during rollout.
|
|
119
|
+
- **Constant-time verification** — HMAC compare uses `crypto.timingSafeEqual` (mirror `safeCompare` in `api.ts`).
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
## Edge Cases & Risks
|
|
124
|
+
|
|
125
|
+
| Scenario | Handling |
|
|
126
|
+
|----------|---------|
|
|
127
|
+
| Token expired mid-long-turn | Route returns 401; treat as a transient auth failure. TTL should exceed the container timeout to avoid this. |
|
|
128
|
+
| System-triggered task (`callerId = "system"`) | Mint a token for `system` too, or allow the shared-secret path for system callers explicitly. |
|
|
129
|
+
| Rollout with old containers (no token) | Header/shared-secret fallback keeps them working until images are rebuilt. |
|
|
130
|
+
| Signing key unset | Derive deterministically from `API_SECRET` (documented) so no new required config. |
|
|
131
|
+
| Attacker replays a captured token | Bound to `{caller, space, exp}` and only valid for that caller's own actions; replay grants nothing beyond what that caller already had, and expires with the turn. |
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## Implementation Checklist
|
|
136
|
+
|
|
137
|
+
### Phase 1 — Goal
|
|
138
|
+
- [x] Goal, User Stories, and MVP Scope written and reviewed
|
|
139
|
+
|
|
140
|
+
### Phase 2 — Architecture
|
|
141
|
+
- [x] Architecture & Data section complete
|
|
142
|
+
- [x] Non-Negotiable Rules defined
|
|
143
|
+
- [x] Edge Cases covered
|
|
144
|
+
- [x] Context for Claude pointers filled in
|
|
145
|
+
|
|
146
|
+
### Implementation
|
|
147
|
+
- [ ] Step 1 — Token helpers
|
|
148
|
+
- [ ] Step 2 — Mint at spawn
|
|
149
|
+
- [ ] Step 3 — Forward in mrctl
|
|
150
|
+
- [ ] Step 4 — Verify host-side
|
|
151
|
+
- [ ] Step 5 — Migrate TradeStation guard
|
|
152
|
+
- [ ] Step 6 — Tests
|
|
153
|
+
- [ ] `bun run check` passes
|
|
154
|
+
- [ ] No secrets committed
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## Open Questions
|
|
159
|
+
|
|
160
|
+
- [ ] Should `system` callers use a token or keep the shared-secret path? — owner: TBD
|
|
161
|
+
- [ ] TTL value — fixed, or derived from the per-turn container timeout? — owner: TBD
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## Retrospective
|
|
166
|
+
|
|
167
|
+
**Residual risks / follow-ups:**
|
|
168
|
+
- none
|
|
169
|
+
|
|
170
|
+
**What changed from the plan:**
|
|
171
|
+
- ...
|
|
172
|
+
|
|
173
|
+
**Key decisions made during implementation:**
|
|
174
|
+
- ...
|
|
175
|
+
|
|
176
|
+
**Architecture impact:**
|
|
177
|
+
- [ ] New package/module added
|
|
178
|
+
- [ ] New data model or schema change
|
|
179
|
+
- [ ] New inter-package interaction
|
|
180
|
+
- [ ] Deployment topology changed
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Bug: Strict YAML schema crashes Mercury on unknown config keys
|
|
2
|
+
|
|
3
|
+
**Status**: Fixed
|
|
4
|
+
**Severity**: major
|
|
5
|
+
**Slug**: strict-yaml-schema-crash
|
|
6
|
+
**Reported**: 2026-07-01
|
|
7
|
+
**Last updated**: 2026-07-01
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Summary
|
|
12
|
+
`mercuryFileSchema` uses `.strict()` on all Zod objects, causing Mercury to crash on startup when `mercury.yaml` contains any unrecognized key — a typo, a key from a newer version, or a renamed field. Combined with PM2's default unlimited restarts and Baileys' session-based auth, this creates a cascade: config error → crash loop → WhatsApp session revoked → extended downtime requiring manual QR re-scan.
|
|
13
|
+
|
|
14
|
+
## Steps to Reproduce
|
|
15
|
+
1. Add any unrecognized key to `mercury.yaml`, e.g. `dm_auto_space: { admin_numbers: ["123"] }` (renamed to `admin_ids` in 0.4.27)
|
|
16
|
+
2. Start Mercury (`mercury run` or `mercury service install`)
|
|
17
|
+
3. Mercury crashes with `Invalid mercury.yaml: dm_auto_space: Unrecognized key: "admin_numbers"`
|
|
18
|
+
4. PM2 restarts Mercury in a crash loop, each restart fails identically
|
|
19
|
+
5. Crash loop causes WhatsApp session to expire (reason=401), requiring re-authentication
|
|
20
|
+
|
|
21
|
+
## Expected Behavior
|
|
22
|
+
Mercury should log a warning about the unknown key and continue starting normally. Invalid *values* (wrong type, out of range) should still crash — the distinction is between "unknown key" (safe to ignore) and "invalid value" (can't run correctly).
|
|
23
|
+
|
|
24
|
+
## Actual Behavior
|
|
25
|
+
Mercury throws an unhandled error and exits. PM2 restart loop compounds the damage by expiring the WhatsApp session.
|
|
26
|
+
|
|
27
|
+
## Impact
|
|
28
|
+
- **Service downtime** — entire Mercury instance goes down, no messages processed
|
|
29
|
+
- **WhatsApp session loss** — crash loop causes Baileys auth to expire, requiring manual QR re-scan
|
|
30
|
+
- **Version upgrade risk** — any field rename between versions (like `admin_numbers` → `admin_ids`) crashes deployments that haven't updated their yaml
|
|
31
|
+
- **Typo intolerance** — a single typo in mercury.yaml takes down the whole service
|
|
32
|
+
|
|
33
|
+
## Root Cause Analysis
|
|
34
|
+
|
|
35
|
+
Two separate concerns are conflated by `.strict()`:
|
|
36
|
+
|
|
37
|
+
1. **Unknown keys** (typo, version mismatch, renamed field) — should warn, not crash. The config isn't invalid, it's just unrecognized. Even with WABA, crashing because of a typo is wrong.
|
|
38
|
+
2. **Invalid values** (wrong type, out of range) — should crash. If `port: "abc"`, the service can't run correctly. Fail fast is correct.
|
|
39
|
+
|
|
40
|
+
The Baileys crash-loop cascade is a separate but compounding problem — on the unofficial WhatsApp protocol, rapid reconnect attempts trigger session revocation by Meta. This makes *any* crash loop catastrophic, not just config-related ones.
|
|
41
|
+
|
|
42
|
+
## Suspected Location
|
|
43
|
+
- `src/config-file.ts` — `mercuryFileSchema` (line 25) and all nested objects use `.strict()`, which rejects unknown keys
|
|
44
|
+
- `src/config-file.ts` — `mergeRawMercuryConfig()` (line 352) calls `safeParse` but throws on failure instead of warning
|
|
45
|
+
|
|
46
|
+
## Fix Plan
|
|
47
|
+
|
|
48
|
+
### Phase 1 — Short term (now)
|
|
49
|
+
|
|
50
|
+
**Config resilience:** Remove `.strict()` from `mercuryFileSchema` and all nested objects (Zod default behavior strips unknown keys silently). Add a post-parse diff that logs warnings for any stripped keys so operators notice typos without crashing.
|
|
51
|
+
|
|
52
|
+
**PM2 restart limit:** Configure `max_restarts` and `min_uptime` in the PM2 ecosystem config so a crash loop stops after N attempts instead of running forever. Send an alert when the limit is hit.
|
|
53
|
+
|
|
54
|
+
### Phase 2 — Medium term
|
|
55
|
+
|
|
56
|
+
**Circuit breaker for container errors:** Container failures (model API errors, malformed responses) should not crash the host process. The host should log the error, reply to the user with a friendly message, and continue.
|
|
57
|
+
|
|
58
|
+
### Phase 3 — Long term (WABA)
|
|
59
|
+
|
|
60
|
+
With WABA, auth is a permanent API token — crash loops no longer revoke auth. At that point, strict validation on values remains correct (fail fast on truly invalid config), while unknown keys should still warn-not-crash regardless of the WhatsApp transport.
|
|
61
|
+
|
|
62
|
+
## Notes
|
|
63
|
+
- Industry standard (Kubernetes, OpenCode/SST) is to warn on unknown keys, not crash. Zod's default behavior is `.strip()` (silently remove unknown keys) — Mercury explicitly opted into `.strict()` which is the most aggressive mode.
|
|
64
|
+
- OpenCode hit the same bug and fixed it by switching to passthrough + warnings (github.com/sst/opencode/issues/6145).
|
|
65
|
+
- The Baileys session loss is a known risk of the unofficial protocol — Meta actively discourages bot usage on consumer WhatsApp. WABA eliminates this class of risk entirely.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## Post-Mortem
|
|
70
|
+
|
|
71
|
+
### Investigation
|
|
72
|
+
Read `src/config-file.ts` — `mercuryFileSchema` and all 13 nested Zod object schemas used `.strict()`. The `mergeRawMercuryConfig()` function called `safeParse()` correctly but threw unconditionally on any parse failure, including unknown keys. The Zod `.strict()` mode treats unknown keys the same as type errors — both produce parse failures.
|
|
73
|
+
|
|
74
|
+
### Root Cause
|
|
75
|
+
`.strict()` on Zod object schemas rejects unknown keys with the same error severity as invalid values. There was no distinction between "unrecognized key" (safe to ignore) and "invalid value" (can't run). A renamed field (`admin_numbers` → `admin_ids`) caused Mercury to crash on startup, PM2's unlimited restarts amplified this into a crash loop, and the rapid reconnection pattern caused WhatsApp/Baileys to revoke the session.
|
|
76
|
+
|
|
77
|
+
### Fix
|
|
78
|
+
- **`src/config-file.ts`**: Replaced all 13 `.strict()` calls with `.strip()` (Zod default — silently removes unknown keys). Added `KNOWN_TOP_KEYS` and `KNOWN_SECTION_KEYS` static key sets, and a `warnUnknownKeys()` function that runs before parsing to log `[WARN]` messages for any unrecognized keys. Invalid values (wrong types, out-of-range numbers) still throw — only unknown keys are tolerated.
|
|
79
|
+
- **`mergeRawMercuryConfig()`**: Added an optional `log` parameter (defaults to `console.warn`) so the warning function is testable. Called `warnUnknownKeys()` before `safeParse()`.
|
|
80
|
+
- **`tests/config.test.ts`**: Added 4 tests: unknown top-level key warns but doesn't crash, unknown nested key warns, invalid value type still throws, valid config produces no warnings.
|
|
81
|
+
|
|
82
|
+
### Lessons
|
|
83
|
+
- Separate "unknown key" from "invalid value" in config validation — they have different severity and different correct responses.
|
|
84
|
+
- Any feature that adds config fields must consider version mismatch — a user on version N with a yaml from version N+1 should not crash.
|
|
85
|
+
- On Baileys, any crash loop is catastrophic — PM2 restart limits should be configured.
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: business-extensions
|
|
3
|
+
description: Dedicated business extensions that wrap raw capabilities (GWS, etc.) with deterministic domain logic — e.g. a barber-appointments extension
|
|
4
|
+
metadata:
|
|
5
|
+
type: idea
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Business Extensions
|
|
9
|
+
|
|
10
|
+
**Status**: Idea
|
|
11
|
+
**Created**: 2026-07-01
|
|
12
|
+
|
|
13
|
+
## Problem
|
|
14
|
+
|
|
15
|
+
Raw capability extensions (GWS Calendar, Gmail, web search) expose generic APIs. Business workflows (appointment booking, customer management) require deterministic logic — availability checks, double-booking prevention, confirmation flows — that can't be reliably delegated to the LLM via prompting alone.
|
|
16
|
+
|
|
17
|
+
Today the only options are: (1) let the LLM orchestrate raw APIs via system prompt instructions (not deterministic enough), or (2) build a full standalone application outside Mercury. Neither is ideal.
|
|
18
|
+
|
|
19
|
+
## Idea
|
|
20
|
+
|
|
21
|
+
A pattern for **business extensions** that wrap raw capabilities with domain-specific, deterministic logic and expose simplified CLIs to the agent.
|
|
22
|
+
|
|
23
|
+
Example: `barber-appointments` extension
|
|
24
|
+
- Wraps GWS Calendar internally
|
|
25
|
+
- Exposes: `book-appointment`, `list-availability`, `cancel-appointment`
|
|
26
|
+
- Enforces business rules in code: no double-booking, business hours only, minimum slot duration, confirmation required
|
|
27
|
+
- Registers as a single Mercury permission (`barber-appointments`)
|
|
28
|
+
- The LLM calls the simplified CLI; the extension handles the hard logic deterministically
|
|
29
|
+
|
|
30
|
+
This separates concerns:
|
|
31
|
+
- **Mercury** = platform (messaging, spaces, permissions, rate limits)
|
|
32
|
+
- **Capability extensions** = raw API access (GWS, web search, etc.) — admin-only or building blocks
|
|
33
|
+
- **Business extensions** = deterministic domain logic wrapping capabilities — customer-facing
|
|
34
|
+
|
|
35
|
+
## Why not just prompt engineering?
|
|
36
|
+
|
|
37
|
+
LLMs are non-deterministic. For a barber shop, a double-booked appointment or a cancelled-without-confirmation is a real-world problem. Business rules must be enforced in code, not hoped for via prompts.
|
|
38
|
+
|
|
39
|
+
## Open questions
|
|
40
|
+
|
|
41
|
+
- Should business extensions be a separate category in the extension system, or just a convention?
|
|
42
|
+
- How do business extensions declare their dependency on capability extensions (e.g., `barber-appointments` needs `gws`)?
|
|
43
|
+
- Should there be a template/scaffold for creating business extensions?
|
|
44
|
+
- How does this relate to "profiles" — is a profile = business extension + config preset?
|
|
45
|
+
|
|
46
|
+
## First candidate
|
|
47
|
+
|
|
48
|
+
`barber-appointments` — Calendar-based appointment booking for a barber shop. Quick win for first customer impression. Gmail integration deferred.
|
|
@@ -0,0 +1,211 @@
|
|
|
1
|
+
# GWS OAuth Setup Runbook
|
|
2
|
+
|
|
3
|
+
How to connect Google Calendar to a customer's Mercury assistant.
|
|
4
|
+
|
|
5
|
+
## Prerequisites
|
|
6
|
+
|
|
7
|
+
- Access to the Tagula GCP project (console.cloud.google.com, project "Tagula")
|
|
8
|
+
- SSH access to the customer's VPS
|
|
9
|
+
- The customer's `MERCURY_API_SECRET` (from their `.env`)
|
|
10
|
+
- The customer's Google account email
|
|
11
|
+
|
|
12
|
+
## Option A: Auth Gateway (preferred — works on mobile + desktop)
|
|
13
|
+
|
|
14
|
+
Use this when `auth.tagula.ai` is set up. Customer just clicks a link.
|
|
15
|
+
|
|
16
|
+
### One-time gateway setup
|
|
17
|
+
|
|
18
|
+
> Skip if the gateway is already running at `auth.tagula.ai`.
|
|
19
|
+
|
|
20
|
+
1. **DNS**: Add an A record for `auth.tagula.ai` → gateway server IP (GoDaddy DNS panel)
|
|
21
|
+
|
|
22
|
+
2. **Server**: SSH into the gateway server and install nginx + certbot:
|
|
23
|
+
```bash
|
|
24
|
+
apt-get update && apt-get install -y nginx certbot python3-certbot-nginx
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
3. **Nginx config** (`/etc/nginx/sites-available/auth-tagula`):
|
|
28
|
+
```nginx
|
|
29
|
+
server {
|
|
30
|
+
listen 80;
|
|
31
|
+
server_name auth.tagula.ai;
|
|
32
|
+
location / {
|
|
33
|
+
proxy_pass http://127.0.0.1:3456;
|
|
34
|
+
proxy_set_header Host $host;
|
|
35
|
+
proxy_set_header X-Forwarded-For $remote_addr;
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
```
|
|
39
|
+
```bash
|
|
40
|
+
ln -s /etc/nginx/sites-available/auth-tagula /etc/nginx/sites-enabled/
|
|
41
|
+
nginx -t && systemctl reload nginx
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
4. **SSL**:
|
|
45
|
+
```bash
|
|
46
|
+
certbot --nginx -d auth.tagula.ai
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
5. **Gateway config**: Copy `gateway-config.example.json` → `gateway-config.json`, fill in:
|
|
50
|
+
- `google.clientId` and `google.clientSecret` from the Tagula GCP project
|
|
51
|
+
- `google.redirectUri` = `https://auth.tagula.ai/oauth/callback`
|
|
52
|
+
|
|
53
|
+
6. **GCP redirect URI**: In GCP Console → APIs & Services → Credentials → Web client → add `https://auth.tagula.ai/oauth/callback` to Authorized redirect URIs
|
|
54
|
+
|
|
55
|
+
7. **Start gateway**:
|
|
56
|
+
```bash
|
|
57
|
+
npm install -g pm2 # if not installed
|
|
58
|
+
pm2 start auth-gateway.js --name auth-gateway
|
|
59
|
+
pm2 save
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Per-customer setup
|
|
63
|
+
|
|
64
|
+
1. **Add customer to gateway config** (`gateway-config.json`):
|
|
65
|
+
```json
|
|
66
|
+
"assistants": {
|
|
67
|
+
"customer-name": {
|
|
68
|
+
"apiUrl": "http://<VPS_IP>:8787",
|
|
69
|
+
"apiSecret": "<MERCURY_API_SECRET from customer's .env>",
|
|
70
|
+
"envVar": "MERCURY_GWS_CREDENTIALS_JSON"
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
```
|
|
74
|
+
Restart gateway: `pm2 restart auth-gateway`
|
|
75
|
+
|
|
76
|
+
2. **Add test user in GCP**: GCP Console → Google Auth Platform → Audience → Add users → add customer's Gmail
|
|
77
|
+
|
|
78
|
+
3. **Install gws extension on customer VPS** (if not already):
|
|
79
|
+
```bash
|
|
80
|
+
ssh root@<VPS_IP>
|
|
81
|
+
# Via dashboard: go to http://<VPS_IP>:8787/dashboard/page/features → Install gws
|
|
82
|
+
# Or via API:
|
|
83
|
+
curl -X POST http://localhost:8787/api/console/extensions/install \
|
|
84
|
+
-H "Authorization: Bearer <API_SECRET>" \
|
|
85
|
+
-H "Content-Type: application/json" \
|
|
86
|
+
-d '{"catalogName": "gws"}'
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
4. **Send the link to customer**:
|
|
90
|
+
```
|
|
91
|
+
https://auth.tagula.ai/connect/gws?assistant=customer-name
|
|
92
|
+
```
|
|
93
|
+
Customer clicks → signs in with Google → sees "Connected!" → done.
|
|
94
|
+
|
|
95
|
+
5. **Verify**: Check Mercury logs on the VPS:
|
|
96
|
+
```bash
|
|
97
|
+
pm2 logs 'mercury run' --lines 20 --nostream | grep -i gws
|
|
98
|
+
```
|
|
99
|
+
Should show: `Loaded extension: gws` and `Installed skill: gws`
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Option B: Manual (ngrok — no gateway needed)
|
|
104
|
+
|
|
105
|
+
Use this before the gateway is set up, or as a one-off.
|
|
106
|
+
|
|
107
|
+
### Setup
|
|
108
|
+
|
|
109
|
+
1. **Install ngrok on customer VPS**:
|
|
110
|
+
```bash
|
|
111
|
+
ssh root@<VPS_IP>
|
|
112
|
+
curl -sSL https://ngrok-agent.s3.amazonaws.com/ngrok.asc \
|
|
113
|
+
| tee /etc/apt/trusted.gpg.d/ngrok.asc >/dev/null
|
|
114
|
+
echo 'deb https://ngrok-agent.s3.amazonaws.com buster main' \
|
|
115
|
+
| tee /etc/apt/sources.list.d/ngrok.list
|
|
116
|
+
apt-get update && apt-get install -y ngrok
|
|
117
|
+
ngrok config add-authtoken <YOUR_NGROK_TOKEN>
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
2. **Write callback script** (`/tmp/oauth-callback.js`):
|
|
121
|
+
```bash
|
|
122
|
+
# Copy from tagula-agents-cloud-console/oauth-callback-standalone.js
|
|
123
|
+
# Update CLIENT_ID and CLIENT_SECRET from GCP
|
|
124
|
+
scp oauth-callback-standalone.js root@<VPS_IP>:/tmp/oauth-callback.js
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
3. **Start callback + ngrok**:
|
|
128
|
+
```bash
|
|
129
|
+
ssh root@<VPS_IP>
|
|
130
|
+
nohup node /tmp/oauth-callback.js > /tmp/oauth-callback.log 2>&1 &
|
|
131
|
+
ngrok http 9999
|
|
132
|
+
```
|
|
133
|
+
Note the ngrok URL (e.g., `https://abc123.ngrok-free.dev`)
|
|
134
|
+
|
|
135
|
+
4. **Configure redirect URI**:
|
|
136
|
+
- Set redirect URI in the callback script:
|
|
137
|
+
```bash
|
|
138
|
+
curl "http://localhost:9999/set-redirect?uri=https://<NGROK_URL>/oauth/callback"
|
|
139
|
+
```
|
|
140
|
+
- Add same URL to GCP: APIs & Services → Credentials → Web client → Authorized redirect URIs
|
|
141
|
+
|
|
142
|
+
5. **Add test user**: GCP → Google Auth Platform → Audience → Add customer's Gmail
|
|
143
|
+
|
|
144
|
+
6. **Send OAuth link to customer**:
|
|
145
|
+
```
|
|
146
|
+
https://accounts.google.com/o/oauth2/v2/auth?client_id=<CLIENT_ID>&redirect_uri=https://<NGROK_URL>/oauth/callback&response_type=code&scope=https://www.googleapis.com/auth/calendar.events&access_type=offline&prompt=consent
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
7. **After customer authorizes**:
|
|
150
|
+
- ngrok free tier shows an interstitial — customer clicks "Visit Site"
|
|
151
|
+
- Check logs: `cat /tmp/oauth-callback.log`
|
|
152
|
+
- Should show `SUCCESS - credentials saved`
|
|
153
|
+
|
|
154
|
+
8. **Move credentials to correct .env** (the callback script writes to `.mercury/.env` but Mercury reads from project root `.env`):
|
|
155
|
+
```bash
|
|
156
|
+
grep MERCURY_GWS /opt/path/to/assistant/.mercury/.env >> /opt/path/to/assistant/.env
|
|
157
|
+
sed -i '/MERCURY_GWS/d' /opt/path/to/assistant/.mercury/.env
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
9. **Restart Mercury and clean up**:
|
|
161
|
+
```bash
|
|
162
|
+
pm2 restart 'mercury run'
|
|
163
|
+
# Kill callback server and ngrok
|
|
164
|
+
kill $(lsof -ti:9999) 2>/dev/null
|
|
165
|
+
pkill ngrok
|
|
166
|
+
rm /tmp/oauth-callback.js /tmp/oauth-callback.log
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
10. **Verify**:
|
|
170
|
+
```bash
|
|
171
|
+
pm2 logs 'mercury run' --lines 20 --nostream | grep -i gws
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
## Option C: Copy-paste URL (desktop only)
|
|
177
|
+
|
|
178
|
+
Simplest, no infrastructure needed. Customer must be on a computer (not mobile).
|
|
179
|
+
|
|
180
|
+
1. **Create Desktop-type OAuth client** in GCP (not Web) — allows `http://localhost` redirect
|
|
181
|
+
2. **Generate OAuth URL** with `redirect_uri=http://localhost:1`
|
|
182
|
+
3. **Send link to customer** — they click, authorize, Google redirects to localhost (fails)
|
|
183
|
+
4. **Customer copies the full URL** from their browser bar and sends it back
|
|
184
|
+
5. **Extract code and exchange**:
|
|
185
|
+
```bash
|
|
186
|
+
# Extract code from the URL the customer sent
|
|
187
|
+
CODE="the_code_from_url"
|
|
188
|
+
curl -s -X POST https://oauth2.googleapis.com/token \
|
|
189
|
+
-d "code=$CODE&client_id=<CLIENT_ID>&client_secret=<CLIENT_SECRET>&redirect_uri=http://localhost:1&grant_type=authorization_code"
|
|
190
|
+
```
|
|
191
|
+
6. **Set env var and restart** — copy the tokens JSON into `.env` as `MERCURY_GWS_CREDENTIALS_JSON='...'`
|
|
192
|
+
|
|
193
|
+
---
|
|
194
|
+
|
|
195
|
+
## Troubleshooting
|
|
196
|
+
|
|
197
|
+
| Symptom | Cause | Fix |
|
|
198
|
+
|---------|-------|-----|
|
|
199
|
+
| `invalid_client` on Google consent | Client not propagated yet, or created in new Auth Platform UI | Wait 10 min, or recreate from legacy APIs & Services → Credentials |
|
|
200
|
+
| Extension installs but doesn't load | `MERCURY_GWS_CREDENTIALS_JSON` not set or in wrong `.env` | Check `grep MERCURY_GWS .env` in project root |
|
|
201
|
+
| `Removed stale skill: gws` in logs | Extension skipped by credential gate | Same as above — credential missing |
|
|
202
|
+
| `Failed to build derived image` | `docker-buildx` not installed | `apt-get install docker-buildx` |
|
|
203
|
+
| Customer's wrong Google account used | Browser auto-selects default account | Add all their accounts as test users, or use incognito |
|
|
204
|
+
| ngrok "Visit Site" interstitial | Free tier warning | Customer clicks "Visit Site" — shows once |
|
|
205
|
+
| `refresh_token_expires_in: 604799` | Token expires in 7 days (testing mode) | Publish the OAuth app in GCP to get non-expiring refresh tokens |
|
|
206
|
+
|
|
207
|
+
## Important notes
|
|
208
|
+
|
|
209
|
+
- **Test mode tokens expire in 7 days.** While the GCP app is in "Testing" status, refresh tokens expire after 7 days. To get permanent tokens, publish the app (requires Google verification for sensitive scopes). For now, re-run the OAuth flow when tokens expire.
|
|
210
|
+
- **docker-buildx** is required on Linux servers for Mercury to build derived images with extension CLIs. Install with `apt-get install docker-buildx`.
|
|
211
|
+
- **Rotate credentials** after sharing them in chat. Go to GCP → Credentials → Web client → rotate client secret. Update `gateway-config.json` and any callback scripts.
|