mercury-agent 0.18.2 → 0.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -205,17 +205,55 @@
205
205
  the owner and a second WhatsApp account, with the snapshot/restore
206
206
  procedure of §5. The bench does not try to make this list empty.
207
207
  - **Alternatives considered:** a Baileys stub; a second bot number in
208
- shadow mode.
208
+ shadow mode; **a test bot** — a snapshot restored on another host
209
+ (M1.6), the candidate Mercury version installed, linked to WhatsApp with
210
+ its *own* number, and driven from the owner's phone (owner's question,
211
+ 2026-09-06).
209
212
  - **Reasoning:** the memory `mercury-member-role-cannot-be-simulated` and
210
213
  the F5 debug doc both record the same wall: `isSelfJid` reads a live socket.
211
214
  D-003 moves the *decisions* out from behind that wall; the socket itself is
212
215
  not our code and is not worth stubbing. A private lab group with a second
213
216
  account is cheap and is what the football space already is until the real
214
217
  group is linked.
218
+
219
+ The test bot is **complementary, not a replacement**, and it is not safe
220
+ as described. It exercises what the shadow cannot — the socket list
221
+ above — but a snapshot linked to a different number breaks on identity:
222
+ every space is reached through `conversations.external_id`, the live
223
+ chat JID (`src/storage/db.ts`, `conversations`), so the copied groups
224
+ belong to a bot that is not a member of them and the copied `main` space
225
+ points at the owner's real number. Three consequences, each of which
226
+ the shadow avoids by having no adapter at all: (1) a copied scheduled
227
+ task whose space resolves to the owner's DM **sends from the test number
228
+ to the owner's real phone**, and one bound to a group the test bot has
229
+ been added to reaches the group; (2) the test bot cannot be added to the
230
+ real groups without reaching real people, so it needs its own groups,
231
+ whose JIDs differ, so the copied roles, tasks, config and workspace no
232
+ longer apply until the snapshot's space→chat links are remapped; (3) D-002
233
+ makes "adapter on" mean "live", so every refusal of D-005 (broker,
234
+ publish, billing, deliver) is off — a copied task with a broker action is
235
+ real. What a test bot cannot do at all: act *as* a specific existing
236
+ member (a non-admin quoting a bot message is the F5 shape), which
237
+ `shadow say --as <member>` replays from the copied `space_roles`. What it
238
+ does better: everything in §5, plus a candidate version on a real socket
239
+ before `promote` (D-013). Where it is redundant: a migration rehearsal
240
+ (apply → diff) needs no socket and is the shadow's own job (M1.4, M3.2).
215
241
  - **Revisit if:** the owner does not want to hold a second number — then the
216
242
  §5 list becomes the standing content of `pending-verification.md` and the
217
- bench's coverage is everything else.
218
- - **Used by:** lab-space-procedure
243
+ bench's coverage is everything else. **Or** the owner wants the test bot —
244
+ then it is M4.2 and lands only with all three preconditions, each denying
245
+ on absence: a new opt-in mode value (never "`MERCURY_SHADOW` unset plus a
246
+ list" — absence is live, D-002) that keeps every D-005 refusal while
247
+ constructing the adapter; a **recipient allowlist** of literal chat JIDs
248
+ on the send side (none exists today — `direct-send.ts` resolves a
249
+ recipient, it does not restrict one), checked at the bridge so all
250
+ fifteen send sites of `docs/live-testing.md` §3 are behind it, empty
251
+ list = nothing leaves; and a `shadow snapshot --remap <live-jid>=<test-jid>`
252
+ that rewrites `conversations.external_id` (and `message_platform_ids`)
253
+ so the copied spaces point at the test groups, refusing any live JID left
254
+ unmapped. Tasks whose space maps to nothing are deactivated in the copy,
255
+ not left to fail silently.
256
+ - **Used by:** lab-space-procedure; test-bot-instance (M4.2, if chosen)
219
257
 
220
258
  ## D-008: No injection route on the live instance in this goal
221
259
  - **Category:** scope
@@ -1,7 +1,7 @@
1
1
  # Roadmap: Rehearsal Bench
2
2
 
3
3
  **Goal**: [rehearsal-bench](goal.md)
4
- **Last updated**: 2026-09-03 (re-scope after the VPS cutover review, [note](../../notes/2026-09-03-release-gate-and-test-environment.md):
4
+ **Last updated**: 2026-09-06 (M4.2 `test-bot-instance` added as an owner's-decision story after the "why not a test bot on a snapshot?" review — D-007 extended with the identity/egress/remap analysis; nothing else moved). Earlier: 2026-09-03 (re-scope after the VPS cutover review, [note](../../notes/2026-09-03-release-gate-and-test-environment.md):
5
5
  M1.3 gains the instance-label prerequisite and the fleet-box shape; M1.6
6
6
  `snapshot-across-hosts` added; M3.2 extended to tagula's space-sync path;
7
7
  M3.3 `sync-rehearse` **moved** to [`release-gate`](../release-gate/roadmap.md)
@@ -99,6 +99,7 @@ in `pending-verification.md` for that fix is replaced by the transcript.
99
99
  | ID | Story | Slug | Depends on | Status |
100
100
  |----|-------|------|------------|--------|
101
101
  | M4.1 | `lab` space: private group of bot + owner + second account; `mercury lab snapshot|restore` = `VACUUM INTO` + tar before, `messages`/`space_roles`/`token_usage` window delete + workspace restore after; `restore` proves itself by re-running `shadow diff` against the pre-snapshot and requiring an empty report | lab-space-procedure | M1.4 | backlog (needs a second WhatsApp number — owner's decision) |
102
+ | M4.2 | **Test-bot instance** (added 2026-09-06, D-007 revisit): a snapshot restored on another host (M1.6) and linked to WhatsApp with its own number, driven from the owner's phone — the socket-only list of §5 on a candidate version, before `promote`. Complementary to the shadow, never a replacement (it cannot act *as* an existing member; a migration rehearsal needs no socket). Lands only with three guards, each denying on absence: a new opt-in mode that constructs the adapter but keeps every D-005 refusal (absence is still live, D-002); a send-side **recipient allowlist** of literal chat JIDs checked at the bridge behind all §3 send sites, empty = nothing leaves; `shadow snapshot --remap <live-jid>=<test-jid>` rewriting `conversations.external_id` + `message_platform_ids`, refusing an unmapped live JID and deactivating tasks whose space maps to nothing. Without the remap a copied task delivers from the test number to the owner's real DM | test-bot-instance | M1.6, M1.4 | backlog (needs a second bot number and a host — owner's decision) |
102
103
 
103
104
  **Checkpoint:** the §5 list of `docs/live-testing.md` is walked once in the
104
105
  lab space and every item is either ticked with the date or moved into M2 as
@@ -153,6 +154,7 @@ M1.1 ─ M1.5 │
153
154
  M2.1 ──────┬─ M2.2 ─┴─ M2.3 ─ M2.4
154
155
  M1.3 ──────┘ └────────────────────► release-gate R3 (smoke runner)
155
156
  M1.4 ─ M4.1
157
+ M1.6 ─ M4.2 (owner's decision)
156
158
  release-gate R5 (sync-rehearse, ex-M3.3) ◄─ M3.1, M2.3
157
159
  ```
158
160
 
@@ -110,3 +110,43 @@
110
110
  - **Revisit if:** a second operator (not tagula) needs the same policy —
111
111
  then it is a Mercury feature, planned as one.
112
112
  - **Used by:** smoke-runner, release-gate-rules
113
+
114
+ ## G-007: Whether Mercury publishes its own agent image — OPEN, owner's call
115
+ - **Category:** architecture / distribution
116
+ - **Status:** **Open.** R0.3 is blocked on it; R0.1, R0.2 and R0.5 are not.
117
+ - **Question:** Mercury's release workflow publishes the npm package and
118
+ nothing else. The agent image every fleet box runs is built and pushed by
119
+ `agent-image.yml` in **tagula-agents**, from Mercury's published npm
120
+ tarball. Mercury's own `ghcr.io/avishai-tsabari/mercury-agent` is pushed
121
+ by hand (`docs/runbooks/publish-checklist.md`, "manual — CI does not do
122
+ this"), is stale, and returned 401/403 unauthenticated on 2026-09-05.
123
+ Should Mercury publish its own image from `release.yml`, and public or
124
+ private?
125
+ - **Evidence (2026-09-05):** `container/Dockerfile` has zero `LABEL` lines;
126
+ `container/build.sh:20` tags `:latest` only; `release.yml` has no
127
+ `packages: write`; `src/config.ts:59` defaults to
128
+ `ghcr.io/avishai-tsabari/mercury-agent:latest`;
129
+ `container-runner.ts:1131-1136` never refreshes a tag that exists locally;
130
+ `container-runner.ts:1104-1117` skips validation for exactly the
131
+ `/mercury-agent:` names that broke. A fresh outside install therefore
132
+ fails `ensureImage()` and must fall back to a ~2.8 GB local `mercury build`
133
+ the README's install steps never mention. The tagula box could not perform
134
+ that fallback: 2 vCPU, 3 GB RAM and 1.9 GB free before the 2026-09-05
135
+ cleanup.
136
+ - **The two answers, and what each commits to:**
137
+ - **Public.** One publishing path, third-party installs work, R0.4's
138
+ version pin becomes safe, `agent-image.yml`'s version-linking half
139
+ retires. Commits to publishing a built artifact of a private repo —
140
+ the same reason the fleet image is private — so it is an open-core
141
+ boundary decision, not a CI one.
142
+ - **Private (or no change).** Nothing outside the fleet can install
143
+ Mercury. Then say so: `docs/container-lifecycle.md:225` currently
144
+ claims "Images are published on each release", which is false today
145
+ either way, and the README's install steps imply an independent
146
+ install that does not exist.
147
+ - **Not in scope of this decision:** the fleet's idle check, database
148
+ snapshot, rollback and stagger. Those are topology and stay in
149
+ tagula-agents whatever is decided here (G-006).
150
+ - **Revisit if:** a second operator appears, or Mercury is open-sourced.
151
+ - **Used by:** mercury-publishes-image (R0.3), image-refresh-and-visibility (R0.4)
152
+
@@ -1,7 +1,7 @@
1
1
  # Roadmap: Release Gate
2
2
 
3
3
  **Goal**: [release-gate](goal.md)
4
- **Last updated**: 2026-09-03 (created from the VPS cutover review; story slugs are `/f-feature-planning` inputs)
4
+ **Last updated**: 2026-09-06 (R0 added: the image supply side R1.1(c) reads from, and nobody owned)
5
5
 
6
6
  > Order is by leverage against the live risk, not by size: R1 protects the
7
7
  > customer box within a session or two and depends on nothing; R2 is the
@@ -11,6 +11,33 @@
11
11
 
12
12
  ## Milestones
13
13
 
14
+ ### Milestone R0: The agent image is a published, self-describing artifact
15
+
16
+ > R1.1(c) asks the image what mercury version it is and which `COPY`
17
+ > sources it carries. **Nothing puts either there today**, and nothing in
18
+ > this repo publishes the image at all, so R1.1(c) has no artifact to read
19
+ > and no milestone owned the gap. This is that milestone. It is small,
20
+ > it depends on nothing, and R1.1(c) is blocked without R0.1.
21
+ >
22
+ > Established 2026-09-05 while releasing 0.18.1/0.18.2 (evidence in
23
+ > [`decisions.md` G-007](decisions.md)):
24
+ > `container/Dockerfile` has **zero** `LABEL` lines; `container/build.sh`
25
+ > tags `:latest` only, never a version; `release.yml` has no image job and
26
+ > no `packages: write` anywhere; the image every fleet box actually runs is
27
+ > built and pushed by `agent-image.yml` in **tagula-agents**, from
28
+ > Mercury's published npm tarball. Mercury's own
29
+ > `ghcr.io/avishai-tsabari/mercury-agent` is pushed by hand per
30
+ > `docs/runbooks/publish-checklist.md` ("CI does not do this"), is stale,
31
+ > and is **not anonymously pullable** (401/403 verified 2026-09-05).
32
+
33
+ | ID | Story | Slug | Depends on | Status |
34
+ |----|-------|------|------------|--------|
35
+ | R0.1 | **The image describes itself.** `container/Dockerfile` + `container/build.sh` stamp, at build time, (a) `org.opencontainers.image.version` and a mercury-namespaced version label taken from `package.json`, not hand-typed, and (b) the **`COPY`-source manifest** R1.1(c) checks against — generated from the Dockerfile's own `COPY` list so the two cannot drift. `build.sh` tags `:<version>` as well as `:latest`. Labels are inherited by the derived `mercury-agent-ext-*` image with no builder change (verified on the tagula box 2026-09-05: derived and base both report `0.18.0-beta.0`), so the host-side check works through the image actually executing | image-self-describing | — | backlog — **blocks R1.1(c)** |
36
+ | R0.2 | **The build-integrity gate moves into this repo.** `agent-image.yml` in tagula-agents verifies Mercury's artifact — every `COPY` source present in the tarball, `bun build container-entry.ts` resolves inside the image — and that check is what would have caught 0.17.0. It belongs in `release.yml` (or `bun run check`) so every consumer building the image gets it, not only this one fleet. The fleet keeps its copy or drops it; either is fine once this exists | build-integrity-in-mercury | — | backlog |
37
+ | R0.3 | **Mercury publishes its own image.** A job in `release.yml`, after `publish-npm`, that installs the published tarball, builds, verifies (R0.2) and pushes `:<version>` + `:latest` with the R0.1 labels. Needs `packages: write`, which no job has today. **Gated on the G-007 decision** (public or private): the answer changes whether this closes the third-party install path or only tidies the internal one. Makes `agent-image.yml`'s *version-linking* half redundant; its nightly updater, idle check, snapshot, rollback and stagger stay where they are | mercury-publishes-image | R0.1, R0.2, G-007 | backlog — **decision first** |
38
+ | R0.4 | **A floating tag stops meaning "frozen at first pull".** `ensureImage()` returns the moment `docker image inspect` succeeds, so the default `:latest` is never refreshed and a host upgraded by npm keeps yesterday's container silently — the trap the owner hit on 2026-09-05. Either pin the default to `:${pkg.version}` (safe **only** after R0.3, or every fresh install fails on a tag nobody published) or re-pull a floating tag on a version change. Either way boot logs the resolved image ref, digest and version label so drift is visible without asking | image-refresh-and-visibility | R0.3 | backlog |
39
+ | R0.5 | **`mercury upgrade` owns both halves.** `upgradeAction` contains no Docker call at all: it stops the service, npm-installs and restarts, leaving the image behind. It should pull the matching image first and refuse when that tag does not exist — image first, host second, the rule the fleet updater already follows and the one 0.17.0 broke | upgrade-pulls-image | R0.1 | backlog |
40
+
14
41
  ### Milestone R1: Preflight
15
42
  > After this, a box that cannot run a container says so at boot and on
16
43
  > demand, with one diagnosis per blocker, and the nightly updater has
@@ -94,6 +121,10 @@ manual check in `pending-verification.md` is replaced by the reports.
94
121
  ## Dependency Graph
95
122
 
96
123
  ```
124
+ R0.1 ─┬─ R1.1(c)
125
+ ├─ R0.5
126
+ └─ R0.3 ◄─ G-007 (decision) ─ R0.4
127
+ R0.2 ─── R0.3
97
128
  R1.1 ─┬─ R1.2
98
129
  ├─ R2.1 ─ R2.2
99
130
  ├─ R4.1 ─ R5.2
@@ -103,7 +134,9 @@ R3.1 ─ R3.2 ─┬─ R3.3
103
134
  rehearsal-bench M2.2 (function half) ─► R3.2
104
135
  ```
105
136
 
106
- R1.1 first, alone. R2 and R4 can run beside R3.1 in separate worktrees.
137
+ R0.1 is the true first story: it is small, depends on nothing, and R1.1(c)
138
+ cannot be built without it. R0.3 waits on a decision, not on code.
139
+ R1.1 otherwise first, alone. R2 and R4 can run beside R3.1 in separate worktrees.
107
140
  R3.2 is the join with the bench. R5 last.
108
141
 
109
142
  ## Cost note
@@ -15,6 +15,8 @@ type ConnectionAuthType =
15
15
  | "credentials-file"
16
16
  | "custom";
17
17
 
18
+ type ConnectionStatus = "connected" | "needs-reauth" | "broken" | "unknown";
19
+
18
20
  type MercuryExt = {
19
21
  cli(opts: { name: string; install: string }): void;
20
22
  permission(opts: { defaultRoles: string[] }): void;
@@ -27,6 +29,7 @@ type MercuryExt = {
27
29
  authType: ConnectionAuthType;
28
30
  credentialEnvVar?: string;
29
31
  scopes?: string[];
32
+ statusCheck?: () => Promise<{ status: ConnectionStatus; detail?: string }>;
30
33
  }): void;
31
34
  on(
32
35
  event: "before_container",
@@ -55,6 +58,59 @@ const gwsEnv = {
55
58
  /** Path where credentials are materialized inside the inner container's own /tmp. */
56
59
  const CREDENTIALS_FILE = "/tmp/gws-credentials.json";
57
60
 
61
+ /**
62
+ * The string fields google-auth-library needs on an ADC "authorized user"
63
+ * document, beyond `type`. `type` is checked separately so its absence can be
64
+ * named on its own — it is the field operators actually hit.
65
+ */
66
+ const REQUIRED_CREDENTIAL_FIELDS = [
67
+ "client_id",
68
+ "client_secret",
69
+ "refresh_token",
70
+ ] as const;
71
+
72
+ /**
73
+ * Describe what is wrong with a candidate credential, or `undefined` when it is
74
+ * a usable ADC "authorized user" document.
75
+ *
76
+ * A parse check is not enough, which is what this guard exists to fix. Google's
77
+ * `/token` response is valid JSON carrying a working `refresh_token`, but has no
78
+ * top-level `type`, so it sails through `JSON.parse` and then fails on *every*
79
+ * `gws` invocation with "the file doesn't contain the field 'type'" — from
80
+ * inside a conversation, where no operator sees it. `{}` passed too. Read the
81
+ * guard by asking what happens when a field is missing, and make absence deny.
82
+ */
83
+ function credentialProblem(raw: string): string | undefined {
84
+ let parsed: unknown;
85
+ try {
86
+ parsed = JSON.parse(raw);
87
+ } catch {
88
+ return "is not valid JSON";
89
+ }
90
+ if (parsed === null || typeof parsed !== "object" || Array.isArray(parsed)) {
91
+ return "is not a JSON object";
92
+ }
93
+ const doc = parsed as Record<string, unknown>;
94
+ if (doc.type === undefined) {
95
+ return 'is missing the "type" field (expected "authorized_user"; note that "token_type" is a different field and does not satisfy it)';
96
+ }
97
+ if (doc.type !== "authorized_user") {
98
+ return `has type ${JSON.stringify(doc.type)} (expected "authorized_user")`;
99
+ }
100
+ const missing = REQUIRED_CREDENTIAL_FIELDS.filter(
101
+ (field) => typeof doc[field] !== "string" || doc[field] === "",
102
+ );
103
+ if (missing.length > 0) {
104
+ return `is missing or has empty ${missing.join(", ")}`;
105
+ }
106
+ return undefined;
107
+ }
108
+
109
+ /** Appended to every rejection so the log names the fix, not just the fault. */
110
+ const CREDENTIAL_RECOVERY =
111
+ 'Expected {"type":"authorized_user","client_id":...,"client_secret":...,"refresh_token":...}. ' +
112
+ "Regenerate with: gws auth login && gws auth export --unmasked";
113
+
58
114
  export default function (mercury: MercuryExt) {
59
115
  // The gws CLI can only take refresh-token credentials from a file, so someone
60
116
  // has to materialize GWS_CREDENTIALS_JSON before the first command runs. The
@@ -81,11 +137,11 @@ export default function (mercury: MercuryExt) {
81
137
  "'if [ -n \"$GWS_CREDENTIALS_JSON\" ]; then' " +
82
138
  `' c="\${GOOGLE_WORKSPACE_CLI_CREDENTIALS_FILE:-${CREDENTIALS_FILE}}"' ` +
83
139
  "' t=\"$c.$$\"' " +
84
- "' (umask 077; printf %s \"$GWS_CREDENTIALS_JSON\" > \"$t\")' " +
140
+ '\' (umask 077; printf %s "$GWS_CREDENTIALS_JSON" > "$t")\' ' +
85
141
  // Credentials changed (rotation, or a different caller in a reused
86
142
  // container) invalidates the cached access token alongside them.
87
- "' cmp -s \"$t\" \"$c\" 2>/dev/null || rm -f \"${GOOGLE_WORKSPACE_CLI_CONFIG_DIR:-$HOME/.config/gws}/token_cache.json\"' " +
88
- "' mv -f \"$t\" \"$c\"' " +
143
+ '\' cmp -s "$t" "$c" 2>/dev/null || rm -f "${GOOGLE_WORKSPACE_CLI_CONFIG_DIR:-$HOME/.config/gws}/token_cache.json"\' ' +
144
+ '\' mv -f "$t" "$c"\' ' +
89
145
  "' GOOGLE_WORKSPACE_CLI_CREDENTIALS_FILE=\"$c\"; export GOOGLE_WORKSPACE_CLI_CREDENTIALS_FILE' " +
90
146
  "'fi' " +
91
147
  "'exec /usr/local/bin/gws-real \"$@\"' " +
@@ -112,21 +168,56 @@ export default function (mercury: MercuryExt) {
112
168
  "https://www.googleapis.com/auth/spreadsheets",
113
169
  "https://www.googleapis.com/auth/userinfo.email",
114
170
  ],
171
+ // Without this, the connection reports "connected" on presence alone, so a
172
+ // credential the CLI cannot use still shows green on the dashboard and the
173
+ // only symptom is a failed tool call mid-conversation. Shape is checked
174
+ // here — validity (is the token still live?) is not, and still has no
175
+ // reader; see docs/bugs/gws-connection-credential-has-no-health-check.md.
176
+ statusCheck: async () => {
177
+ const raw = process.env[gwsEnv.env.credentials];
178
+ // Match the runtime's own presence default so an unconfigured connection
179
+ // reads the same with or without this probe.
180
+ if (!raw) {
181
+ return {
182
+ status: "unknown",
183
+ detail: `${gwsEnv.env.credentials} is not set`,
184
+ };
185
+ }
186
+ const problem = credentialProblem(raw);
187
+ if (problem) {
188
+ return {
189
+ status: "broken",
190
+ detail: `${gwsEnv.env.credentials} ${problem}`,
191
+ };
192
+ }
193
+ return { status: "connected" };
194
+ },
115
195
  });
116
196
 
117
197
  // Inner containers don't share the outer container's volume — they mount host
118
198
  // filesystem paths. So we can't write a credentials file from this hook and
119
199
  // have the inner container see it. Instead, pass the target path via env var
120
200
  // and let the skill materialize the file from GWS_CREDENTIALS_JSON at runtime.
121
- mercury.on("before_container", async () => {
201
+ mercury.on("before_container", async (event, ctx) => {
122
202
  const raw = process.env[gwsEnv.env.credentials];
123
203
  if (!raw) return undefined;
124
204
 
125
- try {
126
- JSON.parse(raw);
127
- } catch {
205
+ // GWS_CREDENTIALS_JSON is delivered by THIS hook, not by mercury.env(), and
206
+ // the two paths are not gated alike: a declared env var is injected only for
207
+ // a caller holding the extension's permission, while a hook's returned env
208
+ // is merged in `emitBeforeContainer` and pushed straight onto the container's
209
+ // `-e` list with no RBAC in between. So `mercury.permission()` above gates
210
+ // the *skill*, not this credential — without the check here the refresh
211
+ // token (gmail.modify, drive, calendar, documents, spreadsheets) reaches
212
+ // every caller in every space. The permission name is the extension name.
213
+ if (!ctx.hasCallerPermission(event.spaceId, event.callerId, "gws")) {
214
+ return undefined;
215
+ }
216
+
217
+ const problem = credentialProblem(raw);
218
+ if (problem) {
128
219
  console.error(
129
- `[gws.before_container] ${gwsEnv.env.credentials} is not valid JSON — skipping`,
220
+ `[gws.before_container] ${gwsEnv.env.credentials} ${problem} — skipping. ${CREDENTIAL_RECOVERY}`,
130
221
  );
131
222
  return undefined;
132
223
  }
@@ -41,22 +41,23 @@ mrctl config set morning.environment sandbox # or: production
41
41
 
42
42
  Only those two values are accepted. A typo is refused at set time.
43
43
 
44
- ## Two things that will look like bugs
44
+ ## Why this connection is not marked `sensitive`
45
45
 
46
- **1. In a WhatsApp group, every turn is blocked until you enable sensitive
47
- connections.** This connection is declared `sensitive: true` because it can
48
- issue legally binding tax documents. In a group-linked space Mercury then
49
- refuses assistant turns unless the space opts in, and asks each caller to
50
- confirm:
46
+ It issues legally binding tax documents, so `sensitive: true` looks right. It is
47
+ deliberately omitted, because the runtime's sensitive guard is host-global and
48
+ space-blind: one sensitive connection anywhere on the host puts *every*
49
+ group-linked space behind a per-message confirmation, and the confirmation is
50
+ consumed on each `yes`, so it repeats for every turn forever. A host with one
51
+ accounting space and thirteen unrelated ones would pay that cost fourteen times
52
+ over.
51
53
 
52
- ```bash
53
- mrctl config set security.sensitive_connections_allowed true
54
- ```
54
+ Nothing load-bearing is lost. The credentials are `hostOnly` and reached only
55
+ through the broker, so they never enter an agent container, and the verbs are
56
+ `admin`-only. Restore the flag once the guard scopes per-space or per-extension.
55
57
 
56
- This is intended, not a fault. In a one-to-one conversation the guard does not
57
- apply.
58
+ ## One thing that will look like a bug
58
59
 
59
- **2. Issuing requires a preview first.** `document-create` refuses without an
60
+ **Issuing requires a preview first.** `document-create` refuses without an
60
61
  `issueToken` returned by `document-preview` of the *same* document. The token is
61
62
  bound to the payload, the environment, the space and the caller, and works once.
62
63
  A retry after a successful create is refused rather than issuing twice.
@@ -93,7 +94,16 @@ mrctl capability morning <action> '<json>'
93
94
  ```
94
95
 
95
96
  `client-search`, `client-create`, `document-types`, `document-preview`,
96
- `document-create`, `document-search`, `document-links`.
97
+ `document-create`, `document-search`, `document-links`, `expense-upload`,
98
+ `expense-draft-search`.
99
+
100
+ **`expense-upload` takes a path, never bytes.** The container names a file
101
+ relative to its own space workspace (`inbox/receipt.pdf`); the host reads it and
102
+ confines the path to `<spacesDir>/<spaceId>/`, resolving symlinks *before* the
103
+ containment check — `inbox/` is written by a lower-trust boundary, so a symlink
104
+ there pointing at the host's `.env` is the attack the guard exists for. A
105
+ refused path returns one generic message and logs the detail host-side at WARN;
106
+ telling the container why would make the guard a filesystem oracle.
97
107
 
98
108
  Full request shapes and the issue flow are in `skill/SKILL.md`; enums and the
99
109
  240-row error table are in `skill/references/`.
@@ -102,8 +112,9 @@ Full request shapes and the issue flow are in `skill/SKILL.md`; enums and the
102
112
 
103
113
  | Area | Why |
104
114
  |---|---|
105
- | Expenses, receipt upload | `POST /expenses` requires an `accountingClassification`, and the "Get Accounting Classifications" endpoint its schema references is absent from the 53 published paths. Also, an uploaded file always becomes a *draft* and there is no API to approve a draft. |
106
- | Webhooks (`payment/received`, `document/created`) | Extensions cannot register HTTP routes. Needs a core route or a polling job. |
115
+ | Creating an expense from typed-in details (`POST /expenses`) | Requires an `accountingClassification`, and the "Get Accounting Classifications" endpoint its schema references is still absent from the published paths (re-verified 2026-09-06). Classifying inline would mean guessing a tax category on the user's behalf. Receipt **upload** sidesteps this entirely and is implemented — it creates a draft Morning parses itself, which a human approves in Morning's UI. |
116
+ | Editing, closing or deleting an expense | Reading and creating drafts is the chore; mutating existing expenses is a larger trust decision. |
117
+ | Webhooks (`payment/received`, `document/created`, `expense-draft/parsed`) | Extensions cannot register HTTP routes. This is why `expense-upload` cannot report whether a draft was ultimately parsed or declined — poll `expense-draft-search` instead. |
107
118
  | Payments API, credit-card tokens | Moving money is a larger trust decision than issuing a document. |
108
119
  | Partners API | For multi-business accountants; this is scoped to one business per key set. |
109
120
 
@@ -12,6 +12,7 @@
12
12
  * without an accounting-classification source.
13
13
  */
14
14
 
15
+ import { isAbsolute, join } from "node:path";
15
16
  import { runAction } from "./lib/morning.js";
16
17
  import { authErrorKey } from "./lib/token.js";
17
18
 
@@ -43,6 +44,9 @@ type ExtCtx = {
43
44
  listExtState(e: string): Array<{ key: string; value: string }>;
44
45
  };
45
46
  getConfig(spaceId: string, key: string): string | null;
47
+ /** Structural subset of `AppConfig` — only what this extension reads. */
48
+ config: { spacesDir: string };
49
+ log: { warn(message: string, meta?: Record<string, unknown>): void };
46
50
  };
47
51
 
48
52
  /** Structural match for `MercuryExtensionAPI` (avoid package subpath imports). */
@@ -131,12 +135,20 @@ export default function (mercury: MercuryExt) {
131
135
  // from the env var rather than from stored state makes the dashboard
132
136
  // correct before the first token is ever minted.
133
137
  credentialEnvVar: CLIENT_ID_VAR,
134
- // This connection can issue tax documents. In a group-linked space the
135
- // runtime then requires explicit admin enablement
136
- // (security.sensitive_connections_allowed) plus per-caller confirmation —
137
- // see src/core/runtime.ts. That operational cost is intended; the README
138
- // says so, because the symptom otherwise reads as a broken extension.
139
- sensitive: true,
138
+ // NOT `sensitive: true`, despite issuing tax documents. The runtime's
139
+ // sensitive guard (src/core/runtime.ts:217) is host-global and
140
+ // space-blind: getActiveSensitiveConnectionName() scans every loaded
141
+ // extension, and checkSensitiveConnectionGuard never looks at the space or
142
+ // the prompt. One sensitive connection anywhere on the host therefore puts
143
+ // EVERY group-linked space behind a per-message confirmation — on a box
144
+ // with fourteen of them, thirteen unrelated spaces pay for this one. Worse,
145
+ // the confirmation does not stick: the pending entry is deleted on `yes`,
146
+ // so the next message asks again, forever.
147
+ //
148
+ // The load-bearing controls are elsewhere and unaffected: the credentials
149
+ // are hostOnly and brokered (rung 4), so they never enter a container, and
150
+ // mercury.permission({defaultRoles:["admin"]}) keeps the verbs admin-only.
151
+ // Restore this line when the guard scopes per-space or per-extension.
140
152
  // Side-effect free and fast, per the ConnectionDef contract (5s timeout):
141
153
  // it reports from stored state and never mints a token or makes a request.
142
154
  statusCheck: async (ctx) => {
@@ -167,6 +179,15 @@ export default function (mercury: MercuryExt) {
167
179
  spaceId: req.spaceId,
168
180
  callerId: req.callerId,
169
181
  environment: ctx.getConfig(req.spaceId, ENVIRONMENT_KEY),
182
+ // `expense-upload` reads a file the container named, so it needs the
183
+ // workspace root to confine that path to the caller's own space.
184
+ // `resolveProjectPath` is not an exported subpath of the package; this
185
+ // mirrors it (src/config.ts) rather than widening the package API for
186
+ // two lines. Keep the two in step.
187
+ spacesDir: isAbsolute(ctx.config.spacesDir)
188
+ ? ctx.config.spacesDir
189
+ : join(process.cwd(), ctx.config.spacesDir),
190
+ logDenial: (message, detail) => ctx.log.warn(message, detail),
170
191
  });
171
192
  });
172
193
 
@@ -15,6 +15,14 @@
15
15
  * credential sets. A production key cannot mint against sandbox.
16
16
  * 3. Sandbox's API host is a `greeninvoice.co.il` name while its token host is
17
17
  * a `morning.dev` name. They do not share a domain; do not "tidy" them.
18
+ *
19
+ * A fourth, added 2026-09-06 and verified the same way: `/file-upload/v1/url`
20
+ * carries PATH-LEVEL `servers` in the spec that override the global ones, so it
21
+ * is not reachable under `apiBase` at all — hence `uploadBase`. That makes three
22
+ * distinct sandbox hosts (`api.sandbox.morning.dev`,
23
+ * `sandbox.d.greeninvoice.co.il`, `api.sandbox.d.greeninvoice.co.il`) which read
24
+ * like typos of one another and are not. `tests/morning-extension.test.ts`
25
+ * asserts all six as literals so a tidy-up cannot collapse them.
18
26
  */
19
27
 
20
28
  export const MORNING_EXT = "morning" as const;
@@ -27,16 +35,24 @@ export interface MorningHosts {
27
35
  readonly tokenUrl: string;
28
36
  /** API base, no trailing slash. Every non-token call is `${apiBase}/<path>`. */
29
37
  readonly apiBase: string;
38
+ /**
39
+ * Base for the expense file-upload endpoint only, no trailing slash. A
40
+ * different origin from `apiBase` and without the `/api/v1` prefix — see the
41
+ * fourth note above.
42
+ */
43
+ readonly uploadBase: string;
30
44
  }
31
45
 
32
46
  const HOSTS: Record<MorningEnvironment, MorningHosts> = {
33
47
  production: {
34
48
  tokenUrl: "https://api.morning.co/idp/v1/oauth/token",
35
49
  apiBase: "https://api.greeninvoice.co.il/api/v1",
50
+ uploadBase: "https://api.morning.co",
36
51
  },
37
52
  sandbox: {
38
53
  tokenUrl: "https://api.sandbox.morning.dev/idp/v1/oauth/token",
39
54
  apiBase: "https://sandbox.d.greeninvoice.co.il/api/v1",
55
+ uploadBase: "https://api.sandbox.d.greeninvoice.co.il",
40
56
  },
41
57
  };
42
58
 
@@ -29,6 +29,7 @@ import {
29
29
  mintIssueToken,
30
30
  } from "./issue-token.js";
31
31
  import { getAccessToken, recordAuthError, type TokenStore } from "./token.js";
32
+ import { expenseUpload as runExpenseUpload } from "./upload.js";
32
33
 
33
34
  export interface CapabilityResult {
34
35
  status?: number;
@@ -43,6 +44,14 @@ export interface ActionContext {
43
44
  spaceId: string;
44
45
  callerId: string;
45
46
  fetchImpl?: typeof fetch;
47
+ /**
48
+ * Root of all space workspaces, project-resolved by the caller. Only
49
+ * `expense-upload` needs it; absent means that verb refuses rather than
50
+ * guessing a root.
51
+ */
52
+ spacesDir?: string;
53
+ /** Host-side WARN channel for refusals that must not be detailed to the caller. */
54
+ logDenial?: (message: string, detail: Record<string, unknown>) => void;
46
55
  }
47
56
 
48
57
  type Body = Record<string, unknown>;
@@ -439,6 +448,73 @@ async function documentSearch(ctx: ActionContext, body: Body) {
439
448
  });
440
449
  }
441
450
 
451
+ async function expenseDraftSearch(ctx: ActionContext, body: Body) {
452
+ const { page, pageSize } = paging(body);
453
+ return apiCall(ctx, "POST", "/expenses/drafts/search", {
454
+ ...pick(body, [
455
+ "fromDate",
456
+ "toDate",
457
+ "description",
458
+ "supplierId",
459
+ "supplierName",
460
+ ]),
461
+ page,
462
+ pageSize,
463
+ });
464
+ }
465
+
466
+ /**
467
+ * Upload a receipt as an expense draft.
468
+ *
469
+ * Two hops to two different hosts, so it cannot be expressed as one `apiCall`.
470
+ * The token PROVIDER is handed over rather than a token: the retry policy lives
471
+ * in `upload.ts`, which is the only place that knows step 1 is safe to repeat
472
+ * and step 2 is not.
473
+ */
474
+ async function expenseUpload(ctx: ActionContext, body: Body) {
475
+ if (!ctx.spacesDir) {
476
+ return {
477
+ status: 503,
478
+ data: {
479
+ error:
480
+ "expense-upload is unavailable: the host did not supply a workspace root.",
481
+ },
482
+ };
483
+ }
484
+
485
+ return runExpenseUpload(
486
+ {
487
+ env: ctx.env,
488
+ spaceId: ctx.spaceId,
489
+ spacesDir: ctx.spacesDir,
490
+ ...(ctx.fetchImpl ? { fetchImpl: ctx.fetchImpl } : {}),
491
+ ...(ctx.logDenial ? { logDenial: ctx.logDenial } : {}),
492
+ },
493
+ body,
494
+ {
495
+ async getToken(forceRefresh: boolean) {
496
+ const result = await getAccessToken(ctx.db, ctx.env, {
497
+ forceRefresh,
498
+ ...(ctx.fetchImpl ? { fetchImpl: ctx.fetchImpl } : {}),
499
+ });
500
+ if (result.ok) return { ok: true as const, token: result.accessToken };
501
+ if (result.kind === "auth_failed") {
502
+ recordAuthError(ctx.db, ctx.env, result.message);
503
+ }
504
+ return {
505
+ ok: false as const,
506
+ result: {
507
+ status: result.kind === "not_configured" ? 503 : 502,
508
+ data: { error: result.message, kind: result.kind },
509
+ },
510
+ };
511
+ },
512
+ recordAuthError: (message: string) =>
513
+ recordAuthError(ctx.db, ctx.env, message),
514
+ },
515
+ );
516
+ }
517
+
442
518
  async function documentLinks(ctx: ActionContext, body: Body) {
443
519
  const id = typeof body.id === "string" ? body.id.trim() : "";
444
520
  if (!id) return usage("document-links requires an id");
@@ -462,6 +538,8 @@ const ACTIONS: Record<
462
538
  "document-create": documentCreate,
463
539
  "document-search": documentSearch,
464
540
  "document-links": documentLinks,
541
+ "expense-upload": expenseUpload,
542
+ "expense-draft-search": expenseDraftSearch,
465
543
  };
466
544
 
467
545
  export const AVAILABLE_ACTIONS = Object.keys(ACTIONS);