@anynotate/protocol 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (8) hide show
  1. package/PROTOCOL.md +45 -3
  2. package/dist/fixtures.d.ts +13 -1
  3. package/dist/fixtures.js +10 -2
  4. package/dist/index.d.ts +60 -5
  5. package/dist/index.js +14 -4
  6. package/package.json +1 -1
  7. package/src/fixtures.ts +10 -2
  8. package/src/index.ts +19 -4
package/PROTOCOL.md CHANGED
@@ -1,12 +1,12 @@
1
- # Anynotate protocol v1
1
+ # Anynotate protocol v2
2
2
 
3
3
  The bridge is an HTTP server on `127.0.0.1`, port `47291` by default (`ANYNOTATE_PORT`). Clients send annotation **bundles**; the bridge stores each one as a folder and delivers it to an AI agent session. Constants and schemas live in `@anynotate/protocol`; request/response examples for the extension-facing endpoints live in `@anynotate/protocol/fixtures` (`conformance`).
4
4
 
5
5
  ## Handshake
6
6
 
7
- `POST /health` (or `GET`) → `{ "ok": true, "bridgeVersion": "0.2.0", "protocol": { "version": 1, "min": 1 } }`.
7
+ `POST /health` (or `GET`) → `{ "ok": true, "bridgeVersion": "0.5.1", "protocol": { "version": 2, "min": 1 } }`.
8
8
 
9
- A client supports a range of protocol versions (the extension: `min 1, max 1`) and calls `compatibility(clientRange, response.protocol)`:
9
+ A client supports a range of protocol versions (the extension: `min 2, max 2`) and calls `compatibility(clientRange, response.protocol)`:
10
10
 
11
11
  - `ok` — the ranges overlap; proceed.
12
12
  - `bridge-too-old` — ask the user to update the bridge; don't send.
@@ -66,12 +66,54 @@ Each bundle is written to `$ANYNOTATE_HOME/inbox/<id>/` (default `~/.anynotate`)
66
66
 
67
67
  `$ANYNOTATE_HOME/inbox/latest` points at the newest bundle.
68
68
 
69
+ ## Intents
70
+
71
+ Each annotation may carry an `intent` telling the agent what the user wants. `INTENTS` lists the current ones; `README.md` puts a directive line under the note's heading instead of naming the intent:
72
+
73
+ | `intent` | Directive in `README.md` |
74
+ |---|---|
75
+ | `explain` | `**Explain** this.` |
76
+ | `change` | `**Change requested.**` |
77
+ | `approve` | `**Approved** — no change needed.` |
78
+
79
+ The v1 values `question`, `bug` and `note` are still accepted and appear in the heading as before, with no directive line. `change` was also a v1 value; it keeps its name and now gets the directive line. No `intent` means no directive either.
80
+
81
+ ### Region notes
82
+
83
+ On pages that draw into a `<canvas>` (online documents, design tools, maps), there is no text or element under the user's pointer worth selecting, so the user draws a rectangle instead. Such a note is an `element` annotation whose `element` is the node under the rectangle (usually the `<canvas>`) and which carries an optional `region`:
84
+
85
+ ```ts
86
+ region?: { x: number; y: number; w: number; h: number }
87
+ ```
88
+
89
+ The rectangle is in CSS pixels, relative to the viewport at capture time; add `viewport.scrollY` for the page offset. `w` and `h` are non-negative. `box` still describes the whole element, and `crops/A<n>.png` shows just the rectangle. `README.md` adds a line under the note's heading, such as `Region: 420×180 at (310, 96) on <canvas#board> · see crops/A3.png`.
90
+
91
+ `region` is only valid on `kind: "element"`; a `text` annotation with a `region` is rejected. It is an additive field: a bridge that predates it parses the annotation as a plain element note and drops `region`, so the agent still gets the comment, element and crop.
92
+
93
+ ### Notes inside frames
94
+
95
+ A note made on text or an element inside a same-origin `<iframe>` carries the frames on its anchor:
96
+
97
+ ```ts
98
+ anchor.frames?: string[]
99
+ ```
100
+
101
+ Each entry is a CSS selector for an `<iframe>`, outermost first, resolved in the document of the frame before it (the first in the top page). `css`, `hosts`, `path` and `quote` then refer to the innermost frame's document, while `box` and `region` stay in the top page's viewport. `README.md` names the frames after the note's selector, such as ``selector: `p#inner` inside iframe `iframe#card` ``. It is additive: a bridge that predates it drops `frames`, and the agent gets a selector without the frame it lives in.
102
+
69
103
  ## Evolution
70
104
 
71
105
  - Additive changes — optional fields, new endpoints — keep `PROTOCOL_VERSION`. Both sides ignore fields they don't know.
72
106
  - Removing, renaming or changing the meaning of anything bumps `PROTOCOL_VERSION`; the bridge keeps accepting the previous version (`PROTOCOL_MIN`) for at least one release.
73
107
  - The bundle format has its own `v` (currently `1`); the agent-facing `README.md` stays human-readable.
74
108
 
109
+ ### Version history
110
+
111
+ - **v1** — handshake, token-only access, native-host token delivery.
112
+ - **v2** — intents `explain`, `change`, `approve` (legacy `question`, `bug`, `note` still accepted). `PROTOCOL_MIN` stays `1`: a v2 bridge accepts v1 clients.
113
+ - Optional `region` on element annotations (additive, `@anynotate/protocol` 0.4.0, bridge 0.5.1).
114
+ - Optional `offscreen: true` on any annotation whose target was not on screen at send time (`box` and crop are from when it was last seen); `README.md` adds `(off-screen when sent)` to its heading (additive, same releases).
115
+ - Optional `anchor.frames` on notes made inside same-origin iframes; `README.md` names the frames after the selector (additive, same releases).
116
+
75
117
  ## Threat model
76
118
 
77
119
  - **Web pages** can reach `127.0.0.1` from the user's browser. They are refused by `Origin` (including preflights), and DNS-rebinding pages by `Host`; a web request without an `Origin` (such as an `<img>` GET) fails the token check.
package/dist/fixtures.d.ts CHANGED
@@ -42,6 +42,7 @@ export declare const responseSchemas: {
42
42
  pull: "pull";
43
43
  }>>;
44
44
  session: z.ZodOptional<z.ZodString>;
45
+ agent: z.ZodOptional<z.ZodString>;
45
46
  summary: z.ZodOptional<z.ZodString>;
46
47
  note: z.ZodOptional<z.ZodString>;
47
48
  }, z.core.$strip>;
@@ -67,8 +68,10 @@ export declare const responseSchemas: {
67
68
  }>;
68
69
  comment: z.ZodString;
69
70
  intent: z.ZodOptional<z.ZodEnum<{
70
- question: "question";
71
+ explain: "explain";
71
72
  change: "change";
73
+ approve: "approve";
74
+ question: "question";
72
75
  bug: "bug";
73
76
  note: "note";
74
77
  }>>;
@@ -86,6 +89,7 @@ export declare const responseSchemas: {
86
89
  path: z.ZodArray<z.ZodString>;
87
90
  near: z.ZodString;
88
91
  hosts: z.ZodOptional<z.ZodArray<z.ZodString>>;
92
+ frames: z.ZodOptional<z.ZodArray<z.ZodString>>;
89
93
  }, z.core.$strip>;
90
94
  element: z.ZodOptional<z.ZodObject<{
91
95
  tag: z.ZodString;
@@ -108,6 +112,13 @@ export declare const responseSchemas: {
108
112
  scrollY: z.ZodNumber;
109
113
  }, z.core.$strip>;
110
114
  crop: z.ZodString;
115
+ region: z.ZodOptional<z.ZodObject<{
116
+ x: z.ZodNumber;
117
+ y: z.ZodNumber;
118
+ w: z.ZodNumber;
119
+ h: z.ZodNumber;
120
+ }, z.core.$strip>>;
121
+ offscreen: z.ZodOptional<z.ZodBoolean>;
111
122
  }, z.core.$strip>>;
112
123
  id: z.ZodString;
113
124
  files: z.ZodObject<{
@@ -130,6 +141,7 @@ export declare const responseSchemas: {
130
141
  pull: "pull";
131
142
  }>>;
132
143
  session: z.ZodOptional<z.ZodString>;
144
+ agent: z.ZodOptional<z.ZodString>;
133
145
  summary: z.ZodOptional<z.ZodString>;
134
146
  note: z.ZodOptional<z.ZodString>;
135
147
  }, z.core.$strip>>;
package/dist/fixtures.js CHANGED
@@ -11,7 +11,7 @@ export const exampleBundleInput = {
11
11
  target: { agent: "claude", sessionId: "s-1", cwd: "/tmp/repo" },
12
12
  overall: "can we make this vegan?",
13
13
  annotations: [{
14
- id: "A1", kind: "text", comment: "is there a substitute for cream?", intent: "question",
14
+ id: "A1", kind: "text", comment: "is there a substitute for cream?", intent: "explain",
15
15
  anchor: {
16
16
  quote: { exact: "200 ml cream", prefix: "stir in ", suffix: " before serving" },
17
17
  css: "section#ingredients > li:nth-child(3)", path: ["main", "section#ingredients", "ul"], near: "Ingredients",
@@ -19,13 +19,21 @@ export const exampleBundleInput = {
19
19
  box: { x: 10, y: 20, w: 100, h: 18 },
20
20
  viewport: { w: 1280, h: 800, dpr: 2, scrollY: 0 },
21
21
  crop: "crops/A1.png",
22
+ }, {
23
+ id: "A2", kind: "element", comment: "make this bowl bigger", intent: "change",
24
+ anchor: { css: "canvas#plating", path: ["main", "figure"], near: "Serving" },
25
+ element: { tag: "canvas", text: "", html: '<canvas id="plating" width="800" height="600"></canvas>', attrs: { id: "plating" } },
26
+ box: { x: 40, y: 300, w: 800, h: 600 },
27
+ viewport: { w: 1280, h: 800, dpr: 2, scrollY: 0 },
28
+ crop: "crops/A2.png",
29
+ region: { x: 310, y: 420, w: 220, h: 140 },
22
30
  }],
23
31
  };
24
32
  export const examplePage = "# Tomato soup\n\nstir in ⟦A1⟧200 ml cream before serving\n";
25
33
  export const responseSchemas = {
26
34
  HealthResponse, SessionsResponse, SendResponse, BundleStatusResponse, OkResponse, ErrorResponse,
27
35
  };
28
- const health = { ok: true, bridgeVersion: "0.2.0", protocol: { version: 1, min: 1 } };
36
+ const health = { ok: true, bridgeVersion: "0.5.1", protocol: { version: 2, min: 1 } };
29
37
  const queued = { state: "queued", at: "2026-09-24T15:32:01.000Z" };
30
38
  const exampleBundle = { ...exampleBundleInput, id: EXAMPLE_BUNDLE_ID, files: { page: "page.md", screenshot: "screenshot.png" } };
31
39
  export const conformance = [
package/dist/index.d.ts CHANGED
@@ -2,6 +2,18 @@ import { z } from "zod";
2
2
  export declare const AGENT_NAME: RegExp;
3
3
  export declare const Agent: z.ZodString;
4
4
  export type Agent = z.infer<typeof Agent>;
5
+ export declare const INTENTS: readonly ["explain", "change", "approve"];
6
+ export declare const LEGACY_INTENTS: readonly ["question", "bug", "note"];
7
+ /** The intent clients should send; parsed annotations may also hold a LEGACY_INTENTS value. */
8
+ export type Intent = (typeof INTENTS)[number];
9
+ /** A rectangle the user drew, in CSS px relative to the viewport at capture time (add `viewport.scrollY` for page y). */
10
+ export declare const Region: z.ZodObject<{
11
+ x: z.ZodNumber;
12
+ y: z.ZodNumber;
13
+ w: z.ZodNumber;
14
+ h: z.ZodNumber;
15
+ }, z.core.$strip>;
16
+ export type Region = z.infer<typeof Region>;
5
17
  export declare const Annotation: z.ZodObject<{
6
18
  id: z.ZodString;
7
19
  kind: z.ZodEnum<{
@@ -10,8 +22,10 @@ export declare const Annotation: z.ZodObject<{
10
22
  }>;
11
23
  comment: z.ZodString;
12
24
  intent: z.ZodOptional<z.ZodEnum<{
13
- question: "question";
25
+ explain: "explain";
14
26
  change: "change";
27
+ approve: "approve";
28
+ question: "question";
15
29
  bug: "bug";
16
30
  note: "note";
17
31
  }>>;
@@ -29,6 +43,7 @@ export declare const Annotation: z.ZodObject<{
29
43
  path: z.ZodArray<z.ZodString>;
30
44
  near: z.ZodString;
31
45
  hosts: z.ZodOptional<z.ZodArray<z.ZodString>>;
46
+ frames: z.ZodOptional<z.ZodArray<z.ZodString>>;
32
47
  }, z.core.$strip>;
33
48
  element: z.ZodOptional<z.ZodObject<{
34
49
  tag: z.ZodString;
@@ -51,6 +66,13 @@ export declare const Annotation: z.ZodObject<{
51
66
  scrollY: z.ZodNumber;
52
67
  }, z.core.$strip>;
53
68
  crop: z.ZodString;
69
+ region: z.ZodOptional<z.ZodObject<{
70
+ x: z.ZodNumber;
71
+ y: z.ZodNumber;
72
+ w: z.ZodNumber;
73
+ h: z.ZodNumber;
74
+ }, z.core.$strip>>;
75
+ offscreen: z.ZodOptional<z.ZodBoolean>;
54
76
  }, z.core.$strip>;
55
77
  export type Annotation = z.infer<typeof Annotation>;
56
78
  export declare const Target: z.ZodObject<{
@@ -80,8 +102,10 @@ export declare const BundleInput: z.ZodObject<{
80
102
  }>;
81
103
  comment: z.ZodString;
82
104
  intent: z.ZodOptional<z.ZodEnum<{
83
- question: "question";
105
+ explain: "explain";
84
106
  change: "change";
107
+ approve: "approve";
108
+ question: "question";
85
109
  bug: "bug";
86
110
  note: "note";
87
111
  }>>;
@@ -99,6 +123,7 @@ export declare const BundleInput: z.ZodObject<{
99
123
  path: z.ZodArray<z.ZodString>;
100
124
  near: z.ZodString;
101
125
  hosts: z.ZodOptional<z.ZodArray<z.ZodString>>;
126
+ frames: z.ZodOptional<z.ZodArray<z.ZodString>>;
102
127
  }, z.core.$strip>;
103
128
  element: z.ZodOptional<z.ZodObject<{
104
129
  tag: z.ZodString;
@@ -121,6 +146,13 @@ export declare const BundleInput: z.ZodObject<{
121
146
  scrollY: z.ZodNumber;
122
147
  }, z.core.$strip>;
123
148
  crop: z.ZodString;
149
+ region: z.ZodOptional<z.ZodObject<{
150
+ x: z.ZodNumber;
151
+ y: z.ZodNumber;
152
+ w: z.ZodNumber;
153
+ h: z.ZodNumber;
154
+ }, z.core.$strip>>;
155
+ offscreen: z.ZodOptional<z.ZodBoolean>;
124
156
  }, z.core.$strip>>;
125
157
  }, z.core.$strip>;
126
158
  export type BundleInput = z.infer<typeof BundleInput>;
@@ -145,8 +177,10 @@ export declare const Bundle: z.ZodObject<{
145
177
  }>;
146
178
  comment: z.ZodString;
147
179
  intent: z.ZodOptional<z.ZodEnum<{
148
- question: "question";
180
+ explain: "explain";
149
181
  change: "change";
182
+ approve: "approve";
183
+ question: "question";
150
184
  bug: "bug";
151
185
  note: "note";
152
186
  }>>;
@@ -164,6 +198,7 @@ export declare const Bundle: z.ZodObject<{
164
198
  path: z.ZodArray<z.ZodString>;
165
199
  near: z.ZodString;
166
200
  hosts: z.ZodOptional<z.ZodArray<z.ZodString>>;
201
+ frames: z.ZodOptional<z.ZodArray<z.ZodString>>;
167
202
  }, z.core.$strip>;
168
203
  element: z.ZodOptional<z.ZodObject<{
169
204
  tag: z.ZodString;
@@ -186,6 +221,13 @@ export declare const Bundle: z.ZodObject<{
186
221
  scrollY: z.ZodNumber;
187
222
  }, z.core.$strip>;
188
223
  crop: z.ZodString;
224
+ region: z.ZodOptional<z.ZodObject<{
225
+ x: z.ZodNumber;
226
+ y: z.ZodNumber;
227
+ w: z.ZodNumber;
228
+ h: z.ZodNumber;
229
+ }, z.core.$strip>>;
230
+ offscreen: z.ZodOptional<z.ZodBoolean>;
189
231
  }, z.core.$strip>>;
190
232
  id: z.ZodString;
191
233
  files: z.ZodObject<{
@@ -216,6 +258,7 @@ export declare const Status: z.ZodObject<{
216
258
  pull: "pull";
217
259
  }>>;
218
260
  session: z.ZodOptional<z.ZodString>;
261
+ agent: z.ZodOptional<z.ZodString>;
219
262
  summary: z.ZodOptional<z.ZodString>;
220
263
  note: z.ZodOptional<z.ZodString>;
221
264
  }, z.core.$strip>;
@@ -234,7 +277,7 @@ export declare const Session: z.ZodObject<{
234
277
  sessionIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
235
278
  }, z.core.$strip>;
236
279
  export type Session = z.infer<typeof Session>;
237
- export declare const PROTOCOL_VERSION = 1;
280
+ export declare const PROTOCOL_VERSION = 2;
238
281
  export declare const PROTOCOL_MIN = 1;
239
282
  export declare const HealthResponse: z.ZodObject<{
240
283
  ok: z.ZodLiteral<true>;
@@ -275,6 +318,7 @@ export declare const SendResponse: z.ZodObject<{
275
318
  pull: "pull";
276
319
  }>>;
277
320
  session: z.ZodOptional<z.ZodString>;
321
+ agent: z.ZodOptional<z.ZodString>;
278
322
  summary: z.ZodOptional<z.ZodString>;
279
323
  note: z.ZodOptional<z.ZodString>;
280
324
  }, z.core.$strip>;
@@ -301,8 +345,10 @@ export declare const BundleStatusResponse: z.ZodObject<{
301
345
  }>;
302
346
  comment: z.ZodString;
303
347
  intent: z.ZodOptional<z.ZodEnum<{
304
- question: "question";
348
+ explain: "explain";
305
349
  change: "change";
350
+ approve: "approve";
351
+ question: "question";
306
352
  bug: "bug";
307
353
  note: "note";
308
354
  }>>;
@@ -320,6 +366,7 @@ export declare const BundleStatusResponse: z.ZodObject<{
320
366
  path: z.ZodArray<z.ZodString>;
321
367
  near: z.ZodString;
322
368
  hosts: z.ZodOptional<z.ZodArray<z.ZodString>>;
369
+ frames: z.ZodOptional<z.ZodArray<z.ZodString>>;
323
370
  }, z.core.$strip>;
324
371
  element: z.ZodOptional<z.ZodObject<{
325
372
  tag: z.ZodString;
@@ -342,6 +389,13 @@ export declare const BundleStatusResponse: z.ZodObject<{
342
389
  scrollY: z.ZodNumber;
343
390
  }, z.core.$strip>;
344
391
  crop: z.ZodString;
392
+ region: z.ZodOptional<z.ZodObject<{
393
+ x: z.ZodNumber;
394
+ y: z.ZodNumber;
395
+ w: z.ZodNumber;
396
+ h: z.ZodNumber;
397
+ }, z.core.$strip>>;
398
+ offscreen: z.ZodOptional<z.ZodBoolean>;
345
399
  }, z.core.$strip>>;
346
400
  id: z.ZodString;
347
401
  files: z.ZodObject<{
@@ -364,6 +418,7 @@ export declare const BundleStatusResponse: z.ZodObject<{
364
418
  pull: "pull";
365
419
  }>>;
366
420
  session: z.ZodOptional<z.ZodString>;
421
+ agent: z.ZodOptional<z.ZodString>;
367
422
  summary: z.ZodOptional<z.ZodString>;
368
423
  note: z.ZodOptional<z.ZodString>;
369
424
  }, z.core.$strip>>;
package/dist/index.js CHANGED
@@ -1,13 +1,17 @@
1
1
  import { z } from "zod";
2
2
  export const AGENT_NAME = /^[a-z][a-z0-9_-]{0,31}$/;
3
3
  export const Agent = z.string().regex(AGENT_NAME);
4
+ export const INTENTS = ["explain", "change", "approve"];
5
+ export const LEGACY_INTENTS = ["question", "bug", "note"];
4
6
  const Box = z.object({ x: z.number(), y: z.number(), w: z.number(), h: z.number() });
7
+ /** A rectangle the user drew, in CSS px relative to the viewport at capture time (add `viewport.scrollY` for page y). */
8
+ export const Region = z.object({ x: z.number(), y: z.number(), w: z.number().nonnegative(), h: z.number().nonnegative() });
5
9
  export const Annotation = z.object({
6
10
  id: z.string().regex(/^A\d+$/),
7
11
  // v1 kinds only; "region" | "draw" are reserved for later tools.
8
12
  kind: z.enum(["text", "element"]),
9
13
  comment: z.string(),
10
- intent: z.enum(["question", "change", "bug", "note"]).optional(),
14
+ intent: z.enum([...INTENTS, ...LEGACY_INTENTS]).optional(),
11
15
  anchor: z.object({
12
16
  quote: z.object({ exact: z.string(), prefix: z.string(), suffix: z.string() }).optional(),
13
17
  position: z.object({ start: z.number().int(), end: z.number().int() }).optional(),
@@ -15,6 +19,8 @@ export const Annotation = z.object({
15
19
  path: z.array(z.string()),
16
20
  near: z.string(),
17
21
  hosts: z.array(z.string()).optional(),
22
+ /** CSS selectors of the iframes holding the note, outermost first, each resolved in the previous frame's document; absent for the top document. `css`, `hosts` and `path` are then inside the innermost frame, while `box` and `region` stay in the top page's viewport. */
23
+ frames: z.array(z.string()).optional(),
18
24
  }),
19
25
  element: z.object({
20
26
  tag: z.string(),
@@ -27,7 +33,10 @@ export const Annotation = z.object({
27
33
  box: Box,
28
34
  viewport: z.object({ w: z.number(), h: z.number(), dpr: z.number(), scrollY: z.number() }),
29
35
  crop: z.string().regex(/^crops\/A\d+\.png$/),
30
- });
36
+ region: Region.optional(),
37
+ /** True when the note's target was not on screen at send time (scrolled out of a virtual list, another tab); `box` and the crop are from when it was last seen. */
38
+ offscreen: z.boolean().optional(),
39
+ }).refine((a) => !a.region || a.kind === "element", { message: "region is only allowed on element annotations", path: ["region"] });
31
40
  export const Target = z.object({
32
41
  agent: Agent,
33
42
  sessionId: z.string().optional(),
@@ -56,6 +65,7 @@ export const Status = z.object({
56
65
  at: z.string(),
57
66
  via: Via.optional(),
58
67
  session: z.string().optional(),
68
+ agent: Agent.optional(),
59
69
  summary: z.string().optional(),
60
70
  note: z.string().optional(),
61
71
  });
@@ -68,9 +78,9 @@ export const Session = z.object({
68
78
  pane: z.string().optional(),
69
79
  sessionIds: z.array(z.string()).optional(),
70
80
  });
71
- // Protocol v1. Additive changes (optional fields, new endpoints) keep the version; removals, renames and
81
+ // Protocol v2. Additive changes (optional fields, new endpoints) keep the version; removals, renames and
72
82
  // semantic changes bump PROTOCOL_VERSION, and the bridge keeps accepting the previous version for a release.
73
- export const PROTOCOL_VERSION = 1;
83
+ export const PROTOCOL_VERSION = 2;
74
84
  export const PROTOCOL_MIN = 1;
75
85
  export const HealthResponse = z.object({
76
86
  ok: z.literal(true),
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@anynotate/protocol",
3
- "version": "0.2.0",
3
+ "version": "0.4.0",
4
4
  "description": "Wire types and zod schemas shared by Anynotate clients and the Anynotate bridge.",
5
5
  "license": "Apache-2.0",
6
6
  "author": "Kun Xu",
package/src/fixtures.ts CHANGED
@@ -16,7 +16,7 @@ export const exampleBundleInput: BundleInput = {
16
16
  target: { agent: "claude", sessionId: "s-1", cwd: "/tmp/repo" },
17
17
  overall: "can we make this vegan?",
18
18
  annotations: [{
19
- id: "A1", kind: "text", comment: "is there a substitute for cream?", intent: "question",
19
+ id: "A1", kind: "text", comment: "is there a substitute for cream?", intent: "explain",
20
20
  anchor: {
21
21
  quote: { exact: "200 ml cream", prefix: "stir in ", suffix: " before serving" },
22
22
  css: "section#ingredients > li:nth-child(3)", path: ["main", "section#ingredients", "ul"], near: "Ingredients",
@@ -24,6 +24,14 @@ export const exampleBundleInput: BundleInput = {
24
24
  box: { x: 10, y: 20, w: 100, h: 18 },
25
25
  viewport: { w: 1280, h: 800, dpr: 2, scrollY: 0 },
26
26
  crop: "crops/A1.png",
27
+ }, {
28
+ id: "A2", kind: "element", comment: "make this bowl bigger", intent: "change",
29
+ anchor: { css: "canvas#plating", path: ["main", "figure"], near: "Serving" },
30
+ element: { tag: "canvas", text: "", html: '<canvas id="plating" width="800" height="600"></canvas>', attrs: { id: "plating" } },
31
+ box: { x: 40, y: 300, w: 800, h: 600 },
32
+ viewport: { w: 1280, h: 800, dpr: 2, scrollY: 0 },
33
+ crop: "crops/A2.png",
34
+ region: { x: 310, y: 420, w: 220, h: 140 },
27
35
  }],
28
36
  };
29
37
  export const examplePage = "# Tomato soup\n\nstir in ⟦A1⟧200 ml cream before serving\n";
@@ -49,7 +57,7 @@ export type Exchange = {
49
57
  response: { schema: SchemaName; example: unknown };
50
58
  };
51
59
 
52
- const health = { ok: true, bridgeVersion: "0.2.0", protocol: { version: 1, min: 1 } };
60
+ const health = { ok: true, bridgeVersion: "0.5.1", protocol: { version: 2, min: 1 } };
53
61
  const queued = { state: "queued", at: "2026-09-24T15:32:01.000Z" };
54
62
  const exampleBundle = { ...exampleBundleInput, id: EXAMPLE_BUNDLE_ID, files: { page: "page.md", screenshot: "screenshot.png" } };
55
63
 
package/src/index.ts CHANGED
@@ -4,14 +4,23 @@ export const AGENT_NAME = /^[a-z][a-z0-9_-]{0,31}$/;
4
4
  export const Agent = z.string().regex(AGENT_NAME);
5
5
  export type Agent = z.infer<typeof Agent>;
6
6
 
7
+ export const INTENTS = ["explain", "change", "approve"] as const;
8
+ export const LEGACY_INTENTS = ["question", "bug", "note"] as const;
9
+ /** The intent clients should send; parsed annotations may also hold a LEGACY_INTENTS value. */
10
+ export type Intent = (typeof INTENTS)[number];
11
+
7
12
  const Box = z.object({ x: z.number(), y: z.number(), w: z.number(), h: z.number() });
8
13
 
14
+ /** A rectangle the user drew, in CSS px relative to the viewport at capture time (add `viewport.scrollY` for page y). */
15
+ export const Region = z.object({ x: z.number(), y: z.number(), w: z.number().nonnegative(), h: z.number().nonnegative() });
16
+ export type Region = z.infer<typeof Region>;
17
+
9
18
  export const Annotation = z.object({
10
19
  id: z.string().regex(/^A\d+$/),
11
20
  // v1 kinds only; "region" | "draw" are reserved for later tools.
12
21
  kind: z.enum(["text", "element"]),
13
22
  comment: z.string(),
14
- intent: z.enum(["question", "change", "bug", "note"]).optional(),
23
+ intent: z.enum([...INTENTS, ...LEGACY_INTENTS]).optional(),
15
24
  anchor: z.object({
16
25
  quote: z.object({ exact: z.string(), prefix: z.string(), suffix: z.string() }).optional(),
17
26
  position: z.object({ start: z.number().int(), end: z.number().int() }).optional(),
@@ -19,6 +28,8 @@ export const Annotation = z.object({
19
28
  path: z.array(z.string()),
20
29
  near: z.string(),
21
30
  hosts: z.array(z.string()).optional(),
31
+ /** CSS selectors of the iframes holding the note, outermost first, each resolved in the previous frame's document; absent for the top document. `css`, `hosts` and `path` are then inside the innermost frame, while `box` and `region` stay in the top page's viewport. */
32
+ frames: z.array(z.string()).optional(),
22
33
  }),
23
34
  element: z.object({
24
35
  tag: z.string(),
@@ -31,7 +42,10 @@ export const Annotation = z.object({
31
42
  box: Box,
32
43
  viewport: z.object({ w: z.number(), h: z.number(), dpr: z.number(), scrollY: z.number() }),
33
44
  crop: z.string().regex(/^crops\/A\d+\.png$/),
34
- });
45
+ region: Region.optional(),
46
+ /** True when the note's target was not on screen at send time (scrolled out of a virtual list, another tab); `box` and the crop are from when it was last seen. */
47
+ offscreen: z.boolean().optional(),
48
+ }).refine((a) => !a.region || a.kind === "element", { message: "region is only allowed on element annotations", path: ["region"] });
35
49
  export type Annotation = z.infer<typeof Annotation>;
36
50
 
37
51
  export const Target = z.object({
@@ -71,6 +85,7 @@ export const Status = z.object({
71
85
  at: z.string(),
72
86
  via: Via.optional(),
73
87
  session: z.string().optional(),
88
+ agent: Agent.optional(),
74
89
  summary: z.string().optional(),
75
90
  note: z.string().optional(),
76
91
  });
@@ -87,9 +102,9 @@ export const Session = z.object({
87
102
  });
88
103
  export type Session = z.infer<typeof Session>;
89
104
 
90
- // Protocol v1. Additive changes (optional fields, new endpoints) keep the version; removals, renames and
105
+ // Protocol v2. Additive changes (optional fields, new endpoints) keep the version; removals, renames and
91
106
  // semantic changes bump PROTOCOL_VERSION, and the bridge keeps accepting the previous version for a release.
92
- export const PROTOCOL_VERSION = 1;
107
+ export const PROTOCOL_VERSION = 2;
93
108
  export const PROTOCOL_MIN = 1;
94
109
 
95
110
  export const HealthResponse = z.object({