@sightmap/sightkick 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sightmap/sightkick",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "sightkick — compile a webmcp.tools.yaml + sightmap corpus into WebMCP tool IR, and install the sightkick agent skills (native binary, no Go toolchain required)",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"homepage": "https://sightmap.org",
|
|
@@ -34,11 +34,11 @@
|
|
|
34
34
|
"access": "public"
|
|
35
35
|
},
|
|
36
36
|
"optionalDependencies": {
|
|
37
|
-
"@sightmap/sightkick-darwin-arm64": "0.
|
|
38
|
-
"@sightmap/sightkick-darwin-x64": "0.
|
|
39
|
-
"@sightmap/sightkick-linux-arm64": "0.
|
|
40
|
-
"@sightmap/sightkick-linux-x64": "0.
|
|
41
|
-
"@sightmap/sightkick-win32-arm64": "0.
|
|
42
|
-
"@sightmap/sightkick-win32-x64": "0.
|
|
37
|
+
"@sightmap/sightkick-darwin-arm64": "0.3.0",
|
|
38
|
+
"@sightmap/sightkick-darwin-x64": "0.3.0",
|
|
39
|
+
"@sightmap/sightkick-linux-arm64": "0.3.0",
|
|
40
|
+
"@sightmap/sightkick-linux-x64": "0.3.0",
|
|
41
|
+
"@sightmap/sightkick-win32-arm64": "0.3.0",
|
|
42
|
+
"@sightmap/sightkick-win32-x64": "0.3.0"
|
|
43
43
|
}
|
|
44
44
|
}
|
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sightkick-authoring
|
|
3
|
+
description: Author a webmcp.tools.yaml — the sightkick manifest that turns a sightmap corpus into named WebMCP tools (atomic view-scoped actions + guidance journeys), then compile it to IR with `sightkick build`. Use when you have (or are building) a `.sightmap/` corpus and want to define the tools an agent can call on that app. Pairs with sightmap-authoring (writes the corpus this reads) and sightkick-debug (runs the compiled tools on a live page).
|
|
4
|
+
activation:
|
|
5
|
+
- a `.sightmap/` corpus exists (or is being authored) and you want to define WebMCP tools over it
|
|
6
|
+
- writing or editing a `webmcp.tools.yaml`
|
|
7
|
+
- a `sightkick build` reports unresolved component/property/view references
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# sightkick-authoring: write a webmcp.tools.yaml
|
|
11
|
+
|
|
12
|
+
sightkick compiles two inputs into a self-contained **IR**: a `.sightmap/` corpus
|
|
13
|
+
(the app's component map — authored with the **`sightmap-authoring`** skill) and a
|
|
14
|
+
**`webmcp.tools.yaml`** manifest (the *tool layer*, authored here). `sightkick
|
|
15
|
+
build` resolves every reference in the manifest against the corpus and reports
|
|
16
|
+
unresolved ones with candidate lists. Once it compiles, drive the tools on a live
|
|
17
|
+
page with the **`sightkick-debug`** skill.
|
|
18
|
+
|
|
19
|
+
**Prerequisite:** the corpus must exist first — tools reference corpus
|
|
20
|
+
**component names** and **declared properties**, so if a name/property isn't in
|
|
21
|
+
the corpus, author it there (sightmap-authoring) before referencing it here.
|
|
22
|
+
|
|
23
|
+
## Mental model
|
|
24
|
+
|
|
25
|
+
- A **tool** is one atomic action at a single point in time — no navigation
|
|
26
|
+
crossing mid-tool. It bundles ordered `steps` (fill/click/…) and/or a
|
|
27
|
+
`returns` read, and yields a structured result.
|
|
28
|
+
- Multi-step flows are **not executed** by a runtime. A **journey** is a
|
|
29
|
+
compile-time ordering over tools that compiles into **guidance breadcrumbs**
|
|
30
|
+
("after `add_task`, consider `list_tasks`") attached to each tool's result. The
|
|
31
|
+
agent sequences; you hand it the map.
|
|
32
|
+
- Tools address elements by **component query** (component identity + extracted
|
|
33
|
+
properties + descendant scope), never raw CSS.
|
|
34
|
+
|
|
35
|
+
## File shape
|
|
36
|
+
|
|
37
|
+
```yaml
|
|
38
|
+
version: 1 # required (must be 1)
|
|
39
|
+
name: myapp # the IR name
|
|
40
|
+
corpus: ./.sightmap # required — path to the corpus dir, relative to this file
|
|
41
|
+
tools: [ ... ] # required, non-empty
|
|
42
|
+
journeys: [ ... ] # optional
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Tools
|
|
46
|
+
|
|
47
|
+
```yaml
|
|
48
|
+
- name: add_task # required, unique
|
|
49
|
+
description: Add a task. # shown in getTools(); a result-shape hint is appended automatically
|
|
50
|
+
mode: live # live (default) drives the DOM; api is opt-in reads-only
|
|
51
|
+
ensure_view: Home # a corpus VIEW name — scopes component resolution to that
|
|
52
|
+
# view (+ globals) AND view-scopes the tool at runtime
|
|
53
|
+
# (it only registers on pages whose route matches)
|
|
54
|
+
params: # become the tool's input schema; referenced as {{name}}
|
|
55
|
+
- name: title
|
|
56
|
+
type: string # string | number | boolean | enum
|
|
57
|
+
required: true
|
|
58
|
+
description: The task title.
|
|
59
|
+
# values: [A, B, C] # required when type: enum
|
|
60
|
+
guard: # optional idempotency guard — exactly one of present/absent
|
|
61
|
+
present: { query: 'TaskItem[title="{{title}}"]' } # SKIP the steps when this exists
|
|
62
|
+
# absent: { query: ... } # SKIP when it does NOT exist
|
|
63
|
+
steps: [ ... ] # ordered actions (below)
|
|
64
|
+
returns: { ... } # the structured result (below)
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
A `live` tool needs **at least one `step` or a `returns`**. `ensure_view` is
|
|
68
|
+
optional but recommended — it both disambiguates component names and controls
|
|
69
|
+
which page the tool appears on.
|
|
70
|
+
|
|
71
|
+
### Steps (each is a single-key mapping: the op)
|
|
72
|
+
|
|
73
|
+
| Step | Body | Does |
|
|
74
|
+
|------|------|------|
|
|
75
|
+
| `fill` | `query`, `value` | Type `value` (supports `{{param}}`) into the matched input. |
|
|
76
|
+
| `click` | `query` | Click the matched element. |
|
|
77
|
+
| `wait_for` | `query`, `timeout_ms` (default 5000) | Wait until the query matches — use after a mutating action to confirm the visible result. |
|
|
78
|
+
| `navigate` | `view` | Client-navigate to a corpus **view** by name. |
|
|
79
|
+
| `goto` | `url` | Navigate to a URL template (`{{param}}` interpolated). |
|
|
80
|
+
|
|
81
|
+
Reads are **not** steps — declare them with `returns`. A tool that ends with a
|
|
82
|
+
mutation should `wait_for` its own visible feedback before returning.
|
|
83
|
+
|
|
84
|
+
### Returns (exactly one of `value` or `list`, or description-only)
|
|
85
|
+
|
|
86
|
+
```yaml
|
|
87
|
+
returns:
|
|
88
|
+
description: One task title. # optional; also folded into the tool description
|
|
89
|
+
value: # a single scalar
|
|
90
|
+
query: 'TaskItem[title="{{title}}"]'
|
|
91
|
+
property: title # a DECLARED corpus property of the matched component
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
```yaml
|
|
95
|
+
returns:
|
|
96
|
+
description: The current task rows.
|
|
97
|
+
list: # an array of objects, one per match
|
|
98
|
+
rows: TaskItem # a compquery; every match is a row
|
|
99
|
+
fields: # outputName: <declared property of the row>
|
|
100
|
+
title: title # scalar shorthand, or: title: { property: title }
|
|
101
|
+
done: done
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
## Component queries (CSS-shaped, over corpus components)
|
|
105
|
+
|
|
106
|
+
- `Component` — by corpus component name.
|
|
107
|
+
- `Component[prop="v"]` — filter on an **extracted property** (not a raw DOM
|
|
108
|
+
attribute). Operators: `=` exact, `^=` prefix, `*=` substring; append ` i` for
|
|
109
|
+
case-insensitive (e.g. `[label="done" i]`).
|
|
110
|
+
- `A B` — descendant; the **last** component is the target, an ancestor predicate
|
|
111
|
+
scopes it (`TaskItem[title="{{title}}"] TaskToggle`). There is **no `>`** child
|
|
112
|
+
combinator — use whitespace.
|
|
113
|
+
- `Component#N` — 0-based occurrence when several match (weak fallback; prefer a
|
|
114
|
+
distinguishing property).
|
|
115
|
+
- `{{param}}` interpolates a tool param into any query/value/url.
|
|
116
|
+
|
|
117
|
+
Every property you filter on or read must be **declared in the corpus**. If a
|
|
118
|
+
label is CSS-uppercased on screen but lowercase in the DOM text, match
|
|
119
|
+
case-insensitively with ` i`.
|
|
120
|
+
|
|
121
|
+
## Journeys → guidance (not execution)
|
|
122
|
+
|
|
123
|
+
```yaml
|
|
124
|
+
journeys:
|
|
125
|
+
- name: add_and_review
|
|
126
|
+
description: Add a task, then review the list.
|
|
127
|
+
steps:
|
|
128
|
+
- add_task # bare tool name
|
|
129
|
+
- tool: list_tasks # or a mapping with a reason
|
|
130
|
+
reason: see the task you just added
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Each journey needs **≥2 steps** to produce guidance edges. A tool shared across
|
|
134
|
+
journeys accumulates the union of its successors. Journeys never navigate or run
|
|
135
|
+
anything — they only shape the breadcrumbs in results.
|
|
136
|
+
|
|
137
|
+
## Worked example (a task-list app)
|
|
138
|
+
|
|
139
|
+
```yaml
|
|
140
|
+
version: 1
|
|
141
|
+
name: tasks
|
|
142
|
+
corpus: ./.sightmap
|
|
143
|
+
tools:
|
|
144
|
+
- name: list_tasks
|
|
145
|
+
description: List the current tasks and whether each is done.
|
|
146
|
+
ensure_view: Home
|
|
147
|
+
returns:
|
|
148
|
+
description: The task rows (title + done-state).
|
|
149
|
+
list:
|
|
150
|
+
rows: TaskItem
|
|
151
|
+
fields:
|
|
152
|
+
title: title
|
|
153
|
+
done: done
|
|
154
|
+
|
|
155
|
+
- name: add_task
|
|
156
|
+
description: Add a task to the list.
|
|
157
|
+
ensure_view: Home
|
|
158
|
+
params:
|
|
159
|
+
- name: title
|
|
160
|
+
type: string
|
|
161
|
+
required: true
|
|
162
|
+
description: The task title.
|
|
163
|
+
steps:
|
|
164
|
+
- fill: { query: NewTaskInput, value: "{{title}}" }
|
|
165
|
+
- click: { query: AddTaskButton }
|
|
166
|
+
- wait_for: { query: 'TaskItem[title="{{title}}"]' }
|
|
167
|
+
returns:
|
|
168
|
+
description: The title of the new task.
|
|
169
|
+
value: { query: 'TaskItem[title="{{title}}"]', property: title }
|
|
170
|
+
|
|
171
|
+
- name: complete_task
|
|
172
|
+
description: Mark a task done by clicking its toggle.
|
|
173
|
+
ensure_view: Home
|
|
174
|
+
params:
|
|
175
|
+
- name: title
|
|
176
|
+
type: string
|
|
177
|
+
required: true
|
|
178
|
+
description: The task to complete.
|
|
179
|
+
steps:
|
|
180
|
+
- click: { query: 'TaskItem[title="{{title}}"] TaskToggle' }
|
|
181
|
+
- wait_for: { query: 'TaskItem[title="{{title}}"] TaskToggle[label="Undo"]' }
|
|
182
|
+
returns:
|
|
183
|
+
value: { query: 'TaskItem[title="{{title}}"]', property: done }
|
|
184
|
+
|
|
185
|
+
journeys:
|
|
186
|
+
- name: add_and_review
|
|
187
|
+
description: Add a task, then review the list.
|
|
188
|
+
steps:
|
|
189
|
+
- add_task
|
|
190
|
+
- tool: list_tasks
|
|
191
|
+
reason: confirm the task you just added
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
## Build & fix
|
|
195
|
+
|
|
196
|
+
```sh
|
|
197
|
+
sightkick build <DIR> -o /tmp/x.ir.json # <DIR> holds webmcp.tools.yaml + the corpus
|
|
198
|
+
sightkick build <DIR> --verify # also checks returns extractors against captured
|
|
199
|
+
# view snapshots; warns on fields empty on every row
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
The compiler is your validator. Common diagnostics and fixes:
|
|
203
|
+
|
|
204
|
+
- **unresolved component / property / view** — the name isn't in the corpus (for
|
|
205
|
+
that view). `build` prints candidates; fix the query, or declare the
|
|
206
|
+
component/property in the corpus (sightmap-authoring). Remember property refs
|
|
207
|
+
resolve against the **row/target** component.
|
|
208
|
+
- **`returns has both value and list`** — pick one.
|
|
209
|
+
- **`live tool needs at least one step or a returns`** — add a step or a read.
|
|
210
|
+
- **`unrecognized step op`** / **`not a single-key mapping`** — each step is one
|
|
211
|
+
op key (`fill`/`click`/`wait_for`/`navigate`/`goto`) with its body.
|
|
212
|
+
- **`--verify` says a field resolves empty on every row** — the declared property
|
|
213
|
+
extracts nothing on the live DOM; fix the property's extractor in the corpus.
|
|
214
|
+
|
|
215
|
+
For `--verify` you need a captured snapshot of the view (`sightmap capture` /
|
|
216
|
+
`snapshot` in the sightmap-browser skill). Once `build` is clean, run the tools
|
|
217
|
+
on a live page with the **sightkick-debug** skill.
|
|
@@ -17,33 +17,40 @@ a page: `sightmap browser eval` injects the runtime bundle, then
|
|
|
17
17
|
|
|
18
18
|
## Prerequisites
|
|
19
19
|
|
|
20
|
-
This skill
|
|
21
|
-
|
|
22
|
-
|
|
20
|
+
This skill uses two CLIs: **`sightkick`** (to build the IR and emit the runtime
|
|
21
|
+
bundle) and **`sightmap`** (to drive the live browser session). Install whichever
|
|
22
|
+
isn't already on your PATH, plus the supporting sightmap skills:
|
|
23
23
|
|
|
24
24
|
```sh
|
|
25
|
+
npm i -g @sightmap/sightkick # the sightkick CLI (build + runtime + skills)
|
|
25
26
|
npx @sightmap/sightmap skills install # or: sightmap skills install (if already on PATH)
|
|
26
27
|
sightmap browser install # Chrome-for-Testing; needs >=152 for native document.modelContext
|
|
27
28
|
```
|
|
28
29
|
|
|
29
30
|
That installs the **`sightmap-browser`** skill (driving a live session) and
|
|
30
31
|
**`sightmap-authoring`** skill (building the `.sightmap/` corpus) alongside this
|
|
31
|
-
one — everything needed to build and test a `webmcp.tools.yaml`.
|
|
32
|
-
|
|
32
|
+
one — everything needed to build and test a `webmcp.tools.yaml`. No repo checkout
|
|
33
|
+
is required; `<CORPUS_DIR>` below is any directory holding a `webmcp.tools.yaml`
|
|
34
|
+
+ `.sightmap/` corpus.
|
|
35
|
+
|
|
36
|
+
To **write** the `webmcp.tools.yaml` (its tool/step/`returns`/`journeys` grammar),
|
|
37
|
+
see the **`sightkick-authoring`** skill — this skill assumes it already exists and
|
|
38
|
+
compiles.
|
|
33
39
|
|
|
34
40
|
## 1. Build the two artifacts
|
|
35
41
|
|
|
42
|
+
Both come straight from the installed `sightkick` CLI:
|
|
43
|
+
|
|
36
44
|
```sh
|
|
37
|
-
# The payload: the
|
|
38
|
-
|
|
45
|
+
# The payload: the runtime bundle (exposes window.__sightkick.load).
|
|
46
|
+
sightkick runtime -o /tmp/sightkick-runtime.js
|
|
39
47
|
|
|
40
48
|
# The IR for the corpus you're testing (any dir with webmcp.tools.yaml + .sightmap/).
|
|
41
|
-
|
|
42
|
-
# e.g. <CORPUS_DIR> = ../examples/search or ../../sites/netlify
|
|
49
|
+
sightkick build <CORPUS_DIR> -o /tmp/x.ir.json
|
|
43
50
|
```
|
|
44
51
|
|
|
45
|
-
Rebuild the IR whenever the corpus/manifest changes
|
|
46
|
-
|
|
52
|
+
Rebuild the IR whenever the corpus/manifest changes. The runtime bundle is
|
|
53
|
+
embedded in the CLI, so re-emit it after upgrading `sightkick`.
|
|
47
54
|
|
|
48
55
|
## 2. Pick a mode, start the session
|
|
49
56
|
|
|
@@ -61,13 +68,16 @@ sightmap browser start --detach --url <SITE_URL> --profile /tmp/sk-dbg
|
|
|
61
68
|
|
|
62
69
|
### Mode B — inspector/Gemini-driven (native WebMCP surface)
|
|
63
70
|
Turn on the blink flags so Chrome exposes the real `document.modelContext`, and
|
|
64
|
-
load the
|
|
65
|
-
|
|
66
|
-
|
|
71
|
+
load the **WebMCP inspector** (a drive-with-Gemini sidebar). Our tools register on
|
|
72
|
+
the native surface, so the inspector reads them like any site's own. The inspector
|
|
73
|
+
isn't shipped with the CLI — use the vendored copy in the sightkick repo
|
|
74
|
+
(`vendor/webmcp-tool/unpacked`, whose `NOTES.md` explains the flag/CfT-version
|
|
75
|
+
rationale) or install it from the Chrome Web Store. Point `<INSPECTOR_DIR>` at its
|
|
76
|
+
unpacked directory:
|
|
67
77
|
|
|
68
78
|
```sh
|
|
69
79
|
sightmap browser start --detach --url <SITE_URL> --profile /tmp/sk-dbg \
|
|
70
|
-
--extensions ~/.sightmap/extension
|
|
80
|
+
--extensions ~/.sightmap/extension,<INSPECTOR_DIR> \
|
|
71
81
|
--chrome-flag=--enable-blink-features=ModelContext,ModelContextTesting \
|
|
72
82
|
--chrome-flag=--enable-features=DevToolsWebMCPSupport
|
|
73
83
|
```
|
|
@@ -85,8 +95,7 @@ sightmap browser eval "typeof document.modelContext" # object
|
|
|
85
95
|
## 3. Inject the runtime + IR
|
|
86
96
|
|
|
87
97
|
```sh
|
|
88
|
-
|
|
89
|
-
sightmap browser eval "$(cat "$BUNDLE")" # sets window.__sightkick
|
|
98
|
+
sightmap browser eval "$(cat /tmp/sightkick-runtime.js)" # sets window.__sightkick
|
|
90
99
|
sightmap browser eval "window.__sightkick.load($(cat /tmp/x.ir.json))"
|
|
91
100
|
```
|
|
92
101
|
|
|
@@ -104,18 +113,27 @@ sleep 1; sightmap browser eval "window.__t" # -> ["tool_a","t
|
|
|
104
113
|
|
|
105
114
|
## 4. Drive the tools
|
|
106
115
|
|
|
107
|
-
### Mode A (
|
|
108
|
-
The
|
|
116
|
+
### Mode A (agent/scripted): drive via `window.__sightkick.call`
|
|
117
|
+
The **reliable** scripted entry point is `window.__sightkick.call(name, args)` — it
|
|
118
|
+
invokes a tool by name and resolves to the raw `ToolResult` directly, regardless
|
|
119
|
+
of whether the page's `document.modelContext` is our polyfill or a native surface:
|
|
109
120
|
|
|
110
121
|
```sh
|
|
111
|
-
sightmap browser eval "window.__r='RUN';
|
|
112
|
-
sleep 2; sightmap browser eval "window.__r" # the ToolResult JSON (ok/value/items/guidance)
|
|
122
|
+
sightmap browser eval "window.__r='RUN';window.__sightkick.call('add_task',{title:'Water the plants'}).then(function(r){window.__r=JSON.stringify(r)}).catch(function(e){window.__r='ERR '+e});'go'"
|
|
123
|
+
sleep 2; sightmap browser eval "window.__r" # the ToolResult JSON (ok/value/items/skipped/guidance)
|
|
113
124
|
```
|
|
114
125
|
|
|
115
|
-
The
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
126
|
+
The `ToolResult` carries `ok`, `value`/`items`, `skipped` (idempotency guard
|
|
127
|
+
hit), and `guidance` (next-step breadcrumbs). The sightkick repo's
|
|
128
|
+
`packages/runtime/eval/run.mjs` is a full scripted example of this loop.
|
|
129
|
+
|
|
130
|
+
> **Don't script `document.modelContext.executeTool` directly.** Its call shape
|
|
131
|
+
> differs by surface: the polyfill takes a bare `{name}`, but a **native**
|
|
132
|
+
> `document.modelContext` (present on Chrome ≥150 even with no blink flags) is
|
|
133
|
+
> stricter and rejects it with `Failed to parse input arguments`, and wraps the
|
|
134
|
+
> result in an envelope (`{content:[{type:'text',text:<ToolResult JSON>}]}`).
|
|
135
|
+
> `window.__sightkick.call` sidesteps both differences — prefer it for Mode A, and
|
|
136
|
+
> leave `executeTool` to the inspector in Mode B.
|
|
119
137
|
|
|
120
138
|
### Mode B (native): drive via the inspector
|
|
121
139
|
Open the inspector's sidebar and prompt Gemini — it enumerates
|