@k71n/agent-probe 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +324 -0
- package/dist/assets/playbook.md +113 -0
- package/dist/assets/skill/SKILL.md +32 -0
- package/dist/cleanup/cleanup-verify.js +233 -0
- package/dist/constants.js +55 -0
- package/dist/evidence/diff.js +0 -0
- package/dist/evidence/evidence-store.js +238 -0
- package/dist/evidence/query.js +97 -0
- package/dist/index.js +16 -0
- package/dist/ingest/contract.js +94 -0
- package/dist/ingest/ingest.js +211 -0
- package/dist/logger.js +18 -0
- package/dist/node-version.js +18 -0
- package/dist/server.js +90 -0
- package/dist/session/run-boundaries.js +44 -0
- package/dist/session/session-manager.js +354 -0
- package/dist/session/state-dir.js +26 -0
- package/dist/tools.js +242 -0
- package/package.json +38 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Catalinm
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,324 @@
|
|
|
1
|
+
# agent-probe
|
|
2
|
+
|
|
3
|
+
**Evidence-based, leave-no-trace debugging for AI coding agents — an MCP server that lets your agent place temporary probes in running code, capture runtime evidence across services, name the root cause, and verifiably clean up after itself.**
|
|
4
|
+
|
|
5
|
+
agent-probe is the local-dev counterpart to production AI debugging. Your agent (Claude Code, Cursor, or any MCP host) drives the whole loop: instrument → reproduce → analyze → remove → verify. Evidence lives in a per-session local SQLite file that is **destroyed when the session ends**. No SDK in your app, no accounts, no dashboards, and no network egress, ever.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Why agent-probe?
|
|
10
|
+
|
|
11
|
+
The hardest bugs are invisible from the code alone: a write succeeds, the dependent read shows nothing, and no error appears anywhere. Static analysis and grepping can't see runtime state — and `console.log` debugging by an agent leaves litter in your tree and noise in your terminal.
|
|
12
|
+
|
|
13
|
+
- **Probes are one-liners, not a library** — a self-contained, fire-and-forget HTTP POST to localhost, wrapped in marker comments. Your app gains zero dependencies.
|
|
14
|
+
- **Probing never perturbs the app** — the server replies 202 *before* validating anything, applies no backpressure, and a dead server changes nothing (proven by an explicit behavior-equivalence test).
|
|
15
|
+
- **Evidence is structured, not scrollback** — timelines across services, bounded queries, and a first-class *diff between a failing run and a working run* that surfaces the discriminating difference directly.
|
|
16
|
+
- **Cleanup is verified, never assumed** — the server scans your workspace itself and *refuses to close the session* while any probe marker remains. `git diff` ends empty.
|
|
17
|
+
- **Everything stays on your machine** — localhost-only ingestion (bound at the kernel level), ephemeral per-session storage, zero telemetry.
|
|
18
|
+
- **Language-agnostic by contract** — anything that can POST JSON conforms: JS/TS, Python, shell/curl, Go, SQL comments for markers, and more.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Features
|
|
23
|
+
|
|
24
|
+
### The debugging loop
|
|
25
|
+
- **Goal-scoped sessions** — `start_session(goal, workspace_root)` opens an isolated investigation with its own evidence store and ingestion port
|
|
26
|
+
- **Runs as first-class boundaries** — tag reproductions (`"buggy"`, `"clean"`), and events are attributed to the run they arrived in; out-of-run events are kept as `"unattributed"`, never lost
|
|
27
|
+
- **Cross-service timelines** — time-ordered events from every service in one view, with honest `seq_tied` flags when ordering rests on arrival rather than causality
|
|
28
|
+
- **Bounded span queries** — filter by run, probe, service, or time range; results are capped server-side with `truncated` + true `total`, so a context window holds hypotheses, not haystacks
|
|
29
|
+
- **Run diffing (the wedge)** — structured presence/absence, ordering inversions, and payload deltas between two runs
|
|
30
|
+
|
|
31
|
+
### Non-perturbation guarantees
|
|
32
|
+
- **202-before-validate ingestion** — a probe never waits on parsing, validation, or storage
|
|
33
|
+
- **Fire-and-forget idiom** — no `await`, no retries, no queues; errors are swallowed at the probe (`.catch(() => {})`)
|
|
34
|
+
- **Caps instead of pressure** — oversized payloads are truncated (event kept), events past the per-session cap are dropped with a warning; the app never feels any of it
|
|
35
|
+
- **Warnings surface on every tool response** — rejected events, truncations, and drops are reported to the agent; silence about dropped evidence would mislead the analysis
|
|
36
|
+
|
|
37
|
+
### Leave no trace
|
|
38
|
+
- **Marker convention** — every probe is wrapped in own-line comment pairs (`// agent-probe-begin p1` … `// agent-probe-end p1`) that survive formatters and are mechanically removable
|
|
39
|
+
- **Server-side cleanup verification** — `verify_cleanup` scans the workspace (git-aware; `node_modules/` and `.git/` always skipped) and returns exact file+line locations
|
|
40
|
+
- **The close gate** — `end_session` is refused with `MARKERS_REMAIN` while markers exist; the override is explicit, user-owned, and always reported in the result
|
|
41
|
+
- **Destructive close** — the session's SQLite file and lock are deleted; orphans from crashes are surfaced and disposed explicitly, never silently resumed
|
|
42
|
+
|
|
43
|
+
### Agent guidance, token-cheap
|
|
44
|
+
- **Pull-once playbook** — conventions, the wire contract, probe strategy, and the removal ritual ship as an MCP resource (`playbook://probes`) read once per session
|
|
45
|
+
- **Optional Agent Skill** — skill-capable hosts get progressive-disclosure triggering (~100 idle tokens); it routes to the playbook and is never required
|
|
46
|
+
|
|
47
|
+
---
|
|
48
|
+
|
|
49
|
+
## Quick start
|
|
50
|
+
|
|
51
|
+
### Install
|
|
52
|
+
|
|
53
|
+
One MCP config entry. Requires **Node >= 22.13** (the server tells you on stderr if yours is older — it uses the built-in `node:sqlite`).
|
|
54
|
+
|
|
55
|
+
**Claude Code**
|
|
56
|
+
|
|
57
|
+
```sh
|
|
58
|
+
claude mcp add agent-probe -- npx -y @k71n/agent-probe@latest
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
**Cursor** (`.cursor/mcp.json`)
|
|
62
|
+
|
|
63
|
+
```json
|
|
64
|
+
{
|
|
65
|
+
"mcpServers": {
|
|
66
|
+
"agent-probe": {
|
|
67
|
+
"command": "npx",
|
|
68
|
+
"args": ["-y", "@k71n/agent-probe@latest"]
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Note that `npx` caches versions: use `@latest` as above, or pin a version. The npm package is scoped (`@k71n/agent-probe`); the tool, bin, and probe markers are plain `agent-probe`.
|
|
75
|
+
|
|
76
|
+
### Try it: the golden demo
|
|
77
|
+
|
|
78
|
+
The repo ships a tiny two-layer app staging the classic silent failure — *the form saves, but the list never updates*. No errors anywhere.
|
|
79
|
+
|
|
80
|
+
```sh
|
|
81
|
+
node examples/golden-demo/api/server.mjs
|
|
82
|
+
# open http://localhost:4280 — save an entry, watch it never appear
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Then tell your agent:
|
|
86
|
+
|
|
87
|
+
> The form saves but the list never updates — find out why.
|
|
88
|
+
|
|
89
|
+
What happens next:
|
|
90
|
+
|
|
91
|
+
1. The agent starts a session and pulls the playbook once.
|
|
92
|
+
2. It inserts marker-wrapped probe lines on both sides of the data boundary (write path + read path).
|
|
93
|
+
3. You reproduce the bug twice — once failing, once working — while the agent captures both runs.
|
|
94
|
+
4. The timeline names the root cause: the entry was **written with `categoryId: null`** while the list **filtered on a category** — a silent field-name mismatch between frontend and backend.
|
|
95
|
+
5. The agent removes every probe, the server verifies the workspace is clean, and the session is destroyed with all its evidence. `git diff` is empty.
|
|
96
|
+
|
|
97
|
+
---
|
|
98
|
+
|
|
99
|
+
## Tool reference
|
|
100
|
+
|
|
101
|
+
The surface is deliberately frozen at **9 tools**.
|
|
102
|
+
|
|
103
|
+
| Tool | Arguments | Description |
|
|
104
|
+
|------|-----------|-------------|
|
|
105
|
+
| `start_session` | `goal`, `workspace_root`, `stale?` | Open a Debug Session scoped to a stated goal. Returns `session_id`, the ingestion `port`, the echoed `workspace_root`, and the playbook URI. Refused with `STALE_SESSION_EXISTS` if orphans exist (resolve with `stale: "dispose"` or `"keep"`), or `INSTANCE_CONFLICT` if another live process holds the workspace. |
|
|
106
|
+
| `start_run` | `tag?` | Arm a Run while the user reproduces the flow. With host elicitation support, waits for the user's confirmation and returns the closed run; otherwise returns `status: "open"` and `end_run` closes it. |
|
|
107
|
+
| `end_run` | `tag?` | Close the open Run (a tag here overwrites one set at start). |
|
|
108
|
+
| `list_runs` | — | Every run with tag, boundaries, and event count. |
|
|
109
|
+
| `get_timeline` | `run`, `limit?` | Time-ordered events across all services for a run (or `"unattributed"`). Same-millisecond events are flagged `seq_tied`. |
|
|
110
|
+
| `get_span` | `run?`, `probe?`, `service?`, `from?`, `to?`, `limit?` | Bounded slice by any filter combination. |
|
|
111
|
+
| `diff_runs` | `a`, `b` | Structured differences between two runs: presence/absence, ordering changes, payload deltas. |
|
|
112
|
+
| `verify_cleanup` | — | Server-side scan of the workspace for residual probe markers; returns exact locations and orphan (unpaired) markers separately. |
|
|
113
|
+
| `end_session` | `override?` | Destroy the session and all stored evidence. Refused with `MARKERS_REMAIN` while markers remain; `override: true` is the user's call and is always reported in the result. |
|
|
114
|
+
|
|
115
|
+
### Response envelope
|
|
116
|
+
|
|
117
|
+
Every successful response is the same shape — even single-item ones:
|
|
118
|
+
|
|
119
|
+
```json
|
|
120
|
+
{ "data": { ... }, "truncated": false, "total": 1, "warnings": ["..."] }
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`truncated`/`total` make result capping honest; `warnings` (when present) carry ingestion rejections, payload truncations, and drops.
|
|
124
|
+
|
|
125
|
+
### Errors
|
|
126
|
+
|
|
127
|
+
Every tool error is `{ code, message, hint }` with `code` drawn from a closed enum:
|
|
128
|
+
|
|
129
|
+
`NO_ACTIVE_SESSION` · `SESSION_ALREADY_ACTIVE` · `STALE_SESSION_EXISTS` · `INSTANCE_CONFLICT` · `MARKERS_REMAIN` · `RUN_NOT_FOUND` · `NO_ACTIVE_RUN` · `INVALID_STATE`
|
|
130
|
+
|
|
131
|
+
The `hint` always says what to do next — errors are written for agents.
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## The probe contract
|
|
136
|
+
|
|
137
|
+
Probes POST JSON to `http://127.0.0.1:<port>/events` (the port comes from `start_session`). snake_case on the wire; optional fields are **omitted** when absent, never null.
|
|
138
|
+
|
|
139
|
+
| Field | Type | Required | Meaning |
|
|
140
|
+
|-------|------|----------|---------|
|
|
141
|
+
| `session_id` | string | yes | from the `start_session` response |
|
|
142
|
+
| `probe_id` | string | yes | matches the probe's marker id |
|
|
143
|
+
| `service` | string | yes | which service emitted it (`"web"`, `"api"`, …) |
|
|
144
|
+
| `file` | string | yes | source file the probe lives in |
|
|
145
|
+
| `line` | int | yes | source line |
|
|
146
|
+
| `ts_probe` | int | yes | epoch **milliseconds** at emission |
|
|
147
|
+
| `payload` | JSON | yes | any JSON value; keep it ≤ 64 KiB (truncated beyond) |
|
|
148
|
+
| `trace_id` | string | no | correlation headroom (W3C-aligned, free-form) |
|
|
149
|
+
| `parent_id` | string | no | correlation headroom (W3C-aligned, free-form) |
|
|
150
|
+
|
|
151
|
+
The server assigns `ts_server` and a monotonic `seq` on arrival. Unknown extra keys are ignored — strictness would break language-agnosticism. No `Content-Type` required.
|
|
152
|
+
|
|
153
|
+
### The idiom (JS/TS)
|
|
154
|
+
|
|
155
|
+
```ts
|
|
156
|
+
// agent-probe-begin p1
|
|
157
|
+
fetch(`http://127.0.0.1:${PORT}/events`, { method: "POST", body: JSON.stringify({ session_id: SID, probe_id: "p1", service: "api", file: "list.ts", line: 42, ts_probe: Date.now(), payload: { categoryId, rowCount } }) }).catch(() => {});
|
|
158
|
+
// agent-probe-end p1
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Markers are **own-line** comment pairs (they survive Prettier/ESLint), both carrying the probe id; the comment leader adapts to the language (`//`, `#`, `--`, `<!-- -->`). Anything that can POST JSON conforms — shell:
|
|
162
|
+
|
|
163
|
+
```sh
|
|
164
|
+
curl -s -X POST http://127.0.0.1:$PORT/events -d "{\"session_id\":\"$SID\",\"probe_id\":\"p3\",\"service\":\"worker\",\"file\":\"job.sh\",\"line\":7,\"ts_probe\":$(date +%s%3N),\"payload\":{\"jobId\":\"$JOB\"}}" >/dev/null 2>&1 &
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
The full conventions — probe strategy for silent write/read failures, the removal ritual, run protocol — live in the bundled playbook (`playbook://probes`), which agents pull once per session.
|
|
168
|
+
|
|
169
|
+
### Limits (enforced, not asserted)
|
|
170
|
+
|
|
171
|
+
| Cap | Value | Behavior past it |
|
|
172
|
+
|-----|-------|------------------|
|
|
173
|
+
| Payload size | 64 KiB | truncated, event kept, warning emitted |
|
|
174
|
+
| Events per session | 100,000 | dropped with warning |
|
|
175
|
+
| Events per query result | 500 | clamped; `truncated: true` + true `total` |
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
## Architecture
|
|
180
|
+
|
|
181
|
+
```
|
|
182
|
+
MCP host (Claude Code, Cursor, …)
|
|
183
|
+
|
|
|
184
|
+
stdio (JSON-RPC) your services
|
|
185
|
+
| (web, api, worker)
|
|
186
|
+
+---------v----------+ |
|
|
187
|
+
| MCP server | marker-wrapped
|
|
188
|
+
| 9 tools + the | one-line probes
|
|
189
|
+
| playbook resource | |
|
|
190
|
+
+---------+----------+ POST /events (fire-and-forget)
|
|
191
|
+
| |
|
|
192
|
+
+-------------+--------------+ +-------v--------+
|
|
193
|
+
| SessionManager | | Ingest listener |
|
|
194
|
+
| state machine, lockfiles, |<--+ 127.0.0.1:ephem |
|
|
195
|
+
| orphan recovery, close | | 202-before- |
|
|
196
|
+
| gate (MARKERS_REMAIN) | | validate |
|
|
197
|
+
+------+--------------+------+ +----------------+
|
|
198
|
+
| |
|
|
199
|
+
+---------v---+ +------v----------+
|
|
200
|
+
| Evidence | | Cleanup verify |
|
|
201
|
+
| store + query| | (workspace scan,|
|
|
202
|
+
| + run diff | | marker pairing)|
|
|
203
|
+
| (SQLite, | +-----------------+
|
|
204
|
+
| per-session,|
|
|
205
|
+
| destroyed |
|
|
206
|
+
| on close) |
|
|
207
|
+
+-------------+
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
### How a session works
|
|
211
|
+
|
|
212
|
+
1. **`start_session`** validates the workspace, takes a PID lockfile (one active session per workspace), creates a per-session SQLite file, and returns the ingestion port + playbook URI.
|
|
213
|
+
2. **The agent instruments** — it inserts whole probe lines into your code, baking the session id and port in as literals. The server never edits your files.
|
|
214
|
+
3. **Ingestion** replies `202` immediately, then parses, validates against the contract, applies caps, and flushes to SQLite once per event-loop tick in a single transaction. Events arriving while a run is open are attributed to it.
|
|
215
|
+
4. **Analysis** runs entirely over the bounded query tools — one shared ordering comparator (`ts_probe`, then `seq`) backs the timeline, spans, and the run diff.
|
|
216
|
+
5. **Cleanup** is a trust split: the *server* reports exact marker locations (fresh scan every time), the *agent* deletes the marked ranges, the server re-verifies. `end_session` only succeeds clean — then closes and unlinks everything.
|
|
217
|
+
|
|
218
|
+
### Crash safety
|
|
219
|
+
|
|
220
|
+
There is no state outside the session's SQLite file and its lockfile. Sudden death leaves only an orphan `.db`; the next `start_session` surfaces it (`STALE_SESSION_EXISTS`, with the goal of the lost investigation in the hint) and the user decides: dispose or keep. Orphans are never silently resumed.
|
|
221
|
+
|
|
222
|
+
### State location
|
|
223
|
+
|
|
224
|
+
Per-user, per-platform (override with `XDG_STATE_HOME`):
|
|
225
|
+
|
|
226
|
+
| Platform | Path |
|
|
227
|
+
|----------|------|
|
|
228
|
+
| Linux | `~/.local/state/agent-probe/` |
|
|
229
|
+
| macOS | `~/Library/Application Support/agent-probe/` |
|
|
230
|
+
| Windows | `%LOCALAPPDATA%/agent-probe/` |
|
|
231
|
+
|
|
232
|
+
Inside: `sessions/<session-id>.db` (deleted on close) and `locks/` (PID lockfiles, reaped when stale).
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
## Development
|
|
237
|
+
|
|
238
|
+
### Setup
|
|
239
|
+
|
|
240
|
+
```sh
|
|
241
|
+
git clone <this repo>
|
|
242
|
+
cd agent-probe
|
|
243
|
+
npm ci
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
### Scripts
|
|
247
|
+
|
|
248
|
+
| Command | What it does |
|
|
249
|
+
|---------|--------------|
|
|
250
|
+
| `npm run dev` | Run the server from source (tsx, stdio) |
|
|
251
|
+
| `npm test` | Full vitest suite (unit + integration) |
|
|
252
|
+
| `npm run test:watch` | Watch mode |
|
|
253
|
+
| `npm run typecheck` | `tsc --noEmit` (strict, `noUncheckedIndexedAccess`) |
|
|
254
|
+
| `npm run lint` | ESLint over the solution |
|
|
255
|
+
| `npm run build` | `tsc` + inject prose assets into `dist/` |
|
|
256
|
+
|
|
257
|
+
### Repository layout
|
|
258
|
+
|
|
259
|
+
```
|
|
260
|
+
src/
|
|
261
|
+
index.ts entry point (Node version gate, then stdio server)
|
|
262
|
+
server.ts MCP server wiring + the playbook resource
|
|
263
|
+
tools.ts the 9 tool registrations (the ONE place tools exist)
|
|
264
|
+
constants.ts single source for names, limits, error codes
|
|
265
|
+
logger.ts stderr-only logging (stdout belongs to MCP framing)
|
|
266
|
+
session/ session state machine, run boundaries, lockfiles, state dirs
|
|
267
|
+
ingest/ POST /events listener, wire contract, caps, warnings
|
|
268
|
+
evidence/ per-session SQLite store, timeline/span queries, run diff
|
|
269
|
+
cleanup/ marker scanning, pairing, removal-range derivation
|
|
270
|
+
integration/ cross-module flow tests + the golden-demo fixture
|
|
271
|
+
assets/
|
|
272
|
+
playbook.md the agent playbook (template; injected at build)
|
|
273
|
+
skill/SKILL.md optional Agent Skill (template; injected at build)
|
|
274
|
+
examples/
|
|
275
|
+
golden-demo/ runnable demo app with the staged silent-failure bug
|
|
276
|
+
build-playbook.mjs build-time placeholder injection (+ drift guards)
|
|
277
|
+
.github/workflows/ ci.yml (tests, greps, packaging gates), release.yml (npm publish)
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
### Design rules the code enforces
|
|
281
|
+
|
|
282
|
+
- **stdout is sacred** — it carries MCP framing only; all diagnostics go to stderr. Lint + CI grep enforce it.
|
|
283
|
+
- **No egress** — the only networking is the localhost ingestion listener; net-client imports are lint-banned, URLs are CI-grepped.
|
|
284
|
+
- **Frozen runtime deps** — `@modelcontextprotocol/sdk` + `zod`, nothing else.
|
|
285
|
+
- **Single-sourced names** — the package name and marker token live in `src/constants.ts`; prose templates use placeholders injected at build, and CI fails on survivors.
|
|
286
|
+
- **Tools never touch SQL** — strict layering: tools → SessionManager → EvidenceStore.
|
|
287
|
+
|
|
288
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for the full list and the PR process.
|
|
289
|
+
|
|
290
|
+
### Releases
|
|
291
|
+
|
|
292
|
+
Pushing a `v*` tag runs `release.yml`: typecheck → tests → lint → build → pack-install smoke (tarball contents + the installed bin must boot) → `npm publish` via OIDC trusted publishing (no tokens, provenance attached automatically).
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
## 3-minute demo
|
|
297
|
+
|
|
298
|
+
<!-- DEMO_RECORDING_URL -->
|
|
299
|
+
*Recording coming with launch.*
|
|
300
|
+
|
|
301
|
+
---
|
|
302
|
+
|
|
303
|
+
## Known limitations
|
|
304
|
+
|
|
305
|
+
Honest ones, each with its mitigation:
|
|
306
|
+
|
|
307
|
+
1. **Timing-sensitive bugs may shift under instrumentation.** Behavior equivalence is proven for application-visible *outputs* (see `src/integration/behavior-equivalence.test.ts`), not microsecond timing. If your bug is a sub-millisecond race, probes can move it.
|
|
308
|
+
2. **Git hygiene: don't commit probed code mid-session.** Markers are plain greppable comments and `verify_cleanup` is the gate — but nothing stops a `git commit` while probes are in place. Finish the loop before committing.
|
|
309
|
+
3. **Container/WSL2 clock skew can disorder cross-service timelines.** `ts_probe` is each app's own clock. Keep services on one clock domain, or read `seq_tied` flags honestly — arrival order is not causality.
|
|
310
|
+
4. **`workspace_root` is agent-supplied and trusted (v1).** The cleanup scan verifies everything *within* it; it does not verify that it *is* your project root. Glance at the echoed path when the session starts.
|
|
311
|
+
5. **One active session per workspace (v1).** A second session against the same workspace is refused while the first holds the lock; stale sessions from crashes are surfaced and disposed explicitly.
|
|
312
|
+
6. **Non-git workspaces over-scan.** Without git's ignore rules the cleanup scan walks everything under `workspace_root`, so stale markers in build output may surface. They're real markers — delete the build artifacts or rebuild.
|
|
313
|
+
|
|
314
|
+
---
|
|
315
|
+
|
|
316
|
+
## Contributing
|
|
317
|
+
|
|
318
|
+
Contributions are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md) for setup, the project's invariants, and the PR process. All participants are expected to follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|
|
319
|
+
|
|
320
|
+
Found a security issue? Please report it privately — see [SECURITY.md](SECURITY.md).
|
|
321
|
+
|
|
322
|
+
## License
|
|
323
|
+
|
|
324
|
+
[MIT](LICENSE)
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# Probe Playbook
|
|
2
|
+
|
|
3
|
+
You are the debugger. The server stores and queries probe evidence; YOU place
|
|
4
|
+
and remove probes. Read this once per session — everything you need is here.
|
|
5
|
+
Never paste this content into tool calls; act on it.
|
|
6
|
+
|
|
7
|
+
## Session flow
|
|
8
|
+
|
|
9
|
+
1. `start_session(goal, workspace_root)` → returns `session_id`, `port`, echoed `workspace_root`. Bake both values into every probe.
|
|
10
|
+
2. Pull this playbook (you just did — don't pull it again).
|
|
11
|
+
3. Instrument: place probes in the code under `workspace_root` (conventions below).
|
|
12
|
+
4. `start_run(tag)` → the user reproduces the bug → the run closes (user confirmation, or call `end_run`).
|
|
13
|
+
5. Analyze: `list_runs`, `get_timeline`, `get_span`, `diff_runs`.
|
|
14
|
+
6. Remove every probe, then `verify_cleanup` until `clean: true` (ritual below).
|
|
15
|
+
7. `end_session` — destroys all stored evidence. Refused with `MARKERS_REMAIN` while markers remain.
|
|
16
|
+
|
|
17
|
+
## Probe conventions — markers are the ground truth
|
|
18
|
+
|
|
19
|
+
Every probe is wrapped in an own-line comment pair carrying the same probe-id:
|
|
20
|
+
|
|
21
|
+
```ts
|
|
22
|
+
// agent-probe-begin p1
|
|
23
|
+
fetch(`http://127.0.0.1:${PORT}/events`, { method: "POST", body: JSON.stringify({ session_id: SID, probe_id: "p1", service: "api", file: "list.ts", line: 42, ts_probe: Date.now(), payload: { categoryId, rowCount } }) }).catch(() => {});
|
|
24
|
+
// agent-probe-end p1
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Rules — each one exists because something breaks without it:
|
|
28
|
+
|
|
29
|
+
- Markers are OWN-LINE comments (never trailing a code line) wrapping COMPLETE statements. Own-line is what survives Prettier/ESLint reformatting.
|
|
30
|
+
- `begin` AND `end` both carry the probe-id. Unique probe-id per probe.
|
|
31
|
+
- The comment leader adapts to the language: `// agent-probe-begin p2` (JS/TS/Go), `# agent-probe-begin p2` (Python/shell), `-- agent-probe-begin p2` (SQL), `<!-- agent-probe-begin p2 -->` (HTML).
|
|
32
|
+
- Probes go ONLY inside `workspace_root` — the server's cleanup verifier scans nothing else, so a probe outside it can never be verified removed.
|
|
33
|
+
- Never edit existing lines to insert a probe; INSERT whole new lines. Removal then restores the file byte-for-byte.
|
|
34
|
+
|
|
35
|
+
## Event contract v1 (the wire shape)
|
|
36
|
+
|
|
37
|
+
`POST http://127.0.0.1:<port>/events` with a JSON body. snake_case on the wire. Optional fields are OMITTED when absent — never null.
|
|
38
|
+
|
|
39
|
+
| field | type | required | meaning |
|
|
40
|
+
|------------|--------|----------|--------------------------------------------------|
|
|
41
|
+
| session_id | string | yes | from the start_session response |
|
|
42
|
+
| probe_id | string | yes | matches the probe's marker id |
|
|
43
|
+
| service | string | yes | which service emitted it ("web", "api", …) |
|
|
44
|
+
| file | string | yes | source file the probe lives in |
|
|
45
|
+
| line | int | yes | source line |
|
|
46
|
+
| ts_probe | int | yes | epoch MILLISECONDS at emission |
|
|
47
|
+
| payload | JSON | yes | any JSON value; keep it ≤ 64 KiB (truncated beyond) |
|
|
48
|
+
| trace_id | string | no | W3C Trace Context headroom |
|
|
49
|
+
| parent_id | string | no | W3C Trace Context headroom |
|
|
50
|
+
|
|
51
|
+
The server assigns `ts_server` and a monotonic `seq` on ingestion. Epoch-ms is a one-liner everywhere: `Date.now()` (JS/TS), `int(time.time() * 1000)` (Python), `$(date +%s%3N)` (shell), `time.Now().UnixMilli()` (Go).
|
|
52
|
+
|
|
53
|
+
## Fire-and-forget — never perturb the app
|
|
54
|
+
|
|
55
|
+
The probe idiom, verbatim — non-awaited, error-swallowed:
|
|
56
|
+
|
|
57
|
+
```ts
|
|
58
|
+
fetch(`http://127.0.0.1:${PORT}/events`, { method: "POST", body: JSON.stringify({ ...event }) }).catch(() => {});
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
- NO `await`, NO retries, NO timeouts-with-retry, NO queues.
|
|
62
|
+
- NO shared logging helpers or wrapper functions — a probe is a self-contained one-liner, or it isn't mechanically removable and isn't language-agnostic.
|
|
63
|
+
- Browser code near a page unload/navigation: add `keepalive: true` so the request survives.
|
|
64
|
+
- `PORT` and `SID` come from the `start_session` response — bake the literal values in.
|
|
65
|
+
|
|
66
|
+
Other languages, same contract (anything that can POST JSON conforms; `Content-Type` not required):
|
|
67
|
+
|
|
68
|
+
```sh
|
|
69
|
+
curl -s -X POST http://127.0.0.1:$PORT/events -d "{\"session_id\":\"$SID\",\"probe_id\":\"p3\",\"service\":\"worker\",\"file\":\"job.sh\",\"line\":7,\"ts_probe\":$(date +%s%3N),\"payload\":{\"jobId\":\"$JOB\"}}" >/dev/null 2>&1 &
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
```python
|
|
73
|
+
# agent-probe-begin p4
|
|
74
|
+
import json, urllib.request
|
|
75
|
+
try: urllib.request.urlopen(urllib.request.Request("http://127.0.0.1:PORT/events", data=json.dumps({"session_id": SID, "probe_id": "p4", "service": "api", "file": "views.py", "line": 88, "ts_probe": int(time.time() * 1000), "payload": {"query": str(qs.query)}}).encode()), timeout=1)
|
|
76
|
+
except Exception: pass
|
|
77
|
+
# agent-probe-end p4
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## Probe strategy: the silent write-path/read-path failure
|
|
81
|
+
|
|
82
|
+
The classic: a write succeeds, the dependent read shows nothing, no error anywhere. Strategy:
|
|
83
|
+
|
|
84
|
+
- Probe BOTH sides of the data boundary. Write side: what was persisted, with which discriminating values (ids, foreign keys, flags, tenant/category fields). Read side: what the query filtered on, and what came back (row count, first row).
|
|
85
|
+
- Put the DISCRIMINATING fields in `payload` — the fields that decide matching (the category id written vs the category id queried), not entire entities.
|
|
86
|
+
- Capture two runs: `start_run(tag: "buggy")` for the failing path, `start_run(tag: "clean")` for a working comparison (a case that does show up).
|
|
87
|
+
- `diff_runs(a, b)` then surfaces presence/absence, ordering changes, and payload deltas directly — you rarely need to read either timeline whole.
|
|
88
|
+
|
|
89
|
+
## Run protocol
|
|
90
|
+
|
|
91
|
+
- `start_run` may wait for the user's confirmation (the host shows a prompt: reproduce now, then confirm) — or it returns `status: "open"` and you call `end_run` when the user says the reproduction is done.
|
|
92
|
+
- Tag runs (`"buggy"`, `"clean"`); a tag at `end_run` overwrites the start tag.
|
|
93
|
+
- Events arriving OUTSIDE any run are kept, not lost: query them with `run: "unattributed"`.
|
|
94
|
+
|
|
95
|
+
## Analysis
|
|
96
|
+
|
|
97
|
+
- `list_runs` — every run with tag, boundaries, event count.
|
|
98
|
+
- `get_timeline(run)` — time-ordered events across all services. Events flagged `seq_tied` arrived inside the same millisecond: their order rests on arrival, NOT on causality. Never read jitter as causality.
|
|
99
|
+
- `get_span(filters)` — bounded slice by run/probe/service/time range. Results are capped; `truncated: true` + `total` tell you when to narrow the filter.
|
|
100
|
+
- `diff_runs(a, b)` — the structured account of differences between two runs.
|
|
101
|
+
|
|
102
|
+
## Cleanup ritual — leave no trace, verified
|
|
103
|
+
|
|
104
|
+
1. Run `verify_cleanup` — the server scans `workspace_root` itself and returns the EXACT file+line of every marker. Trust this list, not your memory.
|
|
105
|
+
2. Per file, delete each marked range INCLUSIVE (begin line through end line, whole lines — never blank them), working bottom-up (highest start line first) so earlier line numbers stay valid.
|
|
106
|
+
3. Re-run `verify_cleanup`. Locations from before any edit are stale — always re-scan, then remove. Repeat until `clean: true`.
|
|
107
|
+
4. Orphans (a begin without its end, or vice versa) are reported separately — fix them by hand; never guess a deletion range from an orphan.
|
|
108
|
+
5. `end_session`. If it returns `MARKERS_REMAIN`, markers still exist — go to 1. `override: true` exists, but it is the USER's decision: ask first, and tell them the override (with the residual locations from the result) — it is always reported, never silent. Evidence is destroyed either way.
|
|
109
|
+
|
|
110
|
+
## Observability
|
|
111
|
+
|
|
112
|
+
- Any tool response may carry `warnings`: rejected events, payload truncations, drops past caps, stale sessions. Read them — silence about dropped evidence would mislead your analysis.
|
|
113
|
+
- No active session → posted events are rejected. Events with a stale `session_id` (from a previous session's probes) are rejected too: stale probes can't pollute a new investigation.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: agent-probe
|
|
3
|
+
description: Drives evidence-based debugging of multi-service applications through the agent-probe MCP server - places temporary probes, captures runtime evidence across services, compares runs, and verifies clean removal. Use when the user says "debug this", asks why something happens at runtime, or hunts a bug spanning more than one service or process.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# agent-probe — evidence-based multi-service debugging
|
|
7
|
+
|
|
8
|
+
## When to use (and when not to)
|
|
9
|
+
|
|
10
|
+
Use this skill when a bug's cause is invisible from the code alone: runtime behavior across services, silent failures, "the write succeeds but the read shows nothing". Do NOT use it for static questions (type errors, syntax, refactoring) or when a single stack trace already explains the failure.
|
|
11
|
+
|
|
12
|
+
## The session loop
|
|
13
|
+
|
|
14
|
+
1. `start_session(goal, workspace_root)` — note the returned `session_id` and `port`.
|
|
15
|
+
2. Pull the playbook resource `playbook://probes` ONCE — it carries every convention you need.
|
|
16
|
+
3. Instrument: place probes in the code, following the playbook's conventions exactly.
|
|
17
|
+
4. `start_run(tag)` → the user reproduces the bug → the run closes (user confirmation, or `end_run`). Capture a failing run AND a working comparison run when possible.
|
|
18
|
+
5. Analyze: `list_runs`, `get_timeline`, `get_span`, `diff_runs`.
|
|
19
|
+
6. Remove every probe, then `verify_cleanup` until it reports clean.
|
|
20
|
+
7. `end_session` — destroys all stored evidence.
|
|
21
|
+
|
|
22
|
+
## Standing disciplines
|
|
23
|
+
|
|
24
|
+
- A probe is a self-contained fire-and-forget one-liner — no shared helpers, no awaits, no retries.
|
|
25
|
+
- Every probe is wrapped in an own-line marker comment pair carrying a unique probe id.
|
|
26
|
+
- Probes go ONLY inside `workspace_root` — nothing outside it can ever be verified removed.
|
|
27
|
+
- Cleanup is verified, never assumed — re-scan until the server reports clean.
|
|
28
|
+
- Ask the user before any `override`, and report it — it destroys evidence while markers remain.
|
|
29
|
+
|
|
30
|
+
## Authority
|
|
31
|
+
|
|
32
|
+
The playbook resource (`playbook://probes`) is the source of truth for conventions, the event contract, and removal mechanics — pull it before placing any probe. If anything here seems to conflict with it, the playbook wins.
|