@reticlehq/server 2.6.0 → 2.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -5
- package/SKILL.md +251 -33
- package/dist/bridge/bridge.d.ts +6 -1
- package/dist/bridge/bridge.js +27 -4
- package/dist/capsule/causal-summary.d.ts +20 -0
- package/dist/capsule/causal-summary.js +31 -6
- package/dist/capsule/divergence.js +7 -1
- package/dist/cli/chromium-hint.d.ts +68 -0
- package/dist/cli/chromium-hint.js +145 -0
- package/dist/cli/cli-doctor.js +31 -36
- package/dist/cli/cli-kill.d.ts +58 -0
- package/dist/cli/cli-kill.js +162 -0
- package/dist/cli/cli-launch.d.ts +18 -0
- package/dist/cli/cli-launch.js +32 -1
- package/dist/cli/cli-parse.d.ts +16 -1
- package/dist/cli/cli-parse.js +32 -9
- package/dist/cli/cli-port.d.ts +8 -0
- package/dist/cli/cli-port.js +36 -1
- package/dist/cli/cli-verify.js +2 -2
- package/dist/cli/doctor-daemon-line.js +10 -1
- package/dist/cli/doctor-sessions-line.d.ts +12 -0
- package/dist/cli/doctor-sessions-line.js +66 -0
- package/dist/cli/port-holder.d.ts +10 -0
- package/dist/cli/port-holder.js +26 -2
- package/dist/cli.js +139 -23
- package/dist/crawl/crawl-tools.js +12 -2
- package/dist/crawl/crawl.d.ts +13 -0
- package/dist/crawl/crawl.js +50 -2
- package/dist/daemon/daemon-resilience.js +2 -2
- package/dist/daemon/loopback-alias.d.ts +32 -0
- package/dist/daemon/loopback-alias.js +80 -0
- package/dist/events/contradictions.d.ts +14 -0
- package/dist/events/contradictions.js +26 -17
- package/dist/events/predicate-eval.d.ts +26 -3
- package/dist/events/predicate-eval.js +284 -95
- package/dist/events/predicate-parse.d.ts +1 -1
- package/dist/events/predicate-parse.js +18 -4
- package/dist/events/predicate.js +170 -18
- package/dist/events/ring-buffer.d.ts +8 -0
- package/dist/events/ring-buffer.js +34 -1
- package/dist/flows/flow-classify.js +1 -1
- package/dist/flows/flow-tools.js +12 -4
- package/dist/flows/heal-run.js +1 -1
- package/dist/honesty/blind-spots.js +6 -2
- package/dist/honesty/honesty.d.ts +11 -0
- package/dist/honesty/honesty.js +9 -1
- package/dist/honesty/unsettled.d.ts +40 -0
- package/dist/honesty/unsettled.js +78 -0
- package/dist/honesty/verified.d.ts +7 -0
- package/dist/honesty/verified.js +27 -4
- package/dist/http-server.js +11 -1
- package/dist/index.js +29 -2
- package/dist/init/agent-rules.d.ts +16 -0
- package/dist/init/agent-rules.js +96 -17
- package/dist/init/astro-patch.d.ts +2 -1
- package/dist/init/astro-patch.js +11 -5
- package/dist/init/capabilities.d.ts +22 -0
- package/dist/init/capabilities.js +76 -0
- package/dist/init/confirm.d.ts +96 -0
- package/dist/init/confirm.js +138 -0
- package/dist/init/cra.js +3 -2
- package/dist/init/detect.d.ts +8 -0
- package/dist/init/detect.js +15 -0
- package/dist/init/non-js-project.d.ts +27 -0
- package/dist/init/non-js-project.js +84 -0
- package/dist/init/plan-framework.d.ts +9 -0
- package/dist/init/plan-framework.js +72 -16
- package/dist/init/plan.d.ts +40 -4
- package/dist/init/plan.js +183 -43
- package/dist/init/run.d.ts +24 -2
- package/dist/init/run.js +181 -46
- package/dist/init/slash-command.d.ts +1 -1
- package/dist/init/slash-command.js +46 -6
- package/dist/init/snippets.d.ts +64 -5
- package/dist/init/snippets.js +220 -31
- package/dist/init/vite-config.js +29 -3
- package/dist/input/network-mock-tools.d.ts +19 -0
- package/dist/input/network-mock-tools.js +29 -4
- package/dist/license/license.js +1 -1
- package/dist/mcp/mcp-proxy.d.ts +69 -3
- package/dist/mcp/mcp-proxy.js +215 -23
- package/dist/mcp/mcp.js +58 -7
- package/dist/mcp/proxy-lifecycle.d.ts +20 -1
- package/dist/mcp/proxy-lifecycle.js +22 -1
- package/dist/pool/browser-pool.d.ts +22 -2
- package/dist/pool/browser-pool.js +44 -4
- package/dist/pool/playwright-launcher.js +10 -2
- package/dist/project/learned-routes.d.ts +24 -0
- package/dist/project/learned-routes.js +63 -0
- package/dist/project/project-store.d.ts +6 -0
- package/dist/project/project-store.js +25 -1
- package/dist/session/live-control-tools.js +39 -2
- package/dist/session/live-control.d.ts +7 -9
- package/dist/session/live-control.js +34 -1
- package/dist/session/no-session-diagnosis.d.ts +23 -4
- package/dist/session/no-session-diagnosis.js +128 -44
- package/dist/session/no-session-watch.js +12 -0
- package/dist/session/ref-provenance.d.ts +54 -0
- package/dist/session/ref-provenance.js +61 -0
- package/dist/session/review-store.d.ts +19 -0
- package/dist/session/review-store.js +34 -5
- package/dist/session/session-lease.js +1 -1
- package/dist/session/session-manager.d.ts +9 -0
- package/dist/session/session-manager.js +11 -0
- package/dist/session/session.d.ts +8 -0
- package/dist/session/session.js +12 -0
- package/dist/status-payload.d.ts +13 -1
- package/dist/status-payload.js +6 -1
- package/dist/telemetry/app-instrumented.d.ts +37 -0
- package/dist/telemetry/app-instrumented.js +54 -0
- package/dist/telemetry/automation-hint.d.ts +17 -0
- package/dist/telemetry/automation-hint.js +71 -0
- package/dist/telemetry/bug-found.d.ts +0 -20
- package/dist/telemetry/bug-found.js +38 -21
- package/dist/telemetry/cli-telemetry.js +7 -1
- package/dist/telemetry/daemon-telemetry.d.ts +4 -4
- package/dist/telemetry/daemon-telemetry.js +8 -4
- package/dist/telemetry/feedback.js +1 -1
- package/dist/telemetry/init-telemetry.js +9 -1
- package/dist/telemetry/install-source.d.ts +36 -0
- package/dist/telemetry/install-source.js +40 -0
- package/dist/telemetry/run-telemetry.js +4 -1
- package/dist/telemetry/session-metrics.d.ts +3 -3
- package/dist/telemetry/session-metrics.js +9 -4
- package/dist/telemetry/telemetry.d.ts +7 -1
- package/dist/telemetry/telemetry.js +17 -2
- package/dist/telemetry/tool-refused.d.ts +19 -0
- package/dist/telemetry/tool-refused.js +55 -0
- package/dist/telemetry/verification-of.js +72 -8
- package/dist/temp-dir.d.ts +27 -0
- package/dist/temp-dir.js +46 -0
- package/dist/tools/act-tools.js +77 -11
- package/dist/tools/assert-grade.d.ts +1 -1
- package/dist/tools/assert-grade.js +27 -5
- package/dist/tools/assert-verdict.js +25 -1
- package/dist/tools/contract-tools.js +4 -0
- package/dist/tools/dynamic-tools.js +1 -1
- package/dist/tools/error-recovery.d.ts +10 -0
- package/dist/tools/error-recovery.js +70 -5
- package/dist/tools/feedback-invite.js +1 -1
- package/dist/tools/invoke-tool.js +149 -32
- package/dist/tools/lease-tools.d.ts +29 -1
- package/dist/tools/lease-tools.js +58 -5
- package/dist/tools/navigate-result.js +1 -1
- package/dist/tools/observe-tools.js +1 -1
- package/dist/tools/observed-nothing.js +1 -1
- package/dist/tools/react-grace.d.ts +48 -0
- package/dist/tools/react-grace.js +87 -0
- package/dist/tools/read-tools.js +20 -4
- package/dist/tools/settle-in-flight.d.ts +79 -0
- package/dist/tools/settle-in-flight.js +106 -0
- package/dist/tools/snapshot-delta.d.ts +7 -0
- package/dist/tools/snapshot-delta.js +130 -6
- package/dist/tools/tool-kit.d.ts +52 -5
- package/dist/tools/tool-kit.js +55 -24
- package/dist/tools/tool-surface.js +28 -9
- package/dist/tools/tools.js +38 -4
- package/dist/visual/capture-cleanup.d.ts +16 -0
- package/dist/visual/capture-cleanup.js +39 -0
- package/dist/visual/visual-tools.d.ts +18 -0
- package/dist/visual/visual-tools.js +24 -5
- package/docs/README.md +56 -11
- package/docs/actions.mdx +87 -0
- package/docs/agent-cheatsheet.md +102 -26
- package/docs/architecture.md +46 -35
- package/docs/benchmarks.md +76 -72
- package/docs/best-practices.mdx +211 -0
- package/docs/capabilities.mdx +387 -0
- package/docs/cli/affected.mdx +51 -0
- package/docs/cli/capsules.mdx +69 -0
- package/docs/cli/cloud.mdx +127 -0
- package/docs/cli/doctor.mdx +76 -0
- package/docs/cli/drive.mdx +55 -0
- package/docs/cli/feedback.mdx +67 -0
- package/docs/cli/gate.mdx +64 -0
- package/docs/cli/help.mdx +92 -0
- package/docs/cli/hunt.mdx +57 -0
- package/docs/cli/identify.mdx +62 -0
- package/docs/cli/init.mdx +94 -0
- package/docs/cli/kill.mdx +91 -0
- package/docs/cli/license.mdx +48 -0
- package/docs/cli/mcp.mdx +61 -0
- package/docs/cli/open.mdx +68 -0
- package/docs/cli/restart.mdx +60 -0
- package/docs/cli/rollback.mdx +35 -0
- package/docs/cli/serve.mdx +62 -0
- package/docs/cli/status.mdx +64 -0
- package/docs/cli/stop.mdx +56 -0
- package/docs/cli/telemetry.mdx +57 -0
- package/docs/cli/update.mdx +52 -0
- package/docs/cli/verify.mdx +57 -0
- package/docs/cli/version.mdx +39 -0
- package/docs/cli/watch.mdx +55 -0
- package/docs/cli.mdx +219 -0
- package/docs/debugging.md +13 -9
- package/docs/deploy-checks.md +15 -11
- package/docs/desktop-apps.md +49 -40
- package/docs/desktop.mdx +188 -0
- package/docs/docs.json +363 -0
- package/docs/enterprise.md +20 -16
- package/docs/faq.mdx +88 -0
- package/docs/fixtures.md +11 -7
- package/docs/flows.md +26 -22
- package/docs/for-agents.md +71 -0
- package/docs/frameworks.mdx +199 -0
- package/docs/gate-plan.md +45 -41
- package/docs/gates.md +23 -19
- package/docs/getting-started.md +76 -65
- package/docs/human-control.md +22 -18
- package/docs/index.mdx +101 -0
- package/docs/install-agentic.mdx +215 -0
- package/docs/install-manual.mdx +194 -0
- package/docs/instrumentation.mdx +232 -0
- package/docs/integration-patterns.md +38 -34
- package/docs/local-registry.md +16 -12
- package/docs/multi-agent-testing.md +16 -12
- package/docs/packages/babel-plugin.mdx +70 -0
- package/docs/packages/browser.mdx +111 -0
- package/docs/packages/core.mdx +106 -0
- package/docs/packages/electron.mdx +87 -0
- package/docs/packages/eslint-plugin.mdx +84 -0
- package/docs/packages/next.mdx +91 -0
- package/docs/packages/react.mdx +91 -0
- package/docs/packages/server.mdx +98 -0
- package/docs/packages/tauri.mdx +63 -0
- package/docs/packages/test.mdx +108 -0
- package/docs/packages/vite-plugin.mdx +96 -0
- package/docs/packages.mdx +217 -0
- package/docs/platform-integration.md +33 -26
- package/docs/predicates.mdx +179 -0
- package/docs/quickstart.mdx +211 -0
- package/docs/recipes.mdx +242 -0
- package/docs/skill-file.mdx +105 -0
- package/docs/state-management.mdx +197 -0
- package/docs/style.css +14 -0
- package/docs/system-map.md +47 -32
- package/docs/telemetry-contract.md +101 -46
- package/docs/telemetry-events.mdx +430 -0
- package/docs/telemetry.md +63 -50
- package/docs/testing.md +10 -6
- package/docs/token-efficiency.md +30 -26
- package/docs/tools-act-and-wait.mdx +123 -0
- package/docs/tools-act-sequence.mdx +120 -0
- package/docs/tools-act.mdx +82 -0
- package/docs/tools-assert.mdx +70 -0
- package/docs/tools-console.mdx +59 -0
- package/docs/tools-inspect.mdx +85 -0
- package/docs/tools-navigate.mdx +63 -0
- package/docs/tools-network.mdx +86 -0
- package/docs/tools-observe.mdx +89 -0
- package/docs/tools-overview.mdx +157 -0
- package/docs/tools-query.mdx +82 -0
- package/docs/tools-session-and-feedback.mdx +91 -0
- package/docs/tools-sessions.mdx +65 -0
- package/docs/tools-snapshot.mdx +75 -0
- package/docs/tools-state.mdx +81 -0
- package/docs/tools-tools-and-run.mdx +125 -0
- package/docs/tools-wait-for.mdx +66 -0
- package/docs/troubleshooting.mdx +193 -0
- package/docs/usage.md +175 -153
- package/docs/vs-chrome-devtools-mcp.mdx +82 -0
- package/docs/vs-playwright-mcp.mdx +85 -0
- package/docs/vs-screenshots.mdx +96 -0
- package/docs/what-is-reticle.mdx +63 -0
- package/docs/why-reticle.mdx +107 -0
- package/package.json +2 -2
- package/docs/matrix/2.5.0/claude-code-machine.json +0 -19
- package/docs/matrix/2.5.0/codex-machine.json +0 -20
- package/docs/matrix/2.5.0/cursor-machine.json +0 -22
- package/docs/matrix/2.5.0/gemini-machine.json +0 -22
- package/docs/matrix/2.5.0/opencode-machine.json +0 -22
- package/docs/matrix/2.5.0/vscode-machine.json +0 -22
- package/docs/matrix/2.5.0/windsurf-machine.json +0 -22
- package/docs/matrix/MATRIX.md +0 -26
- package/docs/matrix/README.md +0 -59
- package/docs/mint.json +0 -61
package/docs/flows.md
CHANGED
|
@@ -1,6 +1,10 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
title: Flows and the recorder
|
|
3
|
+
description: 'Record an interactive run once and replay it forever as a git-checked, self-healing program anchored on meaning, not selectors.'
|
|
4
|
+
icon: repeat
|
|
5
|
+
---
|
|
2
6
|
|
|
3
|
-
Reticle turns an interactive run into a **git-checked, replayable program** stored under `.reticle/`. Flows are anchored on **meaning** (testid + signal), not volatile element refs or coordinates, so they survive refactors
|
|
7
|
+
Reticle turns an interactive run into a **git-checked, replayable program** stored under `.reticle/`. Flows are anchored on **meaning** (testid + signal), not volatile element refs or coordinates, so they survive refactors, and when an anchor does drift, Reticle tells you _why_ and can repair it. This is what makes Reticle "the project's living test suite a human seeds and an agent maintains."
|
|
4
8
|
|
|
5
9
|
> All of this is on-disk and human-readable, so flows are reviewed in PRs and diffed like code.
|
|
6
10
|
|
|
@@ -18,7 +22,7 @@ When you record or save a contract, Reticle writes a git-checked workspace next
|
|
|
18
22
|
|
|
19
23
|
The server resolves `.reticle/` from the working directory it runs in (your project root). A fresh agent can read `.reticle/contract.json` to learn the testable surface **without grepping your source**.
|
|
20
24
|
|
|
21
|
-
## The contract
|
|
25
|
+
## The contract: advertise the testable surface
|
|
22
26
|
|
|
23
27
|
In your app, declare what's testable (see [Step 6 in Getting Started](getting-started.md)):
|
|
24
28
|
|
|
@@ -40,7 +44,7 @@ Then persist it to disk so it's committed and any agent can read it:
|
|
|
40
44
|
|
|
41
45
|
## Create a flow
|
|
42
46
|
|
|
43
|
-
**(a) Agent-recorded
|
|
47
|
+
**(a) Agent-recorded.** The agent drives, then saves:
|
|
44
48
|
|
|
45
49
|
```jsonc
|
|
46
50
|
reticle_record {action:"start"}({ recordingName: "create-task" })
|
|
@@ -49,7 +53,7 @@ reticle_record {action:"stop"}({ recordingName: "create-task" })
|
|
|
49
53
|
reticle_flow_save({ flowName: "create-task" }) // → .reticle/flows/create-task.json
|
|
50
54
|
```
|
|
51
55
|
|
|
52
|
-
**(b) Human-recorded (the recorder toolbar)
|
|
56
|
+
**(b) Human-recorded (the recorder toolbar).** With the presenter on (`present: true`), the floating panel hosts a recorder: a human clicks the golden path in the page and Reticle captures each interaction as a **semantic-anchored** step (testid, else role+name), then persists it via `reticle_flow_save_recorded`. The agent then runs and maintains it. _(First cut: structured annotations only, see below; free natural-language annotations are future work.)_
|
|
53
57
|
|
|
54
58
|
### What a flow file looks like
|
|
55
59
|
|
|
@@ -71,7 +75,7 @@ reticle_flow_save({ flowName: "create-task" }) // → .reticle/flows/cr
|
|
|
71
75
|
}
|
|
72
76
|
```
|
|
73
77
|
|
|
74
|
-
Each step binds to a **semantic anchor**, never a `eXX` ref: a `testid`/`signal` when available, else an auto-derived `component` anchor (component name + source `file:line`) for an element with no testid
|
|
78
|
+
Each step binds to a **semantic anchor**, never a `eXX` ref: a `testid`/`signal` when available, else an auto-derived `component` anchor (component name + source `file:line`) for an element with no testid, so the flow stays stable with zero hand-added testids. Only when none of those resolve is a step kept `degraded: true` (a last-resort "add a testid here" marker) rather than silently dropped.
|
|
75
79
|
|
|
76
80
|
## Run a flow
|
|
77
81
|
|
|
@@ -81,17 +85,17 @@ reticle_flow {action:"load"}({ flowName: "create-task" }) // → the flow JSON
|
|
|
81
85
|
reticle_flow_replay({ flowName: "create-task" }) // re-resolve each anchor against the LIVE DOM, run it
|
|
82
86
|
```
|
|
83
87
|
|
|
84
|
-
**Watch it replay on the page.** When the presenter is on (`present: true`), a replay isn't silent
|
|
88
|
+
**Watch it replay on the page.** When the presenter is on (`present: true`), a replay isn't silent. Each step drives the real page, so the synthetic cursor flies to the element, the focus ring lands, and the activity log streams the journey live. You (or a teammate) literally watch the saved journey re-walk itself on your app, then see the verdict land. It's the fastest way to _see_ that a flow still works, not just read a green checkmark.
|
|
85
89
|
|
|
86
90
|
`reticle_flow_replay` returns a status:
|
|
87
91
|
|
|
88
|
-
- `ok
|
|
89
|
-
- `drift
|
|
90
|
-
- `error
|
|
92
|
+
- `ok`: every anchor resolved and every `expect` held.
|
|
93
|
+
- `drift`: an anchor missed (a testid was renamed, or a signal never fired). The result is **legible**, never a blind failure: `{ step, anchor, drift: { reasonKind: "testid_not_found", nearest: "send-message" } }`. (This is the "whose fault is it" principle.)
|
|
94
|
+
- `error`: the flow file is missing/invalid, or a resolved action failed. Runtime failures include the failed step and a top-level error envelope.
|
|
91
95
|
|
|
92
|
-
A testid-_preserving_ refactor (you moved markup but kept the testids) still replays green. A step whose element has **no testid** is anchored on its component + source location (`{ kind: "component", component, source: { file, line } }`)
|
|
96
|
+
A testid-_preserving_ refactor (you moved markup but kept the testids) still replays green. A step whose element has **no testid** is anchored on its component + source location (`{ kind: "component", component, source: { file, line } }`), an auto-derived stable anchor, so a flow records cleanly with zero hand-added testids and replay re-resolves it via `reticle_query by:'component'`.
|
|
93
97
|
|
|
94
|
-
### The decision envelope
|
|
98
|
+
### The decision envelope: what to do next, not just pass/fail
|
|
95
99
|
|
|
96
100
|
On a `drift` or `error`, the replay result carries a `decision` an agent can act on directly:
|
|
97
101
|
|
|
@@ -105,11 +109,11 @@ decision: {
|
|
|
105
109
|
}
|
|
106
110
|
```
|
|
107
111
|
|
|
108
|
-
This is the feedback a human reviewer used to give
|
|
112
|
+
This is the feedback a human reviewer used to give, made machine-actionable, so the agent decides its next move without one.
|
|
109
113
|
|
|
110
114
|
## Verify the whole suite in one call
|
|
111
115
|
|
|
112
|
-
`reticle_flow_verify` replays **every** saved flow (or a named subset) deterministically
|
|
116
|
+
`reticle_flow_verify` replays **every** saved flow (or a named subset) deterministically, with no LLM per flow, and returns one consolidated verdict. This is the regression check to run after any change:
|
|
113
117
|
|
|
114
118
|
```jsonc
|
|
115
119
|
reticle_flow_verify()
|
|
@@ -119,14 +123,14 @@ reticle_flow_verify()
|
|
|
119
123
|
// whatChanged: "...", whereInSource: "src/...:NN", nextAction: "..." }] }
|
|
120
124
|
```
|
|
121
125
|
|
|
122
|
-
Passing flows are counted; only failures carry detail (token-cheap). Build → `reticle_flow_verify` → fix from each failure's `nextAction` → repeat
|
|
126
|
+
Passing flows are counted; only failures carry detail (token-cheap). Build → `reticle_flow_verify` → fix from each failure's `nextAction` → repeat: the autonomous regression loop.
|
|
123
127
|
|
|
124
|
-
## Self-healing
|
|
128
|
+
## Self-healing: the agent maintains the flow
|
|
125
129
|
|
|
126
|
-
When a testid is renamed, the flow drifts. `reticle_flow_heal` proposes
|
|
130
|
+
When a testid is renamed, the flow drifts. `reticle_flow_heal` proposes (and optionally applies) the nearest-match rebind, so flows don't rot:
|
|
127
131
|
|
|
128
132
|
```jsonc
|
|
129
|
-
reticle_flow_heal({ flowName: "create-task" }) // PROPOSE only
|
|
133
|
+
reticle_flow_heal({ flowName: "create-task" }) // PROPOSE only, never writes
|
|
130
134
|
// → { status: "drift", applied: false,
|
|
131
135
|
// proposals: [{ step: 0, from: "add-tassk", to: "add-task", confidence: 0.8 }] }
|
|
132
136
|
|
|
@@ -134,19 +138,19 @@ reticle_flow_heal({ flowName: "create-task", apply: true }) // rewrite the anch
|
|
|
134
138
|
// → { status: "healed", applied: true, proposals: [...] }
|
|
135
139
|
```
|
|
136
140
|
|
|
137
|
-
With `apply: false` the flow file is **never modified
|
|
141
|
+
With `apply: false` the flow file is **never modified**; you get the proposed diff to review. With `apply: true` Reticle rewrites the drifted anchor(s) to the confident nearest match and a subsequent replay passes. A drift with **no** confident nearest match leaves the file untouched.
|
|
138
142
|
|
|
139
143
|
## Annotations (structured)
|
|
140
144
|
|
|
141
145
|
`reticle_annotate` attaches a structured annotation that compiles into the flow, so replay is a _checked_ re-run, not a blind macro:
|
|
142
146
|
|
|
143
147
|
- `assert-signal` / `assert-visible` → a step `expect` predicate (the invariant).
|
|
144
|
-
- `mark-dynamic` → a `flow.dynamic[]` entry
|
|
145
|
-
- `success-state` → `flow.success` (the golden end condition). Pass `signal`/`testid`, or `statePath` (+ `store`, `equals`) to make the golden condition a **store-truth** assertion
|
|
148
|
+
- `mark-dynamic` → a `flow.dynamic[]` entry; replay asserts the region's _presence_ but **not its words** (the LLM-output case: assert `caption:generated`, ignore the caption text).
|
|
149
|
+
- `success-state` → `flow.success` (the golden end condition). Pass `signal`/`testid`, or `statePath` (+ `store`, `equals`) to make the golden condition a **store-truth** assertion against the app's own source of truth, which no DOM read can reach (e.g. `statePath: "deployments.0.status", equals: "live"` fails the flow if a deploy only _looks_ shipped on screen). State assertions are graded as consequences, so they satisfy the business-outcome oracle.
|
|
146
150
|
|
|
147
151
|
## Flows are your test suite
|
|
148
152
|
|
|
149
|
-
`.reticle/` flows can be executed as CI specs
|
|
153
|
+
`.reticle/` flows can be executed as CI specs via `@reticlehq/test`'s `flowsAsSpecs`, replayed with their `expect`/`success` predicates and skipping `dynamic` regions. See [Testing with Reticle](testing.md).
|
|
150
154
|
|
|
151
155
|
## Tool reference
|
|
152
156
|
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Docs for agents
|
|
3
|
+
description: Fetch any page as plain Markdown, pull the whole site as one file, or hand a page straight to your agent.
|
|
4
|
+
icon: robot
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
These docs are built to be read by a program, not just a person. Nothing here needs an API key, a scraper, or an HTML parser. Every page has a plain-text form at a predictable URL.
|
|
8
|
+
|
|
9
|
+
The site lives at **`https://docs.reticle.sh`**.
|
|
10
|
+
|
|
11
|
+
## Fetch one page
|
|
12
|
+
|
|
13
|
+
Append `.md` to any documentation URL and you get the source Markdown, with the page's frontmatter and no site chrome:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
curl https://docs.reticle.sh/getting-started.md
|
|
17
|
+
curl https://docs.reticle.sh/agent-cheatsheet.md
|
|
18
|
+
curl https://docs.reticle.sh/usage.md
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
### What comes back
|
|
22
|
+
|
|
23
|
+
Every Markdown page is prepended with two things before its content, so a single fetch orients you without a second call:
|
|
24
|
+
|
|
25
|
+
```
|
|
26
|
+
> ## Documentation Index
|
|
27
|
+
> Fetch the complete documentation index at: https://docs.reticle.sh/llms.txt
|
|
28
|
+
> Use this file to discover all available pages before exploring further.
|
|
29
|
+
|
|
30
|
+
> ## Agent Instructions
|
|
31
|
+
> Reticle is a dev-only, localhost-only verification layer for AI coding agents...
|
|
32
|
+
> Only `reticle_act_and_wait` and `reticle_assert` produce a verdict...
|
|
33
|
+
> A verdict of `verified: "unknown"` is not a pass...
|
|
34
|
+
> Package names are scoped `@reticlehq/*` and the CLI is `reticle`...
|
|
35
|
+
|
|
36
|
+
# reticle_query
|
|
37
|
+
...
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The four rules are the ones worth knowing before you drive anything, and they arrive whether or not you asked for them. You do not have to read this page first.
|
|
41
|
+
|
|
42
|
+
The slug is the filename in [`docs/`](https://github.com/reticlehq/reticle/tree/main/docs), so `docs/token-efficiency.md` in the repo is `/token-efficiency` on the site and `/token-efficiency.md` as raw text.
|
|
43
|
+
|
|
44
|
+
## Fetch the index, or everything
|
|
45
|
+
|
|
46
|
+
| URL | What it is | Use it when |
|
|
47
|
+
| --- | --- | --- |
|
|
48
|
+
| [`/llms.txt`](https://docs.reticle.sh/llms.txt) | Every page title and URL, and nothing else | You want to pick the right page before spending tokens on it |
|
|
49
|
+
| [`/llms-full.txt`](https://docs.reticle.sh/llms-full.txt) | The entire documentation as one file (hundreds of KB, and growing) | You are seeding a context window or an index once |
|
|
50
|
+
|
|
51
|
+
Start with `llms.txt`. It is small enough to read in full and tells you which single page answers the question, which is almost always cheaper than pulling `llms-full.txt`.
|
|
52
|
+
|
|
53
|
+
## Pull a page into your harness
|
|
54
|
+
|
|
55
|
+
Every page has a menu in its top-right corner: copy the page as Markdown, or open it directly in Claude, ChatGPT, Cursor or VS Code with the source already attached. That is the fastest route when a human is driving and wants the agent to have the page.
|
|
56
|
+
|
|
57
|
+
> None of this is Reticle itself. These endpoints serve the **documentation**. Reticle runs on your machine and verifies your app; you get it with `npx @reticlehq/server init`. See [Getting started](/getting-started).
|
|
58
|
+
|
|
59
|
+
## Which page to read
|
|
60
|
+
|
|
61
|
+
You rarely need more than one.
|
|
62
|
+
|
|
63
|
+
| You want to | Read |
|
|
64
|
+
| --- | --- |
|
|
65
|
+
| Install Reticle and get a first verdict | [Getting started](/getting-started) |
|
|
66
|
+
| Get fluent in one screen | [Agent cheat sheet](/agent-cheatsheet) |
|
|
67
|
+
| Look up a specific tool, flag, or workflow | [Complete usage guide](/usage) |
|
|
68
|
+
| Understand what Reticle is doing under the hood | [Architecture](/architecture) |
|
|
69
|
+
| Wire it into a desktop app | [Desktop apps](/desktop-apps) |
|
|
70
|
+
| Make the checks repeatable in CI | [Specs for CI](/testing) |
|
|
71
|
+
| Work on Reticle itself | [Gates](/gates), then [System map](/system-map) |
|
|
@@ -0,0 +1,199 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Frameworks
|
|
3
|
+
description: Which frameworks Reticle supports, which are wired but unverified, and the exact wiring for each one.
|
|
4
|
+
icon: layer-group
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
The core of Reticle is framework-neutral. DOM, network, console, routing, animations and source mapping work anywhere JavaScript runs.
|
|
8
|
+
|
|
9
|
+
What varies is two things: how `connect()` gets into your app, and whether you get component identity on top.
|
|
10
|
+
|
|
11
|
+
**Source pointer** is the `file:line` on a verdict, the thing an agent follows straight to the code after a red. It comes from the build plugin stamping your JSX, not from the SDK, so it is a property of how the framework is wired rather than of the framework. Remix and Astro render their own HTML, so the plugin's connect-injection cannot fire there, and both examples used to drop the plugin entirely, which quietly cost them the stamping too. Both now wire it with `inject: false`, which keeps the half that applies. A row marked `unproven` has no example app, so nobody has checked.
|
|
12
|
+
|
|
13
|
+
## Where each framework stands
|
|
14
|
+
|
|
15
|
+
We separate "we tested this" from "this should work". Four frameworks have an example app and a CI gate. The rest are wired and documented but nothing proves they still work.
|
|
16
|
+
|
|
17
|
+
Two columns say different things and it is worth keeping them apart. **CI gate** `yes` means an example app is driven to a verdict on every change. **install only** means the install gate scaffolds that stack from scratch, runs `init`, boots it and waits for a session, so the setup is proven and the drive is not. Vue is there because 2.8.0 nearly shipped an installer that left a Vue app connecting 0% of the time, and the gate that exists for exactly that had only React scaffolds.
|
|
18
|
+
|
|
19
|
+
**Source pointer** is the `file:line` on a verdict. It needs the build plugin to stamp the source, which it does for JSX and, separately, for Svelte components. Vue single-file components are neither, so a Vue app gets every other signal and no `source` field. Measured, not assumed: a Svelte counter returns `src/lib/Counter.svelte:5` and the same drive on Vue returns none.
|
|
20
|
+
|
|
21
|
+
| Framework | `init` detects | Example app | CI gate | Source pointer | Status |
|
|
22
|
+
| --- | :-: | :-: | :-: | :-: | --- |
|
|
23
|
+
| **React + Vite** | yes | yes | yes | yes | Supported |
|
|
24
|
+
| **Next.js** | yes | yes | yes | yes | Supported |
|
|
25
|
+
| **Remix** | yes | yes | yes | yes | Supported |
|
|
26
|
+
| **Astro** | yes | yes | yes | yes | Supported |
|
|
27
|
+
| **SvelteKit** | yes | no | no | unproven | Wired, unverified |
|
|
28
|
+
| **Nuxt** | yes | no | no | unproven | Wired, unverified |
|
|
29
|
+
| **Create React App** | yes | no | no | unproven | Wired, unverified |
|
|
30
|
+
| **Plain HTML** | yes | no | no | no | Wired, unverified |
|
|
31
|
+
| **Vue (non-Nuxt)** | as Vite | no | install only | no | Install gated, drive unverified |
|
|
32
|
+
| **Preact** | as Vite | no | no | unproven | Sensor, React adapter via `preact/compat` |
|
|
33
|
+
|
|
34
|
+
<Warning>
|
|
35
|
+
"Wired, unverified" means exactly that. The recipe exists and was written from a real integration,
|
|
36
|
+
but there is no app in CI exercising it, so nothing will tell us if it breaks. If one of these
|
|
37
|
+
does not register a session, that is a bug worth reporting.
|
|
38
|
+
</Warning>
|
|
39
|
+
|
|
40
|
+
`init` detects the UI library separately from the framework. That distinction exists because detection used to stop at "vite is in package.json", which installed the React adapter into Vue and Preact apps and reported all green with nothing attached.
|
|
41
|
+
|
|
42
|
+
## React on Vite
|
|
43
|
+
|
|
44
|
+
The shortest path, and the one with the most in it.
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npm i -D @reticlehq/react @reticlehq/vite-plugin
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
import { defineConfig } from 'vite';
|
|
52
|
+
import react from '@vitejs/plugin-react';
|
|
53
|
+
import { reticle } from '@reticlehq/vite-plugin';
|
|
54
|
+
|
|
55
|
+
export default defineConfig({
|
|
56
|
+
plugins: [reticle(), react()],
|
|
57
|
+
});
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
That is the whole install. The plugin injects `connect()` and stamps source locations, which is what turns a DOM node into `src/components/Login.tsx:81`.
|
|
61
|
+
|
|
62
|
+
## Next.js
|
|
63
|
+
|
|
64
|
+
Keeps SWC. No Babel migration.
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
npm i -D @reticlehq/react @reticlehq/next
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
```js
|
|
71
|
+
const { withReticle } = require('@reticlehq/next');
|
|
72
|
+
|
|
73
|
+
module.exports = withReticle({
|
|
74
|
+
/* your config */
|
|
75
|
+
});
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
<Note>
|
|
79
|
+
`@reticlehq/next` is CommonJS on purpose. `next.config.js` is loaded by Node before any ESM
|
|
80
|
+
transform, so an ESM-only helper fails at the least helpful moment.
|
|
81
|
+
</Note>
|
|
82
|
+
|
|
83
|
+
## Remix and Astro
|
|
84
|
+
|
|
85
|
+
Both run on Vite, so both use the Vite plugin exactly as React does. Astro needs the React integration if you want component identity in `.jsx` islands. Static `.astro` markup gets DOM, network and routing without it.
|
|
86
|
+
|
|
87
|
+
## SvelteKit
|
|
88
|
+
|
|
89
|
+
SvelteKit renders through `app.html` and never triggers Vite's `index.html` injection, so the plugin cannot auto-connect. A client hook is the reliable path, and it is what `init` writes.
|
|
90
|
+
|
|
91
|
+
```ts
|
|
92
|
+
// src/hooks.client.ts
|
|
93
|
+
import { reticle } from '@reticlehq/browser';
|
|
94
|
+
|
|
95
|
+
if (import.meta.env.DEV) reticle.connect();
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The Vite plugin still stamps `data-reticle-source` into `.svelte` components, so verdicts carry `file:line`.
|
|
99
|
+
|
|
100
|
+
For state, use the `svelteStore` adapter. See [State management](/state-management).
|
|
101
|
+
|
|
102
|
+
## Nuxt
|
|
103
|
+
|
|
104
|
+
Nuxt owns its own Vite instance and renders its own HTML. There is no `vite.config` to patch and no `index.html` to inject into. The idiom is a dev-only client plugin.
|
|
105
|
+
|
|
106
|
+
```ts
|
|
107
|
+
// plugins/reticle.client.ts
|
|
108
|
+
export default defineNuxtPlugin(() => {
|
|
109
|
+
if (!import.meta.dev) return;
|
|
110
|
+
void import('@reticlehq/browser').then(({ reticle }) => {
|
|
111
|
+
reticle.connect();
|
|
112
|
+
});
|
|
113
|
+
});
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
Three traps, all reported from the field rather than imagined:
|
|
117
|
+
|
|
118
|
+
<AccordionGroup>
|
|
119
|
+
<Accordion title="The .client suffix is load-bearing">
|
|
120
|
+
It is what keeps the plugin out of the server bundle. Without it you are running the SDK during
|
|
121
|
+
SSR, where `window` does not exist.
|
|
122
|
+
</Accordion>
|
|
123
|
+
<Accordion title="Guard on import.meta.dev, never on hostname">
|
|
124
|
+
`window.location.hostname === 'localhost'` fails twice over here. `window` does not exist in
|
|
125
|
+
SSR, and the check is false on any hosts-file alias or LAN address. `import.meta.dev` resolves
|
|
126
|
+
at build time and does not care what host you develop on.
|
|
127
|
+
</Accordion>
|
|
128
|
+
<Accordion title="Restart the dev server">
|
|
129
|
+
A running dev server does not pick up a new plugin. It will not appear in
|
|
130
|
+
`.nuxt/plugins/client.mjs`, and the app comes up with no SDK at all and no error saying so.
|
|
131
|
+
</Accordion>
|
|
132
|
+
</AccordionGroup>
|
|
133
|
+
|
|
134
|
+
Use `@reticlehq/browser`, the framework-neutral sensor. You get DOM, network, console, routing and source `file:line`. What you do not get is React component identity, which is the only thing the React adapter adds.
|
|
135
|
+
|
|
136
|
+
<Tip>
|
|
137
|
+
Developing on something other than `localhost`? Add `allowNonLocalhost: true` to the connect call.
|
|
138
|
+
Without it the SDK loads and then refuses, and the only sign is one line in the browser console.
|
|
139
|
+
</Tip>
|
|
140
|
+
|
|
141
|
+
## Vue
|
|
142
|
+
|
|
143
|
+
Vue apps outside Nuxt are Vite apps, so `init` finds them. Install the sensor rather than the React kit.
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
npm i -D @reticlehq/browser
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
```ts
|
|
150
|
+
import { reticle } from '@reticlehq/browser';
|
|
151
|
+
if (import.meta.env.DEV) reticle.connect();
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
Pinia is a first-class store. See [State management](/state-management).
|
|
155
|
+
|
|
156
|
+
## Preact
|
|
157
|
+
|
|
158
|
+
Detected as its own UI library, because a Preact app using `preact/compat` aliases React and would otherwise be handed the React kit with nothing to attach to.
|
|
159
|
+
|
|
160
|
+
The sensor works. If you use `preact/compat`, the React adapter can attach through the alias, but that path is not gated.
|
|
161
|
+
|
|
162
|
+
## Create React App and plain bundlers
|
|
163
|
+
|
|
164
|
+
No config file to patch, so `connect()` goes in your entry module.
|
|
165
|
+
|
|
166
|
+
```ts
|
|
167
|
+
// src/index.tsx
|
|
168
|
+
import { reticle } from '@reticlehq/browser';
|
|
169
|
+
if (process.env.NODE_ENV !== 'production') reticle.connect();
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
The same shape works for webpack, Parcel, and the Vue and Svelte CLIs.
|
|
173
|
+
|
|
174
|
+
## Plain HTML
|
|
175
|
+
|
|
176
|
+
```html
|
|
177
|
+
<script type="module">
|
|
178
|
+
import { reticle } from '/node_modules/@reticlehq/browser/dist/index.js';
|
|
179
|
+
reticle.connect();
|
|
180
|
+
</script>
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
Keep it out of your production template. Reticle self-disables when the build reports `NODE_ENV=production`, but a second lock on a door leading to your users is a reasonable number.
|
|
184
|
+
|
|
185
|
+
## Desktop is a different shape
|
|
186
|
+
|
|
187
|
+
Electron and Tauri invert the direction: your app dials the daemon, and there is no URL for an agent to open. Reticle also reaches the main-process and Rust IPC boundary, which the renderer cannot see.
|
|
188
|
+
|
|
189
|
+
<Card title="Desktop apps" icon="display" href="/desktop">
|
|
190
|
+
Electron and Tauri, IPC observation, and the CSP trap whose failure is completely silent.
|
|
191
|
+
</Card>
|
|
192
|
+
|
|
193
|
+
## Not supported
|
|
194
|
+
|
|
195
|
+
Angular and Solid appear in dependency detection but have no wiring, no adapter and no gate. The sensor may work if you call `connect()` yourself. Nothing proves it, so we are not going to list them as supported.
|
|
196
|
+
|
|
197
|
+
<Card title="Something missing?" icon="comments" href="/tools-session-and-feedback">
|
|
198
|
+
A framework you need is a `gap` report. It is the signal that decides what gets built.
|
|
199
|
+
</Card>
|
package/docs/gate-plan.md
CHANGED
|
@@ -1,4 +1,8 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
title: Release gate plan
|
|
3
|
+
description: 'What has to be true before a PR merges and before a release ships, and what is built so far.'
|
|
4
|
+
icon: list-check
|
|
5
|
+
---
|
|
2
6
|
|
|
3
7
|
> What has to be true before a PR merges and before a release ships, why each piece exists, and what is built so far. Derived from [`system-map.md`](./system-map.md); the harness rules every tier obeys are in [`harness-rules.md`](../apps/e2e/harness-rules.md).
|
|
4
8
|
|
|
@@ -17,17 +21,17 @@
|
|
|
17
21
|
|
|
18
22
|
| Tier | Question | Who | When | Blocks |
|
|
19
23
|
| --- | --- | --- | --- | --- |
|
|
20
|
-
| 0
|
|
21
|
-
| 1
|
|
22
|
-
| 2
|
|
23
|
-
| 3
|
|
24
|
-
| 4
|
|
24
|
+
| 0: repo | is the code internally consistent | Actions | every PR | merge |
|
|
25
|
+
| 1: install | does it install into an app nobody instrumented | Actions, scaffolded apps | PRs touching install/wire | merge |
|
|
26
|
+
| 2: ecosystem | does it survive third-party apps at real scale | `reticle-fixtures`, dispatch | 3×/week + release | release |
|
|
27
|
+
| 3: conformance | does each MCP client actually work | humans + headless CLIs | release candidate | the tag |
|
|
28
|
+
| 4: canary | does a real model still succeed | nightly, live LLM | nightly | nothing (alerts) |
|
|
25
29
|
|
|
26
|
-
**Tier 1 vs 2.** Tier 1 scaffolds a pristine app at CI time (`npm create vite`)
|
|
30
|
+
**Tier 1 vs 2.** Tier 1 scaffolds a pristine app at CI time (`npm create vite`): clean by construction, seconds to produce, catches install _regressions_, fast enough to block a PR. Tier 2 uses vendored production apps and catches install _complexity_. Conflating them produces a gate too slow to block and too shallow to trust.
|
|
27
31
|
|
|
28
32
|
**Tier 3 is the [CNCF conformance](https://github.com/cncf/k8s-conformance) model:** contributors do not _claim_ a client works, they run one command and submit its machine-generated output as a PR, which a bot validates before a human looks. Self-reported pass/fail cannot gate anything.
|
|
29
33
|
|
|
30
|
-
**No LLM in the merge path.** Deterministic tests on every commit; live-model evals nightly and alert-only. The `record → save → verify → heal` chain is what makes this possible
|
|
34
|
+
**No LLM in the merge path.** Deterministic tests on every commit; live-model evals nightly and alert-only. The `record → save → verify → heal` chain is what makes this possible: a flow is recorded once by an agent, then replayed forever with no model.
|
|
31
35
|
|
|
32
36
|
## The matrix, factored
|
|
33
37
|
|
|
@@ -35,64 +39,64 @@
|
|
|
35
39
|
|
|
36
40
|
## Phases
|
|
37
41
|
|
|
38
|
-
### Phase 0
|
|
42
|
+
### Phase 0: the harness contract and the map (**done**)
|
|
39
43
|
|
|
40
44
|
Nothing below is trustworthy until a gate result can be distinguished from a gate artifact.
|
|
41
45
|
|
|
42
|
-
- [x] `docs/system-map.md
|
|
43
|
-
- [x] `apps/e2e/harness-rules.md
|
|
44
|
-
- [x] `apps/e2e/gate-harness.mjs
|
|
46
|
+
- [x] `docs/system-map.md`: topology, connection sequence, tool graph, fragility inventory
|
|
47
|
+
- [x] `apps/e2e/harness-rules.md`: the four rules, with the incidents that produced each
|
|
48
|
+
- [x] `apps/e2e/gate-harness.mjs`: the rules as code, `portHolders`, `freePortSafely`, `startOwnedDaemon`, `watchTransport`, `attributeOutcome`, plus a self-check
|
|
45
49
|
- [x] `run.mjs` and `ci.yml` no longer kill client sockets on the bridge port
|
|
46
50
|
|
|
47
|
-
### Phase 1
|
|
51
|
+
### Phase 1: close the silent gaps already in reach
|
|
48
52
|
|
|
49
|
-
- [x] **Trace-shape assertions
|
|
50
|
-
- [x] **Daemon truthfulness (the port half)
|
|
51
|
-
- [x] **`daemon_alive` heartbeat
|
|
52
|
-
- [x] **`observation_lost
|
|
53
|
-
- [x] **Telemetry chokepoint coverage
|
|
54
|
-
- [x] **Specified transport faults** in place of `kill -9
|
|
53
|
+
- [x] **Trace-shape assertions**: `apps/e2e/trace-shape.mjs` + `trace-shape-test`. One root span per callId, no nested span without a completed parent (the hang signature), no undeclared parentless `browser.command`, no `ok:false` without an error. Calibrated against a real 494-span run: silent there, fires on all four fault shapes.
|
|
54
|
+
- [x] **Daemon truthfulness (the port half)**: `daemon/port-presence.ts` gives a three-state answer (`daemon` / `foreign` / `free`) from two probes and no platform code. `serve` now refuses a foreign port and waits for a real bind before claiming success; `status` and `doctor` name the obstacle. Guarded by `daemon-port-honesty-test`, which squats the port and includes the free-port control. (#105, #112, #115)
|
|
55
|
+
- [x] **`daemon_alive` heartbeat**: `daemon/heartbeat.ts` beats unconditionally on a fixed cadence (30s, `RETICLE_HEARTBEAT_MS` to override) and `classifyDaemonLife` reads a log back into `alive` / `clean` / `signalled` / `died_silently` / `unknown`. A heartbeat nobody interprets is only log volume, so the reader ships with it and is exported for the gate. Guarded by `daemon-heartbeat-test`, which SIGKILLs a real daemon and includes the tidy-shutdown control. (#123)
|
|
56
|
+
- [x] **`observation_lost`**: a new `VerifiedReason`, matched above the `pass === false` clause, so a lost connection grades UNKNOWN instead of being reported as a failed assertion. Signalled by a structured flag from `waitForPredicate`, never by matching on `failureReason` (which is free prose about the app everywhere else it is produced). Threaded through the assert and wait paths too, not just act. (#124)
|
|
57
|
+
- [x] **Telemetry chokepoint coverage**: `tools/dispatch-chokepoint.test.ts` scans for any `.handler(` call outside `runTool` and fails unless it is declared with a stated reason and cost. Three exist today: family folding in `merge-tools.ts` (counted as the family, not the member), `verify-change-tools.ts` (inner `flow_verify` uncounted), and the bridge test harness. Two of those are known observability gaps; the value is that they are now _known_ and a fourth cannot arrive unnoticed. Verified by introducing a bypass and watching it redden.
|
|
58
|
+
- [x] **Specified transport faults** in place of `kill -9`: `apps/e2e/fault-proxy.mjs` (no dependency; toxiproxy is not installed and the battery is deliberately dependency-free) gives none / reset-peer / blackhole / latency / truncate over `node:net`, each proven distinguishable by its own self-check. `transport-faults-test` puts it between the MCP proxy and a real daemon and asserts the product claim: every call is ANSWERED and the stdio server survives. It also separates the two unanswered-call populations by timing: a queued call answered by the 20s queue timer, a call broken in flight answered in ~600ms via `sse_aborted`. A toxic breaks the connection and never the process, so the self-inflicted `kill -9` confusion is unreachable here.
|
|
55
59
|
|
|
56
|
-
### Phase 2
|
|
60
|
+
### Phase 2: the install gate
|
|
57
61
|
|
|
58
|
-
- [x] **Tier 1
|
|
62
|
+
- [x] **Tier 1, all three `init` paths**: `apps/e2e/install-gate.mjs` (`pnpm gate:install`), wired into CI as its own `install-gate` job. Scaffolds `npm create vite`, `create-next-app` (app router) and `create-next-app --no-app` (pages router), publishes this checkout to a local Verdaccio, lets **`init` do its own dependency install** from it, boots each app, opens it in a real browser and POLLS for a session. **3/3 scaffolds, 8/8 assertions each.** The pages-router path (no `app/` root layout, so connect must mount via `pages/_app`) is the one that once did nothing at all, silently.
|
|
59
63
|
- `⚠` is an absolute zero, not a tolerated exemption. The earlier `file:`-wired version had to pass `--no-install` and then argue away the `⚠` it produced, which left the step most likely to regress as the one step untested.
|
|
60
|
-
- A `package-lock.json` check confirms the SDK came from the local registry. Without it a scope typo silently measures PUBLISHED code
|
|
61
|
-
- **Negative control, run FIRST in CI** (`pnpm gate:install:self-test`): every scaffold is mis-wired to a port the daemon is not on and every one must go RED. Verified
|
|
64
|
+
- A `package-lock.json` check confirms the SDK came from the local registry. Without it a scope typo silently measures PUBLISHED code, which is exactly the defect this work found in `docs/local-registry.md` (`@reticle:` vs `@reticlehq:`).
|
|
65
|
+
- **Negative control, run FIRST in CI** (`pnpm gate:install:self-test`): every scaffold is mis-wired to a port the daemon is not on and every one must go RED. Verified: all three fail on the session assertion alone (7 passed, 1 failed), attributed FAIL rather than INCONCLUSIVE because the bridge was provably up. A guard that has never failed is not a guard.
|
|
62
66
|
- `file:` wiring is a dead end and the reason is worth keeping: npm symlinks it, which Vite resolves and Next does not; `--install-links` copies instead and then cannot resolve `workspace:*` at all.
|
|
63
|
-
- [◐] **Tier 2
|
|
67
|
+
- [◐] **Tier 2: built, NOT live.** The sending half is `fixtures-dispatch` in `ci.yml` (on `main` only, never fails the workflow; it is a notification, and the verdict lands in the fixtures repo's run). The receiving half is a template at [`docs/fixtures-dispatch-receiver.yml`](./fixtures-dispatch-receiver.yml), kept in the _sending_ repo so the two ends of the contract cannot drift apart.
|
|
64
68
|
- **Blocked on two things neither of which can be done from inside this repo:** a fine-grained PAT with Actions:write on `reticle-fixtures` added as `FIXTURES_DISPATCH_TOKEN`, and the template copied into that repo. Until the token exists the job says so and exits 0.
|
|
65
69
|
- **It has never run.** Every line is unverified. Tier 2 appearing in this plan is not Tier 2 being live, and the checkbox is deliberately ◐ rather than ✗ or ✓.
|
|
66
|
-
- [x] **Committed baseline of what `init` plans
|
|
67
|
-
- The `target` is in the fingerprint and is the load-bearing field. Leaving it out
|
|
68
|
-
- Proven by tampering: changing one committed entry to `vite.config.js` reddens with a readable expected/got, while every other assertion stays green
|
|
69
|
-
- [x] **Risk routing
|
|
70
|
-
- The reason this was deferred is now solved rather than ignored: a job skipped by an `if:` reports **skipped**, and a branch-protection rule requiring `install-gate` directly would block every docs-only PR forever, waiting on a job that will never run. So nothing is required directly
|
|
70
|
+
- [x] **Committed baseline of what `init` plans**: `apps/e2e/install-baseline.json`, one line per step (`mark title → target`) per scaffold, diffed on every run; `--update-baseline` re-records it so a change arrives as a reviewable diff. "Zero ⚠" is an absolute and cannot carry this alone: a step silently changing mark (`✓ → ℹ`) still passes it, and a step DISAPPEARING from the plan passes it most comfortably of all, because the thing that would have warned you is the thing that is gone. A threshold answers "is this bad"; a baseline answers "is this DIFFERENT", and silent regressions are almost always the second question.
|
|
71
|
+
- The `target` is in the fingerprint and is the load-bearing field. Leaving it out (which the first version did, on an assumption about absolute paths that was simply wrong) made the app-router and pages-router baselines byte-identical, so a regression mounting the pages-router app into the wrong file would have diffed clean. `Mount ReticleDev → app/layout.tsx` vs `→ pages/_app.tsx` is the whole distinction.
|
|
72
|
+
- Proven by tampering: changing one committed entry to `vite.config.js` reddens with a readable expected/got, while every other assertion stays green, which is the point, since a one-character target change is invisible to both "zero ⚠" and "a session appeared".
|
|
73
|
+
- [x] **Risk routing**: a `changes` job computes the diff with plain `git diff --name-only` (no third-party action: the question is "what changed", and a paths-filter dependency for a one-line diff is a supply-chain decision for nothing) and gates `install-gate` on it. It **fails open**: an unresolvable base ref runs every tier, because the cost of a false positive is CI time and the cost of a false negative is the 0%-connect class shipping again.
|
|
74
|
+
- The reason this was deferred is now solved rather than ignored: a job skipped by an `if:` reports **skipped**, and a branch-protection rule requiring `install-gate` directly would block every docs-only PR forever, waiting on a job that will never run. So nothing is required directly: a single aggregate `gate` job is, and it passes when every dependency succeeded **or was deliberately skipped**, and fails on anything else. `always()` on that job is load-bearing; without it the aggregate is itself skipped whenever a dependency is, reproducing the problem one level up.
|
|
71
75
|
- Routing verified against twelve representative paths without touching CI.
|
|
72
76
|
|
|
73
|
-
### Phase 3
|
|
77
|
+
### Phase 3: the client matrix
|
|
74
78
|
|
|
75
79
|
**Blocked on `init` learning each client.** Today only Claude Code (`claude mcp add`) and Cursor (`~/.cursor/mcp.json`) are first-class; everything else gets a printed JSON snippet, so a `v2.5.0-<client>` artifact would measure the contributor's copy-paste rather than the product.
|
|
76
80
|
|
|
77
|
-
- [x] **`init` support per client
|
|
81
|
+
- [x] **`init` support per client**: `init/mcp-clients.ts` registers Reticle with **seven** clients, not two. Every path and shape was read from that client's own documentation rather than recalled: Cursor/Windsurf/Gemini share `mcpServers`, VS Code uses `servers`, OpenCode uses `mcp` with the command as an **array**, Codex is TOML. A single "write mcp.json" assuming the Cursor shape produces a file three of them ignore: an install that reports success and registers nothing.
|
|
78
82
|
- Detection is conservative and one-directional: we write into a config a client **already has**, never creating `~/.gemini` or `~/.codeium` for somebody who does not use them.
|
|
79
83
|
- Codex is deliberately not auto-written. Editing TOML without a parser risks every _other_ server in the user's file; it returns MANUAL with an exact block.
|
|
80
|
-
- [x] **Per-client compat script
|
|
84
|
+
- [x] **Per-client compat script**: `apps/e2e/client-compat.mjs`. Extracts the command from the config **exactly the way that client would**, runs it, and speaks MCP to it. 7/7 clients, 23 assertions, each advertising 18 tools.
|
|
81
85
|
- It proves only the half a machine can prove, and the verdict is named `runnable-unverified` so the distinction survives into the matrix. Whether the client _reads_ that path needs the real client, which is the submitted half.
|
|
82
|
-
- [x] **Submission flow
|
|
83
|
-
- [x] **Generated `MATRIX.md
|
|
86
|
+
- [x] **Submission flow**: `apps/e2e/matrix.mjs --validate`, run by CI's `matrix-records` job on every PR. The validator refuses what cannot be acted on: `works` without `checks.toolsVisible` (the self-report the flow exists to replace), `broken` without the client's verbatim error, or any record with no host or version. Its self-check runs first, because a validator that has never refused anything is not a validator. Contributor guide in [`docs/matrix/README.md`](./matrix/README.md).
|
|
87
|
+
- [x] **Generated `MATRIX.md`**: built from submitted records, prettier-ignored because a generated file that fails `format:check` teaches people to skip the gate. **The current honest state is seven clients and ZERO ticks**: `◐` means a runnable entry was written where the client documents it and _nobody has run that client_, and the generated legend says so rather than rounding `◐` up to `✅`.
|
|
84
88
|
|
|
85
|
-
### Phase 4
|
|
89
|
+
### Phase 4: rates, not booleans
|
|
86
90
|
|
|
87
91
|
Connection stability is a rate. A pass/fail cannot express "breaks a lot", and a number that is not recorded cannot regress.
|
|
88
92
|
|
|
89
|
-
- [x] **Soak with a recorded stability rate
|
|
90
|
-
- **Idle time between rounds is the point**, not a pause for politeness. 150 back-to-back calls finish in seconds and measure throughput; the disconnect users actually report happens while nobody is calling
|
|
93
|
+
- [x] **Soak with a recorded stability rate**: `apps/e2e/soak.mjs` (`pnpm gate:soak`), run inside the e2e battery where a paired app already exists, and appended to `bench/soak-history.jsonl` so the rate can regress against itself. One held-open MCP link for the whole run, never a reconnect per call (harness rule 3). First recorded row: **120/120 answered, 0 link drops over 39s**.
|
|
94
|
+
- **Idle time between rounds is the point**, not a pause for politeness. 150 back-to-back calls finish in seconds and measure throughput; the disconnect users actually report happens while nobody is calling: the agent is thinking and the next call finds the link gone. Keep-alives, idle shutdown and proxy timeouts all live in that gap, so a soak with no idle never enters it. `--idle-ms 30000 --rounds 60` is the half-hour release soak.
|
|
91
95
|
- Attribution is 3-valued: a transport that did not stay up reports INCONCLUSIVE and claims nothing about the product (harness rule 4).
|
|
92
|
-
- [x] **Per-tool latency and failure budgets
|
|
93
|
-
- **Latency is recorded, NOT gated, and that is a deliberate deviation from the wording of this plan.** CLAUDE.md: timing assertions are a bug. A p95 budget in wall-clock ms is a statement about the machine, goes red only under parallel CI load, and teaches everyone to re-run
|
|
94
|
-
- The warn carries a 50ms floor, added because the first real run shouted `reticle_sessions p95 8ms vs 1ms (>4x)
|
|
96
|
+
- [x] **Per-tool latency and failure budgets**, same run, `bench/TOOL-PROFILE.md`. Answers "which tool is breaking": `tool-surface-sweep` proves each tool is callable **once**, and one call cannot have a failure rate.
|
|
97
|
+
- **Latency is recorded, NOT gated, and that is a deliberate deviation from the wording of this plan.** CLAUDE.md: timing assertions are a bug. A p95 budget in wall-clock ms is a statement about the machine, goes red only under parallel CI load, and teaches everyone to re-run; a gate people re-run has stopped working. A >4x p95 jump prints a loud WARN instead. What IS gated: answer rate (absolute) and per-tool failure rate (vs the recorded baseline), both deterministic.
|
|
98
|
+
- The warn carries a 50ms floor, added because the first real run shouted `reticle_sessions p95 8ms vs 1ms (>4x)`: true, imperceptible, and exactly how a warning channel dies.
|
|
95
99
|
- It profiles the **6 repeatable read-only tools, not all 48 advertised**, and says so in the generated file. Repeating a mutating tool measures the fixture drifting rather than the tool degrading.
|
|
96
|
-
- Eleven-case self-check (`node apps/e2e/soak.mjs --self-check`), in CI's `matrix-records` job. It needs no build and no app
|
|
100
|
+
- Eleven-case self-check (`node apps/e2e/soak.mjs --self-check`), in CI's `matrix-records` job. It needs no build and no app, proven by running it in an empty directory, which is how a static import of the built CLI was caught.
|
|
97
101
|
- [ ] false-green scorecard as a standing gate against apps we did not write (#130)
|
|
98
102
|
- **Genuinely blocked here, not deferred by choice.** "Apps we did not write" is the whole content of the check, and this repo contains only apps we wrote. It belongs to the fixtures repo (Tier 2) and needs `FIXTURES_DISPATCH_TOKEN`.
|