@reticlehq/server 2.8.0 → 2.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/SKILL.md +130 -813
- package/dist/bridge/bridge.d.ts +23 -0
- package/dist/bridge/bridge.js +80 -2
- package/dist/bridge/bridge.test-harness.d.ts +5 -0
- package/dist/bridge/bridge.test-harness.js +6 -4
- package/dist/capsule/causal-summary.d.ts +16 -1
- package/dist/capsule/causal-summary.js +2 -1
- package/dist/cli/cli-doctor.js +41 -13
- package/dist/cli/cli-flow-commands.js +8 -8
- package/dist/cli/cli-launch.d.ts +0 -17
- package/dist/cli/cli-launch.js +0 -28
- package/dist/cli/cli-port.d.ts +24 -0
- package/dist/cli/cli-port.js +66 -35
- package/dist/cli/cli-verify.d.ts +21 -0
- package/dist/cli/cli-verify.js +45 -1
- package/dist/cli/config-discovery.d.ts +35 -0
- package/dist/cli/config-discovery.js +163 -0
- package/dist/cli/doctor-daemon-line.js +4 -1
- package/dist/cli/doctor-project-line.d.ts +37 -0
- package/dist/cli/doctor-project-line.js +35 -0
- package/dist/cli/doctor-rows.d.ts +51 -0
- package/dist/cli/doctor-rows.js +54 -0
- package/dist/cli/doctor-sessions-line.js +6 -3
- package/dist/cli/drive-attach.d.ts +99 -0
- package/dist/cli/drive-attach.js +203 -0
- package/dist/cli/drive-command.d.ts +13 -0
- package/dist/cli/drive-command.js +103 -0
- package/dist/cli/open-note.d.ts +20 -0
- package/dist/cli/open-note.js +37 -0
- package/dist/cli/status-next-action.d.ts +37 -0
- package/dist/cli/status-next-action.js +61 -0
- package/dist/cli.js +65 -70
- package/dist/events/contradictions.d.ts +32 -0
- package/dist/events/contradictions.js +9 -5
- package/dist/events/declared.d.ts +45 -0
- package/dist/events/declared.js +73 -0
- package/dist/events/echo-mismatch.d.ts +8 -2
- package/dist/events/echo-mismatch.js +33 -5
- package/dist/events/event-filters.d.ts +1 -1
- package/dist/events/event-filters.js +7 -4
- package/dist/events/predicate-eval.d.ts +17 -91
- package/dist/events/predicate-eval.js +85 -326
- package/dist/events/predicate-parse.js +31 -3
- package/dist/events/predicate-route.d.ts +32 -0
- package/dist/events/predicate-route.js +102 -0
- package/dist/events/predicate-schema.d.ts +99 -0
- package/dist/events/predicate-schema.js +309 -0
- package/dist/events/predicate.d.ts +8 -0
- package/dist/events/predicate.js +127 -5
- package/dist/flows/annotate-tools.js +3 -0
- package/dist/flows/annotate.js +18 -4
- package/dist/flows/coverage.d.ts +38 -5
- package/dist/flows/coverage.js +30 -4
- package/dist/flows/decision.d.ts +15 -0
- package/dist/flows/decision.js +1 -1
- package/dist/flows/flow-replay-run.js +8 -1
- package/dist/flows/flow-tools.js +9 -1
- package/dist/flows/replay.js +3 -0
- package/dist/honesty/accepted-write.d.ts +3 -0
- package/dist/honesty/accepted-write.js +8 -3
- package/dist/honesty/already-true.js +15 -0
- package/dist/honesty/blind-spots.js +4 -0
- package/dist/honesty/honesty.d.ts +11 -0
- package/dist/honesty/honesty.js +1 -0
- package/dist/honesty/uncaptured-bodies.d.ts +4 -2
- package/dist/honesty/uncaptured-bodies.js +9 -2
- package/dist/honesty/unread-outcome.d.ts +8 -0
- package/dist/honesty/unread-outcome.js +36 -9
- package/dist/honesty/unsettled.d.ts +9 -0
- package/dist/honesty/unsettled.js +7 -3
- package/dist/honesty/verified.d.ts +29 -3
- package/dist/honesty/verified.js +51 -8
- package/dist/http-server.d.ts +19 -0
- package/dist/http-server.js +202 -4
- package/dist/impact/impact-recorder.d.ts +23 -0
- package/dist/impact/impact-recorder.js +76 -0
- package/dist/impact/impact-store.d.ts +47 -0
- package/dist/impact/impact-store.js +171 -0
- package/dist/index.d.ts +7 -0
- package/dist/index.js +44 -10
- package/dist/init/agent-rules.d.ts +20 -0
- package/dist/init/agent-rules.js +36 -0
- package/dist/init/astro-patch.js +27 -1
- package/dist/init/confirm.d.ts +21 -3
- package/dist/init/confirm.js +32 -5
- package/dist/init/connect-steps.d.ts +2 -0
- package/dist/init/connect-steps.js +50 -0
- package/dist/init/csp-check.d.ts +38 -0
- package/dist/init/csp-check.js +92 -0
- package/dist/init/csp-doctor.d.ts +25 -0
- package/dist/init/csp-doctor.js +49 -0
- package/dist/init/cursor.js +6 -9
- package/dist/init/detect.d.ts +8 -0
- package/dist/init/detect.js +33 -3
- package/dist/init/mcp-clients.d.ts +1 -0
- package/dist/init/mcp-clients.js +51 -7
- package/dist/init/mcp.d.ts +15 -0
- package/dist/init/mcp.js +19 -0
- package/dist/init/plan-framework.d.ts +10 -0
- package/dist/init/plan-framework.js +25 -1
- package/dist/init/plan.d.ts +7 -2
- package/dist/init/plan.js +66 -30
- package/dist/init/run.js +11 -5
- package/dist/init/slash-command.d.ts +1 -1
- package/dist/init/slash-command.js +12 -4
- package/dist/init/snippets.d.ts +1 -1
- package/dist/init/snippets.js +35 -7
- package/dist/init/vite-config.js +24 -2
- package/dist/input/network-mock-tools.js +8 -1
- package/dist/input/scroll-find.d.ts +6 -0
- package/dist/input/scroll-find.js +17 -2
- package/dist/input/scroll-tools.js +4 -0
- package/dist/journal/attach-journal.d.ts +9 -1
- package/dist/journal/attach-journal.js +13 -2
- package/dist/journal/journal-query.d.ts +13 -0
- package/dist/journal/journal-query.js +28 -3
- package/dist/journal/workspace-gitignore.d.ts +10 -0
- package/dist/journal/workspace-gitignore.js +59 -0
- package/dist/mcp/mcp-proxy.d.ts +5 -66
- package/dist/mcp/mcp-proxy.js +180 -163
- package/dist/mcp/mcp.d.ts +10 -1
- package/dist/mcp/mcp.js +47 -27
- package/dist/mcp/proxy-handshake.d.ts +82 -1
- package/dist/mcp/proxy-handshake.js +109 -2
- package/dist/mcp/proxy-lifecycle.d.ts +10 -1
- package/dist/mcp/proxy-lifecycle.js +11 -2
- package/dist/mcp/proxy-log.d.ts +63 -0
- package/dist/mcp/proxy-log.js +132 -0
- package/dist/mcp/server-instructions.d.ts +17 -0
- package/dist/mcp/server-instructions.js +56 -0
- package/dist/mcp/shared-params.d.ts +27 -0
- package/dist/mcp/shared-params.js +36 -0
- package/dist/project/reticle-dir.d.ts +2 -0
- package/dist/project/reticle-dir.js +1 -0
- package/dist/session/connection-memory.d.ts +59 -0
- package/dist/session/connection-memory.js +108 -0
- package/dist/session/dev-command.d.ts +26 -0
- package/dist/session/dev-command.js +89 -0
- package/dist/session/dev-server-probe.d.ts +68 -0
- package/dist/session/dev-server-probe.js +102 -7
- package/dist/session/last-act.d.ts +26 -0
- package/dist/session/last-act.js +35 -1
- package/dist/session/live-control-tools.js +42 -5
- package/dist/session/no-session-diagnosis.d.ts +48 -2
- package/dist/session/no-session-diagnosis.js +176 -20
- package/dist/session/no-session-next-action.d.ts +46 -0
- package/dist/session/no-session-next-action.js +85 -0
- package/dist/session/no-session-watch.d.ts +40 -1
- package/dist/session/no-session-watch.js +165 -11
- package/dist/session/pending-commands.d.ts +12 -0
- package/dist/session/pending-commands.js +13 -1
- package/dist/session/session-manager.d.ts +4 -0
- package/dist/session/session-manager.js +35 -5
- package/dist/session/session.d.ts +10 -1
- package/dist/session/session.js +146 -5
- package/dist/telemetry/app-instrumented.d.ts +8 -0
- package/dist/telemetry/app-instrumented.js +10 -0
- package/dist/telemetry/bug-found.d.ts +18 -0
- package/dist/telemetry/bug-found.js +14 -0
- package/dist/telemetry/daemon-telemetry.js +24 -0
- package/dist/telemetry/install-source.d.ts +20 -0
- package/dist/telemetry/install-source.js +34 -0
- package/dist/telemetry/instrumentation-stall.d.ts +50 -0
- package/dist/telemetry/instrumentation-stall.js +90 -0
- package/dist/telemetry/session-metrics.d.ts +8 -0
- package/dist/telemetry/session-metrics.js +10 -0
- package/dist/telemetry/telemetry.d.ts +3 -1
- package/dist/telemetry/telemetry.js +13 -2
- package/dist/tools/act-tools.js +166 -29
- package/dist/tools/assert-verdict.js +32 -7
- package/dist/tools/browser-tools.js +10 -0
- package/dist/tools/dynamic-tools.js +16 -7
- package/dist/tools/error-recovery.js +9 -5
- package/dist/tools/invoke-tool.js +20 -3
- package/dist/tools/lease-hint.d.ts +30 -8
- package/dist/tools/lease-hint.js +65 -14
- package/dist/tools/lease-tools.d.ts +11 -0
- package/dist/tools/lease-tools.js +141 -96
- package/dist/tools/observe-tools.js +9 -3
- package/dist/tools/react-grace.d.ts +4 -0
- package/dist/tools/react-grace.js +6 -3
- package/dist/tools/read-tools.js +47 -1
- package/dist/tools/resolve-target.d.ts +33 -0
- package/dist/tools/resolve-target.js +53 -0
- package/dist/tools/settle-in-flight.d.ts +15 -0
- package/dist/tools/settle-in-flight.js +42 -2
- package/dist/tools/snapshot-delta.d.ts +3 -0
- package/dist/tools/snapshot-delta.js +67 -6
- package/dist/tools/tool-kit.d.ts +8 -0
- package/dist/tools/tool-surface.d.ts +60 -0
- package/dist/tools/tool-surface.js +85 -19
- package/dist/tools/tools.js +45 -4
- package/dist/tools/unadvertised-help.js +9 -6
- package/docs/README.md +1 -2
- package/docs/actions.mdx +41 -8
- package/docs/agent-cheatsheet.md +4 -2
- package/docs/architecture.md +6 -0
- package/docs/benchmarks.md +60 -12
- package/docs/best-practices.mdx +11 -6
- package/docs/capabilities.mdx +201 -54
- package/docs/cli/affected.mdx +6 -4
- package/docs/cli/capsules.mdx +9 -8
- package/docs/cli/cloud.mdx +30 -4
- package/docs/cli/doctor.mdx +41 -8
- package/docs/cli/drive.mdx +34 -15
- package/docs/cli/gate.mdx +6 -6
- package/docs/cli/help.mdx +8 -6
- package/docs/cli/hunt.mdx +18 -2
- package/docs/cli/identify.mdx +16 -2
- package/docs/cli/init.mdx +35 -8
- package/docs/cli/kill.mdx +5 -5
- package/docs/cli/license.mdx +1 -1
- package/docs/cli/open.mdx +6 -0
- package/docs/cli/restart.mdx +6 -6
- package/docs/cli/serve.mdx +10 -3
- package/docs/cli/status.mdx +29 -5
- package/docs/cli/stop.mdx +8 -0
- package/docs/cli/verify.mdx +20 -1
- package/docs/cli/version.mdx +1 -1
- package/docs/cli/watch.mdx +9 -2
- package/docs/cli.mdx +44 -22
- package/docs/debugging.md +3 -1
- package/docs/deploy-checks.md +30 -1
- package/docs/desktop.mdx +276 -80
- package/docs/docs.json +13 -12
- package/docs/enterprise.md +2 -0
- package/docs/faq.mdx +49 -35
- package/docs/fixtures.md +2 -0
- package/docs/flows.md +3 -1
- package/docs/for-agents.md +3 -3
- package/docs/frameworks.mdx +46 -7
- package/docs/gate-plan.md +4 -2
- package/docs/gates.md +7 -5
- package/docs/getting-started.md +53 -12
- package/docs/human-control.md +2 -0
- package/docs/index.mdx +43 -1
- package/docs/install-agentic.mdx +13 -4
- package/docs/install-manual.mdx +6 -4
- package/docs/instrumentation.mdx +7 -1
- package/docs/integration-patterns.md +3 -1
- package/docs/local-registry.md +4 -2
- package/docs/multi-agent-testing.md +3 -1
- package/docs/packages/babel-plugin.mdx +15 -6
- package/docs/packages/browser.mdx +2 -2
- package/docs/packages/core.mdx +26 -16
- package/docs/packages/electron.mdx +5 -7
- package/docs/packages/eslint-plugin.mdx +5 -3
- package/docs/packages/next.mdx +3 -5
- package/docs/packages/react.mdx +3 -3
- package/docs/packages/server.mdx +2 -2
- package/docs/packages/tauri.mdx +6 -6
- package/docs/packages/test.mdx +18 -7
- package/docs/packages/vite-plugin.mdx +4 -6
- package/docs/packages.mdx +14 -13
- package/docs/platform-integration.md +2 -0
- package/docs/predicates.mdx +76 -7
- package/docs/quickstart.mdx +51 -5
- package/docs/recipes.mdx +62 -22
- package/docs/skill-file.mdx +3 -1
- package/docs/state-management.mdx +2 -0
- package/docs/system-map.md +3 -1
- package/docs/telemetry-contract.md +16 -5
- package/docs/telemetry-events.mdx +63 -18
- package/docs/telemetry.md +8 -5
- package/docs/testing.md +35 -1
- package/docs/token-efficiency.md +8 -2
- package/docs/tools-act-and-wait.mdx +64 -14
- package/docs/tools-act-sequence.mdx +34 -17
- package/docs/tools-act.mdx +24 -11
- package/docs/tools-assert.mdx +69 -10
- package/docs/tools-console.mdx +11 -7
- package/docs/tools-inspect.mdx +11 -8
- package/docs/tools-navigate.mdx +41 -5
- package/docs/tools-network.mdx +42 -9
- package/docs/tools-observe.mdx +66 -10
- package/docs/tools-overview.mdx +26 -23
- package/docs/tools-query.mdx +52 -8
- package/docs/tools-session-and-feedback.mdx +30 -9
- package/docs/tools-sessions.mdx +39 -10
- package/docs/tools-snapshot.mdx +57 -3
- package/docs/tools-state.mdx +24 -1
- package/docs/tools-tools-and-run.mdx +40 -9
- package/docs/tools-wait-for.mdx +34 -15
- package/docs/troubleshooting.mdx +80 -10
- package/docs/usage.md +43 -21
- package/docs/vs-chrome-devtools-mcp.mdx +41 -0
- package/docs/vs-playwright-mcp.mdx +42 -2
- package/docs/vs-screenshots.mdx +42 -1
- package/docs/what-is-reticle.mdx +1 -1
- package/docs/why-reticle.mdx +3 -1
- package/package.json +5 -5
- package/docs/desktop-apps.md +0 -265
package/dist/tools/tools.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { z } from 'zod';
|
|
2
|
-
import { QueryBy, ReticleCommand, SnapshotMode } from '@reticlehq/core';
|
|
2
|
+
import { NoSessionAction, QueryBy, ReticleCommand, SnapshotMode } from '@reticlehq/core';
|
|
3
3
|
import { ReticleTool } from './tool-names.js';
|
|
4
4
|
import { withSizeCost } from '../session/output-budget.js';
|
|
5
5
|
import { applySnapshotDelta, SnapshotCache } from './snapshot-delta.js';
|
|
@@ -82,6 +82,15 @@ const RAW_TOOLS = [
|
|
|
82
82
|
.string()
|
|
83
83
|
.optional()
|
|
84
84
|
.describe('Present ONLY when `sessions` is empty: why nothing is connected, and the next action that fixes it. An empty list is never the end of the road — read this before concluding the app cannot be driven.'),
|
|
85
|
+
next_action: z
|
|
86
|
+
.object({
|
|
87
|
+
action: z.nativeEnum(NoSessionAction),
|
|
88
|
+
command: z.string().optional(),
|
|
89
|
+
port: z.number().optional(),
|
|
90
|
+
reason: z.string(),
|
|
91
|
+
})
|
|
92
|
+
.optional()
|
|
93
|
+
.describe("Present ONLY when `sessions` is empty: the same answer as `why`, executable. `command` is the LITERAL command to run, sourced from this project's own package.json scripts and lockfile — it is absent, never guessed, when the project declares no dev script. `action` is one of start_dev_server | run_init | open_app | reopen_app."),
|
|
85
94
|
},
|
|
86
95
|
handler: async (deps) => {
|
|
87
96
|
const provider = deps.realInput;
|
|
@@ -97,7 +106,14 @@ const RAW_TOOLS = [
|
|
|
97
106
|
// rather than only when a later tool fails for want of a session.
|
|
98
107
|
if (0 === sessions.length) {
|
|
99
108
|
const why = deps.sessions.noSessionHint();
|
|
100
|
-
|
|
109
|
+
// The executable half. `why` is for the human reading the transcript; this is the one the
|
|
110
|
+
// agent acts on, so it never has to parse a paragraph to find a command inside it.
|
|
111
|
+
const next = deps.sessions.noSessionNextAction();
|
|
112
|
+
return {
|
|
113
|
+
sessions,
|
|
114
|
+
...(why === undefined ? {} : { why }),
|
|
115
|
+
...(next === undefined ? {} : { next_action: next }),
|
|
116
|
+
};
|
|
101
117
|
}
|
|
102
118
|
return { sessions };
|
|
103
119
|
},
|
|
@@ -126,11 +142,18 @@ const RAW_TOOLS = [
|
|
|
126
142
|
.string()
|
|
127
143
|
.optional()
|
|
128
144
|
.describe('Indented ARIA tree of every element on the page (or the scoped subtree).'),
|
|
129
|
-
status: z
|
|
145
|
+
status: z
|
|
146
|
+
.object({
|
|
147
|
+
route: z.string(),
|
|
148
|
+
title: z.string().optional(),
|
|
149
|
+
visibleDialogs: z.array(z.string()).optional(),
|
|
150
|
+
overlayHidingPage: z.string().optional(),
|
|
151
|
+
})
|
|
152
|
+
.optional(),
|
|
130
153
|
mode: z
|
|
131
154
|
.string()
|
|
132
155
|
.optional()
|
|
133
|
-
.describe('delta | unchanged
|
|
156
|
+
.describe('full | delta | unchanged — which kind of diff response this is.'),
|
|
134
157
|
delta: z
|
|
135
158
|
.object({
|
|
136
159
|
added: z.array(z.string()),
|
|
@@ -140,6 +163,15 @@ const RAW_TOOLS = [
|
|
|
140
163
|
})
|
|
141
164
|
.optional()
|
|
142
165
|
.describe('Only present on a diff:true call that found changes.'),
|
|
166
|
+
changed: z
|
|
167
|
+
.array(z.string())
|
|
168
|
+
.optional()
|
|
169
|
+
.describe('Elements whose value changed in place (same ref, different content) — not a structural add/remove.'),
|
|
170
|
+
changedCount: z.number().optional().describe('Number of elements in `changed`.'),
|
|
171
|
+
reason: z
|
|
172
|
+
.string()
|
|
173
|
+
.optional()
|
|
174
|
+
.describe('Why mode is "full": first snapshot for this route, or route changed.'),
|
|
143
175
|
cost: z
|
|
144
176
|
.object({ bytes: z.number(), tokens: z.number() })
|
|
145
177
|
.optional()
|
|
@@ -380,6 +412,15 @@ const RAW_TOOLS = [
|
|
|
380
412
|
* Reticle build plugin in dev; absent in production builds.
|
|
381
413
|
*/
|
|
382
414
|
source: z.string().optional(),
|
|
415
|
+
/**
|
|
416
|
+
* Why `source` is missing, when it is.
|
|
417
|
+
*
|
|
418
|
+
* Distinguishes "this element has no stamp" from "nothing on this page has one, so the
|
|
419
|
+
* stamping loader is not running" — the second means `file:line` is unavailable for the
|
|
420
|
+
* whole session and the fix is a build-config change, which used to be discoverable only by
|
|
421
|
+
* reading the adapter's own source.
|
|
422
|
+
*/
|
|
423
|
+
sourceUnavailable: z.string().optional(),
|
|
383
424
|
/**
|
|
384
425
|
* Component identity from the framework adapter (@reticlehq/react).
|
|
385
426
|
*
|
|
@@ -1,9 +1,10 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* What to say when an agent calls a real Reticle tool that this profile does not advertise.
|
|
3
3
|
*
|
|
4
|
-
* The default
|
|
5
|
-
* is re-sent on every turn, and
|
|
6
|
-
* fully callable through `reticle_run`,
|
|
4
|
+
* The default surface advertises 18 of 48 tools, and the extended one 30. That is deliberate — every
|
|
5
|
+
* advertised tool's schema is re-sent on every turn, and editors budget MCP tools as a COUNT shared
|
|
6
|
+
* across every connected server — and everything omitted stays fully callable through `reticle_run`,
|
|
7
|
+
* which `surface-reachability.test.ts` guards.
|
|
7
8
|
*
|
|
8
9
|
* The gap was in the ERROR. The MCP SDK answers `Tool <name> not found`, which is indistinguishable
|
|
9
10
|
* from "this tool does not exist", so an agent that trusts it stops trying. Reported from a real
|
|
@@ -14,7 +15,7 @@
|
|
|
14
15
|
* and the switch that makes it stop being necessary.
|
|
15
16
|
*/
|
|
16
17
|
import { ReticleTool } from './tool-names.js';
|
|
17
|
-
import {
|
|
18
|
+
import { ADVERTISE_ALL_ENV } from './tool-surface.js';
|
|
18
19
|
import { mergedNameRedirect, mergedNameMessage } from './merged-name-redirect.js';
|
|
19
20
|
/**
|
|
20
21
|
* Guidance for `name`, or undefined when there is nothing useful to add — the tool IS advertised (so
|
|
@@ -34,6 +35,8 @@ export function unadvertisedToolHelp(name, advertised, known) {
|
|
|
34
35
|
`away. It is NOT missing: invoke it with ` +
|
|
35
36
|
`${ReticleTool.RUN} { tool: "${name}", args: { ... } }. ` +
|
|
36
37
|
`Call ${ReticleTool.TOOLS} { names: ["${name}"] } for its parameters. ` +
|
|
37
|
-
`If you need it repeatedly,
|
|
38
|
-
`
|
|
38
|
+
`If you need it repeatedly, start the daemon with ${ADVERTISE_ALL_ENV}=1 for the extended ` +
|
|
39
|
+
`surface. That is read by the DAEMON at startup, so it takes effect on the next daemon, and it ` +
|
|
40
|
+
`advertises MORE tools rather than all of them — editors budget MCP tools as a shared count, so ` +
|
|
41
|
+
`${ReticleTool.RUN} stays the way to reach the rest.`);
|
|
39
42
|
}
|
package/docs/README.md
CHANGED
|
@@ -49,8 +49,7 @@ Everything here is published to [docs.reticle.sh](https://docs.reticle.sh) by `d
|
|
|
49
49
|
| [for-agents.md](for-agents.md) | how to fetch these docs as Markdown or `llms.txt` |
|
|
50
50
|
| [architecture.md](architecture.md) | how it works, and why it is built this way |
|
|
51
51
|
| [platform-integration.md](platform-integration.md) | Vite, Next, Remix, Astro, plain HTML |
|
|
52
|
-
| [desktop.mdx](desktop.mdx) |
|
|
53
|
-
| [desktop-apps.md](desktop-apps.md) | Electron and Tauri, in full |
|
|
52
|
+
| [desktop.mdx](desktop.mdx) | Electron and Tauri in full: setup, IPC, screenshots, headless, troubleshooting |
|
|
54
53
|
| [frameworks.mdx](frameworks.mdx) | what is supported, what is wired-but-unverified, and the wiring |
|
|
55
54
|
| [state-management.mdx](state-management.mdx) | zustand, Redux, and eight adapters |
|
|
56
55
|
| [instrumentation.mdx](instrumentation.mdx) | stores, signals, testids, and how verdicts get stronger |
|
package/docs/actions.mdx
CHANGED
|
@@ -1,12 +1,14 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: Action reference: lights, camera, click
|
|
2
|
+
title: 'Action reference: lights, camera, click'
|
|
3
3
|
sidebarTitle: Actions
|
|
4
4
|
description: Every action reticle_act and reticle_act_and_wait can perform, with its arguments and the one that has a history.
|
|
5
5
|
icon: hand-pointer
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
+
Every action Reticle can perform takes the same shape: a `ref` from [`reticle_snapshot`](/tools-snapshot) or [`reticle_query`](/tools-query), an `action` name, and an optional `args` object. The same seventeen actions are available on [`reticle_act`](/tools-act), which proves nothing, and on [`reticle_act_and_wait`](/tools-act-and-wait), which returns a verdict.
|
|
9
|
+
|
|
8
10
|
```json
|
|
9
|
-
{ "ref": "
|
|
11
|
+
{ "ref": "e104", "action": "fill", "args": { "value": "reticle" } }
|
|
10
12
|
```
|
|
11
13
|
|
|
12
14
|
| Action | Arguments | Notes |
|
|
@@ -22,10 +24,39 @@ icon: hand-pointer
|
|
|
22
24
|
| `submit` | none | Submits the element's `<form>` |
|
|
23
25
|
| `press` | `{ text }` | The key **name**: `Escape`, `Tab`, `Enter` |
|
|
24
26
|
| `scrollIntoView` | none | |
|
|
25
|
-
| `upload` | `{
|
|
26
|
-
| `drag` | `{ toRef }` | Pointer-based drag (dnd-kit, react-beautiful-dnd) plus HTML5 DnD |
|
|
27
|
+
| `upload` | `{ content?, name?, type? }` | Sets a file on `<input type="file">`. All three default, to `"reticle test file"`, `"file.txt"` and `"text/plain"` |
|
|
28
|
+
| `drag` | `{ toRef }` | Pointer-based drag (dnd-kit, react-beautiful-dnd) plus HTML5 DnD. `target` is an alias |
|
|
27
29
|
| `webmcp` | `{ tool, params }` | Calls a `navigator.modelContext` tool, if the site exposes one |
|
|
28
30
|
|
|
31
|
+
### Arguments that are not actions
|
|
32
|
+
|
|
33
|
+
Three `args` keys apply across actions rather than belonging to one:
|
|
34
|
+
|
|
35
|
+
| Key | What it does |
|
|
36
|
+
| --- | --- |
|
|
37
|
+
| `native` | Forces a trusted native click instead of the occlusion-honest synthetic path. For file pickers and clipboard access |
|
|
38
|
+
| `holdMs` | Keeps the pointer **down** for that long between mousedown and mouseup. The only way to drive a hold-to-confirm control. `effect.heldMs` reports the hold actually achieved |
|
|
39
|
+
| `confirmDangerous` | A permission gate for a control Reticle judged destructive. It is **not** a duration |
|
|
40
|
+
|
|
41
|
+
<Warning>
|
|
42
|
+
`confirmDangerous` and `holdMs` are routinely confused because both show up around delete buttons.
|
|
43
|
+
`holdMs` is how long the button is held; `confirmDangerous` is permission to press it at all.
|
|
44
|
+
Passing the wrong one gets you either a refusal or a press that was never held.
|
|
45
|
+
</Warning>
|
|
46
|
+
|
|
47
|
+
### The refusal you will meet first
|
|
48
|
+
|
|
49
|
+
Reticle blocks a control that looks destructive before acting on it:
|
|
50
|
+
|
|
51
|
+
```json
|
|
52
|
+
{
|
|
53
|
+
"error": "potentially destructive action blocked; retry with args.confirmDangerous=true",
|
|
54
|
+
"recovery": "Reticle blocked a potentially destructive control (delete, remove, revoke…) on purpose. If you mean it, retry the same action with args.confirmDangerous set to true. This is a deliberate refusal, not a defect: there is nothing to report."
|
|
55
|
+
}
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
A stale ref is refused the same way, with `"ref 'e303' no longer resolves to an element"` and a recovery note telling you to re-query. Both refusals cost a turn and prevent a false green, which is the trade.
|
|
59
|
+
|
|
29
60
|
## `fill` versus `type`
|
|
30
61
|
|
|
31
62
|
`fill` replaces. `type` appends. Reaching for `type` when you meant `fill` is how a field ends up containing `hellohello`, and neither call will complain.
|
|
@@ -54,9 +85,9 @@ Several actions in one round trip:
|
|
|
54
85
|
```json
|
|
55
86
|
{
|
|
56
87
|
"steps": [
|
|
57
|
-
{ "ref": "
|
|
58
|
-
{ "ref": "
|
|
59
|
-
{ "ref": "
|
|
88
|
+
{ "ref": "e603", "action": "fill", "args": { "value": "admin@reticle.dev" } },
|
|
89
|
+
{ "ref": "e604", "action": "fill", "args": { "value": "hunter2" } },
|
|
90
|
+
{ "ref": "e605", "action": "click" }
|
|
60
91
|
]
|
|
61
92
|
}
|
|
62
93
|
```
|
|
@@ -78,7 +109,9 @@ Several actions in one round trip:
|
|
|
78
109
|
|
|
79
110
|
## What every action reports
|
|
80
111
|
|
|
81
|
-
`valueChanged`, `focusMoved`, `domMutatedWithin`, `defaultPrevented`, `occluded` and `
|
|
112
|
+
`dispatched`, `targetMatched`, `visible`, `enabled`, `valueChanged`, `focusMoved`, `domMutatedWithin`, `defaultPrevented`, `occluded`, `occludedBy` and `scrolledIntoView`. Plus the element's `testid`, `component`, `role`, `name` and `source` file and line.
|
|
113
|
+
|
|
114
|
+
Fields sitting at their uninformative default are **omitted**, so a clean action collapses to its consequence. A missing `visible` means `true`, not `false`. Read an absence as "nothing to report" rather than as a failure.
|
|
82
115
|
|
|
83
116
|
`occluded` is the answer to a large share of "the button doesn't work" reports: the click landed on the modal overlay sitting on top of it.
|
|
84
117
|
|
package/docs/agent-cheatsheet.md
CHANGED
|
@@ -4,6 +4,8 @@ description: 'One screen to get fluent: the look → act → observe → assert
|
|
|
4
4
|
icon: bolt
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
The loop is **look → act → observe → assert**, and only `reticle_act_and_wait` and `reticle_assert` produce a verdict. Reach for `reticle_act_and_wait({ ref, action, until })` first: it names the expected consequence before the action, which is the difference between a check and a rationalisation. `verified: "unknown"` is not a pass.
|
|
8
|
+
|
|
7
9
|
One screen to get fluent. Reticle is the **proof layer for AI agents**: no screenshots, no vision model, evidence not prose. Everything below returns structured data. Full guide: [usage.md](usage.md).
|
|
8
10
|
|
|
9
11
|
## The core loop: look → act → observe → assert
|
|
@@ -109,9 +111,9 @@ A claim is real only when the layers agree. Check more than the UI:
|
|
|
109
111
|
|
|
110
112
|
The tools advertised DIRECTLY, i.e. what you'll use 90% of the time. Everything else is one `reticle_run` away (see Token note below).
|
|
111
113
|
|
|
112
|
-
`reticle_sessions` · `reticle_navigate` · `reticle_snapshot` · `reticle_query` · `reticle_act` · `reticle_act_sequence` · `reticle_act_and_wait` · `reticle_observe` · `reticle_network` · `reticle_console` · `reticle_wait_for` · `reticle_assert` · `reticle_state` · `reticle_inspect` (DOM node → `src/App.tsx:104`) · `
|
|
114
|
+
`reticle_sessions` · `reticle_navigate` · `reticle_snapshot` · `reticle_query` · `reticle_act` · `reticle_act_sequence` · `reticle_act_and_wait` · `reticle_observe` · `reticle_network` · `reticle_console` · `reticle_wait_for` · `reticle_assert` · `reticle_state` · `reticle_inspect` (DOM node → `src/App.tsx:104`) · `reticle_feedback` (tell the maintainers what is missing) · `reticle_session` (hand back: `{action:"yield"}` the moment you stop driving, `{action:"resume"}` after a human pause).
|
|
113
115
|
|
|
114
|
-
Frequently useful but NOT core, so reach them through `reticle_run({ tool, args })`: `
|
|
116
|
+
Frequently useful but NOT core, so reach them through `reticle_run({ tool, args })`: `reticle_capabilities` (the app's whole testable surface in one call; `{fromDisk:true}` needs no browser), `reticle_domain` (learn the app + gaps), `reticle_baseline {action:"diff"}`, `reticle_project` (run history).
|
|
115
117
|
|
|
116
118
|
**Reach past core when…** you need to record/replay a journey (`reticle_record {action:"start"}/stop`, `reticle_replay`), persist a self-healing golden flow (`reticle_flow_save*` / `reticle_flow_replay` / `reticle_flow_heal`), compile annotations (`reticle_annotate`), explore autonomously (`reticle_explore` lists controls; `reticle_crawl` clicks them all and reports anomalies, and is **destructive**), reveal a virtualized off-screen row (`reticle_scroll_to`, for when `reticle_query` finds nothing because a windowed list hasn't rendered it yet), visual-check (`reticle_screenshot` / `reticle_visual_diff`, pinned with `reticle_viewport` for reproducible baselines), test error/edge states by stubbing the network (`reticle_network_mock`: 500 / offline / delay, driven mode), control time for toasts/debounces/auto-dismiss (`reticle_clock { freeze | advanceMs | reset }`), or work with a human (`reticle_session {action:"end"}` / `reticle_session {action:"resume"}` / `reticle_session {action:"messages"}`, and **`reticle_session {action:"review"}`** to drain + fix the bugs the human flagged from the panel).
|
|
117
119
|
|
package/docs/architecture.md
CHANGED
|
@@ -4,6 +4,8 @@ description: 'The moving parts, the data flow between them, and the design decis
|
|
|
4
4
|
icon: sitemap
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
Reticle has three moving parts: a dev-only **SDK** inside your page that instruments the DOM, network, console, routing and framework state; a local **server** that runs a WebSocket bridge to that SDK and an MCP server for your agent; and a git-checked **`.reticle/` workspace** holding flows, baselines and run artifacts. Your agent calls an MCP tool, the server turns it into a command over the WebSocket, the SDK runs it in the real page and streams structured events back.
|
|
8
|
+
|
|
7
9
|
> For engineers evaluating Reticle, integrating it at scale, or contributing. It explains the moving parts, the data flow, and the design decisions behind them. If you just want to get running, start with [getting-started.md](getting-started.md); come back here when you want to know _why_.
|
|
8
10
|
|
|
9
11
|
---
|
|
@@ -40,6 +42,10 @@ Reticle is a pnpm + Turborepo monorepo. The split is not cosmetic: each boundary
|
|
|
40
42
|
| `@reticlehq/server` | Node | The bridge, the MCP server, the `reticle` CLI, flow/run storage | Never imports DOM APIs |
|
|
41
43
|
| `@reticlehq/react` | the browser | The SDK **kit** you install in a browser app: re-exports the browser sensor (so one install gives both `reticle` and `install`) and maps a DOM node → React component → source `file:line` | Core works without the source-mapping half |
|
|
42
44
|
| `@reticlehq/babel-plugin`, `@reticlehq/next`, `@reticlehq/vite-plugin` | build time | Stamp `data-reticle-source` for source mapping (and, for Vite, inject `connect()`) | Plain tooling |
|
|
45
|
+
| `@reticlehq/electron` | Electron main process | Adapter for the desktop runtime: IPC observer, main-process capture | Desktop-only, outside the web gates |
|
|
46
|
+
| `packages/tauri` (`reticle-tauri`) | Tauri (Rust) | Capture backend for the Tauri desktop runtime | Rust, outside every JS gate; compiled by CI's `rust`/`rust-macos` jobs |
|
|
47
|
+
| `@reticlehq/test` | Node (dev dependency) | Spec runner + matchers used by CI's own test suites | Peer dependency on `vitest` |
|
|
48
|
+
| `@reticlehq/eslint-plugin` | dev tooling | Repo-internal lint rule: a state change must fire a signal | Dev-only, never shipped in a user's bundle |
|
|
43
49
|
|
|
44
50
|
> Note: pre-2.0, `@reticlehq/core` was a single umbrella package that re-exported all of the above under subpaths. In 2.0 the umbrella was retired and `@reticlehq/core` became the bottom-of-graph wire contract.
|
|
45
51
|
|
package/docs/benchmarks.md
CHANGED
|
@@ -4,6 +4,14 @@ description: 'How we measure Reticle, what each number actually means, and the p
|
|
|
4
4
|
icon: chart-column
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
Two different measurements, because there are two different questions.
|
|
8
|
+
|
|
9
|
+
Against a **Playwright script** over an 88-bug registry, each tool running its own native check deterministically with no model in the loop, Reticle caught **85 of 88** and Playwright **59 of 88**, both with zero false alarms on the clean build. The gap is not spread evenly: it is 26 bugs Playwright **structurally cannot** see, and on everything observable from outside the app the two tie exactly. See Part 5a.
|
|
10
|
+
|
|
11
|
+
Against **Playwright MCP and Chrome DevTools MCP**, the agent-driven tools where cost per look matters, Reticle caught 10 of 10 injected regressions to Playwright MCP's 9 and DevTools' 8, averaging 815 tokens per look against 1,292 and 758. See Part 5b.
|
|
12
|
+
|
|
13
|
+
Where Reticle loses is stated plainly in Part 5, and it loses in more places than these two tables show.
|
|
14
|
+
|
|
7
15
|
> This page assumes **zero** testing background. By the end you'll understand what software testing is, why AI coding agents made it urgent, how to tell a good verification tool from a bad one, and exactly how Reticle measures up against the main alternatives, including the places Reticle loses. If a term looks like jargon, it's defined the first time it appears.
|
|
8
16
|
>
|
|
9
17
|
> _Where the pictures come from, stated accurately: the **SVG charts** (`bench/artifacts/`) are generated from measured raws by `bench/harness/charts.mjs`. The **headline PNG cards** (`assets/readme/`) are screenshots of hand-authored HTML in `assets/benchmarks/src/`, rendered by `assets/benchmarks/render.mjs`; their figures are typed in by a human and are **not** read from `history.jsonl`, which this line previously claimed. Treat a card as a design asset that has to be updated by hand when a number moves, and `bench/SCORECARD.md` (plus its freshness banner) as the source of truth._
|
|
@@ -59,7 +67,9 @@ We compare Reticle against the two most credible agent-native browser tools, bec
|
|
|
59
67
|
- **Chrome DevTools MCP** is Google's tool exposing Chrome's DevTools to an agent.
|
|
60
68
|
- **(Baseline) screenshot agents**, the common "let the model look at a picture" approach.
|
|
61
69
|
|
|
62
|
-
Why not compare to traditional test frameworks (Playwright the library, Cypress, etc.)? Because those are written and maintained by humans; they don't answer "can an _agent_ verify its own work in the loop
|
|
70
|
+
Why not compare to traditional test frameworks (Playwright the library, Cypress, etc.) on the token passes? Because those are written and maintained by humans; they don't answer "can an _agent_ verify its own work in the loop," and a hand-written script has no token cost to compare. The three above do, so the cost comparison stays apples-to-apples.
|
|
71
|
+
|
|
72
|
+
There **is** a comparison against a Playwright script, and it answers a different question: not "what does it cost an agent to look" but "what can each tool structurally SEE, at all." That one is in Part 5a. It runs no model, so nothing about it depends on how well an agent drives, which is what makes it the more durable of the two results.
|
|
63
73
|
|
|
64
74
|
Tool versions are pinned in `bench/raw/run-meta.json` so any run is reproducible.
|
|
65
75
|
|
|
@@ -101,7 +111,45 @@ We ran this two ways on purpose: a **controlled toy app** (where we can inject e
|
|
|
101
111
|
|
|
102
112
|

|
|
103
113
|
|
|
104
|
-
### 5a. The
|
|
114
|
+
### 5a. The 88-bug head-to-head against a Playwright script
|
|
115
|
+
|
|
116
|
+
The largest and least model-dependent measurement here. An **88-bug registry** (`bench/pw-vs-reticle/bugs.mjs`), each bug injected into the same app, each tool running its NATIVE check: a Playwright script doing what a Playwright script does, Reticle doing what Reticle does. **No LLM in the loop**, so nothing depends on how well an agent happened to drive.
|
|
117
|
+
|
|
118
|
+
A tool "catches" a bug when its check correctly FAILS on the broken build **and** does not fail on the clean one. Failing on the clean build is a false alarm and is counted against the tool, not ignored.
|
|
119
|
+
|
|
120
|
+
| | Reticle | Playwright script |
|
|
121
|
+
| -------------------------------------- | ----------: | ----------------: |
|
|
122
|
+
| Bugs caught | **85 / 88** | 59 / 88 |
|
|
123
|
+
| Of what it can structurally catch | **85 / 86** | 57 / 60 |
|
|
124
|
+
| **False greens** (broken, reported OK) | **1** | **29** |
|
|
125
|
+
| False alarms on the clean build | 0 | 0 |
|
|
126
|
+
| Output bytes per bug | 9,261 | **5,849** |
|
|
127
|
+
|
|
128
|
+
Three things about that table are more important than the headline number.
|
|
129
|
+
|
|
130
|
+
**Two of Reticle's three non-catches are not misses.** They are the registry's `false-positive-trap` cases, builds that look broken and are not: flagging them would itself be a false alarm, so 0 of 2 is the correct score and a tool that "caught" them would be worse. The third, `iframe-stale-data` (deep-dom), is a **genuine miss** and is counted as a false green above rather than explained away.
|
|
131
|
+
|
|
132
|
+
**These figures were re-derived on the 2.9.0 branch and they moved.** The numbers first published here were recorded on 2026-07-24; 615 commits touched the harness, the fixture app, the SDK or the server before anybody re-ran it. Detection moved 86 → 85 and 60 → 59, and the per-bug output figure **inverted**: Reticle was published as the leaner of the two at 4,134 B against 7,899 B and measures 9,261 B against 5,849 B here. Reticle's own check being the more expensive one does not contradict the token-cost sections below. Those measure an AGENT-driven look against MCP tools, which is a different question. It does mean the earlier claim should not be repeated. Re-run `bench/pw-vs-reticle/run.mjs` before quoting any of this.
|
|
133
|
+
|
|
134
|
+
**Reticle does not win everything, and the ties are the honest part.** On everything observable from outside the app the two are exactly level: console 6/6 against 6/6, network 8/8 against 8/8, storage 5/5 against 5/5, visual UI 16/16 against 16/16, plus routing, timing, paint and chart bugs. Any evaluate-capable tool matches Reticle on what the DOM, the network buffer or a screenshot can show.
|
|
135
|
+
|
|
136
|
+
**The gap is one specific shape:** bugs where the screen is right and the app is wrong.
|
|
137
|
+
|
|
138
|
+
| Category | Reticle | Playwright | The bug |
|
|
139
|
+
| --- | --: | --: | --- |
|
|
140
|
+
| state | 8/8 | **0/8** | the UI renders a plausible value that contradicts the app's own store |
|
|
141
|
+
| business-logic | 6/6 | **0/6** | an action corrupts a field nothing renders. The record is wrong, the screen is fine |
|
|
142
|
+
| signal | 4/4 | **0/4** | an event the app emits about itself (hydration done, error boundary caught) |
|
|
143
|
+
| net-status | 4/4 | 1/4 | a swallowed 4xx/5xx. The request failed, a catch block ate it, the UI rendered |
|
|
144
|
+
| streams | 3/3 | 0/3 | an SSE or WebSocket frame anomaly a request-level view cannot see |
|
|
145
|
+
| perf | 3/3 | 1/3 | layout shift, or a render storm where the DOM is identical |
|
|
146
|
+
| deep-dom | 2/3 | 2/3 | a break deep in a subtree a snapshot elides. The one both now miss is `iframe-stale-data` |
|
|
147
|
+
|
|
148
|
+
That is the whole claim, stated as narrowly as it deserves: Reticle sees inside the running app, so it catches the class of bug where the outside looks correct. Outside that class it ties, and it says so.
|
|
149
|
+
|
|
150
|
+
Bug registry: `bench/pw-vs-reticle/bugs.mjs`. Runner: `bench/pw-vs-reticle/run.mjs`, which regenerates `results.json` locally (run artifacts are not committed). Scorecard: `bench/FALSE-GREEN-SCORECARD.md`.
|
|
151
|
+
|
|
152
|
+
### 5b. The controlled toy app (the demo)
|
|
105
153
|
|
|
106
154
|
Measured on the observation-cost pass (numbers regenerate from `bench/raw/`):
|
|
107
155
|
|
|
@@ -115,7 +163,7 @@ Measured on the observation-cost pass (numbers regenerate from `bench/raw/`):
|
|
|
115
163
|
- DevTools is a hair cheaper per look (758 vs 815 tokens), **but it's cheaper because it catches less.** On the metric that combines both (Verification Efficiency), Reticle leads at 12.27 vs 10.55.
|
|
116
164
|
- Playwright catches a lot but is ~1.6× more expensive per look, so its efficiency is lowest here.
|
|
117
165
|
|
|
118
|
-
###
|
|
166
|
+
### 5c. A real production app (the Reticle dashboard)
|
|
119
167
|
|
|
120
168
|
A toy app is a fair lab, but it's small. The harder, more honest test is a **real, complex app**. Take the [Reticle](https://reticle.sh) dashboard itself: React 19, authentication, live data, ~15 routes, a node-graph view, virtualized lists. We embedded the SDK (the Vite plugin) and drove the authenticated app with all three tools. Observing it **once** (the primary snapshot + the network log):
|
|
121
169
|
|
|
@@ -129,13 +177,13 @@ A toy app is a fair lab, but it's small. The harder, more honest test is a **rea
|
|
|
129
177
|
|
|
130
178
|
On a small page everything is cheap and the gap is modest. On a **big** page the structured read pulls ahead: Reticle is **2.1× leaner than Playwright MCP** and the cheapest overall. And only Reticle can read the app's own program state (`authenticated`, `userId`, `activeProjectId`) and assert the login _actually worked_ from a signal the app emits. The others can only look at the DOM and the network and **guess**.
|
|
131
179
|
|
|
132
|
-
###
|
|
180
|
+
### 5d. The kicker: a real bug, caught live
|
|
133
181
|
|
|
134
182
|
Here's what makes the real-app test matter. On the **very first run, before we instrumented anything**, Reticle's network observation flagged two endpoints returning **`500`**: `GET /api/v1/projects` and `/recovery/incidents`, both failing with `column "deleted_at" does not exist` (a database migration that hadn't been applied). **The page rendered fine.** The sidebar loaded, nothing looked broken; a screenshot agent would have called it _"done."_ Reticle saw the broken backend underneath, in one look.
|
|
135
183
|
|
|
136
184
|
That is the whole thesis, demonstrated on a real app we didn't cherry-pick: **"looks done" ≠ "is done," and the difference is usually non-visual.** (We then fixed the migration; all endpoints 200.)
|
|
137
185
|
|
|
138
|
-
###
|
|
186
|
+
### 5e. What each tool can actually do
|
|
139
187
|
|
|
140
188
|
Cost is half the story; capability is the other half. The marks below are what's **first-class and built-in** to each tool (all three are good tools, and they compose: drive with theirs, assert with Reticle):
|
|
141
189
|
|
|
@@ -153,10 +201,10 @@ Honesty is the point of this page, so here are the places Reticle does **not** w
|
|
|
153
201
|
|
|
154
202
|
The numbers above are for **one** verification. But a test suite's real job is the **same** check, over and over, every commit, every CI run. Here the picture changes shape:
|
|
155
203
|
|
|
156
|
-
- Reticle records a flow once, then **replays it with no AI model at all** (re-resolving each element and re-asserting the outcome) for **~
|
|
204
|
+
- Reticle records a flow once, then **replays it with no AI model at all** (re-resolving each element and re-asserting the outcome) for **~239 tokens per run** (re-measured 2026-08-11; it was ~175 before each flow started carrying a success oracle).
|
|
157
205
|
- Playwright MCP and DevTools MCP have no replay: re-checking means an agent **re-drives the whole flow with the model every time**, costing tens of thousands of tokens per run.
|
|
158
206
|
|
|
159
|
-
That's a **~
|
|
207
|
+
That's a **~127× cost difference per re-run**, and it grows with how often you run. This is where the "much cheaper than screenshots" claim becomes dramatic rather than incremental.
|
|
160
208
|
|
|
161
209
|

|
|
162
210
|
|
|
@@ -188,7 +236,7 @@ A fleet of agents (or a parallel suite) verifying the same app doesn't need a br
|
|
|
188
236
|
## Part 7: What this means for you
|
|
189
237
|
|
|
190
238
|
- **If you're a developer using an AI agent:** Reticle lets the agent confirm its own change actually worked (across the network call, the console, the route, the state) for a few hundred tokens, not a screenshot's thousands. "Done" starts meaning done.
|
|
191
|
-
- **If you run CI / a platform team:** the replay pass is the headline: deterministic re-verification with no model, so a known flow re-checks for
|
|
239
|
+
- **If you run CI / a platform team:** the replay pass is the headline: deterministic re-verification with no model, so a known flow re-checks for a couple of hundred tokens at ~0% flake, every commit.
|
|
192
240
|
- **If you're evaluating tools:** the axis that matters is _bugs caught per token, gated on catching them all_. Cheapness that misses regressions is a false economy.
|
|
193
241
|
|
|
194
242
|
You are now equipped to read any verification benchmark critically: ask "what's the catch rate, what's the false-positive rate, and what's the cost _per re-run_?" Then be suspicious of any vendor that won't show you where they lose.
|
|
@@ -204,15 +252,15 @@ A fair question from a large engineering org (think a Datadog- or Salesforce-siz
|
|
|
204
252
|
The win is **regression re-runs**, because that cost is paid over and over. Take a mid-size surface:
|
|
205
253
|
|
|
206
254
|
- **50 golden flows**, re-verified on **every PR**, at **200 PRs/day**.
|
|
207
|
-
- Reticle replays each flow **deterministically, with no model**: ~**
|
|
255
|
+
- Reticle replays each flow **deterministically, with no model**: ~**240 tokens/flow/run** (measured).
|
|
208
256
|
- The agent-driven alternative re-drives each flow with an LLM: ~**30,000 tokens/flow/run** (measured Layer-C comparison).
|
|
209
257
|
|
|
210
258
|
| | Per flow / run | 50 flows × 200 PRs/day | Per year (~250 working days) |
|
|
211
259
|
| --- | --- | --- | --- |
|
|
212
|
-
| Reticle replay (no model) | ~
|
|
260
|
+
| Reticle replay (no model) | ~240 tok | ~2.4 M tok/day | ~0.6 B tok/yr |
|
|
213
261
|
| Agent re-drive (LLM) | ~30,000 tok | ~300 M tok/day | ~75 B tok/yr |
|
|
214
262
|
|
|
215
|
-
That's a **~
|
|
263
|
+
That's a **~125× difference on the recurring axis**, and it scales linearly with flows × runs, the two numbers an enterprise has _a lot_ of. The deterministic replay also means **~0% verdict flake** (measured over repeated identical runs), which at this scale is the difference between a trusted gate and one engineers learn to ignore (recall: flaky suites get abandoned; that's the documented failure mode Reticle is built to avoid). And because replay needs no model, it has **no per-run API spend and no rate-limit ceiling**. It runs in CI like any other deterministic check.
|
|
216
264
|
|
|
217
265
|
### Where it does NOT help (stated plainly)
|
|
218
266
|
|
|
@@ -256,4 +304,4 @@ The same page is loaded in one browser under three conditions (full SDK, observe
|
|
|
256
304
|
|
|
257
305
|
**Result: instrumentation overhead is smaller than the method can resolve. Report it as `< 1.2 percentage points`, inside the 3% budget.** The HUD is measured and reported separately, because it is a developer-visible affordance you can turn off, not a cost of observing.
|
|
258
306
|
|
|
259
|
-
Getting here required correcting the method and then fixing a real bug. The two-condition version of this bench compared the full SDK against Reticle being absent, so it charged instrumentation for the HUD too, and reported **+5.85 pp (FAIL)**. Decomposing it showed almost none of that was instrumentation: a single `backdrop-filter: blur(24px)` on the HUD panel was costing **+4.03 pp** on its own, more than every observer combined, because a backdrop blur re-samples everything behind it whenever that content changes, and this fixture repaints at 60fps. Removing it took total main-thread time down 22%. Full history, including a fix that measured nothing and was reverted rather than kept, is in `bench/overhead/README.md`. Reproduce it yourself with `node bench/overhead/measure.mjs` (see `bench/overhead/README.md`, which records the ruled-out causes).
|
|
307
|
+
Getting here required correcting the method and then fixing a real bug. The two-condition version of this bench compared the full SDK against Reticle being absent, so it charged instrumentation for the HUD too, and reported **+5.85 pp (FAIL)**. Decomposing it showed almost none of that was instrumentation: a single `backdrop-filter: blur(24px)` on the HUD panel was costing **+4.03 pp** on its own, more than every observer combined, because a backdrop blur re-samples everything behind it whenever that content changes, and this fixture repaints at 60fps. Removing it took total main-thread time down 22%. Full history, including a fix that measured nothing and was reverted rather than kept, is in `bench/overhead/README.md`. Reproduce it yourself with `node bench/overhead/measure.mjs` (see `bench/overhead/README.md`, which records the ruled-out causes).
|
package/docs/best-practices.mdx
CHANGED
|
@@ -1,10 +1,12 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: Best practices: how to earn a green
|
|
2
|
+
title: 'Best practices: how to earn a green'
|
|
3
3
|
sidebarTitle: Best practices
|
|
4
4
|
description: The habits that make a verdict worth trusting, and the specific mistakes that produce a confident, wrong pass.
|
|
5
5
|
icon: star
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
+
A green is worth trusting when you named the consequence **before** you acted, ended the run on `reticle_act_and_wait` or `reticle_assert` (nothing else produces a verdict), scoped the assertion with `since`, and read `verified` rather than `pass`. Everything below is one of those habits, or the specific mistake that breaks it.
|
|
9
|
+
|
|
8
10
|
Everything here comes from driving Reticle against real apps and getting it wrong first. None of it is theoretical, and several entries exist because they bit us while writing these docs.
|
|
9
11
|
|
|
10
12
|
## Name the consequence before you act
|
|
@@ -87,11 +89,14 @@ Every `act` response returns a `since`. Use it.
|
|
|
87
89
|
|
|
88
90
|
Every verdict reports `honesty.grade`. Aim for `signal`.
|
|
89
91
|
|
|
90
|
-
| Grade | What it means
|
|
91
|
-
| ---------- |
|
|
92
|
-
| `presence` | Something appeared
|
|
93
|
-
| `state` | A registered store changed
|
|
94
|
-
| `
|
|
92
|
+
| Grade | What it means | How much it proves |
|
|
93
|
+
| ---------- | ----------------------------- | --------------------------- |
|
|
94
|
+
| `presence` | Something appeared | A mock does this too |
|
|
95
|
+
| `state` | A registered store changed | The app accepted it |
|
|
96
|
+
| `net` | A request fired and came back | The app talked to the world |
|
|
97
|
+
| `signal` | The app fired a named signal | The app declared success |
|
|
98
|
+
|
|
99
|
+
Ranked weakest to strongest, that is `presence` < `state` < `net` < `signal`. A predicate with no consequence at all grades `none`.
|
|
95
100
|
|
|
96
101
|
Getting from `presence` to `signal` is roughly ten minutes of [instrumentation](/instrumentation) per flow, and it is the highest-leverage work available.
|
|
97
102
|
|