@reticlehq/server 2.7.0 → 2.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -5
- package/SKILL.md +130 -778
- package/dist/bridge/bridge.d.ts +23 -0
- package/dist/bridge/bridge.js +80 -2
- package/dist/bridge/bridge.test-harness.d.ts +5 -0
- package/dist/bridge/bridge.test-harness.js +6 -4
- package/dist/capsule/causal-summary.d.ts +36 -1
- package/dist/capsule/causal-summary.js +33 -7
- package/dist/capsule/divergence.js +7 -1
- package/dist/cli/chromium-hint.d.ts +33 -5
- package/dist/cli/chromium-hint.js +82 -12
- package/dist/cli/cli-doctor.js +65 -34
- package/dist/cli/cli-flow-commands.js +8 -8
- package/dist/cli/cli-kill.d.ts +58 -0
- package/dist/cli/cli-kill.js +162 -0
- package/dist/cli/cli-parse.d.ts +16 -1
- package/dist/cli/cli-parse.js +32 -9
- package/dist/cli/cli-port.d.ts +32 -0
- package/dist/cli/cli-port.js +82 -35
- package/dist/cli/cli-verify.d.ts +21 -0
- package/dist/cli/cli-verify.js +47 -3
- package/dist/cli/config-discovery.d.ts +35 -0
- package/dist/cli/config-discovery.js +163 -0
- package/dist/cli/doctor-daemon-line.js +4 -1
- package/dist/cli/doctor-project-line.d.ts +37 -0
- package/dist/cli/doctor-project-line.js +35 -0
- package/dist/cli/doctor-rows.d.ts +51 -0
- package/dist/cli/doctor-rows.js +54 -0
- package/dist/cli/doctor-sessions-line.d.ts +12 -0
- package/dist/cli/doctor-sessions-line.js +69 -0
- package/dist/cli/drive-attach.d.ts +99 -0
- package/dist/cli/drive-attach.js +203 -0
- package/dist/cli/drive-command.d.ts +13 -0
- package/dist/cli/drive-command.js +103 -0
- package/dist/cli/open-note.d.ts +20 -0
- package/dist/cli/open-note.js +37 -0
- package/dist/cli/port-holder.d.ts +10 -0
- package/dist/cli/port-holder.js +26 -2
- package/dist/cli/status-next-action.d.ts +37 -0
- package/dist/cli/status-next-action.js +61 -0
- package/dist/cli.js +119 -39
- package/dist/crawl/crawl-tools.js +12 -2
- package/dist/crawl/crawl.d.ts +13 -0
- package/dist/crawl/crawl.js +50 -2
- package/dist/events/contradictions.d.ts +46 -0
- package/dist/events/contradictions.js +35 -22
- package/dist/events/declared.d.ts +45 -0
- package/dist/events/declared.js +73 -0
- package/dist/events/echo-mismatch.d.ts +8 -2
- package/dist/events/echo-mismatch.js +33 -5
- package/dist/events/event-filters.d.ts +1 -1
- package/dist/events/event-filters.js +7 -4
- package/dist/events/predicate-eval.d.ts +17 -72
- package/dist/events/predicate-eval.js +117 -236
- package/dist/events/predicate-parse.js +31 -3
- package/dist/events/predicate-route.d.ts +32 -0
- package/dist/events/predicate-route.js +102 -0
- package/dist/events/predicate-schema.d.ts +99 -0
- package/dist/events/predicate-schema.js +309 -0
- package/dist/events/predicate.d.ts +8 -0
- package/dist/events/predicate.js +296 -22
- package/dist/flows/annotate-tools.js +3 -0
- package/dist/flows/annotate.js +18 -4
- package/dist/flows/coverage.d.ts +38 -5
- package/dist/flows/coverage.js +30 -4
- package/dist/flows/decision.d.ts +15 -0
- package/dist/flows/decision.js +1 -1
- package/dist/flows/flow-classify.js +1 -1
- package/dist/flows/flow-replay-run.js +8 -1
- package/dist/flows/flow-tools.js +21 -5
- package/dist/flows/heal-run.js +1 -1
- package/dist/flows/replay.js +3 -0
- package/dist/honesty/accepted-write.d.ts +3 -0
- package/dist/honesty/accepted-write.js +8 -3
- package/dist/honesty/already-true.js +15 -0
- package/dist/honesty/blind-spots.js +10 -2
- package/dist/honesty/honesty.d.ts +11 -0
- package/dist/honesty/honesty.js +1 -0
- package/dist/honesty/uncaptured-bodies.d.ts +4 -2
- package/dist/honesty/uncaptured-bodies.js +9 -2
- package/dist/honesty/unread-outcome.d.ts +8 -0
- package/dist/honesty/unread-outcome.js +36 -9
- package/dist/honesty/unsettled.d.ts +49 -0
- package/dist/honesty/unsettled.js +82 -0
- package/dist/honesty/verified.d.ts +36 -3
- package/dist/honesty/verified.js +54 -11
- package/dist/http-server.d.ts +19 -0
- package/dist/http-server.js +202 -4
- package/dist/impact/impact-recorder.d.ts +23 -0
- package/dist/impact/impact-recorder.js +76 -0
- package/dist/impact/impact-store.d.ts +47 -0
- package/dist/impact/impact-store.js +171 -0
- package/dist/index.d.ts +7 -0
- package/dist/index.js +47 -10
- package/dist/init/agent-rules.d.ts +36 -0
- package/dist/init/agent-rules.js +132 -22
- package/dist/init/astro-patch.d.ts +2 -1
- package/dist/init/astro-patch.js +38 -6
- package/dist/init/capabilities.d.ts +22 -0
- package/dist/init/capabilities.js +76 -0
- package/dist/init/confirm.d.ts +114 -0
- package/dist/init/confirm.js +165 -0
- package/dist/init/connect-steps.d.ts +2 -0
- package/dist/init/connect-steps.js +50 -0
- package/dist/init/cra.js +3 -2
- package/dist/init/csp-check.d.ts +38 -0
- package/dist/init/csp-check.js +92 -0
- package/dist/init/csp-doctor.d.ts +25 -0
- package/dist/init/csp-doctor.js +49 -0
- package/dist/init/cursor.js +6 -9
- package/dist/init/detect.d.ts +8 -0
- package/dist/init/detect.js +33 -3
- package/dist/init/mcp-clients.d.ts +1 -0
- package/dist/init/mcp-clients.js +51 -7
- package/dist/init/mcp.d.ts +15 -0
- package/dist/init/mcp.js +19 -0
- package/dist/init/non-js-project.js +29 -5
- package/dist/init/plan-framework.d.ts +10 -0
- package/dist/init/plan-framework.js +78 -16
- package/dist/init/plan.d.ts +47 -6
- package/dist/init/plan.js +228 -69
- package/dist/init/run.d.ts +24 -2
- package/dist/init/run.js +157 -45
- package/dist/init/slash-command.d.ts +1 -1
- package/dist/init/slash-command.js +54 -6
- package/dist/init/snippets.d.ts +45 -5
- package/dist/init/snippets.js +194 -42
- package/dist/init/vite-config.js +53 -5
- package/dist/input/network-mock-tools.d.ts +19 -0
- package/dist/input/network-mock-tools.js +37 -5
- package/dist/input/scroll-find.d.ts +6 -0
- package/dist/input/scroll-find.js +17 -2
- package/dist/input/scroll-tools.js +4 -0
- package/dist/journal/attach-journal.d.ts +9 -1
- package/dist/journal/attach-journal.js +13 -2
- package/dist/journal/journal-query.d.ts +13 -0
- package/dist/journal/journal-query.js +28 -3
- package/dist/journal/workspace-gitignore.d.ts +10 -0
- package/dist/journal/workspace-gitignore.js +59 -0
- package/dist/license/license.js +1 -1
- package/dist/mcp/mcp-proxy.d.ts +35 -30
- package/dist/mcp/mcp-proxy.js +306 -104
- package/dist/mcp/mcp.d.ts +10 -1
- package/dist/mcp/mcp.js +70 -34
- package/dist/mcp/proxy-handshake.d.ts +82 -1
- package/dist/mcp/proxy-handshake.js +109 -2
- package/dist/mcp/proxy-lifecycle.d.ts +11 -2
- package/dist/mcp/proxy-lifecycle.js +12 -3
- package/dist/mcp/proxy-log.d.ts +63 -0
- package/dist/mcp/proxy-log.js +132 -0
- package/dist/mcp/server-instructions.d.ts +17 -0
- package/dist/mcp/server-instructions.js +56 -0
- package/dist/mcp/shared-params.d.ts +27 -0
- package/dist/mcp/shared-params.js +36 -0
- package/dist/pool/browser-pool.d.ts +22 -2
- package/dist/pool/browser-pool.js +44 -4
- package/dist/project/learned-routes.d.ts +24 -0
- package/dist/project/learned-routes.js +63 -0
- package/dist/project/project-store.d.ts +6 -0
- package/dist/project/project-store.js +25 -1
- package/dist/project/reticle-dir.d.ts +2 -0
- package/dist/project/reticle-dir.js +1 -0
- package/dist/session/connection-memory.d.ts +59 -0
- package/dist/session/connection-memory.js +108 -0
- package/dist/session/dev-command.d.ts +26 -0
- package/dist/session/dev-command.js +89 -0
- package/dist/session/dev-server-probe.d.ts +68 -0
- package/dist/session/dev-server-probe.js +102 -7
- package/dist/session/last-act.d.ts +26 -0
- package/dist/session/last-act.js +35 -1
- package/dist/session/live-control-tools.js +42 -5
- package/dist/session/no-session-diagnosis.d.ts +67 -2
- package/dist/session/no-session-diagnosis.js +236 -27
- package/dist/session/no-session-next-action.d.ts +46 -0
- package/dist/session/no-session-next-action.js +85 -0
- package/dist/session/no-session-watch.d.ts +40 -1
- package/dist/session/no-session-watch.js +174 -8
- package/dist/session/pending-commands.d.ts +12 -0
- package/dist/session/pending-commands.js +13 -1
- package/dist/session/ref-provenance.d.ts +54 -0
- package/dist/session/ref-provenance.js +61 -0
- package/dist/session/session-lease.js +1 -1
- package/dist/session/session-manager.d.ts +4 -0
- package/dist/session/session-manager.js +35 -5
- package/dist/session/session.d.ts +10 -1
- package/dist/session/session.js +146 -5
- package/dist/telemetry/app-instrumented.d.ts +8 -0
- package/dist/telemetry/app-instrumented.js +10 -0
- package/dist/telemetry/automation-hint.d.ts +17 -0
- package/dist/telemetry/automation-hint.js +71 -0
- package/dist/telemetry/bug-found.d.ts +15 -17
- package/dist/telemetry/bug-found.js +52 -21
- package/dist/telemetry/cli-telemetry.js +7 -1
- package/dist/telemetry/daemon-telemetry.js +24 -0
- package/dist/telemetry/feedback.js +1 -1
- package/dist/telemetry/init-telemetry.js +9 -1
- package/dist/telemetry/install-source.d.ts +56 -0
- package/dist/telemetry/install-source.js +74 -0
- package/dist/telemetry/instrumentation-stall.d.ts +50 -0
- package/dist/telemetry/instrumentation-stall.js +90 -0
- package/dist/telemetry/run-telemetry.js +4 -1
- package/dist/telemetry/session-metrics.d.ts +8 -0
- package/dist/telemetry/session-metrics.js +15 -0
- package/dist/telemetry/telemetry.d.ts +7 -1
- package/dist/telemetry/telemetry.js +26 -2
- package/dist/telemetry/tool-refused.d.ts +19 -0
- package/dist/telemetry/tool-refused.js +55 -0
- package/dist/telemetry/verification-of.js +46 -7
- package/dist/temp-dir.d.ts +27 -0
- package/dist/temp-dir.js +46 -0
- package/dist/tools/act-tools.js +209 -28
- package/dist/tools/assert-grade.d.ts +1 -1
- package/dist/tools/assert-grade.js +27 -5
- package/dist/tools/assert-verdict.js +49 -7
- package/dist/tools/browser-tools.js +10 -0
- package/dist/tools/contract-tools.js +4 -0
- package/dist/tools/dynamic-tools.js +16 -7
- package/dist/tools/error-recovery.d.ts +10 -0
- package/dist/tools/error-recovery.js +73 -9
- package/dist/tools/feedback-invite.js +1 -1
- package/dist/tools/invoke-tool.js +159 -26
- package/dist/tools/lease-hint.d.ts +30 -8
- package/dist/tools/lease-hint.js +65 -14
- package/dist/tools/lease-tools.d.ts +40 -1
- package/dist/tools/lease-tools.js +181 -83
- package/dist/tools/navigate-result.js +1 -1
- package/dist/tools/observe-tools.js +10 -4
- package/dist/tools/observed-nothing.js +1 -1
- package/dist/tools/react-grace.d.ts +52 -0
- package/dist/tools/react-grace.js +90 -0
- package/dist/tools/read-tools.js +67 -5
- package/dist/tools/resolve-target.d.ts +33 -0
- package/dist/tools/resolve-target.js +53 -0
- package/dist/tools/settle-in-flight.d.ts +94 -0
- package/dist/tools/settle-in-flight.js +146 -0
- package/dist/tools/snapshot-delta.d.ts +10 -0
- package/dist/tools/snapshot-delta.js +194 -9
- package/dist/tools/tool-kit.d.ts +8 -0
- package/dist/tools/tool-surface.d.ts +60 -0
- package/dist/tools/tool-surface.js +86 -1
- package/dist/tools/tools.js +61 -8
- package/dist/tools/unadvertised-help.js +9 -6
- package/docs/README.md +55 -11
- package/docs/actions.mdx +120 -0
- package/docs/agent-cheatsheet.md +95 -29
- package/docs/architecture.md +52 -35
- package/docs/benchmarks.md +127 -75
- package/docs/best-practices.mdx +216 -0
- package/docs/capabilities.mdx +534 -0
- package/docs/cli/affected.mdx +53 -0
- package/docs/cli/capsules.mdx +70 -0
- package/docs/cli/cloud.mdx +153 -0
- package/docs/cli/doctor.mdx +109 -0
- package/docs/cli/drive.mdx +74 -0
- package/docs/cli/feedback.mdx +67 -0
- package/docs/cli/gate.mdx +64 -0
- package/docs/cli/help.mdx +94 -0
- package/docs/cli/hunt.mdx +73 -0
- package/docs/cli/identify.mdx +76 -0
- package/docs/cli/init.mdx +121 -0
- package/docs/cli/kill.mdx +91 -0
- package/docs/cli/license.mdx +48 -0
- package/docs/cli/mcp.mdx +61 -0
- package/docs/cli/open.mdx +74 -0
- package/docs/cli/restart.mdx +60 -0
- package/docs/cli/rollback.mdx +35 -0
- package/docs/cli/serve.mdx +69 -0
- package/docs/cli/status.mdx +88 -0
- package/docs/cli/stop.mdx +64 -0
- package/docs/cli/telemetry.mdx +57 -0
- package/docs/cli/update.mdx +52 -0
- package/docs/cli/verify.mdx +76 -0
- package/docs/cli/version.mdx +39 -0
- package/docs/cli/watch.mdx +62 -0
- package/docs/cli.mdx +241 -0
- package/docs/debugging.md +16 -10
- package/docs/deploy-checks.md +45 -12
- package/docs/desktop.mdx +384 -0
- package/docs/docs.json +364 -0
- package/docs/enterprise.md +22 -16
- package/docs/faq.mdx +102 -0
- package/docs/fixtures.md +13 -7
- package/docs/flows.md +29 -23
- package/docs/for-agents.md +71 -0
- package/docs/frameworks.mdx +238 -0
- package/docs/gate-plan.md +48 -42
- package/docs/gates.md +28 -22
- package/docs/getting-started.md +129 -77
- package/docs/human-control.md +24 -18
- package/docs/index.mdx +143 -0
- package/docs/install-agentic.mdx +224 -0
- package/docs/install-manual.mdx +196 -0
- package/docs/instrumentation.mdx +238 -0
- package/docs/integration-patterns.md +41 -35
- package/docs/local-registry.md +19 -13
- package/docs/multi-agent-testing.md +19 -13
- package/docs/packages/babel-plugin.mdx +79 -0
- package/docs/packages/browser.mdx +111 -0
- package/docs/packages/core.mdx +116 -0
- package/docs/packages/electron.mdx +85 -0
- package/docs/packages/eslint-plugin.mdx +86 -0
- package/docs/packages/next.mdx +89 -0
- package/docs/packages/react.mdx +91 -0
- package/docs/packages/server.mdx +98 -0
- package/docs/packages/tauri.mdx +63 -0
- package/docs/packages/test.mdx +119 -0
- package/docs/packages/vite-plugin.mdx +94 -0
- package/docs/packages.mdx +218 -0
- package/docs/platform-integration.md +35 -26
- package/docs/predicates.mdx +248 -0
- package/docs/quickstart.mdx +257 -0
- package/docs/recipes.mdx +282 -0
- package/docs/skill-file.mdx +107 -0
- package/docs/state-management.mdx +199 -0
- package/docs/style.css +14 -0
- package/docs/system-map.md +50 -33
- package/docs/telemetry-contract.md +106 -52
- package/docs/telemetry-events.mdx +475 -0
- package/docs/telemetry.md +63 -55
- package/docs/testing.md +44 -6
- package/docs/token-efficiency.md +37 -27
- package/docs/tools-act-and-wait.mdx +173 -0
- package/docs/tools-act-sequence.mdx +137 -0
- package/docs/tools-act.mdx +95 -0
- package/docs/tools-assert.mdx +129 -0
- package/docs/tools-console.mdx +63 -0
- package/docs/tools-inspect.mdx +88 -0
- package/docs/tools-navigate.mdx +99 -0
- package/docs/tools-network.mdx +119 -0
- package/docs/tools-observe.mdx +145 -0
- package/docs/tools-overview.mdx +160 -0
- package/docs/tools-query.mdx +126 -0
- package/docs/tools-session-and-feedback.mdx +112 -0
- package/docs/tools-sessions.mdx +94 -0
- package/docs/tools-snapshot.mdx +129 -0
- package/docs/tools-state.mdx +104 -0
- package/docs/tools-tools-and-run.mdx +156 -0
- package/docs/tools-wait-for.mdx +85 -0
- package/docs/troubleshooting.mdx +263 -0
- package/docs/usage.md +209 -165
- package/docs/vs-chrome-devtools-mcp.mdx +123 -0
- package/docs/vs-playwright-mcp.mdx +125 -0
- package/docs/vs-screenshots.mdx +137 -0
- package/docs/what-is-reticle.mdx +63 -0
- package/docs/why-reticle.mdx +109 -0
- package/package.json +5 -5
- package/docs/desktop-apps.md +0 -256
- package/docs/matrix/2.5.0/claude-code-machine.json +0 -19
- package/docs/matrix/2.5.0/codex-machine.json +0 -20
- package/docs/matrix/2.5.0/cursor-machine.json +0 -22
- package/docs/matrix/2.5.0/gemini-machine.json +0 -22
- package/docs/matrix/2.5.0/opencode-machine.json +0 -22
- package/docs/matrix/2.5.0/vscode-machine.json +0 -22
- package/docs/matrix/2.5.0/windsurf-machine.json +0 -22
- package/docs/matrix/MATRIX.md +0 -26
- package/docs/matrix/README.md +0 -59
- package/docs/mint.json +0 -61
package/docs/system-map.md
CHANGED
|
@@ -1,16 +1,33 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
title: System map
|
|
3
|
+
description: 'How a tool call travels from a coding agent to your app and back, and which of those failures are silent.'
|
|
4
|
+
icon: map
|
|
5
|
+
---
|
|
2
6
|
|
|
3
|
-
|
|
7
|
+
A tool call crosses five processes and four hops: coding agent → `reticle mcp` proxy (stdio) → daemon on `:4400` (HTTP POST out, SSE back) → browser SDK (WebSocket) → your app. Each hop has its own failure vocabulary and its own way of lying, and the ones marked **silent** below fail with no error reaching either the agent or you.
|
|
8
|
+
|
|
9
|
+
> How a tool call gets from a coding agent to your app and back, what each hop can do wrong, and which of those failures are **silent**. Written for anyone touching the transport, the daemon, the bridge, or a gate, and for contributors trying to work out where their change lives.
|
|
4
10
|
>
|
|
5
11
|
> Companion pages: [`harness-rules.md`](../apps/e2e/harness-rules.md) (what a gate must do about all this) and [`telemetry-contract.md`](./telemetry-contract.md) (the rules for anything that emits).
|
|
6
12
|
|
|
7
|
-
## 1. Topology
|
|
8
|
-
|
|
9
|
-
```
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
13
|
+
## 1. Topology: 5 processes, 4 hops
|
|
14
|
+
|
|
15
|
+
```mermaid
|
|
16
|
+
flowchart LR
|
|
17
|
+
C["coding agent<br/>(client)"]
|
|
18
|
+
P["reticle mcp<br/>(proxy)"]
|
|
19
|
+
D["daemon :4400<br/>(bridge + tools)"]
|
|
20
|
+
S["browser SDK<br/>@reticlehq/browser"]
|
|
21
|
+
A["your app"]
|
|
22
|
+
F[(".reticle/<br/>flows · capsules<br/>baselines · runs")]
|
|
23
|
+
|
|
24
|
+
C -- "stdio / JSON-RPC" --> P
|
|
25
|
+
P -- "HTTP POST" --> D
|
|
26
|
+
D -- "SSE" --> P
|
|
27
|
+
D -- "WebSocket" --> S
|
|
28
|
+
S --> D
|
|
29
|
+
S --> A
|
|
30
|
+
D <--> F
|
|
14
31
|
```
|
|
15
32
|
|
|
16
33
|
Each hop has its own failure vocabulary, its own recovery, and its own way of lying. Nothing stated this end to end until this page, which is why gate results have been contradictory.
|
|
@@ -23,39 +40,39 @@ Each hop has its own failure vocabulary, its own recovery, and its own way of ly
|
|
|
23
40
|
| 2 | proxy probes `:4400` (`probeDaemon`, plain TCP, 500ms) | proxy | a squatter that accepts and never serves looks alive |
|
|
24
41
|
| 3 | absent → spawn daemon detached, poll to `DAEMON_READY_TIMEOUT_MS` (10s) | proxy | slow CI; raise with `RETICLE_DAEMON_READY_TIMEOUT_MS` |
|
|
25
42
|
| 4 | proxy opens SSE, waits for the `endpoint` frame → POST URL | proxy | an empty frame used to read as "connected"; now refused (`buildSessionUrl`) |
|
|
26
|
-
| 5 | `initialize
|
|
43
|
+
| 5 | `initialize`, answered **locally** after `LOCAL_HANDSHAKE_MS` (12s) when the daemon cannot | proxy | any client with a startup budget under ~12s loses Reticle for the whole session |
|
|
27
44
|
| 6 | daemon binds `:4400`, writes the pid file, starts bridge + idle watcher | daemon | `EADDRINUSE` → child exits 1, **parent already exited 0** |
|
|
28
45
|
| 7 | app calls `reticle.connect()` → `ws://localhost:4400/reticle` | SDK | wrong port; missing pairing token; mixed content (Safari throws synchronously on `new WebSocket`) |
|
|
29
46
|
| 8 | HELLO: `sessionId`, `projectId`, `protocolVersion`, capabilities | SDK → bridge | version mismatch → `1008` close → **the SDK stops retrying, permanently** |
|
|
30
47
|
| 9 | bridge authenticates: pairing token / loopback tier / Origin allow-list | bridge | another project's daemon answers → `authFailureReason` names it |
|
|
31
|
-
| 10 | Session registered, `everConnected = true` | daemon |
|
|
48
|
+
| 10 | Session registered, `everConnected = true` | daemon | n/a |
|
|
32
49
|
| 11 | tool call → `runTool` → `session.send` → WS command → SDK → reply | all | any hop above, mid-flight |
|
|
33
50
|
|
|
34
|
-
**Sequencing hazard:** steps 6 and 7 race. The product absorbs it internally
|
|
51
|
+
**Sequencing hazard:** steps 6 and 7 race. The product absorbs it internally (the first live tool call blocks briefly for a session to appear rather than failing), but a gate that boots an app and immediately asks whether a session exists is outside that protection and has been burned by it. Poll for the session; never sample once.
|
|
35
52
|
|
|
36
|
-
## 3. The tool graph
|
|
53
|
+
## 3. The tool graph: what produces what
|
|
37
54
|
|
|
38
55
|
Three different counts, all measured, and the difference between them matters:
|
|
39
56
|
|
|
40
57
|
| Count | What it is |
|
|
41
58
|
| --- | --- |
|
|
42
59
|
| **68** | name constants in `ReticleTool` |
|
|
43
|
-
| **48** | tools advertised under `RETICLE_ADVERTISE_ALL_TOOLS=1
|
|
44
|
-
| **
|
|
60
|
+
| **48** | tools advertised under `RETICLE_ADVERTISE_ALL_TOOLS=1`, i.e. what `tool-surface-sweep-test` drives (51 calls) |
|
|
61
|
+
| **19** | advertised by default; the rest are reached through `reticle_run` |
|
|
45
62
|
|
|
46
|
-
68 → 48 is **family folding**: `reticle_baseline`, `reticle_session`, `reticle_record`, `reticle_flow` and `reticle_lease` each absorb their members behind an `action` parameter. So "every tool is callable" is asserted over 48 surfaces, not 68 behaviours
|
|
63
|
+
68 → 48 is **family folding**: `reticle_baseline`, `reticle_session`, `reticle_record`, `reticle_flow` and `reticle_lease` each absorb their members behind an `action` parameter. So "every tool is callable" is asserted over 48 surfaces, not 68 behaviours; a family member reachable only through an `action` value the sweep never passes is not covered by it. Worth closing when the per-tool budget lands (see [`gate-plan.md`](./gate-plan.md), Phase 4).
|
|
47
64
|
|
|
48
65
|
| Token | Produced by | Consumed by | Invalidated by |
|
|
49
66
|
| --- | --- | --- | --- |
|
|
50
67
|
| `sessionId` | `reticle_sessions`, HELLO | ~every tool | tab close; **survives reload** via `sessionStorage` |
|
|
51
|
-
| `ref` (`e115`) | `snapshot`, `query`, `scroll_to` | `inspect`, `act`, `act_and_wait`, `state` | any re-render
|
|
68
|
+
| `ref` (`e115`) | `snapshot`, `query`, `scroll_to` | `inspect`, `act`, `act_and_wait`, `state` | any re-render; refuses cleanly rather than clicking the new occupant |
|
|
52
69
|
| `actionId` (`a2`) | `act`, `act_and_wait` | `observe`, `network`, `console` filters | window expiry |
|
|
53
70
|
| `since` cursor | every read | every read | ring-buffer eviction |
|
|
54
|
-
| `capsuleSaved` | `act_and_wait` | `replay` |
|
|
71
|
+
| `capsuleSaved` | `act_and_wait` | `replay` | n/a |
|
|
55
72
|
| flow | `record_start` → acts → `flow_save_recorded` | `flow_verify`, `flow_replay`, `flow_heal`, `affected`, `gate` | DOM drift → `flow_heal` |
|
|
56
73
|
| baseline | `baseline_save`, `viewport` | `visual_diff`, `diff` | viewport change |
|
|
57
74
|
| `session_lease` | direct tool results | the agent (`yield` / `end`) | **dropped on the `reticle_run` path** (#119) |
|
|
58
|
-
| run artifact | `run_export`, `reticle verify` | CI, `reticle gate` |
|
|
75
|
+
| run artifact | `run_export`, `reticle verify` | CI, `reticle gate` | n/a |
|
|
59
76
|
|
|
60
77
|
Two structural facts:
|
|
61
78
|
|
|
@@ -66,16 +83,16 @@ Two structural facts:
|
|
|
66
83
|
|
|
67
84
|
**Silent** = fails with no error surfaced to the agent or the user.
|
|
68
85
|
|
|
69
|
-
### Hop 1
|
|
86
|
+
### Hop 1: client ↔ proxy
|
|
70
87
|
|
|
71
88
|
| Failure | Silent | Guarded by |
|
|
72
89
|
| --- | --- | --- |
|
|
73
|
-
| proxy killed externally (the `lsof -ti` recipe) | **yes, total** |
|
|
90
|
+
| proxy killed externally (the `lsof -ti` recipe) | **yes, total** | `reticle kill` frees the port and leaves the agent's own proxy alone. See [harness rule 1](../apps/e2e/harness-rules.md) |
|
|
74
91
|
| proxy uncaught exception | no | `installProxyResilience` + `proxyLog` |
|
|
75
92
|
| proxy exits when the reconnect budget is spent | _was the worst bug in the product_ | `onReconnectBudgetSpent` → DORMANT, never exit |
|
|
76
93
|
| client closes stdin with calls in flight | no | `SHUTDOWN_DRAIN_MS` (5s) |
|
|
77
94
|
|
|
78
|
-
### Hop 2
|
|
95
|
+
### Hop 2: proxy ↔ daemon
|
|
79
96
|
|
|
80
97
|
Three distinct populations of unanswered call, **all covered**:
|
|
81
98
|
|
|
@@ -93,35 +110,35 @@ Still fragile:
|
|
|
93
110
|
| stale tool catalog served while dormant | **yes** | `ToolCatalogCache` answers `tools/list` from cache; after a version change a client can hold a catalog the daemon no longer implements |
|
|
94
111
|
| handshake replay regresses | **yes** | the daemon builds a fresh `McpServer` per SSE connection, so every `tools/call` would hit an uninitialized server |
|
|
95
112
|
|
|
96
|
-
### Hop 3
|
|
113
|
+
### Hop 3: daemon lifecycle
|
|
97
114
|
|
|
98
115
|
| Failure | Silent | Note |
|
|
99
116
|
| --- | --- | --- |
|
|
100
|
-
| `serve` exits 0 on `EADDRINUSE` | **yes** | #115
|
|
117
|
+
| `serve` exits 0 on `EADDRINUSE` | **yes** | #115: the parent reports the spawn, the child reports the bind, nothing joins them |
|
|
101
118
|
| liveness is the pid file, never the port | **yes** | `daemon/daemon.ts:156` |
|
|
102
|
-
| idle shutdown fires mid-install | **yes** | 5min base / 30min attached
|
|
119
|
+
| idle shutdown fires mid-install | **yes** | 5min base / 30min attached; already scored three fixtures as install failures |
|
|
103
120
|
| `isUselessDaemon` ends a daemon that served nothing | **yes** | lifetime facts, but a gate that installs for six minutes before its first call is that daemon |
|
|
104
121
|
| a daemon spawned inside a tool call is reaped when the call ends | **yes** | `SIGTERM` within ~1s, `detached` or not |
|
|
105
122
|
| no `daemon_alive` heartbeat | **fixed** | a killed daemon and a tidy exit used to read identically; the daemon now beats every 30s and `classifyDaemonLife` turns silence into `died_silently` (#123) |
|
|
106
123
|
| no port arbitration | **yes** | a second daemon can bind a port a first one held |
|
|
107
124
|
|
|
108
|
-
### Hop 4
|
|
125
|
+
### Hop 4: bridge ↔ SDK
|
|
109
126
|
|
|
110
127
|
| Failure | Silent | Note |
|
|
111
128
|
| --- | --- | --- |
|
|
112
129
|
| protocol mismatch → `1008` | no, but **terminal** | the SDK stops retrying by design; the app is dark until reload (#127) |
|
|
113
|
-
| another project's daemon answers | no | `authFailureReason`
|
|
114
|
-
| `rejectAll('session disconnected')` | **fixed** | used to become `verified:"no" / assertion_failed
|
|
130
|
+
| another project's daemon answers | no | `authFailureReason` is evidence-based, only claiming it when the daemon has demonstrably served another project |
|
|
131
|
+
| `rejectAll('session disconnected')` | **fixed** | used to become `verified:"no" / assertion_failed`, i.e. Reticle blaming the app, by file and line, for its own lost connection. Now `verified:"unknown" / observation_lost` (#124) |
|
|
115
132
|
| reaper ends a quiet session | no | revivable on the agent's next action |
|
|
116
133
|
| opaque origins (`tauri://localhost`, `file://`) | no | kept verbatim in the allow-list rather than collapsed to `"null"` |
|
|
117
134
|
|
|
118
|
-
### Hop 5
|
|
135
|
+
### Hop 5: SDK internals
|
|
119
136
|
|
|
120
137
|
| Failure | Silent | Note |
|
|
121
138
|
| --- | --- | --- |
|
|
122
139
|
| fixed 1s reconnect, no backoff, unbounded | **yes** | and the noise lands in the console channel we report to the agent (#116) |
|
|
123
140
|
| `onUnreachable` suppressed after the first success | **yes** | a **mid-session** outage never warns at all |
|
|
124
|
-
| offline queue overflow (`MAX_QUEUE = 500`) | no | gap marker stamped at the **oldest** drop
|
|
141
|
+
| offline queue overflow (`MAX_QUEUE = 500`) | no | gap marker stamped at the **oldest** drop, the honest floor for where the hole starts |
|
|
125
142
|
| `sessionStorage` throws in a sandboxed iframe | no | falls back; a session must still start |
|
|
126
143
|
|
|
127
144
|
## 5. Where the gates are
|
|
@@ -136,8 +153,8 @@ Still fragile:
|
|
|
136
153
|
| tools | surface, schema, refusals | `tool-surface-sweep-test`, `tool-fuzz-test` | latency/failure budget |
|
|
137
154
|
| verdict | honesty | `release-smoke-test` | transport ≠ app (#124); false-green rate |
|
|
138
155
|
| telemetry | silent absence | `telemetry-events-test`, `telemetry-contract.test.ts` | chokepoint coverage |
|
|
139
|
-
| install | pristine-surface wiring |
|
|
140
|
-
| client | per-agent MCP compatibility |
|
|
156
|
+
| install | pristine-surface wiring | nothing | the whole class |
|
|
157
|
+
| client | per-agent MCP compatibility | nothing | the whole class |
|
|
141
158
|
|
|
142
159
|
Nine of eleven layers have a gate. The two that do not are exactly the two that need a second repo and a human. The plan for both is in [`gate-plan.md`](./gate-plan.md).
|
|
143
160
|
|
|
@@ -146,7 +163,7 @@ Nine of eleven layers have a gate. The two that do not are exactly the two that
|
|
|
146
163
|
`runTool` opens one `tool.handler` root span per call and every stage underneath inherits it. With `RETICLE_TRACE=1` that makes three assertions available from any existing battery run, at the cost of one log parser:
|
|
147
164
|
|
|
148
165
|
- exactly one root span at `depth: 0` per tool call
|
|
149
|
-
- **zero orphan spans
|
|
166
|
+
- **zero orphan spans**; 23 orphans on one healthy run is the documented signature of a hung call
|
|
150
167
|
- `settleTiming` always fires; it sits in a `finally` so a thrown call cannot leak a concurrency slot
|
|
151
168
|
|
|
152
169
|
Today that output is produced and discarded.
|
|
@@ -1,12 +1,16 @@
|
|
|
1
|
-
|
|
1
|
+
---
|
|
2
|
+
title: Telemetry contract
|
|
3
|
+
description: 'Read this before adding a tool, an event, a finding kind, or a failure path. Telemetry fails silently.'
|
|
4
|
+
icon: file-contract
|
|
5
|
+
---
|
|
2
6
|
|
|
3
|
-
> For anyone
|
|
7
|
+
> For anyone (human or agent) adding a tool, an event, a finding kind, or a failure path to Reticle.
|
|
4
8
|
>
|
|
5
9
|
> The rules here are enforced by `packages/server/src/telemetry/telemetry-contract.test.ts`. If you break one, that test tells you which and where. This page is why.
|
|
6
10
|
|
|
7
11
|
## Why this has its own contract
|
|
8
12
|
|
|
9
|
-
**Telemetry fails silently.** Nothing throws when an event is missed. No test goes red. No user complains. The data is simply, permanently absent
|
|
13
|
+
**Telemetry fails silently.** Nothing throws when an event is missed. No test goes red. No user complains. The data is simply, permanently absent, and you find out months later when someone asks a question the data cannot answer, about a period you can never re-collect.
|
|
10
14
|
|
|
11
15
|
That has already happened here twice, and both times the code looked correct:
|
|
12
16
|
|
|
@@ -19,11 +23,11 @@ So the rule is not "remember to add telemetry". The rule is that the guard lives
|
|
|
19
23
|
|
|
20
24
|
### 1. Everything routes through a chokepoint
|
|
21
25
|
|
|
22
|
-
Tool usage, timing, errors, verifications and bugs are all recorded in **one place
|
|
26
|
+
Tool usage, timing, errors, verifications and bugs are all recorded in **one place**: `runTool` in `tools/invoke-tool.ts`. Adding a tool to `TOOLS` is all it takes to be instrumented.
|
|
23
27
|
|
|
24
28
|
Do **not** add telemetry inside a tool handler. If you find yourself wanting to, the metric probably belongs at the chokepoint, read off the result.
|
|
25
29
|
|
|
26
|
-
> The one exception is a path that genuinely does not go through `runTool
|
|
30
|
+
> The one exception is a path that genuinely does not go through `runTool`. Today that is only the verification runner (`reticle verify`, the HTTP verify surface), which has its own reporter in `telemetry/run-telemetry.ts`. **If you add a second dispatch path, it needs the same treatment**, and until it has one it is invisible. That gap existed for real: CI-found bugs were uncounted.
|
|
27
31
|
|
|
28
32
|
### 2. Names say what happened
|
|
29
33
|
|
|
@@ -31,20 +35,20 @@ Do **not** add telemetry inside a tool handler. If you find yourself wanting to,
|
|
|
31
35
|
|
|
32
36
|
## Counting defects: instances vs distinct
|
|
33
37
|
|
|
34
|
-
`bug_found` fires once per OCCURRENCE. A defect hit five times in a session is five events, which is the right raw signal
|
|
38
|
+
`bug_found` fires once per OCCURRENCE. A defect hit five times in a session is five events, which is the right raw signal; frequency is what says which classes of defect actually cost anybody anything. But it means a naive count answers "how often were defects hit", not "how many defects were found", while looking like it answers the second.
|
|
35
39
|
|
|
36
40
|
So every `bug_found` carries **`repeat`**: false the first time a KIND is seen in a session, true after. Count `repeat: false` for **distinct defects**; count everything for **instances**. Measured on a real app, the instance count was more than double the defect count. Publishing instances as defects inflates the claim accordingly.
|
|
37
41
|
|
|
38
42
|
The denominator is **`verification_completed`**, which fires per verdict with `via`, `verified`, `passed` and `falseGreenCaught`. Defects per verification is the honest rate; raw defect counts grow with usage and say nothing on their own.
|
|
39
43
|
|
|
40
|
-
**And `repeat` only means anything if the session remembers.** `SessionMetrics.reset()` runs at every periodic flush and used to clear the seen-kinds set with the window counters
|
|
44
|
+
**And `repeat` only means anything if the session remembers.** `SessionMetrics.reset()` runs at every periodic flush and used to clear the seen-kinds set with the window counters, so the same defect, re-found after a flush, reported `repeat: false` again. Sessions in the data run to 11.5 hours. Window counters zero on a flush; session-lifetime memory does not. (`session-window.test.ts`)
|
|
41
45
|
|
|
42
46
|
Two rules follow, and both are gated:
|
|
43
47
|
|
|
44
48
|
- **`repeat` is set at the EMISSION site, never by the classifier.** `bugsInResult` is a pure function over one tool result and cannot know what a session has already seen; if it ever grows a `repeat` field it will be guessing, and the guess becomes the published number. (`telemetry-contract.test.ts`)
|
|
45
|
-
- **Session-scoped, and it cannot be otherwise.** The payload carries no selector, URL or app detail by design, so the same defect in two sessions is unrecognisable as one
|
|
49
|
+
- **Session-scoped, and it cannot be otherwise.** The payload carries no selector, URL or app detail by design, so the same defect in two sessions is unrecognisable as one, and making it recognisable would require collecting exactly what this event refuses to collect.
|
|
46
50
|
|
|
47
|
-
The old set failed this so badly it confused its own authors
|
|
51
|
+
The old set failed this so badly it confused its own authors: `invoke` meant "the CLI ran" while `tool` meant "a tool was called", which is the opposite of how both read. A name that has to be looked up is a name that gets misread on a dashboard a year from now.
|
|
48
52
|
|
|
49
53
|
### 3. Names, never values
|
|
50
54
|
|
|
@@ -65,7 +69,7 @@ There is one narrow exception, and it is explicit rather than heuristic: paramet
|
|
|
65
69
|
If a set of kinds already exists in `@reticlehq/core`, **import it**. Do not re-list it.
|
|
66
70
|
|
|
67
71
|
```ts
|
|
68
|
-
// ✗ correct today, wrong the moment core gains a member
|
|
72
|
+
// ✗ correct today, wrong the moment core gains a member, and silent about it
|
|
69
73
|
const KINDS = new Set(['ui-advanced-request-failed', 'signal-contradicted' /* …10 more */]);
|
|
70
74
|
|
|
71
75
|
// ✓ cannot drift
|
|
@@ -78,40 +82,52 @@ A copied enum is a drift hazard anywhere. It is a **correctness** hazard when th
|
|
|
78
82
|
|
|
79
83
|
Every send is wrapped and best-effort. A telemetry failure must not fail a tool call, a verification, a daemon start, or `reticle init`.
|
|
80
84
|
|
|
81
|
-
The single exception is `daemon_stopped`, which is **awaited
|
|
85
|
+
The single exception is `daemon_stopped`, which is **awaited**, because the process exits immediately after and the send would otherwise be killed. Even then a failure resolves rather than throws.
|
|
82
86
|
|
|
83
87
|
## The event kinds, all of them
|
|
84
88
|
|
|
85
|
-
`TelemetryEventKind` in `@reticlehq/core` is the closed list. Seven of these went undocumented here for months
|
|
89
|
+
`TelemetryEventKind` in `@reticlehq/core` is the closed list. Seven of these went undocumented here for months (the doc described only the session-lifecycle half), so anyone building a dashboard from this page could not know that the transport and install events existed at all. A kind nobody documents is a kind nobody queries, and telemetry that nobody queries is telemetry nobody notices has stopped arriving. `telemetry-contract.test.ts` now fails when a kind is missing from this table.
|
|
86
90
|
|
|
87
91
|
| Kind | When | What it answers |
|
|
88
92
|
| --- | --- | --- |
|
|
89
93
|
| `reticle_installed` | first-ever run on a machine | install count, and the new-user curve |
|
|
90
|
-
| `cli_command_run` | a human ran a `reticle` subcommand | human intent
|
|
94
|
+
| `cli_command_run` | a human ran a `reticle` subcommand | human intent: `verify`/`gate` mean something very different from `status`. Never emitted for the internal `_daemon` spawn |
|
|
91
95
|
| `daemon_started` | the daemon came up | active sessions, DAU/WAU/MAU |
|
|
92
|
-
| `daemon_stopped` | clean exit | the rich session roll-up. **Count sessions with this one
|
|
96
|
+
| `daemon_stopped` | clean exit | the rich session roll-up. **Count sessions with this one**; see below |
|
|
93
97
|
| `session_progress` | periodic flush from a LIVE daemon | same payload, `final: false`. Sum work across both |
|
|
94
98
|
| `verification_completed` | a verdict was produced | the product's reason to exist: was an app actually verified |
|
|
95
|
-
| `project_profiled` | once per daemon start | stack, size, and how deeply the feature surface is used
|
|
99
|
+
| `project_profiled` | once per daemon start | stack, size, and how deeply the feature surface is used (activation vs retention) |
|
|
96
100
|
| `version_changed` | the running version differs from last seen | upgrade adoption, and whether a nudge caused it |
|
|
97
101
|
| `runtime_crashed` | an unhandled failure in the daemon | stability |
|
|
98
102
|
| `feedback_submitted` | `reticle feedback`, or an agent's report | the qualitative channel |
|
|
99
103
|
| `identified` | `reticle identify` | joins anonymous machine ids to a person who volunteered one |
|
|
100
104
|
| `mcp_client_connected` | an MCP client attached | how many sessions are agent-driven at all |
|
|
101
|
-
| `app_instrumented` | the first app carrying the SDK reached this daemon | **the funnel step everything turns on
|
|
105
|
+
| `app_instrumented` | the first app carrying the SDK reached this daemon | **the funnel step everything turns on**; see below |
|
|
106
|
+
| `instrumentation_stalled` | the daemon waited and no app ever arrived | the same funnel step, seen from the failure side; see below |
|
|
102
107
|
| `mcp_connection_lost` | the proxy lost its daemon | **the transport-stability metric.** The disconnect that makes a user reopen `/mcp` is invisible without it |
|
|
103
108
|
| `init_completed` | `reticle init` finished | does install actually work, outside the fixtures gate |
|
|
104
109
|
| `bug_found` | a defect was detected in the app under test | the value delivered, as opposed to the work done |
|
|
110
|
+
| `tool_refused` | a tool could not do what was asked | WHY the largest cohort in the funnel goes quiet. See below |
|
|
105
111
|
|
|
106
|
-
## The install has two halves
|
|
112
|
+
## The install has two halves: `app_instrumented`
|
|
107
113
|
|
|
108
114
|
Reticle is only usable when both halves are done: the MCP server is registered so the agent has the tools, and the SDK is loaded by a running page so there is something for those tools to look at. They are done by different commands, at different times, often in different directories. Almost everyone completes the first. The second is where the users go.
|
|
109
115
|
|
|
110
|
-
Nothing measured the second. `daemon_started` and `mcp_client_connected` describe the agent half. `session_appConnects` describes the app half but is a **window counter
|
|
116
|
+
Nothing measured the second. `daemon_started` and `mcp_client_connected` describe the agent half. `session_appConnects` describes the app half but is a **window counter**: it resets on every flush, so a user whose app connected in one window reads zero in every other. The population it under-counts is precisely the population being measured, and a funnel built on it reported fewer instrumented users than there were users calling tools, which is impossible on its face.
|
|
111
117
|
|
|
112
118
|
`app_instrumented` fires **once per daemon run**, on the first session-ready only, so `daemon_started` → `app_instrumented` is a rate rather than an inference and a reloading page cannot inflate it. It carries `initialized` (had `init` run here), `agentAttached` (was an agent already waiting), and `msToFirstApp` (how long the daemon sat with nothing wired). It deliberately carries no stack and no framework: `project_profiled` already reports both for the same run, and the two join on `sessionId`.
|
|
113
119
|
|
|
114
|
-
|
|
120
|
+
### The counterpart: `instrumentation_stalled`
|
|
121
|
+
|
|
122
|
+
`app_instrumented` fires on success and only on success, so the larger population, runs where no app ever arrives, exists in the data as an **absence**. An absence is the one thing telemetry cannot interpret: it cannot separate "nothing was ever wired" from "the process was killed before it could say so" from "we never sent the event". Since the funnel breaks at this step and nowhere else, this is the one silence worth converting into a signal.
|
|
123
|
+
|
|
124
|
+
`instrumentation_stalled` fires **once per daemon run**, only after the daemon has been up past a generous threshold, and **never once an app has connected**. A run that starts slowly and then works reports nothing at all. It carries the same three facts as its success counterpart so the two are directly comparable, with `msWaited` in place of `msToFirstApp`.
|
|
125
|
+
|
|
126
|
+
Read it against `agentAttached`, which is what makes it actionable rather than merely sad. A stalled daemon with no agent attached is somebody who installed and walked away, and is not a defect. A stalled daemon **with** an agent attached is the failure the product actually has: the tools are loaded, something is asking for them, and there is nothing on the other end to drive.
|
|
127
|
+
|
|
128
|
+
It deliberately carries no diagnosis. Whether a dev server is listening is a live probe, and running one on a timer to enrich a metric would be collecting for the metric's sake. The diagnosis belongs where a human or an agent asks for it.
|
|
129
|
+
|
|
130
|
+
What neither event can see is **why** an app never connected. Every cause for that is page-side (the non-localhost gate, a port mismatch, a stale build, a dev server never restarted), where the daemon has no visibility. That needs the SDK to report its own refusals, and it is the next thing to build.
|
|
115
131
|
|
|
116
132
|
## Sessions: `daemon_stopped` vs `session_progress`
|
|
117
133
|
|
|
@@ -119,49 +135,87 @@ Count sessions with **`daemon_stopped`** (`final: true`). It fires once, at a cl
|
|
|
119
135
|
|
|
120
136
|
A running daemon rolls its window up every 5 minutes as **`session_progress`** (`final: false`), same payload shape. Sum work across both; count sessions with neither summed nor doubled.
|
|
121
137
|
|
|
122
|
-
This split exists because the flush used to be emitted AS `daemon_stopped
|
|
138
|
+
This split exists because the flush used to be emitted AS `daemon_stopped`, an event named for an exit, fired while the process was alive. Anything counting sessions over the raw event therefore over-stated, and the two populations it merged are opposites: a flush comes from a daemon that has served tool calls, while a clean exit comes from one that has not, because a daemon that has served a tool never idle-exits and so never reaches a clean shutdown. A funnel over the raw event describes active sessions at one end and abandoned ones at the other.
|
|
123
139
|
|
|
124
|
-
The flush interval is also the **bound on what is lost**: nothing calls shutdown when a working daemon is finally killed, so its last partial window dies with it.
|
|
140
|
+
The flush interval is also the **bound on what is lost**: nothing calls shutdown when a working daemon is finally killed, so its last partial window dies with it. The old 30-minute interval was long enough to lose most of a typical session that way. Only non-empty windows emit, so a short interval costs nothing on the daemons that never serve a tool.
|
|
125
141
|
|
|
126
142
|
## The session summary's newer fields
|
|
127
143
|
|
|
128
|
-
Four counters and one flag were added because the data could not answer questions we were already asking. All are properties on events that already exist
|
|
144
|
+
Four counters and one flag were added because the data could not answer questions we were already asking. All are properties on events that already exist (no new kinds), and all four counters are **omitted rather than sent as zero**, so a field's presence is itself the signal.
|
|
129
145
|
|
|
130
146
|
| Field | On | Means |
|
|
131
147
|
| --- | --- | --- |
|
|
132
|
-
| `noSessionErrors` | session summary | tool calls that failed because there was no app to reach
|
|
148
|
+
| `noSessionErrors` | session summary | tool calls that failed because there was no app to reach: no session, no session by that id, or several with none named. The largest drop-off in the funnel; it was previously reachable only by unpacking `errors[]`. |
|
|
133
149
|
| `consecutiveRepeats` | session summary | longest back-to-back run per tool name. `toolCounts` reports five useful calls and five retries of one failing call identically, and those are opposite facts. |
|
|
134
|
-
| `abandonedActions` | session summary | actions driven with no verdict AFTER them
|
|
135
|
-
| `
|
|
136
|
-
| `verification.
|
|
137
|
-
| `verification.
|
|
138
|
-
| `verification.
|
|
139
|
-
| `
|
|
150
|
+
| `abandonedActions` | session summary | actions driven with no verdict AFTER them (the trailing unsettled run, not `actions - verifications`). That difference ignores order, so a verdict that drove nothing (a `flow_verify` over saved flows) silently paid for an abandoned action elsewhere. |
|
|
151
|
+
| `endedWithVerdict` | session summary (final only) | did this session ever produce a verdict. The headline metric, and previously the only thing in the payload that had to be COMPUTED, from lifetime counters sitting next to windowed ones, which is a subtraction that gets read wrong. Sent as `false` rather than omitted: a session that drove an app and never asked whether it worked is the finding. |
|
|
152
|
+
| `verification.browser` | `verification_completed` | `headless` \| `headed` \| `attached`: who DROVE the browser. `attached` (Reticle launched nothing, the SDK connected from a browser somebody else opened) is the common case in production, so on its own this is mostly "somebody's own browser". |
|
|
153
|
+
| `verification.brand` | `verification_completed` | WHICH browser it was: `chrome` \| `edge` \| `arc` \| `dia` \| `brave` \| `opera` \| `firefox` \| `safari` \| `other`, the closed `BrowserBrand` list in core. The axis `engine` cannot answer, since Chrome, Edge, Arc, Dia and Brave are all `blink`. The SDK reads `navigator.userAgentData.brands` (and the UA string on Firefox/Safari, which expose no `userAgentData`) and normalises IN THE PAGE: a raw brand or UA string is unbounded and fingerprintable and never leaves. Anything unrecognised is `other`. **Omitted rather than `"unknown"`** when the page did not say. A desktop webview has no brand and an older SDK does not report one, and a guess is indistinguishable from a measurement on a dashboard. |
|
|
154
|
+
| `verification.reason` | `verification_completed` | WHICH clause of `decideVerified` produced the verdict, from core's closed `VerifiedReason`: `inconclusive` \| `observation_lost` \| `assertion_failed` \| `contradicted` \| `already_true` \| `unclean_capture` \| `vacuous_grade` \| `outcome_pending` \| `outcome_unread` \| `unsettled` (the page never went idle and no consequence was declared) \| `evidence_incomplete` (the assertion held, but a channel's outcome had not arrived when the window closed) \| `proved`. See below. |
|
|
155
|
+
| `verification.uncleanLoss` | `verification_completed` | WHAT was lost when `reason` is `unclean_capture`, from core's closed `CaptureLoss`: `buffer_loss` (our server ring buffer evicted evidence from the window) \| `transport_gap` (our browser queue overflowed) \| `blind_spot` (a boundary in the page, such as a cross-origin frame or a closed shadow root) \| `other`. Three owners, three fixes, and one bar on a dashboard until this existed. ONE value, not a list: a multi-value property is not something a breakdown can group by, so the first is sent, ours before the page's. **Absent whenever the capture was clean**, so its presence is itself the signal. Reported as `other` rather than omitted when the block says dirty and names nothing: a gap there would read as "no unclean verdicts happened". |
|
|
156
|
+
| `bug.attribution` | `bug_found` | `app` \| `request` \| `reticle`: whose fault the defect was. **Absent means unclassified**, never `app`. See below. |
|
|
140
157
|
| `outage.stage` / `outage.reason` / `outage.attempts` | `mcp_connection_lost` | which stage of the outage, why the stream went away (closed `OutageReason`, `other` for anything unnamed), and how many reconnects had been tried. See below. |
|
|
158
|
+
| `installSource` | `reticle_installed`, `init_completed` | WHICH published route brought this install in, from core's closed `InstallSource`. Read from one self-declared marker (`RETICLE_INSTALL_SOURCE`) and NEVER inferred, so `unknown` is expected to dominate until every channel's own copy of the install command carries it. See below. |
|
|
159
|
+
| `init.confirmation` | `init_completed` | what `init` SAW after writing, from core's closed `InitConfirmation`: `connected` (an app carrying the SDK reached the daemon while it watched, and it is the only value that means installed) \| `no_daemon` (nothing was listening, so no session could arrive) \| `no_session` (a daemon was up and no app connected inside the window). **Absent means it never looked**, which is every scripted run: `init` waits only when a human is at the terminal. Read absent as "not measured", never as a failure to connect. |
|
|
160
|
+
| `automation` | every event | ADVISORY hint that the run looks automated when `CI` does not say so, from core's closed `AutomationHint`: `container` \| `hosted_workspace` \| `no_tty`. `ci` reads one environment variable set only by a runner, so a gate driven from a cloud sandbox lands as a human at a machine. **Never a filter**: people work in containers, in Codespaces, and over ssh with no terminal, and dropping a row because this is set drops real users. Absent means nothing looked automated, not that a human was present. |
|
|
141
161
|
| `tzOffsetMin` | every event | minutes offset from UTC. One integer, no location. |
|
|
142
162
|
| `versionChange.nudged` | `version_changed` | an agent had been told about exactly this version recently, so the nudge plausibly caused the update. The daemon that nudges and the `reticle update` that acts are different processes, so a marker file joins them. |
|
|
143
163
|
|
|
144
|
-
## Why
|
|
164
|
+
## Why they stopped: `tool_refused`
|
|
165
|
+
|
|
166
|
+
The refusal path computes a precise diagnosis, hands it to the agent as prose, and throws it away. So the biggest cohort in the funnel, the users who attach an agent and never drive, emitted nothing at all and was reachable only by subtracting two other numbers. Half of issue #172.
|
|
167
|
+
|
|
168
|
+
- `refusal_tool`: which tool, from our own fixed namespace. Never app data.
|
|
169
|
+
- `refusal_reason`: the closed `RefusalReason`: `no_session` | `no_match` | `unsupported` | `bad_args` | `not_ready` | `other`. Four different owners, and one undifferentiated "they stopped" number is actionable by none of them.
|
|
170
|
+
- `refusal_retried`: the call immediately before this one was the same tool, also refused.
|
|
171
|
+
|
|
172
|
+
Two things about it are deliberate and easy to get wrong later.
|
|
173
|
+
|
|
174
|
+
**The reason is derived from the recovery table, not from a second list of patterns.** `error-recovery.ts` is the one place a thrown message becomes a next action; `REASON_OF` is a `Record` over its keys, so a recovery added without a reason does not compile. A parallel regex list would have been correct the day it was written and silently wrong at the next addition. That is rule 4, on the exact kind of value that cannot be recovered afterwards.
|
|
175
|
+
|
|
176
|
+
**`retried` lands on the RETRY, not on the first refusal.** Reporting it the other way round means holding the first event back until the next call reveals whether one came, which loses it entirely for an agent that gives up, and that agent is the whole population this event exists to describe. So a refusal is sent the moment it happens, and the retry that follows carries the flag. Count `retried: true` for retries; the ratio against all refusals is whether our diagnosis gets anybody unstuck.
|
|
177
|
+
|
|
178
|
+
Capped at 50 per daemon run. Volume is part of this taxonomy's design and a stuck agent is exactly the shape that produces hundreds; `consecutiveRepeats` on the session summary still reports how long the loop ran.
|
|
179
|
+
|
|
180
|
+
## Which route brought them in: `installSource`
|
|
181
|
+
|
|
182
|
+
Four install routes ship at once (the SKILL.md paste URL, an `npx skills add` package, a Claude Code plugin, and docs.reticle.sh), and not one install could be attributed to any of them. Every decision about where to spend distribution effort was made blind.
|
|
183
|
+
|
|
184
|
+
There is exactly ONE mechanism, and it is a declaration rather than a detection: the channel sets `RETICLE_INSTALL_SOURCE` on the process that runs the install, and `install-source.ts` narrows it against core's `InstallSource`. Anything unrecognised reports `unknown`; an echo would put whatever somebody exported onto the wire.
|
|
185
|
+
|
|
186
|
+
Nothing infers. Three things look like signals and are not: `npm_config_user_agent` says npm ran us, and all four routes go through npx; a `.claude-plugin/` directory or an installed skill folder says a route is PRESENT, not that it ran `init`, and both are present on any machine that tried more than one; and which command ran first says nothing about who told the user to run it.
|
|
187
|
+
|
|
188
|
+
So `plugin` is the only route detectable without anybody typing anything (the plugin registers the MCP server and sets the marker in its `env`). `skill_file`, `npx_skill`, `docs_site` and `readme` are detectable only where that channel's own published copy of the install command carries the marker, and each of those is a separately published artifact. `cli_direct` is not detectable at all.
|
|
189
|
+
|
|
190
|
+
**`unknown` is therefore expected to be the largest bucket, and shrinking it is a distribution job rather than a classifier job.** Read a small `unknown` as a marker that spread. A guessed attribution would be worse than none: it is the number distribution decisions get steered on, and once a guess sits in the same column as a measurement the two cannot be told apart.
|
|
191
|
+
|
|
192
|
+
## Why a verdict came out that way: `verification.reason`
|
|
145
193
|
|
|
146
194
|
`verified` has three values. The rule that produces it has **eleven clauses**. Everything in between was thrown away at the moment it was known.
|
|
147
195
|
|
|
148
|
-
Captured against the real classifier: `verified: 'unknown'` covered "the agent malformed the call", "the consequence was already true", "the app answered 202", "a 2xx body went unread", "the capture was not clean", "nothing was asserted at a real grade" and "the page never settled"
|
|
196
|
+
Captured against the real classifier: `verified: 'unknown'` covered "the agent malformed the call", "the consequence was already true", "the app answered 202", "a 2xx body went unread", "the capture was not clean", "nothing was asserted at a real grade" and "the page never settled": **seven causes, two wire payloads**. They belong to three different owners (the agent, the app, Reticle) and need opposite responses: teach the agent, wait and re-check, or ship a fix. On a dashboard they were one bar. `verified: 'no'` collapsed the same way: "channels disagree" (Reticle earning its keep) and "the agent's predicate failed" were the same string.
|
|
149
197
|
|
|
150
|
-
`VerifiedReason` lives in `@reticlehq/core` and is the **single list**. `decideVerified` returns a member from every clause, so a new clause cannot compile without one; `verified.test.ts` drives
|
|
198
|
+
`VerifiedReason` lives in `@reticlehq/core` and is the **single list**. `decideVerified` returns a member from every clause, so a new clause cannot compile without one; `verified.test.ts` drives every member and fails if a member exists that no clause produces. `verification-of.ts` narrows the result field against `Object.values(VerifiedReason)`, and anything else is dropped rather than forwarded, because a string nobody can group by is worse than a gap. **Nothing re-lists these**, including the battery spec, which imports the enum from core's build.
|
|
151
199
|
|
|
152
200
|
Optional on purpose: a suite verdict (`flow_verify`) is a pass/fail with no clause behind it, and an older sender has none. Absent means unclassified.
|
|
153
201
|
|
|
154
|
-
## Whose defect it was
|
|
202
|
+
## Whose defect it was: `bug.attribution`
|
|
203
|
+
|
|
204
|
+
It shipped twice and was wrong both times. A real drive found that across two full runs EVERY `attribution: 'app'` was a misattribution, while the one defect that genuinely was a bad agent predicate carried none. A single session would have published "2 defects in the app" against a true count of 0. So it was removed, and then absence made "nobody classified this" and "we looked and could not tell" the same fact, which is the other half of the same problem.
|
|
205
|
+
|
|
206
|
+
It is back with two rules, and both are the thing the earlier versions lacked. Issue #122.
|
|
207
|
+
|
|
208
|
+
**Always present.** `unclassified` is a value, not a gap. Absence would mean an old sender or a path that forgot; a value means the classifier ran and declined. Count `app` for defects in anybody's product; exclude `unclassified` rather than folding it in.
|
|
155
209
|
|
|
156
|
-
`
|
|
210
|
+
**`app` requires positive evidence.** Something the app itself did: a request that came back failed, a signal the app fired carrying data that disagrees with its own screen, a written field echoed back changed. Never "nothing else explained it". That line already exists in core, as `ABSENCE_DERIVED_CONTRADICTIONS`, the kinds inferred from something NOT having happened inside a window Reticle chose the end of, and it is the same line that decides whether a verdict may say `no`. Reusing it rather than inventing a second judgement beside it is the point: every historical misattribution was an absence-derived kind, so this rule produces zero of them on the data that broke the last two versions.
|
|
157
211
|
|
|
158
|
-
|
|
212
|
+
Everything else is `unclassified` on purpose. A failed `element.present` covers "the button is missing", "the API is down" and "the agent mistyped a testid" identically. A console error can be the app, a browser extension, or a framework's dev overlay, and one of those was a real false positive here. A replay regression says a flow that used to pass no longer does, which is a regression somewhere, and the row does not say whether the app changed or a selector strategy of ours did.
|
|
159
213
|
|
|
160
|
-
|
|
214
|
+
Driver-side causes need no bucket of their own: a stale ref, a malformed call and a lost session produce no `bug_found` at all, because those paths are excluded upstream in `bugsInResult` before anything is counted.
|
|
161
215
|
|
|
162
|
-
## What is NOT a crash
|
|
216
|
+
## What is NOT a crash: expected disconnects
|
|
163
217
|
|
|
164
|
-
`runtime_crashed` answers exactly one question: is Reticle stable. One real session put **nine** events into it, all `write EPIPE
|
|
218
|
+
`runtime_crashed` answers exactly one question: is Reticle stable. One real session put **nine** events into it, all `write EPIPE`: the MCP client closed its half of the stdio pipe and the next `process.stdout.write` failed, which is how a client is supposed to leave.
|
|
165
219
|
|
|
166
220
|
`daemon-resilience.ts` matches `err.code` against `EPIPE` / `ECONNRESET` / `ERR_STREAM_DESTROYED` and logs `reticle_daemon_client_disconnected` (or `reticle_mcp_proxy_client_disconnected`) instead of emitting. Two rules make this safe rather than a hole:
|
|
167
221
|
|
|
@@ -170,19 +224,19 @@ It comes back when a verdict carries the reason it came out that way. Today the
|
|
|
170
224
|
|
|
171
225
|
A disconnect is also **no longer fatal for the daemon**: Node's "process state is undefined" guidance is about a throw that escaped everything, not about writing to a socket somebody closed, and exiting there let one departing client take down the daemon serving every other agent.
|
|
172
226
|
|
|
173
|
-
## Did MCP stay up
|
|
227
|
+
## Did MCP stay up: the `outage` block
|
|
174
228
|
|
|
175
|
-
The transport-stability metric shipped with an **empty payload** for months, and it is the exact failure this page opens with. `reportMcpOutage` passed `{ outage: { stage, reason, attempts } }`, `TelemetryExtra` declared the field, and it typechecked
|
|
229
|
+
The transport-stability metric shipped with an **empty payload** for months, and it is the exact failure this page opens with. `reportMcpOutage` passed `{ outage: { stage, reason, attempts } }`, `TelemetryExtra` declared the field, and it typechecked. But `emit()` builds its event from an **explicit allow-list of keys** and `outage` was not on it, nor in the `blocks` flattening map, nor in core's `TelemetryEventSchema`. Two deliberately different outages produced byte-identical events. Nothing threw, no test went red, and the data for that whole period cannot be recovered.
|
|
176
230
|
|
|
177
231
|
The lesson is not "wire the field". It is that **the battery asserted the event ARRIVED and never that it carried anything**, and a kind-only assertion cannot see an empty payload. When you add an event kind, the live check has to assert the FIELDS.
|
|
178
232
|
|
|
179
|
-
- `stage`
|
|
180
|
-
- `reason`
|
|
181
|
-
- `attempts
|
|
233
|
+
- `stage` is `first` (this session lost MCP at all) or `budget_spent` (it stopped retrying). These are the two facts the event exists to separate: the share of sessions that lose MCP, and the share where it never came back on its own.
|
|
234
|
+
- `reason` is the closed `OutageReason`: `sse_ended` | `sse_error` | `sse_aborted` | `sse_closed` | `connect_error` | `other`. The proxy's own reason strings are free text that also feeds a log, so `mcp-outage.ts` narrows them and reports **`other`** for anything unnamed. A classifier that cannot say "I don't know" lies instead, and an unbounded string must never reach the wire.
|
|
235
|
+
- `attempts`: consecutive reconnects tried when this was reported.
|
|
182
236
|
|
|
183
237
|
Still true and worth knowing when you query it: `mcp_connection_lost` carries **no `sessionId`** (it fires from the proxy process, not the daemon), and is capped at two per proxy process by design.
|
|
184
238
|
|
|
185
|
-
## Recording locally instead of sending
|
|
239
|
+
## Recording locally instead of sending: `RETICLE_TELEMETRY_FILE`
|
|
186
240
|
|
|
187
241
|
Set it to a path and every event is appended there as one JSON object per line, and **nothing is sent**. The payload is the one the wire would have carried, built by the same code and redacted by the same rules, so what a run records is what a user would have sent.
|
|
188
242
|
|
|
@@ -191,7 +245,7 @@ It exists for two reasons that pull the same way:
|
|
|
191
245
|
- **A release sweep is not a user.** Driving dozens of sessions through a gate emits real `daemon_started` / `verification_completed` / `bug_found` events, indistinguishable in PostHog from people. Test runs polluting the numbers is the same class of error as counting `cli_command_run { mcp }` as human intent: the metric stops describing what it claims to.
|
|
192
246
|
- **Verifying telemetry should not need a hand-rolled HTTP server.** Ad-hoc harnesses are how a check ends up measuring nothing.
|
|
193
247
|
|
|
194
|
-
One deliberate exception to the rules above: `RETICLE_TELEMETRY_FILE` keeps telemetry ENABLED inside a Reticle source checkout. The checkout guard exists to stop us phoning home, and writing a local file is not phoning home
|
|
248
|
+
One deliberate exception to the rules above: `RETICLE_TELEMETRY_FILE` keeps telemetry ENABLED inside a Reticle source checkout. The checkout guard exists to stop us phoning home, and writing a local file is not phoning home, while a release sweep is driven from exactly there, so a sink that inherited the guard would record nothing and look like it had worked.
|
|
195
249
|
|
|
196
250
|
`sent: true` from `reticle_feedback` means the record landed in the file, which is the honest reading of "captured" for a recorded run. An unwritable path degrades to a no-op and reports `false`; it never takes the daemon down.
|
|
197
251
|
|
|
@@ -200,13 +254,13 @@ One deliberate exception to the rules above: `RETICLE_TELEMETRY_FILE` keeps tele
|
|
|
200
254
|
| You are adding | Do this | Enforced by |
|
|
201
255
|
| --- | --- | --- |
|
|
202
256
|
| **A tool** | Add it to `TOOLS`. Nothing else. If its name implies a verdict (`assert`/`verify`), also add it to `VERIFICATION_TOOLS` | `telemetry-contract.test.ts` |
|
|
203
|
-
| **A verdict-producing tool** | `VERIFICATION_TOOLS
|
|
257
|
+
| **A verdict-producing tool** | Add it to `VERIFICATION_TOOLS`. Otherwise it emits no `verification_completed` and stops counting toward the product's headline metric | ✓ |
|
|
204
258
|
| **A contradiction / anomaly kind** | Add it to core's enum only. `bug-found.ts` derives from it | ✓ |
|
|
205
259
|
| **A new finding shape** in a tool result | Teach `bugsInResult` the field. Add a case to the contract test | ✓ |
|
|
206
|
-
| **A failure path** (connect, install, crash) | Classify it into an enum with an explicit `OTHER` bucket
|
|
207
|
-
| **An event kind** | Add to `TelemetryEventKind` + a payload schema + emit it + add a live check to `apps/e2e/specs/telemetry-events-test.mjs` that asserts the **fields**, not just that it arrived | partly
|
|
208
|
-
| **A block on `TelemetryExtra`** | Also add it to the `emit()` event build, the `blocks` flattening map, AND `TelemetryEventSchema`. Missing any one of the three drops the payload in silence
|
|
209
|
-
| **A dispatch path** that bypasses `runTool` | Give it a reporter like `run-telemetry.ts`, or it is invisible | ✗ **not enforced
|
|
260
|
+
| **A failure path** (connect, install, crash) | Classify it into an enum with an explicit `OTHER` bucket; a classifier that cannot say "I don't know" lies instead | ✓ |
|
|
261
|
+
| **An event kind** | Add to `TelemetryEventKind` + a payload schema + emit it + add a live check to `apps/e2e/specs/telemetry-events-test.mjs` that asserts the **fields**, not just that it arrived | partly; the live check is on you |
|
|
262
|
+
| **A block on `TelemetryExtra`** | Also add it to the `emit()` event build, the `blocks` flattening map, AND `TelemetryEventSchema`. Missing any one of the three drops the payload in silence (see `outage`) | ✗ **not enforced; be careful** |
|
|
263
|
+
| **A dispatch path** that bypasses `runTool` | Give it a reporter like `run-telemetry.ts`, or it is invisible | ✗ **not enforced; be careful** |
|
|
210
264
|
|
|
211
265
|
## Verifying it actually works
|
|
212
266
|
|
|
@@ -217,10 +271,10 @@ pnpm test:unit # the contract test + the finger
|
|
|
217
271
|
node apps/e2e/specs/telemetry-events-test.mjs # fires every event at a real endpoint, checks it lands
|
|
218
272
|
```
|
|
219
273
|
|
|
220
|
-
The second is the one that matters. It drives the real built modules against a real capture server
|
|
274
|
+
The second is the one that matters. It drives the real built modules against a real capture server (real network, real process semantics, real redaction) and asserts each event **arrives**. Half its checks are leak checks, asserting that secrets, passwords, customer emails and home directories are _absent_.
|
|
221
275
|
|
|
222
276
|
**Both halves are mutation-tested.** Reintroducing the fire-and-forget bug fails 9 checks; disabling redaction fails 3. A guard that cannot fail is theatre, so these are periodically proven to bite.
|
|
223
277
|
|
|
224
278
|
## The privacy line, in one sentence
|
|
225
279
|
|
|
226
|
-
We measure **that** something happened and **what class** of thing it was
|
|
280
|
+
We measure **that** something happened and **what class** of thing it was, never **what** it was, in whose app, or containing what.
|