@tech-leads-club/harness-toolkit 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +95 -0
- package/LICENSE +84 -0
- package/NOTICE +9 -0
- package/README.md +480 -0
- package/bin/tlc +10 -0
- package/bin/tlc-build +80 -0
- package/bin/tlc-cli.ts +1509 -0
- package/bin/tlc-exec +10 -0
- package/bin/tlc-exec.cmd +4 -0
- package/bin/tlc-exec.d.mts +60 -0
- package/bin/tlc-exec.mjs +248 -0
- package/bin/tlc.cmd +4 -0
- package/bin/tlc.mjs +7 -0
- package/bin/write-user-hooks.d.mts +40 -0
- package/bin/write-user-hooks.mjs +123 -0
- package/capabilities/catalog.json +418 -0
- package/config.example.json +51 -0
- package/dist/compact-before.mjs +7839 -0
- package/dist/doctor.mjs +8263 -0
- package/dist/help-topic.mjs +82 -0
- package/dist/init-project.mjs +993 -0
- package/dist/install-runtime.mjs +969 -0
- package/dist/lessons-cli.mjs +7258 -0
- package/dist/obs-cli.mjs +7025 -0
- package/dist/price-lookup.mjs +221 -0
- package/dist/prompt-submit.mjs +7860 -0
- package/dist/refresh-model-prices.mjs +158 -0
- package/dist/response-after.mjs +7879 -0
- package/dist/run.mjs +7830 -0
- package/dist/session-end.mjs +7937 -0
- package/dist/session-start.mjs +8056 -0
- package/dist/shim.mjs +52 -0
- package/dist/stop.mjs +8494 -0
- package/dist/subagent-start.mjs +7861 -0
- package/dist/subagent-stop.mjs +7851 -0
- package/dist/support.mjs +7045 -0
- package/dist/tlc-cli.mjs +8063 -0
- package/dist/tool-after.mjs +8113 -0
- package/dist/tool-before.mjs +7985 -0
- package/dist/tool-failure.mjs +7850 -0
- package/dist/uninstall-runtime.mjs +1307 -0
- package/docs/architecture.md +270 -0
- package/docs/concepts.md +562 -0
- package/docs/coverage.md +55 -0
- package/docs/decisions/ad-001.md +32 -0
- package/docs/decisions/ad-002.md +51 -0
- package/docs/decisions/ad-003.md +30 -0
- package/docs/decisions/ad-004.md +37 -0
- package/docs/decisions/ad-005.md +36 -0
- package/docs/decisions/ad-006.md +49 -0
- package/docs/decisions/ad-007.md +36 -0
- package/docs/decisions/ad-008.md +54 -0
- package/docs/decisions/ad-009.md +61 -0
- package/docs/decisions/ad-010.md +45 -0
- package/docs/decisions/ad-011.md +59 -0
- package/docs/decisions/ad-012.md +71 -0
- package/docs/decisions/ad-013.md +87 -0
- package/docs/decisions/ad-014.md +56 -0
- package/docs/decisions/ad-015.md +33 -0
- package/docs/decisions/ad-016.md +98 -0
- package/docs/decisions/ad-017.md +65 -0
- package/docs/decisions/ad-018.md +77 -0
- package/docs/decisions/ad-019.md +75 -0
- package/docs/decisions/ad-020.md +88 -0
- package/docs/decisions/ad-021.md +57 -0
- package/docs/decisions/ad-022.md +120 -0
- package/docs/decisions/ad-023.md +87 -0
- package/docs/decisions/ad-024.md +85 -0
- package/docs/decisions/ad-025.md +92 -0
- package/docs/decisions/ad-026.md +131 -0
- package/docs/decisions/ad-027.md +90 -0
- package/docs/decisions/ad-028.md +90 -0
- package/docs/decisions/ad-029.md +73 -0
- package/docs/decisions/ad-030.md +97 -0
- package/docs/decisions/ad-031.md +94 -0
- package/docs/decisions/ad-032.md +83 -0
- package/docs/decisions/ad-033.md +89 -0
- package/docs/decisions/ad-034.md +86 -0
- package/docs/decisions/ad-035.md +86 -0
- package/docs/decisions/ad-036.md +68 -0
- package/docs/decisions/ad-037.md +47 -0
- package/docs/decisions/ad-038.md +52 -0
- package/docs/decisions/ad-039.md +69 -0
- package/docs/decisions/ad-040.md +89 -0
- package/docs/decisions/ad-041.md +98 -0
- package/docs/decisions/ad-042.md +82 -0
- package/docs/decisions/ad-043.md +79 -0
- package/docs/decisions/ad-044.md +61 -0
- package/docs/decisions/ad-045.md +94 -0
- package/docs/decisions/ad-046.md +111 -0
- package/docs/decisions/ad-047.md +96 -0
- package/docs/decisions/ad-048.md +85 -0
- package/docs/decisions/ad-049.md +66 -0
- package/docs/decisions/ad-050.md +94 -0
- package/docs/decisions/ad-051.md +69 -0
- package/docs/decisions/ad-052.md +69 -0
- package/docs/decisions/ad-053.md +78 -0
- package/docs/decisions/ad-054.md +98 -0
- package/docs/decisions/ad-055.md +74 -0
- package/docs/decisions/ad-056.md +85 -0
- package/docs/decisions/ad-057.md +68 -0
- package/docs/decisions/ad-058.md +97 -0
- package/docs/decisions/ad-059.md +82 -0
- package/docs/decisions/ad-060.md +75 -0
- package/docs/decisions/ad-061.md +68 -0
- package/docs/decisions/ad-062.md +72 -0
- package/docs/decisions/ad-063.md +84 -0
- package/docs/decisions/ad-064.md +79 -0
- package/docs/decisions/ad-065.md +81 -0
- package/docs/decisions/ad-066.md +111 -0
- package/docs/decisions/ad-067.md +64 -0
- package/docs/decisions/ad-068.md +79 -0
- package/docs/decisions/ad-069.md +74 -0
- package/docs/decisions/ad-070.md +86 -0
- package/docs/decisions/ad-071.md +93 -0
- package/docs/decisions/ad-072.md +82 -0
- package/docs/decisions/ad-073.md +102 -0
- package/docs/decisions/ad-074.md +91 -0
- package/docs/decisions/ad-075.md +79 -0
- package/docs/decisions/ad-076.md +102 -0
- package/docs/decisions/ad-077.md +94 -0
- package/docs/decisions/ad-078.md +84 -0
- package/docs/decisions/ad-079.md +73 -0
- package/docs/decisions/ad-080.md +86 -0
- package/docs/decisions/ad-081.md +70 -0
- package/docs/decisions/ad-082.md +79 -0
- package/docs/decisions/ad-083.md +88 -0
- package/docs/decisions/index.md +111 -0
- package/docs/diagnose.md +248 -0
- package/docs/index.md +52 -0
- package/docs/init.md +101 -0
- package/docs/lessons.md +294 -0
- package/docs/log.md +139 -0
- package/docs/measure.md +178 -0
- package/docs/providers/claude-code.md +118 -0
- package/docs/providers/cursor.md +110 -0
- package/docs/providers/index.md +71 -0
- package/docs/troubleshooting.md +138 -0
- package/model-aliases.json +12 -0
- package/model-prices.cursor.json +410 -0
- package/model-prices.json +1 -0
- package/package.json +57 -0
- package/skills/harness-init/SKILL.md +241 -0
- package/skills/harness-init/evals/trigger_evals.json +86 -0
- package/skills/harness-init/references/capabilities.md +113 -0
- package/src/contracts/capabilities.ts +30 -0
- package/src/contracts/decision.ts +21 -0
- package/src/contracts/effort.ts +15 -0
- package/src/contracts/harness-event.ts +87 -0
- package/src/contracts/index.ts +6 -0
- package/src/contracts/policy-defaults.ts +14 -0
- package/src/contracts/wiring.ts +20 -0
- package/src/core/attest/attest.service.ts +0 -0
- package/src/core/capability/capability.service.ts +62 -0
- package/src/core/capability/capability.store.ts +51 -0
- package/src/core/capability/capability.types.ts +56 -0
- package/src/core/comment-policy/comment-policy.service.ts +307 -0
- package/src/core/comment-policy/comment-policy.types.ts +6 -0
- package/src/core/comment-policy/comment-resolvability.ts +134 -0
- package/src/core/comment-policy/comment-syntax.catalog.ts +310 -0
- package/src/core/comment-policy/comment-syntax.store.ts +65 -0
- package/src/core/comment-policy/comment-syntax.types.ts +24 -0
- package/src/core/core.facade.ts +461 -0
- package/src/core/duplication/duplication.service.ts +219 -0
- package/src/core/floor/floor.catalog.ts +54 -0
- package/src/core/floor/floor.paths.ts +90 -0
- package/src/core/floor/floor.policy-surface.ts +311 -0
- package/src/core/floor/floor.service.ts +306 -0
- package/src/core/floor/floor.tokenize.ts +179 -0
- package/src/core/floor/floor.verb.ts +28 -0
- package/src/core/floor/index.ts +1 -0
- package/src/core/gate/gate.artifact.ts +157 -0
- package/src/core/gate/gate.command.ts +130 -0
- package/src/core/gate/gate.findings.ts +201 -0
- package/src/core/gate/gate.inputs.ts +0 -0
- package/src/core/gate/gate.lock.ts +283 -0
- package/src/core/gate/gate.service.ts +28 -0
- package/src/core/gate/gate.types.ts +67 -0
- package/src/core/handoff/handoff.service.ts +43 -0
- package/src/core/handoff/handoff.store.ts +59 -0
- package/src/core/handoff/handoff.types.ts +89 -0
- package/src/core/index.ts +52 -0
- package/src/core/integrity/state-seal.ts +81 -0
- package/src/core/lesson/lesson.authored.ts +96 -0
- package/src/core/lesson/lesson.credit.ts +51 -0
- package/src/core/lesson/lesson.garden.ts +283 -0
- package/src/core/lesson/lesson.link.ts +93 -0
- package/src/core/lesson/lesson.score.ts +61 -0
- package/src/core/lesson/lesson.select.ts +209 -0
- package/src/core/lesson/lesson.service.ts +110 -0
- package/src/core/lesson/lesson.store.ts +317 -0
- package/src/core/lesson/lesson.sync.ts +70 -0
- package/src/core/lesson/lesson.types.ts +92 -0
- package/src/core/lesson/lesson.validity.ts +44 -0
- package/src/core/observability/observability.report.ts +326 -0
- package/src/core/observability/observability.service.ts +328 -0
- package/src/core/observability/observability.store.ts +269 -0
- package/src/core/observability/observability.types.ts +193 -0
- package/src/core/observability/observability.why.ts +206 -0
- package/src/core/observe/observe.service.ts +76 -0
- package/src/core/plan/plan.detect.ts +47 -0
- package/src/core/plan/plan.service.ts +81 -0
- package/src/core/plan/plan.types.ts +25 -0
- package/src/core/policy/policy.defaults.ts +103 -0
- package/src/core/policy/policy.guard.ts +34 -0
- package/src/core/policy/policy.integrity.ts +238 -0
- package/src/core/policy/policy.loader.ts +118 -0
- package/src/core/policy/policy.operator.ts +82 -0
- package/src/core/policy/policy.posture.ts +79 -0
- package/src/core/policy/policy.rails.ts +45 -0
- package/src/core/policy/policy.types.ts +157 -0
- package/src/core/presence/presence.service.ts +111 -0
- package/src/core/presence/presence.store.ts +59 -0
- package/src/core/presence/presence.types.ts +9 -0
- package/src/core/release/release.decisions.ts +147 -0
- package/src/core/release/release.seen.ts +38 -0
- package/src/core/shell-policy/shell-policy.service.ts +215 -0
- package/src/core/shell-policy/shell-policy.stall.ts +57 -0
- package/src/core/shell-policy/shell-policy.types.ts +26 -0
- package/src/core/ship/ship.ledger.ts +76 -0
- package/src/core/ship/ship.service.ts +137 -0
- package/src/core/ship/ship.types.ts +19 -0
- package/src/core/stagnation/stagnation.resolution.ts +87 -0
- package/src/core/stagnation/stagnation.service.ts +20 -0
- package/src/core/stagnation/stagnation.store.ts +49 -0
- package/src/core/stagnation/stagnation.types.ts +12 -0
- package/src/core/subagent-policy/subagent-policy.parent-model.ts +202 -0
- package/src/core/subagent-policy/subagent-policy.service.ts +139 -0
- package/src/core/subagent-policy/subagent-policy.types.ts +22 -0
- package/src/core/supply-chain/supply-chain.catalog.ts +51 -0
- package/src/core/supply-chain/supply-chain.service.ts +217 -0
- package/src/core/turn/turn.activity.ts +94 -0
- package/src/core/turn/turn.autopilot.ts +131 -0
- package/src/core/turn/turn.failure-signals.ts +193 -0
- package/src/core/turn/turn.loop-counter.ts +71 -0
- package/src/core/turn/turn.types.ts +14 -0
- package/src/core/untrusted/untrusted.detect.ts +53 -0
- package/src/core/untrusted/untrusted.recall.ts +87 -0
- package/src/core/untrusted/untrusted.service.ts +129 -0
- package/src/core/untrusted/untrusted.store.ts +75 -0
- package/src/core/untrusted/untrusted.types.ts +42 -0
- package/src/entrypoints/compact-before.ts +14 -0
- package/src/entrypoints/prompt-submit.ts +25 -0
- package/src/entrypoints/response-after.ts +60 -0
- package/src/entrypoints/run.ts +179 -0
- package/src/entrypoints/session-end.ts +58 -0
- package/src/entrypoints/session-start.ts +172 -0
- package/src/entrypoints/shim.ts +46 -0
- package/src/entrypoints/stop.ts +876 -0
- package/src/entrypoints/subagent-start.ts +16 -0
- package/src/entrypoints/subagent-stop.ts +32 -0
- package/src/entrypoints/support.ts +182 -0
- package/src/entrypoints/tool-after.ts +111 -0
- package/src/entrypoints/tool-before.ts +186 -0
- package/src/entrypoints/tool-failure.ts +26 -0
- package/src/platform/backoff.ts +49 -0
- package/src/platform/cli-output.ts +30 -0
- package/src/platform/env-scope.ts +22 -0
- package/src/platform/fs-atomic.ts +142 -0
- package/src/platform/fs-jsonl.ts +24 -0
- package/src/platform/git.ts +187 -0
- package/src/platform/paths.ts +80 -0
- package/src/platform/pricing.ts +229 -0
- package/src/platform/process.ts +64 -0
- package/src/platform/sanitize.ts +23 -0
- package/src/platform/screen.ts +46 -0
- package/src/platform/style.ts +103 -0
- package/src/providers/claude/claude.capabilities.ts +25 -0
- package/src/providers/claude/claude.detect.ts +14 -0
- package/src/providers/claude/claude.inbound.ts +252 -0
- package/src/providers/claude/claude.lessons-view.ts +33 -0
- package/src/providers/claude/claude.outbound.ts +65 -0
- package/src/providers/claude/claude.policy-defaults.ts +9 -0
- package/src/providers/claude/claude.transcript.ts +62 -0
- package/src/providers/claude/claude.wiring.ts +245 -0
- package/src/providers/claude/index.ts +17 -0
- package/src/providers/cursor/cursor.capabilities.ts +28 -0
- package/src/providers/cursor/cursor.detect.ts +14 -0
- package/src/providers/cursor/cursor.inbound.ts +203 -0
- package/src/providers/cursor/cursor.lessons-view.ts +30 -0
- package/src/providers/cursor/cursor.outbound.ts +43 -0
- package/src/providers/cursor/cursor.policy-defaults.ts +9 -0
- package/src/providers/cursor/cursor.wiring.ts +214 -0
- package/src/providers/cursor/index.ts +17 -0
- package/src/providers/index.ts +9 -0
- package/src/providers/provider.degrade.ts +152 -0
- package/src/providers/provider.port.ts +20 -0
- package/src/providers/provider.registry.ts +28 -0
- package/tools/doctor.ts +631 -0
- package/tools/help-topic.ts +74 -0
- package/tools/init-project.ts +277 -0
- package/tools/install-runtime.ts +164 -0
- package/tools/lessons-cli.ts +474 -0
- package/tools/obs-cli.ts +175 -0
- package/tools/price-lookup.ts +57 -0
- package/tools/refresh-model-prices.ts +210 -0
- package/tools/test-env.mjs +28 -0
- package/tools/test-env.names.d.mts +1 -0
- package/tools/test-env.names.mjs +14 -0
- package/tools/uninstall-runtime.ts +463 -0
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-058 — Gates diff against the turn's base, and language knowledge is one table"
|
|
4
|
+
description: "A turn that commits used to erase its own violations, because every stop-time gate diffed against HEAD. The comparison point is now the revision the turn started at. Comment syntax moves from three hand-written regexes to one catalog module, after measuring that a grammar-based parser would not remove per-language knowledge."
|
|
5
|
+
tags: [decision, comments, gates, git, languages]
|
|
6
|
+
timestamp: "2026-08-10"
|
|
7
|
+
migration: "If a project has the comment gate on, expect it to start firing on turns that commit and on languages it never covered. Nothing changes in your config."
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# AD-058 — Gates diff against the turn's base, and language knowledge is one table
|
|
11
|
+
|
|
12
|
+
- **status**: active
|
|
13
|
+
- **date**: 2026-08-10
|
|
14
|
+
|
|
15
|
+
## Decision
|
|
16
|
+
|
|
17
|
+
1. **`listChangedRepoFiles` and `listAddedLines` take a base revision**, and `stop` passes the sha recorded at
|
|
18
|
+
`prompt.submit`. Absent, the base is `HEAD` — the previous behaviour exactly.
|
|
19
|
+
2. **Comment syntax is one catalog module**, `comment-syntax.catalog.ts`: line prefixes, block pairs and
|
|
20
|
+
continuation prefixes per extension. Adding a language is an entry, never code.
|
|
21
|
+
3. **No parser, and no runtime dependency.** Measured against the alternatives below.
|
|
22
|
+
4. **An unknown extension yields no findings and is named** in the observation record, rather than reading as
|
|
23
|
+
"the property held".
|
|
24
|
+
5. **`tlc harness policy accept` says which project it applied to**, gains `--all`, and says so plainly when the
|
|
25
|
+
project it ran in has no recorded baseline.
|
|
26
|
+
|
|
27
|
+
## Why
|
|
28
|
+
|
|
29
|
+
**A commit inside the turn erased the violation.** `listAddedLines` ran `git diff --unified=0 HEAD`, so once a
|
|
30
|
+
turn committed, its own added lines were "pre-existing" and the file list was empty — which made
|
|
31
|
+
`codeTargets.length > 0` skip the comment gate outright. Reported from a project with
|
|
32
|
+
`comments: { enabled: true, mode: "strict" }` where a turn added a docstring and a JSDoc block and nothing
|
|
33
|
+
fired; the task was named "schema v2 + tests + commit". This was never comment-specific: the plan gate and the
|
|
34
|
+
ship gate read the same list.
|
|
35
|
+
|
|
36
|
+
The turn boundary is `prompt.submit`, which already resets the untrusted-content marker for the same reason, so
|
|
37
|
+
the base has an obvious producer and one writer.
|
|
38
|
+
|
|
39
|
+
**A grammar would not have removed the per-language knowledge.** The research finding that decided this: tree-sitter
|
|
40
|
+
grammars name comment nodes inconsistently — `comment`, `line_comment`, `block_comment`, and `string` for a Python
|
|
41
|
+
docstring — so a per-language kind allowlist is required whichever tool is used. What a parser adds is cost, not
|
|
42
|
+
coverage:
|
|
43
|
+
|
|
44
|
+
| Option | What it costs | Verdict |
|
|
45
|
+
| --- | --- | --- |
|
|
46
|
+
| `@ast-grep/napi` | platform-specific native binaries across eight targets; only JS-ecosystem languages built in, the rest via `registerDynamicLanguage` | Refused: [/decisions/ad-012.md](/decisions/ad-012.md) ships no binary |
|
|
47
|
+
| tree-sitter WASM | one grammar `.wasm` per language, megabytes into a 1.2 MB package, **plus** the kind allowlist | Refused: cost without the benefit |
|
|
48
|
+
| Catalog module | one table, zero dependencies, zero I/O | Chosen |
|
|
49
|
+
|
|
50
|
+
So the shape that answers the objection is not a parser. It is **one scanner and one table**: the per-language
|
|
51
|
+
knowledge exists in exactly one place, as data, and the code that reads it never grows.
|
|
52
|
+
|
|
53
|
+
**A module rather than a data file.** The first version read `comment-syntax.json` from the runtime home and
|
|
54
|
+
promptly broke its own suite, because the suite runs with a deliberately empty runtime home
|
|
55
|
+
([/decisions/ad-042.md](/decisions/ad-042.md)). A hook is a process, so a data file is one more read per
|
|
56
|
+
invocation and one more path that resolves differently under Bun, under `dist/` and under a test. Bundled, it
|
|
57
|
+
costs nothing and behaves identically everywhere. The trade-off is real and stated below.
|
|
58
|
+
|
|
59
|
+
**Following the modular rules the repository already applies**: the catalog is the external vocabulary, and
|
|
60
|
+
`comment-syntax.store.ts` is the anti-corruption layer over it. The scanner asks "how does this file write a
|
|
61
|
+
comment" and is answered in the domain's own type; it knows nothing about extensions, languages or file formats.
|
|
62
|
+
The aggregate stays flat — catalog, store, types, service, all in `comment-policy/`.
|
|
63
|
+
|
|
64
|
+
## Trade-offs
|
|
65
|
+
|
|
66
|
+
**A delimiter scanner cannot see a comment inside a string literal, and cannot track a block across
|
|
67
|
+
non-contiguous diff hunks.** Neither could the regexes it replaces. What it can do is stop guessing: `#` is a
|
|
68
|
+
comment in Python and a private field in TypeScript, and the catalog is what knows the difference.
|
|
69
|
+
|
|
70
|
+
**Adding a language now requires a release.** With a JSON file an operator could have added one locally. That
|
|
71
|
+
route also made the scanner depend on a file that may not exist, and the language a project needs is a change
|
|
72
|
+
worth shipping to everyone rather than carrying privately.
|
|
73
|
+
|
|
74
|
+
**Coverage is finite and now visible.** Forty-one languages, a hundred-odd extensions. A file outside that set
|
|
75
|
+
produces no findings, and the observation record names its extension — which is the honest reading, but it is
|
|
76
|
+
still a gap rather than a guarantee.
|
|
77
|
+
|
|
78
|
+
**The gate will start firing where it was silent.** A project with the gate on and turns that commit will now be
|
|
79
|
+
blocked by comments it had been landing. That is the fix working, and it is the reason this decision carries a
|
|
80
|
+
migration note.
|
|
81
|
+
|
|
82
|
+
**`--all` reduces typing, not authority.** It expands to what diverged at that moment, so it is still per source
|
|
83
|
+
and still refuses without a terminal. The four locks of
|
|
84
|
+
[/decisions/ad-030.md](/decisions/ad-030.md) are untouched — they govern who clears a divergence and from where,
|
|
85
|
+
never how much they type.
|
|
86
|
+
|
|
87
|
+
## Not decided here
|
|
88
|
+
|
|
89
|
+
- **Making acceptance machine-wide.** The runtime config is a machine-scoped file, so a divergence in it blocks
|
|
90
|
+
every project at once and has to be accepted in each. Fixing that needs a registry of projects the harness
|
|
91
|
+
does not have.
|
|
92
|
+
- **Regenerating the catalog from upstream language-configuration files.** The shape allows it; a tool that
|
|
93
|
+
fetches and rewrites the module is a separate change.
|
|
94
|
+
- **Tracking block state across hunks.** It would need the whole file rather than the diff, and the gate is
|
|
95
|
+
about what the turn added.
|
|
96
|
+
- **Whether `doctor` should report a runtime older than its package.** Noted in
|
|
97
|
+
[/decisions/ad-056.md](/decisions/ad-056.md) and still open.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-059 — A rail never writes a field it reads, and a counter reads the plane its events land on"
|
|
4
|
+
description: "The idle-turn gate blocked the same turn four times in a row. Two defects compounded: its activity counter read the signal plane, where a successful tool call never lands, and the rail wrote the `blockers` that its own precondition reads. Neither is specific to that rail, so both are stated as rules."
|
|
5
|
+
tags: [decision, observability, gates, idle-turn]
|
|
6
|
+
timestamp: "2026-08-10"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-059 — A rail never writes a field it reads, and a counter reads the plane its events land on
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-10
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`readTurnActivity` reads both planes.** `obs.jsonl` carries the turn boundary; `debug.jsonl` carries the
|
|
17
|
+
work. The window boundary is therefore a timestamp rather than a position in one file.
|
|
18
|
+
2. **The counted kinds are the ones a turn actually produces**: `tool.end`, `shell.end`, `file.edit`,
|
|
19
|
+
`file.read`, `mcp.end`, alongside the `.start` and `.fail` kinds already listed.
|
|
20
|
+
3. **The idle-turn rail no longer writes `blockers`.** It writes `next_action` and the failure category. No rail
|
|
21
|
+
may write a field that its own trigger reads.
|
|
22
|
+
4. Both are asserted by tests that a mutant kills, because both failures were invisible to a green suite.
|
|
23
|
+
|
|
24
|
+
## Why
|
|
25
|
+
|
|
26
|
+
**The counter read the one plane its events cannot reach.** `resolveObsLevel` sends a successful tool call to
|
|
27
|
+
`debug`, and `SIGNAL_KINDS` does not contain `tool.end`, `shell.end` or `file.edit`. `readTurnActivity` read
|
|
28
|
+
`obs.jsonl` — the signal plane — so `toolCalls` was structurally zero for every turn whose work went well.
|
|
29
|
+
Measured on this repository's own state:
|
|
30
|
+
|
|
31
|
+
| Plane | `tool.end` | `shell.end` | `file.edit` |
|
|
32
|
+
| --- | --- | --- | --- |
|
|
33
|
+
| `obs.jsonl` — what the counter read | 0 | 0 | 0 |
|
|
34
|
+
| `debug.jsonl` | 322 | 1909 | 981 |
|
|
35
|
+
|
|
36
|
+
Three thousand records of work in the file next to the one being read. The rail's stated property — that it
|
|
37
|
+
counts events the harness recorded and so no wording can satisfy it
|
|
38
|
+
([/decisions/ad-027.md](/decisions/ad-027.md)) — was exactly inverted: **nothing** could satisfy it.
|
|
39
|
+
|
|
40
|
+
**The rail manufactured its own precondition.** `endedWithoutActing` fires only with open work, and open work is
|
|
41
|
+
`blockers || previous_gaps || pending || in_progress`. The rail then wrote
|
|
42
|
+
`blockers: "Turn ended with open work and nothing attempted."`. So one firing guaranteed the next, and it
|
|
43
|
+
returned `continue` before reaching the clean-stop path that clears `blockers` — the operator saw the same
|
|
44
|
+
BLOCKED four times in a row, with the agent running tools and reporting a clean tree between each one.
|
|
45
|
+
|
|
46
|
+
Either defect alone would have been survivable: a wrong counter with no self-feeding write stops when the
|
|
47
|
+
blocker clears, and a self-feeding write with a working counter stops as soon as the agent does anything. Together
|
|
48
|
+
they made a rail with no exit.
|
|
49
|
+
|
|
50
|
+
**Both generalise, which is why they are decisions and not fixes.** A counter that reads one plane is a bug any
|
|
51
|
+
rail can repeat, since which plane an event lands on is decided by `resolveObsLevel` and not by the rail. A rail
|
|
52
|
+
writing a field its trigger reads is a loop any rail can build. The two rules are cheap to check in review and
|
|
53
|
+
neither had been stated.
|
|
54
|
+
|
|
55
|
+
## Trade-offs
|
|
56
|
+
|
|
57
|
+
**Reading two planes doubles the tail read.** Two files, 500 records each, once per stop. The stop hook already
|
|
58
|
+
runs lint and test, so this is not where the time goes — but it is not free, and a third plane would be worth
|
|
59
|
+
questioning.
|
|
60
|
+
|
|
61
|
+
**`debug.jsonl` is written because one code path forces it on.** `debugEnabled` is `false` in the default config
|
|
62
|
+
and the tool entrypoints record with `OBS_CONFIG_AUDIT`, which sets it true so the audit trail persists
|
|
63
|
+
([/decisions/ad-016.md](/decisions/ad-016.md) item 7). The counter now depends on that arrangement. It is
|
|
64
|
+
deliberate and tested, but it is a dependency on a decision made for another reason.
|
|
65
|
+
|
|
66
|
+
**The rail is now weaker against one shape.** A turn that opens a file and answers without changing anything
|
|
67
|
+
counts as activity, because `file.read` counts. That is intended — reading is attempting — but it means "asked
|
|
68
|
+
instead of acting" no longer catches an agent that reads and then asks.
|
|
69
|
+
|
|
70
|
+
**Nothing retroactively clears a stuck handoff.** A project already carrying
|
|
71
|
+
`blockers: "Turn ended with open work and nothing attempted."` keeps it until a clean stop clears it, which now
|
|
72
|
+
happens as soon as a turn does anything. No migration note, because the next successful turn fixes it.
|
|
73
|
+
|
|
74
|
+
## Not decided here
|
|
75
|
+
|
|
76
|
+
- **Whether `resolveObsLevel` should promote tool events to signal.** It should not: the signal plane exists to
|
|
77
|
+
be readable, and 1909 shell calls would drown it. The consumer moving to the right plane is the correct fix.
|
|
78
|
+
- **A dedicated per-turn activity counter** instead of mining the obs planes. Cheaper to read and one more thing
|
|
79
|
+
to keep in step with what actually happened; the planes are already the record of record.
|
|
80
|
+
- **Whether the idle-turn rail should be on by default.** Unchanged: off, and offered with its trade-off.
|
|
81
|
+
- **Clearing `blockers` on a turn that only answers.** That is the rail's whole subject, and deciding it here
|
|
82
|
+
would be deciding it twice.
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-060 — The gate records the environment it ran under, and names it only once the cheap explanations are spent"
|
|
4
|
+
description: "A gate that fails because the hook's environment points its fixtures at the real project is indistinguishable, from the follow-up, from a gate that fails because the code is wrong. It cost four stop loops. The artifact now records which project-scoping variables were set, and the follow-up names them from the second attempt — never the first, because they are set on every hook invocation."
|
|
5
|
+
tags: [decision, gates, observability, environment]
|
|
6
|
+
timestamp: "2026-08-12"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-060 — The gate records the environment it ran under, and names it only once the cheap explanations are spent
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-12
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **Every gate artifact records which project-scoping variables were set** when it ran, by name, as an empty
|
|
17
|
+
list when none were. Always, as a fact.
|
|
18
|
+
2. **`gate.outcome` carries the same reading**, so `obs report` and `attest` can answer it after the fact.
|
|
19
|
+
3. **The follow-up names them from the second attempt onward**, never the first.
|
|
20
|
+
4. **The names live in `src/platform/env-scope.ts`**, once. `core/` asks which are set and never spells one,
|
|
21
|
+
because `tools/check-boundaries.ts` forbids a vendor identifier there.
|
|
22
|
+
5. **A test asserts `tools/test-env.names.mjs` lists the same names.** It is a `.mjs` module loaded by
|
|
23
|
+
`node --import` and cannot import the TypeScript one, so agreement is asserted rather than shared.
|
|
24
|
+
|
|
25
|
+
## Why
|
|
26
|
+
|
|
27
|
+
**Measured, on this repository, against its own author.** The suite passed 1298/1298 from a shell and failed
|
|
28
|
+
nine assertions from inside the hook, four stop loops in a row. The cause was that the project's
|
|
29
|
+
`grind.testCommand` omitted the import that neutralises the provider's project-directory variable — the exact
|
|
30
|
+
hazard `tools/test-env.mjs` documents in prose:
|
|
31
|
+
|
|
32
|
+
> Inside a hook that variable is always set, so 22 tests that build a fixture in a temp directory silently read
|
|
33
|
+
> policy and state from the real repository instead. The suite passed from a shell and failed from inside a hook.
|
|
34
|
+
|
|
35
|
+
The harness held both halves of that fact — the gate's command and output, and the variables in its own
|
|
36
|
+
environment — and joined neither. The follow-up said "fix the test findings", the autopilot said "do not invent
|
|
37
|
+
a different plan", and the lesson store said "do not delete tests". Every one of those was right and none of
|
|
38
|
+
them was the answer.
|
|
39
|
+
|
|
40
|
+
**The second attempt, not the first, and that is the whole design.** A project-scoping variable is set on every
|
|
41
|
+
hook invocation under some providers, so an unconditional note would appear on every gate failure this product
|
|
42
|
+
ever reports. That is the alarm [/decisions/ad-034.md](/decisions/ad-034.md) removed from the update digest —
|
|
43
|
+
one that fires on a healthy run is one the reader learns to scroll past, and it takes the next real one with it.
|
|
44
|
+
By the second attempt the cheap explanations are spent, and the escalation machinery for exactly that already
|
|
45
|
+
exists.
|
|
46
|
+
|
|
47
|
+
**A fact in the record, a hypothesis in the follow-up.** The note says the variables were set and names the
|
|
48
|
+
command that settles it. It does not say they caused the failure, because nothing in the harness can know that:
|
|
49
|
+
whether a variable changes an outcome depends on what the command does with it. Recording is free and always
|
|
50
|
+
correct; diagnosing would be a guess that either accuses a healthy run or excuses a real failure.
|
|
51
|
+
|
|
52
|
+
## Trade-offs
|
|
53
|
+
|
|
54
|
+
**The note can be wrong-footed.** A genuine code failure on the second attempt now carries a paragraph about the
|
|
55
|
+
environment. It costs a few hundred characters and one command to rule out; the alternative cost four loops.
|
|
56
|
+
|
|
57
|
+
**Two declarations of one list.** The hermetic loader cannot import the TypeScript module, so the names exist
|
|
58
|
+
twice with a test holding them together. One of them was already missing an entry once, which is the evidence
|
|
59
|
+
that the test is the load-bearing part.
|
|
60
|
+
|
|
61
|
+
**It does not fix the underlying divergence.** A project can still configure a gate command that differs from
|
|
62
|
+
the one its own suite needs, and the harness must not read or rewrite it — that is the operator's, on the policy
|
|
63
|
+
surface. This makes the divergence visible on the second failure instead of the fourth.
|
|
64
|
+
|
|
65
|
+
**`scopedEnv` is absent on an artifact written before this shipped**, so the first failure after an upgrade
|
|
66
|
+
reports no environment. The reader treats absent as unknown rather than as none.
|
|
67
|
+
|
|
68
|
+
## Not decided here
|
|
69
|
+
|
|
70
|
+
- **Running the gate a second time with the variables cleared.** It would answer the question outright and
|
|
71
|
+
double the cost of every failure to do it. If the note proves insufficient, this is the next step.
|
|
72
|
+
- **`doctor` comparing the project's gate command against anything.** There is nothing correct to compare it
|
|
73
|
+
to — the harness does not know what a given repository's suite needs.
|
|
74
|
+
- **Recording the full environment.** Only variables that name *which project* are recorded. Anything wider is
|
|
75
|
+
a secrets surface.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-061 — A decision that refuses names its rule, and degrading preserves it"
|
|
4
|
+
description: "`rule` becomes required on deny and ask. Of sixteen producers, four carried one, so the report that attributes refusals by rule had holes exactly where operators were confused. Degrade preserved none at all — the one place a decision changes shape."
|
|
5
|
+
tags: [decision, contracts, observability, rules]
|
|
6
|
+
timestamp: "2026-08-12"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-061 — A decision that refuses names its rule, and degrading preserves it
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-12
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`rule` is required on `deny` and `ask`.** The type is the enforcement.
|
|
17
|
+
2. **`degrade` preserves the rule it was given** and never replaces it. The one rule it may name is its own,
|
|
18
|
+
`rewrite-unavailable`, for the conversion where the incoming decision refused nothing.
|
|
19
|
+
3. **Each aggregate owns its rule names as a constant**, following `SHELL_RULES`, which already did.
|
|
20
|
+
4. **One rule per reason, not one per aggregate.** A spawn refused by the allowlist and a spawn refused because
|
|
21
|
+
the parent is in Fast mode are different facts.
|
|
22
|
+
5. `allow`, `context`, `continue`, `abstain` and `rewriteInput` are untouched — they refuse nothing.
|
|
23
|
+
|
|
24
|
+
## Why
|
|
25
|
+
|
|
26
|
+
**A teammate could not tell harness behaviour from model behaviour**, and the command that answers that —
|
|
27
|
+
`tlc harness obs report` — attributes refusals by rule. With `rule` optional and four producers of sixteen
|
|
28
|
+
carrying one, the table was blank precisely where he had been confused: the subagent allowlist refused his
|
|
29
|
+
spawn and the report could not say so.
|
|
30
|
+
|
|
31
|
+
**The type is the enforcement, and that is a measured claim.** Making the field optional again produces zero
|
|
32
|
+
compiler errors across the whole repository — nothing else would catch a producer that forgot. Flipping it to
|
|
33
|
+
required produced 32 errors in seven files in one command, which is the complete list, obtained without writing
|
|
34
|
+
a checker that would then need keeping in step.
|
|
35
|
+
|
|
36
|
+
**Degrade was the worst case and had no attribution at all.** `ask` becomes `deny` where the provider cannot
|
|
37
|
+
ask; `rewriteInput` becomes `ask` where it cannot rewrite. Both dropped the rule. That is the one place a
|
|
38
|
+
decision changes shape between the rail that made it and the operator who reads it, so it is exactly where the
|
|
39
|
+
attribution matters most — and inventing a rule there would have blamed the transport for a rail's decision.
|
|
40
|
+
|
|
41
|
+
**One rule per reason.** Five distinct refusals shared one `block()` helper in the subagent aggregate. Naming
|
|
42
|
+
them `subagent-allowlist`, `subagent-parent-fast`, `subagent-blocked-pattern`, `subagent-model-required` and
|
|
43
|
+
`subagent-min-effort` is what lets a report say which one fired, which was the original complaint.
|
|
44
|
+
|
|
45
|
+
## Trade-offs
|
|
46
|
+
|
|
47
|
+
**Every future producer must name a rule.** That is the point, and it is a real constraint on anyone adding a
|
|
48
|
+
rail: the name appears in `rule=…` in messages operators read, so it is a small piece of public interface
|
|
49
|
+
rather than an internal label.
|
|
50
|
+
|
|
51
|
+
**Test fixtures now carry a rule too** — twenty-six of them. They say `test-deny` and `test-ask`, which is
|
|
52
|
+
noise in the fixtures and the price of the type being unconditional. An optional field with a checker would
|
|
53
|
+
have spared them and would not have been enforced anywhere the checker did not look.
|
|
54
|
+
|
|
55
|
+
**Nothing prevents an empty or duplicated name.** The type accepts any string. A test asserts the declared
|
|
56
|
+
constants are non-empty, kebab-case and unique; a producer that inlines a literal is outside it.
|
|
57
|
+
|
|
58
|
+
**The rule set is not enumerable from one place.** A closed union in `contracts/` would give that, and would
|
|
59
|
+
make adding a rail a two-file change for no gain the required field does not already provide. `tlc harness why`
|
|
60
|
+
will group by whatever it finds rather than by a list it has to be told.
|
|
61
|
+
|
|
62
|
+
## Not decided here
|
|
63
|
+
|
|
64
|
+
- **A closed union of rule names.** Rejected above; revisit if `why` needs to render a rule the operator has
|
|
65
|
+
never seen with an explanation rather than a name.
|
|
66
|
+
- **Recording every refusal in obs.** `rule` being present makes attribution possible; whether every rail also
|
|
67
|
+
writes an obs record is a separate gap, and the next thing to check.
|
|
68
|
+
- **Renaming existing rules.** `secret-access` and its siblings already appear in messages people have read.
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-062 — One command answers whether the harness did that, and says so plainly when it did not"
|
|
4
|
+
description: "Hooks are invisible by construction: the harness answers the host and the host decides whether to show it. `tlc harness why` prints the last decisions with the rule behind each, derived from the two obs planes, and states in words when there were none — which is the reading no other command gives."
|
|
5
|
+
tags: [decision, observability, cli, legibility]
|
|
6
|
+
timestamp: "2026-08-12"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-062 — One command answers whether the harness did that, and says so plainly when it did not
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-12
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`tlc harness why [n]`** prints the last n harness decisions, newest first: the event answered, the verdict,
|
|
17
|
+
the rule, and one line of detail. Default 10.
|
|
18
|
+
2. **It is a first-class verb**, not `obs why`. It routes to the obs entry, and the alias is the interface.
|
|
19
|
+
3. **A window with no decision says so in words**, and that sentence is the feature.
|
|
20
|
+
4. **It derives from the two obs planes and nothing else.** No new producer, no second index.
|
|
21
|
+
5. **Read-only, `--json` like every read command.**
|
|
22
|
+
|
|
23
|
+
## Why
|
|
24
|
+
|
|
25
|
+
**The complaint is structural, not a bug.** A hook answers the host; the host decides whether to render it. So
|
|
26
|
+
a rail can fire and the operator sees only that the agent behaved differently — "só muda o comportamento e não
|
|
27
|
+
fala". Reverse-engineering which behaviour was a rail is the state two people on this team were in.
|
|
28
|
+
|
|
29
|
+
**The two commands that exist answer other questions.** `obs report` is a session rollup — right for "what did
|
|
30
|
+
this session cost and what fired", wrong for "what just happened". `obs live` is a firehose of every recorded
|
|
31
|
+
event. Neither is what someone reaches for in the moment.
|
|
32
|
+
|
|
33
|
+
**The empty case is the whole point.** Every other command shows what happened; none of them can say *nothing
|
|
34
|
+
here was mine*. An empty table would leave the reader exactly as unsure as before, so the absence of decisions
|
|
35
|
+
is rendered as a sentence:
|
|
36
|
+
|
|
37
|
+
> No harness decision in this window. Whatever you just saw was the model, not a rail — the harness allowed
|
|
38
|
+
> everything it was asked about.
|
|
39
|
+
|
|
40
|
+
**It reads both planes for the same reason the idle-turn counter now does.** The signal plane holds the
|
|
41
|
+
refusals, the debug plane holds the allows, and an allow is a decision: "it let it through" is exactly the
|
|
42
|
+
reading someone is looking for ([/decisions/ad-059.md](/decisions/ad-059.md)).
|
|
43
|
+
|
|
44
|
+
**It required [/decisions/ad-061.md](/decisions/ad-061.md) first.** With `rule` optional, this command would
|
|
45
|
+
have rendered "deny, unattributed" precisely on the rail that started the complaint.
|
|
46
|
+
|
|
47
|
+
## Trade-offs
|
|
48
|
+
|
|
49
|
+
**It shows decisions, never reasoning.** The rule name is a pointer; the reasoning is in the decision record it
|
|
50
|
+
cites. Rendering an explanation inline would put a second copy of every rationale in the CLI, to drift from the
|
|
51
|
+
one in `docs/decisions/`.
|
|
52
|
+
|
|
53
|
+
**Activity is excluded, and that is a judgement.** A turn reading forty files is not the harness doing
|
|
54
|
+
something, so it is not listed — but if the interesting question ever becomes "what did the agent do", this is
|
|
55
|
+
the wrong command and `obs live` is the right one.
|
|
56
|
+
|
|
57
|
+
**Historical records read as `unattributed`.** Anything written before rules were required carries none. It
|
|
58
|
+
resolves itself as those records age out of the window; until then it is a name for "the record does not say"
|
|
59
|
+
rather than a blank, because a blank reads as "no rule applied".
|
|
60
|
+
|
|
61
|
+
**A record that no producer writes cannot be shown.** The command is only as complete as the obs planes. Every
|
|
62
|
+
refusal is recorded today — `recordRefusal` covers the non-shell path and the shell rail owns its own — but a
|
|
63
|
+
rail added later that decides without recording would be invisible here, and nothing in the build would catch
|
|
64
|
+
that.
|
|
65
|
+
|
|
66
|
+
## Not decided here
|
|
67
|
+
|
|
68
|
+
- **Failing the gate on a rail that decides without recording.** It is the same shape as the required `rule`
|
|
69
|
+
and would close the last gap above; it needs a way to enumerate rails that is not a hand-written list.
|
|
70
|
+
- **Showing the follow-up text.** A blocked stop appears as a gate outcome, not as the paragraph the agent
|
|
71
|
+
received. The paragraph is in the transcript; duplicating it here would be a second copy that can disagree.
|
|
72
|
+
- **A `--since` window by time.** `n` was enough for the question people are actually asking.
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-063 — One palette for human output, and a checker keeping it out of everything else"
|
|
4
|
+
description: "The CLI's human surfaces share one truecolor palette and symbol set, written without a dependency. Colour is off unless stdout is a terminal, and `check-boundaries` refuses the style module in `providers/` and `entrypoints/` — an escape reaching parsed output already sent an agent to fix a file that does not exist."
|
|
5
|
+
tags: [decision, cli, output, boundaries]
|
|
6
|
+
timestamp: "2026-08-12"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-063 — One palette for human output, and a checker keeping it out of everything else
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-12
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`src/platform/style.ts` owns the palette and the symbols.** One place, so a surface cannot invent its own.
|
|
17
|
+
2. **No dependency.** Truecolor escapes written directly, because this package has none and ships no binary
|
|
18
|
+
([/decisions/ad-012.md](/decisions/ad-012.md)).
|
|
19
|
+
3. **Colour is off unless stdout is a TTY**, and off under `NO_COLOR` or `--no-color`.
|
|
20
|
+
4. **`providers/` and `entrypoints/` may not import it**, enforced by `tools/check-boundaries.ts` with the rule
|
|
21
|
+
`styles-machine-readable-output`.
|
|
22
|
+
5. **`why` gained substance with the paint**: a verdict summary, a date on any record not from today, and the
|
|
23
|
+
`rule=` prefix kept because that is what people grep for.
|
|
24
|
+
|
|
25
|
+
## Why
|
|
26
|
+
|
|
27
|
+
**The output was asked about and the answer was no.** `tlc harness why` shipped as ten lines of
|
|
28
|
+
undifferentiated monospace, six of them `rule=unattributed`, every timestamp reading `19:23:05` with no date —
|
|
29
|
+
so a window spanning three days looked like one burst. Colour was the smaller half of the fix.
|
|
30
|
+
|
|
31
|
+
**The TTY check is load-bearing, not politeness.** This CLI's output is redirected into files, quoted into gate
|
|
32
|
+
follow-ups, and read by the repository's own checkers. An escape in any of those is a defect, and it is not
|
|
33
|
+
hypothetical: a colour escape ends `[39m`, `39m` matched the path pattern in gate output, and the autopilot told
|
|
34
|
+
an agent by name to fix `39msrc/entrypoints/__test__/tool-after.test.ts` — a file that does not exist. That was
|
|
35
|
+
output the harness *read*; this decision adds output the harness *writes*, so the rule gets a checker rather
|
|
36
|
+
than a convention.
|
|
37
|
+
|
|
38
|
+
**The boundary is where it is because of what those directories do.** `entrypoints/` renders text that goes to
|
|
39
|
+
a host and to `--json`; `providers/` serialises the decision the host parses. Neither has a human reader. The
|
|
40
|
+
check costs one import scan and removes a whole class of defect from review.
|
|
41
|
+
|
|
42
|
+
## Trade-offs
|
|
43
|
+
|
|
44
|
+
**Two ways to print now.** A human surface takes a `Style`; everything else does not. A renderer shared between
|
|
45
|
+
the two has to take the style as a parameter and default to plain, which is one more thing to get right — and
|
|
46
|
+
the checker only catches the import, not a styled string threaded through by hand.
|
|
47
|
+
|
|
48
|
+
**Truecolor assumes a modern terminal.** No 256-colour or 16-colour fallback: a terminal that cannot do
|
|
49
|
+
truecolor shows the escape's effect wrong rather than degrading. `NO_COLOR` is the escape hatch, and adding
|
|
50
|
+
fallback tiers would be palette code for a case nobody on this team has.
|
|
51
|
+
|
|
52
|
+
**The palette is not configurable.** One set of hex values, consistent with the team's other tooling. A theme
|
|
53
|
+
option would be a config surface for a preference nobody has asked for.
|
|
54
|
+
|
|
55
|
+
**`why`'s summary counts verdicts, not importance.** Three floor denials of the same command read as "3 denied",
|
|
56
|
+
which is accurate and not the same as "one thing went wrong three times". Grouping would be a judgement about
|
|
57
|
+
which repeats matter.
|
|
58
|
+
|
|
59
|
+
## The standard is structural
|
|
60
|
+
|
|
61
|
+
A screen describes its content as a `Screen` — a title, a summary, sections of label/value rows — and one
|
|
62
|
+
renderer in `src/platform/screen.ts` decides spacing, alignment and colour. A screen cannot be off-standard
|
|
63
|
+
because it cannot express layout.
|
|
64
|
+
|
|
65
|
+
`tools/check-screens.ts` is a gate step at budget **zero**: every exported `*Text` renderer either goes through
|
|
66
|
+
`render` or takes a `Style`. A new one that does neither fails the gate. It was shipped at a budget of fourteen
|
|
67
|
+
and lowered in two passes, because a checker that merely allowed a list would have hidden the remainder.
|
|
68
|
+
|
|
69
|
+
**The plain form is the contract.** Colour wraps; it never reshapes. `obs live` stays tab-separated with a full
|
|
70
|
+
ISO timestamp because that output is piped — `cut -f2` gives the kind — and three tests caught a restyle that
|
|
71
|
+
had quietly dropped wording the reader needed: `DIVERGED mid-session`, `N pass / N fail`, and the phrase that
|
|
72
|
+
says what a garden verdict means. Layout moved; text did not.
|
|
73
|
+
|
|
74
|
+
`obs report` produces two things from one rollup: `sessionReportMarkdown` for the file people paste into a pull
|
|
75
|
+
request, and `sessionReportScreen` for the terminal. One string for both would have put escapes in the artifact.
|
|
76
|
+
|
|
77
|
+
## Not decided here
|
|
78
|
+
|
|
79
|
+
- **A test that the two `obs report` renderings agree.** They read the same rollup and can drift in what they
|
|
80
|
+
choose to show; nothing currently asserts they agree on the numbers.
|
|
81
|
+
- **`usageText` staying plain.** It is thrown as a `UsageError` and printed on the error path, which may be
|
|
82
|
+
redirected. Colouring an error message is the same hazard as colouring an artifact.
|
|
83
|
+
- **Boxes, spinners, progress.** Hook output is not interactive and the CLI is not a TUI.
|
|
84
|
+
- **A 256-colour fallback.** Named above as a real limit rather than an oversight.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-064 — A snapshot is assigned, never accumulated, and a table does not list what it cannot count"
|
|
4
|
+
description: "The session report claimed 102.7M output tokens against 559k input. The transcript reader sums the tail of the transcript and that total is attached to every tool event, so the rollup was summing 3,488 snapshots of a sliding window. It also listed Bash as 0 ok / 23 fail, a row that can only ever show failures."
|
|
5
|
+
tags: [decision, observability, cost, reporting]
|
|
6
|
+
timestamp: "2026-08-12"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-064 — A snapshot is assigned, never accumulated, and a table does not list what it cannot count
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-12
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **A token reading is assigned to the rollup, not added.** Same for the cost estimate.
|
|
17
|
+
2. **The row says what the number is**: the latest reading over the recent transcript, not a session total.
|
|
18
|
+
3. **A shell tool is not listed in the tools table.** Its successes are recorded as shell events, and the
|
|
19
|
+
`shell` row already answers for it.
|
|
20
|
+
4. Both are asserted by tests that a mutant kills.
|
|
21
|
+
|
|
22
|
+
## Why
|
|
23
|
+
|
|
24
|
+
**The number was wrong by two orders of magnitude and looked authoritative.** 102,780,796 output tokens against
|
|
25
|
+
559,896 input, with the cost shown as `$0.0000`. Three layers stacked:
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
claude.transcript.ts readTail(transcriptPath, tailLines) reads the tail
|
|
29
|
+
claude.transcript.ts inputTokens += … sums every record in it
|
|
30
|
+
tool-after.ts attaches that total to every event
|
|
31
|
+
observability.service rollup.output_tokens += outTok sums it again, 3,488 times
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
The reader's result is a **snapshot of a sliding window**, not a delta. Measured: consecutive values oscillate —
|
|
35
|
+
27515, 24764, 29082, 24819 — and repeat within one turn, which is a snapshot re-read, not a counter. Summing
|
|
36
|
+
snapshots multiplies each turn's figure by the number of tool calls in it.
|
|
37
|
+
|
|
38
|
+
**Last-per-turn was measured and rejected.** Taking the final reading of each turn and summing those gives 4.66M
|
|
39
|
+
output tokens and 2.76 **billion** cache reads. Also snapshots, also summed. The only correct operation on a
|
|
40
|
+
gauge is to take it, so the rollup takes the latest.
|
|
41
|
+
|
|
42
|
+
**And the label had to change with it.** "Input tokens / Output tokens" reads as a session total, which is the
|
|
43
|
+
reading that made the sum look reasonable to write. It now says what it is. A number whose meaning is stated is
|
|
44
|
+
usable; one that implies a meaning it does not have is worse than none
|
|
45
|
+
([/decisions/ad-034.md](/decisions/ad-034.md)).
|
|
46
|
+
|
|
47
|
+
**`Bash: 0 ok, 23 fail` was structurally impossible to get right.** `rollup.tools` is fed by `tool.start`,
|
|
48
|
+
`tool.end` and `tool.fail`. A successful shell call is `shell.end`; a failed one is `tool.fail`. So the row
|
|
49
|
+
counted every failure and no success — after roughly 1,900 successful shell calls in the same session. The
|
|
50
|
+
`shell` row two lines above reports `1874 allow / 0 ask / 60 deny`, correctly. Two sources for one fact is how
|
|
51
|
+
this broke, so one of them goes.
|
|
52
|
+
|
|
53
|
+
## Trade-offs
|
|
54
|
+
|
|
55
|
+
**The token figure is now honest and small.** It reports the recent transcript window rather than the session,
|
|
56
|
+
which is less than an operator wants. It is the number the harness can actually justify, and the label says so.
|
|
57
|
+
|
|
58
|
+
**A true session total needs per-record accounting.** Counting each usage record exactly once means tracking a
|
|
59
|
+
cursor into the transcript per session, or reading the whole file. The tail read exists because hook latency is
|
|
60
|
+
a product property ([/decisions/ad-012.md](/decisions/ad-012.md)), and reading a long transcript on every tool
|
|
61
|
+
call would spend it.
|
|
62
|
+
|
|
63
|
+
**The cost estimate inherits the same limitation.** It is the estimate for the latest reading, not the session's
|
|
64
|
+
bill. It was previously the sum of estimates over snapshots, which was wrong in the same way and by the same
|
|
65
|
+
factor.
|
|
66
|
+
|
|
67
|
+
**Dropping the shell row loses its duration.** `tools[name].ms` accumulated for a shell tool would have been a
|
|
68
|
+
per-call timing the shell row does not carry. It was always zero here, because only `tool.end` writes it and a
|
|
69
|
+
shell call never produces one.
|
|
70
|
+
|
|
71
|
+
## Not decided here
|
|
72
|
+
|
|
73
|
+
- **Per-record token accounting for a true session total.** The right fix and a bigger one: a cursor per
|
|
74
|
+
session, or a single whole-transcript read at session end where latency does not matter.
|
|
75
|
+
- **Whether `SHELL_TOOLS` should come from the provider adapter.** It is a three-name set in core today, which
|
|
76
|
+
`check-boundaries` allows because none of the names is a vendor identifier. A fourth host with a differently
|
|
77
|
+
named shell tool would need it moved.
|
|
78
|
+
- **Reporting cache-read tokens at all.** They dominate the real cost and are not shown; showing a number that
|
|
79
|
+
is also a snapshot would repeat the mistake in a new column.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-065 — The obs bus has a contract, and the gate checks both sides of it"
|
|
4
|
+
description: "Seven defects in one week shared one shape: a consumer reading where the producer does not write. Each became a decision record and the pattern never became an object, so it was rediscovered by hand seven times. A kind consumed and never emitted now fails the gate, and so does a consumer reading a plane its kinds never land on."
|
|
5
|
+
tags: [decision, observability, contracts, gates]
|
|
6
|
+
timestamp: "2026-08-12"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-065 — The obs bus has a contract, and the gate checks both sides of it
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-12
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **A kind a consumer counts and no producer emits fails the gate.**
|
|
17
|
+
2. **A consumer declares its kinds and its planes**, and a kind landing on a plane the consumer does not read
|
|
18
|
+
fails. The plane comes from `resolveObsLevel` — the real function — so a change to the plane rules cannot
|
|
19
|
+
leave the check stale.
|
|
20
|
+
3. **A kind emitted and read by nothing is reported, not failed.** It costs a write per event and may be a rail
|
|
21
|
+
half-built; that is the operator's judgement.
|
|
22
|
+
4. **No registry file.** Emit sites are read from the source and consumers declare their own sets, so there is
|
|
23
|
+
nothing to keep in step.
|
|
24
|
+
|
|
25
|
+
## Why
|
|
26
|
+
|
|
27
|
+
**One pattern, seven instances, never materialised.** A consumer reading where the producer does not write:
|
|
28
|
+
`gate.outcome` and `policy.deny` consumed by the report and emitted by nothing
|
|
29
|
+
([/decisions/ad-027.md](/decisions/ad-027.md)); `format.enabled` in the catalog and read by no entrypoint
|
|
30
|
+
([/decisions/ad-033.md](/decisions/ad-033.md)); `progressiveHandoff` written and never read; the idle-turn
|
|
31
|
+
counter reading the signal plane while its kinds land on debug ([/decisions/ad-059.md](/decisions/ad-059.md));
|
|
32
|
+
`latestSessionId` ordering UUIDs by name; the comment gate diffing against `HEAD` after the turn had committed
|
|
33
|
+
([/decisions/ad-058.md](/decisions/ad-058.md)); the rollup summing a snapshot
|
|
34
|
+
([/decisions/ad-064.md](/decisions/ad-064.md)).
|
|
35
|
+
|
|
36
|
+
Every one was recorded. None of it stopped the next. Recording a fact is not the same as materialising the
|
|
37
|
+
pattern, and this repository already has two proofs that materialising works —
|
|
38
|
+
[/decisions/ad-041.md](/decisions/ad-041.md) made a union member read and never written a build failure, and
|
|
39
|
+
`check-screens.ts` did the same for a screen outside the standard.
|
|
40
|
+
|
|
41
|
+
**The practice transfers; the implementation does not exist.** Event-driven work treats a contract as a build
|
|
42
|
+
gate rather than as documentation — validate the producer's record, verify the consumer against it, fail the
|
|
43
|
+
pipeline on a mismatch. Orphan detection is the mirror: enumerate what is declared, check for references, attach
|
|
44
|
+
a cost to what nothing uses. Searching found no tooling that does either statically for an in-process bus, so
|
|
45
|
+
the pattern is borrowed and the code is ours.
|
|
46
|
+
|
|
47
|
+
**`ObsKind` was a schema, not a contract.** A closed union says which kinds exist. It does not say who emits
|
|
48
|
+
one, who reads it, or **which plane it lands on** — and the plane is exactly what the idle-turn rail got wrong.
|
|
49
|
+
`resolveObsLevel` decides it deterministically from the kind, which is what makes the check possible without a
|
|
50
|
+
runtime.
|
|
51
|
+
|
|
52
|
+
## Trade-offs
|
|
53
|
+
|
|
54
|
+
**It catches three of the seven.** The dead-kind class and the plane mismatch. It does nothing for the two
|
|
55
|
+
diff-base defects or the two aggregation defects, and saying otherwise would be the false coverage this exists
|
|
56
|
+
to remove. Those need a relation this cannot enumerate mechanically yet.
|
|
57
|
+
|
|
58
|
+
**The consumer list is hand-maintained.** One entry today. A consumer that never registers is invisible to the
|
|
59
|
+
check — the same shape as the defect it guards against, one level up. It is small enough to see in review and
|
|
60
|
+
large enough to be worth saying out loud.
|
|
61
|
+
|
|
62
|
+
**Emit sites are found by matching `kind:` literals.** A kind assembled at runtime would read as unemitted and
|
|
63
|
+
fail a consumer that reads it. Nothing does that today, and the failure is loud rather than silent.
|
|
64
|
+
|
|
65
|
+
**The orphan list found a defect on its first honest run.** Registering the session rollup and `why` took it
|
|
66
|
+
from thirteen to five, and two of the five were real: `policy.observe` and `cost.session_alert` were emitted and
|
|
67
|
+
read by nothing. Observation mode exists to produce a reading an operator acts on
|
|
68
|
+
([/decisions/ad-027.md](/decisions/ad-027.md)) and no command showed one — the rail recorded into a plane
|
|
69
|
+
nobody read. Both are now decisions in `why`, and three lifecycle kinds remain unread, which is fine and stated.
|
|
70
|
+
|
|
71
|
+
**An inline consumer cannot have a plane mismatch**, and the contract says so rather than passing it silently.
|
|
72
|
+
The rollup is fed at `recordObs` time, before plane routing; `why` and the activity counter read plane files.
|
|
73
|
+
Marking the difference is what separates "checked and fine" from "not checked".
|
|
74
|
+
|
|
75
|
+
## Not decided here
|
|
76
|
+
|
|
77
|
+
- **`generation.end`, `session.end` and `subagent.end`.** Emitted, read by nothing. They are lifecycle markers
|
|
78
|
+
and may stay that way; the report names them so it is a decision rather than an oversight.
|
|
79
|
+
- **Validating attributes per kind.** The payload half of a contract, and a much larger change. The rollup reads
|
|
80
|
+
`attrs.permission`, `attrs.rule` and others with no declaration that a producer sets them.
|
|
81
|
+
- **A runtime check.** This is a build gate; a hook has no budget for it.
|