@tech-leads-club/harness-toolkit 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +95 -0
- package/LICENSE +84 -0
- package/NOTICE +9 -0
- package/README.md +480 -0
- package/bin/tlc +10 -0
- package/bin/tlc-build +80 -0
- package/bin/tlc-cli.ts +1509 -0
- package/bin/tlc-exec +10 -0
- package/bin/tlc-exec.cmd +4 -0
- package/bin/tlc-exec.d.mts +60 -0
- package/bin/tlc-exec.mjs +248 -0
- package/bin/tlc.cmd +4 -0
- package/bin/tlc.mjs +7 -0
- package/bin/write-user-hooks.d.mts +40 -0
- package/bin/write-user-hooks.mjs +123 -0
- package/capabilities/catalog.json +418 -0
- package/config.example.json +51 -0
- package/dist/compact-before.mjs +7839 -0
- package/dist/doctor.mjs +8263 -0
- package/dist/help-topic.mjs +82 -0
- package/dist/init-project.mjs +993 -0
- package/dist/install-runtime.mjs +969 -0
- package/dist/lessons-cli.mjs +7258 -0
- package/dist/obs-cli.mjs +7025 -0
- package/dist/price-lookup.mjs +221 -0
- package/dist/prompt-submit.mjs +7860 -0
- package/dist/refresh-model-prices.mjs +158 -0
- package/dist/response-after.mjs +7879 -0
- package/dist/run.mjs +7830 -0
- package/dist/session-end.mjs +7937 -0
- package/dist/session-start.mjs +8056 -0
- package/dist/shim.mjs +52 -0
- package/dist/stop.mjs +8494 -0
- package/dist/subagent-start.mjs +7861 -0
- package/dist/subagent-stop.mjs +7851 -0
- package/dist/support.mjs +7045 -0
- package/dist/tlc-cli.mjs +8063 -0
- package/dist/tool-after.mjs +8113 -0
- package/dist/tool-before.mjs +7985 -0
- package/dist/tool-failure.mjs +7850 -0
- package/dist/uninstall-runtime.mjs +1307 -0
- package/docs/architecture.md +270 -0
- package/docs/concepts.md +562 -0
- package/docs/coverage.md +55 -0
- package/docs/decisions/ad-001.md +32 -0
- package/docs/decisions/ad-002.md +51 -0
- package/docs/decisions/ad-003.md +30 -0
- package/docs/decisions/ad-004.md +37 -0
- package/docs/decisions/ad-005.md +36 -0
- package/docs/decisions/ad-006.md +49 -0
- package/docs/decisions/ad-007.md +36 -0
- package/docs/decisions/ad-008.md +54 -0
- package/docs/decisions/ad-009.md +61 -0
- package/docs/decisions/ad-010.md +45 -0
- package/docs/decisions/ad-011.md +59 -0
- package/docs/decisions/ad-012.md +71 -0
- package/docs/decisions/ad-013.md +87 -0
- package/docs/decisions/ad-014.md +56 -0
- package/docs/decisions/ad-015.md +33 -0
- package/docs/decisions/ad-016.md +98 -0
- package/docs/decisions/ad-017.md +65 -0
- package/docs/decisions/ad-018.md +77 -0
- package/docs/decisions/ad-019.md +75 -0
- package/docs/decisions/ad-020.md +88 -0
- package/docs/decisions/ad-021.md +57 -0
- package/docs/decisions/ad-022.md +120 -0
- package/docs/decisions/ad-023.md +87 -0
- package/docs/decisions/ad-024.md +85 -0
- package/docs/decisions/ad-025.md +92 -0
- package/docs/decisions/ad-026.md +131 -0
- package/docs/decisions/ad-027.md +90 -0
- package/docs/decisions/ad-028.md +90 -0
- package/docs/decisions/ad-029.md +73 -0
- package/docs/decisions/ad-030.md +97 -0
- package/docs/decisions/ad-031.md +94 -0
- package/docs/decisions/ad-032.md +83 -0
- package/docs/decisions/ad-033.md +89 -0
- package/docs/decisions/ad-034.md +86 -0
- package/docs/decisions/ad-035.md +86 -0
- package/docs/decisions/ad-036.md +68 -0
- package/docs/decisions/ad-037.md +47 -0
- package/docs/decisions/ad-038.md +52 -0
- package/docs/decisions/ad-039.md +69 -0
- package/docs/decisions/ad-040.md +89 -0
- package/docs/decisions/ad-041.md +98 -0
- package/docs/decisions/ad-042.md +82 -0
- package/docs/decisions/ad-043.md +79 -0
- package/docs/decisions/ad-044.md +61 -0
- package/docs/decisions/ad-045.md +94 -0
- package/docs/decisions/ad-046.md +111 -0
- package/docs/decisions/ad-047.md +96 -0
- package/docs/decisions/ad-048.md +85 -0
- package/docs/decisions/ad-049.md +66 -0
- package/docs/decisions/ad-050.md +94 -0
- package/docs/decisions/ad-051.md +69 -0
- package/docs/decisions/ad-052.md +69 -0
- package/docs/decisions/ad-053.md +78 -0
- package/docs/decisions/ad-054.md +98 -0
- package/docs/decisions/ad-055.md +74 -0
- package/docs/decisions/ad-056.md +85 -0
- package/docs/decisions/ad-057.md +68 -0
- package/docs/decisions/ad-058.md +97 -0
- package/docs/decisions/ad-059.md +82 -0
- package/docs/decisions/ad-060.md +75 -0
- package/docs/decisions/ad-061.md +68 -0
- package/docs/decisions/ad-062.md +72 -0
- package/docs/decisions/ad-063.md +84 -0
- package/docs/decisions/ad-064.md +79 -0
- package/docs/decisions/ad-065.md +81 -0
- package/docs/decisions/ad-066.md +111 -0
- package/docs/decisions/ad-067.md +64 -0
- package/docs/decisions/ad-068.md +79 -0
- package/docs/decisions/ad-069.md +74 -0
- package/docs/decisions/ad-070.md +86 -0
- package/docs/decisions/ad-071.md +93 -0
- package/docs/decisions/ad-072.md +82 -0
- package/docs/decisions/ad-073.md +102 -0
- package/docs/decisions/ad-074.md +91 -0
- package/docs/decisions/ad-075.md +79 -0
- package/docs/decisions/ad-076.md +102 -0
- package/docs/decisions/ad-077.md +94 -0
- package/docs/decisions/ad-078.md +84 -0
- package/docs/decisions/ad-079.md +73 -0
- package/docs/decisions/ad-080.md +86 -0
- package/docs/decisions/ad-081.md +70 -0
- package/docs/decisions/ad-082.md +79 -0
- package/docs/decisions/ad-083.md +88 -0
- package/docs/decisions/index.md +111 -0
- package/docs/diagnose.md +248 -0
- package/docs/index.md +52 -0
- package/docs/init.md +101 -0
- package/docs/lessons.md +294 -0
- package/docs/log.md +139 -0
- package/docs/measure.md +178 -0
- package/docs/providers/claude-code.md +118 -0
- package/docs/providers/cursor.md +110 -0
- package/docs/providers/index.md +71 -0
- package/docs/troubleshooting.md +138 -0
- package/model-aliases.json +12 -0
- package/model-prices.cursor.json +410 -0
- package/model-prices.json +1 -0
- package/package.json +57 -0
- package/skills/harness-init/SKILL.md +241 -0
- package/skills/harness-init/evals/trigger_evals.json +86 -0
- package/skills/harness-init/references/capabilities.md +113 -0
- package/src/contracts/capabilities.ts +30 -0
- package/src/contracts/decision.ts +21 -0
- package/src/contracts/effort.ts +15 -0
- package/src/contracts/harness-event.ts +87 -0
- package/src/contracts/index.ts +6 -0
- package/src/contracts/policy-defaults.ts +14 -0
- package/src/contracts/wiring.ts +20 -0
- package/src/core/attest/attest.service.ts +0 -0
- package/src/core/capability/capability.service.ts +62 -0
- package/src/core/capability/capability.store.ts +51 -0
- package/src/core/capability/capability.types.ts +56 -0
- package/src/core/comment-policy/comment-policy.service.ts +307 -0
- package/src/core/comment-policy/comment-policy.types.ts +6 -0
- package/src/core/comment-policy/comment-resolvability.ts +134 -0
- package/src/core/comment-policy/comment-syntax.catalog.ts +310 -0
- package/src/core/comment-policy/comment-syntax.store.ts +65 -0
- package/src/core/comment-policy/comment-syntax.types.ts +24 -0
- package/src/core/core.facade.ts +461 -0
- package/src/core/duplication/duplication.service.ts +219 -0
- package/src/core/floor/floor.catalog.ts +54 -0
- package/src/core/floor/floor.paths.ts +90 -0
- package/src/core/floor/floor.policy-surface.ts +311 -0
- package/src/core/floor/floor.service.ts +306 -0
- package/src/core/floor/floor.tokenize.ts +179 -0
- package/src/core/floor/floor.verb.ts +28 -0
- package/src/core/floor/index.ts +1 -0
- package/src/core/gate/gate.artifact.ts +157 -0
- package/src/core/gate/gate.command.ts +130 -0
- package/src/core/gate/gate.findings.ts +201 -0
- package/src/core/gate/gate.inputs.ts +0 -0
- package/src/core/gate/gate.lock.ts +283 -0
- package/src/core/gate/gate.service.ts +28 -0
- package/src/core/gate/gate.types.ts +67 -0
- package/src/core/handoff/handoff.service.ts +43 -0
- package/src/core/handoff/handoff.store.ts +59 -0
- package/src/core/handoff/handoff.types.ts +89 -0
- package/src/core/index.ts +52 -0
- package/src/core/integrity/state-seal.ts +81 -0
- package/src/core/lesson/lesson.authored.ts +96 -0
- package/src/core/lesson/lesson.credit.ts +51 -0
- package/src/core/lesson/lesson.garden.ts +283 -0
- package/src/core/lesson/lesson.link.ts +93 -0
- package/src/core/lesson/lesson.score.ts +61 -0
- package/src/core/lesson/lesson.select.ts +209 -0
- package/src/core/lesson/lesson.service.ts +110 -0
- package/src/core/lesson/lesson.store.ts +317 -0
- package/src/core/lesson/lesson.sync.ts +70 -0
- package/src/core/lesson/lesson.types.ts +92 -0
- package/src/core/lesson/lesson.validity.ts +44 -0
- package/src/core/observability/observability.report.ts +326 -0
- package/src/core/observability/observability.service.ts +328 -0
- package/src/core/observability/observability.store.ts +269 -0
- package/src/core/observability/observability.types.ts +193 -0
- package/src/core/observability/observability.why.ts +206 -0
- package/src/core/observe/observe.service.ts +76 -0
- package/src/core/plan/plan.detect.ts +47 -0
- package/src/core/plan/plan.service.ts +81 -0
- package/src/core/plan/plan.types.ts +25 -0
- package/src/core/policy/policy.defaults.ts +103 -0
- package/src/core/policy/policy.guard.ts +34 -0
- package/src/core/policy/policy.integrity.ts +238 -0
- package/src/core/policy/policy.loader.ts +118 -0
- package/src/core/policy/policy.operator.ts +82 -0
- package/src/core/policy/policy.posture.ts +79 -0
- package/src/core/policy/policy.rails.ts +45 -0
- package/src/core/policy/policy.types.ts +157 -0
- package/src/core/presence/presence.service.ts +111 -0
- package/src/core/presence/presence.store.ts +59 -0
- package/src/core/presence/presence.types.ts +9 -0
- package/src/core/release/release.decisions.ts +147 -0
- package/src/core/release/release.seen.ts +38 -0
- package/src/core/shell-policy/shell-policy.service.ts +215 -0
- package/src/core/shell-policy/shell-policy.stall.ts +57 -0
- package/src/core/shell-policy/shell-policy.types.ts +26 -0
- package/src/core/ship/ship.ledger.ts +76 -0
- package/src/core/ship/ship.service.ts +137 -0
- package/src/core/ship/ship.types.ts +19 -0
- package/src/core/stagnation/stagnation.resolution.ts +87 -0
- package/src/core/stagnation/stagnation.service.ts +20 -0
- package/src/core/stagnation/stagnation.store.ts +49 -0
- package/src/core/stagnation/stagnation.types.ts +12 -0
- package/src/core/subagent-policy/subagent-policy.parent-model.ts +202 -0
- package/src/core/subagent-policy/subagent-policy.service.ts +139 -0
- package/src/core/subagent-policy/subagent-policy.types.ts +22 -0
- package/src/core/supply-chain/supply-chain.catalog.ts +51 -0
- package/src/core/supply-chain/supply-chain.service.ts +217 -0
- package/src/core/turn/turn.activity.ts +94 -0
- package/src/core/turn/turn.autopilot.ts +131 -0
- package/src/core/turn/turn.failure-signals.ts +193 -0
- package/src/core/turn/turn.loop-counter.ts +71 -0
- package/src/core/turn/turn.types.ts +14 -0
- package/src/core/untrusted/untrusted.detect.ts +53 -0
- package/src/core/untrusted/untrusted.recall.ts +87 -0
- package/src/core/untrusted/untrusted.service.ts +129 -0
- package/src/core/untrusted/untrusted.store.ts +75 -0
- package/src/core/untrusted/untrusted.types.ts +42 -0
- package/src/entrypoints/compact-before.ts +14 -0
- package/src/entrypoints/prompt-submit.ts +25 -0
- package/src/entrypoints/response-after.ts +60 -0
- package/src/entrypoints/run.ts +179 -0
- package/src/entrypoints/session-end.ts +58 -0
- package/src/entrypoints/session-start.ts +172 -0
- package/src/entrypoints/shim.ts +46 -0
- package/src/entrypoints/stop.ts +876 -0
- package/src/entrypoints/subagent-start.ts +16 -0
- package/src/entrypoints/subagent-stop.ts +32 -0
- package/src/entrypoints/support.ts +182 -0
- package/src/entrypoints/tool-after.ts +111 -0
- package/src/entrypoints/tool-before.ts +186 -0
- package/src/entrypoints/tool-failure.ts +26 -0
- package/src/platform/backoff.ts +49 -0
- package/src/platform/cli-output.ts +30 -0
- package/src/platform/env-scope.ts +22 -0
- package/src/platform/fs-atomic.ts +142 -0
- package/src/platform/fs-jsonl.ts +24 -0
- package/src/platform/git.ts +187 -0
- package/src/platform/paths.ts +80 -0
- package/src/platform/pricing.ts +229 -0
- package/src/platform/process.ts +64 -0
- package/src/platform/sanitize.ts +23 -0
- package/src/platform/screen.ts +46 -0
- package/src/platform/style.ts +103 -0
- package/src/providers/claude/claude.capabilities.ts +25 -0
- package/src/providers/claude/claude.detect.ts +14 -0
- package/src/providers/claude/claude.inbound.ts +252 -0
- package/src/providers/claude/claude.lessons-view.ts +33 -0
- package/src/providers/claude/claude.outbound.ts +65 -0
- package/src/providers/claude/claude.policy-defaults.ts +9 -0
- package/src/providers/claude/claude.transcript.ts +62 -0
- package/src/providers/claude/claude.wiring.ts +245 -0
- package/src/providers/claude/index.ts +17 -0
- package/src/providers/cursor/cursor.capabilities.ts +28 -0
- package/src/providers/cursor/cursor.detect.ts +14 -0
- package/src/providers/cursor/cursor.inbound.ts +203 -0
- package/src/providers/cursor/cursor.lessons-view.ts +30 -0
- package/src/providers/cursor/cursor.outbound.ts +43 -0
- package/src/providers/cursor/cursor.policy-defaults.ts +9 -0
- package/src/providers/cursor/cursor.wiring.ts +214 -0
- package/src/providers/cursor/index.ts +17 -0
- package/src/providers/index.ts +9 -0
- package/src/providers/provider.degrade.ts +152 -0
- package/src/providers/provider.port.ts +20 -0
- package/src/providers/provider.registry.ts +28 -0
- package/tools/doctor.ts +631 -0
- package/tools/help-topic.ts +74 -0
- package/tools/init-project.ts +277 -0
- package/tools/install-runtime.ts +164 -0
- package/tools/lessons-cli.ts +474 -0
- package/tools/obs-cli.ts +175 -0
- package/tools/price-lookup.ts +57 -0
- package/tools/refresh-model-prices.ts +210 -0
- package/tools/test-env.mjs +28 -0
- package/tools/test-env.names.d.mts +1 -0
- package/tools/test-env.names.mjs +14 -0
- package/tools/uninstall-runtime.ts +463 -0
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-035 — A lesson learned by reasoning can be written down, and the harness never learns where lessons come from"
|
|
4
|
+
description: "The lesson store had one producer — gate stagnation — so a pattern noticed across several changes, an operator's correction or an incident had no way in, while the mechanism that would carry it already ran every session. `tlc harness lessons add` is that route. It reads no documentation, knows no decision-record convention, and assumes no directory, because the harness runs in many products and one project's filing habits are not a feature."
|
|
5
|
+
tags: [decision, lessons, cli, coupling]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-035 — A lesson learned by reasoning can be written down
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`tlc harness lessons add "<instruction>"` writes a lesson**, with optional `--gate`, `--avoid`, `--prefer` and
|
|
17
|
+
`--tokens`.
|
|
18
|
+
2. **It is `source: "manual"`** — a value already present in `LessonSource` and produced by nothing.
|
|
19
|
+
3. **It is `active` immediately**, not `candidate`. A candidate exists because the automatic producer guesses from
|
|
20
|
+
output; an author is not guessing.
|
|
21
|
+
4. **With no `--gate` it takes a neutral gate**, so a retry for a specific gate is not falsely boosted. Session-mode
|
|
22
|
+
injection carries it regardless.
|
|
23
|
+
5. **The id is a hash of the normalised instruction**, so rewriting the same lesson updates it in place.
|
|
24
|
+
6. **A lesson authored inside an agent session records that in its category.** Recorded, not refused.
|
|
25
|
+
7. **The harness reads no documentation to find lessons.** No decision-record convention, no directory, no file
|
|
26
|
+
format. It ships the mechanism; a project decides what feeds it.
|
|
27
|
+
|
|
28
|
+
## Why
|
|
29
|
+
|
|
30
|
+
The harness injects lessons at session start and on retry, ranked, with recurrence decay and gardening ([/decisions/ad-023.md](/decisions/ad-023.md)). That
|
|
31
|
+
machinery is the right shape for exactly one purpose: carrying what was learned back to the next turn. And it had a
|
|
32
|
+
single producer — `recordLessonFromFailure`, triggered by the same gate fingerprint appearing twice.
|
|
33
|
+
|
|
34
|
+
So everything learned any other way stayed out. A pattern noticed across several changes, a code review, a production
|
|
35
|
+
incident, an operator's correction: none of it could enter the one channel built to deliver it. The measured cost is
|
|
36
|
+
this repository's own week — the same defect family shipped **five times** (a consumer written before its producer),
|
|
37
|
+
each instance recorded in its own decision record, and the pattern never written anywhere that loads before work
|
|
38
|
+
starts. Thirty-four decision records, none of them injected.
|
|
39
|
+
|
|
40
|
+
`LessonSource` already contained `"manual"`, produced by nothing — the union anticipated this route and it was never
|
|
41
|
+
built. Sixth instance of the family, found while fixing the family.
|
|
42
|
+
|
|
43
|
+
## The coupling this deliberately refuses
|
|
44
|
+
|
|
45
|
+
The obvious next step was to let a decision record feed the store: put a `lesson:` field in the frontmatter next to
|
|
46
|
+
the `migration:` field, and have the harness read `docs/decisions/`. It was proposed and rejected by the operator, and
|
|
47
|
+
the rejection is right.
|
|
48
|
+
|
|
49
|
+
`docs/decisions/ad-NNN.md` with OKF frontmatter is **this repository's filing habit**, decided in AD-013 for its own
|
|
50
|
+
documentation bundle. Another product using this harness may keep ADRs in `doc/adr/0001-*.md`, or in a wiki, or
|
|
51
|
+
nowhere. A harness that reads one layout to harvest lessons would make every install carry one project's conventions —
|
|
52
|
+
the same mistake as the earlier coupling to an external methodology's vocabulary, which AD-025 had to undo.
|
|
53
|
+
|
|
54
|
+
The line is: **the harness ships the mechanism, the project decides what feeds it.** If this repository wants its
|
|
55
|
+
decision records to produce lessons, that is a script in this repository calling `lessons add` — not behaviour in the
|
|
56
|
+
tool that every other product installs.
|
|
57
|
+
|
|
58
|
+
## Trade-offs
|
|
59
|
+
|
|
60
|
+
**An agent can write its own injected context.** That is a self-modification surface, and refusing it was considered.
|
|
61
|
+
It loses to the alternative: the reason five patterns went unrecorded is that recording required the operator to
|
|
62
|
+
transcribe what the agent had learned, and that never happens. The mitigations are that the source and the
|
|
63
|
+
in-session marker are recorded, `lessons list` shows them, and `garden` decays and quarantines them like anything
|
|
64
|
+
else. The store already carried harness-authored content from gate output, so the boundary being crossed is narrower
|
|
65
|
+
than it looks.
|
|
66
|
+
|
|
67
|
+
**`active` from the start skips the promotion threshold**, which exists to stop a one-off gate failure from becoming
|
|
68
|
+
advice. An authored lesson has no recurrence to accumulate, so waiting for `promoteHitCount` would mean it never
|
|
69
|
+
activates. The cost is that a badly-written authored lesson is injected immediately; `garden` and `lessons list` are
|
|
70
|
+
the recourse.
|
|
71
|
+
|
|
72
|
+
**The neutral gate is a string that reads in the rendered line**, so an operator sees `[any/active]`. A separate
|
|
73
|
+
"applies to everything" flag would be cleaner and would touch the ranking code, which is not worth it for the
|
|
74
|
+
difference.
|
|
75
|
+
|
|
76
|
+
## Not decided here
|
|
77
|
+
|
|
78
|
+
- **`scope` is read by nothing.** It is a single-value union on every lesson and no ranking or selection consults it.
|
|
79
|
+
Found while building this, left alone — removing a persisted field is its own change, and an authored lesson carries
|
|
80
|
+
the same value rather than inventing an unread second one.
|
|
81
|
+
- **A repository-local bridge from decision records to lessons.** Legitimate for this repository, and it belongs here
|
|
82
|
+
rather than in the shipped tool. Not built.
|
|
83
|
+
- **Whether the floor should refuse `lessons add` in-session.** It does not today, by the reasoning above. If a
|
|
84
|
+
self-written lesson ever causes harm, the marker is what makes that case findable.
|
|
85
|
+
- **De-duplicating an authored lesson against an automatic one** saying the same thing in different words. The ids
|
|
86
|
+
cannot collide, so both would be injected.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-036 — A lesson names what makes it true, and stops being injected when that is gone"
|
|
4
|
+
description: "A lesson is a claim about a repository, and the repository outlives the claim. `tools/check-dist-fresh.ts` gets renamed and the lesson naming it turns from useful into misleading, with the authority of a lesson. A lesson may now carry refs — repository-relative `path` or `path:symbol` — and `garden` withholds it when one stops resolving. An unreadable file is deferred, never stale."
|
|
5
|
+
tags: [decision, lessons, staleness]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-036 — A lesson names what makes it true
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **A lesson carries zero or more `refs`**, each a repository-relative `{ path, symbol? }`, written with
|
|
17
|
+
`tlc harness lessons add --ref path[:symbol]`. The flag may repeat.
|
|
18
|
+
2. **Four statuses**: `present`, `path-missing`, `symbol-missing`, `unreadable`.
|
|
19
|
+
3. **`unreadable` is deferred, never stale.** A file this process cannot open is not evidence that the lesson
|
|
20
|
+
stopped being true — it is evidence that the check could not run.
|
|
21
|
+
4. **`garden` sets `staleReason` and clears it** when every ref resolves again. A rename can be reverted.
|
|
22
|
+
5. **A stale lesson is withheld from injection** in both session and retry mode, and is not deleted.
|
|
23
|
+
6. **A lesson with no refs is never stale.** Most lessons are about conduct, and demanding a ref would make
|
|
24
|
+
authors invent one.
|
|
25
|
+
7. **A symbol is checked by substring, not by parsing.** The question is whether the name still appears in the
|
|
26
|
+
file.
|
|
27
|
+
8. **Staleness is persisted only for the project tier.** The global tier is decided per repository at selection
|
|
28
|
+
time — see AD-040.
|
|
29
|
+
|
|
30
|
+
## Why
|
|
31
|
+
|
|
32
|
+
The harness could expire a lesson by attention and never by truth: quarantine after ninety idle days, prune after
|
|
33
|
+
a hundred and eighty. Neither asks whether the lesson is still *correct*.
|
|
34
|
+
|
|
35
|
+
The concrete case is in this repository's own memory. A recorded lesson says to run `tools/check-dist-fresh.ts`
|
|
36
|
+
before committing. On the day that file is renamed the lesson keeps being injected, and it is now worse than
|
|
37
|
+
absent: it sends the next turn looking for something that does not exist, ranked above lessons that are true.
|
|
38
|
+
Nothing in the harness could notice, because nothing in the harness knew the lesson was about a file.
|
|
39
|
+
|
|
40
|
+
The mechanism is taken from a prior implementation that stores linked symbols on a memory and marks the
|
|
41
|
+
row stale when a linked symbol disappears. Two things were kept and one was dropped. Kept: the `path_renamed` /
|
|
42
|
+
`symbol_deleted` distinction, and the treatment of an unreadable file as deferred rather than stale — the
|
|
43
|
+
conservative direction. Dropped: the LSP-backed symbol checker. That implementation's own fallback is
|
|
44
|
+
`text.includes(symbol)`, which is the whole mechanism at zero dependency and one file read.
|
|
45
|
+
|
|
46
|
+
## The hazard an absolute path would create
|
|
47
|
+
|
|
48
|
+
A ref is repository-relative and an absolute one never resolves. Otherwise a global lesson naming
|
|
49
|
+
`/home/me/repos/a/tools/gate.ts` would report `present` in every product on that machine, because the path
|
|
50
|
+
resolves regardless of which repository is open. Refs describe a repository, not a disk.
|
|
51
|
+
|
|
52
|
+
## Trade-offs
|
|
53
|
+
|
|
54
|
+
**Substring matching accepts a false `present`.** A symbol mentioned in a comment, or a name that is a substring
|
|
55
|
+
of another, keeps a lesson alive that a rename should have retired. The cost is one stale lesson; the alternative
|
|
56
|
+
is a parser per language per turn, which the hook's latency budget forbids.
|
|
57
|
+
|
|
58
|
+
**Refs are author-supplied and never inferred.** Deriving them from gate output was considered and refused:
|
|
59
|
+
guessed links would mark true lessons stale, which is a worse failure than no links at all.
|
|
60
|
+
|
|
61
|
+
**Staleness is reported, not auto-deleted.** A broken ref is a filesystem observation, and a revert or a branch
|
|
62
|
+
switch undoes it. Only an expired validity window — which the author declared — is pruned ([/decisions/ad-037.md](/decisions/ad-037.md)).
|
|
63
|
+
|
|
64
|
+
## Not decided here
|
|
65
|
+
|
|
66
|
+
- **Whether a stale lesson should eventually be pruned.** Today it is withheld and reported forever. The existing
|
|
67
|
+
decay path does not reach it, because decay measures recurrence.
|
|
68
|
+
- **Refs on core lessons.** They ship with none; the six are about conduct, not about files.
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-037 — A lesson can be true for a period, and an unparseable bound withholds it"
|
|
4
|
+
description: "Some knowledge has a known end: pin the formatter until the toolchain moves. A lesson may now carry `validFrom` / `validTo`, and is injected only inside that window. An unparseable bound fails the window rather than being ignored, because the conservative side is the one where a claim nobody can evaluate does not reach the turn."
|
|
5
|
+
tags: [decision, lessons, validity]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-037 — A lesson can be true for a period
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **Optional `validFrom` and `validTo`**, ISO strings, written with `tlc harness lessons add --until <iso>`.
|
|
17
|
+
2. **Active when** `(validFrom absent or <= now) and (validTo absent or > now)`. The lower bound is inclusive, the
|
|
18
|
+
upper is exclusive.
|
|
19
|
+
3. **An unparseable bound fails the window.** The lesson is withheld.
|
|
20
|
+
4. **An empty string is absent, not invalid.**
|
|
21
|
+
5. **`garden` prunes an expired lesson.** Unlike a broken ref, the end was declared by the author.
|
|
22
|
+
6. **`validityReason` names the state** — `active`, `pending`, `expired`, `invalid` — so a report can distinguish
|
|
23
|
+
"not yet" from "no longer".
|
|
24
|
+
|
|
25
|
+
## Why
|
|
26
|
+
|
|
27
|
+
The store had two exits, both driven by idle time. Neither can express "this was true and now is not", which is
|
|
28
|
+
the normal fate of a lesson about a version, a migration in progress, or a workaround for a bug that will be
|
|
29
|
+
fixed. Without it, the author's choice is to inject something they know will expire, or not record it.
|
|
30
|
+
|
|
31
|
+
Taken from a prior validity filter, including its conservative rule: an invalid ISO string fails the window.
|
|
32
|
+
Ignoring an unparseable bound would inject the lesson — treating a broken declaration as no declaration — and the
|
|
33
|
+
author who wrote `--until "next tuesday"` intended a limit. Failing closed surfaces the typo through
|
|
34
|
+
`lessons list` and `doctor` instead of silently disregarding it.
|
|
35
|
+
|
|
36
|
+
## Trade-offs
|
|
37
|
+
|
|
38
|
+
**Pruning on expiry deletes knowledge.** A lesson that expires is gone, not quarantined. That is the difference
|
|
39
|
+
from staleness: a filesystem observation can be wrong or reverted, and a date the author wrote cannot.
|
|
40
|
+
|
|
41
|
+
**`validFrom` has no CLI flag.** Only `--until` is exposed, because the case for a lesson that starts later has
|
|
42
|
+
not appeared. The field is read wherever validity is evaluated, so a future flag needs no new logic.
|
|
43
|
+
|
|
44
|
+
## Not decided here
|
|
45
|
+
|
|
46
|
+
- **Reminding the operator before a window closes.** `doctor` reports it after the fact.
|
|
47
|
+
- **A relative form** such as `--until 30d`. Absolute ISO only, so a stored record means one thing.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-038 — Promotion counts distinct sessions, because one stuck session is one observation"
|
|
4
|
+
description: "A candidate lesson was promoted on `hitCount >= promoteHitCount`, and `hitCount` counted fingerprint recurrences with no session identity. One session hammering the same failure five times promoted a lesson that had occurred once in the world. Promotion now counts the distinct sessions that produced it; `hitCount` keeps counting recurrence and no longer decides."
|
|
5
|
+
tags: [decision, lessons, promotion]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-038 — Promotion counts distinct sessions
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **A lesson records the distinct session keys that produced it**, capped at twelve.
|
|
17
|
+
2. **`garden` promotes a candidate when the distinct session count reaches `promoteHitCount`.**
|
|
18
|
+
3. **`hitCount` keeps counting raw recurrence** and is still displayed. It no longer decides promotion.
|
|
19
|
+
4. **A record with no session keys falls back to `hitCount`.** Every store already on disk predates the field, and
|
|
20
|
+
freezing those lessons as candidates forever would be a silent regression.
|
|
21
|
+
5. **Quarantine uses the same count**, so the promotion threshold means one thing in both directions.
|
|
22
|
+
6. **An authored lesson carries no session keys** and never consults them — it is already `active` ([/decisions/ad-035.md](/decisions/ad-035.md)).
|
|
23
|
+
|
|
24
|
+
## Why
|
|
25
|
+
|
|
26
|
+
`promoteHitCount` defaults to 2 and reads as "seen twice". It was not: `recordLessonFromFailure` incremented
|
|
27
|
+
`hitCount` on every recurrence of the same gate fingerprint, and a single stuck session produces that recurrence by
|
|
28
|
+
definition — the stagnation rail exists because sessions repeat themselves. So the threshold measured persistence
|
|
29
|
+
within one session, and promoted advice from a single observation to active guidance injected at every session
|
|
30
|
+
start.
|
|
31
|
+
|
|
32
|
+
A prior pattern miner makes the distinction explicitly: it aggregates escalation events into
|
|
33
|
+
`uniqueSessionCount` and requires that to cross `patternMiningThreshold` before writing a gotcha, deriving
|
|
34
|
+
importance from the same number. The rule is the transferable part — the recurrence that justifies a durable claim
|
|
35
|
+
is recurrence *across* sessions.
|
|
36
|
+
|
|
37
|
+
## Trade-offs
|
|
38
|
+
|
|
39
|
+
**The cap can lose a promotion signal.** Twelve keys is a bounded record; a lesson recurring across more sessions
|
|
40
|
+
counts twelve. The threshold is a small number, so the cap cannot prevent a promotion that should happen.
|
|
41
|
+
|
|
42
|
+
**Distinct sessions is still not distinct causes.** Two sessions on the same branch hitting the same wall are two
|
|
43
|
+
observations of one situation. Counting causes would need a notion of cause the harness does not have.
|
|
44
|
+
|
|
45
|
+
**Legacy fallback lets an old record promote on within-session recurrence.** Accepted deliberately: the alternative
|
|
46
|
+
punishes existing stores for a field that did not exist when they were written.
|
|
47
|
+
|
|
48
|
+
## Not decided here
|
|
49
|
+
|
|
50
|
+
- **Weighting a session by how much work it contained.** A one-turn session and a fifty-turn session count the
|
|
51
|
+
same.
|
|
52
|
+
- **Whether `hitCount` should stay.** It is displayed and it is honest about what it measures, so it stays.
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-039 — A lesson is graded by the gate it was injected for, and unproven is not a passing reading"
|
|
4
|
+
description: "Nothing in the harness measured whether a lesson ever helped. Injection was recorded; what happened next was not. The lessons injected for a failing gate are now credited by that gate's next run — passed is `helped`, failed is `neutral` — and a lesson nothing has graded reads `unproven`, which `doctor` reports as a warning rather than folding into a zero rate."
|
|
5
|
+
tags: [decision, lessons, measurement, observability]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-039 — A lesson is graded by the gate it was injected for
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **Three counters on a lesson**: `injectedCount`, `helpedCount`, `neutralCount`.
|
|
17
|
+
2. **When lessons are injected on a retry for gate G**, their ids and G are written to the handoff as a pending
|
|
18
|
+
credit.
|
|
19
|
+
3. **On the next run of gate G**: passed → `helpedCount`, failed → `neutralCount`.
|
|
20
|
+
4. **The gate name is compared.** Without it, lessons injected for `lint` would be graded by whichever gate ran
|
|
21
|
+
next, which is `test`, and read as help the lesson never gave.
|
|
22
|
+
5. **Consumed exactly once.** The pending credit is cleared whether or not any lesson matched.
|
|
23
|
+
6. **`helpRate` is `null` over zero graded injections, never zero.** Zero reads as "measured and it never helped".
|
|
24
|
+
7. **`unproven` is a `doctor` warning, not an `ok` row** — and only for a lesson that has actually been injected.
|
|
25
|
+
8. **Core lessons are never credited.** They ship inside the runtime, identical in every install, and there is no
|
|
26
|
+
file to write a counter to.
|
|
27
|
+
|
|
28
|
+
## Why
|
|
29
|
+
|
|
30
|
+
The harness ranks lessons by recurrence, decay, gate match and trigger tokens. Every one of those is a proxy for
|
|
31
|
+
usefulness. None of them is usefulness. So a lesson could be injected at every session start for months, spend its
|
|
32
|
+
share of the budget, and no number anywhere would say whether it ever changed an outcome.
|
|
33
|
+
|
|
34
|
+
The signal was already within reach. `gate.outcome` began being emitted in AD-027, and the retry path already knows
|
|
35
|
+
which lessons it selected and which gate is failing. Joining the two is the difference between "this lesson matches
|
|
36
|
+
the failing gate" and "this lesson was present when that gate started passing".
|
|
37
|
+
|
|
38
|
+
The `null`-over-zero rule is taken from a prior rate KPI helper, which returns `value: null` when the denominator
|
|
39
|
+
is zero, and from the `met: boolean | null` of its KPI targets. That third state is the point. A capability with
|
|
40
|
+
`attempted > 0` and `effective == 0` is wired and broken; one with `attempted == 0` is **unproven**, and reading the
|
|
41
|
+
second as healthy is how this repository shipped a report that printed a truthful-looking `0 / 0` for every gate it
|
|
42
|
+
had ever run.
|
|
43
|
+
|
|
44
|
+
## What this is not
|
|
45
|
+
|
|
46
|
+
**It is not causal.** A gate passing after a lesson was injected does not establish that the lesson caused it — the
|
|
47
|
+
turn also read the gate output, changed files, and may have ignored the lesson entirely. `neutral` likewise does not
|
|
48
|
+
mean the lesson was wrong; it means the gate failed again.
|
|
49
|
+
|
|
50
|
+
The claim is narrower and still worth having: a lesson with many `neutral` gradings and no `helped` ones has been
|
|
51
|
+
present at many failures and no recoveries, and an operator can act on that. A properly causal answer needs the same
|
|
52
|
+
task run with and without the lesson, which requires task repetition, and real work does not repeat — the reason
|
|
53
|
+
AD-027 built observation mode instead of an A/B probe.
|
|
54
|
+
|
|
55
|
+
## Trade-offs
|
|
56
|
+
|
|
57
|
+
**A credit can land on the wrong turn.** If the gate is not reached on the next stop — no changed files, a cap hit,
|
|
58
|
+
a skip flag — the credit waits, and a later run grades it. The gate name bounds the error; time does not.
|
|
59
|
+
|
|
60
|
+
**The handoff carries session state that outlives a session.** A pending credit written at the end of one session is
|
|
61
|
+
graded in the next. That is intended: the retry loop it belongs to often spans a compaction.
|
|
62
|
+
|
|
63
|
+
## Not decided here
|
|
64
|
+
|
|
65
|
+
- **Ranking by effectiveness.** The counters are recorded and reported and do not yet feed `rankScore`. Boosting on
|
|
66
|
+
a non-causal signal would make the ranking self-confirming, which is the hazard already documented for
|
|
67
|
+
`lastAccessedAt`.
|
|
68
|
+
- **Auto-quarantining a lesson with many neutral gradings.** Reported; the operator decides.
|
|
69
|
+
- **Grading session-start injections.** Only the retry path has a gate to compare against.
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-040 — Three lesson tiers, and nothing crosses between products by itself"
|
|
4
|
+
description: "Four of the five patterns recorded on 2026-08-04 are general engineering discipline and were locked in one repository's store, so they would not reach the same operator working in another product. A global tier — this machine, every product — sits between the shipped core and the project store. Promotion into it is an operator command and never automatic, and `projectScoped` is replaced by the tier it duplicated."
|
|
5
|
+
tags: [decision, lessons, tiers, coupling]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-040 — Three lesson tiers
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **Three tiers.** `core` ships inside the runtime, identical everywhere, immutable. `global` is
|
|
17
|
+
`<runtime home>/state/lessons.json` — this machine, every product. `project` is
|
|
18
|
+
`<repo>/.tlc/harness/state/lessons.json`.
|
|
19
|
+
2. **`allLessons` returns core + global + project**, deduplicated by id with the nearer tier winning.
|
|
20
|
+
3. **`tlc harness lessons add --global`** writes to the global tier.
|
|
21
|
+
4. **`tlc harness lessons promote <id>`** copies a project lesson into the global tier. Operator-invoked only.
|
|
22
|
+
5. **No automatic promotion.** A lesson learned in product A never reaches product B unattended.
|
|
23
|
+
6. **`projectBoost` favours the project tier**, so local knowledge outranks knowledge carried in.
|
|
24
|
+
7. **`tier` is derived on read** from the store the lesson came out of, never trusted from the file.
|
|
25
|
+
8. **`projectScoped` is removed.** It was a persisted boolean meaning "gets the boost" that read as "lives in the
|
|
26
|
+
project store"; with three tiers the two stop coinciding.
|
|
27
|
+
9. **Staleness for a global lesson is decided per repository at selection time**, not persisted.
|
|
28
|
+
10. **`garden` sweeps both writable tiers.** Decay, promotion and expiry apply identically; only staleness differs.
|
|
29
|
+
|
|
30
|
+
## Why
|
|
31
|
+
|
|
32
|
+
The measured gap is this repository's own week. Five patterns were recorded on 2026-08-04. One is local — this
|
|
33
|
+
repository's CI sets `TLC_HOME`. The other four are not about this repository at all:
|
|
34
|
+
|
|
35
|
+
- run the gate itself, never an approximation of its steps
|
|
36
|
+
- read the operator's output before anyone else tests
|
|
37
|
+
- a surviving mutant is usually one that never landed
|
|
38
|
+
- a consumer written before its producer reads as working
|
|
39
|
+
|
|
40
|
+
Those are true in `platform`, in `fitness-portal`, in any repository. They were in a store that only this
|
|
41
|
+
repository reads, so the same mistakes were queued to repeat everywhere else — and the harness is installed into
|
|
42
|
+
many products by design.
|
|
43
|
+
|
|
44
|
+
A prior system has the same three-layer shape, and its personal layer explicitly holds cross-project — "knowledge I
|
|
45
|
+
apply across any project" — with promotion drawn as `Personal → Team → Project`, one direction, operator-invoked.
|
|
46
|
+
The shape transferred; the middle layer did not. That system's Layer 2 is a synced team database. This harness has no
|
|
47
|
+
server and no sync, so its middle tier is *this machine* — the operator, across products. The precedent for the
|
|
48
|
+
location already existed: `obs.globalSpool` writes to the same runtime home for exactly the same reason.
|
|
49
|
+
|
|
50
|
+
## Why promotion is never automatic
|
|
51
|
+
|
|
52
|
+
It auto-promotes personal → team on decay and access thresholds, and that is defensible inside one project
|
|
53
|
+
where both layers describe the same codebase. Across products it is not. A lesson mined from a gate failure in
|
|
54
|
+
product A names product A's test runner, its file layout, its stack. Injected into product B it is confident,
|
|
55
|
+
specific and wrong.
|
|
56
|
+
|
|
57
|
+
Thresholds cannot separate the two cases, because the signal they measure — recurrence in A — says nothing about
|
|
58
|
+
applicability in B. Only the operator knows. So the tier is opt-in per lesson, twice: `--global` when writing, or
|
|
59
|
+
`promote` afterwards.
|
|
60
|
+
|
|
61
|
+
The residual risk is a global lesson that is genuinely irrelevant in some product. Two things bound it: the ranking
|
|
62
|
+
boost keeps project lessons above it, and a global lesson carrying refs withholds itself where those refs do not
|
|
63
|
+
resolve ([/decisions/ad-036.md](/decisions/ad-036.md)). That withholding is why the tier is safe rather than merely convenient — a lesson can state the
|
|
64
|
+
conditions under which it applies, and the absence of those conditions silences it.
|
|
65
|
+
|
|
66
|
+
## Why the rendered line names the tier
|
|
67
|
+
|
|
68
|
+
An injected lesson renders as `[gate/status/tier]`. A turn reading `[test/active/global]` can tell the advice was
|
|
69
|
+
written about a different repository and weigh it accordingly. The alternative — hiding the provenance — asks the
|
|
70
|
+
model to treat carried-in advice with the same authority as local knowledge.
|
|
71
|
+
|
|
72
|
+
## Trade-offs
|
|
73
|
+
|
|
74
|
+
**`projectScoped` removal touches every stored record.** No migration is needed, because the tier is derived from
|
|
75
|
+
the file the record was read from, and the removed field is simply ignored.
|
|
76
|
+
|
|
77
|
+
**The global store is per machine, not per operator account.** Two accounts on one machine with the same
|
|
78
|
+
`TLC_HOME` share it. `TLC_HOME` already defines the runtime, and every other runtime-home artefact has the same
|
|
79
|
+
property.
|
|
80
|
+
|
|
81
|
+
**A promoted lesson exists in two stores.** `promote` copies rather than moves, so the project keeps its own copy
|
|
82
|
+
and the dedup rule resolves to it. Moving would remove local knowledge as a side effect of sharing it.
|
|
83
|
+
|
|
84
|
+
## Not decided here
|
|
85
|
+
|
|
86
|
+
- **A team tier.** It needs sync, a server and a trust model. Out of scope.
|
|
87
|
+
- **`lessons demote`**, and deleting a global lesson from within a project. Editing the global store is the
|
|
88
|
+
operator's own act, at its printed path.
|
|
89
|
+
- **Whether the global tier should decay faster.** It uses the same `decayLambda`.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-041 — A member something reads and nothing writes fails the gate"
|
|
4
|
+
description: "This repository has shipped the same defect eight times: a consumer written before its producer, which reads as working because the consumer's default is plausible. `tools/check-wiring.ts` refuses it mechanically — every member of every exported string-literal union that is read anywhere must be written somewhere. Inventories are discovered, never registered, so the check cannot itself be forgotten."
|
|
5
|
+
tags: [decision, gate, tooling, dead-code]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-041 — A member something reads and nothing writes fails the gate
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`tools/check-wiring.ts` is a step of `tlc harness test`** and exits non-zero on any finding.
|
|
17
|
+
2. **Inventories are discovered, never registered.** Every exported string-literal union in the tracked non-test
|
|
18
|
+
source is scanned. A registry would need an entry per union, and the missing entry is the same class of omission
|
|
19
|
+
this exists to catch.
|
|
20
|
+
3. **The rule is one-directional: consumed and never produced.** Written-and-never-read is dead data, a lesser
|
|
21
|
+
smell, and often legitimate — a field emitted for a human to read in JSON.
|
|
22
|
+
4. **Occurrences are classified by preceding context**, and consumer context wins. A producer is a value position
|
|
23
|
+
(`: "x"`, `return "x"`, `=> "x"`, `("x"`, `, "x"`, `["x"`); a consumer is an unambiguous read (`=== "x"`,
|
|
24
|
+
`!== "x"`, `case "x"`, `includes("x")`, `has("x")`).
|
|
25
|
+
5. **Collection membership is a producer, not a read.** The same syntax declares an inventory.
|
|
26
|
+
6. **The file set is git's index**, so an untracked scratch file cannot change the verdict.
|
|
27
|
+
7. **`CONFIG_FACING` is the only escape hatch**, per union rather than per member, for types whose values arrive in
|
|
28
|
+
the operator's `config.json`. Each entry carries a reason and is printed in the clean report.
|
|
29
|
+
8. **The finding names the type, the member, the declaring file and every reader**, and states the two ways out:
|
|
30
|
+
write it, or delete it and the branches that read it.
|
|
31
|
+
|
|
32
|
+
## Why
|
|
33
|
+
|
|
34
|
+
The measured record, each instance its own decision or fix: `attrs.permission` ([/decisions/ad-026.md](/decisions/ad-026.md)), `gate.outcome` and
|
|
35
|
+
`policy.deny` ([/decisions/ad-027.md](/decisions/ad-027.md)), `observe.rails` values ([/decisions/ad-029.md](/decisions/ad-029.md)), `format.enabled` ([/decisions/ad-033.md](/decisions/ad-033.md)), `LessonSource "manual"`
|
|
36
|
+
([/decisions/ad-035.md](/decisions/ad-035.md)), and — found by this check on its first run — `FailureCategory "infra"` and the whole `GenAiOutcome`
|
|
37
|
+
union.
|
|
38
|
+
|
|
39
|
+
Every instance has the same shape. The consumer exists, the producer does not, and the consumer's default is a
|
|
40
|
+
plausible value: a counter reads zero, a flag reads false, a `case` is never entered. Tests pass, because each half
|
|
41
|
+
is tested separately and no test asserts that the two are connected. The session report printed `Gates pass/fail:
|
|
42
|
+
0 / 0` for every gate this harness had ever run, and it looked correct.
|
|
43
|
+
|
|
44
|
+
Prose does not fix this. Six decision records say some version of "wire the producer in the same change", and the
|
|
45
|
+
seventh instance shipped anyway. The August 2026 practitioner consensus on the same failure mode says the same
|
|
46
|
+
thing structurally: prompt-level instructions to double-check do not work reliably because the checker is the
|
|
47
|
+
maker, and what works is an out-of-band audit that cross-references declarations against registrations. That is
|
|
48
|
+
exactly a hook-audit, generalised — and the same writing warns that adding one rule per new defect produces rule
|
|
49
|
+
inflation. Hence one discovered, data-driven check rather than eight.
|
|
50
|
+
|
|
51
|
+
## What the first run found
|
|
52
|
+
|
|
53
|
+
Two real defects, on a repository whose gate was green:
|
|
54
|
+
|
|
55
|
+
- **`FailureCategory "infra"`** — two `case "infra":` branches, in the retry suggestion and in autopilot, that
|
|
56
|
+
nothing could ever enter. `classifyGateFailure` never returns it. Its text told the operator to check that lint
|
|
57
|
+
and test CLIs are on `PATH`, which is the case AD-021 already decided classifies as `config` — a duplicate
|
|
58
|
+
without a producer. Deleted.
|
|
59
|
+
- **`GenAiOutcome`** — a five-member union on `ObsEvent.gen_ai.outcome`, populated by nothing. `usageGenAi` is the
|
|
60
|
+
only producer of that object and never sets it. Sweeping the same field revealed four more never-written
|
|
61
|
+
siblings: `operation_name`, `provider_name`, `request_model`, `reasoning_tokens`. All five removed from the
|
|
62
|
+
shipped event schema.
|
|
63
|
+
|
|
64
|
+
## The two calibration errors, both mine, both measured
|
|
65
|
+
|
|
66
|
+
The first draft excluded the declaring file and required both directions. It reported **85 findings across 129
|
|
67
|
+
members** — 66% noise, because producers legitimately live beside their type and many members are written without
|
|
68
|
+
ever being compared. A check that cries wolf gets disabled, which is the defect family AD-034 named.
|
|
69
|
+
|
|
70
|
+
The second draft included the declaring file and required only "consumed, not produced": **2 findings**, one real,
|
|
71
|
+
one false. The false one was `SpoolStream "audit"`, where `appendSpoolRecord(root, "audit", record)` is a producer
|
|
72
|
+
that the comma-adjacent consumer pattern read as a comparison.
|
|
73
|
+
|
|
74
|
+
Fixing that exposed a worse error in the opposite direction. Adding `=` and `(` to the producer alternatives made
|
|
75
|
+
`=== "x"` and `includes("x")` match as *producers* — so any member compared anywhere counted as written, and the
|
|
76
|
+
check would have passed on nearly every dead rail it exists to catch. The sensor was weaker than its own
|
|
77
|
+
measurement claimed. Classifying each occurrence by its preceding context, with consumer context tested first, is
|
|
78
|
+
what makes the two unambiguous; both holes have a test.
|
|
79
|
+
|
|
80
|
+
## Trade-offs
|
|
81
|
+
|
|
82
|
+
**It matches strings, not types.** Two unions sharing a member string are conflated, and a member appearing in a
|
|
83
|
+
comment or a test fixture name counts. Accepted: the alternative is the TypeScript compiler API, and the
|
|
84
|
+
string-based version found two real defects on its first run with one false positive.
|
|
85
|
+
|
|
86
|
+
**`CONFIG_FACING` can be widened to silence a finding.** It is per union, so it cannot hide one dead member of an
|
|
87
|
+
internal union — only state that a whole type is operator-supplied. Both entries are printed on every clean run,
|
|
88
|
+
because a silent exemption reads as a passing check.
|
|
89
|
+
|
|
90
|
+
**Config-facing unions are therefore unchecked.** Nothing validates that `appendFiles` in a config file is one of
|
|
91
|
+
the three legal values. That is a real gap and it is not this check's subject.
|
|
92
|
+
|
|
93
|
+
## Not decided here
|
|
94
|
+
|
|
95
|
+
- **Validating config-supplied union values at load.** The gap above.
|
|
96
|
+
- **Non-union inventories** — `OBSERVABLE_RAILS` and similar constant arrays. Their invariant is documented and
|
|
97
|
+
tested at the call site; extending discovery to array constants would need a producer notion per constant.
|
|
98
|
+
- **Never-written optional scalar fields** that are not union members. The `gen_ai` sweep was done by hand.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Decision
|
|
3
|
+
title: "AD-042 — The suite gets an empty runtime home, and one renderer renders a lesson"
|
|
4
|
+
description: "Two defects the previous change shipped, both found by running it rather than by any check. `TLC_HOME` was deliberately left alone in the hermetic setup, which was safe only while nothing machine-wide lived under it — the global lesson tier does, so a test reading `allLessons` read whichever lessons the developer had promoted. And `support.ts` carried a copy of the lesson renderer, so the tier added to the core block never reached the text a model receives."
|
|
5
|
+
tags: [decision, testing, lessons, hermetic]
|
|
6
|
+
timestamp: "2026-08-04"
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# AD-042 — The suite gets an empty runtime home, and one renderer renders a lesson
|
|
10
|
+
|
|
11
|
+
- **status**: active
|
|
12
|
+
- **date**: 2026-08-04
|
|
13
|
+
|
|
14
|
+
## Decision
|
|
15
|
+
|
|
16
|
+
1. **`tools/test-env.mjs` points `TLC_HOME` at a fresh empty temp directory.** Redirected, not deleted — deleting
|
|
17
|
+
it would send every test at the developer's real `~/.tlc/harness`, which is the opposite of hermetic.
|
|
18
|
+
2. **A test that needs the real runtime says so.** `e2e-consistency.test.ts` drives the actual launcher and CLI, so
|
|
19
|
+
it passes `TLC_HOME: repoRoot` explicitly.
|
|
20
|
+
3. **`renderLessonLine` in `src/entrypoints/support.ts` delegates to the core renderer**, exposed on the facade as
|
|
21
|
+
`coreFacade.lesson.renderLessonBlock`. The copy is deleted.
|
|
22
|
+
4. **An end-to-end test asserts the tier appears in the text the hook returns**, not only in the core block.
|
|
23
|
+
|
|
24
|
+
## Why the runtime home had to change
|
|
25
|
+
|
|
26
|
+
`TLC_HOME` was excluded from the hermetic setup on an explicit rationale, written in
|
|
27
|
+
`tools/test-env.names.mjs`: it names *which runtime*, not which project, CI sets it deliberately, and the suite
|
|
28
|
+
exercises it.
|
|
29
|
+
|
|
30
|
+
That was true, and it stopped being sufficient the moment something machine-wide lived under it. AD-040 put the
|
|
31
|
+
global lesson tier at `<runtime home>/state/lessons.json`. From then on, any test calling `allLessons` without
|
|
32
|
+
pinning a home read whichever lessons the operator of that machine happened to have promoted.
|
|
33
|
+
|
|
34
|
+
The failure profile is what makes this worth a decision record: **green on a fresh machine, green in CI, green on
|
|
35
|
+
mine at commit time** — because my global store was empty — and red minutes later, when I promoted five lessons
|
|
36
|
+
into it. The gate ran three times and passed three times before the store had contents. Nothing about the change
|
|
37
|
+
was wrong; the test's assumption was, and the assumption only became visible once the feature was used.
|
|
38
|
+
|
|
39
|
+
`obs.globalSpool` ([/decisions/ad-018.md](/decisions/ad-018.md)) had already put a file under the runtime home and did not expose this, because no test
|
|
40
|
+
asserts a count over the spool. The lesson store is read by `allLessons`, which almost every lesson test calls.
|
|
41
|
+
|
|
42
|
+
## Why one renderer
|
|
43
|
+
|
|
44
|
+
`support.ts` carried `renderLessonLine`, a duplicate of `renderLessonBlock`, with a comment explaining the choice:
|
|
45
|
+
the core's version is presentation-only and was not on the facade, so reaching into core internals was avoided by
|
|
46
|
+
copying it.
|
|
47
|
+
|
|
48
|
+
The copy is the one the model actually receives. So `[gate/status/tier]` — added in AD-040 precisely so a turn can
|
|
49
|
+
tell that a lesson was written about a different repository — rendered in `lessons list` and in nothing an agent
|
|
50
|
+
ever saw. Two derivations of one string is the same defect as a consumer without a producer, pointed sideways: both
|
|
51
|
+
halves work, and the one that matters is the one nobody checked.
|
|
52
|
+
|
|
53
|
+
Exposing it on the facade is the sanctioned route ([/decisions/ad-004.md](/decisions/ad-004.md)). "Presentation does not belong in core" was the right
|
|
54
|
+
instinct applied to the wrong thing — the block's shape *is* a core contract, because ranking, budgeting and the
|
|
55
|
+
synced markdown all depend on its length.
|
|
56
|
+
|
|
57
|
+
## How both were found
|
|
58
|
+
|
|
59
|
+
By running the thing and reading the output — the practice AD-034 records, not a mechanism. `check-wiring` ([/decisions/ad-041.md](/decisions/ad-041.md))
|
|
60
|
+
catches a declared member nothing writes; neither of these is that shape. Stated plainly: after this change the
|
|
61
|
+
harness mechanically refuses one defect family, and the other two found here still depend on someone exercising the
|
|
62
|
+
feature and reading what comes out.
|
|
63
|
+
|
|
64
|
+
## Trade-offs
|
|
65
|
+
|
|
66
|
+
**Every test now pays a `mkdtemp` at startup.** One syscall per test process, twice per gate run.
|
|
67
|
+
|
|
68
|
+
**A test that silently depended on the ambient runtime home now fails.** Three did, all in the E2E file, and all
|
|
69
|
+
three were better off stating the dependency. A future test that needs the real runtime will fail loudly rather
|
|
70
|
+
than pass for an ambient reason.
|
|
71
|
+
|
|
72
|
+
**The temp home is not cleaned up.** The OS reclaims `TMPDIR`; a directory per test process with nothing in it is
|
|
73
|
+
cheaper to leak than to track.
|
|
74
|
+
|
|
75
|
+
## Not decided here
|
|
76
|
+
|
|
77
|
+
- **Whether CI should keep setting `TLC_HOME` to the workspace.** It still does, and the setup module now overrides
|
|
78
|
+
it for the suite while `check-dist-fresh` and the CLI steps still read it.
|
|
79
|
+
- **A mechanical check that a test does not read machine state.** It would need to know which paths are
|
|
80
|
+
machine-wide; the hermetic setup covers the two that exist.
|
|
81
|
+
- **Auditing every other machine-wide read for the same assumption.** The runtime home is now empty for the suite,
|
|
82
|
+
which covers the class rather than the instances.
|