agent-bios 0.9.9 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DEPENDENCIES.md +3 -3
- package/README.md +10 -2
- package/claude/CLAUDE.md +4 -41
- package/claude/guides/cli-multi-model-workflow.md +7 -7
- package/claude/guides/coding-staged-workflow.md +51 -49
- package/claude/guides/concept-economy.md +187 -0
- package/claude/guides/documentation-hygiene.md +112 -0
- package/claude/guides/review-request.md +9 -7
- package/claude/guides/session-distill-workflow.md +3 -3
- package/claude/guides/tooling-gotchas.md +10 -0
- package/claude/guides/verification-discipline.md +166 -0
- package/claude/hooks/tooling-gotchas-hook.py +323 -13
- package/codex/AGENTS.md +4 -41
- package/codex/guides/cli-multi-model-workflow.md +7 -7
- package/codex/guides/coding-staged-workflow.md +51 -49
- package/codex/guides/concept-economy.md +187 -0
- package/codex/guides/documentation-hygiene.md +112 -0
- package/codex/guides/review-request.md +9 -7
- package/codex/guides/session-distill-workflow.md +3 -3
- package/codex/guides/tooling-gotchas.md +10 -0
- package/codex/guides/verification-discipline.md +166 -0
- package/compose/assemble.py +10 -1
- package/compose/check-domains.py +882 -6
- package/compose/domains.json +10 -44
- package/install.sh +176 -16
- package/launch/agent-launch.py +790 -173
- package/launch/agent-launch.toml +56 -107
- package/launch/i18n/en.toml +66 -0
- package/launch/i18n/ja.toml +63 -0
- package/launch/i18n/ko.toml +63 -0
- package/package.json +9 -4
- package/provenance.json +1 -0
- package/wrappers/codex-run.sh +1 -1
- package/claude/hooks/__pycache__/tooling-gotchas-hook.cpython-314.pyc +0 -0
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
---
|
|
2
|
+
guide_id: documentation-hygiene
|
|
3
|
+
language: en
|
|
4
|
+
status: active
|
|
5
|
+
use_when:
|
|
6
|
+
- writing a comment, and deciding whether it earns its place
|
|
7
|
+
- recording why something changed, an alternative that was rejected, or a migration's rationale
|
|
8
|
+
- deciding whether an active document should link to a historical note
|
|
9
|
+
- authoring a rule, guideline, or instruction others will follow
|
|
10
|
+
- choosing where change history and implementation context belong
|
|
11
|
+
core_rules:
|
|
12
|
+
- prose about the past and prose about the present need different addresses, not different tenses
|
|
13
|
+
- a comment earns its place by carrying what the code cannot say about itself
|
|
14
|
+
- phrase a rule as the behavior you want, because a prohibition describes everything except what to do
|
|
15
|
+
- a continuously overwritten "current state" file claims to be now and is no particular time
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
# Documentation Hygiene
|
|
19
|
+
|
|
20
|
+
A scoped extension of the global Documentation Hygiene section. The subject is placement: **prose
|
|
21
|
+
about the past and prose about the present need different addresses.**
|
|
22
|
+
|
|
23
|
+
The cost this manages is misreading, not tidiness. A sentence describing how something used to
|
|
24
|
+
work, sitting beside code that works differently now, reads as a present fact — to a person
|
|
25
|
+
skimming and to a model retrieving. Nothing marks it as history except a tense, and a tense is not
|
|
26
|
+
a signal anyone checks. Moving it is cheaper than maintaining the reader's suspicion.
|
|
27
|
+
|
|
28
|
+
## The two addresses
|
|
29
|
+
|
|
30
|
+
**Active surfaces** — runtime code, comments, the docs someone reads to operate the thing — carry
|
|
31
|
+
current behavior, current decisions, current contracts, current authority, current failure
|
|
32
|
+
handling. Nothing else.
|
|
33
|
+
|
|
34
|
+
**Isolated paths** — `docs/`, `design/`, `archive/`, `deprecated/` or whatever the repository uses
|
|
35
|
+
— carry backward-compatibility notes, deprecated behavior, migration rationale, historical
|
|
36
|
+
alternatives, change narratives, and handoff logs.
|
|
37
|
+
|
|
38
|
+
The test for any sentence: *if this stopped being true, would anyone notice?* Active surfaces are
|
|
39
|
+
where the answer must be yes, because something breaks. History is where the answer is no, which
|
|
40
|
+
is exactly why it must not sit in the first place.
|
|
41
|
+
|
|
42
|
+
## Comments carry what the code cannot
|
|
43
|
+
|
|
44
|
+
A comment earns its place by saying something the code cannot say about itself:
|
|
45
|
+
|
|
46
|
+
- Non-obvious current behavior — why this looks wrong and is right.
|
|
47
|
+
- An invariant a reader could break without noticing they broke it.
|
|
48
|
+
- A constraint that comes from outside the file: a protocol, a rate limit, an ordering another
|
|
49
|
+
system depends on.
|
|
50
|
+
- A risk that still applies. Not one that used to.
|
|
51
|
+
|
|
52
|
+
What does not earn its place: restating the line below it, narrating the change that introduced it
|
|
53
|
+
("changed this to fix the bug"), or describing a contract that has since moved. That last one is
|
|
54
|
+
not stale documentation — it is a **second, false authority**, and to whoever reads it first it is
|
|
55
|
+
simply the answer.
|
|
56
|
+
|
|
57
|
+
Comments describing superseded behavior are the highest-yield deletion in most files. They are
|
|
58
|
+
also the hardest to find, because nothing fails when they are wrong.
|
|
59
|
+
|
|
60
|
+
## History has its own address
|
|
61
|
+
|
|
62
|
+
When you have something worth recording that is not current behavior — why an alternative was
|
|
63
|
+
rejected, what a migration was compensating for, what a session handed off — write it in an
|
|
64
|
+
isolated path rather than beside the code.
|
|
65
|
+
|
|
66
|
+
Two properties make such a record useful:
|
|
67
|
+
|
|
68
|
+
- **It is written once.** A record that is edited whenever things change stops describing any
|
|
69
|
+
particular moment. If it must be updated, the honest move is a new record that supersedes the
|
|
70
|
+
old one, not an overwrite.
|
|
71
|
+
- **It carries its own point in time.** A timestamp in the filename cannot rot, because the age is
|
|
72
|
+
on the label. A file called "current state" makes a claim it cannot keep.
|
|
73
|
+
|
|
74
|
+
That second point generalizes past records. Any continuously overwritten "the state of things"
|
|
75
|
+
document claims to be now and is, in practice, no particular time — the last person to touch it
|
|
76
|
+
decided how current it is, and nobody else can tell. Where the underlying facts are derivable,
|
|
77
|
+
give the reader the command that re-derives them instead of a number someone typed.
|
|
78
|
+
|
|
79
|
+
## Linking back
|
|
80
|
+
|
|
81
|
+
Link from active material to a historical note only when the current task needs that history, or
|
|
82
|
+
when the reference genuinely helps a future maintainer — usually because the present design looks
|
|
83
|
+
arbitrary without it.
|
|
84
|
+
|
|
85
|
+
A link is a small permanent cost: it invites the reader to leave, and it must stay resolvable. An
|
|
86
|
+
active document that links to five historical notes has partly become one.
|
|
87
|
+
|
|
88
|
+
## Writing rules people follow
|
|
89
|
+
|
|
90
|
+
Phrase a rule as the behavior you want, not as the thing you are afraid of. A prohibition
|
|
91
|
+
describes everything except what to do, and leaves the reader to invent the positive form —
|
|
92
|
+
usually at the moment they have the least attention to spare.
|
|
93
|
+
|
|
94
|
+
- "Pin the interpreter when a bash-specific feature is needed" beats "don't rely on the default
|
|
95
|
+
shell".
|
|
96
|
+
- "State the assumption you are proceeding under" beats "don't guess".
|
|
97
|
+
|
|
98
|
+
Two more properties of a rule that survives contact:
|
|
99
|
+
|
|
100
|
+
- **It says when it fires.** A rule with no trigger is advice, and advice is followed when
|
|
101
|
+
convenient.
|
|
102
|
+
- **It says what it costs.** A rule whose expense is hidden gets quietly dropped the first time
|
|
103
|
+
someone is in a hurry, and nobody records that it was dropped.
|
|
104
|
+
|
|
105
|
+
## Where change history belongs
|
|
106
|
+
|
|
107
|
+
Prefer the established homes over inventing one per change: a changelog for released behavior, a
|
|
108
|
+
current-state dashboard for architecture and risk, handoff notes for what a session left unfinished.
|
|
109
|
+
|
|
110
|
+
Keep their jobs distinct. A changelog that accumulates design rationale becomes unreadable as a
|
|
111
|
+
changelog; a dashboard that accumulates history stops being current. When a document starts
|
|
112
|
+
answering a question it was not built for, that is a signal to split it, not to add a section.
|
|
@@ -26,10 +26,11 @@ verification_focus:
|
|
|
26
26
|
This guide is a scoped extension of the global Coding Guidelines. Use it when
|
|
27
27
|
composing what you ask a reviewer for — the request, the evidence bar, the
|
|
28
28
|
verdict shape. It does not cover when to review, how deep, or what counts as
|
|
29
|
-
material (the
|
|
30
|
-
which reviewer kind to route to
|
|
31
|
-
|
|
32
|
-
|
|
29
|
+
material (the severity ladder and review loop in `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/coding-staged-workflow.md` own that),
|
|
30
|
+
which reviewer kind to route to — the convergence heuristic in
|
|
31
|
+
`${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/verification-discipline.md` owns that. Phrasing a prompt for a specific model
|
|
32
|
+
family is out of scope here; where that guidance ships, the rule that needs it
|
|
33
|
+
points at it.
|
|
33
34
|
|
|
34
35
|
The rules below are derived from ~330 real multi-lens review sessions run in
|
|
35
36
|
this environment. That corpus is one model family in practice, so nothing here
|
|
@@ -80,8 +81,9 @@ So state the floor in the request: **do not report a finding you would rate low.
|
|
|
80
81
|
It will be discarded; spend the effort on a medium-or-above finding instead.**
|
|
81
82
|
This is not a quality bar on the reviewer, it is a cost decision — the corpus
|
|
82
83
|
shows the discard happens regardless, so the only question is whether you pay to
|
|
83
|
-
generate it first. (What counts as each severity is the
|
|
84
|
-
|
|
84
|
+
generate it first. (What counts as each severity is the ladder in
|
|
85
|
+
`${CLAUDE_CONFIG_DIR:-$HOME/.claude}/guides/coding-staged-workflow.md`, not
|
|
86
|
+
this guide's.)
|
|
85
87
|
|
|
86
88
|
## Forbid carry-forward findings
|
|
87
89
|
|
|
@@ -128,7 +130,7 @@ nicely: the submit schema refuses output without them. The result is a corpus
|
|
|
128
130
|
where reviewers are right about existence (0.3% of issues end unresolved after
|
|
129
131
|
argument) and where "found nothing" is a verified statement rather than silence.
|
|
130
132
|
|
|
131
|
-
This is the general
|
|
133
|
+
This is the general capability-boundary rule applied to review:
|
|
132
134
|
when output must have a property, make it unavailable without it. If your review
|
|
133
135
|
route has a schema, put the anchor there. If it does not, the demand belongs in
|
|
134
136
|
the request — but expect the weaker result that a request-only rule gives you.
|
|
@@ -50,9 +50,9 @@ Run in order; each stage reads the previous stage's `out/`:
|
|
|
50
50
|
2. `digest.py` — one secret-redacted digest per session with deterministic
|
|
51
51
|
6-criteria signals. Screen ALL digests; triage orders, never drops.
|
|
52
52
|
3. Provider-affine screening against the concatenated live baseline
|
|
53
|
-
(CLAUDE.md + guides): `screen-claude.js` (
|
|
54
|
-
`screen-codex.js` (
|
|
55
|
-
text, not memory.
|
|
53
|
+
(CLAUDE.md + guides): `screen-claude.js` (Claude sessions) and
|
|
54
|
+
`screen-codex.js` (Codex sessions), both dynamic-workflow scripts.
|
|
55
|
+
Novelty is judged against real baseline text, not memory.
|
|
56
56
|
4. `consolidate.js` — dedup + independent novelty verification. Rank by
|
|
57
57
|
strength (recurrence × materiality), never by self-reported confidence.
|
|
58
58
|
5. `bundle_final.py` — tiered bundle. Merge new candidates into
|
|
@@ -108,6 +108,16 @@ depends on it, pin it explicitly instead of trusting the environment.
|
|
|
108
108
|
|
|
109
109
|
## Git operations
|
|
110
110
|
|
|
111
|
+
- **A stale local base inflates the range**: before reasoning about what a branch
|
|
112
|
+
contains or opening a PR, `git fetch`, then ask against the remote rather than the
|
|
113
|
+
local tracking ref — `git log origin/<base>..HEAD` for which commits are yours, and
|
|
114
|
+
the merge-base form below for the diff. On a shared repo the local base lags until
|
|
115
|
+
you pull, so `<base>..HEAD` quietly folds in work that already merged. When a range
|
|
116
|
+
looks surprisingly large, suspect the base before the branch.
|
|
117
|
+
- **"Mergeable" is measured against the base, not against siblings**: the platform
|
|
118
|
+
flag says each PR merges into the base, and two PRs can both be clean while
|
|
119
|
+
conflicting with each other. Before choosing a merge order, diff their changed-file
|
|
120
|
+
sets and simulate the sequence.
|
|
111
121
|
- **Two-dot diff semantics**: `git diff A..B` is a direct snapshot
|
|
112
122
|
comparison — unlike `git log A..B` it excludes nothing, so a lagging
|
|
113
123
|
merge-base injects unrelated upstream changes into the diff. For PR/review
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
---
|
|
2
|
+
guide_id: verification-discipline
|
|
3
|
+
language: en
|
|
4
|
+
status: active
|
|
5
|
+
use_when:
|
|
6
|
+
- deciding how much verification a change deserves, before spending on a slow or expensive run
|
|
7
|
+
- choosing what to run for a domain — code, ontology, config/data, spreadsheets, docs, a release
|
|
8
|
+
- building the case space for a check, or deciding what its expected answer should be
|
|
9
|
+
- a check came back green, empty, or fast, and you are about to believe it
|
|
10
|
+
- running independent or adversarial review, and judging what its agreement is worth
|
|
11
|
+
core_rules:
|
|
12
|
+
- a check is only evidence if it could have failed — assert a non-empty subject before any "no bad X" claim
|
|
13
|
+
- enumerate the case space from the artifact that defines it, and record real output instead of typing an expectation
|
|
14
|
+
- proportion depth to cost, risk, and information gain; a single-user tool does not warrant production assurance
|
|
15
|
+
- same-kind reviewers share blind spots, so their shared "clean" is an absence of objection, not verification
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
# Verification Discipline
|
|
19
|
+
|
|
20
|
+
A scoped extension of the global Verification Discipline section. Its subject is not "did you
|
|
21
|
+
test it" but the harder question underneath: **could this check have failed?** Everything below
|
|
22
|
+
is a way of answering that before the result is believed rather than after it is quoted.
|
|
23
|
+
|
|
24
|
+
The global rules that stay always-loaded are the ones whose moment does not announce itself — you
|
|
25
|
+
believe you are finished, and that belief is the failure. This guide is what you open once you
|
|
26
|
+
know you are verifying.
|
|
27
|
+
|
|
28
|
+
## Proportion the depth before you spend
|
|
29
|
+
|
|
30
|
+
Verification has a cost and an information yield, and they are not correlated by default. Decide
|
|
31
|
+
the depth first:
|
|
32
|
+
|
|
33
|
+
- Diagnose in code before running anything expensive, and replay the changed deterministic logic
|
|
34
|
+
over persisted real artifacts rather than re-running the whole pipeline to observe it.
|
|
35
|
+
- Probe at N=1 with the inputs precondition-checked. A single well-chosen case that reaches the
|
|
36
|
+
real path outranks a hundred that stop short of it.
|
|
37
|
+
- Reserve the full design-review-plus-live-verification treatment for first-of-kind work and for
|
|
38
|
+
changes that move authority — who may decide, who may write, what is irreversible.
|
|
39
|
+
- Proportion assurance to the deployment context. A single-user tool operating on its owner's own
|
|
40
|
+
data does not warrant production-grade assurance, and treating it as if it did buys nothing
|
|
41
|
+
while delaying delivery. Prefer shipping.
|
|
42
|
+
|
|
43
|
+
The failure this prevents is not under-testing. It is spending the verification budget on the
|
|
44
|
+
cheap half of the risk and having nothing left for the part that could actually hurt.
|
|
45
|
+
|
|
46
|
+
## The static floor
|
|
47
|
+
|
|
48
|
+
Run the broad, cheap checks first and let them fail before anything slower starts: typecheck,
|
|
49
|
+
lint, build, format, schema and config validation, graph validation, workbook structure checks,
|
|
50
|
+
import boundaries, and security checks where they exist. These are a floor, not a verdict — they
|
|
51
|
+
prove the artifact is well-formed, never that it behaves.
|
|
52
|
+
|
|
53
|
+
## Verification Menus
|
|
54
|
+
|
|
55
|
+
Pick the narrowest reliable mix that proves the changed behavior, meaning, or contract. Inside the
|
|
56
|
+
mix, the unit to add is the narrowest reliable runtime or semantic test that proves it — narrowest
|
|
57
|
+
meaning the smallest test that would fail if the change were wrong, which is not the same as the
|
|
58
|
+
cheapest one to write.
|
|
59
|
+
|
|
60
|
+
- Code: a layered mix of unit tests, integration tests for E2E segments, targeted E2E for changed flows, and full E2E for release or high-risk changes.
|
|
61
|
+
- Ontology: static graph checks, concept economy gates, changed-path integration checks, and competency-question E2E checks.
|
|
62
|
+
- Config or data: real parsers, schema checks, fixture validation, and sample transformations.
|
|
63
|
+
- Spreadsheets: static workbook checks, fixture-based output checks, cross-sheet flow checks, visual/layout checks, and real Microsoft Excel engine recalculation for formula-dependent results.
|
|
64
|
+
- Docs: links, terminology, current behavior alignment, and references to isolated historical notes.
|
|
65
|
+
- Release or distribution: after publishing to multiple independently writable channels (signed manifest, object storage, release host, embedded updater), digest-verify every referenced object against the staging original per channel — publish success and upload order are not evidence — and run the real installer/updater through its default path.
|
|
66
|
+
- A/B or on/off measurements: before accepting a null result, verify the arms actually received different treatment in the mechanism under test — a shared default or unconditional upstream step can silently apply the treatment to both arms.
|
|
67
|
+
- Model-behavior guardrails: verify by changed behavior, not recitation — a staged battery from named-trigger cases through disguised, deconfounded, category-wide, and single-variable framings; a clean pass means "no known defect", so re-run the battery when the model changes.
|
|
68
|
+
- Branch/version test builds against real data: explicitly separate every state sink the app touches (files, DB, OS-level stores that ignore env overrides), confirm the launch path propagates the isolation to child processes, and back up live data before the first run — a mismatched schema that drops unknown fields on write is data loss, not a no-op.
|
|
69
|
+
- Irreversible capture switches: when activation itself has unreproducible cost (a capture window that cannot be replayed), prove the downstream consumption path against existing samples before enabling — reversibility of the code path alone is not enough.
|
|
70
|
+
|
|
71
|
+
## Deriving the case space
|
|
72
|
+
|
|
73
|
+
Which scenarios exist is semantic work: derive them from the diff, the user impact, the
|
|
74
|
+
concept impact, and the failure modes. Running them is not — tools and code execute the
|
|
75
|
+
cases and report the evidence. Keeping that split is what stops a suite from being a
|
|
76
|
+
record of what someone imagined.
|
|
77
|
+
|
|
78
|
+
A check has two authored halves, and they rot differently. The **verdict** — what the
|
|
79
|
+
answer should be — rots by encoding a belief that was wrong from the start. The
|
|
80
|
+
**space** — which cases exist — rots by staying still while the thing it covers grows.
|
|
81
|
+
Recording the verdict is common practice; deriving the space is the half usually left
|
|
82
|
+
hand-written, and a suite can have every expectation derived and still cover a set
|
|
83
|
+
someone typed once.
|
|
84
|
+
|
|
85
|
+
- Make the criterion falsifiable before you make it green. Prefer a signal that fails when
|
|
86
|
+
the mechanism is wrong — a negative or contrast control. Where no existing gate can judge
|
|
87
|
+
a criterion, build the executable judge or do not claim the criterion met: a criterion
|
|
88
|
+
nothing can fail is a description of the work, not a check on it.
|
|
89
|
+
- Record the verdict, do not type it. Run the real path and store what came back;
|
|
90
|
+
drift then shows as a diff instead of as a belief someone has to re-justify.
|
|
91
|
+
- Enumerate the space from the artifact that defines it — the config's entries, the
|
|
92
|
+
schema's fields, the router's routes, the installer's call sites. Adding one there
|
|
93
|
+
should widen coverage with no edit here.
|
|
94
|
+
- Derive the exemption rule too. If some cases legitimately have no answer, decide that
|
|
95
|
+
from a property the artifact carries, never from a list of names: the list is the
|
|
96
|
+
authored space coming back through a side door, and it absorbs the regression where
|
|
97
|
+
a case that should have an answer stops having one.
|
|
98
|
+
- Dedupe on the tuple that actually determines the outcome, and report how many
|
|
99
|
+
collapsed. A coverage count that hides its own truncation reads as more than it is.
|
|
100
|
+
- Split by cost, not by space. When the real path needs money, credentials, or a
|
|
101
|
+
network, run a cheap stand-in on every commit and the real one on demand — both from
|
|
102
|
+
the **same enumeration**, so the two can never disagree about which cases exist.
|
|
103
|
+
- Derivation moves authorship rather than removing it: the extractor and the invariants
|
|
104
|
+
are still written by hand. Give them a negative control, or the derived suite is just
|
|
105
|
+
a larger unfalsifiable one.
|
|
106
|
+
- Planting a violation to prove a control fires is a write into the working tree, and
|
|
107
|
+
the restore is not atomic with it: if the probe can time out, abort, or be
|
|
108
|
+
interrupted, a restore sitting after it never runs and the plant survives into a
|
|
109
|
+
commit. Plant in a copy where the shape allows it, and when it must be in place, snapshot
|
|
110
|
+
first and restore from the snapshot as its own step rather than trusting the probe to finish.
|
|
111
|
+
|
|
112
|
+
## When a green means nothing
|
|
113
|
+
|
|
114
|
+
A passing check and a check that never ran look identical from outside. These are the shapes that
|
|
115
|
+
produce a green with no evidence behind it:
|
|
116
|
+
|
|
117
|
+
- **The empty subject.** Any "no bad X" or "all X satisfy P" claim over an empty set is
|
|
118
|
+
vacuously true. Assert the entity-under-test set has cardinality greater than zero **before**
|
|
119
|
+
the claim, and make the gate itself refuse to report clean when it judged nothing.
|
|
120
|
+
- **The fixture that misses the guard.** For a test touching a branch you are adding or deleting,
|
|
121
|
+
confirm its inputs satisfy the live branch's entry guard. A copied fixture that fails the new
|
|
122
|
+
guard routes silently into the about-to-be-deleted dead branch and stays green after the real
|
|
123
|
+
behavior breaks.
|
|
124
|
+
- **The permissive fallback in the checker.** A `a || b` inside a gate absorbs a wrong assumption
|
|
125
|
+
and keeps passing. Checker code must assert the shape it expects and fail loud.
|
|
126
|
+
- **The suspiciously fast or empty run.** When a check goes green unexpectedly quickly, or reports
|
|
127
|
+
nothing at all, dump what it actually ran over before believing it. A harness that crashed early
|
|
128
|
+
and one that found nothing produce the same exit code.
|
|
129
|
+
- **The control that went quiet.** A negative control indexing a live list stops testing when that
|
|
130
|
+
list empties, and says nothing about it. New controls build their own subject; resolving an item
|
|
131
|
+
means re-reading the controls for ones that have gone silent.
|
|
132
|
+
|
|
133
|
+
The discipline that covers all five: after adding a check, revert the fix it guards and watch the
|
|
134
|
+
check fail. A control that survives a faithful revert was never testing the thing it names.
|
|
135
|
+
|
|
136
|
+
## Keeping E2E honest
|
|
137
|
+
|
|
138
|
+
E2E is where flakiness is mistaken for environment noise and then ignored. Keep it deterministic
|
|
139
|
+
with fixed data, resilient selectors, isolated external dependencies, and explicit waits rather
|
|
140
|
+
than sleeps. A flaky E2E is not a weaker test; it is a test whose result carries no information,
|
|
141
|
+
and a suite that people re-run until it passes has been switched off without anyone deciding to.
|
|
142
|
+
|
|
143
|
+
## Independent review, and what agreement is worth
|
|
144
|
+
|
|
145
|
+
For non-trivial designs and high-risk changes, run independent adversarial review across distinct
|
|
146
|
+
lenses — ideally on the design, before implementation, when a finding is still cheap to act on.
|
|
147
|
+
Then re-verify each finding against real code before acting on it: a reviewer reasons from what it
|
|
148
|
+
was shown, and what it was shown may be wrong.
|
|
149
|
+
|
|
150
|
+
Apply the **convergence heuristic by reviewer kind** — judge the result by reviewer kind, not by count:
|
|
151
|
+
|
|
152
|
+
- Same-kind convergence is high confidence but blind-spot-sharing. Two reviewers of the same kind
|
|
153
|
+
agreeing that something is clean is an absence of objection, not verification.
|
|
154
|
+
- Different-kind divergence is the expected signal, not a problem to resolve. Act on the union of
|
|
155
|
+
what they found rather than the intersection.
|
|
156
|
+
- An orchestrated workflow's self-reported all-green is never sufficient on its own. Re-run the
|
|
157
|
+
diff inspection and the verification suite yourself.
|
|
158
|
+
|
|
159
|
+
The cheapest way to buy real independence is a different provider; after that a different model;
|
|
160
|
+
after that strictly higher effort. A reviewer run at lower effort than the work it checks buys
|
|
161
|
+
nothing — cheaper is not another perspective.
|
|
162
|
+
|
|
163
|
+
## Reporting
|
|
164
|
+
|
|
165
|
+
Before calling the work done, state the checks that ran, their results, and any risk left
|
|
166
|
+
unverified. "Unverified" is a legitimate outcome and a useful one; silence about it is not.
|