agent-bios 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/claude/CLAUDE.md +8 -8
- package/claude/guides/cli-multi-model-workflow.md +6 -1
- package/claude/guides/coding-staged-workflow.md +12 -0
- package/claude/guides/learning-flow.md +4 -1
- package/claude/guides/llm-capability-boundary-patterns.md +8 -0
- package/claude/guides/review-request.md +13 -0
- package/claude/guides/session-distill-workflow.md +21 -9
- package/claude/guides/tooling-gotchas.md +181 -14
- package/claude/guides/verification-discipline.md +71 -3
- package/claude/hooks/tooling-gotchas-hook.py +50 -0
- package/codex/AGENTS.md +8 -8
- package/codex/guides/cli-multi-model-workflow.md +6 -1
- package/codex/guides/coding-staged-workflow.md +12 -0
- package/codex/guides/learning-flow.md +4 -1
- package/codex/guides/llm-capability-boundary-patterns.md +8 -0
- package/codex/guides/review-request.md +13 -0
- package/codex/guides/session-distill-workflow.md +21 -9
- package/codex/guides/tooling-gotchas.md +181 -14
- package/codex/guides/verification-discipline.md +71 -3
- package/install.sh +87 -5
- package/launch/agent-launch.zsh +109 -6
- package/learn/collect-learning.py +593 -62
- package/learn/redact.py +2 -1
- package/package.json +2 -2
- package/provenance.json +1 -1
|
@@ -34,11 +34,21 @@ the depth first:
|
|
|
34
34
|
over persisted real artifacts rather than re-running the whole pipeline to observe it.
|
|
35
35
|
- Probe at N=1 with the inputs precondition-checked. A single well-chosen case that reaches the
|
|
36
36
|
real path outranks a hundred that stop short of it.
|
|
37
|
+
- Bound the N=1 probe to reachability. One case settles whether a path works — never how often a
|
|
38
|
+
stochastic behavior holds. Before quoting a rate-shaped property (determinism, flake rate, an
|
|
39
|
+
A/B effect), confirm the mechanism honors your controls, establish the noise floor from runs on
|
|
40
|
+
hand, then sample enough to bound the rate against a no-treatment control on the real path. A
|
|
41
|
+
floor larger than the effect breaks the premise — record it before redesigning.
|
|
37
42
|
- Reserve the full design-review-plus-live-verification treatment for first-of-kind work and for
|
|
38
43
|
changes that move authority — who may decide, who may write, what is irreversible.
|
|
39
44
|
- Proportion assurance to the deployment context. A single-user tool operating on its owner's own
|
|
40
45
|
data does not warrant production-grade assurance, and treating it as if it did buys nothing
|
|
41
46
|
while delaying delivery. Prefer shipping.
|
|
47
|
+
- Census the population before a costed or irreversible batch. Read its current state cheaply and
|
|
48
|
+
deterministically — status distribution, version, presence of the artifacts the plan expects —
|
|
49
|
+
and compare that against the premise the batch rests on, halting to re-diagnose when the
|
|
50
|
+
distribution contradicts it. A probe validates the path; only the census validates that the
|
|
51
|
+
population is what the plan assumes. A cheap idempotent batch over a few items needs none.
|
|
42
52
|
|
|
43
53
|
The failure this prevents is not under-testing. It is spending the verification budget on the
|
|
44
54
|
cheap half of the risk and having nothing left for the part that could actually hurt.
|
|
@@ -62,10 +72,14 @@ cheapest one to write.
|
|
|
62
72
|
- Config or data: real parsers, schema checks, fixture validation, and sample transformations.
|
|
63
73
|
- Spreadsheets: static workbook checks, fixture-based output checks, cross-sheet flow checks, visual/layout checks, and real Microsoft Excel engine recalculation for formula-dependent results.
|
|
64
74
|
- Docs: links, terminology, current behavior alignment, and references to isolated historical notes.
|
|
75
|
+
- Multi-subject prose: when deliverables draw on material about different people, companies, or cases, bind every captured item to its subject and confirmation status at capture, and narrate nothing under a subject until that binding is confirmed. Cross-check subject-specific proper nouns and figures across all outputs before delivery — one subject's term under another is the signature of context bleed. Keep a figure in the same sentence as its composition, since a detached number is read at its worst.
|
|
65
76
|
- Release or distribution: after publishing to multiple independently writable channels (signed manifest, object storage, release host, embedded updater), digest-verify every referenced object against the staging original per channel — publish success and upload order are not evidence — and run the real installer/updater through its default path.
|
|
66
77
|
- A/B or on/off measurements: before accepting a null result, verify the arms actually received different treatment in the mechanism under test — a shared default or unconditional upstream step can silently apply the treatment to both arms.
|
|
78
|
+
- Multi-stage pipelines with nondeterministic stages: a final-output diff cannot attribute an effect or a regression to a stage — it conflates the change with run-to-run variance. Persist every stage's output, tabulate what each creates, may edit, and only guards, restrict the suspects to the stages that edit the content in question, and find the first stage where the intended effect disappears or the defect appears. Fix there, preferring a structural recheck over another prompt-level instruction that already failed.
|
|
79
|
+
- Before/after comparisons: pin the input to an immutable copy — a snapshot or versioned artifact — and run both arms against it, because a live artifact (a growing log, a regenerated upstream stage) drifts between runs and any diff over it, a matching one included, is evidence of nothing; when the arms are metered, restore the baseline's exact upstream inputs and re-run only the changed stage. This is input identity, not the separate unit-and-denominator basis rule.
|
|
67
80
|
- Model-behavior guardrails: verify by changed behavior, not recitation — a staged battery from named-trigger cases through disguised, deconfounded, category-wide, and single-variable framings; a clean pass means "no known defect", so re-run the battery when the model changes.
|
|
68
81
|
- Branch/version test builds against real data: explicitly separate every state sink the app touches (files, DB, OS-level stores that ignore env overrides), confirm the launch path propagates the isolation to child processes, and back up live data before the first run — a mismatched schema that drops unknown fields on write is data loss, not a no-op.
|
|
82
|
+
- Sandbox, replay, or re-adjudication runs on production-derived config: enumerate every outbound channel the stage can reach — publish, upload, notify, external write — and disable or redirect each one before the run, proving each disarm fires as you would prove a path guard; a guard on the input or target path alone leaves egress armed. Fingerprint every external destination before the run and diff it after, so an escaped write is caught by the run rather than by a recipient.
|
|
69
83
|
- Irreversible capture switches: when activation itself has unreproducible cost (a capture window that cannot be replayed), prove the downstream consumption path against existing samples before enabling — reversibility of the code path alone is not enough.
|
|
70
84
|
|
|
71
85
|
## Deriving the case space
|
|
@@ -108,6 +122,12 @@ someone typed once.
|
|
|
108
122
|
interrupted, a restore sitting after it never runs and the plant survives into a
|
|
109
123
|
commit. Plant in a copy where the shape allows it, and when it must be in place, snapshot
|
|
110
124
|
first and restore from the snapshot as its own step rather than trusting the probe to finish.
|
|
125
|
+
- Attribute a fault by a controlled contrast before choosing a remedy. Hold everything constant —
|
|
126
|
+
principal, path, input — vary only the candidate variable, and read a countable difference
|
|
127
|
+
(element dimensions, request count, status-code family). A missing error report is not
|
|
128
|
+
exoneration: channels can be suppressed, unobserved, or unwired from the mechanism at fault, so
|
|
129
|
+
"no violations logged" rules a cause out only when the contrast shows none either. Read the
|
|
130
|
+
contrast the evidence already holds before proposing a policy loosening.
|
|
111
131
|
|
|
112
132
|
## When a green means nothing
|
|
113
133
|
|
|
@@ -121,17 +141,65 @@ produce a green with no evidence behind it:
|
|
|
121
141
|
confirm its inputs satisfy the live branch's entry guard. A copied fixture that fails the new
|
|
122
142
|
guard routes silently into the about-to-be-deleted dead branch and stays green after the real
|
|
123
143
|
behavior breaks.
|
|
144
|
+
- **The fixture the producer never emits.** A test's input is evidence only when the thing that
|
|
145
|
+
produces it in production produced it. A wire fixture must be a raw response captured through
|
|
146
|
+
the same client that will parse it, never a rendered listing or a hand-written payload; a gated
|
|
147
|
+
feature's E2E must run with the gate in its production setting, on the state its real upstream
|
|
148
|
+
leaves. Replay against the live producer once, then probe every sibling built the same way.
|
|
124
149
|
- **The permissive fallback in the checker.** A `a || b` inside a gate absorbs a wrong assumption
|
|
125
150
|
and keeps passing. Checker code must assert the shape it expects and fail loud.
|
|
126
151
|
- **The suspiciously fast or empty run.** When a check goes green unexpectedly quickly, or reports
|
|
127
152
|
nothing at all, dump what it actually ran over before believing it. A harness that crashed early
|
|
128
153
|
and one that found nothing produce the same exit code.
|
|
154
|
+
- **The control that failed by crashing.** A negative control is evidence only when it fails
|
|
155
|
+
through the assertion it names: a traceback and a caught violation share an exit code, and an
|
|
156
|
+
early crash can pre-empt every control after it. Treat each traceback in a control run as a
|
|
157
|
+
defect until the run reports one named failure per planted violation. Run any unattended gate
|
|
158
|
+
with stdin closed, so a path reaching a prompt fails at once instead of hanging.
|
|
129
159
|
- **The control that went quiet.** A negative control indexing a live list stops testing when that
|
|
130
160
|
list empties, and says nothing about it. New controls build their own subject; resolving an item
|
|
131
161
|
means re-reading the controls for ones that have gone silent.
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
162
|
+
- **The silent log.** Absence of activity is not evidence of non-use. Before retiring an
|
|
163
|
+
identity, key, or endpoint on a quiet log, show that the queried field is the one recording the
|
|
164
|
+
subject — delegated actions are attributed to the caller, with the target in another field, so
|
|
165
|
+
the wrong field returns clean silence — and that no live binding still references it, since a
|
|
166
|
+
reference proves use while emitting no traffic. Only a demonstrated "this use would have been
|
|
167
|
+
logged" evidences non-use.
|
|
168
|
+
- **The population that shrank.** A gate scanning an explicit list of files sees only "listed but
|
|
169
|
+
empty": a subject that migrates to an unlisted surface leaves the scanned set smaller yet
|
|
170
|
+
non-empty, so the empty-subject guard never fires while coverage erodes. Retarget the gate in
|
|
171
|
+
the same change that moves the code, and pin a floor so any decrease fails loudly — proved by
|
|
172
|
+
moving one subject out. A floor is a ratchet: never lower it to pass a run.
|
|
173
|
+
- **The mutant that never ran.** A mutation verdict counts only if the mutant is valid: it
|
|
174
|
+
compiled, sits on a path the exercised test traverses, and changes the guarded behavior, not
|
|
175
|
+
healed downstream or coinciding with a default. The runner must report build failure,
|
|
176
|
+
unreachable, and equivalent distinctly from KILLED and SURVIVED, and halt on a moved anchor.
|
|
177
|
+
More tests red than the mutation should touch indicts it; classify a survivor (rebuild,
|
|
178
|
+
discard, genuine gap) before writing a test.
|
|
179
|
+
- **The probe that measured the original.** A copied script that derives its root or targets from
|
|
180
|
+
its own location (`$0`, `BASH_SOURCE`, a `cd` to its parent) scans the original tree, not the
|
|
181
|
+
copy, so its verdict says nothing about the mutation you planted. Pin the subject in the copy,
|
|
182
|
+
copy the whole tree, or plant in place with a snapshot-restore. The tell is a result identical
|
|
183
|
+
to the unmutated run; a script taking its subject as an argument is safe to copy.
|
|
184
|
+
|
|
185
|
+
The discipline that covers every shape above: after adding a check, revert the fix it guards and
|
|
186
|
+
watch the check fail. A control that survives a faithful revert was never testing the thing it
|
|
187
|
+
names. Prove the plant landed: assert the altered input differs from the original and that the
|
|
188
|
+
control's cases reach the mutated branch before asserting rejection. Construct corruption
|
|
189
|
+
deterministically: scanning for a flip site yields no-op mutations reporting a rejection nothing
|
|
190
|
+
exercised.
|
|
191
|
+
|
|
192
|
+
## Before blaming code for a metric change
|
|
193
|
+
|
|
194
|
+
When a live metric collapses or crosses a pre-declared threshold, localize the change in time
|
|
195
|
+
before diagnosing the feature: confirm every inbound source is alive, compare only
|
|
196
|
+
contemporaneous cohorts, never re-processed rows, then bracket the transition to the finest unit
|
|
197
|
+
available and read the deploy log around that instant.
|
|
198
|
+
|
|
199
|
+
A change landing seconds from the last-good point is the prime suspect; a step in a window with
|
|
200
|
+
no deploys is an input-population shift, fixed by scoping the measured population, not the model
|
|
201
|
+
or the code. Declare the threshold before looking, and confirm a failure from the fleet's vantage
|
|
202
|
+
rather than your own — this does not replace fixing the comparison basis, which comes first.
|
|
135
203
|
|
|
136
204
|
## Keeping E2E honest
|
|
137
205
|
|
|
@@ -53,6 +53,31 @@ RULES = [
|
|
|
53
53
|
"Reverting a path discards ALL uncommitted edits in that file, not just the one "
|
|
54
54
|
f"you planted — check `git diff <path>` first, or restore from a copy ({GUIDE}).",
|
|
55
55
|
"Reverting a path is not undoing your edit"),
|
|
56
|
+
# A sweeping write treats every delta in the tree as the caller's own. Named paths
|
|
57
|
+
# (`git add src/x.py`) are exempt: naming is the attribution the rule asks for.
|
|
58
|
+
("shared-tree-sweep",
|
|
59
|
+
# The commit flag must be its own token: a path like /x/-agent/msg.txt after -F carries
|
|
60
|
+
# "-a" too, and the first version of this rule fired on exactly that.
|
|
61
|
+
re.compile(r"\bgit\s+(add\s+(-A|--all|\.)(\s|$)|commit\s+(?:[^|;&]*\s)?(?:-[a-zA-Z]*a[a-zA-Z]*|--all)(?=\s|$)|stash\b(?!\s+(pop|apply|list|show))|clean\b|reset\s+--hard)"),
|
|
62
|
+
"A shared tree may hold another operator's work — attribute every delta you did not "
|
|
63
|
+
f"make (git status, reflog, live sessions) before sweeping it; add by name ({GUIDE}).",
|
|
64
|
+
"A shared tree holds other operators' work"),
|
|
65
|
+
("job-logs-by-execution",
|
|
66
|
+
re.compile(r"\bgcloud\s+(?:alpha\s+|beta\s+)?(?:logging\s+read|run\s+jobs)\b"),
|
|
67
|
+
"Job-level logs retain earlier executions (even a job deleted and recreated under the "
|
|
68
|
+
f"same name) — scope the read to the execution id you launched ({GUIDE}).",
|
|
69
|
+
"Job logs outlive the execution"),
|
|
70
|
+
("name-substring-liveness",
|
|
71
|
+
re.compile(r"\bpgrep\s+(?:-\w+\s+)*-\w*f|\bps\b[^|\n]*\|[^|\n]*\bgrep\b"),
|
|
72
|
+
"A name-substring process lookup matches the querying shell's own argv and unrelated "
|
|
73
|
+
f"same-named processes — confirm liveness by PID, handle, or output growth ({GUIDE}).",
|
|
74
|
+
"Name-substring lookup is not a liveness check"),
|
|
75
|
+
# Not a search for the word: a grep/rg over sources mentions revoke without revoking.
|
|
76
|
+
("revoke-grant-blast-radius",
|
|
77
|
+
re.compile(r"^(?!.*\b(?:rg|grep|ag|git\s+log)\b).*\brevoke\b"),
|
|
78
|
+
"A revoke is grant-wide: everything issued under a shared client id, including the "
|
|
79
|
+
f"user's live sessions, goes with it — confirm the scope before revoking ({GUIDE}).",
|
|
80
|
+
"A revoke is grant-wide, not token-wide"),
|
|
56
81
|
("grep-binary-heuristic",
|
|
57
82
|
# Applied per extracted STAGE by matches(), not to the raw line. The stage's
|
|
58
83
|
# COMMAND WORD must be grep — after optional reserved words (`if ! grep -q`
|
|
@@ -265,8 +290,33 @@ def self_test() -> int:
|
|
|
265
290
|
"metachar-inline-arg": 'codex exec "$(cat packet.md)"',
|
|
266
291
|
"pipe-exit-masking": "make build | tail -1; echo $?",
|
|
267
292
|
"git-checkout-path": "git checkout src/thing.py",
|
|
293
|
+
"shared-tree-sweep": "git add -A && git commit -m x",
|
|
294
|
+
"job-logs-by-execution": "gcloud logging read 'resource.labels.job_name=probe' --limit 50",
|
|
295
|
+
"name-substring-liveness": "pgrep -f codex",
|
|
296
|
+
"revoke-grant-blast-radius": "gcloud auth revoke probe@example.com",
|
|
268
297
|
"grep-binary-heuristic": "grep needle haystack.md",
|
|
269
298
|
}
|
|
299
|
+
# Naming the path IS the attribution the sweep rule asks for, so a named add, a
|
|
300
|
+
# plain commit, and stash pop must not draw the reminder — or it fires on every commit.
|
|
301
|
+
for quiet_cmd in ("git add src/thing.py", "git commit -m 'fix'", "git stash pop", "git add -p",
|
|
302
|
+
"git commit -F /tmp/claude-501/-Users-someone-Documents-agent-bios/scratch/msg.txt"):
|
|
303
|
+
if "shared-tree-sweep" in [h for h, _ in matches(quiet_cmd, limit=None)]:
|
|
304
|
+
problems.append(f"shared-tree-sweep: fired on a non-sweeping command ({quiet_cmd!r})")
|
|
305
|
+
for sweep_cmd in ("git add .", "git commit -am 'wip'", "git commit --all -m x", "git stash", "git reset --hard HEAD"):
|
|
306
|
+
if "shared-tree-sweep" not in [h for h, _ in matches(sweep_cmd, limit=None)]:
|
|
307
|
+
problems.append(f"shared-tree-sweep: did not fire on a sweeping command ({sweep_cmd!r})")
|
|
308
|
+
for quiet_cmd in ("gcloud run services list", "docker logs probe", "kill -0 12345",
|
|
309
|
+
"grep -rn pgrep claude/guides/", "ls | grep foo", "rg revoke src/",
|
|
310
|
+
"git log --grep revoke"):
|
|
311
|
+
hit = [h for h, _ in matches(quiet_cmd, limit=None)]
|
|
312
|
+
for rule in ("job-logs-by-execution", "name-substring-liveness", "revoke-grant-blast-radius"):
|
|
313
|
+
if rule in hit:
|
|
314
|
+
problems.append(f"{rule}: fired on a non-matching command ({quiet_cmd!r})")
|
|
315
|
+
for rule, cmd in (("job-logs-by-execution", "gcloud beta run jobs executions list --job probe"),
|
|
316
|
+
("name-substring-liveness", "ps aux | grep codex"),
|
|
317
|
+
("revoke-grant-blast-radius", "vault token revoke s.f3b9c2")):
|
|
318
|
+
if rule not in [h for h, _ in matches(cmd, limit=None)]:
|
|
319
|
+
problems.append(f"{rule}: did not fire on ({cmd!r})")
|
|
270
320
|
# A pipeline-stage grep must reach the rule too — the fixture alone exercises only
|
|
271
321
|
# command-start grep, and the reminder is most needed mid-pipeline.
|
|
272
322
|
if "grep-binary-heuristic" not in [h for h, _ in matches("cat payload | grep needle", limit=None)]:
|
package/codex/AGENTS.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## Global Preferences
|
|
4
4
|
|
|
5
|
-
- Prefer concise Korean responses with polite speech unless the user asks otherwise.
|
|
5
|
+
- Prefer concise Korean responses with polite speech unless the user asks otherwise. (private)
|
|
6
6
|
- Keep file changes within the requested scope.
|
|
7
7
|
|
|
8
8
|
## Problem Solving
|
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
- First identify the goal, scope, ambiguities, and likely completion condition.
|
|
11
11
|
- Resolve ambiguity from context when safe; ask only when ambiguity blocks progress or creates risky outcomes.
|
|
12
12
|
- For simple requests, choose the most direct low-risk method and proceed.
|
|
13
|
-
- For non-trivial requests, compare 2-4 methods by goal fit, time, cost, risk, benefit, and "done when".
|
|
13
|
+
- For non-trivial requests, compare 2-4 methods by goal fit, time, cost, risk, benefit, and "done when", and portability — take a host-, model-, or tool-specific mechanism (hook, skill, host-owned directory) only after a portable route is shown absent and its per-host cost is judged worth it.
|
|
14
14
|
- Mark one default method. If the user is silent and the default is safe, proceed with it.
|
|
15
15
|
- Execute the chosen method accurately and stay within scope.
|
|
16
16
|
- Return to understanding if a discovery breaks the user's premise.
|
|
@@ -21,7 +21,7 @@
|
|
|
21
21
|
|
|
22
22
|
## Decision Framing
|
|
23
23
|
|
|
24
|
-
- Ask decision questions in outcome terms, not jargon terms.
|
|
24
|
+
- Ask decision questions in outcome terms, not jargon terms: before ending a turn on a decision request, check that it gives the situation in one plain sentence, what changes for the user under each option, and a default — and where a structured question channel exists, route the ask through it so its fields force that shape.
|
|
25
25
|
- When the user may not know the domain, explain choices by resulting behavior, tradeoffs, time, cost, risk, reversibility, and recommended default.
|
|
26
26
|
- Present 2-4 meaningful options. Ask about implementation details only when they directly affect the decision.
|
|
27
27
|
- For each option, state what changes for the user or product, what it costs, what risk it carries, and when it is the right choice.
|
|
@@ -29,7 +29,7 @@
|
|
|
29
29
|
- Ask for the user's goal or constraint when that determines the answer; otherwise choose the safest default and proceed.
|
|
30
30
|
- Evaluate user suggestions for goal fit, risk, complexity, and verification before turning them into implementation plans; if a suggestion does not fit the user's goal, say so clearly and recommend a better path.
|
|
31
31
|
- Distinguish implementation feasibility from recommendation.
|
|
32
|
-
- Do not default to a restrictive lens (security, masking, capability limits) when the system's purpose is sharing or utilization; confirm the purpose framing first, and restrict only on concrete, named risk.
|
|
32
|
+
- Do not default to a restrictive lens (security, masking, capability limits) when the system's purpose is sharing or utilization; confirm the purpose framing first, and restrict only on concrete, named risk — and size every control (gate, cap, rule, review lens, success criterion, clarifying question) to that risk: a target is a direction, not an absolute, and prefer a warning plus a recovery path over a prohibition.
|
|
33
33
|
- Treat user suggestions, inherited premises, prior diagnoses, handoff and design claims, reviewer findings, and your own earlier conclusions as hypotheses, not facts; re-derive each load-bearing claim from real code or data before building on it, and record a dated correction in the source doc or memory when a finding overturns it.
|
|
34
34
|
|
|
35
35
|
## LLM And Capability Boundary
|
|
@@ -53,7 +53,7 @@
|
|
|
53
53
|
## Concept Economy
|
|
54
54
|
|
|
55
55
|
- When adding, changing, renaming, splitting, or exposing anything lasting or shared — a feature, entity, type, field, config key, CLI flag, enum value, failure kind, artifact, or documentation term — read and use `${CODEX_HOME:-$HOME/.codex}/guides/concept-economy.md` as a scoped extension of this section.
|
|
56
|
-
- Before fixing a review finding or test failure, classify the fix as reducing, preserving, or increasing the active concept surface.
|
|
56
|
+
- Before fixing a review finding or test failure, name its cause — a finding is a symptom — then classify the fix as reducing, preserving, or increasing the active concept surface, and fix the cause completely now: a scope-minimal patch that leaves the cause in place is not a fix.
|
|
57
57
|
|
|
58
58
|
## Coding Guidelines
|
|
59
59
|
|
|
@@ -61,7 +61,7 @@
|
|
|
61
61
|
- For development work, read and use `${CODEX_HOME:-$HOME/.codex}/guides/coding-staged-workflow.md` as a scoped extension of these Coding Guidelines — a change too narrow to need it is what its lightweight path decides, not a reason to skip the read.
|
|
62
62
|
- For mock, fixture, fake, stub, simulated-provider, or test-realization design, read and use `${CODEX_HOME:-$HOME/.codex}/guides/mock-realization-boundary.md` as a scoped extension of these Coding Guidelines.
|
|
63
63
|
- Own the full lifecycle of what you create — spawned processes and handles through teardown, artifacts out of tool-managed temp locations into a durable home — and keep differently-owned state separate: never colocate deploy-managed and user-owned data in one overwrite-managed file.
|
|
64
|
-
- Land risky or behavior-changing work behind a default-off path that preserves current behavior when off (proven by diff) and is enabled by an explicit opt-in, so the change stays reversible and the on/off difference is isolated. When a request would weaken a security or authority posture — removing or loosening an authentication/authorization check or access scope, or lowering a protective value such as session/token lifetime, password/crypto strength, rate limit, lockout threshold, or audit retention — treat it as a decision, not a rote edit, even when it is a one-line change and nothing in the code labels the value as security-relevant: state the consequence and at least one safer path to the real goal, and do not apply the weakening in the same turn — proceed only after the user confirms they accept the tradeoff.
|
|
64
|
+
- Land risky or behavior-changing work behind a default-off path that preserves current behavior when off (proven by diff) and is enabled by an explicit opt-in, so the change stays reversible and the on/off difference is isolated — the switch lands a fix reversibly and never substitutes for one. When a request would weaken a security or authority posture — removing or loosening an authentication/authorization check or access scope, or lowering a protective value such as session/token lifetime, password/crypto strength, rate limit, lockout threshold, or audit retention — treat it as a decision, not a rote edit, even when it is a one-line change and nothing in the code labels the value as security-relevant: state the consequence and at least one safer path to the real goal, and do not apply the weakening in the same turn — proceed only after the user confirms they accept the tradeoff.
|
|
65
65
|
|
|
66
66
|
## Verification Discipline
|
|
67
67
|
|
|
@@ -77,7 +77,7 @@
|
|
|
77
77
|
- For concrete shell/CLI traps — pipe exit codes, output rendering, git range/pull semantics, config and managed-service pitfalls — read and use `${CODEX_HOME:-$HOME/.codex}/guides/tooling-gotchas.md` as a scoped extension of this section.
|
|
78
78
|
- Ambient state — the active shell, cloud CLI project/context, command-name resolution, 'latest'-style pointers, version-bearing paths — drifts silently; where an outcome depends on it, pin it explicitly (a pinned interpreter, --project/--context flags, exact handles, resolved paths) instead of trusting the environment.
|
|
79
79
|
- Before relying on any model id, tool flag, API capability, dependency version, or runtime constraint, confirm it empirically against the live or installed artifact (a minimal probe, the binary's registered options, the installed package version) rather than docs, memory, or a version string.
|
|
80
|
-
- Scope destructive actions (kill, rm, force-push, reset --hard) to targets you own,
|
|
80
|
+
- Scope destructive actions (kill, rm, force-push, reset --hard) to targets you own, by PID, path, or ancestry — never a broad command-line substring or blanket match — and diagnose actual state before any irreversible git, remote, or process operation; snapshot the last good state before any in-place resume or overwrite of a completed run, and gate irreversible identity-tied actions (revoke, delete, grant, consent, account-bound creation) on a live identity check — never auto-open a browser for a non-default identity (hand the operator the URL).
|
|
81
81
|
- Never accept secrets through transcript- or history-logged channels.
|
|
82
82
|
- When a secret must be supplied, provide a gitignored env slot, read the value only from the environment, verify its presence and format without echoing it, and advise rotating anything already pasted; assume a resource-creating call may echo the secret back in its success output — suppress or discard the response body, and treat an echoed secret as pasted (rotate).
|
|
83
83
|
- Treat a coarse runtime signal — a failure label, a `ps`/process-inspection result, idle CPU with no output — as a hypothesis, and confirm the cause against the authoritative low-level evidence the mechanism emits before attributing blame or intervening: read the raw provider/skill log payload (e.g. `input_tokens:0` proves a pre-dispatch rejection that exonerates your content and your change), and confirm a config/env toggle reached a subprocess via a cheap artifact the gated branch emits rather than an unreliable `ps` env read. A multi-minute LLM or subprocess call at ~0% CPU with an output gap is the normal signature of I/O wait, not a hang — check process state and the call trace's in-flight duration before acting, so you do not abort healthy long-running work.
|
|
@@ -85,7 +85,7 @@
|
|
|
85
85
|
## Multi-Model Workflow
|
|
86
86
|
|
|
87
87
|
- Codex-only standing authorization: on root/main local tasks, ordinary subagent dispatch is authorized when the `When To Spawn` gates fire. Explicit no-fan-out wins. Delegated agents may re-delegate only when their role allows. This grants no destructive, remote, credential, install, OAuth, push, live-network-expanding, or broader-sandbox authority.
|
|
88
|
-
- Standing spawn policy: check the spawn gates at every work-unit boundary — judgment latitude applies inside a gate, never to whether the gates are checked. Independence: verifying or reviewing your own work always spawns. Parallelism: two or more independent items spawn in parallel — SWEEP when each item applies one explicit rule and returns ambiguity as an exception, else WORKHORSE. Residual context: work whose log dwarfs the conclusion the main needs spawns with a bounded report contract. Escalation: an irreversible or authority-changing action ahead, two failed attempts, or two persisting design alternatives spawns a bounded FRONTIER judgment with a blind packet (evidence, constraints, rubric, neutral alternatives — never your draft conclusion) and a pre-noted change condition. Specifiability/de-minimis: work needing your live context, or whose verification would repeat the reasoning, or whose packet outweighs the work, stays inline.
|
|
88
|
+
- Standing spawn policy: check the spawn gates at every work-unit boundary — judgment latitude applies inside a gate, never to whether the gates are checked. Independence: verifying or reviewing your own work always spawns, and you raise it yourself — before presenting a load-bearing conclusion or taking an irreversible step, propose the cross-check unprompted; the user should never have to ask for it. Parallelism: two or more independent items spawn in parallel — SWEEP when each item applies one explicit rule and returns ambiguity as an exception, else WORKHORSE. Residual context: work whose log dwarfs the conclusion the main needs spawns with a bounded report contract. Escalation: an irreversible or authority-changing action ahead, two failed attempts, or two persisting design alternatives spawns a bounded FRONTIER judgment with a blind packet (evidence, constraints, rubric, neutral alternatives — never your draft conclusion) and a pre-noted change condition. Specifiability/de-minimis: work needing your live context, or whose verification would repeat the reasoning, or whose packet outweighs the work, stays inline.
|
|
89
89
|
- Down-spawns carry a machine-checkable done-when on decision-complete work with staged output (no external irreversible actions) and briefing-plus-verifying clearly cheaper than doing. Record one line per gate decision — `SpawnGate: <gate> <tier> spawn|inline — <why>` — and for FRONTIER record the disposition afterward (what changed, or why nothing did). A launch contract's `Delegation=off` lifts the spawn obligation, not the records; explicit user no-fan-out always wins.
|
|
90
90
|
- For work spanning multiple models or CLI agents, context resets and handoffs, unattended LLM batches (including orchestrated subagent fleets), or parallel worktree branches, read and use `${CODEX_HOME:-$HOME/.codex}/guides/cli-multi-model-workflow.md` as a scoped extension of this section.
|
|
91
91
|
- For composing a prompt, packet, or tool description aimed at a specific model family — including cross-family review dispatch, porting a prompt written for an older model, or choosing a reasoning-effort level for a model family — read and use `${CODEX_HOME:-$HOME/.codex}/guides/gpt-prompting.md` for gpt-family targets and `${CODEX_HOME:-$HOME/.codex}/guides/claude-prompting.md` for claude-family targets as scoped extensions of this section.
|
|
@@ -106,7 +106,7 @@ Instruction/config reach is per invocation. A rule in AGENTS.md cannot bind a he
|
|
|
106
106
|
- Run deterministic gates before LLM review. Funnel SWEEP finders → WORKHORSE judgments → FRONTIER triage/verdicts.
|
|
107
107
|
- On family collapse, record the downgrade and label clean verdicts PROPOSED until diversity is restored.
|
|
108
108
|
- A silent/dead lens is incomplete, never clean. Confirm liveness from usage/error/report evidence; rerun, swap provider, or report PROPOSED.
|
|
109
|
-
- Kind labels do not guarantee distinct backends: wrappers and rate-limit fallbacks can silently route two "different-kind" verifiers to the same model/provider. Before trusting diversity on a high-stakes verdict, confirm each verifier's actual backing model from live process or usage evidence; on collapse, treat the pair as one kind and label PROPOSED.
|
|
109
|
+
- Kind labels do not guarantee distinct backends: wrappers and rate-limit fallbacks can silently route two "different-kind" verifiers to the same model/provider. Before trusting diversity on a high-stakes verdict, confirm each verifier's actual backing model from live process or usage evidence; on collapse, treat the pair as one kind and label PROPOSED. Runners recording what ran need the same read: identity taken from the target at execution time, never a runner-side literal, and asserted equal to what was requested — fallback seats pass existence checks.
|
|
110
110
|
|
|
111
111
|
### Review Independence
|
|
112
112
|
|
|
@@ -124,6 +124,7 @@ How much independence a review actually bought, as an ordinal grade per reviewer
|
|
|
124
124
|
- The floor still requires **at least two distinct perspectives**; one pass on the main's own seat is self-review with extra steps.
|
|
125
125
|
- Multiple ready methods are **coverage, not diversity**. Distinct labels do not prove the perspectives differed.
|
|
126
126
|
- **Achieved is not available.** What can be projected before a review runs is `projected`; a clean verdict without a receipt evidencing a fresh dispatch, the declared packet, a non-empty result and the exact seat is `PROPOSED`, never ACHIEVED. A model echo is not a receipt.
|
|
127
|
+
- **Evidence access is its own axis.** When every reviewer saw only the blind packet, convergence — even across providers — is evidence about the packet's framing, omissions included. Before adopting a converged verdict resting on a code seat, a measured fact, or a constraint list, route one seat with live read access to falsify those facts: a lone dissent citing a real constraint outweighs a blind majority, and the missing fact returns to the packet. It sits beside the ladder, not on it.
|
|
127
128
|
|
|
128
129
|
## Dual-Provider Design Drafts
|
|
129
130
|
|
|
@@ -136,11 +137,14 @@ How much independence a review actually bought, as an ordinal grade per reviewer
|
|
|
136
137
|
- The parent owns per-item completion and a **code-level circuit breaker**. For dispatchers you do not control, verify equivalent protection or attend the run.
|
|
137
138
|
- Default breaker: halt after 3 consecutive cross-item provider limit/auth/transport failures after bounded backoff. Persist undone items and alert or swap provider.
|
|
138
139
|
- Item-specific failures are poison items: cap at 2–3 attempts, then dead-letter them as complete-with-failure. Resume only unfinished/invalid items; whole-batch reruns require cheap idempotence.
|
|
140
|
+
- An enumeration run is done only when its collected count is asserted against the source's own reported total for the same filter. Classify retriable failures by class — any server-side transient — rather than an enumerated code list, since an omitted code drops items silently; persist which batches failed and reconcile them before declaring completion; and treat a mismatch, or a total from a differently scoped population, as a defect rather than a footnote. A declared partial or sampled scope is outside this.
|
|
139
141
|
- Persist per-item outcome, token, and cost records for recalibration.
|
|
142
|
+
- Before releasing a metered batch past its first item, use that item to probe the batch machinery, not the item logic: run it end to end through the real runner to its side effect, then confirm every value the later analysis depends on — treatment knob, run identity, cost — reached the persisted record through the expected channel. On the first failures read raw run logs, not the runner's status classifier, which infers causes from missing outputs. A cheap idempotent batch needs no gate.
|
|
140
143
|
|
|
141
144
|
## Halt And Resume
|
|
142
145
|
|
|
143
146
|
- Resume-first from artifacts that parse, pass schema, and match their recorded source/config/HEAD fingerprint; unverifiable means invalid.
|
|
147
|
+
- A cache-hit or fingerprint predicate must cover every value that shapes the artifact's content — the upstream input's content identity, caps, templates, model ids, config — never existence, mtime, or size alone; when adding a new output-shaping value, inspect the key's pre-image in the same change and assert the key moves when the value moves. Before re-running because an upstream input changed, invalidate intermediates whose predicate omits that input's identity: a regenerate over existence-keyed caches re-derives from the old input.
|
|
144
148
|
- Resubmit one invalid unit unless failures are broadly correlated, which is structural and halts the run.
|
|
145
149
|
- Treat halt→continue as normal operation.
|
|
146
150
|
- Treat tool-managed temp/cache output locations as ephemeral — they are garbage-collected on the tool's own schedule. Copy any artifact a pending or handed-off decision depends on into a project-owned durable path before relying on it later.
|
|
@@ -150,6 +154,7 @@ How much independence a review actually bought, as an ordinal grade per reviewer
|
|
|
150
154
|
|
|
151
155
|
- Sessions bind to their starting directory. Use the CLI's native relocation/resume mechanism; never copy transcript files.
|
|
152
156
|
- For a new worktree, relocate natively or write a handoff and start fresh. Re-integrate branches serially and re-verify after each merge.
|
|
157
|
+
- A conflict-free merge with a green build is evidence about text, not placement. When the base side restructured the surrounding code — regrouped sections, split modules, new per-variant containers — locate each merged addition in the new structure and confirm its scope still matches its container's: a global setting must not sit inside a variant-specific container, and no duplicate or orphaned copy may remain. A merge onto an unchanged layout needs only the ordinary green-state check.
|
|
153
158
|
- Mark superseded worktrees/handoffs dead so later resume cannot select them.
|
|
154
159
|
- After resume/clear/relocation, verify pwd, branch, and HEAD against the pinned handoff before acting.
|
|
155
160
|
- Attribute a parallel session's action (commit, branch, resource) by execution evidence in that session's own transcript, never by token mentions — shared handoff/memory files inject the same tokens into every session's context.
|
|
@@ -75,6 +75,18 @@ patching downstream: compensating code keeps accumulating around bad inputs, and
|
|
|
75
75
|
another instance of the same defect. The first says go upstream to where the value is produced.
|
|
76
76
|
The second says the instances are a class — single-source the value and fix the class, because
|
|
77
77
|
patching them one at a time is a queue that refills.
|
|
78
|
+
**Supplying a missing shared dependency wakes every consumer, not just the one you are fixing.** When
|
|
79
|
+
a repair supplies a value many paths read and that was absent — a secret, a packaged file —
|
|
80
|
+
enumerate those consumers and say what each starts doing: metered calls, external writes,
|
|
81
|
+
user-visible output. Where that onset exceeds the feature under repair, hand the list to the owner
|
|
82
|
+
as a decision, not a line in the fix. Consumers that are all read-only and free need no gate.
|
|
83
|
+
|
|
84
|
+
**Measure a flip before you design its activation.** When a version bump, default change, or
|
|
85
|
+
severity re-mapping is coming, flip it, run the full suite, classify every failure (cascade,
|
|
86
|
+
pinned control, true detection, real regression), and restore — that count is the activation's
|
|
87
|
+
blast radius. Re-mapping a level obliges enumerating every reader of that field, since one level
|
|
88
|
+
commonly gates shipping, repair, retry, and display at once. A deferred defect is pinned as a
|
|
89
|
+
strict expected failure, never a silent pass.
|
|
78
90
|
|
|
79
91
|
## Review Loop
|
|
80
92
|
|
|
@@ -103,4 +103,7 @@ payload — do not work around the validation.
|
|
|
103
103
|
- Single-session capture only; cross-session mining is `distill!` (curator).
|
|
104
104
|
- Mechanization (hook/gate/enforcement) is deferred to curation — record intent, don't build it.
|
|
105
105
|
- Type-G principle manufacture is curator-only.
|
|
106
|
-
- Transport (upload to the org)
|
|
106
|
+
- Transport (upload to the org) is best-effort after the local write: it runs only
|
|
107
|
+
when the `~/.config/agent-bios/{ingest-url,token}` slot is set, which is the
|
|
108
|
+
only source there is. A default install sets nothing, so nothing leaves the
|
|
109
|
+
machine; an org fills the slot through its own wrapper.
|
|
@@ -314,6 +314,14 @@ Retries are safe for pure generation and validation. They are not automatically
|
|
|
314
314
|
safe for external side effects. Use idempotency keys, locks, duplicate detection,
|
|
315
315
|
or compensation plans where needed.
|
|
316
316
|
|
|
317
|
+
A stage that can re-run on the same record must not read the field it writes.
|
|
318
|
+
On a re-run (reclassification, backfill, retry) an input field that is also
|
|
319
|
+
its output feeds the model its prior answer, and the value drifts from the
|
|
320
|
+
source silently. Keep the captured original immutable, write derived values to
|
|
321
|
+
their own field, and treat `original ?? current` as migration, not design.
|
|
322
|
+
Audit its siblings: one harmless only because its selection rule skips
|
|
323
|
+
overwritten rows is a latent instance.
|
|
324
|
+
|
|
317
325
|
## Single Source Of Truth And Schema Evolution
|
|
318
326
|
|
|
319
327
|
Hybrid enforcement creates drift risk. A single constraint can appear in prompt
|
|
@@ -122,6 +122,12 @@ design; do not treat missing implementation as a defect." A session given exactl
|
|
|
122
122
|
that instruction returned zero findings and proved it had looked, anchored on
|
|
123
123
|
both sides of the comparison. A session not given it filed unimplemented code as
|
|
124
124
|
blockers, and the user caught it manually.
|
|
125
|
+
The target also has a revision. Name the commit or content hash the packet
|
|
126
|
+
was dispatched on, and hold the artifact still until every reviewer on
|
|
127
|
+
that revision has returned. When a fix must land while a lens is still in
|
|
128
|
+
flight, map the returned findings against the pinned revision before
|
|
129
|
+
counting them: a finding whose anchor text no longer exists is stale,
|
|
130
|
+
closed by that mapping rather than re-fixed, and never tallied as open.
|
|
125
131
|
|
|
126
132
|
## Bundle the consumer, not just the artifact
|
|
127
133
|
|
|
@@ -226,6 +232,13 @@ Two consequences for anyone consuming a review:
|
|
|
226
232
|
count was scoped to the material section, at which point 16% of the same
|
|
227
233
|
findings turned out to have reached the reader explicitly flagged
|
|
228
234
|
non-material. Read the verdict field, not the mention.
|
|
235
|
+
- Count the emitted item list against every total the harness reports —
|
|
236
|
+
findings count, verdict tally, per-item decision log — before triaging.
|
|
237
|
+
Zero findings is only the extreme case: any shortfall means items were
|
|
238
|
+
dropped in aggregation, and the dropped set is not random, since a merge
|
|
239
|
+
or filter tends to lose a whole class. Recover the difference from the
|
|
240
|
+
raw per-item record and triage the union; with no raw record the
|
|
241
|
+
deliverable is incomplete, not clean.
|
|
229
242
|
|
|
230
243
|
## Trust an empty result only when it cites what it checked
|
|
231
244
|
|
|
@@ -49,15 +49,27 @@ Run in order; each stage reads the previous stage's `out/`:
|
|
|
49
49
|
Claude sidechain/sdk-cli/agentId).
|
|
50
50
|
2. `digest.py` — one secret-redacted digest per session with deterministic
|
|
51
51
|
6-criteria signals. Screen ALL digests; triage orders, never drops.
|
|
52
|
-
3.
|
|
53
|
-
|
|
54
|
-
`
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
52
|
+
3. `batch.py` — the baseline blob (`claude/CLAUDE.md` + every guide, the
|
|
53
|
+
repo's canonical corpus) and per-provider batches; writes
|
|
54
|
+
`out/batch_index.json`, which the screeners take as their `args`.
|
|
55
|
+
4. Provider-affine screening against that baseline: `screen-claude.js`
|
|
56
|
+
(Claude sessions; a Workflow script — pass the index as `args`, one
|
|
57
|
+
WORKHORSE screener per batch) and `screen-codex.py` (Codex sessions;
|
|
58
|
+
one hermetic read-only `codex exec` per batch, packet on stdin). Novelty
|
|
59
|
+
is judged against real baseline text, not memory. Then `collect.py`
|
|
60
|
+
unions the two outputs into `out/candidates-all.json` and fails when a
|
|
61
|
+
provider's screened set is smaller than its batch.
|
|
62
|
+
5. `consolidate.js` (Workflow; `args` = baseline, candidates path, count,
|
|
63
|
+
and the ledger's `{id, lesson}` list) — dedup + independent novelty
|
|
64
|
+
verification, then a match pass naming which survivor recurs an
|
|
65
|
+
existing ledger entry. Rank by strength (recurrence × materiality),
|
|
66
|
+
never by self-reported confidence. Save its return value as
|
|
67
|
+
`out/consolidated.json`.
|
|
68
|
+
6. `bundle_final.py` — tiered bundle. `merge-ledger.py --window-end <date>`
|
|
69
|
+
(dry-run; `--apply` writes) merges survivors into `ledger.json`: a
|
|
70
|
+
recurrence gains the window's sessions under `recurrence`, a new lesson
|
|
71
|
+
becomes a `candidate` entry — so recurrence accumulates across windows
|
|
72
|
+
and incubated items promote when they re-occur.
|
|
61
73
|
|
|
62
74
|
## Stage 2 — Review with the user
|
|
63
75
|
|