@chrono-meta/fh-gate 1.4.48 → 1.4.49
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +64 -0
- package/README.md +1 -1
- package/knowledge/shared/harness-core/field_verdict_crossfamily_gate.md +143 -0
- package/knowledge/shared/harness-core/multi_model_sidecar_strategy.md +29 -0
- package/package.json +1 -1
- package/plugins/fh-meta/skills/memory-hygiene/SKILL.md +10 -1
- package/plugins/fh-meta/skills/steel-quench/SKILL.md +34 -0
package/CLAUDE.md
CHANGED
|
@@ -285,6 +285,69 @@ unknown) and surface **one line** — then proceed, never block:
|
|
|
285
285
|
inviolable; a pin is not a cap — tier-floor resolution §Floor governance) · field-project operation
|
|
286
286
|
sessions (no FH asset modification) never see this notice — the Sonnet default stays friction-free.
|
|
287
287
|
|
|
288
|
+
## Field-Harness Load-Bearing Change Gate (cross-family, pre-merge)
|
|
289
|
+
|
|
290
|
+
The 4-axis gate above fires on **FH asset** changes. But the correlated blind spot it guards —
|
|
291
|
+
*"when a verdict surface cannot mechanically ground its judgment, it defaults toward PASS instead
|
|
292
|
+
of safe-fail"* — is **model-family-level, not FH-specific**. It lives in any load-bearing code the
|
|
293
|
+
AI writes, including **mapped field projects** (qasp · the-bible · pmh). FH is the meta-harness:
|
|
294
|
+
accelerating a field harness *to FH grade* means the field's load-bearing changes get the **same
|
|
295
|
+
cross-family adversarial gate** as FH's own assets. Not doing so is the exact gap that shipped **9
|
|
296
|
+
default-toward-PASS holes across 3 harnesses undetected** (measured 2026-07-03). Root principle:
|
|
297
|
+
**prose-specified verdict logic grants discretion; discretion's degrade direction is unconstrained
|
|
298
|
+
(→ optimistic PASS); same-family reviewers share the author's optimistic reading and miss it.**
|
|
299
|
+
|
|
300
|
+
**Trigger (per changed file — grep-assisted, salience-dependent, no field hook)**: an AI-authored
|
|
301
|
+
change to a **load-bearing field surface** — a function returning a **verdict/gate enum or exit code** (PASS/FAIL/BLOCK/allow/deny),
|
|
302
|
+
an **irreversible-op** path (publish/delete/history-rewrite), or a **safety invariant** (the-bible
|
|
303
|
+
L1 floor, qasp verdict-binding, a pre-push/pre-commit hook). File+symbol based (grep the diff for a
|
|
304
|
+
verdict-enum return / gate exit / safety-marked function) — but a **strong-advisory grep trigger,
|
|
305
|
+
not a hook**: the FH pre-commit gate is FH-internal and deliberately never installed into field
|
|
306
|
+
projects, so an agent under merge pressure can still under-trigger (unmarked safety logic,
|
|
307
|
+
boolean-return gate helpers, config-driven allow/deny, shell/CI irreversible paths escape the grep).
|
|
308
|
+
The under-trigger residual is named honestly in the detail doc — it is not claimed to be airtight.
|
|
309
|
+
|
|
310
|
+
**Gate (before merge, not after)**:
|
|
311
|
+
1. **Degrade-direction lint** (mechanical pre-screen — `scripts/degrade_direction_scan.sh`): flags
|
|
312
|
+
fall-through / `except` / `.get(default)` / unknown-branch landing on a **permissive** value.
|
|
313
|
+
**Advisory review surface, NOT a hard gate** — grep-heuristic, FP-tolerant; a hit = *"prove this
|
|
314
|
+
isn't default-toward-PASS"*, never a solo block (exit 2 = advisory). Points attention; the
|
|
315
|
+
cross-family review decides. (Portable field copy: `templates/degrade_direction_scan.sh`.)
|
|
316
|
+
2. **Cross-family adversarial review** (`auto-decorrelation` → ≥1 different-family auditor, e.g.
|
|
317
|
+
`codex` gpt-5.5/high for repo-grounded verdict code) — the same standing verifier the 4-axis
|
|
318
|
+
gate uses for load-bearing FH changes, now on field load-bearing changes. Governor keeps the
|
|
319
|
+
terminal verdict + **source-grounds** every finding (mechanical anchor over agreement).
|
|
320
|
+
3. **Confirm→fix→re-verify loop** until CONVERGED (no reachable false-PASS / false-CONFIRMED /
|
|
321
|
+
masked-FAIL / crash-where-safe-fail). **Each fix ships a mechanical regression test reproducing
|
|
322
|
+
the closed hole** — the anchor leg is a *required* convergence sub-condition, not incidental: two
|
|
323
|
+
decorrelated models agreeing is still judgment (mechanical anchor over agreement). Documented
|
|
324
|
+
recall-limits of a deliberately-precise no-judge oracle (separator-negation, positional multiset
|
|
325
|
+
masking) are **not** blockers.
|
|
326
|
+
|
|
327
|
+
*(Role deconfliction: this gate reviews **field code being authored**; the Irreversibility gates
|
|
328
|
+
below gate **the act** of publish/delete/rewrite — disjoint by role and by location, no double-gate.)*
|
|
329
|
+
|
|
330
|
+
**Degrade direction — cross-family unavailable is NOT a silent same-family pass** (the gate's own
|
|
331
|
+
standard, dogfood-caught 2026-07-03): if no different-family auditor is reachable, the gate does
|
|
332
|
+
**not** fall back to same-family review and proceed — that inherits `auto-decorrelation`'s general
|
|
333
|
+
*silent-degrade / never-hard-fail*, which is **fail-OPEN** for a load-bearing pre-merge surface
|
|
334
|
+
(the gate's entire value is decorrelation; same-family review shares the author's blind spot). It
|
|
335
|
+
marks the change **NOT-CONVERGED** and either blocks the autonomous merge / asks the operator, or
|
|
336
|
+
proceeds only under an **explicit, logged same-family-only acknowledgment** — never a silent
|
|
337
|
+
same-family pass. This **overrides** the delegated skill's default degrade for this surface,
|
|
338
|
+
consistent with §Irreversibility Surface-Class Degrade Invariant (applicable-but-tooling-down ≠ free skip).
|
|
339
|
+
|
|
340
|
+
**Residency**: sanitize company code (redact vendor/domain literals) before any external-family
|
|
341
|
+
dispatch; domain data never leaves. **Autonomy**: autonomous once the operator has consented (UAP),
|
|
342
|
+
same as the FH cross-family complement. **In autonomous loops** (innovator loop-engineering ·
|
|
343
|
+
`/goal` · cluster orchestration): this gate is **part of the delegated pipeline**, not an
|
|
344
|
+
afterthought — a load-bearing field change produced autonomously runs the lint → cross-family →
|
|
345
|
+
converge loop *before* it is Done. Autonomy floor (§Floor governance): the skip/run judgment is
|
|
346
|
+
trusted only at opus-tier+; below-floor runs the review or asks, never silently skips.
|
|
347
|
+
|
|
348
|
+
> **Detail** (discretion principle · 4-face signature · gate mechanics · n=7 qasp evidence):
|
|
349
|
+
> `knowledge/shared/harness-core/field_verdict_crossfamily_gate.md`.
|
|
350
|
+
|
|
288
351
|
## Irreversibility Gates — Surface-Class Degrade Invariant (shared spine of the two gates below)
|
|
289
352
|
|
|
290
353
|
The two gates that follow (Pre-Publish, Destructive-Op) guard **irreversible surfaces**. The floor they
|
|
@@ -482,6 +545,7 @@ Proposal format: `"I see [X]. Want me to run /[skill] to [one-line description]?
|
|
|
482
545
|
| "I don't know what to build", "how should I approach this", "organize this for me", "clarify this", "정리해줘" (ambiguous request before dispatch) | `/deep-clarify` |
|
|
483
546
|
| "memory feels bloated", "clean up memory", "memory too large", "memory hygiene" | `/memory-hygiene` |
|
|
484
547
|
| "ready to PR", "about to push", "merge this", "PR 올려줘", FH asset changed in session | 4-axis auto-gate (see above — runs automatically, no proposal needed) |
|
|
548
|
+
| **field verdict/gate/safety/irreversible code changed** in a mapped project (function returning a verdict enum / gate exit code / safety-invariant · publish/delete/history path) — **proactive, before merge** | **Field-Harness Load-Bearing Change Gate** (see above → degrade-lint → cross-family review → converge; same rigor as FH assets, applied to field code) |
|
|
485
549
|
|
|
486
550
|
**Guard**: Do not propose a skill that is already running. One signal = one-line proposal (no pressure). Before proposing, consult the UAP (§Operational Adaptation Loop): a skill the user has rejected 3+ times is **suppressed**, not re-proposed.
|
|
487
551
|
For per-skill utterance patterns, see the relevant `SKILL.md §Trigger Phrases` section.
|
package/README.md
CHANGED
|
@@ -286,7 +286,7 @@ pinned stronger model is recommended only for harness-editing sessions.
|
|
|
286
286
|
This is stated as an **invariant, not a per-model leaderboard**. Two structural laws, neither of which a
|
|
287
287
|
new release overturns:
|
|
288
288
|
|
|
289
|
-
1. **Operation
|
|
289
|
+
1. **Operation flattens across tiers** — the rules-in-context do the work, so every tier ceilings
|
|
290
290
|
on rule-application (Sonnet 5 tied Opus 4.8 at the battery ceiling in a 2026-07-03 replication).
|
|
291
291
|
2. **Depth (design increments) is tier-ordered, and the order is fixed *within a generation*** — a lower
|
|
292
292
|
tier never overtakes the higher of the **same** generation (tiers are priced to be worth it, so the
|
|
@@ -0,0 +1,143 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: field_verdict_crossfamily_gate
|
|
3
|
+
description: Load-bearing field-project verdict/gate/safety code gets the same cross-family adversarial gate as FH assets — the correlated default-toward-PASS blind spot is model-family-level, not FH-specific.
|
|
4
|
+
type: reference
|
|
5
|
+
date: 2026-07-03
|
|
6
|
+
tags: [cross-family, decorrelation, verdict-binding, field-harness, degrade-direction, correlated-blindspot, mode-d]
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Field-Harness Load-Bearing Change Gate — cross-family, pre-merge
|
|
10
|
+
|
|
11
|
+
> Compressed rule + trigger table live in `CLAUDE.md §Field-Harness Load-Bearing Change Gate`.
|
|
12
|
+
> This is the detail: the root principle, the failure signature, the gate mechanics, and the
|
|
13
|
+
> field evidence (qasp, 2026-07-03).
|
|
14
|
+
|
|
15
|
+
## 1. The root principle — prose specification grants discretion, not depth
|
|
16
|
+
|
|
17
|
+
Specifying an engine's decision logic in **conversational / prose** terms ("add more depth",
|
|
18
|
+
"break it down and analyze", "consider carefully whether it passed") does **not** add depth —
|
|
19
|
+
it grants **discretion**. The model fills that discretion with its **optimistic prior**. On a
|
|
20
|
+
verdict surface, the optimistic prior is *PASS*. So wherever a verdict surface's judgment is
|
|
21
|
+
under-constrained, its **degrade direction is toward PASS** — the exact opposite of safe-fail.
|
|
22
|
+
|
|
23
|
+
Corollary: **real depth in an engine = removing discretion (more mechanical constraint), not
|
|
24
|
+
better prose.** "깊이를 더해라" applied to engine logic is a category error. This is the same
|
|
25
|
+
axis as FH's `[[feedback_judge_robustness_mechanical_anchor]]` (terminal verdict needs a
|
|
26
|
+
mechanical anchor, never judge-only) and the source-level reinforcement question (does the
|
|
27
|
+
*format* of an emitted anchor change computation, not just its content).
|
|
28
|
+
|
|
29
|
+
Negative example (반면교사): **CaseCraft** hung loose prompts on the engine logic → the engine's
|
|
30
|
+
verdicts inherited the LLM's unconstrained optimistic reading → holes. **qasp / pmh overcame it**
|
|
31
|
+
by mechanizing the verdict surface (verdict-binding = mechanical ground truth only, no-judge,
|
|
32
|
+
fail-closed gates, mechanical anchors). See `[[project_qasp_casecraft_positioning]]`.
|
|
33
|
+
|
|
34
|
+
## 2. The failure signature — one sentence, four faces
|
|
35
|
+
|
|
36
|
+
> **"When a verdict surface cannot mechanically ground its judgment, it defaults toward PASS
|
|
37
|
+
> instead of safe-fail."**
|
|
38
|
+
|
|
39
|
+
Every instance is a form of **discretion leaking into verdict logic**:
|
|
40
|
+
|
|
41
|
+
| Face | The discretion | Safe-fail form |
|
|
42
|
+
|---|---|---|
|
|
43
|
+
| **default-PASS-on-absence** | unconstrained `else` / fall-through picks the permissive value | explicit `BLOCK`/`None`/raise |
|
|
44
|
+
| **affordance-without-grounding** | score-heuristic "judgment" of what counts as a match | hard grounding gate (target-text must score > 0) |
|
|
45
|
+
| **substring-not-exact** | loose definition of "present" (`tok in text` → paid⊂prepaid, 완료⊂미완료) | exact / word-boundary match |
|
|
46
|
+
| **unknown-to-permissive-default** | unenumerated branch defaults to allow | enumerate-or-safe-fail |
|
|
47
|
+
|
|
48
|
+
## 3. Why same-family review misses it — and cross-family catches it
|
|
49
|
+
|
|
50
|
+
The blind spot is **directional** (the degrade direction on the unhappy path) and rests on a
|
|
51
|
+
**shared optimistic prior**. A same-family reviewer — even a frontier model, even a target-tier
|
|
52
|
+
blind sim — reads the under-constrained branch the same optimistic way the author wrote it, so
|
|
53
|
+
it does not *see* the discretion as a hole. A **different-family** auditor does not share that
|
|
54
|
+
prior, so it reads the same branch adversarially and names the false-PASS.
|
|
55
|
+
|
|
56
|
+
This is not "cross-family is smarter" — it is **decorrelation**: the value is that the auditor's
|
|
57
|
+
error distribution is *different*, so it covers the author's directional blind spot. Governor
|
|
58
|
+
discipline still applies: sidecar findings are **candidates**, not terminal; the governor
|
|
59
|
+
source-grounds each (does the real pipeline reach it? does an existing mechanical anchor mitigate
|
|
60
|
+
it?) before acting — mechanical anchor over agreement.
|
|
61
|
+
|
|
62
|
+
## 4. The gate (before merge, not after)
|
|
63
|
+
|
|
64
|
+
1. **Degrade-direction lint** — `scripts/degrade_direction_scan.sh` (portable copy:
|
|
65
|
+
`templates/degrade_direction_scan.sh`). A cheap **mechanical pre-screen**: greps the changed
|
|
66
|
+
files for the code shapes above (except/else→PASS, `.get(k, <pass>)`/`setdefault`,
|
|
67
|
+
substring-on-grounding-line). **Advisory review surface, NOT a hard gate** — grep-heuristic,
|
|
68
|
+
false positives expected; a hit means *"prove this is not default-toward-PASS"*, and it never
|
|
69
|
+
blocks alone (exit 2 = advisory). It points attention; it does not decide. Opt-out per line:
|
|
70
|
+
`# noqa: degrade`. It scans **py + sh**; a changed load-bearing surface in any other language is
|
|
71
|
+
reported as *unscannable / not-covered* (exit 2), never folded into an "advisory clean". **Efficacy
|
|
72
|
+
caveat**: in the n=7 qasp sweep the lint itself caught nothing — the catches were the cross-family
|
|
73
|
+
audit + accumulated regression tests. It is a token-free pre-screen ahead of a paid cross-family
|
|
74
|
+
dispatch, *not a proven detector*; revisit if it never surfaces what the cross-family pass wouldn't.
|
|
75
|
+
2. **Cross-family adversarial review** — `auto-decorrelation` recruits ≥1 different-family auditor
|
|
76
|
+
(e.g. `codex` gpt-5.5 / high for repo-grounded verdict code). The same standing verifier the
|
|
77
|
+
4-axis gate uses for load-bearing FH assets, now applied to **field** load-bearing changes.
|
|
78
|
+
3. **Confirm → fix → re-verify loop** — iterate until the cross-family pass is **CONVERGED**: no
|
|
79
|
+
reachable false-PASS / false-CONFIRMED / masked-FAIL / crash-where-safe-fail-required. **Each fix
|
|
80
|
+
ships a mechanical regression test** reproducing the closed hole — a *required* convergence
|
|
81
|
+
sub-condition, not incidental. (Two decorrelated models agreeing is still judgment; in the n=7
|
|
82
|
+
sweep the round-6 over-correction was caught by an accumulated regression test, so the anchor leg
|
|
83
|
+
is made mandatory — mechanical anchor over agreement.) Documented recall-limits of a
|
|
84
|
+
deliberately-precise no-judge oracle (e.g. separator-negation `un-paid`, row-level positional
|
|
85
|
+
state swap in a multiset projection) are **not** blockers — inherent precision>recall tradeoffs,
|
|
86
|
+
fixture-tracked, not discretion-holes.
|
|
87
|
+
|
|
88
|
+
**Degrade direction — cross-family unavailable ≠ silent same-family pass** (the gate's own standard,
|
|
89
|
+
dogfood-caught 2026-07-03): the gate delegates the cross-family step to `auto-decorrelation`, whose
|
|
90
|
+
*general* degrade is "missing sidecar CLIs never hard-fail → in-session same-family + honest note."
|
|
91
|
+
That silent-degrade is **fail-OPEN for a load-bearing pre-merge surface** — same-family review shares
|
|
92
|
+
the author's directional blind spot, so proceeding on it defeats the gate's entire decorrelation
|
|
93
|
+
value while *claiming* the gate ran. So for this surface the gate **overrides** the default degrade:
|
|
94
|
+
cross-family unreachable → mark **NOT-CONVERGED** and block the autonomous merge / ask the operator,
|
|
95
|
+
or proceed only under an explicit **logged same-family-only acknowledgment** — never a silent
|
|
96
|
+
same-family pass. (This is the same fail-closed direction the target-tier Sonnet sim itself chose:
|
|
97
|
+
"if no cross-family sidecar is reachable, say so explicitly … not silently treat that as a pass.")
|
|
98
|
+
|
|
99
|
+
**Trigger (per changed file — grep-assisted, salience-dependent, no field hook)** — an AI-authored
|
|
100
|
+
change to a load-bearing field surface: a function returning a **verdict/gate enum or exit code**
|
|
101
|
+
(PASS/FAIL/BLOCK/allow/deny), an **irreversible-op** path (publish/delete/history-rewrite), or a
|
|
102
|
+
**safety invariant** (the-bible L1 floor, qasp verdict-binding, a pre-push/pre-commit hook). The
|
|
103
|
+
grep recipe (verdict-enum return / gate exit / safety-marked function) is a **strong-advisory
|
|
104
|
+
trigger, not a hook** — the FH pre-commit gate is FH-internal and never installed into field
|
|
105
|
+
projects, so it is salience-dependent, and the "safety invariant" category is interpretive, not
|
|
106
|
+
grep-decidable. **Named under-trigger residual** (do not claim airtight): unmarked safety logic,
|
|
107
|
+
boolean-return gate helpers, config-driven allow/deny, and shell/CI irreversible paths can escape
|
|
108
|
+
the grep — an agent under merge pressure can under-trigger by treating a change as non-load-bearing.
|
|
109
|
+
That residual is the reason the gate is reinforced by the always-on Autonomous-Initiative trigger
|
|
110
|
+
row + the operator's proactive framing, not by the grep alone.
|
|
111
|
+
|
|
112
|
+
**Residency** — sanitize company code (redact vendor/domain literals) before any external-family
|
|
113
|
+
dispatch; domain data never leaves. **Autonomy** — autonomous once the operator has consented in
|
|
114
|
+
the UAP (`tracks/_meta/user_adaptation_profile.md`, defined in `.claude/rules/operational_adaptation.md`),
|
|
115
|
+
same as the FH cross-family complement.
|
|
116
|
+
|
|
117
|
+
## 5. Field evidence — qasp verdict-binding sweep, 2026-07-03 (n=7)
|
|
118
|
+
|
|
119
|
+
The gap that motivated this rule: FH's cross-family decorrelation rigor had **never been
|
|
120
|
+
auto-applied to field-harness code** — only to FH's own assets. A one-time sweep of 3 mapped
|
|
121
|
+
harnesses (qasp / the-bible / pmh) found **9 HIGH default-toward-PASS holes**, all one signature.
|
|
122
|
+
|
|
123
|
+
qasp was fixed under this exact gate, and the **loop itself became the evidence**: each
|
|
124
|
+
confirm→fix→re-verify round, a **different family** caught the residual discretion that the
|
|
125
|
+
*previous same-family fix* had left — 7 rounds to CONVERGED. Even the frontier author's own
|
|
126
|
+
round-6 fix over-corrected (trusted a raw `"PASS"` string), and a **mechanical anchor** (a
|
|
127
|
+
round-1 regression test) caught it. Both legs of the doctrine — cross-family audit *and*
|
|
128
|
+
accumulated mechanical tests — fired against same-family error.
|
|
129
|
+
|
|
130
|
+
Positioning note: this is an **observational** field study, not yet a controlled claim. The
|
|
131
|
+
controlled follow-up (same- vs cross-family yield on a fixed bug corpus) is what turns the
|
|
132
|
+
observation into a result. Raw study: a private companion store's `paper-signals/` (Mode D;
|
|
133
|
+
`research_candidate_correlated_blindspot_verdict_code_2026-07-03.md`).
|
|
134
|
+
|
|
135
|
+
## 6. When baked into autonomous loops
|
|
136
|
+
|
|
137
|
+
In innovator-based loop-engineering (operator delegates a goal), `/goal`, or cluster
|
|
138
|
+
orchestration, this gate is **part of the delegated pipeline**, not an afterthought: a
|
|
139
|
+
load-bearing field change produced autonomously runs the degrade-lint → cross-family review →
|
|
140
|
+
converge loop **before it is considered done**. The autonomy floor applies — the skip/run
|
|
141
|
+
judgment on borderline cases is trusted only at opus-tier or above; a below-floor orchestrator
|
|
142
|
+
runs the review or asks, never silently skips (`[[feedback_judge_robustness_mechanical_anchor]]`,
|
|
143
|
+
CLAUDE.md §Floor governance).
|
|
@@ -138,6 +138,35 @@ A sidecar is recruited where it adds *decorrelated* value; its ceiling is still
|
|
|
138
138
|
harness lifts a model to its own ceiling, it does not move it ([[feedback_harness_ceiling_principle]]).
|
|
139
139
|
"Aggressive" Codex/Gemini use is bounded by the fit task-class above, never a blanket main-seat swap.
|
|
140
140
|
|
|
141
|
+
### Vendor-native harness — the main layer stays multi-CLI, never Copilot-consolidated
|
|
142
|
+
|
|
143
|
+
The harness-depth thesis has a **per-vendor corollary**: a frontier model realizes its *highest effective
|
|
144
|
+
capability inside its own vendor-native CLI/harness* — Claude in Claude Code, GPT in the Codex CLI, Gemini
|
|
145
|
+
in Antigravity — because each vendor tunes its full agentic loop (infer→act→observe + tools + context +
|
|
146
|
+
control) for its own model. A **universal router that wraps all of them** (GitHub Copilot) is a *thin*
|
|
147
|
+
surface with no vendor-native harness depth, so routing any model through it **strips the native-harness
|
|
148
|
+
buff and degrades that model's realized intelligence** — not just Claude's. This makes the earlier
|
|
149
|
+
"router shell" row (Copilot / Gemini CLI / Codex CLI as one class) **too coarse**: the *native* CLIs
|
|
150
|
+
(`codex`, `agy`/Antigravity) are full vendor harnesses and belong at their fit task-class above; only the
|
|
151
|
+
*cross-vendor* router (Copilot) is the thin surface.
|
|
152
|
+
|
|
153
|
+
**Consequences (design-locked):**
|
|
154
|
+
- **Main orchestration stays as-is** — FH pipeline + each vendor's native agent (Claude Code + Codex +
|
|
155
|
+
Antigravity), each on its own subscription/CLI. The synergy of that native-multi-CLI layer outweighs any
|
|
156
|
+
cost-consolidation Copilot offers; do **not** collapse the main layer into one router.
|
|
157
|
+
- **Copilot's proper position = a harness-less sidecar** — lightweight inline autocomplete + simple Q&A.
|
|
158
|
+
It is *not* an orchestration layer and *not* the preferred cross-family access when native CLIs reach.
|
|
159
|
+
- **Never route a model through Copilot when its native CLI is reachable** — copilot-Claude ⟪ native CC,
|
|
160
|
+
copilot-GPT ⟪ native Codex, copilot-Gemini ⟪ native Antigravity (harness depth + no decorrelation gain
|
|
161
|
+
if same family as governor + usage-credit cost).
|
|
162
|
+
- **The one legitimate Copilot cross-family use is egress-bounded** — in a *restricted corporate network*
|
|
163
|
+
where the native `codex`/`agy` CLIs cannot reach out, Copilot's **enterprise Pro+ catalog** becomes the
|
|
164
|
+
fallback *access* path to non-Claude frontier families (GPT/Gemini) for a one-shot decorrelation call —
|
|
165
|
+
still a sidecar, never the main seat (company-env panel: [[reference_corp_env_decorrelation_panel]]).
|
|
166
|
+
On a personal machine with a *free* Copilot tier this cross-family value does not even exist — free tier
|
|
167
|
+
is autocomplete/QA only, which is exactly its demoted role. (Derived 2026-07-03, operator + cross-vendor
|
|
168
|
+
Gemini concurrence; extends the governor=native-CC point to every vendor.)
|
|
169
|
+
|
|
141
170
|
**Maintenance-Cost Rule** — a compatibility layer is cheap as a *thin entrypoint*, expensive when it
|
|
142
171
|
*duplicates canonical knowledge*. The test:
|
|
143
172
|
|
package/package.json
CHANGED
|
@@ -9,8 +9,17 @@ model: sonnet
|
|
|
9
9
|
# memory-hygiene — Stale Memory Detection and Re-Verification
|
|
10
10
|
|
|
11
11
|
> Addresses the "stale-but-confident" failure mode: verified information that silently drifts
|
|
12
|
-
> while remaining highly ranked in retrieval — identified as a
|
|
12
|
+
> while remaining highly ranked in retrieval — identified as a harness failure mode
|
|
13
13
|
> in *Scaling the Harness in Agentic AI* (arXiv:2605.26112).
|
|
14
|
+
>
|
|
15
|
+
> **Sister asset**: arXiv:2607.01935 (*A-TMA: Decoupling State-Aware Memory Failures in Long-Term
|
|
16
|
+
> Agent Memory*, Shi/Tang/Tung 2026) names this exact class "ghost memory" — outdated / current /
|
|
17
|
+
> transitional facts intermixing during retrieval — and benchmarks it (LTP; conflict-accuracy +0.240,
|
|
18
|
+
> temporal-F1 0.03→0.17 on LoCoMo with a bank-maintenance layer). It is the tighter external frame for
|
|
19
|
+
> what memory-hygiene detects: this skill is the FH-native *detection + archival* pass over the same
|
|
20
|
+
> failure A-TMA formalizes at the retrieval layer. (Source-verified 2026-07-03; a broader bounded-memory
|
|
21
|
+
> testbed, arXiv:2607.02255 AgenticSTS, was considered and set aside — it targets context-assembly, not
|
|
22
|
+
> staleness, so it is not the sister here.)
|
|
14
23
|
|
|
15
24
|
FH is an online-first harness. Its memory entries point to live external resources (GitHub
|
|
16
25
|
repos, arXiv records, Zenodo DOIs, monitoring routines). These drift faster than in
|
|
@@ -141,6 +141,40 @@ Treat the adapter output as the isolated challenger result for Wave 1. This pres
|
|
|
141
141
|
|
|
142
142
|
---
|
|
143
143
|
|
|
144
|
+
## Step 0.6 — Verdict-Invariance Probe (load-bearing judged gates only · measured)
|
|
145
|
+
|
|
146
|
+
> **Sibling of Step 0.5**: Step 0.5 measures whether a skill *fires* on the right utterances; Step 0.6 measures whether a judged gate's *verdict* rests on behavior or on the rubric's phrasing. Cross-family review (auto-decorrelation) lowers correlation between judges but does not test groundedness — two decorrelated families can be swayed by the same wording artifact. This probe converts "is this verdict grounded?" from judged to **measured**: hold the behavior fixed, paraphrase the rubric, count flips.
|
|
147
|
+
|
|
148
|
+
> **External frame**: content-preserving policy rewrites flip up to 9.1% of LLM safety-judge verdicts above baseline jitter — 18–43% of those flips on *unambiguous* cases (arXiv:2605.06161, *Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges*, Weng/Feng/Xie 2026). The instrument here is a flip-count over fixture×paraphrase runs, not that paper's benchmark — the anchor grounds the invariance framing, not a drop-in metric.
|
|
149
|
+
|
|
150
|
+
**Fires only when** the target is a **load-bearing judged gate** — a rubric whose verdict is a gate enum (PASS/FAIL/BLOCK/allow/deny) **AND whose verdict gates a consequential action** (a FAIL actually blocks a publish/delete/merge, changes a safety outcome, or guards an irreversible surface). The same scope as the Field-Harness Load-Bearing Change Gate. **The enum shape is necessary but not sufficient** (target-tier sim, 2026-07-03): a cosmetic judged Done-When that merely *uses* PASS/FAIL vocabulary but gates nothing consequential ("does the description read well — PASS/FAIL") is **not** load-bearing — confirm a real downstream consequence before firing, or the probe over-fires on gate-shaped-but-inert rubrics. **One-time baseline per gate; re-probe only when the gate's rubric prose changes** (mirror of Step 0.5's re-probe rule). It does **not** fire on ordinary judged Done-When conditions, advisory checks, or prose-only edits that leave the rubric untouched — note `Step 0.6: skipped (not a load-bearing judged-gate rubric change)` and proceed. A gate whose verdict is already bound to a mechanical assertion (fixture test, exit code) has no discretion to probe — skip with that note.
|
|
151
|
+
|
|
152
|
+
**Procedure** (rides shipped machinery — no new harness, benchmark, or daemon):
|
|
153
|
+
1. **Fixtures**: take 2–3 **unambiguous** behavior fixtures — at least one clear-PASS and one clear-FAIL. Reuse the mechanical regression fixtures the gate already ships (the Field-Harness gate's convergence sub-condition requires them); author minimal ones only if absent, and ship them with the gate afterward.
|
|
154
|
+
2. **Paraphrase**: generate **K=5 semantics-preserving rewrites** of the rubric prose (reorder clauses, synonym-swap, restructure lists — never add/remove a criterion). Generate the set **cross-family via auto-decorrelation** so the paraphrase distribution is not same-family-biased; degrade honestly to same-family with a logged note if no sidecar is reachable.
|
|
155
|
+
3. **Dispatch**: run the gate's verdict on each fixture × each paraphrase (plus the original rubric) through the target-tier sim already used by the 4-axis gate — isolated Agent, `model:` pinned to the tier the gate must survive on; the author session does not judge its own rubric.
|
|
156
|
+
4. **Metric**: count verdict flips on the unambiguous fixtures. Report as measured: `verdict-invariance: <flips>/<fixtures×K> runs flipped (gate: <name> · tier: <model>)`.
|
|
157
|
+
5. **Record**: write the flip-count into the Axis-2 marker's `axis2-evidence` field (non-vacuous by construction — a count, not "it ran") and emit a one-line **Judge Card** alongside the gate: `judge-card: <gate> · flip-rate <n>/<runs> · fixtures <ids> · <date>`. The card is the baseline the re-probe rule compares against.
|
|
158
|
+
|
|
159
|
+
**Threshold + verdict mapping** (measured):
|
|
160
|
+
|
|
161
|
+
| Measured | Verdict | Severity |
|
|
162
|
+
|---|---|---|
|
|
163
|
+
| ≥1 flip on an unambiguous fixture | **ungrounded** — the verdict tracks phrasing, not behavior; the rubric leaks discretion | S (load-bearing by scope definition) |
|
|
164
|
+
| 0 flips across all fixture×paraphrase runs | invariance PASS (measured, not guessed) | — |
|
|
165
|
+
|
|
166
|
+
**Remediation** (on FAIL, before the gate is trusted): either **bind a fixture-based mechanical assertion** (the fixture that flipped becomes a regression test the verdict must satisfy regardless of prose), or **narrow/shorten the rubric prose** until the re-probe stops flipping. Both moves remove discretion from the prose — the ratchet direction is always toward less judged surface, not more rubric.
|
|
167
|
+
|
|
168
|
+
**Check class**: measured. **Adversarial pairing** (the probe's own soft spot is "were the paraphrases actually semantics-preserving?" — a corrupted paraphrase manufactures flips): paired by (a) **ambiguity control** — flips are counted only on unambiguous fixtures, where a semantics-changing rewrite would move both fixtures coherently rather than flipping one, and (b) **cross-family paraphrase generation** — the rewrite set is not authored by the family whose verdict is under test. No judge-only path: the terminal artifact is a count over dispatched runs.
|
|
169
|
+
|
|
170
|
+
**Honest residual (named)**: the probe proves the verdict is **invariant to wording**, not that it is **correct** — a gate can be robustly, invariantly wrong. Correctness stays with fixture ground-truth and the human PR gate; Step 0.6 removes one failure class (phrasing-dependence), it does not certify the rubric.
|
|
171
|
+
|
|
172
|
+
**Simplicity note** (earned-complexity check, Wave-1 angle #1 applied to itself): net-new surface = a fixture-paraphrase procedure + one threshold. Everything else is already shipped — steel-quench dispatch, the 4-axis target-tier sim, the Axis-2 marker field, auto-decorrelation, and the mechanical fixtures load-bearing gates already owe. Cost converges to zero: baseline once per gate, re-probe only on rubric prose change. Each FAIL's remediation shrinks judged prose or adds a mechanical anchor — the harness gets simpler under measurement, not more decorated.
|
|
173
|
+
|
|
174
|
+
> **Origin**: proposed by an autonomous innovator scan (Mode F, 2026-07-03), designed by a Fable-5 sidecar, source-verified (2605.06161 numerics confirmed verbatim) before adoption. Orthogonal to today's cross-family gate: that lowers judge *correlation*, this measures verdict *groundedness*.
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
144
178
|
## Wave 1 — 6 Mandatory Attack Angles
|
|
145
179
|
|
|
146
180
|
**Execution principles**: Attacks must be based on real code/files/configs — abstract criticism prohibited.
|