@chrono-meta/fh-gate 1.4.28 → 1.4.30

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@chrono-meta/fh-gate",
3
- "version": "1.4.28",
3
+ "version": "1.4.30",
4
4
  "description": "FH runtime adapters — run FH governance, skills, and agents via Claude or Codex with machine-parseable gates.",
5
5
  "license": "MIT",
6
6
  "keywords": [
@@ -8,12 +8,16 @@ model: sonnet
8
8
 
9
9
  # edit-manifest — Predict-Verify Loop for Harness Edits
10
10
 
11
- > Implements two mechanisms from frontier harness research:
12
- > - **Change Manifest** (AHE, arXiv:2604.25850): every edit declares a falsifiable prediction;
13
- > the next iteration verifies it against actual outcomes.
14
- > - **Validation Gate** (SkillOpt, arXiv:2605.23904): candidate edits are accepted only when
15
- > the selection-split score strictly improves. Rejected edits are retained as negative
16
- > feedback for future proposals — not silently discarded.
11
+ > Implements two mechanisms from frontier harness research, named as in their source papers (both terms
12
+ > are the papers' own verified 2026-06-17 against arXiv full text):
13
+ > - **Change Manifest** (AHE, arXiv:2604.25850): each edit attaches a manifest entry declaring a
14
+ > falsifiable predicted impact (expected fixes + at-risk regressions); the next round intersects it
15
+ > with observed task-level deltas to produce a per-edit verdict. (Paper: "the Evolve Agent produces a
16
+ > change manifest…".)
17
+ > - **Validation Gate** (SkillOpt, arXiv:2605.23904): a candidate is accepted only when its
18
+ > selection-split score is strictly greater than the current selection score (ties rejected); rejected
19
+ > edits are kept in an epoch-local buffer, not silently discarded. (Paper: "validation gate" /
20
+ > "held-out gate".)
17
21
 
18
22
  FH's regression_guard.sh catches backward regressions. edit-manifest closes the forward
19
23
  loop: did the edit actually improve what it claimed to improve?
@@ -201,7 +205,7 @@ Apply `verified_at`, `verification_note`, `gate_decision` to each resolved entry
201
205
 
202
206
  ## Validation Gate Logic
203
207
 
204
- Mirrors SkillOpt's selection-split gate:
208
+ Mirrors SkillOpt's validation gate (the held-out gate over its selection-split score):
205
209
 
206
210
  ```
207
211
  IF verified evidence shows improvement:
@@ -253,6 +257,6 @@ non-vacuous by the **mandatory cited observation** (no citation → stays pendin
253
257
  ## References
254
258
 
255
259
  - Theoretical basis: AHE (arXiv:2604.25850) §4 change manifest + prediction falsifiability
256
- - Validation gate: SkillOpt (arXiv:2605.23904) §3.3 selection-split gate + rejected-edit buffer
260
+ - Validation gate: SkillOpt (arXiv:2605.23904) §3.3 validation/held-out gate over the selection-split score + rejected-edit buffer
257
261
  - Integrates with: `harvest-loop` Step 0-c · `verify-bidirectional` · 3-axis auto-gate (CLAUDE.md)
258
262
  - Manifest file: `tracks/_meta/edit_manifest.yaml`
@@ -106,12 +106,47 @@ literal span — never on a model's agreement.
106
106
 
107
107
  | Citation form | Resolved URL |
108
108
  |---|---|
109
- | `arXiv:NNNN.NNNNN` / `arXiv:NNNN.NNNNNvK` | `https://arxiv.org/abs/NNNN.NNNNN` |
109
+ | `arXiv:NNNN.NNNNN` / `arXiv:NNNN.NNNNNvK` | `https://arxiv.org/abs/NNNN.NNNNN` (abstract — the **canonical identifier** surface). For a section-/body-cited claim, prefer full text **first**, falling back in order `https://arxiv.org/html/NNNN.NNNNN` → `https://arxiv.org/pdf/NNNN.NNNNN` → `/abs/` (HTML is a *partial* backfill — see *Surface selection* below) |
110
110
  | bare DOI `10.xxxx/...` | `https://doi.org/10.xxxx/...` |
111
111
  | `http(s)://...` | use as-is (HTTPS-upgrade handled by WebFetch) |
112
112
  | Named source, no URL ("paper X shows Y") | **one** `WebSearch` to locate the canonical URL → then WebFetch it. Do **not** verify against the search snippet alone — the snippet is not the source. |
113
113
  | version token `pkg x.y.z` | the package registry/release page (npm/PyPI/GitHub releases) for that exact version |
114
114
 
115
+ **Surface selection — the abstract is the wrong surface for a §-cited mechanism.** The `/abs/` page
116
+ carries only the abstract; a claim that cites a section or attributes a *named body-level mechanism* to
117
+ the paper (e.g. "§4.2 stale-but-confident", "the paper's Change Manifest") returns NONE / MENTIONS on
118
+ `/abs/` even when fully supported in the body — an **artificial Unsupported** (measured 2026-06-17
119
+ in-the-wild dogfood: a `/abs/`-only fetch falsely flagged a body-supported claim; it resolved only after
120
+ escalating to full text). Two mechanical, judgment-free rules cover this:
121
+
122
+ 1. **Mechanical first-surface trigger** — if the claim text carries a section/figure marker (`§N`,
123
+ "section N", "Table N", "Figure N") → fetch full text **first**. Otherwise fetch `/abs/` first. (Do
124
+ *not* pre-decide "is this a body-level mechanism?" — that is a salience-dependent judgment; the
125
+ `§`-marker is the only mechanical signal, and the NONE-escalation below catches the rest.)
126
+ 2. **Abstract NONE/MENTIONS escalates** (the self-correcting backstop) — *any* `/abs/` fetch that
127
+ returns **NONE or MENTIONS** escalates once to full text before an Unsupported verdict (see Decision
128
+ rules). A true headline claim already returned ASSERTS on `/abs/`, so it never escalates; only a
129
+ NONE/MENTIONS does, and the escalation is cheap insurance. (A MENTIONS on the abstract is the common
130
+ case — the body may ASSERT what the abstract only named; measured 2026-06-17 wild case 2604.25850 was
131
+ a /abs MENTIONS that became a full-text ASSERTS. A **NEGATES** does *not* escalate — it is
132
+ polarity-final: a body span asserting the same relation cannot coexist with an abstract that refutes
133
+ it; a *different, narrower* body claim is a citation-granularity error, still Unsupported, not an
134
+ escalation target.) This subsumes rule 1's misses — a body claim that *didn't* carry a `§` marker
135
+ still gets full text via its abstract NONE/MENTIONS.
136
+
137
+ **HTML is a partial backfill — fall back, never Phantom.** arXiv HTML exists only for LaTeX-source
138
+ papers (roughly post-2023) and some conversions fail, so `/html/{id}` can 404 for a perfectly real
139
+ paper. A 404 on `/html/` (or `/pdf/`) is **surface-absence, not identifier-absence**: fall back
140
+ `/html/` → `/pdf/` → `/abs/`. Only a 404 on **`/abs/`** (the canonical identifier surface) is Phantom.
141
+
142
+ **Anchor the full-text fetch.** Full text is large; pass the cited section number / mechanism term *into*
143
+ the WebFetch prompt (`"…the span in §4.2 that states…"`) so the span search is anchored, not whole-paper
144
+ — this makes the `§`-citation the retrieval anchor, not merely a routing flag, and guards against a long
145
+ paper's relevant span falling outside the fetch model's attention (a full-text false-NONE).
146
+
147
+ Reserve `/abs/` for claims the abstract itself states (headline numbers, the paper's stated contribution
148
+ — e.g. a top-line stat like "39–77% factual support" is abstract-level and `/abs/` is correct).
149
+
115
150
  **WebFetch prompt template** (ask for a *span*, not a *verdict* — this keeps the anchor non-model):
116
151
 
117
152
  ```
@@ -126,6 +161,12 @@ Polarity is load-bearing: a page saying "X does NOT hold" or "earlier work claim
126
161
  contains a span lexically matching the claim. Asking for the label keeps the *retriever* mechanical while
127
162
  surfacing the stance the *judge* needs — a NEGATES/MENTIONS span is **not** grounding.
128
163
 
164
+ **Coinage at fetch time:** when the claim's key term is a known FH coinage (a name FH invented that the
165
+ source does *not* use — e.g. a label like "Drift Gate" for a mechanism the paper describes only in its
166
+ own words), pass the *mechanism description* — not the coinage string — into the `<exact claim>` slot, so
167
+ the retriever searches for the **relation**, not a label that isn't on the page. The Coinage-vs-source-term decision rule (below) then only adjudicates the returned
168
+ span; it never has to overturn a NONE the prompt itself caused.
169
+
129
170
  **Span-evidence format** (a Grounded external verdict is invalid without this):
130
171
 
131
172
  ```
@@ -150,9 +191,22 @@ Grounded: N / Unsupported: N / Unreachable: N / Phantom: N
150
191
  **Decision rules**:
151
192
  - Span labelled **ASSERTS** that expresses the claimed relation → **Grounded ✅** (record span).
152
193
  - Span labelled **NEGATES or MENTIONS** → **Unsupported 🟠** (a negating or merely-mentioning span is not
153
- grounding — the false-Grounded trap).
194
+ grounding — the false-Grounded trap). **One exception — a MENTIONS from the abstract (`/abs/`) surface
195
+ escalates first** (per the NONE/MENTIONS escalation below) before being finalized: the body may ASSERT
196
+ what the abstract only mentioned. A **NEGATES** is final (the source refutes the claim — the body will
197
+ not undo it); a MENTIONS from a full-text surface is also final.
154
198
  - Fetch succeeded, content readable, span = NONE or off-claim → **Unsupported 🟠** (cited-but-not-verified).
155
- - Identifier does not resolve at all (404 on a specific arXiv id, dead DOI, fabricated) **Phantom ❌**.
199
+ **Surface-escalation first an abstract NONE/MENTIONS is the mechanical trigger (no judgment):** if the
200
+ only surface fetched was the abstract (`/abs/`), re-fetch full text (fall back `/html/{id}` → `/pdf/{id}`)
201
+ before classifying — *any* abstract NONE/MENTIONS escalates once, you do **not** first decide whether the
202
+ claim "looks body-level" (that judgment is the salience leak this rule removes). A body-supported claim
203
+ must not be failed on an abstract-only NONE/MENTIONS. This is the second-surface escalation, **extended
204
+ from Unreachable to Unsupported-on-abstract** (2026-06-17 dogfood finding #2). Only after a full-text
205
+ surface *also* returns NONE/MENTIONS is the verdict Unsupported. (Single-pass: escalate once, then
206
+ classify — no loop.)
207
+ - Identifier does not resolve on the **canonical** surface (`/abs/` 404, dead DOI, fabricated id) →
208
+ **Phantom ❌**. A 404 on `/html/` or `/pdf/` while `/abs/` resolves is **surface-absence, not
209
+ identifier-absence** — fall back per *Surface selection*, never Phantom.
156
210
  - Identifier plausibly real but fetch blocked in this environment (paywall/403/timeout/cross-host) →
157
211
  **Unreachable ⏳** — provisional; note the limit, route to a second surface or the human gate; never auto-Phantom.
158
212
  - `WebFetch`/`WebSearch` **absent/disabled in the environment** (not one source — the capability) → Step 2-E
@@ -160,6 +214,46 @@ Grounded: N / Unsupported: N / Unreachable: N / Phantom: N
160
214
  (do not mark every claim Unreachable + CONDITIONAL_PASS — that falsely implies grounding was attempted).
161
215
  - **Never** upgrade NONE to Grounded because a second model "thinks it's probably right" (agreement bias).
162
216
 
217
+ **Coinage-vs-source-term (the external analogue of Step 2's format normalization — but *judged*, not
218
+ mechanical).** FH sometimes attributes a *coined* name to an external source (e.g. "the paper's Drift
219
+ Gate") when the paper uses different words for the same mechanism. A name
220
+ mismatch alone is not automatically Unsupported — but the path to rescuing it is **deliberately narrow**,
221
+ because this is one rationalization away from the agreement-bias trap above. The failure direction must
222
+ stay conservative (over-flag, never false-Grounded):
223
+
224
+ - **Possessive/attributive phrasing is presumed a terminology claim.** "the paper's X", "its X",
225
+ "X, per [paper]" → literal absence of the coinage on the source = **Unsupported 🟠**, *unless the FH
226
+ artifact separately states the underlying relation in non-coinage words* — "separately" means
227
+ **anywhere in the artifact, including an inline gloss of the coined term** (e.g. "**X** (the spec's
228
+ accept-only-on-improvement rule)"); proximity does not matter, only that the relation is stated in
229
+ non-coinage words, so the mechanism is independently legible without the label. A runner may **not**
230
+ reclassify a bare "the paper's X" (a coinage with *no* such gloss anywhere) as a mechanism claim on its
231
+ own reading — the mechanism claim must already be spelled out **in the artifact** before the span-judged
232
+ path opens. (Closes the laundering move: relabeling a terminology claim as a mechanism claim to save it.)
233
+ An inline gloss grants only *eligibility* for that path — Grounded still comes **only from the fetched
234
+ source span** (de-label test below), never from the gloss itself; a gloss the source does not support
235
+ is artifact-internal, fails de-label, and cannot launder a coinage into Grounded.
236
+ - **When the mechanism IS spelled out, ground on the relation, not the string — mechanical de-label
237
+ test:** blank out *both* labels (FH's coinage and the source's term); the span must still literally
238
+ state the operative relation the claim asserts — the same inputs→outputs / the same conditional / the
239
+ same action. If, with both names blanked, the span no longer states the claim, the equivalence was
240
+ reader-supplied → **Unsupported 🟠**. (Worked shape: blank FH's coined label *and* the source's own
241
+ term; a fetched span that still literally states the operative relation — e.g. "accepted only when the
242
+ score strictly improves" — in the source's own words is Grounded; if the equivalence only holds once
243
+ you read FH's label back in, it was reader-supplied → Unsupported.)
244
+ - **No self-granted Grounded on a contested match.** If the de-label test is contested rather than clean,
245
+ the runner may **not** write Grounded — the *only* path to Grounded for a disputed term-match is
246
+ **escalate to `/steel-quench` Wave 1** and let the isolated adversary surface the span or reject it.
247
+ Default otherwise = Unsupported. (Removes the runner's discretion to self-select the lenient branch.)
248
+ - **Recommended conservative resolution (publish-facing).** For a publish-facing citation the safer fix
249
+ is to require the artifact to **quote the source's own term** (e.g. "what the source itself calls X")
250
+ instead of asserting an unverified label as the source's — removing the judgment entirely. Reserve the
251
+ judged de-label path for low-stakes internal claims; match effort to stakes.
252
+
253
+ Unlike Step 2's value-format normalization (mechanical: `300s` ≡ `300 seconds`), coinage-vs-term is
254
+ **judged** — Grounded only via a clean de-label span or a passed steel-quench escalation, never a
255
+ reader's semantic equivalence assertion.
256
+
163
257
  ---
164
258
 
165
259
  ## §Step3-Detail