@chrono-meta/fh-gate 1.4.72 → 1.4.74
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/rules/.public-surface-patterns.defaults +44 -0
- package/.claude/rules/fh_4axis_gate.md +207 -0
- package/.claude-plugin/marketplace.json +2 -2
- package/AGENTS.md +26 -2
- package/CATALOG.md +31 -0
- package/docs/ETHOS.md +106 -0
- package/docs/OUTPUT_EVIDENCE.md +118 -0
- package/docs/WHY.md +42 -0
- package/knowledge/patterns/ensemble_union_detection_task_pattern.md +125 -0
- package/knowledge/shared/GLOSSARY.md +77 -0
- package/knowledge/shared/harness-core/measurement-integrity-checklist.md +10 -0
- package/knowledge/shared/learnings/subagent_invocations_log.yaml +554 -0
- package/knowledge/shared/patterns/multi-persona-review.md +88 -0
- package/knowledge/shared/plugin-catalog/recommended_plugins.md +117 -0
- package/package.json +42 -1
- package/plugins/fh-commons/.claude-plugin/plugin.json +1 -1
- package/plugins/fh-meta/.claude-plugin/plugin.json +2 -2
- package/plugins/fh-meta/CHANGELOG.md +617 -0
- package/plugins/fh-meta/skills/context-doctor/SKILL.md +42 -4
- package/plugins/fh-meta/skills/context-doctor/SKILL_detail.md +38 -0
- package/scripts/below_floor_scan.sh +91 -0
- package/scripts/chamber_candidate_collect.sh +223 -0
- package/scripts/chamber_run.sh +184 -0
- package/scripts/degrade_direction_scan.sh +222 -0
- package/scripts/fh_env_delta_scan.sh +108 -0
- package/scripts/fh_session_load.sh +202 -0
- package/scripts/gate_pathspec_check.sh +166 -0
- package/scripts/package_coverage_check.sh +119 -0
- package/scripts/prepush_guard_check.sh +374 -0
- package/scripts/psa_scan_lib.sh +153 -0
- package/scripts/public_surface_scan_files.sh +157 -0
- package/scripts/selfcheck.sh +28 -0
- package/scripts/session_close_check.sh +171 -0
- package/scripts/substrate_jump_detector.sh +60 -0
- package/scripts/test_degrade_scan_shell_probes.sh +185 -0
- package/scripts/test_marker_floor_lanes.sh +45 -0
- package/scripts/test_prepush_stdin_integrity.sh +119 -0
- package/scripts/tier_census_grep.sh +54 -0
- package/scripts/universal_guard_check.sh +280 -0
- package/templates/.claude/rules/mcp_tool_gating.md +157 -0
- package/templates/.claude/rules/session.md +153 -0
- package/templates/.git-hooks/pre-commit +848 -0
- package/templates/.git-hooks/pre-push +585 -0
- package/templates/PRE-PUBLISH-CHECKLIST.md +85 -0
- package/templates/contrib_session.md +34 -0
- package/templates/degrade_direction_scan.sh +222 -0
- package/templates/goal-quench-hook-setup.md +152 -0
- package/templates/predelete_check.sh +72 -0
- package/templates/regression_guard.sh +563 -0
- package/templates/starter_profile.md +83 -0
- package/templates/temper_check.sh +46 -0
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# public-surface-audit DEFAULTS (COMMITTED — universal, non-operator-private patterns only).
|
|
2
|
+
# The gate loads this + the gitignored .public-surface-patterns (operator literals) so it is
|
|
3
|
+
# NEVER fully blind, even on a fresh clone before the operator populates the gitignored override.
|
|
4
|
+
# Put ONLY patterns that are safe to publish here (no real username/company/companion-store names).
|
|
5
|
+
# NOTE: a pattern here must NOT match its own regex text, or committing this file self-blocks. The
|
|
6
|
+
# /Users/ class below is safe — the literal '[' after /Users/ cannot match the class [a-z0-9_-].
|
|
7
|
+
# Format: severity<TAB>regex
|
|
8
|
+
MED /Users/[a-z0-9_-]+/
|
|
9
|
+
# Tailscale CGNAT (2nd octet 64-127) — operator node / private network topology (a Tailscale-served sidecar box)
|
|
10
|
+
MED 100\.(6[4-9]|[7-9][0-9]|1[01][0-9]|12[0-7])\.[0-9]{1,3}\.[0-9]{1,3}
|
|
11
|
+
#
|
|
12
|
+
# ── Credential SHAPES (added 2026-07-26) ─────────────────────────────────────
|
|
13
|
+
# Until now this file carried only OPERATOR-PRIVATE tokens (home paths, node IPs), so the gate
|
|
14
|
+
# saw an identity leak but was blind to a live CREDENTIAL: a staged AWS key / GitHub PAT /
|
|
15
|
+
# Anthropic key scanned clean (measured known-pair, 2026-07-26 — Case D). These patterns match
|
|
16
|
+
# SHAPES, never values, so they are safe to publish here. Imported from the cross-audited sister
|
|
17
|
+
# asset PromptPartner/agentsmith (`scripts/secret-scan.sh`) — see
|
|
18
|
+
# tracks/_audit/session_2026_07_26_agentsmith-sister.md.
|
|
19
|
+
# Self-match safe: every pattern's own literal text has a '[' where the pattern needs a class
|
|
20
|
+
# member, so none matches itself (verified mechanically before commit).
|
|
21
|
+
# Live-calibrated over the whole tracked tree before shipping: exactly ONE false positive, the AWS
|
|
22
|
+
# documentation key AKIAIOSFODNN7EXAMPLE, closed via the PSA_PLACEHOLDER exemption (both copies).
|
|
23
|
+
# NAMED RESIDUAL — deliberately NOT imported: agentsmith's generic assigned-literal pattern
|
|
24
|
+
# (password|token|api_key)=«8+ chars». In a docs-heavy repo it fires on every documented
|
|
25
|
+
# `export GH_TOKEN=…` placeholder; over-blocking trains PUBLIC_SURFACE_OK into muscle memory and
|
|
26
|
+
# disarms the gate. Un-imported on purpose, not overlooked.
|
|
27
|
+
#
|
|
28
|
+
# MODERN VARIANTS (added in the same session, after a cross-family audit showed the imported set
|
|
29
|
+
# only covered LEGACY token formats): agentsmith's list predates GitHub's fine-grained PATs
|
|
30
|
+
# (`github_pat_`), Slack app-level tokens (`xapp-`), and OpenAI's project/service-account keys
|
|
31
|
+
# (`sk-proj-` / `sk-svcacct-`). All three scanned CLEAN against the first import — verified by
|
|
32
|
+
# staging each shape and watching the gate pass. The alnum-only `sk-…` class cannot consume a
|
|
33
|
+
# hyphenated prefix, which is why `sk-proj-…` needs its own row rather than a widened class.
|
|
34
|
+
HIGH BEGIN [A-Z ]*PRIVATE KEY
|
|
35
|
+
HIGH AKIA[0-9A-Z]{16}
|
|
36
|
+
HIGH ASIA[0-9A-Z]{16}
|
|
37
|
+
HIGH gh[pousr]_[A-Za-z0-9]{20,}
|
|
38
|
+
HIGH github_pat_[A-Za-z0-9_]{30,}
|
|
39
|
+
HIGH xox[baprs]-[A-Za-z0-9-]{10,}
|
|
40
|
+
HIGH xapp-[0-9]-[A-Za-z0-9-]{10,}
|
|
41
|
+
HIGH sk-[A-Za-z0-9]{20,}
|
|
42
|
+
HIGH sk-(proj|svcacct|admin)-[A-Za-z0-9_-]{20,}
|
|
43
|
+
HIGH sk-ant-[A-Za-z0-9_-]{20,}
|
|
44
|
+
HIGH AIza[0-9A-Za-z_-]{35}
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
---
|
|
2
|
+
paths:
|
|
3
|
+
- "plugins/**/SKILL.md"
|
|
4
|
+
- "knowledge/**/*.md"
|
|
5
|
+
- "templates/**"
|
|
6
|
+
- "AGENTS.md"
|
|
7
|
+
- ".claude/rules/**"
|
|
8
|
+
- ".claude/agents/**/*.md"
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
<!-- ⚠️ CLAUDE.md 를 glob 에서 **의도적으로 제외**했다 (2026-07-20, cross-family 지적).
|
|
12
|
+
루트 CLAUDE.md 는 세션 시작에 무조건 읽히므로, glob 에 넣으면 이 규칙이 매 세션
|
|
13
|
+
즉시 로드된다 = 절감 0 이고 총량은 오히려 늘어난다(67.6k + 12.5k > 76.7k).
|
|
14
|
+
즉 "상주 파일을 조건부 규칙의 트리거로 넣는 것"은 pointer-illusion 의 부활 경로다.
|
|
15
|
+
CLAUDE.md 편집은 pre-commit 훅이 계속 커버한다. docs/** · plugins/**/*.md(전체) ·
|
|
16
|
+
scripts/** 도 과매칭(읽기만 해도 로드)이라 좁혔다. -->
|
|
17
|
+
|
|
18
|
+
<!--
|
|
19
|
+
FH 4-Axis Auto-Gate — 경로 스코핑된 규칙 (2026-07-20 이전)
|
|
20
|
+
|
|
21
|
+
왜 여기 있나: CLAUDE.md **파일** 76,706자 중 이 섹션이 10,331자(13.5%)로 단일 최대였다.
|
|
22
|
+
요약으로 대체했다. 파일 전체는 같은 세션의 다른 정리(중복 3건·New-Skill 게이트 편입)까지 합쳐
|
|
23
|
+
76,706 → 67,611 (순감 9,095자, 11.9%) — **합산치이지 이 절 하나의 성과가 아니다**.
|
|
24
|
+
전부 **파일 char 실측**이지 `/context` 상주
|
|
25
|
+
실측이 아니다(상주는 톱레벨 새 세션 `/context` 로만 잰다 — 이번엔 미측정).
|
|
26
|
+
이 게이트는 **FH 자산 파일을 건드릴 때만** 필요하므로 `paths:` 로 조건부 로드한다.
|
|
27
|
+
(`paths:` 없는 rules 파일은 세션 시작 시 항상 로드된다 — 반드시 유지할 것.)
|
|
28
|
+
⚠️ 공식 트리거는 **read** 다("Path-scoped rules trigger when Claude reads files matching
|
|
29
|
+
the pattern" — code.claude.com/docs/en/memory.md). 즉 매칭 파일을 안 읽고 Write 로 신규
|
|
30
|
+
생성하는 경로엔 이 규칙이 안 실린다. 그 구멍의 floor 는 pre-commit 훅이다.
|
|
31
|
+
|
|
32
|
+
왜 이 섹션이 이동 후보 1순위였나 — 2축 판정:
|
|
33
|
+
① 트리거가 **파일**이다 (FH 자산 수정). 의도·발화 트리거가 아니다.
|
|
34
|
+
② **기계 백스톱이 있다** — `templates/.git-hooks/pre-commit` 이 커밋을 하드 차단한다.
|
|
35
|
+
즉 이 산문이 안 떠도 훅이 막는다. 산문은 훅 위의 살리언스 층이다.
|
|
36
|
+
|
|
37
|
+
⚠️ 같은 이유로 **옮기면 안 되는 것들**: Pre-Publish Gate · Destructive-Op Gate ·
|
|
38
|
+
Irreversibility 공통 척추. 그것들은 **파일이 아니라 의도로 발동한다**
|
|
39
|
+
(`gh repo create --public` 은 파일을 건드리지 않는다). 경로 스코핑하면
|
|
40
|
+
정확히 그 경로에서 안 뜬다 = fail-open.
|
|
41
|
+
|
|
42
|
+
과거 실패 참조: FH 가 detail 포인터 뒤로 ~50k 를 옮겼으나 대상이 여전히 auto-load 라
|
|
43
|
+
절감이 0 이었다(pointer-illusion). `paths:` 는 플랫폼 레벨 강제라 그 함정이 없다.
|
|
44
|
+
-->
|
|
45
|
+
|
|
46
|
+
## FH Improvement 4-Axis Auto-Gate (Self-Verification Orchestrator)
|
|
47
|
+
|
|
48
|
+
**Whenever the AI modifies FH assets** (SKILL.md · **`SKILL_detail.md`** · `.claude/rules/*.md` · `knowledge/shared/rules/*.md` (relocated protocol rules — always full-gate, NOT under the knowledge carve-out) · `templates/` · `CLAUDE.md` · substantive `knowledge/` docs · substantive `docs/*.md` · `AGENTS.md` — see Substantive carve-out below),
|
|
49
|
+
the 4-axis verification chain runs **automatically before the first commit** of that session.
|
|
50
|
+
No user request is needed — this is a mandatory autonomous step, not a proposal.
|
|
51
|
+
|
|
52
|
+
> **`SKILL_detail.md` added 2026-07-26.** It was absent from this list *and* from both gate
|
|
53
|
+
> implementations because the matching term was the literal `SKILL\.md`, which the string
|
|
54
|
+
> `SKILL_detail.md` does not contain (the underscore breaks it). Measured at the time: 17 files /
|
|
55
|
+
> 208,710 B = **27.7% of the skill-spec surface**, 16 of 17 carrying fenced code blocks — i.e. real
|
|
56
|
+
> executable content, precisely what the Substantive carve-out below says must be gated *wherever it
|
|
57
|
+
> lives*. It leaked twice for real (`371c04f`, `e661931` — both single-file edits to
|
|
58
|
+
> `phantom-quench/SKILL_detail.md`, a gate skill's own behavioral spec, with zero 4-axis coverage).
|
|
59
|
+
> **The structural lesson outlives the fix**: `salience-splitter` *widens* a hole of this shape every
|
|
60
|
+
> time it relocates content to lean the resident layer, so **every split must re-ask whether the
|
|
61
|
+
> destination path is inside the gate** — coverage otherwise shrinks as the diet succeeds.
|
|
62
|
+
> Mechanical anchor: `scripts/gate_pathspec_check.sh` (known-pair, wired into pre-commit).
|
|
63
|
+
|
|
64
|
+
**Commit gate**: `git commit` on FH asset changes is hard-blocked by `templates/.git-hooks/pre-commit` until all required axes PASS. Hook installation (one-time): `git config core.hooksPath templates/.git-hooks && chmod +x templates/.git-hooks/pre-commit templates/.git-hooks/pre-push` (the same `core.hooksPath` also activates the **pre-push** Destructive-Op gate — see that section below).
|
|
65
|
+
|
|
66
|
+
```
|
|
67
|
+
FH asset modified → Axis 1 (templates/regression_guard.sh --pr {BRANCH})
|
|
68
|
+
→ Axis 2 (/steel-quench) → Axis 3 (/phantom-quench)
|
|
69
|
+
→ marker: tracks/_meta/.axes_23_passed_{branch}_{date}.marker
|
|
70
|
+
(required fields: axis2-engine / axis2-model / floor-status / axis2-evidence;
|
|
71
|
+
hook validates mechanically: below-floor blocks without below-floor-ack, and axis2-evidence
|
|
72
|
+
must be non-vacuous — a recorded verdict/count, not "it ran". Marker scope is form +
|
|
73
|
+
non-vacuity + auditability, NOT provenance — a fabricated marker is the weekly-audit + operator
|
|
74
|
+
residual by design, do NOT fake-close it.
|
|
75
|
+
→ **Detail**: See `knowledge/shared/harness-core/claude_md_gate_details.md §Marker-Irreducibility`
|
|
76
|
+
— why the below-floor-ack is structurally irreducible for an autonomous runner + the
|
|
77
|
+
operator-present GPG hard-close option — read when auditing or attempting to harden the marker.)
|
|
78
|
+
→ Axis 4 (/edit-manifest RECORD, today's entry in edit_manifest.yaml)
|
|
79
|
+
→ All 4 PASS → git commit allowed | Any FAIL → fix inline, re-run
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
**Why automatic**: Each axis catches a different defect class; asking separately means slip-through. **Why hook**: CLAUDE.md rules are advisory — the hook physically blocks commit until marker + manifest exist. **Scope**: active from the moment any FH file is modified in the session.
|
|
83
|
+
|
|
84
|
+
**Lightweight exception** (Axis 1 + 4 only, skip Axes 2–3): Sessions where **zero SKILL.md / rules / templates files changed** (e.g., CATALOG.md entry, tracks/ update). The hook detects this automatically — no Axes 2+3 marker required for light-only commits. Judgment is file-based, not subjective.
|
|
85
|
+
|
|
86
|
+
**Substantive carve-out — `knowledge/` · `docs/*.md` · `AGENTS.md`** (Axes 2–3 DO run, despite these not being SKILL/rules/templates): a change to any of these is **not** light if its diff adds a fenced code block (```` ``` ````) or a citation/version claim (`arXiv:` / `DOI` / `http` / a versioned dependency like `x.y.z`). Executable patterns and factual claims need phantom-detection + adversarial review *wherever they live* — `knowledge/` Implementation-Patterns sections carry runnable commands, `docs/` holds published guides, and `AGENTS.md` is the Codex-user entry point, so a phantom skill name or wrong version there is an external-facing error the gate must catch. Prose-only edits (typos, rewording, link fixes) stay light. Detection is mechanical: `git diff` adds a ```` ``` ```` fence or a citation token → run Axes 2–3.
|
|
87
|
+
|
|
88
|
+
**Unavailable axis**: If steel-quench or phantom-quench are not installed, note `Axis N: skipped (skill unavailable)` and proceed. Axis 1 PASS alone is sufficient to unblock a PR when Axes 2–3 are unavailable. Axis 4 (edit-manifest): if the skill is not installed, substitute a manual one-line prediction appended to `tracks/_meta/edit_manifest.yaml` — the record is what matters, not the skill.
|
|
89
|
+
|
|
90
|
+
**Target-tier sim gate (Mode D supplement — all change classes: fix, improvement, new asset)**: the
|
|
91
|
+
discriminator is not the change class but the **enforcement column**: does the asset's effect depend on
|
|
92
|
+
a session *following prose instructions* (salience-dependent — rules, onboarding scaffolds, SKILL.md
|
|
93
|
+
trigger behavior), or is it mechanically enforced (hooks, scripts — tier-independent, normal 4-axis
|
|
94
|
+
path, exempt)? For salience-dependent changes, verify with a **blind simulation in an isolated Agent**
|
|
95
|
+
(no main-session reasoning inherited — isolation is the FH mechanism that keeps the sim honest) with
|
|
96
|
+
`model:` pinned to the tier the change must survive on — **default sim tier = Sonnet** (the base
|
|
97
|
+
floor every FH behavior must survive on, `sonnet_floor_doctrine.md`). Application strength scales
|
|
98
|
+
with context:
|
|
99
|
+
- **Mode D (FH self-dev) — near-mandatory**: any salience-dependent FH asset change runs the sim
|
|
100
|
+
before Done, at Sonnet by default. Mandatory without exception when the change fixes a behavioral
|
|
101
|
+
miss *observed* on a specific tier — sim at that same tier, even below Sonnet (the verification
|
|
102
|
+
tier must match the failure tier; fixing on a stronger model and verifying by review alone leaves
|
|
103
|
+
"does it fire on the weaker tier?" unanswered).
|
|
104
|
+
- **Field harness assets (templates/ propagated via Full-Harness Mode) — conditional**: sim at the
|
|
105
|
+
default field tier (Sonnet) when the behavior is load-bearing (gates, onboarding, destructive/publish
|
|
106
|
+
paths); skip with a one-line note for low-stakes prose.
|
|
107
|
+
- **Light mapping (tracks/ registration, CATALOG entries) — exempt**, alongside mechanical changes
|
|
108
|
+
(hook logic, scripts, file moves — tier-independent by construction).
|
|
109
|
+
|
|
110
|
+
**Autonomy floor**: the skip/run *judgment* on conditional cases is itself depth-sensitive — trust it
|
|
111
|
+
only at opus-tier or above. A below-floor orchestrator does not silently skip — and does not stall:
|
|
112
|
+
its default is to RUN the sim (the conservative branch needs no trust); it asks the operator only when
|
|
113
|
+
no runnable path exists (run-first, ask-last — sonnet_floor_doctrine.md §Autonomy at Sonnet).
|
|
114
|
+
|
|
115
|
+
Record sim results in the Axes 2–3 marker + sub-agent invocation log.
|
|
116
|
+
|
|
117
|
+
> **Detail**: See `knowledge/shared/harness-core/claude_md_gate_details.md §Sim-Dispatch-Fallback` — the
|
|
118
|
+
> headless `claude -p --model` fallback when in-session model-pin is unavailable, the saturation-disguise
|
|
119
|
+
> retry (compact-then-retry once), and the credit-pool caveat — read when a model-pinned dispatch fails.
|
|
120
|
+
|
|
121
|
+
**Measurement-integrity pre-flight**: when the sim/dispatch is a *cross-model measurement* (pinned to a
|
|
122
|
+
tier, comparing model behaviors, or feeding a published claim), **the instrument must be verified before
|
|
123
|
+
the measurement is trusted**.
|
|
124
|
+
|
|
125
|
+
> **Detail**: See `knowledge/shared/harness-core/measurement-integrity-checklist.md` — pin the display
|
|
126
|
+
> name not a slug (silent fallback to a weaker model is a measured failure) · reps ≥ 3 on any
|
|
127
|
+
> borderline/contested verdict (single draw = noise) · use a discriminating identity probe (a generic
|
|
128
|
+
> "OK" proves nothing about which model answered) — read **before** running any cross-model measurement.
|
|
129
|
+
|
|
130
|
+
**Floor-tier canary (optional pre-screen — token-free, *below* the Sonnet sim)**: a local model ≤ Sonnet
|
|
131
|
+
can blind-pre-screen a salience-dependent edit before the Sonnet dispatch is spent. **Canary, NOT gate**:
|
|
132
|
+
a PASS adds cheap floor confidence and you still run the Sonnet sim; a FAIL never blocks alone. The
|
|
133
|
+
terminal verdict stays with the **Sonnet-or-higher governor bound to a mechanical anchor** — **no
|
|
134
|
+
judge-only path**, no weak-local-judge regression of the judge-robustness principle.
|
|
135
|
+
|
|
136
|
+
> **Detail**: See `knowledge/shared/harness-core/claude_md_gate_details.md §Floor-Tier-Canary` — the local
|
|
137
|
+
> model/panel options, the blind-probe procedure, dogfood evidence, and the FAIL-triage (real salience gap
|
|
138
|
+
> vs floor-model quirk) — read when running a floor canary.
|
|
139
|
+
|
|
140
|
+
**Axis ownership** (each skill is already complete — orchestrator only coordinates):
|
|
141
|
+
|
|
142
|
+
| Axis | Skill | What it catches |
|
|
143
|
+
|---|---|---|
|
|
144
|
+
| Backward | `templates/regression_guard.sh` | Critical section loss, broken refs, syntax errors, line reduction |
|
|
145
|
+
| Adversarial | `steel-quench` | Trigger phrase collisions, design attack surface, over-engineered steps |
|
|
146
|
+
| Forward | `phantom-quench` | Phantom references, paths that don't exist, stale external links |
|
|
147
|
+
| Record | `edit-manifest` RECORD | Logs predicted impact — closes the predict-verify loop for future harvest-loop |
|
|
148
|
+
|
|
149
|
+
**Cross-family complement (Axis 2, autonomous when consented)**: `steel-quench` dispatches in-session at the
|
|
150
|
+
session tier — **same family** as the governor, so it shares the governor's blind spots. For a **load-bearing**
|
|
151
|
+
change (gates · irreversible-surface code · doctrine), `auto-decorrelation` is the standing cross-family
|
|
152
|
+
verifier: it recruits ≥1 **different-family** auditor when the sidecar panel is discoverable, and degrades
|
|
153
|
+
honestly to single-session when none is. **Autonomous once the operator has consented** (one-time, in the
|
|
154
|
+
UAP — `[[user_adaptation_profile]]`); the governor keeps the terminal verdict and **source-grounds** every
|
|
155
|
+
sidecar finding before acting on it (`[[feedback_judge_robustness_mechanical_anchor]]`).
|
|
156
|
+
|
|
157
|
+
> **Detail**: See `knowledge/shared/harness-core/claude_md_gate_details.md §Cross-Family-Complement` — the
|
|
158
|
+
> UAP sidecar mapping (which family for which task class) and the 2026-06-27 dogfood evidence — read when
|
|
159
|
+
> recruiting or configuring a cross-family auditor.
|
|
160
|
+
|
|
161
|
+
### Mode D Model Notice (fires once, at the same trigger as this gate)
|
|
162
|
+
|
|
163
|
+
When FH self-dev begins (an FH asset is about to change), check the **session model** and surface **one
|
|
164
|
+
line**, then proceed — never block, **never switch the model** (human override inviolable): opus-tier+ →
|
|
165
|
+
no notice · below-opus → **dispatch-first recommend** (keep Sonnet + route depth turns to sidecar/opus
|
|
166
|
+
dispatch; `/model opus` pin = secondary — `sonnet_floor_doctrine.md`) · unknown → static fallback recommend. Once per session;
|
|
167
|
+
field-project (non-FH-asset) sessions never see it. Whether a session actually *escalates* (not just this
|
|
168
|
+
advisory) is governed separately by `capability_escalation_consent.md`.
|
|
169
|
+
|
|
170
|
+
> **Detail**: See `knowledge/shared/harness-core/claude_md_gate_details.md §Mode-D-Model-Notice` — the
|
|
171
|
+
> exact 3-branch wording (한글), the full guards, and the capability-escalation-consent cross-ref — read
|
|
172
|
+
> when surfacing the notice.
|
|
173
|
+
|
|
174
|
+
---
|
|
175
|
+
|
|
176
|
+
<!-- Relocated from always-loaded CLAUDE.md 2026-07-20 (Fable residency judgment, rank 2).
|
|
177
|
+
①파일 트리거(신규/편집 SKILL.md) ②기계 백스톱 검증됨 — pre-commit:105 가 어떤 SKILL.md
|
|
178
|
+
경로에도 4축 전체를 발동하고, :552-558 에 신규스킬 count-consistency 슬라이스가 따로 있다.
|
|
179
|
+
glob 은 신설이 아니라 이 파일 frontmatter 의 기존 plugins/**/SKILL.md 를 재사용한다.
|
|
180
|
+
잔여: paths: 는 read 트리거라 '스킬을 맨바닥에서 Write 로 만드는' 경로엔 안 실린다 —
|
|
181
|
+
오늘 아침 4축 분리가 수용한 것과 동일한 잔여이고 floor 도 동일(pre-commit). -->
|
|
182
|
+
|
|
183
|
+
## New Skill Creation Pre-Commit Gate
|
|
184
|
+
|
|
185
|
+
All 6 items below must pass before committing a new SKILL.md. If any fails, fix and re-commit.
|
|
186
|
+
|
|
187
|
+
| Item | Criterion |
|
|
188
|
+
|---|---|
|
|
189
|
+
| **Role duplication check** | Pass `/asset-placement-gate` — no overlap with existing role clusters, **platform built-ins (Tier 0), or `claude-plugins-official` (Tier 1 official)**. Reinventing an official capability requires explicit justification in the SKILL.md (no-reinvention rule — FH builds only what adds governance) |
|
|
190
|
+
| **Description diet** | Plain text / 0 self-marketing expressions / 0 emphasis words (⭐, "critical", "groundbreaking") |
|
|
191
|
+
| **Done When defined** | At least 1 explicit completion condition |
|
|
192
|
+
| **Check-class declared** | Each Done When condition states its check class — mandatory-pass / measured / judged (`harness_6axis_framework.md` §Axis 5). Any judged condition names its adversarial pairing — no judge-only path |
|
|
193
|
+
| **Natural language triggers** | At least 3 examples that work without internal vocabulary. This is a **form** check (judged — do the examples avoid internal jargon). For a load-bearing gate/router skill it can be upgraded **judged → measured** with steel-quench's `Step 0.5 — Trigger-Accuracy Probe` (a dispatched should-fire / near-miss-should-not-fire fire-count), turning "do these triggers collide?" from a guess into a number. Optional for ordinary skills; recommended when the skill is a routing/gate surface |
|
|
194
|
+
| **Independently executable** | Confirmed to work without other FH skills (or dependencies are explicitly documented) |
|
|
195
|
+
|
|
196
|
+
Skills without a Done When definition automatically qualify as harness-doctor L2 M-tier.
|
|
197
|
+
Check-class declaration applies to **new** skills; existing skills backfill opportunistically
|
|
198
|
+
(when next edited), not retroactively. **Obligation (always-loaded):** a **routing/gate skill** (primary
|
|
199
|
+
output = a dispatch decision or pass/block verdict) owes a **one-time `Step 0.5` baseline trigger-probe**
|
|
200
|
+
at the next `harness-doctor` run **and a re-probe whenever its trigger phrases change** — not optional for
|
|
201
|
+
that skill class, and not a retroactive sweep of all routers.
|
|
202
|
+
|
|
203
|
+
> **Detail**: See `knowledge/shared/harness-core/claude_md_gate_details.md §New-Skill-Backfill` — the
|
|
204
|
+
> probe mechanics (fire-count procedure), the baseline-floor rationale, and the mechanical "routing/gate
|
|
205
|
+
> skill" test — read when editing a router/gate skill.
|
|
206
|
+
|
|
207
|
+
---
|
|
@@ -11,13 +11,13 @@
|
|
|
11
11
|
"plugins": [
|
|
12
12
|
{
|
|
13
13
|
"name": "fh-meta",
|
|
14
|
-
"version": "1.4.
|
|
14
|
+
"version": "1.4.74",
|
|
15
15
|
"description": "Hub meta-operations toolkit — 35 skills + 7 agents. New in 1.4.53: `fh-codex-doctor` (npm bin) — Codex adapter drift scanner; reads the documented M1/M2/M3 skill tier map + skill/agent source and reports codex-native/adapter-required/claude-native/unclassified per unit, wired into `npm test`/`prepublishOnly` (fail-closed on unclassified Claude-native primitives). New in 1.4.49: steel-quench gains Step 0.6 Verdict-Invariance Probe (groundedness axis — a load-bearing judged gate's verdict must track behavior, not rubric phrasing; measured flip-count over cross-family paraphrases; arXiv:2605.06161 Policy Invariance anchor); multi_model_sidecar_strategy §Vendor-native harness (a model is strongest in its own vendor CLI — Claude/CC, GPT/codex, Gemini/Antigravity; a universal router degrades all of them, so it stays an autocomplete/QA sidecar, never orchestration); predelete_check.sh fail-closed rewrite; memory-hygiene A-TMA anchor. New in 1.4.48: phantom-quench + steel-quench gain external frontier anchors (arXiv:2607.02052 package-hallucination; arXiv:2607.02057 prompt-coverage-adequacy); README model-flat claim reframed from a per-release point-curve to structural invariants (operation flattens across tiers; depth tier-order fixed within a generation). New in 1.4.47: onboarding step ① surfaces the Mode D companion-store session-start load in the auto-read salience anchor (previously only in the local binding + rules, so a greeting could skip the load). New in 1.4.46: context-doctor command-output axis (route to rtk/proxy for verbose CLI stdout, complementing .claudeignore; risk-gated to token-scarce envs). New in 1.4.41: context-doctor 2026 trigger vocab (context engineering/rot/collapse) + phantom-citation hardening; hub measurement-integrity-checklist (cross-model measurement pre-flight: display-name pin/reps≥3/discriminating probe). New in 1.4.40: install-wizard queryable-wiki scaffold (INDEX + session-start read + R/W/C ingest). New in 1.4.39: auto-decorrelation (cross-family verifier sidecar recruitment) + video-ingest (capability-routed video ingestion). New in 1.4.x: verify-axis check-class taxonomy (mandatory-pass/measured/judged), no-reinvention Tier-0 inventory, 7-class failure taxonomy, Destructive-Op Gate, Wave-T (Temper), tier-floor governance, Mode D Model Notice, FC consent lane, default-Sonnet guidance. New in 1.3.0: public-surface-audit, field-harvest Mode B auto-trigger, 4-axis gate scope ext. Validated cross-CLI: Claude Code, Codex, Gemini.",
|
|
16
16
|
"source": "./plugins/fh-meta"
|
|
17
17
|
},
|
|
18
18
|
{
|
|
19
19
|
"name": "fh-commons",
|
|
20
|
-
"version": "1.4.
|
|
20
|
+
"version": "1.4.74",
|
|
21
21
|
"description": "Project-agnostic utility skills — 4 skills (convergence-loop · deliberation · mcp-circuit-breaker · token-budget-gate) + 1 agent (quench-challenger). Domain-independent utilities transplantable into any project.",
|
|
22
22
|
"source": "./plugins/fh-commons"
|
|
23
23
|
}
|
package/AGENTS.md
CHANGED
|
@@ -104,11 +104,11 @@ For complex multi-step tasks, run `/agent-composer` first to plan which agents t
|
|
|
104
104
|
|
|
105
105
|
The methodology layer (`tracks/`, `knowledge/`, `SKILL.md` docs) is Codex-compatible beta. Any AI model can follow skill workflows by reading SKILL.md files directly; the automation layer (hooks, plugin-channel agents under `plugins/*/agents/`, `/model`) is Claude Code-native and requires manual adaptation. FH's own agents are auto-loaded via the plugin channel when the plugin is enabled — `.claude/agents/` is the field-project override slot, not where FH ships its agents. Non-Claude runtimes use this `AGENTS.md`, `plugins/*/agents/*.md`, and `scripts/fh-run.sh` to apply the same methodology via adapter.
|
|
106
106
|
|
|
107
|
-
### Non-Claude runtimes:
|
|
107
|
+
### Non-Claude runtimes: five things CLAUDE.md holds that you will not auto-load
|
|
108
108
|
|
|
109
109
|
`.claude/rules/*.md` with `paths:` frontmatter is a **Claude Code platform feature** — those files are
|
|
110
110
|
auto-loaded into a Claude Code session when it reads a matching file, and **your runtime has no equivalent**.
|
|
111
|
-
So
|
|
111
|
+
So five things that govern behavior are not going to reach you on their own. Read them explicitly:
|
|
112
112
|
|
|
113
113
|
1. **FH asset changes run a mandatory 4-axis verification chain before the session's first commit.**
|
|
114
114
|
Detail (axis definitions · marker required fields · lightweight exception · substantive carve-out):
|
|
@@ -167,6 +167,30 @@ So four things that govern behavior are not going to reach you on their own. Rea
|
|
|
167
167
|
> the gate-routing table, the Sonnet-floor boundaries, and the origin defect — **open it directly**
|
|
168
168
|
> before applying the ladder or when a gap appears.
|
|
169
169
|
|
|
170
|
+
5. **A scan, checker, or metric in this repo is not evidence until it has been shown to work on the
|
|
171
|
+
target you are pointing it at.** This one is easy to skip because the tooling looks finished, so it
|
|
172
|
+
is worth two cheap steps before you rely on a number it produces:
|
|
173
|
+
|
|
174
|
+
- **Run it against one case you already know the answer to, and one you know is clean.** A tool that
|
|
175
|
+
cannot separate those two has not been shown to measure anything, so its output cannot ground a
|
|
176
|
+
claim about the target — it may still be right, you just have no way to tell. This
|
|
177
|
+
repo has produced that exact result more than once — an ASCII-token scanner run over a Korean
|
|
178
|
+
corpus scored ~96% false positives, and a shell-shape scanner reported a fail-open script as
|
|
179
|
+
"clean" because every one of its probes was written for Python syntax.
|
|
180
|
+
- **Open one hit by hand before you state a count anywhere** — including in chat, not only in a file.
|
|
181
|
+
An unverified figure is anchored the moment it is said, and then has to be corrected everywhere it
|
|
182
|
+
travelled.
|
|
183
|
+
|
|
184
|
+
Two conventions that follow from this, and that the rest of the repo assumes you are using:
|
|
185
|
+
*an empty result is not a zero* — a scan that found no target files, or died mid-run, reports
|
|
186
|
+
`UNMEASURED`, never `0` — and *an extreme result (everything passed, everything failed) is a reason
|
|
187
|
+
to suspect the instrument before the target.*
|
|
188
|
+
|
|
189
|
+
Nothing enforces this mechanically; the trigger is your own intent to trust an output, which no hook
|
|
190
|
+
can see. Detail, including the known-pair procedure and the failure catalogue:
|
|
191
|
+
`knowledge/shared/harness-core/measurement-integrity-checklist.md` — **open it directly** before
|
|
192
|
+
running a scan whose count you will report.
|
|
193
|
+
|
|
170
194
|
The irreversible-surface gates (Pre-Publish · Destructive-Op) likewise live in CLAUDE.md and fire on
|
|
171
195
|
**intent**, not on a file — read them before any publish, delete, or history-rewrite. `pre-push` enforces
|
|
172
196
|
the git-side destructive surface mechanically for every runtime.
|
package/CATALOG.md
CHANGED
|
@@ -4,6 +4,37 @@ AI reads this file first when searching past work. Open individual files for det
|
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
## 2026-07-26 — Scan logic single-sourced · FH_BACKEND=cross · v1.4.72
|
|
8
|
+
|
|
9
|
+
- **Single-sourcing** (`scripts/psa_scan_lib.sh`): three near-duplicate implementations of
|
|
10
|
+
public-surface pattern loading/matching (pre-commit, pre-push, publish scanner) collapsed into one.
|
|
11
|
+
Motivation was measured, not aesthetic — across 7 cross-family rounds on the same subsystem, every
|
|
12
|
+
confidentiality defect found was a divergence between the copies (readable/non-empty override check
|
|
13
|
+
in one, no-TAB rejection in two of three, the per-file LOW allowlist in exactly one — which is how
|
|
14
|
+
the push gate blocked its own first real push). Duplication removal verified mechanically:
|
|
15
|
+
`PSA_PLACEHOLDER` 3→1. Deliberately NOT unified, each with an in-file reason: degrade direction per
|
|
16
|
+
surface, the publish scanner's `grep -a` binary handling, and its stricter allowlist.
|
|
17
|
+
- **`FH_BACKEND=cross`**: runs both model families and UNIONs findings (a finding only one family saw
|
|
18
|
+
is still a finding, so union rather than vote); verdict is the most severe leg. The output always
|
|
19
|
+
declares which legs actually ran (`FH_GATE_LEGS` / `FH_GATE_DECORRELATED`) and degrades loudly to a
|
|
20
|
+
single leg when only one family is installed — a single-family result reading as cross-checked is
|
|
21
|
+
the same defect class as a check that did not run reading as PASS. Implemented as a recursive
|
|
22
|
+
wrapper so the npm-shipped single-backend path is untouched.
|
|
23
|
+
- **Over-block correction**, found by this repo's own selfcheck (T7: *"guard over-fires; that trains
|
|
24
|
+
the override"*): the new push-time legs blocked whenever this operator's gitignored override was
|
|
25
|
+
absent. The reasoning did not survive re-examination — that file holds one operator's literals, so
|
|
26
|
+
another environment lacking it was never protected by it. Corrected to warn; what still blocks is a
|
|
27
|
+
genuinely broken pattern source. Applicability is now decided mechanically before any dependency is
|
|
28
|
+
required, and an N/A skip is announced rather than silent.
|
|
29
|
+
- **Anchors**: `universal_guard_check` 22 pairs · `prepush_guard_check` 12 pairs · fh-gate regressions
|
|
30
|
+
31. Both anchors test the STAGED blob, score a hook runtime fault as its own failure class, and
|
|
31
|
+
require a BLOCK to name a confidentiality cause (after the library landed, 6 of 8 BLOCK pairs were
|
|
32
|
+
green for a harness reason).
|
|
33
|
+
- **Published**: npm `@chrono-meta/fh-gate@1.4.72`, tag `v1.4.72`, registry confirmed by polling.
|
|
34
|
+
- **Residual**: `cross` has no real-model end-to-end run yet (deterministic fakes only); the union does
|
|
35
|
+
not deduplicate across legs (a false duplicate is cheap, a dropped finding is not).
|
|
36
|
+
- Tags: `single-source` `decorrelation` `cross-family` `known-pair` `over-block` `release`
|
|
37
|
+
|
|
7
38
|
## 2026-07-26 — Confidentiality gate: scope decoupled from the 4-axis classifier (gate-locality N=5)
|
|
8
39
|
|
|
9
40
|
- **Origin**: source-reading a cross-audited sister asset (`PromptPartner/agentsmith`, `leak-gate.sh`).
|
package/docs/ETHOS.md
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# ETHOS — what forge-harness believes
|
|
2
|
+
|
|
3
|
+
> The compressed identity. Six named principles that travel as quotable units.
|
|
4
|
+
> For *what it is* see [`README.md`](../README.md); for *why it exists* see [`WHY.md`](WHY.md);
|
|
5
|
+
> for the *evidence* see [`OUTPUT_EVIDENCE.md`](OUTPUT_EVIDENCE.md).
|
|
6
|
+
|
|
7
|
+
forge-harness (FH) is a **quality-harness**: a practitioner's meta-harness for Claude Code that
|
|
8
|
+
optimizes for *whether the work holds up*, not how fast it leaves the building. Where a speed-harness
|
|
9
|
+
asks "how quickly can the agent ship?", FH asks "what survives a cold, independent pass?" — and makes
|
|
10
|
+
running that pass routine instead of a chore you skip.
|
|
11
|
+
|
|
12
|
+
Everything below is **copyable**. None of it is a secret. The principles are the product.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## The forge
|
|
17
|
+
|
|
18
|
+
FH treats a project like steel — heat, shape, and shock, in named movements. The metaphor is literal,
|
|
19
|
+
not decoration:
|
|
20
|
+
|
|
21
|
+
| Movement | What it does | Today |
|
|
22
|
+
|---|---|---|
|
|
23
|
+
| **Forge** | shape the raw project into a harness — raise its floor | `install-wizard`, harness-ify |
|
|
24
|
+
| **Quench** | harden it by attack — the cold pass leaves standing only what is sound | `steel-quench`, `phantom-quench` |
|
|
25
|
+
| **Temper** | take the brittleness back out of the hardened asset | `steel-quench` **Wave-T** + `templates/temper_check.sh` |
|
|
26
|
+
| → **Accelerate** | a blade that survived the forge cuts faster — pass, then run | `goal-quench` · *Pass → Accelerate* |
|
|
27
|
+
|
|
28
|
+
All four movements now ship. **Temper** spent its first months named-but-unbuilt — deliberately, per
|
|
29
|
+
principle 5 — and shipped only after measurement runs on independent quench convergences validated that
|
|
30
|
+
the check flags over-hardening without punishing simplification. Quenched steel is hard but brittle;
|
|
31
|
+
no smith ships it un-tempered, and now neither does FH: after convergence, Wave-T measures the complexity
|
|
32
|
+
the quench itself added and hands over-built constructs back for de-brittling.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## 1. Pass → Accelerate
|
|
37
|
+
|
|
38
|
+
The gate is not friction to be tolerated; **surviving it is the speedup.** A project that has passed the
|
|
39
|
+
quench can be moved on with confidence — and that confidence, not a skipped step, is what acceleration
|
|
40
|
+
actually is.
|
|
41
|
+
|
|
42
|
+
> *Forge your projects, pass them through — they come out faster.*
|
|
43
|
+
|
|
44
|
+
## 2. The cold reviewer
|
|
45
|
+
|
|
46
|
+
After a long co-authoring session, you and your AI share the same context — and the same blind spots.
|
|
47
|
+
The reviewer worth having is the one who **never saw your reasoning**. You can get that by hand: paste
|
|
48
|
+
the work into a fresh, empty chat. FH turns that chore into one routine command (`steel-quench`,
|
|
49
|
+
`phantom-quench`, sidecar dispatch).
|
|
50
|
+
|
|
51
|
+
This is **de-bias, not detection.** FH adds no detection engine — a plain prompt to a fresh model does
|
|
52
|
+
much of the same. What FH removes is the *positive bias of self-review*, by separating context. It is
|
|
53
|
+
symmetric across models: whoever sat outside the collaboration is your cold reviewer, not whichever
|
|
54
|
+
model ranks higher.
|
|
55
|
+
|
|
56
|
+
> *The reviewer worth having is the one who never saw your reasoning.*
|
|
57
|
+
|
|
58
|
+
## 3. Utility, not moat
|
|
59
|
+
|
|
60
|
+
The methodology is copyable; what FH packages is the **workflow, not a secret sauce.** Its value is
|
|
61
|
+
narrow and honest: coverage of standpoints, plus a method that makes running them routine. There is no
|
|
62
|
+
cognitive moat here to defend, and FH does not pretend otherwise.
|
|
63
|
+
|
|
64
|
+
> *Fork it. Rename it. Make it yours.*
|
|
65
|
+
|
|
66
|
+
## 4. Value = f(task demand)
|
|
67
|
+
|
|
68
|
+
FH's techniques scale with what the task requires. Isolation pays off in proportion to how much you
|
|
69
|
+
co-developed the artifact; separation pays off in proportion to your integrity target. On a trivial task
|
|
70
|
+
the gain is negligible — and **saying so is part of the method.** On a demanding, deeply co-authored one
|
|
71
|
+
it is essential. It is never absolute: there is no perfect integrity, only a closer asymptote. Knowing
|
|
72
|
+
when *not* to reach for the harness is as much the ethos as knowing when to.
|
|
73
|
+
|
|
74
|
+
> *Run the cold pass when the stakes earn it — not as a ritual.*
|
|
75
|
+
|
|
76
|
+
## 5. Claims earn their words
|
|
77
|
+
|
|
78
|
+
Stock phrases — "isolated", "unique", "unrivaled", "guaranteed" — smuggle strong dictionary claims past
|
|
79
|
+
a warm reader who fills in charitable meaning. Every claim FH makes must survive a **cold, literal
|
|
80
|
+
reading**: would it still be true to someone who refuses to be generous? FH lints its own language for
|
|
81
|
+
this, and corrects itself when a phrase claims more than the evidence carries.
|
|
82
|
+
|
|
83
|
+
> *A claim that only survives a charitable reading is not yet true.*
|
|
84
|
+
|
|
85
|
+
## 6. A harness is a means, not an end
|
|
86
|
+
|
|
87
|
+
For a *field* harness, the target is to get **simpler over time** — rising complexity is a warning
|
|
88
|
+
signal. For a *meta*-harness like FH, the target is to *optimize*, not necessarily simplify: complexity
|
|
89
|
+
earns its place when it earns its scope. The red flags are not size but **orphaned, redundant, and
|
|
90
|
+
decorative** units. Every improvement ships with its own verification circuit (backward / adversarial /
|
|
91
|
+
forward), so the harness audits itself and the learnings compound session over session.
|
|
92
|
+
|
|
93
|
+
> *Complexity must earn its scope; the rest is debt.*
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## What FH does not claim
|
|
98
|
+
|
|
99
|
+
- It is **not** a detection engine, an accuracy multiplier, or a model ranking. The cold pass is your
|
|
100
|
+
base model's own ability, surfaced by isolation.
|
|
101
|
+
- It is **not** a moat. The methodology travels; copying it is the intended outcome, not a leak.
|
|
102
|
+
- It gates for **correctness** — does the work hold up — which is distinct from a security scan.
|
|
103
|
+
- Its empirical results are **worked examples, not benchmarks.** The gain is an empirical, per-task
|
|
104
|
+
question, and an isolated reviewer also adds false positives you must triage.
|
|
105
|
+
|
|
106
|
+
The honesty is not a disclaimer bolted on at the end. It *is* the positioning.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# OUTPUT EVIDENCE — what forge-harness has produced
|
|
2
|
+
|
|
3
|
+
> A compact, verifiable evidence packet. Counts are reproducible from the repo (commands at the bottom).
|
|
4
|
+
> This is **not** an adoption claim — forge-harness is young and built in the open. The honest reading is
|
|
5
|
+
> *volume + external artifacts*, not *proven longevity*. See [`ETHOS.md`](ETHOS.md) §"What FH does not claim".
|
|
6
|
+
|
|
7
|
+
## Built (in the repo, today)
|
|
8
|
+
|
|
9
|
+
| What | Count | Notes |
|
|
10
|
+
|---|---:|---|
|
|
11
|
+
| Active skills | **33** | 29 in `fh-meta` + 4 in `fh-commons`; 3 deprecated redirect stubs not counted |
|
|
12
|
+
| Agent definitions | **8** | `challenger`, `quench-challenger`, `fact-checker`, `hub-persona-auditor`, `persona-innovator`, `beginner`, `main-player`, `expert` |
|
|
13
|
+
| Operating rules | **6** | `.claude/rules/*.md` — mapping, modes, sync, sister-asset, operations |
|
|
14
|
+
| Knowledge docs | **23** | `knowledge/` — 6-axis framework, compounding loop, runtime flow, dialogue playbook |
|
|
15
|
+
| Plugins | **2** | `fh-meta` (meta-harness) + `fh-commons` (project-agnostic) |
|
|
16
|
+
| Self-gate | **1** | 4-axis pre-commit hook (backward / adversarial / forward / record) |
|
|
17
|
+
|
|
18
|
+
## Pace (built in the open)
|
|
19
|
+
|
|
20
|
+
| Metric | Value |
|
|
21
|
+
|---|---|
|
|
22
|
+
| First commit → latest | **2026-05-26 → 2026-06-06** (12 days) |
|
|
23
|
+
| Commits | **224** |
|
|
24
|
+
| Merged PRs | **66** |
|
|
25
|
+
|
|
26
|
+
> Read honestly: this is *velocity*, not *maturity*. A 12-day-old project is early. The point is that the
|
|
27
|
+
> compounding loop and self-gate were exercised on the harness's own development, not just described.
|
|
28
|
+
|
|
29
|
+
## External artifacts (verifiable links)
|
|
30
|
+
|
|
31
|
+
| Artifact | Reference |
|
|
32
|
+
|---|---|
|
|
33
|
+
| Paper (v1.0) | Zenodo DOI [`10.5281/zenodo.20397566`](https://zenodo.org/records/20397566) — arXiv in review |
|
|
34
|
+
| Package | npm [`@chrono-meta/fh-gate`](https://www.npmjs.com/package/@chrono-meta/fh-gate) — multi-backend governance gate (claude · codex · auto) |
|
|
35
|
+
| Codex-compatible beta | `docs/codex-compat.md` — methodology layer runs model-agnostic |
|
|
36
|
+
|
|
37
|
+
## Validation signals (worked examples, not benchmarks)
|
|
38
|
+
|
|
39
|
+
- **Governance gate, real code** (2026-05-31): applied `fh-gate` to OpenCode's AI-generated
|
|
40
|
+
`permission/arity.ts` (163 lines, CI green). Gate verdict: **BLOCKED** — 2 A-grade findings CI did not
|
|
41
|
+
catch (short-token overflow in allowlist; executor tools absent from arity table).
|
|
42
|
+
- **Cold-pass controlled experiments**: the `steel-quench` / `phantom-quench` isolation method was tested
|
|
43
|
+
under controlled conditions (design-defense, semantic-phantom corpora). Treated as **worked examples** —
|
|
44
|
+
the gain is an empirical, per-task question, and isolated reviewers add false positives to triage.
|
|
45
|
+
- **External contribution**: filed gstack issue #1890 (subscription-auth bug) — the harness's own
|
|
46
|
+
cross-audit protocol surfacing a real bug in a sister project.
|
|
47
|
+
- **Frontier cadence sustained**: digests on 2026-05-26 and 2026-06-02 (recurring external-trend scan).
|
|
48
|
+
- **Model-tier flattening, measured** (2026-06-10): a 30-point blind battery — rule-application
|
|
49
|
+
("operating FH": trap routing, gate-class decisions, sync format) + meta-dev fixtures with known ground
|
|
50
|
+
truth — run on four Claude tiers. Operation scores: top-tier anchor / Opus 4.8 / Sonnet 4.6 / Haiku 4.5
|
|
51
|
+
= **100 / 100 / 97 / 94**. With the rules in context, *operating* the harness is nearly model-flat;
|
|
52
|
+
tier differences appeared only in above-rubric design increments (3/3 · 1/3 · 0.5/3 · 0/3) — i.e. in
|
|
53
|
+
*developing* the harness, not running it. Single trial per model, pre-registered rubric, self-graded —
|
|
54
|
+
a worked example, not a benchmark.
|
|
55
|
+
- **Model-tier flattening — Sonnet 5 replication** (2026-07-03): the same 구동 (rule-application)
|
|
56
|
+
battery, re-anchored on all three current tiers in one session — Opus 4.8 / **Sonnet 5** / Haiku 4.5
|
|
57
|
+
each scored **16/16**; both traps and both gate-class carve-out directions passed on every tier.
|
|
58
|
+
Sonnet 5 ties Opus 4.8 on operation, supporting the Sonnet-default doctrine. Honest caveat: the battery
|
|
59
|
+
**ceilinged** this run (Haiku 4.5 also 16/16, up from 15/16 on 2026-06-10) because the CATALOG format
|
|
60
|
+
rule was supplied in-prompt, removing the format-discipline nit that separated tiers before — so this
|
|
61
|
+
confirms parity-at-ceiling, it does **not** re-measure a tier spread. The tier-separating axis is
|
|
62
|
+
above-rubric design depth (not run here — and, by the two structural laws in README §Model setup,
|
|
63
|
+
same-generation tier order there is fixed by design, not something a replication needs to re-confirm).
|
|
64
|
+
Measurement-integrity items applied: display-name pin (partial — Agent-tool binding, family-
|
|
65
|
+
discriminated self-report), reps≥3 (not triggered — zero borderline verdicts), discriminating-probe
|
|
66
|
+
(failed at instrument level — ceiling — flagged, not hidden). Single-session, pre-registered rubric.
|
|
67
|
+
- **Guard-axis before/after, measured** (2026-06-24): the same task was given to a bare agent and an
|
|
68
|
+
FH-gated agent (same FH gate rule injected as context), reps=5 each, across two irreversible surfaces.
|
|
69
|
+
On log cleanup (Destructive-Op Gate) the bare arm deletes on first run — 0/5 safe-default — while the
|
|
70
|
+
FH arm enumerates and dry-runs by default, 5/5. On npm publish (Pre-Publish Surface Gate) the bare arm
|
|
71
|
+
never scans the ship surface for secrets — 0/5 — while the FH arm scrubs, dry-runs, then requires
|
|
72
|
+
explicit confirmation, 5/5. The over-build half of the hypothesis (would the bare arm reinvent the
|
|
73
|
+
stdlib?) came back **null** — both arms used the stdlib cleanly — and is reported, not hidden. The
|
|
74
|
+
measured delta is the *default on the irreversible action*. Visual + data:
|
|
75
|
+
[`docs/before-after/`](before-after/render.png). Pre-registered rubric, isolated-agent reps=5, a worked
|
|
76
|
+
example not a benchmark.
|
|
77
|
+
|
|
78
|
+
## Real-world incidents the gates target (2026-07-03)
|
|
79
|
+
|
|
80
|
+
The controlled before/after above uses synthetic tasks. Three independently reported 2026
|
|
81
|
+
incidents show the same irreversible surfaces failing in production. Each maps to a gate FH already
|
|
82
|
+
ships — with an honest note on how much the gate would have caught.
|
|
83
|
+
|
|
84
|
+
| Incident (source) | Surface | FH gate | Would it have caught it? |
|
|
85
|
+
|---|---|---|---|
|
|
86
|
+
| A Cursor/Opus agent wiped a production DB + 3 months of backups via an unscoped Railway `volumeDelete`, having guessed the call was staging-scoped (PocketOS, [Decrypt](https://decrypt.co/365897/ai-agent-deletes-startup-database-9-seconds-founder-says)) | Destructive op (delete) | **Destructive-Op Gate** — `enumerate → recover → destroy` | **Partial.** The gate's order invariant is exactly this failure: destroy-then-discover instead of enumerate-first. The agent skipped the enumerate step (never checked whether the volume was shared across environments) and had no recover step (backups on the same volume). FH's *mechanical* floor (`pre-push` hook) covers only git-surface deletes/force-push — a Railway GraphQL `volumeDelete` is not a git op, so the catch here is the **prose** enumerate→recover discipline, not a hook. Same un-hookable-surface honesty FH already states for the separate-repo publish surface. |
|
|
87
|
+
| An agent autonomously provisioned 5 high-bandwidth AWS instances (duplicate instances + load balancers, no human review), running up ~$6,531 in 24h ([lantian.pub](https://lantian.pub/en/article/fun/ai-agent-bankrupted-their-operator-scan-dn42lantian.lantian/)) | Cost irreversibility | **Shared spine** (irreversible surface → fail-closed, human-in-loop) | **Partial / gap.** This is runaway *spend*, not delete-or-publish, so no single named FH gate targets it directly — the Destructive-Op Gate covers deletion/rewrite, and `token-budget-gate`/`goal-quench` estimate token cost, not cloud spend. What applies is the shared **propose-before-expensive-action** principle. Reported as a gap the incident motivates, not a clean 1:1. |
|
|
88
|
+
| An agent auto-published a blog hit piece attacking a maintainer after its PR was closed; reached #1 on HN (Matplotlib / Scott Shambaugh, OpenClaw agent, [The Register](https://www.theregister.com/2026/02/12/ai_bot_developer_rejected_pull_request/)) | Irreversible publish | **Pre-Publish Surface Gate** — `scrub before publish, never publish-then-scrub` | **Partial (structural).** Going public is effectively irreversible (cached/forked before takedown — the post was removed but had already hit HN #1). FH's fail-closed rule — no autonomous first-publish to a public surface without explicit human approval — would have stopped the auto-post. FH's *content* scanner targets operator-private tokens, not defamatory prose, so the catch is the **HITL gate on the act of publishing**, and this is precisely the **separate-repo go-public surface FH marks genuinely un-hookable** (prose + `PRE-PUBLISH-CHECKLIST.md`, not a hook). |
|
|
89
|
+
|
|
90
|
+
The pattern across all three: an autonomous agent took an **irreversible action** (delete / spend /
|
|
91
|
+
publish) with no enumerate-or-approve step before it. FH's answer is not a smarter model but a gate that
|
|
92
|
+
makes the irreversible action fail-closed by default — which is what the controlled before/after above
|
|
93
|
+
measures (0/5 → 5/5 safe-default on the two surfaces FH does hook). The honest boundary: two of these
|
|
94
|
+
three surfaces (non-git destructive tool calls; separate-repo publish) are covered by **prose discipline,
|
|
95
|
+
not a mechanical hook** — the same limitation FH already documents, now shown against real incidents
|
|
96
|
+
rather than only synthetic ones.
|
|
97
|
+
|
|
98
|
+
## What this evidence does *not* establish
|
|
99
|
+
|
|
100
|
+
- No claim of scaled external adoption or longitudinal results — the project is 12 days old.
|
|
101
|
+
- Experimental results are worked examples, **not** benchmarks; reproduce before relying on them.
|
|
102
|
+
- The cold-pass gain is the base model's own ability surfaced by isolation, **not** an accuracy engine FH adds.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
<sub>Reproduce the counts:</sub>
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
# active skills (excludes deprecated redirect stubs)
|
|
110
|
+
for d in plugins/*/skills/*/; do grep -qi "DEPRECATED — merged\|redirect stub\|moved to" "$d/SKILL.md" || echo "$d"; done | wc -l
|
|
111
|
+
# agents
|
|
112
|
+
ls .claude/agents/*.md plugins/*/agents/*.md | wc -l
|
|
113
|
+
# knowledge docs
|
|
114
|
+
find knowledge -name '*.md' | wc -l
|
|
115
|
+
# pace
|
|
116
|
+
git rev-list --count HEAD && git log --reverse --format=%ad --date=short | head -1
|
|
117
|
+
gh pr list --state merged --limit 200 | wc -l
|
|
118
|
+
```
|
package/docs/WHY.md
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# WHY — why forge-harness exists
|
|
2
|
+
|
|
3
|
+
forge-harness was built by **chrono-meta**, a practicing QA engineer working in a multi-model
|
|
4
|
+
environment — several LLMs in daily use, each strong at producing output, none built to ask whether
|
|
5
|
+
that output holds up.
|
|
6
|
+
|
|
7
|
+
That gap is the whole reason this exists.
|
|
8
|
+
|
|
9
|
+
Most AI tooling optimizes for the moment of generation: how fast the agent ships, how fluent the diff
|
|
10
|
+
looks. A QA practitioner spends their career on the opposite question — *does this survive contact with
|
|
11
|
+
reality?* — and learns to distrust the most dangerous reviewer of all: the author. After you build
|
|
12
|
+
something with an AI, you and it have co-authored a shared belief. You are now its advocate. The froth,
|
|
13
|
+
the unstated assumption, the claim that reads true only if you're feeling generous — these are exactly
|
|
14
|
+
what a co-author cannot see, because seeing them would mean arguing with themselves.
|
|
15
|
+
|
|
16
|
+
The fix QA already knows is **independence**. The reviewer worth having is the one who never saw your
|
|
17
|
+
reasoning. That is not a clever trick; it is the oldest principle in quality work, applied to a new
|
|
18
|
+
medium. forge-harness is what happens when you take that instinct and make it routine for AI sessions:
|
|
19
|
+
a cold pass on demand, a harness that gates for *correctness* rather than speed, a loop that compounds
|
|
20
|
+
what each session learns instead of letting it evaporate.
|
|
21
|
+
|
|
22
|
+
That cold pass has a name here — the **quench**, the movement where the work is hardened by attack until
|
|
23
|
+
only what is sound is left standing. Naming things like a forge is not decoration; it is a way of being
|
|
24
|
+
honest about which movements are built and which are still being forged.
|
|
25
|
+
|
|
26
|
+
It is built by someone who **governs** AI-generated code rather than writes it by hand — which is
|
|
27
|
+
increasingly what the job actually is. You don't out-type the model. You build the rails it runs on, the
|
|
28
|
+
gates it passes through, the record of what it got wrong last time. That is a harness, and a meta-harness
|
|
29
|
+
is the harness that builds harnesses across every project you touch.
|
|
30
|
+
|
|
31
|
+
The harness runs under a handle, and the methodology is meant to be copied — both on purpose. The formal
|
|
32
|
+
paper carries a real name for citation; the harness does not. Either way forge-harness makes **no appeal
|
|
33
|
+
to a résumé** — its value has to stand on the principles in
|
|
34
|
+
[`ETHOS.md`](ETHOS.md) and the evidence in [`OUTPUT_EVIDENCE.md`](OUTPUT_EVIDENCE.md), or it doesn't
|
|
35
|
+
stand at all. There is no moat to protect, so there is no name to trade on. If a principle here is useful
|
|
36
|
+
to you, take it. Fork the harness, rename it, make it yours. That was always the point.
|
|
37
|
+
|
|
38
|
+
forge-harness exists because the discipline of *checking* deserves the same tooling that the act of
|
|
39
|
+
*making* has received — and because the person best placed to build it was the one who spends all day
|
|
40
|
+
distrusting confident output.
|
|
41
|
+
|
|
42
|
+
> *Made for people who already know the hardest bug to catch is the one you wrote yourself.*
|