@chrono-meta/fh-gate 1.4.41 → 1.4.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +5 -3
- package/CATALOG.md +6 -0
- package/CLAUDE.md +65 -130
- package/docs/CONTRIBUTING.md +2 -2
- package/knowledge/shared/dialogue/ai_dialogue_playbook.md +137 -0
- package/knowledge/shared/dialogue/claude_code_runtime_flow.md +170 -0
- package/knowledge/shared/dialogue/memory_intent_recall.md +209 -0
- package/knowledge/shared/harness-core/claude_md_gate_details.md +170 -0
- package/knowledge/shared/harness-core/companion_store_pluggable_cross_audit_2026-06-11.md +118 -0
- package/knowledge/shared/harness-core/crucible_mode.md +112 -0
- package/knowledge/shared/harness-core/deep_research_capability_ladder.md +122 -0
- package/knowledge/shared/harness-core/fh_detail_protocols.md +163 -0
- package/knowledge/shared/harness-core/fh_ecosystem_positioning.md +147 -0
- package/knowledge/shared/harness-core/fh_opencode_governance_wrapper.md +163 -0
- package/knowledge/shared/harness-core/fh_synergy_playbook.md +217 -0
- package/knowledge/shared/harness-core/gate_locality_principle.md +57 -0
- package/knowledge/shared/harness-core/goal_quench_anthropic_issue.md +104 -0
- package/knowledge/shared/harness-core/harness_6axis_framework.md +136 -0
- package/knowledge/shared/harness-core/harness_design_decision_lens.md +108 -0
- package/knowledge/shared/harness-core/harness_frontier_diagnosis_2026-06-02.md +102 -0
- package/knowledge/shared/harness-core/hub_compounding_loop.md +109 -0
- package/knowledge/shared/harness-core/hub_maturity_roadmap.md +201 -0
- package/knowledge/shared/harness-core/hybrid_orchestration_architecture_roadmap.md +196 -0
- package/knowledge/shared/harness-core/live_surface_automation_pattern.md +110 -0
- package/knowledge/shared/harness-core/measurement-integrity-checklist.md +59 -0
- package/knowledge/shared/harness-core/meta_harness_engineering_definition.md +116 -0
- package/knowledge/shared/harness-core/multi_model_sidecar_strategy.md +651 -0
- package/knowledge/shared/harness-core/persona_container_schema.md +172 -0
- package/knowledge/shared/harness-core/return_path_gate.md +120 -0
- package/knowledge/shared/harness-core/self_evolution_routine.md +268 -0
- package/knowledge/shared/harness-core/skill_quality_rubric.md +71 -0
- package/knowledge/shared/harness-core/tpa_schema.md +136 -0
- package/package.json +3 -2
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: goal-quench-anthropic-issue
|
|
3
|
+
description: Draft Anthropic GitHub issue — native /goal token budget + quality verification hook. Reference for when arXiv number is confirmed.
|
|
4
|
+
type: reference
|
|
5
|
+
date: 2026-05-31
|
|
6
|
+
tags: [goal, anthropic, feature-request, token-budget, quality-gate]
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# [Feature Request] /goal — native token budget control + quality verification hook
|
|
10
|
+
|
|
11
|
+
## Summary
|
|
12
|
+
|
|
13
|
+
`/goal` is powerful but ships two structural gaps: no token budget enforcement and a binary completion evaluator (Haiku yes/no) with no quality signal. This issue proposes three native flags to close these gaps, with a userspace proof-of-concept as reference.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Problem
|
|
18
|
+
|
|
19
|
+
### 1. Token explosion with no mid-run intervention
|
|
20
|
+
|
|
21
|
+
`/goal` runs until Haiku says "done" or context is exhausted. There is no mechanism to:
|
|
22
|
+
- Set a token ceiling before the run
|
|
23
|
+
- Intervene at a percentage threshold (e.g., 80% consumed)
|
|
24
|
+
- Save and checkpoint progress when budget runs low
|
|
25
|
+
|
|
26
|
+
Real-world symptom: users report running `/goal "finish all of this"`, going to sleep, and waking to a fully exhausted context with no recovery path for incomplete work.
|
|
27
|
+
|
|
28
|
+
### 2. Completion ≠ quality
|
|
29
|
+
|
|
30
|
+
Haiku's binary evaluator (`done? yes/no`) checks whether the stated goal condition is satisfied — not whether the output is correct, well-structured, or regression-free.
|
|
31
|
+
|
|
32
|
+
A session can reach `done = yes` while having introduced bugs, phantom references, or broken existing behavior. There is no hook for a quality gate on the completion verdict.
|
|
33
|
+
|
|
34
|
+
### 3. No checkpoint / resume
|
|
35
|
+
|
|
36
|
+
When budget exhaustion forces a stop, there is no structured record of what was completed vs. what remains. The next session starts cold.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Proposed Native Flags
|
|
41
|
+
|
|
42
|
+
### `--budget <N>`
|
|
43
|
+
|
|
44
|
+
Enforce a token ceiling for the `/goal` session.
|
|
45
|
+
|
|
46
|
+
Behavior:
|
|
47
|
+
- At 70% of `N`: surface a warning to the user — "Budget at 70%. Recommend re-prioritizing remaining tasks."
|
|
48
|
+
- At 85% of `N`: pause the session. Prompt: "Budget at 85%. Options: (a) continue / (b) reduce scope / (c) stop and save."
|
|
49
|
+
- At 95% of `N`: force stop. Commit completed work. Output structured summary: `Completed: [...] | Remaining: [...]`
|
|
50
|
+
|
|
51
|
+
### `--verify <command>`
|
|
52
|
+
|
|
53
|
+
Run a shell command when Haiku returns `done = yes`. Accept the completion verdict only if the command exits 0.
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
# Example: test suite must pass before /goal accepts "done"
|
|
57
|
+
/goal "all tests pass" --verify "npm test"
|
|
58
|
+
|
|
59
|
+
# Example: FH quality gate before accepting completion
|
|
60
|
+
/goal "refactor complete" --verify "claude -p 'run pipeline-conductor --quick'"
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
This separates the concerns that a single evaluator cannot serve simultaneously:
|
|
64
|
+
- **Haiku**: completion detection (fast, cheap, every turn)
|
|
65
|
+
- `--verify` command: quality gate (once, on completion, user-defined)
|
|
66
|
+
|
|
67
|
+
The same principle that motivated separating the Haiku evaluator from Claude's self-assessment (avoiding cognitive bias) applies here: the completion judge should not also be the quality judge.
|
|
68
|
+
|
|
69
|
+
### `--checkpoint`
|
|
70
|
+
|
|
71
|
+
Auto-commit on structured sub-goal boundaries. Requires Haiku to output sub-goal markers (not just yes/no), which is a separate RFC — listed here for completeness.
|
|
72
|
+
|
|
73
|
+
---
|
|
74
|
+
|
|
75
|
+
## Reference Implementation
|
|
76
|
+
|
|
77
|
+
**forge-harness `goal-quench`** implements a userspace version of `--budget` + `--verify` using:
|
|
78
|
+
- Pre-run: `token-budget-gate` skill estimates cost and sets thresholds
|
|
79
|
+
- Mid-run: thresholds injected as session instructions (Claude self-enforces)
|
|
80
|
+
- Post-run: Stop hook detects `/goal` completion → triggers `pipeline-conductor --quick`
|
|
81
|
+
|
|
82
|
+
Limitations of the userspace approach (why native support is needed):
|
|
83
|
+
- Mid-run token ceiling cannot be hard-enforced without native hook
|
|
84
|
+
- Stop hook cannot invoke Claude recursively (verification triggers on next session, not immediately)
|
|
85
|
+
- Checkpoint requires manual commit — no structured sub-goal output from Haiku
|
|
86
|
+
|
|
87
|
+
**Paper reference**: forge-harness: A Meta-Harness Engineering Platform for Terminal-Native Claude Code Workflows. Zenodo DOI: 10.5281/zenodo.20397566. arXiv: [pending number].
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## Adoption signal
|
|
92
|
+
|
|
93
|
+
`/goal` was introduced in Claude Code Week 20 (2026-05-11), absorbing the community `claude-goal` project (Stop hook implementation) 11 days after OpenAI Codex CLI v0.128.0 shipped a similar feature. Community demand is established. This request addresses the production-readiness gap that community implementations cannot close without native runtime access.
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## Related issues / PRs
|
|
98
|
+
|
|
99
|
+
- `claude-goal` (community Stop hook implementation): [link if available]
|
|
100
|
+
- OpenAI Codex CLI `/goal` reference: v0.128.0 release notes
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
*Drafted: 2026-05-31. Submit after arXiv number confirmed.*
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# Harness 6-Axis Framework
|
|
2
|
+
|
|
3
|
+
> Top-level meta-framework for forge-harness operations. The 6 axes form a decision tree that governs all harness-level work — from initial structure through continuous improvement.
|
|
4
|
+
|
|
5
|
+
**Core principle (field harness)**: "A good harness gets simpler over time. If it's getting more complex, something is wrong."
|
|
6
|
+
|
|
7
|
+
**Meta-harness variant**: A good meta-harness *optimizes* over time — complexity is justified when it earns its scope. Red flags: orphaned skills (never invoked), redundant overlap (two skills doing the same thing), decorative structure (exists but doesn't change behavior). Complexity itself is not the warning signal.
|
|
8
|
+
|
|
9
|
+
**Completeness model (frame-wise ultimate)**: A harness is never *finally* complete — it is **frame-wise ultimate**. A *frame* is the scope one verification cycle can currently see: per-asset, an edit-manifest RECORD (Axis 3) → verify (Axis 5) cycle; for FH-as-a-whole, a release/version milestone (cf. a model's knowledge cutoff = its release frame's timestamp). A frame is *closed* — ultimate **for that frame** — when every **visible** gap is resolved or **named**: a named gap is evidence the frame closed (you saw the edge), not evidence against it; the only signal of incompleteness is an **un-named** gap. This reframes the existing completion-claim discipline (Axis 5 before "done", residuals declared) — growth is not the opposite of completeness but its **ladder**: Axis 6 carries one frame's closure into the next frame's baseline. **Two guards keep this honest, not self-congratulatory:** (1) never declare a frame closed before the gap-surfacing pass (Axis 5) has run — premature closure (e.g. "downloads ≠ validated usage"); (2) the verdict "are all *visible* gaps named?" is **judged, so it is adversarially paired** (Axis 5's challenger, or an external sister-GT) — never self-asserted, because an un-named gap is invisible *to the self-judge by construction*; only a non-self pass expands what is visible (mechanical-anchor / Non-Model-Ground principle, §Scope Hierarchy's no-self-commit kin). An *irreducible* gap (visible but unclosable — e.g. the 4-axis gate's autonomous-runner self-verification residual, documented honest in CLAUDE.md §FH 4-Axis Gate) closes its frame **at the ceiling**; the next frame opens only when a new capability changes what is visible.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## The 6 Axes
|
|
14
|
+
|
|
15
|
+
| Axis | Name | Question it answers |
|
|
16
|
+
|:---:|---|---|
|
|
17
|
+
| **1** | **Structure** | Where does this work live? (hub vs. project, rules vs. skills, knowledge vs. tracks) |
|
|
18
|
+
| **2** | **Context** | What does the AI need to know before starting? (session card, CATALOG, relevant docs) |
|
|
19
|
+
| **3** | **Plan** | What is the intended change and its predicted impact? (edit-manifest RECORD) |
|
|
20
|
+
| **4** | **Execute** | What is the minimal, reversible action? (direct edit, agent dispatch, parallel dispatch) |
|
|
21
|
+
| **5** | **Verify** | Did the change do what was predicted? (regression guard, adversarial, source-grounding) |
|
|
22
|
+
| **6** | **Improve** | What pattern is worth keeping? (harvest-loop, field-harvest, compounding loop) |
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Decision Tree (Condensed)
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
New work arrives
|
|
30
|
+
│
|
|
31
|
+
▼ Axis 1 — Structure
|
|
32
|
+
Where does this live?
|
|
33
|
+
├── Hub meta (rules/skills/templates) → FH 4-axis auto-gate applies
|
|
34
|
+
├── Field project → route via Agent dispatch or direct edit
|
|
35
|
+
└── Cross-project knowledge → knowledge/shared/
|
|
36
|
+
|
|
37
|
+
▼ Axis 2 — Context
|
|
38
|
+
What must the AI know?
|
|
39
|
+
├── Read session_card (reference_next_session_starter.md)
|
|
40
|
+
├── Read CATALOG.md → candidate files only
|
|
41
|
+
└── Load LOCAL_SKILL_REGISTRY if cross-project dispatch
|
|
42
|
+
|
|
43
|
+
▼ Axis 3 — Plan
|
|
44
|
+
What will change?
|
|
45
|
+
└── edit-manifest RECORD entry: branch, change, predicted_impact, verify_next
|
|
46
|
+
|
|
47
|
+
▼ Axis 4 — Execute
|
|
48
|
+
Minimum viable action:
|
|
49
|
+
├── Direct edit (simple, known file, absolute path)
|
|
50
|
+
├── Single Agent dispatch (field task, one project)
|
|
51
|
+
└── Parallel Agent dispatch (2+ independent tasks — no asking, just dispatch)
|
|
52
|
+
|
|
53
|
+
▼ Axis 5 — Verify
|
|
54
|
+
Did it work?
|
|
55
|
+
├── Axis 1 (backward): regression_guard.sh
|
|
56
|
+
├── Axis 2 (adversarial): steel-quench
|
|
57
|
+
├── Axis 3 (forward): phantom-quench
|
|
58
|
+
├── Axis 4 (record): confirm edit-manifest entry exists
|
|
59
|
+
└── Before claiming "done": attach (a) evidence (b) failure-checks run
|
|
60
|
+
(c) residual risk — bare "completed" does not pass (see Completion-claim discipline)
|
|
61
|
+
|
|
62
|
+
▼ Axis 6 — Improve
|
|
63
|
+
Worth keeping as a pattern?
|
|
64
|
+
├── 3+ repeats → skill-candidate tag → field-harvest
|
|
65
|
+
├── Session end → harvest-loop (weekly cycle)
|
|
66
|
+
└── Compounding: hub_compounding_loop.md
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## Axis 5 — FH 4-Axis Verification Gate (detail)
|
|
72
|
+
|
|
73
|
+
Applies automatically when any FH asset is modified (SKILL.md, rules, templates, CLAUDE.md, substantive knowledge/ docs).
|
|
74
|
+
|
|
75
|
+
| Gate axis | Tool | Class | What it catches |
|
|
76
|
+
|---|---|---|---|
|
|
77
|
+
| **Backward** | `regression_guard.sh` | mandatory-pass | Critical section loss, broken refs, syntax errors, line reduction |
|
|
78
|
+
| **Adversarial** | `steel-quench` | judged | Trigger phrase collisions, design attack surface, over-engineered steps |
|
|
79
|
+
| **Forward** | `phantom-quench` | judged | Phantom references, paths that don't exist, stale external links |
|
|
80
|
+
| **Record** | `edit-manifest RECORD` | mandatory-pass | Logs predicted impact — closes the predict-verify loop |
|
|
81
|
+
|
|
82
|
+
**Check classes**: every verify check is one of three classes — **mandatory-pass**
|
|
83
|
+
(deterministic; blocks on fail), **measured** (quantitative; tracked, not blocking alone —
|
|
84
|
+
e.g. `token-budget-gate`, goal-quench calibration), **judged** (LLM-judge emitting verdict +
|
|
85
|
+
cited evidence + a corrective action — a judge score without a fix path is unactionable, and
|
|
86
|
+
self-judges grade leniently). **Judged rule**: a judge verdict alone never passes — it must be
|
|
87
|
+
paired with adversarial re-verification (`steel-quench` / `verify-bidirectional`), and its
|
|
88
|
+
cited evidence is itself subject to `phantom-quench`. (Taxonomy adapted from external
|
|
89
|
+
supervisor-loop discourse, 2026-06; FH adds the judged-pairing rule and evidence
|
|
90
|
+
re-verification, which the source leaves open.)
|
|
91
|
+
|
|
92
|
+
**Completion-claim discipline** (judged-class sharpening): a "done" / "passed" claim is itself a judged
|
|
93
|
+
verdict and must carry three things, not just an assertion — **(a) an evidence artifact** (the output,
|
|
94
|
+
diff, or run that shows it), **(b) the enumerated failure-checks actually run** (which negative cases
|
|
95
|
+
were tested, not only that it "works"), and **(c) the explicit residual risk** (what could still be
|
|
96
|
+
wrong). A bare "completed" with none of these is an ungrounded judge verdict and does not pass. Each of
|
|
97
|
+
(a)–(c) must be **non-vacuous** — a named artifact, an enumerated case list, and a specific risk; "it
|
|
98
|
+
works" / "tested" / "none known" are vacuous fills and fail the discipline (same non-vacuity bar as the
|
|
99
|
+
CLAUDE.md §marker rule: a recorded verdict/count, not "it ran"). Bounded scope: (b) means the negative
|
|
100
|
+
cases you *actually ran*, not all conceivable ones; (c) means the one or two risks you can name now, not
|
|
101
|
+
an exhaustive proof of safety. Applies to every skill's Done When and to `goal-quench` /
|
|
102
|
+
`pipeline-conductor` completion gates. (Harvested as independent-convergence reinforcement from sister
|
|
103
|
+
assets — oh-my-claudecode "Ralph" Done-When + book/19689's verification-before-completion-claim theme;
|
|
104
|
+
cross-audit `tracks/_audit/session_2026_06_14_wikidocs-deep-sweep.md`.)
|
|
105
|
+
|
|
106
|
+
**Hard gate**: git pre-commit hook (`templates/.git-hooks/pre-commit`) blocks commit until marker + manifest entry exist.
|
|
107
|
+
|
|
108
|
+
**Lightweight exception** (Axis 1 + 4 only): sessions where zero SKILL.md/rules/templates files changed.
|
|
109
|
+
|
|
110
|
+
---
|
|
111
|
+
|
|
112
|
+
## Scope Hierarchy
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
Hub common principles (CLAUDE.md)
|
|
116
|
+
└── Project CLAUDE.md
|
|
117
|
+
└── Domain session rules (.claude/rules/session.md)
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Lower levels cannot override higher. AI contribution → PR proposal only (no direct commit to shared repos without explicit user approval).
|
|
121
|
+
|
|
122
|
+
---
|
|
123
|
+
|
|
124
|
+
## Related
|
|
125
|
+
|
|
126
|
+
- `harness_design_decision_lens.md` — orthogonal companion: the 7 architectural-bet decisions (which design point at Axis 1 / Axis 4) + default-bias checklist + scaffolding-removal method
|
|
127
|
+
- `crucible_mode.md` — total-immersion absorption stance: chains Axis 5 (the melt) + Axis 6 (the rebirth) with an unmeltable identity core; used when a whole corpus on a core FH axis is absorbed
|
|
128
|
+
- `hub_compounding_loop.md` — Axis 6 automation (weekly/monthly/quarterly cycles)
|
|
129
|
+
- `ai_dialogue_playbook.md` — Axis 2 dialogue principles (how to ask, delegate, record)
|
|
130
|
+
- `claude_code_runtime_flow.md` — Axis 4 runtime behavior (chronological session flow)
|
|
131
|
+
- `.claude/rules/operations.md` — Sub-agent operations, weekly cycle detail
|
|
132
|
+
|
|
133
|
+
**External sibling (independent convergence)**
|
|
134
|
+
|
|
135
|
+
- arXiv:2603.25723 (*Natural-Language Agent Harnesses*, NLAH) — external academic sibling that independently converges on the same core thesis: a harness control layer can be an executable natural-language object, not code. NLAH measures the natural-language-harness form empirically; FH governs and compounds it.
|
|
136
|
+
- arXiv:2606.06324 (*HarnessFix / ETCLOVG*, 2026-06) — sibling on the **orthogonal** axis: where NLAH and FH describe the harness as a *process/control* object, HarnessFix supplies a *component taxonomy* of what a deployed harness contains (7 layers — Execution · Tooling · Context · Lifecycle · Observability · Verification · Governance). Its **V layer maps onto FH's Axis-5 gate on 3 of its 4 functions** (intermediate validation → steel/phantom-quench · final-output eval → completion-claim discipline · regression testing → regression_guard); its *readiness-check* function maps to FH pre-flight gates (install-doctor / asset-placement-gate) that sit outside Axis-5. Its named **Observability** layer is a structural axis FH lacks — FH's nearest coverage is *retrospective audit* (weekly_audit, subagent_invocations_log), not runtime observability (import candidate for `harness-doctor`). Cross-audit: `tracks/_audit/session_2026_06_19_harnessfix-etclovg-cross-audit.md`.
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
# Harness Design-Decision Lens
|
|
2
|
+
|
|
3
|
+
> A complement to `harness_6axis_framework.md`. The 6-axis framework is a **lifecycle** decomposition —
|
|
4
|
+
> it answers *when, in the flow of a piece of work*, each concern applies (structure → context → plan →
|
|
5
|
+
> execute → verify → improve). This lens is **orthogonal**: it answers *which architectural bet to place*
|
|
6
|
+
> at Axis 1 (Structure) and Axis 4 (Execute), where the 6-axis tree says "route it" but not "which design
|
|
7
|
+
> point." Use the two together — the 6-axis says *which stage*; this lens says *which trade-off setting*.
|
|
8
|
+
>
|
|
9
|
+
> **Open this doc when** you are about to add agents, tools, permissions, or harness thickness — or when
|
|
10
|
+
> the 6-axis tree says "route it" (Axis 1 / Axis 4) but not *which way*. For lifecycle questions (what
|
|
11
|
+
> stage am I in, what must I verify), stay in the 6-axis framework.
|
|
12
|
+
|
|
13
|
+
**Provenance (independent-convergence harvest — not clone-and-own):** the decision-table framing was
|
|
14
|
+
harvested from the sister asset `wikidocs.net/book/19689` 「하네스 엔지니어링 백과사전」 Ch12 (governor
|
|
15
|
+
source-closed 2026-06-14; cross-audit `tracks/_audit/session_2026_06_14_wikidocs-deep-sweep.md`). **The
|
|
16
|
+
single net-new increment from the harvest is the orthogonal-bets *framing* itself** — presenting harness
|
|
17
|
+
architecture as ~7 trade-off axes set against the 6-axis lifecycle. The governor source-close ruled the
|
|
18
|
+
individual decisions, the contrarian checklist, and the scaffolding-removal method **ALREADY-HAVE** (FH
|
|
19
|
+
embodies them — see column 3 and the `measured`/judged check classes). They are reproduced below only to
|
|
20
|
+
make the already-placed bets *explicit and operational in one place*, not as novel imports. Where this
|
|
21
|
+
doc restates an existing FH principle, it says so.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## The seven decisions (architectural bets)
|
|
26
|
+
|
|
27
|
+
Each is a spectrum, not a right answer. The third column names the FH asset that already embodies the
|
|
28
|
+
bet — so this is a *map of bets FH is already placing*, made explicit, plus the two FH was placing only
|
|
29
|
+
implicitly (agent-count, reasoning-strategy).
|
|
30
|
+
|
|
31
|
+
| Decision | Spectrum | Where FH already places the bet | "Which point, when" |
|
|
32
|
+
|---|---|---|---|
|
|
33
|
+
| **Agent count** | single ↔ multi-agent | `agent-composer` (single vs parallel dispatch) | Single by default; split only on genuinely independent sub-tasks (agent-composer's per-wave fan-out cap, ≤4). Isolation cost rises with agent count. |
|
|
34
|
+
| **Reasoning strategy** | ReAct (think each step) ↔ plan-then-execute | 6-axis Axis 3 (Plan) → Axis 4 (Execute); `goal-quench` decomposition | Plan-then-execute for parallelizable / known work; ReAct for genuinely exploratory steps. |
|
|
35
|
+
| **Context strategy** | strong compression ↔ rich context | `context-doctor`, compact-before-saturation reflex, layered memory (MEMORY.md index / `memory/*.md` keyword-load / `tracks/` archive) | Compress on long sessions; keep rich context where verification needs the evidence intact (phantom-quench tension). |
|
|
36
|
+
| **Verification** | computational ↔ judged | check-class taxonomy: mandatory-pass / measured / **judged** (`harness_6axis_framework.md` Axis 5) | Prefer computational anchors; a judged verdict never passes alone (judge-robustness rule). |
|
|
37
|
+
| **Permissions** | permissive ↔ restrictive | `mcp_tool_gating` allow / ask / deny; Destructive-Op + Pre-Publish gates | Restrictive on irreversible / external-facing actions; permissive on reversible local work. |
|
|
38
|
+
| **Tool scope** | all tools always ↔ staged minimal | ToolSearch (deferred-tool fetch), `mcp_tool_gating` | Stage tools in; broad always-on exposure degrades selection accuracy. (Vercel measured 17 specialized tools → 80% success vs 2 general-purpose tools → 100%, at lower token + latency: "We removed 80% of our agent's tools", vercel.com/blog/we-removed-80-percent-of-our-agents-tools, accessed 2026-06-14.) |
|
|
39
|
+
| **Harness thickness** | thin (trust model) ↔ thick (control by code/rules) | field principle "simpler over time" vs meta principle "complexity earns its scope" (`harness_6axis_framework.md` Core principle) | FH already holds a *more nuanced* field-vs-meta split than a single thin/thick dial. Thicker buys predictability but encodes "things the model can't do" — assumptions that age as the model improves (design intuition, not a benchmarked claim). |
|
|
40
|
+
|
|
41
|
+
The decisions are **not independent**: more agents raises the cost of context isolation + verification;
|
|
42
|
+
always-on tools raises the stakes of permission design; a thicker harness raises the carrying cost of
|
|
43
|
+
stale assumptions.
|
|
44
|
+
|
|
45
|
+
**Orthogonality is partial, not total.** Two of the seven bets — *reasoning strategy* and *verification* —
|
|
46
|
+
are placed *at* a 6-axis stage (Axis 3→4 for reasoning, Axis 5 for verification), so for those the lens
|
|
47
|
+
and the lifecycle touch rather than run perpendicular. The lens is still the right tool for "which setting"
|
|
48
|
+
(plan-vs-ReAct, computational-vs-judged); the 6-axis is the right tool for "at which stage." The orthogonal
|
|
49
|
+
claim holds strongly for the other five bets and loosely for these two.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## Default-bias checklist (the contrarian inversions)
|
|
54
|
+
|
|
55
|
+
> *Already-embodied (governor: ALREADY-HAVE) — restated here as a one-place design-time pass, not a novel
|
|
56
|
+
> import. FH already runs these instincts through `steel-quench` and the meta-harness red-flag scan.*
|
|
57
|
+
|
|
58
|
+
Four intuitions that feel right while building and fail in operation. Run this as a quick adversarial
|
|
59
|
+
pass on any harness design (it is the design-time analogue of `steel-quench`'s attack lenses):
|
|
60
|
+
|
|
61
|
+
1. **"More tools = more capable."** Often false — broad tool exposure hurts selection accuracy. Prefer
|
|
62
|
+
staged, single-purpose tools over a wide general surface.
|
|
63
|
+
2. **"Reason at every step (ReAct always)."** Often false — for parallelizable / known work,
|
|
64
|
+
plan-then-execute is faster and cheaper.
|
|
65
|
+
3. **"Broader permissions = faster."** False under operations — the cost lands as irreversible mistakes,
|
|
66
|
+
not saved time. Gate the irreversible.
|
|
67
|
+
4. **"Thicker harness = safer."** True short-term, debt long-term — a thick harness encodes "things the
|
|
68
|
+
model can't do," and those assumptions age. Safety that does not get revisited becomes a burden.
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## Scaffolding-removal method (operationalizes "simpler over time")
|
|
73
|
+
|
|
74
|
+
> *The principle is already FH's (governor: ALREADY-HAVE — 6-axis core principle). What is operationalized
|
|
75
|
+
> here is the **method** the principle left unstated, consistent with FH's `measured` check class.*
|
|
76
|
+
|
|
77
|
+
FH's core principle states the *goal* — "a good harness gets simpler over time" — but not the *method*.
|
|
78
|
+
A testable one, consistent with FH's verification identity:
|
|
79
|
+
|
|
80
|
+
> **Remove scaffolding one piece at a time, and compare.** When simplifying, ablate a single support at a
|
|
81
|
+
> time and check what actually holds quality before removing the next. Removing several at once loses the
|
|
82
|
+
> signal of which support was load-bearing.
|
|
83
|
+
|
|
84
|
+
This is the disciplined form of the meta-harness red-flag scan (orphaned / redundant / decorative units):
|
|
85
|
+
do not bulk-delete suspected-decorative structure — ablate-one, measure, then proceed. It pairs with the
|
|
86
|
+
`measured` check class (the comparison is the measurement).
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## How it plugs into the 6-axis flow
|
|
91
|
+
|
|
92
|
+
- **Axis 1 (Structure)** — when deciding where work lives and how it is shaped, consult the seven
|
|
93
|
+
decisions for the architectural bet, and the default-bias checklist before committing to "more"
|
|
94
|
+
(agents, tools, permissions, thickness).
|
|
95
|
+
- **Axis 4 (Execute)** — the agent-count and reasoning-strategy decisions set the dispatch shape
|
|
96
|
+
(direct / single agent / parallel) the 6-axis tree otherwise leaves to habit.
|
|
97
|
+
- **Axis 6 (Improve)** — apply the scaffolding-removal method when the simplification principle fires.
|
|
98
|
+
|
|
99
|
+
**Check class:** this doc is a *judged* design aid (it informs human/AI design choices, it does not gate
|
|
100
|
+
mechanically). Its adversarial pairing is the default-bias checklist above + `steel-quench` on any design
|
|
101
|
+
it informs — no judge-only path.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## Related
|
|
106
|
+
- `harness_6axis_framework.md` — the lifecycle framework this lens complements (Core principle: thickness; Axis 5: check-class taxonomy)
|
|
107
|
+
- `tracks/_audit/session_2026_06_14_wikidocs-deep-sweep.md` — the governor-closed cross-audit this harvest came from
|
|
108
|
+
- `.claude/rules/auto_project_mapping.md` — Full-Harness Mode, where thickness/permission bets are placed per project
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: harness-frontier-diagnosis-2026-06-02
|
|
3
|
+
description: Frontier digest anchored on FH's 3-layer identity (Control Tower · Frontier→Org Propagation · AI Collaboration Guide) + Core Axis. External AI/harness-engineering signal from 2026-06 translated into per-identity strengthening candidates, with simplicity guards.
|
|
4
|
+
type: frontier-diagnosis
|
|
5
|
+
date: 2026-06-02
|
|
6
|
+
engine: websearch
|
|
7
|
+
tags: [frontier, identity, harness-engineering, multi-agent, a2a, mcp, context-engineering, observability, v2-paper]
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Harness Frontier Diagnosis — 2026-06-02
|
|
11
|
+
|
|
12
|
+
> Identity ② asset (`harness_frontier_diagnosis_*.md`). Collects the global AI/harness-engineering
|
|
13
|
+
> frontier and **translates it for FH operations** — anchored on FH's three identities + Core Axis.
|
|
14
|
+
> Engine: WebSearch (no `ANTHROPIC_API_KEY`; outbound curl blocked → forced downgrade per `frontier-digest` skill).
|
|
15
|
+
|
|
16
|
+
## FH Identity Anchor (from CLAUDE.md §Identity)
|
|
17
|
+
|
|
18
|
+
| # | Identity | One-line role |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| ① | **Control Tower** | Command HQ that coordinates all connected projects |
|
|
21
|
+
| ② | **Frontier → Org Propagation** | Absorb global frontier thinking, translate it into org language |
|
|
22
|
+
| ③ | **AI Collaboration Guide** | Accumulate/distribute token-efficiency + dialogue methodology |
|
|
23
|
+
| Axis | **Harness Engineering (How)** | The 6-axis methodology that realizes the three above |
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Frontier Highlights (2026-06)
|
|
28
|
+
|
|
29
|
+
**1. "Harness Engineering" named the 4th paradigm of AI engineering.**
|
|
30
|
+
The arc prompt → context → harness is now an explicit industry framing: *"Agents aren't hard; the
|
|
31
|
+
Harness is hard."* The widely-cited claim is that **~65% of enterprise AI failures trace to harness
|
|
32
|
+
defects** — Context Drift, Schema Misalignment, State Degradation — not model capability. This is
|
|
33
|
+
direct external validation of FH's whole thesis (`meta_harness_engineering_definition.md`,
|
|
34
|
+
`fh_ecosystem_positioning.md`). → **Core Axis**.
|
|
35
|
+
|
|
36
|
+
**2. Agent interoperability standardized: A2A "Agent Cards" + MCP registry under Linux Foundation.**
|
|
37
|
+
A2A standardizes how agents *discover* each other's capabilities (Agent Cards); MCP launched a
|
|
38
|
+
community server registry (Nov 2025). Production topology data: orchestrator-worker is ~70% of
|
|
39
|
+
deployments, but **centralized multi-agent coordination carries ~+285% token overhead** and the
|
|
40
|
+
practical team size is **3–4 agents** before coordination cost dominates. → **① Control Tower**.
|
|
41
|
+
|
|
42
|
+
**3. Observability is the bottleneck for self-improving harnesses (Agentic Harness Engineering).**
|
|
43
|
+
AHE's central claim: *agents cannot reliably improve a black-box harness* — the evolution loop needs
|
|
44
|
+
the harness's components, experiences, and decisions to be observable and verifiable. Eval-driven:
|
|
45
|
+
the 2026 Coding Agent Index benchmarks **model+harness pairs**, not models alone. Context research
|
|
46
|
+
adds the "lost in the middle" effect (10–30% accuracy drop on mid-context information) and the
|
|
47
|
+
hierarchical-context remedy (L1 always-on / L2 session / L3 on-demand) + prompt compression.
|
|
48
|
+
→ **② Frontier Propagation** + **③ AI Collaboration Guide**.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## Per-Identity Strengthening Candidates
|
|
53
|
+
|
|
54
|
+
### ① Control Tower
|
|
55
|
+
|
|
56
|
+
| Candidate | Frontier basis | FH hook |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| **Machine-readable `agent-card`-style capability registry** for the FH agents (capability / input-output contract, synced to actual file counts) | A2A Agent Card = the discovery standard | Closes the "count drift / no canonical registry" gap already flagged in `fh_ecosystem_positioning.md` |
|
|
59
|
+
| **Coordination-overhead budget** in `context-bridge-dispatch`: parallel-fan-out cap (3–4) + capability-aware routing | Centralized = +285% tokens; team size caps at 3–4 | `plugins/fh-meta/skills/context-bridge-dispatch`, `agent-composer` |
|
|
60
|
+
|
|
61
|
+
### ② Frontier → Org Propagation
|
|
62
|
+
|
|
63
|
+
| Candidate | Frontier basis | FH hook |
|
|
64
|
+
|---|---|---|
|
|
65
|
+
| Add a **harness-defect taxonomy axis** (Context Drift / Schema Misalignment / State Degradation) to structural diagnosis | "65% of AI failures = harness defects" | `plugins/fh-meta/skills/harness-doctor` |
|
|
66
|
+
| Add **observability / eval hooks** to the evolution loop so self-improvement is glass-box, not black-box (eval-driven, model+harness benchmarking style) | AHE: observability is the self-improvement bottleneck | `plugins/fh-meta/skills/harvest-loop`, `harness-doctor` |
|
|
67
|
+
|
|
68
|
+
### ③ AI Collaboration Guide
|
|
69
|
+
|
|
70
|
+
| Candidate | Frontier basis | FH hook |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| Formalize **L1/L2/L3 context hierarchy** + critical-info-at-start-and-end placement as a dialogue norm | "Lost in the middle" 10–30% degradation | `plugins/fh-meta/skills/context-doctor`, `CHEATSHEET.md` |
|
|
73
|
+
| Add a **prompt-compression pass** (LLMLingua-style) to further shrink the install footprint | 100K→20K near-lossless compression cases | `plugins/fh-meta/skills/context-doctor` |
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Warning Signals
|
|
78
|
+
|
|
79
|
+
- **Agent-proliferation temptation.** Centralized multi-agent = +285% tokens; it only pays off with
|
|
80
|
+
genuine specialization / parallelism / critique. FH's own principle — *"a good harness gets simpler
|
|
81
|
+
over time"* — is the built-in guard. Do not add agents to chase the trend.
|
|
82
|
+
- **"Harness Engineering" is becoming a buzzword** (awesome-lists, "4th paradigm" marketing). Cite the
|
|
83
|
+
external convergence as validation, but treat it as a complexity-creep risk, not a mandate to expand.
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Provenance (WebSearch sources, 2026-06-02)
|
|
88
|
+
|
|
89
|
+
- Epsilla — *The Third Evolution: Why Harness Engineering Replaced Prompting in 2026* — https://www.epsilla.com/blogs/harness-engineering-evolution-prompt-context-autonomous-agents
|
|
90
|
+
- Faros.ai — *Harness Engineering: Making AI Coding Agents Work in 2026* — https://www.faros.ai/blog/harness-engineering
|
|
91
|
+
- Adnan Masood — *Agent Harness Engineering — The Rise of the AI Control Plane* — https://medium.com/@adnanmasood/agent-harness-engineering-the-rise-of-the-ai-control-plane-938ead884b1d
|
|
92
|
+
- getstream.io — *Top AI Agent Protocols in 2026 — MCP, A2A, ACP & More* — https://getstream.io/blog/ai-agent-protocols/
|
|
93
|
+
- Zylos Research — *Agent Interoperability Protocols 2026: MCP, A2A, ACP and the Path to Convergence* — https://zylos.ai/research/2026-03-26-agent-interoperability-protocols-mcp-a2a-acp-convergence
|
|
94
|
+
- codebridge.tech — *Multi-Agent Systems & AI Orchestration Guide 2026* — https://www.codebridge.tech/articles/mastering-multi-agent-orchestration-coordination-is-the-new-scale-frontier
|
|
95
|
+
- Micheal Lanham — *Multi-Agent in Production in 2026: What Actually Survived* — https://medium.com/@Micheal-Lanham/multi-agent-in-production-in-2026-what-actually-survived-f86de8bb1cd1
|
|
96
|
+
- Inference Weekly — *Agentic Harness Engineering (AHE): Evolving Coding-Agent Harnesses with Observability-Driven Automation* — https://medium.com/@harshit.sinha0910/agentic-harness-engineering-ahe-evolving-coding-agent-harnesses-with-observability-driven-297481226663
|
|
97
|
+
- Observability-Driven Automatic Evolution of Coding-Agent Harnesses — https://arxiv.org/pdf/2604.25850
|
|
98
|
+
- TokenMix — *LLM Context Window 2026: 128K to 10M Tokens* — https://tokenmix.ai/blog/llm-context-window-explained
|
|
99
|
+
- dasroot.net — *Token Optimization Strategies for Cost-Effective LLM Applications* — https://dasroot.net/posts/2026/04/token-optimization-llm-costs-prompt-engineering/
|
|
100
|
+
|
|
101
|
+
> Raw collected signal + the improvement-signal processing checklist (working log) are kept in the
|
|
102
|
+
> private companion store, per the public/private split policy — not committed to this public repo.
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# Hub Compounding Loop
|
|
2
|
+
|
|
3
|
+
> Axis-6 automation: the mechanism by which the forge-harness hub improves itself over time through structured feedback cycles.
|
|
4
|
+
|
|
5
|
+
**Principle**: Each session's learnings are absorbed back into the hub so the harness evolves on its own — without requiring manual re-triggering.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Cycle Overview
|
|
10
|
+
|
|
11
|
+
| Cadence | Trigger | Key actions |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| **Per-session** | Session close ("wrap up", "done", "good work") | harvest-loop → card update → push check |
|
|
14
|
+
| **Weekly** | 7 days since last `frontier_digest_*.md` | `/frontier-digest` → CATALOG entry + 6-candidate implementation |
|
|
15
|
+
| **Monthly** | 30 days since last harness-doctor run | `/harness-doctor` → L1~L4 diagnosis + M/S/R prescription |
|
|
16
|
+
| **Quarterly** | ~90 days | Sister asset sync, Phase transition gate review |
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Per-Session Close Chain (Automatic — Not Skippable)
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
Closing phrase detected
|
|
24
|
+
→ ① git diff check
|
|
25
|
+
→ ② if diff exists → harvest-loop
|
|
26
|
+
→ ③ card update (reference_next_session_starter.md) — independent obligation
|
|
27
|
+
→ ④ unpushed commits → propose "push?"
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Card update is NOT a sub-step of harvest-loop — runs even if harvest-loop is skipped.
|
|
31
|
+
|
|
32
|
+
**Real-time tracking**: Complete S-tier/A-tier items get immediately appended to
|
|
33
|
+
`tracks/_meta/fh_completed_{YYYY-MM-DD}.md` (before context compression).
|
|
34
|
+
harvest-loop Step 0-b uses this file — relying on LLM memory after compression causes omissions.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## harvest-loop Pipeline (8 Steps)
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
field-harvest (pattern extraction)
|
|
42
|
+
→ contention-layer (collision signals)
|
|
43
|
+
→ [persona-devil-advocate + persona-innovator] (parallel)
|
|
44
|
+
→ synthesizer (devil/innovator collision harvest)
|
|
45
|
+
→ Critic isolated Agent (SAGE critique)
|
|
46
|
+
→ harness-doctor (health check)
|
|
47
|
+
→ verify-bidirectional (consistency validation)
|
|
48
|
+
→ curator (skill lifecycle management)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Session learnings automatically absorbed back into FH ecosystem.
|
|
52
|
+
|
|
53
|
+
**In main dev env**: runs automatically at session end.
|
|
54
|
+
**For external FH users**: proposes execution first.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Weekly Audit Cycle (Phase 1.5)
|
|
59
|
+
|
|
60
|
+
1. `./tracks/_audit/_scanner.sh "7 days ago"` — aggregates: commits, tags, stale files, sub-agent invocation log, self-asset references
|
|
61
|
+
2. Copy `_template_weekly.md` → `weekly_audit_YYYY-MM-DD.md`
|
|
62
|
+
3. Propose 3-tier improvements (🟥mandatory / 🟧strong / 🟩recommended)
|
|
63
|
+
|
|
64
|
+
**Phase 2 (skill-ized)**: `/harvest-loop` automates the above (manual ~10 min → auto ~3 min target).
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## Cadence Files (Session-Start Auto-Detection)
|
|
69
|
+
|
|
70
|
+
| File | Cadence | Auto-propose condition |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| `tracks/_meta/frontier_digest_*.md` | 7 days | Propose `/frontier-digest` at session start if 7+ days |
|
|
73
|
+
| `tracks/_meta/*harness_doctor*.md` | 30 days | Propose `/harness-doctor` at session start if 30+ days |
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## 3-Phase Maturity Roadmap
|
|
78
|
+
|
|
79
|
+
| Phase | Name | Criteria |
|
|
80
|
+
|---|---|---|
|
|
81
|
+
| **Phase I** | Entering Maturity | 5-criteria gate — consistent weekly audit, no critical debt, harvest-loop running |
|
|
82
|
+
| **Phase II** | Frontier Following | frontier-digest cadence + sister asset sync + external PR evidence |
|
|
83
|
+
| **Phase III** | Frontier Leading | 6 indicators + writing guide for org-level propagation |
|
|
84
|
+
|
|
85
|
+
**Shared condition for all transitions**: optimization principle — field harness: getting simpler over time; meta-harness: complexity justified by scope (no orphaned/redundant/decorative units).
|
|
86
|
+
|
|
87
|
+
Detailed frame: `hub_maturity_roadmap.md`.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## FH Improvement Signal Recording
|
|
92
|
+
|
|
93
|
+
When friction is detected during a session, record it for the next session's awareness:
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
tracks/_meta/fh_signal_{YYYY_MM_DD}_{source}.md
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Fields: friction point, FH registration candidate, status (pending hub review).
|
|
100
|
+
|
|
101
|
+
**Guard**: 1 file per session (append if same date+source). Structural improvements only — no minor typos.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## Related
|
|
106
|
+
|
|
107
|
+
- `harness_6axis_framework.md` — Axis 6 is the "Improve" step that feeds into this loop
|
|
108
|
+
- `.claude/rules/operations.md` — Sub-agent invocation log, weekly audit scanner detail
|
|
109
|
+
- `.claude/rules/sync_push_protocols.md` — Session Sync Protocol (how learnings enter the loop)
|