@chrono-meta/fh-gate 1.4.40 → 1.4.42
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/CLAUDE.md +74 -131
- package/docs/CONTRIBUTING.md +2 -2
- package/knowledge/shared/dialogue/ai_dialogue_playbook.md +137 -0
- package/knowledge/shared/dialogue/claude_code_runtime_flow.md +170 -0
- package/knowledge/shared/dialogue/memory_intent_recall.md +209 -0
- package/knowledge/shared/harness-core/claude_md_gate_details.md +170 -0
- package/knowledge/shared/harness-core/companion_store_pluggable_cross_audit_2026-06-11.md +118 -0
- package/knowledge/shared/harness-core/crucible_mode.md +112 -0
- package/knowledge/shared/harness-core/deep_research_capability_ladder.md +122 -0
- package/knowledge/shared/harness-core/fh_detail_protocols.md +163 -0
- package/knowledge/shared/harness-core/fh_ecosystem_positioning.md +147 -0
- package/knowledge/shared/harness-core/fh_opencode_governance_wrapper.md +163 -0
- package/knowledge/shared/harness-core/fh_synergy_playbook.md +217 -0
- package/knowledge/shared/harness-core/gate_locality_principle.md +57 -0
- package/knowledge/shared/harness-core/goal_quench_anthropic_issue.md +104 -0
- package/knowledge/shared/harness-core/harness_6axis_framework.md +136 -0
- package/knowledge/shared/harness-core/harness_design_decision_lens.md +108 -0
- package/knowledge/shared/harness-core/harness_frontier_diagnosis_2026-06-02.md +102 -0
- package/knowledge/shared/harness-core/hub_compounding_loop.md +109 -0
- package/knowledge/shared/harness-core/hub_maturity_roadmap.md +201 -0
- package/knowledge/shared/harness-core/hybrid_orchestration_architecture_roadmap.md +196 -0
- package/knowledge/shared/harness-core/live_surface_automation_pattern.md +110 -0
- package/knowledge/shared/harness-core/measurement-integrity-checklist.md +54 -0
- package/knowledge/shared/harness-core/meta_harness_engineering_definition.md +116 -0
- package/knowledge/shared/harness-core/multi_model_sidecar_strategy.md +650 -0
- package/knowledge/shared/harness-core/persona_container_schema.md +172 -0
- package/knowledge/shared/harness-core/return_path_gate.md +120 -0
- package/knowledge/shared/harness-core/self_evolution_routine.md +268 -0
- package/knowledge/shared/harness-core/skill_quality_rubric.md +71 -0
- package/knowledge/shared/harness-core/tpa_schema.md +136 -0
- package/package.json +3 -2
- package/plugins/fh-meta/skills/context-doctor/SKILL.md +4 -0
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: harness-frontier-diagnosis-2026-06-02
|
|
3
|
+
description: Frontier digest anchored on FH's 3-layer identity (Control Tower · Frontier→Org Propagation · AI Collaboration Guide) + Core Axis. External AI/harness-engineering signal from 2026-06 translated into per-identity strengthening candidates, with simplicity guards.
|
|
4
|
+
type: frontier-diagnosis
|
|
5
|
+
date: 2026-06-02
|
|
6
|
+
engine: websearch
|
|
7
|
+
tags: [frontier, identity, harness-engineering, multi-agent, a2a, mcp, context-engineering, observability, v2-paper]
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Harness Frontier Diagnosis — 2026-06-02
|
|
11
|
+
|
|
12
|
+
> Identity ② asset (`harness_frontier_diagnosis_*.md`). Collects the global AI/harness-engineering
|
|
13
|
+
> frontier and **translates it for FH operations** — anchored on FH's three identities + Core Axis.
|
|
14
|
+
> Engine: WebSearch (no `ANTHROPIC_API_KEY`; outbound curl blocked → forced downgrade per `frontier-digest` skill).
|
|
15
|
+
|
|
16
|
+
## FH Identity Anchor (from CLAUDE.md §Identity)
|
|
17
|
+
|
|
18
|
+
| # | Identity | One-line role |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| ① | **Control Tower** | Command HQ that coordinates all connected projects |
|
|
21
|
+
| ② | **Frontier → Org Propagation** | Absorb global frontier thinking, translate it into org language |
|
|
22
|
+
| ③ | **AI Collaboration Guide** | Accumulate/distribute token-efficiency + dialogue methodology |
|
|
23
|
+
| Axis | **Harness Engineering (How)** | The 6-axis methodology that realizes the three above |
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Frontier Highlights (2026-06)
|
|
28
|
+
|
|
29
|
+
**1. "Harness Engineering" named the 4th paradigm of AI engineering.**
|
|
30
|
+
The arc prompt → context → harness is now an explicit industry framing: *"Agents aren't hard; the
|
|
31
|
+
Harness is hard."* The widely-cited claim is that **~65% of enterprise AI failures trace to harness
|
|
32
|
+
defects** — Context Drift, Schema Misalignment, State Degradation — not model capability. This is
|
|
33
|
+
direct external validation of FH's whole thesis (`meta_harness_engineering_definition.md`,
|
|
34
|
+
`fh_ecosystem_positioning.md`). → **Core Axis**.
|
|
35
|
+
|
|
36
|
+
**2. Agent interoperability standardized: A2A "Agent Cards" + MCP registry under Linux Foundation.**
|
|
37
|
+
A2A standardizes how agents *discover* each other's capabilities (Agent Cards); MCP launched a
|
|
38
|
+
community server registry (Nov 2025). Production topology data: orchestrator-worker is ~70% of
|
|
39
|
+
deployments, but **centralized multi-agent coordination carries ~+285% token overhead** and the
|
|
40
|
+
practical team size is **3–4 agents** before coordination cost dominates. → **① Control Tower**.
|
|
41
|
+
|
|
42
|
+
**3. Observability is the bottleneck for self-improving harnesses (Agentic Harness Engineering).**
|
|
43
|
+
AHE's central claim: *agents cannot reliably improve a black-box harness* — the evolution loop needs
|
|
44
|
+
the harness's components, experiences, and decisions to be observable and verifiable. Eval-driven:
|
|
45
|
+
the 2026 Coding Agent Index benchmarks **model+harness pairs**, not models alone. Context research
|
|
46
|
+
adds the "lost in the middle" effect (10–30% accuracy drop on mid-context information) and the
|
|
47
|
+
hierarchical-context remedy (L1 always-on / L2 session / L3 on-demand) + prompt compression.
|
|
48
|
+
→ **② Frontier Propagation** + **③ AI Collaboration Guide**.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## Per-Identity Strengthening Candidates
|
|
53
|
+
|
|
54
|
+
### ① Control Tower
|
|
55
|
+
|
|
56
|
+
| Candidate | Frontier basis | FH hook |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| **Machine-readable `agent-card`-style capability registry** for the FH agents (capability / input-output contract, synced to actual file counts) | A2A Agent Card = the discovery standard | Closes the "count drift / no canonical registry" gap already flagged in `fh_ecosystem_positioning.md` |
|
|
59
|
+
| **Coordination-overhead budget** in `context-bridge-dispatch`: parallel-fan-out cap (3–4) + capability-aware routing | Centralized = +285% tokens; team size caps at 3–4 | `plugins/fh-meta/skills/context-bridge-dispatch`, `agent-composer` |
|
|
60
|
+
|
|
61
|
+
### ② Frontier → Org Propagation
|
|
62
|
+
|
|
63
|
+
| Candidate | Frontier basis | FH hook |
|
|
64
|
+
|---|---|---|
|
|
65
|
+
| Add a **harness-defect taxonomy axis** (Context Drift / Schema Misalignment / State Degradation) to structural diagnosis | "65% of AI failures = harness defects" | `plugins/fh-meta/skills/harness-doctor` |
|
|
66
|
+
| Add **observability / eval hooks** to the evolution loop so self-improvement is glass-box, not black-box (eval-driven, model+harness benchmarking style) | AHE: observability is the self-improvement bottleneck | `plugins/fh-meta/skills/harvest-loop`, `harness-doctor` |
|
|
67
|
+
|
|
68
|
+
### ③ AI Collaboration Guide
|
|
69
|
+
|
|
70
|
+
| Candidate | Frontier basis | FH hook |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| Formalize **L1/L2/L3 context hierarchy** + critical-info-at-start-and-end placement as a dialogue norm | "Lost in the middle" 10–30% degradation | `plugins/fh-meta/skills/context-doctor`, `CHEATSHEET.md` |
|
|
73
|
+
| Add a **prompt-compression pass** (LLMLingua-style) to further shrink the install footprint | 100K→20K near-lossless compression cases | `plugins/fh-meta/skills/context-doctor` |
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Warning Signals
|
|
78
|
+
|
|
79
|
+
- **Agent-proliferation temptation.** Centralized multi-agent = +285% tokens; it only pays off with
|
|
80
|
+
genuine specialization / parallelism / critique. FH's own principle — *"a good harness gets simpler
|
|
81
|
+
over time"* — is the built-in guard. Do not add agents to chase the trend.
|
|
82
|
+
- **"Harness Engineering" is becoming a buzzword** (awesome-lists, "4th paradigm" marketing). Cite the
|
|
83
|
+
external convergence as validation, but treat it as a complexity-creep risk, not a mandate to expand.
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Provenance (WebSearch sources, 2026-06-02)
|
|
88
|
+
|
|
89
|
+
- Epsilla — *The Third Evolution: Why Harness Engineering Replaced Prompting in 2026* — https://www.epsilla.com/blogs/harness-engineering-evolution-prompt-context-autonomous-agents
|
|
90
|
+
- Faros.ai — *Harness Engineering: Making AI Coding Agents Work in 2026* — https://www.faros.ai/blog/harness-engineering
|
|
91
|
+
- Adnan Masood — *Agent Harness Engineering — The Rise of the AI Control Plane* — https://medium.com/@adnanmasood/agent-harness-engineering-the-rise-of-the-ai-control-plane-938ead884b1d
|
|
92
|
+
- getstream.io — *Top AI Agent Protocols in 2026 — MCP, A2A, ACP & More* — https://getstream.io/blog/ai-agent-protocols/
|
|
93
|
+
- Zylos Research — *Agent Interoperability Protocols 2026: MCP, A2A, ACP and the Path to Convergence* — https://zylos.ai/research/2026-03-26-agent-interoperability-protocols-mcp-a2a-acp-convergence
|
|
94
|
+
- codebridge.tech — *Multi-Agent Systems & AI Orchestration Guide 2026* — https://www.codebridge.tech/articles/mastering-multi-agent-orchestration-coordination-is-the-new-scale-frontier
|
|
95
|
+
- Micheal Lanham — *Multi-Agent in Production in 2026: What Actually Survived* — https://medium.com/@Micheal-Lanham/multi-agent-in-production-in-2026-what-actually-survived-f86de8bb1cd1
|
|
96
|
+
- Inference Weekly — *Agentic Harness Engineering (AHE): Evolving Coding-Agent Harnesses with Observability-Driven Automation* — https://medium.com/@harshit.sinha0910/agentic-harness-engineering-ahe-evolving-coding-agent-harnesses-with-observability-driven-297481226663
|
|
97
|
+
- Observability-Driven Automatic Evolution of Coding-Agent Harnesses — https://arxiv.org/pdf/2604.25850
|
|
98
|
+
- TokenMix — *LLM Context Window 2026: 128K to 10M Tokens* — https://tokenmix.ai/blog/llm-context-window-explained
|
|
99
|
+
- dasroot.net — *Token Optimization Strategies for Cost-Effective LLM Applications* — https://dasroot.net/posts/2026/04/token-optimization-llm-costs-prompt-engineering/
|
|
100
|
+
|
|
101
|
+
> Raw collected signal + the improvement-signal processing checklist (working log) are kept in the
|
|
102
|
+
> private companion store, per the public/private split policy — not committed to this public repo.
|
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
# Hub Compounding Loop
|
|
2
|
+
|
|
3
|
+
> Axis-6 automation: the mechanism by which the forge-harness hub improves itself over time through structured feedback cycles.
|
|
4
|
+
|
|
5
|
+
**Principle**: Each session's learnings are absorbed back into the hub so the harness evolves on its own — without requiring manual re-triggering.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Cycle Overview
|
|
10
|
+
|
|
11
|
+
| Cadence | Trigger | Key actions |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| **Per-session** | Session close ("wrap up", "done", "good work") | harvest-loop → card update → push check |
|
|
14
|
+
| **Weekly** | 7 days since last `frontier_digest_*.md` | `/frontier-digest` → CATALOG entry + 6-candidate implementation |
|
|
15
|
+
| **Monthly** | 30 days since last harness-doctor run | `/harness-doctor` → L1~L4 diagnosis + M/S/R prescription |
|
|
16
|
+
| **Quarterly** | ~90 days | Sister asset sync, Phase transition gate review |
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Per-Session Close Chain (Automatic — Not Skippable)
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
Closing phrase detected
|
|
24
|
+
→ ① git diff check
|
|
25
|
+
→ ② if diff exists → harvest-loop
|
|
26
|
+
→ ③ card update (reference_next_session_starter.md) — independent obligation
|
|
27
|
+
→ ④ unpushed commits → propose "push?"
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Card update is NOT a sub-step of harvest-loop — runs even if harvest-loop is skipped.
|
|
31
|
+
|
|
32
|
+
**Real-time tracking**: Complete S-tier/A-tier items get immediately appended to
|
|
33
|
+
`tracks/_meta/fh_completed_{YYYY-MM-DD}.md` (before context compression).
|
|
34
|
+
harvest-loop Step 0-b uses this file — relying on LLM memory after compression causes omissions.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## harvest-loop Pipeline (8 Steps)
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
field-harvest (pattern extraction)
|
|
42
|
+
→ contention-layer (collision signals)
|
|
43
|
+
→ [persona-devil-advocate + persona-innovator] (parallel)
|
|
44
|
+
→ synthesizer (devil/innovator collision harvest)
|
|
45
|
+
→ Critic isolated Agent (SAGE critique)
|
|
46
|
+
→ harness-doctor (health check)
|
|
47
|
+
→ verify-bidirectional (consistency validation)
|
|
48
|
+
→ curator (skill lifecycle management)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Session learnings automatically absorbed back into FH ecosystem.
|
|
52
|
+
|
|
53
|
+
**In main dev env**: runs automatically at session end.
|
|
54
|
+
**For external FH users**: proposes execution first.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Weekly Audit Cycle (Phase 1.5)
|
|
59
|
+
|
|
60
|
+
1. `./tracks/_audit/_scanner.sh "7 days ago"` — aggregates: commits, tags, stale files, sub-agent invocation log, self-asset references
|
|
61
|
+
2. Copy `_template_weekly.md` → `weekly_audit_YYYY-MM-DD.md`
|
|
62
|
+
3. Propose 3-tier improvements (🟥mandatory / 🟧strong / 🟩recommended)
|
|
63
|
+
|
|
64
|
+
**Phase 2 (skill-ized)**: `/harvest-loop` automates the above (manual ~10 min → auto ~3 min target).
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## Cadence Files (Session-Start Auto-Detection)
|
|
69
|
+
|
|
70
|
+
| File | Cadence | Auto-propose condition |
|
|
71
|
+
|---|---|---|
|
|
72
|
+
| `tracks/_meta/frontier_digest_*.md` | 7 days | Propose `/frontier-digest` at session start if 7+ days |
|
|
73
|
+
| `tracks/_meta/*harness_doctor*.md` | 30 days | Propose `/harness-doctor` at session start if 30+ days |
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## 3-Phase Maturity Roadmap
|
|
78
|
+
|
|
79
|
+
| Phase | Name | Criteria |
|
|
80
|
+
|---|---|---|
|
|
81
|
+
| **Phase I** | Entering Maturity | 5-criteria gate — consistent weekly audit, no critical debt, harvest-loop running |
|
|
82
|
+
| **Phase II** | Frontier Following | frontier-digest cadence + sister asset sync + external PR evidence |
|
|
83
|
+
| **Phase III** | Frontier Leading | 6 indicators + writing guide for org-level propagation |
|
|
84
|
+
|
|
85
|
+
**Shared condition for all transitions**: optimization principle — field harness: getting simpler over time; meta-harness: complexity justified by scope (no orphaned/redundant/decorative units).
|
|
86
|
+
|
|
87
|
+
Detailed frame: `hub_maturity_roadmap.md`.
|
|
88
|
+
|
|
89
|
+
---
|
|
90
|
+
|
|
91
|
+
## FH Improvement Signal Recording
|
|
92
|
+
|
|
93
|
+
When friction is detected during a session, record it for the next session's awareness:
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
tracks/_meta/fh_signal_{YYYY_MM_DD}_{source}.md
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Fields: friction point, FH registration candidate, status (pending hub review).
|
|
100
|
+
|
|
101
|
+
**Guard**: 1 file per session (append if same date+source). Structural improvements only — no minor typos.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## Related
|
|
106
|
+
|
|
107
|
+
- `harness_6axis_framework.md` — Axis 6 is the "Improve" step that feeds into this loop
|
|
108
|
+
- `.claude/rules/operations.md` — Sub-agent invocation log, weekly audit scanner detail
|
|
109
|
+
- `.claude/rules/sync_push_protocols.md` — Session Sync Protocol (how learnings enter the loop)
|
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Hub Maturity 3-Phase Roadmap Frame (template)
|
|
3
|
+
description: Long-term evolution path frame for the hub. Phase I (entering maturity) → Phase II (frontier following) → Phase III (frontier leading) 3-stage model. Fixes the gap, output, completion criteria, and transition conditions for each phase as a quarterly re-diagnosis reference document. The maturity axis is the parent frame referenced by monthly level snapshots and quarterly re-diagnosis. **This file is a template — in actual hub operation, add project-specific information (timing, asset names, identifiers) to write an operating copy**.
|
|
4
|
+
type: reference
|
|
5
|
+
date: 2026-04-28
|
|
6
|
+
tags: [harness, maturity, roadmap, 3-phase, frontier-tracking, evolution, compounding, long-term, hub, strategic, template-frame]
|
|
7
|
+
scope: hub-template
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Hub Maturity 3-Phase Roadmap (frame)
|
|
11
|
+
|
|
12
|
+
## Why this document exists
|
|
13
|
+
|
|
14
|
+
If the monthly level snapshot captures **"where are we now"**, this document captures **"where are we going and when do we transition"**. If the quarterly re-diagnosis captures **current position vs industry frontier**, this document captures **the evolution of the relationship with the frontier itself** (follower → leader).
|
|
15
|
+
|
|
16
|
+
**Core vision**:
|
|
17
|
+
|
|
18
|
+
> The hub model matures enough to **enter the maturity phase**, **follow** the frontier's advancement direction periodically, and after that, the path where **we become the frontier**.
|
|
19
|
+
|
|
20
|
+
This vision is made explicit as 3 phases so that at each quarterly re-diagnosis, "which phase are we in · what are the conditions for the next phase transition" can be judged.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 1. 3-Phase Overview
|
|
25
|
+
|
|
26
|
+
| Phase | Scope | Core gap (example) | Representative output (example) | Standard duration (example) |
|
|
27
|
+
|---|---|---|---|---|
|
|
28
|
+
| **I. Entering maturity** | Infrastructure/asset building + routine establishment | Operational gap axes remain · automation not established · 0 external propagation | Weekly audit automation · operations guide · 1-2 external assets · sub-agent judgment · self-diagnosis warning reduction | 3-6 months |
|
|
29
|
+
| **II. Frontier following** | Quarterly re-diagnosis routine + gap auto-detection | Single/dual input sources · no auto gap detection · cadence not confirmed | Quarterly frontier diagnosis 2 times + monthly brief established + external scan cadence confirmed | 5-8 months |
|
|
30
|
+
| **III. Frontier leading** | Self-invented outbound propagation + self-evolving | 0 open-source/presentation record · no self-evolving loop · 0 industry citations | Public refactor · blog/presentation 10+ per year · industry citation case accumulation | Ongoing (no completion) |
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## 2. Current Position (fill in operating copy)
|
|
35
|
+
|
|
36
|
+
Fill in the following format in the operating copy:
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
**Phase X · ~N% progress** — infrastructure/asset accumulation status · N remaining completion gaps
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
### Achievement table (example format)
|
|
43
|
+
|
|
44
|
+
| Area | Status | Basis |
|
|
45
|
+
|---|---|---|
|
|
46
|
+
| 6-axis framework + feedback loop | ✅ | (corresponding asset path) |
|
|
47
|
+
| Package structure alignment | ✅ | (realignment session) |
|
|
48
|
+
| ... | ... | ... |
|
|
49
|
+
|
|
50
|
+
### Gaps remaining
|
|
51
|
+
|
|
52
|
+
See §3 Phase I completion criteria.
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 3. Phase I Completion Criteria (5 measurable criteria)
|
|
57
|
+
|
|
58
|
+
Phase II entry gate passed when all 5 are met.
|
|
59
|
+
|
|
60
|
+
| # | Condition | Measurement method (frame) |
|
|
61
|
+
|---|---|---|
|
|
62
|
+
| 1 | **Weekly audit automation established** | audit skill run 3+ times + manual N min → auto N min measured |
|
|
63
|
+
| 2 | **Axis N operations guide established** | orchestration guide draft + mode switch 3+ cases accumulated (if gap axis exists) |
|
|
64
|
+
| 3 | **External propagation N cases** | Distributed to external channel + external response received (generally 2 cases recommended) |
|
|
65
|
+
| 4 | **Sub-agent pilot promotion/deprecation judgment** | 2+ week observation + invocation log-based judgment (`accepted ≥ 60%` / `rejected ≥ 40%` / `invocation count ≥ N`) |
|
|
66
|
+
| 5 | **Self-diagnosis warning reduction** | Quarterly self-diagnosis warnings N items → 1 or fewer |
|
|
67
|
+
|
|
68
|
+
### Completion checklist derived Decision
|
|
69
|
+
|
|
70
|
+
- Phase I completion date = earliest date all 5 criteria are met
|
|
71
|
+
- Completion confirmation event = **Phase II entry meeting** (1 separate session, §4 entry condition check)
|
|
72
|
+
- No Phase II output (e.g., external GitHub scan) before Phase I completion — simplification principle violation
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## 4. Phase II (Frontier Following) — Entry conditions, cadence, outputs
|
|
77
|
+
|
|
78
|
+
### 4.1 Entry conditions
|
|
79
|
+
|
|
80
|
+
- Phase I completion 5 criteria **all** met
|
|
81
|
+
- Monthly level snapshot updated 2+ consecutive times
|
|
82
|
+
- §2 current position re-judged as "Phase II · 0%"
|
|
83
|
+
|
|
84
|
+
### 4.2 Cadence 3-option comparison
|
|
85
|
+
|
|
86
|
+
| Option | Cycle | Pros | Cons |
|
|
87
|
+
|---|---|---|---|
|
|
88
|
+
| **(a) Quarterly only** | 3 months | Simple | Gap detection delayed 3 months (slow if frontier moves fast) |
|
|
89
|
+
| **(b) Quarterly + monthly light scan** | 3 months + 4 weeks | Gap detection within 1 month + 10 min addition to existing monthly routine (minimum invasive) | (None — recommended) |
|
|
90
|
+
| (c) Trigger-based | When stagnation detected | Resource efficient | Stagnation detection criteria + auto-alerts + trigger tags all require new infra. **Simplification principle violation risk** |
|
|
91
|
+
|
|
92
|
+
→ **Recommended: (b) quarterly + monthly**. Joining existing monthly routine = minimum invasive. Consistent with simplification principle.
|
|
93
|
+
|
|
94
|
+
### 4.3 Representative outputs (frame)
|
|
95
|
+
|
|
96
|
+
| Output | Cycle | Content |
|
|
97
|
+
|---|---|---|
|
|
98
|
+
| **Quarterly frontier diagnosis #N** | 3 months | 6-axis level change vs previous edition + 3-5 new frontier techniques integrated |
|
|
99
|
+
| **Monthly brief** | 4 weeks | 10-min scan of GitHub trending, Anthropic/OpenAI blog, and similar — 3-5 line summary |
|
|
100
|
+
| **External GitHub scan actual operation** | Monthly | Monthly 10-min scan of public repos. Sprint Contract in 5 lines |
|
|
101
|
+
| **Auto gap detection** | Scanner feature addition | Flag "frontier diagnosis not updated > 90 days" |
|
|
102
|
+
|
|
103
|
+
### 4.4 Completion conditions
|
|
104
|
+
|
|
105
|
+
- Quarterly re-diagnosis 2 times + monthly brief 6+ consecutive productions
|
|
106
|
+
- 1+ time **own methodology back-referenced from frontier** found
|
|
107
|
+
- 2+ axes in monthly level snapshot sustained as "leading" judgment
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## 5. Phase III (Frontier Leading) — Entry conditions, 6 indicators, outputs
|
|
112
|
+
|
|
113
|
+
### 5.1 Entry conditions
|
|
114
|
+
|
|
115
|
+
- Phase II completion 3 conditions all met
|
|
116
|
+
- 3+ of N self-invented assets **observed as original concepts** for similar industry concepts
|
|
117
|
+
- Public seed repository external contribution record started (at least one of fork/issue/star)
|
|
118
|
+
|
|
119
|
+
### 5.2 6 Leading indicators
|
|
120
|
+
|
|
121
|
+
| # | Indicator | Measurement |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| 1 | Public seed repository record | star/fork/issue count |
|
|
124
|
+
| 2 | Blog/presentation | 2-3 per quarter · 10+ per year |
|
|
125
|
+
| 3 | Industry citation cases | External articles/seminars citing this hub/methodology |
|
|
126
|
+
| 4 | External organization adoption | Other companies/departments |
|
|
127
|
+
| 5 | Self-evolving loop demonstration | Skill generates skills |
|
|
128
|
+
| 6 | Self-invented industry original recognition | Adopted as industry term/frame |
|
|
129
|
+
|
|
130
|
+
### 5.3 Representative outputs
|
|
131
|
+
|
|
132
|
+
- **Public seed repository refactor** — from personal seed to collaboratable template. Team customization + common protocol separation
|
|
133
|
+
- **10+ blog/presentations per year** — externalize 1 self-invented asset per quarter
|
|
134
|
+
- **Self-evolving loop MVP** — audit skill proposes own skill generation + user approval → auto-generate → usage observation → deprecate/improve
|
|
135
|
+
|
|
136
|
+
### 5.4 Phase III has no "completion"
|
|
137
|
+
|
|
138
|
+
Phase III is an ongoing state. Instead of completion criteria, **3 maintenance conditions**:
|
|
139
|
+
- 3+ of 6 indicators continuously rising
|
|
140
|
+
- "Frontier level maintained" judgment in quarterly re-diagnosis (no regression)
|
|
141
|
+
- Self-diagnosis 8-item checklist failure signal 0 maintained
|
|
142
|
+
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
## 6. Common principles for transitions
|
|
146
|
+
|
|
147
|
+
### 6.1 Consistent simplification principle
|
|
148
|
+
|
|
149
|
+
Phase transitions are **methodology/abstraction level rises**, not **increases in file/skill/rule count**. At each phase transition, do the following first:
|
|
150
|
+
|
|
151
|
+
- [ ] Self-diagnosis 8-item checklist check — confirm 0 failure signals
|
|
152
|
+
- [ ] Files unreferenced 6+ months → move to `archive/` or consolidate
|
|
153
|
+
- [ ] Clean up previous phase temporary outputs made unnecessary by new phase entry
|
|
154
|
+
- [ ] Confirm CLAUDE.md and CATALOG within 200 lines
|
|
155
|
+
- [ ] Re-confirm optimization principle: field harness → simpler over time; meta-harness → complexity earns its scope (purge orphaned/redundant/decorative units)
|
|
156
|
+
|
|
157
|
+
### 6.2 Transition deferred on simplification failure
|
|
158
|
+
|
|
159
|
+
Above checklist not passed → **phase transition deferred**. Perform previous phase remaining work + simplification work for 1 additional month then re-check.
|
|
160
|
+
|
|
161
|
+
### 6.3 Phase regression possible
|
|
162
|
+
|
|
163
|
+
**Regression** from Phase II to Phase I also allowed. E.g., frontier following routine missed 2 consecutive times → "manual re-establishment" re-perform part of Phase I. Do not force linear progression.
|
|
164
|
+
|
|
165
|
+
---
|
|
166
|
+
|
|
167
|
+
## 7. This roadmap update cycle
|
|
168
|
+
|
|
169
|
+
| Trigger | Update content |
|
|
170
|
+
|---|---|
|
|
171
|
+
| Quarterly re-diagnosis | §2 current position re-judgment + §3·§4·§5 criteria change check |
|
|
172
|
+
| Phase transition event | Record corresponding phase completion + initialize next phase progress |
|
|
173
|
+
| Self-invented asset recognized as industry original | §5.1 entry condition counter update |
|
|
174
|
+
| Simplification principle violation detected | §6 transition deferral activation record |
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## 8. Operating copy writing guide (for template users)
|
|
179
|
+
|
|
180
|
+
When moving this frame to an operating copy:
|
|
181
|
+
|
|
182
|
+
1. **Fill §2 current position** — add asset list and basis material paths
|
|
183
|
+
2. **Concretize §3 5 criteria** — add `current` column + `target date` column (e.g., `2026-05-17`)
|
|
184
|
+
3. **Recommend §4.2 cadence (b)** — if adopting other option, need to prove §6.1 simplification gate passed
|
|
185
|
+
4. **Add current counter to §5 6 indicators** — starting from all 0 is natural
|
|
186
|
+
5. **Update §2 at each quarterly re-diagnosis** + record cumulative changes in §7 trigger table
|
|
187
|
+
|
|
188
|
+
---
|
|
189
|
+
|
|
190
|
+
## 9. Related assets
|
|
191
|
+
|
|
192
|
+
- 6-axis framework — `knowledge/shared/harness-core/harness_6axis_framework.md` (frame premise)
|
|
193
|
+
- Monthly level snapshot — `knowledge/shared/harness-core/harness_level_snapshot_*.md` (current position)
|
|
194
|
+
- Quarterly re-diagnosis — `knowledge/shared/harness-core/harness_frontier_diagnosis_*.md` (following basis)
|
|
195
|
+
- Feedback automation — `knowledge/shared/harness-core/hub_compounding_loop.md` (Phase transition gate input)
|
|
196
|
+
|
|
197
|
+
---
|
|
198
|
+
|
|
199
|
+
## 10. One-line conclusion
|
|
200
|
+
|
|
201
|
+
**Phase I completion → Phase II frontier following (b)cadence → Phase III leading. Simplification principle is the common gate for each transition**.
|
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: hybrid-orchestration-architecture-roadmap
|
|
3
|
+
description: Proposed (not-yet-implemented) architecture roadmap for fh as an intelligent hybrid orchestration engine — Claude Code as main driver, other CLIs/APIs as runtime-discovered sidecars, with Zero-Config standalone fallback for plugin-only users. Design intent + reconciliation against existing FH assets.
|
|
4
|
+
type: roadmap
|
|
5
|
+
date: 2026-06-09
|
|
6
|
+
status: proposed (design intent — NOT implemented; see §Status & Reconciliation)
|
|
7
|
+
tags: [hybrid-orchestration, sidecar, zero-config, jit-probing, install-wizard, roadmap, proposed]
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Hybrid Orchestration Architecture — Roadmap (Proposed)
|
|
11
|
+
|
|
12
|
+
> **Status: PROPOSED design intent, not current behavior.** This document captures a
|
|
13
|
+
> forward-looking architecture for fh. Several components it describes — a `config.json`
|
|
14
|
+
> engine-topology file, runtime JIT engine probing, an install-wizard that auto-builds an
|
|
15
|
+
> engine map — **do not exist in FH today**. Read §Status & Reconciliation first to see
|
|
16
|
+
> what is already shipped vs what is aspirational. Nothing here should be cited as a
|
|
17
|
+
> current feature.
|
|
18
|
+
>
|
|
19
|
+
> **Source**: operator design doc (2026-06-09), reconciled to FH conventions on ingest —
|
|
20
|
+
> pinned model versions → `{model-name}` placeholders (per `multi_model_sidecar_strategy.md`
|
|
21
|
+
> §Generalization #4); illustrative pseudo-code marked as such (FH's real layer is markdown
|
|
22
|
+
> methodology + Claude-native automation, not a Python runtime engine).
|
|
23
|
+
|
|
24
|
+
## 1. Overview
|
|
25
|
+
|
|
26
|
+
fh aims to be a **hybrid orchestration** harness: **Claude Code as the main driver (core
|
|
27
|
+
controller)**, with third-party CLIs / APIs mapped in as **sidecars** according to whatever
|
|
28
|
+
the user's local environment makes available. The goal is to escape single-model dependency
|
|
29
|
+
and **progressively enhance** output by discovering the user's own resources (subscription
|
|
30
|
+
CLIs, API keys) at runtime — while still working in a fully **Zero-Config** state for users
|
|
31
|
+
who copy only a single plugin/skill rather than installing the whole framework.
|
|
32
|
+
|
|
33
|
+
This is the design north star. The mechanism for *delegating to sidecars* is already
|
|
34
|
+
validated and shipped (see `multi_model_sidecar_strategy.md`); the *automatic
|
|
35
|
+
discovery/topology* layer below is the proposed addition.
|
|
36
|
+
|
|
37
|
+
## 2. Model roles (provider-agnostic)
|
|
38
|
+
|
|
39
|
+
FH convention forbids pinning version numbers or benchmark figures into durable assets
|
|
40
|
+
(they age and become phantom claims), so roles are stated by function, not by pinned model:
|
|
41
|
+
|
|
42
|
+
| Role | Engine slot | Function | Maps to FH skill family |
|
|
43
|
+
|---|---|---|---|
|
|
44
|
+
| **Core Driver** | `{primary-model}` (strongest available; CC host) | Full code edit, architecture, final synthesis | steel-quench, refactor, design consensus |
|
|
45
|
+
| **Context/Analysis Sidecar** | `{large-context-sidecar}` | Bulk log/API-spec parsing → distilled clues | harvest-loop bulk scan |
|
|
46
|
+
| **Test/Util Sidecar** | `{fast-util-sidecar}` | Boilerplate, unit tests, repetitive drops | speed-run / scaffolding |
|
|
47
|
+
|
|
48
|
+
> Capability/benchmark comparisons between providers change every release cycle — consult a
|
|
49
|
+
> live source (`/frontier-digest`, or the `claude-api` skill for Claude specifics) at decision
|
|
50
|
+
> time rather than trusting any number frozen into this doc. No SWE-bench figures are recorded
|
|
51
|
+
> here by design.
|
|
52
|
+
|
|
53
|
+
## 3. Main driver + sidecar dual channel
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
┌─────────────────────────────┐
|
|
57
|
+
│ user terminal (fh) │
|
|
58
|
+
└──────────────┬──────────────┘
|
|
59
|
+
│ [intelligent routing]
|
|
60
|
+
┌───────────────────┼───────────────────┐
|
|
61
|
+
▼ (bulk / low-cost) ▼ (final reasoning) ▼ (fast unit-test / repeat)
|
|
62
|
+
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
|
63
|
+
│ context sidecar │ │ Claude Code (main)│ │ util sidecar │
|
|
64
|
+
└────────┬────────┘ └────────┬─────────┘ └────────┬────────┘
|
|
65
|
+
└────────────────────┼─────────────────────┘
|
|
66
|
+
▼
|
|
67
|
+
[final quality verification + apply]
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
- **Claude Code (main)** keeps full project context/architecture and is the only writer of
|
|
71
|
+
final code. This matches FH's existing rule: *"Host is always single"*
|
|
72
|
+
(`multi_model_sidecar_strategy.md` §Mechanism).
|
|
73
|
+
- **Sidecars** are stateless one-shot `Bash`-invoked processes whose stdout is folded back in
|
|
74
|
+
by the calling skill — **already the shipped mechanism**, not new.
|
|
75
|
+
|
|
76
|
+
## 4. 3-Tier routing protocol (proposed ordering)
|
|
77
|
+
|
|
78
|
+
When a skill needs a sidecar engine, probe in priority order:
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
[sidecar request]
|
|
82
|
+
├── Tier 1: subscription CLI (zero marginal cost)
|
|
83
|
+
│ discover logged-in binaries: `claude -p`, `aider`, `gemini`, `codex`, `gh copilot`
|
|
84
|
+
├── Tier 2: native API call (pay-per-use)
|
|
85
|
+
│ env keys: GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY
|
|
86
|
+
└── Tier 3: self-contained sub-agent split (fallback)
|
|
87
|
+
no external sidecar → Claude Code spawns isolated sub-agent / prompt-chunking
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
> **Relation to the shipped fallback chain**: `multi_model_sidecar_strategy.md`
|
|
91
|
+
> §Implementation-Patterns already defines a 3-tier *fallback* chain (Copilot CLI → corporate
|
|
92
|
+
> endpoint → direct Gemini/Codex). The ordering here is framed by **cost/access tier** rather
|
|
93
|
+
> than network reachability. These are two lenses on the same fan-out; if implemented, they
|
|
94
|
+
> should be unified into one routing table, not maintained as two competing lists. Tier 3
|
|
95
|
+
> ("Claude internal sub-agent fallback") is the genuinely new contribution — it guarantees the
|
|
96
|
+
> chain never hard-fails even with zero external resources.
|
|
97
|
+
|
|
98
|
+
## 5. Static inheritance + dynamic JIT fallback (proposed)
|
|
99
|
+
|
|
100
|
+
```
|
|
101
|
+
execution request (skill / agent)
|
|
102
|
+
│
|
|
103
|
+
▼
|
|
104
|
+
[config.json present?] ← PROPOSED file; does NOT exist in FH today
|
|
105
|
+
├── Yes → static-inheritance mode → load priority map → run immediately
|
|
106
|
+
└── No → Zero-Config standalone mode → runtime JIT probing:
|
|
107
|
+
1. local subscription CLI scan (Tier 1)
|
|
108
|
+
2. system API-key env scan (Tier 2)
|
|
109
|
+
3. main-driver sub-agent fallback (Tier 3)
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
**5.1 Static config inheritance** — if an install-wizard run had persisted an engine map,
|
|
113
|
+
skills would adopt it as default and skip per-call scan overhead. *(Proposed: install-wizard
|
|
114
|
+
does not generate such a file today.)*
|
|
115
|
+
|
|
116
|
+
**5.2 Runtime JIT probing** — if no config exists (or a tool was added after install), the
|
|
117
|
+
harness probes the live environment on-the-fly via the Tier 1→2→3 protocol and binds the best
|
|
118
|
+
path. All probes failing → safe descent to Tier 3 (main model handles it).
|
|
119
|
+
|
|
120
|
+
**5.3 Zero-Config standalone self-reliance** — the key requirement for **Mode C** (plugin/skill
|
|
121
|
+
copied without the full framework, see `.claude/rules/modes_and_value.md`). With no
|
|
122
|
+
`config.json`, the module must **not error** — it switches to Zero-Config standalone mode and
|
|
123
|
+
JIT-probes for whatever local tools exist, then proceeds quietly. Falls back to Tier 3 if none.
|
|
124
|
+
|
|
125
|
+
```python
|
|
126
|
+
# ILLUSTRATIVE resolution logic — NOT shipped code.
|
|
127
|
+
# FH's actual layer is markdown methodology + Claude-native automation (skills/rules/hooks),
|
|
128
|
+
# not a Python engine. This sketch only shows the intended decision order.
|
|
129
|
+
def resolve_sidecar_engine(preferred="{util-sidecar}"):
|
|
130
|
+
cfg = load_static_config_safe() # Step 1: static config (if any)
|
|
131
|
+
if cfg and cfg.get("mapped_engine"):
|
|
132
|
+
return cfg["mapped_engine"]
|
|
133
|
+
if shutil.which("aider") or shutil.which("{cli}"): # Tier 1: subscription CLI
|
|
134
|
+
return "CLI_WRAPPER_MODE"
|
|
135
|
+
if os.environ.get("{PROVIDER}_API_KEY"): # Tier 2: API key
|
|
136
|
+
return "NATIVE_API_MODE"
|
|
137
|
+
return "CLAUDE_SUBAGENT_FALLBACK" # Tier 3: peaceful fallback
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
## 6. Intelligent install-wizard (proposed extension)
|
|
141
|
+
|
|
142
|
+
Today `install-wizard` sets up the periodic-audit notification structure (zshrc hook +
|
|
143
|
+
sentinels + session-start mtime detection). It does **not** build an engine topology. The
|
|
144
|
+
proposed extension would add, at install time:
|
|
145
|
+
|
|
146
|
+
- **Binary probing** — scan `$PATH` for available CLIs/dependencies.
|
|
147
|
+
- **Profile mapping** — write the discovered hybrid config (the proposed `config.json`).
|
|
148
|
+
- **Sanity check** — a light sidecar call to confirm the pipeline actually works.
|
|
149
|
+
|
|
150
|
+
Onboarding UX phrasing (proposed):
|
|
151
|
+
- Minimal env → *"Configured a Claude-Code-only harness. All skills run safely via context
|
|
152
|
+
splitting, no API key needed."*
|
|
153
|
+
- Expansion nudge → *"To enable cheaper large-analysis skills, register a sidecar API key later;
|
|
154
|
+
fh will auto-detect it on the next run and apply sidecar orchestration."*
|
|
155
|
+
|
|
156
|
+
## 7. sim-conductor & core-skill improvement directions (proposed)
|
|
157
|
+
|
|
158
|
+
- **7.1 `claude -p` non-interactive pipe runtime** — exploit CC's non-interactive flag; pipe
|
|
159
|
+
stdout/stdin between agents; reuse session cache via a harness wrapper.
|
|
160
|
+
- **7.2 Hybrid context bridge** — normalize heterogeneous sidecar output (API JSON vs CLI
|
|
161
|
+
streamed text); embed an extractor (code-block / JSON-structure parser) in a bridge layer so
|
|
162
|
+
callers always get structured responses regardless of invocation path.
|
|
163
|
+
- **7.3 Built-in sub-agent injection** — for minimal envs with no third-party CLI/API: place
|
|
164
|
+
worker prompt specs (e.g. `steel-quench-worker.md`, `harvest-loop-worker.md`) into
|
|
165
|
+
`.claude/agents/`, so the harness spawns Claude's own isolated-context sub-agent pool for
|
|
166
|
+
parallel work. *(Note: per `operations.md`, personal agents in shared repos should be kept
|
|
167
|
+
local via `.git/info/exclude` — an installer-placed agent must respect that boundary.)*
|
|
168
|
+
|
|
169
|
+
---
|
|
170
|
+
|
|
171
|
+
## Status & Reconciliation (read this before citing anything)
|
|
172
|
+
|
|
173
|
+
| Component | In FH today? | Where / note |
|
|
174
|
+
|---|---|---|
|
|
175
|
+
| Sidecar delegation via `Bash` (stateless, host-single) | ✅ Shipped + validated | `multi_model_sidecar_strategy.md` (Experiment 1·2) |
|
|
176
|
+
| Cross-provider perspective-diversity rationale | ✅ Shipped | same doc + `steel-quench` Wave 5 |
|
|
177
|
+
| 3-tier *fallback* chain (network-reachability lens) | ✅ Shipped | same doc §Implementation-Patterns |
|
|
178
|
+
| 3-tier *routing* by cost/access (Tier1 CLI→Tier2 API→Tier3 subagent) | 🟡 Partial | reorders the shipped chain; Tier-3 subagent fallback is new |
|
|
179
|
+
| `config.json` engine-topology file | ❌ Proposed | does not exist |
|
|
180
|
+
| Runtime JIT engine probing | ❌ Proposed | does not exist |
|
|
181
|
+
| Zero-Config standalone auto-heal (Mode C) | ❌ Proposed | concept aligns with Mode C in `modes_and_value.md` |
|
|
182
|
+
| install-wizard builds engine topology / sanity-check | ❌ Proposed | install-wizard today = zshrc hook + sentinels only |
|
|
183
|
+
| Hybrid context bridge / built-in sub-agent injection | ❌ Proposed | sim-conductor roadmap item |
|
|
184
|
+
|
|
185
|
+
**Conflicts resolved on ingest** (per FH conventions): pinned model versions + SWE-bench
|
|
186
|
+
figures dropped → `{model-name}` placeholders (avoids phantom/stale claims); Python
|
|
187
|
+
`engine_resolver.py` marked ILLUSTRATIVE (FH has no Python runtime layer); every non-shipped
|
|
188
|
+
component tagged ❌ Proposed above so this roadmap can never be mistaken for current behavior.
|
|
189
|
+
|
|
190
|
+
## References
|
|
191
|
+
|
|
192
|
+
- `knowledge/shared/harness-core/multi_model_sidecar_strategy.md` — shipped sidecar mechanism + fallback chain (this roadmap extends, does not replace, it)
|
|
193
|
+
- `plugins/fh-meta/skills/install-wizard/SKILL.md` — current install behavior (the topology extension would build on this)
|
|
194
|
+
- `.claude/rules/modes_and_value.md` — Mode C (plugin/skill-only) that Zero-Config self-reliance targets
|
|
195
|
+
- `.claude/rules/operations.md` — sub-agent boundary rules the built-in-injection item must respect
|
|
196
|
+
- `plugins/fh-meta/skills/frontier-digest/SKILL.md` — live source for current model capability/benchmark comparison (do not freeze numbers here)
|