@chrono-meta/fh-gate 1.4.41 → 1.4.42
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +2 -2
- package/CLAUDE.md +65 -130
- package/docs/CONTRIBUTING.md +2 -2
- package/knowledge/shared/dialogue/ai_dialogue_playbook.md +137 -0
- package/knowledge/shared/dialogue/claude_code_runtime_flow.md +170 -0
- package/knowledge/shared/dialogue/memory_intent_recall.md +209 -0
- package/knowledge/shared/harness-core/claude_md_gate_details.md +170 -0
- package/knowledge/shared/harness-core/companion_store_pluggable_cross_audit_2026-06-11.md +118 -0
- package/knowledge/shared/harness-core/crucible_mode.md +112 -0
- package/knowledge/shared/harness-core/deep_research_capability_ladder.md +122 -0
- package/knowledge/shared/harness-core/fh_detail_protocols.md +163 -0
- package/knowledge/shared/harness-core/fh_ecosystem_positioning.md +147 -0
- package/knowledge/shared/harness-core/fh_opencode_governance_wrapper.md +163 -0
- package/knowledge/shared/harness-core/fh_synergy_playbook.md +217 -0
- package/knowledge/shared/harness-core/gate_locality_principle.md +57 -0
- package/knowledge/shared/harness-core/goal_quench_anthropic_issue.md +104 -0
- package/knowledge/shared/harness-core/harness_6axis_framework.md +136 -0
- package/knowledge/shared/harness-core/harness_design_decision_lens.md +108 -0
- package/knowledge/shared/harness-core/harness_frontier_diagnosis_2026-06-02.md +102 -0
- package/knowledge/shared/harness-core/hub_compounding_loop.md +109 -0
- package/knowledge/shared/harness-core/hub_maturity_roadmap.md +201 -0
- package/knowledge/shared/harness-core/hybrid_orchestration_architecture_roadmap.md +196 -0
- package/knowledge/shared/harness-core/live_surface_automation_pattern.md +110 -0
- package/knowledge/shared/harness-core/measurement-integrity-checklist.md +54 -0
- package/knowledge/shared/harness-core/meta_harness_engineering_definition.md +116 -0
- package/knowledge/shared/harness-core/multi_model_sidecar_strategy.md +650 -0
- package/knowledge/shared/harness-core/persona_container_schema.md +172 -0
- package/knowledge/shared/harness-core/return_path_gate.md +120 -0
- package/knowledge/shared/harness-core/self_evolution_routine.md +268 -0
- package/knowledge/shared/harness-core/skill_quality_rubric.md +71 -0
- package/knowledge/shared/harness-core/tpa_schema.md +136 -0
- package/package.json +3 -2
|
@@ -0,0 +1,650 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: multi-model-sidecar-strategy
|
|
3
|
+
description: Pattern for invoking other AI models (Gemini, Codex, Copilot CLI) as sidecars from within a Claude Code / FH session via Bash tool. Token economy, model-access fallback, and adversarial diversity use cases.
|
|
4
|
+
date: 2026-05-31
|
|
5
|
+
tags: [multi-model, sidecar, token-economy, model-access, adversarial, internally-validated]
|
|
6
|
+
status: mechanism-validated (cross-provider grader confirmed 2026-06-02)
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
> **Validation status** (updated 2026-06-02): mechanism validated by cross-provider grader.
|
|
10
|
+
>
|
|
11
|
+
> 2026-06-01 steel-quench (Issue #47): mechanism confirmed runnable — implementation shipped in PR #36/#37. Original empirical claims (Experiment 1·2) were an internal same-session self-report; raw transcripts not retained, codex grader blocked by network policy.
|
|
12
|
+
>
|
|
13
|
+
> 2026-06-02 update: Gemini 0.41.2 cross-provider grader run on `pipeline-conductor/SKILL.md` (retained transcript: `tracks/_meta/grader_gemini_pipeline_conductor_2026_06_02.txt`). Gemini found 3 S-grade findings (interaction deadlock, PR-approval deadlock, cadence-lock deadlock); Claude Sonnet-4.6 previously found 3 different S-grade findings (model conflict, invocation contradiction, self-referential sweep). **Zero overlap across 6 S-grade findings** — validates the non-overlapping failure modes claim and perspective diversity mechanism. Provider-identity diversity is empirically confirmed; specific Experiment 2 finding counts on goal-quench (original target) are not directly re-run. Record: `tracks/_meta/grader_gemini_pipeline_conductor_2026_06_02.txt`.
|
|
14
|
+
|
|
15
|
+
# Multi-Model Sidecar Strategy
|
|
16
|
+
|
|
17
|
+
## Thesis grounding — hierarchy of differentiation
|
|
18
|
+
|
|
19
|
+
FH's paper thesis: **the harness (specialized shell) is the durable layer; the model (core) converges across providers.**
|
|
20
|
+
|
|
21
|
+
Empirical observation from 3-round orchestrator-swap experiment (Claude → Gemini → Codex as orchestrator, others as sidecars, same FH skill as target):
|
|
22
|
+
- **Process differs** — each orchestrator highlights different angles (Gemini: state machine + audit methodology; Codex: implementation-level parsing and dependency contracts).
|
|
23
|
+
- **Results converge** — 3 critical issues appeared in every round regardless of orchestrator identity (freshness guard pseudocode, BLOCKED deadlock, interim-commit false CLEAN).
|
|
24
|
+
|
|
25
|
+
This is precisely what the thesis predicts. But the experiment revealed a second-order effect not captured in the original framing:
|
|
26
|
+
|
|
27
|
+
**Process divergence → cross-wave delta → better convergence.**
|
|
28
|
+
|
|
29
|
+
The sidecar pattern's value is not parallelism — it is the *delta*. Each orchestrator's unique process angle produces non-overlapping findings. When synthesized, the combined result exceeds what any single-model run achieves. This is the mechanism by which the sidecar pattern compounds into a quality improvement, not just a coverage check.
|
|
30
|
+
|
|
31
|
+
The enabling condition for this entire chain is the harness:
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
Harness (FH)
|
|
35
|
+
→ consistent SKILL.md format loadable in all 3 CLIs
|
|
36
|
+
→ same skill runs under each orchestrator with comparable evaluation protocol
|
|
37
|
+
→ cross-wave delta is structurally comparable (apples-to-apples)
|
|
38
|
+
→ synthesis is valid
|
|
39
|
+
→ convergence improves
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Without the harness, "multi-model adversarial review" is three separate unstructured prompts — no comparable protocol, no valid synthesis. The harness is not one of the differentiators in a hierarchy. **It is the activation condition that makes the hierarchy operate.**
|
|
43
|
+
|
|
44
|
+
Refined hierarchy:
|
|
45
|
+
|
|
46
|
+
| Level | What it is | Role |
|
|
47
|
+
|---|---|---|
|
|
48
|
+
| **Harness (present vs absent)** | Structured skill methodology, memory, verification protocol | **Activation condition** — makes all lower levels function |
|
|
49
|
+
| Orchestrator-swap sidecar | Process divergence → cross-wave delta → better convergence | Compounds quality via synthesis |
|
|
50
|
+
| Model A vs Model B (no harness) | Process differs, results converge weakly | Small, unstructured |
|
|
51
|
+
|
|
52
|
+
**The sidecar pattern is a quality compounding mechanism, not merely a process tool.** Process divergence produces insights that single-model runs miss. Those insights, when synthesized by the harness skill, produce a convergence that is richer than any single model's output. The harness is what makes this synthesis structurally valid.
|
|
53
|
+
|
|
54
|
+
Two shells — do not conflate:
|
|
55
|
+
|
|
56
|
+
| Shell | What it is | Layer |
|
|
57
|
+
|---|---|---|
|
|
58
|
+
| **Harness shell** (FH, forge-harness) | Methodology, rules, memory, skills — *specialization* accumulated over time | Durable orchestration layer — thesis subject |
|
|
59
|
+
| **Router shell** (Copilot CLI, Gemini CLI, Codex CLI) | Thin routing / access tool that selects and forwards to a model | Delivery / access mechanism — not a harness |
|
|
60
|
+
|
|
61
|
+
FH is the orchestrator. A router shell (or direct model CLI) is one kind of sidecar. Calling Copilot CLI "the harness" is a category error.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Native cross-CLI portability (validated)
|
|
66
|
+
|
|
67
|
+
FH's SKILL.md format is shared across all three major AI CLIs. FH skills load natively in each environment without conversion or adaptation:
|
|
68
|
+
|
|
69
|
+
| CLI | Load mechanism | Status |
|
|
70
|
+
|---|---|---|
|
|
71
|
+
| **Claude Code** | `.claude/plugins/` native | ✅ Primary environment |
|
|
72
|
+
| **Codex CLI** | `SKILL.md` plugin (`fh-meta@forge-harness`) | ✅ Validated |
|
|
73
|
+
| **Gemini CLI** | `gemini skills install <path> --consent` | ✅ Validated — ⚠️ direct `gemini` CLI EOL 2026-06-18 |
|
|
74
|
+
| **Antigravity CLI** (`agy`) | `agy plugin install <local-path>` — Claude `.claude-plugin/` layout handled without conversion (bulk `agy plugin import claude` also exists) | ✅ Validated 2026-06-11 (agy 1.0.7) — fh-meta imported (32 skills + 7 agents); print-mode load probe reproduced phantom-quench purpose/trigger/Phantom-Claim definition from SKILL.md |
|
|
75
|
+
|
|
76
|
+
This means FH is not just model-agnostic in theory — the methodology layer physically runs in multiple CLI environments without modification. This is the empirical foundation for the cross-CLI portability claim.
|
|
77
|
+
|
|
78
|
+
> **Gemini-CLI → Antigravity migration (2026-06-18)**: **closed 2026-06-11, ahead of EOL** — same cheap
|
|
79
|
+
> gate Codex/Gemini passed. `agy plugin install ./plugins/fh-meta` (agy 1.0.7) imported 32 skills +
|
|
80
|
+
> 7 agents straight from the Claude `.claude-plugin/` layout (note: standalone `agy plugin validate`
|
|
81
|
+
> expects a root `plugin.json`, but `install` does not). Load probe in print mode answered
|
|
82
|
+
> phantom-quench's purpose, a trigger phrase, and the Phantom-Claim definition accurately and named
|
|
83
|
+
> steel-quench as a second visible skill. Two same-day measurements:
|
|
84
|
+
> (1) `agy -p` (print mode) **auto-approves tool calls** — it ran a shell command and read a file
|
|
85
|
+
> outside the workspace with no permission prompt (nonce-verified, not hallucinated). Headless sidecar
|
|
86
|
+
> dispatch therefore works like `codex exec` — no approval mode required; permission prompts belong to
|
|
87
|
+
> interactive mode.
|
|
88
|
+
> (2) Post-EOL caveat on the Gemini row: the binary outlives the backend, so `command -v gemini` keeps
|
|
89
|
+
> passing after 2026-06-18 while pipes degrade to empty output behind `2>/dev/null` guards — a silent
|
|
90
|
+
> degradation. Once EOL hits, sidecar team detection should probe `agy` first (follow-up:
|
|
91
|
+
> `fh_signal_2026-06-11_fh-direct`).
|
|
92
|
+
|
|
93
|
+
> **Load ≠ full parity**: "loads and lists as Enabled" means the SKILL.md *methodology* is readable and runnable as guidance. Skills whose steps dispatch a sub-agent (`Agent` tool, `fh-commons:*` challengers) or depend on slash-commands/hooks are **Claude-native** — on Gemini/Codex they degrade to manual methodology, not automated execution. Cross-CLI portability covers the *methodology layer*; the automation layer (sub-agents, hooks, slash commands) requires Claude Code as host. See `README.md §2-layer architecture`.
|
|
94
|
+
|
|
95
|
+
> **Check-class taxonomy = the model-portability map**: the Axis-5 check classes
|
|
96
|
+
> (`harness_6axis_framework.md` §Axis 5) double as the portability map. **Mandatory-pass** checks —
|
|
97
|
+
> and the mechanical subset of **measured** checks (scripts and greps that run with no model) — are
|
|
98
|
+
> model-independent *by construction*; model-produced measured scores (e.g. tier batteries) are
|
|
99
|
+
> tracked, not assumed flat. **Judged** checks are where model choice matters most, and
|
|
100
|
+
> judged-pairing bounds that sensitivity. Two consequences: (1) pushing a workflow's checks toward
|
|
101
|
+
> mechanical classes is a high-leverage portability move wherever mechanization is feasible — a
|
|
102
|
+
> mechanized check ports to any backend the moment it ships; (2) when auditing what survives a
|
|
103
|
+
> backend/tier swap, classify by check class first and re-validate the judged steps (the same
|
|
104
|
+
> discipline as §Floor governance's below-floor re-run rule, below) instead of re-testing
|
|
105
|
+
> everything. Judged-class tier sensitivity is what `docs/OUTPUT_EVIDENCE.md` actually measured —
|
|
106
|
+
> the "by construction" claim rests on the no-model property, not on that single trial.
|
|
107
|
+
|
|
108
|
+
## Orchestrator–sidecar model strategy
|
|
109
|
+
|
|
110
|
+
Use the strongest available model as orchestrator; delegate subsidiary tasks to lighter sidecar models for token economy:
|
|
111
|
+
|
|
112
|
+
| Role | Model selection | Rationale |
|
|
113
|
+
|---|---|---|
|
|
114
|
+
| **Orchestrator** | Strongest available (CC=Opus, Gemini=Pro, Codex=GPT-5.5) | Design, judgment, synthesis |
|
|
115
|
+
| **Sidecar** | Lighter versions (Gemini Flash, GPT-4o-mini, etc.) | Repetitive verification, adversarial passes, token-efficient delegation |
|
|
116
|
+
|
|
117
|
+
This combination can be freely mixed across CLIs — e.g., Gemini Pro orchestrating with Claude Haiku as sidecar, or CC Opus orchestrating with Gemini Flash. FH methodology works regardless of which combination is chosen.
|
|
118
|
+
|
|
119
|
+
## Runtime Authority — single governor + routed sidecars
|
|
120
|
+
|
|
121
|
+
> **Canonical doctrine (single source — runtime files point here, do not restate per-runtime).** The
|
|
122
|
+
> main/governor runtime is **context-specific and must be explicit**. Other models are
|
|
123
|
+
> **capability-routed sidecars** unless a document explicitly defines a limited-runtime entrypoint.
|
|
124
|
+
> **Sidecar findings are evidence candidates, not terminal verdicts, until source-closed by the governor
|
|
125
|
+
> *via a mechanical anchor* (a local hit / literal source span / passing gate) — never governor agreement
|
|
126
|
+
> alone** (the no-judge-only-path / mechanical-anchor principle, CLAUDE.md §FH Improvement 4-Axis Auto-Gate).
|
|
127
|
+
|
|
128
|
+
FH is a **multi-runtime harness with explicit runtime authority**, not Claude-only:
|
|
129
|
+
|
|
130
|
+
| Runtime | Authority | Fit task-class (aggressive *within* it, not blanket) |
|
|
131
|
+
|---|---|---|
|
|
132
|
+
| **Claude Code** | default **action/governor** — Claude-native automation, hooks, agents, MCP writeback, terminal verdict | orchestration · design-depth · synthesis · the writeback/commit path |
|
|
133
|
+
| **Codex** | **audit / repo-grounded sidecar** | file reads · grep/source-close · diff & patch · gate execution · phantom/backtrace (`fh-run` / `codex exec`). **NOT** discovery/design-depth — below-floor there |
|
|
134
|
+
| **Gemini / agy** | **breadth / multimodal sidecar** | wide alt-generation · multimodal/video/image · exploratory critique · non-code artifacts. Outputs stay **source candidates** until grounded |
|
|
135
|
+
| **Other runtimes** | **portable entrypoint only** (`AGENTS.md`) | apply FH methodology via adapter; do **not** inherit Claude-native automation or writeback authority unless explicitly scoped |
|
|
136
|
+
|
|
137
|
+
A sidecar is recruited where it adds *decorrelated* value; its ceiling is still set by the governor — the
|
|
138
|
+
harness lifts a model to its own ceiling, it does not move it ([[feedback_harness_ceiling_principle]]).
|
|
139
|
+
"Aggressive" Codex/Gemini use is bounded by the fit task-class above, never a blanket main-seat swap.
|
|
140
|
+
|
|
141
|
+
**Maintenance-Cost Rule** — a compatibility layer is cheap as a *thin entrypoint*, expensive when it
|
|
142
|
+
*duplicates canonical knowledge*. The test:
|
|
143
|
+
|
|
144
|
+
| Pattern | Cost | Verdict |
|
|
145
|
+
|---|---|---|
|
|
146
|
+
| Runtime-specific `AGENTS.md` with capability routing + stop lines | Low | ✅ Good |
|
|
147
|
+
| Sidecar-invocation docs pointing to canonical skills/agents | Low | ✅ Good |
|
|
148
|
+
| Generated wrappers with drift checks | Medium | Use only if a need is proven |
|
|
149
|
+
| Manual copies of agent/skill bodies per runtime | High | ✗ Avoid |
|
|
150
|
+
| "Every model can be main" governance language | High (hidden) | ✗ Avoid |
|
|
151
|
+
|
|
152
|
+
Keep canonical assets **single-source**: the method/agent/skill body lives once; runtime-specific files
|
|
153
|
+
only describe *what auto-loads · what does not · allowed operations · handoff/sidecar invocation · hard
|
|
154
|
+
stop lines*.
|
|
155
|
+
|
|
156
|
+
## Scope vs steel-quench Wave 5
|
|
157
|
+
|
|
158
|
+
This document is the **rationale layer** (why sidecars, when, what value, what boundaries). `steel-quench/SKILL.md` Wave 5 (Multi-Team Adversarial Panel) is the **implementation layer** — the runnable team-formation + parallel-dispatch + cross-team-synthesis steps. They are not redundant: a skill cites this doc for *why* and *when*; Wave 5 (and any other caller) owns the *how*. If the how appears in two places, Wave 5 is canonical and this doc defers to it.
|
|
159
|
+
|
|
160
|
+
## The capability
|
|
161
|
+
|
|
162
|
+
**Any FH user can delegate tasks to other models as sidecars** from within a session. The host orchestrator is always the primary CLI + FH. The sidecar is invoked via `Bash` tool — no special integration required.
|
|
163
|
+
|
|
164
|
+
### Available sidecar paths (use whichever your environment allows)
|
|
165
|
+
|
|
166
|
+
| Sidecar | Invocation | Model access |
|
|
167
|
+
|---|---|---|
|
|
168
|
+
| **Gemini CLI** | `echo "prompt" \| gemini --skip-trust` or `gemini -p "prompt"` | Gemini family. ⚠️ **the direct `gemini` CLI is being sunset (vendor EOL 2026-06-18)** → its successor is the Antigravity router-shell (`agy`) or the Gemini API (Tier 2). `--skip-trust` is required outside a trusted directory (headless). A *pure-text* prompt needs no tool-permission flag. |
|
|
169
|
+
| **Antigravity CLI** (`agy`) | `echo "prompt" \| agy` or `agy -p "prompt" --model "<name>"` | **Router-shell** (route-to-model selector, same class as Copilot — *not* a harness): model-selectable across providers (e.g. Gemini 3.x · Claude Sonnet/Opus · GPT-OSS-120B). For cross-provider **diversity** pick a *non-Claude* model from a Claude host (Claude→Claude = no divergence — see §Boundaries). Agentic: `-p` pre-flights tool permission, so a headless tool-using run needs `--dangerously-skip-permissions` (or run under the host's approval mode); a pure-text run does not. |
|
|
170
|
+
| **Codex CLI** | `npx @openai/codex exec "prompt"` (or `codex exec --skip-git-repo-check -`) | GPT-4o / GPT-5.5 (non-interactive exec mode — true headless, no permission pre-flight) |
|
|
171
|
+
| **Copilot CLI** (`gh copilot`) | `gh copilot -- -p "prompt" --allow-all-tools` | Copilot model catalog — **subscription-dependent**: preview = claude-haiku-4.5 + gpt-5-mini; international/enterprise subscription = GPT-5.5, Claude Opus, etc. Verify: `gh copilot -- -p "list available models"`. Router-shell (agentic) — same permission-preflight note as `agy`. |
|
|
172
|
+
|
|
173
|
+
### Binary names churn — probe by capability, never pin a name
|
|
174
|
+
|
|
175
|
+
The Gemini-CLI→Antigravity migration (direct `gemini` EOL 2026-06-18) is the live proof: a Tier-1
|
|
176
|
+
entry pinned to the literal `gemini` binary goes stale on a fixed date. The resolution protocol below
|
|
177
|
+
therefore probes for *whichever route exists* (direct CLI · router-shell · API), and Gemini access
|
|
178
|
+
simply migrates `gemini` → (`agy` | API) without changing the methodology. **Router-shells (`agy`,
|
|
179
|
+
`gh copilot`) are a Tier-1 *class*, not a harness** (§Two shells) — they select+forward a model; FH
|
|
180
|
+
governs which/when.
|
|
181
|
+
|
|
182
|
+
---
|
|
183
|
+
|
|
184
|
+
## Sidecar Engine Resolution Protocol (Zero-Config default)
|
|
185
|
+
|
|
186
|
+
**Problem this solves**: skills across FH say *"Gemini sidecar if available"* / *"if external
|
|
187
|
+
CLIs available"* without a shared definition of how "available" is decided. This is the canonical
|
|
188
|
+
resolution recipe — every sidecar-invoking skill resolves engine availability through it, so a
|
|
189
|
+
user who configured nothing still gets intelligent multi-model use, and a plugin-only (Mode C)
|
|
190
|
+
user never hits a hard error.
|
|
191
|
+
|
|
192
|
+
**Core principle — discovery is automatic; invocation stays value-gated.** Probing the
|
|
193
|
+
environment is cheap (shell `command -v` + env-var check, near-zero cost), so it runs **by
|
|
194
|
+
default** on every sidecar-eligible step. *Actually invoking* a sidecar still passes the value
|
|
195
|
+
test in §When NOT to invoke — intelligent use, not indiscriminate fan-out. "Default multi-AI"
|
|
196
|
+
means FH auto-knows what is available and uses it when the task warrants it, never that every
|
|
197
|
+
task sprays calls to every model.
|
|
198
|
+
|
|
199
|
+
**Resolution order (Tier 1 → 2 → 3)** — bind the first tier that resolves:
|
|
200
|
+
|
|
201
|
+
```bash
|
|
202
|
+
# Tier 1 — subscription / logged-in CLI (zero marginal cost; preferred).
|
|
203
|
+
# Probe by CAPABILITY, never pin a fixed binary name — names churn: the direct `gemini`
|
|
204
|
+
# CLI is sunset 2026-06-18, its Tier-1 successor is the `agy` (Antigravity) router-shell.
|
|
205
|
+
# Direct provider CLIs + router-shells (agy/gh-copilot) are all Tier-1 routes.
|
|
206
|
+
for cli in gemini agy codex aider; do command -v "$cli" >/dev/null 2>&1 && echo "tier1:$cli"; done
|
|
207
|
+
command -v gh >/dev/null 2>&1 && gh copilot --help >/dev/null 2>&1 && echo "tier1:gh-copilot"
|
|
208
|
+
# (codex may be `npx @openai/codex` when not on PATH; agy/gh-copilot are router-shells —
|
|
209
|
+
# model-selectable, so pick a non-Claude model from a Claude host for genuine diversity)
|
|
210
|
+
|
|
211
|
+
# Tier 2 — native API key (pay-per-use; only if no Tier-1 CLI)
|
|
212
|
+
for k in GEMINI_API_KEY OPENAI_API_KEY ANTHROPIC_API_KEY; do
|
|
213
|
+
[ -n "${!k:-}" ] && echo "tier2:$k"
|
|
214
|
+
done
|
|
215
|
+
|
|
216
|
+
# Tier 1.5 — OAuth-proxy ("anti-api" class): a local OpenAI-compatible server that fronts a
|
|
217
|
+
# logged-in CLI/OAuth session, letting any OpenAI-spec client use that subscription as a
|
|
218
|
+
# backend. Verified instance: `hermes proxy start` ("forwards OpenAI-compatible requests to
|
|
219
|
+
# an OAuth-authenticated provider; external apps point at the proxy with any bearer token" —
|
|
220
|
+
# upstreams: Nous Portal, xAI Grok). Sits between Tier 1 (the session itself) and Tier 2
|
|
221
|
+
# (a raw key): zero marginal cost like Tier 1, API-shaped like Tier 2.
|
|
222
|
+
# Probe detects the capability only — login state is shown by `hermes proxy status` and a
|
|
223
|
+
# non-logged-in upstream reads "not logged in" (substring-greps for "logged in" false-match it).
|
|
224
|
+
command -v hermes >/dev/null 2>&1 && echo "tier1.5:hermes-proxy (candidate — verify an upstream is logged in via: hermes proxy status)"
|
|
225
|
+
# Two cautions, always surfaced when this rung binds: (a) wrapping an OAuth session into an
|
|
226
|
+
# API surface may sit outside the provider's ToS — operator's call, per provider; (b) from
|
|
227
|
+
# 2026-06-15, proxied **Claude-subscription** usage = third-party-agent class = the metered
|
|
228
|
+
# hard-capped credit pool, not the subscription pool — prefer non-Claude upstreams here.
|
|
229
|
+
|
|
230
|
+
# Tier 3 — guaranteed fallback: Claude Code's own isolated sub-agent (always available)
|
|
231
|
+
# No external resource → orchestrator spawns an Agent(subagent_type=…) / prompt-chunking.
|
|
232
|
+
# This tier never fails, so the chain has no hard-error state. Same-provider, so it serves
|
|
233
|
+
# model-access/parallelism, NOT cross-provider diversity (see §Boundaries).
|
|
234
|
+
echo "tier3:claude-subagent" # used when Tiers 1–2 resolve nothing
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
**Verdict mapping**:
|
|
238
|
+
- Tier 1/2 resolved **and cross-provider** → genuine diversity wave (primary use case).
|
|
239
|
+
- Only Tier 3 available → no diversity; proceed with the Claude sub-agent (no error, reduced value).
|
|
240
|
+
- The resolution result is **advisory** to the caller's own value test — a resolved engine is
|
|
241
|
+
*usable*, not *mandatory*.
|
|
242
|
+
|
|
243
|
+
**Relation to the §Implementation-Patterns fallback chain**: that chain degrades *when a chosen
|
|
244
|
+
path is blocked* (network / quota); this protocol decides *what exists in the first place*. Same
|
|
245
|
+
fan-out, two moments — resolve first (this), degrade-on-failure second (that). Do not duplicate
|
|
246
|
+
the tier list into callers; cite this section.
|
|
247
|
+
|
|
248
|
+
**Skills that resolve through this protocol** (wired 2026-06-09): `goal-quench` (Step D sidecar
|
|
249
|
+
routing), `steel-quench` (Wave 5 / runtime-adapter fallback), `harvest-loop` (Step 3.5-X
|
|
250
|
+
cross-validation). Other sidecar-using skills (`sim-conductor`, `pipeline-conductor`,
|
|
251
|
+
`agent-composer`) inherit by reference — when they say "if available", availability = this
|
|
252
|
+
protocol's verdict.
|
|
253
|
+
|
|
254
|
+
### Video-extraction capability-gated router (added 2026-06-15)
|
|
255
|
+
|
|
256
|
+
Video understanding is a **task-specific specialization** of the resolution ladder above — the
|
|
257
|
+
general "which engine" question, narrowed to "which path can actually *watch* this video". Two axes
|
|
258
|
+
decide the route: **capability** (what's installed) and **length** (frame-extraction tools go sparse
|
|
259
|
+
past ~10 min).
|
|
260
|
+
|
|
261
|
+
| Condition (probe in order) | Route | Why |
|
|
262
|
+
|---|---|---|
|
|
263
|
+
| `command -v gemini` **or** `agy` (or `GEMINI_API_KEY`) | **Gemini / agy sidecar** — URL-native ingest, **no frame cap**, handles long/dense video | Native multimodal; the operator-verified YouTube-URL path (`gemini --skip-trust -p "<URL>"`). ⚠ direct `gemini` EOL 2026-06-18 → `agy` successor / Gemini API (same Tier-floor migration as the text ladder) |
|
|
264
|
+
| Claude-only host (no Gemini/agy), `ffmpeg`+`yt-dlp` present | **claude-video `/watch`** — ffmpeg frames (≤2fps/100, **best ≤10 min**) → Claude's own vision Read; captions via yt-dlp/Whisper | No separate multimodal runtime, **no Gemini-EOL exposure** — Claude watches natively. **3rd-party executable — review before install** (`/plugin marketplace add bradautomates/claude-video`; brew installs + Whisper egress) |
|
|
265
|
+
| Cloud / text-only sandbox (host blocks video) | **remote transcript MCP** | URL hosts 403 in cloud VMs; transcript-only is the floor (the daily-routine sandbox case) |
|
|
266
|
+
|
|
267
|
+
**2nd axis — length**: short/medium → claude-video is fine and keeps everything in-Claude; long or
|
|
268
|
+
visually-dense → prefer Gemini/agy (claude-video's frame cap goes sparse past ~10 min).
|
|
269
|
+
|
|
270
|
+
**Governance invariant (do not drop)**: whichever route runs, the **governor never trusts a
|
|
271
|
+
multimodal "comprehensive read"** as a terminal verdict — a video claim (timestamped summary) is a
|
|
272
|
+
*claim*, cross-checked on a second surface (e.g. Codex clicking the timestamp, or a caption grep)
|
|
273
|
+
before it anchors a decision (`cross_runtime_routing` debate-loop; judge-robustness "judged →
|
|
274
|
+
mechanical"). claude-video is the *executor* FH lacked; FH's cross-check discipline is the
|
|
275
|
+
*governance* claude-video has no equivalent for — compose, don't trust. (Sister-asset intake
|
|
276
|
+
recorded in the private companion store's sister-asset signals.)
|
|
277
|
+
|
|
278
|
+
### Tier-floor resolution — the model dimension (added 2026-06-10)
|
|
279
|
+
|
|
280
|
+
The ladder above resolves **which engine**; this subsection resolves **which model tier within it**.
|
|
281
|
+
Same shape: an asset declares a requirement, the environment resolves it, and resolution never
|
|
282
|
+
hard-fails.
|
|
283
|
+
|
|
284
|
+
**Asset-side floor**: an asset may declare a minimum model tier via its existing
|
|
285
|
+
`model:`/`complexity_routing` frontmatter — that declaration is a **floor, not a pin**. Floors are
|
|
286
|
+
measured-or-justified, never asserted (same discipline as check-class declaration): e.g.
|
|
287
|
+
`quench-challenger` floors at opus (adversarial increment-finding is the product — the
|
|
288
|
+
depth-sensitive class), while Wave-T and `harness-doctor` floor at sonnet (measured: T-3 blind
|
|
289
|
+
3/3 on both tiers, 2026-06-10), and mechanical scripts have no floor at all (measured tier-flat).
|
|
290
|
+
|
|
291
|
+
**Composition order — engine first, floor second (does NOT reorder the engine ladder)**: the Tier 1→2→3
|
|
292
|
+
Engine Resolution above keeps full authority over *which engine* runs — including its **perspective-
|
|
293
|
+
diversity-first** doctrine (a cross-provider Tier-1 CLI beats a same-provider Claude sub-agent whenever
|
|
294
|
+
diversity is the point, e.g. Wave 5). Tier-floor resolution then applies **within the chosen engine**:
|
|
295
|
+
it asks "does this engine reach the asset's floor?", never "should we switch to a Claude sub-agent
|
|
296
|
+
because it has opus?". A floor is satisfied by the chosen engine's **strongest floor-equivalent tier**
|
|
297
|
+
(e.g. an external CLI's top model for an opus-floored asset) — floors measure strength, not provider.
|
|
298
|
+
|
|
299
|
+
**Floor resolution within the chosen engine**:
|
|
300
|
+
- **F1 — engine reaches the floor**: dispatch at the floor tier (native CC `Agent(model=…)` when the
|
|
301
|
+
chosen engine is the Claude sub-agent; the CLI/API's floor-equivalent tier when it is external).
|
|
302
|
+
The session's default model stays untouched — FH **never switches the user's session model**;
|
|
303
|
+
floors apply only to FH's own dispatches.
|
|
304
|
+
- **F2 — engine tops out below the floor** (e.g., a Sonnet-only API-routed environment, or an external
|
|
305
|
+
CLI whose available tiers sit below floor-equivalent): **run anyway at the engine's best tier +
|
|
306
|
+
mandatory below-floor flag.** Never hard-fail — the analog of Tier 3's no-hard-error guarantee.
|
|
307
|
+
The output header carries e.g. `challenger: sonnet (below-floor; floor=opus)`, and a **judged verdict
|
|
308
|
+
produced below floor is auto-tagged a re-quench candidate** for when a floor-tier becomes available —
|
|
309
|
+
degraded delivery is explicit, never silent coverage. Do **not** abandon a diversity-chosen engine
|
|
310
|
+
merely because it is below floor — a below-floor cross-provider pass still buys the diversity the
|
|
311
|
+
step exists for; flag it and proceed.
|
|
312
|
+
|
|
313
|
+
**Floor governance** (from the design's own first quench — challenger findings, both tiers):
|
|
314
|
+
- **Cross-provider equivalence is opt-in with evidence, never assumed**: an external engine's tier
|
|
315
|
+
counts as **below-floor by default** until a measured equivalence entry exists (recorded in the floor
|
|
316
|
+
table above with its evidence citation, same discipline as the floor declarations themselves). The
|
|
317
|
+
backend×tier ladder measurement is exactly what produces these entries — name-matching or vibes never
|
|
318
|
+
satisfies a floor.
|
|
319
|
+
- **Below-floor judged verdicts are provisional, and the queue has a consumer**: a judged-class verdict
|
|
320
|
+
produced below floor does **not** count as gate-PASS evidence for promotion/merge until either a
|
|
321
|
+
floor-tier re-run or an explicit operator acceptance. The **weekly audit** (run at the highest
|
|
322
|
+
available tier per the cadence-compensation policy) is the standing consumer: it scans recent quench
|
|
323
|
+
records for `below-floor` markers and re-runs or explicitly writes them off — flags terminate, they
|
|
324
|
+
do not decorate. For the hub's own 4-axis gate this is now **mechanically enforced**: the Axes 2+3
|
|
325
|
+
pass marker must carry structured floor fields (engine / producing model / floor-status) and the
|
|
326
|
+
pre-commit hook blocks a below-floor marker that lacks an explicit operator ack — self-flagging in
|
|
327
|
+
prose proved salience-fragile at any tier (the adversarial-floor-enforcement signal's A/B finding).
|
|
328
|
+
- **`floor: hard` for depth-critical judged roles** (quench-challenger, apex-review): for these, the
|
|
329
|
+
floor outranks diversity — prefer a floor-meeting engine (incl. the native Tier-3 sub-agent at floor)
|
|
330
|
+
over a below-floor diversity engine. Only when *no* engine anywhere reaches the floor does F2 apply
|
|
331
|
+
(never-hard-fail preserved). When a below-floor engine is deliberately chosen over an available
|
|
332
|
+
floor-meeting one (diversity genuinely the point), record a one-line `diversity_rationale`
|
|
333
|
+
(the existing `sidecar_rationale` convention) — no rationale, floor-meeting engine wins the tie.
|
|
334
|
+
- **Tier-flat / floor claims age**: no-floor and floor declarations cite their basis and are
|
|
335
|
+
re-validated opportunistically when the asset next changes (the standing backfill rule) — a floor
|
|
336
|
+
without a cited basis is treated as unjustified by reviewers.
|
|
337
|
+
|
|
338
|
+
**Human override is inviolable — and a pin is not a cap**: if the operator pins a session default
|
|
339
|
+
(stronger or weaker), FH follows it for **session turns**; floors govern FH's **own sub-agent
|
|
340
|
+
dispatches** and a session pin does not lower them — that separation *is* the Sonnet-main +
|
|
341
|
+
Opus-dispatch doctrine (pinned-sonnet sessions still dispatch floored agents at opus).
|
|
342
|
+
|
|
343
|
+
**Field depth-escalation (the upward complement)**: floors push *dispatches* up automatically, but
|
|
344
|
+
main-thread depth on a field session has no floor — so the templates bundle carries a Field
|
|
345
|
+
Depth-Escalation Notice (templates CLAUDE.md): on strain signals (repeated correction loops,
|
|
346
|
+
non-decomposable design reasoning, user stuck), the session AI proposes the cheaper rung first —
|
|
347
|
+
an opus sidecar dispatch for packageable units — and a session pin only for inherently main-thread
|
|
348
|
+
depth. Advisory, once per session, never auto-switches; the symmetric counterpart of the hub's
|
|
349
|
+
Mode D Model Notice, placed field-side so the proposal does not depend on mid-session rule recall. If the operator
|
|
350
|
+
explicitly caps dispatches too ("sonnet only, everywhere"), that is an environment limit: F2 applies,
|
|
351
|
+
flags and all. Pinning a stronger-than-opus default strengthens harness *self-development* in
|
|
352
|
+
particular (measured: the tier cliff sits on design-increment work) — mostly relevant to harness
|
|
353
|
+
developers (Mode D), not field operation.
|
|
354
|
+
|
|
355
|
+
---
|
|
356
|
+
|
|
357
|
+
## Empirical validation
|
|
358
|
+
|
|
359
|
+
### Experiment 1 — Sidecar invocation (2026-05-31)
|
|
360
|
+
|
|
361
|
+
1. **Corporate network** — Copilot CLI sidecar (CC standalone = Sonnet-only on restricted network). Copilot CLI's model catalog provided access to Codex, Gemini, and Claude Opus.
|
|
362
|
+
|
|
363
|
+
2. **Direct Gemini CLI sidecar** — `echo "prompt" | gemini` inside a Claude Code session. Adversarial review of `pipeline-conductor` Done When: 3 structural gaps (Amnesia Loophole, Activity vs Integrity gap, Escalation Paradox).
|
|
364
|
+
|
|
365
|
+
3. **Direct Codex CLI sidecar** — `npx @openai/codex exec "prompt"`. Same target: 3 non-overlapping issues (scope definition gap, skip justification absent, report persistence).
|
|
366
|
+
|
|
367
|
+
4. **Gemini native skill load** — `gemini skills install <fh-skill-path> --consent`. All 32 FH skills loaded and listed as Enabled. FH SKILL.md format is natively compatible with Gemini CLI.
|
|
368
|
+
|
|
369
|
+
---
|
|
370
|
+
|
|
371
|
+
### Experiment 2 — 3-round orchestrator-swap (2026-05-31)
|
|
372
|
+
|
|
373
|
+
**Setup**: Same FH skill (`goal-quench/SKILL.md`, post-patch) reviewed under 3 orchestrator configurations. Each round: orchestrator runs Wave 1 (3 primary findings), sidecar adds 3 non-overlapping findings, orchestrator synthesizes final verdict.
|
|
374
|
+
|
|
375
|
+
| Round | Orchestrator | Sidecar | Verdict | Orchestrator focus |
|
|
376
|
+
|---|---|---|---|---|
|
|
377
|
+
| 1 (prior) | Claude Opus | Gemini + Codex | 6 issues → patched | State machine + hook reliability |
|
|
378
|
+
| 2 | Gemini | Codex | FAIL — 4A + 2H | State machine loop + audit methodology |
|
|
379
|
+
| 3 | Codex | Gemini | FAIL — 4A + 2B | Implementation parsing + dependency contract |
|
|
380
|
+
|
|
381
|
+
**Convergence (appeared in every round)**:
|
|
382
|
+
- Freshness guard is pseudocode — no actual timestamp comparison
|
|
383
|
+
- `.pending` surviving BLOCKED creates an infinite-loop deadlock
|
|
384
|
+
- `git diff HEAD` yields false CLEAN if `/goal` makes interim commits
|
|
385
|
+
|
|
386
|
+
**Divergence (orchestrator-specific)**:
|
|
387
|
+
- Gemini orchestrator: state-machine completeness, audit methodology validity, phantom telemetry
|
|
388
|
+
- Codex orchestrator: `grep/cut` parsing fragility, `token-budget-gate` invocation contract absent
|
|
389
|
+
|
|
390
|
+
**Cross-wave delta finding**: 3 common issues + 3 orchestrator-specific = 6 total per round. No round found all 6 on its own. Single-model review (Claude alone, Round 1 on original) found a different 3 — correct but incomplete. Multi-model synthesis identified 12 distinct issues across 3 rounds, with 3 confirmed-critical by convergence.
|
|
391
|
+
|
|
392
|
+
**Conclusion**: Process divergence is the mechanism by which sidecar diversity compounds into better convergence. The harness (FH skill format + consistent evaluation protocol) is the activation condition — without it, the three results are structurally incomparable and the delta cannot be synthesized.
|
|
393
|
+
|
|
394
|
+
```bash
|
|
395
|
+
# Minimal sidecar call pattern (validated)
|
|
396
|
+
echo "You are an adversarial reviewer. Identify the 3 most critical gaps in this SKILL.md Done When: $(tail -30 path/to/SKILL.md)" | gemini
|
|
397
|
+
```
|
|
398
|
+
|
|
399
|
+
This is **not a prototype** — it is a confirmed, runnable pattern.
|
|
400
|
+
|
|
401
|
+
---
|
|
402
|
+
|
|
403
|
+
## When to distribute
|
|
404
|
+
|
|
405
|
+
Three use cases, ordered by primacy under a **full-subscription environment** (Claude Max + Gemini Pro/Advanced + GPT-4o Plus — all strong models available simultaneously):
|
|
406
|
+
|
|
407
|
+
| Use case | Primacy | Sidecar role | Example |
|
|
408
|
+
|---|---|---|---|
|
|
409
|
+
| **Perspective diversity** | **Primary** — valid regardless of model tier | Each model's process angle produces non-overlapping findings; cross-wave delta improves convergence quality | steel-quench Wave 5: Claude primary → Gemini + Codex sidecars → synthesize delta |
|
|
410
|
+
| **Model-access fallback** | Secondary — situational | Reach a stronger model when the CC host is downgraded on a restricted network | CC standalone = Sonnet-only on a restricted network → Copilot sidecar reaches Opus |
|
|
411
|
+
| **Token economy** | Tertiary — relevant when models differ in cost tier | Offload subsidiary tasks to a lighter or separately-billed model | Delegate claim extraction to Gemini Flash while Opus handles synthesis |
|
|
412
|
+
|
|
413
|
+
**Full-subscription case (validated 2026-05-31)**: When all three providers are at premium tier (Claude Max / Gemini Pro / GPT-4o Plus), token economy is not the primary motivation. The orchestrator-swap experiment showed that even when all models are strong, **process divergence still produces non-overlapping findings** — Gemini focused on state-machine completeness, Codex on implementation parsing fragility. Neither found what the other found. This means perspective diversity is an intrinsic property of model identity, not a function of capability tier.
|
|
414
|
+
|
|
415
|
+
**Implication**: In a full-subscription environment, sidecar invocation should default to the strongest available sidecar model, not a lightweight one. The goal is maximum perspective divergence, not cost savings.
|
|
416
|
+
|
|
417
|
+
### Surface-Specialized Orchestration (operator observation, 2026-06-13)
|
|
418
|
+
|
|
419
|
+
Sidecar value has evolved from generic "model diversity" to **surface-specialized orchestration**. Routing is determined not by model capability alone, but by which runtime natively governs the **task surface**. Each runtime excels on its own surface, but FH fundamentally distrusts their self-reports (fluency) and binds them via gate/review.
|
|
420
|
+
|
|
421
|
+
Observed surface split across the Claude Code / Gemini / Codex family:
|
|
422
|
+
|
|
423
|
+
| Runtime | Surface & Role | FH routing criteria |
|
|
424
|
+
|---|---|---|
|
|
425
|
+
| **Claude Code / Opus** | **FH governor**. Public/private hygiene, hook/gate, claim-scope review, repo governance | The task depends on terminal/repository governance, `.claude/agents/` dispatch, or FH-native automation. |
|
|
426
|
+
| **Codex** | **Web-flow pilot**. Direct click, page navigation, login state, guest onboarding, browser UI iteration | The task depends on live web-flow automation (requires Codex app/runtime with Browser connectors). |
|
|
427
|
+
| **Gemini / Antigravity** | **Multimodal ingestion**. Video, screen recording, image-based UI/UX observation, time-flow data | The task surface is visual/temporal (video/recording/image). Among the three runtimes, **only Gemini natively ingests video/multimodal input** (CC and Codex do not) — an established capability boundary confirmed across prior experiments, not a self-report. The routing choice for visual/temporal surfaces. (FH states the capability boundary; specific performance benchmarks are a separate measurement.) |
|
|
428
|
+
|
|
429
|
+
**FH Governance Meta-Rule**: The orchestrating harness synthesizes these surfaces but **never trusts a runtime's self-report**. A live pilot's "success" or a multimodal ingestion's "comprehensive read" is always cross-checked by FH gates.
|
|
430
|
+
|
|
431
|
+
|
|
432
|
+
This is **affordance routing, not provider ranking**. `codex exec` remains a headless text/CLI sidecar;
|
|
433
|
+
the web-automation value appears when the Codex session has a Browser/Chrome control surface. A Claude
|
|
434
|
+
Code host should therefore treat Codex as the preferred handoff for web-flow probes only after probing
|
|
435
|
+
that connector capability, and should fall back to the normal text sidecar role when it is absent.
|
|
436
|
+
|
|
437
|
+
**FH-local workspace note**: some operator setups load the public forge-harness mirror and a private
|
|
438
|
+
companion store (the `*-be` pattern) as sibling repositories inside one working folder to form a personal
|
|
439
|
+
FH environment. Capability routing should be resolved at the workspace level in that setup: preserve which
|
|
440
|
+
repository owns the artifact or change, but choose the runtime by the surface being tested (public FH docs
|
|
441
|
+
and governance in the mirror; backend/runtime/experiment work in the companion store; live web-flow probes
|
|
442
|
+
through a Browser/Chrome-capable Codex session when available).
|
|
443
|
+
|
|
444
|
+
> **v2 paper candidate**: Does model tier affect the quality of perspective divergence, or is the divergence pattern stable across tiers? The orchestrator-swap experiment used entry/mid-tier sidecars — replicating with all-premium models (Gemini Pro, GPT-4o, Claude Opus) would test whether diversity compounds further or plateaus. This is a natural follow-on experiment for the v2 empirical section.
|
|
445
|
+
|
|
446
|
+
### Debate Circulation Loop + governor source-verification (methodology, promoted 2026-06-14)
|
|
447
|
+
|
|
448
|
+
Surface-specialized routing answers *which runtime does the work*. The **Debate Circulation Loop**
|
|
449
|
+
answers *how to trust the result when several runtimes disagree* — it is the verification methodology
|
|
450
|
+
that rides on top of surface routing, and the model-level expression of the judge-robustness principle
|
|
451
|
+
(a judged verdict reached only by a judge is foolable; bind it to a mechanical anchor).
|
|
452
|
+
|
|
453
|
+
**The loop** — used when a question genuinely spans multiple runtimes (compatibility calls,
|
|
454
|
+
cross-surface mining, contested claims):
|
|
455
|
+
|
|
456
|
+
1. **Relay** the same question to each runtime (CC governor + the surface-appropriate sidecars).
|
|
457
|
+
2. **Mutual critique** — each runtime critiques the others' output. The key asymmetry: *a runtime's
|
|
458
|
+
own blind spot is invisible to itself but visible to a peer.* Divergence is the signal, not noise.
|
|
459
|
+
3. **Governor closes with source** — CC/Opus does not average the opinions (a debate is *also* judged,
|
|
460
|
+
so a vote is still judge-only). It **closes the loop against a source of ground truth**: re-reads
|
|
461
|
+
the cited file, runs the command, checks the live surface. Only the source-verified residue is kept.
|
|
462
|
+
4. **Promote refined, quarantine raw** — only the governor-closed conclusion is eligible for public
|
|
463
|
+
promotion; raw runtime output stays in the private companion store first (drift guard).
|
|
464
|
+
|
|
465
|
+
**Standing clause — distrust the self-report, the gate is the backstop.** A runtime's claim about its
|
|
466
|
+
*own* strength is the least reliable thing it emits. Origin: a sidecar self-reported "strong
|
|
467
|
+
public/private separation" and was then disproven by an actual private-store leak the *governor* caught
|
|
468
|
+
— after which the sidecar accepted the catch and self-modeled the blind spot. Fluency is not accuracy;
|
|
469
|
+
this is why every runtime output terminates at an FH gate/review, never at the runtime's own verdict.
|
|
470
|
+
|
|
471
|
+
**Cross-surface verification (the strongest form)** — make the *verifier* live on a different surface
|
|
472
|
+
than the *claimer*, so the check is mechanical rather than a second opinion:
|
|
473
|
+
|
|
474
|
+
| Role | Runtime | Why |
|
|
475
|
+
|---|---|---|
|
|
476
|
+
| Claim producer | Gemini (multimodal) | reads the video/image, emits a timestamped claim |
|
|
477
|
+
| Claim verifier | Codex (web-flow) | independently clicks that timestamp in a browser → ground-truth compare |
|
|
478
|
+
| Governor | CC / Opus | records, adjudicates, closes against source |
|
|
479
|
+
|
|
480
|
+
A timestamp *read by a model* is judged; the same timestamp *clicked on a different surface* is a
|
|
481
|
+
mechanical anchor. Verifier-≠-claimer is an independent verification path, not self-verification — the
|
|
482
|
+
multi-runtime realization of "judged → mechanical."
|
|
483
|
+
|
|
484
|
+
**When to run it (cost guard).** The loop is expensive (N runtime calls + governor close). Reserve it
|
|
485
|
+
for genuine multi-runtime decisions; trivial questions take a single runtime + a gate. Tier note:
|
|
486
|
+
trust the loop's *strategic* reasoning only from frontier-top sidecars (Codex/Gemini Pro high-reasoning
|
|
487
|
+
tiers); a below-floor model (e.g. a Flash backend) is a producer, not a trusted critic — its critique
|
|
488
|
+
is itself subject to §Floor governance.
|
|
489
|
+
|
|
490
|
+
> Live instance (this doc's own promotion, 2026-06-14): mining two external sources for FH-reinforcement
|
|
491
|
+
> ran *as* the loop — Gemini ingested a web book the governor's fetcher was 403-blocked from, Codex ran
|
|
492
|
+
> the adversarial governance critique of `oh-my-claudecode` import candidates, and CC closed both against
|
|
493
|
+
> FH's own asset inventory (most candidates resolved to ALREADY-HAVE, a healthy sister-asset result).
|
|
494
|
+
|
|
495
|
+
---
|
|
496
|
+
|
|
497
|
+
## Mechanism (how it works)
|
|
498
|
+
|
|
499
|
+
```
|
|
500
|
+
FH / Claude Code (orchestrator)
|
|
501
|
+
│
|
|
502
|
+
│ Bash tool
|
|
503
|
+
▼
|
|
504
|
+
Sidecar process
|
|
505
|
+
├── Gemini CLI → Gemini model
|
|
506
|
+
├── OpenAI/Codex CLI → GPT-4o / Codex model
|
|
507
|
+
└── Copilot CLI → model catalog (Codex / Gemini / Claude Opus)
|
|
508
|
+
│
|
|
509
|
+
└── stdout → back to Claude Code session → integrated by the skill
|
|
510
|
+
```
|
|
511
|
+
|
|
512
|
+
- **Host is always single**: Claude Code session owns the conversation, memory, and file state.
|
|
513
|
+
- **Sidecar is stateless**: each call is a one-shot prompt → response. No persistent sidecar context.
|
|
514
|
+
- **Integration is inline**: the skill reads stdout and folds it into its own output or verdict.
|
|
515
|
+
- **Not an agent dispatch**: sidecar calls bypass `.claude/agents/` entirely. No AGENTS.md entry needed.
|
|
516
|
+
|
|
517
|
+
---
|
|
518
|
+
|
|
519
|
+
## Integration with FH skills
|
|
520
|
+
|
|
521
|
+
Sidecar calls are coordinated inline by the calling skill, not by a central dispatcher. Recommended pattern:
|
|
522
|
+
|
|
523
|
+
```bash
|
|
524
|
+
# Inside a skill Step (AI executes this via Bash tool):
|
|
525
|
+
SIDECAR_RESULT=$(echo "${PROMPT}" | gemini 2>/dev/null)
|
|
526
|
+
# Then fold $SIDECAR_RESULT into the skill's output or verdict.
|
|
527
|
+
```
|
|
528
|
+
|
|
529
|
+
Suggested integration points:
|
|
530
|
+
- `steel-quench` — 2nd-challenger pass after Wave 1 (primary Claude challenger)
|
|
531
|
+
- `pipeline-conductor` — cross-check a verdict with a sidecar before elevating to CONDITIONAL_PASS
|
|
532
|
+
- `sim-conductor` — persona simulation from a different model's perspective
|
|
533
|
+
- `phantom-quench` — Gemini reads source files as a secondary back-tracer
|
|
534
|
+
|
|
535
|
+
---
|
|
536
|
+
|
|
537
|
+
## Boundaries
|
|
538
|
+
|
|
539
|
+
- The harness (FH) is the specialization layer. Do not treat the sidecar as a second harness.
|
|
540
|
+
- Each sidecar call is independent (no shared context with main session by default).
|
|
541
|
+
- Sidecar model output is **untrusted input** — the orchestrating skill validates before accepting.
|
|
542
|
+
- Cost: sidecar API calls are billed separately (Gemini API key, OpenAI key, Copilot subscription).
|
|
543
|
+
- **Provider identity gates perspective diversity** — the primary use case (diversity) only holds when the sidecar is a *different provider*. Reaching Claude Opus through Copilot CLI from a Claude host is **Claude → Claude**: it serves model-access fallback (secondary use case) but produces **no genuine process divergence** — same pre-training distribution, same blind spots. Do not count a same-provider sidecar as a diversity wave. Diversity requires a cross-provider sidecar (e.g. Claude host → Gemini/Codex).
|
|
544
|
+
|
|
545
|
+
---
|
|
546
|
+
|
|
547
|
+
## Implementation Patterns
|
|
548
|
+
|
|
549
|
+
**Context**: §1–8 establish *why* sidecar distribution works and its validated patterns. This section adds *executable* implementation guidance reverse-harvested from a sister-harness `sidecar-orchestrator` (2026-06-01), generalized for any environment. These are **reference** patterns for any caller — they do **not** supersede §Scope vs steel-quench Wave 5: for the steel-quench multi-team case specifically, Wave 5 remains the canonical implementation.
|
|
550
|
+
|
|
551
|
+
> **CLI-syntax caveat**: Exact sidecar invocation flags differ by CLI and version. The forms below use FH's validated baselines (`gh copilot suggest`, `echo … | gemini`, `npx @openai/codex exec` — see §The capability). Flags such as `--model` are catalog/version-dependent — verify with the CLI's own `--help` before relying on them.
|
|
552
|
+
|
|
553
|
+
### Three-tier fallback chain
|
|
554
|
+
|
|
555
|
+
When the primary sidecar path is blocked (network restrictions, API outages, rate limits), degrade to secondary/tertiary options so execution continues rather than failing silently.
|
|
556
|
+
|
|
557
|
+
| Priority | Sidecar | Access | Trigger condition |
|
|
558
|
+
|---|---|---|---|
|
|
559
|
+
| **1 (Primary)** | Copilot CLI (`gh copilot`) | Copilot model catalog | Default — widest catalog |
|
|
560
|
+
| **2 (Fallback)** | Corporate AI endpoint | Internal models (if any) | Priority 1 unreachable (503, timeout, quota) |
|
|
561
|
+
| **3 (Last resort)** | Direct CLI (Gemini / Codex) | Public API key | Priorities 1–2 unreachable + external network OK |
|
|
562
|
+
|
|
563
|
+
**Empirical grounding**: some corporate networks block direct LLM-provider APIs but allow GitHub CLI routing → `gh copilot` was the only path to premium models there. Without a fallback tier the workflow blocks entirely.
|
|
564
|
+
|
|
565
|
+
### Executable patterns
|
|
566
|
+
|
|
567
|
+
**Pattern 1 — Single-model sidecar**
|
|
568
|
+
```bash
|
|
569
|
+
SIDECAR_RESULT=$(gh copilot suggest "Adversarial reviewer: 3 most critical gaps in this Done When: $(tail -30 path/to/SKILL.md)" 2>&1)
|
|
570
|
+
EXIT_CODE=$?
|
|
571
|
+
[[ $EXIT_CODE -ne 0 ]] && echo "⚠️ sidecar failed (exit $EXIT_CODE): $SIDECAR_RESULT" # escalate or fail gracefully
|
|
572
|
+
```
|
|
573
|
+
Model selection, if the CLI exposes it, is catalog-dependent — check `gh copilot --help`.
|
|
574
|
+
|
|
575
|
+
**Pattern 2 — Cross-provider parallel ensemble** (real diversity comes from *distinct providers*, not one router)
|
|
576
|
+
```bash
|
|
577
|
+
echo "$PROMPT" | gemini > raw_gemini.txt 2>&1 &
|
|
578
|
+
npx @openai/codex exec "$PROMPT" > raw_codex.txt 2>&1 &
|
|
579
|
+
gh copilot suggest "$PROMPT" > raw_copilot.txt 2>&1 &
|
|
580
|
+
wait # block until all sidecars finish
|
|
581
|
+
# orchestrator synthesizes cross-wave delta from raw_*.txt
|
|
582
|
+
```
|
|
583
|
+
> Routing the *same* provider twice (e.g. Claude → Claude via Copilot) is model-access fallback, **not a diversity wave** — same pre-training, same blind spots (see §Boundaries). For divergence, use genuinely different providers.
|
|
584
|
+
|
|
585
|
+
**Orchestrator-swap variant** (FH Experiment 2): rotate orchestrator identity across waves — host runs Wave 1, a different-provider sidecar challenges Wave 1, a third challenges Wave 2, then the host synthesizes all three.
|
|
586
|
+
|
|
587
|
+
**Pattern 3 — Corporate endpoint fallback (Priority 2)**
|
|
588
|
+
```bash
|
|
589
|
+
CORPORATE_RESULT=$(curl -s -X POST "${CORPORATE_AI_ENDPOINT}" \
|
|
590
|
+
-H "Authorization: Bearer ${CORPORATE_AI_TOKEN}" \
|
|
591
|
+
-H "Content-Type: application/json" \
|
|
592
|
+
-d "{\"model\":\"${CORPORATE_MODEL}\",\"prompt\":\"${PROMPT}\"}" | jq -r .response)
|
|
593
|
+
[[ -z "$CORPORATE_RESULT" ]] && echo "❌ corporate endpoint failed — check token in .env" # escalate to Priority 3
|
|
594
|
+
```
|
|
595
|
+
Configure `CORPORATE_AI_ENDPOINT` / `CORPORATE_MODEL` / `CORPORATE_AI_TOKEN` in `.env` (git-excluded).
|
|
596
|
+
|
|
597
|
+
**Pattern 4 — Direct Gemini/Codex CLI (Priority 3)**
|
|
598
|
+
```bash
|
|
599
|
+
echo "${PROMPT}" | gemini > raw_gemini.txt
|
|
600
|
+
npx @openai/codex exec "${PROMPT}" > raw_codex.txt
|
|
601
|
+
```
|
|
602
|
+
Requires the respective CLI installed and authenticated.
|
|
603
|
+
|
|
604
|
+
### Error-handling checklist
|
|
605
|
+
|
|
606
|
+
```bash
|
|
607
|
+
RESULT=$(sidecar_command 2>&1); EXIT_CODE=$?
|
|
608
|
+
if [[ $EXIT_CODE -ne 0 ]]; then
|
|
609
|
+
echo "⚠️ sidecar failed (exit $EXIT_CODE)"
|
|
610
|
+
if [[ "$RESULT" =~ (unknown|invalid)\ model ]]; then echo " → model name error — check catalog: gh copilot --help"
|
|
611
|
+
elif [[ "$RESULT" =~ (503|timeout) ]]; then echo " → network/API outage — escalate to Priority 2"
|
|
612
|
+
elif [[ "$RESULT" =~ (rate\ limit|quota\ exceeded) ]]; then echo " → rate limited — retry after delay or use Priority 3"
|
|
613
|
+
else echo " → unknown failure: $RESULT"; fi
|
|
614
|
+
fi
|
|
615
|
+
```
|
|
616
|
+
**Critical**: never assume sidecar success — always capture `$EXIT_CODE` and parse `stderr` for failure signals.
|
|
617
|
+
|
|
618
|
+
### Three-layer persistence protocol
|
|
619
|
+
|
|
620
|
+
Sidecar results are high-risk for **compression aging** (AgingBench, arXiv:2605.26302) — single-session outputs that vanish if not structurally anchored. Persist in 3 independent layers so at least one survives compression/refactoring:
|
|
621
|
+
|
|
622
|
+
1. **Full result file** — `tracks/_meta/sidecar_{target}_{YYYY_MM_DD}.md` with frontmatter (`type: sidecar-review`, `models`, `target`, `priority`) + tiered findings (M/S/R).
|
|
623
|
+
2. **Memory reference entry** — one applicability-phrased line in the durable memory store (`~/.claude/.../memory/`) so next session recalls it on intent match (`memory_intent_recall.md`).
|
|
624
|
+
3. **CATALOG search entry** — 3-line summary + Decision/Open, pointing at the Layer-1 file.
|
|
625
|
+
|
|
626
|
+
Missing any layer = compression risk. (Path conventions adapt per project — see Generalization below.)
|
|
627
|
+
|
|
628
|
+
### When NOT to invoke (simplification guard)
|
|
629
|
+
|
|
630
|
+
**Skip the sidecar** for: single-file review (host sufficient), simple design decisions (no architectural complexity), repetitive non-judgment tasks (shell faster than an AI sidecar). **Invoke** only when 2+ hold: complex/multi-skill architecture · external knowledge needed (arXiv, frontier, cross-domain) · genuine multi-model perspective diversity required.
|
|
631
|
+
|
|
632
|
+
> For *when to distribute by value tier* (perspective diversity / model-access / token economy), the canonical guidance is §When to distribute above — not repeated here.
|
|
633
|
+
|
|
634
|
+
### Generalization guidelines (sister-harness → FH)
|
|
635
|
+
|
|
636
|
+
1. **Corporate endpoint → generic fallback**: a sister-harness's internal gateway → user-configured `.env` endpoint.
|
|
637
|
+
2. **Approval mode → consent gate**: prompt the user before a sidecar sends internal code/patterns to an external API.
|
|
638
|
+
3. **Persistence paths → project structure**: the sister-harness's `tracks/_meta/`·`memory/`·`CATALOG.md` → adapt to the host project's equivalents.
|
|
639
|
+
4. **Model names → CLI-agnostic**: replace pinned names (`claude-opus-4.x`, `gpt-5.x`) with `{model-name}` placeholders + "check CLI help for catalog".
|
|
640
|
+
|
|
641
|
+
---
|
|
642
|
+
|
|
643
|
+
## References
|
|
644
|
+
|
|
645
|
+
- `README.md §Architecture — 2-layer design` — sidecar note in Automation layer section
|
|
646
|
+
- `AGENTS.md §2-Layer Architecture Context` — sidecar note distinguishing Bash invocation from agent dispatch
|
|
647
|
+
- FH paper (Zenodo DOI: 10.5281/zenodo.20397566, arXiv: submit/7657304) — harness-as-durable-layer thesis
|
|
648
|
+
- A sister-harness `sidecar-orchestrator` SKILL.md (2026-06-01) — gh copilot + corporate endpoint + 3-tier fallback + 3-layer persistence
|
|
649
|
+
- arXiv:2605.26302 AgingBench — compression aging defense rationale
|
|
650
|
+
- `hybrid_orchestration_architecture_roadmap.md` — proposed (not-yet-implemented) architecture direction that would generalize this sidecar strategy into a hybrid orchestration engine
|