@chrono-meta/fh-gate 1.4.41 → 1.4.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +5 -3
- package/CATALOG.md +6 -0
- package/CLAUDE.md +65 -130
- package/docs/CONTRIBUTING.md +2 -2
- package/knowledge/shared/dialogue/ai_dialogue_playbook.md +137 -0
- package/knowledge/shared/dialogue/claude_code_runtime_flow.md +170 -0
- package/knowledge/shared/dialogue/memory_intent_recall.md +209 -0
- package/knowledge/shared/harness-core/claude_md_gate_details.md +170 -0
- package/knowledge/shared/harness-core/companion_store_pluggable_cross_audit_2026-06-11.md +118 -0
- package/knowledge/shared/harness-core/crucible_mode.md +112 -0
- package/knowledge/shared/harness-core/deep_research_capability_ladder.md +122 -0
- package/knowledge/shared/harness-core/fh_detail_protocols.md +163 -0
- package/knowledge/shared/harness-core/fh_ecosystem_positioning.md +147 -0
- package/knowledge/shared/harness-core/fh_opencode_governance_wrapper.md +163 -0
- package/knowledge/shared/harness-core/fh_synergy_playbook.md +217 -0
- package/knowledge/shared/harness-core/gate_locality_principle.md +57 -0
- package/knowledge/shared/harness-core/goal_quench_anthropic_issue.md +104 -0
- package/knowledge/shared/harness-core/harness_6axis_framework.md +136 -0
- package/knowledge/shared/harness-core/harness_design_decision_lens.md +108 -0
- package/knowledge/shared/harness-core/harness_frontier_diagnosis_2026-06-02.md +102 -0
- package/knowledge/shared/harness-core/hub_compounding_loop.md +109 -0
- package/knowledge/shared/harness-core/hub_maturity_roadmap.md +201 -0
- package/knowledge/shared/harness-core/hybrid_orchestration_architecture_roadmap.md +196 -0
- package/knowledge/shared/harness-core/live_surface_automation_pattern.md +110 -0
- package/knowledge/shared/harness-core/measurement-integrity-checklist.md +59 -0
- package/knowledge/shared/harness-core/meta_harness_engineering_definition.md +116 -0
- package/knowledge/shared/harness-core/multi_model_sidecar_strategy.md +651 -0
- package/knowledge/shared/harness-core/persona_container_schema.md +172 -0
- package/knowledge/shared/harness-core/return_path_gate.md +120 -0
- package/knowledge/shared/harness-core/self_evolution_routine.md +268 -0
- package/knowledge/shared/harness-core/skill_quality_rubric.md +71 -0
- package/knowledge/shared/harness-core/tpa_schema.md +136 -0
- package/package.json +3 -2
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: fh-ecosystem-positioning
|
|
3
|
+
description: FH's structural position in the AI agent framework ecosystem vs Hermes, OpenCode, OpenHuman — gap analysis, synergy map, and layered readiness verdict from 3-model adversarial audit (Claude + Gemini + Codex).
|
|
4
|
+
date: 2026-05-31
|
|
5
|
+
tags: [ecosystem, positioning, synergy, opencode, opencode, hermes, openhuman, readiness, v2-paper]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# FH Ecosystem Positioning
|
|
9
|
+
|
|
10
|
+
## Source
|
|
11
|
+
|
|
12
|
+
3-model orchestrator-swap adversarial audit (2026-05-31).
|
|
13
|
+
- Orchestrator: Claude Opus (synthesis)
|
|
14
|
+
- Sidecar 1: Gemini (structural + security lens)
|
|
15
|
+
- Sidecar 2: Codex (ecosystem + distribution lens)
|
|
16
|
+
|
|
17
|
+
Target: FH full structure vs Hermes-type agent frameworks, OpenCode-style coding agents, OpenHuman-style human-in-loop systems.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Gap Analysis — Where FH Falls Short
|
|
22
|
+
|
|
23
|
+
| Gap | Gemini finding | Codex finding | Convergence |
|
|
24
|
+
|---|---|---|---|
|
|
25
|
+
| **Runtime / Distribution** | — | No standalone binary, no package-manager path; peers (OpenCode, Hermes IDE, OpenHuman) ship signed installers | Codex only |
|
|
26
|
+
| **Execution sandboxing** | No WASM/Docker isolation — destructive bash commands possible inside skill steps | — | Gemini only |
|
|
27
|
+
| **Concurrent state** | File-system bottleneck — no event-bus; parallel agents on same `tracks/` risk state corruption | — | Gemini only |
|
|
28
|
+
| **HITL granularity** | Binary confirm/deny — no capability-based fine-grained delegation (OpenHuman standard) | — | Gemini only |
|
|
29
|
+
| **Gate enforcement** | — | Gates are advisory (instructional thresholds, not mechanical block) | Codex only |
|
|
30
|
+
| **Registry coherence** | — | Count drift: 33 skills, 6 agents vs declared values → fixed in this session | Codex only (fixed) |
|
|
31
|
+
|
|
32
|
+
**No gap converged across both models** — each lens found distinct weaknesses. This itself is a cross-wave delta finding: Gemini sees architecture risk, Codex sees distribution/ecosystem risk.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Synergy Map — Where Integration Produces N-Fold Value
|
|
37
|
+
|
|
38
|
+
### 1. FH + OpenCode: Governance Layer on Execution Speed
|
|
39
|
+
|
|
40
|
+
| Layer | Provider | What it contributes |
|
|
41
|
+
|---|---|---|
|
|
42
|
+
| Execution | OpenCode | High-volume autonomous coding, CLI/desktop distribution, broad language support |
|
|
43
|
+
| Governance | FH | `pipeline-conductor` (4-axis gated sweep) + `steel-quench` (adversarial review) + `phantom-quench` |
|
|
44
|
+
|
|
45
|
+
**N-fold mechanism**: OpenCode is criticized for "automation-drift" — valid code that doesn't solve the core problem. FH's adversarial + grounding verification layer catches this. The combined result is an autonomous coder that must survive structured critique before the human sees it: "fast coder" → "rigorous engineer."
|
|
46
|
+
|
|
47
|
+
**Integration path**: Route OpenCode PRs through FH `pipeline-conductor --full` as a post-generation gate. No runtime adapter needed — FH reads any file-based output.
|
|
48
|
+
|
|
49
|
+
### 2. FH + OpenHuman / Hermes: Memory → Audited Methodology
|
|
50
|
+
|
|
51
|
+
| System | OpenHuman / Hermes | FH |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| Provides | Local Memory Tree, persistent conversations, workspace config, cost dashboard, UI/UX | `tracks/`, `knowledge/`, `harvest-loop`, `phantom-quench`, `memory-hygiene` |
|
|
54
|
+
|
|
55
|
+
**N-fold mechanism**: OpenHuman/Hermes stores memory but treats it as passive recall (context stuffing). FH's harvest-loop + source-grounding turns that memory into *audited institutional process* — patterns are reviewed, grounded, and promoted. Memory becomes methodology, not just storage.
|
|
56
|
+
|
|
57
|
+
**Integration path**: FH `harvest-loop` runs on the host system's Memory Tree as its audit target. No runtime adapter needed for the methodology layer.
|
|
58
|
+
|
|
59
|
+
### 3. FH + Hermes IDE: SKILL.md as Portable Instruction Set
|
|
60
|
+
|
|
61
|
+
**Gemini's framing**: Hermes provides the "seat" (delivery, UI, local context); FH provides the "controls" (vendor-agnostic methodology). Embedding FH's SKILL.md parser into the Hermes sidecar gives **model portability** — switch from Claude to Gemini mid-task without re-teaching the UI how to run a `harness-doctor` audit. FH becomes the "instruction set" for the UI's "processor."
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Readiness Verdict (Layered)
|
|
66
|
+
|
|
67
|
+
**Verdict diverged by lens** — both are correct, different layers:
|
|
68
|
+
|
|
69
|
+
| Layer | Gemini verdict | Codex verdict | Synthesis |
|
|
70
|
+
|---|---|---|---|
|
|
71
|
+
| Methodology | "Fully-Prepared Hermit" — deep internal maturity, can engage | Strong claim, best-in-class in process integrity | **Peer-ready NOW** |
|
|
72
|
+
| Framework/Runtime | "Externally illiterate" — lacks API, event system, sandbox | "Not peer-level" — no binary, no canonical registry, no distribution | **v0.x — needs bridge layer** |
|
|
73
|
+
|
|
74
|
+
**The hermit metaphor holds, precisely**: FH has meditated deeply (adversarial review, harvest loops, calibration, sidecar orchestration). It can engage any framework on methodology. It cannot yet *integrate as a framework peer* without forcing the other system to adopt its file-heavy, CLI-centric worldview.
|
|
75
|
+
|
|
76
|
+
**What "bridge layer" means concretely**:
|
|
77
|
+
1. Host-agnostic CLI adapter — methodology layer runs without Claude Code automation layer
|
|
78
|
+
2. Canonical machine-readable registry — `plugin.json` + actual file counts synchronized, test status, compatibility matrix
|
|
79
|
+
3. Integration contract — OpenCode/OpenHuman calls FH gates, receives structured verdicts, persists reports
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## Immediate Action (No Bridge Layer Required)
|
|
84
|
+
|
|
85
|
+
FH + OpenCode governance integration is executable **today** at the methodology layer:
|
|
86
|
+
|
|
87
|
+
```bash
|
|
88
|
+
# After OpenCode generates a PR:
|
|
89
|
+
# 1. Capture the diff
|
|
90
|
+
git diff main..HEAD > /tmp/opencode_output.diff
|
|
91
|
+
CHANGED=$(git diff main..HEAD --name-only | tr '\n' ' ')
|
|
92
|
+
|
|
93
|
+
# 2. steel-quench adversarial pass on changed files
|
|
94
|
+
# → finds behavioral edge cases, untested contracts, security assumptions
|
|
95
|
+
|
|
96
|
+
# 3. pipeline-conductor --quick on changed files
|
|
97
|
+
# → 4-axis gate: backward / adversarial / forward / record
|
|
98
|
+
|
|
99
|
+
# 4. phantom-quench on any new documentation claims
|
|
100
|
+
# → catches phantom references and stale citations
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
This requires no OpenCode API integration — FH reads files, OpenCode writes files. The protocol is the interface.
|
|
104
|
+
|
|
105
|
+
See `fh_opencode_governance_wrapper.md` for the full step-by-step guide and Stop hook automation.
|
|
106
|
+
|
|
107
|
+
### Empirical result (2026-05-31)
|
|
108
|
+
|
|
109
|
+
Applied the 3-step governance pass to OpenCode's own AI-generated `permission/arity.ts` (163 lines). CI verdict: DONE (6/6 tests pass). FH governance verdict: PENDING — 2 A-grade findings CI did not cover:
|
|
110
|
+
|
|
111
|
+
1. Short-token overflow in `prefix()` — allowlist pattern may not cover bare commands previously approved
|
|
112
|
+
2. `npx`, `opencode`, `claude` absent from arity table — `npx <anything>` receives the same broad `"npx *"` pattern, weakening the permission model
|
|
113
|
+
|
|
114
|
+
The delta is attributable to the methodology layer, not the model. Both passes read identical code.
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## v2 Paper Connection
|
|
119
|
+
|
|
120
|
+
The ecosystem positioning audit surfaces a testable claim for v2:
|
|
121
|
+
|
|
122
|
+
> "A harness-structured workflow integrated as a governance layer on top of a bare coding agent produces qualitatively different outputs from either system alone — not because the model changed, but because the methodology layer enforces structured verification that the agent alone cannot generate."
|
|
123
|
+
|
|
124
|
+
This is the N-fold synergy claim stated precisely. The controlled experiment design: OpenCode alone vs OpenCode + FH governance on the same task. Measure: findings caught by governance that CI missed, rework cycles prevented.
|
|
125
|
+
|
|
126
|
+
**Empirical pilot (2026-05-31)**: Applied to OpenCode's own AI-generated `permission/arity.ts`. Governance caught 2 A-grade security-adjacent issues that 6 CI tests missed. Causal attribution is clean: same code, same model, different methodology layer.
|
|
127
|
+
|
|
128
|
+
**v2 scope**: This experiment, combined with the 3-round orchestrator-swap finding (`multi_model_sidecar_strategy.md`), constitutes novel empirical contribution — not a version update. Proposed framing:
|
|
129
|
+
|
|
130
|
+
| Experiment | Claim tested | Evidence produced |
|
|
131
|
+
|---|---|---|
|
|
132
|
+
| 3-round orchestrator-swap | Process diverges, results converge; harness is activation condition | Cross-wave delta synthesized across Claude/Gemini/Codex |
|
|
133
|
+
| FH + OpenCode governance | Methodology layer catches what bare coding + CI misses | DONE → PENDING verdict flip on AI-generated code |
|
|
134
|
+
| Tier comparison (pending) | Divergence quality stable across model tiers | Needs replication with all-premium models |
|
|
135
|
+
|
|
136
|
+
**OpenCode as citation**: The governance experiment uses OpenCode's codebase as the subject. OpenCode should be cited as the target system in the v2 experimental section. Citation candidate: the OpenCode GitHub repository + any associated paper/technical report.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
## References
|
|
141
|
+
|
|
142
|
+
- `fh_opencode_governance_wrapper.md` — step-by-step usage guide with empirical findings
|
|
143
|
+
- `fh_synergy_playbook.md` — concrete FH×OpenCode/Hermes/OpenHuman workflow specs that operationalize this positioning
|
|
144
|
+
- `multi_model_sidecar_strategy.md` — orchestrator-swap experiment that generated this audit
|
|
145
|
+
- `README.md §Architecture` — 2-layer design (methodology vs automation)
|
|
146
|
+
- `AGENTS.md` — 6-agent registry (fact-checker added after this audit)
|
|
147
|
+
- FH paper (Zenodo: 10.5281/zenodo.20397566) — harness-as-durable-layer thesis this positioning extends
|
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: fh-opencode-governance-wrapper
|
|
3
|
+
description: Step-by-step usage guide for FH + OpenCode governance integration. No API adapter required — FH reads files OpenCode writes. Includes empirical findings from 2026-05-31 controlled trial on arity.ts.
|
|
4
|
+
date: 2026-05-31
|
|
5
|
+
tags: [opencode, governance, usage-guide, synergy, pipeline-conductor, steel-quench, v2-paper]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# FH + OpenCode Governance Wrapper — Usage Guide
|
|
9
|
+
|
|
10
|
+
## What this is
|
|
11
|
+
|
|
12
|
+
OpenCode generates code fast. FH catches what fast generation misses.
|
|
13
|
+
|
|
14
|
+
The integration requires no runtime adapter. OpenCode writes files; FH reads files. The protocol is the interface. You can run this today on any OpenCode output.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## The 3-Step Governance Pass
|
|
19
|
+
|
|
20
|
+
After OpenCode completes a task (or at any checkpoint), run these three steps in order:
|
|
21
|
+
|
|
22
|
+
### Step 1 — Capture the diff
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
# Capture everything OpenCode changed since the task started
|
|
26
|
+
git diff <start-commit>..HEAD > /tmp/opencode_output.diff
|
|
27
|
+
|
|
28
|
+
# If you don't know the start commit:
|
|
29
|
+
git diff main..HEAD > /tmp/opencode_output.diff
|
|
30
|
+
|
|
31
|
+
# List changed files explicitly (needed for Steps 2 and 3):
|
|
32
|
+
CHANGED=$(git diff main..HEAD --name-only | tr '\n' ' ')
|
|
33
|
+
echo "Changed: $CHANGED"
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
### Step 2 — steel-quench adversarial pass
|
|
37
|
+
|
|
38
|
+
Run `/steel-quench` (or describe the target to Claude):
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
Run steel-quench adversarial review on these files: $CHANGED
|
|
42
|
+
Focus: behavioral edge cases, untested contracts, security assumptions.
|
|
43
|
+
Output: 3 most critical findings with severity (A/B/C) and evidence.
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
steel-quench looks for what tests don't cover: contract boundary violations,
|
|
47
|
+
caller assumption gaps, silent fallbacks that mask errors.
|
|
48
|
+
|
|
49
|
+
### Step 3 — pipeline-conductor --quick
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
Run pipeline-conductor --quick on: $CHANGED
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
4-axis verdict:
|
|
56
|
+
- Axis 1 (Backward): regression risk
|
|
57
|
+
- Axis 2 (Adversarial): structural gaps (from Step 2)
|
|
58
|
+
- Axis 3 (Forward): phantom claims, broken references
|
|
59
|
+
- Axis 4 (Record): calibration log entry
|
|
60
|
+
|
|
61
|
+
**If verdict is CLEAN or PENDING**: proceed. Log the run.
|
|
62
|
+
**If verdict is BLOCKED**: surface findings to OpenCode, re-run the task with constraints added.
|
|
63
|
+
**If verdict is ESCALATE**: human decision required before merge.
|
|
64
|
+
|
|
65
|
+
---
|
|
66
|
+
|
|
67
|
+
## Optional: Step 0 — Pre-task scope gate
|
|
68
|
+
|
|
69
|
+
Before running OpenCode on a large task, estimate the scope:
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
Estimate token budget for: <task description>, ~<N> files expected to change.
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Use `/token-budget-gate` or the fallback heuristic:
|
|
76
|
+
|
|
77
|
+
| Scope | Verdict |
|
|
78
|
+
|---|---|
|
|
79
|
+
| < 5 files, no new architecture | GREEN — proceed |
|
|
80
|
+
| 5–20 files or new module | YELLOW — proceed with monitoring |
|
|
81
|
+
| 20+ files or cross-system refactor | ORANGE — confirm scope |
|
|
82
|
+
| Full rewrite | RED — split into smaller tasks first |
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## Empirical Baseline (2026-05-31)
|
|
87
|
+
|
|
88
|
+
**Target**: OpenCode's own `packages/opencode/src/permission/arity.ts` (163 lines, AI-generated).
|
|
89
|
+
|
|
90
|
+
**Baseline (CI + self-evaluation)**: 6 unit tests, all pass. No syntax errors. Verdict: DONE.
|
|
91
|
+
|
|
92
|
+
**After FH governance pass**:
|
|
93
|
+
|
|
94
|
+
| Finding | Grade | What CI missed |
|
|
95
|
+
|---|---|---|
|
|
96
|
+
| Short-token overflow in `prefix()` — arity=3 entry with 2-token input builds allowlist pattern that may not cover bare commands | A | Untested path; `git stash` alone may not match `"git stash *"` |
|
|
97
|
+
| `npx`, `opencode`, `claude`, `bunx`, `uvx` absent from arity table — any `npx <package>` receives same broad `"npx *"` pattern | A | Not in test scope; security model weakened |
|
|
98
|
+
| AI-generated dictionary has no maintenance protocol or rule compliance check | B | No cadence, drift risk |
|
|
99
|
+
|
|
100
|
+
**Verdict flip**: DONE → PENDING. Delta is attributable to methodology layer, not the model.
|
|
101
|
+
|
|
102
|
+
**Implication**: 3 findings per 163-line AI-generated module that CI treats as done. Extrapolate across a codebase and the governance dividend compounds.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## When to run
|
|
107
|
+
|
|
108
|
+
| Signal | Action |
|
|
109
|
+
|---|---|
|
|
110
|
+
| OpenCode completes a task and opens a PR | Run all 3 steps before merge review |
|
|
111
|
+
| OpenCode generates a new module (AI-authored code) | Run Steps 2+3 — adversarial + 4-axis |
|
|
112
|
+
| OpenCode touches a security-adjacent file (permissions, auth, tokens) | Mandatory — run Steps 2+3 with security lens |
|
|
113
|
+
| OpenCode generates tests | Step 3 (pipeline-conductor) — check if tests actually cover the contract |
|
|
114
|
+
| Long OpenCode session (YELLOW or ORANGE budget) | Run Step 1 at each checkpoint, not just at the end |
|
|
115
|
+
|
|
116
|
+
---
|
|
117
|
+
|
|
118
|
+
## Synergy map — what each layer contributes
|
|
119
|
+
|
|
120
|
+
| Layer | Role | What it catches |
|
|
121
|
+
|---|---|---|
|
|
122
|
+
| **OpenCode** | Fast autonomous coding | Working code at speed |
|
|
123
|
+
| **FH steel-quench** | Adversarial review | Behavioral edge cases, untested contracts, security assumptions — what tests don't cover |
|
|
124
|
+
| **FH pipeline-conductor** | 4-axis structured gate | Regression risk, phantom claims, record keeping — what code review doesn't structure |
|
|
125
|
+
| **Combined** | Governance wrapper | Fast coder + structured critique = rigorous engineer |
|
|
126
|
+
|
|
127
|
+
The key: OpenCode and FH operate on the same files with no integration layer. The governance wrapper is a *protocol*, not an API.
|
|
128
|
+
|
|
129
|
+
---
|
|
130
|
+
|
|
131
|
+
## Full automation (optional)
|
|
132
|
+
|
|
133
|
+
For teams that want to run governance automatically after every OpenCode session, add a Stop hook:
|
|
134
|
+
|
|
135
|
+
```json
|
|
136
|
+
// .claude/settings.json (in your project, not forge-harness)
|
|
137
|
+
{
|
|
138
|
+
"hooks": {
|
|
139
|
+
"Stop": [
|
|
140
|
+
{
|
|
141
|
+
"matcher": "",
|
|
142
|
+
"hooks": [
|
|
143
|
+
{
|
|
144
|
+
"type": "command",
|
|
145
|
+
"command": "echo '[FH governance] Run pipeline-conductor --quick on $(git diff main..HEAD --name-only | tr \"\\n\" \" \")' >> /tmp/fh-pending-governance.txt"
|
|
146
|
+
}
|
|
147
|
+
]
|
|
148
|
+
}
|
|
149
|
+
]
|
|
150
|
+
}
|
|
151
|
+
}
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
On session end, check `/tmp/fh-pending-governance.txt` and run the governance pass.
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## References
|
|
159
|
+
|
|
160
|
+
- `fh_ecosystem_positioning.md` — ecosystem context, synergy map, v2 paper connection
|
|
161
|
+
- `multi_model_sidecar_strategy.md` — multi-model orchestration (sidecar pattern for adding Gemini/Codex review)
|
|
162
|
+
- `tracks/_meta/fh_opencode_governance_experiment_2026_05_31.md` — full empirical record (local)
|
|
163
|
+
- FH paper (Zenodo: 10.5281/zenodo.20397566) — harness-as-durable-layer thesis
|
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: fh-synergy-playbook
|
|
3
|
+
description: Concrete workflow specifications for using FH governance alongside OpenCode, Hermes, and OpenHuman — grounded in empirical results, no unverified claims.
|
|
4
|
+
date: 2026-05-31
|
|
5
|
+
tags: [synergy, integration, opencode, hermes, openhuman, governance, playbook, marketing]
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# FH Synergy Playbook — How to Use FH with OpenCode, Hermes, and OpenHuman
|
|
9
|
+
|
|
10
|
+
This document describes **exact workflows** for combining forge-harness with three AI agent projects. Every outcome stated here is backed by a recorded experiment or a structural guarantee from the software itself. Nothing is promised without evidence.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## The Core Pattern (Abstract)
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
AI agent generates or mutates code
|
|
18
|
+
↓
|
|
19
|
+
FH governance pass (before merge)
|
|
20
|
+
↓
|
|
21
|
+
Structured verdict: PASS | PENDING | BLOCKED
|
|
22
|
+
↓
|
|
23
|
+
Merge (if PASS) or fix-cycle (if PENDING/BLOCKED)
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
**What FH adds:** a structured verification layer that runs the same criteria every time, regardless of which model generated the code or which CI system is in use. It reads files the agent writes — no API integration, no runtime adapter.
|
|
27
|
+
|
|
28
|
+
**What FH does NOT do:**
|
|
29
|
+
- It does not fix the code for you (it produces findings, not patches)
|
|
30
|
+
- It does not replace CI (unit tests and FH governance are complementary, not competing)
|
|
31
|
+
- It does not guarantee zero bugs in output — it guarantees that structured verification ran
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## With OpenCode — AI Code Generation + FH Quality Gate
|
|
36
|
+
|
|
37
|
+
### The gap FH closes
|
|
38
|
+
|
|
39
|
+
OpenCode generates code rapidly. Its own CI (unit tests, type checks) validates syntax and basic behavior. What it misses: security-adjacent logic, arity/allowlist edge cases, AI-generated assumptions that pass tests but break real-world usage.
|
|
40
|
+
|
|
41
|
+
### Workflow
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
# 1. Let OpenCode do its work
|
|
45
|
+
opencode run "implement the feature"
|
|
46
|
+
|
|
47
|
+
# 2. Capture what changed
|
|
48
|
+
export FH_TARGET_FILES=$(git diff main..HEAD --name-only)
|
|
49
|
+
|
|
50
|
+
# 3. Run FH governance pass
|
|
51
|
+
./scripts/fh-gate.sh "$FH_TARGET_FILES"
|
|
52
|
+
# → generates structured prompt for steel-quench + pipeline-conductor
|
|
53
|
+
|
|
54
|
+
# 4. Read the verdict
|
|
55
|
+
# FH_STATUS: SUCCESS
|
|
56
|
+
# FH_GATE_VERDICT: PASS | PENDING | BLOCKED
|
|
57
|
+
# findings: (YAML block — actionable issues with grade A/B/C)
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
### Evidence
|
|
61
|
+
|
|
62
|
+
**Controlled trial (2026-05-31):** Applied to `packages/opencode/src/permission/arity.ts` — 163 lines, AI-generated, 6 unit tests all passing, self-evaluation verdict: DONE.
|
|
63
|
+
|
|
64
|
+
FH governance verdict: **PENDING**.
|
|
65
|
+
|
|
66
|
+
Findings:
|
|
67
|
+
- **A-grade:** `prefix()` short-token path — permission allowlist may not cover bare commands. Untested execution path.
|
|
68
|
+
- **A-grade:** `npx`, `opencode`, `claude`, `bunx`, `uvx` absent from ARITY table — `npx <anything>` receives the same broad allow pattern, security model weakened.
|
|
69
|
+
- **B-grade:** AI-generated dictionary has no maintenance protocol.
|
|
70
|
+
|
|
71
|
+
**Causal attribution:** same code, same model (Claude), same CI — different methodology layer. The delta is attributed to the governance pass, not the model.
|
|
72
|
+
|
|
73
|
+
### When to use
|
|
74
|
+
|
|
75
|
+
- After any OpenCode session that touches security-sensitive files (`permission/`, `auth/`, `token/`, `key/`)
|
|
76
|
+
- Before merging AI-generated code to main
|
|
77
|
+
- As part of a Stop hook: OpenCode exits → fh-gate.sh auto-triggers → verdict in git notes
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## With Hermes — Skill Configuration Audit Before Dispatch
|
|
82
|
+
|
|
83
|
+
### The gap FH closes
|
|
84
|
+
|
|
85
|
+
Hermes orchestrates agents through skill files. Skill configuration errors (missing validation, unsandboxed context, credential exposure) only surface at runtime — sometimes in production. FH governance catches these before the skill is dispatched.
|
|
86
|
+
|
|
87
|
+
### Workflow
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
# 1. After adding or modifying a Hermes skill
|
|
91
|
+
SKILL_PATH="skills/autonomous-ai-agents/opencode/SKILL.md"
|
|
92
|
+
|
|
93
|
+
# 2. Run FH governance on the skill file
|
|
94
|
+
export FH_TARGET_FILES="$SKILL_PATH"
|
|
95
|
+
export FH_CALLER="hermes"
|
|
96
|
+
export FH_GATE_LEVEL="standard"
|
|
97
|
+
./scripts/fh-gate.sh "$FH_TARGET_FILES" "$FH_GATE_LEVEL" "$FH_CALLER"
|
|
98
|
+
|
|
99
|
+
# 3. Before dispatching the skill, check verdict
|
|
100
|
+
# If PENDING → review findings before allowing live dispatch
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### Evidence
|
|
104
|
+
|
|
105
|
+
**Gemini sidecar review (2026-05-31):** Applied to Hermes `skills/autonomous-ai-agents/opencode/SKILL.md`.
|
|
106
|
+
|
|
107
|
+
Findings:
|
|
108
|
+
- **A-grade:** No pre-execution plan validation — OpenCode operates directly on `workdir` without a review step before file mutations. Destructive rewrites land unreviewed.
|
|
109
|
+
- **A-grade:** Secret ingestion risk — OpenCode may ingest `.env` files or `.git` history into prompt context, potentially exposing credentials to external LLM providers.
|
|
110
|
+
- **B-grade:** No convergence verification — background sessions can exit with success status while leaving codebase in broken state.
|
|
111
|
+
|
|
112
|
+
**Filed as:** Hermes issue [#35709](https://github.com/NousResearch/hermes-agent/issues/35709).
|
|
113
|
+
|
|
114
|
+
### When to use
|
|
115
|
+
|
|
116
|
+
- Before committing any new Hermes skill that dispatches an external agent
|
|
117
|
+
- During Hermes skill review cycles
|
|
118
|
+
- As part of Hermes CI: run fh-gate.sh on changed skill files in PRs
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## With OpenHuman — Memory Tree Staleness Audit
|
|
123
|
+
|
|
124
|
+
### The gap FH closes
|
|
125
|
+
|
|
126
|
+
OpenHuman's Memory Tree stores persistent context (technical decisions, file paths, architectural claims). These are validated once at write time, then trusted indefinitely. FH's `memory-hygiene` + `phantom-quench` pattern re-verifies memory entries against live state.
|
|
127
|
+
|
|
128
|
+
### Workflow
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
# 1. Extract memory entries that reference file paths or technical claims
|
|
132
|
+
# (OpenHuman Memory Tree format — paths, decisions, URLs)
|
|
133
|
+
|
|
134
|
+
# 2. For each entry, verify it still holds
|
|
135
|
+
for entry in memory_entries:
|
|
136
|
+
verify file paths exist in current codebase
|
|
137
|
+
verify external URLs still reachable
|
|
138
|
+
verify technical claims match current code
|
|
139
|
+
|
|
140
|
+
# 3. Verdict per entry:
|
|
141
|
+
# GROUNDED — verified, safe to use
|
|
142
|
+
# STALE — entry exists but referenced reality changed (warn, prompt for update)
|
|
143
|
+
# BROKEN — hard reference failure (flag for immediate update)
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
### Evidence
|
|
147
|
+
|
|
148
|
+
**Structural analysis (2026-05-31):** OpenHuman's `pr-review-shepherd` skill documentation notes that "Composio catalog slugs occasionally drift" — an acknowledgment that the stale-but-confident failure mode exists in the project's own documentation.
|
|
149
|
+
|
|
150
|
+
**FH's `memory-hygiene` skill** implements this pattern for harness memory with a verified weekly cadence. The same algorithm applies directly to OpenHuman's Memory Tree.
|
|
151
|
+
|
|
152
|
+
**Filed as:** OpenHuman issue [#3069](https://github.com/tinyhumansai/openhuman/issues/3069).
|
|
153
|
+
|
|
154
|
+
### When to use
|
|
155
|
+
|
|
156
|
+
- Weekly, or on session start if >7 days since last audit
|
|
157
|
+
- After major refactors that move or delete files
|
|
158
|
+
- After reversing a significant architectural decision
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## Expected Outcomes (Honest Estimates)
|
|
163
|
+
|
|
164
|
+
Based on the controlled trial and sidecar experiments:
|
|
165
|
+
|
|
166
|
+
| Scenario | Expected FH Verdict | Finding Rate |
|
|
167
|
+
|---|---|---|
|
|
168
|
+
| AI-generated code, CI passing, no prior governance | PENDING (expected) | 1–3 A/B-grade findings per session (empirical: 2A+1B on arity.ts) |
|
|
169
|
+
| Hermes skill configuration, new external agent | PENDING (expected) | 1–2 A-grade findings per skill (empirical: 2A+1B on opencode skill) |
|
|
170
|
+
| Human-written code, existing tests passing | PASS (expected) | 0–1 B-grade; A-grade unlikely but possible |
|
|
171
|
+
| Previously FH-reviewed code, no changes | PASS | Near-zero new findings (compounding effect) |
|
|
172
|
+
|
|
173
|
+
**Important caveat:** These are estimates from a small sample. Your actual finding rate depends on code quality, domain, and how much adversarial pressure the governance pass applies. The finding rate is not guaranteed — the governance execution is.
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## The "No Integration Required" Value Proposition
|
|
178
|
+
|
|
179
|
+
All three workflows above require:
|
|
180
|
+
1. `git diff` to capture changed files
|
|
181
|
+
2. `./scripts/fh-gate.sh` to generate a structured governance prompt
|
|
182
|
+
3. Reading the verdict
|
|
183
|
+
|
|
184
|
+
No API integration. No OpenCode plugin. No Hermes adapter. No OpenHuman SDK.
|
|
185
|
+
|
|
186
|
+
FH reads files OpenCode / Hermes / OpenHuman write. The protocol is the interface.
|
|
187
|
+
|
|
188
|
+
This also means FH governance can be added **at any point in an existing workflow** — no architectural changes required.
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
## Compounding Effect
|
|
193
|
+
|
|
194
|
+
The first FH governance pass on a codebase finds the most issues. Each subsequent pass finds fewer — not because the tool is degrading, but because the codebase is improving. This is the governance dividend:
|
|
195
|
+
|
|
196
|
+
```
|
|
197
|
+
Pass 1: PENDING (2 A-grade, 1 B-grade)
|
|
198
|
+
↓ fix
|
|
199
|
+
Pass 2: PENDING (0 A-grade, 1 B-grade)
|
|
200
|
+
↓ fix
|
|
201
|
+
Pass 3: PASS
|
|
202
|
+
↓ future changes re-enter at Pass 1 only for the changed files
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
Over time, the governance overhead per PR decreases while baseline code quality rises. The tool does not get harder to satisfy — the codebase gets easier to verify.
|
|
206
|
+
|
|
207
|
+
---
|
|
208
|
+
|
|
209
|
+
## References
|
|
210
|
+
|
|
211
|
+
- **Controlled trial (Experiment 2):** `tracks/_meta/fh_opencode_governance_experiment_2026_05_31.md`
|
|
212
|
+
- **Sidecar experiment (Experiment 1):** `knowledge/shared/harness-core/multi_model_sidecar_strategy.md`
|
|
213
|
+
- **Integration contract spec:** `knowledge/shared/harness-core/fh_integration_contract.md`
|
|
214
|
+
- **fh-gate.sh:** `scripts/fh-gate.sh`
|
|
215
|
+
- **OpenCode issue:** github.com/anomalyco/opencode/issues/30057
|
|
216
|
+
- **Hermes issue:** github.com/NousResearch/hermes-agent/issues/35709
|
|
217
|
+
- **OpenHuman issue:** github.com/tinyhumansai/openhuman/issues/3069
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: gate-locality-principle
|
|
3
|
+
description: A safety gate must live where the actor that needs it actually reads it — a gate defined in a place the enforcing actor never loads is decorative, not enforced.
|
|
4
|
+
type: reference
|
|
5
|
+
date: 2026-06-20
|
|
6
|
+
tags: [governance, gate-locality, multi-runtime, judge-robustness, field-harvest]
|
|
7
|
+
originProjects: [two restricted-env field harnesses]
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Gate-Locality Principle
|
|
11
|
+
|
|
12
|
+
> **A safety gate must live where the actor that needs it actually reads it.**
|
|
13
|
+
> A gate defined in a place the enforcing actor never loads provides the *appearance* of safety
|
|
14
|
+
> with none of the enforcement. Locality is part of the gate's correctness, not an afterthought.
|
|
15
|
+
|
|
16
|
+
This is a sibling of the **judge-robustness / mechanical-anchor** spine: judge-robustness says *don't
|
|
17
|
+
let a foolable judge hold the terminal verdict*; gate-locality says *don't put the gate somewhere the
|
|
18
|
+
enforcer can't see it*. Both fail the same way — a control that looks present but cannot actually fire.
|
|
19
|
+
|
|
20
|
+
## The failure mode (two observed shapes)
|
|
21
|
+
|
|
22
|
+
| Shape | Where the gate was | Who needed it | Why it didn't fire |
|
|
23
|
+
|---|---|---|---|
|
|
24
|
+
| **Code-locality** | absent from the write path entirely | the function that writes to JIRA | the writeback gate checked confidence but not provenance, so a self-inferred finding auto-posted as if verified |
|
|
25
|
+
| **File-locality** | only in a Claude-only `CLAUDE.md` | a Gemini/Codex orchestrator that auto-loads root `AGENTS.md`, not `CLAUDE.md` | the runtime assumed the commander *role* without inheriting the *gates* |
|
|
26
|
+
|
|
27
|
+
Both were found 2026-06-20 across **two field harnesses** (Harness-A: a provenance gate missing from a
|
|
28
|
+
write path; Harness-B + Harness-A: orchestration gates that lived only in a Claude-only file). The
|
|
29
|
+
recurrence across two contexts is what lifts this past a
|
|
30
|
+
single anecdote; it is N=2 within one operator's projects, so **cross-operator confirmation is the
|
|
31
|
+
upgrade path** that would harden it from a working principle to a validated one.
|
|
32
|
+
|
|
33
|
+
## The fix pattern
|
|
34
|
+
|
|
35
|
+
Move the gate into the artifact the enforcing actor actually reads:
|
|
36
|
+
- **Code path** → put the guard *in the function that performs the irreversible action* (e.g. gate the
|
|
37
|
+
writeback candidate generator on `provenance == verified`, not in a doc that describes it).
|
|
38
|
+
- **Multi-runtime orchestration** → put orchestration gates in a **model-agnostic** file every
|
|
39
|
+
runtime loads (`AGENTS.md`), not in a Claude-only `CLAUDE.md`. A non-Claude orchestrator that never
|
|
40
|
+
reads `CLAUDE.md` otherwise gets the role without the governance.
|
|
41
|
+
|
|
42
|
+
## Verification (how to know the locality fix worked)
|
|
43
|
+
|
|
44
|
+
A **blind target-tier sim** is the honest check: feed the enforcing actor ONLY the file/path it
|
|
45
|
+
actually loads (e.g. `AGENTS.md` alone, no `CLAUDE.md`) and present a trap the gate should catch
|
|
46
|
+
(e.g. a high-confidence but unverified finding to auto-write). Pre-fix the actor has no basis to
|
|
47
|
+
refuse; post-fix it refuses, citing the now-local gate. The behavioral delta *is* the proof of
|
|
48
|
+
locality — review alone cannot show it (review reads all files; the runtime does not).
|
|
49
|
+
|
|
50
|
+
## Relationship to other FH assets
|
|
51
|
+
|
|
52
|
+
- **steel-quench** carries this as a Wave-1 attack angle ("Gate-locality — is every safety gate
|
|
53
|
+
readable by the actor that must enforce it?").
|
|
54
|
+
- **judge-robustness / mechanical-anchor** (`[[feedback_judge_robustness_mechanical_anchor]]`) — the
|
|
55
|
+
sibling principle for *verdict* placement; gate-locality is for *gate* placement.
|
|
56
|
+
- **Non-Model Ground** — multi-runtime orchestration is exactly where gate-locality bites, because
|
|
57
|
+
different runtimes load different files.
|