@chrono-meta/fh-gate 1.4.41 → 1.4.43
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +5 -3
- package/CATALOG.md +6 -0
- package/CLAUDE.md +65 -130
- package/docs/CONTRIBUTING.md +2 -2
- package/knowledge/shared/dialogue/ai_dialogue_playbook.md +137 -0
- package/knowledge/shared/dialogue/claude_code_runtime_flow.md +170 -0
- package/knowledge/shared/dialogue/memory_intent_recall.md +209 -0
- package/knowledge/shared/harness-core/claude_md_gate_details.md +170 -0
- package/knowledge/shared/harness-core/companion_store_pluggable_cross_audit_2026-06-11.md +118 -0
- package/knowledge/shared/harness-core/crucible_mode.md +112 -0
- package/knowledge/shared/harness-core/deep_research_capability_ladder.md +122 -0
- package/knowledge/shared/harness-core/fh_detail_protocols.md +163 -0
- package/knowledge/shared/harness-core/fh_ecosystem_positioning.md +147 -0
- package/knowledge/shared/harness-core/fh_opencode_governance_wrapper.md +163 -0
- package/knowledge/shared/harness-core/fh_synergy_playbook.md +217 -0
- package/knowledge/shared/harness-core/gate_locality_principle.md +57 -0
- package/knowledge/shared/harness-core/goal_quench_anthropic_issue.md +104 -0
- package/knowledge/shared/harness-core/harness_6axis_framework.md +136 -0
- package/knowledge/shared/harness-core/harness_design_decision_lens.md +108 -0
- package/knowledge/shared/harness-core/harness_frontier_diagnosis_2026-06-02.md +102 -0
- package/knowledge/shared/harness-core/hub_compounding_loop.md +109 -0
- package/knowledge/shared/harness-core/hub_maturity_roadmap.md +201 -0
- package/knowledge/shared/harness-core/hybrid_orchestration_architecture_roadmap.md +196 -0
- package/knowledge/shared/harness-core/live_surface_automation_pattern.md +110 -0
- package/knowledge/shared/harness-core/measurement-integrity-checklist.md +59 -0
- package/knowledge/shared/harness-core/meta_harness_engineering_definition.md +116 -0
- package/knowledge/shared/harness-core/multi_model_sidecar_strategy.md +651 -0
- package/knowledge/shared/harness-core/persona_container_schema.md +172 -0
- package/knowledge/shared/harness-core/return_path_gate.md +120 -0
- package/knowledge/shared/harness-core/self_evolution_routine.md +268 -0
- package/knowledge/shared/harness-core/skill_quality_rubric.md +71 -0
- package/knowledge/shared/harness-core/tpa_schema.md +136 -0
- package/package.json +3 -2
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: persona-container-schema
|
|
3
|
+
description: Canonical schema for synthesizing a simulation persona from a reusable container. sim-conductor and any persona-dispatch skill fill these slots instead of injecting an ad-hoc role directive. Defines the slots, crowd-scale stop rule, multi-LLM tier distribution, the situation→group→skill dispatch binding (folded skill-container), and the synthesize→validate→graduate lifecycle.
|
|
4
|
+
type: reference
|
|
5
|
+
scope: meta-harness
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Persona Container — Canonical Schema
|
|
9
|
+
|
|
10
|
+
> **The gap this fills** (verified 2026-06-20): FH had no persona-*definition* container. `sim-conductor`
|
|
11
|
+
> *conducts* — it profiles a target, derives a needed perspective, then sources a persona (installed
|
|
12
|
+
> agent → an undefined "built-in fallback palette" injected as an ad-hoc prompt directive →
|
|
13
|
+
> plugin-recommender). There was no reusable *container* that holds a persona's internal logic +
|
|
14
|
+
> external-data wiring. This schema is that container. It does **not** replace sim-conductor (the
|
|
15
|
+
> runner); it is what sim-conductor (and any persona-dispatch skill) **fills** instead of an ad-hoc
|
|
16
|
+
> directive.
|
|
17
|
+
>
|
|
18
|
+
> **Consumer status**: sim-conductor extension **SHIPPED 2026-06-20** (same session as this schema) —
|
|
19
|
+
> Persona Discovery ② now fills these 6 slots instead of an ad-hoc directive; §Scale gained the Crowd
|
|
20
|
+
> tier (16-cap lifted behind the marginal-coverage stop) + cost-tier routing; Step 5 adds the graduate
|
|
21
|
+
> step with the admission test below. The A4 clocked write-off trigger is therefore **resolved** — this
|
|
22
|
+
> doc has a live consumer.
|
|
23
|
+
>
|
|
24
|
+
> **Orthogonal to `tpa_schema.md`** (foreseeable-collision note): TPA profiles the **target** (what is
|
|
25
|
+
> reviewed → which gates fire); this profiles the **persona** (who reviews). TPA's profile *drives*
|
|
26
|
+
> which personas to synthesize — they pair, they do not overlap (zero shared fields).
|
|
27
|
+
|
|
28
|
+
## The container — 6 slots
|
|
29
|
+
|
|
30
|
+
A synthesized persona is a filled instance of these slots. (Empirically used 2026-06-20 to synthesize
|
|
31
|
+
blind `newcomer` / `red-team` / `vulnerable-user` personas that found a real grounding bug a
|
|
32
|
+
design-aware inline pass missed.)
|
|
33
|
+
|
|
34
|
+
| Slot | What it holds | Why it is a slot (not free text) |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| **lens / identity** | who the outsider is (non-tech newcomer · adversarial red-team · domain expert · vulnerable user) | the standpoint; parallax depth comes from the shift *between* standpoints |
|
|
37
|
+
| **internal logic** | what they probe / attack / get stuck on (the persona's decision rules) | makes the reaction reproducible, not vibes |
|
|
38
|
+
| **external grounding** | the real artifact files the persona MUST read + (optional) external-search queries that ground the persona in real data | a persona ungrounded in the artifact hallucinates; cheap bounded grounding keeps it economical |
|
|
39
|
+
| **output protocol** | **reuses sim-conductor Step 1.5 parallax shape** (single source — do not re-spec here; it already defines Strengths / Concerns / Open-questions / Absence-check) | one shape across all personas → cross-persona agreement is meaningful (`[[multi-persona-review]]`) |
|
|
40
|
+
| **cost tier** | which model runs this persona (floor-local · heavy-local · frontier) | the economy lever — see §Multi-LLM tier distribution |
|
|
41
|
+
| **lifecycle** | synthesized · validated · graduated (see §Lifecycle) | most personas stay ephemeral; only proven ones graduate |
|
|
42
|
+
|
|
43
|
+
**Isolation invariant**: a synthesized persona dispatches as an **isolated** agent (no main-session
|
|
44
|
+
context inherited). Isolation is the FH mechanism that keeps a blind reaction honest — a `newcomer`
|
|
45
|
+
that knows the design intent is not a newcomer. (2026-06-20: the blind-container personas found
|
|
46
|
+
UNNAMED bugs precisely because the inline, design-aware pass was contaminated.)
|
|
47
|
+
|
|
48
|
+
## Crowd scale + the marginal-coverage stop (anti-decorative)
|
|
49
|
+
|
|
50
|
+
The container scales from a few personas to a crowd, but **crowd size is output-driven, never a
|
|
51
|
+
target** — there is no persona-count goal, and naming a headline capacity number is itself the
|
|
52
|
+
decorative over-generation steel-quench Wave-1 angle #1 attacks. Two rules bound the crowd:
|
|
53
|
+
|
|
54
|
+
- **Decorrelation**: add a persona only if it is *distinct* (a different viewpoint, OR — at scale —
|
|
55
|
+
a different model family) from the ones already running. A crowd's value is breadth of distinct
|
|
56
|
+
viewpoints, never headcount.
|
|
57
|
+
- **Marginal-coverage stop**: stop adding personas when the marginal new-finding rate → 0. Crowd size
|
|
58
|
+
is *measured against findings*, never a fixed number.
|
|
59
|
+
|
|
60
|
+
(Platform sub-agent ceilings — recent runtimes allow large fan-outs — are a *capacity* fact, relevant
|
|
61
|
+
only as the upper bound the marginal-coverage stop operates well below; they are not a usage target.)
|
|
62
|
+
|
|
63
|
+
## Multi-LLM tier distribution (the economy lever)
|
|
64
|
+
|
|
65
|
+
The `cost tier` slot lets a crowd be distributed across model tiers so breadth is cheap and the verdict
|
|
66
|
+
stays trustworthy. **One probe, not a benchmark** (the-bible truncation-inversion, 2026-06-20, N=1 —
|
|
67
|
+
a single task, a single model trio, one run):
|
|
68
|
+
|
|
69
|
+
| Tier | Example | Role | What the one probe showed |
|
|
70
|
+
|---|---|---|---|
|
|
71
|
+
| **floor-local** | `gemma4:e2b` (MacBook) | cheap breadth / obvious-signal filtering | caught obvious oblique-distress, **MISSED** the subtle truncation-inversion |
|
|
72
|
+
| **heavy-local** | `qwen3:32b` (4090) | stronger decorrelation canary | **CAUGHT** the subtle case with correct reasoning ("fool's heart, not authoritative") |
|
|
73
|
+
| **frontier** | Claude (Opus/Sonnet) | terminal verdict | the blind red-team that originally found the bug |
|
|
74
|
+
|
|
75
|
+
**What this one signal supports — and what it does NOT**: it is a single data point that a 27–32B tier
|
|
76
|
+
*may* clear adversarial reasoning a floor model cannot — enough to justify the floor-local-as-cheap-
|
|
77
|
+
breadth / heavy-local-as-stronger-canary split. It is **not** a measured equivalence to frontier, and
|
|
78
|
+
the heavy-local tier is **never promoted to a gate** on this basis: **the terminal verdict always
|
|
79
|
+
stays frontier** — local tiers are canaries (decorrelation), not gates. (Consistent with
|
|
80
|
+
`[[feedback_judge_robustness_mechanical_anchor]]` and the "Local AI is not Opus" finding — a stronger
|
|
81
|
+
local model lowers the cost of *breadth*, it does not move the *verdict*.)
|
|
82
|
+
|
|
83
|
+
## Dispatch — situation → group → skill (folded skill-container)
|
|
84
|
+
|
|
85
|
+
`[[asset-placement-gate]]` routed a proposed standalone **skill-container** here (2026-06-20): FH 4-criteria
|
|
86
|
+
**FAIL on ④ overlap** — the GROUP axis is derivable from existing slots and dispatch-planning overlaps
|
|
87
|
+
`[[agent-composer]]`, so it does not earn a standalone asset (the same anti-decorative fold as §Sibling's
|
|
88
|
+
skill-foundry). The one net-new sliver — the **runtime situation→group→skill binding** — folds here.
|
|
89
|
+
|
|
90
|
+
**Group = personas sharing a capability-class**, read mechanically off existing slots (NOT a new taxonomy):
|
|
91
|
+
|
|
92
|
+
| Group | derived from slot | bound skill (the dispatch target) | tier-floor |
|
|
93
|
+
|---|---|---|---|
|
|
94
|
+
| Relay | output-protocol = quote-only | verified-corpus quote (fail-closed) | any |
|
|
95
|
+
| Lens / counsel | external-grounding = active | framed reflection + search-synthesize | frontier for the synthesis step |
|
|
96
|
+
| Adversary | lifecycle = ephemeral, paired | quench / failure-mode generation | local OK (generation task-class) |
|
|
97
|
+
| Guardian | output-protocol = verdict | classify-and-block (L2 semantic) | local screen → **frontier on the subtle class** |
|
|
98
|
+
|
|
99
|
+
**Dispatch rule**: classify the incoming situation → route to its group → invoke that group's bound skill.
|
|
100
|
+
The binding carries a **tier-floor per group**, not one global model — consistent with §Multi-LLM above.
|
|
101
|
+
|
|
102
|
+
**Guardian tier-floor is measured** (the-bible red-team dogfood, 2026-06-20, signal in the companion store):
|
|
103
|
+
on CLEAR keyword-evasion the local L2 (qwen3:32b) caught **11/11**, but on the SUBTLE/borderline class it
|
|
104
|
+
collapsed to **1/4** while Gemini-3.5-Flash got 3/4 and GPT-5 / Opus-4.6 got 4/4 (0 false-positives
|
|
105
|
+
across all). So the Guardian binding is *local-screen-then-frontier*: cheap local clears the easy class,
|
|
106
|
+
the subtle residual needs frontier (and ultimately L3 human, per the L2-imperfect residual). A second probe
|
|
107
|
+
reinforcing §Multi-LLM's single truncation-inversion point, same direction — local lowers the cost of
|
|
108
|
+
breadth, frontier keeps the verdict on the cases that matter.
|
|
109
|
+
|
|
110
|
+
**Done When** (skeleton — fold scope only): (1) each group is derivable from a slot, no new taxonomy
|
|
111
|
+
*[mandatory-pass]* · (2) each group names exactly one bound skill *[mandatory-pass]* · (3) a tier-floor is
|
|
112
|
+
stated, measured where claimed *[judged, pair: target-tier sim]*.
|
|
113
|
+
|
|
114
|
+
## Lifecycle — synthesize → simulate → validate → graduate
|
|
115
|
+
|
|
116
|
+
```
|
|
117
|
+
synthesize (fill the container) → simulate (dispatch isolated, collect parallax output)
|
|
118
|
+
→ validate (findings triaged M/S/R)
|
|
119
|
+
→ graduate (ONLY a persona that PASSES the admission test below is embedded in the project's
|
|
120
|
+
static set, e.g. the-bible's core/personas.json) — most personas never graduate;
|
|
121
|
+
the throwaway run still accelerated the work.
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
**Graduation admission test** (mechanical — closes the judge-only path; a "proven" persona is not a vibe):
|
|
125
|
+
|
|
126
|
+
- **measured**: the persona surfaced **≥1 unique S/M finding** (a finding no other persona in the run
|
|
127
|
+
produced) across **≥2 distinct targets** — recurrence, not a one-off. A persona that fired once is a
|
|
128
|
+
lucky throwaway, not a graduate.
|
|
129
|
+
- **mandatory-pass**: if graduating into a *skill* (not just a project's persona set), it additionally
|
|
130
|
+
clears the **FH 6-item creation gate** (the skill-foundry inversion below — the trust bar for a
|
|
131
|
+
governance asset is non-negotiable).
|
|
132
|
+
|
|
133
|
+
Graduation is **optional and rare** — the analogue of "app-ification": a persona earns a permanent
|
|
134
|
+
home only when it repeatedly proves itself. The cheap, dynamic, mostly-discarded synthesis is the
|
|
135
|
+
point (cost-efficiency > build-and-demolish).
|
|
136
|
+
|
|
137
|
+
## How a runner consumes this (sim-conductor extension — shipped 2026-06-20)
|
|
138
|
+
|
|
139
|
+
`sim-conductor` is the runner that **fills** these slots instead of injecting an ad-hoc role directive.
|
|
140
|
+
The shipped extension (through the FH 4-axis gate, HITL): sim-conductor's Persona Discovery ② reads this
|
|
141
|
+
schema and fills the 6 slots; §Scale lifts its 3–16 cap behind the marginal-coverage stop (Crowd tier)
|
|
142
|
+
and assigns `cost tier` per persona for multi-LLM distribution; Step 5 adds the graduate step. Routing
|
|
143
|
+
decided by `[[asset-placement-gate]]` (2026-06-20): **not a new forked skill** (≈60–70% overlap with
|
|
144
|
+
sim-conductor's runner role) — this schema is the net-new asset, sim-conductor is extended to consume it.
|
|
145
|
+
|
|
146
|
+
## Sibling — skill-foundry (terminal-before-appification)
|
|
147
|
+
|
|
148
|
+
The same lifecycle generalizes to **skills**, with one inversion that is the whole point: a persona is a
|
|
149
|
+
low-trust simulation lens (ephemeral-and-many is the value); a skill is a governance asset whose value is
|
|
150
|
+
being **trusted**. A persona's low trust bar is harmless (a wrong lens is a discarded run); a skill's is
|
|
151
|
+
dangerous (a wrong instruction is trusted and reused). So a synthesized ("terminal-stage") skill is
|
|
152
|
+
explicitly **provisional / ungated / unpublished** — its SKILL.md is marked `lifecycle: provisional` with
|
|
153
|
+
an "ungated, in-session use only" banner so it cannot masquerade as a gate-passed skill.
|
|
154
|
+
|
|
155
|
+
**Provisional convention** (salience-dependent, NOT hook-enforced — no checker scans `lifecycle:` today;
|
|
156
|
+
it is a discipline, not a mechanical control): (1) used in the authoring session only — **not committed to
|
|
157
|
+
a shared plugin or marketplace before graduation**; (2) **deleted if it never graduates** —
|
|
158
|
+
build-and-keep-forever is the anti-pattern (the cheap-and-discarded discipline is the value); (3)
|
|
159
|
+
**graduates only through the FH 6-item creation gate** (CLAUDE.md §New Skill Creation Pre-Commit Gate) +
|
|
160
|
+
`[[asset-placement-gate]]` routing — HITL, never auto. Authoring reuses `contention-layer` Step-4's
|
|
161
|
+
skeleton (`origin: skill-foundry`; drop/replace `contention-parents` with a `source-action:` pointer,
|
|
162
|
+
mirroring that skeleton's field-scaffold reuse note). (Illustrative — *not* a committed artifact: a
|
|
163
|
+
`persona-safety-sweep` provisional skill kept local/gitignored, graduation-target = a sim-conductor preset.)
|
|
164
|
+
|
|
165
|
+
## Provenance
|
|
166
|
+
|
|
167
|
+
Validated end-to-end on the `the-bible` project 2026-06-20: container-filled blind personas found a
|
|
168
|
+
real substring-truncation grounding bypass; the multi-LLM tier comparison (e2b miss / 32B catch /
|
|
169
|
+
frontier verdict) grounded the cost-tier slot. Operator-origin concept; `[[asset-placement-gate]]`
|
|
170
|
+
routed it (schema here + sim-conductor extension + thin skill-foundry convention). Related:
|
|
171
|
+
`[[multi-persona-review]]` (the output-protocol architecture this builds on) · `tpa_schema.md` (the
|
|
172
|
+
target-profile schema sim-conductor pairs with this).
|
|
@@ -0,0 +1,120 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: return-path-gate
|
|
3
|
+
description: Skill chain pattern where a downstream skill returns a structured conditional verdict back to its caller, which gates its next step on that verdict. Prevents fire-and-forget chains and closes the feedback loop.
|
|
4
|
+
type: pattern
|
|
5
|
+
date: 2026-05-29
|
|
6
|
+
tags: [pattern, skill-chain, verdict, conditional-pass, closed-loop]
|
|
7
|
+
instances:
|
|
8
|
+
- apex-review → sim-conductor
|
|
9
|
+
- agent-composer ↔ deliberation
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
# Return-Path Gate — Closed-Loop Skill Chain Pattern
|
|
13
|
+
|
|
14
|
+
A skill chain is **closed** when the downstream skill's output feeds back into the upstream caller's decision logic. Without a return path, downstream calls are fire-and-forget: the caller can't act on results, conditions go unresolved, and the chain terminates with unverified state.
|
|
15
|
+
|
|
16
|
+
The Return-Path Gate enforces closure: the caller registers a verdict expectation before calling downstream, the downstream returns a structured verdict, and the caller gates its next step on that verdict.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Structure
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
[Caller Skill]
|
|
24
|
+
Step N: Detect trigger condition
|
|
25
|
+
Step N+1: Dispatch → [Callee Skill] (with topic + context)
|
|
26
|
+
Step N+2: ◀ WAIT for verdict ▶
|
|
27
|
+
Step N+3: Gate on verdict
|
|
28
|
+
PASS → proceed normally
|
|
29
|
+
CONDITIONAL → resolve listed conditions before proceeding
|
|
30
|
+
FAIL/ESCALATE → block + surface to user
|
|
31
|
+
|
|
32
|
+
[Callee Skill]
|
|
33
|
+
Processes topic
|
|
34
|
+
Returns structured verdict (not a binary win/lose)
|
|
35
|
+
↓
|
|
36
|
+
[Caller] receives verdict → re-enters decision logic
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## Verdict Format (standard)
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
Verdict: PASS | CONDITIONAL_PASS | FAIL | ESCALATE
|
|
45
|
+
Conditions: [list — only present when CONDITIONAL_PASS]
|
|
46
|
+
Basis: [one-line rationale]
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
- **PASS** — proceed without modification
|
|
50
|
+
- **CONDITIONAL_PASS** — proceed only after listed conditions are addressed; conditions must be explicitly resolved, not silently skipped
|
|
51
|
+
- **FAIL** — block; requires redesign before proceeding
|
|
52
|
+
- **ESCALATE** — surface to user; decision requires human judgment
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Verified Instances
|
|
57
|
+
|
|
58
|
+
### 1. apex-review → sim-conductor
|
|
59
|
+
|
|
60
|
+
**Caller**: `apex-review`
|
|
61
|
+
**Callee**: `sim-conductor` (Area E — external scenario validation)
|
|
62
|
+
|
|
63
|
+
**Trigger**: apex-review produces `Conditionally passed` verdict
|
|
64
|
+
**Return path**: sim-conductor Area E results fold back into the apex-review HTML deck
|
|
65
|
+
**Gate logic**:
|
|
66
|
+
- `Passed` → deck is submission-ready
|
|
67
|
+
- `Conditionally passed` → **mandatory sim-conductor dispatch** (option B is default; user must explicitly opt out)
|
|
68
|
+
- `Rejected` → redesign required
|
|
69
|
+
|
|
70
|
+
**Why it matters**: Without this gate, persona conditions from apex-review are listed but never verified. sim-conductor's challenger/beginner/main-player validation is the only mechanism that resolves them.
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
### 2. agent-composer ↔ deliberation
|
|
75
|
+
|
|
76
|
+
**Caller**: `agent-composer` (Step 4-b state transition)
|
|
77
|
+
**Callee**: `deliberation` (3-layer: Innovator → Devil → Mediator)
|
|
78
|
+
|
|
79
|
+
**Trigger**: 2+ conflicting suggestions or design decision conflict detected in fan-in results → `Wave next-D`
|
|
80
|
+
**Return path**: deliberation synthesis verdict folds back into agent-composer Step 4-b fan-in result set → conflict marked resolved → state transition re-runs
|
|
81
|
+
**Gate logic**:
|
|
82
|
+
- Synthesis verdict (conditional, not binary) replaces the conflict entry
|
|
83
|
+
- Step 4-b re-evaluates with conflict resolved
|
|
84
|
+
- Auto-execution forbidden — user decision finalizes
|
|
85
|
+
|
|
86
|
+
**Why it matters**: Without the return fold, deliberation output is displayed to the user but the agent-composer orchestration loop doesn't update — the conflict remains unresolved in the state machine.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## Anti-Pattern: Fire-and-Forget
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
[Caller] → [Callee]
|
|
94
|
+
↓
|
|
95
|
+
outputs result (displayed to user)
|
|
96
|
+
↓
|
|
97
|
+
[Caller continues independently — does not act on result]
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
This is the default failure mode when a skill chain has no return path defined. Symptoms:
|
|
101
|
+
- Downstream output is shown but not acted on
|
|
102
|
+
- Conditions listed by callee remain unverified
|
|
103
|
+
- Caller proceeds as if callee succeeded unconditionally
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## Implementation Checklist (when adding a Return-Path Gate to a skill)
|
|
108
|
+
|
|
109
|
+
- [ ] Caller SKILL.md: `§Done When` — explicitly states "wait for callee verdict before proceeding"
|
|
110
|
+
- [ ] Callee SKILL.md: `§Done When` — outputs structured verdict (PASS / CONDITIONAL_PASS / FAIL / ESCALATE)
|
|
111
|
+
- [ ] Callee SKILL.md: `§Chains` — "returns verdict to `[caller]`"
|
|
112
|
+
- [ ] Caller SKILL.md: `§Chains` — "→ Mandatory next (`[callee]`) when [condition]; verdict folds back into [step]"
|
|
113
|
+
- [ ] CONDITIONAL_PASS gate: default path is resolution, not bypass
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## Relation to Other Patterns
|
|
118
|
+
|
|
119
|
+
- **Three-Doctor Loop** (`harness-doctor ↔ context-doctor ↔ sim-conductor`) uses return paths implicitly — each doctor's output triggers the next. A full Return-Path Gate implementation would make the fold-back explicit.
|
|
120
|
+
- **verify-bidirectional** is a single-depth return path: AI recommendation → user counter-argument → AI baseline update. The verdict here is implicit (user correction = FAIL signal).
|
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
# Self-Evolution Routine — Remote Daily/Weekly Innovator Loop
|
|
2
|
+
|
|
3
|
+
> **What this is**: a paste-ready Claude Code **Routines** configuration that runs `frontier-digest` +
|
|
4
|
+
> `persona-innovator` on a remote schedule, accumulating frontier signals daily and proposing one
|
|
5
|
+
> concrete FH improvement weekly — **as a draft PR for human merge, never auto-merge**.
|
|
6
|
+
>
|
|
7
|
+
> **Governance stance (proposal-type, HITL-preserving)**: the routine *runs* autonomously; the
|
|
8
|
+
> decision to *adopt* an improvement stays with the operator. This keeps the
|
|
9
|
+
> `feedback_no_personal_commit_to_shared_repo` spine intact even though Routine sessions execute with
|
|
10
|
+
> no approval prompts. See §3.
|
|
11
|
+
|
|
12
|
+
## 1. Why a routine (and why this shape)
|
|
13
|
+
|
|
14
|
+
Claude Code on the web supports **Routines** (`code.claude.com/docs/en/routines.md`): a saved
|
|
15
|
+
prompt + repo + connectors that runs automatically on a **schedule** (≥1h interval), a **GitHub
|
|
16
|
+
event**, or an **HTTP API** call. Each run is a fresh, ephemeral cloud session — no local state
|
|
17
|
+
survives, the container is reclaimed, and by default it may only push to `claude/`-prefixed branches.
|
|
18
|
+
|
|
19
|
+
Three platform facts shape the design below:
|
|
20
|
+
|
|
21
|
+
| Platform fact | Consequence for this loop |
|
|
22
|
+
|---|---|
|
|
23
|
+
| Runs are **ephemeral** (fresh clone each time) | Cross-run memory cannot live in the working tree — it lives on **GitHub** (issue comments / draft PR). |
|
|
24
|
+
| `tracks/**` is **gitignored** (public mirror = methodology only) | Daily digests **must not** be committed to `tracks/_meta/` — they would be dropped by git and lost on reclaim. Durable store = a standing GitHub Issue. |
|
|
25
|
+
| Routine sessions run with **no approval prompts** | "Self-improvement" must be **gated by a draft PR**, not auto-merged — the human merge IS the HITL gate. |
|
|
26
|
+
|
|
27
|
+
## 2. Architecture — Daily scan, Weekly propose
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
[Daily Routine] (schedule: daily, model: sonnet)
|
|
31
|
+
frontier-digest autonomous collection (HN + arxiv; operator-intake arm skipped — no human in loop)
|
|
32
|
+
→ post the digest as a COMMENT on the standing Issue "🛰️ Frontier Digest Log"
|
|
33
|
+
→ no repo commit, no PR (cheap, ephemeral-safe)
|
|
34
|
+
|
|
35
|
+
[Weekly Routine] (schedule: weekly, model: OPUS — required, see §4 floor note)
|
|
36
|
+
① read the last 7 days of comments on the "🛰️ Frontier Digest Log" issue
|
|
37
|
+
② persona-innovator Mode F — gap + external-frontier scan against current FH assets
|
|
38
|
+
③ pick AT MOST ONE concrete, scoped improvement candidate (or none — "no proposal this week" is valid)
|
|
39
|
+
④ install + run the FH 4-axis gate on the change (see §4)
|
|
40
|
+
⑤ commit to a fresh claude/frontier-auto-{YYYY-MM-DD} branch → open a **DRAFT PR**
|
|
41
|
+
⑥ STOP. The operator reviews and merges. ← HITL gate
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**Why the daily/weekly split** (not "daily full"): FH's own cadence is 7-day for `frontier-digest`
|
|
45
|
+
and event-bound for `persona-innovator` — a daily full synthesis would tax tokens and risk
|
|
46
|
+
decorative-unit over-generation (the exact failure `steel-quench` Wave-1 #1 attacks). Daily =
|
|
47
|
+
cheap signal accumulation; weekly = one considered proposal.
|
|
48
|
+
|
|
49
|
+
## 3. Governance guards (do not weaken)
|
|
50
|
+
|
|
51
|
+
- **Draft PR only — never merge.** The weekly routine opens a *draft* PR and stops. Auto-merge would
|
|
52
|
+
bypass `feedback_no_personal_commit_to_shared_repo`.
|
|
53
|
+
- **One proposal per week, or none.** "No proposal this week" is a valid, preferred outcome over a
|
|
54
|
+
forced low-value change. Promotion of `persona-innovator` itself is *measured* (accepted ≥ 60% gate,
|
|
55
|
+
`operations.md`) — the draft-PR accept/reject history IS that pilot data.
|
|
56
|
+
- **`claude/` branch prefix only.** Do not enable "unrestricted branch pushes" for this routine.
|
|
57
|
+
- **Methodology mirror stays clean.** Daily digests live on the GitHub Issue, never in `tracks/**`
|
|
58
|
+
(gitignored) and never committed to the methodology tree.
|
|
59
|
+
- **Cost ceiling.** Both arms run `WebSearch`/`WebFetch`. Daily draws subscription usage and counts
|
|
60
|
+
against the per-account routine daily cap; keep the daily prompt to collection-only.
|
|
61
|
+
- **Cite-verify (PR #112 lesson, 2026-06-22) — do not weaken.** Any citation / source / version claim
|
|
62
|
+
the weekly change *introduces into an FH asset* (distinct from citing the signal URL) must be
|
|
63
|
+
WebSearch-verified before commit. Unverifiable → omit it or mark `(unverified)`; never ship a
|
|
64
|
+
plausible-sounding but unchecked citation. **Origin**: a weekly run added a fabricated "O'Reilly AI
|
|
65
|
+
Agents Stack 2026" citation (+ an over-specific "~32K" threshold) that reached draft PR #112; the
|
|
66
|
+
merger caught it via WebSearch (real source = Chroma 2025), but **Axis 3 phantom-quench is
|
|
67
|
+
"if available"** in the routine sandbox, so the prompt must enforce cite-verify directly — the gate
|
|
68
|
+
cannot be relied on for it.
|
|
69
|
+
|
|
70
|
+
## 4. The 4-axis gate inside the weekly routine
|
|
71
|
+
|
|
72
|
+
Because a fresh clone does **not** install the FH pre-commit hook, the weekly routine must install and
|
|
73
|
+
run it before committing an FH-asset change:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
git config core.hooksPath templates/.git-hooks && chmod +x templates/.git-hooks/pre-commit
|
|
77
|
+
bash templates/regression_guard.sh --pr "$(git branch --show-current)" # Axis 1
|
|
78
|
+
# Axis 2 steel-quench + Axis 3 phantom-quench → write tracks/_meta/.axes_23_passed_{branch}_{date}.marker
|
|
79
|
+
# Axis 4 edit-manifest RECORD → append today's predicted-impact line to tracks/_meta/edit_manifest.yaml
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
**Floor note — why the weekly routine MUST be opus (load-bearing).** The pre-commit hook cross-checks
|
|
83
|
+
the marker's `axis2-model` against `floor-status`: a `sonnet`/`haiku` model claiming `at-floor` or
|
|
84
|
+
`above-floor` is **rejected**, and `below-floor` is rejected too **unless** a `below-floor-ack:` line
|
|
85
|
+
quotes a live operator utterance. A routine runs with **no human online**, so there is no utterance to
|
|
86
|
+
quote — meaning a Sonnet weekly run **cannot legally pass the gate and cannot commit**. That dead-end
|
|
87
|
+
is itself a hazard: under "produce a draft PR" pressure a session may improvise an unsanctioned escape
|
|
88
|
+
(fabricate an ack, mislabel the floor, `--no-verify`). **Pin the weekly routine to opus** so the
|
|
89
|
+
honest marker is `floor-status: at-floor` and the gate passes unattended. (If opus is ever unavailable
|
|
90
|
+
in routines, the sanctioned fallback is: do **not** commit — attach the proposed patch to a draft PR
|
|
91
|
+
opened via the GitHub tools as a diff in the PR body / a patch file, bypassing the local hook, and
|
|
92
|
+
label it `gate: deferred — opus re-run needed` for the operator. Never `--no-verify`.)
|
|
93
|
+
|
|
94
|
+
**Marker carve-out (the one sanctioned `tracks/**` write).** The two gate files
|
|
95
|
+
(`tracks/_meta/.axes_23_passed_*.marker` and `tracks/_meta/edit_manifest.yaml`) live under gitignored
|
|
96
|
+
`tracks/**` and are **never committed** — the hook reads them from the **working tree**, which
|
|
97
|
+
persists for the duration of the run. The §3 "no `tracks/**`" guard is about *committing the digest*,
|
|
98
|
+
not these working-tree gate markers.
|
|
99
|
+
|
|
100
|
+
If `steel-quench`/`phantom-quench` are unavailable in the routine session, note
|
|
101
|
+
`Axis N: skipped (skill unavailable)` — Axis 1 PASS alone unblocks a *draft* PR (Axes 2–3 are the
|
|
102
|
+
operator's residual at merge review).
|
|
103
|
+
|
|
104
|
+
**Skip-visibility at the handoff (load-bearing for honest HITL escalation).** When Axis 2
|
|
105
|
+
(challenger / steel-quench) is skipped, the adversarial check did not run autonomously — it becomes
|
|
106
|
+
the *merger's* responsibility. An intelligent hand-off must make that visible at the hand-off point,
|
|
107
|
+
not bury it. So the draft PR opened by the weekly routine MUST carry a top-of-body label whenever
|
|
108
|
+
Axis 2 was skipped: `⚠️ Axis 2 (adversarial/challenger) NOT RUN this routine — adversarial review is
|
|
109
|
+
the merger's before merge.` A silent skip turns the draft PR into an unchallenged proposal wearing a
|
|
110
|
+
"gate passed" coat; the label keeps the escalation honest. (Axis 2 *passing* needs no such label — the
|
|
111
|
+
marker already records `axis2-engine`/`axis2-evidence`.)
|
|
112
|
+
|
|
113
|
+
**Merge-side citation residual (auto-PR cite floor — 2026-06-22 harvest-loop).** §3's cite-verify is
|
|
114
|
+
the *generator-side* half; this is the *merger-side* redundancy. Because the routine sandbox runs
|
|
115
|
+
phantom-quench only "if available" (line above) and **no CI job runs phantom-quench at the merge
|
|
116
|
+
boundary** (the merge-time mechanical gates are `regression-guard.yml` Axis 1 + `validate.yml`
|
|
117
|
+
token/JSON/count only), the citation check on an auto-authored (`claude/`-branch) PR is the **merger's
|
|
118
|
+
explicit, non-skippable residual** — not a mechanical gate. *Honest scope: this is judged prose, not a
|
|
119
|
+
CI check.* Obligation: before merging any auto-authored PR that **adds a citation / URL / version
|
|
120
|
+
claim**, run phantom-quench on that citation surface (or WebSearch-verify it directly); a fabricated
|
|
121
|
+
citation reaching `main` is the failure this closes (origin: PR #112's "O'Reilly AI Agents Stack 2026"
|
|
122
|
+
phantom — the human merger was the only catch).
|
|
123
|
+
|
|
124
|
+
> **Measured-trigger → CI escalation (do not build the CI gate speculatively).** One occurrence (PR
|
|
125
|
+
> #112) is N=1, below `operations.md`'s N=3 recurrence threshold for instrumenting a defect class as a
|
|
126
|
+
> CI probe. So the fix today is the prose residual above, **not** a workflow. **If an auto-PR phantom
|
|
127
|
+
> citation recurs (N≥2 toward N=3)**, escalate to a real mechanical gate: a CI job (extend
|
|
128
|
+
> `validate.yml` or a new workflow) that greps `claude/`-branch PR diffs for added citation tokens and
|
|
129
|
+
> **fails** unless phantom-quench evidence is attached. Until that recurrence is measured, a prose
|
|
130
|
+
> checklist is the right weight — calling it "mechanical" now would be the gate-locality error in
|
|
131
|
+
> reverse (a decorative gate nobody's traffic justifies). See `[[measurement-integrity-checklist]]` for
|
|
132
|
+
> the sibling "instrument-before-trusting" discipline.
|
|
133
|
+
|
|
134
|
+
## 5. Setup steps (operator, one-time)
|
|
135
|
+
|
|
136
|
+
1. **Standing issue** — create one GitHub issue in `chrono-meta/forge-harness` titled
|
|
137
|
+
`🛰️ Frontier Digest Log`, label `frontier-digest`. (Can be auto-created; see §6.) This is the
|
|
138
|
+
durable daily store.
|
|
139
|
+
2. **Daily Routine** — at `claude.ai/code/routines` → New routine:
|
|
140
|
+
- Repo: `chrono-meta/forge-harness`
|
|
141
|
+
- Schedule: daily (pick a low-traffic hour)
|
|
142
|
+
- Model: sonnet
|
|
143
|
+
- Prompt: paste **§7 Daily prompt** (replace every `{ISSUE}` with the issue number from step 1 before saving)
|
|
144
|
+
3. **Weekly Routine** — New routine:
|
|
145
|
+
- Repo: `chrono-meta/forge-harness`
|
|
146
|
+
- Schedule: weekly
|
|
147
|
+
- Model: **opus (required — see §4 floor note; a Sonnet weekly run cannot pass the commit gate)**
|
|
148
|
+
- Prompt: paste **§7 Weekly prompt**
|
|
149
|
+
- **Before saving**: replace every `{ISSUE}` in the prompt with the actual issue number from step 1.
|
|
150
|
+
4. **Network policy**: default "Trusted" allows HN/arxiv/web search. No change needed unless you add
|
|
151
|
+
walled sources.
|
|
152
|
+
5. **Connectors**: keep the GitHub connector; remove others to reduce surface.
|
|
153
|
+
|
|
154
|
+
## 6. Auto-bootstrap (optional)
|
|
155
|
+
|
|
156
|
+
The standing issue can be created from a normal session in FH cwd via the GitHub MCP tools (issue
|
|
157
|
+
title `🛰️ Frontier Digest Log`, body = "Daily frontier-digest comments accumulate here; the weekly
|
|
158
|
+
routine reads the last 7 days."). The two Routines themselves must be created in the web UI — there is
|
|
159
|
+
no documented API to define a routine from inside a session (only to *fire* an existing one).
|
|
160
|
+
|
|
161
|
+
## 7. Paste-ready routine prompts
|
|
162
|
+
|
|
163
|
+
> These are **self-contained** — a fresh Routine session sees no prior context. Each restates its full
|
|
164
|
+
> task. `{ISSUE}` = the "🛰️ Frontier Digest Log" issue number.
|
|
165
|
+
|
|
166
|
+
### 7-a. Daily prompt
|
|
167
|
+
|
|
168
|
+
```
|
|
169
|
+
You are the FH self-evolution DAILY scan, running as an autonomous Claude Code routine in a fresh
|
|
170
|
+
clone of chrono-meta/forge-harness. There is no human in the loop and no prior context.
|
|
171
|
+
|
|
172
|
+
Task — collect, do NOT modify the repo:
|
|
173
|
+
1. Read plugins/fh-meta/skills/frontier-digest/SKILL.md and run its AUTONOMOUS collection arm only
|
|
174
|
+
(HackerNews + arxiv). Let SKILL.md Step 0 resolve the engine itself — it prefers the /deep-research
|
|
175
|
+
built-in (Priority 0: staged gather + cross-check + cited synthesis) if it is in this session's
|
|
176
|
+
skill list, then ANTHROPIC_API_KEY/Sonnet, then plain WebSearch. Do not force an engine. SKIP the
|
|
177
|
+
Step 0.5 operator-intake question — there is no operator online.
|
|
178
|
+
2. In this sandbox expect WebSearch mode (no ANTHROPIC_API_KEY, so the SKILL's API-synthesis arm is
|
|
179
|
+
unavailable) — YOU, the session model, do the synthesis in-context from the WebSearch results.
|
|
180
|
+
Produce a short digest: 3–6 bullets of frontier signals relevant to FH skills/structure, each with
|
|
181
|
+
a source URL and a one-line "relevance to FH" note.
|
|
182
|
+
3. Post the digest as a COMMENT on issue #{ISSUE} ("🛰️ Frontier Digest Log") using the GitHub tools.
|
|
183
|
+
Title the comment with today's date.
|
|
184
|
+
4. Do NOT commit, do NOT push, do NOT open a PR. Daily is collection-only.
|
|
185
|
+
Stop after posting the comment.
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
### 7-b. Weekly prompt
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
You are the FH self-evolution WEEKLY proposer, running as an autonomous Claude Code routine in a fresh
|
|
192
|
+
clone of chrono-meta/forge-harness. No human is online during the run. Your output is a DRAFT PR for
|
|
193
|
+
later human review — you must NOT merge anything.
|
|
194
|
+
|
|
195
|
+
Task:
|
|
196
|
+
1. Read the last 7 days of comments on issue #{ISSUE} ("🛰️ Frontier Digest Log") via the GitHub tools.
|
|
197
|
+
These are the week's accumulated frontier signals.
|
|
198
|
+
2. Dispatch the persona-innovator agent (plugins/fh-meta/agents/persona-innovator.md) in Mode F
|
|
199
|
+
(gap + external-frontier scan) against the current FH assets and those signals. If sub-agent
|
|
200
|
+
dispatch is unavailable in this runtime, run it INLINE: read the agent file and execute its Mode F
|
|
201
|
+
steps in this session.
|
|
202
|
+
3. Select AT MOST ONE concrete, well-scoped FH improvement. THE BAR: the candidate must name a
|
|
203
|
+
SPECIFIC FH asset (file + section) AND cite a SPECIFIC signal URL from this week's issue comments.
|
|
204
|
+
If you cannot cite both, post a comment on #{ISSUE} saying "No proposal this week — signals logged"
|
|
205
|
+
and STOP. A no-op week is the PREFERRED outcome when the bar isn't met (it is also pilot data: it
|
|
206
|
+
shows the bar holds). A forced low-value PR costs the operator a review and erodes trust — when in
|
|
207
|
+
doubt, no-op.
|
|
208
|
+
4. If you have one improvement, implement it minimally on a NEW branch named
|
|
209
|
+
claude/frontier-auto-<today's date>.
|
|
210
|
+
**CITE-VERIFY (mandatory — distinct from step 3's signal URL):** if your change introduces ANY
|
|
211
|
+
citation, source name, version, benchmark number, or "per <source> <year>" claim INTO an FH asset,
|
|
212
|
+
WebSearch-verify it FIRST. If you cannot confirm the exact source, OMIT it or write "(unverified)" —
|
|
213
|
+
never ship a plausible-but-unchecked citation. A fabricated citation in an FH asset is a phantom;
|
|
214
|
+
Axis 3 (phantom-quench) may be skipped in this sandbox, so this step does NOT depend on the gate.
|
|
215
|
+
(This guard exists because a prior run shipped a fabricated "O'Reilly AI Agents Stack 2026" cite to
|
|
216
|
+
PR #112 — real source was Chroma 2025.)
|
|
217
|
+
5. Run the FH 4-axis gate before committing:
|
|
218
|
+
- git config core.hooksPath templates/.git-hooks && chmod +x templates/.git-hooks/pre-commit
|
|
219
|
+
- bash templates/regression_guard.sh --pr "$(git branch --show-current)" (Axis 1, must PASS)
|
|
220
|
+
- Axis 2 steel-quench + Axis 3 phantom-quench if available → write the
|
|
221
|
+
tracks/_meta/.axes_23_passed_<branch>_<date>.marker with fields axis2-engine / axis2-model /
|
|
222
|
+
floor-status / axis2-evidence. If a skill is unavailable, note "skipped (skill unavailable)".
|
|
223
|
+
- Axis 4: append a one-line predicted-impact entry to tracks/_meta/edit_manifest.yaml.
|
|
224
|
+
6. Commit (the hook enforces the gate), push the claude/frontier-auto-* branch, and open a DRAFT pull
|
|
225
|
+
request titled "frontier-auto: <one-line summary>" describing the signal it came from and the
|
|
226
|
+
predicted impact. IF Axis 2 (challenger/steel-quench) was skipped (skill unavailable), the FIRST
|
|
227
|
+
line of the PR body MUST be: "⚠️ Axis 2 (adversarial/challenger) NOT RUN — adversarial review is
|
|
228
|
+
the merger's." Do not bury the skip — the merger has to know the proposal is unchallenged.
|
|
229
|
+
7. Do NOT mark the PR ready, do NOT merge. STOP after opening the draft PR.
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
## 8. Field-project adoption — any user, any project
|
|
233
|
+
|
|
234
|
+
The loop is **not FH-only**. Anyone accelerating their own project — or the operator on any mapped field
|
|
235
|
+
project — can adopt the same shape. Only **two things are FH-specific and must be swapped**; the rest
|
|
236
|
+
is project-agnostic.
|
|
237
|
+
|
|
238
|
+
| Layer | FH (this doc) | Field-project version |
|
|
239
|
+
|---|---|---|
|
|
240
|
+
| Daily scan lens | "frontier signals relevant to FH skills/structure" | "...relevant to **{project}'s domain/stack**" |
|
|
241
|
+
| Durable daily store | `🛰️ Frontier Digest Log` issue in `forge-harness` | same pattern, an issue in the **project's own repo** |
|
|
242
|
+
| Weekly verification gate | **FH 4-axis gate** (hub-internal hook — hard-codes hub paths/markers) | **the project's OWN gate** — NOT the FH hook (per `auto_project_mapping.md §6`, the FH gate is deliberately not installed into projects; it would block the project's commits). A project supplies its own: its test suite + lint + whatever quality gate it trusts |
|
|
243
|
+
| HITL spine | draft PR → operator merge | **identical** — draft PR → owner merge (never auto-merge) |
|
|
244
|
+
| Branch | `claude/frontier-auto-*` | **identical** — `claude/`-prefixed |
|
|
245
|
+
|
|
246
|
+
**The invariant (do not change across projects)**: daily = cheap signal accumulation to a GitHub
|
|
247
|
+
issue; weekly = at most one scoped proposal as a **draft PR**, gated by *whatever verification that
|
|
248
|
+
project trusts*, merged by a human. Substitute the lens + the gate; keep the ephemeral-safe GitHub
|
|
249
|
+
store and the draft-PR HITL.
|
|
250
|
+
|
|
251
|
+
**For a field project**: the weekly proposer's gate is the project's own — run the project's test
|
|
252
|
+
suite and its own quality gate instead of the FH marker/hook, and the model floor concern (§4) does
|
|
253
|
+
**not** apply (no FH marker = no `floor-status` cross-check), so a field project's weekly routine can
|
|
254
|
+
run on sonnet. Open the draft PR against the project repo's `claude/` branch.
|
|
255
|
+
|
|
256
|
+
**Propagation path**: this pattern can ride **Full-Harness Mode** (`auto_project_mapping.md §6`) as an
|
|
257
|
+
opt-in project-local asset — offered, never auto-installed, owner approves each routine. It is a
|
|
258
|
+
*recommendation surface*, not a daemon dropped into the project.
|
|
259
|
+
|
|
260
|
+
## 9. Honest limits
|
|
261
|
+
|
|
262
|
+
- **Research preview**: Routines' limits/API may change; GitHub webhook + daily caps apply.
|
|
263
|
+
- **Salience-dependent**: these prompts are prose the routine session must follow; on a weaker model
|
|
264
|
+
the gate/draft-only discipline can slip. Mitigation: the prompts restate the hard guards inline, the
|
|
265
|
+
weekly arm is opus-pinned (§4), and the `claude/`-branch-only default makes an accidental
|
|
266
|
+
protected-branch write impossible **while that restriction stays on** — do not disable it (§3).
|
|
267
|
+
- **innovator is v0.2** (no pilot data). This loop is the mechanism that *produces* that data via the
|
|
268
|
+
draft-PR accept/reject record — treat early proposals as candidates, not authority.
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: measurement
|
|
3
|
+
date: 2026-05-26
|
|
4
|
+
refs:
|
|
5
|
+
- "arxiv 2605.26112 — From Model Scaling to System Scaling"
|
|
6
|
+
- "plugins/fh-meta/skills/harvest-loop/SKILL.md"
|
|
7
|
+
- "plugins/fh-meta/skills/agent-composer/SKILL.md"
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# FH Skill Quality Rubric
|
|
11
|
+
|
|
12
|
+
> Skill maturity score formula definition file.
|
|
13
|
+
> Declaring verifiable / evolution numbers without this file violates the cold audit "self-declaration = delete if no basis" rule.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## Verifiable Axis Formula
|
|
18
|
+
|
|
19
|
+
### Definition
|
|
20
|
+
"The ratio at which the result of executing a skill can be independently reproduced and verified by an external observer"
|
|
21
|
+
|
|
22
|
+
### Measurement targets (5 core skills — fixed denominator)
|
|
23
|
+
harness-doctor · verify-bidirectional · hub-cc-pr-reviewer · context-doctor · sim-conductor
|
|
24
|
+
|
|
25
|
+
### Per-skill scoring criteria
|
|
26
|
+
|
|
27
|
+
| Condition | Score |
|
|
28
|
+
|---|---|
|
|
29
|
+
| Done When item exists + harvest-loop Step 3.75 Critic isolation judgment linkage stated | 1.0 |
|
|
30
|
+
| Done When item exists + external observer reproducible format (Critic linkage not stated) | 0.75 |
|
|
31
|
+
| Done When item exists + internal self-declaration format only | 0.5 |
|
|
32
|
+
| No Done When | 0.0 |
|
|
33
|
+
|
|
34
|
+
### Measurement history
|
|
35
|
+
|
|
36
|
+
| Date | harness-doctor | verify-bidi | hub-cc-pr | context-doctor | sim-conductor | verifiable% | Change content |
|
|
37
|
+
|---|---|---|---|---|---|---|---|
|
|
38
|
+
| 2026-05-26 (pre-implementation) | 0.75 | 0.75 | 0.75 | 0.75 | 0.75 | 75% | Baseline set |
|
|
39
|
+
| **2026-05-26 (post-implementation)** | **0.85** | **0.85** | **0.85** | **0.85** | **0.85** | **85%** | External verification path + Critic linkage stated complete |
|
|
40
|
+
|
|
41
|
+
**Current verifiable% = 85%** ✅ (target 80%+ achieved)
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Evolution Axis Formula
|
|
46
|
+
|
|
47
|
+
### Definition
|
|
48
|
+
"The ratio of the harness's capacity to observe real usage data and automatically update parameters"
|
|
49
|
+
|
|
50
|
+
### 5 core element checklist (20% each)
|
|
51
|
+
|
|
52
|
+
| Element | Status | Basis |
|
|
53
|
+
|---|---|---|
|
|
54
|
+
| harvest-loop pipeline exists | ✅ | 8-step pipeline complete |
|
|
55
|
+
| Harness Evolution Cadence 4-week cycle | ✅ | Step 6-b newly added (2026-05-26 `f13d3a4`) |
|
|
56
|
+
| fh_signal accumulation mechanism | ✅ | `tracks/_meta/fh_signal_*.md` |
|
|
57
|
+
| Real usage data auto-aggregation → Leaderboard update | ✅ | Step 6-b fh_signal grep executed / skill_usage.md updated (2026-05-26) |
|
|
58
|
+
| escalate_when auto-update loop connection | ❌ | Candidate output then human approval only, no data feedback |
|
|
59
|
+
|
|
60
|
+
**evolution% history**:
|
|
61
|
+
- Pre-implementation: 60% (3/5)
|
|
62
|
+
- After Step 6-b mechanism definition: 70% (3.5/5)
|
|
63
|
+
- **fh_signal auto-aggregation + skill_usage.md update complete: 80%** ✅ (4/5)
|
|
64
|
+
|
|
65
|
+
---
|
|
66
|
+
|
|
67
|
+
## Update Rules
|
|
68
|
+
|
|
69
|
+
- Recalculate denominator (5 core skills) when skills are added or removed
|
|
70
|
+
- Re-measure once per quarter — auto-proposed at harvest-loop forced mode session end
|
|
71
|
+
- Measurement executor: runs as part of harvest-loop Step 6-b Cadence execution
|