@chrono-meta/fh-gate 1.4.72 → 1.4.74

package/knowledge/patterns/ensemble_union_detection_task_pattern.md

@@ -0,0 +1,125 @@
+---
+name: ensemble-union-detection-task
+description: "Detection tasks benefit from ensemble UNION (2.5–3.0x coverage lift); generation tasks do not (voting, ~0x). Measured, domain-agnostic."
+type: pattern
+status: VALIDATED
+tags: [ensemble, union, detection, decorrelation, composition-over-scale]
+---
+
+# Ensemble Union Pattern for Detection Tasks
+
+> **Measurement basis**: a PRD-review detection protocol (n=3 real product PRDs) + a generation-task
+> contrast, measured on a fixed open-weight 3-model roster. Detection lift +11.3 findings (2.8x; per-PRD
+> range 2.5x–3.0x); generation lift ~0.
+
+## Pattern
+
+**Detection tasks benefit from an ensemble UNION strategy, not voting.**
+
+- **Detection** (finding defects, gaps, issues): union of all model findings → **2.5x–3.0x** more coverage.
+- **Generation** (creating one artifact): voting for consensus → **~0x lift** (no benefit).
+
+## Mechanism
+
+Each model has **different blind spots**. Model A finds category-X defects, B finds Y, C finds Z.
+`Union(A, B, C)` = superset of all findings → coverage expansion. Voting would **discard minority
+findings**, losing exactly the diversity that makes the panel worth running. (This is the
+composition-over-scale lever: the gain is model *diversity*, not model size — and it holds only while the
+errors are decorrelated. A shared/correlated blind spot is not closed by union; diagnose correlated vs
+independent first.)
+
+## Measured Results
+
+Anchored on a PRD-review detection protocol (3 real product PRDs; 3-model open-weight union vs each single
+model):
+
+| Metric | Union (E) | Best single | Lift |
+|---|:--:|:--:|:--:|
+| Mean findings / PRD | 18.0 | 6.7 | **+11.3 (2.8x)** |
+
+Per-PRD lift ranged **+10.7 to +12.0 (2.5x–3.0x)** — union was 18.0 on every PRD while the best single
+model never exceeded 7.3, because each model surfaced *different* findings. A generation-task contrast
+(create one artifact) showed **~0 lift** from the same union: consensus/voting adds nothing when the goal
+is one output rather than coverage. (The detection→UNION / generation→VOTING split is the validated
+pattern; the +11.3 detection anchor is the measured number, the generation-0 is the qualitative contrast.)
+
+**Arms** (illustrative open-weight roster): three distinct open-weight models spanning ≥2 model families
+(e.g. `qwen3.5-122b`, `qwen3-next-80b`, `gpt-oss-120b`). Family diversity is the point — a same-family
+trio shares blind spots. `E` = union(the three).
+
+## Implementation
+
+```python
+# Detection task → UNION
+result = ensemble_request(
+    prompt="Review this document and classify findings into bins A/B/C/D",
+    models=[MODEL_A, MODEL_B, MODEL_C],   # ≥2 families
+    strategy="UNION",
+    temperature=0.0,
+)
+# result.findings = all unique findings across models
+```
+
+### Deduplication logic
+
+```python
+def compute_union(responses):
+    """Union of findings across models, deduplicated by normalized description."""
+    seen, union = set(), []
+    for response in responses:
+        for finding in extract_findings(response):
+            key = finding.get("description", "")[:50].lower()
+            if key and key not in seen:
+                seen.add(key)
+                union.append(finding)
+    return {"findings": union, "total": len(union)}
+```
+
+## Token economics
+
+Detection is where a cheap local/open-weight union beats a single frontier call on **both** axes —
+more findings **and** lower cost — because union trades false-positive rate (filtered downstream) for
+coverage:
+
+| Option | Relative cost | Findings/item | Note |
+|---|:--:|:--:|---|
+| Frontier alone | baseline | 6.7 | single strong model |
+| Open-weight union (3-model) | ~1/19 | 18.0 | ~19x cheaper + ~2.8x more findings |
+
+## Constraints
+
+- **False positives**: union includes *all* findings → higher FP rate. Mitigate with a frontier filter
+  (union output → one strong-model validation pass), expert review of high-priority findings only, and a
+  downstream precision measurement against ground truth.
+- **Decorrelation precondition**: union only helps when the panel's errors are *independent*. If a blind
+  spot is shared (correlated), stacking more models does not close it — escalate to a human/ground anchor
+  instead. Diagnose correlated-vs-independent (reps ≥ 3) before prescribing union.
+
+## When to apply
+
+| Task type | Examples | Strategy |
+|---|---|:--:|
+| **Detection** | Bug finding · document review · audit · red-team · structure/traceability check | **UNION** |
+| **Generation** | Artifact creation · synthesis · single-answer tasks | **VOTING** |
+
+**Rule of thumb**: "find all X" → UNION. "create one Y" → VOTING.
+
+## Cross-domain applicability
+
+Domain-agnostic — applies to any detection task where (1) multiple independent judges have different
+blind spots, (2) the goal is coverage maximization not consensus, (3) false positives can be filtered
+downstream. Examples beyond QA: code review (security · performance · maintainability), multi-specialist
+diagnosis, content moderation (hate speech · spam · misinformation), fraud detection (multiple risk
+models).
+
+## Done When
+
+- [x] Validated on a PRD-review detection protocol (+11.3, 2.8x; per-PRD +10.7 to +12.0) + a generation contrast (~0 lift). *[measured]*
+- [x] Deduplication logic specified. *[mandatory-pass]*
+- [x] Decorrelation precondition + FP mitigation documented. *[judged — pair: correlated-blindspot diagnosis]*
+- [ ] (Optional) Precision/FP-rate measurement against ground truth.
+
+---
+
+**One-line takeaway**: Detection → ensemble UNION (2.5–3.0x lift). Generation → voting (~0x). The lever is
+family diversity, not model size, and only while errors stay decorrelated.

package/knowledge/shared/GLOSSARY.md

@@ -0,0 +1,77 @@
+# GLOSSARY — forge-harness Key Term Definitions
+
+> One-line definitions of FH-specific vocabulary appearing in skills, agents, and documentation.
+> Reference point for new user onboarding. Keep in sync with the README key terms table.
+
+---
+
+## Hub Structure
+
+| Term | Definition |
+|---|---|
+| **Meta-Harness** | A persistent hub in a Claude Code environment that connects the work, learnings, and patterns of N projects for mutual reinforcement. Not a simple storage — a connection layer through which knowledge flows between projects. |
+| **Meta Hub** | The role of coordinating all field projects from the meta-harness cwd. Hub = common standards and feedback center / field projects = execution sites. |
+| **Launch Pad Effect** | Using the meta-harness not as a final destination but as a launch pad — even a brief pass-through generates setup, pattern sharing, and speed-up effects. |
+| **Transit Acceleration Value** | The meta-harness's core value — passing through itself accelerates the starting line. Acceleration effect occurs the moment you pass through, without requiring absorption or permanent setup. |
+| **Shared Skill Pool** | Removes the cost of each team/harness independently reinventing the same skills/agents. The meta-harness provides a common pool and each project draws from it. |
+
+---
+
+## Operating Modes
+
+| Term | Definition |
+|---|---|
+| **Mode A** | Full harness clone + use all skills/agents. Run directly from hub cwd. |
+| **Mode B** | Partial harness clone or fork. Select and use specific skills/agents only. |
+| **Mode C** | Install only fh-meta via `claude plugin install` without cloning hub. Independent plugin method. |
+| **Path B generalization** | Generalizing skill behavior to work in external user environments without organization-specific infrastructure dependencies. |
+
+---
+
+## Diagnostic Skill Triangle
+
+| Term | Definition |
+|---|---|
+| **Three-Doctor Loop** | Pattern where harness-doctor (structure diagnosis) + context-doctor (token/context diagnosis) + sim-conductor (simulation/ideation scan) form a self-renewing closed loop of diagnosis→prescription→re-diagnosis. |
+| **harness-doctor** | Harness structure L1~L4 diagnostic skill. L1 structural completeness · L2 complexity · L3 Drift · L4 connection diagnosis. |
+| **context-doctor** | Token waste diagnostic skill. `.claudeignore` auto-generation, large file detection, `/clear` timing guidance. |
+| **sim-conductor** | Meta-simulation automation skill. Area A (external user) · B (internal audit) · C (ideation scan) · D (code/session/skill/memory verification) · E (quality examination). |
+| **install-doctor** | Plugin install pre/post conflict, duplicate, and silent overwrite risk diagnostic skill. |
+
+---
+
+## Prescription Tiers
+
+| Term | Definition |
+|---|---|
+| **M-tier (Mandatory)** | Requires immediate action. Risk of functional failure or data loss if left unaddressed. |
+| **S-tier (Strongly recommended)** | Strongly recommended improvement. Quality degradation and drift accumulation if left unaddressed. |
+| **R-tier (Recommended)** | Recommended optimization item. Efficiency improvement when resolved. |
+
+---
+
+## Design Principles
+
+| Term | Definition |
+|---|---|
+| **Simplification Guard** | Mandatory matching of existing assets before adding new ones. Additions rejected without "N+ real-use observations". The execution mechanism of the "a good harness gets simpler over time" principle. |
+| **Description diet** | Removing self-marketing vocabulary (iteration counts, version history, emphasis words, owner names) from skill frontmatter descriptions to make them readable by external users. |
+| **Layer A auto-read** | The 4 files CLAUDE.md automatically reads at session start (CATALOG.md · latest track file · MEMORY.md · next session starter card). Only works in meta-harness cwd. |
+| **Layer A fallback** | Alternate path when Layer A silent-skips in non-meta-harness cwd environments. Manual CATALOG.md read or adding Layer A reference to project CLAUDE.md. |
+| **silent overwrite** | Risk of overwriting existing settings without user awareness. Detected in advance by install-doctor. |
+| **drift** | Phenomenon of growing gap between design intent and actual behavior. Checked periodically as harness-doctor L3 item. |
+
+---
+
+## Evolution Concepts
+
+| Term | Definition |
+|---|---|
+| **cascade α** | Stage where FH skills are first autonomously executed by internal users (including owner). |
+| **cascade β** | Stage where FH skills are autonomously executed by users other than the owner (quasi-external). First achieved by an external user. |
+| **cross-project skill bus** | Structure for centrally managing skills/agents of local projects through FH and enabling cross-project cross-calling. |
+| **field harvest** | Process of feeding patterns discovered in field project work back (pull) to FH. Automated with `/field-harvest` skill. |
+
+---
+
+*Updated: 2026-05-26*

package/knowledge/shared/harness-core/measurement-integrity-checklist.md

@@ -120,6 +120,8 @@ here.** Three shapes, all observed 2026-07-20 in a single session:
 | n+7 | **Instrument sees only part of its own declared surface** | An "always-loaded footprint" scan summed files rooted at `$TARGET`, silently omitting the auto-loaded memory index living outside it — **61% of the real resident surface** | A known-positive (a file you *know* is resident) fails to appear in the sum |
 | n+8 | **A cheap proxy substituted for the real property** | Index-line/topic-file **size ratio** used as a proxy for *content coverage*; minimum ratio 3.7× read as "safe" — while an entry whose file was 3.7× larger still lacked every fact the index carried | One known case checked by content, not size, inverts the verdict immediately |
 | n+9 | **Language / encoding assumption mismatch** | An **ASCII-token scanner run over a Korean corpus**: the index wrote `catch`, `MERGED`, `expert-system`; the files wrote `잡았다`, `머지`, `케이스크래프트` → every token scored as missing. **~96% false positives** | One known-negative (an entry you know is fully covered) scores as "missing" → mismatch exposed |
+| n+10 | **Accepted into the scanned set, but no probe matches that shape** — the instrument counts the file as covered and reports it *clean* | `degrade_direction_scan.sh` collected `.sh` files while every probe was Python-shaped (`except:` / `.get(k, True)` / `if not x:`). A known-positive bash file with four default-toward-PASS constructs scored **0/4**, and the output read `no default-toward-PASS smells in 1 scanned py/sh file` (2026-07-28) | The known-positive of *that shape* scores 0 while the summary says "scanned" — the tell is coverage claimed without detection demonstrated |
+| n+11 | **The collection predicate, not the probe, is what silently drops the target** | Same scan: git hooks are named `pre-push` (no extension) under `.git-hooks` (dotted directory), and the extension test ran against the **full path**, so FH's own mechanical floor reported `no scannable (py/sh) target files`, exit 0 | Point the instrument at a directory you *know* holds a positive; an empty file list is the finding, not a clean result |
 
 Secondary false-positive sources in the same run, worth checking explicitly: **whitespace/hyphen
 variants** (`3주새` vs `3주 새`), and treating a line's **navigational annotation** (`(detail …, archive)`)
@@ -134,6 +136,14 @@ as a factual claim.
   `UNMEASURED`, never `0`.
 - An instrument that produces an **impossible value** (all-pass, all-fail, or a self-scan in which the
   running tool does not detect itself) is suspect **before** its target is. Suspect the instrument first.
+- **"Scanned" is a claim, and it is separate from "covered."** A scanner that admits a file type into its
+  scanned set owes a known-positive *of that type*; without one, its clean message is a false clean, which
+  is strictly worse than an honest "not covered" (n+10). Same for the collection step: an empty file list
+  is a calibration failure to report, never a clean run (n+11).
+- **A probe that flags the prescribed remedy is a defect in the probe.** Measured 2026-07-28: flagging
+  `${count:-0}` — the integer sanitization that closes the `pipefail`-fallback class — would push an author
+  to delete the fix. Hand-check a sample of hits before shipping a probe; on this corpus 6/6 sampled hits
+  were false positives and the scoping that followed cut the load from 73 to 14.
 
 ### Done When