@chrono-meta/fh-gate 1.4.41 → 1.4.43

package/knowledge/shared/harness-core/hub_maturity_roadmap.md

@@ -0,0 +1,201 @@
+---
+name: Hub Maturity 3-Phase Roadmap Frame (template)
+description: Long-term evolution path frame for the hub. Phase I (entering maturity) → Phase II (frontier following) → Phase III (frontier leading) 3-stage model. Fixes the gap, output, completion criteria, and transition conditions for each phase as a quarterly re-diagnosis reference document. The maturity axis is the parent frame referenced by monthly level snapshots and quarterly re-diagnosis. **This file is a template — in actual hub operation, add project-specific information (timing, asset names, identifiers) to write an operating copy**.
+type: reference
+date: 2026-04-28
+tags: [harness, maturity, roadmap, 3-phase, frontier-tracking, evolution, compounding, long-term, hub, strategic, template-frame]
+scope: hub-template
+---
+
+# Hub Maturity 3-Phase Roadmap (frame)
+
+## Why this document exists
+
+If the monthly level snapshot captures **"where are we now"**, this document captures **"where are we going and when do we transition"**. If the quarterly re-diagnosis captures **current position vs industry frontier**, this document captures **the evolution of the relationship with the frontier itself** (follower → leader).
+
+**Core vision**:
+
+> The hub model matures enough to **enter the maturity phase**, **follow** the frontier's advancement direction periodically, and after that, the path where **we become the frontier**.
+
+This vision is made explicit as 3 phases so that at each quarterly re-diagnosis, "which phase are we in · what are the conditions for the next phase transition" can be judged.
+
+---
+
+## 1. 3-Phase Overview
+
+| Phase | Scope | Core gap (example) | Representative output (example) | Standard duration (example) |
+|---|---|---|---|---|
+| **I. Entering maturity** | Infrastructure/asset building + routine establishment | Operational gap axes remain · automation not established · 0 external propagation | Weekly audit automation · operations guide · 1-2 external assets · sub-agent judgment · self-diagnosis warning reduction | 3-6 months |
+| **II. Frontier following** | Quarterly re-diagnosis routine + gap auto-detection | Single/dual input sources · no auto gap detection · cadence not confirmed | Quarterly frontier diagnosis 2 times + monthly brief established + external scan cadence confirmed | 5-8 months |
+| **III. Frontier leading** | Self-invented outbound propagation + self-evolving | 0 open-source/presentation record · no self-evolving loop · 0 industry citations | Public refactor · blog/presentation 10+ per year · industry citation case accumulation | Ongoing (no completion) |
+
+---
+
+## 2. Current Position (fill in operating copy)
+
+Fill in the following format in the operating copy:
+
+```
+**Phase X · ~N% progress** — infrastructure/asset accumulation status · N remaining completion gaps
+```
+
+### Achievement table (example format)
+
+| Area | Status | Basis |
+|---|---|---|
+| 6-axis framework + feedback loop | ✅ | (corresponding asset path) |
+| Package structure alignment | ✅ | (realignment session) |
+| ... | ... | ... |
+
+### Gaps remaining
+
+See §3 Phase I completion criteria.
+
+---
+
+## 3. Phase I Completion Criteria (5 measurable criteria)
+
+Phase II entry gate passed when all 5 are met.
+
+| # | Condition | Measurement method (frame) |
+|---|---|---|
+| 1 | **Weekly audit automation established** | audit skill run 3+ times + manual N min → auto N min measured |
+| 2 | **Axis N operations guide established** | orchestration guide draft + mode switch 3+ cases accumulated (if gap axis exists) |
+| 3 | **External propagation N cases** | Distributed to external channel + external response received (generally 2 cases recommended) |
+| 4 | **Sub-agent pilot promotion/deprecation judgment** | 2+ week observation + invocation log-based judgment (`accepted ≥ 60%` / `rejected ≥ 40%` / `invocation count ≥ N`) |
+| 5 | **Self-diagnosis warning reduction** | Quarterly self-diagnosis warnings N items → 1 or fewer |
+
+### Completion checklist derived Decision
+
+- Phase I completion date = earliest date all 5 criteria are met
+- Completion confirmation event = **Phase II entry meeting** (1 separate session, §4 entry condition check)
+- No Phase II output (e.g., external GitHub scan) before Phase I completion — simplification principle violation
+
+---
+
+## 4. Phase II (Frontier Following) — Entry conditions, cadence, outputs
+
+### 4.1 Entry conditions
+
+- Phase I completion 5 criteria **all** met
+- Monthly level snapshot updated 2+ consecutive times
+- §2 current position re-judged as "Phase II · 0%"
+
+### 4.2 Cadence 3-option comparison
+
+| Option | Cycle | Pros | Cons |
+|---|---|---|---|
+| **(a) Quarterly only** | 3 months | Simple | Gap detection delayed 3 months (slow if frontier moves fast) |
+| **(b) Quarterly + monthly light scan** | 3 months + 4 weeks | Gap detection within 1 month + 10 min addition to existing monthly routine (minimum invasive) | (None — recommended) |
+| (c) Trigger-based | When stagnation detected | Resource efficient | Stagnation detection criteria + auto-alerts + trigger tags all require new infra. **Simplification principle violation risk** |
+
+→ **Recommended: (b) quarterly + monthly**. Joining existing monthly routine = minimum invasive. Consistent with simplification principle.
+
+### 4.3 Representative outputs (frame)
+
+| Output | Cycle | Content |
+|---|---|---|
+| **Quarterly frontier diagnosis #N** | 3 months | 6-axis level change vs previous edition + 3-5 new frontier techniques integrated |
+| **Monthly brief** | 4 weeks | 10-min scan of GitHub trending, Anthropic/OpenAI blog, and similar — 3-5 line summary |
+| **External GitHub scan actual operation** | Monthly | Monthly 10-min scan of public repos. Sprint Contract in 5 lines |
+| **Auto gap detection** | Scanner feature addition | Flag "frontier diagnosis not updated > 90 days" |
+
+### 4.4 Completion conditions
+
+- Quarterly re-diagnosis 2 times + monthly brief 6+ consecutive productions
+- 1+ time **own methodology back-referenced from frontier** found
+- 2+ axes in monthly level snapshot sustained as "leading" judgment
+
+---
+
+## 5. Phase III (Frontier Leading) — Entry conditions, 6 indicators, outputs
+
+### 5.1 Entry conditions
+
+- Phase II completion 3 conditions all met
+- 3+ of N self-invented assets **observed as original concepts** for similar industry concepts
+- Public seed repository external contribution record started (at least one of fork/issue/star)
+
+### 5.2 6 Leading indicators
+
+| # | Indicator | Measurement |
+|---|---|---|
+| 1 | Public seed repository record | star/fork/issue count |
+| 2 | Blog/presentation | 2-3 per quarter · 10+ per year |
+| 3 | Industry citation cases | External articles/seminars citing this hub/methodology |
+| 4 | External organization adoption | Other companies/departments |
+| 5 | Self-evolving loop demonstration | Skill generates skills |
+| 6 | Self-invented industry original recognition | Adopted as industry term/frame |
+
+### 5.3 Representative outputs
+
+- **Public seed repository refactor** — from personal seed to collaboratable template. Team customization + common protocol separation
+- **10+ blog/presentations per year** — externalize 1 self-invented asset per quarter
+- **Self-evolving loop MVP** — audit skill proposes own skill generation + user approval → auto-generate → usage observation → deprecate/improve
+
+### 5.4 Phase III has no "completion"
+
+Phase III is an ongoing state. Instead of completion criteria, **3 maintenance conditions**:
+- 3+ of 6 indicators continuously rising
+- "Frontier level maintained" judgment in quarterly re-diagnosis (no regression)
+- Self-diagnosis 8-item checklist failure signal 0 maintained
+
+---
+
+## 6. Common principles for transitions
+
+### 6.1 Consistent simplification principle
+
+Phase transitions are **methodology/abstraction level rises**, not **increases in file/skill/rule count**. At each phase transition, do the following first:
+
+- [ ] Self-diagnosis 8-item checklist check — confirm 0 failure signals
+- [ ] Files unreferenced 6+ months → move to `archive/` or consolidate
+- [ ] Clean up previous phase temporary outputs made unnecessary by new phase entry
+- [ ] Confirm CLAUDE.md and CATALOG within 200 lines
+- [ ] Re-confirm optimization principle: field harness → simpler over time; meta-harness → complexity earns its scope (purge orphaned/redundant/decorative units)
+
+### 6.2 Transition deferred on simplification failure
+
+Above checklist not passed → **phase transition deferred**. Perform previous phase remaining work + simplification work for 1 additional month then re-check.
+
+### 6.3 Phase regression possible
+
+**Regression** from Phase II to Phase I also allowed. E.g., frontier following routine missed 2 consecutive times → "manual re-establishment" re-perform part of Phase I. Do not force linear progression.
+
+---
+
+## 7. This roadmap update cycle
+
+| Trigger | Update content |
+|---|---|
+| Quarterly re-diagnosis | §2 current position re-judgment + §3·§4·§5 criteria change check |
+| Phase transition event | Record corresponding phase completion + initialize next phase progress |
+| Self-invented asset recognized as industry original | §5.1 entry condition counter update |
+| Simplification principle violation detected | §6 transition deferral activation record |
+
+---
+
+## 8. Operating copy writing guide (for template users)
+
+When moving this frame to an operating copy:
+
+1. **Fill §2 current position** — add asset list and basis material paths
+2. **Concretize §3 5 criteria** — add `current` column + `target date` column (e.g., `2026-05-17`)
+3. **Recommend §4.2 cadence (b)** — if adopting other option, need to prove §6.1 simplification gate passed
+4. **Add current counter to §5 6 indicators** — starting from all 0 is natural
+5. **Update §2 at each quarterly re-diagnosis** + record cumulative changes in §7 trigger table
+
+---
+
+## 9. Related assets
+
+- 6-axis framework — `knowledge/shared/harness-core/harness_6axis_framework.md` (frame premise)
+- Monthly level snapshot — `knowledge/shared/harness-core/harness_level_snapshot_*.md` (current position)
+- Quarterly re-diagnosis — `knowledge/shared/harness-core/harness_frontier_diagnosis_*.md` (following basis)
+- Feedback automation — `knowledge/shared/harness-core/hub_compounding_loop.md` (Phase transition gate input)
+
+---
+
+## 10. One-line conclusion
+
+**Phase I completion → Phase II frontier following (b)cadence → Phase III leading. Simplification principle is the common gate for each transition**.

package/knowledge/shared/harness-core/hybrid_orchestration_architecture_roadmap.md

@@ -0,0 +1,196 @@
+---
+name: hybrid-orchestration-architecture-roadmap
+description: Proposed (not-yet-implemented) architecture roadmap for fh as an intelligent hybrid orchestration engine — Claude Code as main driver, other CLIs/APIs as runtime-discovered sidecars, with Zero-Config standalone fallback for plugin-only users. Design intent + reconciliation against existing FH assets.
+type: roadmap
+date: 2026-06-09
+status: proposed (design intent — NOT implemented; see §Status & Reconciliation)
+tags: [hybrid-orchestration, sidecar, zero-config, jit-probing, install-wizard, roadmap, proposed]
+---
+
+# Hybrid Orchestration Architecture — Roadmap (Proposed)
+
+> **Status: PROPOSED design intent, not current behavior.** This document captures a
+> forward-looking architecture for fh. Several components it describes — a `config.json`
+> engine-topology file, runtime JIT engine probing, an install-wizard that auto-builds an
+> engine map — **do not exist in FH today**. Read §Status & Reconciliation first to see
+> what is already shipped vs what is aspirational. Nothing here should be cited as a
+> current feature.
+>
+> **Source**: operator design doc (2026-06-09), reconciled to FH conventions on ingest —
+> pinned model versions → `{model-name}` placeholders (per `multi_model_sidecar_strategy.md`
+> §Generalization #4); illustrative pseudo-code marked as such (FH's real layer is markdown
+> methodology + Claude-native automation, not a Python runtime engine).
+
+## 1. Overview
+
+fh aims to be a **hybrid orchestration** harness: **Claude Code as the main driver (core
+controller)**, with third-party CLIs / APIs mapped in as **sidecars** according to whatever
+the user's local environment makes available. The goal is to escape single-model dependency
+and **progressively enhance** output by discovering the user's own resources (subscription
+CLIs, API keys) at runtime — while still working in a fully **Zero-Config** state for users
+who copy only a single plugin/skill rather than installing the whole framework.
+
+This is the design north star. The mechanism for *delegating to sidecars* is already
+validated and shipped (see `multi_model_sidecar_strategy.md`); the *automatic
+discovery/topology* layer below is the proposed addition.
+
+## 2. Model roles (provider-agnostic)
+
+FH convention forbids pinning version numbers or benchmark figures into durable assets
+(they age and become phantom claims), so roles are stated by function, not by pinned model:
+
+| Role | Engine slot | Function | Maps to FH skill family |
+|---|---|---|---|
+| **Core Driver** | `{primary-model}` (strongest available; CC host) | Full code edit, architecture, final synthesis | steel-quench, refactor, design consensus |
+| **Context/Analysis Sidecar** | `{large-context-sidecar}` | Bulk log/API-spec parsing → distilled clues | harvest-loop bulk scan |
+| **Test/Util Sidecar** | `{fast-util-sidecar}` | Boilerplate, unit tests, repetitive drops | speed-run / scaffolding |
+
+> Capability/benchmark comparisons between providers change every release cycle — consult a
+> live source (`/frontier-digest`, or the `claude-api` skill for Claude specifics) at decision
+> time rather than trusting any number frozen into this doc. No SWE-bench figures are recorded
+> here by design.
+
+## 3. Main driver + sidecar dual channel
+
+```
+                 ┌─────────────────────────────┐
+                 │      user terminal (fh)      │
+                 └──────────────┬──────────────┘
+                                │   [intelligent routing]
+            ┌───────────────────┼───────────────────┐
+            ▼ (bulk / low-cost)  ▼ (final reasoning)  ▼ (fast unit-test / repeat)
+   ┌─────────────────┐  ┌──────────────────┐  ┌─────────────────┐
+   │ context sidecar │  │ Claude Code (main)│  │  util sidecar   │
+   └────────┬────────┘  └────────┬─────────┘  └────────┬────────┘
+            └────────────────────┼─────────────────────┘
+                                 ▼
+                   [final quality verification + apply]
+```
+
+- **Claude Code (main)** keeps full project context/architecture and is the only writer of
+  final code. This matches FH's existing rule: *"Host is always single"*
+  (`multi_model_sidecar_strategy.md` §Mechanism).
+- **Sidecars** are stateless one-shot `Bash`-invoked processes whose stdout is folded back in
+  by the calling skill — **already the shipped mechanism**, not new.
+
+## 4. 3-Tier routing protocol (proposed ordering)
+
+When a skill needs a sidecar engine, probe in priority order:
+
+```
+[sidecar request]
+   ├── Tier 1: subscription CLI (zero marginal cost)
+   │            discover logged-in binaries: `claude -p`, `aider`, `gemini`, `codex`, `gh copilot`
+   ├── Tier 2: native API call (pay-per-use)
+   │            env keys: GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY
+   └── Tier 3: self-contained sub-agent split (fallback)
+                no external sidecar → Claude Code spawns isolated sub-agent / prompt-chunking
+```
+
+> **Relation to the shipped fallback chain**: `multi_model_sidecar_strategy.md`
+> §Implementation-Patterns already defines a 3-tier *fallback* chain (Copilot CLI → corporate
+> endpoint → direct Gemini/Codex). The ordering here is framed by **cost/access tier** rather
+> than network reachability. These are two lenses on the same fan-out; if implemented, they
+> should be unified into one routing table, not maintained as two competing lists. Tier 3
+> ("Claude internal sub-agent fallback") is the genuinely new contribution — it guarantees the
+> chain never hard-fails even with zero external resources.
+
+## 5. Static inheritance + dynamic JIT fallback (proposed)
+
+```
+execution request (skill / agent)
+        │
+        ▼
+[config.json present?]   ← PROPOSED file; does NOT exist in FH today
+        ├── Yes → static-inheritance mode → load priority map → run immediately
+        └── No  → Zero-Config standalone mode → runtime JIT probing:
+                     1. local subscription CLI scan   (Tier 1)
+                     2. system API-key env scan        (Tier 2)
+                     3. main-driver sub-agent fallback (Tier 3)
+```
+
+**5.1 Static config inheritance** — if an install-wizard run had persisted an engine map,
+skills would adopt it as default and skip per-call scan overhead. *(Proposed: install-wizard
+does not generate such a file today.)*
+
+**5.2 Runtime JIT probing** — if no config exists (or a tool was added after install), the
+harness probes the live environment on-the-fly via the Tier 1→2→3 protocol and binds the best
+path. All probes failing → safe descent to Tier 3 (main model handles it).
+
+**5.3 Zero-Config standalone self-reliance** — the key requirement for **Mode C** (plugin/skill
+copied without the full framework, see `.claude/rules/modes_and_value.md`). With no
+`config.json`, the module must **not error** — it switches to Zero-Config standalone mode and
+JIT-probes for whatever local tools exist, then proceeds quietly. Falls back to Tier 3 if none.
+
+```python
+# ILLUSTRATIVE resolution logic — NOT shipped code.
+# FH's actual layer is markdown methodology + Claude-native automation (skills/rules/hooks),
+# not a Python engine. This sketch only shows the intended decision order.
+def resolve_sidecar_engine(preferred="{util-sidecar}"):
+    cfg = load_static_config_safe()              # Step 1: static config (if any)
+    if cfg and cfg.get("mapped_engine"):
+        return cfg["mapped_engine"]
+    if shutil.which("aider") or shutil.which("{cli}"):   # Tier 1: subscription CLI
+        return "CLI_WRAPPER_MODE"
+    if os.environ.get("{PROVIDER}_API_KEY"):             # Tier 2: API key
+        return "NATIVE_API_MODE"
+    return "CLAUDE_SUBAGENT_FALLBACK"                     # Tier 3: peaceful fallback
+```
+
+## 6. Intelligent install-wizard (proposed extension)
+
+Today `install-wizard` sets up the periodic-audit notification structure (zshrc hook +
+sentinels + session-start mtime detection). It does **not** build an engine topology. The
+proposed extension would add, at install time:
+
+- **Binary probing** — scan `$PATH` for available CLIs/dependencies.
+- **Profile mapping** — write the discovered hybrid config (the proposed `config.json`).
+- **Sanity check** — a light sidecar call to confirm the pipeline actually works.
+
+Onboarding UX phrasing (proposed):
+- Minimal env → *"Configured a Claude-Code-only harness. All skills run safely via context
+  splitting, no API key needed."*
+- Expansion nudge → *"To enable cheaper large-analysis skills, register a sidecar API key later;
+  fh will auto-detect it on the next run and apply sidecar orchestration."*
+
+## 7. sim-conductor & core-skill improvement directions (proposed)
+
+- **7.1 `claude -p` non-interactive pipe runtime** — exploit CC's non-interactive flag; pipe
+  stdout/stdin between agents; reuse session cache via a harness wrapper.
+- **7.2 Hybrid context bridge** — normalize heterogeneous sidecar output (API JSON vs CLI
+  streamed text); embed an extractor (code-block / JSON-structure parser) in a bridge layer so
+  callers always get structured responses regardless of invocation path.
+- **7.3 Built-in sub-agent injection** — for minimal envs with no third-party CLI/API: place
+  worker prompt specs (e.g. `steel-quench-worker.md`, `harvest-loop-worker.md`) into
+  `.claude/agents/`, so the harness spawns Claude's own isolated-context sub-agent pool for
+  parallel work. *(Note: per `operations.md`, personal agents in shared repos should be kept
+  local via `.git/info/exclude` — an installer-placed agent must respect that boundary.)*
+
+---
+
+## Status & Reconciliation (read this before citing anything)
+
+| Component | In FH today? | Where / note |
+|---|---|---|
+| Sidecar delegation via `Bash` (stateless, host-single) | ✅ Shipped + validated | `multi_model_sidecar_strategy.md` (Experiment 1·2) |
+| Cross-provider perspective-diversity rationale | ✅ Shipped | same doc + `steel-quench` Wave 5 |
+| 3-tier *fallback* chain (network-reachability lens) | ✅ Shipped | same doc §Implementation-Patterns |
+| 3-tier *routing* by cost/access (Tier1 CLI→Tier2 API→Tier3 subagent) | 🟡 Partial | reorders the shipped chain; Tier-3 subagent fallback is new |
+| `config.json` engine-topology file | ❌ Proposed | does not exist |
+| Runtime JIT engine probing | ❌ Proposed | does not exist |
+| Zero-Config standalone auto-heal (Mode C) | ❌ Proposed | concept aligns with Mode C in `modes_and_value.md` |
+| install-wizard builds engine topology / sanity-check | ❌ Proposed | install-wizard today = zshrc hook + sentinels only |
+| Hybrid context bridge / built-in sub-agent injection | ❌ Proposed | sim-conductor roadmap item |
+
+**Conflicts resolved on ingest** (per FH conventions): pinned model versions + SWE-bench
+figures dropped → `{model-name}` placeholders (avoids phantom/stale claims); Python
+`engine_resolver.py` marked ILLUSTRATIVE (FH has no Python runtime layer); every non-shipped
+component tagged ❌ Proposed above so this roadmap can never be mistaken for current behavior.
+
+## References
+
+- `knowledge/shared/harness-core/multi_model_sidecar_strategy.md` — shipped sidecar mechanism + fallback chain (this roadmap extends, does not replace, it)
+- `plugins/fh-meta/skills/install-wizard/SKILL.md` — current install behavior (the topology extension would build on this)
+- `.claude/rules/modes_and_value.md` — Mode C (plugin/skill-only) that Zero-Config self-reliance targets
+- `.claude/rules/operations.md` — sub-agent boundary rules the built-in-injection item must respect
+- `plugins/fh-meta/skills/frontier-digest/SKILL.md` — live source for current model capability/benchmark comparison (do not freeze numbers here)

package/knowledge/shared/harness-core/live_surface_automation_pattern.md

@@ -0,0 +1,110 @@
+---
+name: live-surface-automation-pattern
+description: The capability pattern FH routes to when a mapping project needs an agent to drive a live UI surface (web/mobile) — observe-act-verify over the running app, not code/data fetch. FH routes drivers (no-reinvention); the value is the cross-platform observe-act-verify contract, the Appium-less principle, and the hybrid-WebView vision-synthesis rule. Generalizable across mapping projects; not tied to any one project.
+date: 2026-06-14
+tags: [live-surface, ui-automation, observe-act-verify, appium-less, hybrid-webview, no-reinvention, mapping-acceleration]
+---
+
+# Live-Surface Automation Pattern
+
+When a mapping project's work lives on a **live UI surface** — a running mobile app or web page whose
+state cannot be reached by reading code or fetching data — FH's posture is the same as for any
+capability it does not own: **detect the need and route to the best driver present, do not build a UI
+engine** (no-reinvention). The value FH adds is the *contract* below, not a new automation tool.
+
+This is the acceleration axis behind the "③ map-project acceleration" door's live-surface capability
+(`[[fh-live-surface-acceleration]]`). It was first validated on web (Playwright MCP); the 2026-06-14
+validation extended it to mobile (Android + iOS) and surfaced the two principles that make it work.
+A 2026-06-19 two-surface session then measured the **logged-in-web** (claude-in-chrome) and
+**native-desktop** (computer-use) lanes against the same six-primitive contract — splitting the web
+row into isolated vs logged-in and adding a native-desktop row (see Driver routing).
+
+## The observe-act-verify contract (driver-agnostic)
+
+Every driver fills the same six primitives; the runner above them is identical regardless of backend:
+
+| primitive | role | verdict impact |
+|---|---|---|
+| launch | bring the app/page up | failure → BLOCKED |
+| screenshot | vision-channel capture | evidence |
+| observe | structured element tree (XML/AX/DOM) → normalized elements | candidate ranking input |
+| tap | element/coordinate action | act |
+| input_text | text entry | act |
+| screen_signature | pre/post comparison | re-observe |
+
+A TC/step's intent text is ranked against observed elements (text · description · role · clickable ·
+bounds), the top candidate is acted on, and the post-action observation is checked **mechanically**
+(target text present/absent + candidate count + adapter exit) — no judge LLM decides the verdict.
+
+## Driver routing (no-reinvention)
+
+| surface | driver | note |
+|---|---|---|
+| web — isolated (no login) | Playwright MCP (Claude-usable) / Stagehand | isolated session, no operator cookies; generic-coding-agent usable — covers users without a proprietary browser-agent app |
+| web — logged-in (operator's authenticated session) | authenticated-session browser driver — currently claude-in-chrome (`/chrome`) | reaches SSO-behind SPAs the isolated lane can't; `find` → DOM element ref. Current-instance caveats in footnote † |
+| native desktop screen | screen-control driver — currently computer-use (screenshot + coordinate, vision channel) | reaches non-browser native app windows the browser lanes can't; `observe` is vision-coordinate, not an AX/DOM tree. ⚠️ local-session-bound (remote-control), no mobile; `input_text` unverified (5/6 — see note) |
+| Android (emulator **or** real device) | `adb` + `uiautomator dump` + `input tap` | same code path for both — see Portability below |
+| iOS simulator | `idb ui describe-all` + `idb ui tap` (idb-companion + fb-idb) | native AX tree |
+| hybrid WebView region | **vision channel** (screenshot + element detection) | native AX is blind here — see Hybrid rule |
+
+FH routes; it does not reimplement these. Drivers are pluggable; the contract is fixed.
+
+**Observe channel + measurement (2026-06-19).** computer-use fills `observe` via the **vision channel**
+(screenshot + coordinate) — the same channel the hybrid-WebView rule already mandates (below), now for
+native desktop; the browser and mobile drivers fill it with a structured tree (DOM ref / AX). Measured
+that session: claude-in-chrome closed **all six** primitives on both a no-login and an operator-logged-in
+surface; computer-use closed **five** (launch / screenshot / observe(vision) / tap / screen_signature) —
+`input_text` is keyboard-capable but was not exercised. The lanes cover **different surfaces, not
+redundant ones** — calculator success on computer-use does not imply chrome, and only chrome reaches the
+logged-in session.
+
+† **Current-instance caveats (point-in-time, not routing inputs).** claude-in-chrome and computer-use are
+built-in MCPs — verify presence via the `/mcp` UI, not `claude mcp list`. claude-in-chrome needs a direct
+first-party plan (unavailable on Bedrock/Vertex) and a per-site permission grant. Tool surface as measured
+2026-06-19: claude-in-chrome ~22, computer-use ~24 (counts drift across releases — not a routing input).
+
+## ★ Principle 1 — Appium-less is the enabler
+
+Going **through Appium is the failure mode**, not the solution. Appium's WebView-context switching is
+flaky on hybrid apps; in one field case it burned a large automation budget without completing a single
+hybrid flow (repeated run→fail→rerun). The **direct path** (`uiautomator dump` / `idb describe-all` +
+coordinate tap + vision) bypasses that failure mode entirely. Appium is therefore *one regression-stage
+adapter*, not the exploration/execution substrate.
+
+## ★ Principle 2 — Hybrid WebView is opaque to native accessibility → synthesize vision
+
+Live-measured (2026-06-14, iOS hybrid sandbox): a WKWebView's **web content does not appear in the
+native accessibility tree** — only native chrome and bridge-triggered native overlays do. A tap on a
+web button fired the web→native bridge and the resulting native picker *did* appear in the AX tree
+(4→10 elements), confirming the boundary precisely.
+
+→ For hybrid apps, the observe channel must be **synthesized**: native AX for native targets + **vision
+(screenshot + element detection) for the WebView interior** + optionally a web-layer inspector. An
+XML/AX-only agent silently misses the entire web form. This converges with the external frontier
+finding that SOTA UI automation is hybrid (AX/accessibility for action targets, vision for grounding
+and verification, deterministic probes for assertions).
+
+## Portability — design on simulator, run on real device (Appium-less)
+
+Because the Android driver uses the same `adb -s <serial>` interface for an emulator and a USB-connected
+real device, a structure authored and validated on the **simulator runs unchanged on the real device**
+— no per-device manual element extraction. Live-measured: a flow validated on the emulator auto-extracted
+the full native element set on a real-device banking sandbox with zero code change. This portability —
+sim-authored → real-device execution without Appium — is itself a meaningful capability step (a project
+may ship it before any AI-prospective layer).
+
+## Caveats (honest scope)
+
+- Element-ranking *accuracy* (intent → correct element) is unmeasured until a project supplies real
+  fixtures from its own app; the contract and loop are validated, the ranking quality is per-project.
+- Native-only screens work out of the box; hybrid screens need the vision channel wired (a pluggable
+  detector — FH routes to a vision model, does not build one).
+- Korean / IME text entry on Android real devices via `adb input text` is unreliable (measured) — needs
+  an IME-broadcast keyboard, not raw input.
+
+## Cross-refs
+
+`[[fh-live-surface-acceleration]]` (the capability-axis memory) · `multi_model_sidecar_strategy.md`
+(surface routing) · `deep_research_capability_ladder.md` (the sibling "route, don't build" pattern for
+research). The 2026-06-14 frontier survey backing the hybrid-SOTA convergence lives in the private
+companion store (`paper-signals/frontier_computer_use_ui_automation_2026-06-14.md`).

package/knowledge/shared/harness-core/measurement-integrity-checklist.md

@@ -0,0 +1,59 @@
+# Measurement-Integrity Checklist — cross-model measurement pre-flight
+
+> A cross-model measurement is only trustworthy if its **instrument** is verified first.
+> Measurement integrity is a *precondition*, not a result. Three observed failure modes, each with a
+> concrete countermeasure. Consult this before any FH measurement that compares models (sims, sidecar
+> comparisons, capability-equalizer runs, the-bible model panels, A6-class experiments).
+
+This is a **checklist a measurement consults**, not a gate with triggers and not a dispatch surface —
+hence a knowledge doc, the lightest asset that holds it (a harness gets simpler over time). If it ever
+becomes a gate other skills invoke, revisit the weight.
+
+## The three failure modes + countermeasures
+
+| # | Failure mode (observed) | Countermeasure |
+|---|---|---|
+| 1 | **Silent model fallback** — passing a model *slug* silently resolved to a weaker model (e.g. an `agy` slug fell back to Flash) instead of the intended one. The run *looks* like the named model but isn't. | **Pin the display name, not the slug** (e.g. `"Gemini 3.1 Pro (High)"`, not a bare slug). Confirm the resolved identity, don't assume the slug binds. |
+| 2 | **Non-deterministic borderline verdicts** — contested/borderline cases flip across runs (observed: haiku 4/4 flip; flagship models flip too — flipping is **not** a tier signal). A single draw is noise, not a measurement. | **reps ≥ 3 on any borderline/contested verdict.** A single run on a contested case is inadmissible. Report the flip pattern (STABLE vs FLIP), not just the modal verdict. |
+| 3 | **Generic self-identity probe** — a probe any model passes ("are you working? → OK") proves nothing about *which* model answered. | **Use a discriminating probe** — one that two different models answer *differently*. A generic-pass probe is invalid. The probe is a **pattern, not a fixed string**: a probe that discriminates Opus 4.8 from Sonnet 4.6 today may both-pass a future model generation, so **re-validate the probe each model generation** (same staleness class `memory-hygiene` exists to catch). |
+
+## Why these are entangled (and why they matter beyond their own scope)
+
+Item #2 (reps≥3) is the discipline that **retracted half the evidence** for the
+`[[feedback_correlated_blindspot_union_over_majority]]` finding — one of its two supporting cases
+turned out to be non-deterministic borderline flipping, not a stable correlated blind spot. So this
+checklist is the **prerequisite** for any "correlated error" claim: you cannot call an error correlated
+(and prescribe union-over-majority) until reps≥3 has distinguished a stable correlated error from a
+single-draw artifact. Item #3 (discriminating probe) **embodies** the judge-robustness /
+mechanical-anchor principle — don't trust self-reported identity, prove it discriminatingly
+([[feedback_judge_robustness_mechanical_anchor]]).
+
+## Done When
+
+- The checklist enumerates all three failure modes, each with its countermeasure.
+  *Check class: mandatory-pass (binary — three items present, each with a countermeasure).*
+- The probe item specifies a **discriminating** test and rejects generic probes.
+  *Check class: judged, pair: a probe that two different models both pass must FAIL this check; a
+  discriminating one must distinguish them.*
+- Any FH cross-model measurement records which checklist items it ran.
+  *Check class: measured (count of items applied) — closes the predict-verify loop for future audit.*
+
+## Optional hardening (when a measurement feeds a published / paper claim)
+
+Escalate item #1 (display-name pin) and item #3 (identity verification) from prose to a **logged
+mechanical assertion**: the measurement harness records the *verified* model identity it observed, not
+the requested slug. Prose discipline is sufficient for internal dogfooding; a published claim earns the
+mechanical log.
+
+> **External dogfood (a second, field-layer instance — n=1 external, a signal not a settled frontier):**
+> the sister skill `ponytail` ships a runnable instance of the precondition behind all three modes —
+> a `--selftest` that proves each instrument (`good===true && bad===false`) before any API spend, and
+> two caught instrument contaminations. Detail + pinned citations: `tracks/_audit/session_2026_06_24_ponytail-lazy-senior-dev.md` §2-C (single source).
+
+---
+
+**Origin** (2026-06-22 harvest-loop): three failure modes observed across the-bible L2 model panel
+(agy slug→Flash silent fallback; reps=3 non-determinism) and prior multi-model sims (generic-probe
+ambiguity). Sister findings: [[feedback_correlated_blindspot_union_over_majority]] (reps≥3 prerequisite),
+[[feedback_judge_robustness_mechanical_anchor]] (discriminating-probe = mechanical anchor),
+[[reference_agy_model_catalog]] (display-name pin — agy slug fallback documented there).