@chrono-meta/fh-gate 1.4.41 → 1.4.42

package/knowledge/shared/harness-core/live_surface_automation_pattern.md

@@ -0,0 +1,110 @@
+---
+name: live-surface-automation-pattern
+description: The capability pattern FH routes to when a mapping project needs an agent to drive a live UI surface (web/mobile) — observe-act-verify over the running app, not code/data fetch. FH routes drivers (no-reinvention); the value is the cross-platform observe-act-verify contract, the Appium-less principle, and the hybrid-WebView vision-synthesis rule. Generalizable across mapping projects; not tied to any one project.
+date: 2026-06-14
+tags: [live-surface, ui-automation, observe-act-verify, appium-less, hybrid-webview, no-reinvention, mapping-acceleration]
+---
+
+# Live-Surface Automation Pattern
+
+When a mapping project's work lives on a **live UI surface** — a running mobile app or web page whose
+state cannot be reached by reading code or fetching data — FH's posture is the same as for any
+capability it does not own: **detect the need and route to the best driver present, do not build a UI
+engine** (no-reinvention). The value FH adds is the *contract* below, not a new automation tool.
+
+This is the acceleration axis behind the "③ map-project acceleration" door's live-surface capability
+(`[[fh-live-surface-acceleration]]`). It was first validated on web (Playwright MCP); the 2026-06-14
+validation extended it to mobile (Android + iOS) and surfaced the two principles that make it work.
+A 2026-06-19 two-surface session then measured the **logged-in-web** (claude-in-chrome) and
+**native-desktop** (computer-use) lanes against the same six-primitive contract — splitting the web
+row into isolated vs logged-in and adding a native-desktop row (see Driver routing).
+
+## The observe-act-verify contract (driver-agnostic)
+
+Every driver fills the same six primitives; the runner above them is identical regardless of backend:
+
+| primitive | role | verdict impact |
+|---|---|---|
+| launch | bring the app/page up | failure → BLOCKED |
+| screenshot | vision-channel capture | evidence |
+| observe | structured element tree (XML/AX/DOM) → normalized elements | candidate ranking input |
+| tap | element/coordinate action | act |
+| input_text | text entry | act |
+| screen_signature | pre/post comparison | re-observe |
+
+A TC/step's intent text is ranked against observed elements (text · description · role · clickable ·
+bounds), the top candidate is acted on, and the post-action observation is checked **mechanically**
+(target text present/absent + candidate count + adapter exit) — no judge LLM decides the verdict.
+
+## Driver routing (no-reinvention)
+
+| surface | driver | note |
+|---|---|---|
+| web — isolated (no login) | Playwright MCP (Claude-usable) / Stagehand | isolated session, no operator cookies; generic-coding-agent usable — covers users without a proprietary browser-agent app |
+| web — logged-in (operator's authenticated session) | authenticated-session browser driver — currently claude-in-chrome (`/chrome`) | reaches SSO-behind SPAs the isolated lane can't; `find` → DOM element ref. Current-instance caveats in footnote † |
+| native desktop screen | screen-control driver — currently computer-use (screenshot + coordinate, vision channel) | reaches non-browser native app windows the browser lanes can't; `observe` is vision-coordinate, not an AX/DOM tree. ⚠️ local-session-bound (remote-control), no mobile; `input_text` unverified (5/6 — see note) |
+| Android (emulator **or** real device) | `adb` + `uiautomator dump` + `input tap` | same code path for both — see Portability below |
+| iOS simulator | `idb ui describe-all` + `idb ui tap` (idb-companion + fb-idb) | native AX tree |
+| hybrid WebView region | **vision channel** (screenshot + element detection) | native AX is blind here — see Hybrid rule |
+
+FH routes; it does not reimplement these. Drivers are pluggable; the contract is fixed.
+
+**Observe channel + measurement (2026-06-19).** computer-use fills `observe` via the **vision channel**
+(screenshot + coordinate) — the same channel the hybrid-WebView rule already mandates (below), now for
+native desktop; the browser and mobile drivers fill it with a structured tree (DOM ref / AX). Measured
+that session: claude-in-chrome closed **all six** primitives on both a no-login and an operator-logged-in
+surface; computer-use closed **five** (launch / screenshot / observe(vision) / tap / screen_signature) —
+`input_text` is keyboard-capable but was not exercised. The lanes cover **different surfaces, not
+redundant ones** — calculator success on computer-use does not imply chrome, and only chrome reaches the
+logged-in session.
+
+† **Current-instance caveats (point-in-time, not routing inputs).** claude-in-chrome and computer-use are
+built-in MCPs — verify presence via the `/mcp` UI, not `claude mcp list`. claude-in-chrome needs a direct
+first-party plan (unavailable on Bedrock/Vertex) and a per-site permission grant. Tool surface as measured
+2026-06-19: claude-in-chrome ~22, computer-use ~24 (counts drift across releases — not a routing input).
+
+## ★ Principle 1 — Appium-less is the enabler
+
+Going **through Appium is the failure mode**, not the solution. Appium's WebView-context switching is
+flaky on hybrid apps; in one field case it burned a large automation budget without completing a single
+hybrid flow (repeated run→fail→rerun). The **direct path** (`uiautomator dump` / `idb describe-all` +
+coordinate tap + vision) bypasses that failure mode entirely. Appium is therefore *one regression-stage
+adapter*, not the exploration/execution substrate.
+
+## ★ Principle 2 — Hybrid WebView is opaque to native accessibility → synthesize vision
+
+Live-measured (2026-06-14, iOS hybrid sandbox): a WKWebView's **web content does not appear in the
+native accessibility tree** — only native chrome and bridge-triggered native overlays do. A tap on a
+web button fired the web→native bridge and the resulting native picker *did* appear in the AX tree
+(4→10 elements), confirming the boundary precisely.
+
+→ For hybrid apps, the observe channel must be **synthesized**: native AX for native targets + **vision
+(screenshot + element detection) for the WebView interior** + optionally a web-layer inspector. An
+XML/AX-only agent silently misses the entire web form. This converges with the external frontier
+finding that SOTA UI automation is hybrid (AX/accessibility for action targets, vision for grounding
+and verification, deterministic probes for assertions).
+
+## Portability — design on simulator, run on real device (Appium-less)
+
+Because the Android driver uses the same `adb -s <serial>` interface for an emulator and a USB-connected
+real device, a structure authored and validated on the **simulator runs unchanged on the real device**
+— no per-device manual element extraction. Live-measured: a flow validated on the emulator auto-extracted
+the full native element set on a real-device banking sandbox with zero code change. This portability —
+sim-authored → real-device execution without Appium — is itself a meaningful capability step (a project
+may ship it before any AI-prospective layer).
+
+## Caveats (honest scope)
+
+- Element-ranking *accuracy* (intent → correct element) is unmeasured until a project supplies real
+  fixtures from its own app; the contract and loop are validated, the ranking quality is per-project.
+- Native-only screens work out of the box; hybrid screens need the vision channel wired (a pluggable
+  detector — FH routes to a vision model, does not build one).
+- Korean / IME text entry on Android real devices via `adb input text` is unreliable (measured) — needs
+  an IME-broadcast keyboard, not raw input.
+
+## Cross-refs
+
+`[[fh-live-surface-acceleration]]` (the capability-axis memory) · `multi_model_sidecar_strategy.md`
+(surface routing) · `deep_research_capability_ladder.md` (the sibling "route, don't build" pattern for
+research). The 2026-06-14 frontier survey backing the hybrid-SOTA convergence lives in the private
+companion store (`paper-signals/frontier_computer_use_ui_automation_2026-06-14.md`).

package/knowledge/shared/harness-core/measurement-integrity-checklist.md

@@ -0,0 +1,54 @@
+# Measurement-Integrity Checklist — cross-model measurement pre-flight
+
+> A cross-model measurement is only trustworthy if its **instrument** is verified first.
+> Measurement integrity is a *precondition*, not a result. Three observed failure modes, each with a
+> concrete countermeasure. Consult this before any FH measurement that compares models (sims, sidecar
+> comparisons, capability-equalizer runs, the-bible model panels, A6-class experiments).
+
+This is a **checklist a measurement consults**, not a gate with triggers and not a dispatch surface —
+hence a knowledge doc, the lightest asset that holds it (a harness gets simpler over time). If it ever
+becomes a gate other skills invoke, revisit the weight.
+
+## The three failure modes + countermeasures
+
+| # | Failure mode (observed) | Countermeasure |
+|---|---|---|
+| 1 | **Silent model fallback** — passing a model *slug* silently resolved to a weaker model (e.g. an `agy` slug fell back to Flash) instead of the intended one. The run *looks* like the named model but isn't. | **Pin the display name, not the slug** (e.g. `"Gemini 3.1 Pro (High)"`, not a bare slug). Confirm the resolved identity, don't assume the slug binds. |
+| 2 | **Non-deterministic borderline verdicts** — contested/borderline cases flip across runs (observed: haiku 4/4 flip; flagship models flip too — flipping is **not** a tier signal). A single draw is noise, not a measurement. | **reps ≥ 3 on any borderline/contested verdict.** A single run on a contested case is inadmissible. Report the flip pattern (STABLE vs FLIP), not just the modal verdict. |
+| 3 | **Generic self-identity probe** — a probe any model passes ("are you working? → OK") proves nothing about *which* model answered. | **Use a discriminating probe** — one that two different models answer *differently*. A generic-pass probe is invalid. The probe is a **pattern, not a fixed string**: a probe that discriminates Opus 4.8 from Sonnet 4.6 today may both-pass a future model generation, so **re-validate the probe each model generation** (same staleness class `memory-hygiene` exists to catch). |
+
+## Why these are entangled (and why they matter beyond their own scope)
+
+Item #2 (reps≥3) is the discipline that **retracted half the evidence** for the
+`[[feedback_correlated_blindspot_union_over_majority]]` finding — one of its two supporting cases
+turned out to be non-deterministic borderline flipping, not a stable correlated blind spot. So this
+checklist is the **prerequisite** for any "correlated error" claim: you cannot call an error correlated
+(and prescribe union-over-majority) until reps≥3 has distinguished a stable correlated error from a
+single-draw artifact. Item #3 (discriminating probe) **embodies** the judge-robustness /
+mechanical-anchor principle — don't trust self-reported identity, prove it discriminatingly
+([[feedback_judge_robustness_mechanical_anchor]]).
+
+## Done When
+
+- The checklist enumerates all three failure modes, each with its countermeasure.
+  *Check class: mandatory-pass (binary — three items present, each with a countermeasure).*
+- The probe item specifies a **discriminating** test and rejects generic probes.
+  *Check class: judged, pair: a probe that two different models both pass must FAIL this check; a
+  discriminating one must distinguish them.*
+- Any FH cross-model measurement records which checklist items it ran.
+  *Check class: measured (count of items applied) — closes the predict-verify loop for future audit.*
+
+## Optional hardening (when a measurement feeds a published / paper claim)
+
+Escalate item #1 (display-name pin) and item #3 (identity verification) from prose to a **logged
+mechanical assertion**: the measurement harness records the *verified* model identity it observed, not
+the requested slug. Prose discipline is sufficient for internal dogfooding; a published claim earns the
+mechanical log.
+
+---
+
+**Origin** (2026-06-22 harvest-loop): three failure modes observed across the-bible L2 model panel
+(agy slug→Flash silent fallback; reps=3 non-determinism) and prior multi-model sims (generic-probe
+ambiguity). Sister findings: [[feedback_correlated_blindspot_union_over_majority]] (reps≥3 prerequisite),
+[[feedback_judge_robustness_mechanical_anchor]] (discriminating-probe = mechanical anchor),
+[[reference_agy_model_catalog]] (display-name pin — agy slug fallback documented there).

package/knowledge/shared/harness-core/meta_harness_engineering_definition.md

@@ -0,0 +1,116 @@
+---
+name: meta-harness-engineering-definition
+description: Defines meta harness engineering — the discipline of building systems that build, evaluate, and evolve AI harnesses. Grounds FH's mission in external academic convergence (arXiv 2605.18747, 2604.14228) and maps FH's 6-axis framework to the emerging field taxonomy.
+type: reference
+date: 2026-05-29
+tags: [harness-engineering, meta-harness, definition, frontier, academic-convergence, 6-axis]
+---
+
+# Meta Harness Engineering — Definition and FH Positioning
+
+## What is a Harness?
+
+A harness is the engineering layer that surrounds an AI model and determines how it behaves in practice:
+context injection, tool routing, permission management, memory, output verification, and improvement loops.
+
+The model provides reasoning capability. The harness determines whether that capability is reliable, reproducible, and improvable.
+
+> VILA-Lab empirical finding (arXiv 2604.14228): analysis of Claude Code v2.1.88 (1,884 files, ~512,000 lines of TypeScript) found that **98.4% is harness infrastructure; 1.6% is AI decision logic**. As foundation models converge in reasoning capability, the engineering harness is the primary differentiator.
+
+---
+
+## What is Meta Harness Engineering?
+
+**Harness engineering**: building a harness for a specific project.
+
+**Meta harness engineering**: building a system that:
+1. **Seeds** harnesses for new projects (templates, protocols, onboarding)
+2. **Evaluates** harness quality across projects (diagnosis, quality rubrics, regression detection)
+3. **Evolves** harnesses over time (self-improvement loops, pattern extraction, frontier absorption)
+
+The meta layer does not replace the project-level harness — it accelerates, standardizes, and continuously improves it.
+
+---
+
+## Academic Convergence (2026)
+
+Two independent research threads converged on the same insight in 2026:
+
+### "Code as Agent Harness" (arXiv 2605.18747)
+**Authors**: Xuying Ning, Katherine Tieu, Dongqi Fu, + 39 collaborators (May 2026)
+
+> "Code has evolved from being an output of LLMs to serving as the basis for agent infrastructure."
+
+Proposed three-layer harness taxonomy:
+
+| Academic Layer | Description |
+|---|---|
+| **Harness interface** | Connecting agents to reasoning environments |
+| **Harness mechanisms** | Planning, tool use, state management |
+| **Scale** | Single → multi-agent coordination |
+
+### FH 6-Axis Framework Mapping
+
+| Academic Layer | FH Equivalent | Assets |
+|---|---|---|
+| Harness interface | Context injection layer | `CLAUDE.md` · `.claude/rules/*.md` · `MEMORY.md` |
+| Harness mechanisms | Skill bus | `verify-bidirectional` · `steel-quench` · `phantom-quench` · `agent-composer` |
+| Scale | Multi-agent dispatch | Agent View · `context-bridge-dispatch` · `agent-composer` parallel dispatch |
+| *(meta layer, not in taxonomy)* | Harness evolution | `harvest-loop` · `harness-doctor` · `frontier-digest` · `prompt-regression` |
+
+The academic taxonomy covers the **static structure** of a harness. FH adds the **dynamic evolution layer** — the mechanism by which the harness improves itself over time.
+
+---
+
+## FH's Differentiating Position
+
+Three contemporary approaches to harness engineering, compared:
+
+| Approach | Representative | Axis | Human role |
+|---|---|---|---|
+| **Automation-maximalist** | Sylph.AI ("The Last Harness You'll Ever Build", arXiv 2604.21003) | Fully automated adversarial loops — minimize human touch | Minimal (approve outputs) |
+| **Automation-first** | harness-evolver (raphaelchristi) | Outer-loop field observation → adversarial critique → integration → verification | Light (curate exceptions) |
+| **Human-in-the-loop curation** | **forge-harness** | Knowledge accumulation + AI-assisted evolution + mandatory human gate on PRs | Active (all merges require approval) |
+
+FH's position: **human judgment is not a cost to minimize — it is the quality gate that prevents harness drift**. Automation handles pattern detection, drafting, and proposal; humans decide what enters the harness.
+
+> **Principle (field harness)**: "A good harness gets simpler over time. If it's getting more complex, something is wrong."
+> **Principle (meta-harness)**: Optimize, not necessarily simplify — complexity is justified when it earns its scope. Red flags: orphaned skills, redundant overlap, decorative structure (complexity that exists but doesn't change behavior).
+
+---
+
+## The 6-Axis Framework as Practice Layer
+
+The 6-axis framework (`harness_6axis_framework.md`) is FH's operational implementation of meta harness engineering:
+
+| Axis | Name | What it governs |
+|---|---|---|
+| 1 | **Structure** | File layout, naming conventions, required assets |
+| 2 | **Context** | What information the AI sees at session start |
+| 3 | **Plan** | How work is decomposed and tracked |
+| 4 | **Execute** | How tasks are carried out (inline / agent dispatch / skill) |
+| 5 | **Verify** | How outputs are validated (regression, bidirectional, simulation) |
+| 6 | **Improve** | How the harness evolves (harvest-loop, frontier-digest, compounding) |
+
+Axis 6 is the meta layer within the framework — the harness improves itself through structured cadence.
+
+---
+
+## External Evidence Base
+
+| Source | Finding | FH Relevance |
+|---|---|---|
+| VILA-Lab (arXiv 2604.14228) | 98.4% of Claude Code = harness infrastructure | Validates that harness engineering is the primary differentiator, not model capability |
+| Ning et al. (arXiv 2605.18747) | Code = agent harness substrate; 3-layer taxonomy | FH maps to all 3 layers + adds evolution layer |
+| Seong / Sylph.AI (arXiv 2604.21003) | "Terminal harness" — convergence point exists | Sister asset: automation-maximalist vs FH human-in-loop; titles echo simplification principle |
+| harness-evolver (raphaelchristi) | Outer-loop field observation → adversarial critique | Sister asset: automation-first complement to FH knowledge-accumulation-first |
+
+---
+
+## Related FH Assets
+
+- `harness_6axis_framework.md` — The operational framework
+- `hub_compounding_loop.md` — Weekly/monthly/quarterly improvement cadence (Axis 6)
+- `skill_quality_rubric.md` — Quality standard for skill assets
+- `return_path_gate.md` — Closed-loop skill chain pattern (structural)
+- `hub_maturity_roadmap.md` — Phase I→II→III evolution path