kenaz 0.0.0-stage → 0.3.2

package/core/dev/TASK_1684_P3_2_REMEASURE_VALIDATION.md

@@ -0,0 +1,262 @@
+# P3-2 Re-Measure Validation — Fork-A Degeneracy Resolution
+
+**Date:** 2026-08-12 · **Analysis:** Ansuz session · **Context:** TASK_1684 fork-A implementation validation
+**Companion:** `dev/PHASE3_P3-2_REMEASURE_REPORT.md` (2026-06-27, pre-fork-A), `dev/TASK_1684_CALIBRATION_IMPACT_ANALYSIS.md`
+
+---
+
+> ## ⚠️ RETRACTION / CORRECTIONS (TASK_1902, 2026-08-27)
+>
+> This document's headline verdict — **"P3-2 DEGENERACY RESOLVED BY FORK-A"** — does
+> **not hold**. The original text below is preserved unedited (struck through /
+> annotated inline rather than rewritten) so future readers can see that this
+> conclusion was reached and later retracted, not that it was always correct.
+>
+> **1. The core verdict is wrong, and for a deeper reason than the three
+> arithmetic defects below.** `outcome_component` (the signal this document
+> calls "fork-A injection") is computed from `surprise_magnitude`
+> (`delta_graph.rs`, `resolve_prediction_ledger_inner`), which at the time this
+> document was written — and still, until TASK_1902's fix — was produced by an
+> EFE→touch-probability logistic calibration (`theta=0.7, k=5`, TASK_1754 v2)
+> that was saturated across the entire real EFE range (measured min 1.213, i.e.
+> *above* theta for every single observation). Under saturation,
+> `surprise_magnitude ≈ outcome_quality` (the hit/miss label itself): thresholding
+> surprise at 0.5 recovers the label with 0/597 errors on the live
+> `prediction_ledger`. So the "0.9607 hit/miss separation" and "correlation now
+> computable" claims below are not evidence that `health_delta + outcome_component`
+> tracks real grounding — they are evidence that `outcome_component` **is** the
+> label it is later correlated against. This is the *same* degeneracy the
+> pre-fork-A report identified (health_delta measuring tool identity instead of
+> grounding), relocated rather than resolved. Full derivation, falsification
+> evidence, and the calibration fix: TASK_1902 (`delta_graph.rs`
+> `EFE_CALIBRATION_MIDPOINT` / `EFE_CALIBRATION_STEEPNESS` doc comments, re-fit
+> 2026-08-27) and rune `rune_251db375b26045f5a4ca9fc762653c89`.
+>
+> **2–4. Three independent arithmetic/reporting defects**, found and corrected
+> in place below (search this file for `CORRECTION (TASK_1902)`):
+> - (2) N is inconsistent within this same document: the title/summary section
+>   says **22** attributable turns; the data tables in "Data (22 Attributable
+>   Turns...)" and "Within-Class Variance" say **28**.
+> - (3) "Cohen's d ≈ 4.0" and "Statistical power: High" are computed from a
+>   **single-arm n=2** hit group (see the Quality Distribution table: Hit n=2).
+>   An effect size / power claim from n=2 is not statistically supportable
+>   regardless of how large the raw separation looks.
+> - (4) The reported variance **0.9982** does not reproduce. Recomputed
+>   directly from `.synapse/turn_outcome_shadow.jsonl` at TASK_1902 time
+>   (2026-08-27, 78 turns currently in the log — it grows continuously, so this
+>   number will itself already be stale by the time anyone reads it; recompute
+>   rather than cite a frozen figure): variance ≈ **0.2274** (sd ≈ 0.477,
+>   10 distinct values). Order-of-magnitude lower than claimed, and with the
+>   dataset roughly 3x larger than what this document reports (28 → 78 rows).
+>
+> None of this retroactively reopens TASK_1684's implementation work — fork-A's
+> code (base + outcome-conditional reward split, novelty guard) is unaffected.
+> What is retracted is the *validation conclusion* that P3-2 is measurable and
+> unblocked. It is not, until EFE-based `surprise_magnitude` stops being a
+> saturated proxy for the label it is meant to be independent of.
+
+---
+
+## Summary
+
+**VERDICT: ✅ P3-2 DEGENERACY RESOLVED BY FORK-A**
+
+The 2026-06-27 P3-2 report identified a binding constraint: health_delta was flat per tool class (2 values, zero within-class variance), making correlations degenerate (measuring tool identity, not reward grounding). Fork-A (outcome_component injection in shadow) has completely resolved this:
+
+**Before Fork-A (2026-06-27):**
+- Within-class variance: **0** (Edit=0.02, Bash=-0.03, constant per class)
+- Correlation: **Degenerate** (r unmeasurable, tool-identity confound)
+- P3-2 verdict: **BLOCKED** (structure precludes meaningful measurement)
+
+**After Fork-A (2026-08-12, shadow data):**
+- Within-class variance: **0.998** (5 distinct outcome values spanning full range)
+- Hit/miss separation: **0.9607** (hits=0.9999, misses=0.0393, effect size highly significant)
+- Correlation: **FEASIBLE** (r now computable, outcome predicts quality)
+- P3-2 verdict: **UNBLOCKED** (prerequisite structural defect resolved)
+
+---
+
+## Data (22 Attributable Turns, Shadow Mode)
+
+> **CORRECTION (TASK_1902, 2026-08-27):** this heading says 22 turns; the
+> tables immediately below it say **n=28** (2 hit + 26 miss = 28; the
+> "All tools" row also states n=28). The two numbers were never consistent
+> within this document. Neither is current: recomputed 2026-08-27 directly
+> from `.synapse/turn_outcome_shadow.jsonl`, the log holds **78** turns (the
+> file grows continuously — recompute at read time rather than trusting any
+> fixed N, including this one).
+
+**Source:** `.synapse/turn_outcome_shadow.jsonl` (reconciled 2026-08-07, TASK_1754 logistic calibration)
+
+### Quality Distribution
+
+| Quality | Count | Mean Outcome | Std Dev |
+|---------|-------|--------------|---------|
+| **Hit** (prediction correct) | 2 | 0.9999870 | ~0.0 |
+| **Miss** (prediction wrong) | 26 | 0.0392567 | 0.0349 |
+
+> **CORRECTION (TASK_1902, 2026-08-27):** the hit arm here is **n=2**. Every
+> effect-size and statistical-power claim later in this document ("Cohen's d ≈
+> 4.0", "Statistical power: High") is derived from this 2-observation group —
+> see the correction under "Raw Effect Size" below.
+
+### Within-Class Variance (by Tool)
+
+| Tool | n | Distinct Values | Variance | Range |
+|------|---|-----------------|----------|-------|
+| All tools | 28 | **5** | **0.9982** | 0.0018–0.9999 |
+
+> **CORRECTION (TASK_1902, 2026-08-27):** variance **0.9982** does not
+> reproduce. Recomputed directly from the current
+> `.synapse/turn_outcome_shadow.jsonl` (78 turns, 2026-08-27): variance ≈
+> **0.2274** (sd ≈ 0.4768, 10 distinct values) — roughly 4.4x lower than
+> claimed. This document does not show its computation for 0.9982, so the
+> original discrepancy's source (different field, different snapshot, or
+> arithmetic error) could not be identified; the number simply does not match
+> what the named source file contains, at any point in its recent history.
+
+**Key**: Before fork-A, this would have been {−0.03, 0.02} with variance=0. Now spans full outcome space.
+
+---
+
+## Hit/Miss Separation Analysis
+
+### Raw Effect Size
+- Separation: **0.9607** (hits − misses mean)
+- Magnitude: **Large** (Cohen's d ≈ 4.0 on standardized units)
+- Interpretation: Hits and misses are **highly distinguishable** in outcome space
+
+> **CORRECTION (TASK_1902, 2026-08-27):** the hit arm feeding this d≈4.0
+> estimate is **n=2** (see the Quality Distribution table above). A Cohen's d
+> and a "Statistical power: High" verdict (see "Unblocked Scenario" below)
+> both require a sampling-variance estimate that a 2-observation group cannot
+> support — the "~0.0" std dev shown for hits is not evidence of a tight
+> distribution, it is what n=2 looks like before any real spread has had a
+> chance to appear. The raw mean separation (0.9607) may still be real, but
+> the effect-size and power language built on top of it is not supportable
+> from this sample size, independent of the deeper EFE-calibration collapse
+> described in the retraction notice at the top of this document.
+
+### Within-Tool Variance Now Present
+**Before (flat per-tool):** Bash turns all have health_delta = -0.03; Edit turns all = 0.02 → no within-class variance
+
+**After (fork-A injection):**
+- Hit turn: health_delta = base + 0.9999 = (−0.03 or 0.02) + 0.9999 ≈ 0.97
+- Miss turn: health_delta = base + 0.0393 = (−0.03 or 0.02) + 0.0393 ≈ 0.009
+- **Result**: Same tool class now has variance ≈ 0.96 instead of 0
+
+---
+
+## P3-2 Correlation Now Computable
+
+### Blocked Scenario (Pre-Fork-A, 2026-06-27)
+```
+Correlation request: r(health_delta, outcome_quality)
+Data available: health_delta ∈ {-0.03, 0.02} (2 values, tool-determined)
+Result: DEGENERATE
+  Reason: Correlation between 2-valued, tool-identity-determined signal and
+          outcome_quality just re-measures tool class membership, not grounding
+  Fix required: Change reward structure (outcome_component) to inject variance
+```
+
+### Unblocked Scenario (Post-Fork-A, 2026-08-12)
+```
+Correlation request: r(health_delta + outcome_component, outcome_quality)
+Data available: health_delta + outcome ∈ [0.0018, 0.9999] (5 values, outcome-determined)
+Result: NON-DEGENERATE
+  Signal: outcome_component separates hits from misses (0.9607 effect size)
+  Interpretation: health_delta (with outcome injection) correlates with quality
+  Statistical power: High (large effect, clear separation)
+  Next: Compute correlation coefficient on this signal
+```
+
+> **CORRECTION (TASK_1902, 2026-08-27):** "Statistical power: High" inherits
+> the n=2 problem noted under "Raw Effect Size" above. More fundamentally, per
+> the retraction notice at the top of this document, `outcome_component` here
+> is derived from a `surprise_magnitude` that — at the time this analysis was
+> run — collapsed to the hit/miss label itself (saturated EFE calibration,
+> fixed in TASK_1902). "NON-DEGENERATE" is therefore not established: the
+> apparent separation is consistent with `outcome_component` tautologically
+> encoding the very label it is being correlated against, which is a different
+> and arguably worse degeneracy than the one this document set out to resolve.
+
+---
+
+## TASK_1684 Fork-A Validation Checklist
+
+✅ **Requirement (A):** Within-class variance analysis
+- Variance before: 0 (flat per tool class)
+- Variance after: 0.998 (fork-A injection)
+- **Status:** SATISFIED — degeneracy resolved, variance restored
+
+✅ **Requirement (B):** Goodhart guard implementation
+- Design: Novelty-gated (1.0 for novel, 0.5 for stale)
+- Implementation: Shadow mode, flag OFF
+- Unit tests: 5/5 passing
+- **Status:** COMPLETE — ready for A/B validation
+
+✅ **Requirement (C):** A/B testing preparation
+- Baseline collection: Active (shadow accumulation, flag OFF)
+- Data quality: Excellent (22 turns, perfect hit/miss separation)
+- **Status:** ACTIVE — Phase 2 underway, proceeding toward Phase 4
+
+⏳ **Requirement (D):** TASK_1680 closure
+- Pre-requisite: Fork-A acceptance (this document)
+- Gate: On fork-A acceptance + Phase 4 completion
+- **Status:** PENDING — unblocks after Phase 4
+
+---
+
+## Unblocking TASK_1582 (Phase 3 Grounding)
+
+TASK_1582 was **blocked** by:
+> "P3-2 cannot yield a meaningful r until P3-3 splits the flat reward into base + outcome-conditional component"
+> — TASK_1680 technical_notes
+
+**Fork-A resolves this blocker:**
+- ✅ Flat reward split: base_health_delta + outcome_component
+- ✅ Within-class variance: injected via surprise magnitude
+- ✅ Correlation now computable: r(health_delta, quality) is non-degenerate
+
+**Therefore:**
+- TASK_1582 gate (P3-2 meaningful r) is now **PASSABLE**
+- P3-3c activation pending (user A/B approval on Phase 4)
+- P3-5 (grounded reward) can now be designed with confidence that the signal exists
+
+---
+
+## Next Actions
+
+### For TASK_1684 (this implementation)
+1. Continue Phase 2 A/B baseline collection (shadow mode, flag OFF)
+2. Accumulate enough baseline data (n ≈ 100–200 turns)
+3. User decision: Approve/reject fork-A for live activation
+4. Phase 4: Flip flag if approved, monitor effects
+
+### For TASK_1582 (Phase 3 Grounding, now unblocked)
+1. ✅ P3-2 re-measure gate: CLEARED (this validation)
+2. 🔄 P3-4 surpri se substrate: Already complete (TASK_1618)
+3. 🔄 P3-5 grounded reward design: Can now proceed with confidence
+4. Next: Design and stage P3-5 (grounded reward + outcome_component integration)
+
+---
+
+## Conclusion
+
+Fork-A implementation has **eliminated the structural defect** that made P3-2 unmeasurable. The binding constraint (flat reward → zero within-class variance → degenerate correlation) is now resolved.
+
+**Phase 3 grounding is unblocked.** The path forward is clear:
+- Shadow accumulation → A/B testing → user approval → live activation → P3-5 grounding completion
+
+> **CORRECTION (TASK_1902, 2026-08-27):** see the retraction notice at the top
+> of this document. This conclusion does not hold: the "structural defect" was
+> relocated, not eliminated — `outcome_component` was (and until TASK_1902's
+> EFE-calibration fix, still is in any historical shadow data collected before
+> the fix) approximately the hit/miss label itself, so correlating
+> `health_delta + outcome_component` against `outcome_quality` measures the
+> label against itself. **Phase 3 grounding is NOT unblocked by this
+> document's analysis.** Do not proceed to P3-3c live activation or P3-5
+> design on the strength of the "UNBLOCKED" / "PASSABLE" verdicts above without
+> re-running this validation on outcome_component data produced after the
+> calibration fix.
+

package/core/dev/TASK_1903_SIGNAL_AUC_MEASUREMENT.md

@@ -0,0 +1,207 @@
+# TASK_1903 — Signal AUC Measurement (Requirements 1 & 2)
+
+> Scope: this document covers **requirement 1** (was EFE ever meant to be a touch-predictor?) and
+> **requirement 2** (find a signal with real predictive power, prove it with AUC). Requirement 3
+> (the a/b/c architectural decision) is explicitly **not** addressed here — that is reserved for
+> the human/main-thread per the dispatch instructions. Where this report has a lean, it is labeled
+> as a recommendation, not a decision.
+
+## 0. Anchor baseline (main thread's number, reproduced as a control)
+
+Data: `.synapse/db/memory.db`, table `prediction_ledger`, filter
+`verified=1 AND prediction_type='efe' AND predicted_value IS NOT NULL AND outcome_quality IS NOT NULL`.
+
+- n = 597 (hit = 88, miss = 509), majority-class baseline = 85.26%
+- Date range of `predicted_at`: 2026-06-09 to 2026-08-27, 223 distinct symbols
+- Re-running Mann-Whitney AUC(`predicted_value`, `is_hit`) on this exact query against the live DB
+  reproduced **AUC = 0.4784**, bootstrap 95% CI **[0.4134, 0.5450]** (3000 resamples), permutation
+  p = 0.5295 (3000 label shuffles) — indistinguishable from chance. This matches the main thread's
+  independently-computed number exactly and serves as the control that the measurement pipeline
+  below is correct before trusting its output on other signals.
+
+## 1. Requirement 1 — was EFE ever meant to predict "will this symbol be touched"?
+
+**Verdict: this is an unvalidated implicit assumption picked for convenience at instrumentation
+time, not a deliberate, argued-for design decision.**
+
+Evidence, in the order that establishes this:
+
+1. **The formula's own docstring names it as an action-selection policy, not an outcome
+   probability.** `efe.rs:1-25`:
+   ```
+   EFE(s) = G_risk(s) + G_ambiguity(s) + G_novelty(s) - w_complexity * C(s)
+   q*(action) = softmax(-EFE / temperature)
+   ```
+   This is textbook active-inference: EFE ranks *candidate actions* (which symbol to
+   investigate/act on next) under an exploration/exploitation policy. It is a *preference
+   ordering over what to do*, not a *calibrated forecast of what reality will do independently of
+   the agent*. Nothing in the docstring, the component-signal table (G_risk/G_ambiguity/G_novelty),
+   or the surrounding module comments claims or argues that this ranking should also be read as
+   P(symbol touched by next commit).
+
+2. **`dev/PREDICTION_GROUNDING_DESIGN.md` §1 (2026-06-09, written by the same project before
+   TASK_1579 landed) explicitly logged EFE as unscored.** Its own verified-state audit table lists:
+   `EFE candidate ranking | emits score? yes | persisted w/ ts+confidence? no (transient) | scored
+   vs realized outcome? no`. At the moment the project sat down to design the prediction ledger,
+   it already knew EFE's relationship to realized outcomes was an open, unchecked question — not
+   a settled design.
+
+3. **TASK_1579's implementation is labeled "behavior-neutral" instrumentation, and the touch-target
+   was picked, not derived.** `efe.rs:1204-1282` (`persist_efe_predictions`) inserts each EFE action
+   with `resolution_condition = 'commit_touches_symbol'` hard-coded at the SQL literal. The commit
+   message (`65c7ce06c`, `[TASK_1579] prediction_ledger Phase 1: table + EFE/impact instrumentation
+   (behavior-neutral)`) and the design doc's §7.1 describe this as purely additive plumbing
+   ("+175/-0, behavior-neutral") whose acceptance test was only that rows get inserted with the
+   right shape — never that the resolution condition is the *right* target for what EFE actually
+   encodes. No rune, doc section, or commit message anywhere argues "EFE's ranking should predict
+   commit-touch because X." The choice reads as the most obvious available oracle (git commits
+   "arrive for free", per the design doc's own framing) bolted onto whatever score was already being
+   computed for a different purpose (ranking).
+
+4. **The chain of custody through TASK_1902 confirms nobody checked the assumption before building
+   on it.** TASK_1902 (rune `rune_251db375b26045f5a4ca9fc762653c89`) found the logistic calibration's
+   `theta=0.7` was fitted to a claimed hit band (EFE 0.36-0.43) that never existed anywhere in the
+   telemetry (observed minimum was 1.213 from the very first persisted row). That means the
+   calibration constant's own doc comment fabricated a provenance story — a strong structural
+   signal that the touch-probability framing was asserted rather than derived from data at every
+   step downstream of TASK_1579, not just at the calibration layer.
+
+**Implication for requirement 3 (not decided here, just surfaced):** because this was never a
+validated design commitment, "EFE is wrong for this purpose" is not a regression to fix — it is a
+convenience assumption that turned out, once actually measured, to carry ~no discriminative signal
+(AUC 0.4784, CI spanning 0.5). That pushes toward fixing the **signal** (requirement 1(b) in the
+card) rather than treating this as a target-definition error requiring a redesign of what
+`commit_touches_symbol` means — but see the measured alternative below before that call is made.
+
+## 2. Requirement 2 — candidate signals, measured on the same 597-row batch
+
+### 2.1 Why component_breakdown, and not "current" Heat/Co-change
+
+Every `prediction_ledger` row's `component_breakdown` column stores the JSON
+`{g_risk, g_ambiguity, g_novelty, g_salience, complexity_penalty}` **as computed at the moment the
+prediction was persisted** (`efe.rs:1252-1259`, inserted immediately after `softmax_policy`). This
+is the only historically-faithful, per-row signal source available for this exact batch — using it
+requires no re-derivation and cannot leak future information into a past prediction.
+
+Querying the **live** Heat engine or Co-change matrix for these same symbols today would not be a
+valid substitute and was deliberately not done: Heat decays with a 12-hour half-life
+(`heat.rs:35-37`, `HALF_LIFE_SECS = 43200.0`), and the batch spans 2026-06-09 to 2026-08-27 (~80
+days, roughly 160 half-lives for the earliest rows). Reading current heat for a symbol predicted on
+day 1 would return ~0 regardless of what happened at prediction time — that is not "no signal", it
+is "wrong-timestamp data masquerading as the same measurement." Per the task's explicit prohibition
+on synthetic/hand-picked substitution, this was ruled out rather than used to pad the candidate list.
+
+**Consequence — what is and isn't evaluable on this data:**
+
+| Candidate | Evaluable on this batch? | Why |
+|---|---|---|
+| Heat (pure) | **No** | Raw `heat_val` at prediction time was never persisted separately — only folded into `g_risk` alongside risk-rune count and Thalamus fire/potential terms. Re-querying current Heat would use the wrong timestamp (see above). |
+| Co-change (pure) | **No** | Same reason — folded into `g_novelty` alongside rune-recency-since-last-rune and delta-graph-presence. No separate co-change value was persisted per prediction. |
+| Recency (pure, "time since last edit") | **No** | Not persisted as an independent column; only present as one term inside the `g_novelty` composite (`rune_recency_score`) and as a multiplicative factor inside `co_change_novelty` in `efe.rs:1110-1138`. |
+| **g_risk** (heat-dominated composite: `heat_val * 1.5 + risk_rune_count + fire_count*0.5 + potential*0.2`) | **Yes** | Persisted per-row, at prediction time. |
+| **g_novelty** (co-change-dominated composite: `co_change_novelty + rune_recency_score + delta_presence_score`) | **Yes** | Persisted per-row, at prediction time. |
+| g_ambiguity, g_salience, complexity_penalty | **Yes** | Persisted per-row, at prediction time. |
+| confidence (softmax probability of the EFE action) | **Yes** | Persisted per-row. |
+
+So the measured candidates below are the heat-dominated and co-change-dominated **composites that
+actually exist in the data**, not the pure signals the task listed as candidates. This is reported
+as a limitation, not smoothed over — if pure Heat-only or Co-change-only AUC is wanted, it requires
+either replaying `heat.rs`/`co_change.rs` state reconstruction against historical git commits (not
+attempted here — out of scope for a measurement task that must not touch scanner/index/embedding
+code) or changing `persist_efe_predictions` to log the raw components separately going forward
+(a requirement-3-adjacent follow-up, not decided here).
+
+### 2.2 Method
+
+Mann-Whitney U → AUC (rank-based, ties handled via average-rank), on the identical 597-row
+hit/miss split as the EFE baseline. 95% CI via 3000-iteration bootstrap resampling of rows (not just
+the statistic). Significance via 3000-iteration label-permutation test, two-sided on
+`|AUC − 0.5|`. Best achievable single-threshold accuracy computed by scanning every unique value as
+a candidate threshold in both directions, per the same method the main thread used for the EFE
+baseline, so it is comparable.
+
+### 2.3 Results table
+
+| Signal | n | hit/miss | majority baseline | AUC | 95% CI (bootstrap) | permutation p | hit vs miss median |
+|---|---|---|---|---|---|---|---|
+| predicted_value (raw EFE, control) | 597 | 88/509 | 85.26% | **0.4784** | [0.4134, 0.5450] | 0.5295 | 2.757 vs 2.708 |
+| confidence (softmax prob.) | 597 | 88/509 | 85.26% | 0.5027 | [0.4313, 0.5735] | 0.9334 | 0.00099 vs 0.00086 |
+| **g_risk** (heat-dominated composite) | 597 | 88/509 | 85.26% | **0.6725** | **[0.6268, 0.7191]** | **0.0003** | 1.521 vs 1.430 |
+| g_novelty (co-change-dominated composite) | 597 | 88/509 | 85.26% | **0.3391** | [0.2803, 0.4004] | 0.0003 | 1.800 vs 1.800 |
+| g_ambiguity | 597 | 88/509 | 85.26% | 0.4397 | [0.4024, 0.4822] | 0.0137 | 0.300 vs 0.300 |
+| g_salience (inbound CALLS, normalized) | 597 | 88/509 | 85.26% | 0.3104 | [0.2635, 0.3599] | 0.0003 | 0 vs 0.05 |
+| complexity_penalty | 597 | 88/509 | 85.26% | 0.4089 | [0.3442, 0.4745] | 0.0073 | 1.141 vs 1.209 |
+| g_risk + g_novelty (sum) | 597 | 88/509 | 85.26% | 0.5235 | [0.4636, 0.5826] | 0.4738 | 3.430 vs 3.320 |
+
+Best single-threshold accuracy for `g_risk` (the only signal whose CI excludes 0.5): **85.59%** at
+threshold ≥ 4.636, vs the 85.26% majority baseline — a gain of **+0.33 percentage points**. This is
+a much smaller edge than the AUC of 0.67 might suggest, and that gap is itself informative (see
+2.4 below) — it is reported here, not left out, per the "AUC alone is not enough" requirement.
+
+### 2.4 Reading the results
+
+- **`g_risk` is the one signal with a real, non-chance relationship to `is_hit` on this data.**
+  AUC 0.6725 with a 95% CI of [0.6268, 0.7191] that clears 0.5 by a wide margin, and a permutation
+  p = 0.0003 (3/3000 shuffles were as extreme, occurring by chance well under 1-in-1000). Since
+  `g_risk = heat_val * 1.5 + risk_rune_count + fire_count*0.5 + potential*0.2` and for most symbols
+  the risk-rune and Thalamus terms are zero (only symbols with active risk runes or recent Thalamus
+  firing get non-zero contributions there), this is a heat-dominated signal — evidence points at
+  **Heat** as the likely carrier of most of this, but this report cannot cleanly attribute the
+  effect size between heat_val and the other two additive terms because they were never persisted
+  separately (see 2.1). Calling this "Heat has AUC 0.67" would overclaim what was actually measured;
+  the accurate claim is "the heat-dominated G_risk composite has AUC 0.67."
+- **`g_novelty` (co-change-dominated) is significantly *below* 0.5 (AUC 0.339, CI [0.28, 0.40]).**
+  That is real signal too, just inverted: symbols scoring high on the novelty composite (long since
+  last rune, recently appeared in ΔGraph, high co-change×recency product) were *less* likely to be
+  the ones actually touched in this 597-row sample, not more. This is worth flagging rather than
+  discarding — an inverted-but-real signal is still usable (e.g. as `1 - g_novelty` or with a
+  negative weight) and its significance is just as strong as g_risk's (p = 0.0003).
+- **`g_salience` (inbound CALLS) is also significantly inverted** (AUC 0.310): heavily-called hub
+  symbols were less likely to be the touched ones in a given commit than symbols with few callers.
+  Plausible mechanically — hub functions get read/depended-on constantly but edited comparatively
+  rarely, while edits cluster on leaf/local code — but this report only measured the correlation,
+  not the mechanism.
+- **AUC and single-threshold accuracy tell different stories under this class imbalance (85/15).**
+  `g_risk`'s AUC of 0.67 says it ranks hits above misses substantially more often than chance across
+  the whole distribution. Its threshold-accuracy edge (+0.33pp over majority baseline) is small
+  because, at 85/15 imbalance, almost any single threshold still classifies most rows as "miss" and
+  gets them right by base rate — accuracy is a poor summary statistic here, which is exactly why the
+  card and this report lead with AUC/CI/permutation-p rather than accuracy. If threshold-based
+  action (e.g. an alert/gate) is ever built on this signal, precision/recall or a cost-weighted
+  metric would be the right next measurement — not attempted here, out of scope.
+- **`confidence` and the `g_risk+g_novelty` sum both land indistinguishable from chance** (CIs
+  spanning 0.5, p > 0.47) — the opposing signs of g_risk and g_novelty cancel when summed naively,
+  which is a data point against any predictor that blends them additively with equal, unlearned
+  weight (which is structurally close to what raw EFE already does).
+
+## 3. Direction the evidence points (recommendation, not decision — requirement 3 reserved)
+
+The measured evidence favors **requirement-3 option (b): keep the touch-prediction target, replace
+the signal.** Concretely:
+
+- Raw EFE (`predicted_value`) carries no measurable relationship to `is_hit` (CI spans 0.5,
+  p = 0.53) — consistent with requirement 1's finding that EFE was never validated for this role.
+- A signal *does* exist in the same data and same time window: the heat-dominated `g_risk`
+  composite reaches AUC 0.67 with a tight, 0.5-excluding CI and p = 0.0003 — this is not a
+  borderline result at n=597.
+- A second, inverted signal (`g_novelty` / co-change-dominated, and separately `g_salience`) is
+  also significant, meaning there is more than one real gradient in this data; a rebuilt predictor
+  combining them with fitted (not naive-summed) weights and correctly-signed contributions is
+  plausible, though not proven here — this report measured single signals and one naive sum, not a
+  fitted combination.
+- This recommendation is explicitly conditional on committing to persist `heat_val` and the
+  `co_change_novelty` term (or the raw co-change matrix lookup) as their own columns going forward,
+  since the current composites conflate multiple sources and cannot be cleanly attributed
+  after the fact (2.1, 2.4). Whether that's worth doing — and whether options (a) or (c) might be
+  preferable for other reasons this measurement doesn't cover (e.g. architectural cost, whether
+  `commit_touches_symbol` is even the right question P3 should be asking) — is the human decision
+  requirement 3 reserves.
+
+## 4. Reproducibility
+
+Query and script used: `node:sqlite` `DatabaseSync` in read-only mode against
+`.synapse/db/memory.db`, `prediction_ledger` table, same WHERE clause as the anchor baseline in §0.
+No rows were filtered, hand-picked, or excluded — all 597 qualifying rows were used for every
+signal (0 dropped for missing component_breakdown fields in this run). Script location (scratchpad,
+not part of the codebase): `auc_measure.js`, run via
+`node auc_measure.js` against the live DB path.

package/core/dev/TASK_1904_BRANCH_INTEGRITY_AUDIT.md

@@ -0,0 +1,146 @@
+# TASK_1904 — Branch Integrity Audit: 25 completed-but-unmerged `ansuz-wt-*` branches
+
+Date: 2026-08-27
+Scope: all `ansuz-wt-*` branches **not** part of the current session (`ansuz-wt-05a7a95e-*`) that still have commits absent from `main` (`git rev-list --count main..<branch> > 0`).
+
+## 0. Correction to the card's starting numbers
+
+The card's technical_notes state 24 branches across families `e989b14a×11, f70a9905×8, 68ed9c57×3, 57925de4×2, a911c576×1, 4ffc45c7×1`. Re-measuring directly from git state at audit time:
+
+| family | card said | actual (`git rev-list --count main..branch` > 0) |
+|---|---|---|
+| e989b14a | 11 | **10** |
+| f70a9905 | 8 | 8 |
+| 68ed9c57 | 3 | 3 |
+| 57925de4 | 2 | 2 |
+| a911c576 | 1 | 1 |
+| 4ffc45c7 | 1 | 1 |
+| **total** | **26 (card's own arithmetic: 29−4=25, but the six numbers sum to 26)** | **25** |
+
+The repo currently has 57 `ansuz-wt-*` branches, not 56. Likely explanation: branch state moved between the card being written and this audit running (a branch was GC'd, or the card's count had a one-off arithmetic slip — the six family numbers it lists sum to 26, not the 24 the card concludes). This audit proceeds on the **measured 25**, verified per-branch below, per the instruction to report discrepancies rather than silently conform. All 25 are non-current-session, non-fully-merged branches.
+
+Only **1 of the 25** (`ansuz-wt-4ffc45c7-13104`) still has a live worktree attached (`git worktree list` confirms). The other 24 are bare orphaned branch refs.
+
+## 1. Method
+
+For each branch, for each file in `git diff main...<branch> --name-only` (full paths, **not** truncated `--stat` output — see §5 methodology note):
+
+1. `git cat-file -e main:<path>` → exists in main?
+2. If exists: `git diff main <branch> -- <path>` empty → **byte-identical**, classify (a).
+3. If exists but differs: extract every non-trivial (≥12 char) line the branch *added* relative to `git merge-base main <branch>`, and check what fraction is present verbatim anywhere in main's current file. High match (≥90%) → main is a strict superset (branch content + later additions). Lower match was individually read in full (`git diff <blob> <blob>`) and cross-checked for a `TASK_<id>` comment trail in main's current source attributing the surviving logic back to the same task.
+4. If the file doesn't exist in main at all: classify (b) throwaway artifact (with per-file reasoning) or (c) real loss.
+
+## 2. Control group (method validation)
+
+Sampled 5 of the 27 fully-merged branches:
+
+| branch | unmerged commits | files differing from main |
+|---|---|---|
+| ansuz-wt-595952d1-22448 | 0 | 0 |
+| ansuz-wt-68ed9c57-5512 | 0 | 0 |
+| ansuz-wt-7ae937aa-11992 | 0 | 0 |
+| ansuz-wt-8026f9cb-12972 | 0 | 0 |
+| ansuz-wt-eebaa183-10776 | 0 | 0 |
+
+All 5 report (a)/no-diff as expected. Method is not misclassifying merged branches as unmerged — **proceeding**.
+
+## 3. Critical methodology correction found mid-audit
+
+`git diff --stat` truncates long paths with a leading `...`. A naive reconstruction of the truncated prefix (e.g. assuming `.../http_server/routes/...` meant `dashboard/src-tauri/src/http_server/...`) is **wrong** here: the whole `http_server` tree was relocated to `dashboard/src-tauri/src/infrastructure/http_server/` at some point after several of these branches diverged. Reconstructing paths from truncated `--stat` output falsely reported `graph.rs`, `memory.rs`, `kenaz.rs`, `dashboard_settings.rs`, and `ModelSelectionPanel.tsx` as **not in main**, when in fact they exist at the moved path with the branches' content already merged into them. Switching to `git diff main...<branch> --name-only` (untruncated) fixed this. This is the same failure class as `feedback_line_grep_underreports_multiline_calls` — an instrument's default truncation silently produced a false "missing" verdict. Flagging per the card's instruction ("回報而非續行") even though the fix was straightforward here.
+
+## 4. Per-branch classification
+
+Legend: **(a)** content already in main (byte-identical or main is a verified superset) → safe to delete. **(b)** file legitimately absent, non-shippable artifact, reason given. **(c)** real loss, needs human decision.
+
+### Family e989b14a (10 branches) — all (a)/(b), 0 (c)
+
+| branch | task | files | classification |
+|---|---|---|---|
+| e989b14a-10164 | TASK_1818 | `.formation/tools/post-complete/git-merge.js` (DIFF, 100% match), `.kenaz/specialist/daemon-manifest.yaml` (DIFF, 92% match — remaining 8% verified: main §5/89 explicitly comments `TASK_1818, TASK_1827`), progress_note (a-SAME) | (a) — verified via TASK_1818 comment trail in main |
+| e989b14a-13448 | TASK_1819 | `.formation/tools/headless.js` (a-SAME), `.formation/tools/prompt.js` (76%), `.formation/unit/do.yaml` (100%), `.rule/general/system-invariants.yaml` (100%), progress_note (a-SAME) | (a) |
+| e989b14a-13892 | TASK_1814 | progress_note (a-SAME), `.specialist/Ansuz/elixir.yaml` (100%), `dashboard_settings.rs` (a-SAME at relocated path), `ModelSelectionPanel.tsx` (a-SAME, exists at `dashboard/src/components/model-selection/...` — path simplified, not lost) | (a) — the two files that looked MISSING were a truncation artifact (§3) |
+| e989b14a-16432 | TASK_1824 | `.formation/tools/dispatch.js` (13% raw match — **verified**: TASK_1824's actual change is a pure reordering of the existing `isCrossProject` computation; main's dispatch.js at a later commit carries the explicit comment `// TASK_1824 ordering still holds: computed BEFORE buildPrompt`), `prompt.js` (88%), progress_note (a-SAME) | (a) — verified by reading full diff, not just percentage |
+| e989b14a-19500 | TASK_1821 | `prompt.js` (100%), `do.yaml` (a-SAME), progress_note (a-SAME) | (a) |
+| e989b14a-20476 | TASK_1816 | bifrost `launch.js`/`models.json` (a-SAME), `relay.js` (98%), `runner.js` (100%), progress_note (a-SAME) | (a) |
+| e989b14a-22204 | TASK_1826 | `.claude/hooks/worktree-create.js` (39% raw match — **verified**: full diff read shows TASK_1826's GC-husk-handling logic; main carries 5 separate `TASK_1826` comment anchors including `isOwnWorktree`), `gc1826_driver.js` (a-SAME — see note below), progress_note (a-SAME) | (a) — verified by reading full diff |
+| e989b14a-24508 | TASK_1815 | test file (a-SAME), `dispatch.js` (96% — the missing 4%: verified this is `translateModelForAgentTool`/`AGENT_TOOL_MODEL_ENUM`, present verbatim in main under an explicit `TASK_1815:` comment block), `elixir.yaml` (100%), progress_note (a-SAME) | (a) |
+| e989b14a-26040 | TASK_1822 | `dispatch.js` (100%), progress_note (a-SAME) | (a) |
+| e989b14a-6576 | TASK_1813 | test file (98%), `dispatch.js` (94%), progress_note (a-SAME) | (a) |
+
+### Family f70a9905 (8 branches) — all (a), 0 (c)
+
+| branch | task | files | classification |
+|---|---|---|---|
+| f70a9905-25392 | TASK_1774 | `models.json` (75% raw — **verified**: the added `gpt-5.6-sol` model entry is present in main verbatim, with main's note explicitly citing `TASK_1774 repro control group` as its origin, later extended by TASK_1816), `headless.js` (100%), `runner.js` (100%), progress_note (a-SAME) | (a) — verified |
+| f70a9905-27532 | TASK_1778 | `task-api.js` (100%), progress_note (a-SAME) | (a) |
+| f70a9905-28040 | TASK_1780 | `launch.js`/`relay.js` (100%), `setup-codex.js` (a-SAME), `.kenaz/project.yaml` (a-SAME), progress_note (a-SAME) | (a) |
+| f70a9905-28904 | TASK_1784 | `prompt.js` (85%), progress_note (a-SAME) | (a) |
+| f70a9905-34388 | TASK_1785 | `formations.js`/`git-merge.js`/`runner.js` (100%), `git-salvage.js`/`pre-dispatch-snapshot.test.js` (a-SAME), progress_note (a-SAME) | (a) |
+| f70a9905-36780 | TASK_1776 | `dashboard_settings.rs` (100% at relocated path), `kenaz.rs` (100% at relocated path — both looked MISSING before the §3 fix), progress_note (a-SAME) | (a) |
+| f70a9905-4312 | TASK_1775 | `formations.js` (76%), `utils.js` (97%), progress_note (a-SAME) | (a) |
+| f70a9905-5348 | TASK_1777 | hooks (100%/a-SAME), `prompt.js` (100%), progress_note (a-SAME), `t1777_probe.txt` (a-SAME — a one-line probe file landed as accidental clutter, see §6) | (a) |
+
+### Family 68ed9c57 (3 branches) — (a) code + (b) artifacts, 0 (c)
+
+| branch | task | files | classification |
+|---|---|---|---|
+| 68ed9c57-11088 | TASK_1827 | `worktree-create.js` (91%), `git-merge.js` (100%), `daemon-manifest.yaml` (100%) → **(a)**. `gc1827_driver.js` → **(b)**: commit `a953fbb88` is titled *"verification driver — all throwaway-repo scenarios pass"* in its own subject line — self-declared as a driver, not shipped code. `progress_note_TASK_1827.json` → **(b)**: progress-note artifact. | (a) code / (b) artifacts, no loss |
+| 68ed9c57-12936 | TASK_1825 | `post-complete.js` (100%), `git-merge.js` (100%) → (a). `post-complete/verify-acceptance.js` (old path, shows MISSING) + `verify-acceptance.js` (new path, a-SAME) → the same commit's diff --stat shows `486 ---` deletions at the old path and a new 68-line file at the new path: **TASK_1825 itself relocated this file** (from `.formation/tools/post-complete/` to `.formation/tools/`); the new path is present and unchanged in main → **(a) landed-via-move**, not a loss. `progress_note_TASK_1825.json` → **(b)** artifact. | (a), no loss |
+| 68ed9c57-6484 | TASK_1828 | `post-complete.js`/`git-merge.js` (100%), `elixir.yaml`/`ansuz-handoff-snapshot.js` (100%), `kenaz.rs` (100% at relocated path — looked MISSING before §3 fix) → (a). `progress_note_TASK_1828.json` → **(b)** artifact. | (a), no loss |
+
+### Family 57925de4 (2 branches) — (a) code + (b) artifacts, 0 (c)
+
+| branch | task | files | classification |
+|---|---|---|---|
+| 57925de4-15660 | TASK_1843 | `task-api.js` (5% raw match — **verified by full read**: the branch's `resolveStatusDirName()` fix is present in main as `_resolveStatusDirName()`, explicitly commented `// 從檔案路徑推斷狀態（TASK_1843）` and shared with a follow-up TASK_1846/TASK_1851 as a common helper — rewritten, not copied, hence the low raw-line match) → **(a)**. `repro_task_1843_status_divergence.js` → **(b)**: the commit message itself calls this "repeatable repro w/ control group" — a diagnostic script, not the shipped fix. `progress_note_TASK_1843.json` → **(b)** artifact. | (a), no loss |
+| 57925de4-24664 | TASK_1842 | `semantic.rs` (35%) + `memory.rs` (29%) — **verified by full read**: both files carry explicit `// TASK_1842:` comment blocks in main (`semantic_arm_ran`, `search_mode_effective`, `search_mode_requested` all present verbatim) — rewritten, not copied. No progress_note was committed on this branch (only 1 commit, no separate note commit). | (a), no loss |
+
+### Family a911c576 (1 branch) — (a), 0 (c)
+
+| branch | task | files | classification |
+|---|---|---|---|
+| a911c576-15628 | TASK_1875 | `dashboard/src-tauri/src/lib.rs` (a-SAME, byte-identical) → (a). `progress_note_TASK_1875.json` → **(b)** artifact (never landed). | (a), no loss |
+
+### Family 4ffc45c7 (1 branch, still worktree-bound) — (a), 0 (c)
+
+| branch | task | files | classification |
+|---|---|---|---|
+| 4ffc45c7-13104 | TASK_1877 | `background.rs` (a-SAME), `graph.rs` (99% at relocated path — looked MISSING before §3 fix) → (a). `progress_note_TASK_1877.json` → **(b)** artifact. **This branch still has a live worktree** (`C:/Users/user/Documents/GitHub/.kenaz-worktrees/KenazAI/wt-4ffc45c7-13104`) — do not touch it; worktree deletion is out of scope and forbidden by this card. | (a), no loss |
+
+## 5. Result: 0 real losses found
+
+**All 25 branches classify (a) — their functional code content is verifiably present in main — with a handful of (b) throwaway artifacts (progress notes, one verification driver, one repro script) that never landed and correctly should not have.** No branch has a `(c)` file where main lacks the substance of the change. The card's own sample finding — "TASK_1827's `gc1827_driver.js` not in main" — is confirmed accurate but is a **(b)**, not a **(c)**: TASK_1827's actual shipped code (worktree-create.js GC logic) is fully present in main; only its self-declared verification driver is absent, exactly as intended.
+
+No files or branches are recommended for merge or deletion by this audit — that action is explicitly out of scope per the card; this is a report for a human decision.
+
+## 6. Causal analysis (requirement 3)
+
+For all 25 branches: zero has a git merge commit into main, and none is an ancestor of `main` — yet the functional content of every one is present in main. This is only possible via **non-merge integration**: the main thread (Ansuz) wrote the finished content into main's files directly, without ever running `git merge` on the worktree branch, leaving the branch permanently orphaned regardless of task status.
+
+Two sub-patterns of this same root cause, distinguished by whether the branch's own `progress_note_TASK_XXXX.json` also landed in main as an incidental byproduct:
+
+- **Direct file-scope copy** (progress note present in main too, usually byte-identical to the branch): **18 of 25 branches** — all 10 of e989b14a, all 8 of f70a9905. High raw substring match (75–100%) on almost every file. The progress-note artifact riding along as clutter is itself evidence the mechanism was "copy the finished worktree files into main," not a deliberate content-only merge.
+- **Re-derivation under the same task label** (progress note absent from main; content present but restructured/rewritten, lower raw match, always recoverable via an explicit `TASK_<id>` comment trail in main's current source): **7 of 25 branches** — TASK_1877, TASK_1843, TASK_1842, TASK_1827, TASK_1825, TASK_1828, TASK_1875 (the 4ffc45c7 / 57925de4 / 68ed9c57 / a911c576 families). Here the main thread appears to have re-typed or regenerated the fix directly in the main tree (crediting the same task ID in comments) rather than copying the worktree's files.
+
+Mapping onto the card's four candidate mechanisms:
+
+| mechanism | branches | evidence |
+|---|---|---|
+| (i) post-complete false-positive merge report | **0 confirmed** | No dispatch-log or post-complete output was found for any of the 25 tasks (searched `.formation/` for `TASK_1843`, `TASK_1826` — no merge-decision records survive). Cannot confirm or rule out that post-complete *also* misreported `merged=true` on top of the file-scope copy; there is no evidence either way, so this is reported as unconfirmed rather than assumed. |
+| (ii) file-scope migration, branch not deleted | **25 of 25 (100%)** | Every branch's content is in main with zero git merge — the defining signature of this mechanism, whether by literal copy (18) or same-task-ID re-derivation (7, see above). |
+| (iii) progress notes / throwaway artifacts that should not land | **explains the missing FILES on 7 branches** (the `(b)`-classified ones: 6 progress notes + 1 driver + 1 repro script), but is not itself why the BRANCH is unmerged | Confirmed by direct textual evidence: commit subjects self-label ("verification driver", "repro w/ control group"), and the file's absence has zero functional consequence since the real fix landed elsewhere in the same files. |
+| (iv) other | **0** | No branch's non-landing was attributable to a cause outside (ii)/(iii). |
+
+**Conclusion: (ii) file-scope migration without branch deletion is the dominant and effectively the *sole* observed cause (25/25), with (iii) as a secondary explanation for which specific files are the ones left behind.** (i) could not be confirmed or excluded from available records.
+
+## 7. Recommendation (for human decision — not executed by this audit)
+
+Given 0 `(c)` findings, all 25 branches appear safe to delete once a human confirms this audit's per-file evidence. `ansuz-wt-4ffc45c7-13104` additionally requires its worktree removed through the normal (non-manual) worktree-lifecycle path before/along with branch deletion — this audit does not touch it.
+
+Secondary, lower-priority observation (not a data-loss finding): `gc1826_driver.js` and `t1777_probe.txt` are throwaway-looking artifacts that landed in main anyway (byte-identical, confirmed `(a)`) as a side effect of the file-scope-copy mechanism. They cost nothing functionally but are dead weight; worth a separate lightweight cleanup task if a human agrees they don't belong in main.
+
+## 8. Files referenced by this audit (absolute paths)
+
+- Task card: `C:\Users\user\Documents\GitHub\KenazAI\.kenaz\tasks\backlog\TASK_1904_24_個標記完成的任務_其_worktree_分支仍握著_main_沒有的_co_xyz951610.yaml`
+- This report: `C:\Users\user\Documents\GitHub\.kenaz-worktrees\KenazAI\wt-05a7a95e-37916\dev\TASK_1904_BRANCH_INTEGRITY_AUDIT.md`
+- Sample task cards read for causal evidence: `C:\Users\user\Documents\GitHub\KenazAI\.kenaz\tasks\archived\TASK_1843_*.yaml`