kenaz 0.3.22 → 0.3.24

package/core/dev/archive/ansuz/investigations/reachability_unified_api_2026-04-11.md

@@ -1,330 +0,0 @@
-# Reachability Unified API — Consumer Audit & Design Proposal
-
-**Task**: TASK_1183  
-**Date**: 2026-04-11  
-**Analyst**: unit agent (backend specialist)  
-**Depends on**: TASK_1181 (Phase 1: INVOKES_COMMAND + ROUTES_TO guards in orphans_impl)  
-**Scope**: `dashboard/src-tauri/src/infrastructure/http_server/routes/mcp_tools/synapse.rs`,  
-`dashboard/src-tauri/src/features/synapse/`
-
----
-
-## Consumer audit
-
-### Methodology
-
-Each consumer was read from source to extract: (a) which edge types it queries, (b) whether it uses NOT EXISTS / EXISTS / count / BFS / adjacency-list over those edges, (c) what confidence threshold it applies (if any), and (d) what semantic claim it is making about reachability.
-
-### Consumer inventory
-
-**C1 — `orphans_impl` / `orphan_cypher_query()`**  
-File: `mcp_tools/synapse.rs:5182–5192`  
-Purpose: Identify function/method symbols with no inbound reachability path.  
-Query (post-TASK_1181 Phase 1):
-```cypher
-MATCH (s:Symbol)
-WHERE s.kind IN ['function', 'method']
-AND NOT EXISTS { MATCH (:Symbol)-[:CALLS]->(s) }
-AND NOT EXISTS { MATCH (:File)-[:EXPORTS]->(s) }
-AND NOT EXISTS { MATCH (:Symbol)-[:INVOKES_COMMAND]->(s) }
-AND NOT EXISTS { MATCH (:Symbol)-[:ROUTES_TO]->(s) }
-RETURN s.id, …
-```
-Post-filter: excludes `STEP_IN_PROCESS` entry-point IDs (loaded separately).  
-Confidence filter: **none** (all CALLS edges regardless of confidence).  
-Semantic claim: "No caller, not exported, not a Tauri command target, not an HTTP route handler."
-
----
-
-**C2 — `synapse_context` (symbol mode, callers)**  
-File: `features/synapse/context.rs:138–161`  
-Purpose: Show explicit callers of a symbol for blast-radius analysis.  
-Query:
-```cypher
-MATCH (caller:Symbol)-[:CALLS]->(target:Symbol)
-WHERE target.name = '<name>'
-RETURN caller.name, caller.kind, caller.file_path, …
-```
-Depth > 1 uses `CALLS*1..N`.  
-Also augments with `LINKS_TO` (document domain — separate query appended to callers list).  
-Confidence filter: **none**.  
-Semantic claim: "Symbols that explicitly CALL this symbol." Does NOT check EXPORTS, INVOKES_COMMAND, or ROUTES_TO.
-
----
-
-**C3 — `community_detector` (code pass)**  
-File: `features/synapse/community_detector.rs:79`  
-Purpose: Partition symbols into communities for coupling analysis.  
-Query:
-```cypher
-MATCH (a:Symbol)-[:CALLS]->(b:Symbol) RETURN a.id, b.id
-```
-Separate LINKS_TO pass for note/document communities.  
-Confidence filter: **none**.  
-Semantic claim: "Structural connectivity for Louvain partitioning." Reachability is not the goal — all CALLS edges participate equally. Orphaned symbols (no CALLS in or out) simply never appear in the adjacency list and receive no community assignment.
-
----
-
-**C4 — `process_detector` (entry point detection)**  
-File: `features/synapse/process_detector.rs:163`  
-Purpose: Find execution entry points (in_degree=0 among CALLS participants) and trace BFS chains.  
-Query:
-```cypher
-MATCH (caller:Symbol)-[r:CALLS]->(callee:Symbol)
-WHERE r.confidence IS NULL OR r.confidence >= 0.5
-RETURN caller.id, callee.id
-```
-Entry points: symbols with `in_degree = 0` in the CALLS subgraph above.  
-Confidence filter: **≥ 0.5 (or NULL for legacy edges)** — the only consumer that applies a confidence threshold.  
-Semantic claim: "Symbol is a root of an execution chain (no higher-level CALLS caller)." Does NOT check EXPORTS, INVOKES_COMMAND, or ROUTES_TO for entry-point classification.  
-**Drift note**: A Tauri handler has `CALLS in_degree = 0`, so process_detector treats it as an entry point. orphans_impl correctly excludes it via INVOKES_COMMAND. These two consumers agree on "no CALLS inbound" but disagree on whether the symbol is "orphaned" — a latent semantic mismatch.
-
----
-
-**C5 — `search.rs` graph_boost / `compute_in_degrees`**  
-File: `features/synapse/scanner.rs:2836`, `search.rs:974`  
-Purpose: Rank search results by symbol popularity.  
-Query (populate `symbol_embeddings.in_degree`):
-```cypher
-MATCH (caller:Symbol)-[:CALLS]->(callee:Symbol)
-RETURN callee.id, count(caller) AS in_deg
-```
-Used as: `(1.0 + in_degree).ln() / (1.0 + max_in_degree).ln()`  
-Confidence filter: **none**.  
-Semantic claim: "Popularity = raw CALLS in-degree." Does NOT account for EXPORTS, INVOKES_COMMAND, or ROUTES_TO. A Tauri handler with heavy frontend usage has `in_degree = 0` → zero graph_boost.
-
----
-
-**C6 — Guardian health score `orphan_ratio` (15% weight)**  
-File: `mcp_tools/synapse.rs:5640–5658`  
-Purpose: Compute codebase health metric — fraction of orphaned functions/methods.  
-Query:
-```cypher
-MATCH (s:Symbol)
-WHERE s.kind IN ['function', 'method']
-AND NOT EXISTS { MATCH (:Symbol)-[:CALLS]->(s) }
-AND NOT EXISTS { MATCH (:File)-[:EXPORTS]->(s) }
-RETURN count(s)
-```
-Confidence filter: **none**.  
-Semantic claim: "Symbol has no inbound CALLS and no EXPORTS." **Frozen at the pre-Phase-1 definition** — does NOT include INVOKES_COMMAND or ROUTES_TO. This means every Tauri handler and HTTP route handler is counted as orphaned in the health score, even though orphans_impl correctly excludes them since TASK_1181.  
-**This drift is LIVE in production.**
-
----
-
-**C7 — EFE `orphan_ratio` approximation**  
-File: `features/synapse/efe.rs:549–555`  
-Purpose: Curiosity-driven exploration priority (EFE = Expected Free Energy).  
-Mechanism: Not a graph query. Approximated via prediction_map — symbols absent from Curiosity prediction data get a neutral orphan signal (`ORPHAN_WEIGHT` constant). No edge-type queries at all.  
-Semantic claim: "Symbol has no curiosity prediction data." Completely decoupled from graph reachability. Included for completeness but not a drift site for the reachability unification problem.
-
----
-
-**C8 — `attention.rs` pain-path BFS**  
-File: `features/synapse/attention.rs:554–562`  
-Purpose: Compute BFS depth from each symbol along CALLS edges (used in pain-path scoring).  
-Query:
-```cypher
-MATCH (a:Symbol)-[:CALLS]->(b:Symbol) RETURN a.id, b.id
-```
-Confidence filter: **none**.  
-Semantic claim: "How deep is this symbol's call chain?" Reachability from the symbol's perspective (forward BFS). Not checking orphan status — not a drift site.
-
----
-
-### Drift matrix
-
-Rows = consumers. Columns = edge types relevant to reachability. Cells:  
-`Q` = queries this edge, `–` = does not query, `F(≥0.5)` = queries with confidence filter.
-
-| Consumer | CALLS (inbound) | EXPORTS | INVOKES_COMMAND | ROUTES_TO | LINKS_TO | Conf. filter | Semantic claim |
-|----------|----------------|---------|-----------------|-----------|----------|--------------|----------------|
-| C1 orphans_impl | Q (NOT EXISTS) | Q (NOT EXISTS) | Q (NOT EXISTS) | Q (NOT EXISTS) | – | none | "unreachable symbol" |
-| C2 synapse_context callers | Q (count/list) | – | – | – | Q (doc only) | none | "explicit CALLS callers" |
-| C3 community_detector | Q (all edges) | – | – | – | Q (separate) | none | "graph connectivity" |
-| C4 process_detector | Q (in_degree=0 → entry) | – | – | – | – | ≥0.5 or NULL | "execution entry point" |
-| C5 search graph_boost | Q (in_degree count) | – | – | – | – | none | "symbol popularity" |
-| C6 guardian health orphan_ratio | Q (NOT EXISTS) | Q (NOT EXISTS) | – | – | – | none | "unreachable symbol" **← STALE** |
-| C7 EFE orphan_ratio | – (approx) | – | – | – | – | N/A | "missing curiosity data" |
-| C8 attention BFS | Q (forward depth) | – | – | – | – | none | "call chain depth" |
-
-### Key drift sites
-
-**Drift Site 1 (CRITICAL — already live)**: C6 vs C1  
-Guardian health orphan_ratio uses `CALLS + EXPORTS` while orphans_impl (post-TASK_1181) uses `CALLS + EXPORTS + INVOKES_COMMAND + ROUTES_TO`. A Tauri `#[tauri::command]` handler appears as orphaned in the health score but not in the orphans tool. This causes the health score's `orphan_count` to be inflated vs what the user observes in the orphans tool.
-
-**Drift Site 2 (LATENT)**: C4 vs C1  
-process_detector treats a Tauri handler as an entry point (CALLS in_degree = 0, participates in CALLS graph only if it calls something). orphans_impl treats it as reachable (via INVOKES_COMMAND). If a Tauri handler calls other functions, it appears in both as an entry point in one context and as "reachable" in the other — consistent for now. But if Phase 2 adds more IPC edge types, process_detector's entry-point definition will silently diverge.
-
-**Drift Site 3 (MODERATE)**: C5 vs C1  
-search graph_boost uses only CALLS in_degree. A Tauri handler with `in_degree=0` (no CALLS inbound) gets zero graph_boost. This means frontend-heavy handlers are ranked poorly in search despite being highly "reached" from the TS layer. This will worsen as INVOKES_COMMAND edges populate.
-
-**Drift Site 4 (INFORMATIONAL)**: C2 vs C1  
-synapse_context reports "0 callers" for symbols with EXPORTS but no CALLS. The user may see a symbol that is not orphaned (has EXPORTS) but appears to have no callers in context. Not technically a bug — context answers a different question ("who explicitly calls me?") — but the UX is confusing without a note.
-
----
-
-## Proposal A: Rust helper `is_reachable`
-
-### Design
-
-New file: `dashboard/src-tauri/src/features/synapse/reachability.rs`
-
-```rust
-/// Canonical reachability check for a single symbol.
-/// All consumers that need to determine "is this symbol reachable?" should call this.
-pub enum ReachabilityMode {
-    /// Strict: only explicit CALLS inbound (legacy pre-Phase-1 behavior).
-    Strict,
-    /// Standard: CALLS + EXPORTS + INVOKES_COMMAND + ROUTES_TO (post-Phase-1).
-    Standard,
-    /// Loose: Standard + LINKS_TO (useful for mixed code+doc consumers).
-    Loose,
-}
-
-/// Returns true if the symbol has at least one inbound reachability edge
-/// matching the requested mode.
-pub fn is_reachable(
-    conn: &kuzu::Connection,
-    symbol_id: &str,
-    mode: ReachabilityMode,
-) -> Result<bool, String> { … }
-
-/// Returns the Cypher NOT EXISTS predicate fragment for bulk queries.
-/// Use this when checking many symbols in a single MATCH, to avoid N+1 queries.
-pub fn unreachable_predicate(mode: ReachabilityMode) -> &'static str { … }
-```
-
-`unreachable_predicate(ReachabilityMode::Standard)` would return the same predicate string that `orphan_cypher_query()` currently builds inline:
-```
-AND NOT EXISTS { MATCH (:Symbol)-[:CALLS]->(s) }
-AND NOT EXISTS { MATCH (:File)-[:EXPORTS]->(s) }
-AND NOT EXISTS { MATCH (:Symbol)-[:INVOKES_COMMAND]->(s) }
-AND NOT EXISTS { MATCH (:Symbol)-[:ROUTES_TO]->(s) }
-```
-
-### Migration plan for each consumer
-
-| Consumer | Change | Mode to use |
-|----------|--------|-------------|
-| C1 orphans_impl | Replace inline Cypher predicate with `unreachable_predicate(Standard)` | Standard |
-| C6 guardian health | Replace inline Cypher predicate with `unreachable_predicate(Standard)` — **fixes Drift Site 1** | Standard |
-| C5 search graph_boost | Optionally augment `compute_in_degrees` with reachability-aware pseudo-degree: add 1 for INVOKES_COMMAND edges | Standard (augmented) |
-| C4 process_detector | No change needed — entry-point detection remains CALLS-only by design. Document explicitly. | Strict (documented) |
-| C2 synapse_context | No change needed — callers list is intentionally CALLS-only for blast-radius semantics | Strict (documented) |
-| C3 community_detector | No change needed — community partitioning is graph-topology, not reachability | N/A |
-
-### Tradeoff
-
-- **Pro**: Zero index-time cost. Changes are purely query-time.
-- **Pro**: Mode enum makes intent explicit and grep-able.
-- **Pro**: `unreachable_predicate()` is a pure function — easy to test.
-- **Pro**: Does not require a schema change. Existing Cypher queries remain valid.
-- **Con**: Consumers must be updated one by one to call the helper. Non-Rust consumers (if any) cannot share it.
-- **Con**: If a new edge type is added (e.g., Phase 3 LINKED_FROM), all callers of `unreachable_predicate(Standard)` need a recompile — but at least they share one definition.
-- **Con**: `is_reachable()` for a single symbol still requires a round-trip to KuzuDB. For bulk queries, `unreachable_predicate()` is more efficient (single MATCH).
-
----
-
-## Proposal B: Virtual `REACHABLE_BY` edge
-
-### Design
-
-At index time (Phase 3 completion hook), compute and insert a virtual `REACHABLE_BY` edge from every "reachable source" to the target symbol. A symbol `s` gets `REACHABLE_BY` edges from every `CALLS`, `EXPORTS`, `INVOKES_COMMAND`, and `ROUTES_TO` source, unified into one edge type.
-
-All consumers then query:
-```cypher
-NOT EXISTS { MATCH (:Symbol|:File)-[:REACHABLE_BY]->(s) }
-```
-
-The `REACHABLE_BY` edge carries a `reason` property (`"CALLS"`, `"EXPORTS"`, `"INVOKES_COMMAND"`, `"ROUTES_TO"`) to allow downstream filtering.
-
-### Schema change required
-
-```
-CREATE REL TABLE REACHABLE_BY (FROM Symbol TO Symbol, reason STRING, source_id STRING)
-CREATE REL TABLE REACHABLE_BY (FROM File TO Symbol, reason STRING, source_id STRING)
-```
-
-KuzuDB requires separate `FROM … TO …` declarations per source node type. This may require two table declarations or a union node type.
-
-### Migration plan
-
-A post-Phase-3 hook runs:
-```cypher
-MATCH (caller:Symbol)-[:CALLS]->(s:Symbol) CREATE (caller)-[:REACHABLE_BY {reason:'CALLS'}]->(s);
-MATCH (f:File)-[:EXPORTS]->(s:Symbol) CREATE (f)-[:REACHABLE_BY {reason:'EXPORTS'}]->(s);
-MATCH (ts:Symbol)-[:INVOKES_COMMAND]->(s:Symbol) CREATE (ts)-[:REACHABLE_BY {reason:'INVOKES_COMMAND'}]->(s);
-MATCH (r:Symbol)-[:ROUTES_TO]->(s:Symbol) CREATE (r)-[:REACHABLE_BY {reason:'ROUTES_TO'}]->(s);
-```
-
-All consumers replace their inline predicates with:
-```cypher
-NOT EXISTS { MATCH ()-[:REACHABLE_BY]->(s) }
-```
-
-### Tradeoff
-
-- **Pro**: Single-token query. All consumers are identical — zero per-consumer migration after schema adoption.
-- **Pro**: Adding a new reachability edge type (e.g., Phase 3 LINKED_FROM) only requires updating the hook, not recompiling consumer code.
-- **Pro**: `REACHABLE_BY` edges can carry metadata (`reason`, `confidence`) surfaced in diagnostics.
-- **Con**: **Index-time cost**: every index run must re-derive and re-insert REACHABLE_BY edges. For large codebases this adds a post-Phase-3 pass of O(E) where E = total CALLS + EXPORTS + INVOKES_COMMAND + ROUTES_TO edges.
-- **Con**: **Schema coupling**: adding the new table to KuzuDB requires a schema migration (schema version bump, potential incompatibility with existing `.synapse/` databases). This is a destructive change for users with existing indexes.
-- **Con**: **Derived data duplication**: REACHABLE_BY edges carry no new information — they are a materialized view of existing edges. Any query that could update source edges (incremental indexing) must also invalidate/update REACHABLE_BY edges. The invalidation logic is non-trivial: when a CALLS edge is removed, the corresponding REACHABLE_BY edge must also be removed.
-- **Con**: **KuzuDB multi-type FROM constraint**: KuzuDB requires `FROM <NodeType>` to be a single type per REL TABLE declaration. Since REACHABLE_BY sources include both `Symbol` (for CALLS, INVOKES_COMMAND, ROUTES_TO) and `File` (for EXPORTS), two separate REL TABLE declarations are needed or a schema workaround (e.g., a `ReachSource` union node). This adds schema complexity.
-- **Con**: Consumers that intentionally need CALLS-only semantics (C4 process_detector, C2 synapse_context) cannot use `REACHABLE_BY` directly and must still query edge-specific tables — meaning both systems must coexist.
-
----
-
-## Tradeoff comparison
-
-| Dimension | Proposal A (Rust helper) | Proposal B (Virtual edge) |
-|-----------|--------------------------|---------------------------|
-| Implementation cost | Low — 1 new file, 2 consumer fixes (C1, C6) | High — schema migration, post-index hook, incremental invalidation |
-| Runtime performance | Same as today (no extra index pass) | Adds O(E) post-Phase-3 pass; queries simplified to single NOT EXISTS |
-| Consistency guarantee | Strong at compile time (shared function) | Strong at index time (materialized) |
-| Flexibility for new edge types | Requires recompile of `reachability.rs` + `unreachable_predicate()` | Only hook update needed |
-| Schema stability | Zero schema changes | Breaking schema change (new REL TABLE + migration) |
-| Incremental index correctness | No new requirements | Must invalidate REACHABLE_BY on every source-edge mutation (complex) |
-| CALLS-only consumers (C2, C4) | Remain CALLS-only by explicit choice — documented | Must bypass REACHABLE_BY, dual-query needed |
-| Urgency (Drift Site 1 fix) | Can fix C6 guardian health in 1 PR | Requires full schema migration first |
-| KuzuDB multi-source FROM constraint | N/A | Needs workaround (2 REL TABLE declarations) |
-
----
-
-## Recommendation
-
-**Adopt Proposal A (Rust helper), with a phased rollout.**
-
-### Rationale
-
-1. **Drift Site 1 (guardian health orphan_ratio) is already live** and causes user-visible inconsistency. Proposal A allows a targeted 2-line fix to C6 without any schema migration or index downtime.
-
-2. **Proposal B's incremental invalidation complexity is unjustified** at current scale. The REACHABLE_BY edges are a derived view that must be kept consistent with four independent source edge types under incremental indexing. The invalidation logic would need to hook into CALLS deletion, EXPORTS deletion, INVOKES_COMMAND deletion, and ROUTES_TO deletion — adding maintenance surface area larger than the problem it solves.
-
-3. **The KuzuDB multi-source FROM constraint** makes Proposal B fragile. The current schema already separates `File → Symbol` (EXPORTS) from `Symbol → Symbol` (CALLS, INVOKES_COMMAND, ROUTES_TO). Unifying them into one REACHABLE_BY edge type requires either a schema hack or two separate tables, defeating the "single-token query" advantage.
-
-4. **CALLS-only consumers are intentional.** C2 (synapse_context) and C4 (process_detector) use CALLS-only semantics by design — they answer "who explicitly calls this?" and "what are execution roots?", respectively. Proposal B cannot serve them, so the two systems must coexist anyway. Proposal A's mode enum documents this explicitly.
-
-5. **Future edge types** (Phase 3: LINKED_FROM for markdown identifier references) can be added to `ReachabilityMode::Standard` in a single place in Proposal A. The compile-time visibility of the change is a feature — it surfaces every consumer that needs updating.
-
-### Recommended implementation task
-
-**TASK_NNNN — Implement `reachability.rs` and fix guardian health orphan_ratio drift**
-
-Scope:
-1. Create `dashboard/src-tauri/src/features/synapse/reachability.rs` with:
-   - `ReachabilityMode` enum (`Strict`, `Standard`, `Loose`)
-   - `unreachable_predicate(mode) -> &'static str` — returns the Cypher NOT EXISTS fragment
-   - `is_reachable_bulk(conn, symbol_ids, mode) -> HashMap<String, bool>` — efficient bulk check
-2. Refactor `orphan_cypher_query()` to call `unreachable_predicate(ReachabilityMode::Standard)`
-3. Fix guardian health orphan_ratio query (synapse.rs:5641–5645) to use `unreachable_predicate(ReachabilityMode::Standard)` — **this is the Drift Site 1 fix**
-4. Add `in_degree` augmentation for INVOKES_COMMAND edges in `compute_in_degrees` (scanner.rs:2836) — optional, lower priority
-5. Document in `process_detector.rs` and `context.rs` that they intentionally use CALLS-only semantics
-
-Priority: P1 (Drift Site 1 causes incorrect health scores in production).  
-Complexity: small (< 100 LOC, no schema changes).
-
----
-
-status: design_complete_implementation_task_proposed