sql-code-graph 1.37.0__tar.gz → 1.38.0__tar.gz

{sql_code_graph-1.37.0 → sql_code_graph-1.38.0}/.gitignore

@@ -70,6 +70,9 @@ anomaly.csv
 
 # Local-only golden lineage ground truth — names private DWH columns
 tests/e2e/golden_lineage.yaml
+# Real-name lineage oracle — names private internal DWH tables; NEVER commit.
+# The committed, anonymized oracle is plan/metrics/lineage_gold_set_synthetic.json.
+plan/metrics/lineage_gold_set.json
 # Local-only scratch script for bootstrapping the golden set
 scratch_bootstrap_golden.py
 .claude/worktrees/

{sql_code_graph-1.37.0 → sql_code_graph-1.38.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sql-code-graph
-Version: 1.37.0
+Version: 1.38.0
 Summary: SQL code graph analyzer and lineage tracer
 Project-URL: Homepage, https://github.com/Warhorze/sql-code-graph
 Project-URL: Repository, https://github.com/Warhorze/sql-code-graph

{sql_code_graph-1.37.0 → sql_code_graph-1.38.0}/docs/cli.md

@@ -473,11 +473,19 @@ and consumers that read the table only via COLUMN_LINEAGE / STAR_SOURCE
 one-hop consumer completeness ``find_table_usages`` already has. Stays
 one-hop (NOT transitive — that is ``downstream``'s job).
 
+With ``--through-temps`` the consumer set is augmented with the downstream
+**temp-collapsed** closure: physical tables reachable from this table only
+by threading through ``kind IN ('cte','derived','temp')`` intermediates,
+computed from the structural COLUMN_LINEAGE hops (read-only; it never
+mutates the graph). This recovers consumers — e.g. the star-into-temp
+final target — that the one-hop default and TEMP_INLINE do not surface.
+
 ### Options
 
 | Option | Type | Required | Repeatable | Default | Description |
 | --- | --- | --- | --- | --- | --- |
 | --raw | BOOLEAN | No | No | False | Disable noise filtering on results |
+| --through-temps | BOOLEAN | No | No | False | Also report consumers reachable only THROUGH a temp/cte/derived intermediate (table-grain temp-collapsed closure). Off by default — the default impact stays one-hop and does not thread temps. |
 
 ## `sqlcg analyze failures`
 

sql_code_graph-1.38.0/plan/metrics/lineage_gold_set_synthetic.json

@@ -0,0 +1,374 @@
+{
+  "name": "lineage_gold_set_synthetic",
+  "version": 1,
+  "created": "2026-06-15",
+  "purpose": "SYNTHETIC anonymized hand-labelled table-level lineage edges. Names are fabricated and map 1:1 onto the synthetic fixtures under tests/fixtures/gold_set/. NO real internal names. Re-keyed straight from the real-name oracle (kept out of git); totals hold: 39 present / 4 gating-join correct-absence / 1 star-into-temp residual. in_current_graph reflects the structural COLUMN_LINEAGE edge (transform != TEMP_INLINE).",
+  "key_convention": "lowercase schema.table; INSERT targets resolve schema via in-file USE SCHEMA",
+  "edges": [
+    {
+      "src": "raw.src_orders",
+      "dst": "mart.f_orders",
+      "file": "int/f_orders.sql",
+      "line": 5,
+      "shape": "single-source INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_compo",
+      "dst": "mart.f_compo",
+      "file": "int/f_compo.sql",
+      "line": 5,
+      "shape": "single-source INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_brand",
+      "dst": "mart.f_brand",
+      "file": "int/f_brand.sql",
+      "line": 5,
+      "shape": "single-source INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_loc",
+      "dst": "mart.f_loc",
+      "file": "int/f_loc.sql",
+      "line": 3,
+      "shape": "single-source INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_fx",
+      "dst": "mart.f_fx",
+      "file": "int/f_fx.sql",
+      "line": 5,
+      "shape": "single-source INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_emp_a",
+      "dst": "stgmart.dim_staff",
+      "file": "template/dim_staff.sql",
+      "line": 50,
+      "shape": "multi-join INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_emp_b",
+      "dst": "stgmart.dim_staff",
+      "file": "template/dim_staff.sql",
+      "line": 54,
+      "shape": "multi-join INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_store",
+      "dst": "stgmart.dim_staff",
+      "file": "template/dim_staff.sql",
+      "line": 58,
+      "shape": "multi-join INSERT...SELECT",
+      "in_current_graph": false,
+      "note": "GATING-JOIN only: bouw.* appears only in JOIN/ON predicates, never projected. Correct that no DATA edge exists. Reclassified: not a recall miss.",
+      "gating_join_correct_absence": true
+    },
+    {
+      "src": "raw.dim_person",
+      "dst": "stgmart.dim_staff",
+      "file": "template/dim_staff.sql",
+      "line": 62,
+      "shape": "multi-join INSERT...SELECT",
+      "in_current_graph": false,
+      "note": "GATING-JOIN only: emp.* used only in ON predicates, not projected. Correct absence, not a recall miss.",
+      "gating_join_correct_absence": true
+    },
+    {
+      "src": "stg.dim_contract",
+      "dst": "stgmart.dim_staff",
+      "file": "template/dim_staff.sql",
+      "line": 66,
+      "shape": "multi-join INSERT...SELECT",
+      "in_current_graph": false,
+      "note": "GATING-JOIN only: wc.* used only in ON predicates, not projected. Correct absence, not a recall miss.",
+      "gating_join_correct_absence": true
+    },
+    {
+      "src": "stg.dim_dept",
+      "dst": "stgmart.dim_staff",
+      "file": "template/dim_staff.sql",
+      "line": 70,
+      "shape": "multi-join INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_ord_hdr",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 26,
+      "shape": "CTE-mediated INSERT...SELECT (base behind ORDER_TABEL CTE)",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_ord_org",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 27,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_weborder",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 29,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_article",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 31,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_date",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 78,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_inbound",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 94,
+      "shape": "CTE/subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_tariff",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 139,
+      "shape": "CTE/subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_colli",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 179,
+      "shape": "CTE/subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_pcs",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 213,
+      "shape": "CTE/subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_returns",
+      "dst": "mart.f_freight_ktn",
+      "file": "int/gmdf/f_freight_ktn.sql",
+      "line": 264,
+      "shape": "CTE/subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_ord_hdr",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": 34,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_ord_org",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": 35,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_weborder",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": 37,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_weborderline",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": 57,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_mail",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": -1,
+      "shape": "subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_voip",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": -1,
+      "shape": "subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_giftcard",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": -1,
+      "shape": "subquery-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.f_freight_ktn",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": -1,
+      "shape": "cross-file: reads sibling f_freight_ktn output",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.f_freight_cost",
+      "dst": "mart.f_freight_misc",
+      "file": "int/gmdf/f_freight_misc.sql",
+      "line": -1,
+      "shape": "cross-file: reads sibling f_freight_cost output",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.src_po",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": 14,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_po",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": 15,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_weborderline",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": 20,
+      "shape": "subquery-mediated INSERT...SELECT",
+      "in_current_graph": false,
+      "note": "FILTER-ONLY: appears only inside a WHERE ... IN (subquery) predicate, no column projected to output. Correct absence, not a recall miss.",
+      "gating_join_correct_absence": true
+    },
+    {
+      "src": "raw.src_invoice",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": 41,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_invoiceline",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": 65,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_article",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": -1,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_date",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": -1,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_inbound",
+      "dst": "mart.f_freight_cost",
+      "file": "int/gmdf/f_freight_cost.sql",
+      "line": -1,
+      "shape": "CTE-mediated INSERT...SELECT",
+      "in_current_graph": true
+    },
+    {
+      "src": "raw.src_emp_backup",
+      "dst": "mart.tmp_emp",
+      "file": "template/dim_person_temp.sql",
+      "line": 18,
+      "shape": "TEMP-chain hop 1: src -> CREATE TEMP",
+      "in_current_graph": true,
+      "note": "structural hop; temp node namespaced dim_person_temp.sql::mart.tmp_emp"
+    },
+    {
+      "src": "mart.tmp_emp",
+      "dst": "mart.dim_person",
+      "file": "template/dim_person_temp.sql",
+      "line": 69,
+      "shape": "TEMP-chain hop 2: temp -> final INSERT",
+      "in_current_graph": true,
+      "note": "structural hop"
+    },
+    {
+      "src": "raw.src_emp_backup",
+      "dst": "mart.dim_person",
+      "file": "template/dim_person_temp.sql",
+      "line": -1,
+      "shape": "TEMP-collapsed transitive (the TRUE business edge / oracle GT)",
+      "in_current_graph": false,
+      "note": "NOT emitted as TEMP_INLINE here because IN-hop is SELECT e.* into temp and the source has no indexed DDL columns -> only volgnr reaches the temp -> composition has no matching column. This is the star-into-temp gap; the structural hops exist but the collapsed edge is absent."
+    },
+    {
+      "src": "raw.src_compet_price",
+      "dst": "stgmart.f_compet_article",
+      "file": "template/f_compet_article.sql",
+      "line": 35,
+      "shape": "TEMP+CTE-mediated; TEMP_INLINE fires (explicit cols)",
+      "in_current_graph": true,
+      "note": "20 TEMP_INLINE collapsed edges emitted here; structural hops also present"
+    },
+    {
+      "src": "stg.dim_art2",
+      "dst": "stgmart.f_compet_article",
+      "file": "template/f_compet_article.sql",
+      "line": -1,
+      "shape": "TEMP-mediated, collapsed via TEMP_INLINE",
+      "in_current_graph": true
+    },
+    {
+      "src": "stg.dim_compet",
+      "dst": "stgmart.f_compet_article",
+      "file": "template/f_compet_article.sql",
+      "line": -1,
+      "shape": "TEMP-mediated, collapsed via TEMP_INLINE",
+      "in_current_graph": true
+    }
+  ],
+  "source_note": "Anonymized 1:1 from the local-only real-name oracle; mapping not committed."
+}

sql_code_graph-1.38.0/plan/research/snowflake_oracle_groundtruth_prep.md

@@ -0,0 +1,245 @@
+# Snowflake ACCESS_HISTORY ground-truth oracle — groundwork research
+
+> **ANONYMIZED for the public repo.** All table/column/file names below are SYNTHETIC and map onto `tests/fixtures/gold_set/` + `plan/metrics/lineage_gold_set_synthetic.json`. The original real-name research (naming private internal DWH objects) is kept local-only and is NOT committed.
+
+
+**Date:** 2026-06-15
+**Status:** Research. Nothing implemented, nothing planned. Read-only investigation on
+live corpus + parser source. Feeds a future planner for the `ACCESS_HISTORY`-as-oracle
+(Tier 3) work in [`observed_usage_overlay_query_history_tableau.md`](observed_usage_overlay_query_history_tableau.md).
+**Author context:** dispatched to answer three gating questions before any oracle plan.
+
+## Lead verdict (FINDING 1 — the gate)
+
+**The graph TODAY retains the FULL per-hop temp trace AND additively emits the collapsed
+edge. Both representations coexist.** The temp-collapse-for-diff primitive can READ
+existing structural hops directly — no preservation work is needed first. VERDICT:
+**per-hop temp edges are already PRESERVED; the diff primitive reads them as-is.**
+
+---
+
+## FINDING 1 — Does the graph retain per-hop temp trace, or collapse it?
+
+### The question
+For a real `src → TEMP → final` chain, does the graph store TWO hops (`src→temp`,
+`temp→final`, temp node having real in/out edges) OR a single collapsed `src→final`
+("TEMP_INLINE"), OR both? This determines whether the eventual temp-collapse-for-diff
+primitive can READ existing hops, or whether per-hop temp edges must first be PRESERVED.
+
+### Method
+- Read the parser's temp pipeline: [`base.py:519-556`](../../src/sqlcg/parsers/base.py)
+  (temp identity / namespacing), [`base.py:1200-1234`](../../src/sqlcg/parsers/base.py)
+  (temp leaf stamped `role="temp"`, namespaced key), and
+  [`ansi_parser.py:264`](../../src/sqlcg/parsers/ansi_parser.py) →
+  [`_emit_transitive_temp_edges`](../../src/sqlcg/parsers/ansi_parser.py) (`:273-…`).
+- Read the recall/health exclusion: [`coverage.py:69,83,368,382`](../../src/sqlcg/cli/coverage.py)
+  (`WHERE cl.transform IS DISTINCT FROM 'TEMP_INLINE'` + `t.kind IN ('cte','derived','temp')`).
+- Read the indexer edge persistence: [`indexer.py:1780`](../../src/sqlcg/indexer/indexer.py)
+  (`edge.transform` stored on every COLUMN_LINEAGE row) and the dedup precedence at
+  [`indexer.py:182-201`](../../src/sqlcg/indexer/indexer.py) (structural never replaced by TEMP_INLINE).
+- **Parsed two REAL corpus files** (no full reindex; one-file `parse_file` calls, free-mem ~1.7GB,
+  one process) with `SnowflakeParser(SchemaResolver(dialect='snowflake'))` on master (v1.36.0).
+
+### Findings (measured fact)
+
+**Example A — [`dim_person_temp.sql`](tests/fixtures/gold_set/template/dim_person_temp.sql)**
+chain: `raw.src_emp_backup → tmp_emp → mart.dim_person`.
+The parser emitted, as distinct table-level structural edges:
+
+```
+CREATE_TABLE  raw.src_emp_backup [table] -> dim_person_temp.sql::mart.tmp_emp [temp]   (line 18)
+INSERT        dim_person_temp.sql::mart.tmp_emp [temp] -> mart.dim_person [table]              (line 69)
+```
+
+So the temp node `dim_person_temp.sql::mart.tmp_emp` (namespaced, `role="temp"`)
+carries a **real incoming edge** and a **real outgoing edge** — TWO hops, fact.
+
+**Example B — [`f_compet_article.sql`](tests/fixtures/gold_set/template/f_compet_article.sql)**
+chain: 4 base tables → `tmp_f_compet_article` → `stgmart.f_compet_article`.
+Parser emitted **5 structural hops** into/out of the temp node **PLUS 20 TEMP_INLINE
+collapsed edges** (`transform='TEMP_INLINE'`), e.g. `raw.src_compet_price → stgmart.f_compet_article`.
+So here **BOTH** the per-hop trace and the collapsed edge exist simultaneously.
+
+### Why the two examples differ (load-bearing nuance)
+In Example A the IN-hop is `SELECT e.*, ROW_NUMBER()… AS volgnr FROM src`. The `e.*`
+star **does not expand** into per-column edges (the source has no indexed DDL columns),
+so the temp's only *named* incoming column is `volgnr`. The OUT-hop consumes 27 explicit
+named columns, none of which is `volgnr`. `_emit_transitive_temp_edges` composes
+`x→temp→y` **by matching temp column NAME** ([`ansi_parser.py:273-308`](../../src/sqlcg/parsers/ansi_parser.py)),
+so **0** TEMP_INLINE edges fire. In Example B the IN-hop projects explicit named columns,
+the names match, and 20 collapsed edges fire.
+
+**Measured fact:** structural per-hop edges are present in BOTH cases. **Inference:** the
+collapsed `src→final` TEMP_INLINE edge is present only when the temp's incoming columns
+are named (not `SELECT *` into a column-less source). This is the **star-into-temp gap**.
+
+### Verdict (FINDING 1)
+- **Per-hop temp trace: ALWAYS PRESERVED.** The temp is a namespaced `role="temp"` node
+  with real `COLUMN_LINEAGE` in and out edges; `transform` is persisted per edge
+  ([`indexer.py:1780`](../../src/sqlcg/indexer/indexer.py)); structural hops are
+  recall/health-counted ([`coverage.py:69,83`](../../src/sqlcg/cli/coverage.py)).
+- **Collapsed `src→final`: present additively when columns are named, absent on star-into-temp;**
+  always excluded from recall/health (`transform='TEMP_INLINE'`,
+  [`coverage.py:368,382,864`](../../src/sqlcg/cli/coverage.py)).
+- **BOTH** representations exist (when TEMP_INLINE fires); structural-only when it doesn't.
+
+**Decision this informs:** the temp-collapse-for-diff primitive can READ existing
+structural hops to build a temp-collapsed closure on OUR side — no schema change, no
+preservation pass needed. It must NOT depend on the existing TEMP_INLINE edge being
+present (the star-into-temp gap means it is sometimes absent); it should compose the
+**structural** hops itself (same `x→temp→y` logic, but keyed on the temp NODE, not on
+column-name match, so star-into-temp chains still collapse at table grain).
+
+---
+
+## FINDING 2 — Edge model for the oracle (design spec, not code)
+
+### Hard invariant (state it, do not violate)
+> **The stored graph MUST retain the full per-hop temp trace. Temp collapse for diffing
+> is COMPARISON-TIME ONLY and NON-DESTRUCTIVE.** The whole DWH is temp-mediated (277/…
+> SQL files create temp tables — grep below); collapsing the stored graph would destroy
+> impact analysis and is irreversible. The oracle diff builds a *transient* temp-collapsed
+> view of both sides at comparison time and discards it.
+
+This invariant is already honoured by the codebase: TEMP_INLINE is *additive* and the
+dedup precedence ([`indexer.py:196-201`](../../src/sqlcg/indexer/indexer.py)) forbids a
+derived edge from shadowing a structural one. The oracle must follow the same discipline
+— never mutate stored hops.
+
+### Both sides temp-collapsed identically before diffing
+A temp-mediated ETL chain must NOT register as a false recall-miss. Per the architect
+review §(b) (verified on-branch), Snowflake's `ACCESS_HISTORY` records `src→temp` and
+`temp→final` as **two separate per-query rows and never the transitive `src→final`** — it
+does not stitch temps across statements. Our graph also stores two structural hops. So:
+
+- **OUR side (`E_ours*`):** compose structural hops through every `role="temp"` /
+  `kind IN ('cte','derived','temp')` node to a bounded-depth fixpoint, then drop edges
+  with a temp endpoint. Equivalent to the existing TEMP_INLINE composition but driven off
+  structural hops (so it is robust to the star-into-temp gap). For the oracle comparison
+  the TEMP_INLINE edges may be *counted in the numerator* (architect §(b) rule 3) — but
+  they stay out of the internal `edge_health_*` self-metric.
+- **ORACLE side (`E_oracle*`):** contract temp nodes out of the per-row oracle edges the
+  same way (architect §(b) rules 1-2), then drop temp-endpoint edges, leaving only
+  persistent endpoints.
+
+Both sides then have identical temp-collapsed semantics → comparable. **TEMP_INLINE counts
+in the oracle numerator** (for temp-mediated chains it is the only edge that matches the
+oracle's collapsed GT); it is reported separately from `edge_health_*`.
+
+### baseSources-derived, persistent endpoints only, OBJECT_DEPENDENCIES as falsifier
+- Derive oracle table edges from **`OBJECTS_MODIFIED[].columns[].baseSources`** collapsed
+  to table grain (architect §(c) correction to §3.1) — NOT the `BASE_OBJECTS_ACCESSED ×
+  OBJECTS_MODIFIED` array cross product, which over-approximates within multi-target queries
+  and artificially depresses recall. Keep the cross product only as a labelled coarse upper bound.
+- Restrict the closed `E_oracle*` to **persistent endpoints** (non-temp, non-transient),
+  classified by a **structural** signal (CREATE TEMP/TRANSIENT DDL via QUERY_HISTORY join,
+  or the ephemeral-objectId heuristic) — **NOT** name patterns and **NOT** `objectDomain`
+  (a TEMPORARY table reports `objectDomain="Table"`; architect §3.2 step-5 correction).
+- **`OBJECT_DEPENDENCIES`** is the exact, non-windowed view-layer falsifier: a view→base
+  edge we have that it contradicts is a parser bug; one it confirms is GT regardless of
+  the window. It does NOT cover temp/transient or one-off DML — complement, not replacement.
+
+### Risks the architect review flagged — quantify what is measurable here
+- **baseSources population rate:** CANNOT be measured without warehouse access (no
+  ACCESS_HISTORY rows on this box). Open. This is the single biggest unknown for the oracle
+  denominator — if `baseSources` is sparsely populated, the oracle under-counts and recall
+  is overstated. Must be measured on a live export before any plan commits.
+- **Dynamic-SQL / IDENTIFIER($var) provenance:** `ACCESS_HISTORY` captures the *executed*
+  objects (a strength), but the producing `QUERY_ID` is the inner execution, complicating the
+  `client_environment`/`QUERY_TAG` provenance join (the session tag is on the outer call).
+  Corpus-side `IDENTIFIER($var)` prevalence is already known from prior research (the 142-file
+  lever; generic-var resolution shipped v1.36.0, +777 resolvable edges). For the oracle this
+  is a name-matching residual, not a blocker.
+- **Star-into-temp gap (NEW, measured here):** when a CREATE TEMP uses `SELECT *` from a
+  column-less source, our TEMP_INLINE edge does not fire (Example A). The structural hops
+  still exist, so a *node-keyed* (not column-keyed) temp collapse on OUR side recovers the
+  table-grain edge. Prevalence: see below — likely material given how common `SELECT e.*`
+  into a temp is in this DWH. The diff primitive MUST collapse on structural hops, not rely
+  on TEMP_INLINE presence.
+
+### Prevalence (measured fact)
+- **277 SQL files** create a temp/transient table
+  (`grep -rilE "create\s+(or replace\s+)?(temporary|temp|transient)\s+table"` on dwh).
+  Temp-mediation is the dominant ETL shape, so the temp-collapse step is not optional —
+  it gates the whole oracle metric (architect: "the single most common pattern in this DWH").
+
+---
+
+## FINDING 3 — Hand-labelled gold set (usable today)
+
+Saved as [`lineage_gold_set.json`](../metrics/lineage_gold_set.json): **44 KNOWN-TRUE
+table→table edges** drawn from 8 real corpus files, each verified by reading the SQL, each
+checked against what the CURRENT parser (master v1.36.0) produces.
+
+### Result (measured fact)
+| Bucket | Count | Meaning |
+|--------|-------|---------|
+| Present as structural edge | **39 / 44** | parser correctly extracts the data edge |
+| Correct-by-design absence (gating/filter join) | **4** | table joined only in ON / WHERE-IN predicate, never projected → correctly NO data edge |
+| Real candidate gap | **1** | `raw.src_emp_backup → mart.dim_person` (the star-into-temp TEMP_INLINE non-firing in Example A; both structural hops present, collapsed edge absent) |
+
+The 4 "absences" were initially logged as misses, then read-verified as the **gating-join**
+distinction the project already ships a metric for (e.g. `stg.dim_store`, `raw.dim_person`,
+`stg.dim_contract` in [`dim_staff.sql`](tests/fixtures/gold_set/template/dim_staff.sql)
+appear only in JOIN/ON predicates; `stg.dim_weborderline` in
+[`f_freight_cost.sql`](tests/fixtures/gold_set/int/gmdf/f_freight_cost.sql)
+only in a `WHERE … IN (subquery)`). These are CORRECT for column/data lineage and are
+labelled `gating_join_correct_absence: true` in the JSON.
+
+### Interpretation
+On unambiguous single- and multi-source `INSERT…SELECT` and CTE-mediated writes the current
+parser is **correct** (39/39 of the data-bearing edges; 4/4 correct absences). The only gap
+is the temp-transitive collapse on star-into-temp — exactly the FINDING-1 mechanism. The
+gold set is a usable **correctness oracle today**: re-run it after any parser change to catch
+regressions, and as the tie-breaker for the temp-closure logic the architect review specified
+(no automated Snowflake source emits the `src→final` temp-collapsed edge directly).
+
+---
+
+## Classification
+
+| Finding | Class | Reasoning |
+|---------|-------|-----------|
+| Per-hop temp trace preserved + TEMP_INLINE additive | **non-issue** (already done) | structural hops always stored; diff primitive reads them as-is |
+| Star-into-temp: TEMP_INLINE doesn't fire on `SELECT *` into column-less source | **coverage lever** (for the oracle), bounded | structural hops exist, so a node-keyed collapse recovers the table edge; only the column-grain collapsed edge is lost |
+| baseSources population rate / dynamic-SQL provenance | **future-proofing / open** | unmeasurable without a live ACCESS_HISTORY export |
+| Gold set (44 edges) | **active asset** | usable today as a regression oracle, independent of ACCESS_HISTORY |
+
+---
+
+## Ranked recommendation (what to build first)
+
+1. **Comparison-time temp-collapse primitive that composes STRUCTURAL hops (node-keyed),
+   non-destructive.** This is the #1 oracle prerequisite and FINDING 1 proves the inputs
+   already exist. Build it to collapse `x→temp→y` at table grain off the structural edges
+   (robust to the star-into-temp gap), NOT off the TEMP_INLINE edges. Validate against the
+   gold set's temp chains. No graph mutation.
+2. **Wire the gold set into a cheap CI/regression check NOW** (44 edges, no warehouse). It
+   already pays off: it pins the 39 correct edges and the 4 gating-join absences so a future
+   refactor can't silently regress them.
+3. **Defer the ACCESS_HISTORY ingest itself until a live `baseSources` export is measured.**
+   The denominator depends entirely on `baseSources` population, which is unmeasured here.
+   Pull a small windowed export first, measure population rate + temp prevalence in the
+   arrays, THEN plan the overlay.
+
+### Risks / what would change the verdict
+- If `baseSources` is sparsely populated on the live account, the oracle denominator is
+  unreliable and Tier 3 should be reconsidered before building. (Verdict on FINDING 1 is
+  unaffected — it is purely about OUR graph.)
+- If the star-into-temp gap turns out to affect a large share of temp chains at *table*
+  grain too (not just column grain), the node-keyed collapse in rec #1 becomes load-bearing
+  rather than a nicety. Measure star-into-temp prevalence (below) before sizing it.
+
+---
+
+## What I could NOT determine
+- **`baseSources` population rate** on the live account — no ACCESS_HISTORY access from this
+  box; needs a live export. This is the gating unknown for the oracle denominator.
+- **Star-into-temp prevalence across the 277 temp files** — I confirmed the mechanism on 1
+  file (Example A) and its absence on another (Example B) but did NOT count how many of the
+  277 temp-creating files use `SELECT *`/`SELECT alias.*` into a column-less-source temp.
+  A grep + per-file parse over the 277 would quantify how much of the temp-collapse must run
+  on structural hops vs. can lean on existing TEMP_INLINE. Recommend measuring before sizing rec #1.
+- **OBJECT_DEPENDENCIES view-layer agreement rate** — needs the live export; not checkable here.
+- **Whether any temp/permanent NAME COLLISION exists in the live arrays** (architect §(b) rule 4
+  danger zone) — needs the export; our side is safe (temps are namespaced) but the oracle side is not.