ripple-sql 0.1.7__tar.gz → 0.1.8__tar.gz

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/CHANGELOG.md

@@ -6,6 +6,12 @@ Notable changes to Ripple. The format follows
 
 ## [Unreleased]
 
+## [0.1.8] - 2026-09-15
+
+### Added
+- `ripple check-file FILE [--source TABLE]`: a file arrives from an external team (csv, tsv, xlsx, parquet), and before anyone loads it Ripple diffs its header against the landing table's declared columns and prints who reads every column that is gone: models, dashboards, hit hardest. The verdict is the first line and the last; a similar name is a hint ("looks like customer_name; a similar name does not prove the same contents"), never an assumption; exit 0 when nothing downstream changes (new columns nobody reads included), 1 when something breaks, 2 when the check could not run. The file binds to its table by `--source`, or by a `files:` pattern the project declares (`meta: {ripple: {files: ["vendor_feed_*.csv"]}}` on a dbt source, or `files:` in `ripple.yml` for a plain SQL folder); with neither, the command prints the closest table and the exact line to add and stops. Header only: no data row is read and nothing leaves the machine. Shaped by a two-model debate; the spec's sample output is the command's own.
+- A dbt source's `columns:` in sources.yml are its declared columns. The graph read only ingested schemas and seeds before, so every dbt source showed as "not declared" and a select star through it stayed unresolved even when the yml listed every column.
+
 ## [0.1.7] - 2026-09-10
 
 ### Fixed
@@ -264,7 +270,8 @@ First public release.
 ### Removed
 - The dark whole-graph canvas that `ripple serve` used to open (`static/index.html`). Every surface now draws one answer.
 
-[Unreleased]: https://github.com/bteh/ripple/compare/v0.1.7...HEAD
+[Unreleased]: https://github.com/bteh/ripple/compare/v0.1.8...HEAD
+[0.1.8]: https://github.com/bteh/ripple/releases/tag/v0.1.8
 [0.1.7]: https://github.com/bteh/ripple/releases/tag/v0.1.7
 [0.1.6]: https://github.com/bteh/ripple/releases/tag/v0.1.6
 [0.1.5]: https://github.com/bteh/ripple/releases/tag/v0.1.5

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ripple-sql
-Version: 0.1.7
+Version: 0.1.8
 Summary: Offline column-level SQL lineage. See what breaks before you merge.
 Project-URL: Homepage, https://github.com/bteh/ripple
 Project-URL: Repository, https://github.com/bteh/ripple
@@ -210,6 +210,53 @@ ripple unresolved                 # which tables block the most coverage, worst
 ripple ingest-schema cols.csv     # CSV with a table,column header (or JSON, or - for stdin)
 ```
 
+## A file arrives from another team
+
+Vendors and business units hand over CSVs and spreadsheets by hand. When a column is renamed or dropped in one of them, the pipeline loads it anyway and dashboards go blank the next morning. Before loading, ask Ripple what this exact file breaks:
+
+```bash
+ripple check-file vendor_feed_2026-09.csv
+```
+
+```
+Stop: 2 columns that 4 models read are missing from this file.
+
+File   vendor_feed_2026-09.csv
+Table  raw.vendor_feed (files: vendor_feed_*.csv)
+Checked: column names only. Not checked: values, types, whether the file loads.
+
+6 columns: 2 unchanged, 2 missing, 4 new
+
+MISSING  customer_name   read by 3 models; hit hardest: dim_customer (2 columns)
+MISSING  amount_usd      read by 2 models
+NEW      cust_nm         looks like customer_name; a similar name does not prove the same contents
+3 more new columns nobody reads yet: region, notes, batch_id
+
+To fix: if cust_nm holds the same data as customer_name, rename that header to customer_name; otherwise restore customer_name. Restore amount_usd. Then run this check again.
+Details: ripple breaks raw.vendor_feed.customer_name
+
+Stop: 2 columns that 4 models read are missing from this file. (exit 1)
+```
+
+The first line and the last are the same sentence, so the vendor's screenshot and the engineer's scrolled terminal both end on the verdict. Exit 0 means nothing downstream changes (a new column nobody reads is fine), 1 means something breaks, 2 means the check could not run. Header only: no data row is read and nothing leaves the machine.
+
+The file finds its table through `--source raw.vendor_feed`, or through a pattern the project declares once:
+
+```yaml
+sources:
+  - name: raw
+    tables:
+      - name: vendor_feed
+        meta:
+          ripple:
+            files: ["vendor_feed_*.csv"]
+        columns:
+          - name: customer_id
+          - name: customer_name
+```
+
+A plain SQL folder keeps the same map in `ripple.yml` (`files: {"vendor_feed_*.csv": raw.vendor_feed}`). The table's declared columns are the contract: `columns:` in sources.yml, a seed, or a list from `ripple ingest-schema`. Reads csv, tsv, xlsx and parquet (parquet needs pyarrow installed). `--json` carries the same facts for an ingest job.
+
 ## Bring your query history (optional)
 
 Your SQL files say what is supposed to happen. The warehouse's query log says what

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/README.md

@@ -170,6 +170,53 @@ ripple unresolved                 # which tables block the most coverage, worst
 ripple ingest-schema cols.csv     # CSV with a table,column header (or JSON, or - for stdin)
 ```
 
+## A file arrives from another team
+
+Vendors and business units hand over CSVs and spreadsheets by hand. When a column is renamed or dropped in one of them, the pipeline loads it anyway and dashboards go blank the next morning. Before loading, ask Ripple what this exact file breaks:
+
+```bash
+ripple check-file vendor_feed_2026-09.csv
+```
+
+```
+Stop: 2 columns that 4 models read are missing from this file.
+
+File   vendor_feed_2026-09.csv
+Table  raw.vendor_feed (files: vendor_feed_*.csv)
+Checked: column names only. Not checked: values, types, whether the file loads.
+
+6 columns: 2 unchanged, 2 missing, 4 new
+
+MISSING  customer_name   read by 3 models; hit hardest: dim_customer (2 columns)
+MISSING  amount_usd      read by 2 models
+NEW      cust_nm         looks like customer_name; a similar name does not prove the same contents
+3 more new columns nobody reads yet: region, notes, batch_id
+
+To fix: if cust_nm holds the same data as customer_name, rename that header to customer_name; otherwise restore customer_name. Restore amount_usd. Then run this check again.
+Details: ripple breaks raw.vendor_feed.customer_name
+
+Stop: 2 columns that 4 models read are missing from this file. (exit 1)
+```
+
+The first line and the last are the same sentence, so the vendor's screenshot and the engineer's scrolled terminal both end on the verdict. Exit 0 means nothing downstream changes (a new column nobody reads is fine), 1 means something breaks, 2 means the check could not run. Header only: no data row is read and nothing leaves the machine.
+
+The file finds its table through `--source raw.vendor_feed`, or through a pattern the project declares once:
+
+```yaml
+sources:
+  - name: raw
+    tables:
+      - name: vendor_feed
+        meta:
+          ripple:
+            files: ["vendor_feed_*.csv"]
+        columns:
+          - name: customer_id
+          - name: customer_name
+```
+
+A plain SQL folder keeps the same map in `ripple.yml` (`files: {"vendor_feed_*.csv": raw.vendor_feed}`). The table's declared columns are the contract: `columns:` in sources.yml, a seed, or a list from `ripple ingest-schema`. Reads csv, tsv, xlsx and parquet (parquet needs pyarrow installed). `--json` carries the same facts for an ingest job.
+
 ## Bring your query history (optional)
 
 Your SQL files say what is supposed to happen. The warehouse's query log says what

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ripple-sql"
-version = "0.1.7"
+version = "0.1.8"
 description = "Offline column-level SQL lineage. See what breaks before you merge."
 readme = "README.md"
 requires-python = ">=3.10"

ripple_sql-0.1.8/src/ripple/check_file.py

@@ -0,0 +1,336 @@
+"""A file arrives from an external team: what breaks if it is loaded.
+
+The file feeds one landing table the graph already knows. Its header is
+diffed against the table's declared columns, and every column that is gone
+gets the blast radius `ripple breaks` would give it. Header only: no data
+row is read, nothing leaves the machine, and a similar name is a hint,
+never an assumption.
+"""
+
+from __future__ import annotations
+
+import csv
+import difflib
+import fnmatch
+import zipfile
+from pathlib import Path
+from xml.etree import ElementTree
+
+from ripple.answer import breaks_answer, hit_hardest
+from ripple.graph import UnknownTarget
+
+HEADER_ONLY = "Checked: column names only. Not checked: values, types, whether the file loads."
+KINDS = ("csv", "tsv", "xlsx", "parquet")
+LOOKS_LIKE = 0.6
+
+
+class CannotCheck(Exception):
+    """The check could not run; the message says why and what to do."""
+
+
+def read_header(path: Path) -> list[str]:
+    """The column names of a file, from its header only."""
+    kind = path.suffix.lower().lstrip(".")
+    if kind in ("csv", "tsv", "txt"):
+        return _delimited(path, "\t" if kind == "tsv" else ",")
+    if kind == "xlsx":
+        return _xlsx(path)
+    if kind == "parquet":
+        return _parquet(path)
+    raise CannotCheck(f"{path.name}: Ripple reads a csv, tsv, xlsx or parquet header, not .{kind}")
+
+
+def _delimited(path: Path, delimiter: str) -> list[str]:
+    # newline="" lets the csv module see a quoted header that spans lines
+    with open(path, encoding="utf-8-sig", errors="replace", newline="") as f:
+        try:
+            header = next(csv.reader(f, delimiter=delimiter))
+        except StopIteration:
+            raise CannotCheck(f"{path.name} is empty; no header row to check") from None
+    # a quoted header that spans lines keeps the file's own line ending
+    return [h.replace("\r\n", "\n").strip() for h in header]
+
+
+def _local(tag: str) -> str:
+    return tag.rsplit("}", 1)[-1]
+
+
+def _xlsx(path: Path) -> list[str]:
+    """Row 1 of the first sheet, without a spreadsheet library."""
+    try:
+        with zipfile.ZipFile(path) as z:
+            names = z.namelist()
+            shared: list[str] = []
+            if "xl/sharedStrings.xml" in names:
+                for si in ElementTree.fromstring(z.read("xl/sharedStrings.xml")):
+                    shared.append("".join(t.text or "" for t in si.iter() if _local(t.tag) == "t"))
+            sheet = next((n for n in sorted(names) if n.startswith("xl/worksheets/sheet")), None)
+            if not sheet:
+                raise CannotCheck(f"{path.name} has no worksheet")
+            root = ElementTree.fromstring(z.read(sheet))
+    except zipfile.BadZipFile:
+        raise CannotCheck(f"{path.name} is not an xlsx workbook") from None
+    row = next((r for r in root.iter() if _local(r.tag) == "row"), None)
+    if row is None:
+        raise CannotCheck(f"{path.name} is empty; no header row to check")
+    header = []
+    for cell in row:
+        if _local(cell.tag) != "c":
+            continue
+        kind = cell.get("t")
+        value = next((v.text or "" for v in cell.iter() if _local(v.tag) in ("v", "t")), "")
+        if kind == "s" and value.isdigit() and int(value) < len(shared):
+            value = shared[int(value)]
+        header.append(value.strip())
+    return header
+
+
+def _parquet(path: Path) -> list[str]:
+    try:
+        import pyarrow.parquet as pq
+    except ImportError:
+        raise CannotCheck(
+            f"{path.name}: reading a Parquet schema needs pyarrow (pip install pyarrow), "
+            "or export the header as csv"
+        ) from None
+    return list(pq.read_schema(path).names)
+
+
+def file_map(root: Path) -> dict[str, str]:
+    """File-name patterns to the tables they feed, from the project.
+
+    A dbt source table carries them as `meta: {ripple: {files: [...]}}`; a
+    plain SQL folder keeps a `files:` map in ripple.yml at its root."""
+    import yaml
+
+    from ripple.loaders.dbt_config import source_tables
+
+    root = Path(root)
+    patterns: dict[str, str] = {}
+    own = root / "ripple.yml"
+    if own.is_file():
+        parsed = yaml.safe_load(own.read_text(encoding="utf-8", errors="replace")) or {}
+        for pattern, table in (parsed.get("files") or {}).items():
+            patterns[str(pattern)] = str(table)
+    for source, table in source_tables(root):
+        meta = (table.get("meta") or {}).get("ripple") or {}
+        for pattern in meta.get("files") or []:
+            patterns[str(pattern)] = f"{source}.{table['name']}"
+    return patterns
+
+
+def bind(
+    file_name: str, explicit: str | None, patterns: dict[str, str]
+) -> tuple[str | None, str | None]:
+    """(table, how) for a file: --source, else the first matching pattern."""
+    if explicit:
+        return explicit, "--source"
+    for pattern, table in patterns.items():
+        if fnmatch.fnmatch(file_name, pattern):
+            return table, f"files: {pattern}"
+    return None, None
+
+
+def find_source(graph, table: str):
+    """The source table by its name or an alias, case-insensitive."""
+    wanted = table.lower()
+    for source in graph.project.sources:
+        if source.name.lower() == wanted or wanted in {a.lower() for a in source.aliases}:
+            return source
+    return None
+
+
+def closest_source(graph, file_name: str) -> str | None:
+    stem = Path(file_name).stem.lower()
+    names = [s.name for s in graph.project.sources]
+    by_tail = {n.rsplit(".", 1)[-1].lower(): n for n in names}
+    match = difflib.get_close_matches(stem, list(by_tail), n=1, cutoff=0.4)
+    return by_tail[match[0]] if match else (names[0] if names else None)
+
+
+def _readers(graph, table: str, column: str) -> dict:
+    try:
+        answer = breaks_answer(graph.breaks(table, column), "model")
+    except UnknownTarget:
+        return {"models": set(), "dashboards": set(), "hit_hardest": []}
+    nodes = [n for n in answer["nodes"] if n["depth"] > 0]
+    return {
+        "models": {n["model"] for n in nodes if n["kind"] == "model"},
+        "dashboards": {n["model"] for n in nodes if n["kind"] == "dashboard"},
+        "hit_hardest": hit_hardest(answer, 1),
+    }
+
+
+def check(
+    graph, table: str, header: list[str], file_name: str, how: str, noun: str = "model"
+) -> dict:
+    """The report: every column's status, who reads the missing ones, the verdict."""
+    source = find_source(graph, table)
+    if source is None:
+        names = [s.name for s in graph.project.sources]
+        near = difflib.get_close_matches(table, names, n=3, cutoff=0.5)
+        hint = f" Did you mean {', '.join(near)}?" if near else ""
+        raise CannotCheck(f"no source table called {table} in this project.{hint}")
+    declared = list(source.declared_columns)
+    if not declared:
+        raise CannotCheck(
+            f"{source.name} has no declared columns, so there is nothing to check the file against. "
+            "Add them to its yml, or run ripple ingest-schema with your warehouse's column list."
+        )
+    present = {h.lower() for h in header}
+    known = {c.lower() for c in declared}
+    missing = [c for c in declared if c.lower() not in present]
+    new = [h for h in header if h.lower() not in known]
+    columns: list[dict] = []
+    for c in declared:
+        if c.lower() in present:
+            columns.append({"name": c, "status": "unchanged"})
+            continue
+        r = _readers(graph, source.name, c)
+        entry = {
+            "name": c,
+            "status": "missing",
+            "readers": {"models": len(r["models"]), "dashboards": len(r["dashboards"])},
+            "hit_hardest": [{"model": m, "columns": n} for m, n in r["hit_hardest"]],
+        }
+        entry["_who"] = r
+        columns.append(entry)
+    read_missing = [c["name"] for c in columns if c["status"] == "missing" and _count(c)]
+    for h in new:
+        like = difflib.get_close_matches(
+            h.lower(), [m.lower() for m in read_missing], n=1, cutoff=LOOKS_LIKE
+        )
+        entry = {"name": h, "status": "new"}
+        if like:
+            entry["looks_like"] = next(m for m in read_missing if m.lower() == like[0])
+        columns.append(entry)
+    models: set[str] = set()
+    dashboards: set[str] = set()
+    for c in columns:
+        if c["status"] == "missing" and _count(c):
+            models |= c["_who"]["models"]
+            dashboards |= c["_who"]["dashboards"]
+    for c in columns:
+        c.pop("_who", None)
+    breaking = len(read_missing)
+    if breaking:
+        verdict = (
+            f"Stop: {_plural(breaking, 'column')} that {_where(len(models), len(dashboards), noun)} "
+            f"read {'is' if breaking == 1 else 'are'} missing from this file."
+        )
+    else:
+        verdict = "Nothing downstream changes."
+    report = {
+        "file": file_name,
+        "table": source.name,
+        "method": how,
+        "checked": "names",
+        "columns": columns,
+        "counts": {
+            "file": len(header),
+            "unchanged": len(declared) - len(missing),
+            "missing": len(missing),
+            "new": len(new),
+        },
+        "verdict": verdict,
+        "exit": 1 if breaking else 0,
+        "noun": noun,
+    }
+    report["lines"] = lines(report)
+    return report
+
+
+def _count(c: dict) -> int:
+    return c["readers"]["models"] + c["readers"]["dashboards"]
+
+
+def _plural(n: int, word: str) -> str:
+    return f"{n} {word}" + ("" if n == 1 else "s")
+
+
+def _where(models: int, dashboards: int, noun: str) -> str:
+    parts = []
+    if models:
+        parts.append(_plural(models, noun))
+    if dashboards:
+        parts.append(_plural(dashboards, "dashboard"))
+    return " and ".join(parts) if parts else "nobody"
+
+
+def lines(report: dict) -> list[str]:
+    """The terminal text: verdict first and last, damage in between."""
+    noun = report.get("noun", "model")
+    counts = report["counts"]
+    listed = [c for c in report["columns"] if c["status"] == "missing" and _count(c)]
+    hinted = [c for c in report["columns"] if c["status"] == "new" and c.get("looks_like")]
+    quiet_new = [
+        c["name"] for c in report["columns"] if c["status"] == "new" and not c.get("looks_like")
+    ]
+    quiet_missing = [
+        c["name"] for c in report["columns"] if c["status"] == "missing" and not _count(c)
+    ]
+    width = max((len(c["name"]) for c in listed + hinted), default=0) + 3
+    out = [report["verdict"], ""]
+    out.append(f"File   {report['file']}")
+    out.append(f"Table  {report['table']} ({report['method']})")
+    out.append(HEADER_ONLY)
+    out.append("")
+    parts = [f"{counts['unchanged']} unchanged"]
+    if counts["missing"]:
+        parts.append(f"{counts['missing']} missing")
+    if counts["new"]:
+        parts.append(f"{counts['new']} new")
+    out.append(f"{_plural(counts['file'], 'column')}: {', '.join(parts)}")
+    out.append("")
+    for c in sorted(listed, key=lambda c: (-_count(c), c["name"])):
+        who = _where(c["readers"]["models"], c["readers"]["dashboards"], noun)
+        line = f"MISSING  {c['name']:<{width}}read by {who}"
+        # worth a name only when one reader is hit in more than one column
+        if c["hit_hardest"] and c["hit_hardest"][0]["columns"] > 1:
+            top = c["hit_hardest"][0]
+            line += f"; hit hardest: {top['model']} ({_plural(top['columns'], 'column')})"
+        out.append(line)
+    for c in hinted:
+        out.append(
+            f"NEW      {c['name']:<{width}}looks like {c['looks_like']}; "
+            "a similar name does not prove the same contents"
+        )
+    if quiet_new:
+        more = "more " if hinted else ""
+        out.append(
+            f"{len(quiet_new)} {more}new column{'s' if len(quiet_new) != 1 else ''} nobody reads yet: {', '.join(quiet_new)}"
+        )
+    if quiet_missing:
+        out.append(
+            f"{len(quiet_missing)} missing column{'s' if len(quiet_missing) != 1 else ''} nobody reads yet: {', '.join(quiet_missing)}"
+        )
+    if listed:
+        out.append("")
+        fixes = []
+        renamed = {c["looks_like"]: c["name"] for c in hinted}
+        for c in listed:
+            if c["name"] in renamed:
+                fixes.append(
+                    f"if {renamed[c['name']]} holds the same data as {c['name']}, rename that header to {c['name']}; "
+                    f"otherwise restore {c['name']}."
+                )
+            else:
+                fixes.append(f"Restore {c['name']}.")
+        out.append("To fix: " + " ".join(fixes) + " Then run this check again.")
+        out.append(f"Details: ripple breaks {report['table']}.{listed[0]['name']}")
+    out.append("")
+    out.append(f"{report['verdict']} (exit {report['exit']})")
+    return out
+
+
+def no_binding_lines(file_name: str, graph) -> list[str]:
+    """What to do when no source claims the file: a hint, never a guess."""
+    guess = closest_source(graph, file_name)
+    out = [f"Cannot check: no source is mapped to {file_name}."]
+    if guess:
+        out.append(
+            f"It looks like {guess}. Run again with --source {guess}, or add to that source's yml:"
+        )
+        out.append(f'  meta: {{ripple: {{files: ["{file_name}"]}}}}')
+        out.append("A pattern like vendor_feed_*.csv covers every drop of the same feed.")
+    return out

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/src/ripple/cli.py

@@ -258,6 +258,43 @@ def cmd_breaks(args) -> None:
     _page_door(args, project, graph, answer, plain_text(breaks_lines(answer, full=True)))
 
 
+def cmd_check_file(args) -> None:
+    from ripple import check_file
+    from ripple.project import find_project_root
+
+    project, graph = _load_graph(args)
+    path = Path(args.file)
+    root = find_project_root(Path(args.path))
+    try:
+        header = check_file.read_header(path)
+        table, how = check_file.bind(path.name, args.source, check_file.file_map(root))
+        if table is None:
+            lines = check_file.no_binding_lines(path.name, graph)
+            print(
+                json.dumps({"file": path.name, "exit": 2, "lines": lines}, indent=2)
+                if args.json
+                else "\n".join(lines)
+            )
+            sys.exit(2)
+        report = check_file.check(graph, table, header, path.name, how, noun_for(project.mode))
+    except check_file.CannotCheck as e:
+        print(
+            json.dumps({"file": path.name, "exit": 2, "lines": [f"Cannot check: {e}"]}, indent=2)
+            if args.json
+            else f"Cannot check: {e}"
+        )
+        sys.exit(2)
+    except OSError as e:
+        print(f"Cannot check: {e}")
+        sys.exit(2)
+    if args.json:
+        print(json.dumps(report, indent=2))
+    else:
+        print("\n".join(report["lines"]))
+    if report["exit"]:
+        sys.exit(report["exit"])
+
+
 def cmd_trace(args) -> None:
     project, graph = _load_graph(args)
     model, column = _parse_target(args.target)
@@ -604,6 +641,15 @@ def main(argv: list[str] | None = None) -> None:
     )
     p_columns.add_argument("model")
 
+    p_check = sub.add_parser(
+        "check-file",
+        help="a file from an external team: what breaks if it is loaded",
+        parents=[common],
+    )
+    p_check.add_argument("file", help="the csv, tsv, xlsx or parquet file that arrived")
+    p_check.add_argument(
+        "--source", help="the table it feeds (raw.vendor_feed); else the project's files map"
+    )
     p_breaks = sub.add_parser("breaks", help="what breaks if this column changes", parents=[common])
     p_breaks.add_argument("target", help="model.column")
     p_breaks.add_argument(
@@ -716,6 +762,7 @@ def main(argv: list[str] | None = None) -> None:
         "models": cmd_models,
         "columns": cmd_columns,
         "breaks": cmd_breaks,
+        "check-file": cmd_check_file,
         "trace": cmd_trace,
         "graph": cmd_graph,
         "ci": cmd_ci,

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/src/ripple/loaders/dbt_config.py

@@ -339,3 +339,47 @@ def _dbt_owned_dirs(project_dir: Path) -> list[Path]:
     owned += _dbt_model_dirs(project_dir)
     owned += [project_dir / name for name in _macro_dir_names(project_dir / "dbt_project.yml")]
     return [d for d in owned if d.is_dir()]
+
+
+def source_tables(root: Path):
+    """Every (source name, table dict) a dbt project declares in its yml.
+
+    Walks the model paths only (dbt_project.yml's model-paths), so a
+    package's or a build folder's yml never counts as this project's."""
+    import yaml
+
+    root = Path(root)
+    if not (root / "dbt_project.yml").is_file():
+        return
+    for model_dir in _dbt_model_dirs(root):
+        for yml in [*files_under(model_dir, "*.yml"), *files_under(model_dir, "*.yaml")]:
+            try:
+                parsed = yaml.safe_load(yml.read_text(errors="replace", encoding="utf-8"))
+            except yaml.YAMLError:
+                continue
+            if not isinstance(parsed, dict):
+                continue
+            for source in parsed.get("sources") or []:
+                if not isinstance(source, dict) or not source.get("name"):
+                    continue
+                for table in source.get("tables") or []:
+                    if isinstance(table, dict) and table.get("name"):
+                        yield str(source["name"]), table
+
+
+def source_columns(root: Path) -> dict[str, list[str]]:
+    """The columns sources.yml declares, as {"schema.table": [names]}.
+
+    dbt users list a source's columns in yml for docs and tests; those are
+    the landing table's contract, and the graph read only ingested schemas
+    and seeds before (a dbt source always showed "not declared")."""
+    out: dict[str, list[str]] = {}
+    for source, table in source_tables(root):
+        names = []
+        for column in table.get("columns") or []:
+            name = column.get("name") if isinstance(column, dict) else column
+            if name:
+                names.append(str(name))
+        if names:
+            out[f"{source}.{table['name']}"] = names
+    return out

{ripple_sql-0.1.7 → ripple_sql-0.1.8}/src/ripple/loaders/identity.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import re
+from pathlib import Path
 
 from ripple.loaders.types import Model, Project
 
@@ -93,14 +94,22 @@ def _apply_ingested_schemas(project: Project, extra_roots: tuple = ()) -> None:
     a new source with declared columns. A model the project actually derives
     keeps its own analysis: ingested schemas fill gaps, they never override.
     """
+    from ripple.loaders.dbt_config import source_columns
     from ripple.schemas import load_schemas
 
     # the CLI and MCP write .ripple/schemas.json at the repo root they were
     # run from; a dbt project in a subfolder (balboa's transform/) has its
-    # own root, and reading only there made every ingest a silent no-op
+    # own root, and reading only there made every ingest a silent no-op.
+    # sources.yml columns come first; an ingest adds to them
     tables: dict = {}
     for root in dict.fromkeys([project.root, *extra_roots]):
-        tables.update(load_schemas(root) or {})
+        tables.update(source_columns(Path(root)))
+    for root in dict.fromkeys([project.root, *extra_roots]):
+        for name, columns in (load_schemas(root) or {}).items():
+            tables[name] = [
+                *tables.get(name, []),
+                *[c for c in columns if c not in tables.get(name, [])],
+            ]
     if not tables:
         return
     # every relation a spelling names, not the first: a monorepo declares the