@altimateai/altimate-code 0.12.7 → 0.12.9

package/CHANGELOG.md

@@ -5,6 +5,102 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.12.9] - 2026-11-04
+
+**Claude Code and other coding agents can call altimate-code's data tools directly.** `altimate-code mcp serve` and the new model-free `altimate-code tool` CLI run the same tools without a second model: reads by default, writes only with `--allow-write` (over MCP, `warehouse_add` can save a connection without it). In projects linked to a workspace, what people teach the agent can become a team rule: they confirm it on a card, the workspace owner reviews it, and approved rules reach every teammate's sessions. `dbt_lineage` and `impact_analysis` now return structured lineage, and column impact traces each downstream model's compiled SQL. Also in this release: `auth logout` removes the saved Altimate gateway key, the analyst stays read-only whatever config grants, and the home screen suggests the Claude Code and Codex plugins.
+
+**Heads-up for support:**
+- **MCP and `tool run` requests are read-only unless `--allow-write`.** The database enforces read-only on PostgreSQL, SQLite and DuckDB; on other warehouses it rests on SQL classification, so stored functions with side effects are not blocked. `DROP DATABASE`, `DROP SCHEMA` and `TRUNCATE` are refused even with `--allow-write`, but that block also reads the SQL text and is best effort. Use a read-only database user for a hard guarantee. `--allow-write` is all or nothing, and statements such as `SET`, `ATTACH`, `COPY ... TO`, `INSTALL` and DuckDB `SUMMARIZE` count as writes.
+- **`mcp serve` allows `warehouse_add` without `--allow-write`**, so a calling agent can save a connection; `tool run` needs `--allow-write` for it. Tool results, including query rows and sample values, go to the calling agent's model provider. Large results are spilled to private local files.
+- **`tool run` starts a per-project background process** that keeps connections warm and exits after 10 idle minutes. `--no-daemon` runs a call in its own process, and `ALTIMATE_TOOL_DAEMON=0` turns the background process off.
+- **Workspace skill syncs can write copies into `<project>/.claude/skills/`** when `ALTIMATE_CLAUDE_SKILL_SYNC=1` is set, which the Claude Code plugin does. Skills you wrote with the same name are never touched; unlinking or signing out removes the copies.
+- **`schema_inspect` without a table** lists schemas, or up to 500 tables with catalog row estimates, instead of scanning.
+- **`learn status` prints `Learning from sessions: on|off (source)`** where it printed `Capture:`, and learn messages say "rule" where they said "lesson". The JSON output only gains a `rules` block.
+- **In a linked project, `training_save` with `kind: "rule"` shows a proposal card** instead of writing training, and `learn bootstrap` / `learn import-reviews` send what they find for review.
+- **Telemetry carries the signed-in user ID and tenant** when the machine is signed in to Altimate, so those events are not anonymous; no email address, hashed or not, is sent. One `GET /users/me` lookup per credential finds them, and none is made while telemetry is off. Rows from machines that are not signed in keep the machine id. Turning telemetry off sends one final `telemetry_opted_out` event (machine id and opt-out source only), then nothing.
+- **The analyst's built-in write tools stay off whatever config grants**, including `"*": "allow"`: file edits, SQL writes, memory saves, shell and delegation. Configs that relied on a grant to give the analyst more power will see those tools denied. `task` from the analyst is limited to `data-explorer`; MCP and custom tools still follow config.
+- **A symlink inside an allowed external folder that points elsewhere now asks** for the folder it points to, for every agent.
+- **`auth logout` removes the saved gateway key**, `~/.altimate/altimate.json`, MCP server entries that carry that key (add the datamate again after signing in), and the cached telemetry identity.
+- **Ctrl+C during browser sign-in exits with status 130**, not 0.
+- **The Altimate LLM Gateway free-token allowance is 1M tokens.**
+- **Read-only `dbt_lineage` calls never start the dbt adapter**, which runs `dbt deps`, writes `target/` and can run macro SQL: MCP and `tool run` calls without `--allow-write`, and plan-mode calls, answer from the manifest. With writes allowed it asks the adapter when the installed dbt integration has `getModelLineage`, and otherwise answers from the manifest and says why.
+- **`impact_analysis` with a column can report models as `uncertain` or `untraced`** instead of a confident answer, for example a `select *` from a parent whose schema.yml doesn't list the column, a NATURAL JOIN, or a model with no compiled SQL. A parent's whole row passed as a value (`to_json(o)`) is not caught yet.
+
+### Added
+
+- **`altimate-code tool` CLI and expanded `mcp serve`.** `tool list`, `schema`, `run`, `context` and `daemon` call the same tools as `mcp serve` through one adapter, one allow-list and one permission layer. Default MCP tools now include `impact_analysis`, `dbt_lineage`, `altimate_core_column_lineage`, `schema_detect_pii`, `data_diff`, `altimate_core_equivalence` and `sql_translate`; `--all-tools` exposes the full list. Workspace tools are on for `tool` commands in linked projects (`--no-workspace-tools` turns them off) and opt-in for `mcp serve` (`--workspace-tools`). `link --claude-md` writes routing guidance into `CLAUDE.md`. (#1609)
+- **Workspace rules.** A proposal card in the TUI (`ctrl+x p`), owner review with approve, edit and "Not a rule", `/rules` and a sidebar `rules:` line, `learn sync`, and approved rules delivered into every session with their `applies_to` paths. See "Workspace rules" in the learn docs. (#1605)
+- **Plugin tips on the home screen** for Claude Code and Codex, and a README section on using altimate-code from them. (#1625)
+- **Workspace telemetry** for linking, usage, the engine and the starter. The telemetry reference lists every new event. (#1513)
+- **Signed-in identity on telemetry and a final opt-out event.** (#1501)
+- **Structured `dbt_lineage`.** It returns the model's nodes, columns and column edges (3 hops each way) as structured content when the dbt integration answers, and its own lineage fields from the manifest otherwise. It finds a model by name, unique_id or project-relative SQL path. `include_graph` adds the whole graph, up to 500 nodes each side; over MCP the graph is left out. (#1637, #1635)
+- **Column reach in `impact_analysis`.** With a column, it traces each downstream model's and snapshot's compiled SQL in dependency order, following renames, and returns `column_reach`: certain readers with the output columns they feed, `uncertain` and `untraced` models, and the exposures on each. It also returns `downstream` and `exposures`. Over MCP both tools declare an output schema and return structured content capped at 60 KB. (#1635)
+
+### Changed
+
+- **The analyst stays read-only.** After config, it denies again every built-in tool that changes state, so a wildcard or explicit grant cannot turn them back on, and it can read the files that ship with global skills. (#1597)
+- **External-directory checks follow symlinks**, so a link in an allowed folder cannot reach a folder that is not allowed. (#1597)
+- **The analyst asks for high reasoning effort** on adaptive-thinking models unless the user or agent config sets one. (#1609)
+
+### Fixed
+
+- **`auth logout` removes the saved Altimate gateway key** and the MCP entries that carry it. Sign-out now also removes the cached telemetry identity, as the telemetry reference says. (#1606)
+- **Ctrl+C during browser sign-in exits 130** instead of reporting success. (#1624)
+- **Docs and the TUI show the 1M free-token allowance.** (#1611)
+
+## [0.12.8] - 2026-10-23
+
+**Claude Code can work through your Altimate workspace.** With the Altimate Code plugin installed and the project linked, tasks Claude hands to altimate-code run with the workspace's skills, knowledge and shared memory, and a headless `run` no longer hangs waiting on a stdin nobody writes to. Also in this release: the curl and PowerShell installers add an `altimate-code` command, the analyst can read workspace memory, `learn status` shows whether a lesson keeps being corrected, `schema_inspect` shows what column values look like, and SQL policy checks fail closed on statements they cannot analyze.
+
+**Heads-up for support:**
+- **`run` waits at most 3 s on a socket stdin.** When `run` already has a message or `--command` and stdin is a socket (an agent's shell tool, a Node `spawn`), it waits 3 s for the first byte, then continues without stdin and prints one line on stderr; input sent later is ignored. Files and shell pipelines (`dbt build | altimate-code run …`) are still read to the end. `ALTIMATE_RUN_STDIN_WAIT_MS` changes the wait (`0` waits for EOF), and `< /dev/null` (`< NUL` in cmd.exe) skips it. On Windows a pipe is not a socket, so a caller that never closes stdin should redirect it (#1577).
+- **`run` may end with a line about unsent memory**, giving the count and `altimate-code workspace sync`. It goes to stderr; scripts that parse stderr will see it.
+- **Plain `run` prompts are no longer quoted.** Arguments containing spaces used to reach the model wrapped in quotes; they now arrive as typed. `--command` keeps its quoting.
+- **`check` policy and semantic checks fail closed.** altimate-core 0.9.0 accepts table-valued functions (`TABLE(fn())`, Snowflake `LATERAL FLATTEN`, DuckDB `read_csv(...)`) that 0.8.0 rejected as invalid, but cannot plan them. Policy now reports `policy_not_evaluated` and treats the statement as blocked; semantics reports it as skipped instead of clean.
+- **`schema_inspect` sends sampled column values to the model provider** by default (see Added). `values: false` turns it off per call, and `ALTIMATE_DISABLE_SCHEMA_VALUES=1` turns it off for every call.
+- **`error_fingerprint` telemetry carries a masked `error_message`**: the first line of the error with every word outside a fixed list of common error terms replaced by `<x>`, at most 200 characters. No identifiers, paths, values or SQL.
+- **Read and bash deny rules match equivalent path spellings** (relative, absolute, after `cd`, through a symlink). A deny rule can now stop a command that used to slip past it; allow rules are unchanged.
+- **Warehouse connections are kept per project directory.** A `serve` process serving two projects no longer shares one project's connections with the other. Global and project connection files load as before; a running server still does not re-read an edited `connections.json`.
+- **`datamate_manager` is not offered in projects linked to a workspace**; integrations there come from the workspace's engine. Existing `datamate-*` entries still load, with a one-time notice naming their file.
+
+### Added
+
+- **Claude Code plugin path to workspaces.** The plugin hands work to `altimate-code run` in the linked workspace; the first run in a project pulls the workspace's skills; `workspace list [--json]` lists the workspaces an account can link to. See the Claude Code guide. (#1552)
+- **`altimate-code` from the standalone installers.** curl installs link `altimate-code` to `altimate`; PowerShell installs add `altimate-code.cmd` and a Git Bash shim. Re-running the installer adds it to an existing install. Scripts on Windows should call `altimate`, since the `.cmd` shim passes arguments through cmd.exe. (#1562)
+- **Column value profiles in `schema_inspect`.** On SQLite, DuckDB, PostgreSQL and MongoDB it adds, per column, distinct count, null share, the most frequent values with counts, uniqueness in the sample and common value shapes, from a bounded sample, so filters match the stored format. MongoDB nested fields are listed by path. (#1542)
+- **`ALTIMATE_DISABLE_SCHEMA_VALUES`** turns those value profiles off for every call, whatever the model asks for. (#1578)
+- **Lesson recurrence in `learn status`.** It shows whether a lesson that was shown still drew a correction on the same topic, and flags recurring lessons to reflection. (#1556)
+- **`/integrations` in the TUI** opens the workspace's integration status directly, or offers the link or the engine install when those are missing. (#1464)
+- **Workspace sync state for every client.** When skills and memory were last checked and what changed is recorded and published, so the IDE extension can show it as the TUI sidebar does. (#1429)
+- **Analyst data analysis** gets query deadlines, result envelopes, a scratch bridge (`sql_execute` `query_from`) and verification checks. (#1441)
+- **`answer_commit` asks once for a missed item.** A commit with fewer fields than the request asked for ("the X and its Y") is rejected once, naming the clause; an unchanged recommit is accepted. (#1553)
+- **`client_connected` telemetry** marks the first client to attach to a running server. (#1528)
+
+### Changed
+
+- **The analyst can read workspace memory** (read, refresh and audit) and is steered to workspace skills. Saving memory blocks stays with builder; `training_save` still works. (#1570)
+- **altimate-core 0.9.0**, with policy and semantic checks failing closed as described above. The messages now say to review such statements by hand. (#1555, #1585)
+- **`datamate_manager` is off in linked projects.** (#1431)
+- **Plain `run` prompt arguments are no longer quoted.** (#1561)
+
+### Fixed
+
+- **Headless `run` no longer hangs under an agent's shell.** (#1552)
+- **"database is locked" when several CLI processes start on a fresh data directory.** Setup is serialized with a lock that the OS releases if its holder dies; after 5 s of waiting, CLI commands such as `run` and `serve` print the lock they are waiting on to stderr; the TUI writes it to the log. (#1541, #1580)
+- **Warehouse connections no longer leak between projects in one server**, including SSH tunnels, which are now owned by the connect attempt that opened them. (#1567)
+- **`warehouse_list` with no connections points at the real config locations** and the `ALTIMATE_CODE_CONN_*` variables. (#1559)
+- **Workspace memory and link checks match a repo's remote across its URL forms**, using the same comparison as the workspace service. (#1462)
+- **A message sent while the assistant is busy attaches the workspace engine.** (#1461)
+- **Prompt caching works across working directories.** The environment section moved after the cached instructions, and tool schemas no longer embed the directory. (#1517)
+- **Gemini:** tool schemas no longer contain empty-string enum values, which Gemini rejected with a 400 on every request; small calls on Gemini 4 use thinking levels instead of a budget it rejects. (#1532, #1534)
+- **Large MongoDB reads are fast again.** Byte-bounded cursors read in 1,000-document batches instead of one document per round trip. (#1480)
+- **DuckDB schema indexing** stays in the current catalog, no longer duplicates schemas from attached catalogs, and stops at a 30 s default deadline with partial results. (#1514)
+- **The analyst reports tool failures as failures**, accepts valid read-only syntax and retrieves long text in full. (#1448)
+- **`run --model` reports a failed provider list as itself**, with the HTTP status, not "Model not found". (#1472)
+- **A failed background file scan is handled** instead of surfacing as an unhandled rejection. (#1471)
+- **The read-only SQL check blocks PostgreSQL replication-slot copy and sync functions.** `pg_copy_logical_replication_slot`, `pg_copy_physical_replication_slot` and `pg_sync_replication_slots` were not on the side-effect list, so a top-level call passed as read-only; the create and drop slot functions were already blocked. Found in this release's review.
+- **`uninstall` on Windows gives PowerShell commands, and says so**, and now also offers to remove the empty `~/.altimate/bin` directory (previously only a legacy `~/.opencode/bin`, with a POSIX `rmdir … 2>/dev/null` line even on Windows). Found in this release's review.
+
 ## [0.12.7] - 2026-10-13
 
 **Workspace skills and knowledge come first.** In a linked project the agent prefers the workspace's own skills over lookalikes in the repo or your home folder, and checks the workspace's knowledge base for best practices without being asked. Linking now ends with a starter that says what the workspace provides and suggests prompts. Also in this release: opt-in automatic promotion for `learn`, session traces for the IDE extension, and Snowflake reconnects written to `opencode.log`.

package/bin/altimate-code

@@ -65,7 +65,13 @@ function run(target) {
     console.error(result.error.message)
     process.exit(1)
   }
-  const code = typeof result.status === "number" ? result.status : 0
+  // A child killed by a signal (e.g. the kernel OOM killer) has no exit status; report it the way
+  // a shell would (128 + signal number) instead of exiting 0 as if the run had succeeded.
+  if (result.signal) {
+    console.error(`altimate-code terminated by ${result.signal}`)
+    process.exit(128 + (os.constants.signals[result.signal] ?? 1))
+  }
+  const code = typeof result.status === "number" ? result.status : 1
   process.exit(code)
 }
 

package/dbt-tools/dist/index.js

@@ -23607,7 +23607,12 @@ async function create(cfg) {
   const command = (root, diag, defer, changed) => new Wt(infra, runtime, provider, python3, cli, term, config, client, root, diag, defer, changed);
   const fusion = (root, diag, defer, changed) => new Yt(infra, factory, cli, runtime, provider, term, root, diag, defer, changed, cloudVariantDetector);
   const adapter = new re(config, factory, core, cloud, fusion, command, cfg.projectRoot, undefined, new ne, new ue(term), new ce(term), new de(term), new se(term), new me(term), new pe(term), new ae(term), new ie(term), new oe(term), term, new le(term, client, config));
-  await adapter.initialize();
+  try {
+    await adapter.initialize();
+  } catch (e2) {
+    await adapter.dispose().catch(() => {});
+    throw e2;
+  }
   return adapter;
 }
 var init_adapter = __esm(() => {

package/package.json

@@ -7,20 +7,20 @@
   "scripts": {
     "postinstall": "bun ./postinstall.mjs || node ./postinstall.mjs"
   },
-  "version": "0.12.7",
+  "version": "0.12.9",
   "license": "MIT",
   "dependencies": {
-    "@altimateai/altimate-core": "0.7.0"
+    "@altimateai/altimate-core": "0.9.0"
   },
   "optionalDependencies": {
-    "@altimateai/altimate-code-windows-x64-baseline": "0.12.7",
-    "@altimateai/altimate-code-linux-x64": "0.12.7",
-    "@altimateai/altimate-code-linux-x64-baseline": "0.12.7",
-    "@altimateai/altimate-code-darwin-x64": "0.12.7",
-    "@altimateai/altimate-code-linux-arm64": "0.12.7",
-    "@altimateai/altimate-code-windows-x64": "0.12.7",
-    "@altimateai/altimate-code-darwin-x64-baseline": "0.12.7",
-    "@altimateai/altimate-code-darwin-arm64": "0.12.7"
+    "@altimateai/altimate-code-windows-x64-baseline": "0.12.9",
+    "@altimateai/altimate-code-darwin-x64-baseline": "0.12.9",
+    "@altimateai/altimate-code-linux-arm64": "0.12.9",
+    "@altimateai/altimate-code-darwin-x64": "0.12.9",
+    "@altimateai/altimate-code-linux-x64-baseline": "0.12.9",
+    "@altimateai/altimate-code-darwin-arm64": "0.12.9",
+    "@altimateai/altimate-code-linux-x64": "0.12.9",
+    "@altimateai/altimate-code-windows-x64": "0.12.9"
   },
   "peerDependencies": {
     "pg": ">=8",

package/skills/altimate-setup/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: altimate-setup
-description: Configure Altimate platform credentials for datamate and API access
+description: Configure Altimate platform credentials for API and workspace access
 ---
 
 # Altimate Setup
@@ -28,4 +28,4 @@ Guide the user through configuring their Altimate platform credentials.
    ```
    Then set permissions to owner-only: `chmod 600 ~/.altimate/altimate.json`
 
-4. **Validate**: Call the `datamate_manager` tool with `operation: "list"` to verify the credentials work. Report success or failure to the user.
+4. **Validate**: If the `datamate_manager` tool is available, call it with `operation: "list"` to verify the credentials work, and report success or failure to the user. If it is not (it is turned off in a project linked to an Altimate workspace), ask the user to run `/workspace` → Refresh instead, which calls the Altimate API with these credentials and reports an error if they don't work.

package/skills/data-question/SKILL.md

@@ -0,0 +1,192 @@
+---
+name: data-question
+description: Answer an analytical question from connected databases, explore data to answer a question, or perform cross-database analysis.
+---
+
+# Data Question
+
+Use the available read-only tools to answer the user's question. Source databases and project files stay unchanged; when exposed, `sql_materialize` may create session-local tables in the `scratch` warehouse. All examples below use synthetic tables. Adapt names and dialects after inspection. This skill is self-contained.
+
+## Workflow
+
+1. **Inspect.** Use `warehouse_list`, `schema_inspect` and small `sql_execute` queries as needed. Inspect actual keys and text before choosing joins, casts or parsing rules: schemas alone hide whitespace, case, prefixes and mixed formats. Preserve meaningful leading zeros; check that normalization does not merge distinct identities.
+2. **Plan.** State the population, row grain, denominator, units and material assumptions. Decide which joins and aggregation levels preserve them. Keep this brief; a trivial lookup does not need a checklist or several validation scans.
+3. **Compute in-engine.** Filter and aggregate close to the source. Derive large intermediate sets with joins/CTEs or the scratch bridge below, rather than copying rows into `IN (...)` or `VALUES`. Small user-supplied constants are fine. Keep LIMIT, sampling and approximate functions exploratory unless the user requests an estimate, or an exact answer would need per-row model labeling of a large population (see "Batch semantic labels, then count"); then say the answer is an estimate. For exact final answers, aggregate the full intended population in SQL.
+4. **Verify relevant invariants.** Check key uniqueness, before/after join counts, missing matches and null share when they affect the answer. Use `sql_check` for these invariants and extraction coverage; for cross-warehouse comparisons, first materialize both sides into `scratch` with `sql_materialize`. Reconcile components with totals and preserve weighting. `SUM(group_mean * contributing_count) / SUM(contributing_count)` recovers the overall mean for disjoint, complete groups using unrounded means and counts of the same non-null measure; `AVG(group_mean)` gives each group equal weight. `sql_analyze` is optional. If available, a read-only `data-explorer` via `task` can independently re-derive a difficult result.
+5. **Answer.** Explain the basis in one or two sentences (population, grain, caveats), retaining useful tables or explanation before it. Finish with `**Answer:** <requested value(s), with units>`. If evidence or available tools are insufficient, finish with `**Answer:** unable to determine — <why>`. Never guess a missing result.
+
+Select needed columns and useful partition filters to control cost; LIMIT need not reduce scanned bytes. Report measured costs, not invented estimates. After a timeout, revise the query or split work along exhaustive, disjoint partitions; a failed query is not a zero count.
+
+## Cross-engine bridge
+
+**Prerequisites:** both source warehouses are connected; `scratch` is available and uses DuckDB; `sql_materialize` is exposed. Source queries use their own engine's dialect. The synthetic source tables here are `orders(order_id, customer_code, amount, status)` on `sales_db` and `customers(customer_code, region)` on `crm_db`, with text customer codes.
+
+After inspecting examples and checking normalization collisions, invoke `sql_materialize` twice:
+
+- `source_warehouse: "sales_db"`, `target_table: "orders_stage"`, `sql`:
+
+  ```sql
+  SELECT order_id, LOWER(TRIM(customer_code)) AS customer_key, amount
+  FROM orders
+  WHERE status = 'paid'
+  ```
+
+- `source_warehouse: "crm_db"`, `target_table: "customers_stage"`, `sql`:
+
+  ```sql
+  SELECT LOWER(TRIM(customer_code)) AS customer_key, region
+  FROM customers
+  ```
+
+These are complete derived relations, without exploratory LIMITs. Check reported materialization completeness/counts against the intended scope. A partial transfer cannot support an exact cross-engine answer. Query with `sql_execute` on warehouse `scratch`, using the returned table names:
+
+```sql
+SELECT c.region, COUNT(*) AS paid_orders,
+       SUM(o.amount) AS total_amount, AVG(o.amount) AS mean_order_amount
+FROM orders_stage AS o
+LEFT JOIN customers_stage AS c ON c.customer_key = o.customer_key
+GROUP BY c.region;
+```
+
+First verify the customer key is unique and quantify unmatched orders with the next pattern. Do not silently remove unmatched orders; decide how they affect the requested population.
+
+**If the bridge is unavailable:** use existing views, federation or an already connected engine that can access both relations. Independent aggregates can be combined only if sufficient for the requested statistic (for example, sums and contributing counts over disjoint partitions). Otherwise state that the cross-engine join cannot be completed with available tools. Do not invent a local scripting/write capability or hand-transcribe a large key list.
+
+## Fan-out and nulls
+
+**Prerequisites:** ordinary SELECT/CTE support; this pattern works in DuckDB and SQLite. `order_id` is the intended unique left-side grain. Run on the warehouse holding both staged relations, or adapt to source tables on one engine.
+
+```sql
+SELECT customer_key, COUNT(*) AS occurrences
+FROM customers_stage
+GROUP BY customer_key
+HAVING COUNT(*) > 1;
+
+WITH joined AS (
+  SELECT o.order_id, o.customer_key AS source_key,
+         c.customer_key AS matched_key
+  FROM orders_stage AS o
+  LEFT JOIN customers_stage AS c ON c.customer_key = o.customer_key
+)
+SELECT
+  (SELECT COUNT(*) FROM orders_stage) AS rows_before,
+  COUNT(*) AS rows_after,
+  COUNT(DISTINCT order_id) AS distinct_orders_after,
+  SUM(CASE WHEN matched_key IS NULL THEN 1 ELSE 0 END) AS unmatched_rows,
+  1.0 * SUM(CASE WHEN source_key IS NULL THEN 1 ELSE 0 END)
+      / NULLIF(COUNT(*), 0) AS source_key_null_share
+FROM joined;
+```
+
+Submit the two SELECTs separately if the connector accepts only one statement. For a many-to-one left join, row counts should match; also verify left-side IDs are unique and non-null. Resolve unintended fan-out at the correct business grain, not by adding an arbitrary DISTINCT. The reported null share is at joined grain; interpret it as source share only after ruling out fan-out. Empty-population ratios are undefined, not zero.
+
+## Many similarly shaped tables
+
+**Prerequisites:** inspect the engine/version and table schemas. `information_schema`, DuckDB `query_table` and `UNION ALL BY NAME` are not portable to SQLite. Confirm compatible types and that all intended tables are included exactly once; name-based unions fill absent columns with NULL. Reconcile union counts with the sum of source counts and account for overlapping row identities.
+
+For DuckDB, first generate a single SELECT from the catalog of synthetic `orders_*` partitions:
+
+```sql
+WITH catalog AS (
+  SELECT '"' || replace(table_catalog, '"', '""') || '"."' ||
+         replace(table_schema, '"', '""') || '"."' ||
+         replace(table_name, '"', '""') || '"' AS qualified_name
+  FROM information_schema.tables
+  WHERE table_catalog = current_database()
+    AND table_schema = 'main' AND table_type = 'BASE TABLE'
+    AND starts_with(table_name, 'orders_')
+)
+SELECT COUNT(*) AS table_count,
+       'SELECT COUNT(*) AS order_count, SUM(amount) AS total_amount ' ||
+       'FROM query_table([' ||
+       string_agg('''' || replace(qualified_name, '''', '''''') || '''',
+                  ', ' ORDER BY qualified_name) || '], true)' AS sql
+FROM catalog;
+```
+
+Check the catalog scope and `table_count`, then execute the complete generated `sql`. `query_table([...], true)` requires a constant list and unions by name; it does not accept a catalog subquery as its argument. It adds no source-table column. If provenance matters, or table names contain embedded double quotes, generate ordinary quoted SELECTs joined with `UNION ALL BY NAME` instead. For two known synthetic partitions:
+
+```sql
+SELECT 'orders_jan' AS source_table, * FROM "main"."orders_jan"
+UNION ALL BY NAME
+SELECT 'orders_feb' AS source_table, * FROM "main"."orders_feb";
+```
+
+For a large family, generate those SELECT branches from the catalog, escaping SQL string literals and identifiers separately; do not enumerate tables from memory. Reserve a provenance column name absent from the input tables. The star here intentionally aligns the inspected partition schemas; project needed columns for the final computation.
+
+SQLite has no `query_table` or `UNION ALL BY NAME`. Query `sqlite_master` to build a complete statement with quoted identifiers and a consistent explicit column order:
+
+```sql
+SELECT COUNT(*) AS table_count,
+       'WITH combined AS (' ||
+       group_concat(
+         'SELECT ' || quote(name) || ' AS source_table, order_id, amount ' ||
+         'FROM main."' || replace(name, '"', '""') || '"',
+         ' UNION ALL '
+       ) || ') SELECT COUNT(*) AS order_count, SUM(amount) AS total_amount ' ||
+       'FROM combined' AS sql
+FROM main.sqlite_master
+WHERE type = 'table' AND substr(name, 1, 7) = 'orders_';
+```
+
+Inspect the catalog, then execute the returned statement. Every SQLite partition must have `order_id` and `amount` with compatible meaning/types, or each generated branch needs explicit casts/NULL placeholders. `UNION ALL` retains rows; `UNION` would silently deduplicate them. Respect the engine's compound-query limit; if batching is necessary, combine disjoint batches using sufficient statistics (sum/count, not averages of averages).
+
+In both generators, zero matching tables yields `table_count = 0` and `sql = NULL`: investigate coverage instead of claiming a zero total. Never execute clipped generated SQL. Read its spill if available, or generate smaller catalog batches if `read` also clips a long line. These queries generate SQL from metadata, not lists of copied business keys.
+
+Dialect references: [DuckDB query_table](https://www.duckdb.org/docs/current/guides/sql_features/query_and_query_table_functions), [DuckDB union-by-name option](https://www.duckdb.org/docs/lts/sql/functions/utility), [SQLite compound SELECT](https://www.sqlite.org/lang_select.html). All essential examples are above; no external file is required.
+
+## Batch semantic labels, then count
+
+Treat inferred labels as new data, not ground truth. Define a fixed rubric, allowed labels and uncertainty policy before classifying. For synthetic `messages(message_id, body)`, a rubric might classify a request to reverse a charge as `refund`, a question about shipping progress as `delivery`, and ambiguous text as `uncertain`. Specify tie-breaking for multi-topic messages.
+
+Inspect text, then define the complete population of stable IDs and count it before classifying. Labeling with a model costs one call per batch, so it grows with the table and can cost far more than the analysis itself. Without the user's confirmation, label at most 500 rows in total by any path: your own reading, subagents, or warehouse AI functions (for example classification functions in Snowflake, Databricks or BigQuery), which are billed per row too. To label more, first tell the user the row count, how the labels would be produced and the size of the job (about one model call per 50 rows, or a per-row charge on their warehouse bill for warehouse AI functions), and wait for their confirmation. When the population is larger than 500 rows, by default label a documented random sample of up to 500 rows (roughly ±4.5 percentage points at 95% confidence for a proportion) and report the result as an estimate with its sample size, not as an exact count. Review uncertain cases consistently.
+
+To reuse the checks below, form `labels(message_id, label, rubric_version)` for the labeled IDs; `population` must cover the same intended IDs (for a sample, the sampled IDs). Small audited annotation batches can use an available permitted ingestion path. When `scratch` and `sql_materialize` are available, these small batches can be expressed as a SELECT relation and materialized there. For example (DuckDB/SQLite SELECT syntax):
+
+```sql
+WITH annotations(message_id, label, rubric_version) AS (
+  VALUES (101, 'refund', 'v1'),
+         (102, 'delivery', 'v1'),
+         (103, 'uncertain', 'v1')
+)
+SELECT * FROM annotations;
+```
+
+These literals are newly inferred annotations, not a copied intermediate database result. Do not scale this into hand-copying thousands of IDs. When the full population cannot be labeled within these limits, label a sample or report the coverage limitation. `sql_materialize` alone does not imply an arbitrary file writer. Never claim labels or counts you did not compute and persist; with no persistence tool, a small complete annotation CTE can still be checked and counted in a read-only query.
+
+**Prerequisites for the following SQL:** DuckDB/SQLite; `population(message_id)` contains exactly the intended unique IDs (the sampled IDs for a sample) and `labels` contains the annotation records in the same engine. They may be scratch tables or CTEs. Run each SELECT separately as needed:
+
+```sql
+-- Must return no rows: includes duplicates across batches.
+SELECT message_id, COUNT(*) AS copies
+FROM labels GROUP BY message_id HAVING COUNT(*) <> 1;
+
+-- Missing IDs, extra IDs and invalid records must all be zero.
+SELECT
+  (SELECT COUNT(*) FROM population AS p
+   WHERE NOT EXISTS (SELECT 1 FROM labels AS l
+                     WHERE l.message_id = p.message_id)) AS missing_ids,
+  (SELECT COUNT(*) FROM labels AS l
+   WHERE NOT EXISTS (SELECT 1 FROM population AS p
+                     WHERE p.message_id = l.message_id)) AS extra_ids,
+  (SELECT COUNT(*) FROM labels
+   WHERE message_id IS NULL OR label IS NULL
+      OR label NOT IN ('refund', 'delivery', 'uncertain')
+      OR rubric_version IS NULL OR rubric_version <> 'v1') AS invalid_records;
+
+-- Only count after the invariants pass; retain uncertain in the denominator.
+SELECT label, COUNT(*) AS messages
+FROM labels GROUP BY label ORDER BY label;
+```
+
+Verify the label counts sum to the `population` count (the sample size when you labeled a sample). Use these computed counts in the answer; do not re-count labels by eye or combine unverified child summaries. For a sample, report proportions with the sample size and the size of the full population, and do not present scaled-up counts as exact.
+
+## Reading spilled results
+
+When `sql_execute` reports `Preview: N of M returned rows ... saved to <path>`, N is the displayed preview and M is the connector-returned count, not necessarily the number of matches. A row-limit warning still applies to the spill. If a JSONL path is supplied, inspect it with `read`, for example:
+
+```json
+{"filePath":"<absolute path reported by sql_execute>","offset":1,"limit":50}
+```
+
+Inspect any metadata/schema record before interpreting row records; preserve encoded types and distinguish NULL from the string "NULL". Continue using line offsets from the read output, accounting for metadata lines. Read may itself clip long lines or total output: recover only the needed fields with a SQL projection/filter instead of guessing missing cell contents. For exact totals, compute `COUNT(*)`, SUM or other requested aggregates in the engine; a spill cannot recover rows discarded by a connector limit. If no spill is available, narrow or aggregate the query and respect its returned-row limit.