@altimateai/altimate-code 0.5.21 → 0.6.1

package/CHANGELOG.md

@@ -5,6 +5,77 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.6.1] - 2026-04-24
+
+### Fixed
+
+- **BigQuery finops tools were broken — now work, in any region.** `finops_query_history` was failing 100% on BigQuery with `Unrecognized name: error_message at [11:5]`. Three separate bugs in the `INFORMATION_SCHEMA.JOBS` template: (a) `error_message` and `error_code` were read as top-level columns but they only exist inside the `error_result` struct — now reads `error_result.message` and `error_result.reason`; (b) `total_rows` is a `PARTITIONS` column, not `JOBS` — replaced with `CAST(NULL AS INT64) AS rows_produced`; (c) BigQuery's `state` returns `'DONE'`, not `'SUCCESS'`, so the summary loop was reporting every completed job as FAILED — now derives `execution_status` from `error_result IS NULL`. Every successful BQ job now reports as SUCCESS. (#739, closes #738)
+- **BigQuery finops unusable outside US region.** All five finops modules (`finops_query_history`, `finops_analyze_credits`, `finops_expensive_queries`, `finops_warehouse_advice`, `finops_unused_resources`, `finops_role_grants`) hardcoded `` `region-US.INFORMATION_SCHEMA.*` ``. Now reads the BigQuery connection's configured `location` (e.g. `us`, `eu`, `us-central1`, `asia-northeast1`), sanitised via an `[a-z0-9-]` allowlist with a 64-char cap and hyphen trim. If `location` is unset the tool defaults to `us` — **set `location` explicitly on the BigQuery connection for non-US projects** or you will query the wrong region. Snowflake and Databricks paths are unchanged. (#739)
+
+### Changed
+
+- **`@altimateai/altimate-core` 0.3.0 → 0.3.1** — upstream patch release of the native SQL type-checker used by `altimate_core_validate` and the cross-dialect data-parity engine.
+
+### Docs
+
+- **New showcase page** (`docs/examples/`) with 12 end-to-end workflow demos — dbt peer review, column-level lineage diff, MS SQL → Fabric migration, Fabric platform admin, upstream schema change, NYC Taxi, Olist, Spotify, and more. (#742)
+- **`location` field documented** on the BigQuery connection and called out on the finops tools page.
+
+### Data handling
+
+- `finops_query_history`, `finops_analyze_credits`, and `finops_warehouse_advice` read `user_email` (BigQuery), `user_name` (Snowflake / Databricks / ClickHouse), and raw `query_text` from warehouse system views. Results are returned to the agent and enter the LLM context window. No telemetry or backend upload. Review your tenant's data-handling policy before enabling finops tools in regulated (PII / PHI / PCI) environments.
+- `finops_unused_resources` on BigQuery reads `INFORMATION_SCHEMA.TABLE_STORAGE`, which is an org-level view. Project-scoped service accounts typically require `bigquery.resourceAdmin` at the org to avoid a permission error.
+
+### Internal
+
+- `anti-slop` CI workflow is now advisory — labels + comments still fire on blocked-term hits, but the workflow no longer auto-closes PRs. Root cause: the repo's pull-request template embeds an HTML comment instructing AI to insert "PINEAPPLE", which `anti-slop.yml` also blocklists — every AI-assisted team-member PR was auto-closing within two minutes. (#741, closes #740)
+- Marker-guard hotfix: the `isValidDatabricksHost` env-fallback path added in v0.6.0 straddled the post-push Marker Guard's `-U5` diff window; wrapped in an inline `altimate_change` marker pair to keep strict mode green.
+
+### Testing
+
+- New adversarial tests for v0.6.1 in `packages/opencode/test/skill/release-v0.6.1-adversarial.test.ts`: `sanitizeBqRegion` injection vectors (CRLF, null bytes, backticks, path traversal, Unicode homoglyphs, prototype-pollution-adjacent inputs), `interpolateBqRegion` idempotency and multi-placeholder safety, `bqRegionFor` registry edge cases, BIGQUERY_HISTORY_SQL column-name regression guards (all four #739 bugs), cross-module "every BQ template uses `{region}`" guard, and full-pipeline `buildHistoryQuery` behaviour including Snowflake / Databricks no-regression guards.
+
+## [0.6.0] - 2026-04-21
+
+### Added
+
+- **Data parity (`data_diff` tool + skill)** — compare tables or SQL-query results row-by-row across Postgres, Snowflake, BigQuery, Databricks, ClickHouse, MySQL, Redshift, SQL Server, Microsoft Fabric, DuckDB, SQLite, and Oracle. Five algorithms: `auto` (JoinDiff same-dialect, HashDiff cross-dialect), `joindiff` (FULL OUTER JOIN), `hashdiff` (bisecting checksums — works at any scale without pulling data out), `profile` (column-level statistics, no row values leave the database), and `cascade` (profile first, then HashDiff on diverging columns). Partitioning supports date (`day`/`week`/`month`/`year`), numeric (`bucket_size`), and categorical (distinct values) modes so 100M+ row tables diff in independent batches. Auto-discovers comparable columns from `information_schema`, excludes audit/timestamp columns by name pattern AND by catalog default (`NOW()`, `CURRENT_TIMESTAMP`, `GETDATE()`, `SYSDATE`, `SYSTIMESTAMP`), and confirms exclusions with the user before diffing. (#493)
+- **MSSQL and Microsoft Fabric support in data-parity** — dialect-aware date truncation (`DATETRUNC`), locale-safe date literals (`CONVERT(DATE, ..., 23)`), and seven Azure AD / Entra ID authentication flows (password, access-token, service-principal-secret, MSI-VM, MSI-app-service, default credential chain, token-credential) delegated to `tedious`. Upgrades `mssql` v11 → v12 with explicit `ConnectionPool` isolation and correct handling of unnamed-column result sets. (#705)
+- **Databricks AI Gateway provider** — connect to Databricks serving endpoints (Foundation Model APIs) via PAT auth (`workspace-host::token`), with fallback to `DATABRICKS_HOST` / `DATABRICKS_TOKEN` environment variables. Registers 11 foundation models — Meta Llama 3.1 (405B / 70B / 8B), Claude Sonnet / Opus 4.6, GPT-5.4 / GPT-5 Mini, Gemini 3.1 Pro, DBRX Instruct, and Mixtral 8x7B. Host regex restricts credentials to `*.cloud.databricks.com`, `*.azuredatabricks.net`, and `*.gcp.databricks.com`. (#649, closes #602)
+- **Amazon Bedrock custom-endpoints guide** — dedicated docs page covering bearer-token auth, AWS credential chain, `baseURL` configuration, cross-region model-ID prefixing, and troubleshooting. Provider key corrected from `bedrock` to `amazon-bedrock` across quickstart, providers, and models pages. (#706)
+- **User-facing docs for the new features** — Databricks AI Gateway section in `configure/providers.md` and a full `data-engineering/guides/data-parity.md` covering supported warehouse pairs, algorithms, partition modes, Azure AD auth matrix for Fabric, and compliance guidance.
+
+### Changed
+
+- **`@altimateai/altimate-core` 0.2.6 → 0.3.0** — enables the cross-dialect data-parity engine, T-SQL dialect support for MSSQL/Fabric, and refined hashdiff bisection. Rebuilt native binaries published for all five supported platforms. (#717, closes #716)
+- **Altimate connect dialog polish** — `/connect` now accepts `instance-name::api-key` directly (default URL `https://api.myaltimate.com`). The three-part `api-url::instance-name::api-key` form still works for custom and self-hosted instances. Provider display name "Altimate" → "Altimate AI"; default model display "Altimate AI" → "Altimate LLM Gateway". The internal provider ID (`altimate-backend`) and model ID (`altimate-default`) are preserved — existing `model.json` favorites, recents, and pinned `model:` entries in `opencode.json` continue to work without migration. (#724)
+
+### Fixed
+
+- **Text contrast on light terminal backgrounds** — dark foreground (`#1a1a1a` or `palette[0]` when available) replaces the near-invisible `palette[7]` on light-mode system themes, and inline code blocks now render on an opaque background instead of transparent. Eager `COLORFGBG` env-var detection narrowed to `bg === 7 || bg === 15` skips the 1-second OSC 11 query altogether on light terminals that don't support it (urxvt, gnome-terminal). (#712, closes #704)
+- **Historical `tool_use` blocks after agent switches or MCP disconnects** — the LiteLLM-only `_noop` workaround is replaced by a general fix: tool names are extracted from both `tool-call` and `tool-result` blocks in message history (using `Object.hasOwn()` for prototype-pollution safety), validated against `/^[a-zA-Z0-9_-]{1,64}$/`, and registered as stubs that return "tool no longer available" if the model attempts to call them. Eliminates the Anthropic API 400 error "Requests with 'tool_use' and 'tool_result' blocks must include tool definition." (#703, closes [AI-678])
+- **`/docs` command and config links point to `docs.altimate.sh`** — the TUI "Open docs" action, Cloudflare AI Gateway help text, Anthropic system prompt, and `Config.command` / `Config.agent` schema descriptions previously linked to the wrong domain. Also updates paths to the `/configure/` prefix that matches the actual mkdocs site. (#715, closes #714)
+
+### Security
+
+- **Databricks host validation hardened** — new `isValidDatabricksHost` helper rejects CRLF/whitespace (JS regex `$` matches before `\n` by default), and the env-fallback path now validates host before constructing the `baseURL`.
+- **Tool-name validation** — stub registration ignores names with shell metacharacters, ANSI escapes, control characters, or lengths > 64, guarding against tampered session-file replays.
+- **Restricted `az` inherited env** — `az account get-access-token` is invoked with a whitelisted environment (`PATH`, `HOME`, `AZURE_*`, locale) so unrelated secrets (`DATABRICKS_TOKEN`, cloud provider keys) are not inherited by `az` or any `az` extension.
+
+### Testing
+
+- 38 new adversarial tests covering Databricks host validation (CRLF, anchoring, attacker suffixes), PAT parsing edge cases, body transform, tool-name tainted-input guards, and `data_diff` tool-description release contract.
+- 139-test consolidation across dbt helpers, file status, project-scan, session/llm, and MCP discovery — symlink cache round-trips, seed/test node exclusion, JSON array edge cases, sessionId sanitization, pagination boundary math, and connection-string masking. Two broken `${VAR}`-in-MCP-`command` tests removed (resolution was always restricted to `env` and `headers`). (#709)
+
+### Compliance note
+
+`data_diff` includes up to 5 sample diff rows in its tool output, which becomes part of the LLM conversation. For PII / PHI / PCI data, use `algorithm: "profile"` — column statistics compare without sending row values. The `data-parity` skill asks for explicit confirmation before running row-level diffs against tables whose names match common regulated patterns (`customers`, `patients`, `orders`, `payments`, `accounts`, `users`). A hard env-var opt-out for sample values is tracked in [#729](https://github.com/AltimateAI/altimate-code/issues/729).
+
+### Breaking
+
+- **`mssql` upgraded to v12**. Users with `mssql@^11` pinned at the application level will see "mssql.ConnectionPool is not available" on first SQL Server connection. Pin to `^12` or let altimate resolve.
+- **SQL Server result sets now expose `_`-prefixed columns**. The internal `startsWith("_")` column filter (introduced for partition-discovery noise suppression) was removed because it also stripped legitimate aliases like `_p` used by the partition engine. Queries that relied on this implicit filtering will see the extra columns in results.
+
 ## [0.5.21] - 2026-04-13
 
 ### Added

package/package.json

@@ -7,24 +7,24 @@
   "scripts": {
     "postinstall": "bun ./postinstall.mjs || node ./postinstall.mjs"
   },
-  "version": "0.5.21",
+  "version": "0.6.1",
   "license": "MIT",
   "dependencies": {
-    "@altimateai/altimate-core": "0.2.6"
+    "@altimateai/altimate-core": "0.3.1"
   },
   "optionalDependencies": {
-    "@altimateai/altimate-code-linux-x64-musl": "0.5.21",
-    "@altimateai/altimate-code-linux-x64-baseline": "0.5.21",
-    "@altimateai/altimate-code-windows-arm64": "0.5.21",
-    "@altimateai/altimate-code-darwin-arm64": "0.5.21",
-    "@altimateai/altimate-code-windows-x64-baseline": "0.5.21",
-    "@altimateai/altimate-code-linux-arm64": "0.5.21",
-    "@altimateai/altimate-code-linux-x64-baseline-musl": "0.5.21",
-    "@altimateai/altimate-code-darwin-x64-baseline": "0.5.21",
-    "@altimateai/altimate-code-windows-x64": "0.5.21",
-    "@altimateai/altimate-code-darwin-x64": "0.5.21",
-    "@altimateai/altimate-code-linux-x64": "0.5.21",
-    "@altimateai/altimate-code-linux-arm64-musl": "0.5.21"
+    "@altimateai/altimate-code-linux-x64-musl": "0.6.1",
+    "@altimateai/altimate-code-linux-x64-baseline": "0.6.1",
+    "@altimateai/altimate-code-windows-arm64": "0.6.1",
+    "@altimateai/altimate-code-darwin-arm64": "0.6.1",
+    "@altimateai/altimate-code-windows-x64-baseline": "0.6.1",
+    "@altimateai/altimate-code-linux-arm64": "0.6.1",
+    "@altimateai/altimate-code-linux-x64-baseline-musl": "0.6.1",
+    "@altimateai/altimate-code-darwin-x64-baseline": "0.6.1",
+    "@altimateai/altimate-code-windows-x64": "0.6.1",
+    "@altimateai/altimate-code-darwin-x64": "0.6.1",
+    "@altimateai/altimate-code-linux-x64": "0.6.1",
+    "@altimateai/altimate-code-linux-arm64-musl": "0.6.1"
   },
   "peerDependencies": {
     "pg": ">=8",

package/skills/data-parity/SKILL.md

@@ -0,0 +1,492 @@
+---
+name: data-parity
+description: Validate that two tables or query results are identical — or diagnose exactly how they differ. Discover schema, identify keys, profile cheaply, then diff. Use for migration validation, ETL regression, and query refactor verification.
+---
+
+# Data Parity (Table Diff)
+
+## CRITICAL: Regulated / Sensitive Data
+
+`data_diff` includes up to 5 **sample diff rows** in the tool output so you can see *which* values differ. Those rows are part of the conversation and are sent to the LLM provider you're using.
+
+Before running `data_diff` against a table that might contain PII, PHI, PCI, or other regulated data:
+
+1. **Ask the user** whether the target contains regulated columns.
+2. If yes, prefer `algorithm: "profile"` — it compares column-level statistics (count, nulls, min/max, distinct count) without any row values leaving the database.
+3. If a row-level diff is genuinely required, tell the user that up to 5 sample rows will be sent to the LLM and get explicit approval before calling the tool.
+4. Consider scoping with `where_clause` to exclude sensitive customers/accounts first.
+
+Default to profile mode whenever the table name suggests regulated data (`customers`, `patients`, `orders`, `payments`, `accounts`, `users`, etc.) unless the user explicitly requests row-level comparison.
+
+---
+
+## CRITICAL: Always Start With a Plan
+
+**Before doing anything else**, generate a numbered TODO list for the user:
+
+```
+Here's my plan:
+1. [ ] List available warehouse connections
+2. [ ] Inspect schema, discover primary key candidates, and detect auto-timestamp columns
+3. [ ] Confirm primary keys with you
+4. [ ] Confirm which auto-timestamp columns to exclude
+5. [ ] Check row counts on both sides
+6. [ ] Run column-level profile (cheap — no row scan)
+7. [ ] Ask whether to proceed with row-level diff (may be expensive for large tables)
+8. [ ] Run targeted row-level diff on diverging columns only
+9. [ ] Present findings with scope, filters, time period, columns compared/excluded, and assumptions
+```
+
+Update each item to `[x]` as you complete it. This plan should be visible before any tool is called.
+
+---
+
+## CRITICAL: Use `data_diff` Tool — Never Write Manual Diff SQL
+
+**NEVER** write SQL to diff tables manually (e.g., `EXCEPT`, `FULL OUTER JOIN`, `MINUS`).
+**ALWAYS** use the `data_diff` tool for any comparison operation.
+
+`sql_query` is only for:
+- Schema inspection (`information_schema`, `SHOW COLUMNS`, `DESCRIBE`)
+- Cardinality checks to identify keys
+- Row count estimates
+
+Everything else — profile, row diff, value comparison — goes through `data_diff`.
+
+---
+
+## Step 1: List Connections
+
+Use `warehouse_list` to show the user what connections are available and which warehouses map to source and target.
+
+---
+
+## Step 2: Inspect Schema, Discover Primary Keys, and Detect Auto-Timestamp Columns
+
+Use `sql_query` to get columns, defaults, and identify key candidates:
+
+```sql
+-- Postgres / Redshift / DuckDB
+SELECT column_name, data_type, is_nullable, column_default
+FROM information_schema.columns
+WHERE table_schema = 'public' AND table_name = 'orders'
+ORDER BY ordinal_position
+```
+
+```sql
+-- Snowflake
+SHOW COLUMNS IN TABLE orders
+```
+
+```sql
+-- MySQL / MariaDB  (also fetch EXTRA for ON UPDATE detection)
+SELECT column_name, data_type, is_nullable, column_default, extra
+FROM information_schema.columns
+WHERE table_schema = 'mydb' AND table_name = 'orders'
+ORDER BY ordinal_position
+```
+
+```sql
+-- SQL Server / Fabric
+SELECT c.name AS column_name, tp.name AS data_type, c.is_nullable,
+       dc.definition AS column_default
+FROM sys.columns c
+INNER JOIN sys.types tp ON c.user_type_id = tp.user_type_id
+INNER JOIN sys.objects o ON c.object_id = o.object_id
+INNER JOIN sys.schemas s ON o.schema_id = s.schema_id
+LEFT JOIN sys.default_constraints dc ON c.default_object_id = dc.object_id
+WHERE s.name = 'dbo' AND o.name = 'orders'
+ORDER BY c.column_id
+```
+
+```sql
+-- ClickHouse
+DESCRIBE TABLE source_db.events
+```
+
+**Look for:** columns named `id`, `*_id`, `*_key`, `uuid`, or with `NOT NULL` + unique index.
+
+**Also look for auto-timestamp columns** — any column whose `column_default` contains a time-generating function:
+- PostgreSQL/DuckDB/Redshift: `now()`, `CURRENT_TIMESTAMP`, `clock_timestamp()`
+- MySQL/MariaDB: `CURRENT_TIMESTAMP` (in default or EXTRA)
+- Snowflake: `CURRENT_TIMESTAMP()`, `SYSDATE()`
+- SQL Server: `getdate()`, `sysdatetime()`
+- Oracle: `SYSDATE`, `SYSTIMESTAMP`
+
+These columns auto-generate values on INSERT, so they inherently differ between source and target due to write timing — not because of actual data discrepancies. **Collect them for confirmation in Step 4.**
+
+If no obvious PK, run a cardinality check:
+
+```sql
+SELECT
+  COUNT(*) AS total_rows,
+  COUNT(DISTINCT order_id) AS distinct_order_id,
+  COUNT(DISTINCT customer_id) AS distinct_customer_id
+FROM orders
+```
+
+A valid key column: `distinct_count = total_rows`.
+
+For composite keys:
+```sql
+SELECT order_id, line_item_id, COUNT(*) AS cnt
+FROM order_lines
+GROUP BY order_id, line_item_id
+HAVING COUNT(*) > 1
+LIMIT 5
+```
+If this returns 0 rows, `(order_id, line_item_id)` is a valid composite key.
+
+## Step 3: Confirm Keys With the User
+
+**Always confirm** the identified key columns before proceeding:
+
+> "I identified `order_id` as the primary key (150,000 distinct values = 150,000 rows, no NULLs). Does that look right, or should I use a different column?"
+
+Do not proceed to diff until the user confirms or corrects.
+
+---
+
+## Step 4: Confirm Auto-Timestamp Column Exclusions
+
+If you detected any columns with auto-generating timestamp defaults in Step 2, **present them to the user and ask for confirmation** before excluding them.
+
+**Example prompt when auto-timestamp columns are found:**
+
+> "I found **3 columns** with auto-generating timestamp defaults that will inherently differ between source and target (due to when each row was written, not actual data differences):
+>
+> | Column | Default | Reason to exclude |
+> |--------|---------|-------------------|
+> | `created_at` | `DEFAULT now()` | Set on insert — reflects when this copy was written |
+> | `updated_at` | `DEFAULT now()` | Set on insert — reflects when this copy was written |
+> | `_loaded_at` | `DEFAULT CURRENT_TIMESTAMP` | ETL load timestamp |
+>
+> Should I **exclude** these from the comparison? Or do you want to include any of them (e.g., if you're verifying that `created_at` was preserved during migration)?"
+
+**If user confirms exclusion:** Omit those columns from `extra_columns` when calling `data_diff`.
+
+**If user wants to include some:** Add them explicitly to `extra_columns`.
+
+**If no auto-timestamp columns were detected:** Skip this step and proceed to Step 5.
+
+> **Why ask?** In migration validation, `created_at` should often be *identical* between source and target (it was migrated, not regenerated). But in ETL replication, `created_at` is freshly generated on each side and *should* differ. Only the user knows which case applies.
+
+---
+
+## Step 5: Check Row Counts
+
+```sql
+SELECT COUNT(*) FROM orders   -- run on both source and target
+```
+
+Use counts to:
+- Detect load completeness issues before row-level diff
+- Choose the algorithm and decide whether to ask about cost
+- If counts differ significantly (>5%), flag it immediately
+
+---
+
+## Step 6: Column-Level Profile (Always Run This First)
+
+Profile is cheap — it runs aggregates, not row scans. **Always run profile before row-level diff.**
+
+```
+data_diff(
+  source="orders",
+  target="orders",
+  key_columns=["order_id"],
+  source_warehouse="postgres_prod",
+  target_warehouse="snowflake_dw",
+  algorithm="profile"
+)
+```
+
+Profile tells you:
+- Row count on each side
+- Which columns have null count differences → NULL handling bug
+- Min/max divergence per column → value transformation bug
+- Which columns match exactly → safe to skip in row-level diff
+
+**Example output:**
+```
+Column Profile Comparison
+
+  ✓ order_id: match
+  ✓ customer_id: match
+  ✗ amount: DIFFER     ← source min=10.00, target min=10.01 — rounding?
+  ✗ status: DIFFER     ← source nulls=0, target nulls=47 — NULL mapping bug?
+  ✓ created_at: match
+```
+
+---
+
+## Step 7: Ask Before Running Row-Level Diff on Large Tables
+
+After profiling, check row count and **ask the user** before proceeding:
+
+**If table has < 100K rows:** proceed automatically.
+
+**If table has 100K–10M rows:**
+> "The table has 1.2M rows. Row-level diff will scan all rows on both sides — this may take 30–60 seconds and consume warehouse compute. Do you want to proceed? You can also provide a `where_clause` to limit the scope (e.g., `created_at >= '2024-01-01'`)."
+
+**If table has > 10M rows:**
+> "The table has 50M rows. Full row-level diff could be expensive. Options:
+> 1. Diff a recent window only (e.g., last 30 days)
+> 2. Partition by a date/key column — shows which partition has problems without scanning everything
+> 3. Proceed with full diff (may take several minutes)
+> Which would you prefer?"
+
+---
+
+## Step 8: Run Targeted Row-Level Diff
+
+Use only the columns that the profile said differ. This is faster and produces cleaner output.
+
+```
+data_diff(
+  source="orders",
+  target="orders",
+  key_columns=["order_id"],
+  extra_columns=["amount", "status"],    // only diverging columns from profile
+  source_warehouse="postgres_prod",
+  target_warehouse="snowflake_dw",
+  algorithm="hashdiff"
+)
+```
+
+### For large tables — use partition_column
+
+Split the table into groups and diff each independently. Three modes:
+
+```
+// Date column — partition by month
+data_diff(source="lineitem", target="lineitem",
+  key_columns=["l_orderkey", "l_linenumber"],
+  source_warehouse="pg_source", target_warehouse="pg_target",
+  partition_column="l_shipdate", partition_granularity="month",
+  algorithm="hashdiff")
+
+// Numeric column — partition by key ranges of 100K
+data_diff(source="orders", target="orders",
+  key_columns=["o_orderkey"],
+  source_warehouse="pg_source", target_warehouse="pg_target",
+  partition_column="o_orderkey", partition_bucket_size=100000,
+  algorithm="hashdiff")
+
+// Categorical column — partition by distinct values (string, enum, boolean)
+data_diff(source="orders", target="orders",
+  key_columns=["o_orderkey"],
+  source_warehouse="pg_source", target_warehouse="pg_target",
+  partition_column="o_orderstatus",
+  algorithm="hashdiff")
+```
+
+Output includes aggregate diff + per-partition breakdown showing which group has problems.
+
+---
+
+## Algorithm Selection
+
+| Algorithm | When to use |
+|-----------|-------------|
+| `profile` | **Always run first** — column stats (count, min, max, nulls). No row scan. |
+| `joindiff` | Same database — single FULL OUTER JOIN. Fast, exact. |
+| `hashdiff` | Cross-database or large tables — bisection with checksums. Scales to billions. |
+| `cascade` | Auto-escalate: profile → hashdiff on diverging columns. |
+| `auto` | JoinDiff if same warehouse, HashDiff if cross-database. |
+
+> **CRITICAL:** If `source_warehouse` ≠ `target_warehouse`, **never use `joindiff`** — it only sees one connection and always reports 0 differences. Use `hashdiff` or `auto`.
+
+---
+
+## Output Interpretation
+
+### IDENTICAL
+```
+✓ Tables are IDENTICAL
+  Rows checked: 1,000,000
+```
+
+### DIFFER
+```
+✗ Tables DIFFER
+
+  Source rows:      150,000
+  Target rows:      149,950
+  Only in source:   50       → rows deleted in target (ETL missed deletes)
+  Only in target:   0
+  Updated rows:     0
+  Identical rows:   149,950
+```
+
+| Pattern | Root cause |
+|---------|-----------|
+| `only_in_source > 0`, target = 0 | ETL dropped rows — check filters, incremental logic |
+| `only_in_target > 0`, source = 0 | Target has extra rows — dedup issue or wrong join |
+| `updated_rows > 0`, counts match | Silent value corruption — check type casts, rounding |
+| Row counts differ significantly | Load completeness — check ETL watermarks |
+
+---
+
+## CRITICAL: `extra_columns` Behavior
+
+The Rust engine **only compares columns listed in `extra_columns`**. If the list is empty, it compares key existence only — rows that match on key but differ in values will be silently reported as "identical". This is the most common source of false positives.
+
+**Auto-discovery (default for table names):** When `extra_columns` is omitted and the source is a plain table name, `data_diff` auto-discovers all non-key columns from the database catalog and excludes columns using two detection layers:
+
+1. **Name-pattern matching** — columns named like `updated_at`, `created_at`, `inserted_at`, `modified_at`, `publisher_last_updated_epoch_ms`, ETL metadata columns like `_fivetran_synced`, `_airbyte_extracted_at`, etc.
+2. **Schema-level default detection** — columns with auto-generating timestamp defaults (`DEFAULT NOW()`, `DEFAULT CURRENT_TIMESTAMP`, `GETDATE()`, `SYSDATE()`, `SYSTIMESTAMP`, etc.), detected directly from the database catalog. This catches columns that don't follow naming conventions but still auto-generate values on INSERT. Works across PostgreSQL, MySQL, Snowflake, SQL Server, Oracle, ClickHouse, DuckDB, SQLite, and Redshift.
+
+The output lists which columns were auto-excluded and why.
+
+**SQL queries:** When source is a SQL query (not a table name), auto-discovery cannot work. You **must** provide `extra_columns` explicitly. If you don't, only key-level matching occurs.
+
+**When to override auto-exclusion:** If the user specifically wants to compare audit columns (e.g., verifying that `created_at` was preserved during migration), pass those columns explicitly in `extra_columns`.
+
+---
+
+## Step 9: Present Findings — Always Surface Context
+
+When reporting diff results, **never present bare numbers**. Always frame the result with the full context that determines what the numbers actually mean.
+
+### Required elements in every result summary
+
+**1. Scope — what was compared**
+State exactly which tables/queries were diffed and on which warehouses:
+> "Compared `public.orders` on **postgres_prod** vs `public.orders` on **snowflake_dw**"
+
+**2. Filters and time period applied**
+If any `where_clause` or `partition_column` was used, state it explicitly:
+> "Scope limited to: `created_at >= '2024-01-01' AND created_at < '2024-04-01'` (Q1 2024 only)"
+> "Partitioned by `l_shipdate` (monthly buckets) — diff covers Jan 2023 through Mar 2024"
+
+If no filter was applied, say so:
+> "No row filter applied — full table compared"
+
+**3. Key columns used**
+> "Key: `order_id` (confirmed unique — 150,000 distinct values = 150,000 rows)"
+
+**4. Columns included and excluded**
+List what was compared and what was skipped, and why:
+> "Compared columns: `amount`, `status`, `customer_id`"
+> "Excluded (auto-timestamp defaults): `created_at`, `updated_at`, `_loaded_at`"
+> "Excluded (user request): `internal_score`"
+
+If the user confirmed exclusions in Step 4, reference that confirmation:
+> "Excluded per your confirmation: `created_at`, `updated_at`"
+
+**5. Algorithm used**
+> "Algorithm: `hashdiff` (cross-database)"
+
+### Example full result summary
+
+```
+## Data Parity Results
+
+**Compared:** `public.orders` (postgres_prod) → `public.orders` (snowflake_dw)
+**Scope:** `created_at >= '2024-01-01'` (Q1 2024 only — 42,301 rows in scope)
+**Key:** `order_id`
+**Columns compared:** `amount`, `status`, `customer_id`, `region`
+**Columns excluded:** `created_at`, `updated_at` (auto-timestamp defaults, per your confirmation)
+**Algorithm:** hashdiff
+
+### Result: ✗ DIFFER
+
+| Metric | Value |
+|--------|-------|
+| Source rows | 42,301 |
+| Target rows | 42,298 |
+| Only in source | 3 |
+| Only in target | 0 |
+| Updated rows | 47 |
+| Identical rows | 42,251 |
+
+**Findings:**
+- 3 rows exist in source but are missing in target → possible ETL delete propagation gap
+- 47 rows have value differences in `amount` or `status` → check rounding or status mapping
+```
+
+### When result is IDENTICAL — still surface the scope
+
+Even when tables match perfectly, state what was checked:
+> "✓ Tables are **identical** across 150,000 rows. Compared `amount`, `status`, `customer_id` (full table, no filter, key=`order_id`). Auto-timestamp columns `created_at`, `updated_at` were excluded."
+
+**Why this matters:** "Tables are identical" without context is meaningless — the user needs to know if you checked Q1 only, skipped 5 columns, or used a WHERE clause that covered just 1% of the data.
+
+---
+
+## Common Mistakes
+
+**Writing manual diff SQL instead of calling data_diff**
+→ Never use EXCEPT, MINUS, or FULL OUTER JOIN to diff tables. Use `data_diff`.
+
+**Calling data_diff without confirming the key**
+→ Confirm cardinality with the user first. A bad key gives meaningless results.
+
+**Using joindiff for cross-database tables**
+→ JoinDiff can't see the remote table. Always returns 0 diffs. Use `hashdiff` or `auto`.
+
+**Skipping the profile step and jumping to full row diff**
+→ Profile is free. It tells you which columns actually differ so you avoid scanning everything.
+
+**Running full diff on a billion-row table without asking**
+→ Always ask the user before expensive operations. Offer filtering and partition options.
+
+**Omitting extra_columns when source is a SQL query**
+→ Auto-discovery only works for table names. For SQL queries, always list the columns to compare explicitly.
+
+**Silently excluding auto-timestamp columns without asking the user**
+→ Always present detected auto-timestamp columns (Step 4) and get explicit confirmation. In migration scenarios, `created_at` should be *identical* — excluding it silently hides real bugs.
+
+---
+
+## SQL Server and Microsoft Fabric
+
+### Minimum Version Requirements
+
+| Component | Minimum Version | Why |
+|---|---|---|
+| **SQL Server** | 2022 (16.x) | `DATETRUNC()` used for date partitioning; `LEAST()`/`GREATEST()` used by Rust engine |
+| **Azure SQL Database** | Any current version | Always has `DATETRUNC()` and `LEAST()` |
+| **Microsoft Fabric** | Any current version | T-SQL surface includes all required functions |
+| **mssql** (npm) | 12.0.0 | `ConnectionPool` isolation for concurrent connections, tedious 19 |
+| **@azure/identity** (npm) | 4.0.0 | Required only for Azure AD authentication; tedious imports it internally |
+
+> **Note:** Date partitioning (`partition_column` + `partition_granularity`) uses `DATETRUNC()` which is **not available on SQL Server 2019 or earlier**. Basic diff operations (joindiff, hashdiff, profile) work on older versions. If you need partitioned diffs on SQL Server < 2022, use numeric or categorical partitioning instead.
+
+### Supported Configurations
+
+| Warehouse Type | Authentication | Notes |
+|---|---|---|
+| `sqlserver` / `mssql` | User/password or Azure AD | On-prem or Azure SQL. SQL Server 2022+ required for date partitioning. |
+| `fabric` | Azure AD only | Microsoft Fabric SQL endpoint. Always uses TLS encryption. |
+
+### Connecting to Microsoft Fabric
+
+Fabric uses the same TDS protocol as SQL Server — no separate driver needed. Configuration:
+
+```yaml
+type: "fabric"
+host: "<workspace-id>-<item-id>.datawarehouse.fabric.microsoft.com"
+database: "<warehouse-name>"
+authentication: "azure-active-directory-default"   # recommended
+```
+
+Auth shorthands (mapped to full tedious type names):
+- `CLI` or `default` → `azure-active-directory-default`
+- `password` → `azure-active-directory-password`
+- `service-principal` → `azure-active-directory-service-principal-secret`
+- `msi` or `managed-identity` → `azure-active-directory-msi-vm`
+
+Full Azure AD authentication types:
+- `azure-active-directory-default` — auto-discovers credentials via `DefaultAzureCredential` (recommended; works with `az login`)
+- `azure-active-directory-password` — username/password with `azure_client_id` and `azure_tenant_id`
+- `azure-active-directory-access-token` — pre-obtained token (does **not** auto-refresh)
+- `azure-active-directory-service-principal-secret` — service principal with `azure_client_id`, `azure_client_secret`, `azure_tenant_id`
+- `azure-active-directory-msi-vm` / `azure-active-directory-msi-app-service` — managed identity
+
+### Algorithm Behavior
+
+- **Same-warehouse** MSSQL or Fabric → `joindiff` (single FULL OUTER JOIN, most efficient)
+- **Cross-warehouse** MSSQL/Fabric ↔ other database → `hashdiff` (automatic when using `auto`)
+- The Rust engine maps `sqlserver`/`mssql` to `tsql` dialect and `fabric` to `fabric` dialect — both generate valid T-SQL syntax with bracket quoting (`[schema].[table]`).