@sqlanvil/cli 1.20.1 → 1.22.0

package/README.md

@@ -2,7 +2,7 @@
 
 **SQL workflow tool for BigQuery, Postgres, Supabase, and MySQL/MariaDB.**
 
-SQLAnvil is an open-source fork of [Dataform OSS](https://github.com/dataform-co/dataform) (Apache 2.0), extended with first-class PostgreSQL and Supabase support, plus adapters for other data sources. Define your data transformations in SQLX, have SQLAnvil compile them to idiomatic SQL, and run them against your warehouse.
+SQLAnvil is an open-source fork of [Dataform OSS](https://github.com/dataform-co/dataform) (Apache 2.0), extended with first-class PostgreSQL, Supabase, and MySQL/MariaDB support. Define your data transformations in SQLX, have SQLAnvil compile them to idiomatic SQL for your warehouse, and run, validate, and document the whole DAG from one CLI.
 
 > **SQLAnvil is not affiliated with or endorsed by Google.** The Dataform name and related marks are trademarks of Google LLC. See [NOTICE](NOTICE) for attribution.
 
@@ -10,10 +10,21 @@ SQLAnvil is an open-source fork of [Dataform OSS](https://github.com/dataform-co
 
 ## Features
 
-- **BigQuery** — full support: partitioning, clustering, labels, materialized views, `MERGE`-based incremental upserts
-- **PostgreSQL** — idiomatic DDL: native partitioning, `INSERT ... ON CONFLICT` upserts, btree/gin/gist/brin indexes, tablespaces, fillfactor
-- **Supabase** — extends Postgres with RLS policies, Realtime publications, pgvector indexes, and Supabase Wrappers _(coming soon)_
-- **MySQL / MariaDB** — portable MySQL DDL: CTAS tables, `CREATE OR REPLACE VIEW`, `ON DUPLICATE KEY UPDATE` incremental upserts (one adapter, validated against both engines)
+**Four first-class warehouses** — each adapter generates its warehouse's own idiomatic SQL, not translated BigQuery:
+
+- **PostgreSQL** — native partitioning (RANGE/LIST/HASH), `INSERT … ON CONFLICT` upserts, btree/hash/gin/gist/brin indexes (unique, partial, `INCLUDE`, opclasses), tablespaces, fillfactor, materialized views with in-place refresh policies
+- **Supabase** — everything Postgres, plus RLS policies, Realtime publications, pgvector indexes, and Supabase Wrappers (FDW) as declarative actions
+- **MySQL / MariaDB** — one adapter validated against both engines: CTAS tables, `ON DUPLICATE KEY UPDATE` incremental upserts, the full `mysql: {}` config block (engine/charset/collation/rowFormat, secondary indexes incl. `FULLTEXT`/`SPATIAL` and prefix lengths, native partitioning), materialized-view emulation
+- **BigQuery** — full upstream support: partitioning, clustering, labels, materialized views, `MERGE`-based incremental upserts
+
+**Beyond transformation** — the pieces most SQL workflow tools leave outside the pipeline:
+
+- **Cross-warehouse sources** — read BigQuery or MySQL/MariaDB from a Postgres/Supabase warehouse via named `connections:`: a live FDW bridge (Supabase Wrappers) or **runner-extract** (the CLI reads the source at run time and materializes a plain `ref()`-able table — keyless BigQuery supported). `sqlanvil introspect` scaffolds the declarations
+- **File imports & exports** — `type: "import"` loads Parquet/CSV/JSON from `s3://`/`gs://`/local into a `ref()`-able table (DuckDB bridge on Postgres/Supabase, native `LOAD DATA` on BigQuery); `type: "export"` writes query results back out to files
+- **Python script actions** — `python:` in `actions.yaml` runs file-staging and glue scripts as first-class DAG nodes: ordered by dependencies, covered by run history, no warehouse credentials injected (scripts stage files; `import` loads them). SQLAnvil validates the declared environment — interpreter version, `requirements.txt` vs installed packages, syntax — and never installs anything
+- **`sqlanvil validate`** — EXPLAIN/dry-run the *whole DAG* against the live warehouse without executing: an isolated shadow schema of empty stubs lets every downstream `ref()` resolve; results classify as PASS / FAIL / BLOCKED so one real error doesn't cascade
+- **Queryable artifacts** — every compile/run writes a Parquet catalog + run history under `target/`; `sqlanvil query` runs SQL over them, `sqlanvil inspect` summarizes, `sqlanvil docs` renders a self-contained HTML catalog
+- **Release-artifact runs** — `sqlanvil run --graph <file>` executes a stored `compile --json` output exactly as compiled; **named environments** (`--environment`) give per-env schema suffixes and vars
 - **SQLX + YAML + JS** — three authoring modes: SQL with config blocks, `actions.yaml` bulk definitions, or the JavaScript API
 
 ---
@@ -24,23 +35,32 @@ SQLAnvil is an open-source fork of [Dataform OSS](https://github.com/dataform-co
 npm install -g @sqlanvil/cli
 sqlanvil init my-project
 cd my-project
-# edit workflow_settings.yaml to configure your warehouse
+# edit workflow_settings.yaml + .df-credentials.json (see below)
 sqlanvil compile
 sqlanvil run
 ```
 
-**`workflow_settings.yaml` example (Postgres):**
+**`workflow_settings.yaml` (Postgres/Supabase):** the warehouse is a flat string; connection
+secrets never live here.
 
 ```yaml
-warehouse:
-  kind: postgres
-  host: localhost
-  port: 5432
-  database: analytics
-  user: sqlanvil_writer
-  password: ${PG_PASSWORD}
-  ssl: disable
-  defaultSchema: public
+warehouse: postgres            # bigquery | postgres | supabase | mysql
+defaultDataset: public         # the Postgres schema
+defaultAssertionDataset: sqlanvil_assertions
+sqlanvilCoreVersion: 1.20.1
+```
+
+**`.df-credentials.json`** (gitignored) holds the connection:
+
+```json
+{
+  "host": "localhost",
+  "port": 5432,
+  "database": "analytics",
+  "user": "sqlanvil_writer",
+  "password": "...",
+  "sslMode": "disable"
+}
 ```
 
 **First action (`definitions/my_view.sqlx`):**
@@ -58,23 +78,29 @@ SELECT 1 AS id, 'hello' AS greeting
 
 ## Documentation
 
-Full documentation at **[sqlanvil.com](https://sqlanvil.com)**.
+Full documentation at **[sqlanvil.com/docs](https://sqlanvil.com/docs/)**.
+
+- [Getting Started](https://sqlanvil.com/docs/getting-started/)
+- Warehouses: [PostgreSQL](https://sqlanvil.com/docs/guides/postgres/) · [Supabase](https://sqlanvil.com/docs/guides/supabase/) · [MySQL/MariaDB](https://sqlanvil.com/docs/guides/mysql/) · [BigQuery](https://sqlanvil.com/docs/guides/bigquery/)
+- [Cross-warehouse sources](https://sqlanvil.com/docs/guides/foreign-wrappers/) · [File Imports](https://sqlanvil.com/docs/guides/imports/) · [File Exports](https://sqlanvil.com/docs/guides/exports/) · [Python Script Actions](https://sqlanvil.com/docs/guides/python-actions/)
+- [Validate](https://sqlanvil.com/docs/guides/validate/) · [Named Environments](https://sqlanvil.com/docs/guides/environments/) · [Artifacts & Catalog](https://sqlanvil.com/docs/guides/artifacts/)
+- [Configs Reference](https://sqlanvil.com/docs/reference/configs/) · [What's New](https://sqlanvil.com/docs/whats-new/)
 
-- [Getting Started](https://sqlanvil.com/getting-started/)
-- [BigQuery Guide](https://sqlanvil.com/guides/bigquery/)
-- [PostgreSQL Guide](https://sqlanvil.com/guides/postgres/)
-- [Supabase Guide](https://sqlanvil.com/guides/supabase/)
-- [Configs Reference](https://sqlanvil.com/reference/configs/)
+Writing sqlanvil projects with an AI agent? Point it at [`AGENTS.md`](AGENTS.md) — it corrects
+the Dataform/BigQuery priors that otherwise produce broken sqlanvil code. Contributor/design
+docs live in the separate [`sqlanvil/docs`](https://github.com/sqlanvil/docs) repo.
 
 ---
 
 ## Project layout
 
 ```
-core/         Compiler + action types (table/view/incremental/assertion/operation/notebook/declaration)
-cli/          CLI entrypoint and per-adapter glue (cli/api/dbadapters/)
+core/         Compiler + action types (table/view/incremental/assertion/operation/declaration/import/export/script/…)
+cli/          CLI entrypoint and per-adapter glue (cli/api/dbadapters/ — bigquery, postgres, supabase, mysql)
 protos/       Protobuf definitions for core/configs/execution/db_adapter
-examples/     Sample SQLAnvil projects
+tests/        Integration specs against real warehouses (Postgres, Supabase, MySQL/MariaDB, BigQuery)
+tools/        Bazel rules + Docker fixtures for the integration databases
+examples/     Sample SQLAnvil projects (see examples/supabase_bigquery_mailing_list)
 scripts/      ./scripts/run is the CLI entrypoint wrapper
 ```
 
@@ -82,28 +108,22 @@ scripts/      ./scripts/run is the CLI entrypoint wrapper
 
 ## Building from source
 
-SQLAnvil uses [Bazel](https://bazel.build) (via Bazelisk). Native Bazel works on macOS
-and Linux (Bazel 7 + bzlmod):
+SQLAnvil uses [Bazel](https://bazel.build) 7 with bzlmod (via Bazelisk). Native Bazel works on
+macOS and Linux:
 
 ```bash
-# Build the proto layer
-bazel build //protos:sqlanvil_proto
-
 # Run the CLI
 ./scripts/run help
 
 # Run tests
 bazel test //core/... //cli/...
-```
 
-A Docker-based build container (`./scripts/docker-bazel`) remains available as an
-optional hermetic environment. See [contributing.md](contributing.md) for full instructions.
-
----
-
-## Documentation
+# Integration tests against a local Docker Postgres
+./tools/postgres/run-postgres-db.sh
+```
 
-Reference docs and design documents live in the separate [`sqlanvil/docs`](https://github.com/sqlanvil/docs) repo, published at [sqlanvil.com/docs](https://sqlanvil.com/docs/).
+A Docker-based build container (`./scripts/docker-bazel`) remains available as an optional
+hermetic environment. See [contributing.md](contributing.md) for full instructions.
 
 ---
 
@@ -111,4 +131,4 @@ Reference docs and design documents live in the separate [`sqlanvil/docs`](https
 
 Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
 
-SQLAnvil is a derivative of Dataform OSS (originally developed by Dataform Co and contributed to by Google LLC). It has been fully renamed and extended with first-class PostgreSQL/Supabase support. Licensed under Apache License 2.0.
+SQLAnvil is a derivative of Dataform OSS (originally developed by Dataform Co and contributed to by Google LLC). It has been fully renamed and extended with first-class PostgreSQL/Supabase/MySQL support. Licensed under Apache License 2.0.

package/bundle.js

@@ -49392,11 +49392,11 @@ function setOrValidateTableEnumType(table) {
 function prune(compiledGraph, runConfig) {
     compiledGraph.tables.forEach(setOrValidateTableEnumType);
     const includedActionNames = computeIncludedActionNames(compiledGraph, runConfig);
-    return Object.assign(Object.assign({}, compiledGraph), { tables: compiledGraph.tables.filter(action => includedActionNames.has(targetAsReadableString(action.target))), assertions: compiledGraph.assertions.filter(action => includedActionNames.has(targetAsReadableString(action.target))), operations: compiledGraph.operations.filter(action => includedActionNames.has(targetAsReadableString(action.target))), exports: compiledGraph.exports.filter(action => includedActionNames.has(targetAsReadableString(action.target))), imports: compiledGraph.imports.filter(action => includedActionNames.has(targetAsReadableString(action.target))), scripts: (compiledGraph.scripts || []).filter(action => includedActionNames.has(targetAsReadableString(action.target))) });
+    return Object.assign(Object.assign({}, compiledGraph), { tables: compiledGraph.tables.filter(action => includedActionNames.has(targetAsReadableString(action.target))), assertions: compiledGraph.assertions.filter(action => includedActionNames.has(targetAsReadableString(action.target))), operations: compiledGraph.operations.filter(action => includedActionNames.has(targetAsReadableString(action.target))), exports: compiledGraph.exports.filter(action => includedActionNames.has(targetAsReadableString(action.target))), imports: compiledGraph.imports.filter(action => includedActionNames.has(targetAsReadableString(action.target))), extracts: (compiledGraph.extracts || []).filter(action => includedActionNames.has(targetAsReadableString(action.target))), scripts: (compiledGraph.scripts || []).filter(action => includedActionNames.has(targetAsReadableString(action.target))) });
 }
 function computeIncludedActionNames(compiledGraph, runConfig) {
     var _a, _b, _c;
-    const allActions = [].concat(compiledGraph.tables, compiledGraph.operations, compiledGraph.assertions, compiledGraph.exports, compiledGraph.imports, compiledGraph.scripts || []);
+    const allActions = [].concat(compiledGraph.tables, compiledGraph.operations, compiledGraph.assertions, compiledGraph.exports, compiledGraph.imports, compiledGraph.extracts || [], compiledGraph.scripts || []);
     const allActionNames = new Set(allActions.map(action => targetAsReadableString(action.target)));
     const allActionsByName = new Map(allActions.map(action => [targetAsReadableString(action.target), action]));
     const hasActionSelector = ((_a = runConfig.actions) === null || _a === void 0 ? void 0 : _a.length) > 0;
@@ -50606,7 +50606,7 @@ function collectEvaluationQueries(queryOrAction, concatenate, queryModifier = (q
         .filter(validationQuery => !!validationQuery.query);
 }
 
-const version = "1.20.1";
+const version = "1.22.0";
 const dataformVersion = "3.0.61";
 
 async function build(compiledGraph, runConfig, dbadapter) {
@@ -50995,10 +50995,10 @@ node_modules/
 function postgresCredentialsTemplate(warehouse) {
     const isSupabase = warehouse === "supabase";
     const template = {
-        host: isSupabase ? "db.<project-ref>.supabase.co" : "localhost",
+        host: isSupabase ? "aws-1-<region>.pooler.supabase.com" : "localhost",
         port: 5432,
         database: "postgres",
-        user: "postgres",
+        user: isSupabase ? "postgres.<your-project-ref>" : "postgres",
         password: "",
         sslMode: isSupabase ? "require" : "disable",
         defaultSchema: "public"
@@ -51041,7 +51041,9 @@ async function init(projectDir, projectConfig) {
         workflowSettings.defaultProject = projectConfig.defaultDatabase;
         workflowSettings.defaultLocation = projectConfig.defaultLocation;
     }
-    workflowSettings.defaultDataset = projectConfig.defaultSchema || "sqlanvil";
+    const isPostgresLike = warehouse === "postgres" || warehouse === "supabase";
+    workflowSettings.defaultDataset =
+        projectConfig.defaultSchema || (isPostgresLike ? "public" : "sqlanvil");
     workflowSettings.defaultAssertionDataset =
         projectConfig.assertionSchema || "sqlanvil_assertions";
     if (isBigQuery && projectConfig.defaultIcebergConfig) {
@@ -51062,6 +51064,17 @@ async function init(projectDir, projectConfig) {
     if (projectConfig.builtinAssertionNamePrefix) {
         workflowSettings.builtinAssertionNamePrefix = projectConfig.builtinAssertionNamePrefix;
     }
+    if (isPostgresLike) {
+        workflowSettings.connections = {
+            bigquery_public: sqlanvil.ConnectionConfig.create({
+                platform: "bigquery",
+                project: "bigquery-public-data",
+                dataset: "geo_us_boundaries",
+                billingProject: "REPLACE_WITH_YOUR_GCP_PROJECT",
+                mode: "runner-extract"
+            })
+        };
+    }
     fs__namespace$1.writeFileSync(workflowSettingsYamlPath, jsYaml.dump(workflowSettings));
     filesWritten.push(workflowSettingsYamlPath);
     fs__namespace$1.writeFileSync(gitignorePath, gitIgnoreContents);
@@ -51070,39 +51083,140 @@ async function init(projectDir, projectConfig) {
         fs__namespace$1.writeFileSync(path__namespace.join(projectDir, CREDENTIALS_FILENAME), warehouse === "mysql" ? mysqlCredentialsTemplate() : postgresCredentialsTemplate(warehouse));
         filesWritten.push(path__namespace.join(projectDir, CREDENTIALS_FILENAME));
     }
-    const definitionsDir = path__namespace.join(projectDir, "definitions");
-    fs__namespace$1.mkdirSync(definitionsDir);
-    dirsCreated.push(definitionsDir);
-    const sourcesDir = path__namespace.join(definitionsDir, "sources");
-    fs__namespace$1.mkdirSync(sourcesDir);
-    dirsCreated.push(sourcesDir);
-    const ecommerceDir = path__namespace.join(sourcesDir, "ecommerce");
-    fs__namespace$1.mkdirSync(ecommerceDir);
-    dirsCreated.push(ecommerceDir);
-    const intermediateDir = path__namespace.join(definitionsDir, "intermediate");
-    fs__namespace$1.mkdirSync(intermediateDir);
-    dirsCreated.push(intermediateDir);
-    const outputsDir = path__namespace.join(definitionsDir, "outputs");
-    fs__namespace$1.mkdirSync(outputsDir);
-    dirsCreated.push(outputsDir);
-    const salesDir = path__namespace.join(outputsDir, "sales");
-    fs__namespace$1.mkdirSync(salesDir);
-    dirsCreated.push(salesDir);
-    const ordersDir = path__namespace.join(outputsDir, "orders");
-    fs__namespace$1.mkdirSync(ordersDir);
-    dirsCreated.push(ordersDir);
-    const marketingDir = path__namespace.join(outputsDir, "marketing");
-    fs__namespace$1.mkdirSync(marketingDir);
-    dirsCreated.push(marketingDir);
-    const testDir = path__namespace.join(definitionsDir, "test");
-    fs__namespace$1.mkdirSync(testDir);
-    dirsCreated.push(testDir);
-    const extraDir = path__namespace.join(definitionsDir, "extra");
-    fs__namespace$1.mkdirSync(extraDir);
-    dirsCreated.push(extraDir);
-    const includesDir = path__namespace.join(projectDir, "includes");
-    fs__namespace$1.mkdirSync(includesDir);
-    dirsCreated.push(includesDir);
+    const mkdir = (...segments) => {
+        const dir = path__namespace.join(projectDir, ...segments);
+        fs__namespace$1.mkdirSync(dir, { recursive: true });
+        dirsCreated.push(dir);
+        return dir;
+    };
+    const write = (filePath, contents) => {
+        fs__namespace$1.writeFileSync(filePath, contents);
+        filesWritten.push(filePath);
+    };
+    const keep = (dir) => write(path__namespace.join(dir, ".gitkeep"), "");
+    mkdir("definitions");
+    const sourcesDir = mkdir("definitions", "sources");
+    const intermediateDir = mkdir("definitions", "intermediate");
+    const salesDir = mkdir("definitions", "outputs", "sales");
+    const reportingDir = mkdir("definitions", "outputs", "reporting");
+    const testDir = mkdir("definitions", "test");
+    keep(mkdir("includes"));
+    const withBigQuerySource = warehouse !== "mysql";
+    write(path__namespace.join(sourcesDir, "app_orders.sqlx"), `-- A declaration: a table that already exists in your warehouse (created by your
+-- application, an import, or another pipeline). Point name/dataset at a real table —
+-- downstream models read it with \${ref("app_orders")}.
+config {
+  type: "declaration",
+  name: "app_orders",
+  description: "Raw orders table owned by the application — replace with one of your tables."
+}
+`);
+    write(path__namespace.join(intermediateDir, "stg_app_orders.sqlx"), `config {
+  type: "view",
+  description: "Staging view over the raw orders: rename, cast, and filter here — downstream models only read staged data."
+}
+
+SELECT
+  order_id,
+  product,
+  quantity,
+  amount,
+  postal_code,
+  ordered_at
+FROM \${ref("app_orders")}
+WHERE amount IS NOT NULL
+`);
+    write(path__namespace.join(salesDir, "daily_sales.sqlx"), `config {
+  type: "table",
+  description: "Sales by day and product, built from the staged orders."
+}
+
+SELECT
+  CAST(ordered_at AS DATE) AS sale_date,
+  product,
+  SUM(quantity) AS units_sold,
+  SUM(amount) AS revenue
+FROM \${ref("stg_app_orders")}
+GROUP BY CAST(ordered_at AS DATE), product
+`);
+    write(path__namespace.join(reportingDir, "product_revenue.sqlx"), `config {
+  type: "view",
+  description: "Revenue by product, straight off the staged orders."
+}
+
+SELECT
+  product,
+  SUM(quantity) AS units_sold,
+  SUM(amount) AS revenue
+FROM \${ref("stg_app_orders")}
+GROUP BY product
+`);
+    write(path__namespace.join(testDir, "assert_sales_amounts_positive.sqlx"), `config {
+  type: "assertion",
+  description: "Business rule: every sales row has positive units and revenue."
+}
+
+-- An assertion FAILS when its query returns rows — select the violations.
+SELECT *
+FROM \${ref("daily_sales")}
+WHERE units_sold <= 0 OR revenue <= 0
+`);
+    if (withBigQuerySource) {
+        write(path__namespace.join(sourcesDir, "bigquery_zip_codes.sqlx"), isBigQuery
+            ? `-- A declaration for a table in another project: Google's public ZIP geo data.
+-- BigQuery reads it natively — \${ref("zip_codes")} resolves to the full name.
+config {
+  type: "declaration",
+  database: "bigquery-public-data",
+  schema: "geo_us_boundaries",
+  name: "zip_codes",
+  description: "US ZIP codes from BigQuery public data."
+}
+`
+            : `-- A cross-warehouse source: Google's public ZIP geo data in BigQuery, read through
+-- the "bigquery_public" connection (see workflow_settings.yaml). With
+-- mode: runner-extract the CLI reads BigQuery at run time and materializes the rows
+-- as a plain table here — \${ref("zip_codes")} works like any other source.
+-- Scaffold real sources with:  sqlanvil introspect bigquery_public <dataset.table>
+config {
+  type: "declaration",
+  connection: "bigquery_public",
+  name: "zip_codes",
+  description: "US ZIP codes from BigQuery public data, materialized by runner-extract.",
+  columnTypes: {
+    zip_code: "text",
+    city: "text",
+    state_code: "text"
+  }
+}
+`);
+        write(path__namespace.join(intermediateDir, "stg_zip_codes.sqlx"), `config {
+  type: "view",
+  description: "Staged ZIP geography from the BigQuery source."
+}
+
+SELECT
+  zip_code,
+  city,
+  state_code
+FROM \${ref("zip_codes")}
+`);
+        write(path__namespace.join(reportingDir, "orders_by_region.sqlx"), `config {
+  type: "table",
+  description: "Orders and revenue by state/city — joins both staged sources."
+}
+
+SELECT
+  z.state_code,
+  z.city,
+  COUNT(*) AS orders,
+  SUM(o.amount) AS revenue
+FROM \${ref("stg_app_orders")} o
+JOIN \${ref("stg_zip_codes")} z
+  ON z.zip_code = o.postal_code
+GROUP BY z.state_code, z.city
+`);
+    }
     return {
         filesWritten,
         dirsCreated
@@ -53427,7 +53541,7 @@ class Runner {
         return actionResult;
     }
     async executeTask(client, task, parentAction, options, action) {
-        var _a, _b, _c, _d, _e, _f, _g;
+        var _a, _b, _c, _d, _e, _f, _g, _h, _j, _k, _l, _m, _o, _p, _q;
         const timer = Timer.start();
         const taskResult = {
             status: sqlanvil.TaskResult.ExecutionStatus.RUNNING,
@@ -53499,7 +53613,14 @@ class Runner {
         }
         else if (task.type === "extract") {
             try {
-                const extractor = ((_d = action === null || action === void 0 ? void 0 : action.extract) === null || _d === void 0 ? void 0 : _d.platform) === "mysql"
+                if (Object.keys((_e = (_d = action === null || action === void 0 ? void 0 : action.extract) === null || _d === void 0 ? void 0 : _d.columnTypes) !== null && _e !== void 0 ? _e : {}).length === 0) {
+                    const ds = ((_f = action === null || action === void 0 ? void 0 : action.extract) === null || _f === void 0 ? void 0 : _f.platform) === "mysql"
+                        ? (_g = action === null || action === void 0 ? void 0 : action.extract) === null || _g === void 0 ? void 0 : _g.database : (_h = action === null || action === void 0 ? void 0 : action.extract) === null || _h === void 0 ? void 0 : _h.dataset;
+                    throw new Error(`declaration "${(_j = action === null || action === void 0 ? void 0 : action.target) === null || _j === void 0 ? void 0 : _j.name}" has no columnTypes — scaffold them with: ` +
+                        `sqlanvil introspect ${(_k = action === null || action === void 0 ? void 0 : action.extract) === null || _k === void 0 ? void 0 : _k.connectionName} ${ds}.${(_l = action === null || action === void 0 ? void 0 : action.extract) === null || _l === void 0 ? void 0 : _l.sourceName} ` +
+                        `--output <its .sqlx file>`);
+                }
+                const extractor = ((_m = action === null || action === void 0 ? void 0 : action.extract) === null || _m === void 0 ? void 0 : _m.platform) === "mysql"
                     ? this.executionOptions.mysqlExtract || runMysqlExtract
                     : this.executionOptions.bigQueryExtract || runBigQueryExtract;
                 await extractor({
@@ -53529,7 +53650,7 @@ class Runner {
                 }), task.type === "operation" ? 1 : options.bigquery.actionRetryLimit + 1 || 1);
                 taskResult.metadata = metadata;
                 if (task.type === "assertion") {
-                    const rowCount = (_e = rows[0]) === null || _e === void 0 ? void 0 : _e[Object.keys(rows[0])[0]];
+                    const rowCount = (_o = rows[0]) === null || _o === void 0 ? void 0 : _o[Object.keys(rows[0])[0]];
                     if (rowCount > 0) {
                         throw new Error(`Assertion failed: query returned ${rowCount} row(s).`);
                     }
@@ -53541,7 +53662,7 @@ class Runner {
                     ? sqlanvil.TaskResult.ExecutionStatus.CANCELLED
                     : sqlanvil.TaskResult.ExecutionStatus.FAILED;
                 taskResult.errorMessage = `${this.graph.projectConfig.warehouse} error: ${e.message}`;
-                if ((_g = (_f = e.metadata) === null || _f === void 0 ? void 0 : _f.bigquery) === null || _g === void 0 ? void 0 : _g.jobId) {
+                if ((_q = (_p = e.metadata) === null || _p === void 0 ? void 0 : _p.bigquery) === null || _q === void 0 ? void 0 : _q.jobId) {
                     taskResult.metadata = {
                         bigquery: {
                             jobId: e.metadata.bigquery.jobId
@@ -54061,6 +54182,596 @@ ${columnRows || '<tr><td colspan="3" class="meta">No documented columns.</td></t
 `;
 }
 
+function containsPath(parent, child) {
+    const rel = path__namespace.relative(parent, child);
+    return rel === "" || (!rel.startsWith("..") && !path__namespace.isAbsolute(rel));
+}
+class OutWriter {
+    constructor(outDir) {
+        this.outDir = outDir;
+    }
+    write(relPath, contents) {
+        const full = path__namespace.resolve(this.outDir, relPath);
+        if (!containsPath(path__namespace.resolve(this.outDir), full)) {
+            throw new Error(`Refusing to write outside the output directory: ${relPath}`);
+        }
+        fs__namespace$1.mkdirSync(path__namespace.dirname(full), { recursive: true });
+        fs__namespace$1.writeFileSync(full, contents);
+    }
+}
+const SAFETY_SKIP = [
+    /^\.git(\/|$)/,
+    /(^|\/)node_modules(\/|$)/,
+    /(^|\/)\.df-credentials[^/]*\.json$/,
+    /service_account[^/]*\.json$/i,
+    /(^|\/)\.env([^/]*)$/,
+    /\.pem$/,
+    /(^|\/)compile_output\.json$/,
+    /(^|\/)\.DS_Store$/,
+    /(^|\/)target(\/|$)/,
+];
+function walkFiles(root) {
+    const out = [];
+    const walk = (rel) => {
+        for (const entry of fs__namespace$1.readdirSync(path__namespace.join(root, rel), { withFileTypes: true })) {
+            const relPath = rel ? `${rel}/${entry.name}` : entry.name;
+            if (SAFETY_SKIP.some(re => re.test(relPath) || re.test(relPath + "/"))) {
+                skippedForSafety.add(relPath);
+                continue;
+            }
+            if (entry.isDirectory()) {
+                walk(relPath);
+            }
+            else if (entry.isFile()) {
+                out.push(relPath);
+            }
+        }
+    };
+    walk("");
+    return out;
+}
+let skippedForSafety;
+function findConfigBlock(source) {
+    const m = /(^|\n)\s*config\s*\{/.exec(source);
+    if (!m)
+        return null;
+    const open = source.indexOf("{", m.index + m[0].indexOf("config"));
+    let depth = 0;
+    let inString = null;
+    for (let i = open; i < source.length; i++) {
+        const ch = source[i];
+        const prev = source[i - 1];
+        if (inString) {
+            if (ch === inString && prev !== "\\")
+                inString = null;
+            continue;
+        }
+        if (ch === '"' || ch === "'" || ch === "`") {
+            inString = ch;
+            continue;
+        }
+        if (ch === "{")
+            depth++;
+        if (ch === "}") {
+            depth--;
+            if (depth === 0) {
+                return {
+                    start: m.index + (m[1] ? m[1].length : 0),
+                    open,
+                    end: i + 1,
+                    body: source.slice(open + 1, i),
+                };
+            }
+        }
+    }
+    return null;
+}
+function jsBlockLines(source) {
+    const lines = new Set();
+    const re = /(^|\n)\s*js\s*\{/g;
+    let m;
+    while ((m = re.exec(source))) {
+        const open = source.indexOf("{", m.index + m[0].length - 1);
+        let depth = 0;
+        let end = -1;
+        for (let i = open; i < source.length; i++) {
+            if (source[i] === "{")
+                depth++;
+            if (source[i] === "}") {
+                depth--;
+                if (depth === 0) {
+                    end = i;
+                    break;
+                }
+            }
+        }
+        if (end < 0)
+            end = source.length - 1;
+        const startLine = source.slice(0, m.index + (m[1] ? m[1].length : 0)).split("\n").length - 1;
+        const endLine = source.slice(0, end).split("\n").length - 1;
+        for (let l = startLine; l <= endLine; l++)
+            lines.add(l);
+    }
+    return lines;
+}
+const str = (field, body) => {
+    const re = new RegExp(`${field}\\s*:\\s*["'\`]([^"'\`]+)["'\`]`);
+    const m = re.exec(body);
+    return m ? m[1] : null;
+};
+function parseSqlxConfig(body) {
+    let database = str("database", body);
+    if (!database && /database\s*:\s*(dataform|sqlanvil)\.projectConfig\.defaultDatabase/.test(body)) {
+        database = "<defaultDatabase>";
+    }
+    return {
+        type: str("type", body),
+        schema: str("schema", body),
+        database,
+        name: str("name", body),
+        description: str("description", body),
+    };
+}
+const GLOBAL_RULES = [
+    {
+        pattern: /\bdataform\.projectConfig\b/g,
+        replacement: "sqlanvil.projectConfig",
+        kind: "rewrite",
+        note: "dataform.projectConfig → sqlanvil.projectConfig (compile global renamed)",
+    },
+];
+const SQL_RULES = [
+    {
+        pattern: /\bSAFE_CAST\s*\(/g,
+        replacement: "CAST(",
+        kind: "rewrite",
+        note: "SAFE_CAST → CAST — Postgres CAST raises on bad input; verify the data or guard it",
+    },
+    {
+        pattern: /\bCURRENT_DATE\s*\(\s*\)/g,
+        replacement: "CURRENT_DATE",
+        kind: "rewrite",
+        note: "CURRENT_DATE() → CURRENT_DATE",
+    },
+    {
+        pattern: /\bCURRENT_DATE\s*\(\s*['"]/g,
+        kind: "flag",
+        note: "CURRENT_DATE('<tz>') — Postgres: (CURRENT_TIMESTAMP AT TIME ZONE '<tz>')::date",
+    },
+    {
+        pattern: /\bSAFE_DIVIDE\s*\(/g,
+        kind: "flag",
+        note: "SAFE_DIVIDE(a, b) — Postgres: a / NULLIF(b, 0)",
+    },
+    {
+        pattern: /\bSAFE_OFFSET\s*\(/g,
+        kind: "flag",
+        note: "array[SAFE_OFFSET(n)] — Postgres arrays are 1-based; use (split_part / arr[n+1])",
+    },
+    { pattern: /\bQUALIFY\b/gi, kind: "flag", note: "QUALIFY — rewrite as a subquery/CTE filter on the window function" },
+    { pattern: /\bUNNEST\s*\(/g, kind: "flag", note: "UNNEST — Postgres unnest() works on arrays only (no STRUCT arrays); verify semantics" },
+    { pattern: /\bSTRUCT\s*[(<]/g, kind: "flag", note: "STRUCT — no Postgres equivalent; consider jsonb or a composite type" },
+    { pattern: /\bARRAY\s*</g, kind: "flag", note: "ARRAY<T> type syntax — Postgres uses T[]" },
+    { pattern: /\bFORMAT_DATE\s*\(/g, kind: "flag", note: "FORMAT_DATE → TO_CHAR (different format tokens)" },
+    { pattern: /\bPARSE_DATE\s*\(/g, kind: "flag", note: "PARSE_DATE → TO_DATE (different format tokens)" },
+    { pattern: /\bFORMAT_TIMESTAMP\s*\(/g, kind: "flag", note: "FORMAT_TIMESTAMP → TO_CHAR (different format tokens)" },
+    {
+        pattern: /\b(DATE_ADD|DATE_SUB|TIMESTAMP_ADD|TIMESTAMP_SUB|DATETIME_ADD|DATETIME_SUB)\s*\(/g,
+        kind: "flag",
+        note: "BigQuery date arithmetic — Postgres uses interval arithmetic (x + INTERVAL '1 day')",
+    },
+    { pattern: /\bDATE_DIFF\s*\(/g, kind: "flag", note: "DATE_DIFF — Postgres: subtraction / EXTRACT(EPOCH FROM ...) depending on the part" },
+    { pattern: /_PARTITIONTIME|_PARTITIONDATE|_TABLE_SUFFIX/g, kind: "flag", note: "BigQuery pseudo-column has no Postgres equivalent" },
+    { pattern: /\bEXPORT\s+DATA\b/gi, kind: "flag", note: "EXPORT DATA — use a sqlanvil `type: \"export\"` action instead" },
+    { pattern: /\bEXECUTE\s+IMMEDIATE\b/gi, kind: "flag", note: "BigQuery scripting (EXECUTE IMMEDIATE) — rewrite as a PL/pgSQL DO block or operations" },
+    { pattern: /\bGENERATE_UUID\s*\(\s*\)/g, kind: "flag", note: "GENERATE_UUID() → gen_random_uuid()" },
+    {
+        pattern: /`[A-Za-z0-9_-]+\.[A-Za-z0-9_$]+\.[A-Za-z0-9_$]+`/g,
+        kind: "flag",
+        note: "backticked BigQuery FQN — use ${ref(...)} (declare it as a source) or a quoted Postgres name",
+    },
+];
+const CONFIG_BQ_KEYS = /^(\s*)(bigquery\s*:|partitionBy\s*:|clusterBy\s*:|requirePartitionFilter\s*:|partitionExpirationDays\s*:|database\s*:)/;
+const sanitize = (s) => s.toLowerCase().replace(/[^a-z0-9_]/g, "_");
+const connectionNameFor = (project) => `bq_${sanitize(project)}`;
+function convertDeclaration(source, parsed, connection, sourceDataset, filePath) {
+    const span = findConfigBlock(source);
+    const description = parsed.description
+        ? `    description: ${JSON.stringify(parsed.description)},\n`
+        : "";
+    const newConfig = `config {
+    type: "declaration",
+    connection: ${JSON.stringify(connection)},
+    schema: ${JSON.stringify(sourceDataset)},
+    name: ${JSON.stringify(parsed.name)},
+${description}    columnTypes: {
+        // SQLANVIL-MIGRATE(TODO): scaffold the columns from the live source:
+        //   sqlanvil introspect ${connection} ${sourceDataset}.${parsed.name} --output ${filePath}
+    },
+}`;
+    const content = source.slice(0, span.start) + newConfig + source.slice(span.end);
+    return {
+        content,
+        findings: [
+            {
+                line: 1,
+                kind: "rewrite",
+                note: `declaration now reads BigQuery through connection "${connection}" (runner-extract); columnTypes need introspect`,
+            },
+        ],
+    };
+}
+function convertTarget(source, jsMode) {
+    const span = jsMode ? null : findConfigBlock(source);
+    const findings = [];
+    const lines = source.split("\n");
+    const jsLines = jsMode ? new Set() : jsBlockLines(source);
+    let configStartLine = -1;
+    let configEndLine = -1;
+    if (span) {
+        configStartLine = source.slice(0, span.start).split("\n").length - 1;
+        configEndLine = source.slice(0, span.end).split("\n").length - 1;
+    }
+    const out = [];
+    let commentingConfigBlockDepth = 0;
+    let swallowLeadingComma = false;
+    for (let i = 0; i < lines.length; i++) {
+        let line = lines[i];
+        const inConfig = i >= configStartLine && i <= configEndLine && configStartLine >= 0;
+        const markersSafe = !jsMode && !jsLines.has(i);
+        const marker = (note) => inConfig ? `// SQLANVIL-MIGRATE: ${note}` : `-- SQLANVIL-MIGRATE: ${note}`;
+        const lineNo = i + 1;
+        const notesForLine = [];
+        for (const rule of GLOBAL_RULES) {
+            rule.pattern.lastIndex = 0;
+            if (rule.pattern.test(line)) {
+                line = line.replace(rule.pattern, rule.replacement);
+                findings.push({ line: lineNo, kind: "rewrite", note: rule.note });
+            }
+        }
+        if (swallowLeadingComma) {
+            swallowLeadingComma = false;
+            if (/^\s*,\s*$/.test(line)) {
+                out.push(`// ${line}`);
+                continue;
+            }
+            if (/^\s*,/.test(line)) {
+                line = line.replace(/,/, " ");
+            }
+        }
+        if (inConfig && commentingConfigBlockDepth === 0) {
+            const m = CONFIG_BQ_KEYS.exec(line);
+            if (m) {
+                const note = `BigQuery-only setting — not applicable on Postgres (partitioning/clustering → postgres: {}; project qualifiers drop)`;
+                findings.push({ line: lineNo, kind: "flag", note });
+                out.push(marker(note));
+                const opens = (line.match(/\{/g) || []).length;
+                const closes = (line.match(/\}/g) || []).length;
+                commentingConfigBlockDepth = Math.max(0, opens - closes);
+                if (commentingConfigBlockDepth === 0)
+                    swallowLeadingComma = !/,\s*$/.test(line) ? true : false;
+                out.push(`// ${line}`);
+                continue;
+            }
+        }
+        else if (commentingConfigBlockDepth > 0) {
+            const opens = (line.match(/\{/g) || []).length;
+            const closes = (line.match(/\}/g) || []).length;
+            commentingConfigBlockDepth = Math.max(0, commentingConfigBlockDepth + opens - closes);
+            if (commentingConfigBlockDepth === 0 && !/,\s*$/.test(line))
+                swallowLeadingComma = true;
+            out.push(`// ${line}`);
+            continue;
+        }
+        if (!inConfig || jsMode) {
+            for (const rule of SQL_RULES) {
+                rule.pattern.lastIndex = 0;
+                if (!rule.pattern.test(line))
+                    continue;
+                if (rule.replacement !== undefined) {
+                    rule.pattern.lastIndex = 0;
+                    line = line.replace(rule.pattern, rule.replacement);
+                    findings.push({ line: lineNo, kind: "rewrite", note: rule.note });
+                    if (markersSafe)
+                        notesForLine.push(rule.note);
+                }
+                else {
+                    findings.push({ line: lineNo, kind: "flag", note: rule.note });
+                    if (markersSafe)
+                        notesForLine.push(rule.note);
+                }
+            }
+        }
+        for (const note of notesForLine) {
+            out.push(marker(note));
+        }
+        out.push(line);
+    }
+    let content = out.join("\n");
+    if (jsMode && findings.length > 0) {
+        const noted = findings.slice(0, 12).map(f => ` *   L${f.line}: ${f.note}`);
+        const more = findings.length > 12 ? [` *   …and ${findings.length - 12} more (see migration-report.json)`] : [];
+        content =
+            ["/* SQLANVIL-MIGRATE — findings in this file (inline markers are unsafe in JS templates):", ...noted, ...more, " */"].join("\n") +
+                "\n" +
+                content;
+    }
+    return { content, findings };
+}
+function readSourceConfig(srcDir) {
+    var _a, _b, _c, _d, _e;
+    const wsPath = path__namespace.join(srcDir, "workflow_settings.yaml");
+    const djPath = path__namespace.join(srcDir, "dataform.json");
+    let raw;
+    if (fs__namespace$1.existsSync(wsPath)) {
+        raw = jsYaml.load(fs__namespace$1.readFileSync(wsPath, "utf8")) || {};
+    }
+    else if (fs__namespace$1.existsSync(djPath)) {
+        raw = JSON.parse(fs__namespace$1.readFileSync(djPath, "utf8"));
+    }
+    else {
+        throw new Error(`${srcDir} does not look like a Dataform project (no workflow_settings.yaml or dataform.json).`);
+    }
+    const s = (k) => (typeof raw[k] === "string" ? raw[k] : null);
+    return {
+        defaultProject: (_a = s("defaultProject")) !== null && _a !== void 0 ? _a : s("defaultDatabase"),
+        defaultDataset: (_c = (_b = s("defaultDataset")) !== null && _b !== void 0 ? _b : s("defaultSchema")) !== null && _c !== void 0 ? _c : "dataform",
+        defaultAssertionDataset: (_d = s("defaultAssertionDataset")) !== null && _d !== void 0 ? _d : s("assertionSchema"),
+        vars: (_e = raw.vars) !== null && _e !== void 0 ? _e : null,
+        raw,
+    };
+}
+async function migrateDataform(opts) {
+    var _a, _b, _c, _d, _e, _f, _g, _h;
+    const srcDir = path__namespace.resolve(opts.srcDir);
+    const outDir = path__namespace.resolve(opts.outDir);
+    if (containsPath(srcDir, outDir)) {
+        throw new Error("The output directory must not be inside the source project.");
+    }
+    if (containsPath(outDir, srcDir)) {
+        throw new Error("The source project must not be inside the output directory.");
+    }
+    if (fs__namespace$1.existsSync(outDir) && fs__namespace$1.readdirSync(outDir).length > 0) {
+        throw new Error(`Output directory ${outDir} is not empty.`);
+    }
+    const sourceConfig = readSourceConfig(srcDir);
+    const writer = new OutWriter(outDir);
+    skippedForSafety = new Set();
+    const files = walkFiles(srcDir);
+    const report = {
+        generator: `sqlanvil migrate-dataform ${version}`,
+        sourceDir: srcDir,
+        sourceConfig: sourceConfig.raw,
+        inventory: { sqlxFiles: 0, jsFiles: 0, byType: {} },
+        connections: [],
+        overlappingSchemas: [],
+        files: [],
+        skippedForSafety: [],
+        warnings: [],
+    };
+    const sqlxFiles = [];
+    for (const rel of files) {
+        if (!rel.endsWith(".sqlx"))
+            continue;
+        const source = fs__namespace$1.readFileSync(path__namespace.join(srcDir, rel), "utf8");
+        const span = findConfigBlock(source);
+        const parsed = span
+            ? parseSqlxConfig(span.body)
+            : { type: null, schema: null, database: null, name: null, description: null };
+        sqlxFiles.push({ rel, source, parsed });
+        report.inventory.sqlxFiles++;
+        const t = (_a = parsed.type) !== null && _a !== void 0 ? _a : "(none)";
+        report.inventory.byType[t] = ((_b = report.inventory.byType[t]) !== null && _b !== void 0 ? _b : 0) + 1;
+    }
+    const resolveDatabase = (db) => { var _a; return !db || db === "<defaultDatabase>" ? (_a = sourceConfig.defaultProject) !== null && _a !== void 0 ? _a : "UNKNOWN_PROJECT" : db; };
+    const declarationSchemas = new Set();
+    const targetSchemas = new Set();
+    const connectionMap = new Map();
+    for (const f of sqlxFiles) {
+        if (f.parsed.type === "declaration") {
+            const dataset = (_c = f.parsed.schema) !== null && _c !== void 0 ? _c : sourceConfig.defaultDataset;
+            const project = resolveDatabase(f.parsed.database);
+            declarationSchemas.add(dataset);
+            const name = connectionNameFor(project);
+            const existing = connectionMap.get(name);
+            if (existing) {
+                existing.declarationCount++;
+                if (!existing.datasets.includes(dataset))
+                    existing.datasets.push(dataset);
+            }
+            else {
+                connectionMap.set(name, {
+                    name,
+                    project,
+                    datasets: [dataset],
+                    declarationCount: 1,
+                    introspectExample: `sqlanvil introspect ${name} <dataset>.<table> --output <declaration.sqlx>`,
+                });
+            }
+        }
+        else if (f.parsed.type) {
+            targetSchemas.add((_d = f.parsed.schema) !== null && _d !== void 0 ? _d : sourceConfig.defaultDataset);
+        }
+    }
+    report.connections = [...connectionMap.values()].sort((a, b) => a.name.localeCompare(b.name));
+    report.overlappingSchemas = [...declarationSchemas].filter(s => targetSchemas.has(s)).sort();
+    for (const rel of files) {
+        const abs = path__namespace.join(srcDir, rel);
+        if (rel === "workflow_settings.yaml" || rel === "dataform.json") {
+            continue;
+        }
+        if (rel.endsWith(".sqlx")) {
+            const info = sqlxFiles.find(f => f.rel === rel);
+            if (info.parsed.type === "declaration") {
+                const dataset = (_e = info.parsed.schema) !== null && _e !== void 0 ? _e : sourceConfig.defaultDataset;
+                const project = resolveDatabase(info.parsed.database);
+                const connection = connectionNameFor(project);
+                const { content, findings } = convertDeclaration(info.source, info.parsed, connection, dataset, rel);
+                writer.write(rel, content);
+                report.files.push({
+                    file: rel,
+                    action: "declaration",
+                    type: "declaration",
+                    status: "rewritten",
+                    findings,
+                    connection,
+                });
+            }
+            else {
+                const { content, findings } = convertTarget(info.source, false);
+                writer.write(rel, content);
+                report.files.push({
+                    file: rel,
+                    action: "target",
+                    type: (_f = info.parsed.type) !== null && _f !== void 0 ? _f : "(none)",
+                    status: findings.length === 0
+                        ? "clean"
+                        : findings.some(f => f.kind === "flag")
+                            ? "flagged"
+                            : "rewritten",
+                    findings,
+                });
+            }
+            continue;
+        }
+        if (rel.endsWith(".js") && (rel.startsWith("definitions/") || rel.startsWith("includes/"))) {
+            report.inventory.jsFiles++;
+            const source = fs__namespace$1.readFileSync(abs, "utf8");
+            const { content, findings } = convertTarget(source, true);
+            writer.write(rel, content);
+            const isIncludes = rel.startsWith("includes/");
+            if (isIncludes && findings.length > 0) {
+                report.warnings.push(`includes helper ${rel} emits BigQuery-flavored SQL — review its ${findings.length} finding(s)`);
+            }
+            report.files.push({
+                file: rel,
+                action: isIncludes ? "includes" : "target",
+                type: "js",
+                status: findings.length === 0
+                    ? "clean"
+                    : findings.some(f => f.kind === "flag")
+                        ? "flagged"
+                        : "rewritten",
+                findings,
+            });
+            continue;
+        }
+        writer.write(rel, fs__namespace$1.readFileSync(abs));
+        report.files.push({ file: rel, action: "copied", status: "clean", findings: [] });
+    }
+    const settings = {
+        warehouse: "supabase",
+        defaultDataset: sourceConfig.defaultDataset.toLowerCase(),
+        defaultAssertionDataset: ((_g = sourceConfig.defaultAssertionDataset) !== null && _g !== void 0 ? _g : "sqlanvil_assertions").toLowerCase(),
+        sqlanvilCoreVersion: (_h = opts.coreVersion) !== null && _h !== void 0 ? _h : version,
+    };
+    if (sourceConfig.vars)
+        settings.vars = sourceConfig.vars;
+    if (report.connections.length > 0) {
+        settings.connections = Object.fromEntries(report.connections.map(c => {
+            var _a;
+            return [
+                c.name,
+                {
+                    platform: "bigquery",
+                    project: c.project,
+                    billingProject: (_a = sourceConfig.defaultProject) !== null && _a !== void 0 ? _a : "REPLACE_WITH_YOUR_GCP_PROJECT",
+                    mode: "runner-extract",
+                },
+            ];
+        }));
+    }
+    writer.write("workflow_settings.yaml", jsYaml.dump(settings));
+    report.skippedForSafety = [...skippedForSafety].sort();
+    if (skippedForSafety.size > 0) {
+        report.warnings.push(`Skipped ${skippedForSafety.size} file(s)/dir(s) for safety (credentials, build artifacts, VCS) — see skippedForSafety`);
+    }
+    writer.write("migration-report.json", JSON.stringify(report, null, 2));
+    writer.write("migration-report.md", renderReportMd(report));
+    return report;
+}
+function renderReportMd(r) {
+    const targets = r.files.filter(f => f.action === "target");
+    const flagged = targets.filter(f => f.status === "flagged");
+    const rewritten = targets.filter(f => f.status === "rewritten");
+    const clean = targets.filter(f => f.status === "clean");
+    const declarations = r.files.filter(f => f.action === "declaration");
+    const lines = [];
+    lines.push(`# Dataform → SQLAnvil migration report`);
+    lines.push("");
+    lines.push(`Generated by ${r.generator} from \`${r.sourceDir}\` (source untouched).`);
+    lines.push("");
+    lines.push(`## Inventory`);
+    lines.push("");
+    lines.push(`- ${r.inventory.sqlxFiles} .sqlx, ${r.inventory.jsFiles} .js`);
+    for (const [t, n] of Object.entries(r.inventory.byType).sort((a, b) => b[1] - a[1])) {
+        lines.push(`  - ${t}: ${n}`);
+    }
+    lines.push("");
+    lines.push(`## Source connections (${r.connections.length})`);
+    lines.push("");
+    lines.push(`Declarations keep reading BigQuery through named \`connections:\` in workflow_settings.yaml ` +
+        `(mode: runner-extract — no Vault, no wrappers). Each declaration needs \`columnTypes\` ` +
+        `before it can extract; scaffold them with \`sqlanvil introspect\`:`);
+    lines.push("");
+    for (const c of r.connections) {
+        lines.push(`- **${c.name}** — project \`${c.project}\`, ${c.declarationCount} declaration(s) across ` +
+            `${c.datasets.length} dataset(s): ${c.datasets.slice(0, 8).map(d => `\`${d}\``).join(", ")}` +
+            (c.datasets.length > 8 ? ", …" : ""));
+        lines.push(`  - \`${c.introspectExample}\``);
+    }
+    if (r.overlappingSchemas.length > 0) {
+        lines.push("");
+        lines.push(`## ⚠ Schemas that are BOTH source and target`);
+        lines.push("");
+        lines.push(`These schemas have declarations AND materializing actions: ${r.overlappingSchemas
+            .map(s => `\`${s}\``)
+            .join(", ")}. ` +
+            `Their declared tables are probably produced by this pipeline (or a sibling) — once those ` +
+            `producers move to Postgres, convert the declarations to plain local declarations instead ` +
+            `of BigQuery connections.`);
+    }
+    lines.push("");
+    lines.push(`## Target files (${targets.length})`);
+    lines.push("");
+    lines.push(`- clean: ${clean.length}`);
+    lines.push(`- rewritten (safe lexical changes only): ${rewritten.length}`);
+    lines.push(`- **flagged (need dialect review): ${flagged.length}** — every location is marked ` +
+        `inline with \`SQLANVIL-MIGRATE:\``);
+    lines.push("");
+    for (const f of flagged) {
+        lines.push(`- \`${f.file}\` (${f.type})`);
+        for (const finding of f.findings.filter(x => x.kind === "flag").slice(0, 6)) {
+            lines.push(`  - L${finding.line}: ${finding.note}`);
+        }
+        const more = f.findings.filter(x => x.kind === "flag").length - 6;
+        if (more > 0)
+            lines.push(`  - …and ${more} more (see migration-report.json)`);
+    }
+    lines.push("");
+    lines.push(`## Declarations (${declarations.length})`);
+    lines.push("");
+    lines.push(`Every declaration was rewritten to ride its connection. The BigQuery data does not move.`);
+    if (r.warnings.length > 0) {
+        lines.push("");
+        lines.push(`## Warnings`);
+        lines.push("");
+        for (const w of r.warnings)
+            lines.push(`- ${w}`);
+    }
+    lines.push("");
+    lines.push(`## Finish the migration`);
+    lines.push("");
+    lines.push(`1. \`sqlanvil compile\` — should already succeed.`);
+    lines.push(`2. Scaffold \`columnTypes\` for the declarations you actually read (see connections above).`);
+    lines.push(`3. \`sqlanvil validate\` against your Postgres/Supabase warehouse — PASS/FAILURE/BLOCKED per ` +
+        `model is the migration to-do list. Work the flagged files until validate is green.`);
+    lines.push(`4. This report is machine-readable (migration-report.json) — pointing an AI agent at it plus ` +
+        `the \`SQLANVIL-MIGRATE:\` markers is the fastest path through the dialect work.`);
+    lines.push("");
+    return lines.join("\n");
+}
+
 const CHECKER_SOURCE = `
 import json, os, sys
 
@@ -56125,7 +56836,7 @@ function runCli() {
             {
                 format: `init [${projectDirOption.name}] [${ProjectConfigOptions.defaultDatabase.name}]` +
                     ` [${ProjectConfigOptions.defaultLocation.name}]`,
-                description: "Create a new sqlanvil project (BigQuery, Postgres, or Supabase).",
+                description: "Create a new sqlanvil project (BigQuery, Postgres, Supabase, or MySQL/MariaDB).",
                 positionalOptions: [
                     projectDirOption,
                     positionalOption(ProjectConfigOptions.defaultDatabase.name, {
@@ -56729,6 +57440,37 @@ function runCli() {
                     }
                     return 0;
                 }
+            },
+            {
+                format: "migrate-dataform <source-dir> <out-dir>",
+                description: "Convert a Dataform/BigQuery project into a sqlanvil-on-Postgres project. The source " +
+                    "directory is READ-ONLY; the converted project + migration-report.{md,json} land in " +
+                    "<out-dir> (must be empty). Sources stay in BigQuery as named connections; target SQL " +
+                    "gets safe rewrites + inline SQLANVIL-MIGRATE markers for dialect review.",
+                positionalOptions: [
+                    positionalOption("source-dir", {
+                        describe: "The Dataform project to convert (never modified)."
+                    }),
+                    positionalOption("out-dir", {
+                        describe: "Where the converted sqlanvil project is written (created; must be empty)."
+                    })
+                ],
+                options: [],
+                processFn: async (argv) => {
+                    const report = await migrateDataform({
+                        srcDir: argv["source-dir"],
+                        outDir: argv["out-dir"]
+                    });
+                    const targets = report.files.filter(f => f.action === "target");
+                    const flagged = targets.filter(f => f.status === "flagged").length;
+                    printSuccess(`Converted ${report.inventory.sqlxFiles} .sqlx file(s): ` +
+                        `${report.connections.length} source connection(s) over ` +
+                        `${report.files.filter(f => f.action === "declaration").length} declaration(s); ` +
+                        `${targets.length} target file(s), ${flagged} flagged for dialect review.`);
+                    print(`Report: ${path__namespace.join(argv["out-dir"], "migration-report.md")}`);
+                    print(`Next: sqlanvil compile ${argv["out-dir"]}`);
+                    return 0;
+                }
             }
         ]
     })

package/package.json

@@ -64,7 +64,7 @@
     "bin": {
         "sqlanvil": "bundle.js"
     },
-    "version": "1.20.1",
+    "version": "1.22.0",
     "name": "@sqlanvil/cli",
     "description": "sqlanvil command line interface.",
     "main": "bundle.js"