@pgpmjs/diff 0.1.1

package/LICENSE

@@ -0,0 +1,23 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Dan Lynch <pyramation@gmail.com>
+Copyright (c) 2025 Constructive <developers@constructive.io>
+Copyright (c) 2020-present, Interweb, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,81 @@
+# @pgpmjs/diff
+
+`pgpm diff` engine: identity-keyed semantic schema diff between any two SQL sources — pgpm modules, SQL dumps, or live databases (dumped via `pg_dump` by the CLI).
+
+Every side normalizes to the same `DiffInputChange[]` seam consumed by the semantic diff driver in `@pgpmjs/transform`, so the comparison is source- and dial-invariant. `deltaChangesToRows` lifts the resulting deltas back onto deployable `PgpmRow`s.
+
+Used by the `pgpm diff` CLI command (`pgpm/cli`). Sibling front doors: `@pgpmjs/export` (live database), `@pgpmjs/import` (dumps).
+
+---
+
+## Education and Tutorials
+
+ 1. 🚀 [Quickstart: Getting Up and Running](https://constructive.io/learn/quickstart)
+Get started with modular databases in minutes. Install prerequisites and deploy your first module.
+
+ 2. 📦 [Modular PostgreSQL Development with Database Packages](https://constructive.io/learn/modular-postgres)
+Learn to organize PostgreSQL projects with pgpm workspaces and reusable database modules.
+
+ 3. ✏️ [Authoring Database Changes](https://constructive.io/learn/authoring-database-changes)
+Master the workflow for adding, organizing, and managing database changes with pgpm.
+
+ 4. 🧪 [End-to-End PostgreSQL Testing with TypeScript](https://constructive.io/learn/e2e-postgres-testing)
+Master end-to-end PostgreSQL testing with ephemeral databases, RLS testing, and CI/CD automation.
+
+ 5. ⚡ [Supabase Testing](https://constructive.io/learn/supabase)
+Use TypeScript-first tools to test Supabase projects with realistic RLS, policies, and auth contexts.
+
+ 6. 💧 [Drizzle ORM Testing](https://constructive.io/learn/drizzle-testing)
+Run full-stack tests with Drizzle ORM, including database setup, teardown, and RLS enforcement.
+
+ 7. 🔧 [Troubleshooting](https://constructive.io/learn/troubleshooting)
+Common issues and solutions for pgpm, PostgreSQL, and testing.
+
+## Related Constructive Tooling
+
+### 📦 Package Management
+
+* [pgpm](https://github.com/constructive-io/constructive/tree/main/pgpm/pgpm): **🖥️ PostgreSQL Package Manager** for modular Postgres development. Works with database workspaces, scaffolding, migrations, seeding, and installing database packages.
+
+### 🧪 Testing
+
+* [pgsql-test](https://github.com/constructive-io/constructive/tree/main/postgres/pgsql-test): **📊 Isolated testing environments** with per-test transaction rollbacks—ideal for integration tests, complex migrations, and RLS simulation.
+* [pglite-test](https://github.com/constructive-io/constructive/tree/main/postgres/pglite-test): **🪶 Drop-in pgsql-test replacement backed by PGlite** — in-process Postgres, no server required, instance-per-suite isolation.
+* [pgsql-seed](https://github.com/constructive-io/constructive/tree/main/postgres/pgsql-seed): **🌱 PostgreSQL seeding utilities** for CSV, JSON, SQL data loading, and pgpm deployment.
+* [supabase-test](https://github.com/constructive-io/constructive/tree/main/postgres/supabase-test): **🧪 Supabase-native test harness** preconfigured for the local Supabase stack—per-test rollbacks, JWT/role context helpers, and CI/GitHub Actions ready.
+* [graphile-test](https://github.com/constructive-io/constructive/tree/main/graphile/graphile-test): **🔐 Authentication mocking** for Graphile-focused test helpers and emulating row-level security contexts.
+* [pg-query-context](https://github.com/constructive-io/constructive/tree/main/postgres/pg-query-context): **🔒 Session context injection** to add session-local context (e.g., `SET LOCAL`) into queries—ideal for setting `role`, `jwt.claims`, and other session settings.
+
+### 🧠 Parsing & AST
+
+* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): **🔄 SQL conversion engine** that interprets and converts PostgreSQL syntax.
+* [libpg-query-node](https://www.npmjs.com/package/libpg-query): **🌉 Node.js bindings** for `libpg_query`, converting SQL into parse trees.
+* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): **📦 Protobuf parser** for parsing PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums.
+* [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): **🏷️ TypeScript enums** for PostgreSQL AST for safe and ergonomic parsing logic.
+* [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): **📝 Type definitions** for PostgreSQL AST nodes in TypeScript.
+* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): **🛠️ AST utilities** for constructing and transforming PostgreSQL syntax trees.
+
+### 📚 Documentation & Skills
+
+* [constructive-skills](https://github.com/constructive-io/constructive-skills): **📖 Platform documentation and AI agent skills** — feature catalog, blueprint reference, SDK guides (i18n, billing, limits, events, uploads, security, entities, search, AI), and deployment guides.
+
+Install skills for AI coding agents:
+
+```bash
+# All platform skills (security, blueprints, codegen, billing, etc.)
+npx skills add constructive-io/constructive-skills
+
+# Individual repo skills (pgpm, testing, CLI, search, etc.)
+npx skills add https://github.com/constructive-io/constructive --skill pgpm
+npx skills add https://github.com/constructive-io/constructive --skill constructive-testing
+```
+
+## Credits
+
+**🛠 Built by the [Constructive](https://constructive.io) team — creators of modular Postgres tooling for secure, composable backends. If you like our work, contribute on [GitHub](https://github.com/constructive-io).**
+
+## Disclaimer
+
+AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND.
+
+No developer or entity involved in creating this software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the code, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value.

package/diff-source.d.ts

@@ -0,0 +1,47 @@
+/**
+ * Input ingestion + migration emission for `pgpm diff`.
+ *
+ * A diff side can be a pgpm module directory, a raw .sql file, or a live
+ * database (handled by the CLI via pg_dump; the dump text lands here through
+ * {@link sqlToDiffChanges}). Every side normalizes to the same
+ * `DiffInputChange[]` seam the semantic diff driver consumes, so the
+ * comparison is source- and dial-invariant.
+ */
+import { PgpmRow } from '@pgpmjs/ast';
+import type { DiffInputChange, SemanticDeltaChange } from '@pgpmjs/transform';
+/** What a diff side spec resolves to. */
+export type DiffSideKind = 'module' | 'sql' | 'database';
+/** A diff side normalized to the semantic diff driver's input seam. */
+export interface DiffSide {
+    kind: DiffSideKind;
+    /** Human-readable label for messages (module name, file, db). */
+    label: string;
+    changes: DiffInputChange[];
+    warnings: string[];
+}
+/**
+ * Classify a side spec: `db:<name>` or a postgres:// / postgresql://
+ * connection string is a live database; a directory containing pgpm.plan is
+ * a module; an existing file is raw SQL.
+ */
+export declare const resolveDiffSideKind: (spec: string) => DiffSideKind;
+/**
+ * Strip the pg_dump session preamble (SET/set_config, psql meta-commands)
+ * so the remaining text is pure DDL the classifier understands. Only whole
+ * lines are stripped — SET clauses inside function bodies are untouched.
+ */
+export declare const stripDumpPreamble: (sql: string) => string;
+/** Wrap raw SQL text (a .sql file or a schema dump) as a single diff change. */
+export declare const sqlToDiffChanges: (sql: string, name: string) => DiffInputChange[];
+/**
+ * Load an on-disk diff side (module directory or .sql file). Live databases
+ * are dumped by the caller and fed through {@link sqlToDiffChanges}.
+ */
+export declare const loadDiffSideFromDisk: (spec: string) => DiffSide;
+/**
+ * Convert the semantic diff's delta changes into writable pgpm rows.
+ * Plan order is the delta order (drops first, then creates/alters); when two
+ * changes derive the same spec path, later occurrences get the alteration
+ * convention (`alterationPathFor`), with the parent added to their deps.
+ */
+export declare const deltaChangesToRows: (changes: SemanticDeltaChange[]) => PgpmRow[];

package/diff-source.js

@@ -0,0 +1,150 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || (function () {
+    var ownKeys = function(o) {
+        ownKeys = Object.getOwnPropertyNames || function (o) {
+            var ar = [];
+            for (var k in o) if (Object.prototype.hasOwnProperty.call(o, k)) ar[ar.length] = k;
+            return ar;
+        };
+        return ownKeys(o);
+    };
+    return function (mod) {
+        if (mod && mod.__esModule) return mod;
+        var result = {};
+        if (mod != null) for (var k = ownKeys(mod), i = 0; i < k.length; i++) if (k[i] !== "default") __createBinding(result, mod, k[i]);
+        __setModuleDefault(result, mod);
+        return result;
+    };
+})();
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.deltaChangesToRows = exports.loadDiffSideFromDisk = exports.sqlToDiffChanges = exports.stripDumpPreamble = exports.resolveDiffSideKind = void 0;
+const naming_spec_1 = require("@pgpmjs/naming-spec");
+const transform_1 = require("@pgpmjs/transform");
+const fs = __importStar(require("fs"));
+const path = __importStar(require("path"));
+/**
+ * Classify a side spec: `db:<name>` or a postgres:// / postgresql://
+ * connection string is a live database; a directory containing pgpm.plan is
+ * a module; an existing file is raw SQL.
+ */
+const resolveDiffSideKind = (spec) => {
+    if (/^postgres(ql)?:\/\//.test(spec) || spec.startsWith('db:'))
+        return 'database';
+    const resolved = path.resolve(spec);
+    if (fs.existsSync(resolved)) {
+        if (fs.statSync(resolved).isDirectory()) {
+            if (fs.existsSync(path.join(resolved, 'pgpm.plan')))
+                return 'module';
+            throw new Error(`${spec}: directory has no pgpm.plan (not a pgpm module)`);
+        }
+        return 'sql';
+    }
+    throw new Error(`${spec}: not a module directory, .sql file, db:<name>, or connection string`);
+};
+exports.resolveDiffSideKind = resolveDiffSideKind;
+/** Lines pg_dump emits that carry no schema content. */
+const DUMP_NOISE = [
+    /^\\/, // psql meta-commands (\connect, \restrict, ...)
+    /^SET\s+[\w.]+\s*(=|TO)\s*[^;]*;?\s*$/i, // session SET preamble
+    /^SELECT\s+pg_catalog\.set_config\(/i
+];
+/**
+ * Strip the pg_dump session preamble (SET/set_config, psql meta-commands)
+ * so the remaining text is pure DDL the classifier understands. Only whole
+ * lines are stripped — SET clauses inside function bodies are untouched.
+ */
+const stripDumpPreamble = (sql) => sql
+    .split('\n')
+    .filter(line => !DUMP_NOISE.some(re => re.test(line.trim())))
+    .join('\n')
+    .trim();
+exports.stripDumpPreamble = stripDumpPreamble;
+/** Wrap raw SQL text (a .sql file or a schema dump) as a single diff change. */
+const sqlToDiffChanges = (sql, name) => [
+    {
+        name,
+        dependencies: [],
+        deploy: (0, exports.stripDumpPreamble)((0, transform_1.stripTransactionWrapper)(sql))
+    }
+];
+exports.sqlToDiffChanges = sqlToDiffChanges;
+/**
+ * Load an on-disk diff side (module directory or .sql file). Live databases
+ * are dumped by the caller and fed through {@link sqlToDiffChanges}.
+ */
+const loadDiffSideFromDisk = (spec) => {
+    const kind = (0, exports.resolveDiffSideKind)(spec);
+    if (kind === 'database') {
+        throw new Error(`${spec}: live database sides must be dumped by the caller`);
+    }
+    const resolved = path.resolve(spec);
+    if (kind === 'module') {
+        const source = (0, transform_1.loadModuleSource)(resolved);
+        return {
+            kind,
+            label: source.name,
+            changes: source.changes.map(change => ({
+                name: change.name,
+                dependencies: change.dependencies,
+                deploy: change.deploy
+            })),
+            warnings: source.warnings
+        };
+    }
+    return {
+        kind,
+        label: path.basename(resolved),
+        changes: (0, exports.sqlToDiffChanges)(fs.readFileSync(resolved, 'utf-8'), path.basename(resolved, '.sql')),
+        warnings: []
+    };
+};
+exports.loadDiffSideFromDisk = loadDiffSideFromDisk;
+/**
+ * Convert the semantic diff's delta changes into writable pgpm rows.
+ * Plan order is the delta order (drops first, then creates/alters); when two
+ * changes derive the same spec path, later occurrences get the alteration
+ * convention (`alterationPathFor`), with the parent added to their deps.
+ */
+const deltaChangesToRows = (changes) => {
+    const counters = new Map();
+    const taken = new Set();
+    return changes.map(change => {
+        let deploy = change.name;
+        const deps = [...change.dependencies];
+        if (taken.has(deploy)) {
+            let n = (counters.get(change.name) ?? 0) + 1;
+            while (taken.has((0, naming_spec_1.alterationPathFor)(change.name, n)))
+                n++;
+            counters.set(change.name, n);
+            deploy = (0, naming_spec_1.alterationPathFor)(change.name, n);
+            if (!deps.includes(change.name))
+                deps.push(change.name);
+        }
+        taken.add(deploy);
+        return {
+            name: deploy,
+            deploy,
+            deps: deps.filter(dep => taken.has(dep) && dep !== deploy),
+            content: change.deploy,
+            revert: change.revert,
+            verify: change.verify
+        };
+    });
+};
+exports.deltaChangesToRows = deltaChangesToRows;

package/esm/diff-source.js

@@ -0,0 +1,109 @@
+import { alterationPathFor } from '@pgpmjs/naming-spec';
+import { loadModuleSource, stripTransactionWrapper } from '@pgpmjs/transform';
+import * as fs from 'fs';
+import * as path from 'path';
+/**
+ * Classify a side spec: `db:<name>` or a postgres:// / postgresql://
+ * connection string is a live database; a directory containing pgpm.plan is
+ * a module; an existing file is raw SQL.
+ */
+export const resolveDiffSideKind = (spec) => {
+    if (/^postgres(ql)?:\/\//.test(spec) || spec.startsWith('db:'))
+        return 'database';
+    const resolved = path.resolve(spec);
+    if (fs.existsSync(resolved)) {
+        if (fs.statSync(resolved).isDirectory()) {
+            if (fs.existsSync(path.join(resolved, 'pgpm.plan')))
+                return 'module';
+            throw new Error(`${spec}: directory has no pgpm.plan (not a pgpm module)`);
+        }
+        return 'sql';
+    }
+    throw new Error(`${spec}: not a module directory, .sql file, db:<name>, or connection string`);
+};
+/** Lines pg_dump emits that carry no schema content. */
+const DUMP_NOISE = [
+    /^\\/, // psql meta-commands (\connect, \restrict, ...)
+    /^SET\s+[\w.]+\s*(=|TO)\s*[^;]*;?\s*$/i, // session SET preamble
+    /^SELECT\s+pg_catalog\.set_config\(/i
+];
+/**
+ * Strip the pg_dump session preamble (SET/set_config, psql meta-commands)
+ * so the remaining text is pure DDL the classifier understands. Only whole
+ * lines are stripped — SET clauses inside function bodies are untouched.
+ */
+export const stripDumpPreamble = (sql) => sql
+    .split('\n')
+    .filter(line => !DUMP_NOISE.some(re => re.test(line.trim())))
+    .join('\n')
+    .trim();
+/** Wrap raw SQL text (a .sql file or a schema dump) as a single diff change. */
+export const sqlToDiffChanges = (sql, name) => [
+    {
+        name,
+        dependencies: [],
+        deploy: stripDumpPreamble(stripTransactionWrapper(sql))
+    }
+];
+/**
+ * Load an on-disk diff side (module directory or .sql file). Live databases
+ * are dumped by the caller and fed through {@link sqlToDiffChanges}.
+ */
+export const loadDiffSideFromDisk = (spec) => {
+    const kind = resolveDiffSideKind(spec);
+    if (kind === 'database') {
+        throw new Error(`${spec}: live database sides must be dumped by the caller`);
+    }
+    const resolved = path.resolve(spec);
+    if (kind === 'module') {
+        const source = loadModuleSource(resolved);
+        return {
+            kind,
+            label: source.name,
+            changes: source.changes.map(change => ({
+                name: change.name,
+                dependencies: change.dependencies,
+                deploy: change.deploy
+            })),
+            warnings: source.warnings
+        };
+    }
+    return {
+        kind,
+        label: path.basename(resolved),
+        changes: sqlToDiffChanges(fs.readFileSync(resolved, 'utf-8'), path.basename(resolved, '.sql')),
+        warnings: []
+    };
+};
+/**
+ * Convert the semantic diff's delta changes into writable pgpm rows.
+ * Plan order is the delta order (drops first, then creates/alters); when two
+ * changes derive the same spec path, later occurrences get the alteration
+ * convention (`alterationPathFor`), with the parent added to their deps.
+ */
+export const deltaChangesToRows = (changes) => {
+    const counters = new Map();
+    const taken = new Set();
+    return changes.map(change => {
+        let deploy = change.name;
+        const deps = [...change.dependencies];
+        if (taken.has(deploy)) {
+            let n = (counters.get(change.name) ?? 0) + 1;
+            while (taken.has(alterationPathFor(change.name, n)))
+                n++;
+            counters.set(change.name, n);
+            deploy = alterationPathFor(change.name, n);
+            if (!deps.includes(change.name))
+                deps.push(change.name);
+        }
+        taken.add(deploy);
+        return {
+            name: deploy,
+            deploy,
+            deps: deps.filter(dep => taken.has(dep) && dep !== deploy),
+            content: change.deploy,
+            revert: change.revert,
+            verify: change.verify
+        };
+    });
+};

package/esm/index.js

@@ -0,0 +1 @@
+export { deltaChangesToRows, loadDiffSideFromDisk, resolveDiffSideKind, sqlToDiffChanges, stripDumpPreamble } from './diff-source';

package/index.d.ts

@@ -0,0 +1,2 @@
+export type { DiffSide, DiffSideKind } from './diff-source';
+export { deltaChangesToRows, loadDiffSideFromDisk, resolveDiffSideKind, sqlToDiffChanges, stripDumpPreamble } from './diff-source';

package/index.js

@@ -0,0 +1,9 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.stripDumpPreamble = exports.sqlToDiffChanges = exports.resolveDiffSideKind = exports.loadDiffSideFromDisk = exports.deltaChangesToRows = void 0;
+var diff_source_1 = require("./diff-source");
+Object.defineProperty(exports, "deltaChangesToRows", { enumerable: true, get: function () { return diff_source_1.deltaChangesToRows; } });
+Object.defineProperty(exports, "loadDiffSideFromDisk", { enumerable: true, get: function () { return diff_source_1.loadDiffSideFromDisk; } });
+Object.defineProperty(exports, "resolveDiffSideKind", { enumerable: true, get: function () { return diff_source_1.resolveDiffSideKind; } });
+Object.defineProperty(exports, "sqlToDiffChanges", { enumerable: true, get: function () { return diff_source_1.sqlToDiffChanges; } });
+Object.defineProperty(exports, "stripDumpPreamble", { enumerable: true, get: function () { return diff_source_1.stripDumpPreamble; } });

package/package.json

@@ -0,0 +1,50 @@
+{
+  "name": "@pgpmjs/diff",
+  "version": "0.1.1",
+  "author": "Constructive <developers@constructive.io>",
+  "description": "pgpm diff — identity-keyed semantic schema diff between pgpm modules, SQL dumps, and live databases",
+  "main": "index.js",
+  "module": "esm/index.js",
+  "types": "index.d.ts",
+  "homepage": "https://github.com/constructive-io/constructive",
+  "license": "MIT",
+  "publishConfig": {
+    "access": "public",
+    "directory": "dist"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/constructive-io/constructive"
+  },
+  "bugs": {
+    "url": "https://github.com/constructive-io/constructive/issues"
+  },
+  "scripts": {
+    "clean": "makage clean",
+    "prepack": "npm run build",
+    "build": "makage build",
+    "build:dev": "makage build --dev",
+    "lint": "eslint . --fix",
+    "test": "jest",
+    "test:watch": "jest --watch"
+  },
+  "keywords": [
+    "sql",
+    "postgres",
+    "postgresql",
+    "pgpm",
+    "pgpmjs",
+    "diff",
+    "schema",
+    "migration"
+  ],
+  "devDependencies": {
+    "makage": "^0.3.0"
+  },
+  "dependencies": {
+    "@pgpmjs/ast": "^0.10.3",
+    "@pgpmjs/naming-spec": "^0.1.2",
+    "@pgpmjs/transform": "^0.19.0"
+  },
+  "gitHead": "c20b6be96b39bf71c0e2df0f4b51e05deacd006b"
+}