dbt-sugar 0.0.0-stage → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Dat Nguyen
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,78 @@
1
- # Temporary Holding Version
1
+ <p align="center">
2
+ <img src="docs/assets/logo.svg" alt="dbt sugar logo" width="220" height="88">
3
+ </p>
2
4
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
5
+ <p align="center"><b>A live local page for dbt v2 runs: what's running, what's next, what to fix, and Dinor Sugar, a pixel dinosaur forever chasing the dbt logo.</b></p>
6
+
7
+ <p align="center">
8
+ <a href="https://github.com/datnguye/dbt-sugar/actions/workflows/ci.yml"><img src="https://github.com/datnguye/dbt-sugar/actions/workflows/ci.yml/badge.svg" alt="CI"></a>
9
+ <a href="docs/spec.md"><img src="https://img.shields.io/badge/docs-spec-blue?style=flat&logo=gitbook&logoColor=white" alt="docs"></a>
10
+ <img src="https://img.shields.io/badge/dbt-v2%20(Fusion)-FF694B?logo=dbt&logoColor=white" alt="dbt v2">
11
+ <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-yellow.svg" alt="License: MIT"></a>
12
+ <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node-22%2B-339933.svg?style=flat&logo=node.js&logoColor=white" alt="node"></a>
13
+ </p>
14
+
15
+ Run `sugar -- dbt build` and a page opens next to your terminal. It shows the phases dbt has finished, a `done/total` progress bar, the nodes running right now (and whether they are slower than last time), what's queued and what each one is waiting on, and a short list of suggestions to make the project healthier. Dinor Sugar, a pixel dinosaur, chases the dbt logo while things go well, sweats at warnings, panics when an error lets the logo escape, and catches it (sunglasses on) when everything is green. It cannot fix your SQL, but it does try.
16
+
17
+ Everything comes from files dbt v2 already writes: `logs/otel*.jsonl`, `logs/dbt.log` and `target/manifest.json`. No API keys, no accounts, nothing leaves your machine.
18
+
19
+ <p align="center">
20
+ <img src="docs/assets/img/demo.gif" alt="dbt sugar: two dbt projects replaying live, the project switcher, a failed run with its blocked nodes, and the Diff tab generating base artifacts for a branch" width="880">
21
+ </p>
22
+
23
+ ## Why dbt sugar?
24
+
25
+ dbt v2 (Fusion) is fast, and its terminal output scrolls by just as fast. When a build has hundreds of nodes, the questions are always the same: *what is it doing now*, *what is it stuck behind*, *will that model that failed yesterday take half the DAG down with it again*, and *what should I fix first*. dbt sugar answers those from the OTel log, live, and remembers past runs in a small local DuckDB file so "slower than usual" and "flaky" actually mean something.
26
+
27
+ ## Install
28
+
29
+ ```bash
30
+ npm install -g dbt-sugar
31
+ ```
32
+
33
+ Requires Node.js 22+ and dbt v2 (Fusion).
34
+
35
+ ## Quickstart
36
+
37
+ ```bash
38
+ sugar -- dbt build --select +orders # run dbt and watch it at http://127.0.0.1:4322
39
+ sugar # watch runs started elsewhere in this project
40
+ sugar -d ./project_a -d ./project_b # several projects; switch between them from the header
41
+ ```
42
+
43
+ When sugar starts dbt for you it sets `DBT_OTEL_FILE_NAME=otel.jsonl`. For runs you start yourself, set it too, since dbt only writes the OTel log when asked. Running several dbt processes at once (hello, agents)? Give each its own log folder, for example `DBT_LOG_PATH=logs/run-$$`, so their `dbt.log` lines don't interleave; `sugar -- dbt …` does this for you. Sugar reads `logs/` and its subfolders and shows each invocation separately.
44
+
45
+ History lives in `~/.dbt-sugar/history.duckdb` (or `$DBT_SUGAR_HOME`). It stores run and node metadata only, no SQL, no data. `sugar --help` lists every option.
46
+
47
+ ## Review a branch: the Diff tab
48
+
49
+ Big PRs are where reviewers lose the plot. The **Diff** tab compares your branch's `target/manifest.json` with the one from `main` and shows what actually changed: added, modified and removed nodes with the reason (SQL, materialization, contract, columns and their types, dependencies, tests), a SQL diff, what sits downstream, a short review checklist, and the `dbt build --select …+` command that rebuilds exactly that.
50
+
51
+ The breaking stuff floats to the top: models removed while something still uses them, column changes on enforced contracts, incremental config changes that need a `--full-refresh`.
52
+
53
+ Pick the **base** branch (defaults to the repo's default branch, e.g. `origin/main`) and the **compare** side (defaults to your current branch's working tree) right on the page. If there are no dbt artifacts for that commit yet, the page shows the exact command and a **Generate** button: sugar checks the commit out into a temporary git worktree and runs `dbt parse --target-path ~/.dbt-sugar/base/<sha>/<project>` there, then removes the worktree. Artifacts are cached per commit, so it only happens once. Run the shown command yourself if you prefer; the page picks the result up either way.
54
+
55
+ It needs whatever `dbt parse` needs (a profile) and reuses your installed `dbt_packages`. The working-tree side is your `target/manifest.json`; if you've never parsed, the page offers Generate for that side too, writing only into sugar's cache (never your `target/` or `logs/`). Nothing has to have run: the Diff tab only needs git and a parse.
56
+
57
+ ## Use it as a library
58
+
59
+ The page is one consumer. The same pieces are exported for hosts such as [Agent Cage](https://github.com/datnguye/agent-cage) that watch many dbt processes:
60
+
61
+ ```js
62
+ import { createDbtSource, createSnapshotBuilder } from 'dbt-sugar';
63
+ import { mascotSvg } from 'dbt-sugar/mascot';
64
+
65
+ const source = createDbtSource({ projectDir: '/path/to/project' });
66
+ const builder = createSnapshotBuilder({ sources: [source], store: null });
67
+ const snapshot = await builder.build();
68
+ ```
69
+
70
+ `store` is optional: pass `null` to skip history, the bundled DuckDB store (`createStore`), or anything with `saveRun`, `previousRun` and `stats`. The full behaviour lives in the **[spec](docs/spec.md)**; how the pieces fit together, from `sugar` starting up to a Diff click in the page, is in **[architecture](docs/architecture.html)** (an animated page; open it in a browser).
71
+
72
+ ## Contributing
73
+
74
+ Bugs, ideas and new suggestion rules are welcome; see the **[Contributing Guide](CONTRIBUTING.md)** and the **[decision records](docs/decisions/README.md)**. `npm run verify` runs the same gates as CI: syntax and layering checks, unit and e2e tests at 100% coverage, and Playwright against a replay of a real jaffle-shop build (`npm run playground` to watch that replay yourself).
75
+
76
+ ## License
77
+
78
+ [MIT](./LICENSE) © Dat Nguyen
package/package.json CHANGED
@@ -1,6 +1,62 @@
1
1
  {
2
2
  "name": "dbt-sugar",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
6
- }
3
+ "version": "1.0.0",
4
+ "description": "A live local page for dbt v2 runs: what's running, what's next, what to fix, a diff for reviewers, and Dinor Sugar chasing the dbt logo.",
5
+ "keywords": [
6
+ "dbt",
7
+ "dbt-fusion",
8
+ "dbt-v2",
9
+ "opentelemetry",
10
+ "monitoring",
11
+ "dashboard",
12
+ "cli"
13
+ ],
14
+ "homepage": "https://github.com/datnguye/dbt-sugar#readme",
15
+ "bugs": {
16
+ "url": "https://github.com/datnguye/dbt-sugar/issues"
17
+ },
18
+ "repository": {
19
+ "type": "git",
20
+ "url": "git+https://github.com/datnguye/dbt-sugar.git"
21
+ },
22
+ "license": "MIT",
23
+ "type": "module",
24
+ "exports": {
25
+ ".": "./src/index.js",
26
+ "./mascot": "./src/web/lib/mascot.js",
27
+ "./view": "./src/web/lib/view.js",
28
+ "./style.css": "./src/web/style.css",
29
+ "./package.json": "./package.json"
30
+ },
31
+ "bin": {
32
+ "sugar": "src/bin/sugar.js"
33
+ },
34
+ "files": [
35
+ "src",
36
+ "!src/scripts",
37
+ "LICENSE",
38
+ "README.md"
39
+ ],
40
+ "engines": {
41
+ "node": ">=22"
42
+ },
43
+ "scripts": {
44
+ "start": "node src/bin/sugar.js",
45
+ "playground": "node test/playground/run.js",
46
+ "check": "node src/scripts/check.mjs",
47
+ "test": "node --test --experimental-test-coverage --test-coverage-include='src/**' --test-coverage-exclude='src/bin/**' --test-coverage-exclude='src/scripts/**' --test-coverage-exclude='src/web/app.js' --test-coverage-lines=100 --test-coverage-branches=100 --test-coverage-functions=100 'test/unit/**/*.test.js' 'test/e2e/*.test.js'",
48
+ "test:unit": "node --test 'test/unit/**/*.test.js'",
49
+ "test:ui": "playwright test",
50
+ "verify": "npm run check && npm test && npm run test:ui",
51
+ "demo:gif": "node src/scripts/demo-gif.mjs",
52
+ "prepare": "git config core.hooksPath .githooks 2>/dev/null || true",
53
+ "ux:shots": "node src/scripts/ux-shots.mjs",
54
+ "spec-guard": "node src/scripts/spec-guard.mjs"
55
+ },
56
+ "devDependencies": {
57
+ "@playwright/test": "^1.56.1"
58
+ },
59
+ "dependencies": {
60
+ "@duckdb/node-api": "2.0.0-r.1"
61
+ }
62
+ }
@@ -0,0 +1,6 @@
1
+ #!/usr/bin/env node
2
+ import { createRequire } from 'node:module';
3
+ import { run } from '../cli/start.js';
4
+
5
+ const { version } = createRequire(import.meta.url)('../../package.json');
6
+ run(process.argv.slice(2), { version });
@@ -0,0 +1,50 @@
1
+ import { parseArgs } from 'node:util';
2
+
3
+ const LOOPBACK = new Set(['127.0.0.1', 'localhost', '::1']);
4
+
5
+ export const USAGE = `Usage: sugar [options] [-- <dbt command>]
6
+
7
+ Watch dbt v2 runs live in the browser: what's running, what's next, what to fix.
8
+
9
+ Options:
10
+ -d, --project-dir <dir> dbt project to watch (repeatable; default .)
11
+ --no-diff turn off the Diff tab (pick branches and build their artifacts on the page)
12
+ -p, --port <n> port to listen on (default 4322, 0 = any free port)
13
+ --host <h> loopback address to bind (default 127.0.0.1)
14
+ --no-open do not open the browser
15
+ -h, --help show this help
16
+ -v, --version print the version
17
+
18
+ Examples:
19
+ sugar -- dbt build --select +orders run dbt and watch it (sets DBT_OTEL_FILE_NAME)
20
+ sugar watch dbt runs started elsewhere
21
+
22
+ Runs started elsewhere need DBT_OTEL_FILE_NAME=otel.jsonl (or any otel*.jsonl name,
23
+ one per process when several dbt processes run at once).`;
24
+
25
+ export function parseCli(argv) {
26
+ const dash = argv.indexOf('--');
27
+ const own = dash === -1 ? argv : argv.slice(0, dash);
28
+ const command = dash === -1 ? [] : argv.slice(dash + 1);
29
+ const { values } = parseArgs({
30
+ args: own,
31
+ allowNegative: true,
32
+ options: {
33
+ 'project-dir': { type: 'string', short: 'd', multiple: true },
34
+ diff: { type: 'boolean', default: true },
35
+ port: { type: 'string', short: 'p' },
36
+ host: { type: 'string', default: '127.0.0.1' },
37
+ open: { type: 'boolean', default: true },
38
+ help: { type: 'boolean', short: 'h', default: false },
39
+ version: { type: 'boolean', short: 'v', default: false }
40
+ }
41
+ });
42
+ const port = values.port === undefined ? 4322 : Number(values.port);
43
+ if (!Number.isInteger(port) || port < 0 || port > 65535) throw new Error(`--port must be 0-65535, got "${values.port}"`);
44
+ if (!LOOPBACK.has(values.host)) throw new Error(`--host must be a loopback address (${[...LOOPBACK].join(', ')})`);
45
+ const projectDirs = values['project-dir'] ?? ['.'];
46
+ if (command.length && projectDirs.length > 1) throw new Error('running a dbt command needs exactly one --project-dir');
47
+ if (dash !== -1 && !command.length) throw new Error('nothing to run after --');
48
+ const diff = values.diff ? {} : null;
49
+ return { host: values.host, open: values.open, help: values.help, version: values.version, port, portExplicit: values.port !== undefined, projectDirs, command, diff };
50
+ }
@@ -0,0 +1,14 @@
1
+ import { spawn } from 'node:child_process';
2
+
3
+ export function openBrowser(url, platform = process.platform, run = spawn) {
4
+ const [command, args] =
5
+ platform === 'darwin' ? ['open', [url]] : platform === 'win32' ? ['cmd', ['/c', 'start', '', url]] : ['xdg-open', [url]];
6
+ try {
7
+ const child = run(command, args, { stdio: 'ignore', detached: true });
8
+ child.on('error', () => {});
9
+ child.unref();
10
+ } catch {
11
+ return false;
12
+ }
13
+ return true;
14
+ }
@@ -0,0 +1,139 @@
1
+ import { spawn as spawnProcess } from 'node:child_process';
2
+ import crypto from 'node:crypto';
3
+ import path from 'node:path';
4
+ import { fileURLToPath } from 'node:url';
5
+ import { createSnapshotBuilder } from '../core/builder.js';
6
+ import { sugarHome } from '../core/home.js';
7
+ import { REF } from '../dbt/base.js';
8
+ import { createDbtSource } from '../dbt/source.js';
9
+ import { createServer } from '../server/http.js';
10
+ import { createWatcher } from '../server/watch.js';
11
+ import { parseCli, USAGE } from './args.js';
12
+ import { openBrowser } from './browser.js';
13
+
14
+ const PORT_TRIES = 10;
15
+ const WEB_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', 'web');
16
+
17
+ export async function openStore({ env, load = () => import('../history/store.js') }) {
18
+ try {
19
+ const { createStore } = await load();
20
+ return createStore({ dir: sugarHome(env) });
21
+ } catch {
22
+ return null;
23
+ }
24
+ }
25
+
26
+ async function listenWithFallback(server, options, port = options.port) {
27
+ try {
28
+ return await server.listen(port, options.host);
29
+ } catch (error) {
30
+ const last = options.portExplicit || options.port === 0 ? options.port : options.port + PORT_TRIES - 1;
31
+ if (error.code !== 'EADDRINUSE' || port >= last) throw error;
32
+ return listenWithFallback(server, options, port + 1);
33
+ }
34
+ }
35
+
36
+ export function runDbt(command, { cwd, env, logDir, spawn = spawnProcess, now = Date.now, onExit }) {
37
+ const logPath = env.DBT_LOG_PATH || path.join(logDir, `sugar-${now()}-${process.pid}`);
38
+ const child = spawn(command[0], command.slice(1), { cwd, stdio: 'inherit', env: { ...env, DBT_LOG_PATH: logPath, DBT_OTEL_FILE_NAME: env.DBT_OTEL_FILE_NAME || 'otel.jsonl' } });
39
+ let done = false;
40
+ const finish = (code) => {
41
+ if (done) return;
42
+ done = true;
43
+ onExit(code);
44
+ };
45
+ child.on('error', () => finish(127));
46
+ child.on('exit', (code) => finish(code ?? 1));
47
+ return child;
48
+ }
49
+
50
+ function validRef(ref) {
51
+ return ref === null || (typeof ref === 'string' && REF.test(ref));
52
+ }
53
+
54
+ export function applySelection(sources, { project, action = 'select', side = null, base = null, head = null } = {}) {
55
+ const source = sources.find((s) => s.dir === project);
56
+ if (!source) return false;
57
+ if (action === 'build') return source.buildDiff(side);
58
+ if (action !== 'select' || !validRef(base) || !validRef(head)) return false;
59
+ return source.selectDiff({ base, head });
60
+ }
61
+
62
+ export async function start(options, { env = process.env, log = console.log, open = openBrowser, spawn = spawnProcess, store = openStore, now = Date.now } = {}) {
63
+ const diff = options.diff && { ...options.diff, cacheDir: path.join(sugarHome(env), 'base') };
64
+ const sources = options.projectDirs.map((projectDir) => createDbtSource({ projectDir, env, diff, archiveRoot: path.join(sugarHome(env), 'runs'), onChange: () => refresh() }));
65
+ const history = await store({ env });
66
+ const builder = createSnapshotBuilder({ sources, store: history, now });
67
+ const server = createServer({ publicDir: WEB_DIR, token: crypto.randomBytes(16).toString('hex'), onSelect: (body) => applySelection(sources, body) });
68
+ let pending = null;
69
+ let again = false;
70
+ function refresh() {
71
+ if (pending) {
72
+ again = true;
73
+ return pending;
74
+ }
75
+ pending = builder.build().then(server.publish).finally(() => {
76
+ pending = null;
77
+ if (again) {
78
+ again = false;
79
+ refresh();
80
+ }
81
+ });
82
+ return pending;
83
+ }
84
+ await refresh();
85
+ const port = await listenWithFallback(server, options);
86
+ const watcher = createWatcher({
87
+ dirs: () => sources.flatMap((s) => s.watchDirs()),
88
+ onChange() {
89
+ refresh();
90
+ watcher.sync();
91
+ }
92
+ });
93
+ const url = `http://${options.host === '::1' ? '[::1]' : options.host}:${port}/`;
94
+ log(`🍬 dbt sugar is open at ${url}`);
95
+ for (const source of sources) log(` watching ${source.name}: ${source.logDir}`);
96
+ log(history ? ` history: ${history.file}` : ' history: off (DuckDB unavailable)');
97
+ if (options.open) open(url);
98
+ const app = { url, port, exitCode: null, refresh };
99
+ if (options.command.length) {
100
+ app.child = runDbt(options.command, {
101
+ cwd: sources[0].dir,
102
+ logDir: sources[0].logDir,
103
+ env,
104
+ spawn,
105
+ onExit(code) {
106
+ app.exitCode = code;
107
+ refresh();
108
+ log(code === 127 ? `sugar: could not start "${options.command[0]}"` : ` ${options.command[0]} exited with ${code}; the page stays open, Ctrl-C to quit`);
109
+ }
110
+ });
111
+ }
112
+ app.close = async () => {
113
+ watcher.close();
114
+ await server.close();
115
+ };
116
+ return app;
117
+ }
118
+
119
+ export async function run(argv, { log = console.log, error = console.error, exit = process.exit, version = '0.0.0', ...deps } = {}) {
120
+ let options;
121
+ try {
122
+ options = parseCli(argv);
123
+ } catch (problem) {
124
+ error(`sugar: ${problem.message}\n\n${USAGE}`);
125
+ return exit(2);
126
+ }
127
+ if (options.help) return log(USAGE);
128
+ if (options.version) return log(version);
129
+ try {
130
+ const app = await start(options, { log, ...deps });
131
+ const stop = () => app.close().then(() => exit(app.exitCode ?? 0));
132
+ process.once('SIGINT', stop);
133
+ process.once('SIGTERM', stop);
134
+ return app;
135
+ } catch (problem) {
136
+ error(problem.code === 'EADDRINUSE' ? `sugar: port ${options.port} is busy; try --port 0` : `sugar: ${problem.message}`);
137
+ return exit(1);
138
+ }
139
+ }
@@ -0,0 +1,67 @@
1
+ import { diffManifests } from './diff.js';
2
+ import { runView } from './snapshot.js';
3
+ import { suggest } from './suggest.js';
4
+
5
+ const RECENT_RUNS = 10;
6
+
7
+ function byActivity(a, b) {
8
+ const live = (r) => (r.status === 'running' ? 0 : 1);
9
+ return live(a) - live(b) || b.startedAt - a.startedAt;
10
+ }
11
+
12
+ function safely(promise, fallback) {
13
+ return promise.then(
14
+ (value) => value,
15
+ () => fallback
16
+ );
17
+ }
18
+
19
+ function diffView({ base, head, git, manifest: worktree }, memo) {
20
+ if (!base) return null;
21
+ const { manifest: baseManifest, ...info } = base;
22
+ const manifest = head ? head.manifest : worktree;
23
+ const view = { ...info, side: 'base', head: head?.ref ? { ref: head.ref, sha: head.sha, status: head.status } : null, compareSource: head?.source ?? 'target', compareWarning: head?.warning ?? null, git, baseAt: baseManifest?.generatedAt ?? null, currentAt: manifest?.generatedAt ?? null, result: null };
24
+ if (!baseManifest) return view;
25
+ if (head && head.status !== 'ready') return { ...view, status: head.status, error: head.error, side: 'head', out: head.out, warning: head.warning };
26
+ if (!manifest) return { ...view, status: 'error', error: 'no target/manifest.json yet: run dbt parse or dbt build' };
27
+ if (memo.base !== baseManifest || memo.current !== manifest) Object.assign(memo, { base: baseManifest, current: manifest, result: diffManifests(baseManifest, manifest) });
28
+ return { ...view, result: memo.result };
29
+ }
30
+
31
+ export function createSnapshotBuilder({ sources, store = null, now = Date.now }) {
32
+ const saved = new Set();
33
+ const memos = new Map(sources.map((source) => [source, {}]));
34
+
35
+ async function project(source) {
36
+ const data = source.read();
37
+ for (const run of data.runs.filter((r) => r.status !== 'running' && !saved.has(r.id))) {
38
+ saved.add(run.id);
39
+ if (store) await safely(store.saveRun(source.dir, run), null);
40
+ }
41
+ const ordered = [...data.runs].sort(byActivity).slice(0, RECENT_RUNS);
42
+ const runs = [];
43
+ for (const run of ordered) {
44
+ const previous = store ? await safely(store.previousRun(source.dir, run.command, run.startedAt), null) : null;
45
+ runs.push({ ...runView(run, { manifest: data.manifest, previous }), log: run.log ?? data.log.get(run.invocationId) ?? [] });
46
+ }
47
+ const history = store ? await safely(store.stats(source.dir), {}) : {};
48
+ return {
49
+ name: source.name,
50
+ dir: source.dir,
51
+ otel: data.found,
52
+ untracked: data.untracked ?? null,
53
+ version: data.manifest?.version ?? null,
54
+ runs,
55
+ suggestions: suggest({ run: ordered[0] ?? null, manifest: data.manifest, history }),
56
+ diff: diffView(data, memos.get(source))
57
+ };
58
+ }
59
+
60
+ return {
61
+ async build() {
62
+ const projects = [];
63
+ for (const source of sources) projects.push(await project(source));
64
+ return { generatedAt: now(), history: Boolean(store), projects };
65
+ }
66
+ };
67
+ }
@@ -0,0 +1,163 @@
1
+ const DIFFABLE = new Set(['model', 'snapshot', 'seed', 'source', 'exposure', 'analysis', 'unit_test', 'test']);
2
+ const RANK = { model: 0, snapshot: 1, seed: 2, source: 3, exposure: 4, analysis: 5, unit_test: 6, test: 7 };
3
+ const TEST_TYPES = new Set(['test', 'unit_test']);
4
+ const BUILDABLE = new Set(['model', 'snapshot', 'seed']);
5
+ const CODE_LIMIT = 1500;
6
+ const CONTEXT = 3;
7
+ const CONFIG_LABEL = { materialized: 'materialized', uniqueKey: 'unique_key', strategy: 'incremental_strategy', onSchemaChange: 'on_schema_change', contract: 'contract.enforced' };
8
+
9
+ export function lineDiff(before, after) {
10
+ const a = before ? before.split('\n') : [];
11
+ const b = after ? after.split('\n') : [];
12
+ if (a.length > CODE_LIMIT || b.length > CODE_LIMIT) return null;
13
+ const table = Array.from({ length: a.length + 1 }, () => new Uint16Array(b.length + 1));
14
+ for (let i = a.length - 1; i >= 0; i -= 1) {
15
+ for (let j = b.length - 1; j >= 0; j -= 1) table[i][j] = a[i] === b[j] ? table[i + 1][j + 1] + 1 : Math.max(table[i + 1][j], table[i][j + 1]);
16
+ }
17
+ const ops = [];
18
+ let i = 0;
19
+ let j = 0;
20
+ while (i < a.length || j < b.length) {
21
+ if (i < a.length && j < b.length && a[i] === b[j]) {
22
+ ops.push({ op: ' ', text: a[i] });
23
+ i += 1;
24
+ j += 1;
25
+ } else if (j < b.length && (i === a.length || table[i][j + 1] >= table[i + 1][j])) {
26
+ ops.push({ op: '+', text: b[j] });
27
+ j += 1;
28
+ } else {
29
+ ops.push({ op: '-', text: a[i] });
30
+ i += 1;
31
+ }
32
+ }
33
+ return ops;
34
+ }
35
+
36
+ function gap(n) {
37
+ return { op: '…', text: `${n} unchanged line${n === 1 ? '' : 's'}` };
38
+ }
39
+
40
+ export function hunks(ops) {
41
+ const near = ops.map((_, k) => ops.slice(Math.max(0, k - CONTEXT), k + CONTEXT + 1).some((o) => o.op !== ' '));
42
+ const out = [];
43
+ let skipped = 0;
44
+ ops.forEach((o, k) => {
45
+ if (near[k]) {
46
+ if (skipped) out.push(gap(skipped));
47
+ skipped = 0;
48
+ out.push(o);
49
+ } else skipped += 1;
50
+ });
51
+ if (skipped) out.push(gap(skipped));
52
+ return out;
53
+ }
54
+
55
+ function own(manifest, node) {
56
+ return DIFFABLE.has(node.type) && (!node.package || !manifest.project || node.package === manifest.project);
57
+ }
58
+
59
+ function descendants(manifest, id) {
60
+ const seen = new Set();
61
+ const queue = [...manifest.nodes.get(id).children];
62
+ while (queue.length) {
63
+ const next = queue.shift();
64
+ if (seen.has(next)) continue;
65
+ seen.add(next);
66
+ queue.push(...(manifest.nodes.get(next)?.children ?? []));
67
+ }
68
+ return [...seen].filter((d) => manifest.nodes.has(d) && !TEST_TYPES.has(manifest.nodes.get(d).type));
69
+ }
70
+
71
+ function columnChanges(before, after) {
72
+ const added = Object.keys(after).filter((c) => !(c in before));
73
+ const removed = Object.keys(before).filter((c) => !(c in after));
74
+ const typed = Object.keys(after).filter((c) => c in before && before[c].type !== after[c].type).map((c) => ({ name: c, from: before[c].type, to: after[c].type }));
75
+ return { added, removed, typed };
76
+ }
77
+
78
+ function compare(base, current, b, c) {
79
+ const reasons = [];
80
+ if ((b.checksum ?? b.code) !== (c.checksum ?? c.code)) reasons.push('code');
81
+ const config = Object.keys(CONFIG_LABEL).filter((k) => b.config[k] !== c.config[k]).map((k) => ({ field: CONFIG_LABEL[k], from: b.config[k], to: c.config[k] }));
82
+ if (config.length) reasons.push('config');
83
+ const columns = columnChanges(b.columns, c.columns);
84
+ if (columns.added.length || columns.removed.length || columns.typed.length) reasons.push('columns');
85
+ const deps = { added: c.parents.filter((p) => !b.parents.includes(p)), removed: b.parents.filter((p) => !c.parents.includes(p)) };
86
+ if (deps.added.length || deps.removed.length) reasons.push('deps');
87
+ if (b.description !== c.description) reasons.push('description');
88
+ const tests = { from: base.testsOf.get(b.id) ?? 0, to: current.testsOf.get(c.id) ?? 0 };
89
+ if (tests.from !== tests.to) reasons.push('tests');
90
+ return { reasons, config, columns, deps, tests };
91
+ }
92
+
93
+ function breakingOf(change, base, current, b, c) {
94
+ const notes = [];
95
+ if (change.change === 'removed') {
96
+ const users = descendants(base, b.id).length;
97
+ if (users) notes.push(`removed while ${users} node${users === 1 ? '' : 's'} in the base depended on it`);
98
+ return notes;
99
+ }
100
+ if (change.change !== 'modified') return notes;
101
+ const contract = b.config.contract || c.config.contract;
102
+ const { removed, typed } = change.columns;
103
+ if (contract && (removed.length || typed.length)) notes.push(`enforced contract changes: ${[...removed.map((r) => `-${r}`), ...typed.map((t) => `${t.name} ${t.from ?? '?'} → ${t.to ?? '?'}`)].join(', ')}`);
104
+ else if (removed.length && change.downstream) notes.push(`drops ${removed.join(', ')}; ${change.downstream} downstream node${change.downstream === 1 ? '' : 's'} may still select ${removed.length === 1 ? 'it' : 'them'}`);
105
+ const incremental = [b.config.materialized, c.config.materialized].includes('incremental');
106
+ if (incremental && change.config.some((c) => ['materialized', 'unique_key', 'incremental_strategy', 'on_schema_change'].includes(c.field))) notes.push('incremental config changed: plan a --full-refresh');
107
+ if (!b.config.contract && c.config.contract) notes.push('contract now enforced: the build fails if column types drift');
108
+ return notes;
109
+ }
110
+
111
+ function byReview(a, b) {
112
+ return Math.sign(b.breaking.length) - Math.sign(a.breaking.length) || RANK[a.type] - RANK[b.type] || a.name.localeCompare(b.name);
113
+ }
114
+
115
+ export function diffManifests(base, current) {
116
+ const ids = new Set([...base.nodes.values()].filter((n) => own(base, n)).map((n) => n.id));
117
+ for (const n of current.nodes.values()) if (own(current, n)) ids.add(n.id);
118
+ const changes = [];
119
+ for (const id of ids) {
120
+ const b = base.nodes.get(id);
121
+ const c = current.nodes.get(id);
122
+ const node = c ?? b;
123
+ const kind = !b ? 'added' : !c ? 'removed' : 'modified';
124
+ const detail = kind === 'modified' ? compare(base, current, b, c) : { reasons: [], config: [], columns: { added: [], removed: [], typed: [] }, deps: { added: [], removed: [] }, tests: null };
125
+ if (kind === 'modified' && !detail.reasons.length) continue;
126
+ const code = kind === 'removed' ? null : lineDiff(kind === 'added' ? '' : b.code, c.code);
127
+ const change = {
128
+ id,
129
+ name: node.name,
130
+ type: node.type,
131
+ file: node.file,
132
+ change: kind,
133
+ ...detail,
134
+ code: code && (kind === 'added' || detail.reasons.includes('code')) && (b?.code || c.code) ? hunks(code) : null,
135
+ downstream: descendants(kind === 'removed' ? base : current, id).length
136
+ };
137
+ change.breaking = breakingOf(change, base, current, b, c);
138
+ changes.push(change);
139
+ }
140
+ changes.sort(byReview);
141
+ const changed = new Set(changes.map((c) => c.id));
142
+ const impacted = new Map();
143
+ for (const change of changes.filter((c) => c.change !== 'removed' && !TEST_TYPES.has(c.type))) {
144
+ for (const id of descendants(current, change.id)) {
145
+ if (!changed.has(id) && !impacted.has(id)) impacted.set(id, { id, name: current.nodes.get(id).name, type: current.nodes.get(id).type, via: change.name });
146
+ }
147
+ }
148
+ const models = changes.filter((c) => c.type === 'model' && c.change !== 'removed');
149
+ const checklist = [
150
+ ...models.filter((c) => !current.testsOf.get(c.id)).map((c) => `${c.name} has no tests`),
151
+ ...models.filter((c) => c.change === 'added' && !current.nodes.get(c.id).description.trim()).map((c) => `${c.name} is new and has no description`),
152
+ ...changes.filter((c) => c.breaking.some((n) => n.includes('full-refresh'))).map((c) => `${c.name}: run with --full-refresh after merge`)
153
+ ];
154
+ const build = changes.filter((c) => c.change !== 'removed' && BUILDABLE.has(c.type)).map((c) => `${c.name}+`);
155
+ const count = (kind) => changes.filter((c) => c.change === kind).length;
156
+ return {
157
+ summary: { added: count('added'), modified: count('modified'), removed: count('removed'), impacted: impacted.size, breaking: changes.filter((c) => c.breaking.length).length, tests: changes.filter((c) => TEST_TYPES.has(c.type)).length },
158
+ changes,
159
+ impacted: [...impacted.values()].sort((a, b) => (RANK[a.type] ?? 9) - (RANK[b.type] ?? 9) || a.name.localeCompare(b.name)),
160
+ checklist,
161
+ command: build.length ? `dbt build --select ${build.join(' ')}` : null
162
+ };
163
+ }
@@ -0,0 +1,6 @@
1
+ import os from 'node:os';
2
+ import path from 'node:path';
3
+
4
+ export function sugarHome(env = process.env, home = os.homedir()) {
5
+ return env.DBT_SUGAR_HOME || path.join(home, '.dbt-sugar');
6
+ }
@@ -0,0 +1,11 @@
1
+ export const MOODS = ['waiting', 'running', 'worried', 'alarmed', 'success', 'failed'];
2
+
3
+ export function moodOf({ status, counts, warnings }) {
4
+ const started = counts.success + counts.warn + counts.error + counts.skipped + counts.running;
5
+ if (status === 'running') {
6
+ if (counts.error) return 'alarmed';
7
+ if (counts.warn || warnings) return 'worried';
8
+ return started ? 'running' : 'waiting';
9
+ }
10
+ return status === 'error' || counts.error || counts.skipped ? 'failed' : 'success';
11
+ }