ci-sdk 0.0.2 → 0.0.3

package/README.md

@@ -2,31 +2,82 @@
 
 Insanely fast CI. No migration. By [Diffling](https://diffling.com).
 
-## scan
+ci-sdk makes the CI you already have skip work your change can't affect: packages whose inputs didn't change, tests your change can't reach, and jobs that already passed on the same code. No pipeline to rewrite, no config, any runner.
 
-See where your GitHub Actions minutes and dollars go, and how much ci-sdk would save. Run it in your repo:
+On [React Router](https://github.com/remix-run/react-router) (10 packages), whole CI jobs (setup, install, build, typecheck, lint, unit tests) ran **40% faster on pushes and 27% faster on pull requests**, with the same results, replaying real commits on GitHub's standard runners. Gains are largest in monorepos with many packages and commits that touch a few of them; repos whose jobs take only seconds gain little.
+
+Needs Node 22.18 or newer. Works with pnpm and npm workspaces (yarn and bun run through their own `run`).
+
+## run
+
+Run a package script in every workspace package, skipping packages whose inputs haven't changed since they last passed:
 
 ```bash
-npx ci-sdk scan
+npx ci-sdk run build
+npx ci-sdk run typecheck
 ```
 
-- Looks at the last 30 days of workflow runs (`--days`, `--max-runs` to change).
-- Estimates savings from your repo's own history: repeated runs, tests a change could not reach (via an oxc import graph), and slow installs.
-- `--markdown` for a PR or Slack, `--json` for tooling.
-- Needs a GitHub token with **Actions: read**. Uses `GITHUB_TOKEN`, `GH_TOKEN`, or `gh auth token`.
-- Runs locally. Your code and CI data never leave your machine.
+- A package reruns when its own files, a workspace package it depends on, or its dependencies in `pnpm-lock.yaml` change. Docs, changelogs, licenses and CI config don't count.
+- Outputs are whatever the script creates outside git. On a cache hit they are restored and the script's log is replayed. No config.
+- Packages run dependencies first, up to 4 at a time. Failures are never cached, and packages that depend on a failure don't run.
+- The cache lives in `node_modules/.cache/ci-sdk`. In CI, entries the run didn't use are pruned, so a saved cache stays small.
+- `--filter './packages/*'` limits it to some folders. `--no-cache` runs everything and refreshes the cache.
+- Environment variables are not part of a package's inputs. If a script's output depends on one, run it with `--no-cache`.
 
 ## test
 
-Run only the tests your changes can reach, using each package's own `test` script:
+Run only the tests your changes can reach:
 
 ```bash
-npx ci-sdk test
+npx ci-sdk test                      # uncommitted and untracked changes
+npx ci-sdk test --since origin/main  # plus commits on your branch
+npx ci-sdk test --since HEAD~1       # in CI: the commit being tested
 ```
 
-- Looks at uncommitted and untracked changes. Add `--since origin/main` to include commits on your branch.
-- `--list` prints the affected test files instead of running them.
 - Follows imports through barrel files, so a change in a shared package only runs the tests that use it.
+- Runs each package's own test script with the affected files. Packages without one run through the root's script, such as `jest` or `vitest run`. `--script test-unit` picks a different script.
 - If a change can affect everything, such as a root config, it runs the full suite.
+- It never reports success for tests it couldn't run: if no script can take the affected files, it fails and names them.
+- `--list` prints the affected test files instead of running them.
+
+## GitHub Action
 
-Needs Node 22.18 or newer.
+Restores tool caches (Turborepo, Next.js, ESLint, `node_modules/.cache`, which includes ci-sdk's own) and tells you when this exact code already passed this job:
+
+```yaml
+steps:
+  - uses: actions/checkout@v6
+    with:
+      fetch-depth: 2 # lets `ci-sdk test --since HEAD~1` see the commit's changes
+  - uses: pnpm/action-setup@v6
+  - uses: actions/setup-node@v7
+    with:
+      node-version: 24
+  - id: ci-sdk
+    uses: jaymalve/ci-sdk@v0.0.3
+  - if: steps.ci-sdk.outputs.already-passed != 'true'
+    run: |
+      pnpm install --frozen-lockfile
+      npx ci-sdk run build
+      npx ci-sdk run typecheck
+      npx ci-sdk test --since HEAD~1
+```
+
+- Add `ci-sdk` to your devDependencies to pin its version; `npx` then uses the installed copy instead of downloading it.
+- `already-passed` is `true` when this job already passed on the same code (same git tree, job and matrix). Skip the rest of the job when it is.
+- Each run saves a fresh cache entry and restores the latest one. On pull requests, that includes the base branch's.
+- On runners that serve the GitHub Actions cache API from their own storage, such as Blacksmith and Depot, the Action uses that automatically.
+
+## scan
+
+See where your GitHub Actions minutes and dollars go, and how much ci-sdk would save. Run it in your repo:
+
+```bash
+npx ci-sdk scan
+```
+
+- Looks at the last 30 days of workflow runs (`--days`, `--max-runs` to change).
+- Estimates savings from your repo's own history: repeated runs, tests a change could not reach (via an oxc import graph), and slow installs.
+- `--markdown` for a PR or Slack, `--json` for tooling.
+- Needs a GitHub token with **Actions: read**. Uses `GITHUB_TOKEN`, `GH_TOKEN`, or `gh auth token`.
+- Runs locally. Your code and CI data never leave your machine.

package/dist/cli.js

@@ -2,17 +2,25 @@
 import { parseArgs } from 'node:util';
 import { getToken, repoFromGit } from './github.js';
 import { json, markdown, text } from './report.js';
+import { runCached } from './run.js';
 import { scan } from './scan.js';
 import { testAffected } from './test.js';
 const HELP = `Usage: ci-sdk <command> [options]
 
 Commands:
+  run      Run a package script in every workspace package, skipping packages whose inputs already passed
   test     Run only the tests your changes can reach
   scan     Show where your GitHub Actions minutes and dollars go, and what ci-sdk would save
 
+run options:  ci-sdk run <script>
+  --filter <glob>      Only packages in matching folders, e.g. './packages/*' (repeatable)
+  --concurrency <n>    Packages at a time (default: up to 4)
+  --no-cache           Run everything and refresh the cache
+
 test options:
   --since <ref>        Also include changes committed since <ref>, e.g. origin/main (default: uncommitted only)
   --list               Print the affected test files instead of running them
+  --script <name>      Script that runs tests and takes file paths (default: test), e.g. test-unit
 
 scan options:
   --repo <owner/name>  Repo to scan (default: this checkout's origin)
@@ -35,17 +43,23 @@ async function main() {
             markdown: { type: 'boolean' },
             since: { type: 'string' },
             list: { type: 'boolean' },
+            script: { type: 'string' },
+            filter: { type: 'string', multiple: true },
+            concurrency: { type: 'string' },
+            'no-cache': { type: 'boolean' },
             help: { type: 'boolean', short: 'h' },
         },
     });
     const command = positionals[0];
-    if (values.help || (command !== 'scan' && command !== 'test')) {
+    if (values.help || !['scan', 'test', 'run'].includes(command) || (command === 'run' && !positionals[1])) {
         console.log(HELP);
         process.exit(values.help ? 0 : 1);
     }
     const log = (m) => process.stderr.write(`ci-sdk: ${m}\n`);
+    if (command === 'run')
+        process.exit(await runCached({ cwd: process.cwd(), script: positionals[1], filter: values.filter, concurrency: values.concurrency ? Number(values.concurrency) : undefined, noCache: values['no-cache'], log }));
     if (command === 'test')
-        process.exit(await testAffected({ cwd: process.cwd(), since: values.since, list: values.list, log }));
+        process.exit(await testAffected({ cwd: process.cwd(), since: values.since, list: values.list, script: values.script, log }));
     const root = process.cwd();
     const repo = values.repo ?? repoFromGit(root);
     if (!repo)

package/dist/lockfile.js

@@ -0,0 +1,83 @@
+import { createHash } from 'node:crypto';
+const sha = (parts) => {
+    const h = createHash('sha256');
+    for (const p of parts)
+        h.update(p).update('\0');
+    return h.digest('hex');
+};
+const ENTRY = /^(\s*)('[^']*'|[^\s:'][^:]*):(?:\s+(.*))?$/;
+const DEP_GROUPS = new Set(['dependencies', 'devDependencies', 'optionalDependencies']);
+/**
+ * Per-package fingerprints from a pnpm v9 lockfile, so a lockfile change reruns only packages whose dependency tree changed.
+ * Reads just the block layout pnpm writes; returns undefined for anything else, and callers then hash the whole file.
+ */
+export function pnpmLockKeys(text) {
+    const docs = text.split(/^---$/m);
+    const mainAt = docs.findLastIndex((d) => /^importers:/m.test(d));
+    if (mainAt < 0 || !/^lockfileVersion: '9\.\d+'/m.test(docs[mainAt]))
+        return undefined;
+    const other = docs.filter((_, i) => i !== mainAt);
+    // Top-level sections of the main document.
+    const sections = new Map();
+    let cur = '';
+    for (const line of docs[mainAt].split('\n')) {
+        const m = /^([^\s#][^:]*):/.exec(line);
+        if (m)
+            sections.set((cur = m[1]), []);
+        else
+            sections.get(cur)?.push(line);
+    }
+    for (const [name, lines] of sections)
+        if (!['importers', 'packages', 'snapshots'].includes(name))
+            other.push(`${name}\n${lines.join('\n')}`);
+    // importers: 2 spaces importer, 4 group, 6 dependency name, 8 specifier/version.
+    const direct = new Map();
+    let imp = '';
+    let group = '';
+    let dep = '';
+    for (const line of sections.get('importers') ?? []) {
+        const m = ENTRY.exec(line);
+        if (!m)
+            continue;
+        const [indent, key, value] = [m[1].length, m[2].replace(/^'(.*)'$/, '$1'), m[3]?.trim()];
+        if (indent === 2)
+            direct.set((imp = key), []);
+        else if (indent === 4)
+            group = key;
+        else if (indent === 6)
+            dep = key;
+        else if (indent === 8 && key === 'version' && DEP_GROUPS.has(group) && value)
+            direct.get(imp).push(node(dep, value));
+    }
+    // snapshots: 2 spaces id, 4 group, 6 `name: version`.
+    const children = new Map();
+    let id = '';
+    for (const line of sections.get('snapshots') ?? []) {
+        const m = ENTRY.exec(line);
+        if (!m)
+            continue;
+        const [indent, key, value] = [m[1].length, m[2].replace(/^'(.*)'$/, '$1'), m[3]?.trim()];
+        if (indent === 2)
+            children.set((id = key), []);
+        else if (indent === 4)
+            group = key;
+        else if (indent === 6 && DEP_GROUPS.has(group) && value)
+            children.get(id).push(node(key, value));
+    }
+    const byImporter = new Map();
+    for (const [importer, roots] of direct) {
+        const seen = new Set();
+        const stack = roots.filter((r) => !r.includes('@link:'));
+        while (stack.length) {
+            const n = stack.pop();
+            if (seen.has(n))
+                continue;
+            seen.add(n);
+            stack.push(...(children.get(n) ?? []));
+        }
+        byImporter.set(importer, sha([...seen].sort()));
+    }
+    return { shared: sha(other), byImporter };
+}
+// A dependency's snapshot id: `name@version`, or the version itself for an alias (`string-width: string-width@4.2.3`).
+const node = (name, version) => (/^(@[^/]+\/)?[^@/(]+@/.test(version) ? version : `${name}@${version}`);

package/dist/run.js

@@ -0,0 +1,257 @@
+import { execFileSync, spawn } from 'node:child_process';
+import { createHash } from 'node:crypto';
+import { cpSync, existsSync, lstatSync, mkdirSync, readdirSync, readFileSync, readlinkSync, renameSync, rmSync, statSync, writeFileSync } from 'node:fs';
+import { availableParallelism } from 'node:os';
+import { delimiter, dirname, join } from 'node:path';
+import { glob } from 'tinyglobby';
+import { pnpmLockKeys } from './lockfile.js';
+import { packageManager, readJson, repoRoot, runScript } from './test.js';
+// Bump when the key recipe changes, so old entries can never hit.
+const VERSION = '2';
+const gitZ = (root, args) => execFileSync('git', args, { cwd: root, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'], maxBuffer: 1 << 28 }).split('\0').filter(Boolean);
+/** Workspace packages from pnpm-workspace.yaml or package.json `workspaces`; a repo without either is one package. */
+export async function workspace(root, scriptName) {
+    const rootPkg = readJson(join(root, 'package.json')) ?? {};
+    let patterns = Array.isArray(rootPkg.workspaces) ? rootPkg.workspaces : (rootPkg.workspaces?.packages ?? []);
+    const yaml = existsSync(join(root, 'pnpm-workspace.yaml')) ? readFileSync(join(root, 'pnpm-workspace.yaml'), 'utf8') : '';
+    // shortcut: reads only the `packages:` list of pnpm-workspace.yaml; use a YAML parser if repos put it in flow style.
+    const block = yaml.match(/^packages:\s*\n((?:\s+-.*\n?|\s*#.*\n?|\s*\n)*)/m)?.[1] ?? '';
+    patterns = [...patterns, ...[...block.matchAll(/^\s+-\s*['"]?([^'"#\n]+?)['"]?\s*$/gm)].map((m) => m[1])];
+    const manifests = patterns.length
+        ? await glob(patterns.filter((p) => !p.startsWith('!')).map((p) => `${p.replace(/\/$/, '')}/package.json`), { cwd: root, ignore: ['**/node_modules/**', ...patterns.filter((p) => p.startsWith('!')).map((p) => p.slice(1))] })
+        : [];
+    const dirs = manifests.length ? manifests.map((m) => m.slice(0, -'package.json'.length)) : [''];
+    const pkgs = dirs.map((dir) => ({ dir, json: readJson(join(root, dir, 'package.json')) ?? {} }));
+    const names = new Set(pkgs.map((p) => p.json.name).filter(Boolean));
+    return pkgs.map(({ dir, json }) => ({
+        dir,
+        name: json.name ?? (dir || '(root)'),
+        script: json.scripts?.[scriptName],
+        deps: Object.keys({ ...json.dependencies, ...json.devDependencies, ...json.peerDependencies, ...json.optionalDependencies }).filter((d) => names.has(d)),
+    }));
+}
+const sha = (...parts) => {
+    const h = createHash('sha256');
+    for (const p of parts)
+        h.update(p).update('\0');
+    return h.digest('hex');
+};
+// Files outside every package that no package script reads: docs, changelogs, licenses, CI and editor config.
+// Without this, a CHANGELOG or docs edit reruns everything. A package's own README or LICENSE still counts, since it is
+// inside the package. shortcut: a build that copies a root README/LICENSE into its output will not rerun when that file changes.
+const READ_BY_NO_SCRIPT = /(^|\/)(LICEN[CS]E[^/]*|[^/]+\.mdx?)$|^\.(github|changeset|vscode|idea)\//i;
+// Release notes no build reads, even inside a package (other markdown there counts: docs packages build from it).
+const RELEASE_NOTES = /(^|\/)(CHANGELOG[^/]*\.mdx?|\.changeset\/.+|\.changes\/.+)$/i;
+/**
+ * One key per package: its own files, every file outside all packages (root configs, scripts), its dependency tree from
+ * the lockfile, and the keys of the workspace packages it depends on. Conservative on purpose: a root config change reruns everything.
+ */
+export function inputKeys(root, pkgs, salt) {
+    const files = gitZ(root, ['ls-files', '-z', '--cached', '--others', '--exclude-standard']);
+    const deleted = new Set(gitZ(root, ['ls-files', '-z', '--deleted']));
+    const present = files.filter((f) => !deleted.has(f));
+    // git hash-object fails on symlinks to folders and on submodule folders, so hash those by their link target or kind.
+    const hashes = new Map();
+    const regular = [];
+    for (const f of present) {
+        const s = lstatSync(join(root, f), { throwIfNoEntry: false });
+        if (s?.isFile())
+            regular.push(f);
+        else
+            hashes.set(f, s?.isSymbolicLink() ? `link:${readlinkSync(join(root, f))}` : s ? 'dir' : 'missing');
+    }
+    if (regular.length) {
+        const out = execFileSync('git', ['hash-object', '--stdin-paths'], { cwd: root, input: regular.join('\n'), encoding: 'utf8', maxBuffer: 1 << 28 }).trim().split('\n');
+        regular.forEach((f, i) => hashes.set(f, out[i]));
+    }
+    // Deepest folder first, so a nested package owns its files.
+    const owners = pkgs.filter((p) => p.dir).sort((a, b) => b.dir.length - a.dir.length);
+    const own = new Map(pkgs.map((p) => [p.dir, []]));
+    const outside = [];
+    // pnpm: key each package on its own dependency tree instead of the whole lockfile, so a lockfile change reruns only
+    // packages whose dependencies changed. Root dependencies count for every package, since package scripts use them.
+    const lock = present.includes('pnpm-lock.yaml') ? pnpmLockKeys(readFileSync(join(root, 'pnpm-lock.yaml'), 'utf8')) : undefined;
+    const treeOf = (dir) => [lock?.byImporter.get('.') ?? '', lock?.byImporter.get(dir.replace(/\/$/, '') || '.') ?? ''];
+    present.forEach((f) => {
+        if (RELEASE_NOTES.test(f) || (lock && f === 'pnpm-lock.yaml'))
+            return;
+        const entry = `${f}:${hashes.get(f)}`;
+        const owner = owners.find((p) => f.startsWith(p.dir));
+        if (owner)
+            own.get(owner.dir).push(entry);
+        else if (!READ_BY_NO_SCRIPT.test(f))
+            outside.push(entry);
+    });
+    if (own.has(''))
+        own.get('').push(...outside);
+    const shared = sha(...outside, lock?.shared ?? '');
+    const byName = new Map(pkgs.map((p) => [p.name, p]));
+    const keys = new Map();
+    const keyOf = (p, seen) => {
+        if (keys.has(p.dir))
+            return keys.get(p.dir);
+        if (seen.has(p.dir))
+            return 'cycle'; // dependency cycles: each member still hashes its own files
+        seen.add(p.dir);
+        const key = sha(VERSION, salt, shared, ...treeOf(p.dir), ...own.get(p.dir), ...p.deps.sort().map((d) => keyOf(byName.get(d), seen)));
+        keys.set(p.dir, key);
+        return key;
+    };
+    for (const p of pkgs)
+        keyOf(p, new Set());
+    return keys;
+}
+/** Files under `dir` that git does not track (outputs live there), with size and mtime; node_modules excluded. */
+function untracked(root, dir) {
+    const files = gitZ(root, ['ls-files', '-z', '--others', '--', dir || '.', ':(exclude,glob)**/node_modules/**']);
+    const stamp = new Map();
+    for (const f of files) {
+        try {
+            const s = statSync(join(root, f));
+            stamp.set(f, `${s.size}:${s.mtimeMs}`);
+        }
+        catch {
+            // vanished while listing
+        }
+    }
+    return stamp;
+}
+/**
+ * Runs the script with sh the way npm and pnpm do, skipping a package-manager launch (~0.35s per package, which adds up
+ * along a dependency chain). Falls back to `<pm> run` where that isn't equivalent: yarn (PnP), bun, Windows, pre/post scripts.
+ */
+export function scriptCommand(pm, root, dir, script) {
+    const json = readJson(join(root, dir, 'package.json')) ?? {};
+    const body = json.scripts?.[script];
+    if (!body || (pm !== 'npm' && pm !== 'pnpm') || process.platform === 'win32' || json.scripts?.[`pre${script}`] || json.scripts?.[`post${script}`])
+        return [...runScript(pm, script, []), process.env];
+    const bins = [];
+    for (let d = join(root, dir);; d = dirname(d)) {
+        bins.push(join(d, 'node_modules/.bin'));
+        if (d === root || d === dirname(d))
+            break;
+    }
+    const env = {
+        ...process.env,
+        PATH: [...bins, process.env.PATH].join(delimiter),
+        npm_lifecycle_event: script,
+        npm_lifecycle_script: body,
+        npm_package_name: json.name,
+        npm_package_version: json.version,
+        npm_package_json: join(root, dir, 'package.json'),
+        // Tools such as wireit pick the package manager to spawn from this prefix.
+        npm_config_user_agent: process.env.npm_config_user_agent?.startsWith(`${pm}/`) ? process.env.npm_config_user_agent : `${pm}/unknown node/${process.version} ${process.platform} ${process.arch}`,
+        INIT_CWD: process.env.INIT_CWD ?? process.cwd(),
+    };
+    return ['sh', ['-c', body], env];
+}
+/** Runs a package script in every workspace package (dependencies first), skipping packages whose inputs already passed. */
+export async function runCached(o) {
+    const root = repoRoot(o.cwd);
+    const pm = packageManager(root);
+    const all = await workspace(root, o.script);
+    const allowed = o.filter?.length ? new Set((await glob(o.filter.map((f) => `${f.replace(/\/$/, '')}/package.json`), { cwd: root, ignore: ['**/node_modules/**'] })).map((m) => m.slice(0, -'package.json'.length))) : undefined;
+    const todo = all.filter((p) => p.script && (!allowed || allowed.has(p.dir)));
+    if (!todo.length) {
+        o.log(`No package has a "${o.script}" script.`);
+        return 0;
+    }
+    const node = process.version;
+    const keys = inputKeys(root, all, [o.script, node, process.platform, process.arch].join('|'));
+    // One folder per script, so pruning after `run build` never touches the entries of `run test`.
+    const cache = join(o.cacheDir ?? join(root, 'node_modules/.cache/ci-sdk'), encodeURIComponent(o.script));
+    const used = new Set();
+    const byName = new Map(all.map((p) => [p.name, p]));
+    const waitingOn = new Map(todo.map((p) => [p.dir, new Set(p.deps.map((d) => byName.get(d).dir).filter((d) => todo.some((t) => t.dir === d && d !== p.dir)))]));
+    const done = new Set();
+    const failed = [];
+    const failedDirs = new Set();
+    let hits = 0;
+    const started = Date.now();
+    const runOne = async (p) => {
+        const key = sha(keys.get(p.dir), p.script);
+        const dir = join(cache, key.slice(0, 2), key);
+        const meta = join(dir, 'meta.json');
+        used.add(join(key.slice(0, 2), key));
+        if (!o.noCache && existsSync(meta)) {
+            const entry = JSON.parse(readFileSync(meta, 'utf8'));
+            for (const f of entry.outputs) {
+                mkdirSync(dirname(join(root, f)), { recursive: true });
+                cpSync(join(dir, 'out', f), join(root, f));
+            }
+            hits++;
+            o.log(`${p.name}: cache hit, ${entry.outputs.length} outputs restored (saved ${(entry.ms / 1000).toFixed(1)}s)`);
+            process.stdout.write(entry.log);
+            return;
+        }
+        const before = untracked(root, p.dir);
+        const t = Date.now();
+        const [cmd, args, env] = scriptCommand(pm, root, p.dir, o.script);
+        let log = '';
+        const code = await new Promise((resolve) => {
+            const child = spawn(cmd, args, { cwd: join(root, p.dir), env, stdio: ['ignore', 'pipe', 'pipe'] });
+            child.stdout.on('data', (d) => (log += d));
+            child.stderr.on('data', (d) => (log += d));
+            child.on('close', (c) => resolve(c ?? 1));
+        });
+        const ms = Date.now() - t;
+        o.log(`${p.name}: ran in ${(ms / 1000).toFixed(1)}s${code ? `, failed with exit code ${code}` : ''}`);
+        process.stdout.write(log);
+        if (code) {
+            failed.push(p.name);
+            failedDirs.add(p.dir);
+            return;
+        }
+        // Outputs are whatever the script created or changed outside git, inferred with no config.
+        const outputs = [...untracked(root, p.dir)].filter(([f, s]) => before.get(f) !== s).map(([f]) => f);
+        const tmp = `${dir}.tmp-${process.pid}`;
+        rmSync(tmp, { recursive: true, force: true });
+        for (const f of outputs) {
+            mkdirSync(dirname(join(tmp, 'out', f)), { recursive: true });
+            cpSync(join(root, f), join(tmp, 'out', f));
+        }
+        mkdirSync(tmp, { recursive: true });
+        writeFileSync(join(tmp, 'meta.json'), JSON.stringify({ log, outputs, ms }));
+        rmSync(dir, { recursive: true, force: true });
+        mkdirSync(dirname(dir), { recursive: true });
+        renameSync(tmp, dir); // atomic: a half-written entry is never read
+    };
+    // Dependencies first, up to `concurrency` packages at a time, like `pnpm -r`.
+    const limit = o.concurrency ?? Math.max(1, Math.min(4, availableParallelism()));
+    const running = new Map();
+    while (done.size < todo.length) {
+        const blocked = (p) => [...waitingOn.get(p.dir)].some((d) => !done.has(d));
+        for (const p of todo) {
+            if (running.size >= limit)
+                break;
+            if (done.has(p.dir) || running.has(p.dir) || blocked(p))
+                continue;
+            if ([...waitingOn.get(p.dir)].some((d) => failedDirs.has(d))) {
+                o.log(`${p.name}: skipped, a dependency failed`);
+                failed.push(p.name);
+                failedDirs.add(p.dir);
+                done.add(p.dir);
+                continue;
+            }
+            running.set(p.dir, runOne(p).then(() => void done.add(p.dir), () => void (failed.push(p.name), failedDirs.add(p.dir), done.add(p.dir))));
+        }
+        if (!running.size) {
+            // A dependency cycle: run the rest in order rather than hang.
+            for (const p of todo.filter((p) => !done.has(p.dir)))
+                waitingOn.set(p.dir, new Set());
+            continue;
+        }
+        await Promise.race(running.values());
+        for (const d of [...running.keys()])
+            if (done.has(d))
+                running.delete(d);
+    }
+    if (o.prune ?? Boolean(process.env.CI)) {
+        for (const shard of existsSync(cache) ? readdirSync(cache) : [])
+            for (const entry of readdirSync(join(cache, shard)))
+                if (!used.has(join(shard, entry)))
+                    rmSync(join(cache, shard, entry), { recursive: true, force: true });
+    }
+    o.log(`${o.script}: ${todo.length} packages, ${hits} from cache, ${failed.length} failed, ${((Date.now() - started) / 1000).toFixed(1)}s`);
+    return failed.length ? 1 : 0;
+}

package/dist/test.js

@@ -10,10 +10,10 @@ export function changedFiles(root, since = 'HEAD') {
     const committed = since === 'HEAD' ? [] : lines(git(root, ['diff', '--name-only', `${since}...HEAD`]));
     return [...new Set([...committed, ...lines(git(root, ['diff', '--name-only', 'HEAD'])), ...lines(git(root, ['ls-files', '--others', '--exclude-standard']))])];
 }
-const packageManager = (root) => existsSync(join(root, 'pnpm-lock.yaml')) ? 'pnpm' : existsSync(join(root, 'yarn.lock')) ? 'yarn' : existsSync(join(root, 'bun.lock')) || existsSync(join(root, 'bun.lockb')) ? 'bun' : 'npm';
+export const packageManager = (root) => existsSync(join(root, 'pnpm-lock.yaml')) ? 'pnpm' : existsSync(join(root, 'yarn.lock')) ? 'yarn' : existsSync(join(root, 'bun.lock')) || existsSync(join(root, 'bun.lockb')) ? 'bun' : 'npm';
 // How each package manager appends extra arguments to a package script.
-const runScript = (pm, script, args) => (pm === 'npm' ? ['npm', ['run', script, '--', ...args]] : [pm, ['run', script, ...args]]);
-const readJson = (file) => {
+export const runScript = (pm, script, args) => (pm === 'npm' ? ['npm', ['run', script, '--', ...args]] : [pm, ['run', script, ...args]]);
+export const readJson = (file) => {
     try {
         return JSON.parse(readFileSync(file, 'utf8'));
     }
@@ -21,20 +21,24 @@ const readJson = (file) => {
         return undefined;
     }
 };
-/** Affected test files grouped by the workspace package that owns them, each run through that package's own test script. */
-export function plan(root, graph, tests) {
+// A script that fans out to every package (turbo, nx, lerna, pnpm -r) can't be narrowed to files.
+const FANS_OUT = /\b(turbo|nx|lerna)\b|\s-r\b|--recursive|--workspaces?\b/;
+// File paths appended to a chained script reach only its last command.
+const CHAINED = /&&|\|\||;/;
+const takesFiles = (script) => Boolean(script) && !FANS_OUT.test(script) && !CHAINED.test(script);
+/** Affected test files grouped by the workspace package that owns them; each group runs through its own script, else the root's. */
+export function plan(root, graph, tests, script = 'test') {
     const byDir = new Map();
     for (const t of tests) {
         const dir = graph.packages.find((p) => t.startsWith(p)) ?? '';
         (byDir.get(dir) ?? byDir.set(dir, []).get(dir)).push(t.slice(dir.length));
     }
+    const rootTakesFiles = takesFiles(readJson(join(root, 'package.json'))?.scripts?.[script]);
     return [...byDir]
         .map(([dir, files]) => {
         const pkg = readJson(join(root, dir, 'package.json'));
-        // A root script that fans out to every package (turbo, nx, lerna, pnpm -r) can't be narrowed to files.
-        const script = pkg?.scripts?.test ?? '';
-        const fansOut = !dir && /\b(turbo|nx|lerna)\b|\s-r\b|--recursive|--workspaces?\b/.test(script);
-        return { dir, name: pkg?.name ?? (dir || '(root)'), files: files.sort(), hasTestScript: Boolean(script) && !fansOut };
+        const via = dir && takesFiles(pkg?.scripts?.[script]) ? 'package' : rootTakesFiles ? 'root' : undefined;
+        return { dir, name: pkg?.name ?? (dir || '(root)'), files: files.sort(), via };
     })
         .sort((a, b) => a.name.localeCompare(b.name));
 }
@@ -71,9 +75,10 @@ export async function testAffected(o) {
                 console.log(t);
             return 0;
         }
-        return exec(runScript(pm, 'test', []), root);
+        return exec(runScript(pm, o.script ?? 'test', []), root);
     }
-    const runs = plan(root, graph, affected);
+    const script = o.script ?? 'test';
+    const runs = plan(root, graph, affected, script);
     o.log(`${changed.length} changed files → ${affected.size} of ${graph.tests.size} test files in ${runs.length} packages`);
     if (o.list) {
         for (const t of [...affected].sort())
@@ -87,15 +92,21 @@ export async function testAffected(o) {
         o.log('Building dependencies failed; not running tests.');
         return 1;
     }
+    // Never report success for tests that did not run.
+    const stranded = runs.filter((r) => !r.via);
+    if (stranded.length) {
+        for (const r of stranded)
+            o.log(`${r.name}: no "${script}" script here or at the root that accepts file paths, so ${r.files.length} affected test files can't run: ${r.files.slice(0, 3).join(', ')}`);
+        o.log(`Not running a partial suite. Pass --script <name> for a script that takes test files, or run the full suite.`);
+        return 1;
+    }
     let failed = 0;
-    for (const r of runs) {
-        if (!r.hasTestScript) {
-            o.log(`${r.name} has no test script that runs single files; skipped ${r.files.length} test files: ${r.files.slice(0, 3).join(', ')}`);
-            continue;
-        }
-        if (exec(runScript(pm, 'test', r.files), join(root, r.dir)) !== 0)
+    for (const r of runs.filter((r) => r.via === 'package'))
+        if (exec(runScript(pm, script, r.files), join(root, r.dir)) !== 0)
             failed++;
-    }
-    o.log(failed ? `${failed} of ${runs.length} packages had failing tests.` : `All affected tests passed (${affected.size} of ${graph.tests.size} test files).`);
+    const viaRoot = runs.filter((r) => r.via === 'root').flatMap((r) => r.files.map((f) => r.dir + f));
+    if (viaRoot.length && exec(runScript(pm, script, viaRoot), root) !== 0)
+        failed++;
+    o.log(failed ? 'Some affected tests failed.' : `All affected tests passed (${affected.size} of ${graph.tests.size} test files).`);
     return failed ? 1 : 0;
 }

package/package.json

@@ -1,10 +1,14 @@
 {
   "name": "ci-sdk",
-  "version": "0.0.2",
+  "version": "0.0.3",
   "description": "Insanely fast CI. No migration. By Diffling.",
   "license": "MIT",
   "type": "module",
   "homepage": "https://ci-sdk.com",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/jaymalve/ci-sdk.git"
+  },
   "bin": {
     "ci-sdk": "dist/cli.js"
   },