@veda-hq/dvara 0.5.0 → 0.5.16

package/README.md

@@ -29,8 +29,21 @@ file in your repository; dvara runs it.
   attaches to each run; it may stop a hung hook and decide the verdict. Without
   one, or when it fails to load, the hook runs exactly as it would anyway.
 
-Every one of these is fail-open: telemetry, quiet output and the watchdog never
-change a hook's verdict.
+- **A stage list, if you keep one.** `stages`, or the JSON file `stageList`
+  names (inside an installed package, say), gives each hook's stage names in
+  order. A passing run that marks a stage its hook's list does not have, or
+  marks one out of the list's order, is refused (exit 1) with the reason. So is
+  a run of a hook in `stageListRequired` (default `pre-push`) that marks a
+  stage and whose list file cannot be read, so a missing package never lets a
+  push that ran its stages through silently; any other hook passes then, with a
+  warning that its stages were not checked. A
+  stage may be marked again where it stands, a name may
+  appear more than once in a list, and dvara's own `startup` and
+  `runtime-selection` need no entry. A run that marks no stage reads no list.
+
+Telemetry, quiet output and a watchdog that fails to load never change a
+hook's verdict. Only a signal, an enforcing watchdog's stop and the stage list
+do.
 
 ## A hook under dvara
 
@@ -73,13 +86,28 @@ rename it in its messages (default "preflight" and "Preflight").
 `.dvara.json` at the repository's root, or the file `DVARA_CONFIG` names. Every
 field is optional; an unknown field is an error.
 
-| Field        | Default                                          | Meaning                                                        |
-| ------------ | ------------------------------------------------ | -------------------------------------------------------------- |
-| `historyDir` | `.dvara`                                         | where the history, the quiet log and scratch files go          |
-| `quietHooks` | `["pre-commit"]`                                 | hooks whose passing output is one line                         |
-| `watchdog`   | `null`                                           | a watchdog module, relative to the root                        |
-| `baseRefs`   | `origin/main`, `origin/develop`, `origin/master` | the refs tried for the base a run is judged against            |
-| `stages`     | `{}`                                             | each hook's stages in order; `dvara stages <hook>` prints them |
+| Field               | Default                                          | Meaning                                                                                           |
+| ------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------- |
+| `historyDir`        | `.dvara`                                         | where the history, the quiet log and scratch files go                                             |
+| `quietHooks`        | `["pre-commit"]`                                 | hooks whose passing output is one line                                                            |
+| `watchdog`          | `null`                                           | a watchdog module, relative to the root                                                           |
+| `baseRefs`          | `origin/main`, `origin/develop`, `origin/master` | the refs tried for the base a run is judged against                                               |
+| `stages`            | `{}`                                             | each hook's stages in order, which a passing run must keep to; `dvara stages <hook>` prints them  |
+| `stageList`         | `null`                                           | a JSON file holding the same map instead, relative to the root and read when a run ends; not both |
+| `stageListRequired` | `["pre-push"]`                                   | hooks a list file that cannot be read refuses; any other hook passes with a warning               |
+
+`records`, an object, holds the settings of `dvara records` and `dvara frozen`
+(below). Each field is optional; an unknown one is an error.
+
+| Field            | Default                        | Meaning                                                                                                                  |
+| ---------------- | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
+| `baseline`       | `.dvara/records-baseline.json` | the per-file baseline, relative to the root; read from git, so it must be tracked (un-ignore it if `.dvara/` is ignored) |
+| `skip`           | `[]`                           | path prefixes no rule reads, such as a checker's own fixtures                                                            |
+| `frozen`         | `{"paths": [], "except": []}`  | `paths`: prefixes `dvara frozen` refuses changing and no rule reads; `except`: exact paths under them that stay editable |
+| `identityPaths`  | `[]`                           | regular expressions for the paths where a commit must be cited by its full id (`EVIDENCE-01`); none by default           |
+| `repository`     | `null`                         | `owner/name` on GitHub: its commit URLs are this repository's, every other repository's are not checked                  |
+| `allowPaths`     | `[]`                           | regular expressions for machine paths that are durable by agreement, matched from where the path starts: anchor with `^` |
+| `packageManager` | `null`                         | `pnpm`, `npm` or `yarn`: the other managers' install commands are refused; `null` checks none                            |
 
 The environment it reads: `DVARA_MODE=preflight` (a preflight's rows get kind
 `preflight`), `DVARA_BACKGROUND=1` (started detached), `DVARA_MEASUREMENT_CONTEXT=manual-benchmark`,
@@ -112,11 +140,78 @@ export function exitDisposition(
 ): { code: number; status: string } | null;
 ```
 
+## The record gate: `dvara records`
+
+Committed text should not cite what stops meaning anything once the session
+that wrote it ends. The gate reads every text file that is not source code, a
+binary, a lockfile (`*.lock`, `pnpm-lock.yaml`, `package-lock.json`), a
+symlink or a submodule, a skipped or frozen path, the baseline or the config,
+and refuses (rule `EVIDENCE-02`, one finding per line):
+
+- a `file://` URL, a `/Users/<name>` or `/home/<name>` home path, a sandbox temp
+  root (`/private/tmp/`, `/private/var/`, `/var/folders/`), a literal `.git/`
+  path, or an agent-process marker ("I am Agent 3");
+- another package manager's install command, when `packageManager` names one.
+  A clause that says not to ("never run `npm install`") is no finding.
+
+Where `identityPaths` match, it also refuses (rule `EVIDENCE-01`) a commit cited
+by less than its full 40-hex id, since git lengthens abbreviations as history
+grows and clones disagree on their length:
+
+- `abbreviated-id`: a 7–39 hex token the line makes a commit (after a commit
+  word, backticked on its own, in an `a..b` range, cut off with an ellipsis, or
+  in a short commit URL of `repository`), unless the line names it another kind
+  of hash (digest, sha256, checksum, …);
+- `abbreviating-command`: a git command that prints abbreviations
+  (`--oneline`, `%h`, `--abbrev-commit`, `rev-parse --short`);
+- `unresolved-id`: a full id the change adds that is not a commit in this
+  repository, compared with the ids the file cited at the merge base. No
+  baseline holds these: they are new by construction.
+
+In Markdown, a fence's captured output is not read for tokens; its command
+lines are (`$ ` lines of a transcript, every line of an `sh` fence). Fences are
+found by CommonMark's rules (mdast-util-from-markdown), so a fence in a quote or
+a list item counts.
+
+Files written before the gate existed are held by the baseline: each file's
+count per rule, which may only fall. A new finding in a legacy file, a fix that
+leaves its allowance in place, an entry for a file that is gone or not read,
+and a baseline edit that raises a count or adds a file are all refused; a
+rename git detects carries its entry forward.
+
+```sh
+dvara records --staged                 # pre-commit: the index, against the first baseRefs that resolves
+dvara records --base origin/main       # CI: the files the branch changed, at HEAD
+dvara records --all [--base REF]       # every tracked file at HEAD
+dvara records --all --write-baseline   # record today's counts
+dvara records --all --survey           # counts per rule and check
+```
+
+Exit 0: clean or within the baseline; 1: refused; 2: could not run. Inside the
+package the gate takes any list of rules, `(path, text, tree) => Finding[]`;
+the command runs the list `rulesFor` in `src/records/rules.ts` returns.
+
+## Frozen paths: `dvara frozen`
+
+A path under one of `records.frozen.paths` is history: no commit adds, changes
+or deletes a file there, except the exact `records.frozen.except` paths.
+
+```sh
+dvara frozen --staged                    # pre-commit, merge commits included
+dvara frozen --base origin/main [--head REF]   # CI: what the branch changes since its merge base
+```
+
+In a merge, a path staged exactly as a merged-in parent has it is that
+parent's change and passes; a hand resolution counts. With no frozen paths
+configured it passes without asking git. Exit 0, 1 or 2 as above.
+
 ## Commands
 
 ```sh
 dvara run <hook-file> [hook args…]   # run a git hook under the supervisor
 dvara stages <hook>                  # the stages .dvara.json lists for it
 dvara shell <stages|keep-going>      # the path of a shell helper
+dvara records <--staged | --all [--base REF] [--write-baseline | --survey] | --base REF>
+dvara frozen <--staged | --base REF [--head REF]>
 dvara --version
 ```

package/dist/cli.js

@@ -3,12 +3,20 @@
 import { readFileSync, realpathSync } from "node:fs";
 import { join, resolve } from "node:path";
 import { fileURLToPath } from "node:url";
-import { loadConfig } from "./config.js";
+import { configName, loadConfig } from "./config.js";
+import { frozen, parseArgs as parseFrozenArgs, USAGE as FROZEN_USAGE, } from "./records/frozen.js";
+import { parseArgs as parseRecordsArgs, records, USAGE as RECORDS_USAGE, } from "./records/gate.js";
+import { runner } from "./records/git.js";
+import { listFor, readStageLists } from "./stages.js";
 import { supervise } from "./supervisor.js";
 export const USAGE = [
     "usage: dvara run <hook-file> [hook args…]   run a git hook under the supervisor",
     "       dvara stages <hook>                  the hook stages the config lists, one per line",
     "       dvara shell <stages|keep-going>      the path of a shell helper to source",
+    "       dvara records <--staged | --all [--base REF] [--write-baseline | --survey] | --base REF>",
+    "                                            the record gate: no new non-durable paths, a baseline that only falls",
+    "       dvara frozen <--staged | --base REF [--head REF]>",
+    "                                            refuse changes to the frozen paths",
     "       dvara --version",
 ].join("\n");
 const SHELL_HELPERS = new Set(["stages", "keep-going"]);
@@ -18,6 +26,38 @@ export const ownVersion = () => {
 };
 /** The path of a shell helper this package ships. */
 export const shellHelper = (name) => resolve(import.meta.dirname, "..", "shell", `${name}.sh`);
+/**
+ * `dvara records` or `dvara frozen` at the root `cwd`: 2 on bad arguments, a
+ * config it cannot read, or git failing.
+ */
+const checkCommand = (command, args, { io, cwd, env }) => {
+    const parsed = command === "records" ? parseRecordsArgs(args) : parseFrozenArgs(args);
+    if ("error" in parsed) {
+        io.err(`dvara ${command}: ${parsed.error}`);
+        io.err(command === "records" ? RECORDS_USAGE : FROZEN_USAGE);
+        return 2;
+    }
+    try {
+        const config = loadConfig(cwd, env);
+        const repo = { cwd, env };
+        if ("mode" in parsed)
+            return records(parsed, repo, io.out, {
+                records: config.records,
+                baseRefs: config.baseRefs,
+                own: [configName(env)],
+            });
+        const result = frozen(parsed, runner(repo), config.records.frozen, env);
+        for (const line of result.stdout)
+            io.out(line);
+        for (const line of result.stderr)
+            io.err(line);
+        return result.code;
+    }
+    catch (error) {
+        io.err(`dvara ${command}: ${error.message}`);
+        return 2;
+    }
+};
 /** Runs one command; resolves with the process's exit code. */
 export const run = async (argv, context) => {
     const { io, cwd, env } = context;
@@ -32,6 +72,8 @@ export const run = async (argv, context) => {
         io.out(shellHelper(rest[0] ?? ""));
         return 0;
     }
+    if (command === "records" || command === "frozen")
+        return checkCommand(command, rest, context);
     if ((command === "run" && rest.length > 0) ||
         (command === "stages" && rest.length === 1)) {
         // Git runs every hook at the top of the working tree.
@@ -44,7 +86,15 @@ export const run = async (argv, context) => {
             return 2;
         }
         if (command === "stages") {
-            for (const name of config.stages[rest[0] ?? ""] ?? [])
+            let lists;
+            try {
+                lists = readStageLists(cwd, config);
+            }
+            catch (error) {
+                io.err(error.message);
+                return 2;
+            }
+            for (const name of lists ? (listFor(lists, rest[0] ?? "") ?? []) : [])
                 io.out(name);
             return 0;
         }

package/dist/config.js

@@ -1,20 +1,120 @@
 // A repository's dvara settings: `.dvara.json` at its root, or the file
 // DVARA_CONFIG names (relative to the root). Every field is optional.
 import { readFileSync } from "node:fs";
-import { resolve } from "node:path";
+import { isAbsolute, posix, resolve } from "node:path";
+import { parseStageLists } from "./stages.js";
+export const PACKAGE_MANAGERS = ["pnpm", "npm", "yarn"];
 export const DEFAULTS = {
     historyDir: ".dvara",
     quietHooks: ["pre-commit"],
     watchdog: null,
     baseRefs: ["origin/main", "origin/develop", "origin/master"],
     stages: {},
+    stageList: null,
+    stageListRequired: ["pre-push"],
+    records: {
+        baseline: ".dvara/records-baseline.json",
+        skip: [],
+        frozen: { paths: [], except: [] },
+        identityPaths: [],
+        repository: null,
+        allowPaths: [],
+        packageManager: null,
+    },
 };
 export const CONFIG_FILE = ".dvara.json";
+/** The config file's name relative to the root, as `loadConfig` reads it. */
+export const configName = (env) => posix.normalize(env.DVARA_CONFIG || CONFIG_FILE);
 const strings = (value, field) => {
     if (!Array.isArray(value) || !value.every((v) => typeof v === "string"))
         throw new Error(`${field} must be a list of strings`);
     return value;
 };
+const isObject = (value) => typeof value === "object" && value !== null && !Array.isArray(value);
+const onlyKnown = (entries, known, field) => {
+    for (const key of Object.keys(entries))
+        if (!known.includes(key))
+            throw new Error(`unknown field ${field}${key}`);
+};
+/** Whether `path`, normalized, leaves the root: `..`, `../x`, `a/../../x`. */
+const escapes = (path) => {
+    const normal = posix.normalize(path);
+    return normal === ".." || normal.startsWith("../");
+};
+/**
+ * Repository-relative path prefixes or paths: none empty, none absolute, none
+ * leaving the root (git prints no path there, so it would match nothing).
+ */
+const repoPaths = (value, field) => {
+    const list = strings(value, field);
+    for (const path of list)
+        if (path === "" || isAbsolute(path) || escapes(path))
+            throw new Error(`${field} must hold repository-relative paths`);
+    return list;
+};
+const patterns = (value, field) => strings(value, field).map((source, i) => {
+    try {
+        return new RegExp(source);
+    }
+    catch (error) {
+        throw new Error(`${field}[${i}] is not a regular expression: ${error.message}`);
+    }
+});
+const parseRecordsConfig = (value) => {
+    if (!isObject(value))
+        throw new Error("records must be an object");
+    onlyKnown(value, Object.keys(DEFAULTS.records), "records.");
+    const records = { ...DEFAULTS.records };
+    if ("baseline" in value) {
+        const path = value.baseline;
+        // Compared with the paths git prints: `./a/../b.json` is `b.json`. A
+        // folder (`.`, `archive/`) names no tracked file, so it is no baseline.
+        const normal = typeof path === "string" ? posix.normalize(path) : "";
+        if (typeof path !== "string" ||
+            path === "" ||
+            isAbsolute(path) ||
+            escapes(path) ||
+            normal === "." ||
+            normal.endsWith("/"))
+            throw new Error("records.baseline must be a file path inside the repository");
+        records.baseline = normal;
+    }
+    if ("skip" in value)
+        records.skip = repoPaths(value.skip, "records.skip");
+    if ("frozen" in value) {
+        const frozen = value.frozen;
+        if (!isObject(frozen))
+            throw new Error("records.frozen must be an object");
+        onlyKnown(frozen, ["paths", "except"], "records.frozen.");
+        records.frozen = {
+            paths: "paths" in frozen
+                ? repoPaths(frozen.paths, "records.frozen.paths")
+                : [],
+            except: "except" in frozen
+                ? repoPaths(frozen.except, "records.frozen.except")
+                : [],
+        };
+    }
+    if ("identityPaths" in value)
+        records.identityPaths = patterns(value.identityPaths, "records.identityPaths");
+    if ("repository" in value) {
+        const repository = value.repository;
+        if (repository !== null &&
+            (typeof repository !== "string" || !/^[\w.-]+\/[\w.-]+$/.test(repository)))
+            throw new Error('records.repository must be "owner/name" or null');
+        records.repository = repository;
+    }
+    if ("allowPaths" in value)
+        records.allowPaths = patterns(value.allowPaths, "records.allowPaths");
+    if ("packageManager" in value) {
+        const manager = value.packageManager;
+        if (manager !== null &&
+            !PACKAGE_MANAGERS.includes(manager))
+            throw new Error(`records.packageManager must be one of ${PACKAGE_MANAGERS.join(", ")}, or null`);
+        records.packageManager = manager;
+    }
+    return records;
+};
 /** The settings in `text` over the defaults; a field of the wrong type is an error, an unknown one too. */
 export const parseConfig = (text) => {
     const raw = JSON.parse(text);
@@ -40,15 +140,27 @@ export const parseConfig = (text) => {
     }
     if ("baseRefs" in entries)
         config.baseRefs = strings(entries.baseRefs, "baseRefs");
-    if ("stages" in entries) {
-        const stages = entries.stages;
-        if (typeof stages !== "object" || stages === null || Array.isArray(stages))
-            throw new Error("stages must map each hook to its stage names");
-        const out = {};
-        for (const [hook, names] of Object.entries(stages))
-            out[hook] = strings(names, `stages.${hook}`);
-        config.stages = out;
+    if ("stages" in entries)
+        config.stages = parseStageLists(entries.stages, "stages");
+    if ("stageList" in entries) {
+        const path = entries.stageList;
+        if (path !== null &&
+            (typeof path !== "string" ||
+                path === "" ||
+                isAbsolute(path) ||
+                escapes(path) ||
+                // Read with the platform's resolve: on Windows `..\x` and `C:x` leave the root.
+                path.includes("\\") ||
+                /^[A-Za-z]:/.test(path)))
+            throw new Error("stageList must be a file path inside the repository, with / separators, or null");
+        if (path !== null && "stages" in entries)
+            throw new Error("give stages or stageList, not both");
+        config.stageList = path;
     }
+    if ("stageListRequired" in entries)
+        config.stageListRequired = strings(entries.stageListRequired, "stageListRequired");
+    if ("records" in entries)
+        config.records = parseRecordsConfig(entries.records);
     return config;
 };
 /**

package/dist/records/baseline.js

@@ -0,0 +1,152 @@
+// The ratchet for files written before the gate existed.
+//
+// A legacy file's current count of findings per rule is recorded here, and the
+// gate holds each count where it is or lower. Three refusals keep the numbers
+// honest:
+//
+// - a file over its recorded count (a new violation in a legacy file);
+// - a file under it (the fix lands with the lower number, so it cannot creep
+//   back up to the old allowance), and an entry for a file that is gone or
+//   that the gate does not read;
+// - a baseline edit that raises a count or adds a file, other than a rename
+//   git detects in the same change carrying its entry forward at the same or
+//   lower counts.
+//
+// The commit that adds the baseline has no earlier one to compare with; the
+// gate then checks every file it names, so each entry must equal the tree.
+import { RULE_IDS } from "./types.js";
+/**
+ * A map keyed by repository path with no prototype: a tracked file named
+ * `__proto__` or `constructor` is an entry like any other.
+ */
+export const pathMap = () => Object.create(null);
+/** A JSON object: not an array, not null. */
+const isJsonObject = (value) => typeof value === "object" && value !== null && !Array.isArray(value);
+const isRuleId = (rule) => RULE_IDS.includes(rule);
+/** The baseline in `text`; `where` names it in errors. Absent or blank is empty. */
+export const parseBaseline = (text, where) => {
+    if (text === null || text.trim() === "")
+        return { files: pathMap() };
+    let raw;
+    try {
+        raw = JSON.parse(text);
+    }
+    catch (error) {
+        throw new Error(`${where}: ${error.message}`);
+    }
+    if (!isJsonObject(raw) || !("files" in raw))
+        throw new Error(`${where}: expected {"files": {...}}`);
+    const files = raw.files;
+    if (!isJsonObject(files))
+        throw new Error(`${where}: "files" must be an object`);
+    const out = pathMap();
+    for (const [path, counts] of Object.entries(files)) {
+        if (!isJsonObject(counts))
+            throw new Error(`${where}: ${path} must map rule ids to counts`);
+        const entry = {};
+        for (const [rule, count] of Object.entries(counts)) {
+            if (!isRuleId(rule))
+                throw new Error(`${where}: ${path} names unknown rule ${rule}`);
+            if (typeof count !== "number" || !Number.isInteger(count) || count < 1)
+                throw new Error(`${where}: ${path} ${rule} must be a positive integer`);
+            entry[rule] = count;
+        }
+        out[path] = entry;
+    }
+    return { files: out };
+};
+/** The baseline as committed: paths sorted, zero counts and empty entries dropped. */
+export const renderBaseline = (baseline) => {
+    const files = pathMap();
+    for (const path of Object.keys(baseline.files).sort()) {
+        const counts = {};
+        for (const rule of RULE_IDS) {
+            const count = baseline.files[path]?.[rule];
+            if (count !== undefined && count > 0)
+                counts[rule] = count;
+        }
+        if (Object.keys(counts).length > 0)
+            files[path] = counts;
+    }
+    return `${JSON.stringify({ files }, null, 2)}\n`;
+};
+export const countFindings = (findings) => {
+    const out = new Map();
+    for (const finding of findings) {
+        const counts = out.get(finding.file) ?? {};
+        counts[finding.rule] = (counts[finding.rule] ?? 0) + 1;
+        out.set(finding.file, counts);
+    }
+    return out;
+};
+/** Compare the checked files' findings with their allowances. */
+export const ratchet = (checked, findings, baseline) => {
+    const result = { over: [], under: [] };
+    const held = findings.filter((f) => f.ratchet !== false);
+    const counts = countFindings(held);
+    // A finding no baseline may hold is over, whatever the file's allowance.
+    const unheld = findings.filter((f) => f.ratchet === false);
+    for (const file of new Set(unheld.map((f) => f.file))) {
+        for (const rule of RULE_IDS) {
+            const mine = unheld.filter((f) => f.file === file && f.rule === rule);
+            if (mine.length === 0)
+                continue;
+            result.over.push({
+                file,
+                rule,
+                allowed: 0,
+                now: mine.length,
+                findings: mine,
+            });
+        }
+    }
+    for (const file of checked) {
+        const now = counts.get(file) ?? {};
+        const allowed = baseline.files[file] ?? {};
+        for (const rule of RULE_IDS) {
+            const n = now[rule] ?? 0;
+            const a = allowed[rule] ?? 0;
+            if (n > a)
+                result.over.push({
+                    file,
+                    rule,
+                    allowed: a,
+                    now: n,
+                    findings: held.filter((f) => f.file === file && f.rule === rule),
+                });
+            else if (n < a)
+                result.under.push({ file, rule, allowed: a, now: n });
+        }
+    }
+    return result;
+};
+/** Baseline entries for files `exists` does not vouch for. */
+export const staleEntries = (baseline, exists) => Object.keys(baseline.files).filter((path) => !exists(path));
+/**
+ * Refusals for a baseline edit that raises any allowance. A new entry is
+ * accepted only as a rename git detected in the same change (`renames`, new
+ * path to old path), whose old entry is removed and whose counts are no
+ * higher: fixing one legacy file must not fund a violation in another.
+ */
+export const baselineIncreases = (before, after, renames = new Map()) => {
+    const problems = [];
+    const carriesForward = (from, counts) => {
+        const old = before.files[from];
+        return (!(from in after.files) &&
+            old !== undefined &&
+            RULE_IDS.every((rule) => (counts[rule] ?? 0) <= (old[rule] ?? 0)));
+    };
+    for (const [path, counts] of Object.entries(after.files)) {
+        const old = before.files[path];
+        if (old === undefined) {
+            const from = renames.get(path);
+            if (from === undefined || !carriesForward(from, counts))
+                problems.push(`${path}: a new baseline entry; fix the file instead of recording its findings`);
+            continue;
+        }
+        for (const rule of RULE_IDS)
+            if ((counts[rule] ?? 0) > (old[rule] ?? 0))
+                problems.push(`${path}: ${rule} raised from ${old[rule] ?? 0} to ${counts[rule] ?? 0}; counts only fall`);
+    }
+    return problems;
+};