@agentx-core/security-sdk 0.1.0 → 0.2.0

package/README.md

@@ -66,9 +66,76 @@ value to that file: the wrapper reduces a call to its shape before anything is w
 `test/wrap.test.ts` asserts the file holds no value from a call carrying a SQL statement, an
 email address and a dollar figure. This is exactly what the Python SDK's `agentx audit` keeps.
 
-The file is yours. Add it to `.gitignore`; the wrapper reminds you once when it creates it. It
-keeps the last 30 days or 10,000 rows, and when it has trimmed, the report says the counts
-describe what was **kept**.
+The files are yours. There are two, side by side: `.agentx-calls.jsonl` is the record, and
+`.agentx-novelty.json` remembers what you had already seen, so the "what changed" block below
+means something. Add both to `.gitignore`; the wrapper names both, once, when it creates the
+record. The record keeps the last 30 days or 10,000 rows, and when it has trimmed, the report
+says the counts describe what was **kept**.
+
+## What changed since you last looked
+
+Running `audit` a second time answers a different question from the first. Under the table it
+prints what is new: a tool you have not seen before, a first call to your database or
+filesystem, an argument that got added or dropped, a bigger amount than any before, a busier
+day, a burst inside one minute, a first call at an hour you have not run at, a first weekend
+call, and a return after a quiet stretch of two days or more.
+
+It only appears when something actually changed. A quiet run prints nothing at all.
+
+## How many tools you wrapped, against how many ran
+
+```
+  4 calls across 2 tools since 2026-09-08 20:52
+  5 tools wrapped. 2 called. 3 never ran.
+```
+
+The second line is the denominator the call list cannot give you: three tools were handed to
+the model and it never reached for them.
+
+## Reading the record in CI
+
+```bash
+npx @agentx-core/security-sdk@^0.2.0 audit --require-calls
+```
+
+Without that flag, a job where the agent never ran leaves no record, `audit` reads zero calls,
+and the step exits 0. A green tick on a check that measured nothing is worse than no check.
+With it, that case exits 2 and says which of the two reasons it was.
+
+| exit | meaning |
+|---|---|
+| 0 | it read the record |
+| 1 | the command line was wrong |
+| 2 | it could not judge anything: the record is unreadable, or `--require-calls` found no calls |
+
+`--json` always prints a complete payload, so a pipeline can gate on the exit code and still
+keep the data.
+
+A worked GitHub Actions job, inline rather than linked, because the npm package ships `dist`
+only and a link would be a dead end:
+
+```yaml
+      - run: npm ci
+
+      # Your agent, with its tools wrapped. It writes the record into whatever
+      # directory it runs in, and audit below reads the one beside it.
+      - name: Run the agent
+        run: node ./scripts/run-agent.js
+
+      # Exits 2 if the record holds no calls, so a run where the agent silently
+      # did nothing cannot pass as a clean audit.
+      - name: What did the agent do
+        run: npx @agentx-core/security-sdk@^0.2.0 audit --require-calls
+
+      - name: Keep the record
+        if: always()
+        run: npx @agentx-core/security-sdk@^0.2.0 audit --json > agentx-audit.json || true
+```
+
+Two things it deliberately does not do. It does not fail on anything the agent *did*: this
+package has no rules and makes no judgement about a call. And it does not gate on the
+what-changed block, because that works off a stored file and a CI container starts clean, so
+everything would look new on every run.
 
 ## Nothing is blocked
 

package/dist/cli.js

@@ -13,6 +13,7 @@ exports.main = main;
 const ledger_1 = require("./ledger");
 const report_1 = require("./report");
 const pulse_1 = require("./pulse");
+const novelty_1 = require("./novelty");
 const USAGE = `
 🔎 agentx audit — what your agent actually did, from the local record.
 
@@ -20,6 +21,7 @@ const USAGE = `
   ${report_1.AUDIT_COMMAND} --calls       one line per call, newest first
   --json                          machine-readable output (always complete)
   --limit N | --all               how many rows the human screen shows
+  --require-calls                 exit 2 if the ledger holds no calls (for CI)
   --help                          this message
 
 The record is written by agentxWatch() into .agentx-calls.jsonl in the folder your agent ran
@@ -43,7 +45,9 @@ function main(argv, io = defaultIo) {
         io.err(USAGE.trim());
         return 1;
     }
-    const opts = { calls: false, json: false };
+    const opts = {
+        calls: false, json: false, requireCalls: false,
+    };
     const die = (message) => {
         io.err(`\n❌ ${message}`);
         io.err(`   Usage:  ${report_1.AUDIT_COMMAND} [--calls] [--json] [--limit N | --all]`);
@@ -55,6 +59,8 @@ function main(argv, io = defaultIo) {
             opts.calls = true;
         else if (tok === "--json")
             opts.json = true;
+        else if (tok === "--require-calls")
+            opts.requireCalls = true;
         else if (tok === "--all")
             opts.all = true;
         else if (tok === "--limit") {
@@ -78,21 +84,80 @@ function main(argv, io = defaultIo) {
     const file = (0, ledger_1.ledgerPath)(env);
     (0, ledger_1.trimLedger)(file);
     const read = (0, ledger_1.readLedger)(file);
+    // 🔴 "I LOOKED AND IT WAS CLEAN" AND "THERE WAS NOTHING TO LOOK AT" ARE THE SAME EXIT CODE
+    // WITHOUT THIS FLAG, and in CI that is the whole problem: a job where the agent never ran
+    // leaves no ledger, this command reads zero rows, and a green tick reports a check that
+    // observed nothing. Exit 2, the same code an unreadable ledger uses, because both mean the
+    // output is not a statement about the agent. Not a new code: 3 is reserved for a gate that
+    // looked and FOUND something, which needs rules this package does not have yet.
+    //
+    // Deliberately BEFORE the `--json` branch's return but AFTER the payload is printed, so the
+    // machine path still emits complete JSON and only the exit code carries the failure.
+    const noCallsToJudge = opts.requireCalls && read.readable && read.rows.length === 0;
     if (opts.json) {
         io.out(JSON.stringify((0, report_1.jsonPayload)(read, opts.calls ? "calls" : "tools", opts), null, 2));
-        return read.readable ? 0 : 2;
+        if (!read.readable)
+            return 2;
+        if (noCallsToJudge) {
+            io.err("\n❌ --require-calls: this ledger holds no calls, so there is nothing to judge.");
+            return 2;
+        }
+        return 0;
     }
     // A human looked. The rung this package exists to make visible, and the one thing `--json`
     // must never climb: a cron job polling it is not a person.
-    (0, pulse_1.markAuditReportRun)(undefined, env);
+    //
+    // 🔴 `--require-calls` DOES NOT CLIMB IT EITHER, for the same reason. Its own usage line says
+    // "for CI", so it is a reliable statement that nobody is reading. `isAutomationContext` catches
+    // the usual thirteen CI variables, but a self-hosted runner, a cron job or a bare container may
+    // set none of them, and this mark is STICKY and cannot be un-climbed: one machine invocation
+    // would record the install converted forever.
+    if (!opts.requireCalls)
+        (0, pulse_1.markAuditReportRun)(undefined, env);
     if (!read.readable) {
         io.err("\n❌ The ledger is on disk but could not be read, so this is not a statement that");
         io.err(`   your agent did nothing.  ${file}`);
         return 2;
     }
-    io.out(opts.calls ? (0, report_1.renderCalls)(read, opts) : (0, report_1.renderTools)(read, opts));
+    // 🔴 THE FAILURE IS THE WHOLE OUTPUT, and the empty screen is skipped rather than printed
+    // above it. `--require-calls` is a CI flag; on a job with no ledger the human screen put twelve
+    // lines of onboarding, including a "here is how to wrap a tool" snippet, in front of the three
+    // lines a build log actually needs. Found in a founder walk. Nothing is lost: the error names
+    // the ledger path itself, and names both reasons including the unwrapped one.
+    if (noCallsToJudge)
+        return requireCallsFailed(io, file);
+    if (opts.calls) {
+        io.out((0, report_1.renderCalls)(read, opts));
+        return 0;
+    }
+    // NOVELTY, on the grouped screen only, matching the Python audit. Read first, print, and move
+    // the mark only afterwards: a reader whose terminal dies mid-screen would otherwise lose that
+    // novelty permanently. `--json` gets none of this on purpose -- "since you last looked" needs
+    // a reader who looks, and a job polling a JSON endpoint has no last look to be since.
+    //
+    // 🔴 AND `--require-calls` GETS NONE OF IT EITHER, WHICH IS THE SAME RULE AS THE PULSE MARK
+    // ABOVE AND WAS MISSED THERE FIRST. There are TWO sticky things a run can consume: the pulse's
+    // "a human read the report", and this mark. The first version of the CI carve-out skipped only
+    // the pulse, four lines up, and left this one consuming. On a build machine that keeps its
+    // workspace between runs -- a self-hosted runner, a Makefile target, a pre-commit hook -- the
+    // CI invocation ate "NEW SINCE YOU LAST LOOKED" and the developer's next `audit` printed
+    // nothing. A flag that says "for CI" in its own usage line must not climb OR consume anything.
+    if (opts.requireCalls) {
+        io.out((0, report_1.renderTools)(read, opts));
+        return 0;
+    }
+    const marks = (0, novelty_1.noveltyPath)(env);
+    const novelty = (0, novelty_1.readNovelty)("report", read.rows, marks);
+    io.out((0, report_1.renderTools)(read, opts, (0, novelty_1.renderNovelty)(novelty)));
+    (0, novelty_1.advanceMark)("report", novelty.current, marks);
     return 0;
 }
+function requireCallsFailed(io, file) {
+    io.err("\n❌ --require-calls: this ledger holds no calls, so there is nothing to judge.");
+    io.err(`   Nothing has been recorded in  ${file}`);
+    io.err("   Either the agent did not run, or its tools are not wrapped with agentxWatch().");
+    return 2;
+}
 // Only auto-run as the real entry point (the published `bin`), never when a test imports `main`.
 if (require.main === module && !process.env.VITEST) {
     process.exitCode = main(process.argv.slice(2));

package/dist/ledger.d.ts

@@ -13,12 +13,29 @@ export interface CallRow {
     quantity: number;
     agent: string;
 }
+/**
+ * The tools that were WRAPPED, whether or not any of them ran. The denominator the call rows
+ * cannot supply: a wrapped tool that never executed appears in no call row, so without this the
+ * screen can say what the agent did and not what it could have done.
+ *
+ * Carries a per-process `session` because a program may wrap several tool maps, and the answer
+ * wanted is "what could this run have called", not "every tool this folder has ever held". The
+ * reader takes the newest roster row's session and unions every row sharing it.
+ */
+export interface RosterRow {
+    k: "roster";
+    ts: number;
+    session: string;
+    tools: string[];
+}
 /** Where the ledger lives. `AGENTX_TS_LEDGER_PATH` moves it (tests, and a shared folder). */
 export declare function ledgerPath(env?: NodeJS.ProcessEnv): string;
 /** Append one call. Returns whether this write CREATED the file (the first-run notice keys on it). */
 export declare function appendCall(row: CallRow, file?: string): {
     created: boolean;
 };
+/** Write this run's roster. Never creates the file: a roster with no calls is not a record of a run. */
+export declare function appendRoster(row: RosterRow, file?: string): void;
 export interface LedgerRead {
     path: string;
     exists: boolean;
@@ -28,6 +45,8 @@ export interface LedgerRead {
     rows: CallRow[];
     /** Cumulative rows dropped by retention. 0 means the rows are everything ever recorded. */
     dropped: number;
+    /** Every tool the most recent run WRAPPED, called or not. Empty on a ledger with no roster row. */
+    roster: string[];
     coversAll: boolean;
     /** Epoch ms of the oldest kept row, or null. */
     windowStart: number | null;
@@ -35,6 +54,12 @@ export interface LedgerRead {
     bytes: number;
     cleanEnd: boolean;
 }
+/**
+ * The newest run's roster: the session id on the latest roster row, then every tool named by any
+ * row sharing it. Sorted, deduplicated, and empty when the ledger holds no roster row at all,
+ * which is every ledger written before this shipped.
+ */
+export declare function latestRoster(rosters: readonly RosterRow[]): string[];
 export declare function readLedger(file?: string): LedgerRead;
 /**
  * Apply retention: drop rows older than RETENTION_DAYS, then the oldest past RETENTION_MAX_ROWS.

package/dist/ledger.js

@@ -36,6 +36,8 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.RETENTION_MAX_ROWS = exports.RETENTION_DAYS = exports.LEDGER_FILE = void 0;
 exports.ledgerPath = ledgerPath;
 exports.appendCall = appendCall;
+exports.appendRoster = appendRoster;
+exports.latestRoster = latestRoster;
 exports.readLedger = readLedger;
 exports.trimLedger = trimLedger;
 /**
@@ -65,8 +67,20 @@ function appendCall(row, file = ledgerPath()) {
     fs.appendFileSync(file, JSON.stringify(row) + "\n", "utf8");
     return { created };
 }
+/** Write this run's roster. Never creates the file: a roster with no calls is not a record of a run. */
+function appendRoster(row, file = ledgerPath()) {
+    try {
+        if (!fs.existsSync(file))
+            return;
+        fs.appendFileSync(file, JSON.stringify(row) + "\n", "utf8");
+    }
+    catch {
+        /* the denominator is a nicety; losing it is never worth a broken exit */
+    }
+}
 function parseLines(text) {
     const rows = [];
+    const rosters = [];
     let dropped = 0;
     for (const line of text.split("\n")) {
         if (!line.trim())
@@ -82,6 +96,9 @@ function parseLines(text) {
         if (r && r.k === "trim" && typeof r.dropped === "number") {
             dropped = Math.max(dropped, r.dropped);
         }
+        else if (r && r.k === "roster" && Array.isArray(r.tools) && typeof r.ts === "number") {
+            rosters.push({ k: "roster", ts: r.ts, session: String(r.session ?? ""), tools: r.tools.map(String) });
+        }
         else if (r && r.k === "call" && typeof r.tool === "string" && typeof r.ts === "number") {
             rows.push({
                 k: "call",
@@ -95,12 +112,29 @@ function parseLines(text) {
             });
         }
     }
-    return { rows, dropped };
+    return { rows, dropped, rosters };
+}
+/**
+ * The newest run's roster: the session id on the latest roster row, then every tool named by any
+ * row sharing it. Sorted, deduplicated, and empty when the ledger holds no roster row at all,
+ * which is every ledger written before this shipped.
+ */
+function latestRoster(rosters) {
+    if (!rosters.length)
+        return [];
+    const newest = rosters.reduce((a, b) => (b.ts >= a.ts ? b : a));
+    const names = new Set();
+    for (const r of rosters) {
+        if (r.session === newest.session)
+            for (const t of r.tools)
+                names.add(t);
+    }
+    return [...names].sort();
 }
 function readLedger(file = ledgerPath()) {
     const base = {
-        path: file, exists: false, readable: true, rows: [], dropped: 0, coversAll: true, windowStart: null,
-        bytes: 0, cleanEnd: true,
+        path: file, exists: false, readable: true, rows: [], dropped: 0, roster: [], coversAll: true,
+        windowStart: null, bytes: 0, cleanEnd: true,
     };
     if (!fs.existsSync(file))
         return base;
@@ -114,12 +148,13 @@ function readLedger(file = ledgerPath()) {
         // statement about the developer's own data.
         return { ...base, readable: false };
     }
-    const { rows, dropped } = parseLines(buf.toString("utf8"));
+    const { rows, dropped, rosters } = parseLines(buf.toString("utf8"));
     rows.sort((a, b) => a.ts - b.ts);
     return {
         ...base,
         rows,
         dropped,
+        roster: latestRoster(rosters),
         coversAll: dropped === 0,
         windowStart: rows.length ? rows[0].ts : null,
         bytes: buf.length,
@@ -145,7 +180,19 @@ function trimLedger(file = ledgerPath(), now = Date.now(), maxAgeDays = exports.
         if (droppedNow === 0)
             return { dropped: 0 };
         const trim = { k: "trim", ts: now, dropped: read.dropped + droppedNow };
-        const lines = [JSON.stringify(trim), ...kept.map((r) => JSON.stringify(r))].join("\n") + "\n";
+        // 🔴 THE ROSTER IS CARRIED THROUGH THE REWRITE, and leaving it out was the first version of
+        // this. `read.rows` holds CALL rows only, so a rewrite built from them alone silently drops
+        // the roster and the screen loses its denominator the first time retention fires. The
+        // carried row is the newest run's, rebuilt from `read.roster`, so the rewrite is also where
+        // older runs' rosters are collapsed away rather than accumulating forever.
+        const carried = read.roster.length
+            ? [{ k: "roster", ts: now, session: "carried", tools: read.roster }]
+            : [];
+        const lines = [
+            JSON.stringify(trim),
+            ...carried.map((r) => JSON.stringify(r)),
+            ...kept.map((r) => JSON.stringify(r)),
+        ].join("\n") + "\n";
         fs.writeFileSync(tmp, lines, "utf8");
         // Other processes write this file too: an agent appending in another terminal, or a second
         // exiting agent running its own trim. Rows appended after the read (the file grew, and what

package/dist/novelty.d.ts

@@ -0,0 +1,138 @@
+import { type CallRow } from "./ledger";
+import { type TargetClass } from "./shape";
+/** The two screens that read novelty, each with its own mark. See decision 2 above. */
+export type Mark = "report" | "session";
+/** How many lines one screen prints before it counts the rest. */
+export declare const MAX_NOVELTY_SHOWN = 6;
+/** Per-tool totals. Every field only ever moves up, which is what makes decision 1 work. */
+export interface ToolTotals {
+    calls: number;
+    /** Every argument name ever seen on this tool. */
+    args: string[];
+    /** Every distinct argument-name COMBINATION, as sorted joined keys. "" is a real member (a call with no arguments). */
+    combos: string[];
+    /** Every surface class ever seen. */
+    classes: TargetClass[];
+    maxAmount: number;
+    /** Most calls on any one local calendar day, and which day. */
+    maxDayCalls: number;
+    maxDay: string;
+    /** Most calls inside any one local minute, and WHICH minute. See `recordChanged`. */
+    maxBurst: number;
+    maxBurstAt: string;
+    /** Every local hour-of-day this tool has run in. */
+    hours: number[];
+    weekend: boolean;
+    /** Epoch ms of the most recent call. The one field that is a time, and it is not what novelty is diffed on. */
+    lastTs: number;
+}
+export interface Totals {
+    tools: Record<string, ToolTotals>;
+    busiest: string;
+    busiestCalls: number;
+    totalCalls: number;
+    distinctTools: number;
+}
+export type NoveltyKind = "tool" | "surface" | "argument" | "dropped" | "amount" | "busiest" | "busiest_day" | "burst" | "hour" | "weekend" | "gap";
+export interface NoveltyItem {
+    kind: NoveltyKind;
+    /** Empty for a fact about the session rather than about one tool. See `renderNovelty`. */
+    tool: string;
+    detail?: unknown;
+}
+/**
+ * Roll the kept rows up into per-tool totals. Pure; takes rows and returns a value.
+ *
+ * ⚠️ THIS IS COMPUTED FROM WHAT THE LEDGER STILL HOLDS, so it can be LOWER than a stored mark
+ * after a trim. That is exactly the case decision 1 exists for, and `diffNovelty` only ever
+ * reports a rise, never a fall.
+ */
+export declare function computeTotals(rows: readonly CallRow[]): Totals;
+/**
+ * What is new in `current` that was not in `previous`. Pure.
+ *
+ * 🔴 EVERY COMPARISON IS "HAS THIS RISEN", NEVER "HAS THIS CHANGED". A trim can only make the
+ * computed totals smaller, and a smaller number is not news.
+ */
+export declare function diffNovelty(previous: Totals | null, current: Totals, rows?: readonly CallRow[]): NoveltyItem[];
+/**
+ * The stored mark, RATCHETED: every maximum and every set only ever grows.
+ *
+ * 🔴 WITHOUT THIS, DECISION 1 AT THE TOP OF THIS FILE IS A COMMENT AND NOT A BEHAVIOUR.
+ * `diffNovelty` only ever reports a rise, but `advanceMark` used to store the freshly computed
+ * totals wholesale, and those are computed over the KEPT rows. So a trim moved the mark DOWN.
+ * Concretely: `chargeCard` runs at 1,000,000 and the mark records it; thirty-one days later that
+ * row is trimmed and the next read stores `maxAmount: 500`; a later call at 900 is then
+ * announced as "largest amount yet: ≥900". The same held for the surface set (a repeat "first
+ * call to your database"), the argument names, the busiest day and the burst.
+ *
+ * `busiest` is deliberately NOT ratcheted: "now your busiest tool" is a claim about the present,
+ * and a genuine change of leader is news each time it happens.
+ *
+ * ⚠️ AND IT IS UNBOUNDED, WHICH IS A KNOWN, DELIBERATE, WRITTEN-DOWN GAP. Before the ratchet the
+ * mark was recomputed from the kept rows, so the ledger's retention capped it as a side effect
+ * nobody had decided; merging removed that cap. A program naming tools dynamically, one per
+ * tenant or per resource, accumulates every name it has ever used.
+ *
+ * 🔴 AN EVICTION RULE WAS WRITTEN AND THEN REVERTED, AND THE REASON IS THE POINT. Dropping tools
+ * whose last call had aged out of the ledger reintroduced this function's own worked example: a
+ * `chargeCard` that runs monthly at 1,000,000 had its entry evicted on day 31 and, on day 32,
+ * was announced as "first time we have seen this tool" with "largest amount yet: ≥900". The
+ * justification, that a tool aged out of the ledger cannot be diffed anyway, was false: the
+ * STORED MARK is the other side of the diff, which is the entire reason totals are stored rather
+ * than read back off the rows. Any eviction keyed on time has this defect for any tool that runs
+ * less often than the window. A cap on the NUMBER of entries would not, and that is what the
+ * backlog row asks for. Growth is slow and hypothetical; the bug the eviction caused was
+ * immediate, so the gap is carried openly instead.
+ */
+export declare function mergeMark(previous: Totals | null, current: Totals): Totals;
+/** The words for one item, in ONE place because two screens print them. "" for anything unrecognised. */
+export declare function formatNoveltyItem(item: NoveltyItem): string;
+/**
+ * The items that actually produce a line. `formatNoveltyItem` returns "" for a kind it cannot
+ * word (a surface with no plain noun, an empty argument list), so a raw `items.length` counts
+ * things no reader will ever see. Both screens count off THIS, so the session line's number and
+ * the report's "(N more not listed)" cannot disagree.
+ */
+export declare function formattableItems(items: readonly NoveltyItem[]): NoveltyItem[];
+export declare function topNovelty(items: readonly NoveltyItem[], limit: number): NoveltyItem[];
+export declare const NOVELTY_FILE = ".agentx-novelty.json";
+/** Beside the ledger, so moving the ledger moves the mark with it. */
+export declare function noveltyPath(env?: NodeJS.ProcessEnv): string;
+export interface NoveltyRead {
+    items: NoveltyItem[];
+    /** True when this screen has no stored mark yet. The header changes; see `renderNovelty`. */
+    firstLook: boolean;
+    /** The totals a later `advanceMark` should store. */
+    current: Totals;
+}
+/**
+ * PURE READ. Works out what is new for one screen and stores nothing. `advanceMark` is the
+ * separate, deliberate second step. See decision 3.
+ */
+export declare function readNovelty(mark: Mark, rows: readonly CallRow[], file?: string): NoveltyRead;
+/**
+ * Store the totals for one screen. Called only after the screen has actually been written.
+ *
+ * 🔴 MERGED INTO THE STORED MARK, NEVER WRITTEN OVER IT. `current` is computed from the rows the
+ * ledger still holds, so writing it wholesale moves the mark DOWN after a trim and re-announces
+ * an ordinary call as a record. See `mergeMark`.
+ */
+export declare function advanceMark(mark: Mark, current: Totals, file?: string): void;
+/**
+ * The ONE line the session-end screen prints: how much is new, and where to read it. "" when
+ * there is nothing to say, so the caller cannot print a bare pointer to an empty report.
+ *
+ * A pure function on purpose. It runs from a process exit hook, where a test cannot see it, and
+ * a sentence is exactly the thing no test reddens unless it is reachable. This one is.
+ */
+export declare function sessionNoveltyLine(read: NoveltyRead): string;
+/**
+ * The block both screens print, as lines. Empty array when there is nothing to say, so a caller
+ * cannot accidentally print a bare header over nothing.
+ *
+ * TWO HEADERS, because they are two different claims. On a ledger we have never marked, every
+ * tool in it is new to US, not new to the agent, and "first call to your database" under a
+ * "since you last looked" header would date a year of history to this afternoon.
+ */
+export declare function renderNovelty(read: NoveltyRead, indent?: string): string[];