@dogfood-lab/ingest 1.4.0 → 1.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -10,7 +10,7 @@
10
10
 
11
11
  Part of the [`testing-os`](https://github.com/dogfood-lab/testing-os) monorepo — the operating system for testing in the AI era.
12
12
 
13
- Runs on the receiving side of the dogfood loop. Receives `repository_dispatch` payloads from consumer repos, validates them through `@dogfood-lab/verify`, persists records under `records/`, and rebuilds the read-side indexes (`latest-by-repo.json`, `failing.json`, `stale.json`) atomically.
13
+ Runs on the receiving side of the dogfood loop. Receives a submission (from a `repository_dispatch` payload or a file), validates it through `@dogfood-lab/verify`, persists the resulting record under `records/`, and rebuilds the read-side indexes (`latest-by-repo.json`, `failing.json`, `stale.json`).
14
14
 
15
15
  ## Install
16
16
 
@@ -20,49 +20,69 @@ npm install @dogfood-lab/ingest
20
20
 
21
21
  ## Usage — programmatic
22
22
 
23
+ The package exports two functions: `ingest` (verify + persist + rebuild indexes) and `verifyOnly` (verify + report where it *would* land, write nothing).
24
+
23
25
  ```js
24
- import { run } from '@dogfood-lab/ingest';
26
+ import { ingest, verifyOnly } from '@dogfood-lab/ingest';
27
+ import { githubProvenance } from '@dogfood-lab/verify';
28
+
29
+ const submission = JSON.parse(/* the dispatch payload's submission object */);
25
30
 
26
- const result = await run({
27
- payloadPath: './incoming.json',
28
- repoRoot: process.cwd(),
29
- provenance: 'github',
31
+ const result = await ingest(submission, {
32
+ repoRoot: process.cwd(), // testing-os repo root the record is written under
33
+ provenance: githubProvenance(process.env.GITHUB_TOKEN), // provider adapter from @dogfood-lab/verify
30
34
  });
31
35
 
32
- console.log(result.persisted_records); // count of new records written
33
- console.log(result.indexes_rebuilt); // boolean
34
- console.log(result.rejected); // array of rejection reasons (if any)
36
+ result.record; // the verified record (null if rejected or a duplicate)
37
+ result.path; // where it was written (null if not written)
38
+ result.written; // boolean did a new record land on disk?
39
+ result.duplicate; // boolean — was this run_id already persisted?
40
+ ```
41
+
42
+ `verifyOnly(submission, options)` runs the same verification but never writes:
43
+
44
+ ```js
45
+ const { record, would_persist_to, verify_only } = await verifyOnly(submission, { repoRoot, provenance });
46
+ // verify_only === true; would_persist_to is the path ingest() would have used.
35
47
  ```
36
48
 
37
49
  ## Usage — CLI
38
50
 
39
51
  ```bash
40
- npx @dogfood-lab/ingest run --payload incoming.json --repo-root .
52
+ node packages/ingest/run.js --provenance=github --file submission.json
41
53
  ```
42
54
 
43
- The CLI exits with structured exit codes:
55
+ Flags:
56
+
57
+ - `--file <path>` / `--payload <path>` — the submission JSON to ingest (both spellings accepted).
58
+ - `--provenance=github` — confirm provenance against the GitHub Actions API (requires `GITHUB_TOKEN` / `GH_TOKEN`). **Production.**
59
+ - `--provenance=stub` — accept the claimed provenance without an API call. **Test/dev only — refused in CI.**
60
+ - `--verify-only` — verify and report, write nothing.
61
+
62
+ Exit codes:
63
+
64
+ - `0` — the record was accepted (and, without `--verify-only`, persisted).
65
+ - `1` — the submission was verified but **not accepted** (rejected by a validator gate).
66
+ - `2` — an operator/runtime fault (missing/unreadable `--file`, missing `--provenance`, missing token in CI, downstream I/O failure). Every exit-2 path emits a structured `logStage('error', …)` event first so a log grep finds the cause.
44
67
 
45
- - `0` ok, all records persisted
46
- - `1` — user error (bad payload shape, missing files, invalid args)
47
- - `2` — runtime error (downstream validator failure, I/O failure)
48
- - `3` — partial success (some records persisted, some rejected)
68
+ There is no exit code 3.
49
69
 
50
70
  ## Pipeline stages
51
71
 
52
72
  | Stage | Module | Output |
53
73
  |---|---|---|
54
- | 1. Load context | `load-context.js` | Reads existing `records/`, `policies/`, prior indexes into memory |
55
- | 2. Validate | delegates to `@dogfood-lab/verify` | Verdict per record: `ok` or `rejection_reasons[]` |
56
- | 3. Persist | `persist.js` | Two-phase atomic write to `records/<repo>/<record-id>.yaml` |
74
+ | 1. Load context | `load-context.js` | Reads existing `records/`, `policies/`, and prior indexes into memory |
75
+ | 2. Verify | delegates to `@dogfood-lab/verify` | A verdict: `accepted`, or `rejection_reasons[]` |
76
+ | 3. Persist | `persist.js` | Atomic write to `records/<org>/<repo>/YYYY/MM/DD/run-<run_id>.json` |
57
77
  | 4. Rebuild indexes | `rebuild-indexes.js` | Regenerates `latest-by-repo.json`, `failing.json`, `stale.json` with crash-safe journaling |
58
78
 
59
- Each stage emits a structured event via `lib/log-stage.js` so the ingest loop is observable end-to-end. Failures at any stage carry a `code`, `message`, and `hint` per the testing-os structured error contract.
79
+ Each stage emits a structured NDJSON event via `lib/log-stage.js` (carrying a `correlation_id`) so the ingest loop is observable end to end. Operator-facing faults carry a `code` / `message` / `hint` per the testing-os [error contract](https://dogfood-lab.github.io/testing-os/handbook/error-codes/).
60
80
 
61
81
  ## Concurrency + crash safety
62
82
 
63
- - **Per-repo locking**: advisory file lock via `@dogfood-lab/findings/lib/file-lock.js` Windows-compatible (uses `linkSync` CAS, not `flock`).
64
- - **Atomic two-phase commit**: `lib/atomic-write.js` writes to a `.tmp-<pid>.json` shadow first, then renames into place with `renameWithRetry` for Windows AV scanner handle-release windows.
65
- - **Idempotent crash recovery**: `cleanupCrashedJournals` runs at every entry point; partial writes from previous crashes are detected and either completed or rolled back.
83
+ - **Race-safe record claim:** `persist.js` claims the canonical record path with `openSync(path, 'wx')` (`O_EXCL`). The first writer wins; a concurrent writer for the same `run_id` loses the race and surfaces a `DuplicateRunIdError` — no torn or double-written record.
84
+ - **Atomic publication:** records and indexes are written to a temp file and `renameWithRetry`'d into place (the retry absorbs the Windows AV-scanner / lock handle-release window).
85
+ - **Crash-recovery for the index group:** the 3-index rebuild is a journaled commit-group; a crash mid-promote is reconciled by the next rebuild. A transiently-unreadable `records/` root or an empty scan over a previously-non-empty corpus is **refused** rather than allowed to overwrite good indexes with empty content (it emits an `index_rebuild_skipped` event instead).
66
86
 
67
87
  ## What testing-os ingest does NOT touch
68
88
 
package/anchor/cli.js ADDED
@@ -0,0 +1,185 @@
1
+ /**
2
+ * anchor CLI handlers — the operator-facing surface for the three anchor verbs,
3
+ * wired into run.js: --anchor-compute, --anchor-post, --anchor-verify.
4
+ *
5
+ * Each handler returns `{ ok, exitCode, lines, event }` so run.js owns the actual
6
+ * console + logStage + process.exit (matching how `--verify-chain` is dispatched
7
+ * via verify-chain.js's verifyChain + formatChainResult). Handlers are
8
+ * operator-legible: no raw stack traces, structured exit codes.
9
+ *
10
+ * Off by default: nothing in the normal ingest/CI path calls these. They are
11
+ * operator-run only.
12
+ *
13
+ * Exit codes:
14
+ * 0 success (compute wrote/idempotent; post landed; verify all-clear)
15
+ * 1 a verification FAILURE (truncation detected, anchor mismatch) — fail loud
16
+ * 2 an operator/precondition error (bad algo, no seed, xrpl missing, etc.)
17
+ *
18
+ * Only --anchor-post reaches the network, and it lazily loads xrpl there; compute
19
+ * and verify never import xrpl.
20
+ */
21
+
22
+ import { computeAnchor, recordAnchorPost } from './compute-root.js';
23
+ import { verifyAnchorState } from './verify-anchor.js';
24
+ import { postAnchor } from './post-anchor.js';
25
+ import { ANCHOR_DEFAULTS, resolveTrustedAccounts } from './config.js';
26
+
27
+ /**
28
+ * --anchor-compute: compute the next anchor and write its append-only manifest.
29
+ * Offline; never imports xrpl.
30
+ *
31
+ * @param {string} repoRoot
32
+ * @param {object} [opts] - { mode, algo, network }
33
+ * @returns {{ ok: boolean, exitCode: number, lines: string[], event: object }}
34
+ */
35
+ export function handleAnchorCompute(repoRoot, opts = {}) {
36
+ try {
37
+ const result = computeAnchor(repoRoot, opts);
38
+ if (result.empty) {
39
+ return {
40
+ ok: true,
41
+ exitCode: 0,
42
+ lines: [`anchor-compute: nothing to anchor — ${result.reason}`],
43
+ event: { stage: 'anchor_compute_complete', empty: true, reason: result.reason },
44
+ };
45
+ }
46
+ const m = result.manifest;
47
+ const lines = [
48
+ `anchor-compute: ${result.written ? 'wrote' : 'idempotent (already present)'} anchor seq ${m.anchor_seq}`,
49
+ ` algo: ${m.algo}`,
50
+ ` leaf_count: ${m.leaf_count}`,
51
+ ` seq_range: [${m.seq_range[0]}, ${m.seq_range[1]}]`,
52
+ ` root: ${m.root}`,
53
+ ` head: ${m.head_digest}`,
54
+ ` prev: ${m.prev_anchor_root || '(genesis)'}`,
55
+ ` manifest: ${result.path}`,
56
+ ];
57
+ for (const w of result.warnings || []) lines.push(` WARNING: ${w}`);
58
+ return {
59
+ ok: true,
60
+ exitCode: 0,
61
+ lines,
62
+ event: {
63
+ stage: 'anchor_compute_complete',
64
+ anchor_seq: m.anchor_seq,
65
+ leaf_count: m.leaf_count,
66
+ root: m.root,
67
+ written: result.written,
68
+ },
69
+ };
70
+ } catch (err) {
71
+ return {
72
+ ok: false,
73
+ exitCode: 2,
74
+ lines: [`anchor-compute failed: ${err.message}`],
75
+ event: { stage: 'error', failed_stage: 'anchor_compute', code: err.code ?? null, message: err.message },
76
+ };
77
+ }
78
+ }
79
+
80
+ /**
81
+ * --anchor-post: compute (if needed) + post the anchor to the XRP Ledger. Needs
82
+ * the optional xrpl package and XRPL_SEED. Lazily loads xrpl; a missing xrpl or
83
+ * missing seed exits 2 with an actionable message.
84
+ *
85
+ * @param {string} repoRoot
86
+ * @param {object} [opts] - { mode, network, wsUrl, seed }
87
+ * @returns {Promise<{ ok: boolean, exitCode: number, lines: string[], event: object }>}
88
+ */
89
+ export async function handleAnchorPost(repoRoot, opts = {}) {
90
+ const network = opts.network || ANCHOR_DEFAULTS.network;
91
+ try {
92
+ const receipt = await postAnchor(repoRoot, {
93
+ ...opts,
94
+ network,
95
+ // Production: persist the on-chain facts back into the manifest file.
96
+ deps: {
97
+ recordPost: (seq, facts) => recordAnchorPost(repoRoot, seq, facts),
98
+ ...(opts.deps || {}),
99
+ },
100
+ });
101
+ if (!receipt.ok) {
102
+ return {
103
+ ok: false,
104
+ exitCode: 1,
105
+ lines: [
106
+ `anchor-post: submission did NOT succeed (engine result: ${receipt.engine_result || 'unknown'}).`,
107
+ ' The anchor did NOT land on-chain. Retry when the network is reachable.',
108
+ ],
109
+ event: { stage: 'error', failed_stage: 'anchor_post', engine_result: receipt.engine_result },
110
+ };
111
+ }
112
+ return {
113
+ ok: true,
114
+ exitCode: 0,
115
+ lines: [
116
+ `anchor-post: anchored seq ${receipt.anchor_seq} on ${receipt.network}`,
117
+ ` tx: ${receipt.tx_hash}`,
118
+ ` root: ${receipt.root}`,
119
+ ` leaf_count: ${receipt.leaf_count}`,
120
+ ` ledger_index: ${receipt.ledger_index ?? '(unavailable)'}`,
121
+ ` close_time: ${receipt.close_time_iso ? receipt.close_time_iso + ' (on-chain clock)' : '(unavailable)'}`,
122
+ ],
123
+ event: {
124
+ stage: 'anchor_post_complete',
125
+ anchor_seq: receipt.anchor_seq,
126
+ tx_hash: receipt.tx_hash,
127
+ ledger_index: receipt.ledger_index,
128
+ },
129
+ };
130
+ } catch (err) {
131
+ // XRPL_NOT_INSTALLED / XRPL_SEED_INVALID / ANCHOR_NOTHING_TO_POST etc.
132
+ const hint = err.hint ? ` (${err.hint})` : '';
133
+ return {
134
+ ok: false,
135
+ exitCode: 2,
136
+ lines: [`anchor-post failed: ${err.message}${hint}`],
137
+ event: { stage: 'error', failed_stage: 'anchor_post', code: err.code ?? null, message: err.message },
138
+ };
139
+ }
140
+ }
141
+
142
+ /**
143
+ * --anchor-verify: run the truncation check (always, offline) and — when a
144
+ * fetched tx is supplied — the on-chain anchor verification. OFFLINE-HONEST: with
145
+ * no tx it reports "XRPL NOT verified" rather than passing the on-chain leg.
146
+ *
147
+ * @param {string} repoRoot
148
+ * @param {object} [opts] - { tx, trustedAnchorAccounts, memoType }
149
+ * @returns {{ ok: boolean, exitCode: number, lines: string[], event: object }}
150
+ */
151
+ export function handleAnchorVerify(repoRoot, opts = {}) {
152
+ const trustedAnchorAccounts = resolveTrustedAccounts(opts.trustedAnchorAccounts);
153
+ const state = verifyAnchorState(repoRoot, {
154
+ tx: opts.tx,
155
+ trustedAnchorAccounts,
156
+ memoType: opts.memoType || ANCHOR_DEFAULTS.memoType,
157
+ });
158
+
159
+ const t = state.truncation;
160
+ const lines = [];
161
+ lines.push(`anchor-verify: ${state.ok ? 'PASS' : 'FAIL'}`);
162
+ lines.push(` anchors: ${t.anchor_count}`);
163
+ lines.push(` chain entries: ${t.chain_count} (head seq ${t.chain_head_seq ?? '(empty)'})`);
164
+ lines.push(` anchored through: seq ${t.anchored_through_seq ?? '(none)'} (≥ ${t.anchored_count} entries)`);
165
+ lines.push(` truncation check: ${t.ok ? 'OK' : 'FAILED'}`);
166
+ if (!t.ok) {
167
+ for (const f of t.failures) lines.push(` - [${f.kind}] ${f.reason}`);
168
+ }
169
+ lines.push(` XRPL on-chain: ${state.xrpl_verified ? 'VERIFIED' : 'NOT verified'}`);
170
+ lines.push(` ${state.note}`);
171
+
172
+ return {
173
+ ok: state.ok,
174
+ exitCode: state.ok ? 0 : 1,
175
+ lines,
176
+ event: {
177
+ stage: state.ok ? 'anchor_verify_complete' : 'error',
178
+ ...(state.ok ? {} : { failed_stage: 'anchor_verify' }),
179
+ truncation_ok: t.ok,
180
+ xrpl_verified: state.xrpl_verified,
181
+ anchored_through_seq: t.anchored_through_seq,
182
+ chain_head_seq: t.chain_head_seq,
183
+ },
184
+ };
185
+ }
@@ -0,0 +1,302 @@
1
+ /**
2
+ * compute-root — Merkle-root the integrity chain entries for the partition to be
3
+ * anchored next, and write an append-only anchor manifest.
4
+ *
5
+ * Adapted from repomesh's compute-root.mjs to testing-os's chain.jsonl. The
6
+ * source ledger here is `indexes/integrity/chain.jsonl` (one line per persisted
7
+ * record: { seq, run_id, repo, status, path, submission_digest, prev_digest,
8
+ * persisted_at }). The Merkle LEAVES are the entries' `submission_digest` values
9
+ * for the partition being anchored — already 64-char hex, exactly the shape the
10
+ * RFC-6962 tree wants.
11
+ *
12
+ * ── Cadence (default SINCE-LAST) ────────────────────────────────────────────
13
+ * Each anchor covers exactly the chain entries appended SINCE the previous
14
+ * anchor manifest (prev-linked), NOT per-entry. Run anchor-compute + anchor-post
15
+ * once per release wave — the unit of attested work — not on a wall-clock timer:
16
+ * - too FREQUENT (per-entry) floods the XRP Ledger with one-leaf anchors and
17
+ * wastes tx fees;
18
+ * - too RARE leaves a long window of only-locally-trusted (truncatable) entries.
19
+ * `--all` produces a single genesis snapshot over the whole chain instead.
20
+ *
21
+ * ── Truncation defense (the gap the offline chain cannot close) ──────────────
22
+ * The manifest records `leaf_count`, `seq_range: [from, to]`, and `head_digest`
23
+ * (the partition's last entry's submission_digest). Once that manifest's root is
24
+ * posted on-chain (post-anchor.js) and confirmed, a later chain that is SHORTER
25
+ * than the anchored (seq_range, leaf_count) is PROVABLY truncated:
26
+ * verify-anchor's verifyChainAgainstAnchors() asserts the live chain reaches at
27
+ * least the highest anchored `to` seq and carries at least the anchored count.
28
+ *
29
+ * This module is OFFLINE: it reads the chain and writes a manifest. It NEVER
30
+ * imports xrpl. Posting to the ledger is post-anchor.js's job.
31
+ */
32
+
33
+ import { existsSync, readFileSync, mkdirSync, readdirSync } from 'node:fs';
34
+ import { join } from 'node:path';
35
+ import { createHash } from 'node:crypto';
36
+
37
+ import { merkleRootForAlgo } from './merkle.js';
38
+ import { readChainManifest } from '../lib/chain-manifest.js';
39
+ import { canonicalize } from '../lib/integrity.js';
40
+ // Anchor manifests live in the shared backing store under indexes/integrity/, so
41
+ // they obey the same torn-write doctrine as records/ + chain.jsonl: write via
42
+ // temp+rename so a reader never sees a half-written manifest. Use the sibling
43
+ // ingest helper (not the findings copy — importing it would close the workspace
44
+ // dependency cycle, see CLAUDE.md).
45
+ import { atomicWriteFileSync } from '../lib/atomic-write.js';
46
+
47
+ /** Manifest format version — bump when the anchor manifest shape changes. */
48
+ export const MANIFEST_VERSION = 1;
49
+
50
+ /** Default Merkle algorithm for new anchors (second-preimage-safe RFC-6962). */
51
+ export const DEFAULT_ALGO = 'sha256-merkle-v2';
52
+
53
+ /** Soft cap: a since-last partition larger than this WARNS of a lapsed cadence. */
54
+ export const OVERSIZED_PARTITION_LEAVES = 5000;
55
+
56
+ /** Repo-root-relative directory holding the append-only anchor manifests. */
57
+ export const ANCHORS_DIR_REL = 'indexes/integrity/anchors';
58
+
59
+ /** Absolute path to the anchors directory for a given repo root. */
60
+ export function anchorsDir(repoRoot) {
61
+ return join(repoRoot, 'indexes', 'integrity', 'anchors');
62
+ }
63
+
64
+ function sha256hex(str) {
65
+ return createHash('sha256').update(str, 'utf-8').digest('hex');
66
+ }
67
+
68
+ /**
69
+ * Read every anchor manifest in seq order. Anchor files are named
70
+ * `anchor-<NNNN>.json` (zero-padded anchor_seq) so a lexical sort is a seq sort.
71
+ * Returns [] when none exist.
72
+ *
73
+ * @param {string} repoRoot
74
+ * @returns {Array<object>} Parsed anchor manifests, ascending by anchor_seq.
75
+ */
76
+ export function readAnchorManifests(repoRoot) {
77
+ const dir = anchorsDir(repoRoot);
78
+ if (!existsSync(dir)) return [];
79
+ const files = readdirSync(dir)
80
+ .filter((f) => /^anchor-\d+\.json$/.test(f))
81
+ .sort();
82
+ const manifests = files.map((f) => JSON.parse(readFileSync(join(dir, f), 'utf-8')));
83
+ manifests.sort((a, b) => a.anchor_seq - b.anchor_seq);
84
+ return manifests;
85
+ }
86
+
87
+ /**
88
+ * Return the highest-seq anchor manifest, or null when none exist.
89
+ *
90
+ * @param {string} repoRoot
91
+ * @returns {object|null}
92
+ */
93
+ export function readLastAnchor(repoRoot) {
94
+ const manifests = readAnchorManifests(repoRoot);
95
+ return manifests.length === 0 ? null : manifests[manifests.length - 1];
96
+ }
97
+
98
+ /**
99
+ * Select the chain entries this anchor will cover.
100
+ *
101
+ * since-last: entries whose `seq` is strictly greater than the highest seq the
102
+ * previous anchor covered (`prevAnchor.seq_range[1]`). Genesis (no prev) covers
103
+ * the whole chain.
104
+ * all: the whole chain (genesis snapshot), regardless of prior anchors.
105
+ *
106
+ * Pure: takes the already-read entries + prev anchor; no I/O. Exposed for tests.
107
+ *
108
+ * @param {Array<object>} entries - All chain entries (chain order).
109
+ * @param {object|null} prevAnchor - The previous anchor manifest, or null.
110
+ * @param {{ mode?: 'since-last'|'all' }} [opts]
111
+ * @returns {Array<object>} The partition entries.
112
+ */
113
+ export function selectPartition(entries, prevAnchor, opts = {}) {
114
+ const mode = opts.mode === 'all' ? 'all' : 'since-last';
115
+ if (mode === 'all' || !prevAnchor) return entries;
116
+ const coveredThrough = prevAnchor.seq_range?.[1];
117
+ if (typeof coveredThrough !== 'number') return entries;
118
+ return entries.filter((e) => e.seq > coveredThrough);
119
+ }
120
+
121
+ /**
122
+ * Build the anchor manifest object for a partition. Pure — no I/O, no network.
123
+ * The `root` + `manifest_hash` are computed here; the network/close-time fields
124
+ * are filled by post-anchor.js after a successful on-chain submission.
125
+ *
126
+ * @param {object} params
127
+ * @param {number} params.anchorSeq - 0,1,2,… ordinal of this anchor.
128
+ * @param {Array<object>} params.partition - The chain entries this anchor covers.
129
+ * @param {object|null} params.prevAnchor - The previous anchor manifest, or null.
130
+ * @param {string} [params.algo] - Merkle algo (default v2).
131
+ * @param {string} [params.network] - 'testnet' | 'mainnet' (default testnet).
132
+ * @param {string} [params.computedAt] - ISO timestamp (default now).
133
+ * @returns {object} The anchor manifest (without on-chain post fields).
134
+ */
135
+ export function buildAnchorManifest({
136
+ anchorSeq,
137
+ partition,
138
+ prevAnchor,
139
+ algo = DEFAULT_ALGO,
140
+ network = 'testnet',
141
+ computedAt = new Date().toISOString(),
142
+ }) {
143
+ if (!Array.isArray(partition) || partition.length === 0) {
144
+ throw new Error('buildAnchorManifest: partition is empty — nothing to anchor');
145
+ }
146
+ const leaves = partition.map((e) => e.submission_digest);
147
+ for (const [i, leaf] of leaves.entries()) {
148
+ if (typeof leaf !== 'string' || !/^[0-9a-fA-F]{64}$/.test(leaf)) {
149
+ throw new Error(`buildAnchorManifest: chain entry[${i}] has a malformed submission_digest: ${leaf}`);
150
+ }
151
+ }
152
+ const root = merkleRootForAlgo(leaves, algo);
153
+ const seqFrom = partition[0].seq;
154
+ const seqTo = partition[partition.length - 1].seq;
155
+ const headDigest = partition[partition.length - 1].submission_digest;
156
+
157
+ // The manifest base is canonicalized + hashed so the on-chain memo can bind to
158
+ // it. Field order is irrelevant (canonicalize sorts keys); the hash is stable.
159
+ const base = {
160
+ anchor_seq: anchorSeq,
161
+ manifest_version: MANIFEST_VERSION,
162
+ algo,
163
+ root,
164
+ leaf_count: leaves.length,
165
+ seq_range: [seqFrom, seqTo],
166
+ head_digest: headDigest,
167
+ prev_anchor_root: prevAnchor ? prevAnchor.root : null,
168
+ network,
169
+ };
170
+ const manifestHash = sha256hex(canonicalize(base));
171
+ return { ...base, manifest_hash: manifestHash, computed_at: computedAt };
172
+ }
173
+
174
+ /**
175
+ * The append-only file path for an anchor of a given seq.
176
+ *
177
+ * @param {string} repoRoot
178
+ * @param {number} anchorSeq
179
+ * @returns {string}
180
+ */
181
+ export function anchorManifestPath(repoRoot, anchorSeq) {
182
+ const padded = String(anchorSeq).padStart(4, '0');
183
+ return join(anchorsDir(repoRoot), `anchor-${padded}.json`);
184
+ }
185
+
186
+ /**
187
+ * Compute the next anchor for a repo root and write its append-only manifest.
188
+ *
189
+ * Reads the chain, selects the partition (since-last by default), builds the
190
+ * manifest (Merkle root + leaf_count + seq_range + head_digest, prev-linked to
191
+ * the prior anchor's root), and writes it to
192
+ * `indexes/integrity/anchors/anchor-<NNNN>.json`.
193
+ *
194
+ * Append-only: write-if-missing, fail-if-exists-with-different-content. An empty
195
+ * partition (nothing new since the last anchor) returns `{ empty: true }` and
196
+ * writes nothing.
197
+ *
198
+ * OFFLINE — never imports xrpl, never touches the network.
199
+ *
200
+ * @param {string} repoRoot - Absolute path to the testing-os repo root.
201
+ * @param {object} [opts]
202
+ * @param {'since-last'|'all'} [opts.mode='since-last']
203
+ * @param {string} [opts.algo='sha256-merkle-v2']
204
+ * @param {string} [opts.network='testnet']
205
+ * @returns {{ empty: boolean, manifest?: object, path?: string, written?: boolean, leaf_count?: number }}
206
+ */
207
+ export function computeAnchor(repoRoot, opts = {}) {
208
+ const mode = opts.mode === 'all' ? 'all' : 'since-last';
209
+ const algo = opts.algo || DEFAULT_ALGO;
210
+ if (algo !== 'sha256-merkle-v1' && algo !== 'sha256-merkle-v2') {
211
+ const e = new Error(`Unknown anchor algo "${algo}" (expected sha256-merkle-v1 or sha256-merkle-v2)`);
212
+ e.code = 'ANCHOR_BAD_ALGO';
213
+ throw e;
214
+ }
215
+ const network = opts.network || 'testnet';
216
+
217
+ const entries = readChainManifest(repoRoot);
218
+ if (entries.length === 0) {
219
+ return { empty: true, reason: 'chain is empty — nothing to anchor' };
220
+ }
221
+
222
+ const existingAnchors = readAnchorManifests(repoRoot);
223
+ const prevAnchor = existingAnchors.length === 0 ? null : existingAnchors[existingAnchors.length - 1];
224
+ const anchorSeq = existingAnchors.length;
225
+
226
+ const partition = selectPartition(entries, prevAnchor, { mode });
227
+ if (partition.length === 0) {
228
+ return { empty: true, reason: 'no chain entries since the last anchor — nothing new to anchor' };
229
+ }
230
+
231
+ const warnings = [];
232
+ if (mode === 'since-last' && partition.length > OVERSIZED_PARTITION_LEAVES) {
233
+ warnings.push(
234
+ `since-last partition has ${partition.length} leaves (> ${OVERSIZED_PARTITION_LEAVES}); ` +
235
+ 'the anchoring cadence has likely lapsed — anchor now, then anchor once per release wave.'
236
+ );
237
+ }
238
+
239
+ const manifest = buildAnchorManifest({ anchorSeq, partition, prevAnchor, algo, network });
240
+
241
+ const path = anchorManifestPath(repoRoot, anchorSeq);
242
+ mkdirSync(anchorsDir(repoRoot), { recursive: true });
243
+ const json = JSON.stringify(manifest, null, 2) + '\n';
244
+
245
+ let written = false;
246
+ if (existsSync(path)) {
247
+ // Append-only: an existing anchor at this seq must be byte-identical (modulo
248
+ // the volatile computed_at field, which we do not let block a re-run).
249
+ const existing = JSON.parse(readFileSync(path, 'utf-8'));
250
+ const stableEqual =
251
+ existing.root === manifest.root &&
252
+ existing.manifest_hash === manifest.manifest_hash &&
253
+ existing.leaf_count === manifest.leaf_count;
254
+ if (!stableEqual) {
255
+ const e = new Error(
256
+ `anchor manifest conflict at ${path}: an anchor for seq ${anchorSeq} already exists ` +
257
+ 'with a different root/hash. Anchor manifests are append-only.'
258
+ );
259
+ e.code = 'ANCHOR_MANIFEST_CONFLICT';
260
+ throw e;
261
+ }
262
+ } else {
263
+ atomicWriteFileSync(path, json);
264
+ written = true;
265
+ }
266
+
267
+ return { empty: false, manifest, path, written, leaf_count: manifest.leaf_count, warnings };
268
+ }
269
+
270
+ /**
271
+ * Record the on-chain facts back into an anchor manifest file after a successful
272
+ * post. This is the production `recordPost` callback post-anchor.js invokes.
273
+ *
274
+ * Patches an `on_chain` block into the manifest at the given seq WITHOUT touching
275
+ * the root / leaf_count / seq_range / head_digest (those are the contract the
276
+ * memo committed to — they must never change after posting). Writing the on-chain
277
+ * receipt is the one mutation an anchor manifest accepts: it ADDS the proof of
278
+ * where the root landed, it does not alter what was committed.
279
+ *
280
+ * @param {string} repoRoot
281
+ * @param {number} anchorSeq
282
+ * @param {object} facts - { tx_hash, ledger_index, close_time_iso, wallet_address, network }
283
+ */
284
+ export function recordAnchorPost(repoRoot, anchorSeq, facts) {
285
+ const path = anchorManifestPath(repoRoot, anchorSeq);
286
+ if (!existsSync(path)) {
287
+ const e = new Error(`cannot record post: no anchor manifest at ${path} (run anchor-compute first)`);
288
+ e.code = 'ANCHOR_MANIFEST_MISSING';
289
+ throw e;
290
+ }
291
+ const manifest = JSON.parse(readFileSync(path, 'utf-8'));
292
+ manifest.on_chain = {
293
+ tx_hash: facts.tx_hash ?? null,
294
+ ledger_index: facts.ledger_index ?? null,
295
+ close_time_iso: facts.close_time_iso ?? null,
296
+ wallet_address: facts.wallet_address ?? null,
297
+ network: facts.network ?? manifest.network,
298
+ posted_at: new Date().toISOString(),
299
+ };
300
+ atomicWriteFileSync(path, JSON.stringify(manifest, null, 2) + '\n');
301
+ return manifest;
302
+ }
@@ -0,0 +1,44 @@
1
+ /**
2
+ * Anchor configuration — network defaults and the trusted-anchor-account
3
+ * allowlist (the authorization root for verification).
4
+ *
5
+ * Any funded XRPL wallet can post a memo, so the trust does not come from the
6
+ * memo — it comes from REQUIRING the posting account to be on an allowlist the
7
+ * verifier controls. BUNDLED_TRUSTED_ACCOUNTS is pinned here so that even if a
8
+ * future config file is fetched/overridden remotely, the account check can never
9
+ * be silently turned off: the resolver UNIONs the bundled list with any
10
+ * operator-supplied accounts (a remote config can ADD, never DROP).
11
+ *
12
+ * The allowlist starts EMPTY by design: testing-os has not minted a production
13
+ * anchor wallet yet. An operator adds their funded wallet's classic address here
14
+ * (and/or passes it via the CLI) when they stand up real anchoring. Verification
15
+ * with an empty allowlist correctly FAILS the on-chain leg — fail closed, never
16
+ * trust an unlisted account.
17
+ */
18
+
19
+ /** Bundled allowlist — pinned, can never be dropped by a remote override. */
20
+ export const BUNDLED_TRUSTED_ACCOUNTS = Object.freeze([]);
21
+
22
+ /** Default network and endpoints. testnet is the default; mainnet is documented. */
23
+ export const ANCHOR_DEFAULTS = Object.freeze({
24
+ network: 'testnet',
25
+ wsUrl: {
26
+ testnet: 'wss://s.altnet.rippletest.net:51233',
27
+ mainnet: 'wss://xrplcluster.com',
28
+ },
29
+ memoType: 'testing-os-anchor-v1',
30
+ });
31
+
32
+ /**
33
+ * Resolve the trusted anchor accounts: the bundled fallback UNION any
34
+ * operator-supplied accounts. The bundled list can never be removed, so an
35
+ * operator-supplied list can only ADD authorized accounts, never disable the
36
+ * check.
37
+ *
38
+ * @param {string[]} [operatorAccounts] - Accounts supplied via CLI/config.
39
+ * @returns {string[]} The de-duplicated union.
40
+ */
41
+ export function resolveTrustedAccounts(operatorAccounts = []) {
42
+ const extra = Array.isArray(operatorAccounts) ? operatorAccounts : [];
43
+ return [...new Set([...BUNDLED_TRUSTED_ACCOUNTS, ...extra])];
44
+ }