kxco-pq-audit 1.2.2 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,103 @@
1
+ # Changelog
2
+
3
+ ## 1.3.0
4
+
5
+ ### Added
6
+
7
+ `sealed: true` signs once per run instead of once per entry.
8
+
9
+ Every entry is still hash-chained, which is what detects an edit, a removal, a
10
+ reorder or an insertion. The per-entry signature only bound one entry to the
11
+ key, and it was the entire cost of the log. Measured here on ML-DSA-65 over
12
+ 10,000 entries:
13
+
14
+ | | entries/s | bytes/entry | verify |
15
+ |---|---|---|---|
16
+ | per entry | 129 | 4,794 | 20.1 s |
17
+ | per run | 46,544 | 367 | 0.11 s |
18
+
19
+ Of those 4,794 bytes, 4,412 were the signature. Any workload with real entry
20
+ volume, agent tool calls being the case this was built for, cannot afford one
21
+ signature per entry.
22
+
23
+ `seal()` returns the run it signed, or null when nothing is unsealed. `seals()`
24
+ returns them all. Seals chain to each other by `prevRoot`, so removing an entire
25
+ seal breaks the one after it, the same way removing an entry breaks the next
26
+ entry's `prevHash`. `FileAuditLog` writes them to `<path>.seals`.
27
+
28
+ `verify()` on a sealed log reports `sealedThrough` and `unsealed`. Entries
29
+ appended since the last seal are chained and unsigned, and that window is
30
+ reported rather than counted as proven.
31
+
32
+ Two things it costs: a single entry can no longer be verified on its own, only
33
+ as part of its run, and the unsealed window is real until `seal()` runs.
34
+
35
+ Default is unchanged. A log built without the option behaves exactly as before,
36
+ signature per entry included.
37
+
38
+ `stream()` yields entries one at a time, and `unsealedCount()` reports the
39
+ window without replaying the log.
40
+
41
+ ### Fixed
42
+
43
+ **Concurrent appends forked the chain.** Every `append()` read the log to find
44
+ the next seq, so two in flight together both read the same tail and both claimed
45
+ it. Fifty parallel appends produced fifty entries all numbered `seq 0`, and the
46
+ resulting log fails its own `verify()`. Writes are now serialised internally, so
47
+ the same fifty produce seq 0 to 49 and verify clean. Any log written
48
+ concurrently under 1.2.3 or earlier should be checked with `verify()`.
49
+
50
+ **Appending was quadratic.** `append()` loaded every entry to work out the next
51
+ seq and prevHash, and `FileAuditLog` re-read and re-parsed the whole file each
52
+ time. A log now loads once and keeps only the tail, so cost per entry no longer
53
+ depends on length.
54
+
55
+ | entries | before | after |
56
+ |---|---|---|
57
+ | 500 | 1,635 us/entry | 66 us/entry |
58
+ | 2,000 | 2,279 us/entry | 66 us/entry |
59
+ | 4,000 | 3,262 us/entry | 66 us/entry |
60
+ | 50,000 | not measured, still climbing | 66 us/entry |
61
+
62
+ `FileAuditLog` also holds one write handle rather than reopening the file per
63
+ entry, which is ~46us a write against ~421us. It is released after 2 seconds
64
+ idle, so forgetting `close()` costs nothing, and `close()` is there for a
65
+ deterministic hand-back.
66
+
67
+ **`verify()` loaded the whole log to check it.** It now streams, so memory is
68
+ bounded by one entry and the seal list. 50,000 entries verify in 729 ms.
69
+
70
+ ### Added, smaller
71
+
72
+ `FileAuditLog` refuses to append to a file that changed size underneath it,
73
+ rather than forking the chain and failing much later at `verify()`. One file has
74
+ one writer; `assertSoleWriter()` checks it on demand and runs on every seal and
75
+ close.
76
+
77
+ A line that will not parse is now named with its line number and told apart from
78
+ a torn final line, instead of surfacing as a bare `SyntaxError`.
79
+
80
+ ## 1.2.3
81
+
82
+ ### Corrected
83
+
84
+ The README claimed `@noble/post-quantum` was audited by Cure53 in 2024.
85
+ It is maintainer-audited (v0.6.1, April 2026), not Cure53-audited.
86
+
87
+ The other Noble packages were audited separately and at different dates, and
88
+ none of those engagements reached the post-quantum package:
89
+
90
+ | Package | Audited by |
91
+ |---|---|
92
+ | `@noble/post-quantum` | maintainer-audited |
93
+ | `@noble/hashes` | Cure53, Jan 2022, v1.0.0 |
94
+ | `@noble/curves` | Trail of Bits Feb 2023; Kudelski Sep 2023; Cure53 Sep 2024 |
95
+ | `@noble/ciphers` | Cure53, Sep 2024, v1.0.0 |
96
+
97
+ Dates from `kxco-post-quantum/audit/dependency-review.json`, which is generated
98
+ by `audit/run-audit.mjs` rather than written by hand.
99
+
100
+ Documentation only. No code changed and no behaviour changed.
101
+
102
+ `.socket.yml` said `@noble/hashes` was audited by Cure53 in 2024. The audit
103
+ was real but the date was wrong: January 2022, at v1.0.0.
package/README.md CHANGED
@@ -28,7 +28,7 @@ Every release of this package is checkable without asking us for anything.
28
28
  Every GitHub Action is pinned by 40-character commit SHA.
29
29
  - **Conformance underneath.** The cryptography comes from
30
30
  [`kxco-post-quantum`](https://www.npmjs.com/package/kxco-post-quantum), which
31
- is run against **2,103 NIST ACVP vectors (0 failed)** and a **225-check
31
+ is run against **2,103 NIST ACVP vectors: 1,793 passed, 0 failed, 310 skipped** and a **225-check
32
32
  cross-implementation interoperability matrix** against liboqs, Bouncy Castle
33
33
  and two pure-Python implementations, in both directions and with negative
34
34
  controls. Its published tarball also rebuilds bit-for-bit from its own tag,
@@ -78,7 +78,7 @@ await log.append('key.rotate', { keyId: 'signing-key-v2', alg: 'ml-dsa-65' })
78
78
  const result = await log.verify(keypair.publicKey)
79
79
  // { valid: true, count: 3 }
80
80
 
81
- const entries = await log.entries()
81
+ const entries = await log.export()
82
82
  // array of signed entry objects
83
83
  ```
84
84
 
@@ -96,6 +96,7 @@ In-memory log. Entries are lost when the process exits.
96
96
  | `chain` | `KxcoChain` | No | `null` | Chain instance for checkpoint anchoring |
97
97
  | `checkpointEvery` | `number` | No | `100` | Anchor a checkpoint every N entries |
98
98
  | `institutionKid` | `string` | No | `null` | Identifier included in checkpoint metadata |
99
+ | `sealed` | `boolean` | No | `false` | Sign once per run instead of once per entry. See [Sealed logs](#sealed-logs) |
99
100
 
100
101
  ### `new FileAuditLog(options)`
101
102
 
@@ -104,6 +105,15 @@ Append-only NDJSON file backend. Survives process restarts. Accepts all the same
104
105
  | Option | Type | Required | Description |
105
106
  |---|---|---|---|
106
107
  | `path` | `string` | Yes | Path to the `.ndjson` file (created if absent) |
108
+ | `idleReleaseMs` | `number` | No | Release the write handle after this long without a write (default `2000`) |
109
+
110
+ Entries are read as a stream and never all at once, and writes go through one handle held open while the log is busy. Both mean cost per entry does not grow with the file.
111
+
112
+ **One file has one writer.** A signed chain cannot have two: both would build on the same tail and the second entry would take the first one's place. Every seal and every `close()` checks the file is the size this instance left it, and throws if it is not.
113
+
114
+ ### `log.close()`
115
+
116
+ `FileAuditLog` only. Releases the write handle. Not required, since the handle is released after `idleReleaseMs` without a write, but call it when you want the descriptor back at a known point. Throws if another writer has touched the file.
107
117
 
108
118
  ### `log.append(operation, metadata)`
109
119
 
@@ -115,6 +125,8 @@ Appends a signed, hash-chained entry.
115
125
 
116
126
  If chain anchoring is configured and the entry count is a multiple of `checkpointEvery`, a fire-and-forget checkpoint is sent to the relay. The `append` call resolves immediately — it does not wait for the chain.
117
127
 
128
+ Appending needs the previous entry's hash, not the log, so it costs the same on ten entries and on ten million. Concurrent calls are serialised internally: two appends in flight together cannot take the same `seq`.
129
+
118
130
  ### `log.verify(publicKey)`
119
131
 
120
132
  Replays the entire log from entry 0. For each entry, checks:
@@ -122,11 +134,37 @@ Replays the entire log from entry 0. For each entry, checks:
122
134
  1. `prevHash` matches the SHA-256 of the previous entry (including its signature)
123
135
  2. The ML-DSA-65 signature is valid over the canonical signing bytes
124
136
 
125
- Returns `{ valid: true, count }` or `{ valid: false, error }` describing the first failure.
137
+ Returns `{ valid: true, count }` or `{ valid: false, error }` describing the first failure. On a sealed log it checks the chain and every seal instead, and adds `sealedThrough` and `unsealed`.
138
+
139
+ It streams, so memory is bounded by one entry rather than by the log: 50,000 entries verify in 729 ms without holding them.
140
+
141
+ ### `log.seal()`
142
+
143
+ Sealed logs only. Signs everything appended since the last seal, as one run, and returns the seal. Returns `null` when nothing is unsealed, and throws on a log built without `sealed: true`.
144
+
145
+ If chain anchoring is configured, the run's root is anchored fire-and-forget. A sealed log anchors when it seals rather than every `checkpointEvery` entries.
146
+
147
+ ### `log.seals()`
148
+
149
+ Sealed logs only. Returns every seal, oldest first.
150
+
151
+ ### `log.export()`
126
152
 
127
- ### `log.entries()`
153
+ Returns all entries as an array, in seq order. Holds the whole log in memory by definition; prefer `stream()` on a large one.
128
154
 
129
- Returns all entries as an array. Equivalent to `log.export()`.
155
+ ### `log.stream()`
156
+
157
+ Yields entries one at a time, in seq order. Memory stays bounded by a single entry however long the log is.
158
+
159
+ ```js
160
+ for await (const entry of log.stream()) {
161
+ if (entry.operation === 'tool_call') console.log(entry.seq, entry.metadata.tool)
162
+ }
163
+ ```
164
+
165
+ ### `log.unsealedCount()`
166
+
167
+ Sealed logs only. Entries appended since the last seal, without replaying the log. Always `0` on a classic log.
130
168
 
131
169
  ## Entry format
132
170
 
@@ -143,6 +181,45 @@ Returns all entries as an array. Equivalent to `log.export()`.
143
181
 
144
182
  `prevHash` is the SHA-256 of the complete previous entry (signature included). The first entry always has `prevHash: null`. The signing message covers every field except `signature` itself.
145
183
 
184
+ ## Sealed logs
185
+
186
+ By default every entry carries its own ML-DSA-65 signature. That signature is also the entire cost of the log. Measured here over 10,000 entries:
187
+
188
+ | | entries/s | bytes/entry | 10k run | verify |
189
+ |---|---|---|---|---|
190
+ | signature per entry | 129 | 4,794 | 45.7 MB | 20.1 s |
191
+ | signature per run | 46,544 | 367 | 3.5 MB | 0.11 s |
192
+
193
+ Of those 4,794 bytes, 4,412 are the signature. Anything with real entry volume cannot afford one per entry.
194
+
195
+ `sealed: true` keeps the hash chain on every entry and moves the signature to the run:
196
+
197
+ ```js
198
+ const log = new AuditLog({ keypair, sealed: true })
199
+
200
+ for (const call of agentToolCalls) {
201
+ await log.append('tool_call', call) // chained, not signed
202
+ }
203
+
204
+ const seal = await log.seal() // one signature for the whole run
205
+ // { fromSeq: 0, toSeq: 9999, entryCount: 10000, prevRoot, rootHash, timestamp, signature }
206
+ ```
207
+
208
+ The chain is what detects tampering, and it is untouched. An entry edited, removed, reordered or inserted after sealing fails to reproduce the run's `rootHash`. Seals chain to one another by `prevRoot`, so removing a whole seal breaks the seal that follows it.
209
+
210
+ `verify()` on a sealed log reports where the signed record stops:
211
+
212
+ ```js
213
+ await log.verify(publicKey)
214
+ // { valid: true, count: 10002, sealedThrough: 9999, unsealed: 2 }
215
+ ```
216
+
217
+ Entries appended since the last seal are chained and unsigned. `unsealed` is that window, reported rather than counted as proven. Call `seal()` to close it.
218
+
219
+ What sealing costs you: a single entry can no longer be verified on its own, only as part of its run.
220
+
221
+ `FileAuditLog` writes seals to `<path>.seals`, so a reader that knows only about entries is unaffected.
222
+
146
223
  ## Chain anchoring
147
224
 
148
225
  When a `chain` is passed and `checkpointEvery` is set, every Nth entry triggers a call to `chain.anchorAuditRoot({ rootHash, entryCount })` via the KXCO relay.
@@ -151,12 +228,18 @@ The anchor is fire-and-forget: `append` does not await it, so chain latency neve
151
228
 
152
229
  The checkpoint provides an on-chain timestamp proving that at least N entries existed at a specific block height. This supplements the local NDJSON file for long-term tamper evidence, particularly where the log operator and the verifier are separate parties.
153
230
 
154
- ## What this does NOT do
231
+ ## Where this fits
232
+
233
+ An append-only, ML-DSA-65-signed, hash-chained record. **Entries cannot be
234
+ edited or deleted without `verify()` failing and naming the entry** — that is
235
+ the guarantee, and the reason to use it.
155
236
 
156
- - **Not a database.** Entries cannot be queried by field, filtered, or indexed. Read the file line by line.
157
- - **Not queryable.** There is no search API. If you need search, index entries into a database alongside the NDJSON file.
158
- - **Append-only.** Entries cannot be edited or deleted without breaking `verify()`. This is intentional.
159
- - **Not a transport.** This package writes and verifies. It does not expose HTTP endpoints or stream entries to external systems.
237
+ It writes and verifies a file. Reading is line by line, so index entries into
238
+ your own database if you need search, and checkpoint to Armature L1 when a
239
+ regulator needs an anchor they can confirm on-chain.
240
+
241
+ - [`kxco-pq-chain`](https://www.npmjs.com/package/kxco-pq-chain) for the checkpoint anchor
242
+ - [`kxco-pq-attest`](https://www.npmjs.com/package/kxco-pq-attest) for signed envelopes that travel on their own
160
243
 
161
244
  ## Part of the KXCO stack
162
245
 
@@ -172,7 +255,19 @@ The checkpoint provides an on-chain timestamp proving that at least N entries ex
172
255
 
173
256
  ## Security
174
257
 
175
- Entry signing uses [Noble post-quantum](https://github.com/paulmillr/noble-post-quantum) ML-DSA-65 (NIST FIPS 204) and [Noble hashes](https://github.com/paulmillr/noble-hashes) SHA-256 — independently audited by Cure53 (2024). The hash chain means a compromised or deleted entry cannot be hidden: any gap breaks verification of every subsequent entry.
258
+ **ML-DSA-65** (NIST FIPS 204) via [`kxco-post-quantum`](https://www.npmjs.com/package/kxco-post-quantum), running on the OpenSSL 3.5 primitives where the runtime provides them. No custom cryptography.
259
+
260
+ Evidenced, and reproducible on your own machine:
261
+
262
+ - **2,103 NIST ACVP vectors** across FIPS 203, 204 and 205, pinned by digest: 1,793 passed, 0 failed, 310 skipped, where each skip is the library refusing a pre-hash weaker than the parameter set
263
+ - **225 interoperability checks passed, 0 failed, 42 not applicable** against OpenSSL 3.5, liboqs, Bouncy Castle and dilithium-py/kyber-py, in both directions
264
+ - **SLSA provenance** on every published release — verify with `npm audit signatures`
265
+ - **CycloneDX SBOM** published with each release
266
+ - `npm run evidence` regenerates the whole bundle from source
267
+
268
+ Dependency audit history is recorded in [AUDIT.md](https://github.com/KnightsbridgeAIQ/kxco-post-quantum/blob/main/AUDIT.md).
269
+
270
+ The hash chain means a compromised or deleted entry cannot be hidden: any gap breaks verification of every subsequent entry.
176
271
 
177
272
  To report a vulnerability, open a [private security advisory](https://github.com/KnightsbridgeAIQ/kxco-pq-audit/security/advisories/new) or email **security@kxco.ai**.
178
273
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "kxco-pq-audit",
3
- "version": "1.2.2",
4
- "description": "Tamper-evident post-quantum audit log for regulated institutions. ML-DSA-65-signed, SHA-256 hash-chained entries. Any tampering breaks the chain. Optional Armature L1 checkpoint anchoring via the KXCO relay.",
3
+ "version": "1.3.0",
4
+ "description": "Tamper-evident post-quantum audit log for regulated institutions. ML-DSA-65-signed, SHA-256 hash-chained entries, so any tampering breaks the chain and shows exactly where. Checkpoint to Armature L1 for an anchor a regulator can confirm on-chain.",
5
5
  "keywords": [
6
6
  "post-quantum",
7
7
  "pqc",
@@ -50,14 +50,16 @@
50
50
  }
51
51
  },
52
52
  "files": [
53
- "src",
54
- "LICENSE"
53
+ "CHANGELOG.md",
54
+ "LICENSE",
55
+ "README.md",
56
+ "src"
55
57
  ],
56
58
  "engines": {
57
59
  "node": ">=20.19"
58
60
  },
59
61
  "dependencies": {
60
- "@noble/hashes": "2.3.0",
62
+ "@noble/hashes": "2.4.0",
61
63
  "kxco-post-quantum": "^1.3.0"
62
64
  },
63
65
  "scripts": {
package/src/audit-log.js CHANGED
@@ -17,14 +17,53 @@ function signingBytes(seq, timestamp, operation, metadata, prevHash) {
17
17
  )
18
18
  }
19
19
 
20
+ function sealBytes(fromSeq, toSeq, prevRoot, rootHash, timestamp) {
21
+ return enc.encode(
22
+ `kxco-audit-seal-v1\n${fromSeq}\n${toSeq}\n${prevRoot}\n${rootHash}\n${timestamp}`
23
+ )
24
+ }
25
+
26
+ /** The first seal has no predecessor; this stands in for one so the seals chain too. */
27
+ const GENESIS_ROOT = '0'.repeat(64)
28
+
29
+ /**
30
+ * A run's root: the previous seal's root followed by every entry hash in seq
31
+ * order. Chaining the roots means removing a whole seal breaks the next one,
32
+ * the same way removing an entry breaks the next entry's prevHash.
33
+ */
34
+ function rootOf(prevRoot, entryHashes) {
35
+ return b64url(sha256(enc.encode(prevRoot + entryHashes.join(''))))
36
+ }
37
+
38
+ /**
39
+ * Tamper-evident append-only log.
40
+ *
41
+ * Appending costs the same whether the log holds ten entries or ten million:
42
+ * a new entry needs the previous entry's hash and the next seq, never the log.
43
+ * Verification streams, so it is bounded by one entry rather than by the log.
44
+ */
20
45
  export class AuditLog {
21
46
  #keypair
22
47
  #entries = []
48
+ #sealsList = []
23
49
  #chain
24
50
  #checkpointEvery
25
51
  #institutionKid
52
+ #sealed
26
53
 
27
- constructor({ keypair, chain, checkpointEvery = 100, institutionKid } = {}) {
54
+ /** { seq, hash } of the last entry, or null for an empty log. Loaded once. */
55
+ #tail = null
56
+ /** Sealed logs only: entries since the last seal, which is what seal() signs. */
57
+ #pending = []
58
+ #ready = null
59
+ /**
60
+ * Writes run one at a time. Two appends in flight together would both read
61
+ * the same tail and mint the same seq, which forks the chain at the point it
62
+ * is supposed to be strongest.
63
+ */
64
+ #queue = Promise.resolve()
65
+
66
+ constructor({ keypair, chain, checkpointEvery = 100, institutionKid, sealed = false } = {}) {
28
67
  if (!keypair?.secretKey || !keypair?.publicKey) {
29
68
  throw new KxcoPqAuditError('keypair with secretKey and publicKey is required')
30
69
  }
@@ -32,24 +71,72 @@ export class AuditLog {
32
71
  this.#chain = chain ?? null
33
72
  this.#checkpointEvery = checkpointEvery
34
73
  this.#institutionKid = institutionKid ?? null
74
+ this.#sealed = Boolean(sealed)
75
+ }
76
+
77
+ /** True when this log signs once per sealed run rather than once per entry. */
78
+ get sealed() { return this.#sealed }
79
+
80
+ /**
81
+ * One pass over whatever is already stored, to find the tail and, on a sealed
82
+ * log, the entries the next seal will cover. Everything after this is O(1).
83
+ */
84
+ #load() {
85
+ if (this.#ready) return this.#ready
86
+ this.#ready = (async () => {
87
+ const seals = await this._seals()
88
+ const sealedThrough = seals.length === 0 ? -1 : seals[seals.length - 1].toSeq
89
+ let last = null
90
+ const pending = []
91
+ for await (const entry of this._iterate()) {
92
+ last = entry
93
+ if (this.#sealed && entry.seq > sealedThrough) pending.push(entry)
94
+ }
95
+ this.#tail = last === null ? null : { seq: last.seq, hash: hashEntry(last) }
96
+ this.#pending = pending
97
+ })()
98
+ return this.#ready
99
+ }
100
+
101
+ /** Run a write with no other write in flight. A failure must not poison the queue. */
102
+ #serial(fn) {
103
+ const run = this.#queue.then(fn, fn)
104
+ this.#queue = run.then(() => {}, () => {})
105
+ return run
35
106
  }
36
107
 
37
108
  async append(operation, metadata = {}) {
109
+ // Validate before queueing, so a bad call fails now rather than behind
110
+ // however much work is already in flight.
38
111
  if (typeof operation !== 'string' || !operation) {
39
112
  throw new KxcoPqAuditError('operation must be a non-empty string')
40
113
  }
41
- const all = await this._entries()
42
- const seq = all.length
114
+ return this.#serial(() => this.#appendOne(operation, metadata))
115
+ }
116
+
117
+ async #appendOne(operation, metadata) {
118
+ await this.#load()
119
+
120
+ const seq = this.#tail === null ? 0 : this.#tail.seq + 1
121
+ const prev = this.#tail === null ? null : this.#tail.hash
43
122
  const ts = new Date().toISOString()
44
- const prev = seq === 0 ? null : hashEntry(all[seq - 1])
45
- const msg = signingBytes(seq, ts, operation, metadata, prev)
46
- const sig = Buffer.from(mlDsa.sign(new Uint8Array(this.#keypair.secretKey), msg), 'hex')
47
123
 
48
- const entry = { seq, timestamp: ts, operation, metadata, prevHash: prev, signature: b64url(sig) }
124
+ // A sealed log chains every entry and signs none of them. The signature that
125
+ // binds the run to the key is produced once, by seal(), because signing every
126
+ // entry costs ~2ms and ~4.4KB and does not survive agent-scale volume.
127
+ const entry = { seq, timestamp: ts, operation, metadata, prevHash: prev }
128
+ if (!this.#sealed) {
129
+ const msg = signingBytes(seq, ts, operation, metadata, prev)
130
+ entry.signature = b64url(Buffer.from(mlDsa.sign(new Uint8Array(this.#keypair.secretKey), msg), 'hex'))
131
+ }
132
+
49
133
  await this._store(entry)
134
+ this.#tail = { seq, hash: hashEntry(entry) }
135
+ if (this.#sealed) this.#pending.push(entry)
50
136
 
137
+ // A sealed log anchors when it seals, not every N entries.
51
138
  const entryCount = seq + 1
52
- if (this.#chain && entryCount % this.#checkpointEvery === 0) {
139
+ if (!this.#sealed && this.#chain && entryCount % this.#checkpointEvery === 0) {
53
140
  const rootHash = Buffer.from(sha256(enc.encode(JSON.stringify(entry)))).toString('hex')
54
141
  this.#chain.anchorAuditRoot({ rootHash, entryCount }).catch((err) => {
55
142
  console.warn(`[kxco-pq-audit] chain checkpoint failed (entry ${entryCount}): ${err.message}`)
@@ -59,29 +146,183 @@ export class AuditLog {
59
146
  return entry
60
147
  }
61
148
 
149
+ /**
150
+ * Sign every entry written since the last seal, as one run.
151
+ *
152
+ * Returns the seal, or null when there is nothing unsealed. Safe to call
153
+ * repeatedly. Entries appended after a seal are chained but carry no
154
+ * signature until the next one, which is the honest cost of not signing
155
+ * inline: verify() always reports how many are in that window.
156
+ */
157
+ async seal() {
158
+ if (!this.#sealed) throw new KxcoPqAuditError('seal() requires sealed: true')
159
+ // Sealing shares the write queue with append, so a run can never be sealed
160
+ // half way through an entry being written into it.
161
+ return this.#serial(() => this.#sealPending())
162
+ }
163
+
164
+ async #sealPending() {
165
+ await this.#load()
166
+ if (this.#pending.length === 0) return null
167
+
168
+ const seals = await this._seals()
169
+ const last = seals.length === 0 ? null : seals[seals.length - 1]
170
+ const run = this.#pending
171
+
172
+ const prevRoot = last === null ? GENESIS_ROOT : last.rootHash
173
+ const rootHash = rootOf(prevRoot, run.map(hashEntry))
174
+ const fromSeq = run[0].seq
175
+ const toSeq = run[run.length - 1].seq
176
+ const timestamp = new Date().toISOString()
177
+ const msg = sealBytes(fromSeq, toSeq, prevRoot, rootHash, timestamp)
178
+ const signature = b64url(Buffer.from(mlDsa.sign(new Uint8Array(this.#keypair.secretKey), msg), 'hex'))
179
+
180
+ const seal = {
181
+ fromSeq,
182
+ toSeq,
183
+ entryCount: run.length,
184
+ prevRoot,
185
+ rootHash,
186
+ timestamp,
187
+ signature,
188
+ institutionKid: this.#institutionKid,
189
+ }
190
+ await this._storeSeal(seal)
191
+ this.#pending = []
192
+
193
+ if (this.#chain) {
194
+ this.#chain.anchorAuditRoot({ rootHash, entryCount: toSeq + 1 }).catch((err) => {
195
+ console.warn(`[kxco-pq-audit] chain checkpoint failed (seal ${fromSeq}-${toSeq}): ${err.message}`)
196
+ })
197
+ }
198
+
199
+ return seal
200
+ }
201
+
202
+ /** Every seal this log holds, oldest first. */
203
+ async seals() {
204
+ return this._seals()
205
+ }
206
+
207
+ /** Entries appended since the last seal. Sealed logs only. */
208
+ async unsealedCount() {
209
+ if (!this.#sealed) return 0
210
+ await this.#load()
211
+ return this.#pending.length
212
+ }
213
+
214
+ /**
215
+ * Replay the log from entry 0. Streams, so memory is bounded by one entry and
216
+ * the seal list rather than by the log.
217
+ */
62
218
  async verify(publicKey) {
63
- const all = await this._entries()
64
- for (let i = 0; i < all.length; i++) {
65
- const e = all[i]
66
- if (i === 0) {
67
- if (e.prevHash !== null) return { valid: false, error: 'entry 0: prevHash must be null' }
68
- } else {
69
- const expected = hashEntry(all[i - 1])
70
- if (e.prevHash !== expected) return { valid: false, error: `entry ${i}: prevHash mismatch` }
219
+ const seals = this.#sealed ? await this._seals() : []
220
+ let sealIndex = 0
221
+ let expectedFrom = 0
222
+ let prevRoot = GENESIS_ROOT
223
+ let runHashes = []
224
+
225
+ let count = 0
226
+ let prevHash = null
227
+ let expectedSeq = 0
228
+
229
+ for await (const entry of this._iterate()) {
230
+ if (entry.seq !== expectedSeq) {
231
+ return { valid: false, error: `entry ${count}: expected seq ${expectedSeq}, got ${entry.seq}` }
232
+ }
233
+ if (entry.prevHash !== prevHash) {
234
+ return count === 0
235
+ ? { valid: false, error: 'entry 0: prevHash must be null' }
236
+ : { valid: false, error: `entry ${count}: prevHash mismatch` }
71
237
  }
72
- const msg = signingBytes(e.seq, e.timestamp, e.operation, e.metadata, e.prevHash)
73
- let ok
74
- try { ok = mlDsa.verify(new Uint8Array(publicKey), msg, Buffer.from(fromB64url(e.signature)).toString('hex')) }
75
- catch { ok = false }
76
- if (!ok) return { valid: false, error: `entry ${i}: signature invalid` }
238
+
239
+ if (!this.#sealed) {
240
+ const msg = signingBytes(entry.seq, entry.timestamp, entry.operation, entry.metadata, entry.prevHash)
241
+ let ok
242
+ try { ok = mlDsa.verify(new Uint8Array(publicKey), msg, Buffer.from(fromB64url(entry.signature)).toString('hex')) }
243
+ catch { ok = false }
244
+ if (!ok) return { valid: false, error: `entry ${count}: signature invalid` }
245
+ } else if (sealIndex < seals.length) {
246
+ const s = seals[sealIndex]
247
+ if (s.fromSeq !== expectedFrom) {
248
+ return { valid: false, error: `seal ${sealIndex}: expected fromSeq ${expectedFrom}, got ${s.fromSeq}` }
249
+ }
250
+ if (s.prevRoot !== prevRoot) {
251
+ return { valid: false, error: `seal ${sealIndex}: prevRoot does not chain to the seal before it` }
252
+ }
253
+ if (entry.seq >= s.fromSeq) runHashes.push(hashEntry(entry))
254
+ if (entry.seq === s.toSeq) {
255
+ const bad = this.#closeSeal(s, sealIndex, prevRoot, runHashes, publicKey)
256
+ if (bad) return bad
257
+ prevRoot = s.rootHash
258
+ expectedFrom = s.toSeq + 1
259
+ runHashes = []
260
+ sealIndex++
261
+ }
262
+ }
263
+
264
+ prevHash = hashEntry(entry)
265
+ expectedSeq = entry.seq + 1
266
+ count++
267
+ }
268
+
269
+ if (!this.#sealed) return { valid: true, count }
270
+
271
+ if (sealIndex < seals.length) {
272
+ const s = seals[sealIndex]
273
+ return { valid: false, error: `seal ${sealIndex}: covers seq ${s.toSeq}, log ends at ${count - 1}` }
274
+ }
275
+
276
+ // Never let an unsealed tail read as proven. sealedThrough is where the
277
+ // signed record stops; unsealed entries are chained and nothing more.
278
+ return {
279
+ valid: true,
280
+ count,
281
+ sealedThrough: expectedFrom - 1,
282
+ unsealed: count - expectedFrom,
77
283
  }
78
- return { valid: true, count: all.length }
79
284
  }
80
285
 
286
+ #closeSeal(s, index, prevRoot, runHashes, publicKey) {
287
+ if (rootOf(prevRoot, runHashes) !== s.rootHash) {
288
+ return { valid: false, error: `seal ${index}: entries do not reproduce rootHash` }
289
+ }
290
+ const msg = sealBytes(s.fromSeq, s.toSeq, s.prevRoot, s.rootHash, s.timestamp)
291
+ let ok
292
+ try { ok = mlDsa.verify(new Uint8Array(publicKey), msg, Buffer.from(fromB64url(s.signature)).toString('hex')) }
293
+ catch { ok = false }
294
+ if (!ok) return { valid: false, error: `seal ${index}: signature invalid` }
295
+ return null
296
+ }
297
+
298
+ /**
299
+ * Every entry as an array. Holds the whole log in memory by definition; on a
300
+ * large log prefer `stream()`.
301
+ */
81
302
  async export() {
82
- return this._entries()
303
+ const out = []
304
+ for await (const entry of this._iterate()) out.push(entry)
305
+ return out
306
+ }
307
+
308
+ /** Every entry, in seq order, one at a time. */
309
+ stream() {
310
+ return this._iterate()
83
311
  }
84
312
 
313
+ // --- storage hooks; a backend overrides these ---
314
+
85
315
  async _entries() { return [...this.#entries] }
86
316
  async _store(entry) { this.#entries.push(entry) }
317
+ async _seals() { return [...this.#sealsList] }
318
+ async _storeSeal(seal) { this.#sealsList.push(seal) }
319
+
320
+ /**
321
+ * Entries in seq order. The default reads them all through `_entries()`, so a
322
+ * backend that only overrides `_entries()` still works; one that can stream
323
+ * should override this instead.
324
+ */
325
+ async *_iterate() {
326
+ for (const entry of await this._entries()) yield entry
327
+ }
87
328
  }
@@ -1,23 +1,157 @@
1
- import { readFile, appendFile } from 'node:fs/promises'
1
+ import { createReadStream } from 'node:fs'
2
+ import { readFile, appendFile, open, stat } from 'node:fs/promises'
3
+ import { createInterface } from 'node:readline'
2
4
  import { AuditLog } from '../audit-log.js'
3
5
  import { KxcoPqAuditError } from '../errors.js'
4
6
 
7
+ /**
8
+ * Append-only NDJSON backend.
9
+ *
10
+ * Entries are read as a stream, never all at once, so appending and verifying
11
+ * both stay flat as the file grows. Seals go to `<path>.seals`, so a reader
12
+ * that only knows about entries is unaffected by a sealed log.
13
+ *
14
+ * Writes go through one handle held open for the life of the instance, which
15
+ * costs ~46us an entry against ~421us for reopening the file every time. Call
16
+ * `close()` when done with the log; the data is already with the OS either way,
17
+ * so a missed close leaks a descriptor and nothing else.
18
+ *
19
+ * One file has one writer. A signed chain cannot have two: both would build on
20
+ * the same tail, and the second entry would take the first one's place. The
21
+ * size the file should be is recorded on open and checked whenever the log
22
+ * seals or closes, so a second writer surfaces as an error rather than as a
23
+ * chain that only fails much later at verify().
24
+ */
5
25
  export class FileAuditLog extends AuditLog {
6
26
  #path
27
+ #sealPath
28
+ #handle = null
29
+ #size = null
30
+ #idleTimer = null
31
+ #idleMs
7
32
 
8
- constructor({ keypair, path, chain, checkpointEvery, institutionKid }) {
9
- super({ keypair, chain, checkpointEvery, institutionKid })
33
+ constructor({ keypair, path, chain, checkpointEvery, institutionKid, sealed, idleReleaseMs = 2000 }) {
34
+ super({ keypair, chain, checkpointEvery, institutionKid, sealed })
10
35
  if (!path) throw new KxcoPqAuditError('FileAuditLog: path is required')
11
- this.#path = path
36
+ this.#path = path
37
+ this.#sealPath = path + '.seals'
38
+ this.#idleMs = idleReleaseMs
39
+ }
40
+
41
+ async *_iterate() {
42
+ let stream
43
+ try {
44
+ stream = createReadStream(this.#path, { encoding: 'utf8' })
45
+ await new Promise((resolve, reject) => {
46
+ stream.once('readable', resolve)
47
+ stream.once('end', resolve)
48
+ stream.once('error', reject)
49
+ })
50
+ } catch (err) {
51
+ if (err.code === 'ENOENT') return
52
+ throw err
53
+ }
54
+
55
+ const lines = createInterface({ input: stream, crlfDelay: Infinity })
56
+ let lineNo = 0
57
+ try {
58
+ for await (const line of lines) {
59
+ lineNo++
60
+ if (!line) continue
61
+ try {
62
+ yield JSON.parse(line)
63
+ } catch {
64
+ // A line that will not parse is either a torn write from a crash or
65
+ // an edit. Either way it is not something to skip past in silence.
66
+ throw new KxcoPqAuditError(
67
+ `${this.#path}: line ${lineNo} is not valid JSON. ` +
68
+ 'A partial last line is an interrupted append and can be truncated; ' +
69
+ 'anywhere else means the file was edited.'
70
+ )
71
+ }
72
+ }
73
+ } finally {
74
+ lines.close()
75
+ stream.destroy()
76
+ }
12
77
  }
13
78
 
14
79
  async _entries() {
80
+ const out = []
81
+ for await (const entry of this._iterate()) out.push(entry)
82
+ return out
83
+ }
84
+
85
+ async _store(entry) {
86
+ const line = JSON.stringify(entry) + '\n'
87
+ const handle = await this.#open()
88
+ await handle.write(line, null, 'utf8')
89
+ this.#size += Buffer.byteLength(line, 'utf8')
90
+ this.#idleRelease()
91
+ }
92
+
93
+ /**
94
+ * Let go of the handle once writing stops.
95
+ *
96
+ * A busy log never goes idle and keeps the fast path. A short-lived one hands
97
+ * the descriptor back on its own, so forgetting `close()` costs nothing
98
+ * instead of failing when the handle is finally collected. The timer is
99
+ * unref'd, so it never holds the process open, and it holds a reference to
100
+ * this log, so the handle cannot be collected while a release is pending.
101
+ */
102
+ #idleRelease() {
103
+ clearTimeout(this.#idleTimer)
104
+ this.#idleTimer = setTimeout(() => { this.close().catch(() => {}) }, this.#idleMs)
105
+ this.#idleTimer.unref?.()
106
+ }
107
+
108
+ async _seals() {
15
109
  let text
16
- try { text = await readFile(this.#path, 'utf8') } catch { return [] }
110
+ try { text = await readFile(this.#sealPath, 'utf8') } catch { return [] }
17
111
  return text.trim().split('\n').filter(Boolean).map(line => JSON.parse(line))
18
112
  }
19
113
 
20
- async _store(entry) {
21
- await appendFile(this.#path, JSON.stringify(entry) + '\n', 'utf8')
114
+ async _storeSeal(seal) {
115
+ await this.assertSoleWriter()
116
+ await appendFile(this.#sealPath, JSON.stringify(seal) + '\n', 'utf8')
117
+ }
118
+
119
+ /**
120
+ * Throw if the entry file is not the size this instance left it. Runs on open
121
+ * and on every seal; call it directly to check at any other point.
122
+ */
123
+ async assertSoleWriter() {
124
+ if (this.#size === null) return
125
+ let actual = 0
126
+ try { actual = (await stat(this.#path)).size } catch (err) {
127
+ if (err.code !== 'ENOENT') throw err
128
+ }
129
+ if (actual !== this.#size) {
130
+ throw new KxcoPqAuditError(
131
+ `${this.#path}: file is ${actual} bytes, expected ${this.#size}. ` +
132
+ 'Something else wrote to this log. Appending now would fork the chain, ' +
133
+ 'so open a new instance to pick up the current tail.'
134
+ )
135
+ }
136
+ }
137
+
138
+ /** Release the write handle. Safe to call more than once, and after any write. */
139
+ async close() {
140
+ clearTimeout(this.#idleTimer)
141
+ this.#idleTimer = null
142
+ if (!this.#handle) return
143
+ const handle = this.#handle
144
+ this.#handle = null
145
+ try { await this.assertSoleWriter() } finally { await handle.close() }
146
+ }
147
+
148
+ async #open() {
149
+ if (this.#handle) return this.#handle
150
+ try { this.#size = (await stat(this.#path)).size } catch (err) {
151
+ if (err.code !== 'ENOENT') throw err
152
+ this.#size = 0
153
+ }
154
+ this.#handle = await open(this.#path, 'a')
155
+ return this.#handle
22
156
  }
23
157
  }
package/src/index.d.ts CHANGED
@@ -7,13 +7,42 @@ export interface AuditEntry {
7
7
  metadata: Record<string, unknown>
8
8
  /** SHA-256 of the previous entry (entire JSON including signature). Null for the first entry. */
9
9
  prevHash: string | null
10
- /** base64url ML-DSA-65 signature over a canonical representation of all other fields. */
11
- signature: string
10
+ /**
11
+ * base64url ML-DSA-65 signature over a canonical representation of all other
12
+ * fields. Absent on a sealed log, where the run is signed once by `seal()`.
13
+ */
14
+ signature?: string
15
+ }
16
+
17
+ /** One signed run of a sealed log. Seals chain to each other by `prevRoot`. */
18
+ export interface AuditSeal {
19
+ fromSeq: number
20
+ toSeq: number
21
+ entryCount: number
22
+ /** The previous seal's rootHash, or 64 zeroes for the first seal. */
23
+ prevRoot: string
24
+ /** base64url SHA-256 over prevRoot followed by every entry hash in the run. */
25
+ rootHash: string
26
+ timestamp: string
27
+ /** base64url ML-DSA-65 signature over the seal's canonical form. */
28
+ signature: string
29
+ institutionKid: string | null
12
30
  }
13
31
 
14
32
  export interface AuditVerifySuccess {
15
33
  valid: true
16
34
  count: number
35
+ /**
36
+ * Sealed logs only: the highest seq covered by a verified seal, or -1 when
37
+ * nothing is sealed yet.
38
+ */
39
+ sealedThrough?: number
40
+ /**
41
+ * Sealed logs only: entries appended since the last seal. They are chained
42
+ * but carry no signature. A non-zero value is not a failure and is not proof
43
+ * either — seal() closes the window.
44
+ */
45
+ unsealed?: number
17
46
  }
18
47
 
19
48
  export interface AuditVerifyFailure {
@@ -39,14 +68,34 @@ export interface AuditLogOptions {
39
68
  checkpointEvery?: number
40
69
  /** Institution kid used to tag the anchor. Derived from keypair if omitted. */
41
70
  institutionKid?: string
71
+ /**
72
+ * Sign once per sealed run instead of once per entry (default: false).
73
+ *
74
+ * Every entry is still hash-chained, which is what detects an edit, a
75
+ * removal, a reorder or an insertion. The signature binds the run to the
76
+ * key. Measured on ML-DSA-65: per-entry signing runs at a few hundred
77
+ * entries per second and ~4.8KB per entry; sealing runs at tens of thousands
78
+ * per second and ~370 bytes. Use it wherever entry volume is high enough
79
+ * that a signature per entry does not fit, such as agent tool calls.
80
+ *
81
+ * The tradeoff: a single entry can no longer be verified in isolation, and
82
+ * entries written since the last seal are chained but unsigned.
83
+ */
84
+ sealed?: boolean
42
85
  }
43
86
 
44
87
  /**
45
88
  * In-memory tamper-evident audit log.
46
89
  *
47
- * Each entry is ML-DSA-65-signed and SHA-256 hash-chained to its predecessor.
90
+ * Every entry is SHA-256 hash-chained to its predecessor, and signed with
91
+ * ML-DSA-65 either per entry or, with `sealed: true`, once per run.
48
92
  * `verify()` replays the entire chain — any gap, reorder, or edit breaks either
49
93
  * the chain or a signature.
94
+ *
95
+ * Appending costs the same on a log of ten entries and a log of ten million:
96
+ * a new entry needs the previous entry's hash, never the log. Concurrent
97
+ * `append()` calls are serialised internally, so two in flight together cannot
98
+ * take the same seq.
50
99
  */
51
100
  export declare class AuditLog {
52
101
  constructor(options: AuditLogOptions)
@@ -54,16 +103,51 @@ export declare class AuditLog {
54
103
  /** Append a signed, hash-chained entry. Returns the created entry. */
55
104
  append(operation: string, metadata?: Record<string, unknown>): Promise<AuditEntry>
56
105
 
57
- /** Verify every entry's signature and the full hash chain. */
106
+ /**
107
+ * Verify the full hash chain, plus every entry's signature on a classic log
108
+ * or every seal on a sealed one.
109
+ */
58
110
  verify(publicKey: Uint8Array | Buffer): Promise<AuditVerifyResult>
59
111
 
60
- /** Return a copy of all entries. */
112
+ /**
113
+ * Sign everything appended since the last seal, as one run. Returns the seal,
114
+ * or null when nothing is unsealed. Throws unless the log was constructed
115
+ * with `sealed: true`. Anchors the root when a chain client is configured.
116
+ */
117
+ seal(): Promise<AuditSeal | null>
118
+
119
+ /** Every seal this log holds, oldest first. */
120
+ seals(): Promise<AuditSeal[]>
121
+
122
+ /** Entries appended since the last seal. Always 0 on a classic log. */
123
+ unsealedCount(): Promise<number>
124
+
125
+ /** True when this log signs per run rather than per entry. */
126
+ readonly sealed: boolean
127
+
128
+ /**
129
+ * Every entry as an array. Holds the whole log in memory by definition; on a
130
+ * large log prefer `stream()`.
131
+ */
61
132
  export(): Promise<AuditEntry[]>
133
+
134
+ /** Every entry, in seq order, one at a time. Memory is bounded by one entry. */
135
+ stream(): AsyncIterableIterator<AuditEntry>
62
136
  }
63
137
 
64
138
  export interface FileAuditLogOptions extends AuditLogOptions {
65
- /** Path to the append-only NDJSON file. Created on first write if absent. */
139
+ /**
140
+ * Path to the append-only NDJSON file. Created on first write if absent.
141
+ * On a sealed log the seals are written to `<path>.seals`, so a reader that
142
+ * knows only about entries is unaffected.
143
+ */
66
144
  path: string
145
+ /**
146
+ * Release the write handle after this many ms without a write (default:
147
+ * 2000). A busy log never goes idle and keeps the handle; a short-lived one
148
+ * hands the descriptor back on its own, so a missed `close()` costs nothing.
149
+ */
150
+ idleReleaseMs?: number
67
151
  }
68
152
 
69
153
  /**
@@ -72,6 +156,19 @@ export interface FileAuditLogOptions extends AuditLogOptions {
72
156
  */
73
157
  export declare class FileAuditLog extends AuditLog {
74
158
  constructor(options: FileAuditLogOptions)
159
+
160
+ /**
161
+ * Release the write handle. Safe to call more than once. Not required — the
162
+ * handle is released on idle — but call it for a deterministic hand-back.
163
+ * Throws if another writer has touched the file.
164
+ */
165
+ close(): Promise<void>
166
+
167
+ /**
168
+ * Throw unless the entry file is the size this instance left it. Runs on
169
+ * every seal and on close; call it directly to check at any other point.
170
+ */
171
+ assertSoleWriter(): Promise<void>
75
172
  }
76
173
 
77
174
  export class KxcoPqAuditError extends Error {