@staix/vault-mem 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -0
- package/README.md +5 -5
- package/package.json +1 -1
- package/src/analysis-input.ts +2 -1
- package/src/analysis-reports.ts +4 -3
- package/src/cli.ts +19 -1
- package/src/org-promote.ts +15 -10
- package/src/registry.ts +15 -3
- package/src/sync.ts +20 -10
- package/src/tidy.ts +25 -11
- package/src/vault-log.ts +22 -2
- package/src/write-validation.ts +2 -1
- package/src/writer.ts +15 -10
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.2.2] - 2026-10-31
|
|
4
|
+
|
|
5
|
+
- Report operations under `reports/` and `ops/` accept and reread files up to 16 MiB, so graph artifacts above 2 MiB can be rebuilt. Notes keep their 256 KiB write / 2 MiB read limits; CLI body input limits are unchanged.
|
|
6
|
+
- `vmem doctor` checks configured remotes and project registries locally, without network access. Remote validation shares sync's origin URL rules, registry validation uses the log-write parser, and failed checks exit 1.
|
|
7
|
+
- A registry update whose directory flush fails after the config was replaced now reports `registry updated but directory fsync failed`, so the caller knows the update committed and need not retry. The lock is still released; a retried `vaults add` remains refused as already registered.
|
|
8
|
+
- `vmem recover` reads at most 4 KiB of the lock owner record; a larger lock file is treated as unreadable (state `unknown`) and is never removed.
|
|
9
|
+
|
|
10
|
+
## [0.2.1] - 2026-10-30
|
|
11
|
+
|
|
12
|
+
- The `tidy` LINT event and the org-promotion SYNC events are bound to the vaults resolved at the start of the run; if a vault id is re-registered before the append, the event is refused with LOG_DRIFT instead of landing in the other vault. The committed notes are not rolled back.
|
|
13
|
+
- Tidy and org promotion bind every note write to the same vaults: a re-registration before a write is refused with VAULT_DRIFT and nothing is written. A bound log append reports any re-registration as LOG_DRIFT, before policy or path checks.
|
|
14
|
+
- Org promotion and `tidy --log` report `applied: true` when a write committed and a later step (lock removal, directory flush) failed; tidy then lists the log in `written`.
|
|
15
|
+
|
|
3
16
|
## [0.2.0] - 2026-10-30
|
|
4
17
|
|
|
5
18
|
- Fine-grained PII policy: `vault.pii` can set `allow`, `warn` or `deny` per category (email, phone, the global `piiPatterns`, named per-vault patterns) and per surface (notes, log, promotion), with an allow list of exact addresses, domains and phone numbers. The `allow`/`deny` shorthand keeps its previous meaning.
|
package/README.md
CHANGED
|
@@ -31,7 +31,7 @@ Durability: the temporary file is fsynced before the rename and the home directo
|
|
|
31
31
|
|
|
32
32
|
Crash recovery: the lock file records the writer's pid, host and start time. Nothing is cleaned up automatically. `vmem recover` previews the state; `vmem recover --confirm` removes the lock only when it names this host and a pid that no longer runs, and then removes leftover `.config-<uuid>.tmp` files. A live lock, a lock from another host, or an unreadable (older-format) lock is never removed and the command exits 1; confirm no writer runs, then delete `config.lock` by hand. No other file in the home or any vault is touched. If cleanup fails after an operation (for example the lock cannot be removed), the original error is kept and the cleanup failure, including a retained lock, is appended to it.
|
|
33
33
|
|
|
34
|
-
Vault fields: `id`, absolute `path` (must contain a real `notes/` directory), `kind` (`personal|org`), `write` (`mcp|cli|none`), optional `pii` (the `allow|deny` shorthand or a policy object, see PII policy below; default `allow` for personal and `deny` for org, and `status`/`vaults list` always report the effective shorthand and `piiPolicy` matrix, including for unhealthy entries), optional `remote` (any nonempty string such as an `https://` or `git@host:path` URL
|
|
34
|
+
Vault fields: `id`, absolute `path` (must contain a real `notes/` directory), `kind` (`personal|org`), `write` (`mcp|cli|none`), optional `pii` (the `allow|deny` shorthand or a policy object, see PII policy below; default `allow` for personal and `deny` for org, and `status`/`vaults list` always report the effective shorthand and `piiPolicy` matrix, including for unhealthy entries), optional `remote` (any nonempty string such as an `https://` or `git@host:path` URL) and optional `registry` (absolute path). For healthy vaults, `doctor` checks locally that the vault is a git repository root with an origin whose fetch and push URLs all match `remote` (URL rewrites are refused), and that `registry` passes the same deny-path, size and project-id validation as log appends. Results appear under each vault's `checks.remote` and `checks.registry`; a failed check exits 1. These checks use no network and do not test reachability. `status` is unchanged. Policy fields are stored for later note-writing stages, not enforced by the read commands. Writer commands enforce write/channel policy and configured deny/PII screening.
|
|
35
35
|
|
|
36
36
|
The only runtime dependency is the exact first-party pin `@staix/agent-mem@0.11.0`; it is not loaded by registry commands. Public library exports cover configuration, home resolution, registry operations, frontmatter, link resolution and the derived index. No console bin or UI contract is advertised before its implementation.
|
|
37
37
|
|
|
@@ -121,9 +121,9 @@ Syntax follows indexed reads: the vault id comes first. `new` supports `--title`
|
|
|
121
121
|
|
|
122
122
|
`--dry-run` plans and validates any writer without creating locks, temporary files, directories or caches. `check` always takes this path, even without `--dry-run`. Move, delete and tidy first return a preview with planned path/hash changes and a fingerprint; confirmation requires that fingerprint. It covers canonical operation input, the canonical vault path/identity and policy, expected SHAs, every scanned note and all touched files, so even a new inbound reference after preview forces another preview. Move rewrites visible wiki/heading/block/embed targets and wiki references in frontmatter across the vault, preserving explicit `|label` text, anchors and code. Alias-only links resolve to no note and are left as written. The move source may be an existing note with a legacy file name (lint L13, for example a dot in the stem); the destination must be a valid stem. Tidy folds Connections sections into Relevant Notes. Delete refuses when it would newly break inbound links or MOC topics; fix those references first.
|
|
123
123
|
|
|
124
|
-
Validation returns named issues with severity and no matched secret text: V01 schema/enums/description; V02 existing MOC topics; V03 newly unresolved links; V04 flat ASCII note paths (maximum 200 bytes) and safe report paths; V05 final Topics placement/content (warning), with explicit append-to-Topics refusal; V06 pinned agent-mem secret regex parity and denied sources/input/snapshot paths; V07 email/phone/configured PII patterns when denied; V08 create duplicates (stem/NFKC H1); V09 256 KiB prospective size, exact patch and expected-SHA/drift; V10 supersession target integrity. Unchanged pre-existing broken links warn, including after a move. `possibleDuplicates` lists candidate paths for warnings from a disposable in-memory FTS5 query of title terms, with BM25 <= -0.000001, up to three paths; no disk cache is written for that check. `LOCKED`, `IO`, `ROLLBACK`, `INDEX` and `CLEANUP` identify operational failures. CLI returns exit 1 when `ok` is false.
|
|
124
|
+
Validation returns named issues with severity and no matched secret text: V01 schema/enums/description; V02 existing MOC topics; V03 newly unresolved links; V04 flat ASCII note paths (maximum 200 bytes) and safe report paths; V05 final Topics placement/content (warning), with explicit append-to-Topics refusal; V06 pinned agent-mem secret regex parity and denied sources/input/snapshot paths; V07 email/phone/configured PII patterns when denied; V08 create duplicates (stem/NFKC H1); V09 256 KiB prospective note size / 16 MiB report size, exact patch and expected-SHA/drift; V10 supersession target integrity. Unchanged pre-existing broken links warn, including after a move. `possibleDuplicates` lists candidate paths for warnings from a disposable in-memory FTS5 query of title terms, with BM25 <= -0.000001, up to three paths; no disk cache is written for that check. `LOCKED`, `IO`, `ROLLBACK`, `INDEX` and `CLEANUP` identify operational failures. CLI returns exit 1 when `ok` is false.
|
|
125
125
|
|
|
126
|
-
A successful receipt reports `applied`, planned path/hash/byte changes, issues and `indexUpdated`. Unrelated malformed legacy YAML is skipped by duplicate hints and by writer index refresh, with a V01 warning naming the unchanged note; normal read/rebuild commands retain strict parsing. Move/tidy still require parseable inputs and name the offending path on refusal. Reads allocate each note's stat size plus one growth-probe byte, capped at the 2 MiB read limit plus one. A failed index refresh rolls back the file changes. Rollback first verifies each file still equals this operation's committed result; external edits are preserved. Failed/refused rollback retains the original backup and reports its relative recovery path. All files are staged before replacement and backups permit rollback of caught multi-file failures. Each individual rename is atomic; a machine/process crash between multiple renames can leave a partially completed batch. The retained lock and `.vmem-<uuid>.tmp` files require inspection and manual recovery after confirming the writer stopped. They are never automatically removed by registry recovery. Uncoordinated editors do not honor the lock; the pipeline rechecks SHAs before replacement, but this does not provide a filesystem transaction against concurrent external edits. Existing notes may be read up to the index's 2 MiB limit; prospective
|
|
126
|
+
A successful receipt reports `applied`, planned path/hash/byte changes, issues and `indexUpdated`. Unrelated malformed legacy YAML is skipped by duplicate hints and by writer index refresh, with a V01 warning naming the unchanged note; normal read/rebuild commands retain strict parsing. Move/tidy still require parseable inputs and name the offending path on refusal. Reads allocate each note's stat size plus one growth-probe byte, capped at the 2 MiB read limit plus one. A failed index refresh rolls back the file changes. Rollback first verifies each file still equals this operation's committed result; external edits are preserved. Failed/refused rollback retains the original backup and reports its relative recovery path. All files are staged before replacement and backups permit rollback of caught multi-file failures. Each individual rename is atomic; a machine/process crash between multiple renames can leave a partially completed batch. The retained lock and `.vmem-<uuid>.tmp` files require inspection and manual recovery after confirming the writer stopped. They are never automatically removed by registry recovery. Uncoordinated editors do not honor the lock; the pipeline rechecks SHAs before replacement, but this does not provide a filesystem transaction against concurrent external edits. Existing notes may be read up to the index's 2 MiB limit; prospective notes are limited to 256 KiB; report operations under `reports/` and `ops/` allow up to 16 MiB for both prospective content and existing-file reads (including staging and rollback rechecks), with at most 10000 flat notes per mutation snapshot. File/directory fsync has the platform guarantees described above. Successful changed mutations now append an event as described below; log failure is reported separately from note success.
|
|
127
127
|
|
|
128
128
|
|
|
129
129
|
# Append-only events
|
|
@@ -140,7 +140,7 @@ Every MCP log append and CLI call with a nonempty `AGENTHUB_PEER_ID`, `CLAUDECOD
|
|
|
140
140
|
|
|
141
141
|
`appendLog` validates the registered canonical vault and write/channel policy, acquires a separate `log.lock` (bounded two-second contention wait), then makes one O_APPEND write and fsyncs the log. New log files are mode 0600 and their parent directory is flushed. Linked/nonregular files and an incomplete existing final line are refused. Cooperating appenders share the lock; a short write is truncated back only when the same file still has the expected size, and its tail still equals our exact prefix, then flushed. Error-after-write cases inspect at most one event's bytes. Failed/ambiguous writes or cleanup are explicit; the receipt reports `appended` (possibly null when unknown), `synced`, `bytesWritten` (possibly null) and `rolledBack` when a partial append was removed. Before reporting sync, the current log and vault identities are rechecked; rotation produces LOG_DRIFT and an unknown current-path append state. A synced event can coexist with a cleanup error. No existing event is rewritten. A crash can retain `log.lock`, and failed short-write recovery can retain an incomplete line; inspect the file and stopped writer before manual recovery. There is no automatic cleanup or new TTY requirement.
|
|
142
142
|
|
|
143
|
-
Successful changed mutations default to EXTRACT, link to CONNECT, move to ROUTE, and supersede/tidy to RETHINK. Default events use project `-`, affected paths and the operation name rather than note text. Library callers can supply `WriteRequest.log` to override the event or `false` to disable it; CLI writers accept `--log JSON` for overrides. Preview/check/no-op/refused mutations never append. The append is bound to the mutation's resolved vault identity/policy, so re-registration cannot route its event to another vault. After a committed mutation, log append is independent: `ok`/`applied` describe the note operation, `log` holds its append receipt, and `logged` is true only when the event was appended and synced. A log error does not roll back a successfully saved note. CLI exits 1 for a log failure even when the note receipt says `ok: true`; inspect the receipt before retrying to avoid duplicate mutations/events. An override is validated at append time, not during a note-only dry run.
|
|
143
|
+
Successful changed mutations default to EXTRACT, link to CONNECT, move to ROUTE, and supersede/tidy to RETHINK. Default events use project `-`, affected paths and the operation name rather than note text. Library callers can supply `WriteRequest.log` to override the event or `false` to disable it; CLI writers accept `--log JSON` for overrides. Preview/check/no-op/refused mutations never append. The append is bound to the mutation's resolved vault identity/policy, so re-registration cannot route its event to another vault; the same binding covers the tidy LINT event and both SYNC events of org promotion, which are refused with LOG_DRIFT instead. A bound append compares the registry before any policy or path check, so a re-registration always reads as LOG_DRIFT. Tidy and org promotion also bind every note write to the vaults resolved at the start of the run: if a vault id is re-registered before a write, that write is refused with VAULT_DRIFT and nothing is written to either vault. After a committed mutation, log append is independent: `ok`/`applied` describe the note operation, `log` holds its append receipt, and `logged` is true only when the event was appended and synced. A log error does not roll back a successfully saved note. CLI exits 1 for a log failure even when the note receipt says `ok: true`; inspect the receipt before retrying to avoid duplicate mutations/events. An override is validated at append time, not during a note-only dry run.
|
|
144
144
|
|
|
145
145
|
Readers never create a home, log or lock, and work with `write: none`. They skip blank/comment lines, return the last matching 1..200 records (default 50), accept an optional canonical TYPE and non-backtracking regex subset up to 256 characters (`|`, `^`, `$`, `.`, character classes, `\d`/`\s`/`\w` and their inverses, escaped literals, and atom `*`/`+`/`?`; groups, backreferences and brace quantifiers are refused), and scan a fixed descriptor-size snapshot. Streaming is capped at 32 MiB; individual historical lines over 4096 bytes and incomplete final lines are omitted with warnings, and the serialized line array stays within 128 KiB. Above 20000 data lines, the manual-archival warning is preserved. Read failure returns `dataLines: null` rather than claiming a count. Golden synthetic fixtures and concurrent/fault persistence tests cover the format and actual append state; no live vault log was read or modified for this increment.
|
|
146
146
|
|
|
@@ -214,7 +214,7 @@ vmem promote-from-amem personal mem:<id> --link-to existing-note # link an exis
|
|
|
214
214
|
|
|
215
215
|
`vmem graph build <vault> [--dry-run]` writes `reports/vault-graph.json`, a dated graph report, graph trend history and GRAPH events through `applyWrite`. Old dated reports are preserved. Trend history appends one row per build, including repeated builds on the same date. Output receipts expose partial report/log failure; these separate writes are not a single transaction. `vmem graph neighbors <vault> <stem> [--hops 1..2] [--limit 1..500]` reads a bounded neighborhood. Matching MCP tools are `vmem_lint`, `vmem_graph_build`, and `vmem_graph_neighbors`; MCP report writes require `write=mcp`.
|
|
216
216
|
|
|
217
|
-
Analysis consumes note sources from the derived SQLite index, including invalid-frontmatter rows retained separately from searchable notes. Normal scans also retain note sources in `analysis_notes`, roughly doubling cached source storage outside the vault; the cache remains derived and can be rebuilt. Read-only analysis and dry runs clone any existing derived cache through a readonly connection, then refresh changed notes in memory. A missing cache is built in memory. They create no cache, locks, reports or events. Auxiliary log, registry and report inputs use bounded plain-file reads. Snapshot limits are 10,000 notes / 64 MiB source, 1,000 reports / 16 MiB combined, and 32 MiB log; oversized inputs fail explicitly. Graph limits are 100,000 edges and 32 deterministic label-propagation rounds. MOCs are excluded from clustering; ties sort by node id. Graph extraction intentionally retains the reference's body-only legacy wiki-link semantics, distinct from the richer navigation resolver.
|
|
217
|
+
Analysis consumes note sources from the derived SQLite index, including invalid-frontmatter rows retained separately from searchable notes. Normal scans also retain note sources in `analysis_notes`, roughly doubling cached source storage outside the vault; the cache remains derived and can be rebuilt. Read-only analysis and dry runs clone any existing derived cache through a readonly connection, then refresh changed notes in memory. A missing cache is built in memory. They create no cache, locks, reports or events. Auxiliary log, registry and report inputs use bounded plain-file reads. Snapshot limits are 10,000 notes / 64 MiB source, 1,000 reports / 16 MiB per file and combined, and 32 MiB log; oversized inputs fail explicitly. Graph limits are 100,000 edges and 32 deterministic label-propagation rounds. MOCs are excluded from clustering; ties sort by node id. Graph extraction intentionally retains the reference's body-only legacy wiki-link semantics, distinct from the richer navigation resolver.
|
|
218
218
|
|
|
219
219
|
`bun scripts/lint-parity.ts` and `bun scripts/graph-parity.ts` compare the synthetic corpus against frozen Python reference oracles. Use `--reference /absolute/path/to/lint.py` or `--reference /absolute/path/to/build-graph.py` to run the unchanged Python tools directly (`PYTHON` selects an interpreter with PyYAML/networkx). Keep those references until a separate retirement decision after two consecutive zero-diff runs. Fixture parity does not establish private-corpus parity or authorize a live vault cutover. Private corpus evidence stays private; no note bodies belong in public fixtures.
|
|
220
220
|
|
package/package.json
CHANGED
package/src/analysis-input.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { MAX_REPORT_BYTES } from "./write-validation.ts";
|
|
1
2
|
import { parseFrontmatter } from "./frontmatter.ts";
|
|
2
3
|
import { sourceWorkspaceRoot, missingSourceWarnings, sourceReferences } from "./source.ts";
|
|
3
4
|
import { parseAnalysisYaml } from "./analysis-yaml.ts";
|
|
@@ -50,7 +51,7 @@ export function analysisInput(home: string, id: string, options: { today?: strin
|
|
|
50
51
|
let bytes = 0;
|
|
51
52
|
for (const name of entries) {
|
|
52
53
|
const file = join(reportsDir,name); permitted(file,config);
|
|
53
|
-
const source = analysisRead(file); if (source !== null) { bytes += Buffer.byteLength(source); if (bytes >
|
|
54
|
+
const source = analysisRead(file, MAX_REPORT_BYTES); if (source !== null) { bytes += Buffer.byteLength(source); if (bytes > MAX_REPORT_BYTES) throw new Error("analysis: reports exceed aggregate bound"); reports[name] = source; }
|
|
54
55
|
}
|
|
55
56
|
let registryIds: string[] | null = null;
|
|
56
57
|
if (vault.registry) {
|
package/src/analysis-reports.ts
CHANGED
|
@@ -1,3 +1,4 @@
|
|
|
1
|
+
import { MAX_REPORT_BYTES } from "./write-validation.ts";
|
|
1
2
|
import type { VaultSpec } from "./config.ts";
|
|
2
3
|
import { lstatSync } from "node:fs";
|
|
3
4
|
import { join } from "node:path";
|
|
@@ -46,10 +47,10 @@ export function writeAnalysisReports(home: string, request: AnalysisRequest, env
|
|
|
46
47
|
}
|
|
47
48
|
const originals = new Map(outputs.map(out => {
|
|
48
49
|
const name = out.path.slice("reports/".length);
|
|
49
|
-
return [out.path, request.kind === "graph" ? snapshot.reports[name] ?? null : analysisRead(join(vault.path,out.path))] as const;
|
|
50
|
+
return [out.path, request.kind === "graph" ? snapshot.reports[name] ?? null : analysisRead(join(vault.path,out.path), MAX_REPORT_BYTES)] as const;
|
|
50
51
|
}));
|
|
51
52
|
unchanged();
|
|
52
|
-
for (const out of outputs) if (analysisRead(join(vault.path,out.path)) !== originals.get(out.path)) throw new Error("analysis: report changed during build");
|
|
53
|
+
for (const out of outputs) if (analysisRead(join(vault.path,out.path), MAX_REPORT_BYTES) !== originals.get(out.path)) throw new Error("analysis: report changed during build");
|
|
53
54
|
const expectedShas = Object.fromEntries(snapshot.notes.map(n => ["notes/"+n.path.replace(/^notes\//,""),sha256(n.source)]));
|
|
54
55
|
// Preflight every output through the same validation pipeline, before any output is written.
|
|
55
56
|
const previews = outputs.map(out => applyWrite(home,{vault:request.vault,operation:{op:"report",...out},dryRun:true,channel:request.channel,expectedShas,log:false},{env}));
|
|
@@ -57,7 +58,7 @@ export function writeAnalysisReports(home: string, request: AnalysisRequest, env
|
|
|
57
58
|
const receipts: WriteReceipt[] = [];
|
|
58
59
|
for (const out of outputs) {
|
|
59
60
|
unchanged();
|
|
60
|
-
if (analysisRead(join(vault.path,out.path)) !== originals.get(out.path)) throw new Error("analysis: report changed during build");
|
|
61
|
+
if (analysisRead(join(vault.path,out.path), MAX_REPORT_BYTES) !== originals.get(out.path)) throw new Error("analysis: report changed during build");
|
|
61
62
|
const receipt = applyWrite(home,{vault:request.vault,operation:{op:"report",...out},channel:request.channel,expectedShas,log:false},{env});
|
|
62
63
|
receipts.push(receipt);
|
|
63
64
|
if (!receipt.ok) break;
|
package/src/cli.ts
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
import { originCheck } from "./sync.ts";
|
|
2
|
+
import { registryIds } from "./vault-log.ts";
|
|
1
3
|
import { runAnalysisCommand } from "./analysis-commands.ts";
|
|
2
4
|
import { runLogCommand } from "./log-commands.ts";
|
|
3
5
|
import { runOrgCommand } from "./org-commands.ts";
|
|
@@ -51,7 +53,23 @@ export function run(args: string[], env: NodeJS.ProcessEnv = process.env): { tex
|
|
|
51
53
|
switch (command) {
|
|
52
54
|
case "status": case "doctor": {
|
|
53
55
|
if (positional.length) throw new Error(`${command}: unexpected arguments`);
|
|
54
|
-
const
|
|
56
|
+
const config = command === "doctor" ? loadRegistry(home.home) : undefined;
|
|
57
|
+
const vaults = registryStatus(home.home).map(vault => {
|
|
58
|
+
if (command !== "doctor" || !vault.healthy || (!vault.remote && !vault.registry)) return vault;
|
|
59
|
+
const checks: Record<string, { ok: boolean; issue?: string }> = {};
|
|
60
|
+
if (vault.remote) {
|
|
61
|
+
try {
|
|
62
|
+
const checked = originCheck(vault.path, vault.remote, env);
|
|
63
|
+
checks.remote = { ok: checked.ok, ...(checked.issue ? { issue: checked.issue } : {}) };
|
|
64
|
+
} catch (error) { checks.remote = { ok: false, issue: error instanceof Error ? error.message : "git failed" }; }
|
|
65
|
+
}
|
|
66
|
+
if (vault.registry) {
|
|
67
|
+
try { registryIds(vault.registry, config!.denyPaths); checks.registry = { ok: true }; }
|
|
68
|
+
catch (error) { checks.registry = { ok: false, issue: error instanceof Error ? error.message : "cannot read project registry" }; }
|
|
69
|
+
}
|
|
70
|
+
return { ...vault, checks };
|
|
71
|
+
});
|
|
72
|
+
const healthy = vaults.every(v => v.healthy && (!("checks" in v) || Object.values(v.checks).every(check => check.ok)));
|
|
55
73
|
result = { ...home, version, stage: "foundation", healthy, vaults };
|
|
56
74
|
if (command === "doctor" && !healthy) code = 1;
|
|
57
75
|
break;
|
package/src/org-promote.ts
CHANGED
|
@@ -9,9 +9,9 @@ import { VaultIndex } from "./vault-index.ts";
|
|
|
9
9
|
import { applyWrite } from "./writer.ts";
|
|
10
10
|
import { notePath } from "./write-operations.ts";
|
|
11
11
|
import { screenContent } from "./write-validation.ts";
|
|
12
|
-
import { appendLog } from "./vault-log.ts";
|
|
12
|
+
import { appendLog, logBinding } from "./vault-log.ts";
|
|
13
13
|
import type { WriteIssue, WriteReceipt } from "./write-types.ts";
|
|
14
|
-
import type { LogReceipt } from "./log-types.ts";
|
|
14
|
+
import type { LogBinding, LogReceipt } from "./log-types.ts";
|
|
15
15
|
import { analysisInput } from "./analysis-input.ts";
|
|
16
16
|
import { lintVault } from "./vault-lint.ts";
|
|
17
17
|
import type { AnalysisSnapshot } from "./analysis-types.ts";
|
|
@@ -182,6 +182,10 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
|
|
|
182
182
|
if (target.kind !== "org" || target.write === "none") return fail(issue("POLICY", "promotion target must be a writable org vault"));
|
|
183
183
|
if (request.channel === "mcp" && target.write !== "mcp") return fail(issue("POLICY", "target vault write policy refuses MCP"));
|
|
184
184
|
if (request.channel === "mcp" && source.write !== "mcp") return fail(issue("POLICY", "source provenance write needs write=mcp on the personal vault"));
|
|
185
|
+
// SYNC events must land in the vaults resolved here, even if the registry is changed before the appends.
|
|
186
|
+
let bindings: Record<"source" | "target", LogBinding>;
|
|
187
|
+
try { bindings = { source: logBinding(source, config), target: logBinding(target, config) }; }
|
|
188
|
+
catch { return fail(issue("V04", "source or target vault path unavailable")); }
|
|
185
189
|
let path: string;
|
|
186
190
|
const index = new VaultIndex(home, source, all.map(v => v.path));
|
|
187
191
|
try { index.scan(); path = `notes/${index.show(request.note, 1).path}`; }
|
|
@@ -207,13 +211,13 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
|
|
|
207
211
|
try { existing = parseFrontmatter(readSource(target, targetPath)).data; } catch { /* unreadable: treated as foreign */ }
|
|
208
212
|
if (existing.promoted_from !== result.source || prior.includes(promotedTo)) return fail(issue("V08", `${target.id} already has ${targetPath}; move or delete it in the org vault first`, targetPath));
|
|
209
213
|
const set = { promoted_to: prior.length ? [...prior, promotedTo] : promotedTo };
|
|
210
|
-
const plan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env });
|
|
214
|
+
const plan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, binding: bindings.source });
|
|
211
215
|
const fingerprint = sha({ recover: plan.fingerprint, promotedTo });
|
|
212
216
|
const recovery: OrgPromoteResult = { ...result, target: `${target.id}:${targetPath}`, screen: { passed: true, issues: [] }, fingerprint, recovery: "provenance-only" };
|
|
213
217
|
if (!plan.ok) return { ...recovery, issues: plan.issues };
|
|
214
218
|
if (request.confirm !== true) return { ...recovery, ok: true, preview: true };
|
|
215
219
|
if (request.fingerprint !== fingerprint) return { ...recovery, issues: [issue("V09", "stale or missing preview fingerprint")] };
|
|
216
|
-
const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", log: false }, { env: options.env });
|
|
220
|
+
const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", log: false }, { env: options.env, binding: bindings.source });
|
|
217
221
|
return { ...recovery, ok: provenance.ok, applied: provenance.applied, provenance, issues: provenance.ok ? [] : provenance.issues };
|
|
218
222
|
}
|
|
219
223
|
const missingTopics = topics.filter(topic => !targetNotes.mocs.has(linkKey(topic)));
|
|
@@ -234,10 +238,10 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
|
|
|
234
238
|
const refusing = screen.filter(found => found.severity === "error");
|
|
235
239
|
result.screen = { passed: refusing.length === 0, issues: screen };
|
|
236
240
|
if (refusing.length) return { ...result, issues: screen };
|
|
237
|
-
const planned = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, piiSurface: "promotion" });
|
|
241
|
+
const planned = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, piiSurface: "promotion", binding: bindings.target });
|
|
238
242
|
const provenanceSet = { promoted_to: prior.length ? [...prior.filter(p => p !== promotedTo), promotedTo] : promotedTo };
|
|
239
243
|
const provenanceText = emitFrontmatter(parsed, provenanceSet);
|
|
240
|
-
const provenancePlan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env });
|
|
244
|
+
const provenancePlan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, binding: bindings.source });
|
|
241
245
|
result.fingerprint = sha({ target: planned.fingerprint, source: provenancePlan.fingerprint, text, provenanceText });
|
|
242
246
|
// Org lint runs on the preview too (the note overlaid on the org snapshot), so a note that would fail lint is never written.
|
|
243
247
|
const previewLint = lintPromoted(home, target, targetPath, text);
|
|
@@ -247,12 +251,13 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
|
|
|
247
251
|
if (request.confirm !== true) return { ...result, ok: true, preview: true, issues: [...planned.issues, ...provenancePlan.issues] };
|
|
248
252
|
if (request.fingerprint !== result.fingerprint) return { ...result, issues: [issue("V09", "stale or missing preview fingerprint")] };
|
|
249
253
|
const event = { type: "SYNC", project: request.logProject ?? "-", source: `${source.id}:${path}`, dest: `${target.id}:${targetPath}`, note: `promoted: ${typeof parsed.data.type === "string" ? parsed.data.type : "note"}` };
|
|
250
|
-
const note = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", log: false }, { env: options.env, piiSurface: "promotion" });
|
|
251
|
-
|
|
252
|
-
|
|
254
|
+
const note = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", log: false }, { env: options.env, piiSurface: "promotion", binding: bindings.target });
|
|
255
|
+
// A create that committed and then failed a later step (cleanup) is still a written org note.
|
|
256
|
+
if (!note.ok || !note.applied) return { ...result, applied: note.applied === true, note, issues: note.issues };
|
|
257
|
+
const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", log: false }, { env: options.env, binding: bindings.source });
|
|
253
258
|
const logs = { source: null as LogReceipt | null, target: null as LogReceipt | null };
|
|
254
259
|
for (const [key, vault] of [["target", target], ["source", source]] as const) {
|
|
255
|
-
try { logs[key] = appendLog(home, { vault: vault.id, event, channel: request.channel ?? "cli" }, { env: options.env }); }
|
|
260
|
+
try { logs[key] = appendLog(home, { vault: vault.id, event, channel: request.channel ?? "cli" }, { env: options.env, binding: bindings[key] }); }
|
|
256
261
|
catch { logs[key] = { ok: false, appended: null, synced: false, bytesWritten: null, issues: [issue("LOG_IO", "log append failed")] }; }
|
|
257
262
|
}
|
|
258
263
|
const lint = lintPromoted(home, target, targetPath, text);
|
package/src/registry.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { closeSync, constants, fsyncSync, mkdirSync, openSync, readdirSync, readFileSync, realpathSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
|
|
1
|
+
import { closeSync, constants, fsyncSync, mkdirSync, openSync, readdirSync, readFileSync, readSync, realpathSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
|
|
2
2
|
import { hostname } from "node:os";
|
|
3
3
|
import { join } from "node:path";
|
|
4
4
|
import { randomUUID } from "node:crypto";
|
|
@@ -50,7 +50,8 @@ function mutate<T>(home: string, update: (config: VmemConfig) => T): T {
|
|
|
50
50
|
try { writeFileSync(fd, text); fsyncSync(fd); } finally { closeSync(fd); }
|
|
51
51
|
renameSync(temporary, join(home, CONFIG_FILE));
|
|
52
52
|
temporary = undefined;
|
|
53
|
-
syncDirectory(home);
|
|
53
|
+
try { syncDirectory(home); }
|
|
54
|
+
catch (error) { throw new Error(`${join(home, CONFIG_FILE)}: registry updated but directory fsync failed: ${message(error)}`, { cause: error }); }
|
|
54
55
|
return result;
|
|
55
56
|
} catch (error) {
|
|
56
57
|
failed = true; primary = error;
|
|
@@ -73,9 +74,20 @@ function mutate<T>(home: string, update: (config: VmemConfig) => T): T {
|
|
|
73
74
|
}
|
|
74
75
|
}
|
|
75
76
|
}
|
|
77
|
+
/** Upper bound for a lock owner record; larger files are treated as unreadable. */
|
|
78
|
+
const MAX_LOCK_BYTES = 4096;
|
|
76
79
|
function lockOwner(lock: string): LockOwner | undefined {
|
|
77
80
|
try {
|
|
78
|
-
const
|
|
81
|
+
const fd = openSync(lock, constants.O_RDONLY);
|
|
82
|
+
let text: string;
|
|
83
|
+
try {
|
|
84
|
+
const buffer = Buffer.alloc(MAX_LOCK_BYTES + 1);
|
|
85
|
+
let length = 0, read: number;
|
|
86
|
+
while (length < buffer.length && (read = readSync(fd, buffer, length, buffer.length - length, null)) > 0) length += read;
|
|
87
|
+
if (length > MAX_LOCK_BYTES) return undefined; // oversized: never trusted
|
|
88
|
+
text = buffer.subarray(0, length).toString("utf8");
|
|
89
|
+
} finally { closeSync(fd); }
|
|
90
|
+
const value = JSON.parse(text) as Partial<LockOwner>;
|
|
79
91
|
if (typeof value.pid === "number" && Number.isInteger(value.pid) && value.pid > 0 && typeof value.host === "string" && typeof value.created === "string") return value as LockOwner;
|
|
80
92
|
} catch { /* unreadable or legacy lock */ }
|
|
81
93
|
return undefined;
|
package/src/sync.ts
CHANGED
|
@@ -52,6 +52,20 @@ function gitIn(cwd: string, env: NodeJS.ProcessEnv): Git {
|
|
|
52
52
|
return out;
|
|
53
53
|
};
|
|
54
54
|
}
|
|
55
|
+
/** Local-only origin validation shared by doctor and sync; never fetches or pushes. */
|
|
56
|
+
export function originCheck(root: string, remote: string, env: NodeJS.ProcessEnv = process.env): { ok: boolean; origin: string | null; fetchUrls: string[]; pushUrls: string[]; issue?: string } {
|
|
57
|
+
const git = gitIn(root, env);
|
|
58
|
+
if (realpathSync(git(["rev-parse", "--show-toplevel"]).stdout.trim()) !== root) throw new Error("vault path is not the root of its git repository");
|
|
59
|
+
const origin = git(["remote", "get-url", "origin"], { allowFail: true });
|
|
60
|
+
const originUrl = origin.status === 0 ? origin.stdout.trim() : null;
|
|
61
|
+
// Every effective fetch and push URL must be the configured remote; URL rewrites are refused outright
|
|
62
|
+
// because git does not report pushInsteadOf rewrites of an explicit pushurl.
|
|
63
|
+
const lines = (args: string[]) => { const out = git(args, { allowFail: true }); return out.status === 0 ? out.stdout.split("\n").map(line => line.trim()).filter(Boolean) : []; };
|
|
64
|
+
const fetchUrls = lines(["remote", "get-url", "--all", "origin"]), pushUrls = lines(["remote", "get-url", "--push", "--all", "origin"]);
|
|
65
|
+
const rewrites = lines(["config", "--get-regexp", "^url\\..*\\.(insteadof|pushinsteadof)$"]);
|
|
66
|
+
const ok = originUrl !== null && rewrites.length === 0 && fetchUrls.length > 0 && pushUrls.length > 0 && [...fetchUrls, ...pushUrls].every(url => sameRemote(url, remote));
|
|
67
|
+
return { ok, origin: originUrl, fetchUrls, pushUrls, ...(!ok ? { issue: originUrl === null ? "repository has no origin remote" : rewrites.length ? "git url insteadOf/pushInsteadOf rewrites are configured; refusing because the push destination cannot be verified" : "an origin fetch or push URL does not match the configured remote; nothing was fetched or pushed" } : {}) };
|
|
68
|
+
}
|
|
55
69
|
function planned(path: string): boolean { return STAGE_FILES.includes(path) || STAGE_PREFIXES.some(prefix => path.startsWith(prefix)); }
|
|
56
70
|
/** Parse `status --porcelain=v1 -z` into changed paths (renames contribute both sides). */
|
|
57
71
|
function changed(git: Git): string[] {
|
|
@@ -83,17 +97,13 @@ export function syncVault(home: string, request: { vault: string; confirm?: bool
|
|
|
83
97
|
if (isDenied(root, config.denyPaths)) return { ...result, issues: [issue("V06", "vault path is denied; nothing was synced")] };
|
|
84
98
|
const git = gitIn(root, options.env ?? process.env);
|
|
85
99
|
try {
|
|
86
|
-
|
|
87
|
-
const
|
|
88
|
-
result.remote.origin =
|
|
89
|
-
// Every effective fetch and push URL must be the configured remote; URL rewrites are refused outright
|
|
90
|
-
// because git does not report pushInsteadOf rewrites of an explicit pushurl.
|
|
91
|
-
const lines = (args: string[]) => { const out = git(args, { allowFail: true }); return out.status === 0 ? out.stdout.split("\n").map(line => line.trim()).filter(Boolean) : []; };
|
|
92
|
-
const fetchUrls = lines(["remote", "get-url", "--all", "origin"]), pushUrls = lines(["remote", "get-url", "--push", "--all", "origin"]);
|
|
93
|
-
const rewrites = lines(["config", "--get-regexp", "^url\\..*\\.(insteadof|pushinsteadof)$"]);
|
|
100
|
+
const checked = originCheck(root, spec.remote, options.env ?? process.env);
|
|
101
|
+
const { fetchUrls, pushUrls } = checked;
|
|
102
|
+
result.remote.origin = checked.origin;
|
|
94
103
|
result.remote.push = pushUrls;
|
|
95
|
-
result.remote.matches =
|
|
96
|
-
if (!
|
|
104
|
+
result.remote.matches = checked.ok;
|
|
105
|
+
if (!checked.ok) return { ...result, issues: [issue("REMOTE", checked.issue!)] };
|
|
106
|
+
const lines = (args: string[]) => { const out = git(args, { allowFail: true }); return out.status === 0 ? out.stdout.split("\n").map(line => line.trim()).filter(Boolean) : []; };
|
|
97
107
|
const branch = git(["symbolic-ref", "--quiet", "--short", "HEAD"], { allowFail: true });
|
|
98
108
|
if (branch.status !== 0) return { ...result, issues: [issue("SYNC", "detached HEAD; check out a branch first")] };
|
|
99
109
|
result.branch = branch.stdout.trim();
|
package/src/tidy.ts
CHANGED
|
@@ -9,7 +9,7 @@ import { extractLinks, normalizeTarget } from "./links.ts";
|
|
|
9
9
|
import { isDenied } from "./platform.ts";
|
|
10
10
|
import { loadRegistry, syncDirectory } from "./registry.ts";
|
|
11
11
|
import { normalizeSource, sourceReferences, sourceWorkspaceRoot, missingSourceWarnings } from "./source.ts";
|
|
12
|
-
import { appendLog, LOG_LOCK } from "./vault-log.ts";
|
|
12
|
+
import { appendLog, logBinding, LOG_LOCK } from "./vault-log.ts";
|
|
13
13
|
import { appendSection, foldConnections, h2Sections } from "./write-operations.ts";
|
|
14
14
|
import { SCHEMA_KEYS, SECRET_PATTERNS, TYPED_KEYS, screenContent } from "./write-validation.ts";
|
|
15
15
|
import { redactPii } from "./pii.ts";
|
|
@@ -300,8 +300,11 @@ function sanitizeLog(text: string, vault: VaultSpec, config: VmemConfig): { text
|
|
|
300
300
|
}).join("\n");
|
|
301
301
|
return { text: out, lines, redactions, formats, changes };
|
|
302
302
|
}
|
|
303
|
-
/**
|
|
304
|
-
|
|
303
|
+
/**
|
|
304
|
+
* Replace the log under its own lock, atomically (the one sanctioned in-place rewrite of the append-only log).
|
|
305
|
+
* `state.committed` turns true once the rename lands, so a caller can tell a later failure (fsync, lock removal) apart.
|
|
306
|
+
*/
|
|
307
|
+
function rewriteLog(root: string, expected: string, next: string, state: { committed: boolean }): void {
|
|
305
308
|
const lock = join(root, LOG_LOCK);
|
|
306
309
|
const fd = openSync(lock, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
|
|
307
310
|
let temp: string | undefined;
|
|
@@ -311,7 +314,7 @@ function rewriteLog(root: string, expected: string, next: string): void {
|
|
|
311
314
|
temp = join(root, `.vmem-log-${randomUUID()}.tmp`);
|
|
312
315
|
const tfd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
|
|
313
316
|
try { writeFileSync(tfd, next); fsyncSync(tfd); } finally { closeSync(tfd); }
|
|
314
|
-
renameSync(temp, join(root, "log")); temp = undefined;
|
|
317
|
+
renameSync(temp, join(root, "log")); temp = undefined; state.committed = true;
|
|
315
318
|
syncDirectory(root);
|
|
316
319
|
} finally {
|
|
317
320
|
if (temp) try { unlinkSync(temp); } catch { /* best effort */ }
|
|
@@ -334,6 +337,8 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
|
|
|
334
337
|
if (vault.write === "none" || (request.channel === "mcp" && vault.write !== "mcp")) return { ...result, issues: [issue("POLICY", "vault write policy refuses this channel")] };
|
|
335
338
|
const root = realpathSync(vault.path);
|
|
336
339
|
if (root !== vault.path || isDenied(root, config.denyPaths)) return { ...result, issues: [issue("V06", "vault path denied or changed")] };
|
|
340
|
+
// The LINT event must land in the vault this run read and wrote, even if the registry is changed meanwhile.
|
|
341
|
+
const binding = logBinding(vault, config);
|
|
337
342
|
const map = request.map ?? { enums: {}, keys: {} };
|
|
338
343
|
// Only flat notes/*.md are read: .obsidian/, templates/ and self/ are never touched.
|
|
339
344
|
let notes: Map<string, string>;
|
|
@@ -369,8 +374,9 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
|
|
|
369
374
|
// Validate every planned note with the writer (dry-run). A note the writer would refuse is reported and skipped
|
|
370
375
|
// (left fully unchanged for manual repair) instead of blocking the rest of the migration.
|
|
371
376
|
for (const change of [...planned]) {
|
|
372
|
-
const dry = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", dryRun: true, log: false }, { env: options.env });
|
|
377
|
+
const dry = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", dryRun: true, log: false }, { env: options.env, binding });
|
|
373
378
|
const errors = dry.issues.filter(found => found.severity === "error");
|
|
379
|
+
if (errors.some(found => found.code === "VAULT_DRIFT")) return { ...result, issues: errors.filter(found => found.code === "VAULT_DRIFT") };
|
|
374
380
|
if (!errors.length) continue;
|
|
375
381
|
planned.splice(planned.indexOf(change), 1);
|
|
376
382
|
result.reported.push({ path: change.path, message: `skipped: the writer refuses the sanitized note (${[...new Set(errors.map(found => found.code))].join(", ")}); repair by hand` });
|
|
@@ -385,18 +391,26 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
|
|
|
385
391
|
if (request.confirm !== true) return { ...result, ok: true, preview: true };
|
|
386
392
|
if (request.fingerprint !== result.fingerprint) return { ...result, issues: [issue("V09", "stale or missing preview fingerprint")] };
|
|
387
393
|
for (const change of planned) {
|
|
388
|
-
const receipt = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", expectedShas: { [change.path]: sha256(change.before) }, log: false }, { env: options.env });
|
|
389
|
-
|
|
394
|
+
const receipt = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", expectedShas: { [change.path]: sha256(change.before) }, log: false }, { env: options.env, binding });
|
|
395
|
+
// Notes written before a failure stay committed, so the result reports them as applied.
|
|
396
|
+
if (!receipt.ok || !receipt.applied) {
|
|
397
|
+
if (receipt.applied) result.written.push(change.path); // committed, then a later step of the write failed
|
|
398
|
+
return { ...result, applied: result.written.length > 0, issues: receipt.issues.length ? receipt.issues : [issue("IO", "note write failed", change.path)] };
|
|
399
|
+
}
|
|
390
400
|
result.written.push(change.path);
|
|
391
401
|
}
|
|
392
402
|
if (logPlan) {
|
|
393
|
-
if (isDenied(join(root, "log"), loadRegistry(home).denyPaths)) return { ...result, issues: [issue("V06", "the vault log is denied by the path policy; log not rewritten")] };
|
|
394
|
-
|
|
395
|
-
|
|
403
|
+
if (isDenied(join(root, "log"), loadRegistry(home).denyPaths)) return { ...result, applied: result.written.length > 0, issues: [issue("V06", "the vault log is denied by the path policy; log not rewritten")] };
|
|
404
|
+
const state = { committed: false };
|
|
405
|
+
try { rewriteLog(root, logPlan.before, logPlan.text, state); result.written.push("log"); }
|
|
406
|
+
catch (error) {
|
|
407
|
+
if (state.committed) result.written.push("log"); // the new log is in place; a later step failed
|
|
408
|
+
return { ...result, applied: result.written.length > 0, issues: [issue("LOG", error instanceof Error ? error.message.slice(0, 200) : "log rewrite failed")] };
|
|
409
|
+
}
|
|
396
410
|
}
|
|
397
411
|
const counted = Object.values(result.counts).reduce((sum, n) => sum + n, 0);
|
|
398
412
|
if (planned.length || logPlan) {
|
|
399
|
-
const receipt = appendLog(home, { vault: vault.id, event: { type: "LINT", project: "-", source: "vmem:tidy", note: result.event! }, channel: request.channel ?? "cli" }, { env: options.env });
|
|
413
|
+
const receipt = appendLog(home, { vault: vault.id, event: { type: "LINT", project: "-", source: "vmem:tidy", note: result.event! }, channel: request.channel ?? "cli" }, { env: options.env, binding });
|
|
400
414
|
if (!receipt.ok) return { ...result, applied: true, issues: receipt.issues };
|
|
401
415
|
}
|
|
402
416
|
return { ...result, ok: true, applied: result.written.length > 0 || counted > 0 };
|
package/src/vault-log.ts
CHANGED
|
@@ -7,6 +7,7 @@ import { canonicalPath, isDenied } from "./platform.ts";
|
|
|
7
7
|
import { formatLogEvent, LOG_TYPES } from "./log-format.ts";
|
|
8
8
|
import type { LogBinding, LogContext, LogReadOptions, LogReadResult, LogReceipt, LogRequest } from "./log-types.ts";
|
|
9
9
|
import type { WriteIssue } from "./write-types.ts";
|
|
10
|
+
import type { VaultSpec, VmemConfig } from "./config.ts";
|
|
10
11
|
export const LOG_LOCK = "log.lock";
|
|
11
12
|
export const MAX_LOG_READ_BYTES = 32 * 1024 * 1024;
|
|
12
13
|
export const LARGE_LOG_LINES = 20000;
|
|
@@ -30,7 +31,7 @@ function boundedPlain(file: string, maximum: number): string {
|
|
|
30
31
|
return text;
|
|
31
32
|
} finally { closeSync(fd); }
|
|
32
33
|
}
|
|
33
|
-
function registryIds(file: string, denyPaths: string[] = []): Set<string> {
|
|
34
|
+
export function registryIds(file: string, denyPaths: string[] = []): Set<string> {
|
|
34
35
|
if (isDenied(canonicalPath(file), denyPaths)) refuse("L09", "registry path is denied");
|
|
35
36
|
let parsed: unknown;
|
|
36
37
|
try { parsed = Bun.YAML.parse(boundedPlain(file, 1024 * 1024)); }
|
|
@@ -64,13 +65,32 @@ function context(home: string, id: string, channel: "cli" | "mcp" | undefined, w
|
|
|
64
65
|
return { vault, config, channel, ...(write && vault.registry ? { projectIds: registryIds(vault.registry, config.denyPaths) } : {}) };
|
|
65
66
|
}
|
|
66
67
|
/** One O_APPEND write per event. A separate cooperative lock permits safe short-write rollback. */
|
|
68
|
+
/** Bind a later append to the vault spec, path policy and root identity resolved now (refused with LOG_DRIFT if they change). */
|
|
69
|
+
export function logBinding(vault: VaultSpec, config: VmemConfig): LogBinding {
|
|
70
|
+
const identity = lstatSync(vault.path);
|
|
71
|
+
return { vault, denyPaths: config.denyPaths ?? [], piiPatterns: config.piiPatterns ?? [], dev: Number(identity.dev), ino: Number(identity.ino) };
|
|
72
|
+
}
|
|
73
|
+
/** True when the registered vault still equals the binding: spec, path policy and root identity. Never throws. */
|
|
74
|
+
export function matchesBinding(config: VmemConfig, id: string, binding: LogBinding): boolean {
|
|
75
|
+
const vault = config.vaults.find(v => v.id === id);
|
|
76
|
+
if (!vault || JSON.stringify(vault) !== JSON.stringify(binding.vault) || JSON.stringify(config.denyPaths ?? []) !== JSON.stringify(binding.denyPaths) || JSON.stringify(config.piiPatterns ?? []) !== JSON.stringify(binding.piiPatterns)) return false;
|
|
77
|
+
try { const identity = lstatSync(vault.path); return Number(identity.dev) === binding.dev && Number(identity.ino) === binding.ino; } catch { return false; }
|
|
78
|
+
}
|
|
67
79
|
export function appendLog(home: string, request: LogRequest, options: { env?: NodeJS.ProcessEnv; now?: Date; binding?: LogBinding } = {}): LogReceipt {
|
|
68
80
|
const result: LogReceipt = { ok: false, appended: false, synced: false, bytesWritten: 0, issues: [] };
|
|
69
81
|
let fd: number | undefined, lockFd: number | undefined, lockFile: string | undefined, identity: { ino: number; dev: number } | undefined;
|
|
70
82
|
const error = (code: string, message: string) => result.issues.push({ code, severity: "error", message });
|
|
71
83
|
try {
|
|
72
84
|
if (!request || typeof request !== "object" || typeof request.vault !== "string" || Object.keys(request).some(key => !["vault", "event", "channel"].includes(key))) refuse("L09", "invalid log request");
|
|
73
|
-
|
|
85
|
+
// A bound append compares the registry first, so any re-registration reads as drift rather than a policy or path error.
|
|
86
|
+
if (options.binding && !matchesBinding(loadRegistry(home), request.vault, options.binding)) refuse("LOG_DRIFT", "vault identity or policy changed after mutation; log not appended");
|
|
87
|
+
let ctx: ReturnType<typeof context>;
|
|
88
|
+
try { ctx = context(home, request.vault, request.channel, true); }
|
|
89
|
+
catch (caught) {
|
|
90
|
+
// A re-registration between the binding check and this second registry read still reads as drift.
|
|
91
|
+
if (options.binding && !matchesBinding(loadRegistry(home), request.vault, options.binding)) refuse("LOG_DRIFT", "vault identity or policy changed after mutation; log not appended");
|
|
92
|
+
throw caught;
|
|
93
|
+
}
|
|
74
94
|
const rootIdentity = lstatSync(ctx.vault.path);
|
|
75
95
|
if (options.binding) {
|
|
76
96
|
const bound = options.binding;
|
package/src/write-validation.ts
CHANGED
|
@@ -34,6 +34,7 @@ const SECRETS = [/-----BEGIN [A-Z ]*PRIVATE KEY-----/, /\bAKIA[0-9A-Z]{16}\b/, /
|
|
|
34
34
|
export const SECRET_PATTERNS: readonly RegExp[] = SECRETS;
|
|
35
35
|
export { EMAIL_PATTERN } from "./pii.ts";
|
|
36
36
|
const MAX_BYTES = 256 * 1024;
|
|
37
|
+
export const MAX_REPORT_BYTES = 16 * 1024 * 1024;
|
|
37
38
|
function issue(code: string, message: string, severity: WriteIssue["severity"] = "error", path?: string): WriteIssue {
|
|
38
39
|
return { code, severity, message, ...(path ? { path } : {}) };
|
|
39
40
|
}
|
|
@@ -209,7 +210,7 @@ export function validateWrite(changes: PlannedChange[], context: ValidationConte
|
|
|
209
210
|
if (change.after === null) final.delete(change.path);
|
|
210
211
|
else if (change.path.startsWith("notes/")) final.set(change.path, change.after);
|
|
211
212
|
if (change.after === null) continue;
|
|
212
|
-
if (Buffer.byteLength(change.after) > MAX_BYTES) issues.push(issue("V09", "prospective file exceeds 256 KiB", "error", change.path));
|
|
213
|
+
if (Buffer.byteLength(change.after) > (report ? MAX_REPORT_BYTES : MAX_BYTES)) issues.push(issue("V09", report ? "prospective report exceeds 16 MiB" : "prospective file exceeds 256 KiB", "error", change.path));
|
|
213
214
|
issues.push(...screenContent(change.after, context.vault, context.config, context.piiSurface).map(value => ({ ...value, path: change.path })));
|
|
214
215
|
try{issues.push(...screenMetadata(parseFrontmatter(change.after).data,context,change.path));}catch{ /* Invalid note YAML is reported separately; reports remain schema-exempt. */ }
|
|
215
216
|
}
|
package/src/writer.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { appendLog } from "./vault-log.ts";
|
|
1
|
+
import { appendLog, matchesBinding } from "./vault-log.ts";
|
|
2
2
|
import type { LogEvent } from "./log-types.ts";
|
|
3
3
|
import { closeSync, constants, fstatSync, fsyncSync, lstatSync, mkdirSync, openSync, readSync, readdirSync, realpathSync, renameSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs";
|
|
4
4
|
import { createHash, randomUUID } from "node:crypto";
|
|
@@ -10,8 +10,9 @@ import { VaultIndex, MAX_NOTE_BYTES } from "./vault-index.ts";
|
|
|
10
10
|
import { parseFrontmatter } from "./frontmatter.ts";
|
|
11
11
|
import { canonicalPath, isDenied } from "./platform.ts";
|
|
12
12
|
import { moveSourcePath, notePath, planOperation, refuse, WriteRefusal } from "./write-operations.ts";
|
|
13
|
-
import { validateWrite } from "./write-validation.ts";
|
|
13
|
+
import { MAX_REPORT_BYTES, validateWrite } from "./write-validation.ts";
|
|
14
14
|
import type { PiiSurface } from "./config.ts";
|
|
15
|
+
import type { LogBinding } from "./log-types.ts";
|
|
15
16
|
import type { ChangeReceipt, PlannedChange, WriteIssue, WriteReceipt, WriteRequest } from "./write-types.ts";
|
|
16
17
|
export const MUTATION_LOCK = "mutation.lock";
|
|
17
18
|
export const MAX_WRITE_BYTES = 256 * 1024;
|
|
@@ -40,22 +41,22 @@ function safePath(root: string, path: string): string {
|
|
|
40
41
|
function stamp(stat: { dev: bigint; ino: bigint; size: bigint; mtimeNs: bigint; ctimeNs: bigint }): string {
|
|
41
42
|
return `${stat.dev}:${stat.ino}:${stat.size}:${stat.mtimeNs}:${stat.ctimeNs}`;
|
|
42
43
|
}
|
|
43
|
-
function readPlain(root: string, path: string): string | null {
|
|
44
|
+
function readPlain(root: string, path: string, maximum = MAX_NOTE_BYTES): string | null {
|
|
44
45
|
const file = safePath(root, path);
|
|
45
46
|
let fd: number;
|
|
46
47
|
try { fd = openSync(file, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK); }
|
|
47
48
|
catch (error) { if ((error as NodeJS.ErrnoException).code === "ENOENT") return null; throw error; }
|
|
48
49
|
try {
|
|
49
50
|
const before = fstatSync(fd, { bigint: true });
|
|
50
|
-
if (!before.isFile() || before.size > BigInt(
|
|
51
|
-
const bytes = Buffer.alloc(Math.min(Number(before.size) + 1,
|
|
51
|
+
if (!before.isFile() || before.size > BigInt(maximum)) refuse("V09", "file exceeds read bound or is not regular", path);
|
|
52
|
+
const bytes = Buffer.alloc(Math.min(Number(before.size) + 1, maximum + 1));
|
|
52
53
|
let size = 0;
|
|
53
54
|
while (size < bytes.length) {
|
|
54
55
|
const count = readSync(fd, bytes, size, bytes.length - size, null); size += count;
|
|
55
56
|
if (!count) break;
|
|
56
57
|
}
|
|
57
58
|
const after = fstatSync(fd, { bigint: true });
|
|
58
|
-
if (size >
|
|
59
|
+
if (size > maximum || stamp(before) !== stamp(after)) refuse("V09", "file changed during read", path);
|
|
59
60
|
const buffer = bytes.subarray(0, size), text = buffer.toString("utf8");
|
|
60
61
|
if (!Buffer.from(text).equals(buffer)) refuse("V01", "note must be valid UTF-8", path);
|
|
61
62
|
return text;
|
|
@@ -112,7 +113,8 @@ function duplicateWarnings(request: WriteRequest, notes: Map<string, string>, ch
|
|
|
112
113
|
} finally { db.close(); }
|
|
113
114
|
}
|
|
114
115
|
/** Single mutation entry point. Preview/dry-run perform no writes, including locks and caches. */
|
|
115
|
-
|
|
116
|
+
/** `options.binding` (internal callers composing several writes) refuses with VAULT_DRIFT unless the registry still matches it. */
|
|
117
|
+
export function applyWrite(home: string, request: WriteRequest, options: { env?: NodeJS.ProcessEnv; piiSurface?: PiiSurface; binding?: LogBinding } = {}): WriteReceipt {
|
|
116
118
|
let lockFd: number | undefined, lockFile: string | undefined, lockIdentity: { ino: number; dev: number } | undefined;
|
|
117
119
|
let index: VaultIndex | undefined;
|
|
118
120
|
const temps = new Set<string>(), createdDirs: string[] = [], committed: PlannedChange[] = [];
|
|
@@ -125,6 +127,7 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
|
|
|
125
127
|
if (request.expectedShas !== undefined && (!request.expectedShas || typeof request.expectedShas !== "object" || Array.isArray(request.expectedShas))) refuse("V09", "expectedShas must be a mapping");
|
|
126
128
|
if (request.channel !== undefined && !["cli", "mcp"].includes(request.channel)) refuse("POLICY", "unknown write channel");
|
|
127
129
|
const config = loadRegistry(home), vault = config.vaults.find(value => value.id === request.vault);
|
|
130
|
+
if (options.binding && !matchesBinding(config, request.vault, options.binding)) refuse("VAULT_DRIFT", "vault identity or policy changed since the run started; nothing written");
|
|
128
131
|
if (!vault) refuse("V04", "unknown vault id");
|
|
129
132
|
if (request.operation.op !== "check" && (vault.write === "none" || (request.channel === "mcp" && vault.write !== "mcp"))) refuse("POLICY", "vault write policy refuses this channel");
|
|
130
133
|
root = realpathSync(vault.path);
|
|
@@ -147,7 +150,7 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
|
|
|
147
150
|
// Planning validates the report's exact relative path before it can be read.
|
|
148
151
|
planOperation(request.operation, notes, null);
|
|
149
152
|
if (isDenied(join(root, request.operation.path), config.denyPaths)) refuse("V06", "report output path is denied");
|
|
150
|
-
reportBefore = readPlain(root, request.operation.path);
|
|
153
|
+
reportBefore = readPlain(root, request.operation.path, MAX_REPORT_BYTES);
|
|
151
154
|
}
|
|
152
155
|
changes = planOperation(request.operation, notes, reportBefore);
|
|
153
156
|
for (const change of changes) if (isDenied(join(root, change.path), config.denyPaths)) refuse("V06", "planned mutation path is denied", change.path);
|
|
@@ -155,6 +158,8 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
|
|
|
155
158
|
if (!issues.some(issue => issue.severity === "error")) issues.push(...duplicateWarnings(request, notes, changes));
|
|
156
159
|
const input = { vault: request.vault, channel: request.channel ?? "cli", operation: request.operation, expectedShas: request.expectedShas ?? {}, log: request.log ?? null };
|
|
157
160
|
const identity = lstatSync(root);
|
|
161
|
+
// The identity every later commit check uses must still be the bound root, not a directory swapped in meanwhile.
|
|
162
|
+
if (options.binding && (Number(identity.dev) !== options.binding.dev || Number(identity.ino) !== options.binding.ino)) refuse("VAULT_DRIFT", "vault root changed since the run started; nothing written");
|
|
158
163
|
fingerprint = sha256(canonical({ input, vault, policy: { denyPaths: config.denyPaths ?? [], piiPatterns: config.piiPatterns ?? [], piiSurface: options.piiSurface ?? "notes" }, root, identity: [identity.dev, identity.ino], files: [...notes].map(([path, text]) => [path, sha256(text)]), changes: receipts(changes) }));
|
|
159
164
|
if (twoStep && request.confirm === true && (typeof request.fingerprint !== "string" || request.fingerprint !== fingerprint)) refuse("V09", "stale or missing preview fingerprint");
|
|
160
165
|
if (issues.some(issue => issue.severity === "error")) return result = { ok: false, applied: false, issues, changes: receipts(changes), fingerprint, indexUpdated: false };
|
|
@@ -191,7 +196,7 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
|
|
|
191
196
|
if (current.size !== notes.size || [...current].some(([path, text]) => notes.get(path) !== text)) refuse("V09", "vault changed while staging");
|
|
192
197
|
for (const change of changes) {
|
|
193
198
|
const file = safePath(root, change.path);
|
|
194
|
-
if (readPlain(root, change.path) !== change.before) refuse("V09", "file changed while staging", change.path);
|
|
199
|
+
if (readPlain(root, change.path, request.operation.op === "report" ? MAX_REPORT_BYTES : MAX_NOTE_BYTES) !== change.before) refuse("V09", "file changed while staging", change.path);
|
|
195
200
|
if (change.after === null) unlinkSync(file);
|
|
196
201
|
else { const temp = staged.get(change.path)!; renameSync(temp, file); temps.delete(temp); }
|
|
197
202
|
committed.push(change);
|
|
@@ -204,7 +209,7 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
|
|
|
204
209
|
for (const change of [...committed].reverse()) {
|
|
205
210
|
try {
|
|
206
211
|
const file = safePath(root, change.path);
|
|
207
|
-
if (readPlain(root, change.path) !== change.after) throw new Error("external write detected before rollback");
|
|
212
|
+
if (readPlain(root, change.path, request.operation.op === "report" ? MAX_REPORT_BYTES : MAX_NOTE_BYTES) !== change.after) throw new Error("external write detected before rollback");
|
|
208
213
|
if (change.before === null) unlinkSync(file);
|
|
209
214
|
else { const backup = backups.get(change.path)!; renameSync(backup, file); temps.delete(backup); }
|
|
210
215
|
} catch {
|