@staix/vault-mem 0.1.3 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (16) hide show
  1. package/CHANGELOG.md +13 -0
  2. package/README.md +47 -6
  3. package/package.json +1 -1
  4. package/src/cli.ts +14 -4
  5. package/src/commands-table.ts +1 -0
  6. package/src/config.ts +44 -5
  7. package/src/log-format.ts +4 -2
  8. package/src/mcp-vault.ts +1 -1
  9. package/src/org-promote.ts +20 -13
  10. package/src/pii.ts +133 -0
  11. package/src/registry.ts +20 -4
  12. package/src/tidy.ts +31 -15
  13. package/src/vault-log.ts +22 -1
  14. package/src/write-types.ts +3 -0
  15. package/src/write-validation.ts +9 -40
  16. package/src/writer.ts +10 -4
package/CHANGELOG.md CHANGED
@@ -1,5 +1,18 @@
1
1
  # Changelog
2
2
 
3
+ ## [0.2.1] - 2026-10-30
4
+
5
+ - The `tidy` LINT event and the org-promotion SYNC events are bound to the vaults resolved at the start of the run; if a vault id is re-registered before the append, the event is refused with LOG_DRIFT instead of landing in the other vault. The committed notes are not rolled back.
6
+ - Tidy and org promotion bind every note write to the same vaults: a re-registration before a write is refused with VAULT_DRIFT and nothing is written. A bound log append reports any re-registration as LOG_DRIFT, before policy or path checks.
7
+ - Org promotion and `tidy --log` report `applied: true` when a write committed and a later step (lock removal, directory flush) failed; tidy then lists the log in `written`.
8
+
9
+ ## [0.2.0] - 2026-10-30
10
+
11
+ - Fine-grained PII policy: `vault.pii` can set `allow`, `warn` or `deny` per category (email, phone, the global `piiPatterns`, named per-vault patterns) and per surface (notes, log, promotion), with an allow list of exact addresses, domains and phone numbers. The `allow`/`deny` shorthand keeps its previous meaning.
12
+ - `vmem vaults pii <id>` shows the effective policy; `--set` previews a change and lists every loosened cell, and `--confirm` applies it. `vaults list` and `vmem_status` report the matrix as `piiPolicy`.
13
+ - `vmem tidy --log` redacts only PII that the policy denies on the log surface, so personal vaults keep historical email addresses; secrets are still redacted everywhere.
14
+ - Org promotion reports a `warn` cell in the target policy as a warning instead of refusing.
15
+
3
16
  ## [0.1.3] - 2026-10-30
4
17
 
5
18
  - `vmem tidy --log` fixes legacy log lines that fail lint L09: a trailing seconds field is dropped, and any other malformed line is kept verbatim behind `# legacy: `.
package/README.md CHANGED
@@ -31,10 +31,51 @@ Durability: the temporary file is fsynced before the rename and the home directo
31
31
 
32
32
  Crash recovery: the lock file records the writer's pid, host and start time. Nothing is cleaned up automatically. `vmem recover` previews the state; `vmem recover --confirm` removes the lock only when it names this host and a pid that no longer runs, and then removes leftover `.config-<uuid>.tmp` files. A live lock, a lock from another host, or an unreadable (older-format) lock is never removed and the command exits 1; confirm no writer runs, then delete `config.lock` by hand. No other file in the home or any vault is touched. If cleanup fails after an operation (for example the lock cannot be removed), the original error is kept and the cleanup failure, including a retained lock, is appended to it.
33
33
 
34
- Vault fields: `id`, absolute `path` (must contain a real `notes/` directory), `kind` (`personal|org`), `write` (`mcp|cli|none`), optional `pii` (`allow|deny`; default `allow` for personal and `deny` for org, and `status`/`vaults list` always report the effective value, including for unhealthy entries), optional `remote` (any nonempty string such as an `https://` or `git@host:path` URL; stored only) and optional `registry` (absolute path; stored only). No sync exists yet, so `remote` and `registry` are not validated against a server and doctor does not check them; reachability checks apply once sync is implemented. Policy fields are stored for later note-writing stages, not enforced by the read commands. Writer commands enforce write/channel policy and configured deny/PII screening.
34
+ Vault fields: `id`, absolute `path` (must contain a real `notes/` directory), `kind` (`personal|org`), `write` (`mcp|cli|none`), optional `pii` (the `allow|deny` shorthand or a policy object, see PII policy below; default `allow` for personal and `deny` for org, and `status`/`vaults list` always report the effective shorthand and `piiPolicy` matrix, including for unhealthy entries), optional `remote` (any nonempty string such as an `https://` or `git@host:path` URL; stored only) and optional `registry` (absolute path; stored only). No sync exists yet, so `remote` and `registry` are not validated against a server and doctor does not check them; reachability checks apply once sync is implemented. Policy fields are stored for later note-writing stages, not enforced by the read commands. Writer commands enforce write/channel policy and configured deny/PII screening.
35
35
 
36
36
  The only runtime dependency is the exact first-party pin `@staix/agent-mem@0.11.0`; it is not loaded by registry commands. Public library exports cover configuration, home resolution, registry operations, frontmatter, link resolution and the derived index. No console bin or UI contract is advertised before its implementation.
37
37
 
38
+ # PII policy
39
+
40
+ `pii` is either the shorthand `allow`/`deny` (every category on every surface) or an object that sets an action per category and, optionally, per surface:
41
+
42
+ ```json
43
+ {"email": {"notes": "allow", "log": "deny"}, "phone": "warn", "custom": "deny",
44
+ "patterns": [{"name": "student-id", "regex": "S\\d{6}", "action": {"notes": "deny", "log": "allow"}}],
45
+ "allow": ["owner@example.org", "@example.org", "+82 10 0000 0000"]}
46
+ ```
47
+
48
+ - Categories:
49
+ - `email`;
50
+ - `phone`;
51
+ - `custom`, the global `piiPatterns`;
52
+ - named per-vault `patterns` (at most 32; names `[a-z][a-z0-9-]{0,31}`, regexes compiled with the `u` flag).
53
+ - Surfaces:
54
+ - `notes`: note and report writes, frontmatter screening, tidy value echo;
55
+ - `log`: log appends and `tidy --log` on existing lines;
56
+ - `promotion`: the screen `vmem promote` applies when this vault is the target.
57
+ - Actions:
58
+ - `allow`: not screened.
59
+ - `warn`: the write proceeds with a V07 warning, and tidy echo prints the value as `[redacted]`.
60
+ - `deny`: V07 error, and `tidy --log` replaces matches in existing lines.
61
+ - Org promotion refuses only `deny` cells and secrets. The promoted note's write into the target vault is screened with the target's `promotion` cells, not its `notes` cells.
62
+ - `tidy --log` replaces the raw denied matches and leaves allow-listed values intact. Screening also checks the NFKC form. If a denied match is zero-length, or the replaced line still contains a denied match (for example a form only NFKC normalization reveals, or a pattern that matches the `[redacted]` marker), the whole line is redacted (fail closed).
63
+ - An omitted category or surface takes the vault default: `allow` for personal vaults, `deny` for org vaults.
64
+ - `allow` entries (at most 64) are exempt everywhere:
65
+ - an exact address, compared case-insensitively;
66
+ - an `@domain` suffix;
67
+ - a phone number standing alone, compared by its digits. Several numbers run together with spaces are never exempt (fail closed);
68
+ - an exact match of a pattern.
69
+ - Secret detection is separate from this policy and always applies.
70
+
71
+ ```sh
72
+ vmem vaults pii personal # effective matrix and allow list
73
+ vmem vaults pii personal --set '{"email":{"log":"deny"}}' # preview: changed cells and every loosened cell
74
+ vmem vaults pii personal --set '{"email":{"log":"deny"}}' --confirm
75
+ ```
76
+
77
+ The policy is changed only through the CLI, under the config lock. MCP clients can read it in `vmem_status` but cannot set it.
78
+
38
79
  # Indexed reads
39
80
 
40
81
  ```sh
@@ -99,7 +140,7 @@ Every MCP log append and CLI call with a nonempty `AGENTHUB_PEER_ID`, `CLAUDECOD
99
140
 
100
141
  `appendLog` validates the registered canonical vault and write/channel policy, acquires a separate `log.lock` (bounded two-second contention wait), then makes one O_APPEND write and fsyncs the log. New log files are mode 0600 and their parent directory is flushed. Linked/nonregular files and an incomplete existing final line are refused. Cooperating appenders share the lock; a short write is truncated back only when the same file still has the expected size, and its tail still equals our exact prefix, then flushed. Error-after-write cases inspect at most one event's bytes. Failed/ambiguous writes or cleanup are explicit; the receipt reports `appended` (possibly null when unknown), `synced`, `bytesWritten` (possibly null) and `rolledBack` when a partial append was removed. Before reporting sync, the current log and vault identities are rechecked; rotation produces LOG_DRIFT and an unknown current-path append state. A synced event can coexist with a cleanup error. No existing event is rewritten. A crash can retain `log.lock`, and failed short-write recovery can retain an incomplete line; inspect the file and stopped writer before manual recovery. There is no automatic cleanup or new TTY requirement.
101
142
 
102
- Successful changed mutations default to EXTRACT, link to CONNECT, move to ROUTE, and supersede/tidy to RETHINK. Default events use project `-`, affected paths and the operation name rather than note text. Library callers can supply `WriteRequest.log` to override the event or `false` to disable it; CLI writers accept `--log JSON` for overrides. Preview/check/no-op/refused mutations never append. The append is bound to the mutation's resolved vault identity/policy, so re-registration cannot route its event to another vault. After a committed mutation, log append is independent: `ok`/`applied` describe the note operation, `log` holds its append receipt, and `logged` is true only when the event was appended and synced. A log error does not roll back a successfully saved note. CLI exits 1 for a log failure even when the note receipt says `ok: true`; inspect the receipt before retrying to avoid duplicate mutations/events. An override is validated at append time, not during a note-only dry run.
143
+ Successful changed mutations default to EXTRACT, link to CONNECT, move to ROUTE, and supersede/tidy to RETHINK. Default events use project `-`, affected paths and the operation name rather than note text. Library callers can supply `WriteRequest.log` to override the event or `false` to disable it; CLI writers accept `--log JSON` for overrides. Preview/check/no-op/refused mutations never append. The append is bound to the mutation's resolved vault identity/policy, so re-registration cannot route its event to another vault; the same binding covers the tidy LINT event and both SYNC events of org promotion, which are refused with LOG_DRIFT instead. A bound append compares the registry before any policy or path check, so a re-registration always reads as LOG_DRIFT. Tidy and org promotion also bind every note write to the vaults resolved at the start of the run: if a vault id is re-registered before a write, that write is refused with VAULT_DRIFT and nothing is written to either vault. After a committed mutation, log append is independent: `ok`/`applied` describe the note operation, `log` holds its append receipt, and `logged` is true only when the event was appended and synced. A log error does not roll back a successfully saved note. CLI exits 1 for a log failure even when the note receipt says `ok: true`; inspect the receipt before retrying to avoid duplicate mutations/events. An override is validated at append time, not during a note-only dry run.
103
144
 
104
145
  Readers never create a home, log or lock, and work with `write: none`. They skip blank/comment lines, return the last matching 1..200 records (default 50), accept an optional canonical TYPE and non-backtracking regex subset up to 256 characters (`|`, `^`, `$`, `.`, character classes, `\d`/`\s`/`\w` and their inverses, escaped literals, and atom `*`/`+`/`?`; groups, backreferences and brace quantifiers are refused), and scan a fixed descriptor-size snapshot. Streaming is capped at 32 MiB; individual historical lines over 4096 bytes and incomplete final lines are omitted with warnings, and the serialized line array stays within 128 KiB. Above 20000 data lines, the manual-archival warning is preserved. Read failure returns `dataLines: null` rather than claiming a count. Golden synthetic fixtures and concurrent/fault persistence tests cover the format and actual append state; no live vault log was read or modified for this increment.
105
146
 
@@ -111,7 +152,7 @@ The vault set is read once at startup: every tool's `vault` property is an enum
111
152
 
112
153
  | tool | arguments (caps) |
113
154
  |---|---|
114
- | `vmem_status` | none; vaults with health, write policy and effective pii |
155
+ | `vmem_status` | none; vaults with health, write policy, effective pii and `piiPolicy` (read-only) |
115
156
  | `vmem_search` | `query`<=1000, `vault`, `limit`<=50, `type`, `domain` |
116
157
  | `vmem_read` | `path` (stem, alias or `notes/` path), `vault`, `maxBytes`<=65536; returns `{path, frontmatter, body, sha256, truncated}` |
117
158
  | `vmem_read_many` | `paths`<=20 (8 KiB each), `vault` |
@@ -192,7 +233,7 @@ vmem sync team --confirm --fingerprint <sha>
192
233
  `vmem promote` copies one note from a personal vault into the local clone of a writable org vault (`kind: org`, `write` not `none`):
193
234
 
194
235
  - **Transform.** The last prose `## Topics` section (never one inside a code fence) is replaced with the target topics, which must name MOCs that already exist in the org vault (source topics by default, `--topics` to override); a missing section is appended. Body wiki links that do not resolve in the org vault become plain text (their label, if any); wiki-valued frontmatter (`project`, `supersedes`, `superseded_by`, `source_ref`) pointing outside it is dropped. All of this is listed in `droppedLinks`. The org note gets `promoted_from: "<personal>:notes/x.md"` and `promoted_at`.
195
- - **Screen (fail closed).** It refuses: secrets; PII under the org vault's policy (org defaults to `deny`, plus configured `piiPatterns`); `type: person` or `domain: people` records; private correspondence (`source`/`source_ref` from `inbox/`, `outlook:`, `telegram(`, `kakao`); and internal meeting notes (`source` under `meetings/`). These are keyword heuristics; the org clone's `git diff` before `vmem sync` is the review surface. Body links that are dropped are also resolved in the personal vault: a link to a person/people, meeting or correspondence note refuses the promotion (reporting only the category and count), so their names never reach the org vault. A refusal or a preview writes nothing in either vault.
236
+ - **Screen (fail closed).** It refuses: secrets; PII denied by the org vault's `promotion` cells (org defaults to `deny`, plus configured `piiPatterns`; a `warn` cell is reported, not refused); `type: person` or `domain: people` records; private correspondence (`source`/`source_ref` from `inbox/`, `outlook:`, `telegram(`, `kakao`); and internal meeting notes (`source` under `meetings/`). These are keyword heuristics; the org clone's `git diff` before `vmem sync` is the review surface. Body links that are dropped are also resolved in the personal vault: a link to a person/people, meeting or correspondence note refuses the promotion (reporting only the category and count), so their names never reach the org vault. A refusal or a preview writes nothing in either vault.
196
237
  - **Confirm.** It requires the current fingerprint. Then it creates the org note through `applyWrite`, records `promoted_to: "<org>:notes/x.md"` on the source, and appends `SYNC <project> personal:notes/x.md → <org>:notes/x.md <em-dash> promoted: <type>` to both vault logs (`--log-project` sets the project, default `-`). The note is also linted against the org vault (`lintVault` over the org snapshot with the note overlaid, scoped to that note): any L01/L02/L13 error refuses the preview, and the lint result is returned again after the write. Vault-wide warnings do not block. A second promotion of the same note needs `--again`, and an existing org note is never overwritten. If the source provenance write failed after the org note was created, a rerun detects the org note's matching `promoted_from` and offers a provenance-only preview and confirm.
197
238
  - **MCP.** `vmem_promote` does the same over MCP and needs `write: mcp` on both vaults.
198
239
 
@@ -234,7 +275,7 @@ Runtime changes intentionally make `release:verify` report stale historical rele
234
275
 
235
276
  - **Two steps.** Without `--confirm` it prints per-rule `counts`, a per-note change list (`key: old -> new`, dropped keys with their values, sections added) and `reported` items that were left unchanged; nothing is written. With `--confirm` the fingerprint must match: it covers the rules, the map file, and the before/after sha of every touched file (and of the log). Each note is then written through `applyWrite` (atomic per note, under the vault lock) and one `LINT` event is appended, for example `tidy: escapes 1, enums 13, keys 78, ...`.
236
277
  - **Never blocked by one note.** A sanitized note that the writer would still refuse (for example an unknown key) is left unchanged, listed under `skipped` and `reported`, and the rest of the run proceeds. Counts and the `LINT` event cover only notes that are written.
237
- - **No value leaks.** Every value echoed into a change line or report is screened: secret-looking values, and email/phone values in `pii: deny` vaults, print as `[redacted]`.
278
+ - **No value leaks.** Every value echoed into a change line or report is screened: secret-looking values, and PII values that the `notes` surface warns on or denies, print as `[redacted]`.
238
279
  - **Restart-safe.** If a run is interrupted, every note is either fully old or fully new; a rerun previews only the rest. A second `--all` run previews zero changes.
239
280
  - **Scope.** Only flat `notes/*.md` and, for `--log`, the vault `log` are touched; `.obsidian/`, `templates/` and `self/` never are. Nothing is committed; commit the receipt paths yourself.
240
281
 
@@ -247,6 +288,6 @@ Runtime changes intentionally make `release:verify` report stale historical rele
247
288
  | `--topics` | append `## Topics` listing the frontmatter topics where no such heading exists; mismatched sections are reported |
248
289
  | `--connections` | fold `## Connections (...)` headings into `## Relevant Notes` |
249
290
  | `--secrets` | replace V06 secret matches with `[redacted]`: in the body the match itself, in frontmatter the whole field value (the key stays, so the YAML stays valid). A match outside a field value, such as a YAML comment, is reported for manual repair. The preview names note and line, never the value. PII in personal vaults is left alone |
250
- | `--log` | replace secret-looking substrings and email addresses in existing log lines with `[redacted]`, under the log lock (refused with V06 when `denyPaths` covers the log). A line that fails the lint L09 structure check loses a trailing seconds field when that makes it valid, and is otherwise kept verbatim behind `# legacy: ` (readers skip `#` lines); no time, zone or TYPE is guessed. Other lines are kept byte for byte. Git history is not rewritten: rotate exposed credentials |
291
+ | `--log` | replace secret-looking substrings in existing log lines with `[redacted]`, and PII values that the vault policy denies on the `log` surface (allow-list entries excepted), under the log lock (refused with V06 when `denyPaths` covers the log). A line that fails the lint L09 structure check loses a trailing seconds field when that makes it valid, and is otherwise kept verbatim behind `# legacy: ` (readers skip `#` lines); no time, zone or TYPE is guessed. Other lines are kept byte for byte. Git history is not rewritten: rotate exposed credentials |
251
292
 
252
293
  `--map <file>` overrides defaults with strict JSON `{"enums": {"confidence|status|type": {"from": "to"}}, "keys": {"key": "drop" | "keep" | {"rename": "key"}}}` (at most 64 KiB and 1000 entries). For `tidy`, `--log` selects the log rule; tidy always writes its own `LINT` event.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@staix/vault-mem",
3
- "version": "0.1.3",
3
+ "version": "0.2.1",
4
4
  "description": "Validated Obsidian-compatible knowledge vaults for agents and people",
5
5
  "license": "MIT",
6
6
  "type": "module",
package/src/cli.ts CHANGED
@@ -7,7 +7,8 @@ import { version } from "../package.json";
7
7
  import { completionScript, help } from "./commands-table.ts";
8
8
  import { runSetup, type SetupMode } from "./setup.ts";
9
9
  import { resolveHome } from "./home.ts";
10
- import { addVault, effectivePii, loadRegistry, recoverRegistry, registryStatus, removeVault } from "./registry.ts";
10
+ import { addVault, effectivePii, loadRegistry, recoverRegistry, registryStatus, removeVault, setVaultPii } from "./registry.ts";
11
+ import { piiPolicy } from "./pii.ts";
11
12
  function parse(args: string[], allowed: string[], values: string[]) {
12
13
  const flags: Record<string, string | boolean> = {}, positional: string[] = [];
13
14
  for (let i = 0; i < args.length; i++) {
@@ -43,8 +44,8 @@ export function run(args: string[], env: NodeJS.ProcessEnv = process.env): { tex
43
44
  if (isReadCommand(args[0]!)) return runReadCommand(args[0]!, args.slice(1), resolveHome(env).home);
44
45
  const command = args[0] === "vaults" ? `vaults ${args[1] ?? ""}` : args[0];
45
46
  const tail = args.slice(args[0] === "vaults" ? 2 : 1);
46
- const options = command === "vaults add" ? ["--kind", "--write", "--pii", "--remote", "--registry"] : [];
47
- const { flags, positional } = parse(tail, ["--json", ...options, ...(command === "vaults remove" || command === "recover" ? ["--confirm"] : [])], options);
47
+ const options = command === "vaults add" ? ["--kind", "--write", "--pii", "--remote", "--registry"] : command === "vaults pii" ? ["--set"] : [];
48
+ const { flags, positional } = parse(tail, ["--json", ...options, ...(command === "vaults remove" || command === "vaults pii" || command === "recover" ? ["--confirm"] : [])], options);
48
49
  const home = resolveHome(env);
49
50
  let result: unknown, code = 0;
50
51
  switch (command) {
@@ -62,7 +63,7 @@ export function run(args: string[], env: NodeJS.ProcessEnv = process.env): { tex
62
63
  break;
63
64
  case "vaults list":
64
65
  if (positional.length) throw new Error("vaults list: unexpected arguments");
65
- result = loadRegistry(home.home).vaults.map(v => ({ ...v, pii: effectivePii(v) }));
66
+ result = loadRegistry(home.home).vaults.map(v => ({ ...v, pii: effectivePii(v), piiPolicy: piiPolicy(v) }));
66
67
  break;
67
68
  case "vaults add": {
68
69
  if (positional.length !== 2) throw new Error("vaults add: requires id and absolute path");
@@ -71,6 +72,15 @@ export function run(args: string[], env: NodeJS.ProcessEnv = process.env): { tex
71
72
  result = { applied: true, vault: addVault(home.home, value) };
72
73
  break;
73
74
  }
75
+ case "vaults pii": {
76
+ if (positional.length !== 1) throw new Error("vaults pii: requires id");
77
+ const raw = flags["--set"];
78
+ if (raw === undefined && flags["--confirm"] === true) throw new Error("vaults pii: --confirm requires --set");
79
+ let value: unknown;
80
+ if (typeof raw === "string") { if (raw === "allow" || raw === "deny") value = raw; else { try { value = JSON.parse(raw); } catch { throw new Error("vaults pii: --set expects allow, deny or a JSON policy object"); } } }
81
+ result = setVaultPii(home.home, positional[0]!, value, flags["--confirm"] === true);
82
+ break;
83
+ }
74
84
  case "vaults remove":
75
85
  if (positional.length !== 1) throw new Error("vaults remove: requires id");
76
86
  result = removeVault(home.home, positional[0]!, flags["--confirm"] === true);
package/src/commands-table.ts CHANGED
@@ -12,6 +12,7 @@ export const COMMANDS = [
12
12
  { name: "vaults list", usage: "vaults list [--json]", mutates: false },
13
13
  { name: "vaults add", usage: "vaults add <id> <absolute-path> --kind personal|org --write mcp|cli|none [--pii allow|deny] [--remote URL] [--registry PATH] [--json]", mutates: true },
14
14
  { name: "vaults remove", usage: "vaults remove <id> [--confirm] [--json]", mutates: "confirm" },
15
+ { name: "vaults pii", usage: "vaults pii <id> [--set allow|deny|JSON] [--confirm] [--json]", mutates: "confirm" },
15
16
  { name: "new", usage: "new <vault> <stem> --type TYPE --domain DOMAIN --topics a,b --description TEXT [--title TEXT] [--source TEXT|JSON] [--confidence TEXT] [--status TEXT] [--body file|-] [--extra JSON] [--dry-run] [--json]", mutates: true },
16
17
  { name: "patch", usage: "patch <vault> <note> --old TEXT --new TEXT [--expected-sha SHA] [--dry-run] [--json]", mutates: true },
17
18
  { name: "append", usage: "append <vault> <note> --section SECTION --markdown TEXT [--expected-sha SHA] [--dry-run] [--json]", mutates: true },
package/src/config.ts CHANGED
@@ -8,10 +8,17 @@ export interface VaultSpec {
8
8
  path: string;
9
9
  kind: "personal" | "org";
10
10
  write: "mcp" | "cli" | "none";
11
- pii?: "allow" | "deny";
11
+ pii?: "allow" | "deny" | PiiSpec;
12
12
  remote?: string;
13
13
  registry?: string;
14
14
  }
15
+ export type PiiAction = "allow" | "warn" | "deny";
16
+ export const PII_SURFACES = ["notes", "log", "promotion"] as const;
17
+ export type PiiSurface = typeof PII_SURFACES[number];
18
+ export type PiiCell = PiiAction | Partial<Record<PiiSurface, PiiAction>>;
19
+ export interface PiiPattern { name: string; regex: string; action?: PiiCell }
20
+ /** Fine-grained PII policy: per category, optionally per surface; omitted cells take the vault default. */
21
+ export interface PiiSpec { email?: PiiCell; phone?: PiiCell; custom?: PiiCell; patterns?: PiiPattern[]; allow?: string[] }
15
22
  export interface VmemConfig { home?: string; vaults: VaultSpec[]; denyPaths?: string[]; piiPatterns?: string[] }
16
23
  const controls = /[\x00-\x1f\x7f]/;
17
24
  function object(value: unknown, keys: string[], label: string): Record<string, unknown> {
@@ -38,6 +45,41 @@ function strings(value: unknown, label: string): string[] | undefined {
38
45
  return text;
39
46
  });
40
47
  }
48
+ const ACTIONS = new Set(["allow", "warn", "deny"]);
49
+ function cell(value: unknown, label: string): PiiCell {
50
+ if (typeof value === "string") {
51
+ if (!ACTIONS.has(value)) throw new Error(`${label}: expected allow, warn or deny`);
52
+ return value as PiiAction;
53
+ }
54
+ const record = object(value, [...PII_SURFACES], label), result: Partial<Record<PiiSurface, PiiAction>> = {};
55
+ for (const surface of PII_SURFACES) if (record[surface] !== undefined) {
56
+ if (typeof record[surface] !== "string" || !ACTIONS.has(record[surface] as string)) throw new Error(`${label}.${surface}: expected allow, warn or deny`);
57
+ result[surface] = record[surface] as PiiAction;
58
+ }
59
+ return result;
60
+ }
61
+ /** Validate `vault.pii`: the `allow`/`deny` shorthand or a PiiSpec object. */
62
+ export function parsePii(value: unknown): "allow" | "deny" | PiiSpec {
63
+ if (value === "allow" || value === "deny") return value;
64
+ if (typeof value === "string") throw new Error("vault.pii: expected allow, deny or a policy object");
65
+ const raw = object(value, ["email", "phone", "custom", "patterns", "allow"], "vault.pii"), spec: PiiSpec = {};
66
+ for (const key of ["email", "phone", "custom"] as const) if (raw[key] !== undefined) spec[key] = cell(raw[key], `vault.pii.${key}`);
67
+ if (raw.patterns !== undefined) {
68
+ if (!Array.isArray(raw.patterns) || raw.patterns.length > 32) throw new Error("vault.pii.patterns: expected at most 32 patterns");
69
+ const names = new Set<string>();
70
+ spec.patterns = raw.patterns.map((item, index) => {
71
+ const p = object(item, ["name", "regex", "action"], `vault.pii.patterns[${index}]`);
72
+ const name = string(p.name, "vault.pii.patterns.name"), regex = string(p.regex, "vault.pii.patterns.regex");
73
+ if (!/^[a-z][a-z0-9-]{0,31}$/.test(name) || names.has(name)) throw new Error(`vault.pii.patterns: invalid or duplicate name ${name.slice(0, 40)}`);
74
+ if (Buffer.byteLength(regex) > 256) throw new Error("vault.pii.patterns.regex: exceeds 256 bytes");
75
+ try { new RegExp(regex, "u"); } catch { throw new Error(`vault.pii.patterns.${name}: invalid regular expression`); }
76
+ names.add(name);
77
+ return { name, regex, ...(p.action !== undefined ? { action: cell(p.action, `vault.pii.patterns.${name}.action`) } : {}) };
78
+ });
79
+ }
80
+ if (raw.allow !== undefined) spec.allow = strings(raw.allow, "vault.pii.allow");
81
+ return spec;
82
+ }
41
83
  export function parseVault(value: unknown): VaultSpec {
42
84
  const v = object(value, ["id", "path", "kind", "write", "pii", "remote", "registry"], "vault");
43
85
  const id = string(v.id, "vault.id");
@@ -45,10 +87,7 @@ export function parseVault(value: unknown): VaultSpec {
45
87
  if (v.kind !== "personal" && v.kind !== "org") throw new Error("vault.kind: expected personal or org");
46
88
  if (v.write !== "mcp" && v.write !== "cli" && v.write !== "none") throw new Error("vault.write: expected mcp, cli or none");
47
89
  const result: VaultSpec = { id, path: absolute(v.path, "vault.path"), kind: v.kind, write: v.write };
48
- if (v.pii !== undefined) {
49
- if (v.pii !== "allow" && v.pii !== "deny") throw new Error("vault.pii: expected allow or deny");
50
- result.pii = v.pii;
51
- }
90
+ if (v.pii !== undefined) result.pii = parsePii(v.pii);
52
91
  if (v.remote !== undefined) result.remote = string(v.remote, "vault.remote");
53
92
  if (v.registry !== undefined) result.registry = absolute(v.registry, "vault.registry");
54
93
  return result;
package/src/log-format.ts CHANGED
@@ -1,4 +1,6 @@
1
1
  import { screenContent } from "./write-validation.ts";
2
+ import { parsePii } from "./config.ts";
3
+ function validPii(value: unknown): boolean { try { parsePii(value); return true; } catch { return false; } }
2
4
  import type { LogContext, LogEvent } from "./log-types.ts";
3
5
  import type { WriteIssue } from "./write-types.ts";
4
6
 
@@ -31,7 +33,7 @@ export function formatLogEvent(event: LogEvent, context: LogContext): { line?: s
31
33
  }
32
34
  if ((context.channel !== undefined && context.channel !== "cli" && context.channel !== "mcp") ||
33
35
  !record(context.vault) || !["personal", "org"].includes(context.vault.kind) ||
34
- (context.vault.pii !== undefined && context.vault.pii !== "allow" && context.vault.pii !== "deny") ||
36
+ (context.vault.pii !== undefined && !validPii(context.vault.pii)) ||
35
37
  !record(context.config) || !Array.isArray(context.config.vaults) ||
36
38
  (context.config.piiPatterns !== undefined && (!Array.isArray(context.config.piiPatterns) || context.config.piiPatterns.some(value => typeof value !== "string"))) ||
37
39
  (context.now !== undefined && !(context.now instanceof Date)) ||
@@ -51,7 +53,7 @@ export function formatLogEvent(event: LogEvent, context: LogContext): { line?: s
51
53
  const env = context.env ?? process.env;
52
54
  const agent = context.channel === "mcp" || AGENT_MARKERS.some(marker => typeof env[marker] === "string" && env[marker]!.length > 0);
53
55
  const line = `${timestamp} ${event.type} ${event.project} ${event.source}${event.dest === undefined ? "" : ` → ${event.dest}`} — ${event.note}${agent ? " (vmem:agent)" : ""}\n`;
54
- const issues = screenContent(line, context.vault, context.config);
56
+ const issues = screenContent(line, context.vault, context.config, "log");
55
57
  if (Buffer.byteLength(line, "utf8") > 1000) issues.push(error("V09", "log line exceeds 1000 UTF-8 bytes including attribution and newline"));
56
58
  return issues.some(issue => issue.severity === "error") ? { issues } : { line, issues };
57
59
  }
package/src/mcp-vault.ts CHANGED
@@ -35,7 +35,7 @@ export function vaultTools(ids: readonly string[], orgIds: readonly string[] = [
35
35
  const twoStep = { confirm: { type: "boolean" }, fingerprint: sha };
36
36
  const orgVault = orgIds.length ? { type: "string", enum: [...orgIds] } : string(32);
37
37
  return [
38
- tool("vmem_status", "Registered vaults with health, write policy and effective pii.", object({})),
38
+ tool("vmem_status", "Registered vaults with health, write policy, effective pii and the piiPolicy matrix (read-only).", object({})),
39
39
  tool("vmem_search", "Full-text (BM25) search over indexed notes. Literal AND terms.", object({ query: string(1000), vault, limit: integer(1, 50), type: string(64), domain: string(64) }, ["query"])),
40
40
  tool("vmem_read", "Read one note by stem, alias or notes/ path: frontmatter, body, sha256. Bounded by maxBytes; section selects an exact H2, offset pages UTF-8 byte ranges, and truncated reads list H2 headings/nextOffset.", object({ path: note, vault, maxBytes: integer(1, 65536), section: string(1024), offset: integer(0, MAX_NOTE_BYTES) }, ["path"])),
41
41
  tool("vmem_read_many", "Read up to 20 notes (8 KiB each).", object({ paths: { type: "array", minItems: 1, maxItems: 20, items: note }, vault }, ["paths"])),
package/src/org-promote.ts CHANGED
@@ -9,9 +9,9 @@ import { VaultIndex } from "./vault-index.ts";
9
9
  import { applyWrite } from "./writer.ts";
10
10
  import { notePath } from "./write-operations.ts";
11
11
  import { screenContent } from "./write-validation.ts";
12
- import { appendLog } from "./vault-log.ts";
12
+ import { appendLog, logBinding } from "./vault-log.ts";
13
13
  import type { WriteIssue, WriteReceipt } from "./write-types.ts";
14
- import type { LogReceipt } from "./log-types.ts";
14
+ import type { LogBinding, LogReceipt } from "./log-types.ts";
15
15
  import { analysisInput } from "./analysis-input.ts";
16
16
  import { lintVault } from "./vault-lint.ts";
17
17
  import type { AnalysisSnapshot } from "./analysis-types.ts";
@@ -61,7 +61,8 @@ const MEETING = [/(^|\/)meetings\//i];
61
61
  * meeting sources. Returns error issues only; any issue refuses promotion.
62
62
  */
63
63
  export function screenForOrg(text: string, data: Record<string, unknown>, target: VaultSpec, config: VmemConfig): WriteIssue[] {
64
- const issues: WriteIssue[] = screenContent(text, target, config).map(found => ({ ...found, severity: "error" as const }));
64
+ // Secrets and deny cells refuse; a warn cell in the target policy is reported without refusing.
65
+ const issues: WriteIssue[] = screenContent(text, target, config, "promotion");
65
66
  if (data.type === "person" || data.domain === "people") issues.push(issue("SCREEN", "personal_data: person or people records are never promoted"));
66
67
  const sources = [...strings(data.source), ...strings(data.source_ref)];
67
68
  if (sources.some(value => MEETING.some(pattern => pattern.test(value)))) issues.push(issue("SCREEN", "internal_meeting_notes: source under meetings/"));
@@ -181,6 +182,10 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
181
182
  if (target.kind !== "org" || target.write === "none") return fail(issue("POLICY", "promotion target must be a writable org vault"));
182
183
  if (request.channel === "mcp" && target.write !== "mcp") return fail(issue("POLICY", "target vault write policy refuses MCP"));
183
184
  if (request.channel === "mcp" && source.write !== "mcp") return fail(issue("POLICY", "source provenance write needs write=mcp on the personal vault"));
185
+ // SYNC events must land in the vaults resolved here, even if the registry is changed before the appends.
186
+ let bindings: Record<"source" | "target", LogBinding>;
187
+ try { bindings = { source: logBinding(source, config), target: logBinding(target, config) }; }
188
+ catch { return fail(issue("V04", "source or target vault path unavailable")); }
184
189
  let path: string;
185
190
  const index = new VaultIndex(home, source, all.map(v => v.path));
186
191
  try { index.scan(); path = `notes/${index.show(request.note, 1).path}`; }
@@ -206,13 +211,13 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
206
211
  try { existing = parseFrontmatter(readSource(target, targetPath)).data; } catch { /* unreadable: treated as foreign */ }
207
212
  if (existing.promoted_from !== result.source || prior.includes(promotedTo)) return fail(issue("V08", `${target.id} already has ${targetPath}; move or delete it in the org vault first`, targetPath));
208
213
  const set = { promoted_to: prior.length ? [...prior, promotedTo] : promotedTo };
209
- const plan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env });
214
+ const plan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, binding: bindings.source });
210
215
  const fingerprint = sha({ recover: plan.fingerprint, promotedTo });
211
216
  const recovery: OrgPromoteResult = { ...result, target: `${target.id}:${targetPath}`, screen: { passed: true, issues: [] }, fingerprint, recovery: "provenance-only" };
212
217
  if (!plan.ok) return { ...recovery, issues: plan.issues };
213
218
  if (request.confirm !== true) return { ...recovery, ok: true, preview: true };
214
219
  if (request.fingerprint !== fingerprint) return { ...recovery, issues: [issue("V09", "stale or missing preview fingerprint")] };
215
- const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", log: false }, { env: options.env });
220
+ const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set }, channel: request.channel ?? "cli", log: false }, { env: options.env, binding: bindings.source });
216
221
  return { ...recovery, ok: provenance.ok, applied: provenance.applied, provenance, issues: provenance.ok ? [] : provenance.issues };
217
222
  }
218
223
  const missingTopics = topics.filter(topic => !targetNotes.mocs.has(linkKey(topic)));
@@ -230,12 +235,13 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
230
235
  const text = emitFrontmatter(parseFrontmatter(parsed.source.slice(0, parsed.source.length - parsed.body.length) + body), frontmatter);
231
236
  result.target = `${target.id}:${targetPath}`; result.text = text; result.droppedLinks = transformed.dropped;
232
237
  const screen = [...screenForOrg(text, parsed.data, target, config), ...screenDroppedLinks(home, source, all, transformed.dropped.filter(link => !link.includes(":")))];
233
- result.screen = { passed: screen.length === 0, issues: screen };
234
- if (screen.length) return { ...result, issues: screen };
235
- const planned = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env });
238
+ const refusing = screen.filter(found => found.severity === "error");
239
+ result.screen = { passed: refusing.length === 0, issues: screen };
240
+ if (refusing.length) return { ...result, issues: screen };
241
+ const planned = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, piiSurface: "promotion", binding: bindings.target });
236
242
  const provenanceSet = { promoted_to: prior.length ? [...prior.filter(p => p !== promotedTo), promotedTo] : promotedTo };
237
243
  const provenanceText = emitFrontmatter(parsed, provenanceSet);
238
- const provenancePlan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env });
244
+ const provenancePlan = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", dryRun: true }, { env: options.env, binding: bindings.source });
239
245
  result.fingerprint = sha({ target: planned.fingerprint, source: provenancePlan.fingerprint, text, provenanceText });
240
246
  // Org lint runs on the preview too (the note overlaid on the org snapshot), so a note that would fail lint is never written.
241
247
  const previewLint = lintPromoted(home, target, targetPath, text);
@@ -245,12 +251,13 @@ export function promoteToOrg(home: string, request: OrgPromoteRequest, options:
245
251
  if (request.confirm !== true) return { ...result, ok: true, preview: true, issues: [...planned.issues, ...provenancePlan.issues] };
246
252
  if (request.fingerprint !== result.fingerprint) return { ...result, issues: [issue("V09", "stale or missing preview fingerprint")] };
247
253
  const event = { type: "SYNC", project: request.logProject ?? "-", source: `${source.id}:${path}`, dest: `${target.id}:${targetPath}`, note: `promoted: ${typeof parsed.data.type === "string" ? parsed.data.type : "note"}` };
248
- const note = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", log: false }, { env: options.env });
249
- if (!note.ok || !note.applied) return { ...result, note, issues: note.issues };
250
- const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", log: false }, { env: options.env });
254
+ const note = applyWrite(home, { vault: target.id, operation: { op: "create", path: targetPath, frontmatter: {}, body: text }, channel: request.channel ?? "cli", log: false }, { env: options.env, piiSurface: "promotion", binding: bindings.target });
255
+ // A create that committed and then failed a later step (cleanup) is still a written org note.
256
+ if (!note.ok || !note.applied) return { ...result, applied: note.applied === true, note, issues: note.issues };
257
+ const provenance = applyWrite(home, { vault: source.id, operation: { op: "frontmatter", path, set: provenanceSet }, channel: request.channel ?? "cli", log: false }, { env: options.env, binding: bindings.source });
251
258
  const logs = { source: null as LogReceipt | null, target: null as LogReceipt | null };
252
259
  for (const [key, vault] of [["target", target], ["source", source]] as const) {
253
- try { logs[key] = appendLog(home, { vault: vault.id, event, channel: request.channel ?? "cli" }, { env: options.env }); }
260
+ try { logs[key] = appendLog(home, { vault: vault.id, event, channel: request.channel ?? "cli" }, { env: options.env, binding: bindings[key] }); }
254
261
  catch { logs[key] = { ok: false, appended: null, synced: false, bytesWritten: null, issues: [issue("LOG_IO", "log append failed")] }; }
255
262
  }
256
263
  const lint = lintPromoted(home, target, targetPath, text);
package/src/pii.ts ADDED
@@ -0,0 +1,133 @@
1
+ import { PII_SURFACES, type PiiAction, type PiiCell, type PiiSurface, type VaultSpec, type VmemConfig } from "./config.ts";
2
+ import type { WriteIssue } from "./write-types.ts";
3
+
4
+ export const PII_CATEGORIES = ["email", "phone", "custom"] as const;
5
+ export type PiiCategory = typeof PII_CATEGORIES[number];
6
+ export type PiiRow = Record<PiiSurface, PiiAction>;
7
+ /** Effective policy: every category and surface resolved, plus named patterns and the allow list. */
8
+ export interface PiiMatrix { email: PiiRow; phone: PiiRow; custom: PiiRow; patterns: { name: string; regex: string; action: PiiRow }[]; allow: string[] }
9
+
10
+ export const EMAIL_PATTERN = /[A-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[A-Z0-9](?:[A-Z0-9.-]*[A-Z0-9])?\.[A-Z]{2,}/i;
11
+ const RANK: Record<PiiAction, number> = { allow: 0, warn: 1, deny: 2 };
12
+
13
+ function row(value: PiiCell | undefined, fallback: PiiAction): PiiRow {
14
+ const out = {} as PiiRow;
15
+ for (const surface of PII_SURFACES) out[surface] = typeof value === "string" ? value : value?.[surface] ?? fallback;
16
+ return out;
17
+ }
18
+ /** Resolve a vault's policy. Omitted cells default to allow for personal and deny for org vaults. */
19
+ export function piiPolicy(vault: VaultSpec): PiiMatrix {
20
+ const fallback: PiiAction = vault.kind === "org" ? "deny" : "allow";
21
+ const pii = vault.pii;
22
+ if (pii === undefined || typeof pii === "string") {
23
+ const action = pii ?? fallback;
24
+ return { email: row(action, action), phone: row(action, action), custom: row(action, action), patterns: [], allow: [] };
25
+ }
26
+ return {
27
+ email: row(pii.email, fallback), phone: row(pii.phone, fallback), custom: row(pii.custom, fallback),
28
+ patterns: (pii.patterns ?? []).map(p => ({ name: p.name, regex: p.regex, action: row(p.action, fallback) })),
29
+ allow: [...(pii.allow ?? [])],
30
+ };
31
+ }
32
+ /** `allow`/`warn`/`deny` when every cell agrees and nothing else is set, else `custom`. */
33
+ export function piiShorthand(vault: VaultSpec): PiiAction | "custom" {
34
+ const m = piiPolicy(vault), cells = PII_CATEGORIES.flatMap(c => PII_SURFACES.map(s => m[c][s]));
35
+ return m.patterns.length || m.allow.length || cells.some(a => a !== cells[0]) ? "custom" : cells[0]!;
36
+ }
37
+ /** Cells (and allow-list or pattern changes) where `next` is more permissive than `prev`. */
38
+ export function loosenedCells(prev: PiiMatrix, next: PiiMatrix): string[] {
39
+ const out: string[] = [];
40
+ for (const c of PII_CATEGORIES) for (const s of PII_SURFACES) if (RANK[next[c][s]] < RANK[prev[c][s]]) out.push(`${c}.${s}: ${prev[c][s]} -> ${next[c][s]}`);
41
+ for (const p of prev.patterns) {
42
+ const n = next.patterns.find(q => q.name === p.name);
43
+ if (!n) { out.push(`patterns.${p.name}: removed`); continue; }
44
+ if (n.regex !== p.regex) out.push(`patterns.${p.name}: regex changed`);
45
+ for (const s of PII_SURFACES) if (RANK[n.action[s]] < RANK[p.action[s]]) out.push(`patterns.${p.name}.${s}: ${p.action[s]} -> ${n.action[s]}`);
46
+ }
47
+ for (const value of next.allow) if (!prev.allow.includes(value)) out.push(`allow: + ${value}`);
48
+ return out;
49
+ }
50
+
51
+ const digitsOf = (value: string) => value.replace(/\D/g, "");
52
+ const TIMESTAMP = /\b[0-9]{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12][0-9]|3[01])[ T](?:[01][0-9]|2[0-3]):[0-5][0-9](?::[0-5][0-9](?:\.[0-9]{1,3})?)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])?\b/g;
53
+ /**
54
+ * Phone runs: exact source ranges that hold a 9-15 digit number. `single` is set when the run is exactly one number
55
+ * (the only form the allow list can exempt). An oversized run (numbers separated by spaces) is screened by each
56
+ * contiguous run of digit-bearing groups and stops at the first valid one; it is never exempt (fails closed).
57
+ */
58
+ export function phoneRuns(text: string): { start: number; end: number; single?: string }[] {
59
+ // A complete ISO calendar timestamp is metadata, not a phone-number candidate; the newline keeps neighbours apart.
60
+ const cleaned = text.replace(TIMESTAMP, match => "\n".repeat(match.length)), out: { start: number; end: number; single?: string }[] = [];
61
+ for (const match of cleaned.matchAll(/(?<![\p{L}\p{N}])\+?\d(?:[\d ().-]{6,}\d)(?![\p{L}\p{N}])/gu)) {
62
+ const digits = digitsOf(match[0]), range = { start: match.index!, end: match.index! + match[0].length };
63
+ if (digits.length >= 9 && digits.length <= 15) { out.push({ ...range, single: digits }); continue; }
64
+ if (digits.length <= 15) continue;
65
+ const groups = match[0].split(/\s+/).map(digitsOf).filter(Boolean);
66
+ search: for (let from = 0; from < groups.length; from++) {
67
+ let total = 0;
68
+ for (let to = from; to < groups.length && total <= 15; to++) { total += groups[to]!.length; if (total >= 9 && total <= 15) { out.push(range); break search; } }
69
+ }
70
+ }
71
+ return out;
72
+ }
73
+ function allowedEmail(value: string, allow: string[]): boolean {
74
+ const lower = value.toLowerCase();
75
+ return allow.some(entry => entry.includes("@") && (entry.startsWith("@") ? lower.endsWith(entry.toLowerCase()) : lower === entry.toLowerCase()));
76
+ }
77
+ function allowedPhone(digits: string, allow: string[]): boolean {
78
+ return allow.some(entry => !entry.includes("@") && digitsOf(entry).length >= 9 && digitsOf(entry) === digits);
79
+ }
80
+ interface Rule { label: string; action: PiiAction; invalid?: boolean; matches: (text: string) => { start: number; end: number }[]; detect: (text: string) => boolean }
81
+ /** The non-allow rules for one surface. `detect` short-circuits; `matches` lists raw non-exempt ranges (zero-length included). */
82
+ function rules(vault: VaultSpec, config: VmemConfig, surface: PiiSurface): Rule[] {
83
+ const m = piiPolicy(vault), out: Rule[] = [];
84
+ const regexRule = (label: string, action: PiiAction, make: () => RegExp, exempt: (value: string) => boolean): Rule => ({
85
+ label, action,
86
+ matches: text => [...text.matchAll(make())].filter(x => !x[0] || !exempt(x[0])).map(x => ({ start: x.index!, end: x.index! + x[0].length })),
87
+ detect: text => { for (const source of [text, text.normalize("NFKC")]) for (const x of source.matchAll(make())) if (!x[0] || !exempt(x[0])) return true; return false; },
88
+ });
89
+ if (m.email[surface] !== "allow") out.push(regexRule("email", m.email[surface], () => new RegExp(EMAIL_PATTERN.source, "gi"), v => allowedEmail(v, m.allow)));
90
+ if (m.phone[surface] !== "allow") {
91
+ const flagged = (text: string) => phoneRuns(text).filter(r => !(r.single && allowedPhone(r.single, m.allow)));
92
+ out.push({ label: "phone", action: m.phone[surface], matches: flagged, detect: text => flagged(text).length > 0 || flagged(text.normalize("NFKC")).length > 0 });
93
+ }
94
+ const custom = (config.piiPatterns ?? []).map((regex, i) => ({ name: `custom-${i + 1}`, regex, action: m.custom[surface], flags: "" }));
95
+ for (const p of [...custom, ...m.patterns.map(p => ({ name: p.name, regex: p.regex, action: p.action[surface], flags: "u" }))]) {
96
+ if (p.action === "allow") continue;
97
+ try { new RegExp(p.regex, p.flags); } catch { out.push({ label: p.name, action: "deny", invalid: true, matches: () => [], detect: () => true }); continue; }
98
+ out.push(regexRule(p.name, p.action, () => new RegExp(p.regex, `${p.flags}g`), v => m.allow.some(a => a.toLowerCase() === v.toLowerCase())));
99
+ }
100
+ return out;
101
+ }
102
+ /** V07 issues for one surface: deny cells are errors, warn cells warnings. Matched values are never returned. */
103
+ export function screenPii(text: string, vault: VaultSpec, config: VmemConfig, surface: PiiSurface): WriteIssue[] {
104
+ const issues: WriteIssue[] = [];
105
+ for (const rule of rules(vault, config, surface)) {
106
+ if (rule.invalid) { if (!issues.some(i => i.message === "invalid PII screening configuration")) issues.push({ code: "V07", severity: "error", message: "invalid PII screening configuration" }); continue; }
107
+ if (!rule.detect(text)) continue;
108
+ const builtin = rule.label === "email" || rule.label === "phone", custom = rule.label.startsWith("custom-");
109
+ const message = rule.action === "deny"
110
+ ? builtin ? "PII is refused by the vault policy" : custom ? "configured PII pattern matched" : `PII pattern ${rule.label} matched`
111
+ : `PII flagged by the vault policy (${builtin || custom ? rule.label.replace(/-\d+$/, "") : `pattern ${rule.label}`}, warn)`;
112
+ if (!issues.some(i => i.message === message)) issues.push({ code: "V07", severity: rule.action === "deny" ? "error" : "warning", message });
113
+ }
114
+ return issues;
115
+ }
116
+ /**
117
+ * Replace raw ranges that a deny cell refuses on this surface with `[redacted]` (tidy --log), leaving allow-listed
118
+ * values intact. Fails closed to a fully redacted text when a denied match is zero-length, or when the returned text
119
+ * itself still contains a denied match (for example a form only NFKC normalization reveals, or a pattern that matches
120
+ * the marker).
121
+ */
122
+ export function redactPii(text: string, vault: VaultSpec, config: VmemConfig, surface: PiiSurface): { text: string; hits: number } {
123
+ const denied = rules(vault, config, surface).filter(rule => rule.action === "deny" && !rule.invalid);
124
+ if (!denied.some(rule => rule.detect(text))) return { text, hits: 0 };
125
+ const ranges = denied.flatMap(rule => rule.matches(text));
126
+ if (ranges.some(r => r.end === r.start)) return { text: "[redacted]", hits: 1 };
127
+ const merged: { start: number; end: number }[] = [];
128
+ for (const r of ranges.sort((a, b) => a.start - b.start || b.end - a.end)) { const last = merged.at(-1); if (last && r.start < last.end) last.end = Math.max(last.end, r.end); else merged.push({ ...r }); }
129
+ let out = "", at = 0;
130
+ for (const r of merged) { out += text.slice(at, r.start) + "[redacted]"; at = r.end; }
131
+ out += text.slice(at);
132
+ return denied.some(rule => rule.detect(out)) ? { text: "[redacted]", hits: 1 } : { text: out, hits: merged.length };
133
+ }
package/src/registry.ts CHANGED
@@ -2,7 +2,8 @@ import { closeSync, constants, fsyncSync, mkdirSync, openSync, readdirSync, read
2
2
  import { hostname } from "node:os";
3
3
  import { join } from "node:path";
4
4
  import { randomUUID } from "node:crypto";
5
- import { CONFIG_FILE, MAX_CONFIG_BYTES, parseConfig, parseVault, readConfig, type VaultSpec, type VmemConfig } from "./config.ts";
5
+ import { CONFIG_FILE, MAX_CONFIG_BYTES, parseConfig, parsePii, parseVault, readConfig, type VaultSpec, type VmemConfig } from "./config.ts";
6
+ import { loosenedCells, piiPolicy, piiShorthand } from "./pii.ts";
6
7
  export const LOCK_FILE = "config.lock";
7
8
  const TEMP = /^\.config-[0-9a-f-]{36}\.tmp$/;
8
9
  export interface LockOwner { pid: number; host: string; created: string }
@@ -11,8 +12,8 @@ export function loadRegistry(home: string): VmemConfig {
11
12
  if (config.home) throw new Error(`${join(home, CONFIG_FILE)}: redirected home cannot contain another home pointer`);
12
13
  return config;
13
14
  }
14
- /** Effective pii policy: explicit value, else deny for org and allow for personal. */
15
- export function effectivePii(vault: VaultSpec): "allow" | "deny" { return vault.pii ?? (vault.kind === "org" ? "deny" : "allow"); }
15
+ /** Effective pii shorthand (`custom` for a mixed policy); the full matrix is `piiPolicy`. */
16
+ export function effectivePii(vault: VaultSpec): "allow" | "warn" | "deny" | "custom" { return piiShorthand(vault); }
16
17
  function message(error: unknown): string { return error instanceof Error ? error.message : String(error); }
17
18
  /**
18
19
  * Directory fsync makes the rename durable on Linux. On macOS fsync(2) on a directory
@@ -157,10 +158,25 @@ export function removeVault(home: string, id: string, confirm = false): { applie
157
158
  };
158
159
  return confirm ? mutate(home, remove) : remove(loadRegistry(home));
159
160
  }
161
+ /** Show or change a vault's PII policy. Without `confirm` it previews; loosened cells are listed either way. */
162
+ export function setVaultPii(home: string, id: string, value: unknown, confirm = false) {
163
+ const next = value === undefined ? undefined : parsePii(value);
164
+ const change = (config: VmemConfig) => {
165
+ const vault = config.vaults.find(v => v.id === id);
166
+ if (!vault) throw new Error(`${join(home, CONFIG_FILE)}: unknown vault id ${id}`);
167
+ const before = piiPolicy(vault);
168
+ if (next === undefined) return { vault: id, pii: piiShorthand(vault), piiPolicy: before };
169
+ const updated: VaultSpec = { ...vault, pii: next }, after = piiPolicy(updated);
170
+ const loosened = loosenedCells(before, after), changed = JSON.stringify(before) !== JSON.stringify(after);
171
+ if (confirm && changed) vault.pii = next;
172
+ return { vault: id, applied: confirm && changed, changed, loosened, pii: piiShorthand(updated), piiPolicy: after, ...(confirm ? {} : { preview: true }) };
173
+ };
174
+ return confirm && next !== undefined ? mutate(home, change) : change(loadRegistry(home));
175
+ }
160
176
  export function registryStatus(home: string) {
161
177
  const config = loadRegistry(home);
162
178
  return config.vaults.map(vault => {
163
- const base = { ...vault, pii: effectivePii(vault) };
179
+ const base = { ...vault, pii: effectivePii(vault), piiPolicy: piiPolicy(vault) };
164
180
  try {
165
181
  const root = realpathSync(vault.path), notes = join(root, "notes");
166
182
  if (root !== vault.path || realpathSync(notes) !== notes || !statSync(notes).isDirectory()) throw new Error("changed");
package/src/tidy.ts CHANGED
@@ -9,9 +9,10 @@ import { extractLinks, normalizeTarget } from "./links.ts";
9
9
  import { isDenied } from "./platform.ts";
10
10
  import { loadRegistry, syncDirectory } from "./registry.ts";
11
11
  import { normalizeSource, sourceReferences, sourceWorkspaceRoot, missingSourceWarnings } from "./source.ts";
12
- import { appendLog, LOG_LOCK } from "./vault-log.ts";
12
+ import { appendLog, logBinding, LOG_LOCK } from "./vault-log.ts";
13
13
  import { appendSection, foldConnections, h2Sections } from "./write-operations.ts";
14
- import { EMAIL_PATTERN, SCHEMA_KEYS, SECRET_PATTERNS, TYPED_KEYS, screenContent } from "./write-validation.ts";
14
+ import { SCHEMA_KEYS, SECRET_PATTERNS, TYPED_KEYS, screenContent } from "./write-validation.ts";
15
+ import { redactPii } from "./pii.ts";
15
16
  import { applyWrite, sha256, snapshot } from "./writer.ts";
16
17
  import type { WriteIssue } from "./write-types.ts";
17
18
 
@@ -281,13 +282,14 @@ function normalizeLogLine(line: string): string | undefined {
281
282
  if (seconds && LOG_STRUCTURE.test(seconds[1]! + seconds[2]!)) return seconds[1]! + seconds[2]! + line.slice(body.length);
282
283
  return `# legacy: ${line}`;
283
284
  }
284
- function sanitizeLog(text: string): { text: string; lines: number; redactions: number; formats: number; changes: string[] } {
285
+ /** PII is redacted where the vault policy denies it on the log surface; secrets always are. */
286
+ function sanitizeLog(text: string, vault: VaultSpec, config: VmemConfig): { text: string; lines: number; redactions: number; formats: number; changes: string[] } {
285
287
  let lines = 0, redactions = 0, formats = 0;
286
288
  const changes: string[] = [];
287
289
  const out = text.split("\n").map((line, index) => {
288
290
  const secret = redact(line);
289
291
  let hits = secret.hits, next = secret.text;
290
- next = next.replace(new RegExp(EMAIL_PATTERN.source, "gi"), () => { hits++; return "[redacted]"; });
292
+ const pii = redactPii(next, vault, config, "log"); hits += pii.hits; next = pii.text;
291
293
  const normalized = normalizeLogLine(next);
292
294
  if (normalized !== undefined) { next = normalized; formats++; }
293
295
  if (hits || normalized !== undefined) {
@@ -298,8 +300,11 @@ function sanitizeLog(text: string): { text: string; lines: number; redactions: n
298
300
  }).join("\n");
299
301
  return { text: out, lines, redactions, formats, changes };
300
302
  }
301
- /** Replace the log under its own lock, atomically (the one sanctioned in-place rewrite of the append-only log). */
302
- function rewriteLog(root: string, expected: string, next: string): void {
303
+ /**
304
+ * Replace the log under its own lock, atomically (the one sanctioned in-place rewrite of the append-only log).
305
+ * `state.committed` turns true once the rename lands, so a caller can tell a later failure (fsync, lock removal) apart.
306
+ */
307
+ function rewriteLog(root: string, expected: string, next: string, state: { committed: boolean }): void {
303
308
  const lock = join(root, LOG_LOCK);
304
309
  const fd = openSync(lock, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
305
310
  let temp: string | undefined;
@@ -309,7 +314,7 @@ function rewriteLog(root: string, expected: string, next: string): void {
309
314
  temp = join(root, `.vmem-log-${randomUUID()}.tmp`);
310
315
  const tfd = openSync(temp, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600);
311
316
  try { writeFileSync(tfd, next); fsyncSync(tfd); } finally { closeSync(tfd); }
312
- renameSync(temp, join(root, "log")); temp = undefined;
317
+ renameSync(temp, join(root, "log")); temp = undefined; state.committed = true;
313
318
  syncDirectory(root);
314
319
  } finally {
315
320
  if (temp) try { unlinkSync(temp); } catch { /* best effort */ }
@@ -332,6 +337,8 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
332
337
  if (vault.write === "none" || (request.channel === "mcp" && vault.write !== "mcp")) return { ...result, issues: [issue("POLICY", "vault write policy refuses this channel")] };
333
338
  const root = realpathSync(vault.path);
334
339
  if (root !== vault.path || isDenied(root, config.denyPaths)) return { ...result, issues: [issue("V06", "vault path denied or changed")] };
340
+ // The LINT event must land in the vault this run read and wrote, even if the registry is changed meanwhile.
341
+ const binding = logBinding(vault, config);
335
342
  const map = request.map ?? { enums: {}, keys: {} };
336
343
  // Only flat notes/*.md are read: .obsidian/, templates/ and self/ are never touched.
337
344
  let notes: Map<string, string>;
@@ -359,7 +366,7 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
359
366
  if (rules.includes("log")) {
360
367
  const before = readLogFile(root);
361
368
  if (before !== null) {
362
- const sanitized = sanitizeLog(before);
369
+ const sanitized = sanitizeLog(before, vault, config);
363
370
  result.log = { lines: sanitized.lines, redactions: sanitized.redactions, formats: sanitized.formats, changes: sanitized.changes };
364
371
  if (sanitized.text !== before) { logPlan = { before, ...sanitized }; result.counts.log = sanitized.redactions + sanitized.formats; }
365
372
  }
@@ -367,8 +374,9 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
367
374
  // Validate every planned note with the writer (dry-run). A note the writer would refuse is reported and skipped
368
375
  // (left fully unchanged for manual repair) instead of blocking the rest of the migration.
369
376
  for (const change of [...planned]) {
370
- const dry = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", dryRun: true, log: false }, { env: options.env });
377
+ const dry = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", dryRun: true, log: false }, { env: options.env, binding });
371
378
  const errors = dry.issues.filter(found => found.severity === "error");
379
+ if (errors.some(found => found.code === "VAULT_DRIFT")) return { ...result, issues: errors.filter(found => found.code === "VAULT_DRIFT") };
372
380
  if (!errors.length) continue;
373
381
  planned.splice(planned.indexOf(change), 1);
374
382
  result.reported.push({ path: change.path, message: `skipped: the writer refuses the sanitized note (${[...new Set(errors.map(found => found.code))].join(", ")}); repair by hand` });
@@ -383,18 +391,26 @@ export function tidyVault(home: string, request: TidyRequest, options: { env?: N
383
391
  if (request.confirm !== true) return { ...result, ok: true, preview: true };
384
392
  if (request.fingerprint !== result.fingerprint) return { ...result, issues: [issue("V09", "stale or missing preview fingerprint")] };
385
393
  for (const change of planned) {
386
- const receipt = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", expectedShas: { [change.path]: sha256(change.before) }, log: false }, { env: options.env });
387
- if (!receipt.ok || !receipt.applied) return { ...result, issues: receipt.issues.length ? receipt.issues : [issue("IO", "note write failed", change.path)] };
394
+ const receipt = applyWrite(home, { vault: vault.id, operation: { op: "sanitize", path: change.path, text: change.after }, channel: request.channel ?? "cli", expectedShas: { [change.path]: sha256(change.before) }, log: false }, { env: options.env, binding });
395
+ // Notes written before a failure stay committed, so the result reports them as applied.
396
+ if (!receipt.ok || !receipt.applied) {
397
+ if (receipt.applied) result.written.push(change.path); // committed, then a later step of the write failed
398
+ return { ...result, applied: result.written.length > 0, issues: receipt.issues.length ? receipt.issues : [issue("IO", "note write failed", change.path)] };
399
+ }
388
400
  result.written.push(change.path);
389
401
  }
390
402
  if (logPlan) {
391
- if (isDenied(join(root, "log"), loadRegistry(home).denyPaths)) return { ...result, issues: [issue("V06", "the vault log is denied by the path policy; log not rewritten")] };
392
- try { rewriteLog(root, logPlan.before, logPlan.text); result.written.push("log"); }
393
- catch (error) { return { ...result, issues: [issue("LOG", error instanceof Error ? error.message.slice(0, 200) : "log rewrite failed")] }; }
403
+ if (isDenied(join(root, "log"), loadRegistry(home).denyPaths)) return { ...result, applied: result.written.length > 0, issues: [issue("V06", "the vault log is denied by the path policy; log not rewritten")] };
404
+ const state = { committed: false };
405
+ try { rewriteLog(root, logPlan.before, logPlan.text, state); result.written.push("log"); }
406
+ catch (error) {
407
+ if (state.committed) result.written.push("log"); // the new log is in place; a later step failed
408
+ return { ...result, applied: result.written.length > 0, issues: [issue("LOG", error instanceof Error ? error.message.slice(0, 200) : "log rewrite failed")] };
409
+ }
394
410
  }
395
411
  const counted = Object.values(result.counts).reduce((sum, n) => sum + n, 0);
396
412
  if (planned.length || logPlan) {
397
- const receipt = appendLog(home, { vault: vault.id, event: { type: "LINT", project: "-", source: "vmem:tidy", note: result.event! }, channel: request.channel ?? "cli" }, { env: options.env });
413
+ const receipt = appendLog(home, { vault: vault.id, event: { type: "LINT", project: "-", source: "vmem:tidy", note: result.event! }, channel: request.channel ?? "cli" }, { env: options.env, binding });
398
414
  if (!receipt.ok) return { ...result, applied: true, issues: receipt.issues };
399
415
  }
400
416
  return { ...result, ok: true, applied: result.written.length > 0 || counted > 0 };
package/src/vault-log.ts CHANGED
@@ -7,6 +7,7 @@ import { canonicalPath, isDenied } from "./platform.ts";
7
7
  import { formatLogEvent, LOG_TYPES } from "./log-format.ts";
8
8
  import type { LogBinding, LogContext, LogReadOptions, LogReadResult, LogReceipt, LogRequest } from "./log-types.ts";
9
9
  import type { WriteIssue } from "./write-types.ts";
10
+ import type { VaultSpec, VmemConfig } from "./config.ts";
10
11
  export const LOG_LOCK = "log.lock";
11
12
  export const MAX_LOG_READ_BYTES = 32 * 1024 * 1024;
12
13
  export const LARGE_LOG_LINES = 20000;
@@ -64,13 +65,32 @@ function context(home: string, id: string, channel: "cli" | "mcp" | undefined, w
64
65
  return { vault, config, channel, ...(write && vault.registry ? { projectIds: registryIds(vault.registry, config.denyPaths) } : {}) };
65
66
  }
66
67
  /** One O_APPEND write per event. A separate cooperative lock permits safe short-write rollback. */
68
+ /** Bind a later append to the vault spec, path policy and root identity resolved now (refused with LOG_DRIFT if they change). */
69
+ export function logBinding(vault: VaultSpec, config: VmemConfig): LogBinding {
70
+ const identity = lstatSync(vault.path);
71
+ return { vault, denyPaths: config.denyPaths ?? [], piiPatterns: config.piiPatterns ?? [], dev: Number(identity.dev), ino: Number(identity.ino) };
72
+ }
73
+ /** True when the registered vault still equals the binding: spec, path policy and root identity. Never throws. */
74
+ export function matchesBinding(config: VmemConfig, id: string, binding: LogBinding): boolean {
75
+ const vault = config.vaults.find(v => v.id === id);
76
+ if (!vault || JSON.stringify(vault) !== JSON.stringify(binding.vault) || JSON.stringify(config.denyPaths ?? []) !== JSON.stringify(binding.denyPaths) || JSON.stringify(config.piiPatterns ?? []) !== JSON.stringify(binding.piiPatterns)) return false;
77
+ try { const identity = lstatSync(vault.path); return Number(identity.dev) === binding.dev && Number(identity.ino) === binding.ino; } catch { return false; }
78
+ }
67
79
  export function appendLog(home: string, request: LogRequest, options: { env?: NodeJS.ProcessEnv; now?: Date; binding?: LogBinding } = {}): LogReceipt {
68
80
  const result: LogReceipt = { ok: false, appended: false, synced: false, bytesWritten: 0, issues: [] };
69
81
  let fd: number | undefined, lockFd: number | undefined, lockFile: string | undefined, identity: { ino: number; dev: number } | undefined;
70
82
  const error = (code: string, message: string) => result.issues.push({ code, severity: "error", message });
71
83
  try {
72
84
  if (!request || typeof request !== "object" || typeof request.vault !== "string" || Object.keys(request).some(key => !["vault", "event", "channel"].includes(key))) refuse("L09", "invalid log request");
73
- const ctx = context(home, request.vault, request.channel, true);
85
+ // A bound append compares the registry first, so any re-registration reads as drift rather than a policy or path error.
86
+ if (options.binding && !matchesBinding(loadRegistry(home), request.vault, options.binding)) refuse("LOG_DRIFT", "vault identity or policy changed after mutation; log not appended");
87
+ let ctx: ReturnType<typeof context>;
88
+ try { ctx = context(home, request.vault, request.channel, true); }
89
+ catch (caught) {
90
+ // A re-registration between the binding check and this second registry read still reads as drift.
91
+ if (options.binding && !matchesBinding(loadRegistry(home), request.vault, options.binding)) refuse("LOG_DRIFT", "vault identity or policy changed after mutation; log not appended");
92
+ throw caught;
93
+ }
74
94
  const rootIdentity = lstatSync(ctx.vault.path);
75
95
  if (options.binding) {
76
96
  const bound = options.binding;
@@ -78,6 +98,7 @@ export function appendLog(home: string, request: LogRequest, options: { env?: No
78
98
  }
79
99
  const formatted = formatLogEvent(request.event, { ...ctx, env: options.env ?? process.env, now: options.now });
80
100
  if (!formatted.line || formatted.issues.some(issue => issue.severity === "error")) { result.issues.push(...formatted.issues); return result; }
101
+ result.issues.push(...formatted.issues); // nonblocking warnings (for example a log-surface PII warn cell)
81
102
  const buffer = Buffer.from(formatted.line), root = ctx.vault.path;
82
103
  lockFile = join(root, LOG_LOCK);
83
104
  const deadline = Date.now() + 2000, sleeper = new Int32Array(new SharedArrayBuffer(4));
package/src/write-types.ts CHANGED
@@ -1,3 +1,4 @@
1
+ import type { PiiSurface } from "./config.ts";
1
2
  import type { LogEvent, LogReceipt } from "./log-types.ts";
2
3
  import type { VaultSpec, VmemConfig } from "./config.ts";
3
4
  export type WriteOp =
@@ -32,6 +33,8 @@ export interface ValidationContext {
32
33
  /** Original flat notes snapshot, paths prefixed with notes/. */
33
34
  notes: Map<string, string>;
34
35
  operation: WriteOp;
36
+ /** PII surface for content screening; `notes` unless an internal caller (org promotion) sets it. */
37
+ piiSurface?: PiiSurface;
35
38
  }
36
39
  export interface ChangeReceipt { path: string; beforeSha: string | null; sha256: string | null; bytes: number }
37
40
  export interface WriteReceipt {
package/src/write-validation.ts CHANGED
@@ -6,6 +6,8 @@ import { parseFrontmatter } from "./frontmatter.ts";
6
6
  import { extractLinks, normalizeTarget, proseLines, resolveLinks, type Link, type LinkNote, type ResolvedLink } from "./links.ts";
7
7
  import { canonicalPath, isDenied } from "./platform.ts";
8
8
  import type { PlannedChange, ValidationContext, WriteIssue } from "./write-types.ts";
9
+ import type { PiiSurface } from "./config.ts";
10
+ import { screenPii } from "./pii.ts";
9
11
  import { moveSourcePath } from "./write-operations.ts";
10
12
 
11
13
  const TYPES = new Set(["insight", "decision", "observation", "person", "project", "method", "moc", "reference"]);
@@ -30,50 +32,16 @@ const WIKI_FIELDS = ["topics", "project", "superseded_by", "supersedes", "source
30
32
  const SECRETS = [/-----BEGIN [A-Z ]*PRIVATE KEY-----/, /\bAKIA[0-9A-Z]{16}\b/, /\bgh[pousr]_[A-Za-z0-9]{30,}/, /\bgithub_pat_[A-Za-z0-9_]{40,}/, /\bsk-[A-Za-z0-9_-]{20,}/,
31
33
  /\bxox[abprs]-[A-Za-z0-9-]{10,}/, /\bAIza[0-9A-Za-z_-]{35}\b/, /\beyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}/, /\b(?:password|passwd|secret|api[_-]?key|token)\s*[:=]\s*\S{8,}/i];
32
34
  export const SECRET_PATTERNS: readonly RegExp[] = SECRETS;
33
- const EMAIL = /[A-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[A-Z0-9](?:[A-Z0-9.-]*[A-Z0-9])?\.[A-Z]{2,}/i;
34
- export const EMAIL_PATTERN = EMAIL;
35
+ export { EMAIL_PATTERN } from "./pii.ts";
35
36
  const MAX_BYTES = 256 * 1024;
36
-
37
37
  function issue(code: string, message: string, severity: WriteIssue["severity"] = "error", path?: string): WriteIssue {
38
38
  return { code, severity, message, ...(path ? { path } : {}) };
39
39
  }
40
- function hasPhone(text: string): boolean {
41
- // A complete ISO calendar timestamp is metadata, not a phone-number candidate.
42
- const withoutTimestamps = text.replace(/\b[0-9]{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12][0-9]|3[01])[ T](?:[01][0-9]|2[0-3]):[0-5][0-9](?::[0-5][0-9](?:\.[0-9]{1,3})?)?(?:Z|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])?\b/g, "\n");
43
- // The newline replacement is outside the candidate class, so neighbours of a timestamp never merge.
44
- const digits = (value: string) => value.replace(/\D/g, "").length;
45
- for (const match of withoutTimestamps.matchAll(/(?<![\p{L}\p{N}])\+?\d(?:[\d ().-]{6,}\d)(?![\p{L}\p{N}])/gu)) {
46
- const count = digits(match[0]);
47
- if (count >= 9 && count <= 15) return true;
48
- if (count <= 15) continue;
49
- // Oversized run, e.g. two phones separated by spaces: evaluate every contiguous run of
50
- // whitespace-separated groups alone so each embedded phone is screened (fails closed).
51
- const groups = match[0].split(/\s+/).filter(Boolean);
52
- for (let start = 0; start < groups.length; start++) {
53
- let total = 0;
54
- for (let end = start; end < groups.length && total <= 15; end++) {
55
- total += digits(groups[end]!);
56
- if (total >= 9 && total <= 15) return true;
57
- }
58
- }
59
- }
60
- return false;
61
- }
62
- /** Screen the whole prospective text, including frontmatter, without returning matched values. */
63
- export function screenContent(text: string, vault: VaultSpec, config: VmemConfig): WriteIssue[] {
40
+ /** Screen the whole prospective text, including frontmatter, without returning matched values. Secrets are always refused; PII follows the vault policy for this surface. */
41
+ export function screenContent(text: string, vault: VaultSpec, config: VmemConfig, surface: PiiSurface = "notes"): WriteIssue[] {
64
42
  const issues: WriteIssue[] = [];
65
43
  if (SECRETS.some(pattern => pattern.test(text))) issues.push(issue("V06", "secret-looking content is refused"));
66
- const pii = vault.pii ?? (vault.kind === "org" ? "deny" : "allow");
67
- if (pii === "deny") {
68
- const normalized = text.normalize("NFKC");
69
- if (EMAIL.test(normalized) || hasPhone(normalized)) issues.push(issue("V07", "PII is refused by the vault policy"));
70
- for (const pattern of config.piiPatterns ?? []) {
71
- try {
72
- if (new RegExp(pattern).test(text)) { issues.push(issue("V07", "configured PII pattern matched")); break; }
73
- } catch { issues.push(issue("V07", "invalid PII screening configuration")); break; }
74
- }
75
- }
76
- return issues;
44
+ return [...issues, ...screenPii(text, vault, config, surface)];
77
45
  }
78
46
 
79
47
  /** Quoted escapes must not hide semantic metadata from the same content policy. */
@@ -81,7 +49,8 @@ function screenMetadata(data:Record<string,unknown>,context:ValidationContext,pa
81
49
  const found=new Map<string,WriteIssue>(),active=new WeakSet<object>();let nodes=0,bytes=0;
82
50
  const inspect=(text:string)=>{
83
51
  bytes+=Buffer.byteLength(text);if(bytes>MAX_BYTES*4)throw new Error("budget");
84
- for(const item of screenContent(text,context.vault,context.config))found.set(item.code,{...item,path});
52
+ // One issue per code; an error is never downgraded by a later warning of the same code.
53
+ for(const item of screenContent(text,context.vault,context.config,context.piiSurface)){const prior=found.get(item.code);if(!prior||(prior.severity!=="error"&&item.severity==="error"))found.set(item.code,{...item,path});}
85
54
  };
86
55
  const visit=(value:unknown,key:string|undefined,depth:number):void=>{
87
56
  if(++nodes>10000||depth>32)throw new Error("budget");
@@ -241,7 +210,7 @@ export function validateWrite(changes: PlannedChange[], context: ValidationConte
241
210
  else if (change.path.startsWith("notes/")) final.set(change.path, change.after);
242
211
  if (change.after === null) continue;
243
212
  if (Buffer.byteLength(change.after) > MAX_BYTES) issues.push(issue("V09", "prospective file exceeds 256 KiB", "error", change.path));
244
- issues.push(...screenContent(change.after, context.vault, context.config).map(value => ({ ...value, path: change.path })));
213
+ issues.push(...screenContent(change.after, context.vault, context.config, context.piiSurface).map(value => ({ ...value, path: change.path })));
245
214
  try{issues.push(...screenMetadata(parseFrontmatter(change.after).data,context,change.path));}catch{ /* Invalid note YAML is reported separately; reports remain schema-exempt. */ }
246
215
  }
247
216
  const sourceRoot=sourceWorkspaceRoot(context.vault);
package/src/writer.ts CHANGED
@@ -1,4 +1,4 @@
1
- import { appendLog } from "./vault-log.ts";
1
+ import { appendLog, matchesBinding } from "./vault-log.ts";
2
2
  import type { LogEvent } from "./log-types.ts";
3
3
  import { closeSync, constants, fstatSync, fsyncSync, lstatSync, mkdirSync, openSync, readSync, readdirSync, realpathSync, renameSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs";
4
4
  import { createHash, randomUUID } from "node:crypto";
@@ -11,6 +11,8 @@ import { parseFrontmatter } from "./frontmatter.ts";
11
11
  import { canonicalPath, isDenied } from "./platform.ts";
12
12
  import { moveSourcePath, notePath, planOperation, refuse, WriteRefusal } from "./write-operations.ts";
13
13
  import { validateWrite } from "./write-validation.ts";
14
+ import type { PiiSurface } from "./config.ts";
15
+ import type { LogBinding } from "./log-types.ts";
14
16
  import type { ChangeReceipt, PlannedChange, WriteIssue, WriteReceipt, WriteRequest } from "./write-types.ts";
15
17
  export const MUTATION_LOCK = "mutation.lock";
16
18
  export const MAX_WRITE_BYTES = 256 * 1024;
@@ -111,7 +113,8 @@ function duplicateWarnings(request: WriteRequest, notes: Map<string, string>, ch
111
113
  } finally { db.close(); }
112
114
  }
113
115
  /** Single mutation entry point. Preview/dry-run perform no writes, including locks and caches. */
114
- export function applyWrite(home: string, request: WriteRequest, options: { env?: NodeJS.ProcessEnv } = {}): WriteReceipt {
116
+ /** `options.binding` (internal callers composing several writes) refuses with VAULT_DRIFT unless the registry still matches it. */
117
+ export function applyWrite(home: string, request: WriteRequest, options: { env?: NodeJS.ProcessEnv; piiSurface?: PiiSurface; binding?: LogBinding } = {}): WriteReceipt {
115
118
  let lockFd: number | undefined, lockFile: string | undefined, lockIdentity: { ino: number; dev: number } | undefined;
116
119
  let index: VaultIndex | undefined;
117
120
  const temps = new Set<string>(), createdDirs: string[] = [], committed: PlannedChange[] = [];
@@ -124,6 +127,7 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
124
127
  if (request.expectedShas !== undefined && (!request.expectedShas || typeof request.expectedShas !== "object" || Array.isArray(request.expectedShas))) refuse("V09", "expectedShas must be a mapping");
125
128
  if (request.channel !== undefined && !["cli", "mcp"].includes(request.channel)) refuse("POLICY", "unknown write channel");
126
129
  const config = loadRegistry(home), vault = config.vaults.find(value => value.id === request.vault);
130
+ if (options.binding && !matchesBinding(config, request.vault, options.binding)) refuse("VAULT_DRIFT", "vault identity or policy changed since the run started; nothing written");
127
131
  if (!vault) refuse("V04", "unknown vault id");
128
132
  if (request.operation.op !== "check" && (vault.write === "none" || (request.channel === "mcp" && vault.write !== "mcp"))) refuse("POLICY", "vault write policy refuses this channel");
129
133
  root = realpathSync(vault.path);
@@ -150,11 +154,13 @@ export function applyWrite(home: string, request: WriteRequest, options: { env?:
150
154
  }
151
155
  changes = planOperation(request.operation, notes, reportBefore);
152
156
  for (const change of changes) if (isDenied(join(root, change.path), config.denyPaths)) refuse("V06", "planned mutation path is denied", change.path);
153
- issues = validateWrite(changes, { vault, config, notes, operation: request.operation });
157
+ issues = validateWrite(changes, { vault, config, notes, operation: request.operation, ...(options.piiSurface ? { piiSurface: options.piiSurface } : {}) });
154
158
  if (!issues.some(issue => issue.severity === "error")) issues.push(...duplicateWarnings(request, notes, changes));
155
159
  const input = { vault: request.vault, channel: request.channel ?? "cli", operation: request.operation, expectedShas: request.expectedShas ?? {}, log: request.log ?? null };
156
160
  const identity = lstatSync(root);
157
- fingerprint = sha256(canonical({ input, vault, policy: { denyPaths: config.denyPaths ?? [], piiPatterns: config.piiPatterns ?? [] }, root, identity: [identity.dev, identity.ino], files: [...notes].map(([path, text]) => [path, sha256(text)]), changes: receipts(changes) }));
161
+ // The identity every later commit check uses must still be the bound root, not a directory swapped in meanwhile.
162
+ if (options.binding && (Number(identity.dev) !== options.binding.dev || Number(identity.ino) !== options.binding.ino)) refuse("VAULT_DRIFT", "vault root changed since the run started; nothing written");
163
+ fingerprint = sha256(canonical({ input, vault, policy: { denyPaths: config.denyPaths ?? [], piiPatterns: config.piiPatterns ?? [], piiSurface: options.piiSurface ?? "notes" }, root, identity: [identity.dev, identity.ino], files: [...notes].map(([path, text]) => [path, sha256(text)]), changes: receipts(changes) }));
158
164
  if (twoStep && request.confirm === true && (typeof request.fingerprint !== "string" || request.fingerprint !== fingerprint)) refuse("V09", "stale or missing preview fingerprint");
159
165
  if (issues.some(issue => issue.severity === "error")) return result = { ok: false, applied: false, issues, changes: receipts(changes), fingerprint, indexUpdated: false };
160
166
  if (preview) return result = { ok: true, applied: false, preview: true, ...(request.operation.op === "link" && !changes.length ? {unchanged:true} : {}), issues, changes: receipts(changes), fingerprint, indexUpdated: false };