@openparachute/vault 0.6.4-rc.1 → 0.6.4-rc.11

package/README.md

@@ -273,6 +273,31 @@ description: |
 
 Sent as the MCP server instruction at session start.
 
+### Schema enforcement & state machines (vault#299)
+
+Tag-field schemas (`update-tag`'s `fields`) are **advisory by default** — a violating write succeeds and the response carries `validation_status` warnings the agent can self-correct on. That's right for organic single-vault use. Once a vault has multiple writers (humans + agents), opt individual fields into **strict** enforcement:
+
+```yaml
+tag: concept-seed
+fields:
+  state:    { type: string, enum: [idea, drafted, scripted, produced, published, killed], strict: true }
+  title:    { type: string, required: true, strict: true }
+  owners:   { type: array,  cardinality: many, strict: true }
+  priority: { type: number }   # advisory (default)
+  notes:    { type: string }   # advisory — free-form stays free-form
+```
+
+`strict: true` flips **all** of that field's declared constraints together — `type` + `enum` + `required` + `cardinality` — from warnings to hard write rejections (all-or-nothing per field). A violating create/update is rejected with a single `schema_validation` error carrying **every** per-field violation (MCP `data.error_type: "schema_validation"` / HTTP 422), and nothing is written. Strictness is **per-field**, so a free-form `notes` field stays advisory on an otherwise-strict tag. Declared strict fields surface in `vault-info` and the connect-time brief so an agent knows the contract before writing.
+
+**Migration bypass.** A token holding the `vault:migrate` scope (broad) or `vault:<name>:migrate` (narrowed) skips strict enforcement — for backfilling/migrating existing non-conforming notes. It's an orthogonal capability (an `admin` token does **not** bypass unless it also holds `migrate`), and every bypassed write is logged to the daemon's structured log (`[schema-bypass] {...}` with the actor/via and waived violations) for later audit.
+
+**Compare-and-set state transitions.** Advance a state machine race-safely in one round trip instead of read → check → conditional update. `update-note`'s `state_transition: { field, from, to }` atomically sets `field` to `to` **only if** it currently equals `from`; otherwise it's rejected with a distinct `transition_conflict` error. A **missing field** counts as a conflict; pass `from: null` to match a field that is absent or explicitly null. A **transition-only** update needs no `if_updated_at`/`force` — the compare-and-set IS the precondition. Combinable with other field updates in the same call (they land in the same atomic UPDATE), but a combined call still needs `if_updated_at`/`force` for the *other* fields — the CAS only guards the transitioned field, not the rest of the merge.
+
+```jsonc
+// advance a seed from idea → drafted, race-safely
+update-note { "id": "Seeds/my-idea", "state_transition": { "field": "state", "from": "idea", "to": "drafted" } }
+```
+
 ## Features
 
 ### Wikilink auto-linking
@@ -321,7 +346,19 @@ triggers:
       send: content
 ```
 
-**Predicate fields**: `tags` (all must match), `has_content` (true/false), `missing_metadata` (keys that must be absent), `has_metadata` (keys that must be present).
+**Predicate fields**: `tags` (all must match), `has_content` (true/false), `missing_metadata` (keys that must be absent), `has_metadata` (keys that must be present), and `metadata` (value-matched — a field → operator-object map using the same operators as `query-notes`: `eq/ne/gt/gte/lt/lte/in/not_in/exists`). All predicate fields are ANDed. Value-matching lets a trigger fire on a specific transition rather than every edit (vault#299):
+
+```yaml
+  - name: announce-published
+    events: [updated]
+    when:
+      tags: [piece]
+      metadata:
+        state: { eq: published }     # fire only when state reaches "published"
+      missing_metadata: [announced_at]
+    action:
+      webhook: https://example.com/announce
+```
 
 **Two-phase markers**: On match, the trigger sets `<name>_pending_at` metadata before calling the webhook, then replaces it with `<name>_rendered_at` on success. This prevents re-entry and concurrent runs.
 
@@ -549,16 +586,12 @@ Range params require content in the response — with `include_content=false` (o
 
 ### Incremental rebuilds: "what changed since X"
 
-The SSG / sync pattern. Two equivalent forms — bracket-style is canonical going forward; the flat form is the same shape that ships through the REST/MCP date filter today.
+The SSG / sync pattern. Bracket-style is the query-string date filter. (The flat `date_field` / `date_from` / `date_to` params were removed in 0.6.4 — vault#288 — and are now ignored.)
 
 ```bash
-# Bracket-style (canonical)
+# Bracket-style (the query-string date filter)
 curl -H "Authorization: Bearer $VAULT_TOKEN" \
   "http://localhost:1940/vault/default/api/notes?meta[updated_at][gte]=2026-04-01T00:00:00Z"
-
-# Flat form (DEPRECATED in 0.4.3; planned removal in a later 0.x per vault#288)
-curl -H "Authorization: Bearer $VAULT_TOKEN" \
-  "http://localhost:1940/vault/default/api/notes?date_field=updated_at&date_from=2026-04-01T00:00:00Z"
 ```
 
 ```jsonc
@@ -910,7 +943,7 @@ cp .env.example .env   # edit with your config
 docker compose up -d
 ```
 
-Optionally set `PARACHUTE_VAULT_NAME` to choose a name for your first vault (defaults to `default`). Lowercase alphanumeric + hyphens or underscores, 2–32 chars.
+Set `PARACHUTE_VAULT_NAME` to auto-create a first vault on startup (name: lowercase alphanumeric + hyphens or underscores, 2–32 chars). Without it the server boots with zero vaults — create one explicitly via the admin wizard, `parachute init --vault-name <name>`, or `parachute-vault create <name>`.
 
 ### Cloud platforms
 
@@ -935,9 +968,11 @@ typical v0.6 self-host wants. If you're not sure, use the hub-managed
 path above. The standalone Blueprint stays in tree because some
 operators specifically want this shape (vault#341).
 
-Optionally set `PARACHUTE_VAULT_NAME` to choose a name for your first
-vault (defaults to `default`). Lowercase alphanumeric + hyphens or
-underscores, 2–32 chars.
+Set `PARACHUTE_VAULT_NAME` to auto-create a first vault on startup (name:
+lowercase alphanumeric + hyphens or underscores, 2–32 chars). Without it
+the server boots with zero vaults — create one explicitly via the admin
+wizard, `parachute init --vault-name <name>`, or `parachute-vault create
+<name>`.
 
 #### Other platforms
 

package/core/src/attribution.test.ts

@@ -0,0 +1,273 @@
+/**
+ * Write-attribution (vault#298) — per-identity (`*_by`) + per-interface
+ * (`*_via`) provenance at the store layer.
+ *
+ * Covers the core contract:
+ *   - create stamps BOTH axes onto created_* AND mirrors into last_updated_*
+ *   - update bumps ONLY last_updated_* (created_* is set-once)
+ *   - a `skipUpdatedAt` machine write does NOT claim authorship
+ *   - no-attribution writes leave the columns NULL (never "")
+ *   - the four query-notes filters (created_by / last_updated_by /
+ *     created_via / last_updated_via)
+ *   - legacy rows written before the v23 migration stay NULL and don't crash
+ *   - the cursor binds the attribution filters (changing them invalidates it)
+ *
+ * The auth → AuthResult → MCP/REST threading is exercised separately in
+ * src/attribution-threading.test.ts (it needs the server-side auth context).
+ */
+import { describe, it, expect, beforeEach } from "bun:test";
+import { Database } from "bun:sqlite";
+import { SqliteStore } from "./store.js";
+import { initSchema, SCHEMA_VERSION } from "./schema.js";
+
+let store: SqliteStore;
+let db: Database;
+
+beforeEach(() => {
+  db = new Database(":memory:");
+  store = new SqliteStore(db);
+});
+
+function rawRow(id: string) {
+  return db
+    .prepare(
+      "SELECT created_by, created_via, last_updated_by, last_updated_via FROM notes WHERE id = ?",
+    )
+    .get(id) as {
+    created_by: string | null;
+    created_via: string | null;
+    last_updated_by: string | null;
+    last_updated_via: string | null;
+  };
+}
+
+describe("write-attribution — schema", () => {
+  it("bumped SCHEMA_VERSION to 23", () => {
+    expect(SCHEMA_VERSION).toBe(23);
+  });
+
+  it("the four columns exist and are indexed", () => {
+    const cols = (db.prepare("PRAGMA table_info(notes)").all() as { name: string }[]).map(
+      (r) => r.name,
+    );
+    expect(cols).toContain("created_by");
+    expect(cols).toContain("created_via");
+    expect(cols).toContain("last_updated_by");
+    expect(cols).toContain("last_updated_via");
+
+    const idx = (
+      db
+        .prepare("SELECT name FROM sqlite_master WHERE type='index' AND tbl_name='notes'")
+        .all() as { name: string }[]
+    ).map((r) => r.name);
+    expect(idx).toContain("idx_notes_created_by");
+    expect(idx).toContain("idx_notes_created_via");
+    expect(idx).toContain("idx_notes_last_updated_by");
+    expect(idx).toContain("idx_notes_last_updated_via");
+  });
+});
+
+describe("write-attribution — per-identity (WHO) + per-interface (VIA)", () => {
+  it("create captures actor + via on BOTH the created_* and last_updated_* pairs", async () => {
+    const note = await store.createNote("hi", { actor: "aaron", via: "mcp" });
+    // Read-model surfaces both axes.
+    expect(note.createdBy).toBe("aaron");
+    expect(note.createdVia).toBe("mcp");
+    // First write IS the most-recent write — mirrored.
+    expect(note.lastUpdatedBy).toBe("aaron");
+    expect(note.lastUpdatedVia).toBe("mcp");
+
+    const row = rawRow(note.id);
+    expect(row.created_by).toBe("aaron");
+    expect(row.created_via).toBe("mcp");
+    expect(row.last_updated_by).toBe("aaron");
+    expect(row.last_updated_via).toBe("mcp");
+  });
+
+  it("update bumps last_updated_* but leaves created_* untouched (set-once)", async () => {
+    const note = await store.createNote("v1", { actor: "aaron", via: "mcp" });
+    const updated = await store.updateNote(note.id, {
+      content: "v2",
+      actor: "mathilda",
+      via: "surface:meeting-ingest",
+    });
+
+    // created_* pins the ORIGINAL author/channel forever.
+    expect(updated.createdBy).toBe("aaron");
+    expect(updated.createdVia).toBe("mcp");
+    // last_updated_* reflects the most recent writer.
+    expect(updated.lastUpdatedBy).toBe("mathilda");
+    expect(updated.lastUpdatedVia).toBe("surface:meeting-ingest");
+
+    const row = rawRow(note.id);
+    expect(row.created_by).toBe("aaron");
+    expect(row.last_updated_by).toBe("mathilda");
+    expect(row.last_updated_via).toBe("surface:meeting-ingest");
+  });
+
+  it("append/prepend updates also record the most-recent author", async () => {
+    const note = await store.createNote("base", { actor: "aaron", via: "mcp" });
+    const after = await store.updateNote(note.id, {
+      append: " +more",
+      actor: "agent:nightly",
+      via: "agent:nightly",
+    });
+    expect(after.content).toBe("base +more");
+    expect(after.createdBy).toBe("aaron");
+    expect(after.lastUpdatedBy).toBe("agent:nightly");
+    expect(after.lastUpdatedVia).toBe("agent:nightly");
+  });
+
+  it("a skipUpdatedAt machine write does NOT claim authorship", async () => {
+    const note = await store.createNote("x", { actor: "aaron", via: "mcp" });
+    const before = rawRow(note.id);
+    // A hook-style marker write: passes actor/via but skipUpdatedAt — should
+    // not bump updated_at NOR rewrite last_updated_*.
+    await store.updateNote(note.id, {
+      metadata: { marker: true },
+      skipUpdatedAt: true,
+      actor: "some-hook",
+      via: "operator",
+    });
+    const after = rawRow(note.id);
+    expect(after.last_updated_by).toBe(before.last_updated_by); // still "aaron"
+    expect(after.last_updated_by).toBe("aaron");
+    expect(after.last_updated_via).toBe("mcp");
+  });
+
+  it("an attribution-less update honestly clears the most-recent author (NULL, not stale)", async () => {
+    const note = await store.createNote("x", { actor: "aaron", via: "mcp" });
+    // A real edit arriving with no attribution context — the latest writer is
+    // unknown; we must NOT leave "aaron" as the apparent last editor.
+    const after = await store.updateNote(note.id, { content: "y" });
+    expect(after.createdBy).toBe("aaron"); // original author preserved
+    expect(after.lastUpdatedBy).toBeNull();
+    expect(after.lastUpdatedVia).toBeNull();
+  });
+});
+
+describe("write-attribution — no context → NULL (never empty string)", () => {
+  it("create without actor/via leaves all four columns NULL", async () => {
+    const note = await store.createNote("anon");
+    const row = rawRow(note.id);
+    expect(row.created_by).toBeNull();
+    expect(row.created_via).toBeNull();
+    expect(row.last_updated_by).toBeNull();
+    expect(row.last_updated_via).toBeNull();
+    expect(note.createdBy).toBeNull();
+    expect(note.createdVia).toBeNull();
+  });
+
+  it("empty / whitespace-only actor/via normalize to NULL", async () => {
+    const note = await store.createNote("e", { actor: "   ", via: "" });
+    const row = rawRow(note.id);
+    expect(row.created_by).toBeNull();
+    expect(row.created_via).toBeNull();
+  });
+});
+
+describe("write-attribution — query filters", () => {
+  beforeEach(async () => {
+    await store.createNote("a", { actor: "aaron", via: "mcp" });
+    await store.createNote("b", { actor: "mathilda", via: "surface:meeting-ingest" });
+    const c = await store.createNote("c", { actor: "aaron", via: "api" });
+    // c's most-recent write is by the nightly agent via mcp.
+    await store.updateNote(c.id, { content: "c2", actor: "agent:nightly", via: "mcp" });
+    await store.createNote("legacy"); // no attribution
+  });
+
+  it("created_by selects only that principal's CREATED notes", async () => {
+    const aaron = await store.queryNotes({ createdBy: "aaron" });
+    expect(aaron.map((n) => n.content).sort()).toEqual(["a", "c2"]);
+  });
+
+  it("last_updated_by selects by MOST-RECENT author (agent's edit on c)", async () => {
+    const byAgent = await store.queryNotes({ lastUpdatedBy: "agent:nightly" });
+    expect(byAgent.map((n) => n.content)).toEqual(["c2"]);
+    // aaron created c but is no longer its last editor.
+    const aaronLatest = await store.queryNotes({ lastUpdatedBy: "aaron" });
+    expect(aaronLatest.map((n) => n.content)).toEqual(["a"]);
+  });
+
+  it("created_via / last_updated_via filter on the interface axis", async () => {
+    const viaSurface = await store.queryNotes({ createdVia: "surface:meeting-ingest" });
+    expect(viaSurface.map((n) => n.content)).toEqual(["b"]);
+    const lastViaMcp = await store.queryNotes({ lastUpdatedVia: "mcp" });
+    // "a" (created via mcp, never updated) + "c2" (last updated via mcp).
+    expect(lastViaMcp.map((n) => n.content).sort()).toEqual(["a", "c2"]);
+  });
+
+  it("an attribution filter never matches legacy/unattributed (NULL) rows", async () => {
+    const any = await store.queryNotes({ createdBy: "aaron" });
+    expect(any.some((n) => n.content === "legacy")).toBe(false);
+  });
+});
+
+describe("write-attribution — legacy backfill (v23 migration)", () => {
+  it("a pre-v23 vault: existing rows stay NULL, new rows attribute, no crash", () => {
+    // Build a v22-shaped notes table by hand (no attribution columns) +
+    // schema_version row, then run initSchema to drive the v23 migration.
+    const legacy = new Database(":memory:");
+    legacy.exec(`
+      CREATE TABLE notes (
+        id TEXT PRIMARY KEY,
+        content TEXT DEFAULT '',
+        path TEXT,
+        metadata TEXT DEFAULT '{}',
+        created_at TEXT NOT NULL,
+        updated_at TEXT,
+        extension TEXT NOT NULL DEFAULT 'md'
+      );
+      CREATE TABLE schema_version (version INTEGER PRIMARY KEY, applied_at TEXT);
+      INSERT INTO schema_version (version, applied_at) VALUES (22, '2026-01-01T00:00:00.000Z');
+      INSERT INTO notes (id, content, created_at, updated_at)
+        VALUES ('old-1', 'pre-existing', '2026-01-01T00:00:00.000Z', '2026-01-01T00:00:00.000Z');
+    `);
+
+    // Should migrate without throwing.
+    initSchema(legacy);
+
+    const ver = (legacy.prepare("SELECT MAX(version) AS v FROM schema_version").get() as {
+      v: number;
+    }).v;
+    expect(ver).toBe(23);
+
+    // Legacy row: attribution columns exist now but are NULL (not fabricated).
+    const old = legacy
+      .prepare("SELECT created_by, last_updated_by, created_via FROM notes WHERE id = ?")
+      .get("old-1") as { created_by: string | null; last_updated_by: string | null; created_via: string | null };
+    expect(old.created_by).toBeNull();
+    expect(old.last_updated_by).toBeNull();
+    expect(old.created_via).toBeNull();
+
+    // A NEW write on the migrated DB attributes correctly.
+    const migratedStore = new SqliteStore(legacy);
+    return migratedStore.createNote("fresh", { actor: "aaron", via: "mcp" }).then((n) => {
+      expect(n.createdBy).toBe("aaron");
+      expect(n.createdVia).toBe("mcp");
+      // The legacy row is still NULL and still queryable.
+      return migratedStore.queryNotes({ createdBy: "aaron" }).then((res) => {
+        expect(res.map((x) => x.content)).toEqual(["fresh"]);
+      });
+    });
+  });
+
+  it("re-running the v23 migration is idempotent (no duplicate-column error)", () => {
+    const again = new Database(":memory:");
+    const s = new SqliteStore(again); // runs migrations once
+    initSchema(again); // run again — must not throw
+    expect(() => s.createNote("ok", { actor: "x", via: "api" })).not.toThrow();
+  });
+});
+
+describe("write-attribution — cursor binding", () => {
+  it("changing an attribution filter between cursor polls invalidates the cursor", async () => {
+    await store.createNote("a", { actor: "aaron", via: "mcp" });
+    const page = await store.queryNotesPaged({ createdBy: "aaron" });
+    // Reusing the cursor under a DIFFERENT createdBy must be rejected, not
+    // silently continue aaron's watermark.
+    await expect(
+      store.queryNotesPaged({ createdBy: "mathilda", cursor: page.next_cursor }),
+    ).rejects.toThrow(/different query/i);
+  });
+});

package/core/src/conformance.test.ts

@@ -0,0 +1,112 @@
+/**
+ * Conformance check (vault#283) — counting existing notes that would violate
+ * a proposed tightening of a tag's field spec. Backs the Schema editor's
+ * "N existing notes violate this" warning.
+ */
+import { describe, it, expect, beforeEach } from "bun:test";
+import { Database } from "bun:sqlite";
+import { SqliteStore } from "./store.js";
+import { countConformanceViolations } from "./conformance.js";
+
+let db: Database;
+let store: SqliteStore;
+
+beforeEach(() => {
+  db = new Database(":memory:");
+  store = new SqliteStore(db);
+});
+
+describe("countConformanceViolations", () => {
+  it("returns zero violations when proposed fields is empty", async () => {
+    await store.createNote("n", { tags: ["task"], metadata: { status: "open" } });
+    const r = countConformanceViolations(db, "task", {});
+    expect(r.total_notes).toBe(1);
+    expect(r.violating_notes).toBe(0);
+    expect(r.checked_fields).toEqual([]);
+  });
+
+  it("counts notes missing a newly-required field", async () => {
+    await store.createNote("has", { tags: ["task"], metadata: { due: "2026-01-01" } });
+    await store.createNote("missing-1", { tags: ["task"], metadata: {} });
+    await store.createNote("missing-2", { tags: ["task"] });
+
+    const r = countConformanceViolations(db, "task", {
+      due: { type: "string", required: true },
+    });
+    expect(r.total_notes).toBe(3);
+    expect(r.violating_notes).toBe(2);
+    expect(r.checked_fields).toEqual(["due"]);
+    // missing_required reasons surface in the sample.
+    expect(r.sample.every((s) => s.fields.some((f) => f.reason === "missing_required"))).toBe(true);
+  });
+
+  it("counts notes whose value falls outside a narrowed enum", async () => {
+    await store.createNote("ok", { tags: ["task"], metadata: { status: "open" } });
+    await store.createNote("bad", { tags: ["task"], metadata: { status: "archived" } });
+
+    const r = countConformanceViolations(db, "task", {
+      status: { type: "string", enum: ["open", "done"] },
+    });
+    expect(r.violating_notes).toBe(1);
+    expect(r.sample[0]?.fields[0]?.reason).toBe("enum_mismatch");
+  });
+
+  it("counts notes whose value contradicts a changed type", async () => {
+    await store.createNote("num", { tags: ["task"], metadata: { count: 3 } });
+    await store.createNote("str", { tags: ["task"], metadata: { count: "three" } });
+
+    const r = countConformanceViolations(db, "task", {
+      count: { type: "integer" },
+    });
+    expect(r.violating_notes).toBe(1);
+    expect(r.sample[0]?.fields[0]?.reason).toBe("type_mismatch");
+  });
+
+  it("includes descendant-tag notes via inheritance", async () => {
+    // dev/log declares dev as a parent → a strict field on dev applies to
+    // dev/log notes too.
+    await store.upsertTagRecord("dev/log", { parent_names: ["dev"] });
+    await store.createNote("parent-note", { tags: ["dev"], metadata: {} });
+    await store.createNote("child-note", { tags: ["dev/log"], metadata: {} });
+
+    const r = countConformanceViolations(db, "dev", {
+      kind: { type: "string", required: true },
+    });
+    expect(r.total_notes).toBe(2);
+    expect(r.violating_notes).toBe(2);
+  });
+
+  it("ignores fields the proposal does not touch", async () => {
+    await store.createNote("n", { tags: ["task"], metadata: { other: 123 } });
+    // We only propose `status`; `other` being weird is irrelevant.
+    const r = countConformanceViolations(db, "task", {
+      status: { type: "string" },
+    });
+    // `status` absent + not required → no violation.
+    expect(r.violating_notes).toBe(0);
+  });
+
+  it("caps the sample at sampleLimit but counts all violations", async () => {
+    for (let i = 0; i < 5; i++) {
+      await store.createNote(`n${i}`, { tags: ["task"], metadata: {} });
+    }
+    const r = countConformanceViolations(
+      db,
+      "task",
+      { req: { type: "string", required: true } },
+      { sampleLimit: 2 },
+    );
+    expect(r.violating_notes).toBe(5);
+    expect(r.sample.length).toBe(2);
+  });
+});
+
+describe("store.countTagConformance", () => {
+  it("proxies to the core helper", async () => {
+    await store.createNote("n", { tags: ["task"], metadata: {} });
+    const r = await store.countTagConformance("task", {
+      due: { type: "string", required: true },
+    });
+    expect(r.violating_notes).toBe(1);
+  });
+});