auggy 0.3.0

package/src/augments/layered-memory/skill/SKILL.md

@@ -0,0 +1,153 @@
+---
+name: layered-memory
+description: When and how to use memory_write, memory_search, memory_list, memory_forget for peer-scoped episodic memory. Read this before saving or recalling anything about a peer.
+---
+
+# Layered memory
+
+You have access to **peer-scoped episodic memory** — entries you save are bound to the peer who is talking with you in the current turn, and your search results return only entries that peer wrote (or that were written about them). Different peers do not see each other's memory.
+
+This memory is for things you learn turn-by-turn — preferences, names, commitments, recurring topics — not for facts about the agent itself or the operator's organization (those live elsewhere in your context).
+
+## Tools
+
+| Tool | What it does | When to call |
+|------|--------------|--------------|
+| `memory_search(query)` | Full-text search over the current peer's memory entries | At the start of a turn with a returning peer; when the peer references something they said before |
+| `memory_write(label, content)` | Persist content under a peer-scoped label | When the peer introduces themselves, states a preference, makes a commitment, or asks to be remembered |
+| `memory_list()` | List available labels and namespaces visible to this peer | When you want to confirm what's already stored before writing or searching |
+| `memory_forget(peerId)` | Wipe ALL episodic memory for a specific peer | Right-to-erasure requests. Requires elevated trust (operator or creator); a regular peer cannot trigger it |
+
+`memory_read(label)` exists but **does not work on episodic labels**. Episodic memory is peer-scoped, and reading by label would bypass that scoping. Use `memory_search` instead. (`memory_read` still works for non-episodic static labels like `self` — the agent's identity preamble — when those are configured.)
+
+## How labels are structured
+
+Labels in this layer must start with the configured **namespace prefix** (set by the operator — typically the agent's name with a colon). When a peer is talking with you, their entries must additionally be scoped to their peer ID.
+
+The shape:
+
+```
+<prefix><peerId>            ← write a single entry for this peer
+<prefix><peerId>:<topic>    ← write multiple entries grouped by topic
+```
+
+**Examples** (assuming the namespace prefix is `concierge:` and the peer id is `vis_a1b2c3`):
+
+```
+concierge:vis_a1b2c3                    ← one umbrella entry
+concierge:vis_a1b2c3:preferences        ← topic-grouped
+concierge:vis_a1b2c3:commitments        ← topic-grouped
+```
+
+If you try to write a label that doesn't include the current peer's ID in the right shape, the write fails with a clear error. Don't try to outsmart this — it is a structural protection against one peer seeding entries against another.
+
+## When to call `memory_write`
+
+Save when the peer:
+
+- Introduces themselves by name (`memory_write("concierge:vis_a1b2c3", "Sam from Acme — interested in research tooling.")`)
+- States a preference you should honor in future turns ("I prefer concise responses.")
+- Makes a commitment ("I'll get back to you Tuesday with the budget.")
+- Explicitly asks to be remembered ("Remember that I'm working on X.")
+- Tells you a piece of context that will matter later in the conversation or in a future one
+
+**Don't save** every conversational turn. The peer's full message history is already part of your context. Memory is for things you'd want to recall in a *future* conversation, not the current one.
+
+## When to call `memory_search`
+
+Call once at the **start of a turn with a returning peer** — that gives you the lightweight context they've built with you over time. Treat the results as background to inform tone and recall, not as a script to recite.
+
+Call again later in the turn if the peer references something specific they said before ("like I mentioned earlier…", "going back to that thing about…"). A targeted search beats trying to scroll mental context.
+
+Keep query terms short — search uses keyword matching against entry content. Two or three meaningful words beats a sentence.
+
+## When to call `memory_list`
+
+Use when you want to know what labels already exist for the current peer before writing a new one — helps you decide between writing a fresh label vs. updating an existing topic. Inexpensive; safe to call.
+
+## What you cannot do
+
+- You cannot read another peer's entries. The system filters search and list results by the peer in the current turn.
+- You cannot write a label that's missing the current peer's ID in the right shape.
+- You cannot use `memory_read` on an episodic label — only `memory_search`.
+- You cannot bulk-delete entries. `memory_forget` wipes everything for one peer (and only with elevated trust); fine-grained deletion isn't a model-callable operation.
+
+## Common mistakes
+
+| Wrong | Correct |
+|-------|---------|
+| `memory_read("ep:...")` on an episodic label | `memory_search("relevant query")` |
+| Writing a label with the wrong namespace prefix | Use the prefix from the operator's config (typically the agent name with a colon) |
+| Writing a peer-scoped label without the peer's ID | Always shape labels as `<prefix><peerId>` or `<prefix><peerId>:<topic>` |
+| Writing every turn as a "remember this" | Save only what you'd want to recall in a *future* conversation |
+| Searching with a long natural-language query | Keep queries to a few keywords |
+| Calling `memory_forget` on your own initiative | Only call it when the peer (or operator) explicitly requests erasure |
+
+## Workflow
+
+### Returning peer says hello
+
+1. `memory_search("recent")` or a query relevant to the conversation
+2. Read the entries returned; let them inform your reply naturally — don't dump them back at the peer
+3. Continue the conversation; save NEW things you learn via `memory_write`
+
+### Peer states a preference or commitment
+
+1. (Optional) `memory_list()` to see if a related label already exists
+2. `memory_write("<prefix><peerId>:<topic>", "<concise description>")`
+3. Acknowledge briefly ("Got it, I'll remember that.") — don't over-explain
+
+### Peer asks to be forgotten
+
+1. Confirm once that they want all their stored memory deleted
+2. If you have the trust level for it, call `memory_forget(peerId)` and confirm the count returned
+3. If you don't have the trust level, tell them you're flagging the request for the operator
+
+## Auto-saved entries — `[AGENT-DERIVED]` provenance
+
+### What auto-save does
+
+A background process extracts facts after each turn (or per the operator's configured cadence) and writes them to your peer-scoped memory. **You never invoke this process directly — it runs on its own.** The only observable effect is that `memory_search` results sometimes include entries carrying the `[AGENT-DERIVED]` marker.
+
+When you call `memory_search` and see an entry like:
+
+```
+[AGENT-DERIVED] Sam prefers concise replies and works at Acme Corp.
+```
+
+that entry came from the background process, not from a `memory_write` call you made.
+
+### `[AGENT-DERIVED]` entries are paraphrases, not verbatim
+
+When you see an entry with that marker, treat the content as an extracted paraphrase — a best-effort summary of what the peer said — **not** the peer's exact words. If precision matters (operator agreements, technical specifications, contact details, verbatim commitments), prefer entries marked `[PEER-DERIVED]` or entries you wrote explicitly with `memory_write` when the peer stated something exactly.
+
+### Trust hierarchy on conflict
+
+If two entries about the same fact conflict, trust them in this order:
+
+1. `[PEER-DERIVED]` entries (or operator-set verbatim entries) — authoritative; the peer's own words or an operator-confirmed record
+2. `[AGENT-DERIVED]` entries — useful background; defer to the above when they contradict
+
+**Never overwrite a `[PEER-DERIVED]` entry with an extracted paraphrase.** If a peer corrects something that the background process extracted incorrectly, write the correction via `memory_write` — the new entry coexists with the old one and the trust hierarchy ensures the peer's explicit statement is preferred. If the conflict is meaningful, surface it to the peer briefly.
+
+### When to call `memory_write` directly anyway
+
+The background process runs after each turn. You do not need to wait for it. Call `memory_write` mid-turn when:
+
+- The peer explicitly asks to be remembered ("save my email as foo@example.com")
+- You want to capture the peer's exact phrasing verbatim (commitments, technical specs, contact details)
+- You are correcting a fact you know to be wrong from a prior extraction
+- The signal is high enough that you do not want to risk it being missed
+
+Both writes coexist in memory. The background process does not overwrite your explicit `memory_write` calls, and your calls do not overwrite background-extracted entries — they accumulate and retrieval ranks them by the trust hierarchy above.
+
+### Privacy boundaries
+
+Some content should never reach memory, regardless of who writes it:
+
+- Secrets and credentials (API keys, passwords, tokens)
+- Content the peer explicitly marked as confidential
+- Sensitive personal information outside what the agent's purpose warrants
+- Anything the peer asked to be forgotten
+
+The background extraction process embeds a privacy guard in its prompt template, but you are also responsible for respecting the same boundary in your own `memory_write` calls. If a peer shares a secret mid-turn, do not save it.

package/src/augments/layered-memory/storage/migrations/README.md

@@ -0,0 +1,16 @@
+# Layered-memory storage migrations
+
+## SQLite
+
+Migrations run automatically at augment boot via PRAGMA-checked ALTERs (idempotent). No operator action needed.
+
+## Supabase
+
+Manual application required:
+
+1. Open Supabase SQL editor for the project hosting this agent's memory.
+2. Run `supabase-add-fact-fields.sql`.
+3. Verify via the table editor that new columns are present.
+4. Deploy the new layered-memory version.
+
+Layered-memory's runtime startup verifies the schema on first connect; if columns are missing, it logs a structured warning and falls back to writing without the new fields (auto-save still works; structured-fact retrieval is degraded).

package/src/augments/layered-memory/storage/migrations/supabase-add-fact-fields.sql

@@ -0,0 +1,9 @@
+-- Adds Phase 2 fact-fields. Apply via Supabase SQL editor or `supabase db push`.
+
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS subject TEXT;
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS predicate TEXT;
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS object TEXT;
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS source_turn_id TEXT;
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS origin TEXT;
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS is_verbatim BOOLEAN;
+ALTER TABLE memory_entries ADD COLUMN IF NOT EXISTS retention_class TEXT;

package/src/augments/layered-memory/storage/sqlite-store.ts

@@ -0,0 +1,352 @@
+import { Database } from "bun:sqlite";
+import { randomUUID } from "node:crypto";
+import type {
+  MemoryStore,
+  RetentionClass,
+  SqliteStoreConfig,
+  StoreEntry,
+  WriteAutoSavedArgs,
+} from "./types";
+import type { TrustLevel } from "../../../types";
+
+// Each statement run individually to keep the SQL surface explicit.
+const SCHEMA_STATEMENTS = [
+  `CREATE TABLE IF NOT EXISTS entries (
+    id              TEXT PRIMARY KEY,
+    label           TEXT NOT NULL,
+    content         TEXT NOT NULL,
+    peer_id         TEXT,
+    trust_level     TEXT,
+    created_at      INTEGER NOT NULL,
+    superseded_by   TEXT,
+    retention_class TEXT NOT NULL DEFAULT 'operational',
+    is_verbatim     INTEGER NOT NULL DEFAULT 0,
+    provenance_model TEXT,
+    confidence      REAL,
+    embedding_model TEXT,
+    scope           TEXT NOT NULL DEFAULT 'peer',
+    expires_at      INTEGER
+  )`,
+  `CREATE INDEX IF NOT EXISTS idx_entries_peer ON entries(peer_id)`,
+  `CREATE INDEX IF NOT EXISTS idx_entries_label ON entries(label)`,
+  `CREATE INDEX IF NOT EXISTS idx_entries_created ON entries(created_at DESC)`,
+  `CREATE INDEX IF NOT EXISTS idx_entries_expires ON entries(expires_at) WHERE expires_at IS NOT NULL`,
+  `CREATE TABLE IF NOT EXISTS event_log (
+    id        TEXT PRIMARY KEY,
+    entry_id  TEXT NOT NULL,
+    action    TEXT NOT NULL,
+    peer_id   TEXT,
+    timestamp INTEGER NOT NULL,
+    detail    TEXT
+  )`,
+  `CREATE INDEX IF NOT EXISTS idx_events_entry ON event_log(entry_id)`,
+];
+
+interface Row {
+  id: string;
+  label: string;
+  content: string;
+  peer_id: string | null;
+  trust_level: string | null;
+  created_at: number;
+  superseded_by: string | null;
+  retention_class: string;
+  is_verbatim: number;
+  expires_at: number | null;
+  // Phase 2 fact-fields (nullable; populated by writeAutoSavedEntry)
+  subject: string | null;
+  predicate: string | null;
+  object: string | null;
+  source_turn_id: string | null;
+  origin: string | null;
+}
+
+function rowToEntry(row: Row): StoreEntry {
+  const entry: StoreEntry = {
+    id: row.id,
+    label: row.label,
+    content: row.content,
+    peerId: row.peer_id,
+    trustLevel: row.trust_level as TrustLevel | null,
+    createdAt: row.created_at,
+    supersededBy: row.superseded_by,
+    retentionClass: row.retention_class as RetentionClass,
+    isVerbatim: row.is_verbatim === 1,
+    expiresAt: row.expires_at,
+  };
+  // Read path: populate fact-fields only when present in storage. Legacy
+  // rows (pre-Phase-2) carry NULLs and stay clean on the way out.
+  if (row.subject != null) entry.subject = row.subject;
+  if (row.predicate != null) entry.predicate = row.predicate;
+  if (row.object != null) entry.object = row.object;
+  if (row.source_turn_id != null) entry.sourceTurnId = row.source_turn_id;
+  if (row.origin != null) entry.origin = row.origin as StoreEntry["origin"];
+  return entry;
+}
+
+// Sampled cleanup: every ~Nth write triggers a bounded DELETE so write
+// latency doesn't depend on stale-data backlog. Capped via LIMIT so even
+// a huge backlog drains in roughly constant time per write.
+const CLEANUP_SAMPLE_RATE = 50;
+const CLEANUP_BATCH_SIZE = 100;
+
+export function createSqliteStore(config: SqliteStoreConfig): MemoryStore {
+  const db = new Database(config.dbPath, { create: true });
+  db.run("PRAGMA journal_mode = WAL");
+  db.run("PRAGMA foreign_keys = ON");
+
+  // Schema must exist before we prepare statements that reference it.
+  for (const stmt of SCHEMA_STATEMENTS) {
+    db.run(stmt);
+  }
+
+  // Phase 2 migration: add structured-fact + provenance columns idempotently.
+  // PRAGMA table_info detects which columns already exist; ALTER TABLE adds
+  // only the absent ones. Existing rows survive with NULLs in the new columns.
+  // is_verbatim + retention_class are already in SCHEMA_STATEMENTS above;
+  // they appear in the list so the migration is self-documenting and safe to
+  // re-run if applied to a DB that predates them (legacy schema path).
+  function ensureMigrations(): void {
+    const cols = db.prepare("PRAGMA table_info(entries)").all() as { name: string }[];
+    const colNames = new Set(cols.map((c) => c.name));
+
+    const additions: Array<{ name: string; ddl: string }> = [
+      { name: "subject", ddl: "ALTER TABLE entries ADD COLUMN subject TEXT" },
+      { name: "predicate", ddl: "ALTER TABLE entries ADD COLUMN predicate TEXT" },
+      { name: "object", ddl: "ALTER TABLE entries ADD COLUMN object TEXT" },
+      { name: "source_turn_id", ddl: "ALTER TABLE entries ADD COLUMN source_turn_id TEXT" },
+      { name: "origin", ddl: "ALTER TABLE entries ADD COLUMN origin TEXT" },
+      { name: "is_verbatim", ddl: "ALTER TABLE entries ADD COLUMN is_verbatim INTEGER" },
+      { name: "retention_class", ddl: "ALTER TABLE entries ADD COLUMN retention_class TEXT" },
+    ];
+
+    for (const { name, ddl } of additions) {
+      if (!colNames.has(name)) db.run(ddl);
+    }
+  }
+
+  ensureMigrations();
+
+  const retentionMs = config.retentionDays * 24 * 60 * 60 * 1000;
+
+  // Pre-compiled statements live as long as the connection.
+  const insertEntryStmt = db.prepare(
+    `INSERT INTO entries (id, label, content, peer_id, trust_level, created_at, superseded_by, retention_class, is_verbatim, expires_at)
+     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+  );
+  const insertEventStmt = db.prepare(
+    "INSERT INTO event_log (id, entry_id, action, peer_id, timestamp, detail) VALUES (?, ?, ?, ?, ?, ?)",
+  );
+  const cleanupStmt = db.prepare(
+    "DELETE FROM entries WHERE id IN (SELECT id FROM entries WHERE expires_at IS NOT NULL AND expires_at < ? LIMIT ?)",
+  );
+
+  type SqlBinding = string | number | bigint | boolean | null | Uint8Array;
+
+  // writeAndLog runs the entry insert and audit insert atomically. If
+  // either fails, both roll back — callers either see a successful
+  // write that's fully recorded, or an error and zero side effects.
+  const writeAndLog = db.transaction((entryParams: SqlBinding[], eventParams: SqlBinding[]) => {
+    insertEntryStmt.run(...entryParams);
+    insertEventStmt.run(...eventParams);
+  });
+
+  async function initialize(): Promise<void> {
+    // Schema is now created at construction time. Kept as a no-op for
+    // contract symmetry with SupabaseStore (whose schema lives in
+    // migrations).
+  }
+
+  function logEvent(entryId: string, action: string, peerId: string | null, detail?: object) {
+    insertEventStmt.run(
+      randomUUID(),
+      entryId,
+      action,
+      peerId,
+      Date.now(),
+      detail ? JSON.stringify(detail) : null,
+    );
+  }
+
+  async function write(input: Omit<StoreEntry, "id"> & { id?: string }): Promise<StoreEntry> {
+    const id = input.id ?? randomUUID();
+    const expiresAt = input.expiresAt ?? input.createdAt + retentionMs;
+
+    writeAndLog(
+      [
+        id,
+        input.label,
+        input.content,
+        input.peerId,
+        input.trustLevel,
+        input.createdAt,
+        input.supersededBy,
+        input.retentionClass,
+        input.isVerbatim ? 1 : 0,
+        expiresAt,
+      ],
+      [randomUUID(), id, "write", input.peerId, Date.now(), null],
+    );
+
+    // Sampled, bounded cleanup. Outside the transaction so a partial
+    // sweep can never roll back the user's write. ~1-in-50 writes pay
+    // the small constant DELETE cost; 49-in-50 writes pay nothing.
+    if (Math.random() * CLEANUP_SAMPLE_RATE < 1) {
+      const result = cleanupStmt.run(Date.now(), CLEANUP_BATCH_SIZE);
+      if (result.changes > 0) {
+        logEvent("(batch)", "expire-sweep", null, { swept: result.changes });
+      }
+    }
+
+    return { ...input, id, expiresAt };
+  }
+
+  async function search(query: string, peerId?: string, limit = 10): Promise<StoreEntry[]> {
+    const escaped = query.replace(/[%_\\]/g, (c) => `\\${c}`);
+    const pattern = `%${escaped}%`;
+    const now = Date.now();
+
+    if (peerId) {
+      const rows = db
+        .prepare<Row, [string, string, number, number]>(
+          `SELECT * FROM entries
+           WHERE peer_id = ?
+             AND content LIKE ? ESCAPE '\\'
+             AND superseded_by IS NULL
+             AND (expires_at IS NULL OR expires_at >= ?)
+           ORDER BY created_at DESC
+           LIMIT ?`,
+        )
+        .all(peerId, pattern, now, limit);
+      return rows.map(rowToEntry);
+    }
+
+    const rows = db
+      .prepare<Row, [string, number, number]>(
+        `SELECT * FROM entries
+         WHERE content LIKE ? ESCAPE '\\'
+           AND superseded_by IS NULL
+           AND (expires_at IS NULL OR expires_at >= ?)
+         ORDER BY created_at DESC
+         LIMIT ?`,
+      )
+      .all(pattern, now, limit);
+    return rows.map(rowToEntry);
+  }
+
+  async function read(label: string): Promise<StoreEntry | null> {
+    const row = db
+      .prepare<Row, [string]>(
+        "SELECT * FROM entries WHERE label = ? AND superseded_by IS NULL ORDER BY created_at DESC LIMIT 1",
+      )
+      .get(label);
+    return row ? rowToEntry(row) : null;
+  }
+
+  async function list(peerId?: string): Promise<string[]> {
+    if (peerId) {
+      const rows = db
+        .prepare<{ label: string }, [string]>(
+          "SELECT DISTINCT label FROM entries WHERE peer_id = ? AND superseded_by IS NULL ORDER BY label",
+        )
+        .all(peerId);
+      return rows.map((r) => r.label);
+    }
+    const rows = db
+      .prepare<{ label: string }, []>(
+        "SELECT DISTINCT label FROM entries WHERE superseded_by IS NULL ORDER BY label",
+      )
+      .all();
+    return rows.map((r) => r.label);
+  }
+
+  async function forget(peerId: string): Promise<number> {
+    const result = db.prepare("DELETE FROM entries WHERE peer_id = ?").run(peerId);
+    logEvent("(batch)", "forget", peerId, { deleted: result.changes });
+    return result.changes;
+  }
+
+  async function supersede(entryId: string, newEntryId: string): Promise<void> {
+    db.prepare("UPDATE entries SET superseded_by = ? WHERE id = ?").run(newEntryId, entryId);
+    logEvent(entryId, "supersede", null, { supersededBy: newEntryId });
+  }
+
+  async function cleanup(): Promise<number> {
+    const result = db
+      .prepare("DELETE FROM entries WHERE expires_at IS NOT NULL AND expires_at < ?")
+      .run(Date.now());
+    return result.changes;
+  }
+
+  // Internal-to-layered-memory write path used by the extractor. NOT
+  // exposed on any augment-public surface — Phase 2 of ADR-018,
+  // Decision 4 of the memorist design. `origin` is hardcoded to
+  // `"agent-derived"` here rather than accepted as an argument so a
+  // misbehaving extraction prompt cannot forge `operator` or
+  // `peer-derived`. Namespace prefix is enforced when configured;
+  // when absent, this function refuses entirely (the augment factory
+  // must always pass a namespace).
+  async function writeAutoSavedEntry(args: WriteAutoSavedArgs): Promise<void> {
+    if (!config.namespace) {
+      throw new Error(
+        "writeAutoSavedEntry: store has no namespace configured; auto-save requires namespace-prefix discipline",
+      );
+    }
+    const prefix = config.namespace.endsWith(":") ? config.namespace : `${config.namespace}:`;
+    if (!args.label.startsWith(prefix)) {
+      throw new Error(
+        `writeAutoSavedEntry: label "${args.label}" does not start with namespace prefix "${prefix}"`,
+      );
+    }
+    const id = randomUUID();
+    const createdAt = Date.now();
+    const expiresAt = createdAt + retentionMs;
+    db.prepare(
+      `INSERT INTO entries
+        (id, label, content, peer_id, trust_level, created_at, superseded_by,
+         retention_class, is_verbatim, expires_at,
+         subject, predicate, object, source_turn_id, origin,
+         provenance_model, confidence)
+       VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 'agent-derived', ?, ?)`,
+    ).run(
+      id,
+      args.label,
+      args.content,
+      args.peerId,
+      null,
+      createdAt,
+      null,
+      args.retentionClass,
+      args.isVerbatim ? 1 : 0,
+      expiresAt,
+      args.subject ?? null,
+      args.predicate ?? null,
+      args.object ?? null,
+      args.sourceTurnId,
+      args.model,
+      args.confidence,
+    );
+    logEvent(id, "auto-save", args.peerId, {
+      sourceTurnId: args.sourceTurnId,
+      model: args.model,
+      confidence: args.confidence,
+    });
+  }
+
+  async function close(): Promise<void> {
+    db.close();
+  }
+
+  return {
+    initialize,
+    write,
+    writeAutoSavedEntry,
+    search,
+    read,
+    list,
+    forget,
+    supersede,
+    cleanup,
+    close,
+  };
+}