@inixiative/archive 0.5.1 → 0.6.0

package/README.md

@@ -29,7 +29,7 @@ bun run archive init
 bun run archive serve
 ```
 
-`init` creates a local database and a private `server.token` under `~/.local/share/archive`. It prints the token's file path, never its value. `serve` binds to loopback port 4411. The server requires bearer authentication for data endpoints; `/health` has no session data.
+`init` creates a local database and a private `server.token` under `~/.local/share/archive`. It prints the token's file path, never its value. `serve` binds to loopback port 4411; with `--sync` it also publishes to its destinations every 30 seconds from the same process. The server requires bearer authentication for data endpoints; `/health` has no session data.
 
 ```sh
 bun run archive preview --directory /path/to/history --source codex
@@ -39,11 +39,11 @@ bun run archive list
 bun run archive search --query 'migration'
 ```
 
-Use `--source claude-code` for Claude histories. Collection matches the exact working-directory metadata recorded by the provider. Repeat `--project-root` to map several checkouts to one project; `--worktrees` also accepts every current git worktree of each root, re-read on each scan. Nested checkouts are separate mappings. Unknown directories, symlinks and sessions without usable metadata are skipped. Incomplete or changing files remain at source and retry on the next scan. The initial collector rescans files every 30 seconds; it deduplicates normalized content rather than maintaining byte-offset ingestion cursors. It retains prior manually assigned tags. It does not upload: run sync separately.
+Use `--source claude-code` for Claude histories. Collection matches the exact working-directory metadata recorded by the provider. Repeat `--project-root` to map several checkouts to one project; `--worktrees` also accepts every current git worktree of each root, re-read on each scan. Nested checkouts are separate mappings. Unknown directories, symlinks and sessions without usable metadata are skipped. Incomplete or changing files remain at source and retry on the next scan. The initial collector rescans files every 30 seconds; it deduplicates normalized content rather than maintaining byte-offset ingestion cursors. It keeps tags captured earlier. It does not upload: run sync separately.
 
 ## Connect to BYO hosting
 
-Deploy the included Dockerfile with a persistent volume mounted at `/data`, HTTPS, and a unique `ARCHIVE_SERVER_TOKEN` of at least 32 characters. Compose bind-mounts `./data` (or `ARCHIVE_DATA_DIR`) at `/data`, so the store survives rebuilds, `down -v` and volume prunes, and binds only to loopback on port 4411 (or `ARCHIVE_PORT`); put an HTTPS reverse proxy in front for remote use. Railway needs a `/data` volume. The Render blueprint provisions a dedicated disk and generated token.
+Every deployment runs the published image `ghcr.io/inixiative/archive` (tags: version, `latest`, commit SHA), built from main for amd64 and arm64. Give it a persistent volume at `/data`, HTTPS, and a unique `ARCHIVE_SERVER_TOKEN` of at least 32 characters; it needs nothing else and creates its store on first start. Compose (`compose.yaml`, also shipped in the npm package) bind-mounts `./data` (or `ARCHIVE_DATA_DIR`) at `/data`, so the store survives rebuilds, `down -v` and volume prunes, and binds only to loopback on port 4411 (or `ARCHIVE_PORT`); put an HTTPS reverse proxy in front for remote use. Pin a version with `ARCHIVE_VERSION`; build from a checkout with `docker compose -f compose.yaml -f compose.build.yaml up -d --build`. The Render blueprint runs the image with a dedicated disk and generated token. Railway deploys the image as a service with a `/data` volume (`railway.json` builds the Dockerfile when deploying from the repository instead).
 
 Place the destination's token in a private file (`chmod 600`, owned by you) or an environment variable, then run:
 
@@ -76,7 +76,7 @@ archive agents status
 
 `add-collector` upserts by `--name` (lowercase letters, digits, `.` and `-`); `--directory` defaults to `~/.claude/projects` or `~/.codex/sessions`, and `--project-root`, `--worktrees` and `--atlas` behave as for `collect`. `remove-collector --name N`, `serve on|off [--port P]` and `sync on|off` edit the same file. `install` writes one unit per agent: `com.inixiative.archive.local` (serve), `com.inixiative.archive.sync` (`sync --watch`) and `com.inixiative.archive.collect.<name>`. Each runs the current Bun with this package's own `src/cli.ts`, restarts on exit (30 second throttle), works in the archive home and logs to `logs/<label>.{out,err}.log` there. Re-running `install` reloads only changed or stopped units and removes Archive units no longer declared; other launchd agents are untouched. `uninstall` removes every Archive unit; `status` reports each unit's load state, PID and last log entry. One archive home per user account: units are named by agent, not by home. Sync reads destination token files itself, so no wrapper script is needed. Re-run `install` after upgrading the package if its install path changes.
 
-Docker Compose is the alternative for the server: `docker compose up -d` bind-mounts `./data` (or `ARCHIVE_DATA_DIR`) at `/data` and binds loopback port 4411. It is a separate store, so connect it as a destination and let the `sync` agent publish to it; do not bind-mount the store the host agents are writing, since SQLite locking across the Docker VM boundary is unreliable. Collectors run on the host, where the session histories are.
+Docker Compose is the alternative for the server: `docker compose up -d` runs `serve --sync`, so the container publishes to the destinations in its own `/data/destinations.json` (token files must live under `/data` too). It is a separate store from the host agents': connect it as a destination and let the `sync` agent publish to it; do not bind-mount the store the host agents are writing, since SQLite locking across the Docker VM boundary is unreliable. Collectors run on the host, where the session histories are.
 
 `@inixiative/archive/agents` exports the pure pieces for integrations: `agentsConfigSchema`, `agentUnits`, `renderPlist`, `renderSystemdUnit` and `planAgents`, plus `installAgents`, `uninstallAgents` and `agentStatus` with an injectable command runner and supervisor directory.
 
@@ -98,17 +98,41 @@ Kingdom takes the owner from the runtime credential; `--owner-model`, `--organiz
 
 Foundry exposes the same commands through `bun run archive`. Its defaults remain `.foundry/archives/archives.sqlite` and `.foundry/archives.json`. Its durable journal capture automatically publishes matching projects once destination credentials are in the viewer's environment. Restart the viewer after changing destination configuration.
 
-## Routing, tagging and Jev
+## Routing
 
-Only explicit `projectId` matches authorize publication. An archive with no matching destination stays local. Multiple matches create deliberate separate copies. `routes` previews these destinations before upload.
+Only explicit `projectId` matches authorize publication. An archive with no matching destination stays local. Multiple matches create deliberate separate copies. `routes` previews these destinations before upload. Tags, references and actors never change ownership or routing.
+
+## Tags, references, actors and retention
+
+The archive owns its tag system. It keeps two different things apart:
+
+- **Tags** are concepts: the `atlas:<concept>` of repository files a session's tool calls touched (collectors with `--atlas`, from `atlas graph --json`), collector `--tag`s, and tags people add. Heuristic categories are offered as `suggestedTags`, computed on read and never stored.
+- **References** point at items in an integration. Collectors record the session's GitHub repository and branch (`owner/repo`, `owner/repo/tree/branch`); linked GitHub pull requests, issues and commits (`owner/repo#24`, `owner/repo@sha`) and Linear issues (`KEY-12`) are found in the text and counted. A link is a mention, not proof of work. The archive's settings list the integrations it references: GitHub and Linear are recognized natively, and any other integration names the link prefix of its items (for example `https://acme.atlassian.net/browse/`); the item is what follows. Prefixes are literal, so settings cannot make matching slow.
 
 ```sh
-bun run archive tag --id ARCHIVE_ID --tag reviewed
+bun run archive tag --id ARCHIVE_ID --tag reviewed --untag debugging
 ```
 
-Collectors add explicit tags from recorded session metadata: `repo:owner/name` and `branch:name` from git context, and with `--atlas` the `atlas:<concept>` of repository files the session's tool calls touched (from `atlas graph --json` in the session's checkout). Suggestions are computed on read and never stored: heuristic categories, plus `reference` suggestions for linked GitHub pull requests, issues and commits (`github:owner/repo#24`, `github:owner/repo@sha`) and Linear issues (`linear:KEY-12`). A link is a mention, not proof of work, so references stay suggestions.
+Tag edits apply to that archive without a new revision, survive re-capture, and do not sync. Offered tags are defined archive-wide or for one actor. Each session can name its **actor**: a `user`, a `service` such as an API-key run, or a `job` (its id is the job name), with the `integrationId` it ran under when there is one. An organization's archive mostly holds its members' work, filterable per person. Entries the model produced record its `model` and `effort` (from Claude Code assistant records and Codex turn context, per turn); listings summarize them per session as `models` and filter by `model` and `effort`.
+
+The **retention** setting (`retentionDays`, off by default) deletes archives whose latest capture is older than that, at server start and hourly. It deletes only from the archive it is set on; deletions never sync.
+
+### HTTP API
+
+All data endpoints are `POST /api/v1/archive/<action>` with a JSON body and `Authorization: Bearer <token>`:
+
+| Action | Body | Returns |
+|---|---|---|
+| `ingest` | `snapshot`, `previousDigest` | id, digest, revision |
+| `read` | `archiveId`, `revision?` | snapshot and chunks |
+| `list` | filters, `limit?`, `beforeId?` | metadata, `nextCursor` |
+| `search` | filters, `query`, `budget?`, `limit?`, `beforeId?` | matching chunks within the token budget |
+| `tag` | `archiveId`, `add?`, `remove?` | tags |
+| `delete` | `archiveId` | deleted |
+| `settings/read`, `settings/update` | `integrations?`, `retentionDays?` | settings |
+| `tags/list`, `tags/define`, `tags/remove` | `tag`, `actorId?`, `description?` | offered tags with usage counts |
 
-Tags and suggestions do not change ownership or routing. Jev integration is not enabled: the next step is labeled, shadow-mode tag/destination proposals, scored for accuracy and cross-organization mistakes before any automatic action. Session content must not be sent to Jev without selecting that service for the relevant ownership boundary.
+Filters are `projectId`, `source`, `tag`, `actorId`, `model`, `effort` and `reference` (`{integration, ref}`). Jev integration is not enabled. Session content must not be sent to Jev without selecting that service for the relevant ownership boundary.
 
 ## Limits
 

package/compose.yaml

@@ -0,0 +1,16 @@
+services:
+  archive:
+    image: ghcr.io/inixiative/archive:${ARCHIVE_VERSION:-latest}
+    command: ["bun", "run", "src/cli.ts", "serve", "--home", "/data", "--hostname", "0.0.0.0", "--sync"]
+    ports:
+      - "127.0.0.1:${ARCHIVE_PORT:-4411}:4411"
+    environment:
+      ARCHIVE_SERVER_TOKEN: ${ARCHIVE_SERVER_TOKEN:?Set a unique random token of at least 32 characters}
+    volumes:
+      - ${ARCHIVE_DATA_DIR:-./data}:/data
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "bun", "-e", "fetch('http://127.0.0.1:4411/health').then(r=>{if(!r.ok)process.exit(1)})"]
+      interval: 30s
+      timeout: 5s
+      retries: 3

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@inixiative/archive",
-  "version": "0.5.1",
+  "version": "0.6.0",
   "type": "module",
   "exports": {
     ".": "./src/index.ts",
@@ -18,7 +18,8 @@
     "./agents": "./src/agents.ts"
   },
   "files": [
-    "src"
+    "src",
+    "compose.yaml"
   ],
   "scripts": {
     "test": "bun test tests",
@@ -36,7 +37,7 @@
   },
   "devDependencies": {
     "@biomejs/biome": "2.5.2",
-    "@inixiative/config": "^0.5.39",
+    "@inixiative/config": "^0.5.42",
     "@types/bun": "1.4.2",
     "lefthook": "2.1.9",
     "typescript": "6.0.3"

package/src/cli.ts

@@ -58,11 +58,13 @@ export async function runCli(args = Bun.argv.slice(2)) {
       worktrees: { type: 'boolean', default: false },
       atlas: { type: 'boolean', default: false },
       tag: { type: 'string', multiple: true },
+      untag: { type: 'string', multiple: true },
       limit: { type: 'string', default: '100' },
       port: { type: 'string' },
       name: { type: 'string' },
       hostname: { type: 'string', default: '127.0.0.1' },
       watch: { type: 'boolean', default: false },
+      sync: { type: 'boolean', default: false },
       remote: { type: 'boolean', default: false },
       destination: { type: 'string' },
       help: { type: 'boolean' },
@@ -78,9 +80,10 @@ export async function runCli(args = Bun.argv.slice(2)) {
       'Archive: init | serve | connect (setup) | preview | import | collect | list | export | tag | routes | sync | search | agents\n' +
         'connect --url HTTPS_URL --project-id ID (--token-env ENV | --token-file PATH) [--kind kingdom [--connection-id ID] [--owner-model M --organization-id UUID --space-id UUID]]\n' +
         'import --file PATH --source codex|claude-code --project-id ID [--tag TAG]\n' +
+        'tag --id ID [--tag TAG ...] [--untag TAG ...]\n' +
         'collect --directory HISTORY --source codex|claude-code --project-root EXACT_CWD [--project-root ...] [--worktrees] [--atlas] --project-id ID [--watch]\n' +
         'sync [--watch] | routes | search --query TEXT [--remote]\n' +
-        'serve --home PATH [--port 4411 --hostname 127.0.0.1]\n' +
+        'serve --home PATH [--port 4411 --hostname 127.0.0.1] [--sync]\n' +
         'agents add-collector --name N --source codex|claude-code --project-id ID --project-root DIR [--project-root ...] [--directory HISTORY] [--worktrees] [--atlas]\n' +
         'agents remove-collector --name N | serve on|off [--port P] | sync on|off | install | uninstall | status\n' +
         'All commands accept --home PATH; integrations may use --store FILE --config FILE.',
@@ -109,8 +112,24 @@ export async function runCli(args = Bun.argv.slice(2)) {
       port: Number(v.port ?? process.env.PORT ?? 4411),
       hostname: v.hostname,
     });
-    output({ listening: instance.server.url.href, store: storePath });
+    output({ listening: instance.server.url.href, store: storePath, sync: v.sync });
+    let stopped = false;
+    // One process owns the store: the server and its sync share it, as in a container.
+    const sync = (async () => {
+      while (v.sync && !stopped) {
+        try {
+          const results = await syncArchives(instance.store, readDestinations(config));
+          const failed = results.filter((r) => r.status.startsWith('failed')).length;
+          if (failed) console.error(JSON.stringify({ sync: 'failed', archives: failed }));
+        } catch {
+          console.error(JSON.stringify({ sync: 'unavailable' }));
+        }
+        for (let i = 0; i < 30 && !stopped; i++) await Bun.sleep(1000);
+      }
+    })();
     const stop = async () => {
+      stopped = true;
+      await sync;
       await instance.close();
       process.exit(0);
     };
@@ -308,14 +327,9 @@ export async function runCli(args = Bun.argv.slice(2)) {
       if (!archive) throw new Error('Valid --id required');
       if (command === 'export') output(archive.snapshot);
       else {
-        if (!v.tag?.length) throw new Error('Tag requires at least one --tag');
-        output(
-          store.capture({
-            ...archive.snapshot,
-            capturedAt: Date.now(),
-            tags: [...new Set([...archive.snapshot.tags, ...v.tag])],
-          }),
-        );
+        if (!v.tag?.length && !v.untag?.length)
+          throw new Error('Tag requires at least one --tag or --untag');
+        output(store.tag(archive.id, { add: v.tag, remove: v.untag }));
       }
     } else throw new Error('Unknown command; use --help');
   } finally {

package/src/collector.ts

@@ -3,7 +3,7 @@ import { isAbsolute, join, resolve } from 'node:path';
 import { importTranscriptFile } from './import-file';
 import { archiveKey } from './index';
 import type { LocalArchiveStore } from './local';
-import { type AtlasFileConcepts, atlasTags, provenanceTags } from './tags';
+import { type AtlasFileConcepts, atlasTags, provenanceReferences } from './tags';
 
 export interface CollectionSource {
   directory: string;
@@ -138,13 +138,14 @@ export async function collectSessions(store: LocalArchiveStore, config: Collecti
         ...new Set([
           ...(old?.snapshot.tags ?? []),
           ...(config.tags ?? []),
-          ...provenanceTags({
-            branch: meta.branch,
-            repository: meta.repository ?? repositories.get(cwd),
-          }),
           ...(graph ? atlasTags(snapshot, graph, roots) : []),
         ]),
       ].slice(0, 100);
+      const references = provenanceReferences({
+        branch: meta.branch,
+        repository: meta.repository ?? repositories.get(cwd),
+      });
+      if (references.length) snapshot.references = references;
       if (store.capture(snapshot).changed) result.imported++;
       else result.unchanged++;
     } catch {

package/src/import.ts

@@ -21,6 +21,8 @@ export function importTranscriptLines(
   const seen = new Map<string, string>();
   let index = -1;
   let contentBytes = 0;
+  // Codex states the model per turn; Claude Code per assistant record.
+  let turn: { model?: string; effort?: string } = {};
   for (const rawLine of lines) {
     index++;
     const line = rawLine.trim();
@@ -48,6 +50,15 @@ export function importTranscriptLines(
       if (sessionId && sessionId !== id) throw new Error('Mixed transcript sessions');
       sessionId = id;
     }
+    const text = (value: unknown) => (typeof value === 'string' && value ? value : undefined);
+    if (options.source === 'codex' && row.type === 'turn_context')
+      turn = { model: text(row.payload?.model), effort: text(row.payload?.effort) };
+    const producer =
+      options.source === 'codex'
+        ? turn
+        : row.type === 'assistant'
+          ? { model: text(row.message?.model), effort: text(row.effort) }
+          : {};
     const timestamp = typeof row.timestamp === 'number' ? row.timestamp : Date.parse(row.timestamp);
     const add = (kind: ArchiveEntry['kind'], content: unknown, suffix: string, callId?: string) => {
       if (typeof content !== 'string' || !content.length) return;
@@ -71,6 +82,12 @@ export function importTranscriptLines(
         timestamp: Number.isFinite(timestamp) ? timestamp : null,
         sourceRef: `line:${index + 1}`,
         ...(callId ? { callId } : {}),
+        ...(['assistant', 'tool-call', 'reasoning-summary'].includes(kind)
+          ? {
+              ...(producer.model ? { model: producer.model } : {}),
+              ...(producer.effort ? { effort: producer.effort } : {}),
+            }
+          : {}),
       });
     };
     if (options.source === 'codex') {

package/src/index.ts

@@ -11,6 +11,22 @@ export const archiveEntrySchema = z.strictObject({
   text: z.string().max(2_000_000),
   timestamp: z.number().finite().nonnegative().nullable(),
   sourceRef: z.string().min(1).max(512),
+  /** The model that produced the entry, and its effort level as the harness names it. */
+  model: z.string().min(1).max(200).optional(),
+  effort: z.string().min(1).max(40).optional(),
+});
+/** Who did the work: a person, a service such as an API-key run, or a job (id = job name). */
+export const archiveActorSchema = z.strictObject({
+  kind: z.enum(['user', 'service', 'job']),
+  id: identifier,
+  name: z.string().min(1).max(200).optional(),
+  /** The integration the work ran under, when there is one. */
+  integrationId: identifier.optional(),
+});
+/** An item in an integration (a GitHub pull request, a Linear issue) the session is about. */
+export const archiveReferenceSchema = z.strictObject({
+  integration: z.string().regex(/^[a-z0-9-]{1,64}$/),
+  ref: z.string().min(1).max(300),
 });
 export const archiveSnapshotSchema = z
   .strictObject({
@@ -20,6 +36,9 @@ export const archiveSnapshotSchema = z
     sessionId: identifier,
     title: z.string().min(1).max(500),
     projectId: identifier.optional(),
+    actor: archiveActorSchema.optional(),
+    /** Recorded references, such as the session's repository and branch. Linked ones are found in the text. */
+    references: z.array(archiveReferenceSchema).max(100).optional(),
     tags: z.array(z.string().min(1).max(120)).max(100),
     goalIds: z.array(identifier).max(100).default([]),
     runIds: z.array(identifier).max(100).default([]),
@@ -39,6 +58,8 @@ export const archiveSnapshotSchema = z
   });
 export type ArchiveSnapshot = z.infer<typeof archiveSnapshotSchema>;
 export type ArchiveEntry = z.infer<typeof archiveEntrySchema>;
+export type ArchiveActor = z.infer<typeof archiveActorSchema>;
+export type ArchiveReference = z.infer<typeof archiveReferenceSchema>;
 export interface ArchiveChunk {
   id: string;
   entryId: string;

package/src/local.ts

@@ -2,6 +2,7 @@ import { Database } from 'bun:sqlite';
 import { randomUUID } from 'node:crypto';
 import { chmodSync, mkdirSync } from 'node:fs';
 import { dirname } from 'node:path';
+import { z } from 'zod';
 import {
   type ArchiveChunk,
   type ArchiveSnapshot,
@@ -11,7 +12,46 @@ import {
   selectChunks,
   snapshotDigest,
 } from './index';
-import { suggestTags } from './tags';
+import {
+  archiveIntegrationSchema,
+  archiveReferences,
+  defaultIntegrations,
+  suggestTags,
+} from './tags';
+
+export const archiveSettingsSchema = z.strictObject({
+  /** Integrations sessions can reference; Kingdom and Foundry display these, Archive owns them. */
+  integrations: z
+    .array(archiveIntegrationSchema)
+    .max(50)
+    .refine(
+      (list) => new Set(list.map((i) => i.key)).size === list.length,
+      'Duplicate integration keys',
+    ),
+  /** Archives whose latest capture is older than this are deleted here. Deletions never sync. */
+  retentionDays: z.number().int().positive().max(36_500).nullable(),
+});
+export type ArchiveSettings = z.infer<typeof archiveSettingsSchema>;
+const defaultSettings: ArchiveSettings = { integrations: defaultIntegrations, retentionDays: null };
+
+const tagSchema = z.string().min(1).max(120);
+/** A conceptual tag this archive offers: archive-wide (no actor) or for one actor. */
+export const tagDefinitionSchema = z.strictObject({
+  tag: tagSchema,
+  actorId: z.string().min(1).max(256).optional(),
+  description: z.string().max(500).optional(),
+});
+export type TagDefinition = z.infer<typeof tagDefinitionSchema>;
+
+export interface ArchiveFilter {
+  projectId?: string;
+  source?: ArchiveSnapshot['source'];
+  tag?: string;
+  actorId?: string;
+  reference?: { integration: string; ref: string };
+  model?: string;
+  effort?: string;
+}
 
 export class LocalArchiveStore {
   private readonly db: Database;
@@ -28,7 +68,11 @@ export class LocalArchiveStore {
       CREATE TABLE IF NOT EXISTS receipts (archive_id TEXT NOT NULL REFERENCES archives(id), destination TEXT NOT NULL,
         digest TEXT NOT NULL, PRIMARY KEY(archive_id, destination));
       CREATE TABLE IF NOT EXISTS outbox (archive_id TEXT NOT NULL REFERENCES archives(id), destination TEXT NOT NULL,
-        pending TEXT NOT NULL, PRIMARY KEY(archive_id, destination));`);
+        pending TEXT NOT NULL, PRIMARY KEY(archive_id, destination));
+      CREATE TABLE IF NOT EXISTS tag_edits (archive_id TEXT NOT NULL REFERENCES archives(id), tag TEXT NOT NULL,
+        added INTEGER NOT NULL, PRIMARY KEY(archive_id, tag));
+      CREATE TABLE IF NOT EXISTS tag_definitions (actor_id TEXT NOT NULL, tag TEXT NOT NULL, description TEXT,
+        PRIMARY KEY(actor_id, tag));`);
     this.db.query("INSERT OR IGNORE INTO settings VALUES ('sourceId', ?)").run(randomUUID());
     this.sourceId = (
       this.db.query("SELECT value FROM settings WHERE key='sourceId'").get() as { value: string }
@@ -82,22 +126,184 @@ export class LocalArchiveStore {
         }
       : undefined;
   }
-  list() {
-    return (
-      this.db.query('SELECT id FROM archives ORDER BY rowid DESC').all() as { id: string }[]
-    ).map(({ id }) => {
-      const archive = this.read(id)!;
-      const { entries, ...snapshot } = archive.snapshot;
-      return {
-        id,
-        revision: archive.revision,
-        digest: archive.digest,
-        ...snapshot,
-        entries: entries.length,
-        suggestedTags: suggestTags(archive.snapshot),
+  /** Computed per revision: references and suggestions scan the whole transcript. */
+  private summaries = new Map<
+    string,
+    ReturnType<LocalArchiveStore['summarize']> & { digest: string }
+  >();
+  private summarize(id: string, revision: number) {
+    const row = this.db
+      .query('SELECT snapshot FROM revisions WHERE archive_id=? AND revision=?')
+      .get(id, revision) as { snapshot: string };
+    const full = archiveSnapshotSchema.parse(JSON.parse(row.snapshot));
+    const { entries, tags, ...snapshot } = full;
+    const models = new Map<string, { model: string; effort?: string; entries: number }>();
+    for (const entry of entries) {
+      if (!entry.model) continue;
+      const key = JSON.stringify([entry.model, entry.effort]);
+      const current = models.get(key) ?? {
+        model: entry.model,
+        ...(entry.effort ? { effort: entry.effort } : {}),
+        entries: 0,
       };
+      current.entries++;
+      models.set(key, current);
+    }
+    return {
+      snapshot,
+      capturedTags: tags,
+      entries: entries.length,
+      models: [...models.values()],
+      suggestedTags: suggestTags(full),
+      references: archiveReferences(full, this.settings().integrations),
+    };
+  }
+  /** Archive metadata, newest first. Tags are the captured tags with this archive's edits applied. */
+  list(filter: ArchiveFilter = {}) {
+    const rows = this.db
+      .query('SELECT id, revision, digest FROM archives ORDER BY rowid DESC')
+      .all() as { id: string; revision: number; digest: string }[];
+    return rows.flatMap(({ id, revision, digest }) => {
+      let summary = this.summaries.get(id);
+      if (summary?.digest !== digest) {
+        summary = { ...this.summarize(id, revision), digest };
+        this.summaries.set(id, summary);
+      }
+      const { snapshot, references, models } = summary;
+      const tags = this.tags(id, summary.capturedTags);
+      const reference = filter.reference;
+      if (
+        (filter.projectId && snapshot.projectId !== filter.projectId) ||
+        (filter.source && snapshot.source !== filter.source) ||
+        (filter.tag && !tags.includes(filter.tag)) ||
+        (filter.actorId && snapshot.actor?.id !== filter.actorId) ||
+        (filter.model && !models.some((m) => m.model === filter.model)) ||
+        (filter.effort &&
+          !models.some(
+            (m) => m.effort === filter.effort && (!filter.model || m.model === filter.model),
+          )) ||
+        (reference &&
+          !references.some(
+            (r) => r.integration === reference.integration && r.ref === reference.ref,
+          ))
+      )
+        return [];
+      return [
+        {
+          id,
+          revision,
+          digest,
+          ...snapshot,
+          tags,
+          entries: summary.entries,
+          models,
+          suggestedTags: summary.suggestedTags,
+          references,
+        },
+      ];
     });
   }
+  private tags(id: string, captured: string[]) {
+    const edits = this.db.query('SELECT tag, added FROM tag_edits WHERE archive_id=?').all(id) as {
+      tag: string;
+      added: number;
+    }[];
+    const removed = new Set(edits.filter((e) => !e.added).map((e) => e.tag));
+    return [...new Set([...captured, ...edits.filter((e) => e.added).map((e) => e.tag)])].filter(
+      (tag) => !removed.has(tag),
+    );
+  }
+  /** Tag or untag without a new revision. Edits belong to this archive and do not sync. */
+  tag(id: string, change: { add?: string[]; remove?: string[] }) {
+    const add = z
+      .array(tagSchema)
+      .max(100)
+      .parse(change.add ?? []);
+    const remove = z
+      .array(tagSchema)
+      .max(100)
+      .parse(change.remove ?? []);
+    const archive = this.read(id);
+    if (!archive) throw new Error('Archive unavailable');
+    const captured = new Set(archive.snapshot.tags);
+    const upsert = this.db.query(
+      'INSERT INTO tag_edits VALUES (?, ?, ?) ON CONFLICT(archive_id, tag) DO UPDATE SET added=excluded.added',
+    );
+    const clear = this.db.query('DELETE FROM tag_edits WHERE archive_id=? AND tag=?');
+    return this.db.transaction(() => {
+      for (const tag of add) captured.has(tag) ? clear.run(id, tag) : upsert.run(id, tag, 1);
+      for (const tag of remove) captured.has(tag) ? upsert.run(id, tag, 0) : clear.run(id, tag);
+      const tags = this.tags(id, archive.snapshot.tags);
+      z.array(tagSchema).max(100, 'An archive holds at most 100 tags').parse(tags);
+      return { id, tags };
+    })();
+  }
+  delete(id: string) {
+    return this.db.transaction(() => {
+      for (const table of ['tag_edits', 'receipts', 'outbox', 'revisions'])
+        this.db.query(`DELETE FROM ${table} WHERE archive_id=?`).run(id);
+      this.summaries.delete(id);
+      return this.db.query('DELETE FROM archives WHERE id=?').run(id).changes > 0;
+    })();
+  }
+  /** Applies the retention setting; returns the deleted archive ids. */
+  prune(now = Date.now()) {
+    const { retentionDays } = this.settings();
+    if (retentionDays === null) return [];
+    const cutoff = now - retentionDays * 86_400_000;
+    const expired = (this.db.query('SELECT id FROM archives').all() as { id: string }[]).filter(
+      ({ id }) => this.read(id)!.snapshot.capturedAt < cutoff,
+    );
+    for (const { id } of expired) this.delete(id);
+    return expired.map(({ id }) => id);
+  }
+  settings(): ArchiveSettings {
+    const row = this.db.query("SELECT value FROM settings WHERE key='archive'").get() as {
+      value: string;
+    } | null;
+    return row ? archiveSettingsSchema.parse(JSON.parse(row.value)) : defaultSettings;
+  }
+  updateSettings(change: Partial<ArchiveSettings>) {
+    const next = archiveSettingsSchema.parse({ ...this.settings(), ...change });
+    this.db
+      .query(
+        "INSERT INTO settings VALUES ('archive', ?) ON CONFLICT(key) DO UPDATE SET value=excluded.value",
+      )
+      .run(JSON.stringify(next));
+    this.summaries.clear();
+    return next;
+  }
+  /** Offered tags with how many archives carry each; an actor sees archive-wide and their own. */
+  tagDefinitions(actorId?: string) {
+    const rows = this.db
+      .query(
+        "SELECT actor_id, tag, description FROM tag_definitions WHERE actor_id IN ('', ?) ORDER BY tag",
+      )
+      .all(actorId ?? '') as { actor_id: string; tag: string; description: string | null }[];
+    const archives = this.list(actorId ? { actorId } : {});
+    return rows.map((row) => ({
+      tag: row.tag,
+      ...(row.actor_id ? { actorId: row.actor_id } : {}),
+      ...(row.description ? { description: row.description } : {}),
+      archives: archives.filter((archive) => archive.tags.includes(row.tag)).length,
+    }));
+  }
+  defineTag(input: TagDefinition) {
+    const definition = tagDefinitionSchema.parse(input);
+    this.db
+      .query(
+        'INSERT INTO tag_definitions VALUES (?, ?, ?) ON CONFLICT(actor_id, tag) DO UPDATE SET description=excluded.description',
+      )
+      .run(definition.actorId ?? '', definition.tag, definition.description ?? null);
+    return definition;
+  }
+  undefineTag(input: { tag: string; actorId?: string }) {
+    return (
+      this.db
+        .query('DELETE FROM tag_definitions WHERE actor_id=? AND tag=?')
+        .run(input.actorId ?? '', input.tag).changes > 0
+    );
+  }
   search(ids: string[], query: string, budget = 2048) {
     return ids.map((id) => {
       const archive = this.read(id);

package/src/server.ts

@@ -1,7 +1,13 @@
 import { timingSafeEqual } from 'node:crypto';
 import { z } from 'zod';
-import { archiveKey, archiveSnapshotSchema, selectChunks, snapshotDigest } from './index';
-import { LocalArchiveStore } from './local';
+import {
+  archiveKey,
+  archiveReferenceSchema,
+  archiveSnapshotSchema,
+  selectChunks,
+  snapshotDigest,
+} from './index';
+import { archiveSettingsSchema, LocalArchiveStore, tagDefinitionSchema } from './local';
 
 const ingestSchema = z.strictObject({
   snapshot: archiveSnapshotSchema,
@@ -10,18 +16,43 @@ const ingestSchema = z.strictObject({
     .regex(/^[a-f0-9]{64}$/)
     .nullable(),
 });
+const archiveId = z.string().regex(/^[a-f0-9]{64}$/);
+const filterFields = {
+  projectId: z.string().max(256).optional(),
+  source: archiveSnapshotSchema.shape.source.optional(),
+  tag: z.string().min(1).max(120).optional(),
+  actorId: z.string().min(1).max(256).optional(),
+  reference: archiveReferenceSchema.optional(),
+  model: z.string().min(1).max(200).optional(),
+  effort: z.string().min(1).max(40).optional(),
+  limit: z.number().int().min(1).max(100).default(20),
+  beforeId: archiveId.optional(),
+};
+const listSchema = z.strictObject(filterFields);
 const searchSchema = z.strictObject({
+  ...filterFields,
   query: z.string().max(1000).default(''),
-  tag: z.string().min(1).max(120).optional(),
   budget: z.number().int().min(16).max(32768).default(2048),
-  projectId: z.string().max(256).optional(),
-  limit: z.number().int().min(1).max(100).default(20),
-  beforeId: z.string().optional(),
 });
-const readSchema = z.strictObject({
-  archiveId: z.string().regex(/^[a-f0-9]{64}$/),
-  revision: z.number().int().positive().optional(),
+const readSchema = z.strictObject({ archiveId, revision: z.number().int().positive().optional() });
+const tagSchema = z.strictObject({
+  archiveId,
+  add: z.array(z.string()).max(100).optional(),
+  remove: z.array(z.string()).max(100).optional(),
 });
+const tagListSchema = z.strictObject({ actorId: z.string().min(1).max(256).optional() });
+const tagRemoveSchema = tagDefinitionSchema.pick({ tag: true, actorId: true });
+
+/** Cursor paging over newest-first archives. */
+function page<T extends { id: string }>(items: T[], limit: number, beforeId?: string) {
+  const cursor = beforeId ? items.findIndex((item) => item.id === beforeId) : -1;
+  if (beforeId && cursor < 0) return undefined;
+  const slice = items.slice(cursor + 1, cursor + 1 + limit);
+  return {
+    items: slice,
+    nextCursor: items.length > cursor + 1 + slice.length ? (slice.at(-1)?.id ?? null) : null,
+  };
+}
 
 /** One deployment, one ownership boundary. No shared cross-tenant database or admin API. */
 export function createArchiveHandler(store: LocalArchiveStore, token: string) {
@@ -71,35 +102,29 @@ export function createArchiveHandler(store: LocalArchiveStore, token: string) {
           archive = store.read(input.archiveId, input.revision);
         return archive ? json({ data: archive }) : json({ error: 'Archive unavailable' }, 404);
       }
+      if (path === '/api/v1/archive/list') {
+        const { limit, beforeId, ...filter } = listSchema.parse(body);
+        const result = page(store.list(filter), limit, beforeId);
+        if (!result) return json({ error: 'Invalid cursor' }, 400);
+        return json({ data: { archives: result.items, nextCursor: result.nextCursor } });
+      }
       if (path === '/api/v1/archive/search') {
-        const input = searchSchema.parse(body);
-        let remaining = input.budget;
+        const { query, budget, limit, beforeId, ...filter } = searchSchema.parse(body);
+        let remaining = budget;
         const matching = store
-          .list()
+          .list(filter)
           .filter(
             (a) =>
-              (!input.projectId || a.projectId === input.projectId) &&
-              (!input.tag || a.tags.includes(input.tag)),
-          )
-          .map((a) => ({
-            a,
-            selected: selectChunks(store.read(a.id)!.chunks, input.query, input.budget),
-          }))
-          .filter(
-            ({ a, selected }) =>
-              !input.query ||
-              selected.matchingChunks > 0 ||
-              [a.title, ...a.tags].some((text) =>
-                text.toLowerCase().includes(input.query.toLowerCase()),
-              ),
+              !query ||
+              selectChunks(store.read(a.id)!.chunks, query, budget).matchingChunks > 0 ||
+              [a.title, ...a.tags].some((text) => text.toLowerCase().includes(query.toLowerCase())),
           );
-        const cursor = input.beforeId ? matching.findIndex(({ a }) => a.id === input.beforeId) : -1;
-        if (input.beforeId && cursor < 0) return json({ error: 'Invalid cursor' }, 400);
-        const page = matching.slice(cursor + 1, cursor + 1 + input.limit);
-        const archives = page.map(({ a }) => {
+        const result = page(matching, limit, beforeId);
+        if (!result) return json({ error: 'Invalid cursor' }, 400);
+        const archives = result.items.map((a) => {
           const selected =
             remaining >= 16
-              ? selectChunks(store.read(a.id)!.chunks, input.query, remaining)
+              ? selectChunks(store.read(a.id)!.chunks, query, remaining)
               : { chunks: [], tokenCount: 0 };
           remaining -= selected.tokenCount;
           return {
@@ -108,20 +133,42 @@ export function createArchiveHandler(store: LocalArchiveStore, token: string) {
             digest: a.digest,
             title: a.title,
             projectId: a.projectId,
+            actor: a.actor,
+            models: a.models,
             tags: a.tags,
             suggestedTags: a.suggestedTags,
+            references: a.references,
             coverage: a.coverage,
             ...selected,
           };
         });
         return json({
-          data: {
-            archives,
-            tokenCount: input.budget - remaining,
-            nextCursor: matching.length > cursor + 1 + page.length ? page.at(-1)?.a.id : null,
-          },
+          data: { archives, tokenCount: budget - remaining, nextCursor: result.nextCursor },
         });
       }
+      if (path === '/api/v1/archive/tag') {
+        const input = tagSchema.parse(body);
+        if (!store.read(input.archiveId)) return json({ error: 'Archive unavailable' }, 404);
+        return json({ data: store.tag(input.archiveId, input) });
+      }
+      if (path === '/api/v1/archive/delete') {
+        const input = readSchema.pick({ archiveId: true }).parse(body);
+        return store.delete(input.archiveId)
+          ? json({ data: { archiveId: input.archiveId, deleted: true } })
+          : json({ error: 'Archive unavailable' }, 404);
+      }
+      if (path === '/api/v1/archive/settings/read') {
+        z.strictObject({}).parse(body);
+        return json({ data: store.settings() });
+      }
+      if (path === '/api/v1/archive/settings/update')
+        return json({ data: store.updateSettings(archiveSettingsSchema.partial().parse(body)) });
+      if (path === '/api/v1/archive/tags/list')
+        return json({ data: { tags: store.tagDefinitions(tagListSchema.parse(body).actorId) } });
+      if (path === '/api/v1/archive/tags/define')
+        return json({ data: store.defineTag(tagDefinitionSchema.parse(body)) });
+      if (path === '/api/v1/archive/tags/remove')
+        return json({ data: { removed: store.undefineTag(tagRemoveSchema.parse(body)) } });
       return json({ error: 'Not found' }, 404);
     } catch (error) {
       if (error instanceof z.ZodError || error instanceof SyntaxError)
@@ -138,6 +185,10 @@ export function startArchiveServer(options: {
   hostname?: string;
 }) {
   const store = new LocalArchiveStore(options.store);
+  // Retention runs at start and hourly; it only ever deletes from this archive.
+  store.prune();
+  const retention = setInterval(() => store.prune(), 3_600_000);
+  retention.unref();
   try {
     const server = Bun.serve({
       port: options.port ?? 4411,
@@ -149,11 +200,13 @@ export function startArchiveServer(options: {
       server,
       store,
       async close() {
+        clearInterval(retention);
         await server.stop(true);
         store.close();
       },
     };
   } catch (error) {
+    clearInterval(retention);
     store.close();
     throw error;
   }

package/src/tags.ts

@@ -1,49 +1,109 @@
-import { type ArchiveSnapshot, categorySuggestions } from './index';
+import { z } from 'zod';
+import { type ArchiveReference, type ArchiveSnapshot, categorySuggestions } from './index';
 
-export type TagSuggestion = { tag: string; origin: 'heuristic' | 'reference' };
+/** A conceptual tag the archive proposes; suggestions never route or authorize. */
+export type TagSuggestion = { tag: string; origin: 'heuristic' };
 
-const references: [RegExp, (match: RegExpExecArray) => string][] = [
-  [
-    /github\.com\/([\w.-]+)\/([\w.-]+)\/(?:pull|issues)\/(\d+)/g,
-    (m) => `github:${m[1]}/${m[2]}#${m[3]}`.toLowerCase(),
-  ],
-  [
-    /github\.com\/([\w.-]+)\/([\w.-]+)\/commit\/([0-9a-f]{7,40})\b/g,
-    (m) => `github:${m[1]}/${m[2]}@${m[3].slice(0, 7)}`.toLowerCase(),
-  ],
-  [
-    /linear\.app\/[\w-]+\/issue\/([A-Za-z][A-Za-z0-9]*-\d+)/g,
-    (m) => `linear:${m[1].toUpperCase()}`,
-  ],
+/**
+ * An integration whose items sessions can reference. GitHub and Linear links are recognized
+ * natively; any other integration names the link prefix of its items, and the item is what
+ * follows. A literal prefix keeps matching linear in the text, whatever settings say.
+ */
+export const archiveIntegrationSchema = z.strictObject({
+  key: z.string().regex(/^[a-z0-9-]{1,64}$/),
+  name: z.string().min(1).max(100),
+  prefix: z.string().min(3).max(300).optional(),
+});
+export type ArchiveIntegration = z.infer<typeof archiveIntegrationSchema>;
+
+export const defaultIntegrations: ArchiveIntegration[] = [
+  { key: 'github', name: 'GitHub' },
+  { key: 'linear', name: 'Linear' },
 ];
 
-/** Issues, pull requests and commits the transcript links to, most-referenced first. A mention is not proof of work, so these stay suggestions. */
-export function referenceSuggestions(snapshot: ArchiveSnapshot, limit = 25): TagSuggestion[] {
-  const counts = new Map<string, number>();
-  for (const entry of snapshot.entries)
-    for (const [pattern, tag] of references)
-      for (const match of entry.text.matchAll(pattern)) {
-        const value = tag(match);
-        counts.set(value, (counts.get(value) ?? 0) + 1);
-      }
-  return [...counts]
-    .sort((a, b) => b[1] - a[1])
-    .slice(0, limit)
-    .map(([tag]) => ({ tag, origin: 'reference' }));
+type Extractor = [RegExp, (match: RegExpExecArray) => string];
+const nativeExtractors: Record<string, Extractor[]> = {
+  github: [
+    [
+      /github\.com\/([\w.-]+)\/([\w.-]+)\/(?:pull|issues)\/(\d+)/g,
+      (m) => `${m[1]}/${m[2]}#${m[3]}`.toLowerCase(),
+    ],
+    [
+      /github\.com\/([\w.-]+)\/([\w.-]+)\/commit\/([0-9a-f]{7,40})\b/g,
+      (m) => `${m[1]}/${m[2]}@${m[3].slice(0, 7)}`.toLowerCase(),
+    ],
+  ],
+  linear: [[/linear\.app\/[\w-]+\/issue\/([A-Za-z][A-Za-z0-9]*-\d+)/g, (m) => m[1].toUpperCase()]],
+};
+
+function extractors(integration: ArchiveIntegration): Extractor[] {
+  if (integration.prefix)
+    return [
+      [
+        new RegExp(`${RegExp.escape(integration.prefix)}([\\w.~#/-]{1,300})`, 'g'),
+        (m) => m[1].replace(/[./]+$/, ''),
+      ],
+    ];
+  return nativeExtractors[integration.key] ?? [];
+}
+
+export type ArchiveReferenceCount = ArchiveReference & { recorded: boolean; mentions: number };
+
+/**
+ * References to the archive's integrations: recorded ones first, then linked ones, most-linked
+ * first. A link is not proof of work, so linked references are evidence, not assignments.
+ */
+export function archiveReferences(
+  snapshot: ArchiveSnapshot,
+  integrations: ArchiveIntegration[] = defaultIntegrations,
+  limit = 25,
+): ArchiveReferenceCount[] {
+  const known = new Set(integrations.map((integration) => integration.key));
+  const found = new Map<string, ArchiveReferenceCount>();
+  const key = (reference: ArchiveReference) =>
+    JSON.stringify([reference.integration, reference.ref]);
+  for (const reference of snapshot.references ?? [])
+    if (known.has(reference.integration))
+      found.set(key(reference), { ...reference, recorded: true, mentions: 0 });
+  for (const integration of integrations)
+    for (const [pattern, ref] of extractors(integration))
+      for (const entry of snapshot.entries)
+        for (const match of entry.text.matchAll(pattern)) {
+          const reference = { integration: integration.key, ref: ref(match) };
+          if (!reference.ref) continue;
+          const current = found.get(key(reference)) ?? {
+            ...reference,
+            recorded: false,
+            mentions: 0,
+          };
+          current.mentions++;
+          found.set(key(reference), current);
+        }
+  return [...found.values()]
+    .sort((a, b) => Number(b.recorded) - Number(a.recorded) || b.mentions - a.mentions)
+    .slice(0, limit);
 }
 
-/** Every computed suggestion for an archive. Suggestions never route or authorize. */
+/** Every computed tag suggestion for an archive. */
 export function suggestTags(snapshot: ArchiveSnapshot): TagSuggestion[] {
-  return [...categorySuggestions(snapshot), ...referenceSuggestions(snapshot)];
+  return categorySuggestions(snapshot);
 }
 
-/** Recorded git context of a session, as explicit tags. */
-export function provenanceTags(context: { repository?: string; branch?: string }): string[] {
-  const repository = context.repository?.match(/github\.com[:/]([\w.-]+\/[\w.-]+?)(?:\.git)?\/?$/);
+/** Recorded git context of a session, as GitHub references to its repository and branch. */
+export function provenanceReferences(context: {
+  repository?: string;
+  branch?: string;
+}): ArchiveReference[] {
+  const repository = context.repository
+    ?.match(/github\.com[:/]([\w.-]+\/[\w.-]+?)(?:\.git)?\/?$/)?.[1]
+    .toLowerCase();
+  if (!repository) return [];
   return [
-    ...(repository ? [`repo:${repository[1].toLowerCase()}`] : []),
-    ...(context.branch && context.branch !== 'HEAD' ? [`branch:${context.branch}`] : []),
-  ].filter((tag) => tag.length <= 120);
+    { integration: 'github', ref: repository },
+    ...(context.branch && context.branch !== 'HEAD'
+      ? [{ integration: 'github', ref: `${repository}/tree/${context.branch}` }]
+      : []),
+  ].filter((reference) => reference.ref.length <= 300);
 }
 
 /** `atlas graph --json`: repository-relative file paths to the concepts they are part of. */