quilltap 4.7.0-dev → 4.7.0-dev.117

package/README.md

@@ -141,12 +141,16 @@ Most subcommands accept `--json` (for piping) and `--limit N`. Names are case-in
 ```bash
 quilltap db --tables                                # List tables in active DB
 quilltap db --count chat_messages                   # Row count
-quilltap db "SELECT id FROM characters LIMIT 5"     # Raw SQL
-quilltap db --repl                                  # Interactive prompt
+quilltap db "SELECT id FROM characters LIMIT 5"     # Raw SQL (read-only)
+quilltap db --repl                                  # Interactive prompt (read-only)
+quilltap db --write "UPDATE characters SET title = 'rival' WHERE id = '...'"
+quilltap db --repl --write                          # Interactive, read-write
 quilltap db --llm-logs --tables                     # Target the LLM logs DB
 quilltap db --mount-points --tables                 # Target the mount index DB
 ```
 
+The database is opened **read-only by default**. Add `--write` to make changes: it opens the database read-write, **claims the instance lock** (`<dataDir>/quilltap.lock`) for the duration, and releases it on exit. It **refuses — with no override — if a running server or another instance holds the lock**, so stop the server first. `--repl` is read-only unless combined with `--write`. Attempting a write without `--write` fails with a hint to re-run with the flag.
+
 In the REPL, `.cols <table>` and `.find <text>` mirror the subcommand helpers.
 
 ## Document Stores (Scriptorium)
@@ -169,14 +173,32 @@ quilltap docs status                            # Per-mount extraction + embeddi
 quilltap docs scan <mount>                                    # Trigger a rescan
 quilltap docs reindex <mount> [path] [--force]                # Re-extract + re-chunk
 quilltap docs embed <mount> [path] [--force] [--wait]         # Enqueue embedding jobs
-quilltap docs write [--force] <mount> <path> [file]           # Stdin or file → mount
+quilltap docs write [--force] [--base64] <mount> <path> [file]  # Stdin or file → mount
+quilltap docs read [--rendered] [--base64] <mount> <path>       # File contents → stdout
 quilltap docs delete <mount> <path>                           # Idempotent delete
 quilltap docs mkdir <mount> <path>                            # Idempotent folder create
 quilltap docs move <srcMount> <srcPath> <dstMount> <dstPath>  # Move (hard-link when possible)
 quilltap docs copy [--force] <srcMount> <srcPath> <dstMount> <dstPath>
+quilltap docs link <srcMount> <srcPath> <dstMount> <dstPath>  # Hard-link (server-required; no byte copy)
+quilltap docs rmdir <mount> <path>                            # Delete an empty folder (server-required)
+quilltap docs mvdir <mount> <fromPath> <toPath>               # Rename/move a folder (server-required)
 ```
 
-Mount arguments accept the mount name (case-insensitive) or a UUID; ambiguous names print candidates and exit non-zero. `--json` is supported by every verb; `reindex` and `embed` refuse to run without a reachable server.
+Mount arguments accept the mount name (case-insensitive) or a UUID; ambiguous names print candidates and exit non-zero. `--json` is supported by every verb; `reindex`, `embed`, `link`, `rmdir`, and `mvdir` refuse to run without a reachable server.
+
+### `--base64` flag
+
+`write --base64`: reads the source bytes and sends them to the server via `PUT /api/v1/mount-points/{mountId}/files/{path}` with `{content: <base64>, encoding: "base64"}`. This is the portable path the file browser uses and handles arbitrary binary files cleanly. Server-required; the direct filesystem fallback is not available with this flag.
+
+`read --base64`: fetches raw bytes via `GET /api/v1/mount-points/{mountId}/files/{path}?encoding=base64` and emits the decoded bytes to stdout, suitable for binary round-trips. Server-required.
+
+### `link`, `rmdir`, `mvdir`
+
+`link` calls `POST /api/v1/mount-points/{srcMountId}?action=link-file` with `{sourcePath, destMountPointId, destPath}`. Creates a true hard link with no byte copy; the server reports back a `strategy` field. Errors: `DEST_EXISTS` (exit 2), `UNSUPPORTED` (cross-storage or cross-device), `SOURCE_NOT_FOUND`.
+
+`rmdir` calls `POST /api/v1/mount-points/{mountId}?action=delete-folder` with `{path}`. Fails with a clear message if the folder is not empty (`NOT_EMPTY` / `CONFLICT`).
+
+`mvdir` calls `POST /api/v1/mount-points/{mountId}?action=move-folder` with `{fromPath, toPath}`. Fails with exit 2 if the destination already exists (`DEST_EXISTS`).
 
 ## Memories
 
@@ -194,6 +216,18 @@ quilltap memories status [--character <name|id>]                       # Per-hol
 
 Shared filter flags apply to `ls`, `find`, `grep`, and `status` where they make sense: `--character`, `--about` (with `self` / `none` shortcuts), `--source`, `--chat` (with `none` for manual entries), `--project`, `--since`, `--until`, `--min-importance`, `--min-reinforced`, `--has-embedding` / `--no-embedding`. Sort flags (`--sort reinforced|importance|created|accessed|reinforcement-count|links`, plus `-r` to reverse) apply to `ls`, `find`, and `grep`. Names accept fuzzy substrings; ambiguous names print candidates and exit 2. `--json` is supported by every verb. The legacy `quilltap db memories --character <name>` verb remains undisturbed.
 
+## Maintenance & Cleanup
+
+`quilltap maintenance` is the manual trigger for the retention sweeps that otherwise run on the server's daily maintenance tick. It reaps data with no bearing on characters, stories, or memories.
+
+```bash
+quilltap maintenance status                  # Read-only: last sweep time + dry-run counts of what would be reaped
+quilltap maintenance status --instance Friday --json
+quilltap maintenance run --instance Friday    # Run the sweeps once (lock-gated; refuses while the server is up)
+```
+
+`maintenance run` is a DB writer: it claims `<dataDir>/quilltap.lock` and **refuses while a running Quilltap server holds it** — stop the server first. Because it can only run with the server down, it performs the sweeps expressible as direct SQL/filesystem work: reaping finished background jobs (COMPLETED after 7 days, DEAD after 30, keyed off `completedAt`), closed terminal sessions older than 30 days plus their transcript files, and orphaned mount-index files. The **stale-chat asset collapse** (superseded story-backgrounds and wardrobe avatars) needs the server's file-storage machinery and runs only on the server's daily tick — `status` reports a stale-chat count so you can see the backlog. Retention windows mirror `lib/background-jobs/maintenance/retention-constants.ts`.
+
 ## Theme Management
 
 The CLI includes theme management commands:

package/bin/quilltap.js

@@ -13,7 +13,7 @@ const {
   loadDbKey,
 } = require('../lib/db-helpers');
 const { resolveInstance } = require('../lib/instances');
-const { resolveModuleDir, ensureNativeModules } = require('../lib/native-modules');
+const { resolveModuleDir, ensureNativeModules, ensureDatabaseNativeModule } = require('../lib/native-modules');
 
 const PACKAGE_DIR = path.resolve(__dirname, '..');
 
@@ -730,9 +730,15 @@ Subcommands (high-level shortcuts; auto-pick the right database):
   Most subcommands also accept --json and --limit N.
 
 Low-level options (legacy; still supported):
+  The database is opened READ-ONLY by default. Add --write to make changes.
   --tables              List all tables in the active database
   --count <table>       Show row count for a table
-  --repl                Interactive SQL prompt (extras: .cols, .find)
+  --repl                Interactive SQL prompt (extras: .cols, .find).
+                        Read-only unless combined with --write.
+  --write               Open the database read-write. Claims the instance lock
+                        for the duration and releases it on exit. Refuses (no
+                        override) if a running server or another instance holds
+                        the lock — stop it first. Works with raw SQL and --repl.
   --json                Emit machine-readable JSON instead of a table
                         (works with --tables, --count, and raw SQL)
   --llm-logs            Target the LLM logs database
@@ -769,6 +775,8 @@ Examples:
   quilltap db "SELECT count(*) FROM characters"
   quilltap db --count messages
   quilltap db --repl
+  quilltap db --write "UPDATE characters SET title = 'rival' WHERE id = '...'"
+  quilltap db --repl --write                           # interactive, read-write
   quilltap db --lock-status
   QUILLTAP_DB_PASSPHRASE=secret quilltap db --tables
 `);
@@ -837,6 +845,7 @@ async function dbCommand(args) {
   let showTables = false;
   let countTable = '';
   let repl = false;
+  let writable = false;
   let sql = '';
   let showHelp = false;
   let lockStatus = false;
@@ -852,6 +861,7 @@ async function dbCommand(args) {
       case '--tables': showTables = true; break;
       case '--count': countTable = cleaned[++i]; break;
       case '--repl': repl = true; break;
+      case '--write': writable = true; break;
       case '--json': asJson = true; break;
       case '--help': case '-h': showHelp = true; break;
       case '--lock-status': lockStatus = true; break;
@@ -919,6 +929,19 @@ async function dbCommand(args) {
     process.exit(1);
   }
 
+  // Read-write opens (`--write`, optionally with `--repl`) must claim the
+  // instance lock first so a server starting mid-operation refuses to run.
+  // Refuses (no override) if a live instance already holds it.
+  if (writable) {
+    const { acquireWriteLock } = require('../lib/lock-helpers');
+    try {
+      acquireWriteLock(dataDir);
+    } catch (err) {
+      console.error(err.message);
+      process.exit(1);
+    }
+  }
+
   // Open database — prefer SQLCipher-capable build
   let Database;
   try {
@@ -926,7 +949,7 @@ async function dbCommand(args) {
   } catch {
     Database = require('better-sqlite3');
   }
-  const db = new Database(dbPath, { readonly: !repl });
+  const db = new Database(dbPath, { readonly: !writable });
 
   if (pepper) {
     const keyHex = Buffer.from(pepper, 'base64').toString('hex');
@@ -971,7 +994,7 @@ async function dbCommand(args) {
     } else if (repl) {
       const readline = require('readline');
       const rl = readline.createInterface({ input: process.stdin, output: process.stdout, prompt: 'quilltap> ' });
-      console.log(`Connected to ${dbPath}`);
+      console.log(`Connected to ${dbPath}${writable ? '  (read-write — instance lock held)' : '  (read-only)'}`);
       console.log('Type .tables, .schema <table>, .cols <table>, .find <text>, or SQL. Ctrl+D to exit.\n');
       rl.prompt();
       rl.on('line', (line) => {
@@ -1033,19 +1056,40 @@ async function dbCommand(args) {
           }
         } catch (err) {
           console.error(`Error: ${err.message}`);
+          if (!writable && /readonly|read-only/i.test(err.message)) {
+            console.error('This REPL is read-only. Exit and re-run as `quilltap db --repl --write` to make changes.');
+          }
         }
         rl.prompt();
       });
       rl.on('close', () => {
         db.close();
+        if (writable) {
+          const { releaseWriteLock } = require('../lib/lock-helpers');
+          releaseWriteLock(dataDir);
+        }
         process.exit(0);
       });
       return; // Don't close db yet — REPL is interactive
     } else {
       printDbHelp();
     }
+  } catch (err) {
+    if (!writable && /readonly|read-only/i.test(err.message)) {
+      console.error('This database was opened read-only, so it cannot be modified.');
+      console.error('Re-run with --write to make changes — it claims the instance lock while it runs');
+      console.error('and refuses if another Quilltap instance is using the database. For example:');
+      console.error(`  quilltap db --write ${sql ? JSON.stringify(sql) : '"UPDATE ..."'}`);
+    } else {
+      console.error(`Error: ${err.message}`);
+    }
+    process.exitCode = 1;
   } finally {
     if (!repl) db.close();
+    if (writable && !repl) {
+      const { releaseWriteLock } = require('../lib/lock-helpers');
+      releaseWriteLock(dataDir);
+    }
   }
 }
 
@@ -1059,7 +1103,7 @@ async function dbCommand(args) {
 // to the subcommand. Each subcommand parses these flags position-independently,
 // so they behave the same before or after the verb.
 const SUBCOMMANDS = new Set([
-  'db', 'themes', 'docs', 'memories', 'instances', 'memory-diff', 'completion', 'logs', 'migrations',
+  'db', 'themes', 'docs', 'memories', 'instances', 'memory-diff', 'completion', 'logs', 'migrations', 'maintenance',
 ]);
 // Global flags that consume the following token as their value.
 const GLOBAL_VALUE_FLAGS = new Set(['-p', '--port', '-d', '--data-dir', '-i', '--instance', '--passphrase']);
@@ -1080,6 +1124,14 @@ const subName = subIdx >= 0 ? cliArgs[subIdx] : '';
 // Everything except the subcommand token itself (leading global flags kept).
 const subArgs = subIdx >= 0 ? [...cliArgs.slice(0, subIdx), ...cliArgs.slice(subIdx + 1)] : [];
 
+// Subcommands load the SQLCipher binding directly and never reach main()'s
+// native-module heal, so self-heal the database ABI here first. Cheap no-op when
+// healthy; rebuilds (with a friendly notice, not an error) only on a real
+// Node-ABI mismatch — e.g. after the user upgrades Node under a cached install.
+if (subName) {
+  ensureDatabaseNativeModule();
+}
+
 if (subName === 'db') {
   dbCommand(subArgs);
 } else if (subName === 'themes') {
@@ -1127,6 +1179,12 @@ if (subName === 'db') {
     console.error(`Error: ${err.message}`);
     process.exit(1);
   });
+} else if (subName === 'maintenance') {
+  const { maintenanceCommand } = require('../lib/maintenance-commands');
+  maintenanceCommand(subArgs).catch(err => {
+    console.error(`Error: ${err.message}`);
+    process.exit(1);
+  });
 } else {
   main();
 }

package/lib/__tests__/qtap-uri.test.js

@@ -0,0 +1,111 @@
+/**
+ * Unit tests for the CLI-local qtap:// codec (packages/quilltap/lib/qtap-uri.js).
+ * Mirrors the server codec's tests (§3.4 / §2.3) — same grammar, same encoding.
+ *
+ * @jest-environment node
+ */
+
+'use strict';
+
+const {
+  isQtapUri,
+  parseQtapUri,
+  formatQtapUri,
+  formatDocStoreUri,
+  QtapUriError,
+} = require('../qtap-uri');
+
+describe('CLI isQtapUri', () => {
+  it('accepts qtap:// (case-insensitive), rejects others', () => {
+    expect(isQtapUri('qtap://self/x.md')).toBe(true);
+    expect(isQtapUri('QTAP://self/x.md')).toBe(true);
+    expect(isQtapUri('https://x')).toBe(false);
+    expect(isQtapUri('self/x')).toBe(false);
+    expect(isQtapUri(undefined)).toBe(false);
+  });
+});
+
+describe('CLI parseQtapUri — worked examples', () => {
+  it('self vault', () => {
+    expect(parseQtapUri('qtap://self/Mail/a.md')).toEqual({
+      scope: 'document_store',
+      mountPoint: 'self',
+      path: 'Mail/a.md',
+    });
+  });
+
+  it('encoded name with space + colon', () => {
+    const p = parseQtapUri('qtap://Project%20Files%3A%20Voyages/Knowledge/x.md');
+    expect(p.mountPoint).toBe('Project Files: Voyages');
+    expect(p.path).toBe('Knowledge/x.md');
+  });
+
+  it('UUID authority', () => {
+    expect(parseQtapUri('qtap://550e8400-e29b-41d4-a716-446655440000/n/t.md').mountPoint).toBe(
+      '550e8400-e29b-41d4-a716-446655440000'
+    );
+  });
+
+  it('project + general scopes', () => {
+    expect(parseQtapUri('qtap://project/Outline.md')).toEqual({ scope: 'project', path: 'Outline.md' });
+    expect(parseQtapUri('qtap://general/S/intro.md')).toEqual({ scope: 'general', path: 'S/intro.md' });
+  });
+
+  it('fragment heading + level', () => {
+    expect(parseQtapUri('qtap://self/Backstory.md#Childhood:2')).toEqual({
+      scope: 'document_store',
+      mountPoint: 'self',
+      path: 'Backstory.md',
+      heading: 'Childhood',
+      level: 2,
+    });
+  });
+
+  it('store root, with and without trailing slash', () => {
+    expect(parseQtapUri('qtap://self/').path).toBe('');
+    expect(parseQtapUri('qtap://self').path).toBe('');
+  });
+
+  it('reserved words win, case-insensitively', () => {
+    expect(parseQtapUri('qtap://SELF/x').mountPoint).toBe('self');
+    expect(parseQtapUri('qtap://Project/x').scope).toBe('project');
+  });
+});
+
+describe('CLI parseQtapUri — errors', () => {
+  it('NOT_A_QTAP_URI / EMPTY_AUTHORITY / BAD_LEVEL', () => {
+    expect(() => parseQtapUri('https://x')).toThrow(QtapUriError);
+    expect(() => parseQtapUri('qtap:///foo')).toThrow(/empty authority/i);
+    expect(() => parseQtapUri('qtap://self/x#H:0')).toThrow(/level/i);
+    expect(() => parseQtapUri('qtap://self/x#H:x')).toThrow(/level/i);
+  });
+});
+
+describe('CLI formatQtapUri — canonical', () => {
+  it('encodes colon as %3A and spaces as %20', () => {
+    expect(
+      formatQtapUri({ scope: 'document_store', mountPoint: 'A: B', path: 'c d.md' })
+    ).toBe('qtap://A%3A%20B/c%20d.md');
+  });
+
+  it('store root has trailing slash', () => {
+    expect(formatQtapUri({ scope: 'document_store', mountPoint: 'self', path: '' })).toBe('qtap://self/');
+  });
+
+  it('formatDocStoreUri builds a store URI', () => {
+    expect(formatDocStoreUri('My Store', 'a/b.md')).toBe('qtap://My%20Store/a/b.md');
+    expect(formatDocStoreUri('My Store', '')).toBe('qtap://My%20Store/');
+  });
+});
+
+describe('CLI round-trip', () => {
+  const cases = [
+    ['qtap://self/Mail/a.md', 'qtap://self/Mail/a.md'],
+    ['qtap://Project%20Files:%20Voyages/k.md', 'qtap://Project%20Files%3A%20Voyages/k.md'],
+    ['qtap://project/Outline.md', 'qtap://project/Outline.md'],
+    ['qtap://self', 'qtap://self/'],
+  ];
+  it.each(cases)('%s → %s', (input, expected) => {
+    expect(formatQtapUri(parseQtapUri(input))).toBe(expected);
+  });
+});

package/lib/completion/bash.template

@@ -113,7 +113,7 @@ _quilltap_complete() {
       local db_flags="--instance --data-dir --passphrase --json --limit --grep \
         --character --project --about --source --chat --message --rendered --field \
         --tail --last --full --from --type --out --id --diverged --blocked --help \
-        --tables --count --repl --llm-logs --mount-points \
+        --tables --count --repl --write --llm-logs --mount-points \
         --lock-status --lock-clean --lock-override"
       if [[ -z "$subverb" ]]; then
         if [[ "$cur" == -* ]]; then

package/lib/completion/fish.template

@@ -109,7 +109,8 @@ complete -c quilltap -n '__quilltap_using_subcommand db' -l 'type' -d 'Filter by
 complete -c quilltap -n '__quilltap_using_subcommand db' -l 'out' -d 'Output directory' -r -F
 complete -c quilltap -n '__quilltap_using_subcommand db' -l 'tables' -d 'List tables (low-level)'
 complete -c quilltap -n '__quilltap_using_subcommand db' -l 'count' -d 'Count rows in table' -x
-complete -c quilltap -n '__quilltap_using_subcommand db' -l 'repl' -d 'Open SQL REPL'
+complete -c quilltap -n '__quilltap_using_subcommand db' -l 'repl' -d 'Open SQL REPL (read-only unless --write)'
+complete -c quilltap -n '__quilltap_using_subcommand db' -l 'write' -d 'Open database read-write (lock-gated)'
 complete -c quilltap -n '__quilltap_using_subcommand db' -l 'llm-logs' -d 'Target llm-logs database'
 complete -c quilltap -n '__quilltap_using_subcommand db' -l 'mount-points' -d 'Target mount-index database'
 complete -c quilltap -n '__quilltap_using_subcommand db' -l 'lock-status' -d 'Show instance lock status'

package/lib/completion/zsh.template

@@ -125,7 +125,8 @@ _quilltap_db() {
     '--blocked[Only characters with vault issues]'
     '--tables[List tables (low-level)]'
     '--count[Count rows in table]:table:'
-    '--repl[Open SQL REPL]'
+    '--repl[Open SQL REPL (read-only unless --write)]'
+    '--write[Open database read-write (lock-gated; refuses if an instance holds the lock)]'
     '--llm-logs[Target llm-logs database]'
     '--mount-points[Target mount-index database]'
     '--lock-status[Show instance lock status]'