quilltap 4.10.0-dev.9 → 4.10.0-dev.92

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -100,15 +100,19 @@ quilltap instances default Friday # Make it the fall-thro
100
100
  quilltap instances rename Friday Weekday # Rename, preserving the stored passphrase
101
101
  quilltap instances remove Friday # Unregister
102
102
  quilltap instances restore-key Friday # Rebuild a lost or passphrase-locked .dbkey
103
+ quilltap instances default --clear # Revert to the OS platform default
104
+ quilltap instances list --json # The registry as JSON, for scripts
103
105
  ```
104
106
 
105
- If an instance's `quilltap.dbkey` goes missing — or its passphrase does — `instances restore-key` rebuilds it. The file only *wraps* the pepper; the pepper itself is the database key, so an operator who kept the one printed at first-run setup can get back in. The pepper is read from `ENCRYPTION_MASTER_PEPPER` or prompted for hidden, never passed as a flag, and it is proved against the encrypted databases on disk before anything is written. Run it with the server down — the command refuses while the instance lock is held. Flags: `--passphrase <pass>` / `--no-passphrase` (the new wrapping), `-d, --data-dir <path>`, `--force`, `-y, --yes`.
107
+ If an instance's `quilltap.dbkey` goes missing — or its passphrase does — `instances restore-key` rebuilds it. The file only *wraps* the pepper; the pepper itself is the database key, so an operator who kept the one printed at first-run setup can get back in. The pepper is read from `ENCRYPTION_MASTER_PEPPER` or prompted for hidden, never passed as a flag, and it is proved against the encrypted databases on disk before anything is written. Run it with the server down — the command refuses while the instance lock is held. Flags: `--passphrase <pass>` / `--no-passphrase` (the new wrapping), `-d, --data-dir <path>`, `--force` (proceed when there is no encrypted database to prove against — a fresh or still-plaintext instance), `-y, --yes`. An existing key file is backed up to `quilltap.dbkey.bak-<timestamp>` first, and a registered instance's stored passphrase is updated to match.
106
108
 
107
- Every subcommand then accepts `--instance <name>` in place of `--data-dir`. The registry lives at `<app-support>/Quilltap/instances.json` (mode 0600; e.g. `~/Library/Application Support/Quilltap/instances.json` on macOS). **Resolution precedence:** `--data-dir` > `--instance` > registered default > `QUILLTAP_DATA_DIR` > the OS platform default. Pass the **instance root** (e.g. `~/iCloud/Quilltap/Friday`), not its `data/` subdirectory.
109
+ **It does not re-encrypt character archive bundles.** Those are keyed on the *passphrase*, not the pepper; only the server's Change Passphrase card rewrites them. Bundles made under a passphrase you have just replaced still want the old one.
110
+
111
+ Every subcommand then accepts `--instance <name>` in place of `--data-dir`. The registry lives at `<app-support>/Quilltap/instances.json` (mode 0600; e.g. `~/Library/Application Support/Quilltap/instances.json` on macOS). **Resolution precedence:** `--data-dir` > `--instance` > registered default > `QUILLTAP_DATA_DIR` > the OS platform default. Pass the **instance root** (e.g. `~/iCloud/Quilltap/Friday`), not its `data/` subdirectory — the CLI appends `data/quilltap.db` itself. `instances list --json` emits the registry (`name`, `path`, `hasPassphrase`, `isDefault`) for scripting.
108
112
 
109
113
  ## Database Tool
110
114
 
111
- The encrypted SQLite databases (main, LLM logs, mount index) can be queried directly via `quilltap db`. There are two modes: high-level subcommands that auto-pick the right database and resolve characters/chats/projects by name, and a low-level path for arbitrary SQL.
115
+ The encrypted SQLite databases (main, LLM logs, mount index) can be queried directly via `quilltap db`. They are SQLCipher-encrypted, so the stock `sqlite3` binary **cannot** open them — this is the way in. There are two modes: high-level subcommands that auto-pick the right database and resolve characters/chats/projects by name, and a low-level path for arbitrary SQL.
112
116
 
113
117
  ### Subcommands
114
118
 
@@ -135,6 +139,27 @@ quilltap db memories --character Friday [--about Amy] [--source AUTO]
135
139
  quilltap db characters status # Per-character vault readiness (--id, --diverged, --blocked)
136
140
  ```
137
141
 
142
+ SQLite columns are **camelCase**, mirroring the Zod/TypeScript types — `createdAt`, `updatedAt`, `chatType`, `messageCount`, `projectId`, *not* `created_at`. When in doubt, run `quilltap db schema <table>`.
143
+
144
+ ### Character Archive
145
+
146
+ Archiving prunes a character down to a tombstone and packs everything else into an encrypted `ARCHIVE` bundle in the file library; rehydrating puts it all back at its original ids.
147
+
148
+ ```bash
149
+ quilltap db characters archives # Archived characters + the bundles on the shelf
150
+ quilltap db characters archive Ariadne --write # Archive her (server must be running)
151
+ quilltap db characters rehydrate Ariadne --write # Wake her again
152
+ quilltap db characters export Ariadne --out /tmp # Plaintext .qtap, archived or live
153
+ ```
154
+
155
+ `archives` is read-only, and flags **loose** bundles — survivors of a "keep archived bundles" wipe, which are importable but not rehydratable.
156
+
157
+ `archive` and `rehydrate` run **through the running server's API** (`--port`, default 3000), because the export pipeline and the unlocked passphrase live only in the server process; the server, not the CLI, holds the instance lock for the duration. `--write` is still required as the explicit opt-in. A rehydrate restores the pruned material at its original ids (skip-if-present), clears the tombstone, and queues re-embedding; the bundle stays in the file library afterwards as a spare copy. On failure — wrong-era passphrase, missing bundle, import refusal — the character stays archived and re-running is safe.
158
+
159
+ `export` writes a **plaintext** `.qtap` and takes no `--write`. For an **archived** character it decrypts the bundle straight off the disk, offline (it tries the internal no-passphrase key, then `QUILLTAP_DB_PASSPHRASE`, then prompts) — the only way to reach packed-away mail, photographs, and summaries without rehydrating. For a **live** character it runs the server's export pipeline, so the server must be up.
160
+
161
+ **Pre-emptive, not recovery:** exporting an archive needs an instance that can still decrypt it. It is no help to someone holding only a restored backup and a forgotten passphrase — which is also why changing an instance's passphrase rewrites every archive bundle. A bundle reported left behind by a partial rewrite still wants the old one.
162
+
138
163
  ### Maintenance and Snapshots
139
164
 
140
165
  ```bash
@@ -149,7 +174,7 @@ quilltap db integrity # cipher_integrity_check + integrity
149
174
  quilltap db integrity llm-logs # one DB; exit 0 ok, 1 issues, 2 open failure
150
175
  ```
151
176
 
152
- `backup` and `integrity` are safe to run while the server is up; `optimize` refuses while a live lock is held. Backups default to `<dataDir>/backups/<timestamp>/` and inherit the source's encryption key transparently.
177
+ `backup` and `integrity` are safe to run while the server is up; `optimize` refuses while a live lock is held (see [Locking](#locking)). Backups default to `<dataDir>/backups/<timestamp>/` and inherit the source's encryption key transparently.
153
178
 
154
179
  Most subcommands accept `--json` (for piping) and `--limit N`. Names are case-insensitive; aliases are searched alongside character names. Ambiguous matches print all candidates and exit non-zero.
155
180
 
@@ -166,10 +191,39 @@ quilltap db --llm-logs --tables # Target the LLM logs DB
166
191
  quilltap db --mount-points --tables # Target the mount index DB
167
192
  ```
168
193
 
169
- The database is opened **read-only by default**. Add `--write` to make changes: it opens the database read-write, **claims the instance lock** (`<dataDir>/quilltap.lock`) for the duration, and releases it on exit. It **refuses — with no override — if a running server or another instance holds the lock**, so stop the server first. `--repl` is read-only unless combined with `--write`. Attempting a write without `--write` fails with a hint to re-run with the flag.
194
+ The database is opened **read-only by default**. Add `--write` to make changes: it opens the database read-write, **claims the instance lock** (`<dataDir>/quilltap.lock`) for the duration, and releases it on exit. It **refuses — with no override — if a running server or another instance holds the lock**, so stop the server first. `--repl` is read-only unless combined with `--write`. Attempting a write without `--write` fails with a hint to re-run with the flag. What "held" means, and how to tell a live lock from a stale one, is [below](#locking).
195
+
196
+ Compressed text columns (`chat_messages.content`, `llm_logs.request` / `response` and friends) are stored as BLOBs. Wrap them in `qt_text()` to read the text: `SELECT qt_text(content) …`.
170
197
 
171
198
  In the REPL, `.cols <table>` and `.find <text>` mirror the subcommand helpers.
172
199
 
200
+ ## Locking
201
+
202
+ Everything that writes to an instance claims `<dataDir>/quilltap.lock` — the same lockfile the server itself uses — for the duration, and releases it on exit. That is `db --write` (including `db --repl --write`), `db optimize`, `maintenance run`, and `instances restore-key`. All four **refuse while the lock is held**: stop the server first. Read-only work — plain `db`, the `docs` read verbs, `memories`, `logs`, `migrations`, `maintenance status`, `db backup`, `db integrity` — never touches the lock and is safe alongside a running instance.
203
+
204
+ ### The five-minute heartbeat window
205
+
206
+ A lock counts as held until its heartbeat is **five minutes** stale, whether or not the process that set it is still alive. Freshness is the fallback for every environment, not just containers: a PID check is not reliable everywhere, and cleaning a *live* instance's lock is much the worse failure.
207
+
208
+ So for up to five minutes after stopping the server, `--write`, `optimize`, `maintenance run`, `restore-key`, and `--lock-clean` all still refuse. That is correct behaviour, not a stale lock. Wait it out; the next startup reclaims the lock regardless.
209
+
210
+ ```bash
211
+ quilltap db --lock-status # Who holds it, and how old the heartbeat is
212
+ quilltap db --lock-clean # Remove a lock whose heartbeat has gone stale
213
+ ```
214
+
215
+ `--lock-status` shows the heartbeat age, which is the tell. `--lock-clean` says so explicitly rather than muttering about liveness, because that arm is only reached once the PID check has come back dead:
216
+
217
+ ```
218
+ Lock heartbeat is still fresh (82s ago). Cannot clean.
219
+ A lock counts as held until its heartbeat is 5 minutes stale, even if its process has gone.
220
+ Wait it out, or use --lock-override to force.
221
+ ```
222
+
223
+ When a live process really does hold the lock, it refuses with "Lock is held by a live Quilltap process" instead.
224
+
225
+ **`--lock-override` exists and is almost never the right answer** — it defeats the protection the lock provides. Reach for `--lock-status` first.
226
+
173
227
  ## Document Stores (Scriptorium)
174
228
 
175
229
  `quilltap docs` exposes the document-store machinery from the command line. Read-only verbs open the mount-index DB directly and work without the server; write and pipeline verbs talk to the running server via `/api/v1/mount-points/[id]`.
@@ -178,6 +232,7 @@ In the REPL, `.cols <table>` and `.find <text>` mirror the subcommand helpers.
178
232
  # Read
179
233
  quilltap docs list # All mounts
180
234
  quilltap docs show <mount> # One mount, with counts
235
+ quilltap docs files <mount> [--folder <path>] # Flat file list for a mount
181
236
  quilltap docs ls <mount> [path] [--links] # POSIX-flavoured listing (alias: dir)
182
237
  quilltap docs tree <mount> [path] # ASCII tree of a folder hierarchy (--depth, --max-nodes)
183
238
  quilltap docs read [--rendered] <mount> <path> # File contents → stdout
@@ -203,7 +258,24 @@ quilltap docs rmdir <mount> <path> # Delete an empty
203
258
  quilltap docs mvdir <mount> <fromPath> <toPath> # Rename/move a folder (server-required)
204
259
  ```
205
260
 
206
- Mount arguments accept the mount name (case-insensitive) or a UUID; ambiguous names print candidates and exit non-zero. `--json` is supported by every verb; `reindex`, `embed`, `link`, `rmdir`, and `mvdir` refuse to run without a reachable server.
261
+ Mount arguments accept the mount name (case-insensitive) or a UUID; ambiguous names print candidates and exit non-zero. `--json` is supported by every verb; `reindex`, `embed`, `link`, `rmdir`, and `mvdir` refuse to run without a reachable server. `grep --semantic` goes through `POST /api/v1/mount-points?action=semantic-search`, because the embedding provider lives in the server; it defaults to `--top 20`, `--threshold 0.5`, `--port 3000`.
262
+
263
+ ### Addressing documents with `qtap://` URIs
264
+
265
+ Anywhere a verb takes a positional `<mount> <relativePath>` pair — `read`, `write`, `delete`, `mkdir`, `ls`/`dir`, `tree`, `files`, `move`, `copy`, `link`, `rmdir`, `mvdir` — you may pass a single `qtap://…` URI in its place:
266
+
267
+ ```bash
268
+ quilltap docs read qtap://notes/today.md
269
+ quilltap docs move qtap://drafts/foo.md qtap://notes/2026/foo.md
270
+ quilltap docs grep --mount qtap://notes/ "TODO"
271
+ quilltap docs find --uri Manifesto
272
+ ```
273
+
274
+ The URI authority is matched name-first, UUID as fallback — the same rule as a bare `<mount>` (`qtap://<store name>/…` or `qtap://<uuid>/…`). Two-target verbs (`move` / `copy` / `link` / `mvdir`) take either two `qtap://` URIs or the four legacy positionals; `find` and `grep` take one via `--mount`.
275
+
276
+ **CLI limitation:** the CLI addresses document stores only. `qtap://self/…` needs a character context, and there is none at a shell prompt, so it is rejected with guidance; `qtap://project/…` and `qtap://general/…` are likewise not CLI-addressable — pass a store name or UUID instead.
277
+
278
+ **Emitting URIs:** `--json` output for `find`, `grep`, `ls`, `files`, and `tree` carries a `uri` field on every row or node. `--uri` switches the text output of `find`, `grep`, and `files` to show the canonical `qtap://` URI as the locator (name form, UUID when the store name is ambiguous).
207
279
 
208
280
  ### `--base64` flag
209
281
 
@@ -213,12 +285,49 @@ Mount arguments accept the mount name (case-insensitive) or a UUID; ambiguous na
213
285
 
214
286
  ### `link`, `rmdir`, `mvdir`
215
287
 
288
+ **`link` vs `copy`:** `docs link` makes two addresses into one document — it shares the content row *and* enrols both link rows in a `linkGroupId`, so a later write through either path repoints both and re-chunks the sibling. `docs copy` produces an independent document that merely shares a deduplicated content row until the first write. The `links` column in `ls` counts group members, not rows that happen to share identical bytes.
289
+
216
290
  `link` calls `POST /api/v1/mount-points/{srcMountId}?action=link-file` with `{sourcePath, destMountPointId, destPath}`. Creates a true hard link with no byte copy; the server reports back a `strategy` field. Errors: `DEST_EXISTS` (exit 2), `UNSUPPORTED` (cross-storage or cross-device), `SOURCE_NOT_FOUND`.
217
291
 
218
292
  `rmdir` calls `POST /api/v1/mount-points/{mountId}?action=delete-folder` with `{path}`. Fails with a clear message if the folder is not empty (`NOT_EMPTY` / `CONFLICT`).
219
293
 
220
294
  `mvdir` calls `POST /api/v1/mount-points/{mountId}?action=move-folder` with `{fromPath, toPath}`. Fails with exit 2 if the destination already exists (`DEST_EXISTS`).
221
295
 
296
+ ### Docker binds
297
+
298
+ `quilltap docs docker-mounts` reports the bind mounts an instance's filesystem and Obsidian stores need in order to be reachable inside a container. Binds are **path-identical** (`-v /host/vault:/host/vault`), so the `basePath` recorded in the database resolves the same inside and out. Stores sharing a path collapse to a single bind, paths nested inside another bind are dropped, and a path that does not exist is **skipped** rather than handed to Docker to fabricate as an empty directory. It warns about macOS paths outside Docker Desktop's default shares and about a Linux uid mismatch, and refuses on Windows, where path-identical binds are not possible. `--format args` puts only the flags on stdout and all advice on stderr, which is what makes it pipeable into a `docker run`.
299
+
300
+ ## Sync a Store to a Directory
301
+
302
+ `quilltap sync <store|qtap://store/> <path>` mirrors a **database-backed** document store and a directory on disk in both directions. Edit a file in your own editor and the next run carries it into the store; edit it in the Scriptorium and the next run carries it out.
303
+
304
+ ```bash
305
+ quilltap sync Lore ~/Documents/lore --dry-run # plan only
306
+ quilltap sync Lore ~/Documents/lore # apply
307
+ quilltap sync Lore ~/Documents/lore --prefer disk
308
+ ```
309
+
310
+ ```text
311
+ --dry-run Plan and print; change nothing on either side
312
+ --direction <which> both (default), to-disk, or to-store
313
+ --prefer <which> newer (default), store, or disk — resolves conflicts
314
+ --no-delete Never propagate a deletion
315
+ --no-manifest Ignore .quilltap-sync.json (first-run rules every time)
316
+ --json Machine-readable plan and results
317
+ -p, --port <n> Server port (default 3000)
318
+ -i, -d, --passphrase The usual instance plumbing (used only to resolve <store>)
319
+ ```
320
+
321
+ Compares by SHA-256 first and modification time second, so equal bytes with unequal clocks are re-stamped rather than re-copied. The side that changed wins; when both changed since the last run it is a `conflict` and nothing happens. After a content action both sides carry the winner's `lastModified` and the **older** of the two `createdAt`s. Deletions propagate only when `.quilltap-sync.json` — a manifest the verb keeps in the directory — proves the entry was there at the last run; on a first run an entry present on one side is created on the other, never deleted.
322
+
323
+ Files and folders whose names begin with a dot are **invisible in both directions**, the manifest being the one exception. A binary's description travels as `<file>.description.md` beside it. Bytes are preserved verbatim: a `.png` pushed from disk stays a `.png`, unlike a Scriptorium upload. Chunks and embedding vectors are never touched by the sync — the store's own post-write hooks re-index.
324
+
325
+ Exit codes: `0` clean, `1` error or failed action, `2` unresolved conflict; `--dry-run` uses the same codes. Report lines go to stdout, warnings and the summary to stderr.
326
+
327
+ Server-required (as `docs write` already is for database stores), and the path is resolved **on the server** — under Docker it must sit inside a bind mount (`quilltap docs docker-mounts`). Refused for a filesystem or Obsidian store (one pointed at its own `basePath`), an archived character's vault, a store mid-conversion or mid-scan, a second concurrent run, and a manifest belonging to another store.
328
+
329
+ A character vault's **keystone files** — `properties.json`, the five required `.md` files, and `Wardrobe/instructions.md` — are never deleted from the store by a sync. The planner reports a `conflict` instead.
330
+
222
331
  ## Memories
223
332
 
224
333
  `quilltap memories` exposes the same Commonplace Book that each character carries — searchable, sortable, graphable, but never writable. All verbs open the main encrypted DB read-only.
@@ -231,10 +340,11 @@ quilltap memories grep -i --max 3 --context 1 "concrete examples" # Pattern
231
340
  quilltap memories show <id|prefix> [--depth N] [--no-related] # Full record + related-memory neighbourhood
232
341
  quilltap memories tree <id|prefix> [--depth N] [--max-nodes N] # ASCII walk of the bidirectional related-memory graph
233
342
  quilltap memories status [--character <name|id>] # Per-holder rollup + dangling-edge check
343
+ quilltap memories validate [--character <name|id>] [--list] # Dangling-edge health check; exit 1 if any remain
234
344
  quilltap memories grep --semantic --character Ariadne "the argument" # Embedding search (server required, one holder)
235
345
  ```
236
346
 
237
- Shared filter flags apply to `ls`, `find`, `grep`, and `status` where they make sense: `--character`, `--about` (with `self` / `none` shortcuts), `--source`, `--chat` (with `none` for manual entries), `--project`, `--since`, `--until`, `--min-importance`, `--min-reinforced`, `--has-embedding` / `--no-embedding`. Sort flags (`--sort reinforced|importance|created|accessed|reinforcement-count|links`, plus `-r` to reverse) apply to `ls`, `find`, and `grep`. Names accept fuzzy substrings; ambiguous names print candidates and exit 2. `--json` is supported by every verb. The legacy `quilltap db memories --character <name>` verb remains undisturbed.
347
+ Shared filter flags apply to `ls`, `find`, `grep`, and `status` where they make sense: `--character`, `--about` (with `self` / `none` shortcuts), `--source`, `--chat` (with `none` for manual entries), `--project`, `--since`, `--until`, `--min-importance`, `--min-reinforced`, `--has-embedding` / `--no-embedding`. Sort flags (`--sort reinforced|importance|created|accessed|reinforcement-count|links`, plus `-r` to reverse) apply to `ls`, `find`, and `grep`. Names accept fuzzy substrings; ambiguous names print candidates and exit 2. `--json` is supported by every verb. `validate` is `status`'s terse twin — read-only, exit 1 if any dangling related-memory edge remains, `--list` to print the offending source IDs and their dangling targets. `grep --semantic` defaults to `--top 20`, `--threshold 0.5`, `--port 3000`, and scopes to **one holder at a time**: `--character all` is rejected. The legacy `quilltap db memories --character <name>` verb remains undisturbed.
238
348
 
239
349
  ## Memory Extraction Dry-Run
240
350
 
package/bin/quilltap.js CHANGED
@@ -11,6 +11,7 @@ const {
11
11
  printDefaultInstanceHint,
12
12
  promptPassphrase,
13
13
  loadDbKey,
14
+ openEncryptedDb,
14
15
  } = require('../lib/db-helpers');
15
16
  const { resolveInstance } = require('../lib/instances');
16
17
  const { resolveModuleDir, ensureNativeModules, ensureDatabaseNativeModule } = require('../lib/native-modules');
@@ -92,6 +93,7 @@ Subcommands:
92
93
  db Query encrypted databases
93
94
  themes Manage theme bundles
94
95
  docs Inspect, read, and export document mounts
96
+ sync <store> <path> Mirror a database-backed store to a directory
95
97
  memories Search, browse, and graph memories
96
98
  instances Register / inspect named Quilltap instances
97
99
  logs Tail or print an instance log file
@@ -486,6 +488,18 @@ function verifyPidIsNode(pid, expectedArgv0) {
486
488
 
487
489
  const HEARTBEAT_FRESH_MS = 5 * 60 * 1000;
488
490
 
491
+ /**
492
+ * The freshness window in words, derived from the constant the check actually
493
+ * uses — so the sentence `--lock-clean` prints cannot drift away from the
494
+ * behaviour it describes.
495
+ */
496
+ function describeFreshWindow() {
497
+ const seconds = Math.round(HEARTBEAT_FRESH_MS / 1000);
498
+ if (seconds % 60 !== 0) return `${seconds} seconds`;
499
+ const minutes = seconds / 60;
500
+ return minutes === 1 ? '1 minute' : `${minutes} minutes`;
501
+ }
502
+
489
503
  /**
490
504
  * Decide whether a lock record belongs to a live process.
491
505
  *
@@ -628,9 +642,17 @@ function handleLockCommand(dataDir, opts) {
628
642
  console.log('Stop the running instance first, or use --lock-override to force.');
629
643
  process.exit(1);
630
644
  } else if (heartbeatFresh) {
645
+ // Reached only when `alive` is false — the arm above has already claimed
646
+ // every confirmed-live case — so this must not assert that anything is
647
+ // running. Freshness alone is the deliberate fallback (bug 126) for
648
+ // environments where PID checks are unreliable; the refusal is right,
649
+ // and what it says is simply what was tested.
631
650
  const ageStr = Math.round(heartbeatAgeMs / 1000) + 's';
632
- console.log(`Lock is still being refreshed (heartbeat ${ageStr} ago) — its holder is alive. Cannot clean.`);
633
- console.log('Stop the running instance first, or use --lock-override to force.');
651
+ console.log(`Lock heartbeat is still fresh (${ageStr} ago). Cannot clean.`);
652
+ console.log(
653
+ `A lock counts as held until its heartbeat is ${describeFreshWindow()} stale, even if its process has gone. ` +
654
+ 'Wait it out, or use --lock-override to force.'
655
+ );
634
656
  process.exit(1);
635
657
  } else if (alive && !isNode) {
636
658
  console.log(`Lock references PID ${lock.pid} which is alive but does NOT look like a Quilltap process.`);
@@ -961,9 +983,10 @@ async function dbCommand(args) {
961
983
  }
962
984
 
963
985
  let dbFilename;
964
- if (useLlmLogs) dbFilename = 'quilltap-llm-logs.db';
965
- else if (useMountPoints) dbFilename = 'quilltap-mount-index.db';
966
- else dbFilename = 'quilltap.db';
986
+ let dbFriendlyName;
987
+ if (useLlmLogs) { dbFilename = 'quilltap-llm-logs.db'; dbFriendlyName = 'LLM logs database'; }
988
+ else if (useMountPoints) { dbFilename = 'quilltap-mount-index.db'; dbFriendlyName = 'mount index database'; }
989
+ else { dbFilename = 'quilltap.db'; dbFriendlyName = 'main database'; }
967
990
  const dbPath = path.join(dataDir, dbFilename);
968
991
 
969
992
  if (!fs.existsSync(dbPath)) {
@@ -993,27 +1016,16 @@ async function dbCommand(args) {
993
1016
  }
994
1017
  }
995
1018
 
996
- // Open database — prefer SQLCipher-capable build
997
- let Database;
1019
+ // Open through the one shared opener. It picks the SQLCipher-capable build,
1020
+ // keys the connection, verifies it, and — the reason the low-level path may
1021
+ // not roll its own — registers qt_text(). Without it raw SQL cannot read
1022
+ // inside a compressed text column, and a --write to chat_messages fails
1023
+ // outright because the search-index triggers call it (bug 162).
1024
+ let db;
998
1025
  try {
999
- Database = require('better-sqlite3-multiple-ciphers');
1000
- } catch {
1001
- Database = require('better-sqlite3');
1002
- }
1003
- const db = new Database(dbPath, { readonly: !writable });
1004
-
1005
- if (pepper) {
1006
- const keyHex = Buffer.from(pepper, 'base64').toString('hex');
1007
- db.pragma(`key = "x'${keyHex}'"`);
1008
- }
1009
-
1010
- try {
1011
- // Verify database is readable
1012
- db.prepare('SELECT 1').get();
1026
+ db = openEncryptedDb(dbPath, pepper, { readonly: !writable, friendlyName: dbFriendlyName });
1013
1027
  } catch (err) {
1014
- console.error(`Cannot open database: ${err.message}`);
1015
- console.error('The database may be encrypted with a different key, or the .dbkey file may be missing.');
1016
- db.close();
1028
+ console.error(err.message);
1017
1029
  process.exit(1);
1018
1030
  }
1019
1031
 
@@ -1154,7 +1166,7 @@ async function dbCommand(args) {
1154
1166
  // to the subcommand. Each subcommand parses these flags position-independently,
1155
1167
  // so they behave the same before or after the verb.
1156
1168
  const SUBCOMMANDS = new Set([
1157
- 'db', 'themes', 'docs', 'memories', 'instances', 'memory-diff', 'recall-replay', 'completion', 'logs', 'migrations', 'maintenance', 'file-verify',
1169
+ 'db', 'themes', 'docs', 'sync', 'memories', 'instances', 'memory-diff', 'recall-replay', 'completion', 'logs', 'migrations', 'maintenance', 'file-verify',
1158
1170
  ]);
1159
1171
  // Global flags that consume the following token as their value.
1160
1172
  const GLOBAL_VALUE_FLAGS = new Set(['-p', '--port', '-d', '--data-dir', '-i', '--instance', '--passphrase']);
@@ -1193,6 +1205,12 @@ if (subName === 'db') {
1193
1205
  } else if (subName === 'docs') {
1194
1206
  const { docsCommand } = require('../lib/docs-commands');
1195
1207
  docsCommand(subArgs);
1208
+ } else if (subName === 'sync') {
1209
+ const { syncCommand } = require('../lib/sync-command');
1210
+ syncCommand(subArgs).catch(err => {
1211
+ console.error(`Error: ${err.message}`);
1212
+ process.exit(1);
1213
+ });
1196
1214
  } else if (subName === 'memories') {
1197
1215
  const { memoriesCommand } = require('../lib/memories-commands');
1198
1216
  memoriesCommand(subArgs).catch(err => {
@@ -108,6 +108,7 @@ describe('every subcommand has its own completion arm', () => {
108
108
  const HELP_SOURCES = {
109
109
  db: ['bin/quilltap.js', 'printDbHelp'],
110
110
  docs: ['lib/docs-commands.js', 'printDocsHelp'],
111
+ sync: ['lib/sync-command.js', 'printSyncHelp'],
111
112
  memories: ['lib/memories-commands.js', 'printMemoriesHelp'],
112
113
  themes: ['lib/theme-commands.js', 'printHelp'],
113
114
  instances: ['lib/instances-commands.js', 'printHelp'],
@@ -0,0 +1,158 @@
1
+ /**
2
+ * How `quilltap sync` renders a plan.
3
+ *
4
+ * The formatter is the operator's whole view of what the verb did, and its
5
+ * two load-bearing claims are easy to get subtly wrong: the second column is
6
+ * the side that CHANGES (so `modify store` means the store is rewritten from
7
+ * disk, not the other way round), and the exit code distinguishes "an
8
+ * unresolved conflict" from "something failed" so a script can tell them apart.
9
+ *
10
+ * @jest-environment node
11
+ */
12
+
13
+ 'use strict';
14
+
15
+ const {
16
+ formatActionLine,
17
+ formatActionLines,
18
+ formatSummary,
19
+ exitCodeFor,
20
+ displayPath,
21
+ formatBytes,
22
+ } = require('../sync-report');
23
+
24
+ function action(over = {}) {
25
+ return { kind: 'create', side: 'disk', relativePath: 'a.md', entryKind: 'file', ...over };
26
+ }
27
+
28
+ function summary(over = {}) {
29
+ return {
30
+ created: 0, modified: 0, deleted: 0, touched: 0,
31
+ described: 0, conflicts: 0, skipped: 0, failed: 0, ...over,
32
+ };
33
+ }
34
+
35
+ describe('one action, one line', () => {
36
+ it('puts the action first and the side that changes second', () => {
37
+ expect(formatActionLine(action({ kind: 'modify', side: 'store', relativePath: 'ch.md' }), false))
38
+ .toBe('modify store ch.md');
39
+ });
40
+
41
+ it('pads the columns so paths line up under each other', () => {
42
+ const lines = formatActionLines([
43
+ action({ kind: 'mkdir', side: 'disk', relativePath: 'lore', entryKind: 'folder' }),
44
+ action({ kind: 'conflict', side: null, relativePath: 'ch.md' }),
45
+ ], false);
46
+ const column = lines.map((l) => l.indexOf(l.trim().split(/\s+/)[2] || ''));
47
+ expect(new Set(column).size).toBe(1);
48
+ });
49
+
50
+ it('marks a folder with a trailing slash so it cannot be mistaken for a file', () => {
51
+ expect(displayPath(action({ entryKind: 'folder', relativePath: 'drafts' }))).toBe('drafts/');
52
+ expect(displayPath(action({ entryKind: 'file', relativePath: 'drafts' }))).toBe('drafts');
53
+ });
54
+
55
+ it('writes an em dash where no side changes', () => {
56
+ expect(formatActionLine(action({ kind: 'conflict', side: null }), false))
57
+ .toContain('conflict —');
58
+ });
59
+
60
+ it('appends the engine’s reason in parentheses', () => {
61
+ expect(formatActionLine(action({ kind: 'modify', side: 'store', reason: 'disk newer by 2h 14m' }), false))
62
+ .toContain('(disk newer by 2h 14m)');
63
+ });
64
+
65
+ it('shows the sha and size on a line that moved bytes', () => {
66
+ const line = formatActionLine(
67
+ action({ sha256: '3f9a' + '0'.repeat(60), sizeBytes: 421888 }), false
68
+ );
69
+ expect(line).toContain('sha 3f9a…');
70
+ expect(line).toContain('412.0 KB');
71
+ });
72
+
73
+ it('says nothing about bytes on a delete', () => {
74
+ expect(formatActionLine(action({ kind: 'delete', side: 'store', sha256: 'x'.repeat(64) }), false))
75
+ .not.toContain('sha');
76
+ });
77
+
78
+ it('leads with the error when an action failed', () => {
79
+ const line = formatActionLine(
80
+ action({ outcome: 'failed', error: 'EACCES', reason: 'store newer' }), false
81
+ );
82
+ expect(line).toContain('FAILED: EACCES');
83
+ expect(line).not.toContain('store newer');
84
+ });
85
+
86
+ it('emits no escape codes when colour is off', () => {
87
+ const lines = formatActionLines(
88
+ [action(), action({ kind: 'conflict', side: null }), action({ kind: 'delete', side: 'store' })],
89
+ false
90
+ );
91
+ expect(lines.join('\n')).not.toMatch(/\x1b\[/);
92
+ });
93
+
94
+ it('tints the line when colour is on', () => {
95
+ expect(formatActionLine(action({ kind: 'conflict', side: null }), true)).toMatch(/\x1b\[31m/);
96
+ });
97
+
98
+ it('caps the path column so one long path does not push every detail off screen', () => {
99
+ const lines = formatActionLines([
100
+ action({ relativePath: 'a/'.repeat(60) + 'deep.md', reason: 'why' }),
101
+ action({ relativePath: 'b.md', reason: 'why' }),
102
+ ], false);
103
+ expect(lines[1].indexOf('(why)')).toBeLessThan(80);
104
+ });
105
+ });
106
+
107
+ describe('the summary', () => {
108
+ it('names only the categories that actually happened', () => {
109
+ expect(formatSummary(summary({ created: 3, modified: 1 }), 800, false))
110
+ .toBe('3 created, 1 modified — 0.8 s');
111
+ });
112
+
113
+ it('says so plainly when there was nothing to do', () => {
114
+ expect(formatSummary(summary(), 120, false)).toBe('Already in step — 0.1 s');
115
+ });
116
+
117
+ it('speaks in the conditional under --dry-run', () => {
118
+ expect(formatSummary(summary({ created: 2 }), 300, true)).toBe('Would do: 2 created — 0.3 s');
119
+ expect(formatSummary(summary(), 300, true)).toBe('Nothing to do — 0.3 s');
120
+ });
121
+
122
+ it('pluralises conflicts', () => {
123
+ expect(formatSummary(summary({ conflicts: 1 }), 0, false)).toContain('1 conflict —');
124
+ expect(formatSummary(summary({ conflicts: 2 }), 0, false)).toContain('2 conflicts');
125
+ });
126
+
127
+ it('reports failures last, where they are hardest to miss', () => {
128
+ const line = formatSummary(summary({ created: 1, failed: 2 }), 0, false);
129
+ expect(line.indexOf('2 failed')).toBeGreaterThan(line.indexOf('1 created'));
130
+ });
131
+ });
132
+
133
+ describe('the exit code', () => {
134
+ it('is 0 when the run was clean', () => {
135
+ expect(exitCodeFor(summary({ created: 5, touched: 2 }))).toBe(0);
136
+ });
137
+
138
+ it('is 2 for an unresolved conflict', () => {
139
+ expect(exitCodeFor(summary({ conflicts: 1 }))).toBe(2);
140
+ });
141
+
142
+ it('is 1 when something failed outright, even alongside a conflict', () => {
143
+ expect(exitCodeFor(summary({ conflicts: 1, failed: 1 }))).toBe(1);
144
+ });
145
+ });
146
+
147
+ describe('byte formatting', () => {
148
+ it('scales through the units', () => {
149
+ expect(formatBytes(512)).toBe('512 B');
150
+ expect(formatBytes(2048)).toBe('2.0 KB');
151
+ expect(formatBytes(5 * 1024 * 1024)).toBe('5.0 MB');
152
+ expect(formatBytes(3 * 1024 ** 3)).toBe('3.00 GB');
153
+ });
154
+
155
+ it('says nothing for a size it was not given', () => {
156
+ expect(formatBytes(undefined)).toBe('');
157
+ });
158
+ });
@@ -61,7 +61,7 @@ _quilltap_complete() {
61
61
  local global_opts="-d --data-dir -i --instance -p --port -o --open -v --version -h --help --update --passphrase"
62
62
 
63
63
  # Top-level subcommands
64
- local top_cmds="db docs themes instances memories memory-diff recall-replay logs migrations maintenance file-verify completion"
64
+ local top_cmds="db docs sync themes instances memories memory-diff recall-replay logs migrations maintenance file-verify completion"
65
65
 
66
66
  # Flags that swallow the word after them. A flat list will not do: -o is the
67
67
  # valueless global --open but themes' valued --output, and `memories`
@@ -71,6 +71,7 @@ _quilltap_complete() {
71
71
  local vf_global=" -d --data-dir -i --instance -p --port --passphrase "
72
72
  local vf_db=" --limit --grep --character --project --about --source --chat --message --field --tail --last --from --type --out --id --count "
73
73
  local vf_docs=" --mount --folder --type --ext --limit --max --context --top --threshold --sort --depth --max-nodes --format "
74
+ local vf_sync=" --direction --prefer "
74
75
  local vf_themes=" -o --output -k --key -n --name "
75
76
  local vf_memories=" -d --data-dir --instance --passphrase --port --character --about --source --chat --project --since --until --min-importance --min-reinforced --sort --limit --in --max --context --depth --max-nodes --top --threshold "
76
77
  local vf_logs=" --stream --tail --grep "
@@ -93,6 +94,7 @@ _quilltap_complete() {
93
94
  case "$subcommand" in
94
95
  db) valued="$vf_global$vf_db" ;;
95
96
  docs) valued="$vf_global$vf_docs" ;;
97
+ sync) valued="$vf_global$vf_sync" ;;
96
98
  themes) valued="$vf_global$vf_themes" ;;
97
99
  memories) valued="$vf_memories" ;;
98
100
  logs) valued="$vf_global$vf_logs" ;;
@@ -197,6 +199,14 @@ _quilltap_complete() {
197
199
  COMPREPLY=($(compgen -W "args json" -- "$cur"))
198
200
  return
199
201
  ;;
202
+ --direction)
203
+ COMPREPLY=($(compgen -W "both to-disk to-store" -- "$cur"))
204
+ return
205
+ ;;
206
+ --prefer)
207
+ COMPREPLY=($(compgen -W "newer store disk" -- "$cur"))
208
+ return
209
+ ;;
200
210
  esac
201
211
 
202
212
  # Subcommand-specific completion
@@ -241,6 +251,18 @@ _quilltap_complete() {
241
251
  _quilltap_docs_positional "$subverb" "$positional_count" "$cur"
242
252
  fi
243
253
  ;;
254
+ sync)
255
+ local sync_flags="--dry-run --direction --prefer --no-delete --no-manifest \
256
+ --json --instance --data-dir --passphrase --port --help"
257
+ if [[ "$cur" == -* ]]; then
258
+ COMPREPLY=($(compgen -W "$sync_flags" -- "$cur"))
259
+ elif [[ "$positional_count" == "1" ]]; then
260
+ _quilltap_lines_compreply "$cur" \
261
+ <<< "$(command quilltap docs list --names-only "${ctx_flags[@]}" 2>/dev/null)"
262
+ else
263
+ COMPREPLY=($(compgen -d -- "$cur"))
264
+ fi
265
+ ;;
244
266
  themes)
245
267
  local themes_verbs="list install uninstall validate export create search update registry"
246
268
  local themes_flags="--instance --data-dir --output -o --help"
@@ -19,7 +19,7 @@ function __quilltap_no_subcommand
19
19
  end
20
20
  for i in (seq 2 (count $cmd))
21
21
  switch $cmd[$i]
22
- case db docs themes instances memories memory-diff recall-replay logs migrations maintenance file-verify completion
22
+ case db docs sync themes instances memories memory-diff recall-replay logs migrations maintenance file-verify completion
23
23
  return 1
24
24
  end
25
25
  end
@@ -31,7 +31,7 @@ function __quilltap_using_subcommand
31
31
  set -l cmd (commandline -opc)
32
32
  for i in (seq 2 (count $cmd))
33
33
  switch $cmd[$i]
34
- case db docs themes instances memories memory-diff recall-replay logs migrations maintenance file-verify completion
34
+ case db docs sync themes instances memories memory-diff recall-replay logs migrations maintenance file-verify completion
35
35
  test "$cmd[$i]" = "$target"
36
36
  return $status
37
37
  end
@@ -55,6 +55,7 @@ end
55
55
 
56
56
  complete -c quilltap -n '__quilltap_no_subcommand' -f -a 'db' -d 'Query encrypted databases'
57
57
  complete -c quilltap -n '__quilltap_no_subcommand' -f -a 'docs' -d 'Inspect and read document mounts'
58
+ complete -c quilltap -n '__quilltap_no_subcommand' -f -a 'sync' -d 'Mirror a database-backed store to a directory'
58
59
  complete -c quilltap -n '__quilltap_no_subcommand' -f -a 'themes' -d 'Manage theme bundles'
59
60
  complete -c quilltap -n '__quilltap_no_subcommand' -f -a 'instances' -d 'Register or inspect instances'
60
61
  complete -c quilltap -n '__quilltap_no_subcommand' -f -a 'memories' -d 'Search and browse memories'
@@ -188,6 +189,22 @@ complete -c quilltap -n '__quilltap_using_subcommand docs' -l 'uri' -d 'Show can
188
189
  complete -c quilltap -n '__quilltap_using_subcommand docs' -l 'base64' -d 'Base64 transfer for binary files'
189
190
  complete -c quilltap -n '__quilltap_using_subverb docs docker-mounts' -l 'format' -d 'Output shape' -x -a 'args json'
190
191
 
192
+ # ---------- sync ----------
193
+ # `quilltap sync <store> <path>`: the first positional is a store, the second a
194
+ # local directory — so the verb is not -f (file completion stays available).
195
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -f -a '(__quilltap_mount_names)' -d 'Document store'
196
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'dry-run' -d 'Plan and print; change nothing'
197
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'direction' -d 'Which side may change' -x -a 'both to-disk to-store'
198
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'prefer' -d 'How to resolve a conflict' -x -a 'newer store disk'
199
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'no-delete' -d 'Never propagate a deletion'
200
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'no-manifest' -d 'Ignore .quilltap-sync.json'
201
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'json' -d 'Machine-readable plan and results'
202
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'port' -s 'p' -d 'Server port' -x
203
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'instance' -s 'i' -d 'Registered instance' -x -a '(__quilltap_instance_names)'
204
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'data-dir' -s 'd' -d 'Data directory' -x -a '(__fish_complete_directories)'
205
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'passphrase' -d 'Database passphrase' -x
206
+ complete -c quilltap -n '__quilltap_using_subcommand sync' -l 'help' -s 'h' -d 'Show help'
207
+
191
208
  # ---------- themes verbs ----------
192
209
  complete -c quilltap -n '__quilltap_using_subcommand themes' -f -a 'list' -d 'List themes'
193
210
  complete -c quilltap -n '__quilltap_using_subcommand themes' -f -a 'install' -d 'Install theme'