memoir-cli 3.12.0 → 3.14.0

package/docs/PROJECT-VIEW-VALIDATION.md

@@ -0,0 +1,136 @@
+# Local project view validation — 2026-09-06
+
+The local project view is implemented and tested in this checkout. Nothing was
+committed, pushed, published to npm, or deployed. Existing Codex and Cursor
+connection settings were preserved. The managed project instructions gained a
+view command; setup backed up their previous contents.
+
+## Use it
+
+Ask the agent **“Open my Memoir project view.”** From this source checkout,
+`node bin/memoir.js work view` opens it in a browser. Keep the terminal process
+running; Ctrl+C stops it. No account or extra package is required beyond the
+repository's existing dependencies.
+
+Search answers, decisions, goals, next steps and check evidence. Correct a record,
+mark a step done, or remove it from the handoff. Removed records can be restored.
+Removal retains local history and is not permanent erasure. Checks cannot be
+edited or restored into a pass; a new authorized execution is required.
+
+## Browser evidence
+
+Actual browser interactions used this project's real local ledger:
+
+| Exercise | Observed result |
+|---|---|
+| Correct the project-view decision | New wording saved at revision 26; earlier revision 22 and its source remained in history |
+| Remove that decision | Revision 27 hid it; a separate agent-resume read confirmed it was absent |
+| Restore through the Removed page | Revision 28 returned the corrected wording to agent context |
+| Save while another session changed the same decision | Concurrent revision 30 blocked the old draft; the browser retained the draft and showed the latest saved version |
+| Compare and save the correction | Revision 31 preserved the reviewed wording and rationale; keyboard focus returned to the card's Correct button |
+| Search the corrected wording | Found the active decision, without unrelated cards |
+| Narrow-screen layout at 390 px | Navigation scrolled within its own row; page content stayed within the viewport |
+| Browser error log after these interactions | No logged browser errors |
+
+Browser records are the same records returned by the project reader, not a second
+memory store. The UI uses text rendering for stored content and contains no
+command execution or settings control.
+
+## Request-security and regression evidence
+
+All **14 local-view test groups passed** in `test-work-view.mjs`. These cover
+loopback binding, per-process authentication, Host rebinding, cross-origin and
+preflight rejection, write method/origin/content type, current branch/revision,
+correction history, stale writes, removal/restore, removed-item stale edits,
+secret and scope rejection, body limits, fixed asset routes, protected receipts,
+text-rendering sinks, and capability invalidation after restarting the view.
+
+The first test run failed because the test's fetch client normalized the hostile
+Host header. Switching that probe to a wire-level HTTP request exercised the
+intended attack and confirmed rejection. That failed receipt remains in history.
+A further review found that an old browser draft could otherwise revive a removed
+item; browser writes now reject it and require explicit restoration.
+
+**23 full suites passed, with zero failures or skips.** This includes the
+existing 17-group handoff integration suite and 19-scenario adversarial suite.
+Of those 19 scenarios, one intentionally demonstrates that a local writer can
+forge a structurally valid receipt; it is a documented limitation, not a defense.
+
+Relevant files changed in the view implementation, so the integration and
+adversarial receipts needed targeted rechecks. Current saved passes are view
+revision 29, integration revision 32 and adversarial revision 33. Ordinary full
+suite output is retained only as local test evidence; it is not represented as
+an automatically captured Memoir receipt.
+
+## Fresh-session continuity
+
+The first fresh Codex CLI session read handoff revision 35. It retrieved delivery
+answer revision 2, privacy answer revision 3, corrected project-view decision
+revision 31, completed build revision 34 and the open validation action revision
+35. It identified the three reusable checks above and named all four changed
+files invalidating documentation receipt 19. It ran no checks and asked no
+questions.
+
+The configured MCP call was refused by Codex's existing never-approval policy.
+Because this probe explicitly prohibited file edits, it used the existing
+read-only project reader instead of the normal CLI resume, which refreshes the
+Markdown preview. No approval policy or app configuration changed. The previous
+Codex/Cursor round trip is documented separately in
+[LOCAL-HANDOFF-VALIDATION.md](LOCAL-HANDOFF-VALIDATION.md); this view exercise does
+not claim a new Cursor agent run.
+
+The second fresh Codex session read revision 38 through the documented CLI
+fallback after the same MCP policy refusal. It recovered the same original
+answers and browser correction, the clarified completed build at revision 37,
+and the new validation decision at revision 38. It correctly recognized all
+four receipts as reusable, including the refreshed documentation receipt 36.
+It performed no check execution and made no project-record changes. The normal
+resume refreshed the generated Markdown preview.
+
+| Observation across the two fresh sessions | Result |
+|---|---|
+| Original answers retained | Both retained revisions 2 and 3 |
+| Corrected project-view decision retained | Both retrieved revision 31 |
+| Completed build recognized | Both reported done; second saw the clarified revision 37 |
+| Expected view/validation decisions missed | None among the explicitly requested records |
+| Questions repeated | 0 |
+| Check executions in the probes | 0 |
+| Stale documentation evidence handled | First named changed files; second reused the targeted replacement |
+
+Both prompts requested a context report and prohibited check execution. These
+counts therefore measure controlled retrieval and instruction following, not
+spontaneous agent behavior or time saved against a baseline. No multi-day usage
+study has been completed. Local event logs/results are in `.memoir/` and remain
+ignored; no transcript was added to the project ledger.
+
+The installed-tarball smoke test passed for version 3.13.3, including the view
+command's help, all three shipped browser assets, authenticated project data and
+unauthenticated denial. The archive was installed only in an isolated temporary
+directory and was not published. Source and test fingerprints did not change
+between the two probes; the documentation receipt was the targeted update.
+
+## Limits and the next useful trial
+
+- A person or agent still starts the view and keeps its process alive. It is an
+  opt-in local HTTP listener on 127.0.0.1, not a background service.
+- Agents must read and save the handoff. It cannot force an agent to respect a
+  decision, prevent every repeated question, or capture unsaved conversation.
+- GitHub does not sync the ignored ledger. Use the same folder and branch on this
+  computer; another checkout needs an explicit future transfer mechanism.
+- The view checks current file fingerprints when refreshed. External systems
+  such as Stripe settings need fresh evidence when relevant conditions change.
+  A documentation-only edit does not invalidate an unrelated code receipt.
+- Keep its temporary access link private. Static pages contain no project data;
+  data and writes require the token. Other same-user processes and an attacker
+  with filesystem access are outside this boundary.
+- Project records remain plaintext, secret detection is heuristic, source labels
+  and check receipts are unauthenticated, and semantic prompt injection remains
+  possible. See [the focused audit](HANDOFF-SECURITY-AUDIT.md).
+
+Use the next three ordinary development tasks as the practical trial. Count
+repeated already-answered questions, identical unnecessary check executions,
+missed saved decisions, and corrections needed. Record only concise project
+observations, not conversation transcripts or credentials. Do not treat a rerun
+after changed relevant files or external conditions as wasted work. This will
+show whether the controlled result holds during real use before expanding to
+more tools or making product performance claims.

package/docs/RELEASE-3.14-VALIDATION.md

@@ -0,0 +1,36 @@
+# 3.14.0 deployment validation
+
+Observed September 6, 2026. Package publication and database deployment are separate operations: check the [publish workflow](https://github.com/camgitt/memoir/actions/workflows/publish.yml) and [npm package](https://www.npmjs.com/package/memoir-cli) for package availability.
+
+## Hosted database
+
+Applied `supabase/migrations/202609050001_backup_versions.sql` to the existing Memoir service through its authenticated SQL editor. Before deployment, the service was healthy and reported a scheduled backup four hours earlier. The metadata inventory contained no duplicate account/version pairs. Existing backup IDs matched after the complete test and cleanup.
+
+Verified the unique version index, counter row-level security, authenticated allocator access, and anonymous denial. Existing metadata policies match `auth.uid()` to `user_id`; storage policies restrict the backup bucket to the authenticated user's folder. No existing policies or customer backups were replaced.
+
+The live test used two temporary accounts created through the admin API without sending email, and synthetic files. Real user tokens exercised:
+
+- Twelve simultaneous allocations with distinct positive versions, plus independent numbering for a second account.
+- Anonymous allocator rejection and denial of direct access to the counter table.
+- Concurrent encrypted uploads, byte-matched restore, and wrong-passphrase rejection.
+- Cross-account metadata read, update, delete and insert denial; storage read, write and delete denial.
+- A locally injected metadata-response failure after a real object upload, followed by a successful new upload and restore. This checks client recovery; it does not simulate a committed-but-lost backend response.
+- Legacy account-key encryption migration, including replacement download and byte comparison before deleting the synthetic original.
+
+The first live run exposed an incorrect Storage deletion endpoint that permissive mocks had accepted. `deleteBackup` now uses the bucket endpoint with a JSON `prefixes` array, matching [Supabase's client contract](https://github.com/supabase/storage-js/blob/master/src/packages/StorageFileApi.ts). Retention and migration regression tests enforce that request shape. The corrected live run passed every check above. Temporary accounts, metadata and stored objects were removed.
+
+An immediate repeat download could return a cached original after deletion. Storage listing, metadata lookup and a fresh uncached download confirmed removal. Provider caches and snapshots may outlive client deletion; this release does not promise immediate provider-wide erasure.
+
+## Local workflow and release checks
+
+The project handoff, recovery interface and adversarial tests are included in the 24-suite runner. Actual Codex → Cursor → Codex observations are in [local handoff validation](LOCAL-HANDOFF-VALIDATION.md). The [browser recovery report](PROJECT-VIEW-DEBUG.md) explains the bugs fixed before release. Local regression checks after the hosted deletion fix passed all 59 cloud assertions and 26 audit groups. GitHub CI runs the full suite and evaluations on Node 18, 20 and 22 across Linux, macOS and Windows; its Ubuntu/Node 22 job also checks the installed package.
+
+Project ledgers, personal memory, local client configuration, deployment credentials and raw deployment logs are excluded from GitHub and the package. The release adds an explicit dependency on the already installed Zod version used by the handoff server.
+
+External settings must be checked again when changed. Passing source tests does not prove npm publishing permission, client MCP approval or hosted policies remain unchanged. Receipt input hashes also become stale when the package manifests change for a release.
+
+CI also exposed an intermittent Git error-classification bug: `403` inside a generated path or timestamp was mistaken for an HTTP authorization failure. The classifier now requires an HTTP diagnostic, with regressions covering both the false match and a real HTTP 403.
+
+## Publication
+
+The existing GitHub publisher was failing authentication before this release. npm requires security-key verification to inspect and repair the trusted-publisher settings. Source and server readiness alone must not be described as a published npm version. The completed workflow and registry version are the publication evidence.

package/docs/RELIABILITY-ROLLOUT.md

@@ -0,0 +1,57 @@
+# Reliability rollout and recovery
+
+Upgrade guidance for version 3.14.0. The Memoir hosted database migration and account-isolation checks completed on September 6, 2026; see [deployment evidence](RELEASE-3.14-VALIDATION.md). Package availability is established by the npm registry and release workflow, separately from database deployment. Preserve a protected, recoverable copy of existing backups before upgrading.
+
+## Compatibility
+
+| Data | New reader | New writer |
+|---|---|---|
+| Legacy local/Git encrypted manifest | Yes | No |
+| Path-bound manifest version 2 | Yes | Yes |
+| Legacy account-ID-keyed/gzip cloud | Yes, with warning | No |
+| User-passphrase cloud MEMOIRC2 | Yes | Hosted migration deployed; self-hosted services must migrate |
+| Supported old session schema | Migrated; future imports rejected | Archives/metadata retained |
+| Legacy workspace tar | Manual inspection only | No |
+| Workspace file manifest version 2 | Separate recovery folder | Explicit --workspace |
+
+Upgrade every syncing client before relying on archives, scopes, or tombstones. Old clients lack these semantics and cannot read new encrypted formats. Downgrades need a preserved legacy snapshot or export. Do not point an old writer at the new backup. Migration cannot reconstruct files already overwritten by old collisions.
+
+## Cloud database change
+
+Review and deploy `supabase/migrations/202609050001_backup_versions.sql` normally. It adds encryption/source metadata, a per-user unique version constraint, and an authenticated atomic allocator.
+
+1. Back up metadata and inventory duplicate `(user_id, version)` pairs. The unique index deliberately fails on duplicates; resolve them with a reviewed mapping. The migration never deletes or silently renumbers backups.
+2. Confirm production row/storage policies restrict all operations to the authenticated account. This source patch does not establish that.
+3. Test simultaneous authenticated allocations: distinct positive versions, anonymous rejection, and no access to another account's state.
+4. Exercise upload, download, wrong-key failure, metadata failure, retry, and migration against a disposable hosted account.
+
+The private counter's security-definer function uses `auth.uid()`, an empty search path, and no caller-supplied user ID. Failed uploads can leave version gaps. Allocation is not a distributed transaction across objects, metadata, and retention.
+
+Stop new writes before backend rollback; preserve columns while new metadata exists. Do not delete an object after an ambiguous metadata response: the write may have committed.
+
+## Cloud key migration
+
+Supply a strong secret through the environment without putting it in project files or shell history. Run `memoir cloud migrate` to review the inventory, then `--apply` after confirming another device can recover the secret.
+
+Each replacement must download and compare byte-for-byte before its old object is deleted. Verification failure retains the original; fix access and rerun. Independently restore on another device afterward. Provider snapshots/logs may retain copies beyond client deletion authority.
+
+Random vault-key wrapping, enrollment, cross-device rotation, hardware-backed storage, and independent protocol review are not implemented by this change.
+
+## Release gates
+
+- Full tests/audit regressions and installed-tarball save/read/resume/encrypted recovery pass.
+- CI confirms the declared OS/Node matrix; a local macOS run cannot establish Windows readiness.
+- Real Claude Code, Codex, and Cursor installations accept configuration and complete save/recall/restart.
+- Hosted migration and account-isolation tests pass.
+- The package owner configures npm's trusted publisher for `camgitt/memoir`, workflow `publish.yml`. The old publication auth failure is not fixed by this source branch.
+- Select and synchronize the release version across package.json, lockfile, and server.json. Publish only the reviewed commit/tag, then repeat the smoke check against the registry artifact.
+
+The release gates above also apply to future deployments. The linked validation report records what was actually observed for this release. No independent security certification is implied.
+
+## Operational limits
+
+Local locks fail when busy. A crash while reaping can leave a `.reaper` file requiring inspection after confirming no live writer. Git conflicts require retry. Native adapter restore still uses file modification times and is not a multi-directory transaction.
+
+Cloud merges retained states, increasing download cost. It cannot recover versions already removed by retention. Strong concurrent-cloud consistency needs a server-side conditional commit/merge protocol and race-tested retention.
+
+Legacy scope migration, renamed projects, unusual remote aliases, and profile isolation remain work. Concurrent hostile parent-directory replacement requires further filesystem review: current validation/no-follow opens are not an openat-based sandbox.

package/docs/RETRIEVAL-INDEX.md

@@ -0,0 +1,45 @@
+# Retrieval index and verification
+
+Memoir uses a process-local inverted index to find lexical candidates and reuse field-token matches. Markdown and session records remain the source of truth. The index is rebuilt from those sources when the process starts or the search cache is cleared; it is not another copy to back up or synchronize.
+
+See [matched measurements and the memory tradeoff](RETRIEVAL-RESULTS.md) for the checked implementation.
+
+The index preserves the existing field weights, conservative English normalization, bidirectional prefix matching, Unicode/CJK tokens, coverage multiplier, scoped IDF, passage extraction, and result budget. It adds no embedding model, remote call, native binary, or runtime dependency. This is a performance and consistency change, not a new semantic-retrieval algorithm or evidence of better task completion.
+
+## Read path
+
+1. Inventory the registered adapters and canonical current records. Inspect file metadata on each query. Canonical revision history is not searched.
+2. Reuse a parsed document only if its modification time, change time, inode, device, and size still match. Cache identity includes the exposed path and adapter, so two adapters cannot borrow each other's labels.
+3. Reevaluate project and lifecycle visibility. Resolve repeated project identities once per query. Hidden, expired, superseded, and unrelated records do not enter the query's postings or IDF population.
+4. Remove absent documents and add changed documents to the postings. Resolve exact/prefix token matches once against the sorted vocabulary, then retain the existing ranking formula.
+5. Validate each returned source's full path and metadata again before releasing a cached passage. If the source disappeared, became a symlink, or changed during the query, skip it; a later query refreshes the content.
+
+Filesystem inventory shares parent-directory checks only within one refresh. Actual content reads still use the validated, no-follow file reader. The parent-check cache is never retained between queries and is never used for writes. This does not establish an atomic filesystem snapshot or eliminate the previously documented hostile parent-directory replacement races.
+
+Current project instructions are discovered from the requested active checkout in normal recall, including checkouts deeper than three directories below the home directory. Explicit discovery-root and all-project diagnostic calls retain their broader discovery scope. Traversal remains limited to three levels beneath the discovery root with existing directory exclusions. Directory entries are cached by metadata, so new instruction files are discovered on the next query without the former 60-second delay.
+
+## Consistency and limits
+
+Save, edit, hide, purge, restore, and external file changes are reflected by the next inventory refresh. Stale backups still go through the canonical store's existing purge/merge rules. A refresh removes missing/unreadable parse entries and evicts outdated postings; clearing the cache cannot resurrect a record that the current sources exclude.
+
+The index is held in memory, so restart and first-query construction are more expensive than warm queries. Every query still inventories source files; latency is not independent of corpus size. Filesystem metadata resolution, network filesystems, corpus size, record length, common versus rare terms, and machine load affect performance. No watcher event is treated as proof that nothing changed. Process memory grows with the parsed corpus and token postings; this implementation is not a disk-backed database for arbitrarily large archives.
+
+The indexed engine and exhaustive reference share the same reader, scope rules, and ranking formula. Their agreement checks index correctness; they are not two independent memory products. Canonical documents now use their source file modification time for tie-breaking, consistently with adapter documents. A tied ranking can therefore differ from the prior branch, which rebuilt canonical documents with a zero modification time.
+
+## Reproduce checks and measurements
+
+`npm test` includes the retrieval-index integration suite. It checks scoring equivalence, same-size external edits with restored modification time, project switching, validity windows, purge plus stale restore, additions/deletions/renames, adapter identity, new project instructions, symlinks, source disappearance during a query, budgets, and source-line parity.
+
+`npm run eval` retains the small development retrieval fixture and adds an exhaustive scoped reference. That fixture is not held out. `searchMemories(query, { engine: 'scan' })` is available internally for diagnostic comparison; MCP continues to use the indexed default.
+
+`npm run bench:retrieval -- --output /absolute/path/report.json` runs synthetic adapter and canonical corpora at 1,000 and 10,000 records. Each corpus has one cold query and 120 warm queries in a fixed mixed cycle. The output includes fixture/harness/source hashes, machine/runtime information, all query timings, median/p95, and process RSS. The fixture has multilingual text and both common- and rare-term queries. It checks a required result and an empty-result case but does not score completed coding tasks.
+
+To compare with an earlier checkout using the same harness and fixtures:
+
+```sh
+node evals/retrieval-performance.mjs --repo /absolute/path/earlier-checkout --kinds adapter --sizes 1000,10000 --samples 120 --output /absolute/path/before.json
+```
+
+That checkout must have its own compatible dependencies available. Run competing measurements sequentially on the same machine. Do not compare a warm API call with a cold process startup or combine different sample counts without labeling them. RSS is process-wide and includes allocations from preceding corpus runs; it is not isolated index memory.
+
+See [the continuity evaluation protocol](../evals/CONTINUITY-PROTOCOL.md) for the separate work required to establish product utility.

package/docs/RETRIEVAL-RESULTS.md

@@ -0,0 +1,26 @@
+# Matched retrieval measurements
+
+Measured 5 September 2026 on macOS arm64, Node 26.7.0, Apple M4 Pro. The earlier audit remediation candidate (`bf30217`) is the baseline; this is not a competitor or published-release comparison. The measured source hashes match implementation commit `a5af53e11e4beb9022e065f7a219ccf309df4d7d`.
+
+Each corpus contains roughly 1 KB of multilingual coding notes per record. Runs used the same harness and normalized fixture hashes, one cold query and 120 warm queries per corpus, with a fixed cycle of common terms, rare terms, prefixes, and a no-match query. Runs were sequential. Timings include the public in-process search API, filesystem inventory/validation, ranking, and passage generation.
+
+| Source | Records | Before median ms | Indexed median ms | Before p95 ms | Indexed p95 ms | Median speedup |
+|---|---:|---:|---:|---:|---:|---:|
+| adapter | 1,000 | 53.6 | 8.4 | 57.9 | 10.0 | 6.4× |
+| adapter | 10,000 | 516.8 | 77.3 | 545.6 | 109.7 | 6.7× |
+| canonical | 1,000 | 359.2 | 9.9 | 368.0 | 11.2 | 36.1× |
+| canonical | 10,000 | — | 92.3 | — | 124.8 | — |
+
+The 10,000-record indexed warm p95 measurements are below the proposed 250 ms budget on this fixture and machine. They are not a universal latency guarantee. Cold indexed queries still took about 1.34 seconds for adapter records and 1.52 seconds for canonical records. There is no matched 10,000-canonical-record baseline in this run.
+
+## Memory tradeoff
+
+A separate fresh-process probe at 10,000 adapter records, after 24 warm queries and an explicit garbage collection, retained 95.5 MiB of JavaScript heap before and 163.8 MiB with the index (about 68.3 MiB extra). Process RSS was 311.2 versus 386.2 MiB. These include the parsed corpus and other process allocations, not only the postings. The longer multi-corpus run had higher transient RSS; its per-corpus snapshots are in the raw evidence.
+
+The speed gain therefore has a memory cost. Large-archive support still needs a memory budget and potentially a disk-backed index. This change does not add semantic search, a native database dependency, or a claim of improved coding outcomes.
+
+## Validation
+
+Twenty local suites passed, including the 26 audit regression groups and 12 retrieval-index groups. The installed tarball passed 11 workflow checks. All 16 development cases agreed with the exhaustive scoped reference, and the dependency audit reported no advisories. Hosted CI is checked separately on the final PR head.
+
+See [raw measurements](../evals/results/retrieval-2026-09-05.json), [implementation and reproduction instructions](RETRIEVAL-INDEX.md), and the [future coding-continuity protocol](../evals/CONTINUITY-PROTOCOL.md).