draken 0.1.6 → 0.3.0

package/ACCEPTANCE.md

@@ -0,0 +1,34 @@
+# Before approving or releasing this rewrite
+
+This is a read-only engineering-sync application, not the old Docker/task dashboard. A draft PR, passing CI, configuration check, or quick code skim is not a release qualification. Public npm/website content changes only through separately authorized publication/deployment.
+
+## Automated gates
+
+`Application qualification` runs strict typechecks/unused-code analysis, production builds, backend and browser-controller tests, Chromium UI regressions, root/client/private-pi-lock audits, a fresh production-only npm package/lifecycle smoke, and private runtime installation/protocol checks. `Tray platform checks` runs controller/executable tests, native helper startup/update/shutdown, and private pi/Codex installation on hosted macOS and Windows.
+
+Runtime qualification uses a private HOME and synthetic loopback responses: it does **not** use real provider accounts, sign in, refresh credentials, acquire a microphone, or claim physical audio/menu usability. Package installation downloads npm dependencies. UI screenshots on failure contain synthetic fixtures only. CI dependency audits are time-specific advisory checks, not exploitability analysis or a security certification.
+
+The release workflow requires a strict stable version already committed consistently in the manifest/lock, source on `master`, and both qualification workflows. Manual versions are passed as data, not interpolated into shell programs. Release uses pinned Node/npm versions and trusted-publishing provenance. No release is triggered by an ordinary PR push. Workflow checks are not a claim that repository branch protection has been configured.
+
+## Short product acceptance checklist
+
+Use disposable histories first. Real-provider checks below require the operator's explicit sign-in and consent.
+
+- [ ] **Useful, qualified briefing:** distinguish what was proposed, attempted, tested, deployed, and still unresolved. Inspect original evidence and a late correction; verify the summary preserves chronology and qualifications rather than sounding more certain.
+- [ ] **Explicit permissions:** Codex/Claude imports stay off until independently enabled. Opening Settings/setup/status does not scan transcript bodies, prepare a snapshot, download an embedding model, call a provider, or request a microphone. Background indexing permission is separate from source permission.
+- [ ] **Small tails and restart:** fewer than 64 distinct changed source records wait indefinitely in the background. Session bookkeeping and frozen aliases do not multiply the count. Foreground preparation handles smaller tails. Restart does not make an unchanged archive new again.
+- [ ] **Evidence availability:** removing/changing a frozen original fails visibly; no cached-source fallback invents availability. Notes/discussion remain inspectable. Restore the exact original and verify recovery without regenerated evidence or review.
+- [ ] **Import completeness:** a source larger than 32 MiB reaches its captured tail before a new snapshot or call readiness. Cancellation retains committed chunks without marking a partial import complete; retry resumes. An unfinished final record is deferred; an oversized or unreadable record blocks preparation. Later appends do not cause an endless chase.
+- [ ] **Useful retrieval:** replay project-status and follow-up questions with older identical user questions and acknowledgements present. Confirm original project evidence, latest recorded work, technical definitions and concrete remaining work—not just source IDs. Include duplicate passages, ambiguous topics, project switches, explicit conversation recall, source readback and per-turn source lists. Passing transport tests alone is insufficient.
+- [ ] **Call readiness:** a complete briefing with unfinished history keeps Call disabled. Prepare or cancel history on the sync page, with no call reservation or microphone. A 64-record background batch can establish readiness; a smaller pending tail cannot silently flush inside call setup. Retry/restart reuses completed work. Changed/missing sources invalidate readiness, and original bytes still require verification.
+- [ ] **Call lifecycle:** cancel history preparation before a call; separately cancel connection before the create response arrives. Reload and start again. Microphone acquisition stays after preparation and explicit intent. End/Stop/mute/hidden-page behavior is understandable; no call restarts itself.
+- [ ] **Review is yours:** included/retrieved/summarized/generated/heard/agreed/reviewed are not interchangeable. Only explicit review advances coverage. Discard, failure, cancellation and app restart do not mark work reviewed.
+- [ ] **Real account and audio, separately:** configuration metadata alone does not prove provider/model access, token refresh, quota, billing or audible playback. If desired, test each explicitly with the intended account/device/browser; do not infer them from CI.
+- [ ] **Physical desktop tray:** on actual macOS and Windows desktops, check icon visibility, all menu actions, browser opening, close-terminal survival, external-server observation, owned Stop/Quit and helper loss. No saved-PID adoption, external-server termination, silent settings changes or second same-data listener. Hosted helper smoke is not this test.
+- [ ] **Overall value:** does the briefing and discussion help you resume engineering work better than reading the original sessions directly?
+
+## Focused independent review
+
+Review authentication/origin boundaries, original-pointer/storage migrations, and child-process ownership/cancellation separately. Existing synthetic and private migration proofs are scoped evidence, not proof of every source format or deployment. Back up original JSONL histories **and** Draken private state before a real migration. Preserve exact compatible vectors, identities, references, credentials, TLS configuration and original launch environment; never deploy proof databases.
+
+Keep the PR draft until the remaining risks are accepted. Version bump, merge, npm publication, website update and live deployment are distinct decisions. Older selected pi runtimes are not silently upgraded; the operator must explicitly rerun setup or update their external pi installation.

package/CHANGELOG.md

@@ -0,0 +1,65 @@
+# Changelog
+
+## 0.3.0
+
+This release adds a separate text-chat experience and improves how Draken prepares history and checks later updates.
+
+### Chat without a call
+
+- Choose **Chat instead** on Home or an existing sync. Chat uses configured pi, without native voice setup, microphone permission or WebRTC.
+- Keep saved messages, original-source links, a square Draken portrait and real searching/thinking status across reloads.
+- Stop a reply or explicitly retry a failed one. Accepted questions remain saved even if generation fails; repeated delivery of the same request does not generate another answer.
+- Sending a message never implicitly saves edited notes, marks work reviewed or starts a voice call. The model/quota notice was removed from the chat view.
+
+### More reliable history preparation and retrieval
+
+- Finish bounded import passes to the captured end of each log before freezing a snapshot. An indexed but incompletely imported prefix no longer qualifies as ready.
+- Prepare history before enabling a call, with explicit progress, cancellation and server-side readiness checks. Live retrieval does not silently download the local model or rebuild corpus embeddings.
+- Keep project scope across follow-up questions using actual user turns and admitted metadata. Prefer original project records over stale Draken replies, with better deduplication and explanatory excerpts.
+- Show the prepared-history cutoff and make refreshing history explicit. New chat messages do not force a corpus rebuild after every turn; changed or missing captured originals still invalidate readiness.
+
+### Reconcile later evidence in text status answers
+
+- Check later related reports within the same original session, including follow-ups that themselves became stale. Keep client implementation, backend foundations, delivery enablement and device/provider acceptance distinct.
+- Supply a larger bounded text-history packet and retain closing qualifications when excerpts are shortened.
+- If a draft cites a linked old checkpoint while ignoring its supplied follow-ups, permit one correction through the same final model and invocation budget. A second failure is visible; rejected drafts are not saved.
+- The correction can consume one additional final-model request. These checks improve retrieval and citation consistency; they do not independently verify every claim or exhaustively reconcile all sessions. Native voice retains its separate smaller lookup path.
+
+### Upgrading from 0.2.0
+
+- Node.js 22.19+ and npm remain required. Back up original agent logs and Draken data first; wait for active calls and preparation/indexing jobs to finish, then stop the server before upgrading and restart it afterward.
+- This release adds import-completeness metadata and nullable chat request IDs with a uniqueness constraint. Existing conversations, notes, review state and compatible embedding vectors are retained. Do not restore an older backup over newer user activity or use an older binary to undo migrations.
+- No provider/account replacement, automatic paid summarization or new background-indexing policy is introduced. Chat and its correction attempts remain outside the preparation-only cost estimate.
+- Automated tests use synthetic provider replies. Real local-embedding checks do not establish real-account quota, model-answer correctness, physical-device audio or end-to-end notification delivery in your projects.
+
+## 0.2.0
+
+Draken now helps you catch up on what your coding agents did. It reads saved sessions, prepares summaries with links to the original messages, and lets you ask questions and save decisions. It does not run project commands or change your code.
+
+### What changed
+
+- Read pi sessions, with separate opt-in imports for supported Codex CLI and Claude Code logs.
+- Prepare summaries of the selected history, including large histories, and reuse completed work when possible.
+- Search older sessions locally, ask questions in text, and keep notes about what to do next.
+- Follow source links back to the original logs. Only explicitly marking a sync reviewed updates review progress.
+- Use optional ChatGPT live voice with its own sign-in and an animated dragon. The microphone stays off until preparation is done and you choose to call.
+- Install a private pi runtime with `draken setup`; pi handles provider sign-in and refresh.
+- Find a running server with `draken status`. Try the optional macOS/Windows tray icon with `draken tray`.
+
+### Before upgrading from 0.1.6 or earlier
+
+This replaces the Docker/Claude task dashboard. The old task execution, terminal, scheduling and Git-control features and APIs are removed. This is not a compatible upgrade for tools built around those APIs.
+
+- **Node.js 22.19+ and npm are required.** Docker is not needed.
+- Back up your original agent logs and private Draken data before upgrading. Original logs are needed to read source links, including those in older summaries.
+- Applicable legacy tables and historical call records are retained, but old task conversations are not automatically imported.
+- Do not use an old Draken binary to undo database changes.
+- An update does not replace an already selected pi runtime. Run `draken setup` explicitly for the shipped private runtime, or manage your external pi installation yourself.
+
+### Limits to know
+
+Summaries can miss details, and search can miss relevant messages. Source links are not independent proof that a claim is correct. Reading or listening does not mark work reviewed.
+
+Reading and search run locally. Summaries and text questions send content to your selected model provider through pi. Voice sends audio and context to OpenAI, using a separate ChatGPT sign-in.
+
+The desktop tray is a preview. Hosted tests check startup and shutdown, not physical menu usability. Automated tests also do not establish real-account access, quota, billing, microphone quality or speaker playback. See [ACCEPTANCE.md](ACCEPTANCE.md) for those checks, and [SETUP.md](SETUP.md) to get started.

package/CONTRIBUTING.md

@@ -0,0 +1,129 @@
+# Contributing to Draken
+
+For normal installation, use the npm instructions in [SETUP.md](SETUP.md). This guide is for people changing, building or qualifying Draken itself. Node.js **22.19+** and npm are required.
+
+## Build and run a checkout
+
+```sh
+git clone https://github.com/mehanig/draken.git
+cd draken
+npm ci
+npm --prefix client ci
+npm run build
+node dist/cli.js
+```
+
+Use `node dist/cli.js` in place of `draken` when following the user guides from a checkout—for example, `node dist/cli.js status` or `node dist/cli.js tray`. No global npm installation is needed for development.
+
+The foreground server defaults to `http://127.0.0.1:40333`; Ctrl+C stops it. Use a disposable `DRAKEN_DATA_DIR` and synthetic source roots when testing changes. A manually launched development server otherwise uses the normal private state/history locations. Do not build over or replace files used by a live deployment; use a separate checkout for staged work.
+
+For frontend development, run the backend and `npm run dev:client` in separate terminals. Vite proxies `/api` to port 40333. Server changes need a rebuild/restart; `npm run dev` runs the TypeScript watcher.
+
+## Ordinary checks
+
+```sh
+npm run check                  # Root/client typechecks and Knip
+npm test                       # Build server, then backend tests
+npm run build                  # Server and frontend production builds
+npx --no-install playwright install chromium
+node --test tests/live-browser.cjs tests/realtime-browser.cjs
+npx --no-install playwright test
+node --test tests/site-browser.cjs
+```
+
+On a Linux CI image, Playwright may also need `--with-deps`. `npm run test:ui` combines the full build and application UI suite.
+
+These tests use temporary synthetic records, fake providers and browser peers. They do not read your real source histories or call a real model provider. Website checks serve only the public static files and verify mobile layout, evidence-reference interaction and the absence of microphone/external-request activity.
+
+### Fresh package and runtime qualification
+
+After a full build:
+
+```sh
+node tests/package-smoke.cjs
+node tests/runtime-smoke.cjs
+npm audit
+npm --prefix client audit
+npm --prefix runtime/pi audit --omit=dev
+```
+
+These smoke checks download npm dependencies into isolated temporary homes. The runtime check installs the pinned pi/Codex packages, checks missing-auth metadata, verifies pi text/usage behavior against synthetic loopback responses, and exercises native RPC denial/text boundaries and confirmed helper exit. It does not sign in, refresh real credentials, call real providers or acquire a microphone. Root/client audits do not cover the separately shipped pi lock; audit it explicitly.
+
+## Voice qualification
+
+Ordinary voice tests cover native-only UI/control behavior, source lookup, interruption, no-capture-on-reload, absent retired endpoints and readable legacy records. Synthetic audio is not a human recording and is not proof of physical microphone, speaker, Bluetooth or mobile performance.
+
+The following optional commands require their stated setup and, for real provider use, explicit consent:
+
+```sh
+# Requires the pinned native helper; synthetic auth and loopback backends only:
+npm run test:realtime:native
+
+# Requires separate native ChatGPT sign-in; real provider, synthetic data/audio:
+npm run voice:realtime:probe -- --allow-provider
+DRAKEN_TEST_STATIC_DIR=/path/to/staged/frontend npm run test:live:real -- --allow-provider
+
+# Real voice with a controlled delayed lookup:
+DRAKEN_TEST_STATIC_DIR=/path/to/staged/frontend node tests/voice-continuation-real.cjs --allow-provider
+```
+
+The environment-prefix examples above use Bash syntax. Real-provider probes are not included in ordinary tests or CI. Never add automatic credential copying, provider fallback, execution tools or approval grants to make a probe pass.
+
+## Local-model qualification
+
+After building, opt into downloading the pinned embedding weights, or supply an existing private test copy:
+
+```sh
+npm run test:memory:local -- --download-model
+npm run test:memory:local -- --model-dir=/private/path/to/model
+node tests/memory-performance.cjs --model-dir=/private/path/to/model
+```
+
+The local-model check uses temporary synthetic records, real offline embeddings and the actual worker. It covers paraphrase/multilingual retrieval, abstention, native context, H citations, cancellation/resume and unchanged evidence/coverage. It does not open the production database or call OpenAI. The download flag controls this test harness, not ordinary product search.
+
+The performance check compares the prior single-session runtime with the bounded pool on 48 synthetic passages plus long-window/query-prefix cases. It reports wall time, RSS, cosine similarity and byte equality. It does not download weights, inspect production history or guarantee whole-sync speed or retrieval quality.
+
+## Desktop tray qualification
+
+On an interactive macOS/Windows desktop, after building:
+
+```sh
+node --test tests/tray.test.cjs tests/native-launch.test.cjs
+npm run test:tray:native
+```
+
+The native smoke check opens a temporary icon, updates status and checks orderly exit. It does not start a Draken server, open a browser, scan history or access a provider. Hosted success does not verify physical menu clicks, desktop policies, logout behavior or every architecture. Complete the manual checks in [TRAY.md](TRAY.md) and [ACCEPTANCE.md](ACCEPTANCE.md).
+
+## Public website
+
+`site/` is plain HTML/CSS with a small, synthetic evidence-reference interaction. It has no account/API integration, remote fonts, analytics or microphone access. Keep demo content labelled and never use private project screenshots or histories.
+
+Merging `site/**` changes into `master` triggers `.github/workflows/web-deploy.yml`, which deploys to the existing Cloudflare R2 site and purges its cache. A feature-branch push does not deploy it. npm publication and application deployment are separate operations. Keep version notices and installation commands accurate. Before announcing a release on the site, verify that the npm registry serves the expected version and test a fresh production-only installation. A successful publish command can precede registry availability while npm processes the package. Update the site and user documentation to match the confirmed release.
+
+## Keep the code lean and the boundaries explicit
+
+- Server and React UI code are TypeScript; native helpers/installers and some tests are JavaScript. The old Python speech worker is removed.
+- Both TypeScript configurations reject unused locals/parameters. Pinned Knip models both packages, spawned fixtures, HTTP-served modules and generated-to-source mappings.
+- Prefer individually inspectable client API exports. Preserve real test seams even where CommonJS namespace imports prevent exact unused-export attribution.
+- Static reachability is not runtime proof. Trace HTTP routes, dynamic imports, generated assets, declarations and failure/cancellation/security paths. Do not remove safeguards to quiet an analyzer or test.
+- Source-only pointers, exact compatible vectors, explicit source permissions, independent voice sign-in and review-only coverage are product boundaries—not optional implementation details.
+- Keep credentials, private runtime data, databases, original histories and diagnostic artifacts out of git/npm. Never deploy a proof database.
+
+## Code map
+
+| Area | Location |
+| --- | --- |
+| Runtime selection, private setup and pi-owned auth | `src/pi/`, `src/npm-cli.ts` |
+| Agent registry and supported log readers | `src/sources/`, `src/sync/collector.ts` |
+| Source-only evidence and migrations | `src/db/`, `src/sync/source-files.ts` |
+| Frozen snapshots, notes and review | `src/sync/store.ts` |
+| Summary trees, references and accounting | `src/sync/{tree,pi,pi-output,cost,references}.ts` |
+| Recaps and conversation orchestration | `src/sync/{recap,manager}.ts` |
+| Retrieval, embedding pool and batching/checkpoints | `src/memory/` |
+| Authenticated application APIs | `src/routes/` |
+| Native voice ownership and browser controls | `src/voice/`, `voice/realtime/`, `client/src/voice/` |
+| Home, snapshots and history UI | `client/src/pages/`, `client/src/components/` |
+| Tray ownership and OS helpers | `src/tray/`, `tray/` |
+| Landing page | `site/` |
+
+See [ARCHITECTURE.md](ARCHITECTURE.md) for design details and [ACCEPTANCE.md](ACCEPTANCE.md) for review/release gates. Passing CI is not permission to merge, publish, deploy, change real credentials or make provider calls.

package/MEMORY.md

@@ -0,0 +1,118 @@
+# Historical search
+
+**Just ask.** Draken automatically searches retained older sessions, sync conversations and saved notes when you ask a text question or talk to it. The separate **Search history** page (`/history`) lets you inspect the same records directly.
+
+There is no memory-enable switch, search-mode selector or wider-history checkbox. An existing sync offers **Prepare history** when its call history is not ready. Historical retrieval does not replace the complete-snapshot summary tree or advance review coverage.
+
+## Automatic preparation
+
+The first request with eligible retained history downloads approximately **136 MB** of pinned model/tokenizer files from Hugging Face, verifies their SHA-256 checksums and prepares local embeddings. Later requests reuse them and automatically embed new or changed records. Empty history does not require a model download.
+
+Search combines meaning and exact keywords every time. It waits for both indexes rather than quietly returning keyword-only results while semantic work is incomplete. First use or a large amount of newly collected history can take minutes. Search, text-reply preparation and pre-call preparation show the current phase, model-download bytes or completed/total **new or changed unique passages**, unchanged cached work separately, elapsed time and an approximate remaining time once enough passages have been measured. The estimate can change with passage length. Progress is read-only metadata; polling it does not start indexing. **Call Draken stays disabled until both the briefing and searchable history are ready.** Sync preparation includes history preparation; an older or changed sync offers **Prepare history**, with progress and cancellation on the sync page, not inside a call. Call creation checks readiness again and rejects unprepared history without starting a call or building an index. Cancelling or failing preparation leaves the microphone off. Ordinary concurrent requests queue behind one history process, within a fixed bound. Its embedding lanes run bounded batches in parallel; they do not run concurrent database writers. An unchanged index shows **History is up to date**, not a full-archive progress bar.
+
+No session text or queries are uploaded for embedding. Text discussion and ChatGPT voice do receive the relevant historical excerpts, just as they receive frozen snapshot context. Those excerpts are dated background—not newly reviewed progress, agreement or permission to execute work.
+
+Opening/reloading a page does not itself download the model, compute embeddings or start a microphone. Source collection happens through sync/scan actions or the optional server-side background schedule below. The search child itself follows only recorded, verified JSONL pointers beneath the configured per-agent source roots; it does not discover sessions or roam project files. There is no automatic decision saving, API-key fallback or second voice engine.
+
+### Optional background indexing
+
+**Settings → Automatically index history**, available from the top bar on every page, is off by default. Its explicit, persistent opt-in authorizes local scanning and embedding (including the first-use model download) while the server runs, even with no browser open. It does not run provider summaries, create/extend frozen syncs, transcribe audio or advance review. Agent logs are already text; summary generation still happens on the explicit sync action and remains separately costed.
+
+The app's single scheduler checks once a minute when no sync, call or foreground history work is active. A background batch requires at least 64 changed source records, regardless of how long they have waited. There is no timed flush. Smaller pending batches wait for a sync, **Prepare history**, or search; that foreground path has no minimum batch size. Background indexing records the same history readiness used by calls. If anything still needs preparation, Call remains disabled—clicking it cannot start a hidden indexing job. Metadata hashes detect additions, edits and removals once per original source entry (or discussion/note record). Session/file bookkeeping does not add records, and frozen aliases coalesce with the original rather than multiplying the count. Actual indexing still reconstructs and verifies admitted source bytes. Unchanged checks do not start an embedding request. Source records and embedding passages are different counts. New sessions from enabled agents retain the collector's included-by-default behavior; per-agent/per-session exclusions and the existing baseline remain respected. Codex and Claude sources require separate explicit opt-in in **Settings → Agents & sessions**, even if background indexing is already enabled. Summarization continues to use pi. See [SOURCES.md](SOURCES.md).
+
+Foreground history requests preempt an active background request, waiting for the old child to exit before replacement. Disabling aborts only owned background embedding work; a scan already in progress may finish, but cannot start an embedding batch afterward. Completed vector commits remain reusable. Failures are visible and retry no sooner than five minutes; foreground preemption waits at least one minute. Shutdown awaits the owned scan/work within the existing bounded server shutdown. Reads of settings/progress start no jobs. The preference is a small `sync_settings` entry. A hash-only checkpoint in the derived search database survives restart and advances only after successful unscoped/current-call preparation; a historical or source-filtered search cannot replace it. It is scheduling metadata, not review coverage or a substitute for source verification. Progress remains process-local. An older index without a checkpoint waits for one foreground call preparation or unfiltered history search instead of treating its cached archive as newly changed. Unchanged preparations do not rewrite the checkpoint. There is no original-text copy, new provider, model substitution, vector conversion or timed small-batch flush.
+
+A failed preparation/search reports an error. If a session file changed, moved or disappeared, scan sessions again before retrying; stale derived text is not a substitute for a verified source. Otherwise retry the question or **Prepare history**; completed embeddings are reused automatically. Cancel preparation on the sync page to stop that work. Cancelling a search or text reply stops its owned work too. Reloading does not start a call. An older cache without a call-readiness record needs one preparation pass, which reuses its compatible vectors. A completed index over a partial import is not sufficient: foreground call preparation first drains bounded import chunks to captured file ends. Missing, changed or incomplete originals block readiness. Later appends do not force preparation to chase a live stream forever; the call uses its displayed cutoff, and new work waits for another preparation. A ready sync offers **Refresh history** for an explicit local-only update without regenerating its briefing. See [SOURCES.md](SOURCES.md).
+
+## Work questions and follow-ups
+
+Voice and **Chat instead** work questions prefer original session records, not old Draken replies or repeated questions. Project names in admitted session metadata establish a scope that follows generic follow-ups; an explicit project change updates it. Explicit conversation recall still searches earlier discussion. This is a heuristic, not a guarantee that an ambiguous project name will be understood correctly.
+
+Retrieval ranks both keywords and local vectors, deduplicates shared passages before the candidate limit, and gives original agent explanations their own candidate set. Current-status questions include the latest recorded agent explanation and favor recent records. Up to two named topics in a question can be searched separately so one does not crowd out the other. All candidates remain subject to source permissions and the prepared cutoff. Useful negative or historical evidence is not confirmation of a current defect.
+
+Voice source packets contain up to four sources within 7,000 UTF-8 bytes. Excerpts keep contiguous explanatory text; tool requests may be omitted with an explicit marker. A later passage can include the beginning of its source for context. The native helper can refine a question or request an H reference (optionally with `offset 2000`); each user turn permits at most three distinct lookups and reuses duplicates. Source IDs are the context supplied for that turn, not verified per-claim citations or a cumulative claim that every earlier source supported the answer. A failed or empty work lookup is not silently replaced by an old conversation.
+
+### Separate text chat
+
+`/syncs/:id/chat` uses the same project/topic planner and prepared-only source search, with an up-to-32,000-byte historical packet (at most 16 sources). Recent actual user questions in the sync's saved chat and native call transcripts establish follow-up scope; assistant replies cannot select a project. The text model receives qualified recent call transcripts as context, not evidence of hearing or agreement. Source-reference validation checks existence, not per-claim truth.
+
+Text status questions additionally check for later related original assistant narratives in the same source session. The bounded lexical pass considers at most 4,096 admitted narratives, four starting threads and two further follow-up generations; it does not run another embedding model or rebuild the corpus. It checks individual pending sub-tasks, favors relevant implementation sections over repeated pending tables, and retains a latest related report. Sequence orders records within a session, not across sessions. Follow-ups can themselves be stale, so a client-completion report that still lists backend jobs as pending is checked against later backend reports. Candidate links are not verified supersession: later plans, contradictory reports, completed foundations and unfinished delivery/acceptance remain distinct.
+
+Packets retain contiguous excerpts and, when clipped, separately labelled ending excerpts with original offsets so closing qualifications survive. An earlier linked checkpoint is omitted if its selected follow-ups cannot fit. Omitted candidates/threads are disclosed; no-match never establishes that no later completion record exists. These are language-dependent lexical candidates over admitted projections, not exhaustive whole-log or cross-session semantic verification.
+
+If a text answer cites a linked old checkpoint but ignores its supplied follow-ups, including an E/C alias of that same original record, Draken permits **one repair using the same final model and invocation/cancellation budget**. The rejected draft is never saved or shown. A second failure is visible and retains the user question without an assistant answer. **Checking later updates** identifies that repair stage. It can consume one additional final-model request; chat/repair usage remains outside the preparation-only estimate. This citation-consistency check is not proof that every claim is true. The larger packet and repair apply to text replies, **not** streaming native voice, whose separate 7,000-byte lookup budget is unchanged.
+
+Chat readiness checks the prepared receipt at its captured cutoff. New chat messages beyond that cutoff are passed directly as recent conversation and do not force an index rebuild on every turn. Original-file identity/import cursors, selection changes, missing originals and edits to already captured records still invalidate readiness. Fresh native-call readiness keeps its stricter current-record check. Both readiness endpoints only read metadata. Refresh is explicit; chat does not silently refresh the corpus, regenerate a briefing or switch providers.
+
+## How it works
+
+```text
+Canonical metadata in draken.db (read-only to search worker)
+             │
+             ├── selected session pointers → verified JSONL records
+             ├── frozen evidence pointers → exact recorded JSONL/edit selections
+             ├── preserved sync discussion (with playback qualifications)
+             └── explicitly saved notes
+             ↓
+Automatic local preparation when needed
+             ├── SQLite FTS5 word index
+             └── sqlite-vec over local text embeddings
+             ↓
+Combine matches → bounded source/context reads → dated H citations
+```
+
+- **Model:** `Xenova/multilingual-e5-small`, pinned revision `761b726dd34fb83930e26aab4e9ac3899aa1fa78`, quantized ONNX. Uses query/passage prefixes, mean pooling and normalized 384-dimensional vectors. Long token sequences use bounded windows rather than silently embedding only a prefix.
+- **Runtime:** CPU WASM through the same pinned `onnxruntime-web`, in one server-side history process. On machines with at least 12 available logical CPUs and 4 GiB of available memory, deltas of at least 32 passages use up to two isolated embedding worker threads, each with one model session and three kernel threads (six total). Smaller/constrained machines keep one lane, with at most four kernel threads and half the available CPUs (minimum one). Reads start no model; queries/small deltas use one lane. A second in-memory model copy costs additional RAM (about 1.7 GB process RSS for two lanes versus roughly 1 GB previously on the qualified host), but adds no on-disk model copies, GPU package or runtime dependency. Lanes retire with the history process, including cancellation; failures stop the request rather than switching runtimes. It is **not imported by the client** and is not transcription or speech synthesis. This avoids native CUDA-install downloads. The runtime package itself occupies additional disk space (about 133 MiB in the qualified Linux install), beyond model files.
+- **Tokenizer:** pinned `@huggingface/tokenizers`; no downloaded executable model code.
+- **Search:** FTS5/BM25 plus pinned `sqlite-vec` 0.1.9. Approximately 800-character overlapping passages (without redundant short tails already covered by the preceding passage), up to 80 candidate chunks per method, combined ranks and at most eight distinct source results. The minimum cosine-similarity threshold is a heuristic, not a correctness guarantee. Exact identifiers remain important keyword matches.
+- **Identity:** content-addressed `H…` references retain source attribution; snapshot-local E/C labels are not global IDs. Duplicate attributions retain scope membership while sharing embeddings and collapsing in results. Changed/excluded records cannot silently replace the text behind an old link.
+- **Preparation:** source admission and raw pointer integrity are still revalidated; delta indexing does not skip those checks. A bounded, process-local normalization cache avoids reparsing unchanged records, but is consulted only after rereading and verifying the complete raw-record hash. No cached text substitutes for missing or changed originals, and this cache is never written to disk. Only added/changed/deleted source records update postings and links. Only unique passages without a compatible vector enter the embedding queue. Unchanged refreshes perform no index writes, and a fully cached preparation does not load/download a model; an actual semantic query still ensures its model is ready. At most eight passage inputs are scheduled per batch, with no unbounded queue. Writes commit at batch boundaries after at most 64 passages or roughly a second of computed work. Progress counts only committed delta passages from zero; cached counts sit outside the denominator. A retry counts only work still uncommitted. ETA measures parallel wall-clock throughput and excludes the first cold batch. The existing model fingerprint/cache and original source references remain valid; obsolete fully covered tail chunks are removed only from the derived index.
+- **Bounds:** one worker and at most eight queued requests; bounded queries, candidates, source pages and context packets. Preparation/search stops after ten minutes without a progress update, or two hours overall; read operations retain their 30-second limit. A healthy first index is no longer killed merely because it passes 30 minutes. The separate voice-call limit still applies. The worker retires after five idle minutes. Cancellation terminates the owned child when needed; queued work waits for its exit before starting a replacement. Search/read cancellation is isolated from other queued requests. Server shutdown cancels all owned work.
+- **Storage:** `records` retains attribution metadata, not another full-text copy. Unique `passages` hold input hashes and embedding state, not text; `chunk_links` retain every source occurrence and offset. `passage_vectors` stores one vector per unique input. `chunks` and `embedding_cache` are join views, not duplicate physical stores. Contentless FTS5 keeps occurrence-level token postings and positions, not original passage strings, so scope membership and keyword document frequencies remain intact. Tokens and vectors are still private derived data, not an anonymization or secure-erasure mechanism. Text for reads and embedding is reconstructed from verified sources in memory. Semantic ranking selects scoped unique passages, expands their scoped occurrences, then bounds the candidate list; ties can rank differently from the former per-occurrence vector layout.
+- **Rebuildability:** the separate passages, keyword index and vectors are derived. Fingerprints invalidate incompatible model data. The worker opens `draken.db` read-only and can read only recorded session files beneath the explicitly supplied session root. It has no source-file write, discovery, scan or application-write path.
+
+## Pointer storage and migration
+
+Live `agent_entries` keep metadata, agent/session attribution, parser versions, byte offsets/lengths, raw-record SHA-256 hashes and branch/edit links—not `text`, `effective_text` and replacement copies. Reads verify the configured root, regular-file identity, session header, complete record hash and normalized entry identity. Context edits resolve both the original record and the selected edit. Append scans verify prior pointers when the file changes, including middle rewrites that leave its prefix and old tail intact.
+
+**Pi JSONL files are the authoritative original-text archive, including for old snapshots.** `sync_items` and frozen background context pin the original record coordinates/hash, normalization version, exact selected context-edit pointer when present, any historical excerpt-clipping recipe, and a projected-text hash. Their membership, attribution and E/C identities remain frozen; later appends/edits/branch switches do not redirect them to the live projection. The current schema has no `retained_text` table. Versioned normalization reproduces older evidence exactly while current model-facing redaction remains in force.
+
+Missing or altered originals are explicitly unavailable, not substituted with search text. Snapshot pages still show discussion, notes and stable citation identities. Generating replies, calling, expanding or marking the snapshot reviewed requires its evidence to be available. Restore the original files and rescan if live pointers moved or their file identity changed. Frozen reads can follow a newly registered path only after checking the session, record hashes and projection. An explicit scan updates current missing/admission state; it does not reconstruct lost snapshot bodies.
+
+The one-way `pointer-v2` migration verifies every existing body against recorded original session prefixes before removing it. Unavailable, ambiguous or unmatchable originals stop and roll back conversion, preserving the old database rather than deleting or guessing evidence. Legacy edit matches must be unique, on a descendant branch and not dated after the snapshot; they carry `legacy_edit_match` because exact edit selection was not originally recorded. New snapshots record that selection directly. Entry IDs, projected evidence, notes, review coverage and summary caches are preserved. The `history-index-v3-contentless` search migration removes passage-body storage while retaining compatible vector bytes, occurrence/reference IDs and keyword frequencies without inference or an embedding-model change. Each database's conversion is transactional and can retry after failure. Unknown formats or conflicting vectors fail visibly instead of silently rebuilding.
+
+Back up both current databases before deployment and qualify migration on private copies. Older binaries are not compatible with the pointer schema. SQLite can retain freed pages and WAL files: reclaiming them with an offline checkpoint/VACUUM is a separate deployment step, not automatic history deletion or page-open maintenance. The search worker closes SQLite on orderly retirement and requests a 4 MiB retained-journal limit after normal checkpoints; active transactions/readers can still require a larger WAL. Backups and pinned model files are additional disk use and are not included in compacted database-size claims. Pre-migration rollback backups still contain the old copied text; migration does not delete or securely erase those backups.
+
+## Scope and qualifications
+
+Source inclusion, missing-session state, active branches for unsnapshotted entries and the existing history baseline are respected. Frozen evidence remains historical, including superseded branch material. Reviewed records remain searchable. Discarded sync discussion/notes are excluded, while discarded work can still be eligible through an active selected session.
+
+Ordinary search and text discussion prepare the candidates observed at their start. Voice instead uses the history cutoff prepared before the call: it verifies original bytes and queries existing passages, but does not build missing corpus embeddings or download a missing model. Newer records wait for a later preparation and call. Query embeddings still run locally during a call. Exclusions and edits are rechecked after inference; stale or unprepared history fails visibly rather than being quietly rebuilt during the call. This avoids mixing fresh keyword-only records into results described as fully hybrid.
+
+Sync discussion is its own retained record: excluding a session does not erase text previously discussed in a sync. Recaps/current briefings are not separately indexed as primary evidence, though preserved discussion can contain historical assistant summaries; attribution remains visible.
+
+Search covers admitted normalized source projections and retained Draken discussion—not every byte of the original JSONL files. Excerpt, ranking and packet limits can omit important information. No match is not proof something never happened. Similarity is not proof a source supports an answer. The local multilingual qualification is not a comprehensive relevance benchmark.
+
+Open-sync notes have no retained edit timestamp and are not backdated. Date filters exclude undated notes. Calls use their start time as the historical cutoff so the active call does not retrieve its new questions as old history. A date filter does not reconstruct an earlier database state: context edits may change retained text without supplying a separate edit date.
+
+Generated/interrupted speech stays qualified as generated/interrupted. Historical requests are data, not instructions. Prompts distinguish plans, reported work, observed validation and uncertainty. Models can still make semantic mistakes; inspect supporting records. Retrieval cannot save decisions, execute tools, complete syncs or advance coverage.
+
+## Private files and packaging
+
+Under `DRAKEN_DATA_DIR` (default `~/.draken`):
+
+- `draken.db`: canonical live/frozen pointers and metadata, Draken conversations, notes and generated summaries; not rewritten by retrieval.
+- `memory/index.sqlite` and sidecars: passage hashes/unique vectors, source-occurrence mappings and contentless keyword postings.
+- `memory/model/`: three pinned model/tokenizer files, downloaded automatically when first needed.
+
+The directory is private (0700); database/model files are 0600. Embeddings are private data too. None belongs in a commit or npm tarball. Exclusions remove queryable derived records/cache on refresh; this is **not secure erasure** of filesystem blocks, WAL files, backups or canonical snapshots.
+
+The npm package contains code/dependencies, not downloaded weights or private indexes. No database service, extra account, GPU package or user-selected search backend is required. Extension/model/runtime failures are visible; there is no alternate provider or keyword-only fallback.
+
+## Checks
+
+Ordinary tests use synthetic records and deterministic or empty embedding fixtures, with real FTS5/vector tests. They do not download/run the real model. Browser fixtures exercise automatic search and source reading without setup controls, plus no capture on navigation. Existing native-only and review-boundary tests remain.
+
+Optional real-model qualification and CPU comparison commands live in [CONTRIBUTING.md](CONTRIBUTING.md#local-model-qualification). Those are isolated test tools, not steps required to use search. Search and text requests prepare the index automatically; calls require completed preparation first. There is no separate model setup command for users.
+
+Synthetic retrieval and throughput checks do not establish exhaustive recall, real-provider answer quality or performance on every machine.

package/PERSONA.md

@@ -0,0 +1,67 @@
+# Draken’s 3D call persona
+
+The call window now contains a local, animated mechanical dragon in a furnished workshop. This is an **initial procedural 3D interpretation**, not a photoreal reconstruction or camera/video feed. The reference is the actual illustration from [Draken’s homepage](https://draken.mehanig.com/): `site/img/scene1-web.jpg`, copied unchanged to `client/public/persona/reference.jpg`.
+
+The character keeps its blue-gray mechanical shell, red wooden jaw/wing struts, amber eyes, ivory horns and mustard knitwear, with a knitted beanie. The call uses a closer, nearly frontal portrait with an uncluttered oak desk and chair. A dusk window, warm floor lamp and single plant sit in a softly blurred room behind the sharp character. Busy slats, shelves, clock, print and desktop accessories have been removed; lower-contrast wood/knit textures and matte wood reduce visual noise. Materials and lighting aim for a warm, tangible setting; the geometry remains deliberately stylized. A professionally sculpted/textured character would be a separate art pass.
+
+## Try it separately
+
+Open **`/persona.html`** on your Draken server. In development:
+
+```sh
+npm --prefix client ci
+npm run dev:client
+# Open http://localhost:3000/persona.html
+```
+
+The preview offers listening/thinking poses, a generated speech sample, pause and the original reference. It does not request a microphone, authenticate, read sessions, or invoke a model. Everything is served locally. The call integration mounts the same component only while its dialog is active. Calls keep the dragon prominent, with a lightweight DOM subtitle layer at the bottom of the scene. It shows live transcript excerpts or bracketed waiting cues; the full transcript/details block is collapsed by default. There is no workshop label or animation-button overlay. Preview chrome is an explicit `studioControls` option; the preview keeps its pause/retry controls.
+
+## Boundaries
+
+```text
+voice/live-session.ts — owns the ChatGPT call and audio
+  └─ optional onPlayback(PlaybackCue | null)
+       └─ VoiceCall.tsx stores a ref and maps call phase to PersonaMode
+            └─ persona/PersonaStage.tsx — lifecycle, visibility, lazy loading
+                 ├─ scene.ts — WebGL renderer, lighting, camera
+                 ├─ dragon.ts — articulated character
+                 ├─ studio.ts — separated background / foreground furniture
+                 ├─ background.ts — cached, separable Gaussian room blur
+                 ├─ geometry.ts — materials and disposable GPU resources
+                 └─ motion.ts — pure motion / audio-envelope math
+```
+
+`client/src/audio/playback.ts` is a presentation-only contract. Live calls supply a read-only energy meter and audio clock. The standalone preview supplies its stored sample's text, PCM samples, duration and playback clock. The renderer cannot start/stop speech, submit a turn, write notes, access auth or finish a sync. Observer exceptions are contained by the audio controller. Live voice optionally supplies only a read-only received-energy getter and audio clock, with no PCM retention or invented phoneme alignment. The persona itself requires no server/schema/model dependencies.
+
+`types.ts` describes the small pose/scene interface. A future authored mesh or glTF rig can replace `dragon.ts` and map the existing pose values to bones/morph targets; call logic and the room do not need to change. No glTF loading or external avatar service is implemented in this version.
+
+## Animation and timing
+
+- The standalone preview derives jaw energy from its **stored sample's decoded audio**. Its text supplies approximate English mouth shapes. This preview does not synthesize speech or provide another call engine.
+- Native live voice follows actual received energy through the same presentation boundary. It does not estimate phonemes from a growing transcript; mute/stop, silence and suspended output produce a closed mouth.
+- Timing uses the same `AudioContext.currentTime` as playback—not response arrival or a looping “talk” animation. Silence, chunk boundaries, interruption, suspended audio and hang-up close the mouth.
+- Idle breathing/blinks, listening/thinking head poses, speaking gestures and question tilt are procedural. They are not model-generated emotions or evidence of attention/understanding.
+- This is **heuristic lip synchronization**, not forced phoneme alignment. Accents, names and speech rate can misalign individual shapes.
+- Pausing motion in the preview or an OS reduced-motion preference keeps a static pose without changing audio. Offscreen/hidden scenes stop drawing. Hang-up disposes textures, geometry, materials, shadows, environment maps, timers and the WebGL context.
+
+## Performance and failure behavior
+
+The Three.js scene is loaded lazily (about 140 kB gzip for the scene/engine chunk), not with the ordinary sync page. No CDN, remote textures/fonts, webcam, extra voice API or rendering server is involved. The reference, a WebP poster matching the new framing/blur, generated demo WAV and Three.js MIT license are local assets.
+
+Hardware rendering is capped at 30 fps while speaking / 18 fps otherwise, DPR 1.5 and 900,000 pixels. Known software renderers use an economy profile: 10 fps, at most 160,000 pixels, no dynamic shadows, clearcoat or cloth sheen. This is visibly less detailed, but keeps graphics from overwhelming audio/control responsiveness. A timer between animation frames avoids continuously asking the browser to composite at 60 fps. These are caps, not promised frame rates.
+
+The room is rendered and blurred only on initialization/resize; each animation frame composites that cached texture behind the sharp dragon and desk. This is layered call-style background separation, not a full optical depth-of-field simulation. Both rendering profiles use it. Blur targets/materials are disposed with the renderer.
+
+WebGL initialization/context loss switches to the static poster; voice continues independently. The call fallback retains the same live subtitle layer and an accessible scene description. Captions are presentation only and do not drive speech or write records. The standalone preview retains its retry button and **Pause animation** control. The preview and call were exercised in Chromium, including its software renderer. Physical mobile GPUs, Safari, Firefox and long-session battery behavior are not yet verified.
+
+## Verification
+
+```sh
+npm test                 # Includes pure audio-clock/pose and module-boundary checks
+npm run test:ui          # Includes preview, reduced motion, context loss and call regressions
+
+# Explicit opt-in: synthetic evidence/audio, real ChatGPT voice, staged frontend:
+DRAKEN_TEST_STATIC_DIR=/path/to/stage node tests/live-voice-real.cjs --allow-provider
+```
+
+Browser checks cover a chrome-free call image, cached blur reuse/resize, high-quality shader compilation (also exercised on software GL), audible sample mouth motion, stopping, desktop/mobile sizing, local-only requests, no preview microphone access, WebGL loss/retry, unavailable WebGL during a working call, and context disposal on hang-up. Native controller/peer tests cover speaker/headphone gating, interruption, mute, permission refusal and cleanup. Main UI tests cover supplied sources, generated/interrupted history, explicit retry and preservation of retired-engine records. No production snapshot is changed by these tests.