trajectories 0.0.0 → 0.1.2

package/docs/research/ledger/docs/research/2026-09-09-native-history-discovery.md

@@ -0,0 +1,180 @@
+# Native History Discovery
+
+## Answer
+
+**Trajectory's normalized format and Cursor/OpenCode transformations exist.
+Native-store discovery and extraction are the missing steps, not a replacement
+canonical format.** Cass implements native readers for both; Obelisk's registered
+providers implement neither at the inspected revision. This is source research,
+not a claim that Ledger's native-store ingestion has now been fixed.
+
+## Trajectory: prepared input, not arbitrary native storage
+
+The inspected `@letta-ai/trajectory` 0.3.0 revision is
+[`21ae92d10579499b183d93929c85da0b91233167`][trajectory-head].
+
+- **Cursor:** captured JSONL with `role` and `message.content`, not SQLite or
+  arbitrary Cursor JSONL. Canonical normalization needs caller-supplied session
+  identity. [Input contract][trajectory-cursor].
+- **OpenCode:** one assembled JSON document containing session `info` and
+  `messages`, each with message `info` and `parts`. Separate native rows/files
+  must first be assembled into that shape. [Input contract][trajectory-opencode].
+- Both sources deliberately return `listing_unavailable` from
+  `listTrajectories()`. [Dispatcher][trajectory-listing].
+
+The [preceding contract investigation](2026-09-09-harness-transcript-contract.md)
+also records successful synthetic normalization probes for both sources. These
+prove the prepared-input transformation, not coverage of actual local histories.
+
+## Cass: native readers before its own normalization
+
+Fresh `main` inspection resolved Cass to
+[`000301c949df5180c0023ef7f6b19767a0cf9fbc`][cass-head], committed
+2026-09-09T00:37:44Z. Its Cursor/OpenCode modules now
+[re-export][cass-cursor] [connectors][cass-opencode] from
+`franken-agent-detection`, pinned to exactly **0.2.3** by
+[Cargo.toml][cass-dependency] and [Cargo.lock][cass-lock].
+The published crate's `.cargo_vcs_info.json` and annotated release tag both
+resolve to [`0feebc55fed8db4d596f330742fe11ca4cb70764`][fad-head]; the reader
+links below use that version-matched source, not the dependency's moving main.
+[Published archive][fad-archive], [release tag object][fad-tag].
+
+### Cursor
+
+- Finds `state.vscdb` in global/workspace storage, including macOS
+  `~/Library/Application Support/Cursor/User/`. Opens read-only; reconstructs
+  Composer sessions from `composerData` plus referenced `bubbleId` rows, with
+  legacy `ItemTable`/conversation layouts. [Discovery][cass-cursor-roots],
+  [database reader][cass-cursor-db], [assembly][cass-cursor-assembly].
+- Separately recognizes
+  `~/.cursor/projects/<project>/agent-transcripts/<sid>/<sid>.jsonl` and
+  role/message content. It explicitly excludes subagent paths and does not
+  decode `agentKv:blob:*`. The source labels this path reporter-documented,
+  not verified there against real Cursor data. [Coverage and exclusions][cass-cursor-agent].
+
+Thus this is implemented, selective coverage, not every Cursor store or a
+universal export format.
+
+### OpenCode
+
+- Reads `opencode.db` and assembles conversations from `session`, `message`,
+  and `part` tables. [Layout][cass-opencode-layout],
+  [session reader][cass-opencode-db], [message/part loading][cass-opencode-rows].
+- Also assembles legacy `storage/session/<project>/<session>.json`,
+  `message/<session>/<message>.json`, and `part/<message>/<part>.json`.
+  It starts from session files, not every JSON file under `storage`, and
+  deduplicates sessions already read from SQLite. [Selection and dedup][cass-opencode-selection],
+  [assembly][cass-opencode-assembly].
+
+Both readers emit their own `NormalizedConversation` / `NormalizedMessage`
+types, not Letta Trajectory records. OpenCode content is flattened for search
+and some control parts are discarded, so Cass's output is not a lossless
+native export to pass straight into Trajectory. [Shared types][cass-types],
+[content projection][cass-opencode-content].
+
+Cass scope: inspected public source and package metadata; no Cass installation,
+test execution, or private-store scan. Existing
+[licensing cautions](2026-09-09-other-inspirations.md) remain; this is not a
+dependency adoption or permission to reuse implementation code.
+
+## Obelisk: Cursor and OpenCode
+
+Audited **2026-09-09, 01:57 UTC**. GitHub's [repository metadata][repo-api] and
+[branch API][branch-api] resolved default branch `main` to
+[`69d9e21a90e55f310376d8f49929b1c16f5a1f0c`][head], committed
+2026-09-03T10:48:53Z. All source links below pin that revision.
+
+**Verdict: neither Cursor nor OpenCode native local-history discovery or
+normalization is implemented by the registered built-in providers.**
+[The exact registration list][builtins], lines 17-23, is `claude`, `codex`,
+`deepseek`, `kimi`, `pi`: Claude Code, Codex, DeepSeek Harness, Kimi Code, and Pi.
+The [registry][registry] enumerates only supplied adapters; the
+[configured runtime][settings] resolves roots from that catalog and reconstructs
+the same built-ins. Adding a directory setting cannot add another source format.
+
+Two misleading matches deserve explicit separation:
+
+- **Cursor:** [editor integration][editor] builds file-opening URLs using the
+  Cursor scheme. It does not discover or decode Cursor history. The contract's
+  `Cursor` type means a resume checkpoint, not the Cursor application.
+- **OpenCode:** the [provider-contract introduction][types] mentions it as a
+  future source. An [activity-label test][label-test] uses `opencode` to verify
+  unknown-provider labeling, not ingestion. Neither occurrence supplies an
+  OpenCode adapter.
+
+**What native discovery actually does:** adapters read harness-owned history
+directly. For example, [Claude discovery][claude] reads project transcripts and
+history-title metadata; [Codex discovery][codex] scans both active and archived
+JSONL transcript directories, using `session_index.jsonl` for title/update
+metadata rather than message bodies. Discovery and decoding remain
+source-specific, not inferred from a directory's name.
+
+**Native-provider contract:** Obelisk defines its own internal
+[`ProviderAdapter` / `TranscriptRecord` contract][types]. An adapter supplies a
+descriptor/default root, watch targets, discovery, parsing, and raw-record
+lookup. `discover(ctx)` returns `IndexUnit` work items with stable keys, session
+identity, and opaque metadata. Context supplies previous cursors, optional
+changed paths, indexed-session provenance, and incomplete-inventory reporting.
+`parse(unit, cursor)` yields canonical session, message, tool, summary, subagent,
+workflow, and update/retraction records, then returns its next opaque checkpoint.
+The [shared indexing caller][indexing] invokes registered adapters and persists
+their records into SQLite. The contract allows non-file work units, including
+database-backed sources; that extensibility is not implemented Cursor/OpenCode
+support.
+
+Scope: public-source inspection only, no clone or runtime verification, and no
+private transcripts, databases, or secrets accessed. Under the AGPL-3.0 boundary,
+this note records behavior only; no upstream code was copied, vendored, or derived.
+
+## Ledger implication
+
+The [current scanner](https://github.com/archeism/ledger/blob/43081aef54c23b4bd017b6c0e11fc42a9a1cb381/packages/core/src/scanner.ts) walks Cursor JSONL
+and OpenCode JSON by extension. It neither reads these SQLite layouts nor
+assembles OpenCode's split native records. Finding telemetry or `session_diff`
+files does not make them valid conversation inputs.
+
+**Proposed boundary, not implemented here:** native discovery and read-only
+extraction/assembly, followed by a validated Trajectory input, canonical
+normalization, and durable Ledger ingestion. Matching Cursor Agent JSONL may
+already fit the input shape; SQLite histories need a deliberate projection.
+Retain native evidence and identities instead of routing through Cass's flattened
+search text. Use source-specific candidate validation, explicit unsupported or
+incomplete coverage, and bounded regression fixtures for native-store versions.
+
+No runtime code, daemon, database, or ingestion state was changed by this note.
+
+[trajectory-head]: https://github.com/letta-ai/trajectory/commit/21ae92d10579499b183d93929c85da0b91233167
+[trajectory-cursor]: https://github.com/letta-ai/trajectory/blob/21ae92d10579499b183d93929c85da0b91233167/src/adapters/cursor/README.md#L3-L27
+[trajectory-opencode]: https://github.com/letta-ai/trajectory/blob/21ae92d10579499b183d93929c85da0b91233167/src/adapters/opencode/README.md#L3-L37
+[trajectory-listing]: https://github.com/letta-ai/trajectory/blob/21ae92d10579499b183d93929c85da0b91233167/src/listing.ts#L63-L97
+[cass-head]: https://github.com/Dicklesworthstone/coding_agent_session_search/commit/000301c949df5180c0023ef7f6b19767a0cf9fbc
+[cass-cursor]: https://github.com/Dicklesworthstone/coding_agent_session_search/blob/000301c949df5180c0023ef7f6b19767a0cf9fbc/src/connectors/cursor.rs#L1-L5
+[cass-opencode]: https://github.com/Dicklesworthstone/coding_agent_session_search/blob/000301c949df5180c0023ef7f6b19767a0cf9fbc/src/connectors/opencode.rs#L1-L5
+[cass-dependency]: https://github.com/Dicklesworthstone/coding_agent_session_search/blob/000301c949df5180c0023ef7f6b19767a0cf9fbc/Cargo.toml#L137
+[cass-lock]: https://github.com/Dicklesworthstone/coding_agent_session_search/blob/000301c949df5180c0023ef7f6b19767a0cf9fbc/Cargo.lock#L1882-L1886
+[fad-head]: https://github.com/Dicklesworthstone/franken_agent_detection/commit/0feebc55fed8db4d596f330742fe11ca4cb70764
+[fad-archive]: https://static.crates.io/crates/franken-agent-detection/franken-agent-detection-0.2.3.crate
+[fad-tag]: https://api.github.com/repos/Dicklesworthstone/franken_agent_detection/git/tags/38662916851d9c5438fae5bfbe622fb958b7aca8
+[cass-cursor-roots]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/cursor.rs#L90-L257
+[cass-cursor-db]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/cursor.rs#L478-L555
+[cass-cursor-assembly]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/cursor.rs#L558-L645
+[cass-cursor-agent]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/cursor.rs#L911-L1010
+[cass-opencode-layout]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/opencode.rs#L1-L10
+[cass-opencode-db]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/opencode.rs#L475-L608
+[cass-opencode-rows]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/opencode.rs#L611-L780
+[cass-opencode-selection]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/opencode.rs#L1026-L1116
+[cass-opencode-assembly]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/opencode.rs#L1333-L1451
+[cass-types]: https://github.com/Dicklesworthstone/coding_agent_session_search/blob/000301c949df5180c0023ef7f6b19767a0cf9fbc/src/connectors/mod.rs#L11-L24
+[cass-opencode-content]: https://github.com/Dicklesworthstone/franken_agent_detection/blob/0feebc55fed8db4d596f330742fe11ca4cb70764/src/connectors/opencode.rs#L1481-L1521
+[repo-api]: https://api.github.com/repos/tommy0103/obelisk
+[branch-api]: https://api.github.com/repos/tommy0103/obelisk/branches/main
+[head]: https://github.com/tommy0103/obelisk/commit/69d9e21a90e55f310376d8f49929b1c16f5a1f0c
+[builtins]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/builtins.ts#L13-L23
+[registry]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/registry.ts#L20-L49
+[settings]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/provider-settings.ts#L98-L138
+[editor]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/app/src/main/file-reference.ts#L7-L82
+[types]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/types.ts#L4-L320
+[label-test]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/tests/app-activity-ledger.test.mjs#L73-L75
+[claude]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/claude.ts#L80-L137
+[codex]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/codex.ts#L77-L151
+[indexing]: https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/provider-indexing.ts#L201-L279

package/docs/research/ledger/docs/research/2026-09-09-native-history-implementation.md

@@ -0,0 +1,137 @@
+# Native History Implementation Verification
+
+## Scope And Status
+
+Implemented in `feat/native-history-discovery`, based on PR7's
+`6782d5575b017eefa35b7714f92c40ce8c690220`, in an isolated worktree. The running
+daemon checkout and its Ledger database were not changed or restarted. This is
+implementation acceptance, not a deployed full-corpus backfill.
+
+The other inspiration is **AgentsView**, alongside Cass and Obelisk. Its
+[pinned native-reader audit](2026-09-09-agentsview-native-discovery.md) supplied
+format and operational evidence. Native readers were independently implemented;
+no upstream reader code was copied and no dependency was added. The installed
+`@letta-ai/trajectory` 0.3.0 remains the canonical normalizer.
+
+| Source | Discovery and assembly |
+| --- | --- |
+| Cursor captures | Validated `role` / `message.content` JSONL, flat or nested including `subagents/`; existing incremental JSONL importer retained. |
+| Cursor IDE | Global/workspace `state.vscdb`; Composer headers joined to bubble IDs, supported inline maps, legacy ItemTable tabs with proved native IDs. Text, reasoning and structured tools are projected into capture JSONL. |
+| OpenCode SQLite | `opencode.db` and channel databases; one native session joined to ordered messages and parts, including native IDs, parent ID, timestamps and structured tool state. |
+| OpenCode legacy | `storage/session/<project>/<id>.json`, `storage/message/<id>/*.json`, `storage/part/<message>/*.json`; assembled export JSON is also accepted in curated roots. |
+
+## Invariants
+
+- SQLite is opened read-only and session assembly uses a single deferred read
+  transaction. Size/count checks precede payload fetches. Defaults are 64 MiB
+  per assembled native session and 100,000 input records. These are input/output
+  bounds, not a bound on SQLite parser memory or total process RSS.
+- Cursor extraction selects chat fields, not credential-bearing Composer fields.
+  OpenCode native-root discovery skips auth/config/token files, extension/cache
+  trees and symlinks before opening candidates. Native stores are never copied
+  wholesale into the repository or mutated by ingestion.
+- DB, WAL and rollback-journal file identities invalidate native snapshots;
+  SHM does not. Legacy fingerprints include message/part files and directories,
+  detecting additions and deletions without a session-metadata timestamp change.
+- Source changes during normalization discard the candidate before commit.
+  Native build uses the same guarded ingestion path as watch, rather than
+  retaining a batch of prepared native sessions across embedding awaits.
+- Shared SQLite paths are not replacement identities. Native session/bubble/
+  message/part IDs remain source-scoped. Content hash hits refresh metadata and
+  provenance without replacing records. Explicit parentless snapshot evidence
+  cannot fall through to a second database read.
+- Failed native assembly preserves previously committed records. Deterministic
+  failures are cached for that snapshot and byte limit; repair or a raised limit
+  makes the source eligible again. Transient lock/read failures remain retryable.
+- Within the discovered page, native watch imports rotate by oldest attempt so
+  a frequently changing newest session cannot starve older candidates.
+
+## Verification
+
+Baseline: `bun test` reported **184 passed, 0 failed**, 1,108 assertions across
+19 files. Final parent-run gates on Bun 1.4.0:
+
+- `bun test`: **245 passed, 0 failed**, 1,478 assertions across 23 files, 17.13 s.
+- `bun run typecheck`: passed.
+- `bun run build`: passed, including TypeScript checking.
+- `git diff --check`: passed.
+
+An earlier full run was 244 passed / 1 failed: the new embedding-retry test
+queried with the parent process's default provider, while the CLI fixture wrote
+vectors for its isolated two-dimensional provider. Aligning and restoring the
+test provider fixed the assertion; the subsequent full run above passed. This
+was a test-isolation correction, not a hidden runtime embedding failure.
+
+Regression coverage includes shared-container isolation, WAL-only updates,
+stable record IDs, FTS replacement, legacy child-file additions, size/schema/
+identity failures, repaired-source retry, transient locks, source changes during
+normalization, metadata refresh, parentage, watch fairness and native CLI build
+racing a newer import. The new discovery, metadata and parentage tests were
+observed failing before their fixes.
+
+Live-store checks use `packages/core/scripts/native-history-smoke.ts`, not the
+daemon database. It considers at most three candidates, caps each assembled
+session at 8 MiB, disables embedding, prints aggregate counts only, and deletes
+its private temporary SQLite store afterward. A metadata-only session is not
+accepted as a successful conversation import. Example from `packages/core`:
+
+```sh
+bun scripts/native-history-smoke.ts opencode "$HOME/.local/share/opencode/opencode.db"
+bun scripts/native-history-smoke.ts cursor "$HOME/Library/Application Support/Cursor/User/globalStorage/state.vscdb"
+```
+
+Both final live-smoke commands exited 0. Each discovered three candidates and
+successfully imported the first into its own temporary database:
+
+| Source | Canonical records | Idempotence and coverage |
+| --- | --- | --- |
+| Cursor | 266: meta 1, user 37, reasoning 67, assistant 51, assistant-tool-call 55, tool 55 | No duplicate IDs; second ingest unchanged; `sessionNeedsIngest` false. Discovery also reported 113 incomplete-evidence and 48 invalid-metadata diagnostics from other entries. No all-Cursor coverage claim. |
+| OpenCode | 2: meta 1, user 1 | Current pending conversation retained; no duplicate IDs; second ingest unchanged; `sessionNeedsIngest` false. No discovery diagnostics in this bounded probe. |
+
+These are small functional acceptance samples, not capacity or performance
+benchmarks. No full backfill or daemon deployment was performed. The running
+checkout remained clean at `6a33ca80e921040981114bf3a94e9995566a0f15`.
+
+Initial real-store attempts exposed two fixture gaps. OpenCode persisted a
+message before its parts; retaining the empty message envelope avoids discarding
+the rest of that session. Cursor's empty-text rows included reasoning-only
+`thinking.text` bubbles and explicit `isPlaceholder: true` bubbles. Reasoning is
+preserved; explicit empty placeholders retain their source position. Arbitrary
+malformed or missing referenced bubbles still fail rather than silently truncate.
+Only shapes/counts, not private transcript text or credential values, were used
+in the investigation output. Synthetic regressions cover the observed shapes.
+
+### WAL Watch Regression
+
+- Initial attempts at `a direct database watch observes WAL changes without
+  periodic reconciliation` timed out after five seconds. The SQLite writer ran
+  in the same Bun test process. Fingerprints changed and the WAL-only
+  `sessionNeedsIngest` regression passed, but these attempts produced no observed
+  filesystem events or watcher `onEvent`, despite explicit database/WAL/journal
+  file watches and a 50 ms setup wait.
+- The revised regression passed when another Bun process performed the SQLite
+  update, matching the harness/daemon process boundary. `reconcileMs: 0` excluded
+  periodic reconciliation as the mechanism delivering this fixture's update.
+- A separate raw filesystem-watch probe observed some events. No root cause was
+  established for the same-process fixture failures; the passing revision does
+  not establish complete filesystem-event delivery across environments.
+- Explicit database/WAL/journal watchers and the 50 ms setup wait were retained;
+  temporary tracing was removed. The production periodic reconciliation floor
+  remains unchanged.
+
+## Remaining Boundaries
+
+- Cursor TXT captures, opaque `agentKv` / `store.db` payloads, unsupported native
+  layouts and attachment-only content are not claimed as supported. Discovery
+  diagnostics distinguish unsupported/incomplete input from no available history.
+- Native SQLite freshness conservatively invalidates the whole container. It
+  does not yet have AgentsView-style per-session content digests. The scan page
+  defaults to 1,000 candidates per source; rotation is only within that page.
+  Full coverage, large-store capacity and fair pagination remain unverified.
+- Filesystem events are best-effort; the production periodic reconciliation
+  floor remains necessary. No guarantee follows from one passing watcher test.
+- Canonical projection is not lossless raw archival. Auxiliary fields and
+  unsupported control parts can be omitted by the normalizer. This work does
+  not restore history already deleted by a harness or coordinate all writers.
+- Legacy JSONL build behavior outside the new native branch is unchanged;
+  native acceptance is not a claim that every older build path is bounded.

package/docs/research/ledger/docs/research/2026-09-09-other-inspirations.md

@@ -0,0 +1,279 @@
+# Other inspirations: source refresh, 2026-09-09
+
+## Outcome
+
+Keep the existing normalization dependency. The smallest useful follow-ups are:
+retained source generations, participating-writer ownership around atomic
+ingestion, fair resumable backfill, and an evidence-bearing agent search envelope.
+These are proposed Ledger changes, not guarantees inherited from another project.
+
+PR2 and PR6 are both merged in main at
+`56b51e2b76d8d37f749bb114f5b73f1e40164cf9`. The
+[integration report](../pr2-pr6-integration.md) records schema-6 migration,
+atomic cursors, staged replacements and bounded JSONL ingestion as tested
+contracts. Preserve that implementation; the remaining adaptations below extend
+it rather than reopening the superseded integration findings.
+
+## Scope and reproducibility
+
+Read [inspirations](../inspirations.md) and the
+[September 7 review](2026-09-07-confer-inspirations.md), then inspected primary
+source and test ranges at the following GitHub commits on September 9, CST.
+Loaded the Confer skill and checked the callable tool inventory and registrar.
+`confer_shell` was unavailable in this session, so the refresh used GitHub
+commit, compare, tree and contents APIs. No queue job or local clones were used.
+There is no new Confer snapshot ID; the immutable GitHub pins are the evidence.
+This document update refreshes only the Ledger comparison to merged code; it
+does not perform new upstream research or change the upstream pins.
+
+| Repository | September 7 pin | Refresh pin | Repository delta |
+| --- | --- | --- | --- |
+| dicklesworthstone/coding_agent_session_search, called `cass` in Confer | `a18a6261a9815204b6ef59f9d186370ebd35dc34` | `e5a492b47bfeef8400e326a8fa03b2870dc0dc87` | 46 commits ahead |
+| kenn-io/agentsview | `2298453833a5c5126a486a2554eac6c35cd2b55f` | `b95262043e49c20108b73f825b3bb26cbf591b58` | 13 commits ahead |
+| tommy0103/obelisk | `69d9e21a90e55f310376d8f49929b1c16f5a1f0c` | `69d9e21a90e55f310376d8f49929b1c16f5a1f0c` | Unchanged |
+| vercel/ai | `c3c189c068baefe62808b4eca5432fe092dc37cb` | `46cea633cf7286386eb4e70f9c04413c57723d35` | 15 commits ahead |
+
+GitHub blob-ID comparisons found the previously cited Cass
+`src/indexer/lexical_reconcile.rs`, AgentsView
+`internal/recall/extract/manager.go`, and AI
+`packages/harness/src/utils/sandbox-channel.ts` unchanged between their old and
+new pins. Freshly verified does not mean newly implemented.
+
+This pass did not inspect private transcript bodies, credentials or runtime
+databases; execute models; run upstream or Ledger tests; or measure performance.
+Trajectory and Letta Code are owned by the separate research worker and are not
+re-researched here. No implementation or upstream source was copied into Ledger.
+
+## Ledger comparison baseline
+
+Final merged main: `56b51e2b76d8d37f749bb114f5b73f1e40164cf9` (PR6 merge),
+including PR2 merge `7fe199ce9d62e4a9e966aff18b7c33eda917ac40`.
+The follow-up branch contains this main commit. The
+[pinned integration report](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/docs/pr2-pr6-integration.md)
+identifies implementation commit `9f52f8a` and records these verified contracts:
+
+- **Schema and identity:** schema 6 distinguishes main-v3, PR2 identity-v4 and
+  PR6 parentage-v4/cursor-v5 layouts by shape. Migration preserves source-scoped
+  sessions, records, FTS, embeddings, parent links and recovered prompt-history
+  identities without double-prefixing. The conflicting v4 meanings are resolved
+  integration history, not an outstanding migration proposal.
+- **Atomic ingestion and replacement:** [records and cursors commit together](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/db.ts#L475-L494).
+  Existing sessions remain searchable while a replacement is staged, including
+  after restart. [Completed promotion](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/db.ts#L556-L618)
+  is transactional and streams staged records in bounded batches. This protects
+  recall during rebuilding, not every historical source generation afterward.
+- **JSONL continuity and limits:** [defaults are a 1 MiB chunk target and a 64 MiB
+  individual-record ceiling](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/ingest.ts#L11-L31).
+  Precise file identity and prefix hashes validate continuity. Incomplete lines
+  do not advance progress; oversized rows warn without consumption. Raising the
+  configured ceiling or ingestion version retries cached failures. These input
+  limits are not peak-RSS guarantees.
+- **Lineage and ordering:** source-snapshot-bound Claude classification, including
+  a sidechain-only first chunk, source-backed parent corrections, whitespace
+  rejection, native OMP identities and diagnostic routing have regressions.
+  Cursor/Copilot ordering uses verified prefix line counts without changing
+  canonical record identities or byte anchors.
+- **Watch and discovery:** [startup/periodic reconciliation and asynchronous
+  shutdown drain](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/watch.ts#L190-L255),
+  missed-root discovery, import budgets, append events and enrichment warnings
+  are covered. [Discovery paginates upstream 1,000-item pages](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/scanner.ts#L52-L80).
+  Pagination and a per-tick budget are not a durable fairness frontier.
+
+The integration report records `bun test`: **152 passed, 0 failed, 959 assertions
+across 17 files**, and `bun run typecheck` passed. These were bounded synthetic
+store tests, not broad personal-history normalization. This documentation update
+checked the report and final source; it did not rerun that suite or observe a
+daemon. Runtime installation and observation remain separate evidence.
+
+The earlier [PR6 ingest findings at `1efa46d`](https://github.com/archeism/ledger/blob/1efa46d9ac7164f2c32b2a47957b6ea732322f8d/packages/core/src/ingest.ts#L95-L217)
+about mtime/size-only skipping and separately persisted cursors no longer describe
+the merged path. Remaining limits from the report still apply: this is a mutable
+snapshot index, non-JSONL adapters may read whole inputs, changed-source prefix
+verification and session hashing require history-linear work, and first-chunk
+metadata/synthesized timestamps do not universally match whole-file normalization.
+
+## Refreshed source findings
+
+### AgentsView: capture, checkpoints and ownership are separate boundaries
+
+The [raw-file capture path](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/internal/rawcapture/files.go#L160-L259)
+copies into a temporary object, hashes and syncs it, and revalidates the source
+handle/path and digest. [Capture publication](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/internal/rawcheckpoint/outbox.go#L655-L790)
+checks reservation ownership and predecessor identity, then records the
+generation, object references and source progress in one write transaction.
+This is stronger evidence than a watcher notification or an updated mtime.
+Its outbox and garbage-collection policy are not Ledger's perpetual retention
+policy; borrow the publication boundary, not assumed archive semantics.
+
+New [parser-checkpoint validation](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/internal/sync/checkpoint.go#L115-L232)
+compares the committed database prefix, source identity, size and tail anchor
+before resuming. Checkpoint/hash state is loaded lazily for append. The fast path
+uses append-trust assumptions, not a proof that every old byte is unchanged;
+the [audit worker](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/cmd/agentsview/sync_worker.go#L191-L205)
+bypasses that shortcut for content verification. The
+[review regression suite](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/internal/sync/checkpoint_review_test.go#L192-L320)
+contains stale-checkpoint and cold-restart cases; this pass read, but did not run,
+them.
+
+[Writable open](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/cmd/agentsview/main.go#L1382-L1406)
+acquires ownership before opening the database. Its
+[worker handoff](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/cmd/agentsview/worker_pass.go#L228-L297)
+closes the writer, releases ownership and reacquires it before reopening.
+[Periodic reconciliation and the separate audit](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/cmd/agentsview/main.go#L2660-L2704)
+are distinct work. Neither an in-process mutex nor an OS supervisor alone gives
+every CLI and daemon writer this ownership protocol.
+
+### Cass: describe incomplete answers and validate the served index
+
+The new [hollow-generation verdict](https://github.com/dicklesworthstone/coding_agent_session_search/blob/e5a492b47bfeef8400e326a8fa03b2870dc0dc87/src/search/asset_state.rs#L1517-L1553)
+compares live-document metadata with a completed, compatible rebuild checkpoint.
+Missing evidence gives no verdict; a severe deficit invalidates readiness.
+The [GH457 regression](https://github.com/dicklesworthstone/coding_agent_session_search/blob/e5a492b47bfeef8400e326a8fa03b2870dc0dc87/tests/gh457_hollow_generation.rs#L165-L286)
+constructs a near-empty published index while its checkpoint still looks complete.
+This is a useful failure case, not comprehensive content-integrity verification.
+
+The [search cursor manifest](https://github.com/dicklesworthstone/coding_agent_session_search/blob/e5a492b47bfeef8400e326a8fa03b2870dc0dc87/src/lib.rs#L33623-L33765)
+exposes exact versus lower-bound counts, returned counts, field projection,
+truncation, partial index state and requested versus realized retrieval mode.
+The cursor itself encodes offset/limit, not a query-bound immutable snapshot.
+Its continuation caveats should not be upgraded into a snapshot guarantee.
+The earlier [source-shape recovery check](https://github.com/dicklesworthstone/coding_agent_session_search/blob/e5a492b47bfeef8400e326a8fa03b2870dc0dc87/src/indexer/lexical_reconcile.rs#L213-L262)
+is unchanged and is still not a cryptographic source-content identity.
+
+### Obelisk: behavioral reference only
+
+At the unchanged pin, the [writer lease](https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/writer-lease.ts#L54-L93)
+and [provider persistence callbacks](https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/provider-indexing.ts#L254-L300)
+make ownership and commit ordering explicit. The
+[subagent/workflow record types](https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/types.ts#L150-L203)
+represent contributors arriving in different orders, with optional evidence.
+The [Codex projection](https://github.com/tommy0103/obelisk/blob/69d9e21a90e55f310376d8f49929b1c16f5a1f0c/packages/core/src/providers/codex.ts#L174-L226)
+folds child messages under a parent session while retaining an agent identity.
+That projection is not a requirement to collapse Ledger's canonical child IDs.
+No code, SQL, implementation structure or dependency is adopted from Obelisk.
+
+### AI Harnesses: replay identity is not archival commitment
+
+The repository delta includes Pi's [filesystem round-trip identity probe](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/packages/harness-pi/src/pi-session.ts#L285-L314),
+not a new durable capture contract. The existing
+[bridge log](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/packages/harness/src/bridge/index.ts#L437-L519)
+mirrors events to disk asynchronously, suppresses append failures and retains a
+turn-scoped replay log. [Monotonic sequence/replay](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/packages/harness/src/bridge/index.ts#L581-L603)
+is useful consumer input identity, not proof of committed Ledger storage.
+[Suspension](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/packages/harness/src/utils/sandbox-channel.ts#L289-L325)
+freezes delivery without cancelling the bridge turn; a pinned checkpoint may
+cause an already-delivered tail to replay.
+
+The [disk-replay test](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/packages/harness/src/bridge/disk-replay.integration.test.ts#L62-L145)
+checks tail replay after the log has been written, not survival of every
+unflushed event. Claude's adapter [drops messages with a parent tool-use ID](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/packages/harness-claude-code/src/bridge/create-emit-stream-event.ts#L213-L218).
+Native-source capture remains necessary for child execution evidence.
+
+## Four minimal adaptations and test gates
+
+### 1. Retain evidence before advancing capture progress
+
+Start with one JSONL source: retain a captured snapshot or bounded captured
+chunks, digest and source-generation identity before normalization progress can
+advance. Replacements must create attributable generations, not erase prior
+evidence. Keep search rebuildable. The merged
+[staging tables](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/schema.ts#L97-L125)
+retain canonical replacement records until promotion, not raw bytes or historical
+generations after promotion. Prefix hashes do not supply that retention boundary.
+Use AgentsView's capture-publication behavior as the implementation reference,
+without importing its replication/outbox subsystem.
+
+Gate: synthetic truncation, same-size rewrite, replacement before append,
+mutation during capture, duplicate import, and crash between object installation
+and metadata publication. Delete the synthetic original and rebuild search from
+retained evidence. Check object and directory durability where applicable.
+See the [upstream replacement fixtures](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/internal/rawcapture/capturer_test.go#L976-L1058).
+Define explicit deletion, exclusion and capacity behavior; source disappearance
+is not authorization to erase captured history.
+
+### 2. Add participating-writer ownership around the merged atomic cursor
+
+Preserve the merged records/cursor transaction, staged replacement and input
+limits; do not add a competing ingestion path. Ingestion-version and configured
+ceiling changes already invalidate cached failures. The remaining adaptation is
+to inventory mutating CLI, import, migration, watch and enrichment entry points
+and give them one cross-process ownership contract. Extend progress invalidation
+only for additional configuration that changes normalization semantics. Use
+AgentsView for implementation reference, not Obelisk-derived code.
+
+New gate: two processes targeting the same database, loss of writer ownership
+and recovery without admitting an overlapping writer. Preserve the integration's
+regressions for atomic rollback, restart during staged replacement, partial
+multibyte/JSONL tails, oversized-row handling, version/ceiling changes and the
+schema-6 layout matrix, including scoped parent/cursor IDs, FTS and foreign keys.
+Embedding failure must not undo committed ingestion. Neither the input ceilings
+nor the passing synthetic suite establish bounded end-to-end memory or IO.
+
+### 3. Persist discovery work and reserve backfill capacity
+
+Separate watcher hints, resumable discovery and the per-tick normalization
+budget. Raising a scan limit or adding pagination within one invocation is not
+durable progress across restarts. The merged watcher still
+[selects a prefix of currently needed work](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/watch.ts#L401-L417).
+The integration report explicitly leaves durable fairness unresolved. A small
+persisted pending-work table with stable enumeration progress and an explicit
+share for older work is preferable to repeatedly rediscovering only the newest
+window.
+
+AgentsView's [keyset candidates and transactional completion](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/internal/db/project_identity_backfill.go#L218-L325)
+provide a bounded precedent for identity backfill, not evidence of universal
+transcript-ingestion fairness. Ledger must define its own scheduling policy.
+
+Gate: more than 1,000 synthetic sessions, a tiny import budget, continuous writes
+to newest sessions, restart mid-page, a repeatedly failing item and an unavailable
+root. The oldest eligible work must eventually finish while live capture still
+progresses. Report enumeration-incomplete, pending and failed states; do not
+label coverage complete merely because one bounded pass returned no new work.
+
+### 4. Expose record evidence, lineage uncertainty and search completeness
+
+Add an opt-in versioned response envelope so existing robot JSONL consumers need
+not break. Return an explicit empty result, `record_id`, retained source locator,
+generation identity, returned count and count precision, partial/failed/pending
+coverage, and requested/realized retrieval mode. The merged
+[robot projection](https://github.com/archeism/ledger/blob/56b51e2b76d8d37f749bb114f5b73f1e40164cf9/packages/core/src/cli.ts#L437-L461)
+still drops record identity and emits no JSONL for zero hits. If pagination is
+added, bind it to query, filters and a defined snapshot/generation, or explicitly
+declare best-effort semantics.
+
+Keep parentage evidence origin and conflict/unknown state rather than making
+nullable display columns the complete relationship contract. Source metadata,
+path-derived layout and a tool-call link are different evidence strengths;
+shared directory proximity alone does not establish delegation. Do not import
+Obelisk's parent-folding projection or assume an AI Harness stream includes all
+children. A later live adapter should scope replay identity by session and bridge
+incarnation as well as sequence, and persist its consumer cursor with evidence.
+
+Gate: empty versus incomplete output, JSON parseability under truncation,
+canonical-record/FTS discrepancy, lexical access during embedding failure,
+cross-provider native-ID collision, child-before-parent arrival, conflicting
+lineage evidence and a cycle. A later live adapter needs duplicate-tail replay,
+sequence-reset and disconnect-without-cancellation fixtures before adoption.
+
+## Licensing and publication boundary
+
+- Obelisk is AGPL-3.0; design reference only. No copying, vendoring or derived
+  implementation without the owner's explicit licensing decision. Rewriting
+  code in another language would not establish independent provenance.
+- Cass's [current LICENSE](https://github.com/dicklesworthstone/coding_agent_session_search/blob/e5a492b47bfeef8400e326a8fa03b2870dc0dc87/LICENSE#L1-L44)
+  includes an OpenAI/Anthropic rider. Do not describe it as ordinary MIT or treat
+  this research as code-reuse clearance. Its observations and failure scenarios
+  are not a proposal to import its engine or implementation.
+- AgentsView's [root license](https://github.com/kenn-io/agentsview/blob/b95262043e49c20108b73f825b3bb26cbf591b58/LICENSE)
+  is MIT; AI's [root license](https://github.com/vercel/ai/blob/46cea633cf7286386eb4e70f9c04413c57723d35/LICENSE)
+  identifies Apache-2.0. Verify the actual adopted files/dependencies and retain
+  applicable notices before any future implementation reuse.
+
+This note owns only this file on `docs/daemon-observation-and-inspirations`.
+It does not update the other worker's research or claim the integrated daemon is
+running, archival, single-writer, fair or memory-bounded. PR2 and PR6 are now both
+merged, and this comparison is pinned to their final integrated main. Runtime
+observation and authorization of a separate follow-up PR remain independent of
+the recorded integration test result. No push or PR is part of this document
+update. New proposal gates above have not been executed here; existing regression
+results are attributed to the integration report.