sessionmem 1.7.0 → 1.8.0

package/docs/automatic-memory.md

@@ -0,0 +1,86 @@
+# Automatic capture and recall
+
+Install once and restart the host so it loads its MCP configuration and guidance.
+The installed flow does not ask the user to run scans, recall commands or save
+commands during ordinary agent work. Host tool permissions must allow the memory
+tools. `sessionmem doctor --adapter <name>` checks the configured flow and gives
+repair hints for missing hooks, stale instructions, disabled MCP wiring and stale
+watcher checkpoints.
+
+## Host audit and behavior
+
+| Host | Capture after install | Session-start recall | Recall on prompts |
+| --- | --- | --- | --- |
+| Claude Code | PostToolUse captures Bash/Edit/Write/MultiEdit; SessionEnd summarizes; login watcher also imports transcripts | SessionStart hook catches up transcripts and injects context | UserPromptSubmit hook retrieves context for the submitted prompt |
+| Codex | Login-started watcher imports local rollout transcripts | Global AGENTS.md and MCP instructions tell the agent to search before work and fetch selected details | Instructions tell the agent to search for a new topic |
+| Cursor | Login-started watcher imports supported local composer SQLite records | Project always-apply rule and global MCP instructions tell the agent to search before work | Same instruction-driven topic recall |
+| Windsurf | Agent calls storeMemory/batchStoreMemory under global rules; no transcript importer | Global rules and MCP instructions | Instruction-driven topic recall |
+| Cline | Agent calls storeMemory/batchStoreMemory under installed rules; no transcript importer | Installed rules and MCP instructions | Instruction-driven topic recall |
+| Antigravity, QCoder adapters | Agent tool saves when the host loads the configured MCP server and guidance; no transcript importer | Instruction-driven; existing adapter file paths are unverified against the native products | Instruction-driven; host loading must be verified |
+| Generic MCP | Agent tool saves when the host reads project AGENTS.md or MCP instructions; no transcript importer | Instruction-driven, dependent on host support | Instruction-driven topic recall |
+
+Only Claude Code has deterministic host hooks in this integration. Instruction-
+driven recall is configured by install, but doctor cannot prove that a model
+executed the instructions or that an unverified native host loaded a guessed
+configuration path. Doctor marks Antigravity/QCoder automation unestablished and
+gives a native-settings verification hint. Its output identifies this distinction. The tests simulate
+an agent following the installed instructions; they do not launch proprietary
+host interfaces or claim that every model obeys instructions.
+
+The watcher starts immediately at install and at login through launchd on macOS,
+a user systemd service on Linux, or Task Scheduler on Windows. It polls supported
+transcripts every five seconds by default. Capture requires readable transcript
+files in a supported format. A missing directory is valid before the first
+session; an unsupported format or unreadable source is a diagnostic failure.
+No user action is needed to import later sessions while the service is healthy.
+
+Claude Code SessionStart and a new MCP server connection also perform an
+incremental catch-up. A session opened immediately after another session can
+therefore recall newly written transcript events before the next watcher poll.
+The MCP server advertises the startup, new-topic recall and decision-save flow
+in its protocol instructions for all hosts. Existing user prose is preserved
+when managed guidance blocks are updated.
+
+Cursor rules are project scoped. The old home-level `.cursor/rules` location did
+not establish a rule for arbitrary projects. Install now writes the rule in the
+current project, repairs `alwaysApply:true`, and supplies the same flow through
+the global MCP server's instructions in other projects. A host that ignores MCP
+instructions needs an active native rule in that project.
+
+Plain-terminal install detection recognizes Claude Code, Codex, Cursor and
+Windsurf CLI executables when on PATH, plus the documented native Claude paths.
+Other hosts still need selection at setup with `--adapter`. Configuration
+directories alone are not treated as evidence of an installed host.
+
+Cline's installed MCP entry preapproves the local search, detail and capture tools
+so these calls do not require a separate user action each time. Deletion and
+external operations are excluded. Other hosts retain their own permission rules.
+
+## Diagnosis and repair
+
+Doctor separately reports hook definitions, MCP startup and tool availability,
+current agent instructions, capture mode and recall mode. For hookless hosts it
+says that recall is instruction-driven and execution is unverified. For hosts
+without transcript support it says that capture consists of agent-driven saves,
+not background transcript import. Hook and guidance configuration are checked
+against the current installed version. Failed guidance writes make install fail
+instead of printing a successful setup message.
+
+Doctor's MCP probe bypasses startup capture so diagnosis does not import
+transcripts or alter the user's memories. Its isolated storage canary rolls back.
+The watcher-health check requires a recent, successful checkpoint for the selected
+database, rather than assuming that a registered service is running.
+
+## CI evidence
+
+`tests/integration/cli/automatic-memory-flow.spec.ts` runs in the CI OS matrix.
+It uses a temporary home and project. The OS supervisor commands are simulated,
+but the registered watcher command performs real transcript ingestion.
+
+The Claude case installs the actual hook definitions, feeds capture events,
+ends the session, starts a new session, and verifies the startup and prompt
+context envelopes contain the stored database decision. The Codex and Cursor
+cases create real-format transcript fixtures, install and run the automatic
+watcher sweep, run doctor, then open a new MCP session. A simulated agent follows
+the installed instructions and verifies selected full details reach its context.
+No manual import or scan command is used.

package/docs/benchmark.md

@@ -5,6 +5,13 @@
 [benchmarks/README.md](../benchmarks/README.md) for the corpus license, selection
 rules, metric definitions, and detailed reproduction instructions.
 
+## Compact tools, temporal retrieval and entity fusion
+
+See the [retrieval evaluation](../benchmarks/retrieval-gaps.md) for fresh 1.7.0
+baselines, per-category before/after tables, complete regression artifacts and
+measured compact-tool token costs. Temporal and source-informed additive experiments remain off by default. Entity
+coverage improves default BM25; optional hybrid results are preserved.
+
 ## Local semantic and competitor comparison
 
 The [measured comparison](../COMPARISON.md) records a new unchanged-main run,

package/docs/local-semantic.md

@@ -1,6 +1,8 @@
 # Optional local semantic retrieval
 
-The default is BM25 plus entity overlap. To add a learned semantic candidate pool,
+The default is BM25 with entity-link coverage. Active MiniLM retains the existing
+entity-overlap fusion because stronger coverage lowered its hit rate and recall.
+To add a learned semantic candidate pool,
 install the optional packages beside sessionmem and enable it:
 
 ```bash

package/docs/retrieval-tools.md

@@ -0,0 +1,144 @@
+# Compact retrieval and optional ranking
+
+## Search, inspect nearby context, then fetch details
+
+The MCP server exposes these tools alongside the existing tools. Project scope
+comes from the server, so callers do not supply `projectId` over MCP.
+
+| Tool | Input | Output |
+| --- | --- | --- |
+| `searchMemoryIndex` | `query`, optional `limit`, `includeHistory`, `tags` | IDs, one-line summaries, kind, capture date, numeric score, `hasMore` |
+| `memoryTimeline` | `memoryId`, or both `fromDate` and `toDate`; optional `limit`, `includeHistory` | Compact rows in capture order; scores are null |
+| `getMemory` | `memoryId` | One complete stored memory, including content and metadata |
+
+Start with `searchMemoryIndex({"query":"database migration","limit":5})`.
+Inspect the surrounding records with
+`memoryTimeline({"memoryId":"selected-id","limit":5})` when chronology matters.
+Fetch only the records needed with `getMemory({"memoryId":"selected-id"})`.
+The existing get-by-ID tool already returns full bodies, so no duplicate tool is
+needed. Existing `retrieveMemories` remains available.
+
+Index and timeline limits default to 10 and accept integers from 1 to 50. Summaries
+are extractive previews of stored text, with whitespace collapsed and at most 160
+characters. They can omit qualifiers from longer memories; read the full body
+before relying on a fact. No summaries or inference are generated for these tools.
+Search scores are rounded to four decimal places. Search uses the same ranker as
+`retrieveMemories`, without its startup formatting or access-counter updates.
+
+Index `hasMore` describes the bounded retrieval candidate pool, not an exhaustive
+store count. Timeline `hasMore` means more visible records exist outside the
+returned window. Around an ID, it balances earlier and later records and fills
+remaining slots from the available side. Timestamp ties use ID ordering. Ranges
+are inclusive UTC ISO timestamps, for example:
+
+```json
+{"fromDate":"2026-01-01T00:00:00Z","toDate":"2026-01-31T23:59:59.999Z","limit":10}
+```
+
+Timeline dates are **capture dates**, not dates mentioned inside prose. Timeline
+includes superseded records by default so it can explain changes; set
+`includeHistory:false` for current records only. Index excludes fully superseded
+records by default. All three tools enforce project scope and expiry. Compact
+search and timeline do not update access counters. Optional local semantic search
+can still backfill its derived vector index, as with ordinary retrieval.
+
+## Optional temporal retrieval
+
+`temporalRetrieval` defaults off. It did not improve the pinned-clock LoCoMo run.
+Enable it for stores with meaningful observation times:
+
+```bash
+sessionmem config set temporalRetrieval true
+```
+
+A deterministic English cue detector recognizes current state (`current`,
+`currently`, `now`, `latest`, `today`, `at present`) and historical questions
+(`before`, `previously`, `formerly`, `prior`, `used to`, `no longer`, `when did`,
+`when was`, `when were`, `in the past`, `historical`, `history`). Dated clauses take
+precedence over current-state cues. With no recognized cue, the previous retrieval
+behavior and scores stay unchanged, even when enabled.
+
+Supported ranges are `YYYY-MM-DD`, `YYYY-MM`, `in YYYY`, `during YYYY`, `yesterday`,
+`last/previous week/month/year`, and numeric day/week offsets such as `2 days ago`.
+Relative dates use the injected retrieval clock in UTC; weeks start on Monday and
+months are calendar months. The detector does not interpret arbitrary natural
+language dates, future plans, locale-specific dates or conversational reference
+times. An invalid date supplies no date signal.
+
+Historical intent includes superseded candidates automatically unless the caller
+explicitly sets `includeHistory:false`. That decision applies to both lexical and
+optional semantic candidates. Explicit `true` also works for current-state queries.
+Existing supersession rules still determine which facts have changed; no new
+contradiction inference is added.
+
+Within relevance, temporal scoring uses a fixed 90% existing relevance and 10%
+time signal. Current-state signals average the non-superseded claim fraction and
+freshness, `1 / (1 + ageDays / 30)`. Historical ranges use the distance from the
+range with the same 30-day scale. `before` and similar previous-state cues favor
+the superseded fraction. Undated historical questions have a neutral time signal.
+The time is the latest extracted claim observation timestamp, falling back to
+original capture time. Editing a row does not invent a new event date.
+
+Past queries neutralize ordinary recency and inactivity-based importance decay,
+so age alone does not suppress historical evidence. Stored importance and access
+boosts still apply. Dates rerank candidates; they are not hard date filters and do
+not guarantee that a relevant memory outside the candidate pool will be found.
+For exact capture-date browsing, use `memoryTimeline`.
+
+## Optional entity-link boost
+
+Entity coverage is enabled by default when retrieval uses BM25, including when
+optional inference is unavailable. It improves all three primary LoCoMo metrics
+in that mode. With active MiniLM it defaults off because hit rate and recall
+regressed. An explicit `entityLinkBoost` value overrides that mode-specific
+choice. To enable it explicitly:
+
+```bash
+sessionmem config set entityLinkBoost true
+```
+
+The query links literal normalized name spans against the existing
+`memory_entities` tables. Explicit aliases cover `postgres`/`postgresql`,
+`node`/`node.js` and `k8s`/`kubernetes`. There is no fuzzy name matching, pronoun
+resolution, entity embedding, or graph traversal. Extraction rules stay unchanged.
+Project, expiry and requested history visibility apply before link scoring.
+
+Each linked `(kind, name)` receives weight
+`1 + log((visibleMemoryCount + 1) / (matchingMemoryCount + 1))`. A memory's entity
+score is its matched weight divided by the total linked weight. Up to 50 highest
+entity-score IDs are admitted to the candidate union, with ID tie breaks. This
+channel can admit rows that were absent from the BM25 or semantic pool.
+
+The fixed fusion is 80% base relevance and 20% linked-entity score. With semantic
+search enabled, base relevance retains the existing 2:1 BM25/positive-cosine blend.
+When no entity is linked, the prior scoring path is retained. The existing 10%
+entity-overlap signal remains in use when coverage is off. Temporal
+fusion, when enabled too, follows entity fusion. Outer relevance, recency and
+importance weights remain 60%, 25% and 15%.
+
+Disable coverage with `sessionmem config set entityLinkBoost false` and temporal
+retrieval with `sessionmem config set temporalRetrieval false`.
+
+## Source-informed fusion alternatives
+
+The optional `retrievalRanking` setting accepts `legacy`, `adaptive-bm25`,
+`hub-additive` and `adaptive-hub-additive`. `legacy` means the existing BM25/dense
+fusion, with the mode-specific entity-coverage default above. Other modes default
+to their stated experiment without coverage, unless explicitly overridden.
+
+`adaptive-bm25` uses a query-length adaptive sigmoid on raw positive FTS5 BM25.
+The additive modes use a hub-damped maximum entity boost from at most eight names,
+plus lexical and optional dense signals, divided by the maximum score of active
+channels. A 0.2 semantic gate applies before additive fusion. The adaptive additive
+variant also enables the sigmoid. These hypotheses did not pass the three-metric
+regression gate and remain opt-in. See the
+[source notes](../benchmarks/retrieval-source-notes.md) for exact parameters,
+algorithm differences and licensing scope. No competitor source or prompt was
+adapted. The current defaults select entity coverage for BM25 and the existing
+fusion for MiniLM; there are no per-query-label or per-category exceptions.
+
+The temporal and entity settings work alongside the existing `semanticSearch`
+setting. Results,
+per-category losses and token measurements are in the
+[retrieval evaluation](../benchmarks/retrieval-gaps.md). They do not establish
+accuracy for generated answers or gains on coding workloads.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sessionmem",
-  "version": "1.7.0",
+  "version": "1.8.0",
   "private": false,
   "type": "module",
   "description": "Local-first MCP memory layer for coding agents across Claude Code, Codex, Cursor, Cline, Windsurf, and other MCP-compatible hosts.",