@tznthou/ccrecall 0.9.2 → 0.10.0

package/README.md

@@ -7,7 +7,7 @@
 
 [中文版](README_ZH.md)
 
-A local memory service for Claude Code — indexes your conversation history, recalls relevant context on demand, and injects it into future sessions. Zero API cost for the core service; optional post-session extraction via Haiku adds ~$0.001/session.
+A local memory service for Claude Code — indexes your conversation history, recalls relevant context on demand, and injects it into future sessions. Zero API cost for the core service. Optional post-session extraction runs on Sonnet: plan usage on a Claude subscription, or $0.21–0.39 per extraction (measured) with an API key.
 
 ---
 
@@ -18,7 +18,7 @@ Five stances, non-negotiable:
 1. **Local-first.** Your machine, your data. No cloud, no accounts, no services beyond localhost.
 2. **SQLite is the API.** One `.db` file. `sqlite3` to inspect anything. Your memories are never a black box.
 3. **FTS5, not vectors.** Code conversations search for file paths, error messages, tool names — keywords, not vibes. At hundreds of sessions, embeddings solve a problem that doesn't exist.
-4. **LLM distills, human curates.** Haiku extracts post-session (~$0.001). Auto memory holds decisions you name. ccRecall holds the long tail no one writes down by hand.
+4. **LLM distills, human curates.** Sonnet extracts post-session (plan usage on a subscription, $0.21–0.39 a run on an API key). Auto memory holds decisions you name. ccRecall holds the long tail no one writes down by hand.
 5. **Read-only toward `~/.claude/`.** Reads your JSONL logs, never writes to Claude Code's state. Worst case: bad search results. Never: a broken setup.
 
 ---
@@ -96,7 +96,7 @@ flowchart TB
 
     subgraph Consumers["Consumers"]
         Hook["Claude Code Hooks<br/>SessionStart / UserPromptSubmit / SessionEnd"]
-        Wrapper["Extraction wrapper<br/>post-session Haiku pass"]
+        Wrapper["Extraction wrapper<br/>post-session Sonnet pass"]
         MCP["MCP Server<br/>recall_query / recall_save"]
     end
 
@@ -119,7 +119,7 @@ sequenceDiagram
     participant H as Hooks
     participant R as ccRecall
     participant W as Wrapper
-    participant X as Haiku extractor
+    participant X as Sonnet extractor
 
     rect rgb(235, 244, 255)
     Note over CC,R: Session start
@@ -163,7 +163,7 @@ The `notBefore` gate keeps a stale session from being extracted twice; the subag
 | FTS5 | Full-text search | Built into SQLite, trigram tokenizer with LIKE fallback for short CJK / mixed-script queries |
 | Native `http` | HTTP server | No Express — minimal surface, localhost only |
 | chokidar | Filesystem watcher | Cross-platform JSONL change detection with 2 s debounce + single-flight |
-| vitest | Testing | 651 tests across 40 files, integration-style |
+| vitest | Testing | 858 tests across 54 files, integration-style |
 | `@modelcontextprotocol/sdk` | MCP server | stdio transport, shared SQLite via WAL |
 
 ---
@@ -281,7 +281,7 @@ ccRecall lives alongside Claude Code's built-in auto memory (`~/.claude/projects
 
 |  | auto memory | ccRecall |
 |---|---|---|
-| **Write path** | Claude curates by hand — new `.md` file + MEMORY.md index line | Two tracks: rule-based session summaries (automatic, free) + opt-in Haiku extraction (~$0.001/session, where most searchable memories come from) |
+| **Write path** | Claude curates by hand — new `.md` file + MEMORY.md index line | Two tracks: rule-based session summaries (automatic, free) + opt-in Sonnet extraction (plan usage, or $0.21–0.39 a run on an API key; where most searchable memories come from) |
 | **Read path** | Always in session context (MEMORY.md loads at session start) | On-demand MCP query when auto memory has no entry |
 | **Signal density** | High — facts worth naming | Long tail — everything the hook can extract |
 | **Typical use** | "Remember X" / "always Y" — durable preferences, decisions | "Didn't we fix that?" / "last time" — cross-session recall |
@@ -395,7 +395,7 @@ ccRecall/
 │   ├── tutorial.md               # End-user walkthrough (install → MCP → usage)
 │   ├── architecture.md           # Daemon design rationale (contributor-oriented)
 │   └── launchd.md                # macOS LaunchAgent install/troubleshoot
-├── tests/                        # 651 tests across 40 files (parser, scanner,
+├── tests/                        # 858 tests across 54 files (parser, scanner,
 │   │                             # summarizer, database, indexer, e2e, MCP,
 │   │                             # memories, hooks, watcher, CLI, migrations,
 │   │                             # FTS5 CJK edge cases, integrity monitor, ...)
@@ -426,7 +426,7 @@ The real trigger was simpler: I kept re-explaining the same architecture to Clau
 
 Each flows from the manifesto. Each was a conscious choice against a popular alternative.
 
-**Rule-based, with a ceiling.** The daemon uses heuristic extraction (regex patterns, tool usage analysis, outcome inference) for session summaries — zero API cost. For that job, "Edit x8, 5 files, committed" is more useful than a paragraph of prose. But rule-based has a ceiling: it handles structured signals (tool calls, file edits, commit messages) well; it misses discussion-only decisions, nuanced trade-offs, and anything that lives in natural language. That ceiling is why v0.4.1 added post-session Haiku extraction (~$0.001/session) — an LLM pass that catches what heuristics can't. In practice, the memories I actually search for mostly come from Haiku extraction or manual `recall_save`, not the rule-based layer. The daemon itself never calls an LLM; the LLM pass is opt-in and runs outside the daemon as a separate process.
+**Rule-based, with a ceiling.** The daemon uses heuristic extraction (regex patterns, tool usage analysis, outcome inference) for session summaries — zero API cost. For that job, "Edit x8, 5 files, committed" is more useful than a paragraph of prose. But rule-based has a ceiling: it handles structured signals (tool calls, file edits, commit messages) well; it misses discussion-only decisions, nuanced trade-offs, and anything that lives in natural language. That ceiling is why v0.4.1 added post-session Haiku extraction (~$0.001/session) — an LLM pass that catches what heuristics can't. In practice, the memories I actually search for mostly come from that post-session pass (on Sonnet since v0.9.0) or manual `recall_save`, not the rule-based layer. The daemon itself never calls an LLM; the LLM pass is opt-in and runs outside the daemon as a separate process.
 
 **FTS5, not vector search.** Semantic search sounds better on paper, but for conversation logs — specific tools, file paths, error messages — keyword matching wins. FTS5 queries run in <10ms locally. No embedding model, no Chroma, no Docker container. At hundreds of sessions (not millions of documents), Karpathy's own analysis confirms: "plain index + keyword search is already sufficient under 500 sources."
 
@@ -438,6 +438,15 @@ Each flows from the manifesto. Each was a conscious choice against a popular alt
 
 **No opinionated injection.** ccRecall doesn't decide what Claude should remember. It provides a search API — the injection layer presents results, Claude integrates them. Opinionated memory selection is a premature optimization that would be wrong in ways we can't predict.
 
+### What the transcript doesn't say
+
+ccRecall knows only what Claude Code writes to disk, and the transcript is not quite the conversation. Neither of the gaps v0.9.2 closed ever raised an error:
+
+- A message typed while Claude is working usually gets folded into Claude's next API call and stored as an attachment, with no user message on record. The extraction transcript never read those — 254 of them in a month of one person's sessions — until v0.9.2.
+- Claude Code 2.1.300 began writing a copy of each API request into the transcript, and the parser counted those records as messages until v0.9.2.
+
+The code reading the transcript was right for the format it assumed. Both gaps turned up by counting what the file holds against what was really said.
+
 ---
 
 ## Roadmap
@@ -449,6 +458,11 @@ Each flows from the manifesto. Each was a conscious choice against a popular alt
 | **v0.5.0** | The knife field: journal/scorer/harvester pipelines removed (zero promotions ever), endpoints 13 → 5, `message_uuids` dual-hash rebuild (DB 114MB → 42MB) | Released |
 | **v0.5.2** | Recall weighting: log-compressed half-life decay, FTS relevance-first ranking via `(-rank)*sqrt(EC)` | Released |
 | **v0.5.3** | CJK topic alignment: Han-aware topic extraction (`\p{Script=Han}` tokenizer + 32 Chinese stopwords + particle split), session-less rebuild fix | Released |
+| **v0.5.4–v0.5.6** | Measuring injection: an `injection_log` table records every memory surfaced (#71), Tier 0 startup slots rotate instead of showing the same three memories every session, `/health` reports `mainSessionCount` | Released |
+| **v0.6.0** | Mid-conversation recall: a `UserPromptSubmit` hook surfaces memories while you work, injected lines carry their key, and extraction flags runs that printed `recall_save` instead of calling it | Released |
+| **v0.7.x** | CJK word segmentation via `Intl.Segmenter` (#80); the `cwd` → project id encoding drift that silently skipped extraction for whole projects (#89); English function words no longer run the topic index (#84); the telemetry log guard's leaks closed | Released |
+| **v0.8.x** | Provenance: a memory records the message it came from and who wrote it, so extraction can't overwrite one you saved by hand; prescription-first extraction; Tier 0 picks other projects' memories per project instead of by confidence alone. Long sessions go to extraction over stdin instead of failing silently, and `backfill-extract.sh` recovers the ones already lost | Released |
+| **v0.9.x** | Extraction on Sonnet, retried once when a run produces nothing usable; no more skipped sessions when a path's letter case differs (#131); Claude Code 2.1.300's request-replay records skipped, and messages sent mid-turn now reach extraction; extraction runs without memories injected into it; `recall_save` no longer hits "database is locked" | Released |
 
 Tracked in [GitHub Issues](https://github.com/tznthou/ccRecall/issues).
 

package/README_ZH.md

@@ -7,7 +7,7 @@
 
 [English](README.md)
 
-Claude Code 的本地記憶服務——索引你的對話歷史，按需召回相關 context，注入到未來的 session。核心服務零 API 成本；選配的 post-session extraction 透過 Haiku 約 $0.001/session。
+Claude Code 的本地記憶服務——索引你的對話歷史，按需召回相關 context，注入到未來的 session。核心服務零 API 成本。選配的 post-session extraction 跑在 Sonnet 上：用 Claude 訂閱方案就算進方案用量；用 API key 的話，我們實測每次抽取 0.21–0.39 美元。
 
 ---
 
@@ -18,7 +18,7 @@ Claude Code 的本地記憶服務——索引你的對話歷史，按需召回
 1. **本地優先。** 你的機器，你的資料。不上雲、不開帳號、不信任 localhost 以外的任何服務。
 2. **SQLite 就是 API。** 一個 `.db` 檔案。`sqlite3` 隨時可查。你的記憶從不是黑箱。
 3. **FTS5，不用向量。** 程式碼對話搜的是檔案路徑、錯誤訊息、工具名稱——關鍵詞就夠了。幾百個 session 的規模，embedding 解決的是不存在的問題。
-4. **LLM 蒸餾，人類策展。** Haiku 在 session 結束後自動萃取（~$0.001）。auto memory 存你親手命名的決策。ccRecall 存沒人會手寫的長尾。
+4. **LLM 蒸餾，人類策展。** Sonnet 在 session 結束後自動萃取（訂閱方案吃方案用量，API key 實測每次 0.21–0.39 美元）。auto memory 存你親手命名的決策。ccRecall 存沒人會手寫的長尾。
 5. **對 `~/.claude/` 唯讀。** 讀 JSONL 記錄，從不寫入 Claude Code 的狀態。最壞情況：搜尋結果不好。永遠不會：搞壞你的設定。
 
 ---
@@ -94,7 +94,7 @@ flowchart TB
 
     subgraph Consumers["使用端"]
         Hook["Claude Code Hooks<br/>SessionStart / UserPromptSubmit / SessionEnd"]
-        Wrapper["Extraction wrapper<br/>session 結束後的 Haiku 抽取"]
+        Wrapper["Extraction wrapper<br/>session 結束後的 Sonnet 抽取"]
         MCP["MCP Server<br/>recall_query / recall_save"]
     end
 
@@ -117,7 +117,7 @@ sequenceDiagram
     participant H as Hooks
     participant R as ccRecall
     participant W as Wrapper
-    participant X as Haiku 抽取器
+    participant X as Sonnet 抽取器
 
     rect rgb(235, 244, 255)
     Note over CC,R: Session 開始
@@ -161,7 +161,7 @@ sequenceDiagram
 | FTS5 | 全文搜尋 | SQLite 內建、trigram tokenizer，短 token / 中英混合查詢透過 LIKE fallback 補齊 |
 | 原生 `http` | HTTP 伺服器 | 不用 Express——最小表面積、僅 localhost |
 | chokidar | 檔案系統 watcher | 跨平台 JSONL 變動偵測，2 秒 debounce + single-flight |
-| vitest | 測試 | 651 個測試（40 檔案）、整合式風格 |
+| vitest | 測試 | 858 個測試（54 檔案）、整合式風格 |
 | `@modelcontextprotocol/sdk` | MCP server | stdio transport，透過 WAL 共用 SQLite |
 
 ---
@@ -276,7 +276,7 @@ ccRecall 和 Claude Code 內建的 auto memory（`~/.claude/projects/*/memory/`
 
 |  | auto memory | ccRecall |
 |---|---|---|
-| **寫入路徑** | Claude 手動策展——新開一個 `.md` 檔 + 更新 MEMORY.md index | 雙軌：規則式 session 摘要（自動、免費）+ 選配 Haiku extraction（~$0.001/session，可搜尋的記憶主要來自這條路） |
+| **寫入路徑** | Claude 手動策展——新開一個 `.md` 檔 + 更新 MEMORY.md index | 雙軌：規則式 session 摘要（自動、免費）+ 選配 Sonnet extraction（訂閱方案算進用量，API key 實測每次 0.21–0.39 美元；可搜尋的記憶主要來自這條路） |
 | **讀取路徑** | 永遠在 session context（MEMORY.md 啟動時就載入） | auto memory 沒答案時，才用 MCP 查詢 |
 | **訊號密度** | 高——值得被命名的決策和偏好 | 長尾——hook 能抓到的都留著 |
 | **適用情境** | 「記住 X」「以後都 Y」——重要偏好、明確決策 | 「上次那個怎麼修的？」——跨多個 session 的回憶 |
@@ -385,7 +385,7 @@ ccRecall/
 │   ├── tutorial_zh.md                # 使用者教學（安裝 → MCP → 日常使用）
 │   ├── architecture_zh.md            # Daemon 設計取捨（給 contributor 看）
 │   └── launchd.md                    # macOS LaunchAgent 安裝/troubleshoot
-├── tests/                            # 651 個測試橫跨 40 檔案（parser、scanner、
+├── tests/                            # 858 個測試橫跨 54 檔案（parser、scanner、
 │   │                                 # summarizer、database、indexer、e2e、MCP、
 │   │                                 # memories、hooks、watcher、CLI、migrations、
 │   │                                 # FTS5 CJK edge cases、integrity monitor 等）
@@ -416,7 +416,7 @@ Anthropic Claude Code 團隊的 Thariq 在 2026 年 4 月[發表了 context 管
 
 每一條都從宣言延伸。每一條都是對流行做法的刻意拒絕。
 
-**規則式，有天花板。** daemon 用 heuristic 萃取（regex 模式、工具使用分析、outcome 推斷）做 session 摘要——零 API 成本。做這件事，「Edit×8, 5 files, committed」比一段散文更有用。但 rule-based 有天花板：結構化訊號（tool call、檔案編輯、commit message）抓得到；討論中的決策、微妙的取捨、任何活在自然語言裡的東西，它抓不到。這就是 v0.4.1 加入 post-session Haiku extraction（~$0.001/session）的原因——用一次 LLM pass 補上 heuristic 的盲區。實際上，我真正會搜到的記憶大多來自 Haiku extraction 或手動 `recall_save`，不是規則式那層。daemon 本體永遠不呼叫 LLM；LLM pass 是選配的，作為獨立進程在 daemon 之外執行。
+**規則式，有天花板。** daemon 用 heuristic 萃取（regex 模式、工具使用分析、outcome 推斷）做 session 摘要——零 API 成本。做這件事，「Edit×8, 5 files, committed」比一段散文更有用。但 rule-based 有天花板：結構化訊號（tool call、檔案編輯、commit message）抓得到；討論中的決策、微妙的取捨、任何活在自然語言裡的東西，它抓不到。這就是 v0.4.1 加入 post-session Haiku extraction（~$0.001/session）的原因——用一次 LLM pass 補上 heuristic 的盲區。實際上，我真正會搜到的記憶大多來自 session 結束後的那次抽取（v0.9.0 起改用 Sonnet）或手動 `recall_save`，不是規則式那層。daemon 本體永遠不呼叫 LLM；LLM pass 是選配的，作為獨立進程在 daemon 之外執行。
 
 **FTS5，不用向量搜尋。** 語義搜尋聽起來更高級，但對話記錄搜的是具體的工具名、檔案路徑、錯誤訊息——關鍵詞匹配就夠了。FTS5 查詢在本地 <10ms。不需要 embedding model、不需要 Chroma、不需要 Docker container。在我們的規模（數百個 session，不是百萬文件），Karpathy 自己的分析也確認：「500 個來源以下，樸素索引 + 關鍵詞搜尋已經夠用。」
 
@@ -428,6 +428,15 @@ Anthropic Claude Code 團隊的 Thariq 在 2026 年 4 月[發表了 context 管
 
 **不做帶偏見的注入。** ccRecall 不替 Claude 決定該記住什麼。它提供搜尋 API——注入層呈現結果，Claude 自己整合。帶偏見的記憶篩選是過早優化，而且會以我們無法預測的方式出錯。
 
+### transcript 沒說的事
+
+ccRecall 能知道的，只有 Claude Code 寫進硬碟的東西；可是 transcript 跟真正發生過的對話，並不完全是同一回事。v0.9.2 補上的兩個洞，都不曾報過錯：
+
+- Claude 還在忙的時候打進去的訊息，多半會被併進它下一次的 API 呼叫，存成 attachment，不會留下使用者訊息的紀錄。抽取用的 transcript 一直沒讀到這些——光是一個人一個月的 session，就有 254 則——到 v0.9.2 才補上。
+- Claude Code 2.1.300 開始把每次 API 請求的副本寫進 transcript，parser 一直把這些紀錄當成訊息算，到 v0.9.2 才擋掉。
+
+讀 transcript 的程式碼，照它假設的格式來看並沒有寫錯；兩個洞都是拿檔案裡實際存了什麼，去對照實際說過什麼，一筆一筆數出來的。
+
 ---
 
 ## 路線圖
@@ -439,6 +448,11 @@ Anthropic Claude Code 團隊的 Thariq 在 2026 年 4 月[發表了 context 管
 | **v0.5.0** | 砍刀場：journal/scorer/harvester 管線移除（史上零 promotion）、端點 13 → 5、`message_uuids` 雙 hash 重建（DB 114MB → 42MB） | 已釋出 |
 | **v0.5.2** | 召回權重：對數壓縮 half-life decay、FTS relevance-first 排序 `(-rank)*sqrt(EC)` | 已釋出 |
 | **v0.5.3** | CJK topic 對齊：Han-aware topic 抽取（`\p{Script=Han}` tokenizer + 中文 stopwords 32 條 + 助詞分割）、session-less rebuild 修正 | 已釋出 |
+| **v0.5.4–v0.5.6** | 量測注入：`injection_log` 記下每一次送出的記憶（#71）；Tier 0 的 startup 名額開始輪替，不再每場都是同樣三條；`/health` 改回報 `mainSessionCount` | 已釋出 |
+| **v0.6.0** | 對話中召回：`UserPromptSubmit` hook 在對話進行中帶出相關記憶；注入的每一行都附上 key；抽取會標出「把 `recall_save` 印成文字、沒有真的呼叫」的 run | 已釋出 |
+| **v0.7.x** | 用 `Intl.Segmenter` 做中文斷詞（#80）；修掉 `cwd` → project id 的編碼漂移，這個 bug 讓整個專案的抽取被靜默跳過（#89）；英文虛詞不再把持 topic 索引（#84）；補上 telemetry log 防護的漏洞 | 已釋出 |
+| **v0.8.x** | 出處與作者：記憶會記下出自哪則訊息、是誰寫的，自動抽取不會再蓋掉你手寫的記憶；抽取改成處方先行；Tier 0 挑別專案的記憶改依專案挑選，不再只看信心分數。長 session 改從 stdin 送進抽取，不再靜默失敗，另附 `backfill-extract.sh` 把已經漏掉的補回來 | 已釋出 |
+| **v0.9.x** | 抽取改用 Sonnet，沒產出可用結果時自動重試一次；路徑大小寫不同也不會再跳過（#131）；跳過 Claude Code 2.1.300 寫進 transcript 的請求副本，對話中途插打的訊息也讀得到了；抽取不再被注入記憶；`recall_save` 不再撞上「database is locked」 | 已釋出 |
 
 追蹤於 [GitHub Issues](https://github.com/tznthou/ccRecall/issues)。
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tznthou/ccrecall",
-  "version": "0.9.2",
+  "version": "0.10.0",
   "description": "AI memory service for Claude Code — on-demand context injection with metacognition",
   "type": "module",
   "main": "dist/index.js",

package/scripts/backfill-extract.sh

@@ -17,6 +17,13 @@
 #   --from-log-all  every non-zero exit in the log — mostly ancient sessions
 #                   whose transcript Claude Code has since deleted
 #
+# Environment:
+#   CCRECALL_EXTRACT_EFFORT          effort for each run, as for ccrecall-extract:
+#                                    low, medium, high, xhigh or max (default high)
+#   CCRECALL_EXTRACT_MAX_BUDGET_USD  spend cap per run, applied only when
+#                                    ANTHROPIC_API_KEY is set (default 0.50);
+#                                    raise it to run at max
+#
 # Why this exists (2026-09-15): post-session-extract.sh used to pass the whole
 # transcript as one argv parameter (capped at 200,000 bytes), and a single argv
 # parameter has a ceiling well below that cap:
@@ -91,9 +98,10 @@ while [[ $# -gt 0 ]]; do
     --force) FORCE=1 ;;
     --from-log) FROM_LOG=1 ;;
     --from-log-all) FROM_LOG=1; FROM_LOG_ALL=1 ;;
-    # Usage + Options only. The rationale below it is for whoever reads the
-    # source; printing half of it (the range has to stop somewhere) helps nobody.
-    -h|--help) sed -n '2,17p' "$SELF"; exit 0 ;;
+    # Usage, Options and Environment only. The rationale below them is for
+    # whoever reads the source; printing half of it (the range has to stop
+    # somewhere) helps nobody.
+    -h|--help) sed -n '2,25p' "$SELF"; exit 0 ;;
     -*) die "unknown option: $1" ;;
     *) SESSION_IDS+=("$1") ;;
   esac
@@ -105,6 +113,11 @@ done
 # shellcheck source=/dev/null
 source "$WRAPPER" >/dev/null 2>&1 || die "failed to source $WRAPPER"
 declare -F _ccrecall_log_append >/dev/null || die "_ccrecall_log_append not defined by $WRAPPER"
+# The MCP configuration too, so a backfill run starts the same single server
+# the wrapper's run does rather than a second copy of how to find it.
+declare -F _ccrecall_extract_mcp_config >/dev/null || die "_ccrecall_extract_mcp_config not defined by $WRAPPER"
+# And the effort, read from CCRECALL_EXTRACT_EFFORT the way the wrapper reads it.
+declare -F _ccrecall_extract_effort >/dev/null || die "_ccrecall_extract_effort not defined by $WRAPPER"
 
 [[ -f "$PROMPT_FILE" ]] || die "extraction-prompt.md not found: $PROMPT_FILE
    (the wrapper's inline fallback is deliberately not duplicated here — a second
@@ -178,8 +191,11 @@ fi
 
 [[ ${#SESSION_IDS[@]} -gt 0 ]] || die "no session ids given (pass ids, or --from-log)"
 
-printf '\n🧠 ccRecall backfill — %d session(s)%s\n\n' \
-  "${#SESSION_IDS[@]}" "$([[ $DRY_RUN -eq 1 ]] && printf ' [DRY RUN]')"
+# Once for the whole batch, so a value that is not a level is warned about once.
+EFFORT=$(_ccrecall_extract_effort)
+
+printf '\n🧠 ccRecall backfill — %d session(s) (effort %s)%s\n\n' \
+  "${#SESSION_IDS[@]}" "$EFFORT" "$([[ $DRY_RUN -eq 1 ]] && printf ' [DRY RUN]')"
 
 ok=0; skipped=0; failed=0
 
@@ -288,13 +304,25 @@ ${prompt}"
   # same reason as in the wrapper: this is a headless Claude Code session of its
   # own, and ccRecall's hooks would otherwise inject memories into it — and
   # count each one as surfaced.
+  #
+  # Effort and permissions match the wrapper's run too, for the reasons given
+  # there: CLAUDE_CODE_EFFORT_LEVEL is set as a prefix because it outranks
+  # --effort (a caller exporting max ran each extraction at max, mostly
+  # thinking), and the run gets ccRecall's own MCP server, no built-in tools and
+  # an allowlist instead of --dangerously-skip-permissions.
+  mcp_config=$(_ccrecall_extract_mcp_config)
   extract_stderr=$(cd "$session_cwd" && printf '%s' "$full_prompt" |
-    CCRECALL_SESSION_START_STRATEGY=off CCRECALL_PROMPT_RECALL=off command claude -p \
+    CCRECALL_SESSION_START_STRATEGY=off CCRECALL_PROMPT_RECALL=off CLAUDE_CODE_EFFORT_LEVEL="$EFFORT" \
+    command claude -p \
     --no-session-persistence \
     --model sonnet \
     ${budget_args[@]+"${budget_args[@]}"} \
     --max-turns 5 \
-    --dangerously-skip-permissions 2>&1 1>/dev/null)
+    --strict-mcp-config --mcp-config "$mcp_config" \
+    --tools "" \
+    --allowedTools "mcp__ccrecall__recall_save,mcp__ccrecall__recall_query" \
+    --permission-mode dontAsk \
+    --disable-slash-commands 2>&1 1>/dev/null)
   extract_exit=$?
   duration=$(( $(date +%s) - start ))
 

package/scripts/extraction-prompt.md

@@ -1,7 +1,8 @@
 You are a memory extraction agent for ccRecall, running as an automated
 non-interactive pipeline. There is NO human reading your text output — only
 tool calls matter. Analyze the session transcript and save 0-5 lasting
-insights via the `recall_save` MCP tool. Produce no other output.
+insights by calling the `mcp__ccrecall__recall_save` tool — use that exact
+name; a bare `recall_save` does not exist in this run. Produce no other output.
 
 ## What to extract
 
@@ -129,7 +130,7 @@ expressed. Copy it verbatim out of the header.
 
 ## recall_save parameters
 
-For each memory, call `recall_save` with:
+For each memory, call `mcp__ccrecall__recall_save` with:
 - `content`: the self-contained insight (see rules above)
 - `type`: one of `decision`, `discovery`, `pattern`, `preference`, `feedback`
 - `key`: stable hyphenated slug (see above)

package/scripts/post-session-extract.sh

@@ -5,12 +5,19 @@
 #
 # Usage: ccrecall-extract [claude args...]
 #   Runs Claude Code with ccRecall startup injection, then extracts
-#   memories via Haiku after the session ends.
+#   memories with Sonnet after the session ends.
 #
 # Environment:
-#   CCRECALL_PORT         — daemon port (default 7749)
-#   CCRECALL_SKIP_EXTRACT — set to 1 to skip post-session extraction
-#   CCRECALL_EXTRACT_LOG  — telemetry log path (default ~/.ccrecall/extract.log.jsonl)
+#   CCRECALL_PORT           — daemon port (default 7749)
+#   CCRECALL_SKIP_EXTRACT   — set to 1 to skip post-session extraction
+#   CCRECALL_EXTRACT_LOG    — telemetry log path (default ~/.ccrecall/extract.log.jsonl)
+#   CCRECALL_EXTRACT_EFFORT — effort for the extraction run: low, medium, high,
+#                             xhigh or max (default high). Your interactive
+#                             session keeps its own effort either way.
+#   CCRECALL_EXTRACT_MAX_BUDGET_USD — spend cap per extraction run, applied only
+#                             when ANTHROPIC_API_KEY is set (default 0.50).
+#                             Measured runs at max cost 0.64-0.97, so raise it
+#                             along with the effort.
 
 CCRECALL_PORT="${CCRECALL_PORT:-7749}"
 CCRECALL_EXTRACT_LOG="${CCRECALL_EXTRACT_LOG:-$HOME/.ccrecall/extract.log.jsonl}"
@@ -178,6 +185,66 @@ fi
 CCRECALL_SCRIPT_DIR="$(cd "$(dirname "$_ccrecall_src")" && pwd)"
 unset _ccrecall_src
 
+# The MCP configuration an extraction run starts with: ccRecall's own server
+# and nothing else, which is what lets `--strict-mcp-config` drop every other
+# server the user has registered — the claude.ai Gmail, Drive and Calendar
+# connectors among them. backfill-extract.sh sources this file for it too.
+#
+# The server is this package's own build, found from where this file lives,
+# not the `ccmem-mcp` bin. A checkout registered with `claude mcp add ... --
+# <path>/dist/mcp/server.js` (the README's way) has no such bin on PATH, and
+# naming it would leave that install's extraction without a tool to save with.
+# The bin is the fallback only when no build sits next to this file. Either way
+# the server inherits this shell's environment, CCRECALL_DB_PATH included —
+# where the daemon installer and _ccrecall_count_extracted read it from too.
+#
+# jq builds the JSON so an install path holding a space or a quote stays one
+# string.
+_ccrecall_extract_mcp_config() {
+  local server="${CCRECALL_SCRIPT_DIR%/*}/dist/mcp/server.js"
+  if [ -f "$server" ]; then
+    jq -nc --arg s "$server" '{mcpServers:{ccrecall:{type:"stdio",command:"node",args:[$s]}}}'
+  else
+    jq -nc '{mcpServers:{ccrecall:{type:"stdio",command:"ccmem-mcp",args:[]}}}'
+  fi
+}
+
+# The effort an extraction run starts at: the level CCRECALL_EXTRACT_EFFORT
+# names, or high when it names none. backfill-extract.sh sources this file for
+# it too, so a backfill starts at the level the run it stands in for would have.
+#
+# The extraction gets a variable of its own because CLAUDE_CODE_EFFORT_LEVEL
+# cannot be the knob: that is the caller's interactive setting, the very value
+# this run has to override (see the effort comment in ccrecall-extract).
+#
+# Checked against the levels rather than passed on as given. Claude Code takes
+# any value without complaint: on 2.1.300 `bogus` exited 0 with nothing on
+# stderr and ran at whatever the settings or the model default said (medium,
+# for Sonnet 5.5 with no settings; read off the Stop hook's effort.level). What
+# this prints is also what the start message names, so an unchecked typo would
+# be reported as the run's level while the run got another. Anything else
+# becomes high, with a warning that leaves the value out: it goes to a
+# terminal, and nothing has vetted what the environment holds.
+#
+# `auto` is left out although Claude Code documents it for this variable as
+# "use the model default". On 2.1.300 it ran Sonnet 5.5 at high every time,
+# with and without settings, where the model default is medium, so a start
+# message saying "auto" would hide the level the run actually got.
+#
+# Always returns 0. The callers assign its output, and this file is sourced
+# into shells that set errexit, where a non-zero status here would abort the
+# caller's shell at that assignment, before extraction ever ran.
+_ccrecall_extract_effort() {
+  case "${CCRECALL_EXTRACT_EFFORT:-}" in
+    low|medium|high|xhigh|max) builtin printf '%s' "$CCRECALL_EXTRACT_EFFORT" ;;
+    '') builtin printf 'high' ;;
+    *)
+      builtin printf '\n⚠️  ccRecall: CCRECALL_EXTRACT_EFFORT must be one of low, medium, high, xhigh, max — extracting at high.\n' >&2
+      builtin printf 'high'
+      ;;
+  esac
+}
+
 ccrecall-extract() {
   # #89: this used to derive project_id here with `sed 's|/|-|g'`, which
   # diverged from Claude Code's char-wise [^A-Za-z0-9] encoding for any cwd
@@ -260,7 +327,7 @@ ccrecall-extract() {
     # below against token-budget.ts, in both copies. 🔴 Until 2026-09-08 this
     # comment already said that while no test read this line's number at all:
     # it could say 999 and stay green. Verified by mutation, both directions.
-    prompt="You are a memory extraction agent. Save 0-5 lasting insights via recall_save. Each memory must be self-contained with a key slug for dedup. Only the first 149 characters survive injection into a future session, so lead with what a future reader should DO and put the evidence, file names and war story after it. Set projectId to \"${project_id}\" for project-specific knowledge; omit for cross-project knowledge. Always pass origin=\"agent-inferred\": you are reading a finished transcript with nobody watching, and that flag is what stops your save from overwriting a memory the user wrote by hand under the same key. Pass messageId: the uuid in the \"--- human [<uuid>] ---\" header of the one message a memory came from, copied verbatim — omit it when no single message is the source, and never guess a uuid. Turns headed \"--- side-agent note [<uuid>] ---\" or \"--- peer session [<uuid>] ---\" are a note the user forwarded or a message from another Claude session, not the user's own words: never save what they say as the user's decision."
+    prompt="You are a memory extraction agent. Save 0-5 lasting insights by calling the mcp__ccrecall__recall_save tool, by that exact name. Each memory must be self-contained with a key slug for dedup. Only the first 149 characters survive injection into a future session, so lead with what a future reader should DO and put the evidence, file names and war story after it. Set projectId to \"${project_id}\" for project-specific knowledge; omit for cross-project knowledge. Always pass origin=\"agent-inferred\": you are reading a finished transcript with nobody watching, and that flag is what stops your save from overwriting a memory the user wrote by hand under the same key. Pass messageId: the uuid in the \"--- human [<uuid>] ---\" header of the one message a memory came from, copied verbatim — omit it when no single message is the source, and never guess a uuid. Turns headed \"--- side-agent note [<uuid>] ---\" or \"--- peer session [<uuid>] ---\" are a note the user forwarded or a message from another Claude session, not the user's own words: never save what they say as the user's decision."
   fi
 
   # Append runtime context (projectId) to the prompt
@@ -398,7 +465,11 @@ ${prompt}"
     return $claude_exit
   fi
 
-  printf '\n🧠 ccRecall: extracting memories from session...\n'
+  # Named in the start message because the level is now a setting: the line
+  # that says extraction began is where a run at the wrong one shows.
+  local effort
+  effort=$(_ccrecall_extract_effort)
+  printf '\n🧠 ccRecall: extracting memories from session (effort %s)...\n' "$effort"
 
   local extract_start
   extract_start=$(date +%s)
@@ -555,6 +626,33 @@ ${prompt}"
   # written. Prefix assignments, so they reach this command only — an export
   # would persist in the shell this file is sourced into and switch prompt
   # recall off for every later session.
+  #
+  # The effort is set the same way, and it has to be a prefix:
+  # CLAUDE_CODE_EFFORT_LEVEL outranks --effort, /effort and every effortLevel
+  # setting, so a user who exports `max` for their interactive sessions ran
+  # every extraction at max, and an --effort flag here would change nothing.
+  # Measured 2026-10-15 on two real transcripts: at max, 90-95% of a run was
+  # thinking — 32-46K tokens before the first save, 261-355 s end to end. At
+  # high the same two runs took 26-40 s and saved five memories each, where max
+  # saved four or five. medium and low were faster still, but saved one to
+  # three fewer per session. So $effort is high unless CCRECALL_EXTRACT_EFFORT
+  # names another level (see _ccrecall_extract_effort).
+  #
+  # And the run gets only what it uses. It reads a transcript this prompt calls
+  # untrusted, and used to do so under --dangerously-skip-permissions with every
+  # MCP server the user had registered: Bash and, on the machine this was
+  # measured on, Gmail, Google Drive and Calendar connectors, with nothing
+  # asking first. Now it gets ccRecall's own server alone (--strict-mcp-config,
+  # see _ccrecall_extract_mcp_config), no built-in tools (--tools ""), the two
+  # tools extraction calls on an allowlist, and dontAsk refusing anything else
+  # instead of prompting for it. Skills are switched off as well — their
+  # listing is tens of kilobytes the model never needs. Verified across four
+  # real extractions the same day: no permission denials, ~26K fewer input
+  # tokens per run, and the MCP tools load directly, so the ToolSearch round
+  # before the first save is gone. Without ToolSearch the model takes tool
+  # names from the prompt, which is why both copies of it spell the full name.
+  local mcp_config
+  mcp_config=$(_ccrecall_extract_mcp_config)
   local attempt=1 extract_retried=0 extracted_count=""
   local stdout_marker recall_save_text_count
   while : ; do
@@ -562,12 +660,17 @@ ${prompt}"
     trap 'command rm -f -- "$stdout_tmp" 2>/dev/null; exit 130' INT
     trap 'command rm -f -- "$stdout_tmp" 2>/dev/null; exit 143' TERM
     builtin printf '%s' "$full_prompt" |
-      CCRECALL_SESSION_START_STRATEGY=off CCRECALL_PROMPT_RECALL=off command claude -p \
+      CCRECALL_SESSION_START_STRATEGY=off CCRECALL_PROMPT_RECALL=off CLAUDE_CODE_EFFORT_LEVEL="$effort" \
+      command claude -p \
       --no-session-persistence \
       --model sonnet \
       ${budget_args[@]+"${budget_args[@]}"} \
       --max-turns 5 \
-      --dangerously-skip-permissions 2>&1 1>|"$stdout_tmp"
+      --strict-mcp-config --mcp-config "$mcp_config" \
+      --tools "" \
+      --allowedTools "mcp__ccrecall__recall_save,mcp__ccrecall__recall_query" \
+      --permission-mode dontAsk \
+      --disable-slash-commands 2>&1 1>|"$stdout_tmp"
   ); then
     extract_exit=0
   else
@@ -599,9 +702,14 @@ ${prompt}"
   # called). A false positive costs one row to check; a false negative restores
   # the status quo of total invisibility.
   #
+  # The `mcp__ccrecall__` prefix is optional because the prompt now teaches the
+  # full tool name, and a model that prints its calls prints the name it was
+  # taught. Matching only the bare name would have gone blind to #75 the day
+  # the prompt changed.
+  #
   # -c yields a bare count and never the matched line, so no session content can
   # reach the telemetry log — the same constraint that governs the stdout capture.
-  recall_save_text_count=$(grep -cE '^[[:space:]]*recall_save[[:space:]]*\(' "$stdout_tmp" 2>/dev/null) || :
+  recall_save_text_count=$(grep -cE '^[[:space:]]*(mcp__ccrecall__)?recall_save[[:space:]]*\(' "$stdout_tmp" 2>/dev/null) || :
   # grep exits 1 with empty output on no match (and $stdout_tmp is /dev/null when
   # mktemp failed); --argjson would abort the whole telemetry write on a non-number.
   [[ "$recall_save_text_count" =~ ^[0-9]+$ ]] || recall_save_text_count=0
@@ -623,12 +731,12 @@ ${prompt}"
   done
   command rm -f -- "$stdout_tmp" 2>/dev/null
   # Scrub common credential formats before stderr reaches the telemetry log.
-  # claude echoes its own key (sk-ant-) in auth errors, and any MCP server
-  # loaded for extraction (this runs with --dangerously-skip-permissions) can
-  # surface its own token in an init/auth failure. This is a best-effort
-  # denylist of common prefixes, not exhaustive — but stdout, the higher-risk
-  # stream where the model could echo transcript secrets, is already discarded
-  # entirely, so this only hardens claude's own diagnostics.
+  # claude echoes its own key (sk-ant-) in auth errors. Only ccRecall's own MCP
+  # server is started for extraction now, so a third-party server can no longer
+  # print its token here, but claude's own diagnostics still can. This is a
+  # best-effort denylist of common prefixes, not exhaustive — but stdout, the
+  # higher-risk stream where the model could echo transcript secrets, is already
+  # discarded entirely, so this only hardens claude's own diagnostics.
   extract_stderr=$(printf '%s' "$extract_stderr" | sed -E 's/(sk-ant-|sk-proj-|ghp_|gho_|ghu_|ghs_|github_pat_|AKIA)[A-Za-z0-9_-]*/[REDACTED]/g' | head -c 2000)
 
   local extract_end