open-memex 0.10.0-alpha.3 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1148,8 +1148,11 @@
1148
1148
  - **D91 — the knowledge graph, deterministic half (§20, 2026-11-06).** Implements the graph layer D88 designed, split at the derivation boundary: this change ships only edges the code can *prove* or derive from structure, with no LLM call anywhere. Edge extraction (`src/graph/extract.ts`, pure): (1) **supersedes** edges from the index chain (`superseded_by`, plus the reverse derived from that column — the index has no `supersedes` column), EXTRACTED/high; (2) **derived-from** edges from the propose-copy's `derived_from` frontmatter (now also an index column, schema v10), EXTRACTED/high; (3) **relates-to** edges from `[[wiki-link]]` body mentions of a node's ULID, EXTRACTED/high, and from bare ULID mentions, INFERRED/high; (4) **relates-to** from shared tags at a fixed ladder — 1 shared tag → low, 2 → medium, ≥3 → high — INFERRED, capped at `MAX_SHARED_TAG_EDGES_PER_NODE = 12` per node and `MAX_SHARED_TAG_NODES = 2000` overall so one universal tag cannot produce an O(n²) edge set; one relational edge per unordered pair, EXTRACTED beats INFERRED on the same pair. Stored in `graph_edges`/`graph_meta` beside the rebuildable index with `CREATE TABLE IF NOT EXISTS` (like `memory_usage`, D77), so the graph is derived, disposable, and never touches Markdown or the source of truth. Every mutating path refreshes it automatically (`refreshGraphQuiet` on the write hooks: add/supersede/soft + hard forget on both hosts, keyword capture, propose, review promotion, CLI add/supersede/status/forget); `refreshGraph` is the explicit `graph build` path and reports honestly (D63 — a missing id fails, findings never invent nodes). **Outline** (`src/graph/outline.ts`, pure): connected components joined **only by medium+ INFERRED shared-tag edges** (the universal-tag collapse guard — a single low/medium tag must not merge unrelated memories; fixture-pinned in `smoke-pure.ts`), rendered `topic (n): titles…` with the cut disclosed at `maxLines` (config `graphOutlineLines`, default 10, cap 30); the compiled outline is stored in `graph_meta` and injected once at session start next to the working-state note (`graphOutline`, default on; injected lines contribute exposure ids to D77 `recordRetrieved`, never usefulness). **Queries** (`src/graph/queries.ts`, pure): `graph neighbours <id> [--depth 1-6]` (BFS, default 1, per-edge receipts with type/provenance/confidence) and `graph path <a> <b>` (shortest path, default depth 4, per-hop receipts). **Lint** (`src/graph/lint.ts`, pure): orphans (active, no topic, no edges), dangling edges (endpoint absent from the index — but an endpoint that exists in *another* scope is a legitimate foreign link, e.g. a propose copy deriving from its personal original, and is skipped), and near-stale pairs (similarity ≥ 0.8, chains excluded) via the existing lifecycle `similarity()`; reports only, nothing auto-resolved — this is the report D78's pre-submit lint consumes (wired in D93). Scope contract: the graph covers the cwd's **project** scope only, no `--scope` flag, `personal` never graphs (§20.5). The interface is **CLI only** (`open-memex graph build|neighbours|path|lint|outline`) — the MCP surface stays at 13 tools; the outline reaches sessions through injection, not a tool (§20.6). Schema bump: added `derived_from TEXT` to `memories` (`SCHEMA_VERSION` 9 → 10, rebuild-from-Markdown migration — the frontmatter field already existed at `markdown.ts`). The **LLM half** — abstracted insights and missing-page alerts that propose new memories — remains open in §18/§20 and must go through the D61 propose-first flow.
1149
1149
  *Status (2026-11-06): 部分实现 (0.9.0-alpha.6) — deterministic half shipped: `src/graph/{extract,outline,queries,lint}.ts` + `src/store/graph.ts`, auto-refresh on every mutating path, outline injection, CLI `graph` group; smoke-pure + test-full (154 checks) green. LLM half (abstracted insights, missing-page → new memories) remains 未实现.*
1150
1150
 
1151
- - **D92 — feedback-weighted ranking, first slice (2026-11-11).** The D79 follow-up that D77-L2 unblocked: the reuse counters now have a *measured*, deliberately small effect on `search()` order. **Measurement first:** commit 1 added a 3-query reuse slice to `scripts/retrieval-eval.ts` — each query has a gold memory carrying `retrieved=5..6, useful=3` counters and a near-tied distractor with the same retrieval count but zero useful confirmations — and seeded that telemetry through the real `recordRetrieved`/`recordUseful` API. Baseline with feedback still off: 60 memories / 47 queries, R@1 0.660 / R@5 0.979 / MRR 0.804. **Mechanism:** `src/retrieve/feedback.ts` (pure, smoke-tested) reorders only inside a near-tied BM25 band (score ≥ **0.75×** the band head), only among hits with `retrieved_count >= 3`, by `reuse_rate` descending, and only within a contiguous run of eligible hits — so an ineligible/new memory keeps its exact BM25 slot and its neighbours cannot leapfrog it. The BM25 score is never changed; `search --explain` still prints it, plus a `feedback:` line listing every promotion (`↑ id +N (reuse …)`) so the trace is auditable. Tiers are reordered independently, so feedback never lifts a draft over reviewed knowledge. Config is a flat key `feedbackWeighting` (**default false**, `SETTABLE_KEYS`, shallow-merge-safe); callers (`memory_search` both hosts, CLI `search`) pass it, and the eval passes both values explicitly. **Result:** weighted R@1 0.702 / R@5 0.979 / MRR 0.825; reuse slice MRR 0.500 → 0.833 (two of three golds promoted 2→1). Band sweep on the fixture: 0.95 → no effect, 0.9/0.85/0.8 → +1 R@1, 0.75 → +2, 0.67 → +3; 0.75 is shipped because it is the widest band that still catches the two strongest reuse signals; 0.67 would also promote a hit ~33% below the band head, which is a relevance gap, not a near-tie. **Guardrails unchanged (D77):** useful only via `memory_useful`, human approve at review, or supersede — passive exposure still increments only `retrieved_count`; `reuse_rate` may exceed 1.0 and is signal strength, never a prune threshold; counters stay in local SQLite and survive index rebuilds. **Explicitly not in this slice:** embedding gate, cross-lingual rerank (D79 item 4, after D78), automatic zombie demotion/eviction (D78), and any change to `memory_useful` or the D77-L3 report's accounting.
1151
+ - **D92 — feedback-weighted ranking, first slice (2026-11-11).** The D79 follow-up that D77-L2 unblocked: the reuse counters now have a *measured*, deliberately small effect on `search()` order. **Measurement first:** commit 1 added a 3-query reuse slice to `scripts/retrieval-eval.ts` — each query has a gold memory carrying `retrieved=5..6, useful=3` counters and a near-tied distractor with the same retrieval count but zero useful confirmations — and seeded that telemetry through the real `recordRetrieved`/`recordUseful` API. Baseline with feedback still off: 60 memories / 47 queries, R@1 0.660 / R@5 0.979 / MRR 0.804. **Mechanism:** `src/retrieve/feedback.ts` (pure, smoke-tested) reorders only inside a near-tied BM25 band (score ≥ **0.75×** the band head), only among hits with `retrieved_count >= 3`, by `reuse_rate` descending, and only within a contiguous run of eligible hits — so an ineligible/new memory keeps its exact BM25 slot and its neighbours cannot leapfrog it. The BM25 score is never changed; `search --explain` still prints it, plus a `feedback:` line listing every promotion (`↑ id +N (reuse …)`) so the trace is auditable. Tiers are reordered independently, so feedback never lifts a draft over reviewed knowledge. Config is a flat key `feedbackWeighting` (**default false**, `SETTABLE_KEYS`, shallow-merge-safe); callers (`memory_search` both hosts, CLI `search`) pass it, and the eval passes both values explicitly. **Result:** weighted R@1 0.702 / R@5 0.979 / MRR 0.825; reuse slice MRR 0.500 → 0.833 (two of three golds promoted 2→1). Band sweep on the fixture: 0.95 → no effect, 0.9/0.85/0.8 → +1 R@1, 0.75 → +2, 0.67 → +3; 0.75 is shipped because it is the widest band that still catches the measured reuse gains; 0.67 would also promote a hit ~33% below the band head, which is a relevance gap, not a near-tie. **Guardrails unchanged (D77):** useful only via `memory_useful`, human approve at review, or supersede — passive exposure still increments only `retrieved_count`; `reuse_rate` may exceed 1.0 and is signal strength, never a prune threshold; counters stay in local SQLite and survive index rebuilds. **Explicitly not in this slice:** embedding gate, cross-lingual rerank (D79 item 4, after D78), automatic zombie demotion/eviction (D78), and any change to `memory_useful` or the D77-L3 report's accounting.
1152
1152
  *Status (2026-11-11): 已实现(第一刀,0.10.0-alpha.2)— `src/retrieve/feedback.ts` + `search()` wiring + `feedbackWeighting` flat key + `search --explain` promotion trace; fixture baseline vs weighted numbers in `scripts/retrieval-eval.ts`. D79 的 embedding gate / 跨语言 rerank / 自动淘汰仍留给后续。*
1153
1153
 
1154
- - **D93 — deterministic pre-submit curation, first slice (2026-11-13).** Implements D78's **auto-apply** tier only, keeping the submit boundary explicit: `submit` now reports graph lint and auto-curates deterministic defects, but the human still chooses the ids and never has a share action performed for them. **Measurement first:** commit 1 (`4db3174`) added `scripts/curate-eval.ts` as a checked-in fixture with no product curation logic; its baseline is 9 lint findings (7 orphans, 1 dangling edge, 1 stale pair), 1 exact duplicate, 1 empty body, 2 title-normalization candidates, and 1 near-duplicate report. **Planner:** `src/curate.ts` (pure, smoke-tested) orders archive-empty → merge-duplicate → normalize-title, then by id. Exact duplicates are `contentHash()`-identical whitespace-normalized bodies; the canonical is the oldest free active record (ties by id), chain-managed records are never chosen, and a duplicate is appended at the oldest tail of the canonical's supersede chain so no history is dropped. Empty / whitespace-only bodies are archived, not deleted. Title normalization rewrites only the first non-empty line (trim + collapse internal whitespace) - line count and every other byte stay intact. Near-duplicates (`similarity() >= 0.8`) remain **report-only**; semantic merge/split and inferred type/tags need a human and are explicitly out of this slice. **Write half:** `src/store/curate.ts` applies actions through the existing lifecycle primitives (`atomicWriteTextSync`, `setStatus`, chain fields, append-only `review_history`) and immediately reindexes. Every applied action emits a receipt in `review_history`; merge and title actions are reversed by `open-memex curate undo <id>`, empty-body archives also by `open-memex status <id> active`. A title undo refuses if the first line changed after curation. **Submit wiring:** validate → branch/conflict preflight → lint report → deterministic apply → re-read and submit only records still `active` → copy + local commit. Lint is report-only and never blocks. A non-active record is history, not a submit candidate (`status is <x>, only active memories can be submitted`). Curated-out outbox files stay in appdata as history and are excluded from the pending count (`outboxHistory` in `sync-status`). If the submit fails before its local commit, applied curation is undone best-effort so the outbox returns to its prior state. `--no-curate` skips the apply tier but still prints lint; the boolean-flag parser accepts it before or after the ids. `graph lint` no longer flags an intentional supersede-chain duplicate as stale. **Result on the fixture:** curated submit reduces the report from 9 findings to 6 (orphans 7→4, dangling 1→1, stale 1→1), auto-applies exactly 3 actions (merge 1, archive 1, title 1), `--no-curate` applies 0, and all 3 undo checks pass; `typecheck`, build, `smoke-pure`, and `test-full` pass. **Still open (D78):** semantic merge/split, batch-confirm screen, inferred type/tags, the undo→confidence feedback loop, and attaching the curation summary to a PR description.
1155
- *Status (2026-11-13): 部分实现(第一刀,0.10.0-alpha.3)— `src/curate.ts` + `src/store/curate.ts` + `submit` wiring + `--no-curate` + `curate undo`; `scripts/curate-eval.ts` pins baseline vs curated numbers. Batch-confirm 与语义合并仍开放。*
1154
+ - **D93 — deterministic pre-submit curation, first slice (2026-11-13).** Implements D78's **auto-apply** tier only, keeping the submit boundary explicit: `submit` now reports graph lint and auto-curates deterministic defects, but the human still chooses the ids and never has a share action performed for them. **Measurement first:** commit 1 (`4db3174`) added `scripts/curate-eval.ts` as a checked-in fixture with no product curation logic; its baseline is 9 lint findings (7 orphans, 1 dangling edge, 1 stale pair), 1 exact duplicate, 1 empty body, 2 title-normalization candidates, and 1 near-duplicate report. **Planner:** `src/curate.ts` (pure, smoke-tested) orders archive-empty → merge-duplicate → normalize-title, then by id. Exact duplicates are `contentHash()`-identical whitespace-normalized bodies; the canonical is the oldest free active record (ties by id), chain-managed records are never chosen, and a duplicate is appended at the oldest tail of the canonical's supersede chain so no history is dropped. Empty / whitespace-only bodies are archived, not deleted. Title normalization rewrites only the first non-empty line (trim + collapse internal whitespace) - line count and every other byte stay intact. Near-duplicates (`similarity() >= 0.8`) remain **report-only**; semantic merge/split and inferred type/tags need a human and are explicitly out of this slice. **Write half:** `src/store/curate.ts` applies actions through the existing lifecycle primitives (`atomicWriteTextSync`, `setStatus`, chain fields, append-only `review_history`) and immediately reindexes. Every applied action emits a receipt in `review_history`; merge and title actions are reversed by `open-memex curate undo <id>`, empty-body archives also by `open-memex status <id> active`. A title undo refuses if the first line changed after curation. **Submit wiring:** validate → branch/conflict preflight → lint report → deterministic apply → re-read and submit only records still `active` → copy + local commit. Lint is report-only and never blocks. The printed `gh pr create` command uses the actual submitted ids/count (not the pre-curation batch) and puts the lint + applied-action summary in the PR body, kept to shell-safe counts and kinds so no nested quotes leak into the command. A non-active record is history, not a submit candidate (`status is <x>, only active memories can be submitted`). Curated-out outbox files stay in appdata as history and are excluded from the pending count (`outboxHistory` in `sync-status`). If the submit fails before its local commit, applied curation is undone best-effort so the outbox returns to its prior state. `--no-curate` skips the apply tier but still prints lint; the boolean-flag parser accepts it before or after the ids. `graph lint` no longer flags an intentional supersede-chain duplicate as stale. **Result on the fixture:** curated submit reduces the report from 9 findings to 6 (orphans 7→4, dangling 1→1, stale 1→1), auto-applies exactly 3 actions (merge 1, archive 1, title 1), `--no-curate` applies 0, and all 3 undo checks pass; `typecheck`, build, `smoke-pure`, and `test-full` pass. **Still open (D78):** semantic merge/split, batch-confirm screen, inferred type/tags, and the undo→confidence feedback loop.
1155
+ *Status (2026-11-13): 部分实现(第一刀,0.10.0-alpha.3)— `src/curate.ts` + `src/store/curate.ts` + `submit` wiring + `--no-curate` + `curate undo` + actual submitted count and lint/curation summary in `gh pr create`; `scripts/curate-eval.ts` pins baseline vs curated numbers. Batch-confirm 与语义合并仍开放。*
1156
+
1157
+ - **D94 - OptMem small items: token cost as spec, wake convention, capture length nudge (2026-11-13).** Closes §18 build item (5) - the three friction-reduction ideas borrowed from kafkaup's OptMem post - as a follow-on to D93 in the same build order. **(1) Token cost as spec (measure first).** `scripts/token-cost.ts` seeds a scratch `OPEN_MEMEX_HOME` with 8 project + 5 personal memories, renders the *actual* `buildContextBlock` output (including the compiled graph outline) and joins the *actual* `TOOL_DESCRIPTIONS` text, and reports characters + tokens estimated as characters / 4 - the rough English-prose ratio, deliberately tokenizer-free (no new dependency). Measured on the checked-in fixture: injection block **2125 chars (~531 tokens)**, 13 MCP tool descriptions **8814 chars (~2204 tokens)**, total **10939 chars (~2735 tokens)**. Both READMEs publish these as approximations with the method stated, plus the fact that matters more than the number: the injection is **once per session, not once per turn**. The script's output and the README table must agree; they change together, never by hand-edit. **(2) `wake` convention - documentation only.** The session-start ritual reuses existing surfaces and adds no mechanism: no new MCP tool (stays 13), no new CLI verb. The ritual, stated identically in README (en + zh), the bundled Agent Skill, and the MCP `SERVER_INSTRUCTIONS`: read the injected `[OPEN-MEMEX]` block (the D89 working-state note and D91 graph outline ride it); if the block is empty or insufficient, call `memory_search` **once** with the task's keywords (2-3 phrasings, both scopes); never re-ask the user for what memory may already hold. §18's original wording mentioned a `memory_recall` tool that does not exist - the documented ritual invents no tool name, and the AGENTS.md wake rule forbids inventing one. **(3) Capture length nudge - hint, never refusal.** Pure `lengthNudge()` (`src/capture/length.ts`, smoke-tested, inclusive boundary: 500 chars = no hint, 501 = hint) appends a one-line hint to the save result of `memory_add` (both hosts via `addMemory`) and CLI `add`; the save itself - redact, dedup, write, graph refresh - is untouched and long content is never refused or truncated. The only hard cap remains keyword capture's `MAX_CAPTURE_CHARS` (2000, D67); what changes is that an over-cap keyword drop, previously visible only in the debug log, is now surfaced to the agent in the D73 hand-off note (`formatCaptureDropBlock`, delivered once per drop like a capture) so it can re-save a shortened, self-contained version - the debug log stays. Deliberately **not** in this change: embedding gate / cross-lingual rerank, three-tier memory zones, auto-capture - each still open in §18; and no rejection path anywhere.
1158
+ *Status (2026-11-13): 已实现(本 PR,0.10.0-alpha.4)- 三刀各自落地:`scripts/token-cost.ts`(可复跑,README 数字与其输出一致);wake 写进 README(中英)+ Agent Skill + MCP 握手,三处一套、无自造工具名;`lengthNudge`(500 字符软阈值,只提示不拒绝)+ 超 2000 硬上限的关键词丢弃进 D73 handoff note,硬上限与保存路径不动。§18 第 (5) 项 → 已实现。*
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "open-memex",
3
- "version": "0.10.0-alpha.3",
3
+ "version": "0.10.0",
4
4
  "mcpName": "io.github.stoneskin/open-memex",
5
5
  "description": "Local-first memory layer and protocol for AI coding agents. Markdown source of truth, SQLite FTS5 index, zero cloud.",
6
6
  "keywords": [