lorekeep 0.4.0__tar.gz → 0.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- lorekeep-0.6.0/.release-please-manifest.json +3 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/AGENTS.md +11 -7
- {lorekeep-0.4.0 → lorekeep-0.6.0}/CHANGELOG.md +14 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/PKG-INFO +6 -4
- {lorekeep-0.4.0 → lorekeep-0.6.0}/README.md +5 -3
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/README.md +3 -2
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/overview.md +4 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/guides/compile.md +12 -5
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/guides/serve.md +8 -1
- lorekeep-0.6.0/docs/guides/wiki.md +179 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/pyproject.toml +1 -1
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/__init__.py +1 -1
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/cli.py +128 -40
- lorekeep-0.6.0/src/lorekeep/importer/codex.py +287 -0
- lorekeep-0.6.0/src/lorekeep/importer/opencode.py +255 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/integrations/codex.py +32 -6
- lorekeep-0.6.0/src/lorekeep/integrations/opencode.py +60 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/paths.py +4 -0
- lorekeep-0.6.0/src/lorekeep/wiki.py +377 -0
- lorekeep-0.6.0/tests/test_import_codex.py +238 -0
- lorekeep-0.6.0/tests/test_import_opencode.py +170 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_session_hook.py +75 -22
- lorekeep-0.6.0/tests/test_wiki.py +577 -0
- lorekeep-0.4.0/.release-please-manifest.json +0 -3
- lorekeep-0.4.0/src/lorekeep/integrations/opencode.py +0 -43
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.commitlintrc.json +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.github/workflows/auto-merge-release-please.yml +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.github/workflows/ci.yml +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.github/workflows/lint-commits.yml +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.github/workflows/release-please.yml +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.gitignore +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.lorekeep/config.yaml.example +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.lorekeep/schema.json +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/.pre-commit-config.yaml +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/CLAUDE.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/LICENSE +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/SECURITY.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/cover.jpeg +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/agent.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/data-model.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/evaluation.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/journal.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/permission.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/pipeline.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/serve-mcp.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/architecture/temporal.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/guides/backup.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/guides/data-home.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/guides/getting-started.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/guides/import.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/superpowers/plans/2026-06-26-lorekeep-dotdir-backup.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/docs/superpowers/specs/2026-06-26-lorekeep-dotdir-layout-design.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/release-please-config.json +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/scripts/check-conventional-commit.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/agent.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/backup.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/compile/__init__.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/compile/extract.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/compile/ingest.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/compile/providers.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/compile/resolve.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/compile/writer.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/config.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/defaults.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/eval/__init__.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/eval/construction.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/eval/gold.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/eval/retrieval.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/facts_io.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/importer/__init__.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/importer/claude.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/importer/cursor.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/integrations/__init__.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/integrations/claude_code.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/integrations/common.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/integrations/cursor.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/integrations/detect.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/journal.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/mcp_server.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/models.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/perm/__init__.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/perm/ns.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/pipeline.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/schema_io.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/store/__init__.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/store/fts.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/src/lorekeep/store/graph.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/conftest.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/claude-session/memory/MEMORY.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/claude-session/memory/lorekeep-design.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/claude-session/memory/lorekeep-project-state.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/claude-session/test.jsonl +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/gold/payments.facts.jsonl +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/raw/backend/payments.md +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/retrieval/questions.json +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/fixtures/schema.json +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_agent_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_agent_detect.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_backup.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_backup_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_compile_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_config.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_construction_eval.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_defaults.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_determinism.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_doctor_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_eval_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_extract.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_fts.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_graph_store.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_import_claude.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_import_cursor.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_ingest.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_init_chain.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_init_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_integrations.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_mcp_add_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_mcp_reload.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_mcp_server.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_mcp_write_tools.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_merge_journals.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_models.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_paths.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_perm.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_pipeline.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_providers.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_resolve.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_retrieval_eval.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_schema_io.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_serve_cli.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_smoke.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_watch_e2e.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_watch_sessions.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/tests/test_writer.py +0 -0
- {lorekeep-0.4.0 → lorekeep-0.6.0}/uv.lock +0 -0
|
@@ -25,12 +25,13 @@ uv run lorekeep <command> # run the CLI in dev mode
|
|
|
25
25
|
| Command | Purpose |
|
|
26
26
|
|---|---|
|
|
27
27
|
| `init` | Bootstrap data home (config + schema + raw/graph dirs) |
|
|
28
|
-
| `compile` | `raw/*.md` → `graph/facts.jsonl` + `manifest.json` (runs the LLM pipeline) |
|
|
28
|
+
| `compile` | `raw/*.md` → `graph/facts.jsonl` + `manifest.json` + `wiki/` (runs the LLM pipeline, auto-generates wiki) |
|
|
29
29
|
| `check` | Validate compiled graph loads, no dangling edges (exit 1 on failure) |
|
|
30
30
|
| `eval` | Tier-1 construction P/R/F1 vs gold corpus + structure metrics |
|
|
31
|
+
| `wiki` | Regenerate `wiki/` from `facts.jsonl` (Obsidian-compatible markdown) |
|
|
31
32
|
| `serve [--transport stdio\|http]` | Run the MCP server (8 read + 5 write tools) |
|
|
32
33
|
| `mcp add --agent claude\|cursor\|codex\|opencode --ns NS` | Write agent MCP config |
|
|
33
|
-
| `import --from claude\|cursor` | Import agent sessions into `raw/`
|
|
34
|
+
| `import --from claude\|cursor\|codex\|opencode` | Import agent sessions into `raw/` |
|
|
34
35
|
| `doctor` | Verify install: graph loads, schema valid, a tool responds |
|
|
35
36
|
| `backup [--init <remote-url>]` | Commit + push `.lorekeep/` to your private backup git repo |
|
|
36
37
|
| `version` | Print version |
|
|
@@ -40,11 +41,11 @@ uv run lorekeep <command> # run the CLI in dev mode
|
|
|
40
41
|
## Architecture: two strictly separated phases
|
|
41
42
|
|
|
42
43
|
```
|
|
43
|
-
COMPILE (offline, curator): raw/<ns>/*.md → ingest → extract(LLM) → resolve → writer → facts.jsonl
|
|
44
|
+
COMPILE (offline, curator): raw/<ns>/*.md → ingest → extract(LLM) → resolve → writer → facts.jsonl → wiki/
|
|
44
45
|
SERVE (runtime, per device): facts.jsonl → GraphStore → ScopedGraph(ns) → MCP → agent
|
|
45
46
|
```
|
|
46
47
|
|
|
47
|
-
`compile` mutates `facts.jsonl
|
|
48
|
+
`compile` mutates `facts.jsonl` and auto-generates `wiki/`; `resolve` regenerates wiki only on actual merge (gated on `merge_count > 0`); `serve` reads `facts.jsonl` and lazily reloads on mtime change. Write tools (propose_fact, link_facts, etc.) append to `pending/` journals; resolve merges them into the graph. Wiki regen is **best-effort** — never blocks `compile` or `resolve`. Wiki builds into a temp dir then `os.rename` swaps into place (atomic — never partially populated).
|
|
48
49
|
|
|
49
50
|
### Compile pipeline (`src/lorekeep/compile/`, orchestrated by `pipeline.py`)
|
|
50
51
|
`ingest` chunks markdown with `path:line` provenance → `extract` calls the LLM provider for schema-constrained nodes/edges/aliases (per-chunk SHA-256 hash cache → unchanged chunks return cached output, giving byte-stable recompiles) → `resolve` collapses alias variants to canonical entities and quarantines invalid facts → `writer` emits **sorted** `facts.jsonl` + `manifest.json`. Failures are skip-and-log (partial compile is valid); errors/quarantine land in the manifest.
|
|
@@ -59,8 +60,11 @@ Pydantic, all `frozen=True`, `extra="forbid"`. `Node` / `Edge` are the two `kind
|
|
|
59
60
|
### Serve (`mcp_server.py`)
|
|
60
61
|
`FastMCP` with 8 read tools (`search`, `get_node`, `neighbors`, `at_time`, `history`, `changes`, `list_namespaces`, `schema`) and 5 write tools (`propose_fact`, `link_facts`, `flag_contradiction`, `update_fact`, `suggest_improvement`). Module-global `ScopedGraph` is set by `configure()`; `_require()` lazy-reloads when `facts.jsonl` mtime changes (so `compile` is visible without reconnecting). Tools are plain functions registered with `@mcp.tool()` but stay directly callable — **tests invoke them directly, no MCP transport**. The writer uses atomic `os.replace` so lazy-reload never reads a half-written file.
|
|
61
62
|
|
|
63
|
+
### Wiki (`wiki.py`)
|
|
64
|
+
Pure JSONL → markdown transform (no LLM). `generate_wiki(graph_dir, wiki_dir)` reads `facts.jsonl` via `GraphStore` → entity pages (`entities/<type>/<slug>.md` with YAML frontmatter, wikilinks, props table), `index.md` (catalog), `overview.md` (stats dashboard), `log.md` (append-only, preserved across regen). Builds into `.wiki-build.tmp` then `os.rename` swaps into place (atomic). `_slug()` replaces `:`/`/` with `-`; collisions raise `ValueError`. YAML scalars quoted via `json.dumps` so IDs like `svc:payments-api` parse correctly. Props table escapes `\|`, collapses newlines, serializes non-strings. Regenerates on every `facts.jsonl` mutation: `compile` (single regen — `_do_auto_resolve` returns bool, compile skips if resolve already regend), `resolve` (gated on merge/flag), daemon auto-resolve (on actual merge). Best-effort — never blocks compile or resolve.
|
|
65
|
+
|
|
62
66
|
### Path resolution (`paths.py`)
|
|
63
|
-
Pure (no I/O), 4-tier precedence high→low: explicit `LOREKEEP_RAW/OUT/CACHE/SCHEMA/CONFIG` env → `LOREKEEP_HOME` → **dev mode** (`.lorekeep/` present in CWD, or `LOREKEEP_DEV=1`; auto-detected in a source checkout) — all data lives under `cwd/.lorekeep/` (`config.yaml`, `schema.json`, `raw/`, `graph/`, `cache.json`), mirroring the `LOREKEEP_HOME` layout → XDG (`platformdirs`). Running `uv run lorekeep …` from the repo uses the repo's own `.lorekeep/` data home with zero migration.
|
|
67
|
+
Pure (no I/O), 4-tier precedence high→low: explicit `LOREKEEP_RAW/OUT/CACHE/SCHEMA/CONFIG/WIKI` env → `LOREKEEP_HOME` → **dev mode** (`.lorekeep/` present in CWD, or `LOREKEEP_DEV=1`; auto-detected in a source checkout) — all data lives under `cwd/.lorekeep/` (`config.yaml`, `schema.json`, `raw/`, `graph/`, `wiki/`, `pending/`, `cache.json`), mirroring the `LOREKEEP_HOME` layout → XDG (`platformdirs`). Running `uv run lorekeep …` from the repo uses the repo's own `.lorekeep/` data home with zero migration.
|
|
64
68
|
|
|
65
69
|
### Provider pluggability (`compile/providers.py`)
|
|
66
70
|
`LiteLLMProvider` (OpenAI / Anthropic / DashScope/Qwen / Ollama via litellm model strings) and `FakeProvider` (tests/offline). Model is set in `config.yaml` as a litellm string.
|
|
@@ -79,11 +83,11 @@ Pure (no I/O), 4-tier precedence high→low: explicit `LOREKEEP_RAW/OUT/CACHE/SC
|
|
|
79
83
|
- **Conventional Commits are enforced** — a `commit-msg` pre-commit hook (`scripts/check-conventional-commit.py`) plus CI (`lint-commits.yml`, checking both commit messages and PR title). Types: `build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test`. Merge commits are exempt.
|
|
80
84
|
- **Releases are automated** — `release-please` runs on every push to `main` (`feat`=minor, `fix`=patch, `!`=major), opens an auto-merging Release PR, tags a GitHub Release on merge, and publishes to PyPI via OIDC trusted publishing (inline publish job). Do not version-bump by hand.
|
|
81
85
|
- **Determinism is a hard requirement** — recompiling unchanged input must be byte-identical (kept green by `test_determinism.py`). Preserve the sorted-output / cache behavior in `writer.py` and `extract.py` when changing the pipeline.
|
|
82
|
-
- **`graph/facts.jsonl
|
|
86
|
+
- **`graph/facts.jsonl`, `graph/manifest.json`, and `wiki/` are gitignored** (regenerated by `compile`); `.lorekeep/schema.json` is committed.
|
|
83
87
|
|
|
84
88
|
## Tests
|
|
85
89
|
|
|
86
|
-
`~
|
|
90
|
+
`~300` tests, no network. Gold corpus in `tests/fixtures/gold/`, raw fixtures in `tests/fixtures/raw/`. Compile/serve/import tests inject `FakeProvider` via `patch_make_provider` / `patch_make_import_provider` conftest fixtures to pin paths and avoid the LLM — follow this pattern for any new CLI test rather than hitting a real provider.
|
|
87
91
|
|
|
88
92
|
## Cursor Cloud specific instructions
|
|
89
93
|
|
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.6.0](https://github.com/manhhailua/lorekeep/compare/v0.5.0...v0.6.0) (2026-06-29)
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
### Features
|
|
7
|
+
|
|
8
|
+
* add Obsidian-compatible wiki output from compiled graph ([#68](https://github.com/manhhailua/lorekeep/issues/68)) ([e3e53f2](https://github.com/manhhailua/lorekeep/commit/e3e53f26f8f97c947bafe4a19485410f4deb14f6))
|
|
9
|
+
|
|
10
|
+
## [0.5.0](https://github.com/manhhailua/lorekeep/compare/v0.4.0...v0.5.0) (2026-06-28)
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
### Features
|
|
14
|
+
|
|
15
|
+
* add import --from codex and --from opencode + hooks for all 4 agents ([#66](https://github.com/manhhailua/lorekeep/issues/66)) ([8a1e364](https://github.com/manhhailua/lorekeep/commit/8a1e364e19cf1b45944cea2ed89d9d7468fec9c6))
|
|
16
|
+
|
|
3
17
|
## [0.4.0](https://github.com/manhhailua/lorekeep/compare/v0.3.1...v0.4.0) (2026-06-28)
|
|
4
18
|
|
|
5
19
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: lorekeep
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.6.0
|
|
4
4
|
Summary: Temporal knowledge graph for AI agents via MCP
|
|
5
5
|
Project-URL: Homepage, https://github.com/manhhailua/lorekeep
|
|
6
6
|
Project-URL: Repository, https://github.com/manhhailua/lorekeep
|
|
@@ -269,7 +269,8 @@ src/lorekeep/
|
|
|
269
269
|
store/{graph,fts}.py GraphStore + optional FTS cache
|
|
270
270
|
perm/ns.py ScopedGraph permission chokepoint
|
|
271
271
|
mcp_server.py FastMCP + 8 read + 5 write tools
|
|
272
|
-
|
|
272
|
+
wiki.py Obsidian-compatible wiki generator
|
|
273
|
+
integrations/{claude_code,cursor,codex,opencode,common}.py
|
|
273
274
|
pipeline.py, cli.py
|
|
274
275
|
eval/{gold,construction,retrieval}.py
|
|
275
276
|
tests/ ~140 tests
|
|
@@ -278,9 +279,9 @@ docs/ README.md index, architecture/, guides/
|
|
|
278
279
|
|
|
279
280
|
## Status
|
|
280
281
|
|
|
281
|
-
**v1 (implemented)** — compile pipeline + serve (store/permission/MCP read/integrations) + import + agent daemon (watch/ingest/lint/suggest/status) + journal + resolve + data-home + dev mode + lazy-reload + backup + eval. Published to PyPI as `lorekeep`.
|
|
282
|
+
**v1 (implemented)** — compile pipeline + serve (store/permission/MCP read+write/4-agent integrations) + import (Claude/Cursor/Codex/opencode) + session-end hooks + agent daemon (watch/ingest/lint/suggest/status) + journal + resolve + data-home + dev mode + lazy-reload + backup + eval + **wiki** (Obsidian-compatible markdown output). Published to PyPI as `lorekeep`.
|
|
282
283
|
|
|
283
|
-
**Phase 2 (planned)** —
|
|
284
|
+
**Phase 2 (planned)** — streamable-HTTP team server, OIDC/SSO, embeddings/hybrid search, scheduled nightly lint/suggest in daemon, schema evolve, full Tier-2 benchmark datasets (HotpotQA/CronQuestions) and the bespoke Tier-3 Lorekeep-Reason eval.
|
|
284
285
|
|
|
285
286
|
## Documentation
|
|
286
287
|
|
|
@@ -291,6 +292,7 @@ The [`docs/`](docs/README.md) index is the entry point.
|
|
|
291
292
|
- [Importing agent sessions](docs/guides/import.md)
|
|
292
293
|
- [Compiling the knowledge graph](docs/guides/compile.md)
|
|
293
294
|
- [Serving the graph to coding agents](docs/guides/serve.md)
|
|
295
|
+
- [Browsing the wiki](docs/guides/wiki.md)
|
|
294
296
|
- [Data home & path resolution](docs/guides/data-home.md)
|
|
295
297
|
- [Backing up the data home](docs/guides/backup.md)
|
|
296
298
|
|
|
@@ -247,7 +247,8 @@ src/lorekeep/
|
|
|
247
247
|
store/{graph,fts}.py GraphStore + optional FTS cache
|
|
248
248
|
perm/ns.py ScopedGraph permission chokepoint
|
|
249
249
|
mcp_server.py FastMCP + 8 read + 5 write tools
|
|
250
|
-
|
|
250
|
+
wiki.py Obsidian-compatible wiki generator
|
|
251
|
+
integrations/{claude_code,cursor,codex,opencode,common}.py
|
|
251
252
|
pipeline.py, cli.py
|
|
252
253
|
eval/{gold,construction,retrieval}.py
|
|
253
254
|
tests/ ~140 tests
|
|
@@ -256,9 +257,9 @@ docs/ README.md index, architecture/, guides/
|
|
|
256
257
|
|
|
257
258
|
## Status
|
|
258
259
|
|
|
259
|
-
**v1 (implemented)** — compile pipeline + serve (store/permission/MCP read/integrations) + import + agent daemon (watch/ingest/lint/suggest/status) + journal + resolve + data-home + dev mode + lazy-reload + backup + eval. Published to PyPI as `lorekeep`.
|
|
260
|
+
**v1 (implemented)** — compile pipeline + serve (store/permission/MCP read+write/4-agent integrations) + import (Claude/Cursor/Codex/opencode) + session-end hooks + agent daemon (watch/ingest/lint/suggest/status) + journal + resolve + data-home + dev mode + lazy-reload + backup + eval + **wiki** (Obsidian-compatible markdown output). Published to PyPI as `lorekeep`.
|
|
260
261
|
|
|
261
|
-
**Phase 2 (planned)** —
|
|
262
|
+
**Phase 2 (planned)** — streamable-HTTP team server, OIDC/SSO, embeddings/hybrid search, scheduled nightly lint/suggest in daemon, schema evolve, full Tier-2 benchmark datasets (HotpotQA/CronQuestions) and the bespoke Tier-3 Lorekeep-Reason eval.
|
|
262
263
|
|
|
263
264
|
## Documentation
|
|
264
265
|
|
|
@@ -269,6 +270,7 @@ The [`docs/`](docs/README.md) index is the entry point.
|
|
|
269
270
|
- [Importing agent sessions](docs/guides/import.md)
|
|
270
271
|
- [Compiling the knowledge graph](docs/guides/compile.md)
|
|
271
272
|
- [Serving the graph to coding agents](docs/guides/serve.md)
|
|
273
|
+
- [Browsing the wiki](docs/guides/wiki.md)
|
|
272
274
|
- [Data home & path resolution](docs/guides/data-home.md)
|
|
273
275
|
- [Backing up the data home](docs/guides/backup.md)
|
|
274
276
|
|
|
@@ -24,8 +24,9 @@ Concepts and design — how the system fits together.
|
|
|
24
24
|
How to use it.
|
|
25
25
|
|
|
26
26
|
- [**Getting started**](guides/getting-started.md) — install → compile → serve → backup in 10 minutes. Start here.
|
|
27
|
-
- [Importing agent sessions](guides/import.md) — Claude Code + Cursor → `raw/`.
|
|
27
|
+
- [Importing agent sessions](guides/import.md) — Claude Code + Cursor + Codex + opencode → `raw/`.
|
|
28
28
|
- [Compiling the knowledge graph](guides/compile.md) — `raw/*.md` → `facts.jsonl` + resolve pending.
|
|
29
|
-
- [Serving the graph to coding agents](guides/serve.md) — wire Claude Code / Cursor / Codex over MCP, write tools, daemon.
|
|
29
|
+
- [Serving the graph to coding agents](guides/serve.md) — wire Claude Code / Cursor / Codex / opencode over MCP, write tools, daemon.
|
|
30
|
+
- [Browsing the wiki](guides/wiki.md) — human-readable Obsidian-compatible markdown view of the graph.
|
|
30
31
|
- [Data home & path resolution](guides/data-home.md) — env / `LOREKEEP_HOME` / dev mode / XDG.
|
|
31
32
|
- [Backing up the data home](guides/backup.md) — `lorekeep backup` to a private git repo: setup, restore, multi-device conflicts.
|
|
@@ -46,6 +46,10 @@ Lorekeep exists to let an agent reason about a domain **systematically and with
|
|
|
46
46
|
│
|
|
47
47
|
▼
|
|
48
48
|
facts.jsonl THE store (sorted, byte-stable)
|
|
49
|
+
│
|
|
50
|
+
├──► wiki/*.md human view (Obsidian-compatible)
|
|
51
|
+
│ atomic swap, regen on
|
|
52
|
+
│ every facts mutation
|
|
49
53
|
│
|
|
50
54
|
▼ (git / S3 sync)
|
|
51
55
|
┌───────────────────────────────────────────┐
|
|
@@ -35,12 +35,17 @@ provider:
|
|
|
35
35
|
uv run lorekeep compile
|
|
36
36
|
```
|
|
37
37
|
|
|
38
|
-
Produces `graph/facts.jsonl` + `graph/manifest.json
|
|
39
|
-
|
|
40
|
-
|
|
38
|
+
Produces `graph/facts.jsonl` + `graph/manifest.json` + auto-generates `wiki/`
|
|
39
|
+
(Obsidian-compatible markdown, regenerated atomically). If `pending/`
|
|
40
|
+
journals exist, compile also merges them via `_do_auto_resolve` — in that
|
|
41
|
+
case wiki regenerates once from the resolved facts (never double).
|
|
42
|
+
Re-running is idempotent: unchanged input yields byte-identical files
|
|
43
|
+
(extraction is cached under `.lorekeep/cache.json`).
|
|
41
44
|
|
|
42
45
|
**What compile does not do:** compile processes only `raw/`. Agent-proposed
|
|
43
46
|
facts in `pending/` journals are merged by `resolve` (see step 5).
|
|
47
|
+
Compile does re-merge pending journals as a convenience, but standalone
|
|
48
|
+
`resolve` is needed if you add journals after compile.
|
|
44
49
|
|
|
45
50
|
## 4. Resolve pending facts (manual)
|
|
46
51
|
|
|
@@ -50,7 +55,8 @@ uv run lorekeep resolve
|
|
|
50
55
|
|
|
51
56
|
Merges all pending agent-proposed facts from `pending/` journals into `facts.jsonl`.
|
|
52
57
|
Facts are gated by confidence: high (≥0.8) auto-merge, medium (0.5-0.8) merge
|
|
53
|
-
with review flag, low (<0.5) quarantine.
|
|
58
|
+
with review flag, low (<0.5) quarantine. Wiki regenerates only if facts
|
|
59
|
+
actually changed (`merge_count > 0` or `flagged_count > 0`).
|
|
54
60
|
|
|
55
61
|
Resolve also runs automatically: the daemon (`lorekeep agent watch`) detects
|
|
56
62
|
new pending entries on every poll cycle (default 60s interval).
|
|
@@ -95,7 +101,8 @@ the next resolve.
|
|
|
95
101
|
|
|
96
102
|
## Next: serve to agents
|
|
97
103
|
|
|
98
|
-
See [serve.md](serve.md) to expose the graph to Claude Code / Cursor / Codex over MCP
|
|
104
|
+
See [serve.md](serve.md) to expose the graph to Claude Code / Cursor / Codex over MCP,
|
|
105
|
+
or [wiki.md](wiki.md) to browse the graph as Obsidian markdown.
|
|
99
106
|
|
|
100
107
|
## Data home
|
|
101
108
|
|
|
@@ -101,13 +101,20 @@ rebuilt automatically. So the workflow is:
|
|
|
101
101
|
|
|
102
102
|
```bash
|
|
103
103
|
<edit raw/.../*.md>
|
|
104
|
-
uvx lorekeep compile # rebuilds facts.jsonl
|
|
104
|
+
uvx lorekeep compile # rebuilds facts.jsonl + regenerates wiki/
|
|
105
105
|
# OR: agent ingest + resolve # propose facts interactively, merge journals
|
|
106
106
|
# OR: lorekeep agent watch # daemon does compile + resolve automatically
|
|
107
107
|
# next query from the agent sees the new graph — NO reconnect needed
|
|
108
|
+
# wiki/ pages also regenerated automatically
|
|
108
109
|
```
|
|
109
110
|
|
|
110
111
|
Connect the MCP server **once**; graph updates via `compile`, `resolve`, or
|
|
111
112
|
the daemon are visible immediately. Reconnect is only needed for **code**
|
|
112
113
|
changes (rare; the serve path is stable) or **scope** changes (`.mcp.json`
|
|
113
114
|
`LOREKEEP_NS`).
|
|
115
|
+
|
|
116
|
+
## Human view: wiki
|
|
117
|
+
|
|
118
|
+
The same `facts.jsonl` is also projected to **Obsidian-compatible markdown**
|
|
119
|
+
in `wiki/` — one page per node, `[[wikilinks]]` for edges, YAML frontmatter
|
|
120
|
+
for Dataview. See [wiki.md](wiki.md).
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# Browsing the wiki (human view)
|
|
2
|
+
|
|
3
|
+
The compiled graph (`facts.jsonl`) is machine-readable — consumed by agents over MCP. The **wiki** is its human-readable projection: Obsidian-compatible markdown pages with wikilinks, YAML frontmatter, and graph view.
|
|
4
|
+
|
|
5
|
+
## Quick start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
uvx lorekeep compile # auto-generates wiki/ after compile
|
|
9
|
+
# OR
|
|
10
|
+
uvx lorekeep wiki # regenerate wiki/ from existing facts.jsonl
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Open the `wiki/` directory in Obsidian:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
# In your data home (default XDG):
|
|
17
|
+
open ~/.local/share/lorekeep/wiki
|
|
18
|
+
|
|
19
|
+
# Or dev mode (repo co-located):
|
|
20
|
+
open .lorekeep/wiki
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
In Obsidian: **File → Open vault** → select the `wiki/` directory.
|
|
24
|
+
|
|
25
|
+
## Lifecycle
|
|
26
|
+
|
|
27
|
+
The wiki is a **derived artifact** — fully regenerable from `facts.jsonl`, never the reverse:
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
raw/*.md → compile → facts.jsonl → wiki/*.md
|
|
31
|
+
↑ ↓
|
|
32
|
+
agent queries human browses
|
|
33
|
+
(MCP tools) (Obsidian)
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
| Consumer | Reads | Format |
|
|
37
|
+
|---|---|---|
|
|
38
|
+
| Agent (MCP) | `facts.jsonl` | JSONL — structured queries, temporal, ns-scoped |
|
|
39
|
+
| Human (Obsidian) | `wiki/` | Markdown — browsable, searchable, graph view |
|
|
40
|
+
|
|
41
|
+
### When wiki regenerates
|
|
42
|
+
|
|
43
|
+
Wiki regenerates on **every `facts.jsonl` mutation**:
|
|
44
|
+
|
|
45
|
+
| Trigger | Who | Wiki regen? |
|
|
46
|
+
|---|---|---|
|
|
47
|
+
| `compile` (raw → facts.jsonl) | Curator | Yes — single regen at end |
|
|
48
|
+
| `compile` + pending resolve | Curator | Yes — `_do_auto_resolve` regens if merge happened; otherwise compile's own regen covers it. **Never double.** |
|
|
49
|
+
| `resolve` (manual, merge happened) | Curator | Yes — gated on `merge_count > 0` or `flagged_count > 0` |
|
|
50
|
+
| `resolve` (quarantine-only, no merge) | Curator | No — facts.jsonl unchanged |
|
|
51
|
+
| Daemon auto-resolve (merge) | Daemon | Yes — `_do_auto_resolve` regens on actual merge |
|
|
52
|
+
| `lorekeep wiki` (manual) | Human | Yes — force regen |
|
|
53
|
+
|
|
54
|
+
### Atomic swap
|
|
55
|
+
|
|
56
|
+
Wiki pages build into a temp sibling directory (`.wiki-build.tmp`), then `os.rename` swaps it into place. The wiki directory is **never partially populated** — Obsidian always sees either the old or the new version, never a mix. This is safe even with the vault open during regeneration.
|
|
57
|
+
|
|
58
|
+
## Output structure
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
wiki/
|
|
62
|
+
├── index.md # catalog of all entities, grouped by type
|
|
63
|
+
├── log.md # append-only generation log
|
|
64
|
+
├── overview.md # graph stats dashboard
|
|
65
|
+
└── entities/
|
|
66
|
+
└── <type>/
|
|
67
|
+
└── <slug>.md # one page per node
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Entity pages
|
|
71
|
+
|
|
72
|
+
Each node becomes a markdown page with:
|
|
73
|
+
|
|
74
|
+
- **YAML frontmatter** — `id`, `type`, `ns`, `valid_from`, `valid_to`, `sources`, `tags` (Obsidian Dataview compatible). All scalars are quoted so IDs containing colons (`svc:payments-api`) parse correctly.
|
|
75
|
+
|
|
76
|
+
Example entity page (`entities/service/svc-payments-api.md`):
|
|
77
|
+
|
|
78
|
+
```markdown
|
|
79
|
+
---
|
|
80
|
+
id: "svc:payments-api"
|
|
81
|
+
type: "service"
|
|
82
|
+
ns: ["backend"]
|
|
83
|
+
valid_from: "2024-01-15"
|
|
84
|
+
valid_to: ""
|
|
85
|
+
sources:
|
|
86
|
+
- "raw/backend/payments.md:3"
|
|
87
|
+
tags: ["service", "backend", "entity"]
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
# payments-api
|
|
91
|
+
|
|
92
|
+
> ID: `svc:payments-api`
|
|
93
|
+
|
|
94
|
+
## Properties
|
|
95
|
+
|
|
96
|
+
| Key | Value |
|
|
97
|
+
|---|---|
|
|
98
|
+
| name | payments-api |
|
|
99
|
+
| lang | go |
|
|
100
|
+
|
|
101
|
+
## Relationships
|
|
102
|
+
|
|
103
|
+
### depends_on →
|
|
104
|
+
|
|
105
|
+
- [[svc-auth]] (2024-01-15 → 2025-03-01)
|
|
106
|
+
|
|
107
|
+
## Timeline
|
|
108
|
+
|
|
109
|
+
- **2024-01-15**: Valid from
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
### Properties table
|
|
113
|
+
|
|
114
|
+
Pipe characters in values are escaped (`\|`), newlines collapsed to spaces, and non-string values (integers, booleans, lists) serialized via `json.dumps` — preventing table corruption.
|
|
115
|
+
|
|
116
|
+
### Slug naming
|
|
117
|
+
|
|
118
|
+
Node IDs are sanitized to filename-safe slugs for wikilinks (`:` → `-`, `/` → `-`). If two distinct node IDs collide to the same slug, `generate_wiki` raises `ValueError` rather than silently overwriting.
|
|
119
|
+
|
|
120
|
+
### index.md
|
|
121
|
+
|
|
122
|
+
Catalog of all entities, grouped by node type (`## Services`, `## Teams`, `## Decisions`). Each entry is a `[[wikilink]]` with a one-line summary.
|
|
123
|
+
|
|
124
|
+
### overview.md
|
|
125
|
+
|
|
126
|
+
Graph-level dashboard: node/edge counts by type, temporal range, namespace breakdown, and compile metadata (run ID, facts hash).
|
|
127
|
+
|
|
128
|
+
### log.md
|
|
129
|
+
|
|
130
|
+
Append-only log of wiki generation events. **Preserved across regenerations** — prior entries survive verbatim. Counts come from the live `GraphStore`, not the (potentially stale) manifest:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
## [2026-06-29T21:53:00Z] wiki | run_id=abc123, 4 nodes, 2 edges
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## Obsidian tips
|
|
137
|
+
|
|
138
|
+
- **Graph view** — all `[[wikilinks]]` render as a force-directed graph. This is the entity-relationship graph visualized.
|
|
139
|
+
- **Backlinks** — Obsidian automatically shows inbound references at the bottom of each page (equivalent to incoming edges).
|
|
140
|
+
- **Dataview** — the YAML frontmatter enables structured queries. Example: list all services written in Go:
|
|
141
|
+
```dataview
|
|
142
|
+
LIST FROM #service WHERE lang = "go"
|
|
143
|
+
```
|
|
144
|
+
- **Tags** — each page is tagged with `[<type>, <ns>, "entity"]`, enabling filtered views.
|
|
145
|
+
|
|
146
|
+
## CLI
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
lorekeep wiki # regenerate wiki/ from facts.jsonl (force)
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
Wiki generation also runs automatically after every `facts.jsonl` mutation:
|
|
153
|
+
|
|
154
|
+
- **`compile`** — always regens (unless `_do_auto_resolve` already regend after merging pending journals; never double).
|
|
155
|
+
- **`resolve`** — regens only if facts actually changed (`merge_count > 0` or `flagged_count > 0`). Quarantine-only resolves skip wiki regen.
|
|
156
|
+
- **Daemon auto-resolve** — regens on actual merge (`_do_auto_resolve` returns `True`).
|
|
157
|
+
|
|
158
|
+
You never need to run `lorekeep wiki` manually unless you want to force a refresh.
|
|
159
|
+
|
|
160
|
+
## Determinism
|
|
161
|
+
|
|
162
|
+
Re-generating the wiki from unchanged `facts.jsonl` yields **byte-identical** pages (entity pages, index, overview). The only exception is `log.md`, which is append-only by design.
|
|
163
|
+
|
|
164
|
+
Wiki generation is **best-effort** — if it fails (e.g. slug collision, disk error), the triggering command (`compile`, `resolve`) still succeeds. `facts.jsonl` is never blocked by a wiki failure.
|
|
165
|
+
|
|
166
|
+
## Path resolution
|
|
167
|
+
|
|
168
|
+
The wiki lives at `<data-home>/wiki/` (same level as `raw/` and `graph/`). Override with:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
LOREKEEP_WIKI=/path/to/wiki uvx lorekeep wiki
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
See [data-home.md](data-home.md) for the full 4-tier path resolution.
|
|
175
|
+
|
|
176
|
+
## Next
|
|
177
|
+
|
|
178
|
+
- [Compile guide](compile.md) — how `facts.jsonl` is produced.
|
|
179
|
+
- [Serve guide](serve.md) — how agents consume the same graph over MCP.
|