@bcmyguest/openmem 0.0.0-stage → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (230) hide show
  1. package/.agents/skills/openmem/SKILL.md +34 -0
  2. package/LICENSE +190 -0
  3. package/README.md +140 -2
  4. package/bin/hindsight-project-index +530 -0
  5. package/cli/coding-agent/access-policy.mjs +38 -0
  6. package/cli/coding-agent/agents.mjs +34 -0
  7. package/cli/coding-agent/bank-from-cwd.mjs +140 -0
  8. package/cli/coding-agent/capabilities.mjs +34 -0
  9. package/cli/coding-agent/context-store.mjs +81 -0
  10. package/cli/coding-agent/context-trim.mjs +897 -0
  11. package/cli/coding-agent/file-io.mjs +72 -0
  12. package/cli/coding-agent/hook-harnesses.mjs +88 -0
  13. package/cli/coding-agent/hook-runner.mjs +284 -0
  14. package/cli/coding-agent/install-access.mjs +119 -0
  15. package/cli/coding-agent/install-context-proxy.mjs +333 -0
  16. package/cli/coding-agent/install-rtk.mjs +60 -0
  17. package/cli/coding-agent/install-shim.mjs +201 -0
  18. package/cli/coding-agent/opencode-config.mjs +107 -0
  19. package/cli/coding-agent/opencode-plugin.mjs +132 -0
  20. package/cli/coding-agent/opencode2-plugin/index.js +299 -0
  21. package/cli/coding-agent/recall-frame.mjs +29 -0
  22. package/cli/coding-agent/remove-upstream-access.mjs +122 -0
  23. package/cli/coding-agent/runtime.mjs +40 -0
  24. package/cli/coding-agent/session-db.mjs +626 -0
  25. package/cli/coding-agent/session-facade.mjs +86 -0
  26. package/cli/coding-agent/toml-edit.mjs +161 -0
  27. package/cli/cognee-mcp.mjs +260 -0
  28. package/cli/context-management.mjs +254 -0
  29. package/cli/feed-sync.mjs +27 -0
  30. package/cli/install-coding-agent.mjs +298 -0
  31. package/cli/install-memory-mcp.mjs +426 -0
  32. package/cli/is-main.mjs +18 -0
  33. package/cli/mcp/context-sql-runner.mjs +58 -0
  34. package/cli/mcp/context-store.mjs +670 -0
  35. package/cli/mcp/context.mjs +61 -0
  36. package/cli/mcp/export/cognee-dataset-name.mjs +12 -0
  37. package/cli/mcp/export/cognee.mjs +119 -0
  38. package/cli/mcp/export/hindsight.mjs +48 -0
  39. package/cli/mcp/export/index.mjs +26 -0
  40. package/cli/mcp/export/mem0.mjs +31 -0
  41. package/cli/mcp/knowledge.mjs +333 -0
  42. package/cli/mcp/llm-synthesize.mjs +50 -0
  43. package/cli/mcp/mem0-scope-id.mjs +22 -0
  44. package/cli/mcp/memory-facade.mjs +521 -0
  45. package/cli/mcp/memory-reflect-consolidate.mjs +120 -0
  46. package/cli/mcp/notebooklm-banks.mjs +21 -0
  47. package/cli/mcp/notebooklm-hindsight-rest.mjs +45 -0
  48. package/cli/mcp/notebooklm-provider.mjs +225 -0
  49. package/cli/mcp/pmm-records.mjs +23 -0
  50. package/cli/mcp/portability.mjs +158 -0
  51. package/cli/mcp/stdio.mjs +65 -0
  52. package/cli/mcp/unified-mcp-install.mjs +65 -0
  53. package/cli/mem0-mcp.mjs +108 -0
  54. package/cli/notebooklm-mcp.mjs +236 -0
  55. package/cli/openmem-mcp.mjs +38 -0
  56. package/cli/openmem.mjs +222 -0
  57. package/cli/roots.mjs +85 -0
  58. package/cli/schedule.mjs +176 -0
  59. package/cli/stack.mjs +438 -0
  60. package/cli/switch-memory-provider.mjs +358 -0
  61. package/cli/wiring-report.mjs +125 -0
  62. package/compose.bench.yaml +12 -0
  63. package/compose.yaml +227 -0
  64. package/config/coding-agent.json +13 -0
  65. package/config/hindsight.env.example +26 -0
  66. package/config/index.yml +54 -0
  67. package/config/litellm.lemond.yaml +40 -0
  68. package/config/litellm.ollama.yaml +33 -0
  69. package/config/notebooklm.yml +33 -0
  70. package/config/openmem.toml.example +124 -0
  71. package/docs/OPENMEM.md +23 -0
  72. package/docs/architecture.md +245 -0
  73. package/docs/assets/README.md +10 -0
  74. package/docs/assets/setup-demo.gif +0 -0
  75. package/docs/coding-agent-hooks.md +75 -0
  76. package/docs/coding-agent-scopes.md +22 -0
  77. package/docs/coding-agents.md +71 -0
  78. package/docs/cognee.md +37 -0
  79. package/docs/compatibility.md +80 -0
  80. package/docs/concepts.md +135 -0
  81. package/docs/context-management.md +255 -0
  82. package/docs/context-proxy.md +115 -0
  83. package/docs/context-research.md +215 -0
  84. package/docs/context-store.md +277 -0
  85. package/docs/hindsight.md +77 -0
  86. package/docs/index-sources-roadmap.md +38 -0
  87. package/docs/index-sources.md +39 -0
  88. package/docs/indexer-sinks.md +122 -0
  89. package/docs/indexing.md +57 -0
  90. package/docs/manual-stack-setup.md +74 -0
  91. package/docs/mcp-install-guide.md +4 -0
  92. package/docs/mcp-setup.md +104 -0
  93. package/docs/mem0.md +141 -0
  94. package/docs/memory-portability.md +73 -0
  95. package/docs/memory-providers.md +246 -0
  96. package/docs/meridian-litellm.md +198 -0
  97. package/docs/model-configuration.md +106 -0
  98. package/docs/notebooklm.md +83 -0
  99. package/docs/npx-install.md +172 -0
  100. package/docs/openmem-config.md +194 -0
  101. package/docs/provider-wizard-hooks.md +222 -0
  102. package/docs/quickstart.md +107 -0
  103. package/docs/setup-wizard.md +117 -0
  104. package/docs/sources-feeds.md +39 -0
  105. package/docs/sources-readwise.md +23 -0
  106. package/docs/sources-zotero.md +22 -0
  107. package/glama.json +4 -0
  108. package/indexer/Dockerfile +38 -0
  109. package/indexer/code_outline.py +146 -0
  110. package/indexer/feed_sync.py +107 -0
  111. package/indexer/index_config.py +290 -0
  112. package/indexer/pytest.ini +5 -0
  113. package/indexer/sinks/__init__.py +148 -0
  114. package/indexer/sinks/_http.py +44 -0
  115. package/indexer/sinks/cognee.py +329 -0
  116. package/indexer/sinks/hindsight.py +107 -0
  117. package/indexer/sinks/mem0.py +247 -0
  118. package/indexer/watch.py +240 -0
  119. package/package.json +96 -4
  120. package/providers/_template/compose.yaml +52 -0
  121. package/providers/cognee/OPERATOR.md +82 -0
  122. package/providers/cognee/compose.yaml +89 -0
  123. package/providers/cognee/lib/clack.mjs +157 -0
  124. package/providers/cognee/lib/coding-agent.mjs +26 -0
  125. package/providers/cognee/lib/cognee-mcp-install.mjs +153 -0
  126. package/providers/cognee/lib/config.mjs +98 -0
  127. package/providers/cognee/lib/env-file.mjs +194 -0
  128. package/providers/cognee/lib/env-setup.mjs +83 -0
  129. package/providers/cognee/lib/llm-env.mjs +382 -0
  130. package/providers/cognee/lib/ports.mjs +25 -0
  131. package/providers/cognee/lib/smoke.mjs +246 -0
  132. package/providers/cognee/lib/stack-env.mjs +134 -0
  133. package/providers/cognee/lib/state.mjs +101 -0
  134. package/providers/cognee/lib/summary.mjs +56 -0
  135. package/providers/cognee/lib/validate.mjs +97 -0
  136. package/providers/cognee/manifest.json +56 -0
  137. package/providers/cognee/wizard.mjs +134 -0
  138. package/providers/graphiti/manifest.json +15 -0
  139. package/providers/hindsight/lib/clack.mjs +347 -0
  140. package/providers/hindsight/lib/coding-agent.mjs +204 -0
  141. package/providers/hindsight/lib/config.mjs +79 -0
  142. package/providers/hindsight/lib/env-setup.mjs +72 -0
  143. package/providers/hindsight/lib/litellm-routes.mjs +196 -0
  144. package/providers/hindsight/lib/meridian.mjs +307 -0
  145. package/providers/hindsight/lib/model-backend.mjs +243 -0
  146. package/providers/hindsight/lib/notebooklm-mcp-install.mjs +127 -0
  147. package/providers/hindsight/lib/ollama-compose.mjs +51 -0
  148. package/providers/hindsight/lib/ports.mjs +30 -0
  149. package/providers/hindsight/lib/probe.mjs +54 -0
  150. package/providers/hindsight/lib/stack-env.mjs +125 -0
  151. package/providers/hindsight/lib/state.mjs +80 -0
  152. package/providers/hindsight/lib/summary.mjs +57 -0
  153. package/providers/hindsight/lib/validate.mjs +81 -0
  154. package/providers/hindsight/manifest.json +58 -0
  155. package/providers/hindsight/wizard.mjs +150 -0
  156. package/providers/letta/manifest.json +29 -0
  157. package/providers/mem0/OPERATOR.md +143 -0
  158. package/providers/mem0/compose.yaml +79 -0
  159. package/providers/mem0/lib/coding-agent.mjs +58 -0
  160. package/providers/mem0/lib/config.mjs +45 -0
  161. package/providers/mem0/lib/configure.mjs +201 -0
  162. package/providers/mem0/lib/dotenv.mjs +71 -0
  163. package/providers/mem0/lib/env-file.mjs +110 -0
  164. package/providers/mem0/lib/http.mjs +69 -0
  165. package/providers/mem0/lib/litellm.mjs +457 -0
  166. package/providers/mem0/lib/mcp-install.mjs +165 -0
  167. package/providers/mem0/lib/secret.mjs +30 -0
  168. package/providers/mem0/lib/smoke.mjs +179 -0
  169. package/providers/mem0/lib/stack-env.mjs +255 -0
  170. package/providers/mem0/lib/state.mjs +67 -0
  171. package/providers/mem0/lib/summary.mjs +88 -0
  172. package/providers/mem0/lib/url.mjs +106 -0
  173. package/providers/mem0/lib/validate.mjs +158 -0
  174. package/providers/mem0/manifest.json +47 -0
  175. package/providers/mem0/wizard.mjs +146 -0
  176. package/providers/zep/manifest.json +29 -0
  177. package/proxy/config.yaml +33 -0
  178. package/proxy/openmem_context/__init__.py +6 -0
  179. package/proxy/openmem_context/adapters.py +504 -0
  180. package/proxy/openmem_context/compress.py +53 -0
  181. package/proxy/openmem_context/core.py +606 -0
  182. package/proxy/openmem_context/embed.py +212 -0
  183. package/proxy/openmem_context/recall.py +49 -0
  184. package/proxy/openmem_context/session.py +70 -0
  185. package/proxy/openmem_context/sessiondb.py +276 -0
  186. package/proxy/openmem_context/settings.py +134 -0
  187. package/proxy/openmem_context/store.py +129 -0
  188. package/proxy/openmem_context/tokens.py +123 -0
  189. package/proxy/openmem_context/ttl.py +78 -0
  190. package/proxy/openmem_context_hook.py +34 -0
  191. package/scripts/check-pack.mjs +219 -0
  192. package/scripts/gen-masks.mjs +83 -0
  193. package/scripts/gen-masks.sh +4 -0
  194. package/scripts/promotion-provider-smoke.mjs +34 -0
  195. package/scripts/run-setup.sh +30 -0
  196. package/scripts/setup.mjs +107 -0
  197. package/scripts/setup.sh +5 -0
  198. package/scripts/spin-up.sh +45 -0
  199. package/scripts/wizard.sh +5 -0
  200. package/server.json +34 -0
  201. package/sync/Dockerfile +27 -0
  202. package/sync/memory_backend.py +87 -0
  203. package/sync/notebooklm_sync.py +664 -0
  204. package/wizard/bin/pm.mjs +97 -0
  205. package/wizard/bin/setup.mjs +33 -0
  206. package/wizard/lib/compose-version.mjs +32 -0
  207. package/wizard/lib/compose.mjs +70 -0
  208. package/wizard/lib/context-step.mjs +140 -0
  209. package/wizard/lib/curl.mjs +103 -0
  210. package/wizard/lib/docker-build.mjs +148 -0
  211. package/wizard/lib/exec.mjs +41 -0
  212. package/wizard/lib/indexer.mjs +6 -0
  213. package/wizard/lib/litellm-env.mjs +55 -0
  214. package/wizard/lib/load-machine-env.mjs +4 -0
  215. package/wizard/lib/log.mjs +32 -0
  216. package/wizard/lib/notebooklm-agent-access.mjs +92 -0
  217. package/wizard/lib/notebooklm.mjs +214 -0
  218. package/wizard/lib/parse-args.mjs +87 -0
  219. package/wizard/lib/paths.mjs +39 -0
  220. package/wizard/lib/pm-config.mjs +371 -0
  221. package/wizard/lib/ports.mjs +94 -0
  222. package/wizard/lib/prompts.mjs +85 -0
  223. package/wizard/lib/provider-wizard/helpers.mjs +98 -0
  224. package/wizard/lib/provider-wizard/run-steps.mjs +129 -0
  225. package/wizard/lib/providers/registry.mjs +262 -0
  226. package/wizard/lib/run-clack.mjs +108 -0
  227. package/wizard/lib/run-wizard.mjs +750 -0
  228. package/wizard/lib/validate-setup.mjs +62 -0
  229. package/wizard/lib/wizard-action.mjs +34 -0
  230. package/wizard/lib/wizard-conf.mjs +71 -0
@@ -0,0 +1,83 @@
1
+ # NotebookLM
2
+
3
+ Opt-in. Syncs notebooks, sources, notes, and (optionally) chats from Google NotebookLM into **`notebooklm::<notebook slug>`** banks on whichever memory provider you run (**Hindsight, mem0, or Cognee** — same `MEMORY_SINK` / indexer sinks as the project indexer). Each notebook gets one bank; duplicate titles get an ID suffix. The service runs only under the `notebooklm` compose profile, so a plain `docker compose up` never starts it unless that profile is enabled.
4
+
5
+ Risks:
6
+
7
+ - The default `client` backend uses the unofficial `notebooklm-py` library (pinned to 0.8.5 with the `markdown` extra). Google can change or block it at any time, and the sync then fails until the library is updated.
8
+ - The stored login is a Google session cookie set. Anyone who reads `~/.config/notebooklm` can act on that Google account. The directory is bind-mounted read-write into the sync container only, and masked from the indexer.
9
+ - Source text and notes are copied into your active memory backend (`notebooklm::*` scope). With Hindsight, retains may invoke the configured model server; mem0/Cognee store chunks via the indexer sink (Cognee runs cognify after each sync batch).
10
+
11
+ Login happens in **`npm run setup`** when you enable NotebookLM: the wizard installs
12
+ `notebooklm-py` (via `uv`) if needed, opens the Google sign-in browser, then writes
13
+ `NOTEBOOKLM_HOME=~/.config/notebooklm` into the process environment and `wizard.conf`.
14
+ Cookies land in `~/.config/notebooklm/profiles/default/storage_state.json` (bind-mounted
15
+ into the sync container). Re-run setup when Google expires the session.
16
+
17
+ Manual equivalent (only if you skip the wizard step):
18
+
19
+ ```sh
20
+ mkdir -p ~/.config/notebooklm
21
+ uv tool install 'notebooklm-py[browser]==0.8.5'
22
+ uv tool run --from 'notebooklm-py[browser]==0.8.5' python -m playwright install chromium
23
+ NOTEBOOKLM_HOME=~/.config/notebooklm notebooklm login # or: ... login --browser-cookies chrome
24
+ NOTEBOOKLM_HOME=~/.config/notebooklm notebooklm auth check --test --json
25
+ ```
26
+
27
+ Enable and start:
28
+
29
+ 1. Create `~/.config/hindsight/notebooklm.yml` with only the keys you change, for example:
30
+
31
+ ```yaml
32
+ enabled: true
33
+ interval_minutes: 60 # default 60
34
+ include_notebooks: ["Work*"] # fnmatch globs, case-insensitive; empty means all
35
+ ```
36
+
37
+ Compose refuses to start if this file is missing; create it first (`touch` is enough when there are no changes). Keys merge with `config/notebooklm.yml`; lists and scalars replace the shipped value.
38
+ 2. Start the service from the repository:
39
+
40
+ ```sh
41
+ docker compose --profile notebooklm up -d notebooklm
42
+ docker compose --profile notebooklm logs -f notebooklm
43
+ ```
44
+
45
+ Without a login, the service logs the login commands and retries every 15 minutes. With `enabled: false` it sleeps and checks again hourly.
46
+
47
+ ### Coding-agent access (`notebooklmAccess`)
48
+
49
+ Synced notebooks land in Hindsight banks named `notebooklm::<slug>`. The coding-agent plugin still uses per-repo `coding-agent::*` banks only. Agents can **list/search** NotebookLM banks on demand (no auto-inject) via an opt-in MCP.
50
+
51
+ **Primary paths** (pick one):
52
+
53
+ 1. **Wizard** — when you enable NotebookLM sync in setup (step 6), the wizard asks whether to allow coding agents to query those banks via MCP (`notebooklm_list_banks` / `notebooklm_search`). Default is yes after sync is enabled. On yes it writes `notebooklmAccess.enabled: true` into a **real** `~/.hindsight/coding-agent.json` (if that path was a symlink to the repo template, it is copied to a regular file first so machine settings never land in `config/coding-agent.json`) and merges a `notebooklm` entry into `~/.cursor/mcp.json`.
54
+ 2. **`npx @bcmyguest/openmem install-mcp`** — same enable + MCP merge when NotebookLM is already on (`notebooklm.yml` `enabled: true`) or with `--notebooklm` / `--notebooklm-only`. See [Coding agents and MCP](coding-agents.md).
55
+
56
+ Restart Cursor Agent after either path. Manual JSON remains a fallback if you skipped the prompt or need to change allowlists later:
57
+
58
+ ```json
59
+ "notebooklmAccess": {
60
+ "enabled": true,
61
+ "banks": ["*"]
62
+ }
63
+ ```
64
+
65
+ - `enabled: false` (shipped default) — the NotebookLM MCP exits immediately; setup does not register it.
66
+ - `banks: ["*"]` — every `notebooklm::*` bank; or list slugs / full ids (e.g. `"sitrep-analysis"` or `"notebooklm::sitrep-analysis"`).
67
+
68
+ MCP tools (stdio server `cli/notebooklm-mcp.mjs`):
69
+
70
+ | Tool | Purpose |
71
+ | --- | --- |
72
+ | `notebooklm_list_banks` | Allowlisted `notebooklm::*` banks (`bank_id`, `fact_count`) |
73
+ | `notebooklm_search` | Fast full-text list on one allowlisted bank (`bank_id` + `query`; optional `mode=recall` for slow LLM synthesis) |
74
+
75
+ When `enabled` is true, the wizard or `install-mcp` merges a `notebooklm` entry into `~/.cursor/mcp.json` (does not overwrite `hindsight` or `codegraph`). To run the server by hand:
76
+
77
+ ```sh
78
+ # after enabling notebooklmAccess (wizard, install-mcp, or hand-edit)
79
+ node cli/notebooklm-mcp.mjs
80
+ # or: openmem-notebooklm-mcp
81
+ ```
82
+
83
+ Enterprise backend (`backend: enterprise`): the documented NotebookLM Enterprise REST API lists recently viewed notebooks only. It returns no source or note text, so this backend syncs notebook metadata only. It is untested against a live Enterprise project.
@@ -0,0 +1,172 @@
1
+ # Installing with npx
2
+
3
+ `openmem` is an npm package whose CLI runs the same setup wizard as `./setup` in a
4
+ checkout. The package name lives in one place, the `name` field of `package.json` (the CLI
5
+ command stays `openmem` if you switch to a scoped name such as
6
+ `openmem`; older installs may still see `openmem`).
7
+
8
+ Requirements: Node ≥ 24 (`node:sqlite` with FTS5, for the session context store), Docker with the compose v2
9
+ plugin (`docker compose version`), `curl`. Linux and macOS are supported; there is no
10
+ bash dependency in the CLI (macOS ships bash 3.2).
11
+
12
+ Wizard phases and flags live in the repository: [docs/setup-wizard.md](setup-wizard.md). Runtime architecture: [architecture.md](architecture.md).
13
+
14
+ ## Install paths
15
+
16
+ ```sh
17
+ npx @bcmyguest/openmem # interactive setup, nothing installed globally
18
+ npx @bcmyguest/openmem setup # same as the bare command
19
+ npx @bcmyguest/openmem configure --yes # non-interactive setup and start
20
+ npx @bcmyguest/openmem up # start stack
21
+ npx @bcmyguest/openmem install-mcp --yes
22
+ npx @bcmyguest/openmem switch mem0 --yes
23
+ npm i -g openmem # then: openmem setup | up | down | ...
24
+ # From a git checkout (package not required on the registry):
25
+ git clone https://github.com/bcmyguest/openmem && cd openmem
26
+ npm install && npm run setup # same wizard; also: ./setup
27
+ node cli/openmem.mjs doctor # any other command
28
+ ```
29
+
30
+ No global `pm` command is installed: it collides with Android's `pm`. `./pm` and `npm run pm`
31
+ exist only inside a checkout; prefer `npx @bcmyguest/openmem …` (or `node cli/openmem.mjs …` in a checkout).
32
+
33
+ ### Commands
34
+
35
+ | Command | What it does |
36
+ | --- | --- |
37
+ | `openmem` / `setup` | interactive setup; extra flags go to the wizard (`--yes`, `--dry-run`, `--no-smoke`, `--rebuild`) |
38
+ | `configure` | non-interactive setup and start (`setup --yes`) |
39
+ | `up` / `down` | start / stop the stack with the saved configuration |
40
+ | `doctor` | validate the configuration without the smoke test |
41
+ | `install-mcp` | merge **openmem** unified MCP (`memory_*`) by default; NotebookLM when sync is on or `--notebooklm`; native mem0/Cognee with `--provider-only`; `--no-unified-mcp` to skip |
42
+ | `install-coding-agent` | `install-mcp` + session hooks for all providers; `--provider-plugin` runs `@vectorize-io/hindsight-coding-agents` instead of OpenMem hooks on Hindsight stacks (saved as `[agents] integration`; `--no-provider-plugin` reverts) |
43
+ | `context <on\|off\|status>` | context management (smart compaction) per agent: saves `[context_proxy.agents]` (`--agents claude-code,opencode1,codex,opencode2`, default all), re-runs `install-coding-agent`, which starts or stops the `context-proxy` service (`--dry-run`); `status` lists each agent, proxy health and pointed agents ([Context management](context-management.md)) |
44
+ | `wiring` | report `MEMORY_PROVIDER`, API URLs, and `mcp.json` alignment (`--json`) |
45
+ | `switch <provider>` | change `MEMORY_PROVIDER` (optional stack restart + MCP; no data migration) |
46
+ | `providers [--json]` | list the provider manifests shipped in the package with their status |
47
+ | `init [--force] [--dry-run] [--json]` | create or upgrade the stack root without running the wizard |
48
+ | `--version`, `--help` | |
49
+
50
+ `--version`, `--help` and `providers` touch nothing on disk. Every other command first
51
+ brings the stack root up to date, then runs the wizard.
52
+
53
+ ## Stack root
54
+
55
+ Under npx the package directory is a read-only, garbage-collected cache, but the wizard
56
+ writes `.env`, `.build-hash` and `compose.override.yaml` and drives compose files and docker
57
+ build contexts relative to its repo root. The CLI therefore keeps a stable, writable
58
+ **stack root**:
59
+
60
+ ```
61
+ ${PM_STACK_ROOT:-${XDG_DATA_HOME:-~/.local/share}/personal-memory/stack}
62
+ ```
63
+
64
+ Layout after `openmem init`:
65
+
66
+ ```
67
+ stack/
68
+ .pm-stack-manifest.json sha256 of every shipped file as last written (do not edit)
69
+ compose.yaml shipped
70
+ compose.bench.yaml shipped
71
+ indexer/ bin/ sync/ shipped (docker build contexts)
72
+ config/ shipped templates: *.example, index.yml, notebooklm.yml,
73
+ litellm.*.yaml, coding-agent.json
74
+ providers/<id>/compose*.yaml shipped provider compose fragments
75
+ .env .build-hash compose.override.yaml personal-memory.toml ... generated: never touched
76
+ ```
77
+
78
+ The wizard code itself (`wizard/`, `scripts/`, provider manifests and hooks) is **not**
79
+ copied: it runs from the package, so `@clack/prompts` resolves from the package's own `node_modules`.
80
+ Per-user config stays where it always was: `~/.config/hindsight/` and
81
+ `~/.config/personal-memory/config.toml`.
82
+
83
+ ### Environment variables
84
+
85
+ | Variable | Meaning |
86
+ | --- | --- |
87
+ | `PM_STACK_ROOT` | Mutable stack root = the wizard's `ctx.repoRoot` (compose project dir, `.env`, generated files, docker build context). Overrides the default location. |
88
+ | `PM_USE_CHECKOUT=1` | Use the package directory itself as the stack root and copy nothing. Wins over `PM_STACK_ROOT`. A directory containing `.git` next to `cli/` behaves this way automatically. |
89
+ | `PM_PACKAGE_ROOT` | Read-only code root (provider manifests and hooks, `scripts/setup.mjs`, `wizard/lib`), never written to. Set by the CLI; defaults to the package directory. |
90
+ | `PM_PROJECT` | compose project name (default `openmem`). |
91
+ | `XDG_DATA_HOME` | Base for the default stack root (must be absolute). |
92
+
93
+ `wizard/bin/setup.mjs` and `wizard/bin/pm.mjs` read `PM_STACK_ROOT` and `PM_PACKAGE_ROOT`
94
+ (defaulting both to the checkout root) and export them to child processes. Running them
95
+ directly from inside an npm package directory without `PM_STACK_ROOT` is refused.
96
+
97
+ ## Upgrades
98
+
99
+ Each command compares the package's shipped files with the stack root using the manifest:
100
+
101
+ - Missing file: created (restored if it was shipped before).
102
+ - Identical: left alone.
103
+ - On disk equals what the previous version wrote: overwritten with the new version
104
+ (atomic write: temp file in the same directory, then rename).
105
+ - Modified by you, or present but not tracked, or a symlink: **kept**, reported as
106
+ `CONFLICT <path>`. The wizard still runs against your version. Merge by hand, or run
107
+ `openmem init --force`, which overwrites it and saves yours as
108
+ `<file>.user-<timestamp>.bak`.
109
+ - No longer shipped: left in place and reported as a note.
110
+ - Generated files (`.env`, `.build-hash`, `compose.override.yaml`, `personal-memory.toml`,
111
+ anything not in the manifest) are never overwritten or deleted.
112
+
113
+ After an upgrade the wizard's `.build-hash` check rebuilds the images when their sources
114
+ changed.
115
+
116
+ ## Uninstall
117
+
118
+ ```sh
119
+ openmem down # stops the stack (docker compose down)
120
+ npm uninstall -g openmem # if installed globally
121
+ rm -rf "${XDG_DATA_HOME:-$HOME/.local/share}/personal-memory" # the stack root
122
+ ```
123
+
124
+ `~/.config/hindsight/` (secrets, `index.yml`) and `~/.config/personal-memory/` are kept
125
+ until you delete them yourself.
126
+
127
+ ## Linux and macOS notes
128
+
129
+ - Linux: the user must be able to run `docker` without sudo. `host.docker.internal`
130
+ is mapped through `extra_hosts` in `compose.yaml`.
131
+ - macOS: use Docker Desktop or Colima with the compose v2 plugin and allow file sharing
132
+ for your home directory (the indexer bind-mounts `$HOME` read-only).
133
+ - Windows is untested; use WSL2.
134
+
135
+ ## Troubleshooting
136
+
137
+ - `CONFLICT <path>`: see Upgrades. It is a warning; the stack still works.
138
+ - `cannot prepare the stack root`: the directory is not writable. Set `PM_STACK_ROOT` to
139
+ a writable path.
140
+ - `read-only ... Run it through the CLI`: you ran `wizard/bin/*.mjs` from inside the npm
141
+ cache. Use `npx @bcmyguest/openmem` or set `PM_STACK_ROOT`.
142
+ - `missing dependencies (@clack/prompts ...)`: reinstall the package (`npm i -g
143
+ openmem`), or in a checkout run `npm install` at the checkout root.
144
+ - `... is not valid JSON` for `.pm-stack-manifest.json`: delete the manifest; shipped
145
+ files that differ from the package are then reported as untracked conflicts instead of
146
+ being overwritten.
147
+ - Port already in use: set `HINDSIGHT_PORT` / `HINDSIGHT_CP_PORT`, or let the wizard pick
148
+ a free port.
149
+
150
+ ## Maintainers: publishing
151
+
152
+ Before the first publish (these are human steps; nothing in this repo does them):
153
+
154
+ 1. **License.** The repository ships `LICENSE` (Apache-2.0) and `package.json` matches.
155
+ 2. **Public source.** `npm publish --provenance` needs a public GitHub repository.
156
+ 3. **Pack gate.** `npm run pack:check` must pass (no operator home paths in shipped files).
157
+ 4. **Create the npm package.** Log in with 2FA (`npm login`), then publish `0.1.0` once by
158
+ hand (`npm publish --access public`). The name `openmem` was verified free on
159
+ 2026-10-19 (so was `openmem`); re-check with `npm view openmem`.
160
+ 5. **Trusted publishing.** On npmjs.com, package Settings -> Trusted Publisher -> GitHub
161
+ Actions: owner `bcmyguest`, repository `openmem`, workflow `release.yml`.
162
+ 6. **Release.** Run the *Prepare release* workflow on `main` (or bump `package.json`, tag
163
+ `vX.Y.Z`, and push the tag). `release.yml` runs CI, then publishes with `--provenance`
164
+ when the version is not already on npm, then creates the GitHub Release.
165
+
166
+ Local rehearsal without publishing:
167
+
168
+ ```sh
169
+ npm run pack:check
170
+ npm pack --pack-destination /tmp/pm-pack
171
+ cd "$(mktemp -d)" && npx --yes /tmp/pm-pack/openmem-*.tgz --version
172
+ ```
@@ -0,0 +1,194 @@
1
+ # Mise-like config (`config.toml`)
2
+
3
+ openmem is a **meta** orchestrator (see [architecture.md](architecture.md) on the site; [memory-providers.md](memory-providers.md) in the repo for contributors). Its
4
+ declarative config is intentionally similar to **[mise](https://mise.jdx.dev/)**:
5
+
6
+ | mise | openmem |
7
+ | --- | --- |
8
+ | `~/.config/mise/config.toml` + `.mise.toml` | `~/.config/personal-memory/config.toml` + `openmem.toml` |
9
+ | `[tools]` plugins | `[memory] provider = "…"` + `providers/*/manifest.json` |
10
+ | `[env]` | Host ports / compose project under `[stack]` |
11
+ | `[tasks]` | `npx @bcmyguest/openmem up` / `down` / `doctor` / `install-mcp` / `switch` |
12
+ | `mise install` applies config | `npx @bcmyguest/openmem configure --yes` applies config |
13
+
14
+ We **do not** replace mise (or mem0/Hindsight native config). You can use both: mise installs
15
+ Node/Docker; this file describes **how agent memory runs on this machine**.
16
+
17
+ ## Files
18
+
19
+ | Path | Purpose |
20
+ | --- | --- |
21
+ | `config/openmem.toml.example` | Shipped template; copy to your user config |
22
+ | `~/.config/personal-memory/config.toml` | **Primary** user config (gitignored on your machine) |
23
+ | `./openmem.toml` | Optional per-checkout overrides (legacy `personal-memory.toml` still loads) |
24
+ | `~/.config/hindsight/wizard.conf` | Last configure run (`MEMORY_PROVIDER`, ports, …); updated when setup applies |
25
+
26
+ ## Precedence
27
+
28
+ 1. CLI flags (`npx @bcmyguest/openmem setup --up`, `--down`, …)
29
+ 2. Environment (`MEMORY_PROVIDER`, `PM_BACKEND`, `HINDSIGHT_PORT`, …)
30
+ 3. `~/.config/personal-memory/config.toml`
31
+ 4. `openmem.toml` / `config/openmem.toml` in the checkout stack root (legacy `personal-memory.toml` names still load)
32
+ 5. Existing `wizard.conf` (provider, ports, model backend from the last wizard run)
33
+
34
+ Interactive `./setup` uses this file for **defaults** (Clack pre-selects); your answers still win.
35
+
36
+ ## Sections
37
+
38
+ - **`[memory]`** — `provider` id (`hindsight`, future `mem0`, …).
39
+ - **`[stack]`** — compose project name and host ports.
40
+ - **`[wizard]`** — default action (`configure`, `up`, `down`).
41
+ - **`[model]`** — meta **LLM routing** for the Hindsight stack only; other providers add
42
+ `[memory.<id>]` tables (see `[memory.mem0]` below).
43
+ - **`[memory.mem0]`** — the mem0 provider (URL-only: you run a mem0 REST server, this stack points
44
+ the indexer at it). Never holds a key.
45
+
46
+ ```toml
47
+ [memory]
48
+ provider = "mem0"
49
+
50
+ [memory.mem0]
51
+ api_url = "http://127.0.0.1:18888" # the server as this host sees it
52
+ # env_file = "~/.config/personal-memory/mem0.env" # optional; where MEM0_API_KEY lives
53
+ ```
54
+
55
+ `api_url` is resolved as: `MEM0_API_URL`, then `PM_MEM0_API_URL`, then this table, then the stack
56
+ `.env`, then `wizard.conf`, then the default `http://127.0.0.1:18888`. The indexer container
57
+ gets the same URL with `127.0.0.1`/`localhost` rewritten to `host.docker.internal`
58
+ (`MEM0_INDEXER_URL` in the stack `.env`). The API key is written by `./setup` (from
59
+ `MEM0_API_KEY` under `--yes`, or a prompt) as the single line `MEM0_API_KEY=your-key-here` in the env file
60
+ (`MEM0_ENV_FILE`, then `env_file`, then `~/.config/personal-memory/mem0.env`; mode 600 in a mode
61
+ 700 directory). You can also create that file by hand; a server run with `AUTH_DISABLED` needs
62
+ none. The key never goes into the stack `.env`, `wizard.conf` or `config.toml`. A non-default
63
+ `env_file` is exported to compose as `MEM0_ENV_FILE` for the run; export it yourself when you run
64
+ `docker compose` by hand.
65
+ - **`[memory.cognee]`** — the Cognee provider (a pinned `cognee/cognee` container on a loopback
66
+ port, single-user mode, no authentication). Never holds a key.
67
+
68
+ ```toml
69
+ [memory]
70
+ provider = "cognee"
71
+
72
+ [memory.cognee]
73
+ port = 8766 # host port; the container listens on 8000
74
+ env_file = "~/.config/personal-memory/cognee.env" # cognify settings (GLiNER/FastEmbed by default)
75
+ ```
76
+
77
+ `port` is resolved as: `COGNEE_PORT`, then this table, then the stack `.env`, then the port
78
+ saved in `wizard.conf`, then the default `8766`. Both env file locations follow the same order:
79
+ the variable (`COGNEE_ENV_FILE`, `COGNEE_INDEXER_ENV_FILE`), then `env_file` /
80
+ `indexer_env_file` in this table, then the stack `.env`, then the defaults
81
+ `~/.config/personal-memory/cognee.env` and `~/.config/personal-memory/cognee-indexer.env`.
82
+ The second file is optional and is the one the indexer container reads; it holds only
83
+ `COGNEE_API_KEY`. A non-default location (`~` is expanded, a relative path is taken relative
84
+ to the stack root) is written to the stack `.env` as a path, so a plain `docker compose` finds
85
+ the same files; the file contents never go there. Keep custom files inside
86
+ `~/.config/personal-memory`: that is the directory hidden from the indexer's read-only `$HOME`
87
+ mount, and setup warns about a file elsewhere under `$HOME`.
88
+
89
+ Documents are stored but not searchable until cognify can run. **Default cognify path** (no
90
+ API key): local GLiNER extraction + FastEmbed embeddings. `./setup` / `--yes` with no LLM
91
+ answers writes that block into the env file (mode 600 in a mode 700 directory), merging into
92
+ any existing file and replacing only the keys it was given; it prints the path and the key
93
+ names, never a value. Where the answers come from:
94
+
95
+ | Variable (environment) | Written to the env file as |
96
+ | --- | --- |
97
+ | `PM_COGNEE_LLM_API_KEY`; under `--yes` or `--dry-run` also plain `LLM_API_KEY` | `LLM_API_KEY` |
98
+ | `PM_COGNEE_LLM_PROVIDER`, `PM_COGNEE_LLM_MODEL`, `PM_COGNEE_LLM_ENDPOINT` | `LLM_PROVIDER`, `LLM_MODEL`, `LLM_ENDPOINT` |
99
+ | `PM_COGNEE_EMBEDDING_PROVIDER`, `_MODEL`, `_ENDPOINT`, `_DIMENSIONS` | the same names without `PM_COGNEE_` |
100
+ | `PM_COGNEE_EMBEDDING_API_KEY`; under `--yes` or `--dry-run` also plain `EMBEDDING_API_KEY` | `EMBEDDING_API_KEY` |
101
+ | `PM_COGNEE_HUGGINGFACE_TOKENIZER` | `HUGGINGFACE_TOKENIZER` |
102
+ | `PM_COGNEE_GRAPH_EXTRACTOR`, `PM_COGNEE_AUTO_FEEDBACK` | `GRAPH_EXTRACTOR`, `AUTO_FEEDBACK` |
103
+
104
+ **Default (GLiNER + FastEmbed)** when no cognify answers are given:
105
+
106
+ ```env
107
+ GRAPH_EXTRACTOR=gliner_demo
108
+ EMBEDDING_PROVIDER=fastembed
109
+ EMBEDDING_MODEL=BAAI/bge-small-en-v1.5
110
+ EMBEDDING_DIMENSIONS=384
111
+ AUTO_FEEDBACK=false
112
+ ```
113
+
114
+ Compose pins `HF_HOME` / `FASTEMBED_CACHE_PATH` under the `cognee-data` volume so model
115
+ downloads survive container recreates. Changing embedding dimensions requires rebuilding
116
+ vectors (prune datasets or recreate the volume).
117
+
118
+ **Local Ollama** (LLM + Ollama embeddings): set `PM_COGNEE_LLM_PROVIDER=ollama` (and optionally
119
+ `PM_COGNEE_LLM_MODEL=...`). The wizard fills in a complete local block when fields are missing:
120
+
121
+ ```bash
122
+ PM_COGNEE_LLM_PROVIDER=ollama PM_COGNEE_LLM_MODEL=llama3.2 ./setup --yes
123
+ ```
124
+
125
+ That writes (defaults only where unset):
126
+
127
+ ```env
128
+ LLM_PROVIDER=ollama
129
+ LLM_API_KEY=ollama
130
+ LLM_ENDPOINT=http://host.docker.internal:11434
131
+ EMBEDDING_PROVIDER=ollama
132
+ EMBEDDING_MODEL=nomic-embed-text:latest
133
+ EMBEDDING_ENDPOINT=http://host.docker.internal:11434/api/embed
134
+ EMBEDDING_DIMENSIONS=768
135
+ ```
136
+
137
+ Ollama LLM + FastEmbed is also valid: set `PM_COGNEE_EMBEDDING_PROVIDER=fastembed` (and model /
138
+ dimensions) alongside `PM_COGNEE_LLM_PROVIDER=ollama`. Use the bare Ollama host (`…:11434`),
139
+ not `…:11434/v1` — Cognee's Ollama path appends `/api/generate` itself. **Host Ollama**
140
+ (listening on the Docker bridge or all interfaces): `host.docker.internal` is correct.
141
+ **Compose Ollama** published as `127.0.0.1:11434` only (this repo's default): that address is
142
+ unreachable via `host.docker.internal` — set `PM_COGNEE_OLLAMA_COMPOSE=1` (or reuse
143
+ `MEMORY_OLLAMA_IN_COMPOSE=1` from a Hindsight Ollama setup) so the wizard writes
144
+ `http://ollama:11434`, and attach the Cognee container to the same Docker network as the
145
+ `ollama` service. Slow local models may need `COGNEE_SKIP_CONNECTION_TEST=true` in
146
+ `cognee.env`. The interactive setup offers GLiNER+FastEmbed (default), Ollama, or an OpenAI
147
+ key (hidden prompt). Do not type the key into a non-Clack `./setup` prompt — put
148
+ `LLM_API_KEY` in `cognee.env` (mode 600) by hand (Ollama accepts any non-empty value), or use
149
+ the UI / `--yes` with `PM_COGNEE_LLM_API_KEY`. Secret variables (including plain `LLM_API_KEY` /
150
+ `EMBEDDING_API_KEY` / `COGNEE_API_KEY`) are removed from the wizard's environment before it
151
+ starts docker. Secrets never go into the stack `.env`, `wizard.conf`, `config.toml` or the
152
+ summary. Authentication is off: `ENABLE_BACKEND_ACCESS_CONTROL` is pinned false in compose and
153
+ overrides the env file (see `providers/cognee/compose.yaml`).
154
+ - **`[tasks.*]`** — descriptions for `npx @bcmyguest/openmem` / `./pm` (see `wizard/bin/pm.mjs` and
155
+ `cli/openmem.mjs`). Built-in tasks:
156
+ - `configure` / `up` / `down` / `doctor` — map to the setup wizard with the matching flags
157
+ - `install-mcp` — runs `cli/install-memory-mcp.mjs` (NotebookLM MCP on Hindsight; mem0/Cognee
158
+ with `--provider-only`; merges into `~/.cursor/mcp.json`)
159
+ - `switch <provider>` — runs `cli/switch-memory-provider.mjs`: sets `MEMORY_PROVIDER` in
160
+ `wizard.conf`, updates `[memory].provider` in `config.toml` when present
161
+ (`updatePmConfigProvider`), optionally restarts the stack and re-points provider MCP.
162
+ Flags: `--yes`, `--dry-run`, `--no-stack`, `--no-mcp`. Does not migrate data between
163
+ backends. See [README — Switch memory provider](../README.md#switch-memory-provider).
164
+ - `feed-sync` — runs `cli/feed-sync.mjs` (RSS + OpenAlex → `feeds.out_dir` in `index.yml`).
165
+ - `install-coding-agent` — unified MCP + session hooks (all providers); `--provider-plugin` / `--no-provider-plugin` set `[agents] integration`; see
166
+ [coding-agent hooks](coding-agent-hooks.md).
167
+
168
+ ## MCP (unified `memory_*` tools)
169
+
170
+ | Variable | Default | Purpose |
171
+ | --- | --- | --- |
172
+ | `PM_MEMORY_ADD_ASYNC` | `true` (implicit) | Hindsight `memory_add` sends `async: true` unless the tool passes `async: false`. Async ingest returns in ~200ms and avoids Cursor/Claude MCP `-32001` timeouts while extraction runs in the background. Set `PM_MEMORY_ADD_ASYNC=false` to force synchronous ingest. |
173
+
174
+ After changing MCP code, re-run `npx @bcmyguest/openmem install-coding-agent --yes` (or copy the repo into `~/.local/share/personal-memory/agent-runtime`) and **restart** the openmem MCP server in Cursor.
175
+
176
+ ## Apply
177
+
178
+ ```bash
179
+ cp config/openmem.toml.example ~/.config/personal-memory/config.toml
180
+ # edit provider, ports, model backend
181
+ npx @bcmyguest/openmem configure --yes
182
+ # from a checkout: node cli/openmem.mjs configure --yes
183
+ ```
184
+
185
+ After a successful configure, the wizard **writes back** `~/.config/personal-memory/config.toml` so
186
+ the file matches what is running (mise-style source of truth).
187
+
188
+ ## Doctor
189
+
190
+ ```bash
191
+ npx @bcmyguest/openmem doctor
192
+ ```
193
+
194
+ Runs the same checks as setup step “Validate memory setup” without mutating the stack.