onemem 0.1.0__tar.gz

onemem-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,41 @@
+name: CI
+
+on:
+  push:
+    branches: [main, release-prep]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Install (all extras + dev)
+        run: pip install -e ".[all,dev]"
+      - name: Run tests
+        run: pytest -q
+      - name: CLI entry point
+        run: men --help
+
+  clean-install:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Build wheel
+        run: |
+          pip install build
+          python -m build
+      - name: Install the built wheel with extras
+        run: |
+          WHEEL=$(ls dist/*.whl)
+          pip install "${WHEEL}[all]"
+      - name: Verify entry points and imports
+        run: |
+          men --help
+          python -c "import onemem, onemem.config, onemem.db, onemem.fact_retrieval, onemem.mcp_server, onemem.pipeline"

onemem-0.1.0/.gitignore

@@ -0,0 +1,27 @@
+.env
+
+# python
+__pycache__/
+*.pyc
+.venv/
+venv/
+.meniscus-dev/
+.pytest_cache/
+*.egg-info/
+.DS_Store
+
+.mcp.json
+
+# benchmark data & frozen memories (large; regenerable via bench/)
+bench/data/
+bench/out/
+bench/memories*/
+bench/*.log
+
+demo/
+
+notes/
+
+# build artifacts
+/dist/
+/build/

onemem-0.1.0/DESIGN.md

@@ -0,0 +1,130 @@
+# oneMEM — Design
+
+A local, structured, long-term memory for AI agents and people. oneMEM receives
+unstructured text (chat turns, notes, imported files), distills it into **atomic facts**,
+stores everything **append-only** in one SQLite file, and makes it retrievable through a
+**deterministic hybrid search** — no LLM in the read path. Any MCP-capable agent reads and
+writes the same memory.
+
+---
+
+## 1. Principles
+
+- **Append-only.** Raw events are never overwritten or deleted. Facts are only ever added.
+  Every fact traces back to the exact source event it came from, so nothing is lost and
+  everything is recoverable.
+- **Deterministic retrieval.** Reading memory is a fixed formula over indexes — no model
+  call. The same query against the same data always returns the same result, and every
+  ranking decision is inspectable with plain SQL.
+- **Local and single-file.** One SQLite database on the user's machine. No server, no
+  cloud, no account. Back it up by copying a file.
+- **Small models at the edges.** An LLM is used only at *write* time (to distill facts) and
+  optionally at *read* time (to phrase an answer). The retrieval itself never calls a model,
+  so a small/cheap model is enough and nothing heavy runs locally.
+
+## 2. Data model
+
+All state lives in one SQLite file:
+
+- **`events`** — raw ingested content, verbatim, timestamped, with a `source`. The source of
+  truth. `extraction_status` tracks whether facts have been distilled yet.
+- **`facts`** — atomic statements distilled from an event by the LLM. Each fact links to its
+  `event_id` (provenance) and carries its text and position.
+- **`entities`** — canonical named things (people, projects, tools) with a normalized form
+  for matching; `entity_aliases` maps surface variants to a canonical entity.
+- **`fact_entity_edges`** — which entities each fact mentions (the entity retrieval door).
+- **`extractions`** — one row per distillation run (provider, model, prompt version), so the
+  provenance of every fact is auditable.
+- **`fact_embeddings`** — a `sqlite-vec` (vec0) virtual table holding each fact's vector.
+- **`facts_fts`** — an FTS5 virtual table over fact text (the keyword door), kept in sync.
+
+Facts and their embeddings/edges are only appended. Correcting the record means adding a new
+event, never mutating an old one.
+
+## 3. Ingestion pipeline
+
+1. **Intake** — content is stored as one or more `events` (large inputs are chunked). Intake
+   always commits first, so an event is durable even if distillation later fails. Duplicate
+   content is deduplicated by a content hash of `(source, content)`.
+2. **Distillation** — a small LLM (via whichever provider is configured — see §7) reads each
+   pending event and extracts atomic facts. Filler turns with nothing worth keeping produce
+   zero facts and are marked complete (not retried forever).
+3. **Entity reconciliation** — entities named by the facts are resolved to canonical rows
+   (via normalization + aliases) and linked through `fact_entity_edges`.
+4. **Embedding** — each fact is embedded locally with `bge-base-en-v1.5` (768-d) and written
+   to `fact_embeddings`; `facts_fts` indexes the text.
+
+Processing runs as a parallel batch and is resumable: if the model becomes unavailable,
+processed events stay done and the rest remain pending for the next run.
+
+## 4. Retrieval — deterministic hybrid fusion
+
+Retrieval scores candidate facts through three independent **doors**, then fuses them. No
+model is involved.
+
+- **Vector door** — cosine similarity between the query embedding and each fact embedding.
+  Queries are embedded with an instruction prefix; facts are embedded plain. A wide candidate
+  pool is fetched (far more than the return limit) so fusion sees a broad set.
+- **Keyword door** — FTS5 (BM25) over fact text. The query is tokenized and OR-matched, so a
+  fact matching *any* term is a candidate; the fact's *rank position* feeds fusion.
+- **Entity door** — facts linked to an explicitly named entity (used when a caller passes an
+  entity argument).
+
+**Fusion (magnitude noisy-OR).** Each door contributes independently; a single strong door
+can win on its own merit:
+
+```
+fused = 1 − (1 − W_VECTOR·vector) · (1 − W_FTS·fts) · (1 − W_ENTITY·entity)
+```
+
+where `vector` is the raw cosine, `fts` is a reciprocal-rank term `RRF_K/(RRF_K + position)`,
+and `entity` is 1 if the entity door matched. Vector is the senior partner; keyword is a
+booster/tiebreaker.
+
+**Adaptive cut.** Results are sorted by fused score and cut on the *shape* of the curve, not
+a fixed `k`: keep every fact scoring at least a ratio of the top score, bounded to
+`[MIN_RETURN, limit]`. A sharp, specific query returns a tight set; a broad query returns
+more. Counting/enumeration questions (detected by trigger phrases like "how many", "every")
+bypass the ratio cut and return the whole plateau.
+
+**Source collapse.** If the facts selected from one event cost as many tokens as the raw
+event, the raw event is returned instead — full fidelity when distillation didn't save
+anything.
+
+## 5. Episodic reconstruction
+
+oneMEM does **not** store episodes or threads. Instead, "what happened around this moment"
+is reconstructed *at read time*: given an anchor (a matched fact, an event id, or a
+timestamp), events in a time window are segmented into sessions by gaps longer than
+`SESSION_GAP_SECONDS`, and the relevant segment is returned with its facts. Episodes are a
+read-time view over the append-only events, never materialized state.
+
+## 6. Interfaces
+
+- **CLI (`men`)** — `init` (interactive setup: provider, key, capture, MCP wiring), `add`,
+  `import`, `process`, `ask`, `watch` (silent transcript capture), `sql`, `tables`, `status`,
+  `doctor`, `help`, and event inspection (`list events`, `show event`). `men ask` plans a
+  query, retrieves deterministically, and phrases an answer.
+- **MCP** — exactly **two** tools, so an agent's surface stays minimal:
+  - `onemem_recall` — the single read entry point. One call; the argument selects the
+    operation (topic search, a time window, a reconstructed session, or the raw source
+    behind a fact). Returns structured facts with timestamps and source ids — not a finished
+    answer.
+  - `onemem_log` — an invisible background write; the agent logs what's worth remembering
+    without announcing it.
+
+## 7. Providers
+
+- **LLM** — bring your own key. Any OpenAI-compatible backend works (OpenRouter, OpenAI,
+  Gemini, Groq, xAI, Hugging Face, a local Ollama server, or any other OpenAI-compatible
+  endpoint via `provider = "custom"`); Anthropic is wired separately against Claude's native
+  API.
+- **Embeddings** — `bge-base-en-v1.5` run **locally** (768-d), no API. Query embeddings use a
+  retrieval instruction prefix.
+
+## 8. What makes it inspectable
+
+Because it is one SQLite file and retrieval is a fixed formula, nothing is hidden. `men sql`
+and `men tables` expose the raw events, facts, entities, edges, and scores — the exact data
+an agent sees. Every fact links to its source event; every extraction records the model that
+produced it. Memory you can audit is memory you can trust.

onemem-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 magic_bubblez
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

onemem-0.1.0/PKG-INFO

@@ -0,0 +1,171 @@
+Metadata-Version: 2.5
+Name: onemem
+Version: 0.1.0
+Summary: One memory. Every AI. You own it. — Local structured memory for AI agents
+Project-URL: Homepage, https://github.com/YOUR_USERNAME/onemem
+Project-URL: Repository, https://github.com/YOUR_USERNAME/onemem
+Project-URL: Issues, https://github.com/YOUR_USERNAME/onemem/issues
+Author: oneMEM
+License-Expression: MIT
+License-File: LICENSE
+Keywords: ai-agents,llm,long-term-memory,mcp,memory,retrieval,sqlite
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.11
+Requires-Dist: click>=8.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: requests>=2.0
+Provides-Extra: all
+Requires-Dist: fastapi>=0.100; extra == 'all'
+Requires-Dist: mcp[cli]<2,>=1.27; extra == 'all'
+Requires-Dist: sentence-transformers>=2.0; extra == 'all'
+Requires-Dist: sqlite-vec>=0.1; extra == 'all'
+Requires-Dist: uvicorn>=0.20; extra == 'all'
+Provides-Extra: api
+Requires-Dist: fastapi>=0.100; extra == 'api'
+Requires-Dist: uvicorn>=0.20; extra == 'api'
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: local-embedding
+Requires-Dist: sentence-transformers>=2.0; extra == 'local-embedding'
+Provides-Extra: mcp
+Requires-Dist: mcp[cli]<2,>=1.27; extra == 'mcp'
+Provides-Extra: vec
+Requires-Dist: sqlite-vec>=0.1; extra == 'vec'
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+# oneMEM
+
+### One memory. Every AI. You own it.
+
+Local, structured memory for AI agents—in one SQLite file on your machine.
+
+[![PyPI](https://img.shields.io/pypi/v/onemem?label=PyPI&color=6C3AED)](https://pypi.org/project/onemem/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-14B8A6)](LICENSE)
+
+</div>
+
+oneMEM gives AI tools a shared local memory. It turns useful context into compact facts and surfaces the minimum amount of memory sufficient for a query. Every connected agent reads and writes the same SQLite file.
+
+```text
+AI agents ───┐
+Code editors ┼── MCP ── oneMEM ── ~/.onemem/onemem.db
+Local tools ─┘
+```
+
+## Install
+
+oneMEM requires Python 3.11 or newer and an API key for any supported language-model provider. Embeddings run locally; there is no embedding service or API key to configure.
+
+```console
+uv tool install "onemem[all]"
+onemem init
+```
+
+`onemem init` does the rest:
+
+1. asks which model provider you want to use;
+2. recommends a model or lets you enter another model ID;
+3. verifies the key and model before saving them;
+4. initializes the local database and embedding model;
+5. offers background capture; and
+6. connects detected AI tools over MCP.
+
+Nothing is uploaded to a oneMEM server. Configuration and credentials stay under `~/.onemem/`; the API key is sent only to the provider you select.
+
+## Try it
+
+Add something directly:
+
+```console
+onemem add "Chose SQLite because it needs zero operations and one-file backups."
+```
+
+Ask for it later:
+
+```console
+onemem ask "What storage did I choose, and why?"
+```
+
+Or ask from a connected agent. It receives two MCP tools:
+
+- `onemem_recall` retrieves relevant memory.
+- `onemem_log` stores something worth remembering.
+
+## MCP setup
+
+oneMEM works with any client that can run a local `stdio` MCP server, including Claude Code, Codex, Cursor, and Windsurf. `onemem init` automatically configures clients whose command-line tools it detects; other clients only need the `onemem-mcp` executable path.
+
+```console
+claude mcp add --scope user onemem -- "$(command -v onemem-mcp)"
+codex mcp add onemem -- "$(command -v onemem-mcp)"
+```
+
+## Commands
+
+| Command | Purpose |
+|---|---|
+| `onemem init` | Complete interactive setup |
+| `onemem add "memory"` | Store one note or observation |
+| `onemem ask "question"` | Retrieve relevant facts and answer a question |
+| `onemem watch --start` | Start capturing in the background |
+| `onemem doctor` | Check the environment |
+| `onemem status` | Show event, fact, entity counts |
+
+## Where data lives
+
+| Path | Contents |
+|---|---|
+| `~/.onemem/onemem.db` | Events, facts, entities, embeddings |
+| `~/.onemem/config.toml` | Active provider, model, and runtime settings |
+| `~/.onemem/.env` | Provider API keys |
+
+## How retrieval works
+
+```text
+session or imported text
+        ↓
+append-only raw event
+        ↓
+compact facts + local embeddings + entity anchors
+        ↓
+deterministic fusion of semantic, keyword, and entity retrieval
+        ↓
+minimum relevant memory returned to the agent
+```
+
+oneMEM uses an LLM only to interpret and compact language and, optionally, to synthesize the final answer. Storage, indexing, and retrieval are ordinary code.
+
+## Benchmarks
+
+Measured on a 100-instance stratified sample of [LongMemEval-S](https://arxiv.org/abs/2410.10813):
+
+| Metric | Result |
+|---|---:|
+| Retrieval recall | **0.89** |
+| Context reduction | **99.1%** |
+| End-to-end answer accuracy | **72%** |
+
+## Development
+
+```console
+git clone https://github.com/YOUR_USERNAME/onemem.git
+cd onemem
+uv sync --all-extras
+uv run pytest -q
+```
+
+## License
+
+MIT — Based on [Meniscus](https://github.com/magic-bubblez/meniscus) by magic_bubblez.

onemem-0.1.0/README.md

@@ -0,0 +1,126 @@
+<div align="center">
+
+# oneMEM
+
+### One memory. Every AI. You own it.
+
+Local, structured memory for AI agents—in one SQLite file on your machine.
+
+[![PyPI](https://img.shields.io/pypi/v/onemem?label=PyPI&color=6C3AED)](https://pypi.org/project/onemem/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-14B8A6)](LICENSE)
+
+</div>
+
+oneMEM gives AI tools a shared local memory. It turns useful context into compact facts and surfaces the minimum amount of memory sufficient for a query. Every connected agent reads and writes the same SQLite file.
+
+```text
+AI agents ───┐
+Code editors ┼── MCP ── oneMEM ── ~/.onemem/onemem.db
+Local tools ─┘
+```
+
+## Install
+
+oneMEM requires Python 3.11 or newer and an API key for any supported language-model provider. Embeddings run locally; there is no embedding service or API key to configure.
+
+```console
+uv tool install "onemem[all]"
+onemem init
+```
+
+`onemem init` does the rest:
+
+1. asks which model provider you want to use;
+2. recommends a model or lets you enter another model ID;
+3. verifies the key and model before saving them;
+4. initializes the local database and embedding model;
+5. offers background capture; and
+6. connects detected AI tools over MCP.
+
+Nothing is uploaded to a oneMEM server. Configuration and credentials stay under `~/.onemem/`; the API key is sent only to the provider you select.
+
+## Try it
+
+Add something directly:
+
+```console
+onemem add "Chose SQLite because it needs zero operations and one-file backups."
+```
+
+Ask for it later:
+
+```console
+onemem ask "What storage did I choose, and why?"
+```
+
+Or ask from a connected agent. It receives two MCP tools:
+
+- `onemem_recall` retrieves relevant memory.
+- `onemem_log` stores something worth remembering.
+
+## MCP setup
+
+oneMEM works with any client that can run a local `stdio` MCP server, including Claude Code, Codex, Cursor, and Windsurf. `onemem init` automatically configures clients whose command-line tools it detects; other clients only need the `onemem-mcp` executable path.
+
+```console
+claude mcp add --scope user onemem -- "$(command -v onemem-mcp)"
+codex mcp add onemem -- "$(command -v onemem-mcp)"
+```
+
+## Commands
+
+| Command | Purpose |
+|---|---|
+| `onemem init` | Complete interactive setup |
+| `onemem add "memory"` | Store one note or observation |
+| `onemem ask "question"` | Retrieve relevant facts and answer a question |
+| `onemem watch --start` | Start capturing in the background |
+| `onemem doctor` | Check the environment |
+| `onemem status` | Show event, fact, entity counts |
+
+## Where data lives
+
+| Path | Contents |
+|---|---|
+| `~/.onemem/onemem.db` | Events, facts, entities, embeddings |
+| `~/.onemem/config.toml` | Active provider, model, and runtime settings |
+| `~/.onemem/.env` | Provider API keys |
+
+## How retrieval works
+
+```text
+session or imported text
+        ↓
+append-only raw event
+        ↓
+compact facts + local embeddings + entity anchors
+        ↓
+deterministic fusion of semantic, keyword, and entity retrieval
+        ↓
+minimum relevant memory returned to the agent
+```
+
+oneMEM uses an LLM only to interpret and compact language and, optionally, to synthesize the final answer. Storage, indexing, and retrieval are ordinary code.
+
+## Benchmarks
+
+Measured on a 100-instance stratified sample of [LongMemEval-S](https://arxiv.org/abs/2410.10813):
+
+| Metric | Result |
+|---|---:|
+| Retrieval recall | **0.89** |
+| Context reduction | **99.1%** |
+| End-to-end answer accuracy | **72%** |
+
+## Development
+
+```console
+git clone https://github.com/YOUR_USERNAME/onemem.git
+cd onemem
+uv sync --all-extras
+uv run pytest -q
+```
+
+## License
+
+MIT — Based on [Meniscus](https://github.com/magic-bubblez/meniscus) by magic_bubblez.

onemem-0.1.0/bench/README.md

@@ -0,0 +1,60 @@
+# Benchmarks
+
+oneMEM is measured on [LongMemEval-S](https://arxiv.org/abs/2410.10813), a long-term-memory
+benchmark of long chat histories with labelled evidence turns. The headline numbers in the
+top-level README come from the scripts here, on a 100-instance stratified sample.
+
+## Reproducing the numbers
+
+### 1. Get the dataset
+
+Download LongMemEval-S (`longmemeval_s.json`) from the official release linked in the
+[paper](https://arxiv.org/abs/2410.10813) and place it at:
+
+```
+bench/data/longmemeval_s.json
+```
+
+### 2. Build the frozen memories (one-time; costs LLM calls)
+
+Building a memory runs one extraction call per chat turn, so we build once and persist each
+instance to `bench/memories/<id>.db` (plus a `.json` sidecar). Every later retrieval experiment
+runs against these frozen memories with **zero** LLM calls, so any measured difference is purely
+the retrieval config.
+
+```bash
+export OPENROUTER_API_KEY="sk-..."
+# The published numbers used deepseek-v4-flash as the extraction model:
+ONEMEM_FREEZE_MODEL=deepseek/deepseek-v4-flash python bench/freeze.py bench/data/longmemeval_s.json 100
+```
+
+The run is spend-guarded (`MAX_RUN_COST_USD` in `config.py`) and resumable — rerun to continue.
+Frozen memories are gitignored (large); you rebuild them locally.
+
+### 3. Retrieval metrics (free — no LLM)
+
+```bash
+python bench/metrics.py       # recall + set-complete (all@k) per question type   -> 0.89 recall
+python bench/reduction.py     # injected vs full-history tokens                    -> ~99.1% reduction
+```
+
+### 4. End-to-end answer accuracy (small; spend-capped)
+
+A reader model answers from the retrieved facts; a judge model grades it against the gold answer.
+Both are set via env vars, and the run is capped at `$2` (`MAX_EVAL_COST_USD`).
+
+```bash
+EVAL_READER=openai/gpt-5 python bench/answer_accuracy.py   # strong reader        -> 72%
+python bench/answer_accuracy.py                            # default small reader -> ~54%
+```
+
+## Which script substantiates which number
+
+| Claim | Script | Notes |
+|---|---|---|
+| Retrieval recall 0.89 | `bench/metrics.py` | fraction of gold evidence in the returned set |
+| Context reduction ~99.1% | `bench/reduction.py` | 1 − injected/full-history tokens |
+| Answer accuracy 72% / ~54% | `bench/answer_accuracy.py` | 72% with a GPT-5 reader, ~54% with a small reader; LLM-judged |
+
+`metrics.py` and `reduction.py` call `search_facts` directly and make no LLM calls, so they are
+fully deterministic and reproduce exactly on identical frozen memories.