contextos-dd 0.1.0__tar.gz

contextos_dd-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,24 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python-version: ["3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: pip install -e ".[dev]"
+      - run: ruff check .
+      - run: mypy contextos
+      - run: pytest -q

contextos_dd-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,34 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install build
+      - run: python -m build
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write   # OIDC for PyPI trusted publishing
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

contextos_dd-0.1.0/.gitignore

@@ -0,0 +1,22 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+venv/
+dist/
+build/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+*.duckdb
+*.duckdb.wal
+.contextos/
+.smoke/
+.smoke_local/
+contextos.html
+ContextOS_PRD_v1.docx
+.claude/
+.env
+.DS_Store

contextos_dd-0.1.0/INSTALL.md

@@ -0,0 +1,233 @@
+# Installing ContextOS
+
+This is the full first-run walkthrough. Most users only need `pip install contextos-dd && contextos install` — but if you want to know exactly what's happening, this is the doc.
+
+---
+
+## Prerequisites
+
+- **Python 3.11 or 3.12**
+- **One of the supported IDEs already installed:** Claude Code, Cursor, Codex CLI, Continue, or Aider
+- **An API key already configured in your IDE** for Anthropic or OpenAI. ContextOS doesn't manage API keys; your IDE keeps doing that.
+- **~500 MB of free disk** for the local models
+- **Internet access** during install (for the model download) — unless you use `--skip-models`
+
+ContextOS works on Windows, macOS, and Linux.
+
+---
+
+## Step 1 — Install the package
+
+```bash
+pip install contextos-dd
+```
+
+This pulls the package and its dependencies (FastAPI, DuckDB, LanceDB, llama-cpp-python, fastembed, etc.). Takes 30–60 seconds depending on your connection.
+
+### Windows note: prebuilt llama-cpp wheel
+
+On Windows, `pip` may try to build `llama-cpp-python` from source and hit Windows' 260-char path limit. Avoid that by installing the prebuilt wheel **first**:
+
+```powershell
+pip install llama-cpp-python --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+pip install contextos-dd
+```
+
+### Old or virtualised CPUs
+
+ContextOS pins `llama-cpp-python==0.2.90` because newer wheels include AVX-512 instructions that crash on Intel 12th/13th-gen consumer CPUs (where AVX-512 is microcode-disabled). If your CPU is older than ~2013, even 0.2.90 may not work — open an issue and we'll add a fallback.
+
+---
+
+## Step 2 — Run `contextos install`
+
+```bash
+contextos install
+```
+
+What happens, in order:
+
+1. **IDE detection.** Looks for these files; patches whichever exist:
+   - `~/.claude/settings.json` (Claude Code)
+   - `~/.cursor/settings.json` (Cursor)
+   - `~/.codex/config.json` (Codex CLI)
+   - `~/.continue/config.json` (Continue)
+   - `~/.aider.conf.yml` (Aider)
+2. **Backup.** Every file we touch is copied to `~/.contextos/backups/<ide>.<timestamp>.<ext>` first.
+3. **Patch.** We set the API base URL to `http://127.0.0.1:9137`.
+4. **Model download (the slow step).** Two files pulled from HuggingFace into `~/.contextos/models/`:
+   - `Qwen2.5-0.5B-Instruct-Q4_K_M.gguf` — ~400 MB
+   - `BAAI/bge-small-en-v1.5` ONNX bundle — ~30 MB
+5. **Daemon start.** The proxy starts on `127.0.0.1:9137` and writes its PID to `~/.contextos/daemon.pid`.
+
+Total time: **2–5 minutes** depending on connection speed.
+
+### Air-gapped or offline install
+
+```bash
+contextos install --skip-models
+```
+
+The daemon will still start and route IDE traffic correctly. Compaction silently no-ops until models are present — you'll lose narrative summaries but still get **DEAD-turn removal and HOT-window trimming (~25–35% savings)**.
+
+To install models later, when you have internet:
+
+```bash
+contextos warmup
+```
+
+Or place the files manually in `~/.contextos/models/` — filenames must match those in step 4.
+
+---
+
+## Step 3 — Verify
+
+```bash
+contextos doctor
+```
+
+Expected output:
+
+```
+Paths
+  OK  data dir   ~/.contextos
+  OK  log dir    ~/.contextos/logs
+  OK  db file    ~/.contextos/ledger.duckdb
+  OK  model dir  ~/.contextos/models
+
+Dependencies
+  OK  llama-cpp-python
+  OK  fastembed
+  OK  lancedb
+  OK  streamlit
+
+Network
+  OK  proxy http://127.0.0.1:9137  daemon running
+  OK  dashboard port 9138 free
+
+IDEs
+  OK  claude-code  ~/.claude/settings.json
+```
+
+Then use your IDE normally. Open the dashboard to see token savings in real time:
+
+```bash
+contextos dashboard
+```
+
+Browser opens `http://127.0.0.1:9138`.
+
+---
+
+## What lives where
+
+```
+~/.contextos/
+  ledger.duckdb          session + turn ledger (DuckDB; ~1 MB per long session)
+  daemon.pid             daemon pid (deleted on clean shutdown)
+  backups/
+    claude-code.20260515T101500Z.json
+    claude-code.latest.json
+    cursor.latest.json
+    ...
+  archive/               LanceDB vector store for retrieval
+  models/
+    Qwen2.5-0.5B-Instruct-Q4_K_M.gguf   ~400 MB
+    models--BAAI--bge-small-en-v1.5/     ~30 MB
+  logs/
+    contextos.log        rolling daemon log (UTF-8)
+```
+
+On Windows these paths live under `%LOCALAPPDATA%\DataDojo\contextos\` (typically `C:\Users\<you>\AppData\Local\DataDojo\contextos\`).
+
+---
+
+## Privacy guarantees, plainly
+
+**ContextOS does not send your code or conversation to any third party we control.** The only outbound HTTP traffic ContextOS initiates:
+
+1. **Model downloads** during `install` or `warmup` — public files from HuggingFace.
+2. **Forwarding your IDE's chat request upstream** — to whatever cloud LLM (`api.anthropic.com`, `api.openai.com`, etc.) your IDE was already configured to use.
+
+That's it. No telemetry, no usage reporting, no cloud sync, no shared memory, no analytics. The proxy reduces the payload before forwarding; everything else stays on disk in `~/.contextos/`.
+
+---
+
+## Uninstall
+
+```bash
+contextos uninstall
+```
+
+- Restores every IDE config we touched (from `~/.contextos/backups/*.latest.*`).
+- Stops the daemon.
+- **Does not** delete the ledger, archive, models, or logs — they stay in case you reinstall later.
+
+To wipe everything:
+
+```bash
+contextos uninstall
+rm -rf ~/.contextos          # Linux / macOS
+Remove-Item -Recurse -Force $env:LOCALAPPDATA\DataDojo\contextos   # Windows
+```
+
+Then `pip uninstall contextos-dd` to remove the package itself.
+
+---
+
+## Troubleshooting
+
+### `contextos doctor` shows daemon stopped
+
+```bash
+contextos start
+```
+
+If that fails, check `~/.contextos/logs/contextos.log`.
+
+### `llama-cpp-python` install errored with `[WinError -1073741795]` or `STATUS_ILLEGAL_INSTRUCTION`
+
+Your CPU doesn't support an instruction the wheel uses (usually AVX-512). ContextOS pins `0.2.90` to avoid this; if it still trips, reinstall:
+
+```bash
+pip install --force-reinstall "llama-cpp-python==0.2.90" --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+```
+
+### Pip errored about Windows long paths
+
+Install the prebuilt wheel first (see Step 1). Or enable Long Paths globally:
+
+```powershell
+# Run once as Administrator:
+New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" `
+  -Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force
+```
+
+### Compaction is too slow / takes ~30s on first batch
+
+The first compaction job loads the model into RAM (~5–10s on CPU). Subsequent jobs are 1–2s. If it's still slow, your model file may be corrupt:
+
+```bash
+rm ~/.contextos/models/Qwen2.5-*.gguf
+contextos warmup
+```
+
+### Model download failed
+
+Check connectivity to `huggingface.co`. Behind a corporate proxy? Set `HF_HUB_ENABLE_HF_TRANSFER=0` and/or `HTTPS_PROXY=http://your-proxy:port` in the environment before running `contextos warmup`.
+
+### Streamlit dashboard says "Ledger not found yet"
+
+You haven't made any API calls through the proxy yet. Make one call from your IDE; refresh the page.
+
+### IDE still hits the cloud directly after install
+
+Restart the IDE. Most IDEs read their settings file at startup and don't notice mid-session changes.
+
+---
+
+## Next steps
+
+- Browse the dashboard: `contextos dashboard`
+- Read the [README](./README.md) for the high-level pitch
+- File issues at https://github.com/DataDojo/context-os/issues

contextos_dd-0.1.0/LICENSE

@@ -0,0 +1,19 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   Copyright 2026 DataDojo
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+   Full license text: https://www.apache.org/licenses/LICENSE-2.0.txt

contextos_dd-0.1.0/PKG-INFO

@@ -0,0 +1,160 @@
+Metadata-Version: 2.4
+Name: contextos-dd
+Version: 0.1.0
+Summary: Local-first, IDE-agnostic agentic memory lifecycle manager
+Project-URL: Homepage, https://github.com/DataDojo/context-os
+Project-URL: Repository, https://github.com/DataDojo/context-os
+Author: DataDojo
+License-Expression: Apache-2.0
+License-File: LICENSE
+Keywords: claude,context,cursor,llm,proxy,tokens
+Classifier: Development Status :: 3 - Alpha
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.11
+Requires-Dist: duckdb>=1.1
+Requires-Dist: fastapi>=0.115
+Requires-Dist: fastembed>=0.4
+Requires-Dist: httpx>=0.27
+Requires-Dist: huggingface-hub>=0.26
+Requires-Dist: lancedb>=0.13
+Requires-Dist: llama-cpp-python==0.2.90
+Requires-Dist: numpy>=1.26
+Requires-Dist: platformdirs>=4.3
+Requires-Dist: pyarrow>=17
+Requires-Dist: pydantic-settings>=2.6
+Requires-Dist: pydantic>=2.9
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.9
+Requires-Dist: tiktoken>=0.8
+Requires-Dist: typer>=0.13
+Requires-Dist: uvicorn[standard]>=0.32
+Provides-Extra: dashboard
+Requires-Dist: polars>=1.12; extra == 'dashboard'
+Requires-Dist: streamlit-autorefresh>=1.0; extra == 'dashboard'
+Requires-Dist: streamlit>=1.40; extra == 'dashboard'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.13; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest>=8.3; extra == 'dev'
+Requires-Dist: ruff>=0.7; extra == 'dev'
+Requires-Dist: types-pyyaml; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# ContextOS
+
+**Local-first agentic memory lifecycle manager for Claude Code, Cursor, Codex, Continue, and Aider.**
+
+ContextOS runs as a background daemon between your IDE and the cloud LLM. It classifies your conversation history by relevance, compresses stale turns into narrative summaries via a small on-device model, and forwards a clean, minimal payload upstream — cutting input tokens 50–70% on long sessions without quality loss.
+
+**Zero bytes of your session content leave your machine** except the request that goes to your chosen cloud LLM (Anthropic, OpenAI) — which is exactly what your IDE was going to send anyway, just smaller.
+
+Built by [DataDojo](https://github.com/DataDojo). Apache-2.0.
+
+---
+
+## Install
+
+```bash
+pip install contextos-dd
+contextos install
+```
+
+That's it. `contextos install`:
+
+1. Detects supported IDE configs (Claude Code, Cursor, Codex CLI, Continue, Aider)
+2. Backs them up to `~/.contextos/backups/`
+3. Patches them to route through the local proxy on `127.0.0.1:9137`
+4. **Downloads two small models** (~430 MB total, one time) to `~/.contextos/models/`
+5. Starts the daemon
+
+The model download happens **once** during install so the first long session doesn't pause for it. For air-gapped or offline installs, use `contextos install --skip-models`.
+
+See [INSTALL.md](./INSTALL.md) for the full first-run walkthrough, paths, privacy guarantees, troubleshooting, and uninstall semantics.
+
+---
+
+## What gets downloaded
+
+| File | Size | Purpose | Where |
+|---|---|---|---|
+| `Qwen2.5-0.5B-Instruct-Q4_K_M.gguf` | ~400 MB | Compaction (summarising stale turns) | `~/.contextos/models/` |
+| `bge-small-en-v1.5` ONNX | ~30 MB | Embeddings (vector recall) | `~/.contextos/models/` |
+
+Both download once, then live on disk forever. Subsequent runs use the cache. Re-pull anytime with `contextos warmup`.
+
+---
+
+## Commands
+
+| Command | What it does |
+|---|---|
+| `contextos install` | Detect IDEs, back up configs, patch URLs, download models, start daemon |
+| `contextos install --skip-models` | Same as above but skip the model download |
+| `contextos warmup` | Download / refresh local models |
+| `contextos status` | Show daemon health and detected IDEs |
+| `contextos start` / `stop` | Daemon lifecycle |
+| `contextos dashboard` | Launch the Streamlit dashboard at `http://127.0.0.1:9138` |
+| `contextos doctor` | Preflight checks: paths, deps, ports, IDE configs |
+| `contextos uninstall` | Restore original IDE configs, stop daemon |
+
+---
+
+## How it works
+
+1. IDE sends API call → `localhost:9137`
+2. Proxy parses messages, classifies turns **HOT / WARM / COLD / DEAD**
+3. **DEAD** (duplicates) dropped, **COLD** runs replaced with a narrative summary from the local Qwen2.5 model
+4. Clean payload forwarded to `api.anthropic.com` / `api.openai.com`
+5. Response streamed back unchanged; trigger phrases like *"as we discussed"* fire a vector recall from a local LanceDB archive for the next turn
+
+---
+
+## Privacy
+
+- **Your code never goes to a third party.** The only outbound network call ContextOS makes is forwarding your IDE's API request to the cloud LLM you already configured your IDE to use.
+- **No telemetry.** Zero. We do not phone home.
+- **No cloud sync.** Sessions, summaries, embeddings, and logs live in `~/.contextos/` on your machine.
+- **No background uploads.** Ever.
+
+The only network traffic ContextOS initiates on its own:
+- Model downloads during `install` or `warmup` — fetched from HuggingFace.
+- Forwarding your IDE's chat requests upstream — exactly the request your IDE built.
+
+---
+
+## Where stuff lives
+
+```
+~/.contextos/
+  ledger.duckdb          # session + turn ledger
+  daemon.pid             # daemon process id
+  backups/               # IDE configs we modified (so uninstall is reversible)
+  archive/               # LanceDB vector store of compacted turns
+  models/                # Qwen2.5 GGUF + bge-small ONNX
+  logs/contextos.log     # rolling daemon log
+```
+
+`contextos uninstall` restores IDE configs and stops the daemon. By default it **does not** delete the ledger or models — they stay in case you reinstall. To wipe everything, delete `~/.contextos/` manually.
+
+---
+
+## Status
+
+**Phase 1 MVP (this package):** proxy, ledger, classifier, compactor, retrieval, Streamlit dashboard, auto-install for 5 IDEs.
+
+**`context-os-pro` (separate package, deferred):** MCP server, team / shared memory, cloud rollups, React dashboard, advanced semantic classifier.
+
+---
+
+## Develop
+
+```bash
+git clone https://github.com/DataDojo/context-os
+cd context-os
+pip install -e ".[dev,dashboard]"
+ruff check .
+pytest -q
+```

contextos_dd-0.1.0/README.md

@@ -0,0 +1,115 @@
+# ContextOS
+
+**Local-first agentic memory lifecycle manager for Claude Code, Cursor, Codex, Continue, and Aider.**
+
+ContextOS runs as a background daemon between your IDE and the cloud LLM. It classifies your conversation history by relevance, compresses stale turns into narrative summaries via a small on-device model, and forwards a clean, minimal payload upstream — cutting input tokens 50–70% on long sessions without quality loss.
+
+**Zero bytes of your session content leave your machine** except the request that goes to your chosen cloud LLM (Anthropic, OpenAI) — which is exactly what your IDE was going to send anyway, just smaller.
+
+Built by [DataDojo](https://github.com/DataDojo). Apache-2.0.
+
+---
+
+## Install
+
+```bash
+pip install contextos-dd
+contextos install
+```
+
+That's it. `contextos install`:
+
+1. Detects supported IDE configs (Claude Code, Cursor, Codex CLI, Continue, Aider)
+2. Backs them up to `~/.contextos/backups/`
+3. Patches them to route through the local proxy on `127.0.0.1:9137`
+4. **Downloads two small models** (~430 MB total, one time) to `~/.contextos/models/`
+5. Starts the daemon
+
+The model download happens **once** during install so the first long session doesn't pause for it. For air-gapped or offline installs, use `contextos install --skip-models`.
+
+See [INSTALL.md](./INSTALL.md) for the full first-run walkthrough, paths, privacy guarantees, troubleshooting, and uninstall semantics.
+
+---
+
+## What gets downloaded
+
+| File | Size | Purpose | Where |
+|---|---|---|---|
+| `Qwen2.5-0.5B-Instruct-Q4_K_M.gguf` | ~400 MB | Compaction (summarising stale turns) | `~/.contextos/models/` |
+| `bge-small-en-v1.5` ONNX | ~30 MB | Embeddings (vector recall) | `~/.contextos/models/` |
+
+Both download once, then live on disk forever. Subsequent runs use the cache. Re-pull anytime with `contextos warmup`.
+
+---
+
+## Commands
+
+| Command | What it does |
+|---|---|
+| `contextos install` | Detect IDEs, back up configs, patch URLs, download models, start daemon |
+| `contextos install --skip-models` | Same as above but skip the model download |
+| `contextos warmup` | Download / refresh local models |
+| `contextos status` | Show daemon health and detected IDEs |
+| `contextos start` / `stop` | Daemon lifecycle |
+| `contextos dashboard` | Launch the Streamlit dashboard at `http://127.0.0.1:9138` |
+| `contextos doctor` | Preflight checks: paths, deps, ports, IDE configs |
+| `contextos uninstall` | Restore original IDE configs, stop daemon |
+
+---
+
+## How it works
+
+1. IDE sends API call → `localhost:9137`
+2. Proxy parses messages, classifies turns **HOT / WARM / COLD / DEAD**
+3. **DEAD** (duplicates) dropped, **COLD** runs replaced with a narrative summary from the local Qwen2.5 model
+4. Clean payload forwarded to `api.anthropic.com` / `api.openai.com`
+5. Response streamed back unchanged; trigger phrases like *"as we discussed"* fire a vector recall from a local LanceDB archive for the next turn
+
+---
+
+## Privacy
+
+- **Your code never goes to a third party.** The only outbound network call ContextOS makes is forwarding your IDE's API request to the cloud LLM you already configured your IDE to use.
+- **No telemetry.** Zero. We do not phone home.
+- **No cloud sync.** Sessions, summaries, embeddings, and logs live in `~/.contextos/` on your machine.
+- **No background uploads.** Ever.
+
+The only network traffic ContextOS initiates on its own:
+- Model downloads during `install` or `warmup` — fetched from HuggingFace.
+- Forwarding your IDE's chat requests upstream — exactly the request your IDE built.
+
+---
+
+## Where stuff lives
+
+```
+~/.contextos/
+  ledger.duckdb          # session + turn ledger
+  daemon.pid             # daemon process id
+  backups/               # IDE configs we modified (so uninstall is reversible)
+  archive/               # LanceDB vector store of compacted turns
+  models/                # Qwen2.5 GGUF + bge-small ONNX
+  logs/contextos.log     # rolling daemon log
+```
+
+`contextos uninstall` restores IDE configs and stops the daemon. By default it **does not** delete the ledger or models — they stay in case you reinstall. To wipe everything, delete `~/.contextos/` manually.
+
+---
+
+## Status
+
+**Phase 1 MVP (this package):** proxy, ledger, classifier, compactor, retrieval, Streamlit dashboard, auto-install for 5 IDEs.
+
+**`context-os-pro` (separate package, deferred):** MCP server, team / shared memory, cloud rollups, React dashboard, advanced semantic classifier.
+
+---
+
+## Develop
+
+```bash
+git clone https://github.com/DataDojo/context-os
+cd context-os
+pip install -e ".[dev,dashboard]"
+ruff check .
+pytest -q
+```

contextos_dd-0.1.0/contextos/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

contextos_dd-0.1.0/contextos/archive/__init__.py

@@ -0,0 +1,3 @@
+from contextos.archive.store import ArchiveStore
+
+__all__ = ["ArchiveStore"]