crewlore 0.1.0__tar.gz

crewlore-0.1.0/.env.example

@@ -0,0 +1,6 @@
+# Bring your own API key. crewlore never ships keys anywhere — they only
+# exist in your environment / your model provider's account.
+#
+# One of the following, matching your `provider` in `.lore/config.yaml`:
+ANTHROPIC_API_KEY=sk-ant-your-key-here
+# OPENAI_API_KEY=sk-your-openai-key-here

crewlore-0.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,34 @@
+---
+name: Bug report
+about: Something behaves unexpectedly
+labels: bug
+---
+
+**What happened**
+
+A short, specific description of the unexpected behavior.
+
+**What you expected**
+
+What you thought should happen instead.
+
+**How to reproduce**
+
+Minimal steps, ideally runnable:
+
+```bash
+# commands
+```
+
+If the issue is with a specific session/transcript, please *do not* paste sensitive content. A reduced synthetic example is best; otherwise describe the shape (which `kind` events, roughly how many, what triggered the bug).
+
+**Environment**
+
+- `crewlore` version (`lore --version` or `pip show crewlore`):
+- Python version:
+- OS:
+- Model provider + model name (if relevant):
+
+**Anything else**
+
+Logs, screenshots, related issues, hunches.

crewlore-0.1.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,21 @@
+---
+name: Feature request
+about: Suggest something crewlore could do
+labels: enhancement
+---
+
+**The problem you're trying to solve**
+
+What's the underlying friction? (Not the solution — the problem.)
+
+**The shape of a solution you're picturing**
+
+How would it work from the user's side? CLI command, config option, file format, …?
+
+**Alternatives considered**
+
+What did you think about and rule out, and why?
+
+**Scope check**
+
+`crewlore`'s job is to compile agent session transcripts into a versioned, plaintext, citable knowledge layer. It is intentionally *not* a coding agent, a vector DB, or a personal-memory store. If your request fits comfortably inside that scope, say so; if it's a stretch, that's fine — just call it out so the discussion can be honest about whether it belongs here or somewhere else.

crewlore-0.1.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,14 @@
+## What this changes
+
+One short paragraph. Focus on the *why* — the *what* is in the diff.
+
+## Test plan
+
+- [ ] `uv run pytest -q` passes locally
+- [ ] `uv run ruff check .` passes locally
+- [ ] If this changes user-visible behavior: README / docs updated
+- [ ] If this changes the on-disk schema: migration / version bump documented in CHANGELOG
+
+## Notes for reviewers
+
+Anything reviewers should look at first, or any decision you'd like a second opinion on.

crewlore-0.1.0/.github/SECURITY.md

@@ -0,0 +1,32 @@
+# Security policy
+
+`crewlore` is an early-stage open-source project. It runs locally and does not host any service on the maintainer's infrastructure — there is no server to attack — but it does process session transcripts that may contain sensitive content, and it ships a secret-scrubber that other people will rely on.
+
+## Reporting a vulnerability
+
+If you find anything in the following list, **please do not file a public issue**. Email **srijan@pavoai.com** with the details and I'll respond within 72 hours:
+
+- A secret-scrub bypass (a real-world secret format that `lore.scrub` passes through un-redacted)
+- A way for `crewlore` to read or leak data from outside its working directory
+- A way for `lore serve` / the MCP server to expose more than the user intended
+- A dependency-chain vulnerability with practical exploitability against `crewlore`'s usage
+
+For ordinary bugs and feature requests, please open a GitHub issue.
+
+## Scope
+
+In scope:
+
+- `crewlore`'s own code (`src/lore/`)
+- Documented behavior of `lore init`, `lore compile`, `lore query`, `lore status`, `lore watch`, `lore serve`
+- The secret-scrubber pattern set in `src/lore/scrub.py` (see also `docs/scrub.md` for the documented coverage contract)
+
+Out of scope:
+
+- Vulnerabilities in the model provider's API (Anthropic / OpenAI)
+- Issues that require physical/local access to the developer's machine
+- Theoretical or non-exploitable patterns
+
+## Disclosure preference
+
+Once a fix is shipped, I will credit reporters in the release notes unless asked not to. Coordinated disclosure window: 7–14 days from a fix landing, depending on severity.

crewlore-0.1.0/.github/dependabot.yml

@@ -0,0 +1,19 @@
+version: 2
+updates:
+  - package-ecosystem: pip
+    directory: "/"
+    schedule:
+      interval: weekly
+      day: monday
+    open-pull-requests-limit: 5
+    labels:
+      - dependencies
+
+  - package-ecosystem: github-actions
+    directory: "/"
+    schedule:
+      interval: monthly
+    open-pull-requests-limit: 3
+    labels:
+      - dependencies
+      - ci

crewlore-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,35 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  test:
+    name: test (py${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+
+      - name: Set up Python ${{ matrix.python-version }}
+        run: uv python install ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: uv sync --all-extras --dev
+
+      - name: Lint (ruff)
+        run: uv run ruff check .
+
+      - name: Test (pytest)
+        run: uv run pytest -q

crewlore-0.1.0/.gitignore

@@ -0,0 +1,31 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+
+# Virtualenv
+.venv/
+venv/
+
+# Tooling caches
+.pytest_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# OS
+.DS_Store
+
+# Local lore data captured during dev (raw sessions never committed)
+.lore/sessions/
+
+# Secrets
+.env
+*.key
+
+# Local-only working dir — strategy notes, launch copy drafts,
+# experiments. Never committed; never pushed.
+local/

crewlore-0.1.0/CHANGELOG.md

@@ -0,0 +1,88 @@
+# Changelog
+
+## 0.1.0 (unreleased)
+
+First working vertical slice — capture → compile → serve → actuate → measure, built test-first.
+
+### Pre-launch hardening
+
+A full adversarial audit before going public surfaced a cluster of fixes, all landed here:
+
+- **Incremental compile cost.** `lore watch` now caches extraction per session (sessions are immutable, so the id is a safe key) — each interval only LLM-extracts newly-ingested sessions instead of re-running the whole corpus against your key. `--rebuild` ignores the cache. The watch loop is now incremental in *cost*, not just idempotent in *output*.
+- **First-run works out of the box.** `anthropic` (the default provider) is now a base dependency, and SDK imports are guarded with an actionable message instead of a raw `ModuleNotFoundError`. `pipx install crewlore` → `lore compile` just works with a key.
+- **Local models are real.** `model.provider: local` + `model.base_url` routes to any OpenAI-compatible endpoint (Ollama, LM Studio, vLLM). Previously "local" was advertised but unimplemented and produced a dead-end error.
+- **`lore serve --mcp`** — the documented MCP flag now exists (it was missing; the copy-paste `mcp.json` failed to launch).
+- **Tool-call arguments are scrubbed.** Secrets passed as tool-call args live in event `meta`, which previously bypassed the scrubber; `meta` is now walked recursively. Coverage broadened (AWS `ASIA`/secret-key assignments, all `xox?-` Slack tokens, quoted multi-word secrets). See [`docs/scrub.md`](docs/scrub.md).
+- **Usage stats moved to a gitignored sidecar** so `lore query` no longer rewrites the git-tracked `claims.jsonl` on every call — `git log .lore/` stays clean.
+- **Robustness:** timestamps are always timezone-aware (a timestampless transcript no longer crashes the actuation loop); a single failing session no longer aborts the whole compile pass.
+- **`lore --version`** flag; one canonical-form definition shared between the fidelity gate and the reported fidelity number; honesty passes on the README (the synthetic rediscovery metric is labelled demo data, the fidelity claim states what the gate does and does not certify, retrieval is described as lexical).
+- Corrected the pydantic-ai example counts (9 scope groupings / 6 topics, matching the committed data).
+
+### Added
+
+- **Schemas** — content-addressed `Claim` IDs (idempotent compile, clean merges); `NSFEvent`, `Anchor`, `Provenance`, `Conflict`, `UsageStats`. Actuation fields (`action`, `status`, `usage`) and a conflict-grouping `topic` baked in.
+- **Store** — `.lore/` layout; sorted, one-object-per-line JSONL for line-oriented diffs/merges; raw sessions gitignored (secrets safety).
+- **Capture** — Claude Code adapter (transcript records → NSF) and a friction/resolution signal gate so trivial one-shot sessions are not compiled.
+- **Compiler** — deterministic dedup/merge, conflict-recording (not merging), authority-by-support, latest-observed; LLM extraction behind a `complete(prompt)->str` seam with a verbatim-anchor fidelity gate.
+- **Serve** — instrumented task-conditioned retrieval (records usage), influence/override feedback, markdown knowledge book, optional MCP server (`[serve]` extra).
+- **Actuation loop** — usage-driven lifecycle: decay unused, retire overridden, reinforce influential.
+- **Measurement** — `fidelity_report` and held-out `replay_report` (preventable-rediscovery rate).
+- **CLI** — `init`, `compile`, `query`, `status` (surfaces utilization as a dumpyard early-warning), `serve`.
+- **Demo** — `scripts/demo.py` runs the full loop on synthetic public-safe data, no API key.
+
+### Provider support
+
+- BYO-key Anthropic and OpenAI; loud `CredentialsError` when no key is configured. Nothing routes through any lore-operated infrastructure.
+
+### Live validation (Anthropic Haiku)
+
+Ran the real extraction path end-to-end and fixed what it surfaced:
+
+- Tolerate ```` ```json ```` fences and prose around the JSON array (models wrap output).
+- Conflicts require matching `kind` (a gotcha and its fix-decision shared a topic and were falsely flagged).
+- Feed existing topic vocabulary back to the extractor so it reuses keys across sessions.
+
+Confirmed working live: extraction returns well-formed claims, the verbatim-anchor fidelity gate holds at 100%, trivial sessions are gated out, and retrieval returns relevant claims. See README "Known limitations" for the residual conflict-coordinate-consistency gap.
+
+### Automatic compilation + secret scrubbing
+
+Compilation is now automatic by default — no human has to remember to run it:
+
+- **Secret scrubber** (`lore.scrub`) runs at ingest: API keys, AWS keys, private-key blocks, and secret assignments are redacted before anything is stored or sent to the model.
+- **Capture-from-transcripts** (`lore.capture.ingest`): reads the coding agent's existing on-disk transcripts, incrementally (skips already-captured sessions). No live hook needed for MVP.
+- **`auto_compile`** = ingest → compile → prune (actuation lifecycle) → re-render the book, in one idempotent pass.
+- **`lore watch`** runs that pass on an interval (`--once` for cron/CI); `lore compile` is the manual escape hatch.
+- **Signal gate** widened to capture procedures/conventions/team-norms, not only friction (was silently dropping "how we do X" / "the rule is Y" sessions).
+
+Live end-to-end on public-safe data (Haiku): 6 transcripts ingested, 2 secrets redacted, 7 compiled claims (decisions/gotchas/procedures), a rendered team-knowledge book, and 2 of 3 held-out sessions re-deriving known knowledge (illustrative demo data, n=3).
+
+### Fidelity-gate contract made explicit (`_canonical_form`)
+
+The fidelity gate's tolerance shape — what counts as a "verbatim" anchor — is now a documented contract instead of an implicit substring check. Driven by real-data findings from capturing pydantic-ai sessions where agent prose contains Markdown decoration and long replies are split across NSF events by tool calls.
+
+- **New module-level `_canonical_form(text)`** in `lore.compile.extractor`. Three transformations applied in order, each with its own line and rationale: strip Markdown decoration (`` ` ``, `*`, `_`); collapse whitespace; lowercase. Full docstring covering the contract.
+- **What's accepted:** Markdown decoration differences, whitespace differences, case differences, quotes that span event boundaries (e.g. a long agent reply split by a tool_call).
+- **What's still rejected:** fabricated content, paraphrase, changed meaningful words, out-of-order stitching of disjoint substrings.
+- **Five adversarial tests** pin down both halves: `test_fidelity_accepts_*` for tolerated variations, `test_fidelity_rejects_*` for content-drift cases.
+- **New spec doc:** [`docs/anchors.md`](docs/anchors.md) — the precise contract, why each transformation exists, what's deliberately preserved, and the v0.2 roadmap item (position-pointer anchors that eliminate the gate entirely).
+- **Haystack excludes `tool_call` events** — their `content` is just the tool name (e.g. `"Read"`), not session prose; including it would break quotes that span agent-message events separated by tool calls.
+
+### Real-data evidence: pydantic-ai example expanded
+
+`docs/examples/pydantic-ai/` now reflects three captured sessions (G1 #5679, G3 #5358, D1 #5536):
+
+- **18 active claims** (7 gotchas, 7 decisions, 3 procedures, 1 style)
+- **9 distinct scope groupings** spanning UI adapters, decorator introspection, durable-execution threat modeling, toolsets, tests, and version policy
+- **100% per-session canonical fidelity** — every anchor verified against its session's content under the explicit contract
+- **0 conflicts** (sessions disjoint in scope)
+- Provenance documents five real-data bugs the capture process found and fixed before publication.
+
+### Renamed from `agent-lore` to `crewlore`
+
+The original working name `agent-lore` collided on PyPI and on GitHub (taken by an unrelated 2018 project). Renamed to `crewlore` — two familiar words, instant comprehension, with the name free on PyPI and the repo live on GitHub. The importable package, CLI command, and `.lore/` directory layout stay the same; only the distribution and GitHub URL change. (Not yet published to PyPI; install from git until the first release is cut.)
+
+### README hero GIF rework
+
+- **Compiled-book renders as Markdown** in the recording (via `rich.markdown.Markdown`) — headings, bullets, and kind tags are now visibly formatted; previously the GIF showed raw text and the "interesting facts" landed weakly.
+- **Architecture diagram is Mermaid**, not ASCII — renders inline on GitHub with the engine + artifact visually emphasized.
+- **Higher-contrast theme + larger font** in `docs/recording.tape` (Catppuccin Mocha, 18pt) for legibility at README hero size.

crewlore-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,65 @@
+# Contributing to crewlore
+
+Thanks for considering a contribution. `crewlore` is small, alpha, and intentionally focused — the best way to help right now is to try it on a real repo, open a [discussion](https://github.com/srijansk/crewlore/discussions) about what worked and what didn't, and (if you're up for it) ship a small, well-scoped PR.
+
+Before substantial changes, please open a discussion or an issue so we can agree on the shape before you write code. A 5-minute conversation usually saves an hour of rework.
+
+## The highest-leverage contribution
+
+**Add a capture adapter for another coding agent.** The architecture is harness-neutral by design; the moat is in compile, and capture is deliberately thin. A new adapter is one self-contained module that maps a harness's session artifacts into the Normalized Session Format (NSF). Look at `src/lore/capture/adapters/claude_code.py` and the tests in `tests/unit/test_claude_code_adapter.py` for the shape — that's the entire surface area.
+
+## Local setup
+
+```bash
+git clone https://github.com/srijansk/crewlore.git
+cd crewlore
+uv venv && source .venv/bin/activate
+uv pip install -e ".[dev]"
+pytest
+```
+
+You should see all tests green. If anything's off, that's a bug — please open an issue.
+
+## Running the tests
+
+```bash
+pytest                          # full suite
+pytest tests/unit               # fast feedback loop
+pytest tests/integration        # store/compile orchestration
+pytest tests/e2e                # the whole vertical, deterministic
+```
+
+The suite is fully deterministic. **No real API calls happen during `pytest`** — LLM extraction is exercised behind an injected `complete(prompt) -> str` callable so tests are fast, hermetic, and free. To exercise the live extraction path, set `ANTHROPIC_API_KEY` (or `OPENAI_API_KEY`) and run `lore compile` against a real repo's transcripts.
+
+## Code style
+
+- Linting and formatting: `ruff check src tests scripts` and `ruff check --fix` for autofixes. Line length 100.
+- Type hints throughout (`from __future__ import annotations` at the top of every module).
+- Pydantic models for any data that crosses a module boundary.
+
+## Test-driven development
+
+The project was built test-first and that's the discipline for contributions. Concretely: write the failing test that captures the desired behavior, verify it fails for the expected reason, then write the smallest implementation that makes it pass. The schema and pipeline tests in `tests/unit/` are good models for the shape.
+
+If you're fixing a bug, please include a regression test that fails on `main` and passes with your fix.
+
+## Pull requests
+
+- Branch from `main`; keep PRs focused on one thing.
+- Include the test that proves the change works (red → green in the same PR).
+- Run `pytest` and `ruff check src tests scripts` before pushing.
+- Reference the issue or discussion the PR addresses.
+
+## Reporting issues
+
+For bugs: please include the command you ran, the observed output, the expected output, and your Python version + OS. If the bug involves real transcripts, **don't paste their contents** — describe the shape (kinds of events, rough size) instead.
+
+For security issues, please email rather than filing publicly.
+
+## Code of conduct
+
+Be excellent to each other. Discussions and reviews are technical and direct, but always respectful — disagree about the work, never about the person.
+
+## License
+
+By contributing you agree your contributions are licensed under the same MIT license that covers the project.

crewlore-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 crewlore Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

crewlore-0.1.0/PKG-INFO

@@ -0,0 +1,251 @@
+Metadata-Version: 2.4
+Name: crewlore
+Version: 0.1.0
+Summary: An open, local-first, harness-agnostic compiler that turns AI-coding-agent sessions into a versioned, plaintext team tribal-knowledge layer in your own git repo.
+Author: crewlore Contributors
+License: MIT License
+        
+        Copyright (c) 2026 crewlore Contributors
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: agents,ai,cli,compiler,knowledge,local-first,mcp,tribal-knowledge
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
+Requires-Python: >=3.10
+Requires-Dist: anthropic>=0.39
+Requires-Dist: pydantic<3.0,>=2.0
+Requires-Dist: pyyaml<7.0,>=6.0
+Requires-Dist: rich<16.0,>=13.0
+Requires-Dist: typer<1.0,>=0.12
+Provides-Extra: dev
+Requires-Dist: openai>=1.0; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff; extra == 'dev'
+Provides-Extra: openai
+Requires-Dist: openai>=1.0; extra == 'openai'
+Provides-Extra: serve
+Requires-Dist: mcp>=1.0; extra == 'serve'
+Description-Content-Type: text/markdown
+
+# crewlore
+
+[![CI](https://github.com/srijansk/crewlore/actions/workflows/ci.yml/badge.svg)](https://github.com/srijansk/crewlore/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![fidelity 100%](https://img.shields.io/badge/fidelity-100%25-success.svg)](docs/examples/pydantic-ai/)
+[![claims compiled 18](https://img.shields.io/badge/claims_compiled-18-informational.svg)](docs/examples/pydantic-ai/)
+
+> **Your coding agents keep relearning what your team already figured out.**
+> `crewlore` compiles agent sessions into a citable, plaintext team-knowledge layer that lives in your git repo. Local-first.
+
+<p align="center">
+  <img src="docs/assets/demo.gif" alt="crewlore in action — sessions compiled into a citable team knowledge book" />
+</p>
+
+```bash
+pipx install git+https://github.com/srijansk/crewlore.git
+```
+
+> **Validated on [`pydantic/pydantic-ai`](https://github.com/pydantic/pydantic-ai)** (17.3k ⭐) · 3 sessions · 18 claims · 100% fidelity · [see receipts →](docs/examples/pydantic-ai/)
+
+## Quickstart
+
+```bash
+cd my-repo
+lore init                      # create .lore/ in your repo
+lore watch                     # automatic: read agent transcripts, scrub secrets,
+                               #   compile to claims, prune — on an interval
+lore query "billing webhook"   # ask the knowledge layer anything, anytime
+```
+
+That's it — engineers keep working in whatever agent they use; `lore` keeps the knowledge layer fresh in the background. Commit `.lore/knowledge` and `.lore/claims` and your teammates inherit it on the next `git pull`.
+
+<details>
+<summary>Trouble installing?</summary>
+
+If `pipx` fails with `Broken Python installation, platform.mac_ver() returned an empty value`, your default Python is a broken install (sometimes seen with very recent Homebrew Python 3.14 builds). Pin a known-working interpreter:
+
+```bash
+pipx install --python python3.13 git+https://github.com/srijansk/crewlore.git
+```
+
+To make pipx default to Python 3.13 going forward: `export PIPX_DEFAULT_PYTHON=$(which python3.13)`.
+
+</details>
+
+### Try it in 30 seconds — no API key
+
+```bash
+git clone https://github.com/srijansk/crewlore.git
+cd crewlore && uv run python scripts/demo.py
+```
+
+The demo runs the full loop on bundled public-safe sessions and prints what it found:
+
+> [!NOTE]
+> **Fidelity — 100%.** Every claim's citation resolves verbatim back to its source.
+> **Conflicts surfaced — 1.** A real disagreement kept with both provenances, not silently merged.
+> **Preventable rediscovery — 2 of 3.** Two of the three held-out follow-up sessions re-derived knowledge the layer already had. (Illustrative demo data — n=3, not a benchmark.)
+
+## See it run on a real codebase: pydantic-ai (17.3k ⭐)
+
+[`docs/examples/pydantic-ai/`](docs/examples/pydantic-ai/) is a committed snapshot of `crewlore` compiled on the public [`pydantic/pydantic-ai`](https://github.com/pydantic/pydantic-ai) repo — 3 Claude Code sessions on real issues, no synthetic data.
+
+- **18 claims** compiled across 9 scope groupings (UI adapters, decorator introspection, durable-execution threat modeling, toolsets, tests, version policy)
+- **100% fidelity** under the explicit [canonical-form contract](docs/anchors.md) — every anchor's quote canonically resolves to a substring of its source session. (Fidelity certifies the *citation* is real, not that the model's *statement* is fully entailed by it — that's what human/PR review of the book is for.)
+- **0 conflicts** because the three sessions covered disjoint scopes — the conflict detector wasn't given anything to flag
+- **Receipts:** the rendered [`book.md`](docs/examples/pydantic-ai/book.md), the raw [`claims.jsonl`](docs/examples/pydantic-ai/claims.jsonl), and full [`provenance.md`](docs/examples/pydantic-ai/provenance.md) (session ids, commit hashes, compile cost, scrub redactions, five real-data bugs the capture surfaced and we fixed before publishing)
+
+## What you get
+
+Raw, messy sessions go in. Out comes a structured, citable **compiled claim** — every one carrying its kind, its scope, the action it implies for future work, and a verbatim **anchor** back to the moment it was discovered:
+
+> **`[gotcha]`** · *services/billing*
+>
+> Billing webhook handler lacks an idempotency check, causing duplicate charges when Stripe retries webhooks.
+>
+> **Do** — dedupe on the Stripe idempotency key before processing.
+>
+> > *anchor* — "the handler has no idempotency check, so when Stripe retries a webhook the charge is processed again."
+
+A human can verify it (the anchor points back to the exact session line); an agent can trust it (the citation is real, not hallucinated). Claims roll up into a knowledge book at `.lore/knowledge/README.md`, grouped by area and committed to your repo alongside your code:
+
+```markdown
+# Team knowledge (compiled by crewlore)
+
+## services/billing
+
+- **[gotcha]** Billing webhook handler lacks an idempotency check; dedupe on the Stripe key.
+  - *Do:* Dedupe on the Stripe idempotency key before processing.
+  - _anchor_ `ses_1#1`: "the handler has no idempotency check, so when Stripe retries a webhook the charge is processed again."
+
+## deployment
+
+- **[procedure]** Run migrations before deploy to prevent missing columns.
+  - *Do:* Run `make migrate` before every deploy.
+```
+
+## How it works
+
+```mermaid
+flowchart LR
+    S["coding agent<br/>sessions"] --> I["ingest + scrub<br/>(transcripts → NSF,<br/>secrets redacted)"]
+    I --> C["compile<br/>(NSF → claims,<br/>verbatim anchors)"]
+    C --> R["<b>.lore/</b> in your repo<br/>(knowledge book + claims,<br/>plaintext, git-versioned)"]
+    R --> SV["serve<br/>(files + MCP query)"]
+    SV --> N["next agent session<br/>inherits the knowledge"]
+
+    SV -. "usage signal" .-> AL["actuation loop<br/>(decay · reinforce · retire)"]
+    AL -. "lifecycle update" .-> R
+
+    classDef engine fill:#4a5d9e,stroke:#1a2c4d,color:#fff,stroke-width:2px
+    classDef artifact fill:#2d6a4f,stroke:#1b4332,color:#fff,stroke-width:2px
+    class C engine
+    class R artifact
+```
+
+> `lore watch` runs ingest → compile → prune automatically, on an interval.
+
+- **Ingest + scrub** — reads the coding agent's existing on-disk transcripts and redacts a curated set of secret patterns (Anthropic / OpenAI / generic `sk-*` API keys, AWS keys, GitHub classic + fine-grained PATs, Google API keys, Slack tokens, HuggingFace tokens, JWTs, connection-string passwords, private-key blocks, and `password=…` assignment shapes) *before* anything is stored or sent to a model. The pattern set is documented in [`docs/scrub.md`](docs/scrub.md).
+- **Compile** — extracts atomic claims, deduplicates them, records disagreements instead of silently overwriting, scores authority by how often a claim recurs, and drops any claim whose citation doesn't resolve verbatim.
+- **Serve** — writes a human- and agent-readable knowledge book to `.lore/knowledge/`, and exposes a query tool (including an optional MCP server) so any agent can pull the relevant slice on demand.
+- **Actuation loop** — every retrieval is recorded, and that usage drives a lifecycle: unused claims decay and archive, contradicted claims are retired, useful claims are reinforced. The store stays small and fresh instead of growing into a pile nobody reads.
+
+The intelligence is in **compile**; ingest and serve are deliberately thin, so supporting another coding agent is a small adapter, not a rewrite. To be precise about the word "compile": extraction is an LLM step (the only non-deterministic part), wrapped in deterministic stages — verbatim-anchor verification, content-addressed dedup, conflict recording, and authority scoring. "Compile" means the repeatable session → claims transform, not that an LLM is absent.
+
+## How it differs
+
+- **vs. hosted memory (Letta, mem0)** — their store lives in someone else's cloud and you can't `git log` it; `crewlore`'s lives in your repo as plaintext.
+- **vs. per-IDE memory (Cursor rules, Claude memory, Continue, Cody)** — tied to one developer, one IDE; `crewlore` is a *team* artifact, committed and reviewed like code.
+- **vs. hand-curated `CLAUDE.md` / `.cursorrules`** — humans write those by hand and they go stale; `crewlore` compiles + reinforces from real sessions and retires what stops being used.
+- **vs. RAG over a vector DB** — RAG retrieves *document chunks*; `crewlore` compiles atomic, citable *claims* with verbatim anchors, so a human or agent can verify the cited source in seconds. (Retrieval today is deterministic lexical overlap, not embeddings — simpler and dependency-free; semantic ranking is on the roadmap.)
+
+## Why this exists
+
+Knowledge discovered inside an agent session is private by default and lost by default. It lives in one developer's transcript, so the next engineer — and every future agent run — re-reads the same files, re-learns the same gotcha, and re-makes a decision the team already made. There's no shared layer that both humans and agents read from, so decisions drift and bugs resurface.
+
+`crewlore` makes that knowledge a first-class, versioned artifact in the place your team already trusts: your git repo.
+
+**What it is:** a compiler that turns sessions into accurate, deduplicated, conflict-aware, provenance-carrying team knowledge, served back to any agent.
+
+**What it isn't:** a hosted service, a vector database, or a personal-memory layer for a single IDE. There's no account, no cloud, and no proprietary store — the compiled knowledge is plaintext you own.
+
+## Your data stays yours
+
+- **Local-first.** Capture, compile, and serve all run on infrastructure you control. Point the compiler at your own model provider or a local OpenAI-compatible model (Ollama, LM Studio, vLLM) via `provider: local` — nothing routes through any `crewlore`-operated service, because there is none.
+- **Plaintext, in your repo.** The knowledge layer is human-readable Markdown and JSONL under `.lore/`, versioned by git. `git log .lore/` is your audit trail.
+- **Secrets never travel.** Scrubbing — of both message content and tool-call arguments — happens at ingest, before storage or any model call. It's a high-precision pattern set (a floor, not a DLP guarantee; see [`docs/scrub.md`](docs/scrub.md)), and raw session captures are git-ignored by default regardless.
+
+## CLI
+
+| Command | What it does |
+|---|---|
+| `lore init` | Create the `.lore/` layout in your repo. |
+| `lore watch` | Automatically ingest → compile → prune on an interval (`--once` for cron/CI). |
+| `lore compile` | Run a single ingest-and-compile pass manually. |
+| `lore query "<task>"` | Retrieve the claims most relevant to a task (records usage). |
+| `lore status` | Show claim/conflict counts and how much of the layer is actually being used. |
+| `lore serve --mcp` | Start an MCP server exposing query-time retrieval to any MCP-speaking agent (Claude Desktop, Cursor, …). Requires `pip install 'crewlore[serve]'`. See [`docs/mcp.md`](docs/mcp.md) for wiring snippets. |
+
+## Configuration
+
+`.lore/config.yaml`:
+
+```yaml
+model:
+  provider: anthropic          # anthropic | openai | local
+  name: claude-sonnet-4-6
+  # For provider: local — point at any OpenAI-compatible endpoint you run:
+  # base_url: http://localhost:11434/v1   # e.g. Ollama, LM Studio, vLLM
+capture:
+  transcripts: ~/.claude/projects
+compile:
+  cadence: auto                # `lore watch` interval below
+  watch_interval_seconds: 300
+```
+
+Bring your own key (`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`); `crewlore` never ships keys anywhere. The default Anthropic provider works out of the box. For OpenAI or a local OpenAI-compatible model, add the SDK: `pipx inject crewlore openai` (or `pip install 'crewlore[openai]'`). With `provider: local` nothing leaves your machine at all — the compile call hits your own endpoint.
+
+## Roadmap & limitations
+
+> [!NOTE]
+> **Status: alpha.** The core is stable and tested end to end. The on-disk schema may change before 1.0 — and because everything is plaintext and git-versioned, breaking format changes will ship with migrations.
+
+- **Stable today:** capture, secret scrubbing, the compile pipeline, retrieval, the actuation loop, and the `.lore/` plaintext format.
+- **In flight:** cross-session conflict alignment — real disagreements are surfaced today, but reliably aligning claims about the same question across independently-compiled sessions is an active area of work.
+- **Planned:** an explicit human approve-before-serve gate (secret scrubbing is already automated), more capture adapters beyond Claude Code, and a real-time capture hook.
+
+## Contributing
+
+Issues, discussions, and PRs welcome. New here? Start a [discussion](https://github.com/srijansk/crewlore/discussions) — adding a capture adapter for another coding agent is the most valuable first contribution and is intentionally small. See [CONTRIBUTING.md](CONTRIBUTING.md) for local setup and the dev loop.
+
+Tests are fully deterministic — no real API calls during `pytest`.
+
+## License
+
+MIT — see [LICENSE](LICENSE).