zettelforge 2.2.0__tar.gz → 2.3.0__tar.gz

{zettelforge-2.2.0 → zettelforge-2.3.0}/.github/workflows/ci.yml

@@ -33,8 +33,9 @@ jobs:
     runs-on: ubuntu-latest
     needs: lint
     strategy:
+      fail-fast: false
       matrix:
-        python-version: ['3.12']
+        python-version: ['3.12', '3.13']
 
     steps:
     - uses: actions/checkout@v6

{zettelforge-2.2.0 → zettelforge-2.3.0}/.github/workflows/docs.yml

@@ -14,8 +14,8 @@ jobs:
   deploy:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
         with:
           python-version: '3.12'
       - run: pip install mkdocs-material

{zettelforge-2.2.0 → zettelforge-2.3.0}/.gitignore

@@ -2,6 +2,9 @@
 config.yaml
 config.yml
 .env
+.env.*
+*.key
+*.pem
 
 # Local planning & task tracking (not committed)
 TODO.md
@@ -62,7 +65,7 @@ htmlcov/
 vectordb/
 .vector_memory.lance/
 .snapshots/
-archive/
+/archive/
 
 # Data directories (runtime, not source)
 /data/

zettelforge-2.3.0/ARCHITECTURE.md

@@ -0,0 +1,42 @@
+# Architecture
+
+Visual diagram: [`docs/architecture-diagram.mmd`](docs/architecture-diagram.mmd)
+Deep explanation: [`docs/explanation/architecture.md`](docs/explanation/architecture.md)
+
+## Storage
+
+ZettelForge uses a `StorageBackend` ABC (33 methods) with pluggable
+implementations. Set `ZETTELFORGE_BACKEND` to select.
+
+- **SQLite** (default since v2.2.0): WAL mode, ACID, zero-config.
+  Notes, knowledge graph, and entity index in one database file.
+- **LanceDB**: Vector index alongside SQLite. 768-dim embeddings
+  via fastembed (nomic-embed-text-v1.5-Q, ONNX, in-process).
+
+## Core Pipeline
+
+1. **Ingestion** — Governance validation → Note construction → Entity
+   extraction (regex fast-path + LLM) → Alias resolution → Storage
+2. **Enrichment** (background) — Causal triple extraction, memory
+   evolution (A-Mem neighbor refinement), HGAM consolidation
+3. **Retrieval** — Intent classification → Blended vector + graph
+   search → Temporal/causal boosting → Cross-encoder reranking
+4. **Synthesis** — RAG-as-Answer with quality validation
+
+## Extension Points
+
+Extensions are optional packages discovered at startup via
+`src/zettelforge/extensions.py`. If installed, they provide
+alternative backends and integrations.
+
+- Knowledge graph: TypeDB STIX 2.1 ontology with inference rules
+- Authentication: Multi-tenant OAuth/JWT
+- Integrations: OpenCTI bi-directional sync, Sigma rule generation
+
+## Why These Boundaries
+
+TypeDB requires a running server. OpenCTI is a complex platform.
+These dependencies should not be required to try ZettelForge.
+
+The default backends are not toy implementations — they are
+production-capable for single-user and small-team deployments.

{zettelforge-2.2.0 → zettelforge-2.3.0}/CHANGELOG.md

@@ -6,6 +6,149 @@ Versioning follows [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [2.3.0] - 2026-04-17
+
+Pluggable LLM provider infrastructure (RFC-002 Phase 1), MCP server
+as a first-class Python module, PyPI discoverability refresh, SEO
+foundations across the docs site, and a full docs-vs-code
+reconciliation. All additions are backward-compatible; no existing
+API changes. Supersedes the never-tagged 2.2.1 metadata patch —
+its PyPI classifier / keyword / image-URL changes are folded in
+below.
+
+### Added
+
+- **Pluggable LLM provider infrastructure (RFC-002 Phase 1)** — new
+  `zettelforge.llm_providers` package with a `@runtime_checkable`
+  `LLMProvider` protocol, a thread-safe registry, and built-in
+  providers for `local` (llama-cpp-python), `ollama`, and `mock`.
+  The public `generate()` signature is unchanged; all 7 existing call
+  sites (`fact_extractor`, `memory_updater`, `synthesis_generator`,
+  `intent_classifier`, `note_constructor`, `entity_indexer`,
+  `memory_evolver`) keep working without modification. Third-party
+  providers can register via the `zettelforge.llm_providers`
+  entry-point group. `openai_compat` and `anthropic` providers land
+  in Phase 2 and Phase 3.
+- **`LLMConfig` expanded** — new `api_key`, `timeout`, `max_retries`,
+  `fallback`, and `extra` fields. `api_key` supports `${ENV_VAR}`
+  references and is redacted from `repr()`. Sensitive keys inside
+  `extra` (matching `key|token|secret|password|credential|auth`) are
+  redacted as well. New env overrides: `ZETTELFORGE_LLM_API_KEY`,
+  `ZETTELFORGE_LLM_TIMEOUT`, `ZETTELFORGE_LLM_MAX_RETRIES`,
+  `ZETTELFORGE_LLM_FALLBACK`.
+- **`LLMProviderConfigurationError`** — new exception surfaced for
+  non-recoverable provider setup problems (bad API key, missing
+  optional SDK) so `generate()` can distinguish "try the fallback"
+  from "stop and report".
+- **`llm_client.reload()` helper** — clears the provider registry
+  and config cache so test suites and long-lived processes can
+  reconfigure the LLM backend without a process restart.
+- **Hardened .gitignore** per GOV-023 — added `.env.*`, `*.key`,
+  `*.pem`.
+- **MCP server as a first-class module** — `python -m zettelforge.mcp`
+  now works out of a `pip install zettelforge` with no git clone
+  required. New package `zettelforge.mcp` (with `server.py`,
+  `__main__.py`, and a console-script entry `zettelforge-mcp`).
+  The previous entry point at `web/mcp_server.py` is retained as a
+  thin backward-compat shim.
+- **Console scripts** — `zettelforge` and `zettelforge-mcp` entry
+  points added to `pyproject.toml`.
+- **How-to guides** — migration (`migrate-jsonl-to-sqlite.md`),
+  benchmark reproduction (`reproduce-benchmarks.md`), troubleshooting
+  (`troubleshoot.md`), and upgrade (`upgrade.md`). Linked from the
+  MkDocs nav.
+- **Design and About sections in the docs nav** — RFC-001, RFC-002,
+  RFC-003 and the origin-story narrative are now discoverable from
+  `docs.threatrecall.ai`.
+- **RFC-003 design proposal (docs only)** — read-path depth routing
+  with a deterministic Quality Gate plus System 1 / System 2 recall
+  paths. Ships with an adversarial-review artifact (4 blockers, 13
+  warnings). No runtime changes yet — implementation deferred.
+- **Archive directory** — `docs/archive/` holds retired v1.0.0-alpha
+  snapshots (`SKILL.md`, `PACKAGE_SUMMARY.md`) with a README explaining
+  their provenance.
+- **`llm_ner` configuration reference** — `docs/reference/configuration.md`
+  now documents `llm_ner.enabled` and the `ZETTELFORGE_LLM_NER_ENABLED`
+  environment override.
+- **Docs SEO foundation** — per-page canonical URLs, OpenGraph and
+  Twitter-card metadata, and a `SoftwareApplication` JSON-LD block on
+  the home page via a `docs/overrides/main.html` theme override. The
+  `softwareVersion` value is sourced from `config.extra.version` in
+  `mkdocs.yml` so it stays in sync with releases.
+- **PyPI classifier refresh** — added `Topic :: Security` (primary
+  filter security engineers use to browse PyPI) and
+  `Topic :: Software Development :: Libraries :: Python Modules`.
+  Existing `Topic :: Scientific/Engineering :: Artificial Intelligence`
+  retained. Development Status stays at `4 - Beta`.
+- **PyPI keyword refresh** — swapped `agent-memory` → `agentic-memory`
+  (emerging category keyword) and `zettelkasten` → `llm-memory`
+  (direct intent match for Mem0/Graphiti discovery traffic). Still
+  10 keywords total; within the PyPI display limit.
+
+### Changed
+
+- **SECURITY.md** — contact updated to `contact@threatrecall.ai`,
+  supported-versions table refreshed to mark `2.3.x` as current and
+  `2.2.x` as the prior minor release; storage section refreshed to
+  reflect SQLite-by-default.
+- **`docs/llms.txt`** — rewritten to match current reality (SQLite
+  default, 19 runtime entity types, correct GOV-003/007/011/012
+  descriptions, MCP invocation).
+- **BENCHMARK_REPORT.md** — CTIBench ATE row updated (F1 = 0.146);
+  architecture summary reframed as SQLite + LanceDB default with
+  TypeDB as an extension; `ctibench_results.json` date bumped.
+- **README** — above-fold rewritten (CTA row, keyword density,
+  PyPI-safe absolute-URL images). Pipeline step 1 entity count
+  corrected from "10 types" to the 19 types `EntityExtractor`
+  actually recognises.
+- **README image paths** — `docs/assets/demo.gif` and
+  `docs/assets/zettelforge_architecture.svg` rewritten to absolute
+  `raw.githubusercontent.com` URLs so the PyPI long description
+  renders correctly (relative paths 404 on the PyPI CDN). Pinned to
+  the `master` ref; can be re-pinned to the `v2.3.0` tag in the
+  next release PR if PyPI-side stability matters.
+- **`docs/superpowers/plans/` renamed to `docs/superpowers/research/`**
+  with a README making clear these are aspirational synthesis, not
+  roadmap commitments. The stray untracked `docs/plans/` directory
+  was removed.
+- **Tutorials and governance-controls reference** — `last_updated`
+  and `version` metadata refreshed.
+- **`zettelforge.ontology` exports** — `TypedEntityStore`,
+  `OntologyValidator`, `get_ontology_store`, `get_ontology_validator`
+  removed from the top-level `__all__` (still importable from
+  `zettelforge.ontology`). They are a parallel store not wired into
+  `MemoryManager`.
+- **`observability.py` and `cache.py` headers** — annotated as
+  currently unwired; kept for future integration.
+- **OCSF `_PRODUCT_VERSION`** — sourced from
+  `importlib.metadata.version("zettelforge")` instead of a hard-coded
+  string, so emitted OCSF events stop drifting when `__version__`
+  bumps.
+- **OpenGraph `og:type`** — `website` on the home page, `article`
+  elsewhere (was unconditionally `article`).
+
+### Fixed
+
+- **OllamaProvider host routing** — now instantiates
+  `ollama.Client(host=self._url)` so the configured URL actually
+  takes effect (previously the module-level `ollama.generate()` call
+  ignored per-instance host).
+- **Provider registry race** — `register()` now checks and mutates
+  under the registry lock, closing a TOCTOU window on concurrent
+  provider registration.
+- **MCP server lazy instantiation** — `MemoryManager` is now created
+  on first tool call rather than at server import time, so `--help`
+  and protocol-handshake tests don't pay the model-load cost.
+
+### Removed
+
+- Six superseded branches that had already been squash-merged into
+  master — `feat/causal-chain-fix-and-demo-gif`,
+  `feat/entity-vocabulary-expansion`,
+  `feature/RFC-001-conversational-entity-extractor`,
+  `fix/intent-classifier-graph-weight`,
+  `fix/p0-production-blockers`, `feat/remember-evolve`.
+
 ## [2.2.0] - 2026-04-16
 
 SQLite default backend, causal chain retrieval, memory evolution, STIX taxonomy alignment, and community-first package cleanup.

{zettelforge-2.2.0 → zettelforge-2.3.0}/PKG-INFO

@@ -1,23 +1,27 @@
 Metadata-Version: 2.4
 Name: zettelforge
-Version: 2.2.0
+Version: 2.3.0
 Summary: ZettelForge: Agentic Memory System with vector search, knowledge graph, and synthesis
 Project-URL: Homepage, https://github.com/rolandpg/zettelforge
-Project-URL: Documentation, https://github.com/rolandpg/zettelforge/blob/main/docs/
+Project-URL: Documentation, https://docs.threatrecall.ai
 Project-URL: Repository, https://github.com/rolandpg/zettelforge
 Project-URL: Issues, https://github.com/rolandpg/zettelforge/issues
+Project-URL: Changelog, https://github.com/rolandpg/zettelforge/blob/master/CHANGELOG.md
 Author-email: Patrick Roland <patrick@groland.com>
 License-Expression: MIT
 License-File: LICENSE
-Keywords: agent-memory,ai-agent,cti,cybersecurity,knowledge-graph,mcp-server,rag,stix,threat-intelligence,zettelkasten
-Classifier: Development Status :: 3 - Alpha
+Keywords: agentic-memory,ai-agent,cti,cybersecurity,knowledge-graph,llm-memory,mcp-server,rag,stix,threat-intelligence
+Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Security
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.10
 Requires-Dist: fastembed>=0.8.0
 Requires-Dist: httpx>=0.25.0
@@ -50,28 +54,34 @@ Description-Content-Type: text/markdown
 
 **The only agentic memory system built for cyber threat intelligence.**
 
-Give your AI agents persistent memory with entity extraction, knowledge graphs, and STIX ontology -- no cloud, no API keys, works offline.
+Persistent memory for AI agents and Claude Code — with CTI entity extraction, STIX knowledge graphs, threat-actor alias resolution, and offline-first RAG. MCP server included. No cloud, no API keys.
 
 [![PyPI](https://img.shields.io/pypi/v/zettelforge)](https://pypi.org/project/zettelforge/)
-[![Downloads](https://static.pepy.tech/badge/zettelforge)](https://pepy.tech/projects/zettelforge)
-[![Stars](https://img.shields.io/github/stars/rolandpg/zettelforge)](https://github.com/rolandpg/zettelforge/stargazers)
-[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/licenses/MIT)
+[![Downloads/month](https://static.pepy.tech/personalized-badge/zettelforge?period=month&units=international_system&left_color=grey&right_color=blue&left_text=downloads%2Fmonth)](https://pepy.tech/projects/zettelforge)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/licenses/MIT)
 [![CI](https://github.com/rolandpg/zettelforge/actions/workflows/ci.yml/badge.svg)](https://github.com/rolandpg/zettelforge/actions)
-[![Contributors](https://img.shields.io/github/contributors/rolandpg/zettelforge)](https://github.com/rolandpg/zettelforge/graphs/contributors)
-[![Last Commit](https://img.shields.io/github/last-commit/rolandpg/zettelforge)](https://github.com/rolandpg/zettelforge/commits/master)
-[![SafeSkill](https://safeskill.dev/api/badge/rolandpg-zettelforge)](https://safeskill.dev/scan/rolandpg-zettelforge)
 
+**[⭐ Star](https://github.com/rolandpg/zettelforge) · [📦 `pip install zettelforge`](https://pypi.org/project/zettelforge/) · [📖 Docs](https://docs.threatrecall.ai/) · [🧪 Hosted](https://threatrecall.ai)**
+
+<p align="center">
+<a href="https://www.buymeacoffee.com/xypher22pr0" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-green.png" alt="Buy Me a Coffee" style="height: 60px !important;width: 217px !important;" ></a>
+</p>
 <p align="center">
-  <img src="docs/assets/demo.gif" width="720" alt="ZettelForge demo — CTI agentic memory in action">
+  <img src="https://raw.githubusercontent.com/rolandpg/zettelforge/master/docs/assets/demo.gif" width="720" alt="ZettelForge demo — CTI agentic memory in action">
 </p>
 
+> If ZettelForge fits a CTI workflow you run, a star is the fastest signal that this category is worth continuing to invest in.
+
 ## Why ZettelForge?
 
 General-purpose memory systems don't understand threat intelligence. They can't tell APT28 from Fancy Bear, don't know that CVE-2024-3094 is the XZ Utils backdoor, and can't track how intelligence evolves across reports. When your agent forgets context between investigations, you end up re-reading the same reports and re-building the same mental models.
 
 ZettelForge was built from the ground up for analysts who think in threat graphs, not chat histories. It extracts CVEs, threat actors, IOCs, and MITRE ATT&CK techniques automatically, resolves aliases across naming conventions, builds a knowledge graph with causal relationships, and retrieves memories using intent-aware blended search -- all offline, with no API keys or cloud dependencies.
 
+>"Memory augmentation closes 33% of the gap between small and large models on CTI tasks (CTI-REALM, Microsoft 2026)." [1]
+
+
 | Feature | ZettelForge | Mem0 | Graphiti | Cognee |
 |---------|------------|------|----------|--------|
 | CTI entity extraction (CVEs, actors, IOCs) | Yes | No | No | No |
@@ -83,6 +93,12 @@ ZettelForge was built from the ground up for analysts who think in threat graphs
 | OCSF audit logging | Yes | No | No | No |
 | MCP server (Claude Code) | Yes | No | No | No |
 
+## Data Pipeline
+<p align="center">
+  <img src="https://raw.githubusercontent.com/rolandpg/zettelforge/master/docs/assets/zettelforge_architecture.svg" width="720" alt="ZettelForge architecture — Extract, Graph, Embed, Recall pipeline">
+</p>
+
+
 ## Features
 
 **Entity Extraction** -- Automatically identifies CVEs, threat actors, IOCs (IPs, domains, hashes, URLs, emails), MITRE ATT&CK techniques, campaigns, intrusion sets, tools, people, locations, and organizations. Regex + LLM NER with STIX 2.1 types throughout.
@@ -148,7 +164,7 @@ mm.remember(
 
 Every `remember()` call triggers a pipeline:
 
-1. **Entity Extraction** -- regex + LLM NER identifies CVEs, actors, tools, campaigns, people, locations, orgs (10 types)
+1. **Entity Extraction** -- regex + LLM NER identifies CVEs, intrusion sets, threat actors, tools, campaigns, ATT&CK techniques, IOCs (IPv4, domain, URL, MD5/SHA1/SHA256, email), people, locations, organizations, events, activities, and temporal references (19 types)
 2. **Knowledge Graph Update** -- entities become nodes, co-occurrence becomes edges, LLM infers causal triples
 3. **Vector Embedding** -- 768-dim fastembed (ONNX, in-process, 7ms/embed) stored in LanceDB
 4. **Supersession Check** -- entity overlap detection marks stale notes as superseded
@@ -206,26 +222,6 @@ python examples/athf_bridge.py /path/to/hunts/
 
 See [examples/athf_bridge.py](examples/athf_bridge.py).
 
-## Architecture
-
-```
-┌──────────────────────────────────────────────────────────────────────┐
-│                           MemoryManager                              │
-│  remember()  remember_with_extraction()  recall()  synthesize()      │
-├──────────┬───────────┬──────────────┬───────────┬────────────────────┤
-│  Note    │  Fact     │   Memory     │  Blended  │   Synthesis        │
-│Constructor│ Extractor │  Updater     │ Retriever │   Generator        │
-│(enrich)  │(Phase 1)  │(Phase 2)     │(vec+graph)│   (RAG)            │
-├──────────┴───────────┴──────────────┼───────────┴────────────────────┤
-│       Entity Indexer + Alias        │  Intent Classifier             │
-│       Resolver                      │  (factual/temporal/causal)     │
-├─────────────────────────────────────┼────────────────────────────────┤
-│   Knowledge Graph (SQLite)          │  LanceDB (Vectors)             │
-│   Entity nodes + edges              │  768-dim fastembed (ONNX)      │
-│   Causal triple inference           │  Zettelkasten notes            │
-│   SQLite WAL (TypeDB via extension) │  IVF_PQ index                  │
-└─────────────────────────────────────┴────────────────────────────────┘
-```
 
 ## Extensions
 
@@ -272,6 +268,12 @@ MIT -- See [LICENSE](LICENSE).
 
 **Made by Patrick Roland**.
 
+## Support the Project
+
+ZettelForge is MIT-licensed. If it's useful in your workflow and you'd like to help keep it maintained:
+
+<a href="https://www.buymeacoffee.com/xypher22pr0" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-green.png" alt="Buy Me a Coffee" style="height: 40px !important;width: 145px !important;" ></a>
+
 ## Acknowledgments
 
 - Inspired by [Zettelkasten](https://en.wikipedia.org/wiki/Zettelkasten) and [A-Mem](https://arxiv.org/abs/2602.10715) (NeurIPS 2025)
@@ -279,3 +281,5 @@ MIT -- See [LICENSE](LICENSE).
 - STIX 2.1 schema informed by [typedb-cti](https://github.com/typedb-osi/typedb-cti)
 - Benchmarked against [LOCOMO](https://snap-research.github.io/locomo/) (ACL 2024) and [CTIBench](https://arxiv.org/abs/2406.07599) (NeurIPS 2024)
 - [LanceDB](https://lancedb.com) | [fastembed](https://github.com/qdrant/fastembed) | [Pydantic](https://pydantic.dev) | [TypeDB](https://typedb.com)
+
+[1]: https://www.microsoft.com/en-us/security/blog/2026/03/20/cti-realm-a-new-benchmark-for-end-to-end-detection-rule-generation-with-ai-agents/

{zettelforge-2.2.0 → zettelforge-2.3.0}/README.md

@@ -2,28 +2,34 @@
 
 **The only agentic memory system built for cyber threat intelligence.**
 
-Give your AI agents persistent memory with entity extraction, knowledge graphs, and STIX ontology -- no cloud, no API keys, works offline.
+Persistent memory for AI agents and Claude Code — with CTI entity extraction, STIX knowledge graphs, threat-actor alias resolution, and offline-first RAG. MCP server included. No cloud, no API keys.
 
 [![PyPI](https://img.shields.io/pypi/v/zettelforge)](https://pypi.org/project/zettelforge/)
-[![Downloads](https://static.pepy.tech/badge/zettelforge)](https://pepy.tech/projects/zettelforge)
-[![Stars](https://img.shields.io/github/stars/rolandpg/zettelforge)](https://github.com/rolandpg/zettelforge/stargazers)
-[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/licenses/MIT)
+[![Downloads/month](https://static.pepy.tech/personalized-badge/zettelforge?period=month&units=international_system&left_color=grey&right_color=blue&left_text=downloads%2Fmonth)](https://pepy.tech/projects/zettelforge)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://opensource.org/licenses/MIT)
 [![CI](https://github.com/rolandpg/zettelforge/actions/workflows/ci.yml/badge.svg)](https://github.com/rolandpg/zettelforge/actions)
-[![Contributors](https://img.shields.io/github/contributors/rolandpg/zettelforge)](https://github.com/rolandpg/zettelforge/graphs/contributors)
-[![Last Commit](https://img.shields.io/github/last-commit/rolandpg/zettelforge)](https://github.com/rolandpg/zettelforge/commits/master)
-[![SafeSkill](https://safeskill.dev/api/badge/rolandpg-zettelforge)](https://safeskill.dev/scan/rolandpg-zettelforge)
 
+**[⭐ Star](https://github.com/rolandpg/zettelforge) · [📦 `pip install zettelforge`](https://pypi.org/project/zettelforge/) · [📖 Docs](https://docs.threatrecall.ai/) · [🧪 Hosted](https://threatrecall.ai)**
+
+<p align="center">
+<a href="https://www.buymeacoffee.com/xypher22pr0" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-green.png" alt="Buy Me a Coffee" style="height: 60px !important;width: 217px !important;" ></a>
+</p>
 <p align="center">
-  <img src="docs/assets/demo.gif" width="720" alt="ZettelForge demo — CTI agentic memory in action">
+  <img src="https://raw.githubusercontent.com/rolandpg/zettelforge/master/docs/assets/demo.gif" width="720" alt="ZettelForge demo — CTI agentic memory in action">
 </p>
 
+> If ZettelForge fits a CTI workflow you run, a star is the fastest signal that this category is worth continuing to invest in.
+
 ## Why ZettelForge?
 
 General-purpose memory systems don't understand threat intelligence. They can't tell APT28 from Fancy Bear, don't know that CVE-2024-3094 is the XZ Utils backdoor, and can't track how intelligence evolves across reports. When your agent forgets context between investigations, you end up re-reading the same reports and re-building the same mental models.
 
 ZettelForge was built from the ground up for analysts who think in threat graphs, not chat histories. It extracts CVEs, threat actors, IOCs, and MITRE ATT&CK techniques automatically, resolves aliases across naming conventions, builds a knowledge graph with causal relationships, and retrieves memories using intent-aware blended search -- all offline, with no API keys or cloud dependencies.
 
+>"Memory augmentation closes 33% of the gap between small and large models on CTI tasks (CTI-REALM, Microsoft 2026)." [1]
+
+
 | Feature | ZettelForge | Mem0 | Graphiti | Cognee |
 |---------|------------|------|----------|--------|
 | CTI entity extraction (CVEs, actors, IOCs) | Yes | No | No | No |
@@ -35,6 +41,12 @@ ZettelForge was built from the ground up for analysts who think in threat graphs
 | OCSF audit logging | Yes | No | No | No |
 | MCP server (Claude Code) | Yes | No | No | No |
 
+## Data Pipeline
+<p align="center">
+  <img src="https://raw.githubusercontent.com/rolandpg/zettelforge/master/docs/assets/zettelforge_architecture.svg" width="720" alt="ZettelForge architecture — Extract, Graph, Embed, Recall pipeline">
+</p>
+
+
 ## Features
 
 **Entity Extraction** -- Automatically identifies CVEs, threat actors, IOCs (IPs, domains, hashes, URLs, emails), MITRE ATT&CK techniques, campaigns, intrusion sets, tools, people, locations, and organizations. Regex + LLM NER with STIX 2.1 types throughout.
@@ -100,7 +112,7 @@ mm.remember(
 
 Every `remember()` call triggers a pipeline:
 
-1. **Entity Extraction** -- regex + LLM NER identifies CVEs, actors, tools, campaigns, people, locations, orgs (10 types)
+1. **Entity Extraction** -- regex + LLM NER identifies CVEs, intrusion sets, threat actors, tools, campaigns, ATT&CK techniques, IOCs (IPv4, domain, URL, MD5/SHA1/SHA256, email), people, locations, organizations, events, activities, and temporal references (19 types)
 2. **Knowledge Graph Update** -- entities become nodes, co-occurrence becomes edges, LLM infers causal triples
 3. **Vector Embedding** -- 768-dim fastembed (ONNX, in-process, 7ms/embed) stored in LanceDB
 4. **Supersession Check** -- entity overlap detection marks stale notes as superseded
@@ -158,26 +170,6 @@ python examples/athf_bridge.py /path/to/hunts/
 
 See [examples/athf_bridge.py](examples/athf_bridge.py).
 
-## Architecture
-
-```
-┌──────────────────────────────────────────────────────────────────────┐
-│                           MemoryManager                              │
-│  remember()  remember_with_extraction()  recall()  synthesize()      │
-├──────────┬───────────┬──────────────┬───────────┬────────────────────┤
-│  Note    │  Fact     │   Memory     │  Blended  │   Synthesis        │
-│Constructor│ Extractor │  Updater     │ Retriever │   Generator        │
-│(enrich)  │(Phase 1)  │(Phase 2)     │(vec+graph)│   (RAG)            │
-├──────────┴───────────┴──────────────┼───────────┴────────────────────┤
-│       Entity Indexer + Alias        │  Intent Classifier             │
-│       Resolver                      │  (factual/temporal/causal)     │
-├─────────────────────────────────────┼────────────────────────────────┤
-│   Knowledge Graph (SQLite)          │  LanceDB (Vectors)             │
-│   Entity nodes + edges              │  768-dim fastembed (ONNX)      │
-│   Causal triple inference           │  Zettelkasten notes            │
-│   SQLite WAL (TypeDB via extension) │  IVF_PQ index                  │
-└─────────────────────────────────────┴────────────────────────────────┘
-```
 
 ## Extensions
 
@@ -224,6 +216,12 @@ MIT -- See [LICENSE](LICENSE).
 
 **Made by Patrick Roland**.
 
+## Support the Project
+
+ZettelForge is MIT-licensed. If it's useful in your workflow and you'd like to help keep it maintained:
+
+<a href="https://www.buymeacoffee.com/xypher22pr0" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-green.png" alt="Buy Me a Coffee" style="height: 40px !important;width: 145px !important;" ></a>
+
 ## Acknowledgments
 
 - Inspired by [Zettelkasten](https://en.wikipedia.org/wiki/Zettelkasten) and [A-Mem](https://arxiv.org/abs/2602.10715) (NeurIPS 2025)
@@ -231,3 +229,5 @@ MIT -- See [LICENSE](LICENSE).
 - STIX 2.1 schema informed by [typedb-cti](https://github.com/typedb-osi/typedb-cti)
 - Benchmarked against [LOCOMO](https://snap-research.github.io/locomo/) (ACL 2024) and [CTIBench](https://arxiv.org/abs/2406.07599) (NeurIPS 2024)
 - [LanceDB](https://lancedb.com) | [fastembed](https://github.com/qdrant/fastembed) | [Pydantic](https://pydantic.dev) | [TypeDB](https://typedb.com)
+
+[1]: https://www.microsoft.com/en-us/security/blog/2026/03/20/cti-realm-a-new-benchmark-for-end-to-end-detection-rule-generation-with-ai-agents/

{zettelforge-2.2.0 → zettelforge-2.3.0}/SECURITY.md

@@ -6,7 +6,7 @@
 
 Report security issues by email to:
 
-**security@zettelforge.dev**
+**contact@threatrecall.ai**
 
 Include in your report:
 - A description of the vulnerability and its potential impact
@@ -39,9 +39,9 @@ backported to the prior minor release.
 
 | Version | Supported |
 |---|---|
-| 2.1.x (current) | Yes — active security support |
-| 2.0.x | Critical fixes only, for 60 days after 2.1.0 release |
-| < 2.0 | No — upgrade required |
+| 2.3.x (current) | Yes — active security support |
+| 2.2.x | Critical fixes only, for 60 days after 2.3.0 release |
+| < 2.2 | No — upgrade required |
 
 ---
 
@@ -53,8 +53,9 @@ The following components are covered by this policy:
 
 - **Memory pipeline** — `remember()`, `recall()`, `synthesize()`, and
   the two-phase extraction pipeline
-- **Storage layer** — JSONL notes store, LanceDB vector index, entity
-  index, knowledge graph JSONL
+- **Storage layer** — SQLite backend (notes, knowledge graph, entity index)
+  and the LanceDB vector index. Legacy JSONL paths still present for
+  migration are also in scope.
 - **MCP server** — all tool handlers exposed to Claude Code and other
   MCP clients
 - **REST API** — all FastAPI endpoints in `src/zettelforge/server.py`
@@ -83,11 +84,16 @@ The following components are covered by this policy:
 
 ### Data at Rest
 
-- Notes are stored as JSONL on local disk. No encryption at rest is
-  applied by ZettelForge itself — encrypt the filesystem or volume at
-  the OS level for sensitive deployments.
-- LanceDB vector index files are stored in the configured data directory
-  and carry the same recommendation.
+- Notes, the knowledge graph, and the entity index are stored in a local
+  SQLite database (WAL mode) under the configured data directory. No
+  encryption at rest is applied by ZettelForge itself — encrypt the
+  filesystem or volume at the OS level for sensitive deployments.
+- LanceDB vector index files live alongside the SQLite database and
+  carry the same recommendation.
+- Legacy v2.1.x deployments that still use JSONL (`notes.jsonl`,
+  `kg_nodes.jsonl`, `kg_edges.jsonl`, `entity_index.json`) should run
+  `scripts/migrate_jsonl_to_sqlite.py` — the JSONL paths are no longer
+  the default but remain supported as a migration input.
 
 ### Injection Defenses