mindforge-memory 6.4.1__tar.gz

mindforge_memory-6.4.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 MindForge
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mindforge_memory-6.4.1/MindForge.py

@@ -0,0 +1,20 @@
+# -*- coding: utf-8 -*-
+"""
+MindForge - AI Agent 终身记忆系统
+=======================================
+四层记忆架构 · 知识图谱引擎 · 多模态支持 · 人格化记忆 · 联邦网络 · AI短剧记忆
+
+兼容入口（自 v5.8.13 起）：全部源码已迁入 `mindforge.*` 命名空间，
+本文件仅为向后兼容保留，使 `from MindForge import MindForge` 在 pip install
+后仍然可用。新代码请统一使用 `from mindforge import MindForge`。
+"""
+
+from mindforge import *  # noqa: F401,F403
+from mindforge import (  # noqa: F401
+    __version__,
+    VERSION_INFO,
+    NEXT_MAJOR_TARGET,
+)
+
+__author__ = "MindForge Project"
+__license__ = "MIT"

mindforge_memory-6.4.1/PKG-INFO

@@ -0,0 +1,281 @@
+Metadata-Version: 2.4
+Name: mindforge-memory
+Version: 6.4.1
+Summary: AI Agent 终身记忆系统 - 四层记忆架构 · 知识图谱 · 多模态支持
+License: MIT
+Project-URL: Homepage, https://github.com/opok-ops/MindForge
+Project-URL: Documentation, https://opok-ops.github.io/MindForge/
+Project-URL: Bug Reports, https://github.com/opok-ops/MindForge/issues
+Project-URL: Source, https://github.com/opok-ops/MindForge
+Keywords: ai,agent,memory,knowledge-graph,llm
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: cryptography<52,>=50.0.2
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Provides-Extra: embedding
+Requires-Dist: sentence-transformers>=2.7.0; extra == "embedding"
+Dynamic: license-file
+
+# MindForge
+
+**A local-first, production-grade memory system for AI agents.**
+
+> **Maintainer** — **Gu Fengyu (顾冯雨)**, born 2008-01-10 · GitHub: [opok-ops](https://github.com/opok-ops) · 抖音: [shidianduo3116](https://www.douyin.com/user/shidianduo3116) · 小红书: [4406811524](https://xhslink.cn/o/3m71WBRPD17) · Feedback: [2638895480@qq.com](mailto:2638895480@qq.com)
+
+MindForge gives AI agents a persistent, structured long-term memory: a four-tier lifecycle model, knowledge graphs, hybrid retrieval, end-to-end encryption, and MCP / CLI / REST interfaces. All data stays on your machine.
+
+> **The story** — one developer, seven months, from v1.0 (March 2026) to v6.4.1: no team, no funding, 1,220 passing tests. [Read the journey →](https://opok-ops.github.io/MindForge/#developer)
+
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Version](https://img.shields.io/badge/version-6.4.1-green.svg)](https://github.com/opok-ops/MindForge/releases)
+[![Stars](https://img.shields.io/github/stars/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/stargazers)
+[![Forks](https://img.shields.io/github/forks/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/forks)
+[![Open Issues](https://img.shields.io/github/issues/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/issues)
+[![Last Commit](https://img.shields.io/github/last-commit/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/commits/master)
+[![CI](https://github.com/opok-ops/MindForge/actions/workflows/ci.yml/badge.svg)](https://github.com/opok-ops/MindForge/actions/workflows/ci.yml)
+[![Documentation](https://img.shields.io/badge/docs-pages-blue.svg)](https://opok-ops.github.io/MindForge/)
+
+[English](README.md) · [中文文档](README.zh-CN.md) · [Live Demo](https://opok-ops.github.io/MindForge/) · [Documentation](docs/) · [Discussions](https://github.com/opok-ops/MindForge/discussions)
+
+---
+
+## Overview
+
+Agent memory solutions that rely on cloud backends introduce latency, cost, and privacy trade-offs. MindForge takes the opposite approach: memory is computed and stored locally, encrypted at rest, and exposed through standard interfaces your existing agents already speak.
+
+Key design decisions:
+
+- **Local-first.** No cloud dependency. Storage runs on SQLite + FTS5 with WAL, a single file that moves with you.
+- **Structured memory.** Four tiers (sensory, short-term, long-term, permanent) with promotion and decay driven by the Ebbinghaus forgetting curve.
+- **Retrieval quality.** Six complementary retrieval paths fused into one ranked result, instead of a single vector index.
+- **Encryption by default.** AES-256-GCM at rest with PBKDF2 key derivation; encrypted backups you can export and trust.
+- **Open interfaces.** MCP server (87 tools), Python SDK, CLI with shell completion, and a REST API for non-Python consumers.
+
+## Quick Start
+
+Install from source, then start writing and reading memories in minutes:
+
+```bash
+git clone https://github.com/opok-ops/MindForge.git
+cd MindForge
+pip install -e .               # zero ML deps: 3-path retrieval (TF-IDF + FTS5 + fuzzy)
+pip install -e ".[embedding]"  # + vector path → full 6-way hybrid search
+```
+
+> Without the `embedding` extra, MindForge runs in zero-dependency mode — fully functional, but paraphrased queries may go unrecalled (no semantic path). With it, a small sentence-transformers model downloads on first vector search.
+
+```python
+from mindforge import MindForge
+
+m = MindForge(db_path="./data/memory.db", encrypted=False)
+m.add("User prefers type-hinted Python code", category="preferences", importance="HIGH")
+result = m.search("coding preferences")
+print(result.chunks[0].content)
+```
+
+CLI:
+
+```bash
+MindForge add "User likes cats" --category preferences --importance HIGH
+MindForge search "what does user like"
+MindForge stats
+```
+
+## What's New
+
+v6.1.3 – v6.4.1 shipped in nine days (2026-11-01 → 11-09), guarded by **1,224 tests** (1219+4 passed, 1 skipped).
+
+- **v6.4.1 — million-character millisecond search** (2026-11-09). Scale benchmark (8,000 memories / ~990k chars) exposed a fuzzy-fallback long tail: search p95 881ms. Root causes: SequenceMatcher wasted on mathematically-impossible matches (ratio = 2M/(m+n), M ≤ m, so n ≥ 4m can never pass 0.4) + a ~50μs/entry Python full-scan tax. Fixed with a math-capped prune + SQLite-level candidate pre-filtering: **search p95 881→30.7ms (28×)**, add p50 2.6ms flat at 10k-scale. New `scripts/bench_scale.py` + CI scale guard (3000 memories, millisecond contract) + 4 behavior-level regression tests pin the semantics.
+
+- **v6.4.0 — resident daemon & performance guards** (2026-11-08). New `mindforge daemon` subcommand: line-JSON over stdio, engine resident in a long-lived process — per-request p50 2.5ms vs 400–700ms for CLI subprocess spawning (two orders of magnitude; 2026-11-09 quiet-machine re-bench: 2.5ms vs 401ms p50, ~160×). CI now guards latency (engine in-process thresholds) and daemon 13-step E2E alongside the 16-step CLI E2E guard.
+- **v6.3.6 — E2E-verified fixes** (2026-11-07). Found by real-bridge E2E testing of dsh-mindforge v3.0: compression TF-IDF index was never hydrated on real stores (`compress` always reported 0 clusters — now hydrated from storage), four v6.3.5 commands ignored `--json` (fixed), `adaptive_search` leaked `session_id` into privacy access control (INTERNAL memories always rejected — permanently empty results, now separated), and `key_file=""` misdetected as encryption. Tests 1210 → 1219.
+- **v6.3.5 — Memory compression & adaptive context** (2026-11-07). Two new capabilities: MemoryCompressor clusters near-duplicate memories (TF-IDF cosine), replaces them with a searchable summary, and restores originals on `decompress` — compressed sources stop appearing in every retrieval path; STRICT memories never participate. AdaptiveContextEngine tracks per-session query patterns and dynamically adjusts the relevance threshold (exploration mode lowers it, focus mode raises it, always bounded). Facade methods + four new CLI commands (`compress`, `compress-list`, `decompress`, `adaptive-search`). Also fixes a P0-class integration issue in the draft compression write path (raw SQL bypassing FTS/embedding/audit — including a plaintext-into-encrypted-store flaw) and a ZeroDivisionError in keyword search.
+- **v6.3.2 — Explain everywhere** (2026-11-06). Retrieval explainability, previously CLI/Python-only, is now aligned across all three interfaces: MCP `memory_search` gains an `explain` parameter and REST `/api/search` accepts `?explain=1` — per-result score breakdowns (per-path scores, winner, fusion details) everywhere, zero overhead and byte-identical responses when not requested.
+- **v6.3.1 — Encryption upgrade: scrypt KDF** (2026-11-06). The default key-derivation function moves from PBKDF2 to scrypt (N=2^16, r=8, p=1 — a 64 MiB memory wall): GPU/ASIC rigs can no longer parallelize offline brute-force; per-derivation cost ~0.3 s. New key files and encrypted exports use scrypt; every historical key file (PBKDF2 600k / 60k / legacy) still decrypts unchanged, and `rekey` migrates an existing store in one step. Also fixes a P0: facade `rekey()` had been broken since v5.8.13 (dead import path), and bumps cryptography to ≥50.0.2.
+- **v6.3.0 — Retrieval explainability** (`search --explain`). Hybrid search used to be a black box: you saw the ranking, not the reasoning. Every result can now carry its full score breakdown — per-path scores (TF-IDF / FTS5 / fuzzy / vector), the winning path, and text+vector fusion weights when both hit. Debug why a memory ranked first — or why another was never recalled at all. Zero overhead when disabled.
+- **v6.2.0 — Growth governance & reproducible benchmarks.** Audit logs get a brake: search-audit debouncing (one row per unique query per 60 s, compliance events never sampled or pruned), `audit-prune` for trimming, and a 7th doctor check that flags runaway growth. Also ships `benchmarks/repro_benchmark.py` — the methodology that killed the "encrypted writes are faster" benchmark artifact: same-N alternating rounds, P50/P95 latency over 50 samples per query type, 8-thread concurrency runs, orphan-repair verification.
+- **v6.1.3 — Self-healing data integrity.** GDPR erasure now removes derived data too — embeddings, notes, review schedules, evolution history, and graph relations (five tables previously survived "the right to be forgotten"). `doctor` gained orphan detection with one-command repair (`--fix`): dangling graph references, orphaned versions, and stray embeddings left behind by older cascade deletes are found and cleaned, with `dry_run` for impact assessment.
+- **v6.1.1 — Follow-up closed loop.** Memories don't just wait to be queried — they nudge. Follow-up items can now trigger proactive reminders when their time comes (87 MCP tools total).
+
+## Features
+
+**v6.0.0 — multi-agent spaces & 6G preview (2026-10-31)**
+
+- **Multi-agent memory spaces** — shared spaces with three-tier roles (owner / editor / reader), last-write-wins conflict resolution, and PRIVATE / STRICT privacy guardrails. Promoted from experimental to a supported capability; available through the `multi_agent` facade, CLI `space-*`, MCP `space_*`, and the v6 port registry (status `AVAILABLE`).
+- **6G AI memory preview** — four capabilities on the preview port, each replacing a former stub: parallel Preview Channels (isolated SQLite stores that never touch the stable database), context distillation (`distill_context` ranks memories HIGH / MEDIUM / LOW under a token budget), incremental index statistics (write-time indexing with near-real-time recall metrics), and per-memory timelines (versions, evolutions, and audit trail). Available through CLI `preview-*`, REST `/api/preview/*`, and MCP `preview_*`.
+- **Agent workflow toolkit** — eight features covering the full memory lifecycle: chat-history import for cold start, timeline visualization, shareable skill packs, memory evolution, outcome-based learning, memory passports for cross-model migration, explainable principled forgetting, and follow-up tracking. Each is exposed consistently across CLI, REST, and MCP.
+- **v6 port registry** — `v6_status()` reports capability readiness; `v6_port(capability)` injects a ready-to-use instance (multi-agent adapter or memory preview), so callers can depend on ports instead of concrete modules.
+
+**Retrieval**
+
+- **Six-way hybrid search** — vector, FTS5, TF-IDF, fuzzy, query expansion, and cross-encoder reranking fused into a single ranked result set.
+- **Knowledge graph** — automatic entity and relation extraction, path finding, and associative recall.
+- **Weighted recall** — multi-factor scoring (coverage, importance, access frequency, time decay) instead of similarity alone.
+
+**Memory lifecycle**
+
+- **Four-tier memory** — sensory buffer, short-term, long-term, and permanent tiers with Ebbinghaus-based decay and periodic consolidation.
+- **Conflict detection** — antonym pairs, attribute inconsistencies, and timeline conflicts, with automatic decay.
+- **Procedural memory (Cases → Skills)** — `record_experience()` stores agent success cases, `distill_skills()` turns them into reusable skill templates (persisted, restart-safe), `skill_match()`/`skill_render()` reuse them on new tasks with parameterized steps (v5.8.0).
+- **Knowledge-graph pipeline** — `auto_extract_graph` hooks into `add()`; `extract_graph()` draws entities/relations into a persisted graph (survives restarts), `graph_related`/`graph_path` query it, STRICT-privacy memories are never extracted (v5.7.9).
+- **Reproducible embedding eval** — `benchmarks/embedding_eval.py` compares hybrid / vector / FTS5 / TF-IDF on a fixed multilingual set with fixed seeds (Recall@5 / MRR@10 / NDCG@10); deterministic `fake` backend runs without any model for CI baselines (v5.7.8).
+- **Agent memory governance** — `agent_pin` freezes decay and pins key memories, `agent_forget` soft-deletes with a reason, `agent_decay_boost` accelerates forgetting, and `expire_memory` closes a fact's validity window (v5.7.7).
+- **Conflict auto-reconcile** — `reconcile_conflicts` closes the stale side of `keep_newer` / `keep_higher_importance` conflicts (Bi-temporal expiry, never deletes), and routes `merge` / `review_needed` to human review (v5.7.7).
+- **Connector framework** — pluggable `json` / `csv` / `markdown` / `file` / `url` connectors with a registry and SSRF guard; extensible via `register_connector` (v5.7.7).
+- **Bi-temporal facts** — `valid_from` / `valid_to` windows with as-of queries (`valid_at`) and `supersede` fact replacement that expires the old fact instead of deleting it, keeping full version history (v5.7.6).
+- **Skill extraction** — clusters memories into reusable slots, steps, and trigger words.
+
+**Privacy and security**
+
+- **Privacy engine** — four isolation levels (PUBLIC / INTERNAL / PRIVATE / STRICT) with fine-grained access control.
+- **At-rest encryption** — AES-256-GCM with PBKDF2-SHA256 key derivation.
+- **Encrypted export / import** — `export-json --password` and `import-json --password` produce encrypted backups that remain unreadable even if the file leaks (v5.7.3).
+- **GDPR toolkit** — `gdpr-report` / `gdpr-export-all` / `gdpr-erase` for data portability and the right to be forgotten, with automatic backup before erasure (v5.7.6).
+- **Memory health report** — `memory_report()` aggregates four-tier counts, TTL expirations, trash, conflict events, FTS consistency, database health, and encryption status, without exposing any plaintext memory content (v5.9.0).
+- **Audit export** — `export_audit()` returns machine-readable audit entries (limit / since / actor filters) for GDPR and forensics; exposed as `GET /api/audit` (v5.9.0).
+- **Privacy health check** — `privacy_report()` aggregates privacy-tier distribution, STRICT count, trash, audit events, encryption / key-file state, and the federated sharing surface, without exposing any plaintext memory content (v5.9.2).
+- **Knowledge-graph export** — `export_graph()` exports entities and relations as structured JSON (schema `mindforge-knowledge-graph/1.0`, memory IDs attached, no plaintext) for external visualization, audit, and migration (v5.9.2).
+- **Privacy-filtered graph export** — `export_graph(max_privacy="INTERNAL")` drops edges sourced from STRICT/PRIVATE memories (fail-closed, isolated nodes pruned); CLI `--privacy` and REST `GET /api/meta/graph?max_privacy=` support the same filter (v5.9.3).
+- **Unknown-privacy grouping** — `privacy_report()` buckets unknown/malformed privacy levels into `unknown_privacy_count` instead of leaking raw level names into the distribution (v5.9.3).
+
+**Integration**
+
+- **MCP server** — 87 tools, drop into Claude Code, OpenClaw, or any MCP client.
+- **Federated memory** — P2P sharing between agents with trust levels, ACLs, and conflict resolution.
+- **REST API** — standard HTTP endpoints for non-Python applications.
+- **CLI** — 200+ commands with bash / zsh / fish completion.
+- **Embedding backends** — sentence-transformers, OpenAI, Ollama, or a custom HTTP endpoint.
+
+## Architecture
+
+![MindForge Architecture](docs/architecture.png)
+
+```
+┌──────────────────────────────────────────────────────┐
+│           MindForge v6.4.1            │
+│  Cognitive Layer  Personality · KnowledgeGraph       │
+│                   MemoryEvolution · FederatedMemory  │
+├──────────────────────────────────────────────────────┤
+│  Function Layer   RecallEngine · HybridSearch        │
+│                   IntentRouter · ConflictDetector    │
+│                   SkillExtractor · SessionFocus       │
+├──────────────────────────────────────────────────────┤
+│  Core Layer       Storage (SQLite+FTS5) · Index     │
+│                   Encryption (AES-256-GCM) · Query  │
+├──────────────────────────────────────────────────────┤
+│  Adapter Layer    OpenClaw · Claude Code · MCP       │
+│                   REST API · CLI · Python SDK        │
+└──────────────────────────────────────────────────────┘
+```
+
+### Four-Tier Memory Model
+
+| Tier | Retention | Capacity | Purpose |
+|------|-----------|----------|---------|
+| Sensory | sec ~ min | ~50 | Input buffer, noise filtering |
+| Short-term | hrs ~ days | ~100 | Working memory, active session |
+| Long-term | wks ~ months | unlimited | Consolidated semantic memory |
+| Permanent | forever | unlimited | Core knowledge, preferences, key experience |
+
+High-value entries strengthen over time through consolidation; low-value entries decay naturally.
+
+## Performance
+
+Benchmarked on i7-12700H / 32GB RAM / NVMe SSD / Python 3.12 / SQLite WAL.
+
+| Operation | 1K | 10K | 100K |
+|-----------|-----|------|------|
+| Write (plaintext) | 0.3 ms | 0.5 ms | 0.9 ms |
+| Write (encrypted) | 0.8 ms | 1.2 ms | 2.1 ms |
+| TF-IDF Search P50 | 4 ms | 12 ms | 38 ms |
+| FTS5 Search P50 | 0.4 ms | 0.9 ms | 3.2 ms |
+| Cross-Encoder Rerank P50 | 5 ms | 15 ms | 45 ms |
+| Consolidate (100 entries) | 45 ms | 120 ms | 580 ms |
+
+Retrieval quality (500 multi-domain memories, 50 annotated queries):
+
+| Metric | TF-IDF | +FTS5+Fuzzy | +Cross-Encoder | +Query Expansion + Vector |
+|--------|--------|-------------|----------------|---------------------------|
+| MRR@10 | 0.62 | 0.71 | **0.82** | **0.85** |
+| NDCG@10 | 0.60 | 0.69 | **0.80** | **0.84** |
+| Recall@10 | 0.70 | 0.80 | **0.88** | **0.92** |
+
+## Integration
+
+**Claude Code**
+
+```python
+from adapters import ClaudeCodeAdapter
+
+adapter = ClaudeCodeAdapter.from_env()
+adapter.remember("User prefers concise code style", ["preferences"])
+context = adapter.get_context("database optimization")
+```
+
+**MCP Server**
+
+```bash
+MindForge-mcp --db-path ./data/memory.db
+```
+
+**REST API**
+
+```bash
+MindForge serve --api --port 9000
+# GET  /api/memories
+# POST /api/memories
+# GET  /api/search?q=...
+# GET  /api/stats
+```
+
+**OpenClaw**
+
+```yaml
+memory:
+  adapter: MindForge
+  adapter_config:
+    db_path: ~/.MindForge/data/memory.db
+    encrypted: true
+    auto_consolidate: true
+```
+
+## Security
+
+- **Encrypted databases**: when a key file is present, set `MINDFORGE_PASSWORD` so the process can unlock the store. This is required for the dsh-mindforge bridge and any non-interactive integration (MCP / REST API / scripts) accessing an encrypted store.
+- **API authentication**: if `MINDFORGE_API_KEY` is configured, unauthenticated callers are treated as `anonymous` and can only reach PUBLIC memories. All read/write paths carry actor identity for authorization and audit.
+- **Access logs are sanitized**: search terms and memory IDs are stripped from logs.
+- **Upgrade note**: v5.5.7 removed the experimental HMAC-XOR downgrade path (config flag `EXPERIMENTAL_HMAC_XOR`, the 「降级加密」 experiment). Blobs created under that flag can no longer be decrypted — back up your database before upgrading if you enabled it.
+
+See [SECURITY.md](SECURITY.md) for the full security policy.
+
+## Roadmap
+
+- [x] **v6.0** — multi-agent memory spaces and the 6G AI memory preview, both promoted from experimental ports to supported capabilities, alongside an eight-feature agent workflow toolkit (released 2026-10-31)
+- [x] **v6.1 – v6.3** — trust & quality: retrieval explainability (`search --explain`), data-integrity self-repair in `doctor`, GDPR-complete erasure, audit-log growth governance, reproducible benchmarks (released 2026-11-05)
+- [ ] Vector database backend (Qdrant / Chroma)
+- [ ] Web UI dashboard
+- [ ] Plugin ecosystem
+
+## Documentation
+
+- [Full documentation](docs/)
+- [Changelog](CHANGELOG.md)
+- [Contributing guide](CONTRIBUTING.md)
+- [Security policy](SECURITY.md)
+
+## License
+
+[MIT](LICENSE) — Copyright (c) 2026 MindForge Project. If MindForge is useful to you, consider starring it on GitHub — it helps the project stay active and visible.

mindforge_memory-6.4.1/README.md

@@ -0,0 +1,251 @@
+# MindForge
+
+**A local-first, production-grade memory system for AI agents.**
+
+> **Maintainer** — **Gu Fengyu (顾冯雨)**, born 2008-01-10 · GitHub: [opok-ops](https://github.com/opok-ops) · 抖音: [shidianduo3116](https://www.douyin.com/user/shidianduo3116) · 小红书: [4406811524](https://xhslink.cn/o/3m71WBRPD17) · Feedback: [2638895480@qq.com](mailto:2638895480@qq.com)
+
+MindForge gives AI agents a persistent, structured long-term memory: a four-tier lifecycle model, knowledge graphs, hybrid retrieval, end-to-end encryption, and MCP / CLI / REST interfaces. All data stays on your machine.
+
+> **The story** — one developer, seven months, from v1.0 (March 2026) to v6.4.1: no team, no funding, 1,220 passing tests. [Read the journey →](https://opok-ops.github.io/MindForge/#developer)
+
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![Version](https://img.shields.io/badge/version-6.4.1-green.svg)](https://github.com/opok-ops/MindForge/releases)
+[![Stars](https://img.shields.io/github/stars/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/stargazers)
+[![Forks](https://img.shields.io/github/forks/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/forks)
+[![Open Issues](https://img.shields.io/github/issues/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/issues)
+[![Last Commit](https://img.shields.io/github/last-commit/opok-ops/MindForge.svg)](https://github.com/opok-ops/MindForge/commits/master)
+[![CI](https://github.com/opok-ops/MindForge/actions/workflows/ci.yml/badge.svg)](https://github.com/opok-ops/MindForge/actions/workflows/ci.yml)
+[![Documentation](https://img.shields.io/badge/docs-pages-blue.svg)](https://opok-ops.github.io/MindForge/)
+
+[English](README.md) · [中文文档](README.zh-CN.md) · [Live Demo](https://opok-ops.github.io/MindForge/) · [Documentation](docs/) · [Discussions](https://github.com/opok-ops/MindForge/discussions)
+
+---
+
+## Overview
+
+Agent memory solutions that rely on cloud backends introduce latency, cost, and privacy trade-offs. MindForge takes the opposite approach: memory is computed and stored locally, encrypted at rest, and exposed through standard interfaces your existing agents already speak.
+
+Key design decisions:
+
+- **Local-first.** No cloud dependency. Storage runs on SQLite + FTS5 with WAL, a single file that moves with you.
+- **Structured memory.** Four tiers (sensory, short-term, long-term, permanent) with promotion and decay driven by the Ebbinghaus forgetting curve.
+- **Retrieval quality.** Six complementary retrieval paths fused into one ranked result, instead of a single vector index.
+- **Encryption by default.** AES-256-GCM at rest with PBKDF2 key derivation; encrypted backups you can export and trust.
+- **Open interfaces.** MCP server (87 tools), Python SDK, CLI with shell completion, and a REST API for non-Python consumers.
+
+## Quick Start
+
+Install from source, then start writing and reading memories in minutes:
+
+```bash
+git clone https://github.com/opok-ops/MindForge.git
+cd MindForge
+pip install -e .               # zero ML deps: 3-path retrieval (TF-IDF + FTS5 + fuzzy)
+pip install -e ".[embedding]"  # + vector path → full 6-way hybrid search
+```
+
+> Without the `embedding` extra, MindForge runs in zero-dependency mode — fully functional, but paraphrased queries may go unrecalled (no semantic path). With it, a small sentence-transformers model downloads on first vector search.
+
+```python
+from mindforge import MindForge
+
+m = MindForge(db_path="./data/memory.db", encrypted=False)
+m.add("User prefers type-hinted Python code", category="preferences", importance="HIGH")
+result = m.search("coding preferences")
+print(result.chunks[0].content)
+```
+
+CLI:
+
+```bash
+MindForge add "User likes cats" --category preferences --importance HIGH
+MindForge search "what does user like"
+MindForge stats
+```
+
+## What's New
+
+v6.1.3 – v6.4.1 shipped in nine days (2026-11-01 → 11-09), guarded by **1,224 tests** (1219+4 passed, 1 skipped).
+
+- **v6.4.1 — million-character millisecond search** (2026-11-09). Scale benchmark (8,000 memories / ~990k chars) exposed a fuzzy-fallback long tail: search p95 881ms. Root causes: SequenceMatcher wasted on mathematically-impossible matches (ratio = 2M/(m+n), M ≤ m, so n ≥ 4m can never pass 0.4) + a ~50μs/entry Python full-scan tax. Fixed with a math-capped prune + SQLite-level candidate pre-filtering: **search p95 881→30.7ms (28×)**, add p50 2.6ms flat at 10k-scale. New `scripts/bench_scale.py` + CI scale guard (3000 memories, millisecond contract) + 4 behavior-level regression tests pin the semantics.
+
+- **v6.4.0 — resident daemon & performance guards** (2026-11-08). New `mindforge daemon` subcommand: line-JSON over stdio, engine resident in a long-lived process — per-request p50 2.5ms vs 400–700ms for CLI subprocess spawning (two orders of magnitude; 2026-11-09 quiet-machine re-bench: 2.5ms vs 401ms p50, ~160×). CI now guards latency (engine in-process thresholds) and daemon 13-step E2E alongside the 16-step CLI E2E guard.
+- **v6.3.6 — E2E-verified fixes** (2026-11-07). Found by real-bridge E2E testing of dsh-mindforge v3.0: compression TF-IDF index was never hydrated on real stores (`compress` always reported 0 clusters — now hydrated from storage), four v6.3.5 commands ignored `--json` (fixed), `adaptive_search` leaked `session_id` into privacy access control (INTERNAL memories always rejected — permanently empty results, now separated), and `key_file=""` misdetected as encryption. Tests 1210 → 1219.
+- **v6.3.5 — Memory compression & adaptive context** (2026-11-07). Two new capabilities: MemoryCompressor clusters near-duplicate memories (TF-IDF cosine), replaces them with a searchable summary, and restores originals on `decompress` — compressed sources stop appearing in every retrieval path; STRICT memories never participate. AdaptiveContextEngine tracks per-session query patterns and dynamically adjusts the relevance threshold (exploration mode lowers it, focus mode raises it, always bounded). Facade methods + four new CLI commands (`compress`, `compress-list`, `decompress`, `adaptive-search`). Also fixes a P0-class integration issue in the draft compression write path (raw SQL bypassing FTS/embedding/audit — including a plaintext-into-encrypted-store flaw) and a ZeroDivisionError in keyword search.
+- **v6.3.2 — Explain everywhere** (2026-11-06). Retrieval explainability, previously CLI/Python-only, is now aligned across all three interfaces: MCP `memory_search` gains an `explain` parameter and REST `/api/search` accepts `?explain=1` — per-result score breakdowns (per-path scores, winner, fusion details) everywhere, zero overhead and byte-identical responses when not requested.
+- **v6.3.1 — Encryption upgrade: scrypt KDF** (2026-11-06). The default key-derivation function moves from PBKDF2 to scrypt (N=2^16, r=8, p=1 — a 64 MiB memory wall): GPU/ASIC rigs can no longer parallelize offline brute-force; per-derivation cost ~0.3 s. New key files and encrypted exports use scrypt; every historical key file (PBKDF2 600k / 60k / legacy) still decrypts unchanged, and `rekey` migrates an existing store in one step. Also fixes a P0: facade `rekey()` had been broken since v5.8.13 (dead import path), and bumps cryptography to ≥50.0.2.
+- **v6.3.0 — Retrieval explainability** (`search --explain`). Hybrid search used to be a black box: you saw the ranking, not the reasoning. Every result can now carry its full score breakdown — per-path scores (TF-IDF / FTS5 / fuzzy / vector), the winning path, and text+vector fusion weights when both hit. Debug why a memory ranked first — or why another was never recalled at all. Zero overhead when disabled.
+- **v6.2.0 — Growth governance & reproducible benchmarks.** Audit logs get a brake: search-audit debouncing (one row per unique query per 60 s, compliance events never sampled or pruned), `audit-prune` for trimming, and a 7th doctor check that flags runaway growth. Also ships `benchmarks/repro_benchmark.py` — the methodology that killed the "encrypted writes are faster" benchmark artifact: same-N alternating rounds, P50/P95 latency over 50 samples per query type, 8-thread concurrency runs, orphan-repair verification.
+- **v6.1.3 — Self-healing data integrity.** GDPR erasure now removes derived data too — embeddings, notes, review schedules, evolution history, and graph relations (five tables previously survived "the right to be forgotten"). `doctor` gained orphan detection with one-command repair (`--fix`): dangling graph references, orphaned versions, and stray embeddings left behind by older cascade deletes are found and cleaned, with `dry_run` for impact assessment.
+- **v6.1.1 — Follow-up closed loop.** Memories don't just wait to be queried — they nudge. Follow-up items can now trigger proactive reminders when their time comes (87 MCP tools total).
+
+## Features
+
+**v6.0.0 — multi-agent spaces & 6G preview (2026-10-31)**
+
+- **Multi-agent memory spaces** — shared spaces with three-tier roles (owner / editor / reader), last-write-wins conflict resolution, and PRIVATE / STRICT privacy guardrails. Promoted from experimental to a supported capability; available through the `multi_agent` facade, CLI `space-*`, MCP `space_*`, and the v6 port registry (status `AVAILABLE`).
+- **6G AI memory preview** — four capabilities on the preview port, each replacing a former stub: parallel Preview Channels (isolated SQLite stores that never touch the stable database), context distillation (`distill_context` ranks memories HIGH / MEDIUM / LOW under a token budget), incremental index statistics (write-time indexing with near-real-time recall metrics), and per-memory timelines (versions, evolutions, and audit trail). Available through CLI `preview-*`, REST `/api/preview/*`, and MCP `preview_*`.
+- **Agent workflow toolkit** — eight features covering the full memory lifecycle: chat-history import for cold start, timeline visualization, shareable skill packs, memory evolution, outcome-based learning, memory passports for cross-model migration, explainable principled forgetting, and follow-up tracking. Each is exposed consistently across CLI, REST, and MCP.
+- **v6 port registry** — `v6_status()` reports capability readiness; `v6_port(capability)` injects a ready-to-use instance (multi-agent adapter or memory preview), so callers can depend on ports instead of concrete modules.
+
+**Retrieval**
+
+- **Six-way hybrid search** — vector, FTS5, TF-IDF, fuzzy, query expansion, and cross-encoder reranking fused into a single ranked result set.
+- **Knowledge graph** — automatic entity and relation extraction, path finding, and associative recall.
+- **Weighted recall** — multi-factor scoring (coverage, importance, access frequency, time decay) instead of similarity alone.
+
+**Memory lifecycle**
+
+- **Four-tier memory** — sensory buffer, short-term, long-term, and permanent tiers with Ebbinghaus-based decay and periodic consolidation.
+- **Conflict detection** — antonym pairs, attribute inconsistencies, and timeline conflicts, with automatic decay.
+- **Procedural memory (Cases → Skills)** — `record_experience()` stores agent success cases, `distill_skills()` turns them into reusable skill templates (persisted, restart-safe), `skill_match()`/`skill_render()` reuse them on new tasks with parameterized steps (v5.8.0).
+- **Knowledge-graph pipeline** — `auto_extract_graph` hooks into `add()`; `extract_graph()` draws entities/relations into a persisted graph (survives restarts), `graph_related`/`graph_path` query it, STRICT-privacy memories are never extracted (v5.7.9).
+- **Reproducible embedding eval** — `benchmarks/embedding_eval.py` compares hybrid / vector / FTS5 / TF-IDF on a fixed multilingual set with fixed seeds (Recall@5 / MRR@10 / NDCG@10); deterministic `fake` backend runs without any model for CI baselines (v5.7.8).
+- **Agent memory governance** — `agent_pin` freezes decay and pins key memories, `agent_forget` soft-deletes with a reason, `agent_decay_boost` accelerates forgetting, and `expire_memory` closes a fact's validity window (v5.7.7).
+- **Conflict auto-reconcile** — `reconcile_conflicts` closes the stale side of `keep_newer` / `keep_higher_importance` conflicts (Bi-temporal expiry, never deletes), and routes `merge` / `review_needed` to human review (v5.7.7).
+- **Connector framework** — pluggable `json` / `csv` / `markdown` / `file` / `url` connectors with a registry and SSRF guard; extensible via `register_connector` (v5.7.7).
+- **Bi-temporal facts** — `valid_from` / `valid_to` windows with as-of queries (`valid_at`) and `supersede` fact replacement that expires the old fact instead of deleting it, keeping full version history (v5.7.6).
+- **Skill extraction** — clusters memories into reusable slots, steps, and trigger words.
+
+**Privacy and security**
+
+- **Privacy engine** — four isolation levels (PUBLIC / INTERNAL / PRIVATE / STRICT) with fine-grained access control.
+- **At-rest encryption** — AES-256-GCM with PBKDF2-SHA256 key derivation.
+- **Encrypted export / import** — `export-json --password` and `import-json --password` produce encrypted backups that remain unreadable even if the file leaks (v5.7.3).
+- **GDPR toolkit** — `gdpr-report` / `gdpr-export-all` / `gdpr-erase` for data portability and the right to be forgotten, with automatic backup before erasure (v5.7.6).
+- **Memory health report** — `memory_report()` aggregates four-tier counts, TTL expirations, trash, conflict events, FTS consistency, database health, and encryption status, without exposing any plaintext memory content (v5.9.0).
+- **Audit export** — `export_audit()` returns machine-readable audit entries (limit / since / actor filters) for GDPR and forensics; exposed as `GET /api/audit` (v5.9.0).
+- **Privacy health check** — `privacy_report()` aggregates privacy-tier distribution, STRICT count, trash, audit events, encryption / key-file state, and the federated sharing surface, without exposing any plaintext memory content (v5.9.2).
+- **Knowledge-graph export** — `export_graph()` exports entities and relations as structured JSON (schema `mindforge-knowledge-graph/1.0`, memory IDs attached, no plaintext) for external visualization, audit, and migration (v5.9.2).
+- **Privacy-filtered graph export** — `export_graph(max_privacy="INTERNAL")` drops edges sourced from STRICT/PRIVATE memories (fail-closed, isolated nodes pruned); CLI `--privacy` and REST `GET /api/meta/graph?max_privacy=` support the same filter (v5.9.3).
+- **Unknown-privacy grouping** — `privacy_report()` buckets unknown/malformed privacy levels into `unknown_privacy_count` instead of leaking raw level names into the distribution (v5.9.3).
+
+**Integration**
+
+- **MCP server** — 87 tools, drop into Claude Code, OpenClaw, or any MCP client.
+- **Federated memory** — P2P sharing between agents with trust levels, ACLs, and conflict resolution.
+- **REST API** — standard HTTP endpoints for non-Python applications.
+- **CLI** — 200+ commands with bash / zsh / fish completion.
+- **Embedding backends** — sentence-transformers, OpenAI, Ollama, or a custom HTTP endpoint.
+
+## Architecture
+
+![MindForge Architecture](docs/architecture.png)
+
+```
+┌──────────────────────────────────────────────────────┐
+│           MindForge v6.4.1            │
+│  Cognitive Layer  Personality · KnowledgeGraph       │
+│                   MemoryEvolution · FederatedMemory  │
+├──────────────────────────────────────────────────────┤
+│  Function Layer   RecallEngine · HybridSearch        │
+│                   IntentRouter · ConflictDetector    │
+│                   SkillExtractor · SessionFocus       │
+├──────────────────────────────────────────────────────┤
+│  Core Layer       Storage (SQLite+FTS5) · Index     │
+│                   Encryption (AES-256-GCM) · Query  │
+├──────────────────────────────────────────────────────┤
+│  Adapter Layer    OpenClaw · Claude Code · MCP       │
+│                   REST API · CLI · Python SDK        │
+└──────────────────────────────────────────────────────┘
+```
+
+### Four-Tier Memory Model
+
+| Tier | Retention | Capacity | Purpose |
+|------|-----------|----------|---------|
+| Sensory | sec ~ min | ~50 | Input buffer, noise filtering |
+| Short-term | hrs ~ days | ~100 | Working memory, active session |
+| Long-term | wks ~ months | unlimited | Consolidated semantic memory |
+| Permanent | forever | unlimited | Core knowledge, preferences, key experience |
+
+High-value entries strengthen over time through consolidation; low-value entries decay naturally.
+
+## Performance
+
+Benchmarked on i7-12700H / 32GB RAM / NVMe SSD / Python 3.12 / SQLite WAL.
+
+| Operation | 1K | 10K | 100K |
+|-----------|-----|------|------|
+| Write (plaintext) | 0.3 ms | 0.5 ms | 0.9 ms |
+| Write (encrypted) | 0.8 ms | 1.2 ms | 2.1 ms |
+| TF-IDF Search P50 | 4 ms | 12 ms | 38 ms |
+| FTS5 Search P50 | 0.4 ms | 0.9 ms | 3.2 ms |
+| Cross-Encoder Rerank P50 | 5 ms | 15 ms | 45 ms |
+| Consolidate (100 entries) | 45 ms | 120 ms | 580 ms |
+
+Retrieval quality (500 multi-domain memories, 50 annotated queries):
+
+| Metric | TF-IDF | +FTS5+Fuzzy | +Cross-Encoder | +Query Expansion + Vector |
+|--------|--------|-------------|----------------|---------------------------|
+| MRR@10 | 0.62 | 0.71 | **0.82** | **0.85** |
+| NDCG@10 | 0.60 | 0.69 | **0.80** | **0.84** |
+| Recall@10 | 0.70 | 0.80 | **0.88** | **0.92** |
+
+## Integration
+
+**Claude Code**
+
+```python
+from adapters import ClaudeCodeAdapter
+
+adapter = ClaudeCodeAdapter.from_env()
+adapter.remember("User prefers concise code style", ["preferences"])
+context = adapter.get_context("database optimization")
+```
+
+**MCP Server**
+
+```bash
+MindForge-mcp --db-path ./data/memory.db
+```
+
+**REST API**
+
+```bash
+MindForge serve --api --port 9000
+# GET  /api/memories
+# POST /api/memories
+# GET  /api/search?q=...
+# GET  /api/stats
+```
+
+**OpenClaw**
+
+```yaml
+memory:
+  adapter: MindForge
+  adapter_config:
+    db_path: ~/.MindForge/data/memory.db
+    encrypted: true
+    auto_consolidate: true
+```
+
+## Security
+
+- **Encrypted databases**: when a key file is present, set `MINDFORGE_PASSWORD` so the process can unlock the store. This is required for the dsh-mindforge bridge and any non-interactive integration (MCP / REST API / scripts) accessing an encrypted store.
+- **API authentication**: if `MINDFORGE_API_KEY` is configured, unauthenticated callers are treated as `anonymous` and can only reach PUBLIC memories. All read/write paths carry actor identity for authorization and audit.
+- **Access logs are sanitized**: search terms and memory IDs are stripped from logs.
+- **Upgrade note**: v5.5.7 removed the experimental HMAC-XOR downgrade path (config flag `EXPERIMENTAL_HMAC_XOR`, the 「降级加密」 experiment). Blobs created under that flag can no longer be decrypted — back up your database before upgrading if you enabled it.
+
+See [SECURITY.md](SECURITY.md) for the full security policy.
+
+## Roadmap
+
+- [x] **v6.0** — multi-agent memory spaces and the 6G AI memory preview, both promoted from experimental ports to supported capabilities, alongside an eight-feature agent workflow toolkit (released 2026-10-31)
+- [x] **v6.1 – v6.3** — trust & quality: retrieval explainability (`search --explain`), data-integrity self-repair in `doctor`, GDPR-complete erasure, audit-log growth governance, reproducible benchmarks (released 2026-11-05)
+- [ ] Vector database backend (Qdrant / Chroma)
+- [ ] Web UI dashboard
+- [ ] Plugin ecosystem
+
+## Documentation
+
+- [Full documentation](docs/)
+- [Changelog](CHANGELOG.md)
+- [Contributing guide](CONTRIBUTING.md)
+- [Security policy](SECURITY.md)
+
+## License
+
+[MIT](LICENSE) — Copyright (c) 2026 MindForge Project. If MindForge is useful to you, consider starring it on GitHub — it helps the project stay active and visible.