zettelforge 2.1.0__tar.gz

zettelforge-2.1.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,30 @@
+---
+name: Bug Report
+about: Report a bug in ZettelForge
+title: ''
+labels: bug
+assignees: ''
+---
+
+**Describe the bug**
+A clear description of what the bug is.
+
+**To reproduce**
+Steps to reproduce the behavior:
+1. ...
+2. ...
+
+**Expected behavior**
+What you expected to happen.
+
+**Environment**
+- ZettelForge version: (e.g., 2.1.0)
+- Python version: (e.g., 3.11)
+- OS: (e.g., Ubuntu 22.04)
+- Backend: (jsonl / typedb)
+- Edition: (community / enterprise)
+
+**Logs / traceback**
+```
+Paste any relevant error output here.
+```

zettelforge-2.1.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: Feature Request
+about: Suggest a feature for ZettelForge
+title: ''
+labels: enhancement
+assignees: ''
+---
+
+**Problem**
+What problem does this feature solve?
+
+**Proposed solution**
+Describe the feature you'd like.
+
+**Alternatives considered**
+Any alternative approaches you've thought about.
+
+**Edition**
+Should this be a Community or Enterprise feature? (See [edition guide](../../src/zettelforge/edition.py) for the boundary.)

zettelforge-2.1.0/.github/SECURITY.md

@@ -0,0 +1,33 @@
+# Security Policy
+
+## Supported Versions
+
+| Version | Supported          |
+|---------|--------------------|
+| 2.1.x   | Yes                |
+| < 2.0   | No                 |
+
+## Reporting a Vulnerability
+
+If you discover a security vulnerability in ZettelForge, please report it
+responsibly. **Do not open a public GitHub issue.**
+
+Email **security@threatengram.com** with:
+
+1. Description of the vulnerability
+2. Steps to reproduce
+3. Potential impact
+4. Suggested fix (if any)
+
+We will acknowledge your report within 48 hours and aim to provide a fix
+within 7 days for critical issues.
+
+## Scope
+
+This policy covers the ZettelForge open-source codebase. For vulnerabilities
+in ThreatRecall Enterprise or hosted services, contact the same email.
+
+## Recognition
+
+We appreciate security researchers who help keep ZettelForge safe. With your
+permission, we will acknowledge your contribution in release notes.

zettelforge-2.1.0/.github/pull_request_template.md

@@ -0,0 +1,18 @@
+## Summary
+
+Brief description of changes.
+
+## Related issue
+
+Fixes #
+
+## Changes
+
+- ...
+
+## Testing
+
+- [ ] Tests pass (`pytest tests/ -v`)
+- [ ] Linting passes (`ruff check src/zettelforge/`)
+- [ ] New tests added for new functionality
+- [ ] No enterprise features added to community code without discussion

zettelforge-2.1.0/.github/workflows/ci.yml

@@ -0,0 +1,76 @@
+name: CI
+
+on:
+  push:
+    branches: [ main, develop ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ['3.10', '3.11', '3.12']
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -e ".[dev]"
+
+    - name: Lint with ruff
+      run: |
+        ruff check src/zettelforge/
+
+    - name: Format check with black
+      run: |
+        black --check src/zettelforge/
+
+    - name: Type check with mypy
+      run: |
+        mypy src/zettelforge/ --ignore-missing-imports
+
+    - name: Test with pytest
+      env:
+        ZETTELFORGE_BACKEND: jsonl
+      run: |
+        pytest tests/ -v --ignore=tests/test_typedb_client.py --cov=zettelforge --cov-report=xml
+
+    - name: Upload coverage
+      uses: codecov/codecov-action@v3
+      with:
+        file: ./coverage.xml
+        fail_ci_if_error: false
+
+  build:
+    runs-on: ubuntu-latest
+    needs: test
+    
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.12'
+
+    - name: Install build dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install build twine
+
+    - name: Build package
+      run: |
+        python -m build
+
+    - name: Check package
+      run: |
+        twine check dist/*

zettelforge-2.1.0/.github/workflows/publish.yml

@@ -0,0 +1,28 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write  # trusted publishing
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.12'
+
+    - name: Install build dependencies
+      run: pip install build
+
+    - name: Build package
+      run: python -m build
+
+    - name: Publish to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1

zettelforge-2.1.0/.gitignore

@@ -0,0 +1,67 @@
+# ZettelForge config (may contain credentials)
+config.yaml
+config.yml
+.env
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual environments
+venv/
+ENV/
+env/
+.venv
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+.tox/
+
+# Runtime data (not source code)
+/notes.jsonl
+/kg_nodes.jsonl
+/kg_edges.jsonl
+/entity_index.json
+*.lance/
+vectordb/
+.vector_memory.lance/
+.snapshots/
+archive/
+
+# Data directories (runtime, not source)
+/data/
+/.amem/
+
+# Logs
+*.log
+
+# OS
+.DS_Store
+Thumbs.db

zettelforge-2.1.0/CHANGELOG.md

@@ -0,0 +1,85 @@
+# Changelog
+
+All notable changes to ZettelForge are documented here.
+Format based on [Keep a Changelog](https://keepachangelog.com/).
+
+## [2.1.0] - 2026-04-12
+
+### Added
+- Open-core edition system (Community MIT + Enterprise BSL-1.1)
+- Edition detection via `THREATENGRAM_LICENSE_KEY` or enterprise package
+- `/api/edition` endpoint showing feature availability
+- `EditionError` for clear upgrade messaging
+- `LICENSE-ENTERPRISE` (BSL-1.1) for enterprise features
+- GitHub issue/PR templates, SECURITY.md, CODE_OF_CONDUCT.md
+
+### Changed
+- Dependencies split: core (MIT) vs `pip install zettelforge[enterprise]`
+- Enterprise-only features: TypeDB STIX ontology, temporal KG queries,
+  advanced synthesis formats, report ingestion, multi-hop traversal,
+  OpenCTI integration, Sigma generation, multi-tenant auth
+- Community gets full memory pipeline: blended retrieval, two-phase
+  extraction, intent routing, causal triples, cross-encoder reranking
+
+## [2.0.0] - 2026-04-09
+
+### Added
+- Hybrid TypeDB + LanceDB architecture
+- STIX 2.1 schema with 9 entity types, 8 relationship types
+- TypeDB alias inference (36 CTI aliases: APT28/Fancy Bear/Strontium, etc.)
+- Report ingestion with auto-chunking (`remember_report()`)
+- In-process embeddings via fastembed (no Ollama needed for embeddings)
+- Local LLM via llama-cpp-python (Qwen 2.5 3B)
+- Conversational entity extraction (person, location, org, event, activity, temporal)
+- Multi-tenant OAuth/JWT authentication
+- ThreatRecall web UI (FastAPI)
+- MCP server for Claude Code integration
+- OpenCTI continuous sync
+- MemoryAgentBench (ICLR 2026) benchmark
+- Configuration system with layered resolution (env > yaml > defaults)
+- Documentation suite (Diataxis framework)
+
+### Changed
+- Knowledge graph backend: TypeDB-first with JSONL fallback
+- Embedding provider: fastembed (in-process ONNX) replaces Ollama HTTP
+- Bumped all benchmarks: CTI 75%, LOCOMO 18%, RAGAS 78.1%
+
+## [1.5.0] - 2026-04-08
+
+### Added
+- GraphRetriever for multi-hop note discovery via knowledge graph
+- BlendedRetriever combining vector + graph results with policy weights
+- Rewritten `recall()` with blended vector + graph retrieval
+- CTIBench (NeurIPS 2024) benchmark adapter
+- RAGAS retrieval quality benchmark
+
+## [1.4.0] - 2026-04-07
+
+### Added
+- FactExtractor (Phase 1): LLM-based salient fact extraction with importance scoring
+- MemoryUpdater (Phase 2): ADD/UPDATE/DELETE/NOOP decision engine
+- `remember_with_extraction()` two-phase pipeline
+
+## [1.3.0] - 2026-04-07
+
+### Added
+- Sigma rule generation from IOCs
+- Microsoft Sentinel rule conversion
+
+## [1.2.0] - 2026-04-07
+
+### Added
+- CTI platform integration (Django CTI database connector)
+- Proactive context injection for agent workflows
+
+## [1.0.0] - 2026-04-06
+
+### Added
+- Core memory pipeline (remember, recall, vector search)
+- Knowledge graph with JSONL persistence
+- Entity extraction (CVE, actor, tool, campaign)
+- Causal triple extraction (LLM-powered)
+- Temporal graph indexing
+- RAG synthesis (direct_answer format)
+- Intent classification and query routing
+- Cross-encoder reranking

zettelforge-2.1.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,40 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior:
+
+* The use of sexualized language or imagery, and sexual attention or advances
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information without explicit permission
+* Other conduct which could reasonably be considered inappropriate
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the project team at **community@threatengram.com**. All complaints
+will be reviewed and investigated promptly and fairly.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/),
+version 2.1, available at
+https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.

zettelforge-2.1.0/CONTRIBUTING.md

@@ -0,0 +1,70 @@
+# Contributing to ZettelForge
+
+Thank you for your interest in contributing to ZettelForge! This document provides guidelines for contributing to the project.
+
+## Development Setup
+
+1. Fork and clone the repository
+2. Create a virtual environment: `python -m venv venv`
+3. Activate it: `source venv/bin/activate` (or `venv\Scripts\activate` on Windows)
+4. Install in development mode: `pip install -e ".[dev]"`
+5. Run tests to verify your setup: `ZETTELFORGE_BACKEND=jsonl pytest tests/ -v --ignore=tests/test_typedb_client.py`
+
+No external services (Ollama, TypeDB) are required for development. Embeddings run in-process via fastembed, and the knowledge graph defaults to JSONL.
+
+## Development Workflow
+
+1. Create a feature branch: `git checkout -b feature/your-feature-name`
+2. Make your changes
+3. Run tests: `pytest tests/ -v`
+4. Run linting: `ruff check src/zettelforge/`
+5. Run formatting: `black src/zettelforge/`
+6. Commit with clear messages
+7. Push and create a pull request
+
+## Community vs Enterprise
+
+ZettelForge uses an open-core model:
+
+- **Community (MIT)** -- everything in `src/zettelforge/` except `enterprise/`. Contributions here are welcome and encouraged.
+- **Enterprise (BSL-1.1)** -- code in `src/zettelforge/enterprise/` and features gated behind `is_enterprise()`. These are maintained by Threatengram.
+
+If you're unsure whether a feature belongs in Community or Enterprise, open an issue to discuss before starting work.
+
+## Code Style
+
+- Follow PEP 8
+- Use type hints where possible
+- Document functions with docstrings
+- Keep functions focused and small
+- Write tests for new functionality
+
+## Testing
+
+- Write tests for new features
+- Ensure all tests pass before submitting PR
+- Tests that require Enterprise features should use the `enable_enterprise` fixture from `tests/conftest.py`
+- Use meaningful test names that describe behavior
+
+## Commit Messages
+
+Use clear, descriptive commit messages:
+
+- `feat: Add entity extraction for CVE patterns`
+- `fix: Correct vector similarity calculation`
+- `docs: Update API reference`
+- `test: Add tests for recall_cve method`
+
+## Pull Request Process
+
+1. Update documentation if needed
+2. Add tests for new functionality
+3. Ensure CI passes
+4. Request review from maintainers
+5. Address review feedback
+
+## Questions?
+
+- Open an issue for bugs or feature requests
+- Start a discussion for questions or ideas
+- Check existing issues before creating new ones

zettelforge-2.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Patrick Roland
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

zettelforge-2.1.0/LICENSE-ENTERPRISE

@@ -0,0 +1,20 @@
+Business Source License 1.1
+
+Licensor: Threatengram, Inc.
+Licensed Work: ThreatRecall Enterprise (zettelforge-enterprise package)
+Change Date: Four years from each release date
+Change License: Apache License, Version 2.0
+
+The Licensed Work is provided under the terms of the Business Source
+License 1.1 (BSL-1.1). See https://mariadb.com/bsl11/ for the full
+license text.
+
+Usage Grant: You may use the Licensed Work for any purpose, including
+production use, provided that you do not offer the Licensed Work as a
+commercial hosted service to third parties without a separate commercial
+agreement with Threatengram, Inc.
+
+All files in this repository (rolandpg/zettelforge) are MIT licensed.
+The enterprise package (rolandpg/zettelforge-enterprise) is BSL-1.1.
+
+Contact: enterprise@threatengram.com

zettelforge-2.1.0/MANIFEST.in

@@ -0,0 +1,7 @@
+include README.md
+include LICENSE
+include LICENSE-ENTERPRISE
+include pyproject.toml
+recursive-include src/zettelforge *.py
+recursive-include tests *.py
+recursive-include docs *.md

zettelforge-2.1.0/PACKAGE_SUMMARY.md

@@ -0,0 +1,174 @@
+# ZettelForge Skill Package - Summary
+
+## What Was Created
+
+A standalone, production-ready Python package for ZettelForge (Agentic Memory) that can be:
+1. Installed as a Python package via pip
+2. Used as an OpenClaw skill
+3. Published to GitHub as an open-source project
+4. Eventually published to PyPI
+
+## Directory Structure
+
+```
+~/.openclaw/workspace/skills/amem/
+├── .github/workflows/ci.yml    # GitHub Actions CI/CD
+├── .git/                        # Git repository
+├── .gitignore                   # Git ignore rules
+├── CONTRIBUTING.md              # Contribution guidelines
+├── LICENSE                      # MIT License
+├── MANIFEST.in                  # Package manifest
+├── README.md                    # Project documentation
+├── SKILL.md                     # OpenClaw skill definition
+├── pyproject.toml               # Python package config
+├── src/amem/                    # Source code
+│   ├── __init__.py             # Package init
+│   ├── entity_indexer.py       # Entity extraction/indexing
+│   ├── memory_manager.py       # Main API
+│   ├── memory_store.py         # JSONL storage
+│   ├── note_constructor.py     # Note enrichment
+│   ├── note_schema.py          # Pydantic schemas
+│   ├── vector_memory.py        # LanceDB vector store
+│   └── vector_retriever.py     # Semantic search
+└── tests/
+    └── test_basic.py           # Unit tests
+```
+
+## Key Features Packaged
+
+| Feature | Status | File |
+|---------|--------|------|
+| Core MemoryNote schema | ✅ | note_schema.py |
+| JSONL storage | ✅ | memory_store.py |
+| Vector storage (LanceDB) | ✅ | vector_memory.py |
+| Semantic retrieval | ✅ | vector_retriever.py |
+| Entity extraction | ✅ | entity_indexer.py |
+| Entity-based lookup | ✅ | memory_manager.py |
+| Main API (remember/recall) | ✅ | memory_manager.py |
+
+## Installation
+
+### Local Development
+
+```bash
+cd ~/.openclaw/workspace/skills/amem
+pip install -e ".[dev]"
+```
+
+### Usage
+
+```python
+from amem import MemoryManager
+
+mm = MemoryManager()
+
+# Store memory
+note, status = mm.remember("CVE-2024-3094 is a backdoor", domain="security_ops")
+
+# Retrieve
+results = mm.recall("XZ backdoor", k=5)
+
+# Entity lookup
+cve_notes = mm.recall_cve("CVE-2024-3094")
+```
+
+## Next Steps to Publish on GitHub
+
+1. **Create GitHub Repository:**
+   ```bash
+   # Option 1: Using GitHub CLI
+   gh repo create rolandpg/amem --public --source=. --remote=origin --push
+   
+   # Option 2: Manual
+   # - Go to https://github.com/new
+   # - Create repo named "amem"
+   # - Follow push instructions
+   ```
+
+2. **Push to GitHub:**
+   ```bash
+   cd ~/.openclaw/workspace/skills/amem
+   git branch -m main
+   git remote add origin https://github.com/rolandpg/amem.git
+   git push -u origin main
+   ```
+
+3. **Set Up CI/CD:**
+   - GitHub Actions workflow already configured
+   - Will run tests on Python 3.10, 3.11, 3.12
+   - Runs linting, type checking, and tests
+
+4. **Future: Publish to PyPI:**
+   ```bash
+   # Build package
+   python -m build
+   
+   # Upload to PyPI (requires account)
+   python -m twine upload dist/*
+   ```
+
+## Differences from Original ZettelForge
+
+| Aspect | Original | Packaged |
+|--------|----------|----------|
+| Location | `~/.openclaw/workspace/memory/` | `~/.openclaw/workspace/skills/amem/` |
+| Hardcoded paths | Yes | Environment configurable |
+| Dependencies | Inline imports | Proper package deps in pyproject.toml |
+| Tests | Phase-based (143 tests) | Simplified basic tests |
+| Synthesis Layer | Phase 7 (complete) | Not included (can add later) |
+| Knowledge Graph | Phase 6 (complete) | Not included (can add later) |
+| Evolution | Multi-phase | Simplified |
+
+## What's NOT Included (Yet)
+
+These advanced features from the original ZettelForge can be added in future versions:
+
+1. **Synthesis Layer (Phase 7)** - RAG-as-answer with LLM generation
+2. **Knowledge Graph (Phase 6)** - Full graph with IEP 2.0
+3. **Note Evolution** - Multi-phase note refinement
+4. **Link Generator** - Automatic note linking
+5. **Alias Resolution** - Entity name normalization
+6. **Cold Archive** - Automatic low-confidence archival
+7. **Burn-in Tests** - Comprehensive stress testing
+
+## Configuration
+
+Environment variables for customization:
+
+```bash
+export AMEM_DATA_DIR=/path/to/data        # Default: ~/.amem
+export AMEM_OLLAMA_URL=http://localhost:11434
+export AMEM_EMBEDDING_MODEL=nomic-embed-text
+```
+
+## Version
+
+Current: **1.0.0-alpha.1**
+
+Following semantic versioning:
+- Alpha: Early testing, API may change
+- Beta: Feature complete, testing bugs
+- 1.0.0: Production ready
+
+## Git Commit
+
+Initial commit hash: `9da469e`
+
+```
+Initial commit: ZettelForge Agentic Memory System v1.0.0-alpha.1
+17 files changed, 2001 insertions(+)
+```
+
+## Ready for Development
+
+The package is ready for:
+- ✅ Local installation and testing
+- ✅ GitHub repository creation
+- ✅ CI/CD pipeline
+- ✅ OpenClaw skill usage
+- 🔄 Future PyPI publication
+
+---
+
+Created: 2026-04-05
+Location: ~/.openclaw/workspace/skills/amem/