agent-os-kernel 1.1.0__py3-none-any.whl → 1.3.0__py3-none-any.whl

modules/emk/CHANGELOG.md

@@ -0,0 +1,41 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+- Hugging Face Hub integration (`emk.hf_utils`) for dataset sharing
+- Reproducible experiment runner (`experiments/reproduce_results.py`)
+- Research paper structure (`paper/whitepaper.md`, `paper/structure.tex`)
+- GitHub Actions CI/CD workflows for testing and PyPI publishing
+- CONTRIBUTING.md with development guidelines
+- PEP 561 `py.typed` marker for typed package support
+- `get_version_info()` function for runtime feature detection
+
+### Changed
+- Enhanced `__init__.py` with better exports and metadata
+- Updated `pyproject.toml` with additional classifiers and keywords
+- Added Python 3.12 support
+
+## [0.1.0] - 2026-01-23
+
+### Added
+- Initial release of EMK (Episodic Memory Kernel)
+- `Episode` schema with immutable, content-addressed storage
+- `VectorStoreAdapter` abstract interface
+- `FileAdapter` for JSONL-based local storage
+- `ChromaDBAdapter` for vector similarity search (optional)
+- `Indexer` utilities for tag generation and search text creation
+- Comprehensive test suite with 71% coverage
+- Basic usage examples
+
+### Security
+- Fixed tempfile.mktemp() usage (CodeQL)
+- Fixed potential metadata mutation issues
+
+[Unreleased]: https://github.com/imran-siddique/emk/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/imran-siddique/emk/releases/tag/v0.1.0

modules/emk/CONTRIBUTING.md

@@ -0,0 +1,295 @@
+# Contributing to EMK
+
+Thank you for your interest in contributing to EMK (Episodic Memory Kernel)! This document provides guidelines and instructions for contributing.
+
+## 📋 Table of Contents
+
+- [Code of Conduct](#code-of-conduct)
+- [Getting Started](#getting-started)
+- [Development Setup](#development-setup)
+- [Making Changes](#making-changes)
+- [Pull Request Process](#pull-request-process)
+- [Coding Standards](#coding-standards)
+- [Testing](#testing)
+- [Documentation](#documentation)
+
+## Code of Conduct
+
+By participating in this project, you agree to maintain a respectful and inclusive environment. Be kind, be constructive, and be patient with others.
+
+## Getting Started
+
+### Prerequisites
+
+- Python 3.8 or higher
+- Git
+- A GitHub account
+
+### Fork and Clone
+
+1. Fork the repository on GitHub
+2. Clone your fork locally:
+   ```bash
+   git clone https://github.com/YOUR-USERNAME/emk.git
+   cd emk
+   ```
+3. Add the upstream remote:
+   ```bash
+   git remote add upstream https://github.com/imran-siddique/emk.git
+   ```
+
+## Development Setup
+
+### Create a Virtual Environment
+
+```bash
+# Create virtual environment
+python -m venv venv
+
+# Activate it
+# On Windows:
+venv\Scripts\activate
+# On macOS/Linux:
+source venv/bin/activate
+```
+
+### Install Development Dependencies
+
+```bash
+# Install package in editable mode with dev dependencies
+pip install -e ".[dev]"
+
+# Or install all optional dependencies
+pip install -e ".[all,dev]"
+```
+
+### Verify Installation
+
+```bash
+# Run tests
+pytest tests/ -v
+
+# Check code style
+black --check .
+ruff check .
+
+# Run type checking
+mypy emk/
+```
+
+## Making Changes
+
+### Branch Naming
+
+Create a descriptive branch name:
+
+```bash
+git checkout -b feature/add-redis-adapter
+git checkout -b fix/episode-id-collision
+git checkout -b docs/improve-readme
+```
+
+### Commit Messages
+
+Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification:
+
+```
+feat: add Redis storage adapter
+fix: resolve episode ID collision on rapid creation
+docs: add ChromaDB usage examples
+test: add integration tests for FileAdapter
+refactor: simplify Indexer tag extraction
+chore: update dependencies
+```
+
+### Keep Commits Atomic
+
+Each commit should represent a single logical change. If you're fixing multiple issues, make multiple commits.
+
+## Pull Request Process
+
+1. **Update your fork** with the latest upstream changes:
+   ```bash
+   git fetch upstream
+   git rebase upstream/main
+   ```
+
+2. **Push your branch** to your fork:
+   ```bash
+   git push origin feature/your-feature
+   ```
+
+3. **Open a Pull Request** on GitHub with:
+   - Clear title describing the change
+   - Description of what and why
+   - Reference to any related issues
+   - Screenshots/examples if applicable
+
+4. **Address review feedback** promptly and push updates
+
+5. **Squash commits** if requested before merge
+
+### PR Checklist
+
+- [ ] Tests pass locally (`pytest tests/ -v`)
+- [ ] Code is formatted (`black .`)
+- [ ] Linting passes (`ruff check .`)
+- [ ] Type hints added for new code
+- [ ] Docstrings added/updated
+- [ ] Documentation updated if needed
+- [ ] CHANGELOG updated for significant changes
+
+## Coding Standards
+
+### Style Guide
+
+We follow [PEP 8](https://pep8.org/) with these tools:
+
+- **Black** for code formatting (line length: 100)
+- **Ruff** for linting
+- **mypy** for type checking
+
+### Type Hints
+
+All public functions must have type hints:
+
+```python
+from typing import List, Optional, Dict, Any
+
+def store(self, episode: Episode, embedding: Optional[np.ndarray] = None) -> str:
+    """Store an episode and return its ID."""
+    ...
+```
+
+### Docstrings
+
+Use Google-style docstrings:
+
+```python
+def generate_episode_tags(episode: Episode) -> List[str]:
+    """
+    Generate searchable tags from an episode.
+    
+    Args:
+        episode: The episode to generate tags from.
+        
+    Returns:
+        List of tags for indexing.
+        
+    Raises:
+        ValueError: If episode content is empty.
+        
+    Example:
+        >>> episode = Episode(goal="Test", action="Run", result="Pass", reflection="Good")
+        >>> tags = generate_episode_tags(episode)
+        >>> print(tags)
+        ['test', 'run', 'pass', 'good']
+    """
+    ...
+```
+
+### Import Order
+
+1. Standard library imports
+2. Third-party imports
+3. Local imports
+
+```python
+from datetime import datetime
+from typing import List, Optional
+
+import numpy as np
+from pydantic import BaseModel
+
+from emk.schema import Episode
+```
+
+## Testing
+
+### Running Tests
+
+```bash
+# Run all tests
+pytest tests/ -v
+
+# Run with coverage
+pytest tests/ -v --cov=emk --cov-report=html
+
+# Run specific test file
+pytest tests/test_schema.py -v
+
+# Run specific test
+pytest tests/test_schema.py::test_episode_immutability -v
+```
+
+### Writing Tests
+
+- Place tests in the `tests/` directory
+- Name test files `test_*.py`
+- Name test functions `test_*`
+- Use descriptive test names that explain what's being tested
+
+```python
+def test_episode_id_is_deterministic_for_same_content():
+    """Episodes with identical content should have identical IDs."""
+    episode1 = Episode(
+        goal="Test",
+        action="Run",
+        result="Pass",
+        reflection="Good",
+        timestamp=datetime(2026, 1, 1, tzinfo=timezone.utc)
+    )
+    episode2 = Episode(
+        goal="Test",
+        action="Run", 
+        result="Pass",
+        reflection="Good",
+        timestamp=datetime(2026, 1, 1, tzinfo=timezone.utc)
+    )
+    assert episode1.episode_id == episode2.episode_id
+```
+
+### Test Coverage
+
+Aim for >80% code coverage. Critical paths (schema, storage) should have >95%.
+
+## Documentation
+
+### README Updates
+
+Update `README.md` if you:
+- Add new features
+- Change public API
+- Add new dependencies
+- Change installation instructions
+
+### Docstring Updates
+
+All public classes, methods, and functions need docstrings.
+
+### Example Updates
+
+Add examples to `examples/` for significant new features.
+
+## Design Principles
+
+When contributing, keep these principles in mind:
+
+1. **Immutability**: Episodes are append-only and cannot be modified
+2. **Minimal Dependencies**: Core functionality should require minimal packages
+3. **No Smart Logic**: EMK stores and retrieves; it doesn't summarize or interpret
+4. **Type Safety**: Use type hints everywhere
+5. **Backward Compatibility**: Don't break existing APIs without deprecation
+
+## Questions?
+
+- Open a [GitHub Issue](https://github.com/imran-siddique/emk/issues) for bugs or feature requests
+- Start a [Discussion](https://github.com/imran-siddique/emk/discussions) for questions
+
+## License
+
+By contributing to EMK, you agree that your contributions will be licensed under the MIT License.
+
+---
+
+Thank you for contributing to EMK! 🎉

modules/emk/IMPLEMENTATION.md

@@ -0,0 +1,174 @@
+# emk Implementation Summary
+
+## Overview
+This repository implements **emk** (Episodic Memory Kernel), a Layer 1 primitive for immutable, append-only storage of agent experiences.
+
+## Implementation Status: ✅ Complete
+
+### Core Components Implemented
+
+#### 1. Episode Schema (`emk/schema.py`)
+- **Purpose**: Immutable data structure for Goal → Action → Result → Reflection pattern
+- **Features**:
+  - Truly immutable (frozen=True)
+  - Auto-generated SHA-256 based episode IDs
+  - Timezone-aware timestamps
+  - JSON serialization/deserialization
+  - Metadata support for flexible tagging
+- **Implementation**: Pydantic BaseModel with model_validator for ID generation
+
+#### 2. VectorStoreAdapter (`emk/store.py`)
+- **Purpose**: Abstract interface for storage implementations
+- **Methods**:
+  - `store(episode, embedding)`: Store an episode
+  - `retrieve(query_embedding, filters, limit)`: Retrieve episodes
+  - `get_by_id(episode_id)`: Get specific episode
+
+#### 3. FileAdapter (`emk/store.py`)
+- **Purpose**: Simple JSONL-based local storage
+- **Features**:
+  - Append-only file storage
+  - Metadata-based filtering
+  - No external dependencies
+  - Suitable for logging and simple use cases
+
+#### 4. ChromaDBAdapter (`emk/store.py`)
+- **Purpose**: Vector database storage with semantic search
+- **Features**:
+  - Embedding-based similarity search
+  - Persistent storage
+  - Metadata filtering
+  - Optional dependency (chromadb extra)
+
+#### 5. Indexer (`emk/indexer.py`)
+- **Purpose**: Utilities for tagging and indexing episodes
+- **Features**:
+  - Tag extraction from text
+  - Episode tag generation
+  - Search text creation for embeddings
+  - Metadata enrichment
+
+### Design Principles Followed
+
+✅ **Immutability**: Episodes are frozen and cannot be modified after creation
+✅ **Append-Only**: No updates or deletes, only additions
+✅ **Minimal Dependencies**: Only numpy, pydantic, and optional chromadb
+✅ **No "Smart" Logic**: Pure storage and retrieval, no summarization
+✅ **Independence**: No dependencies on caas or agent-control-plane
+
+### Dependency Compliance
+
+**Required**:
+- pydantic >= 2.0.0 ✅
+- numpy >= 1.20.0 ✅
+
+**Optional**:
+- chromadb >= 0.4.0 (for vector search) ✅
+
+**Development**:
+- pytest >= 7.0.0 ✅
+- pytest-cov >= 4.0.0 ✅
+- black >= 23.0.0 ✅
+- ruff >= 0.1.0 ✅
+
+**Forbidden** (verified not present):
+- caas ✅
+- agent-control-plane ✅
+
+### Testing
+
+**Coverage**: 71% (32/33 tests passing, 1 skipped)
+- Schema: 100% coverage, 8 tests
+- FileAdapter: 100% coverage, 9 tests
+- Indexer: 100% coverage, 10 tests
+- Integration: 100% coverage, 5 tests
+
+**Skipped Tests**:
+- ChromaDBAdapter tests (requires optional chromadb installation)
+
+### Security
+
+**CodeQL Scan**: ✅ 0 vulnerabilities found
+
+**Security Fixes Applied**:
+- Fixed tempfile.mktemp() usage → NamedTemporaryFile
+- Fixed metadata mutation issues → use .get() instead of .pop()
+- Fixed potential IndexError → added empty list checks
+
+### Code Quality
+
+**Issues Fixed**:
+- Made Episode truly immutable with frozen=True
+- Fixed ChromaDB metadata handling for consistent access
+- Prevented metadata mutation in ChromaDB results processing
+- Added proper empty list validation
+
+### Package Build
+
+**Status**: ✅ Successfully builds
+- Source distribution: emk-0.1.0.tar.gz
+- Wheel distribution: emk-0.1.0-py3-none-any.whl
+- Installation verified: pip install works correctly
+
+### Documentation
+
+1. **README.md**: Comprehensive documentation with examples
+2. **Docstrings**: All public APIs documented
+3. **Examples**: `examples/basic_usage.py` demonstrates all features
+4. **License**: MIT License included
+
+### Next Steps for Users
+
+1. Install: `pip install emk` (when published to PyPI)
+2. Optional: `pip install emk[chromadb]` for vector search
+3. See README.md for usage examples
+4. See examples/basic_usage.py for complete demo
+
+### Publishing Checklist
+
+- [x] Package builds successfully
+- [x] All tests pass
+- [x] Security scan clean
+- [x] Documentation complete
+- [x] Examples provided
+- [x] License included
+- [ ] Publish to PyPI (user action required)
+
+## Architecture Diagram
+
+```
+emk/
+├── schema.py          # Episode data structure
+├── store.py           # Storage adapters
+│   ├── VectorStoreAdapter (abstract)
+│   ├── FileAdapter (JSONL)
+│   └── ChromaDBAdapter (optional)
+└── indexer.py         # Tagging and indexing utilities
+```
+
+## Usage Pattern
+
+```python
+from emk import Episode, FileAdapter, Indexer
+
+# Store episodes
+store = FileAdapter("memories.jsonl")
+episode = Episode(
+    goal="...",
+    action="...",
+    result="...",
+    reflection="..."
+)
+store.store(episode)
+
+# Retrieve and filter
+episodes = store.retrieve(filters={"user_id": "123"})
+
+# Index for search
+tags = Indexer.generate_episode_tags(episode)
+search_text = Indexer.create_search_text(episode)
+```
+
+## Conclusion
+
+The emk package is fully implemented, tested, and ready for use. It provides a solid foundation for episodic memory storage in agent systems while maintaining strict adherence to the design principles outlined in the PRD.

modules/emk/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Imran Siddique
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

modules/emk/MANIFEST.in

@@ -0,0 +1,8 @@
+include README.md
+include LICENSE
+include pyproject.toml
+recursive-include emk *.py
+recursive-exclude tests *
+recursive-exclude examples *
+global-exclude __pycache__
+global-exclude *.py[cod]

modules/emk/README.md

@@ -0,0 +1,135 @@
+# emk - Episodic Memory Kernel
+
+> **Part of [Agent OS](https://github.com/imran-siddique/agent-os)** - Kernel-level governance for AI agents
+
+[![PyPI](https://img.shields.io/pypi/v/emk)](https://pypi.org/project/emk/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Build Status](https://img.shields.io/github/actions/workflow/status/imran-siddique/emk/ci.yml?branch=main)](https://github.com/imran-siddique/emk/actions)
+
+**An immutable, append-only ledger for agent experiences. Part of the Agent OS ecosystem.**
+
+---
+
+## Why emk?
+
+Agents that cannot remember their past repeat the same mistakes. Traditional databases let you update or delete records—this is dangerous for agent memory. What if a bug overwrites critical learning? What if you need to audit agent decisions?
+
+We built `emk` to **subtract the problem of mutable memory.** Instead of complex database transactions and state management, we provide a single abstraction: the immutable Episode. Once written, never changed. This constraint eliminates entire classes of bugs and enables reliable agent learning systems.
+
+**Scale by Subtraction:** By removing the ability to modify history, we remove the need for complex versioning, rollback logic, and concurrency controls. The result is a memory system that scales reliably.
+
+---
+
+## Installation
+
+```bash
+pip install emk
+```
+
+---
+
+## Quick Start
+
+```python
+from emk import Episode, FileAdapter
+
+store = FileAdapter("agent_memory.jsonl")
+episode = Episode(goal="Query user data", action="SELECT * FROM users", 
+                  result="200 rows", reflection="Query was fast")
+store.store(episode)
+```
+
+That's it. Five lines. Your agent's experience is now permanently recorded.
+
+---
+
+## Memory Management Features 🆕
+
+`emk` now includes powerful memory management features to address the challenge: **Infinite memory leads to infinite retrieval costs and confusing context.**
+
+### Memory Decay & Compression (Sleep Cycle)
+
+Automatically summarize old episodes into semantic rules:
+
+```python
+from emk import MemoryCompressor
+
+compressor = MemoryCompressor(store, age_threshold_days=30)
+result = compressor.compress_old_episodes()
+# Compress 1000 episodes → 20 semantic rules
+```
+
+**Benefits:** 50-100x storage reduction, faster retrieval, preserved knowledge without overwhelming detail.
+
+### Negative Memory (Anti-Patterns)
+
+Explicitly track failures to avoid repeating mistakes:
+
+```python
+# Mark failures
+failed = episode.mark_as_failure(reason="Connection timeout")
+store.store(failed)
+
+# Query both successes AND failures
+patterns = store.retrieve_with_anti_patterns()
+print(f"What works: {len(patterns['successes'])}")
+print(f"What to avoid: {len(patterns['failures'])}")
+```
+
+**Benefits:** Prune search space instantly, learn from both successes and failures, "DO NOT TOUCH" vectors.
+
+📖 **Full documentation:** See [docs/MEMORY_MANAGEMENT.md](docs/MEMORY_MANAGEMENT.md) and [examples/memory_features_demo.py](examples/memory_features_demo.py)
+
+---
+
+## Architecture
+
+`emk` sits at **Layer 1 (Primitives)** of the Agent OS stack. It provides the foundational storage abstraction that higher layers depend on:
+
+- **Layer 1 (Primitives):** `emk` stores raw episodes. `caas` (Context-as-a-Service) reads from `emk` to build working memory. `cmvk` (Cryptographic Message Verification Kernel) may sign episodes for audit trails.
+- **Layer 2 (Infrastructure):** `amb` (Agent Message Bus) transports messages. `iatp` (Inter-Agent Trust Protocol) verifies trust. These layers may log their own episodes into `emk`.
+- **Layer 3 (Framework):** `agent-control-plane` orchestrates agents. `scak` (Self-Correction and Alignment Kernel) uses `emk` to learn from mistakes.
+
+`emk` has **zero** dependencies on other Agent OS components. It is the foundation, not the application.
+
+---
+
+## The Ecosystem Map
+
+`emk` is one component of the modular Agent OS. Here's how the pieces fit together:
+
+| Layer              | Component                  | Purpose                                   | Repository                                      |
+|--------------------|----------------------------|-------------------------------------------|-------------------------------------------------|
+| **Primitives**     | `caas`                     | Context-as-a-Service (Working Memory)     | [caas](https://github.com/imran-siddique/caas)  |
+| **Primitives**     | `cmvk`                     | Cryptographic Verification Kernel         | [cmvk](https://github.com/imran-siddique/cmvk)  |
+| **Primitives**     | `emk`                      | Episodic Memory Kernel                    | [emk](https://github.com/imran-siddique/emk)    |
+| **Infrastructure** | `iatp`                     | Inter-Agent Trust Protocol                | [iatp](https://github.com/imran-siddique/iatp)  |
+| **Infrastructure** | `amb`                      | Agent Message Bus                         | [amb](https://github.com/imran-siddique/amb)    |
+| **Infrastructure** | `atr`                      | Agent Tool Registry                       | [atr](https://github.com/imran-siddique/atr)    |
+| **Framework**      | `agent-control-plane`      | Agent Orchestration and Control           | [agent-control-plane](https://github.com/imran-siddique/agent-control-plane) |
+| **Framework**      | `scak`                     | Self-Correction and Alignment Kernel      | [scak](https://github.com/imran-siddique/scak)  |
+
+Each component is independently useful. Together, they form a complete Agent Operating System.
+
+---
+
+## Citation
+
+If you use `emk` in your research or production systems, please cite:
+
+```bibtex
+@software{emk2024,
+  author       = {Siddique, Imran},
+  title        = {emk: Episodic Memory Kernel for Agent Systems},
+  year         = {2024},
+  publisher    = {GitHub},
+  url          = {https://github.com/imran-siddique/emk},
+  note         = {Part of the Agent OS ecosystem}
+}
+```
+
+---
+
+## License
+
+MIT License - see [LICENSE](LICENSE) for details.