zer0lint 0.1.0__tar.gz

zer0lint-0.1.0/PKG-INFO

@@ -0,0 +1,185 @@
+Metadata-Version: 2.4
+Name: zer0lint
+Version: 0.1.0
+Summary: mem0 extraction optimizer: diagnose failures and generate custom prompts for your system
+Requires-Python: >=3.9
+Requires-Dist: httpx>=0.24.0
+Requires-Dist: mem0ai>=0.1.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: typer[all]>=0.9.0
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.1.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# zer0lint
+
+**AI memory extraction diagnostics.** Find out why your AI agent forgets what matters — and fix it.
+
+[![PyPI](https://img.shields.io/badge/coming%20soon-PyPI-blue)](https://pypi.org/project/zer0lint/)
+[![License](https://img.shields.io/badge/license-Apache%202.0-green)](LICENSE)
+[![Made by Hermes Labs](https://img.shields.io/badge/made%20by-Hermes%20Labs-purple)](https://hermes-labs.ai)
+
+---
+
+## The Problem
+
+You set up mem0 (or Zep, or LangMem) and your AI agent still seems forgetful. Technical decisions, version numbers, experiment results — it stores them, but can't recall them.
+
+**Why?** mem0's default extraction prompt is designed for personal assistants: favorite coffee shops, dietary preferences, weekend plans. When your agent stores "Redis upgraded to v7.2.4" or "security audit scored 7.2/10", it silently drops the specifics.
+
+**Proven gap (our testing):**
+- Default mem0 prompt → **70% recall** on technical facts
+- Wrong domain prompt (personal) → **40% recall**
+- zer0lint technical prompt → **90% recall**
+
+That's a **+20pp improvement** from getting the extraction prompt right.
+
+---
+
+## Quick Start
+
+```bash
+pip install zer0lint  # coming soon
+
+# Check your current extraction health
+zer0lint check --config ~/.mem0/config.json
+
+# Diagnose and fix
+zer0lint generate --config ~/.mem0/config.json
+
+# Dry run (see what would change without applying)
+zer0lint generate --config ~/.mem0/config.json --dry-run
+```
+
+---
+
+## What It Does
+
+### `zer0lint check`
+Tests your current mem0 config against domain-relevant synthetic facts. Returns a score and status.
+
+```
+zer0lint v0.2.0 — extraction health check
+Config : ~/.mem0/config.json
+Model  : qwen3.5:4b
+Prompt : default (mem0 built-in)
+
+Score  : 4/5 (80%) — HEALTHY
+
+  ✅ Version update
+  ✅ CI status
+  ✅ Model upgrade
+  ✅ Configuration
+  ⚠  API endpoint
+```
+
+Statuses: **HEALTHY** (≥80%) · **ACCEPTABLE** (60–79%) · **DEGRADED** (40–59%) · **CRITICAL** (<40%)
+
+### `zer0lint generate`
+3-phase diagnostic + fix:
+
+1. **Baseline** — test your config as-is
+2. **Re-test** — inject zer0lint's domain-aware extraction prompt at config level
+3. **Apply** — if improved, write the validated prompt to your config
+
+```
+zer0lint v0.2.0 — extraction optimizer
+
+[1/3] Baseline — testing current config as-is...
+  Baseline score: 4/5 (80%)
+
+[2/3] Re-testing with zer0lint technical extraction prompt...
+  Improved score: 5/5 (100%)
+  Improvement: +20pp
+
+Results:
+  Before : 4/5 (80%)
+  After  : 5/5 (100%)
+  Δ      : +20pp
+
+✅ Fix applied to config.
+   Backup: ~/.mem0/config.backup.2026-03-23T14:20:00
+```
+
+---
+
+## Why Config-Level Injection Matters
+
+**Important finding:** In mem0 v1.x, passing a prompt via `memory.add(..., prompt=X)` has **no measurable effect** on retrieval quality. The extraction prompt must live in the config (`custom_fact_extraction_prompt` field) to actually work.
+
+zer0lint writes the validated prompt directly to your config — this is the correct fix.
+
+---
+
+## Supported Systems
+
+| System | Status | Notes |
+|---|---|---|
+| mem0 (v1.x) | ✅ Supported | Full check + generate |
+| Zep / Graphiti | 🔜 Planned | v0.3 |
+| LangMem | 🔜 Planned | v0.4 |
+| Generic adapter | 🔜 Planned | BYOC callables |
+
+---
+
+## Test Results (2026-03-23)
+
+Horse race across models (5 technical + research facts):
+
+| Model | Default prompt | zer0lint prompt | Δ |
+|---|---|---|---|
+| qwen3.5:4b | 80% | **100%** | +20pp |
+| mistral:7b | 40% | 40% | 0pp |
+
+**mistral:7b** produces malformed JSON regardless of prompt — model quality is the bottleneck. zer0lint detects this and recommends switching models.
+
+Scale test (10 facts, 5 domains):
+
+| | Score | % |
+|---|---|---|
+| Default | 7/10 | 70% |
+| zer0lint | 9/10 | **90%** | 
+
+---
+
+## Installation
+
+```bash
+# From source (current)
+git clone https://github.com/roli-lpci/zer0lint
+cd zer0lint
+pip install -e .
+
+# PyPI (coming soon)
+pip install zer0lint
+```
+
+**Requirements:** Python 3.9+, mem0 v1.x, an Ollama or cloud LLM configured in your mem0 config.
+
+---
+
+## How It Works
+
+zer0lint reads your existing mem0 config, borrows whatever LLM you already have configured, and runs a controlled recall test. No new API keys, no new models, no cloud calls beyond what you already have.
+
+The extraction prompt it generates is **domain-aware** — tuned for technical/research work rather than personal assistant use cases. It's validated against synthetic facts before being applied, and your original config is always backed up.
+
+---
+
+## Built by Hermes Labs
+
+zer0lint is part of the [Hermes Labs](https://hermes-labs.ai) AI agent tooling suite:
+
+- **[lintlang](https://github.com/roli-lpci/lintlang)** — Static linter for AI agent tool descriptions and prompts
+- **[Little Canary](https://github.com/roli-lpci/little-canary)** — Prompt injection detection
+- **[Suy Sideguy](https://github.com/roli-lpci/suy-sideguy)** — Runtime policy enforcement for agents
+- **zer0lint** — Memory extraction diagnostics ← you are here
+
+---
+
+## License
+
+Apache 2.0

zer0lint-0.1.0/README.md

@@ -0,0 +1,169 @@
+# zer0lint
+
+**AI memory extraction diagnostics.** Find out why your AI agent forgets what matters — and fix it.
+
+[![PyPI](https://img.shields.io/badge/coming%20soon-PyPI-blue)](https://pypi.org/project/zer0lint/)
+[![License](https://img.shields.io/badge/license-Apache%202.0-green)](LICENSE)
+[![Made by Hermes Labs](https://img.shields.io/badge/made%20by-Hermes%20Labs-purple)](https://hermes-labs.ai)
+
+---
+
+## The Problem
+
+You set up mem0 (or Zep, or LangMem) and your AI agent still seems forgetful. Technical decisions, version numbers, experiment results — it stores them, but can't recall them.
+
+**Why?** mem0's default extraction prompt is designed for personal assistants: favorite coffee shops, dietary preferences, weekend plans. When your agent stores "Redis upgraded to v7.2.4" or "security audit scored 7.2/10", it silently drops the specifics.
+
+**Proven gap (our testing):**
+- Default mem0 prompt → **70% recall** on technical facts
+- Wrong domain prompt (personal) → **40% recall**
+- zer0lint technical prompt → **90% recall**
+
+That's a **+20pp improvement** from getting the extraction prompt right.
+
+---
+
+## Quick Start
+
+```bash
+pip install zer0lint  # coming soon
+
+# Check your current extraction health
+zer0lint check --config ~/.mem0/config.json
+
+# Diagnose and fix
+zer0lint generate --config ~/.mem0/config.json
+
+# Dry run (see what would change without applying)
+zer0lint generate --config ~/.mem0/config.json --dry-run
+```
+
+---
+
+## What It Does
+
+### `zer0lint check`
+Tests your current mem0 config against domain-relevant synthetic facts. Returns a score and status.
+
+```
+zer0lint v0.2.0 — extraction health check
+Config : ~/.mem0/config.json
+Model  : qwen3.5:4b
+Prompt : default (mem0 built-in)
+
+Score  : 4/5 (80%) — HEALTHY
+
+  ✅ Version update
+  ✅ CI status
+  ✅ Model upgrade
+  ✅ Configuration
+  ⚠  API endpoint
+```
+
+Statuses: **HEALTHY** (≥80%) · **ACCEPTABLE** (60–79%) · **DEGRADED** (40–59%) · **CRITICAL** (<40%)
+
+### `zer0lint generate`
+3-phase diagnostic + fix:
+
+1. **Baseline** — test your config as-is
+2. **Re-test** — inject zer0lint's domain-aware extraction prompt at config level
+3. **Apply** — if improved, write the validated prompt to your config
+
+```
+zer0lint v0.2.0 — extraction optimizer
+
+[1/3] Baseline — testing current config as-is...
+  Baseline score: 4/5 (80%)
+
+[2/3] Re-testing with zer0lint technical extraction prompt...
+  Improved score: 5/5 (100%)
+  Improvement: +20pp
+
+Results:
+  Before : 4/5 (80%)
+  After  : 5/5 (100%)
+  Δ      : +20pp
+
+✅ Fix applied to config.
+   Backup: ~/.mem0/config.backup.2026-03-23T14:20:00
+```
+
+---
+
+## Why Config-Level Injection Matters
+
+**Important finding:** In mem0 v1.x, passing a prompt via `memory.add(..., prompt=X)` has **no measurable effect** on retrieval quality. The extraction prompt must live in the config (`custom_fact_extraction_prompt` field) to actually work.
+
+zer0lint writes the validated prompt directly to your config — this is the correct fix.
+
+---
+
+## Supported Systems
+
+| System | Status | Notes |
+|---|---|---|
+| mem0 (v1.x) | ✅ Supported | Full check + generate |
+| Zep / Graphiti | 🔜 Planned | v0.3 |
+| LangMem | 🔜 Planned | v0.4 |
+| Generic adapter | 🔜 Planned | BYOC callables |
+
+---
+
+## Test Results (2026-03-23)
+
+Horse race across models (5 technical + research facts):
+
+| Model | Default prompt | zer0lint prompt | Δ |
+|---|---|---|---|
+| qwen3.5:4b | 80% | **100%** | +20pp |
+| mistral:7b | 40% | 40% | 0pp |
+
+**mistral:7b** produces malformed JSON regardless of prompt — model quality is the bottleneck. zer0lint detects this and recommends switching models.
+
+Scale test (10 facts, 5 domains):
+
+| | Score | % |
+|---|---|---|
+| Default | 7/10 | 70% |
+| zer0lint | 9/10 | **90%** | 
+
+---
+
+## Installation
+
+```bash
+# From source (current)
+git clone https://github.com/roli-lpci/zer0lint
+cd zer0lint
+pip install -e .
+
+# PyPI (coming soon)
+pip install zer0lint
+```
+
+**Requirements:** Python 3.9+, mem0 v1.x, an Ollama or cloud LLM configured in your mem0 config.
+
+---
+
+## How It Works
+
+zer0lint reads your existing mem0 config, borrows whatever LLM you already have configured, and runs a controlled recall test. No new API keys, no new models, no cloud calls beyond what you already have.
+
+The extraction prompt it generates is **domain-aware** — tuned for technical/research work rather than personal assistant use cases. It's validated against synthetic facts before being applied, and your original config is always backed up.
+
+---
+
+## Built by Hermes Labs
+
+zer0lint is part of the [Hermes Labs](https://hermes-labs.ai) AI agent tooling suite:
+
+- **[lintlang](https://github.com/roli-lpci/lintlang)** — Static linter for AI agent tool descriptions and prompts
+- **[Little Canary](https://github.com/roli-lpci/little-canary)** — Prompt injection detection
+- **[Suy Sideguy](https://github.com/roli-lpci/suy-sideguy)** — Runtime policy enforcement for agents
+- **zer0lint** — Memory extraction diagnostics ← you are here
+
+---
+
+## License
+
+Apache 2.0

zer0lint-0.1.0/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "zer0lint"
+version = "0.1.0"
+description = "mem0 extraction optimizer: diagnose failures and generate custom prompts for your system"
+readme = "README.md"
+requires-python = ">=3.9"
+dependencies = [
+    "mem0ai>=0.1.0",
+    "typer[all]>=0.9.0",
+    "rich>=13.0.0",
+    "httpx>=0.24.0",
+    "pydantic>=2.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0",
+    "pytest-cov>=4.0",
+    "ruff>=0.1.0",
+]
+
+[project.scripts]
+zer0lint = "zer0lint.cli:app"
+
+[tool.ruff]
+line-length = 100
+target-version = "py39"
+
+[tool.ruff.lint]
+select = ["E", "F", "W", "I"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

zer0lint-0.1.0/tests/__init__.py

@@ -0,0 +1 @@
+"""Tests for zer0lint."""

zer0lint-0.1.0/tests/test_fixer.py

@@ -0,0 +1,138 @@
+"""Tests for config application and backup."""
+
+import json
+import tempfile
+from pathlib import Path
+
+import pytest
+
+from zer0lint.fixer import (
+    apply_prompt,
+    backup_config,
+    detect_extraction_model,
+    detect_vector_store,
+)
+
+
+def test_backup_config():
+    """Test config backup creation."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.json"
+        config_path.write_text('{"test": "data"}')
+
+        backup_path = backup_config(config_path)
+
+        assert Path(backup_path).exists()
+        assert "backup" in backup_path
+
+
+def test_apply_prompt_new():
+    """Test applying a new prompt to config."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.json"
+        config = {
+            "llm": {"provider": "ollama"},
+            "custom_fact_extraction_prompt": "old prompt",
+        }
+        config_path.write_text(json.dumps(config))
+
+        result = apply_prompt(config_path, "new prompt", backup=True)
+
+        assert result["success"] is True
+        assert result["backup_path"] is not None
+
+        # Verify new prompt was written
+        updated = json.loads(config_path.read_text())
+        assert updated["custom_fact_extraction_prompt"] == "new prompt"
+
+
+def test_apply_prompt_no_backup():
+    """Test applying prompt without backup."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.json"
+        config_path.write_text('{"custom_fact_extraction_prompt": "old"}')
+
+        result = apply_prompt(config_path, "new", backup=False)
+
+        assert result["success"] is True
+        assert result["backup_path"] is None
+
+
+def test_apply_prompt_file_not_found():
+    """Test error handling for missing config."""
+    with pytest.raises(FileNotFoundError):
+        apply_prompt("/nonexistent/path/config.json", "new prompt")
+
+
+def test_detect_extraction_model_ollama():
+    """Test model detection for Ollama config."""
+    config = {
+        "llm": {
+            "provider": "ollama",
+            "config": {"model": "mistral:7b", "ollama_base_url": "http://localhost:11434"},
+        }
+    }
+
+    model = detect_extraction_model(config)
+
+    assert model == "mistral:7b"
+
+
+def test_detect_extraction_model_openai():
+    """Test model detection for OpenAI config."""
+    config = {
+        "llm": {
+            "provider": "openai",
+            "model": "gpt-4o",
+        }
+    }
+
+    model = detect_extraction_model(config)
+
+    assert model == "gpt-4o"
+
+
+def test_detect_extraction_model_unknown():
+    """Test fallback for unknown config structure."""
+    config = {"llm": {}}
+
+    model = detect_extraction_model(config)
+
+    assert model == "unknown"
+
+
+def test_detect_vector_store_chroma():
+    """Test vector store detection for Chroma."""
+    config = {
+        "vector_store": {
+            "provider": "chroma",
+            "config": {"collection_name": "mem0", "path": ".mem0_chroma"},
+        }
+    }
+
+    store = detect_vector_store(config)
+
+    assert store == "chroma"
+
+
+def test_detect_vector_store_qdrant():
+    """Test vector store detection for Qdrant."""
+    config = {
+        "vector_store": {
+            "provider": "qdrant",
+            "config": {"collection_name": "mem0", "url": "http://localhost:6333"},
+        }
+    }
+
+    store = detect_vector_store(config)
+
+    assert store == "qdrant"
+
+
+def test_detect_vector_store_unknown():
+    """Test fallback for unknown vector store."""
+    config = {"vector_store": {}}
+
+    store = detect_vector_store(config)
+
+    assert store == "unknown"

zer0lint-0.1.0/tests/test_sampler.py

@@ -0,0 +1,92 @@
+"""Tests for memory sampling."""
+
+from unittest.mock import MagicMock
+
+import pytest
+
+from zer0lint.sampler import MemorySample, analyze_sample_content, sample_memories
+
+
+def test_memory_sample_model():
+    """Test MemorySample dataclass."""
+    sample = MemorySample(id="1", content="API runs on port 8421", metadata={"tag": "tech"})
+    assert sample.id == "1"
+    assert sample.content == "API runs on port 8421"
+    assert sample.metadata["tag"] == "tech"
+
+
+def test_sample_memories_with_search():
+    """Test sampling memories via search method."""
+    mock_memory = MagicMock()
+    mock_memory.search.return_value = [
+        {"id": "1", "memory": "API runs on port 8421", "metadata": None},
+        {"id": "2", "memory": "Using gpt-4o-mini", "metadata": None},
+    ]
+
+    samples = sample_memories(mock_memory, limit=30)
+
+    assert len(samples) == 2
+    assert samples[0].id == "1"
+    assert "port 8421" in samples[0].content
+    mock_memory.search.assert_called_once()
+
+
+def test_sample_memories_fallback_to_direct():
+    """Test fallback to direct store access when search fails."""
+    mock_memory = MagicMock()
+    mock_memory.search.side_effect = Exception("Search failed")
+    mock_memory.vector_store.get_all.return_value = [
+        {"id": "1", "content": "Memory 1", "text": None},
+        {"id": "2", "content": "Memory 2", "text": None},
+    ]
+
+    samples = sample_memories(mock_memory, limit=30)
+
+    assert len(samples) >= 1
+    assert samples[0].id == "1"
+
+
+def test_sample_memories_empty():
+    """Test handling empty memories."""
+    mock_memory = MagicMock()
+    mock_memory.search.return_value = []
+    mock_memory.vector_store.get_all.side_effect = Exception("No store")
+
+    samples = sample_memories(mock_memory)
+
+    assert samples == []
+
+
+def test_analyze_sample_content_technical():
+    """Test pattern analysis for technical content."""
+    samples = [
+        MemorySample(id="1", content="API server runs on port 8421"),
+        MemorySample(id="2", content="Updated to gpt-4o-mini model"),
+        MemorySample(id="3", content="Redis version 7.2.4 deployed"),
+    ]
+
+    patterns = analyze_sample_content(samples)
+
+    assert "technical" in patterns
+    assert any(kw in patterns["technical"] for kw in ["port", "version", "model"])
+
+
+def test_analyze_sample_content_mixed():
+    """Test pattern analysis with mixed content."""
+    samples = [
+        MemorySample(id="1", content="API on port 8421"),
+        MemorySample(id="2", content="Diagnosis: acute bronchitis"),
+        MemorySample(id="3", content="Contract clause specifies indemnification"),
+    ]
+
+    patterns = analyze_sample_content(samples)
+
+    # Should detect multiple domains
+    detected_domains = list(patterns.keys())
+    assert len(detected_domains) > 0
+
+
+def test_analyze_sample_content_empty():
+    """Test pattern analysis with no samples."""
+    patterns = analyze_sample_content([])
+    assert patterns == {}