stateloom 0.1.0__tar.gz

stateloom-0.1.0/.claude/plans/v0.2-roadmap.md

@@ -0,0 +1,92 @@
+# StateLoom v0.2 Roadmap: Enterprise Scale & Distributed Governance
+
+## The Vision: "The Standard Library for Agent State"
+
+While v0.1 focused on the **individual developer's productivity** (Replay, Local Dashboard, Cost Tracking), v0.2 transforms StateLoom into **enterprise-grade infrastructure**. The goal is to provide a unified governance and observability layer that works across distributed microservices, multi-cloud environments, and complex agent orchestration frameworks.
+
+---
+
+## Pillar 1: OpenTelemetry (OTel) & Distributed Tracing
+
+StateLoom will adopt OpenTelemetry as its underlying observability standard. This allows it to "speak the same language" as the rest of the enterprise stack (Datadog, Honeycomb, New Relic).
+
+### 1.1 Trace Context Propagation
+- **Session-to-Trace Mapping:** Automatically link an StateLoom `session_id` to an OTel `TraceID`.
+- **Header Injection:** Automatically inject `x-stateloom-session-id` and `traceparent` headers into outbound `httpx` and `requests` calls.
+- **Microservice Continuity:** If Service A starts a session and calls Service B (a researcher agent), Service B will automatically detect and resume the same StateLoom session from the OTel context.
+
+### 1.2 OTLP Exporter
+- **Export to Any Backend:** Provide a native OTLP exporter to pipe StateLoom metrics (cost, PII detections, loop flags) into standard enterprise dashboards.
+- **Custom Attributes:** Attach `stateloom.model`, `stateloom.cost`, and `stateloom.session_status` as attributes to every OTel span.
+
+### 1.3 Auto-Instrumentation
+- **Web Framework Middleware:** Provide one-line `StateLoomMiddleware` for **FastAPI**, **Flask**, and **Django**.
+- **Zero-Touch Sessions:** A new session is automatically created at the start of an HTTP request and closed when the response is sent.
+
+---
+
+## Pillar 2: Distributed State & Governance
+
+Moving beyond single-process agents to horizontally scaled production environments.
+
+### 2.1 Pluggable State Backends
+- **Redis & Postgres:** Support Redis (for high-speed budget locking) and Postgres (for long-term session storage) as alternatives to local SQLite.
+- **Distributed Locking:** Implement distributed locks for **Budget Enforcement**. If five workers are executing steps for one session, they must all share a single atomic budget counter.
+
+### 2.2 Centralized Policy Management (The Control Plane)
+- **Dynamic Policy Sync:** The SDK should be able to fetch `stateloom.yaml` from a remote URL at startup.
+- **Policy Hot-Reload:** Update PII rules or budget limits across an entire fleet of agents in real-time without redeploying code.
+
+### 2.3 Advanced PII with Microsoft Presidio
+- **Presidio Integration:** Formalize the `stateloom[presidio]` extra to use NLP-based PII detection (better than regex for addresses, names, and entities).
+- **Custom Scrubbing:** Allow enterprises to define custom scrubbing logic for proprietary data types (e.g., internal part numbers).
+
+### 2.4 Zero-Trust Data Architecture (BYOD Audit Trails)
+- **Data Routing:** StateLoom acts as a secure router. Metadata (cost, tokens, latency) is synced to the StateLoom Control Plane for analytics, but **raw payloads** (prompts/responses) are routed directly to the customer's own VPC storage (AWS S3, Azure Blob, GCS, or Snowflake).
+- **End-to-End Encryption:** Payloads are encrypted at the SDK level using customer-managed keys (AWS KMS / Azure Key Vault) before being persisted.
+- **Compliance-Grade Logging:** Generate immutable, cryptographically signed audit logs of every LLM interaction, stored entirely within the customer's security boundary.
+- **Why it wins:** This architecture bypasses the most difficult stage of enterprise procurement—the Security Review—by ensuring StateLoom never has "custody" of sensitive customer data.
+
+---
+
+## Pillar 3: The "Developer Wedge" V2 (CLI & Replay)
+
+Hardening the local-to-production debugging loop.
+
+### 3.1 Production Incident Triage (`stateloom pull`)
+- **Remote Session Pull:** Fetch a failed production session from the control plane/database to a local machine.
+- **Automatic Masking:** The control plane applies PII masking *before* the session is downloaded to a developer's machine to maintain GDPR/SOC2 compliance.
+
+### 3.2 Regression Testing (`stateloom test`)
+- **Golden Run Pinning:** "Pin" a successful session as a "Golden Run."
+- **CI/CD Integration:** A new `stateloom test` command that replays the Golden Runs against the current code. If the agent fails to reach the same conclusion or exceeds the "Golden Cost," the CI pipeline fails.
+
+### 3.3 Replay "Fuzzing"
+- **Non-Deterministic Analysis:** Automatically run a replay 10 times with different temperatures to find "flaky" agent steps that lead to divergent outcomes.
+
+---
+
+## Pillar 4: AI Economics (Local Offloading)
+
+Maximizing ROI through model cascading.
+
+### 4.1 Intelligent Auto-Router V2
+- **Semantic Complexity Scoring:** Use a local embedding model to grade the "difficulty" of a prompt before routing.
+- **Fallback Logic:** If a local model (Ollama) returns a low-confidence score, automatically re-route the request to GPT-4o and log the failure to improve the router's training data.
+
+### 4.2 Local Cache Proxy
+- **Session-Independent Caching:** Allow exact-match caching across *different* sessions for identical requests (e.g., "What are our return policies?").
+
+---
+
+## Technical Milestones
+
+1. **v0.2.1 (The OTel Bridge):** Implement `ContextVar` synchronization with OTel and basic OTLP exporter.
+2. **v0.2.2 (The Scale Layer):** Implement Redis storage backend and distributed budget locking.
+3. **v0.2.3 (The Policy Plane):** Add remote config fetching and Presidio PII support.
+4. **v0.2.4 (The CLI Suite):** Finalize `stateloom pull` and `stateloom test` workflows.
+
+---
+
+## Success Metric for v0.2
+"A developer can deploy a LangGraph agent to a Kubernetes cluster with 10 replicas, and StateLoom will correctly enforce a single $5 budget across the entire fleet while piping the results into the company's existing Datadog dashboard."

stateloom-0.1.0/.claude/settings.local.json

@@ -0,0 +1,39 @@
+{
+  "permissions": {
+    "allow": [
+      "WebSearch",
+      "Bash(python -m pytest:*)",
+      "WebFetch(domain:github.com)",
+      "WebFetch(domain:developers.openai.com)",
+      "Bash(curl:*)",
+      "Bash(tar:*)",
+      "Bash(python3:*)",
+      "Bash(ruff format:*)",
+      "Bash(ruff check:*)",
+      "Bash(python -m ruff check:*)",
+      "Bash(python -m ruff format:*)",
+      "Bash(ls:*)",
+      "Bash(find:*)",
+      "Bash(__NEW_LINE_f878715c604ee769__ echo \"\")",
+      "Bash(__NEW_LINE_155e46961a51060b__ echo \"\")",
+      "Bash(python:*)",
+      "Bash(xargs:*)",
+      "Bash(grep:*)",
+      "Bash(done)",
+      "Bash(__NEW_LINE_48e957917c204c8a__ echo \"=== CLI COMMAND NAME ===\")",
+      "Bash(__NEW_LINE_9f1f5c11b909831e__ echo \"\")",
+      "Bash(__NEW_LINE_2b0e96ce7f84692b__ echo \"\")",
+      "Bash(__NEW_LINE_94f4a257f596dc85__ echo \"\")",
+      "Bash(__NEW_LINE_973c1a54780550e2__ echo \"\")",
+      "Bash(echo:*)",
+      "Bash(__NEW_LINE_5749ae0392ae1f60__ echo \"\")",
+      "Bash(__NEW_LINE_9ed2b214ac25caa6__ echo \"\")",
+      "Bash(EOF)",
+      "Bash(bash /tmp/grep_queries.sh)",
+      "Bash(wc:*)",
+      "Bash(pytest:*)",
+      "WebFetch(domain:arxiv.org)",
+      "Bash(/tmp/shadow_code_summary.md:*)"
+    ]
+  }
+}

stateloom-0.1.0/.github/dependabot.yml

@@ -0,0 +1,39 @@
+version: 2
+updates:
+  - package-ecosystem: pip
+    directory: "/"
+    schedule:
+      interval: weekly
+      day: monday
+    open-pull-requests-limit: 10
+    groups:
+      dev-deps:
+        patterns:
+          - "pytest*"
+          - "ruff"
+          - "mypy"
+          - "respx"
+        update-types:
+          - "minor"
+          - "patch"
+      core-deps:
+        patterns:
+          - "pydantic*"
+          - "fastapi"
+          - "uvicorn*"
+          - "httpx"
+          - "rich"
+          - "wrapt"
+          - "tiktoken"
+          - "websockets"
+          - "pyyaml"
+          - "click"
+        update-types:
+          - "patch"
+
+  - package-ecosystem: github-actions
+    directory: "/"
+    schedule:
+      interval: weekly
+    labels:
+      - ci

stateloom-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,211 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    name: Lint
+    runs-on: ubuntu-latest
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install ruff
+        run: pip install ruff>=0.1
+
+      - name: Ruff check
+        run: ruff check src/ tests/
+
+      - name: Ruff format check
+        run: ruff format --check src/ tests/
+
+  typecheck:
+    name: Type Check
+    runs-on: ubuntu-latest
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,metrics,prompts,encryption,migrations]"
+
+      - name: mypy
+        run: mypy src/stateloom/
+
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    needs: lint
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.13"]
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,metrics,prompts,encryption,migrations]"
+
+      - name: Run tests
+        run: |
+          pytest tests/ \
+            --ignore=tests/test_production \
+            --ignore=tests/test_e2e \
+            --ignore=tests/test_store/test_postgres_store.py \
+            --ignore=tests/test_cache/test_redis_cache_store.py \
+            --ignore=tests/test_integration \
+            -v --tb=short \
+            --cov=stateloom --cov-report=xml --cov-report=term-missing --cov-fail-under=80
+
+      - name: Upload coverage
+        if: matrix.python-version == '3.13'
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-report
+          path: coverage.xml
+
+  test-production:
+    name: Production Tests
+    runs-on: ubuntu-latest
+    needs: lint
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,metrics,prompts,encryption,migrations]"
+
+      - name: Run production tests
+        run: |
+          pytest tests/test_production/ tests/test_e2e/ \
+            -v --tb=short \
+            --ignore-glob="**/test_postgres*" \
+            --ignore-glob="**/test_redis*" \
+            || true
+
+  test-integration:
+    name: Integration Tests
+    runs-on: ubuntu-latest
+    needs: lint
+    services:
+      postgres:
+        image: postgres:16
+        env:
+          POSTGRES_USER: stateloom_test
+          POSTGRES_PASSWORD: stateloom_test
+          POSTGRES_DB: stateloom_test
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd="pg_isready -U stateloom_test"
+          --health-interval=10s
+          --health-timeout=5s
+          --health-retries=5
+      redis:
+        image: redis:7
+        ports:
+          - 6379:6379
+        options: >-
+          --health-cmd="redis-cli ping"
+          --health-interval=10s
+          --health-timeout=5s
+          --health-retries=5
+    env:
+      STATELOOM_TEST_POSTGRES_URL: postgresql://stateloom_test:stateloom_test@localhost:5432/stateloom_test
+      STATELOOM_TEST_REDIS_URL: redis://localhost:6379/0
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,metrics,prompts,encryption,migrations,postgres,redis]"
+
+      - name: Run integration tests
+        run: |
+          pytest \
+            tests/test_store/test_postgres_store.py \
+            tests/test_cache/test_redis_cache_store.py \
+            -v --tb=short
+
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    needs: [test, typecheck]
+    steps:
+      - name: Harden Runner
+        uses: step-security/harden-runner@v2
+        with:
+          egress-policy: audit
+
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Install build tools
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Verify wheel installs
+        run: |
+          pip install dist/*.whl
+          python -c "import stateloom; print(stateloom.__version__)"
+
+      - name: Upload build artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/

stateloom-0.1.0/.github/workflows/dependabot-auto-merge.yml

@@ -0,0 +1,34 @@
+name: Dependabot Auto-Merge
+
+on:
+  pull_request:
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  auto-merge:
+    name: Auto-Merge Dependabot PRs
+    runs-on: ubuntu-latest
+    if: github.actor == 'dependabot[bot]'
+    steps:
+      - name: Fetch Dependabot metadata
+        id: metadata
+        uses: dependabot/fetch-metadata@v2
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Auto-approve patch and minor updates
+        if: steps.metadata.outputs.update-type != 'version-update:semver-major'
+        run: gh pr review --approve "$PR_URL"
+        env:
+          PR_URL: ${{ github.event.pull_request.html_url }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Auto-merge patch and minor updates
+        if: steps.metadata.outputs.update-type != 'version-update:semver-major'
+        run: gh pr merge --auto --squash "$PR_URL"
+        env:
+          PR_URL: ${{ github.event.pull_request.html_url }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

stateloom-0.1.0/.github/workflows/dependency-review.yml

@@ -0,0 +1,18 @@
+name: Dependency Review
+on: [pull_request]
+
+permissions:
+  contents: read
+
+jobs:
+  dependency-review:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Dependency Review
+        uses: actions/dependency-review-action@v4
+        with:
+          fail-on-severity: moderate
+          comment-summary-in-pr: always

stateloom-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,71 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: write
+  id-token: write
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Install build tools
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Upload build artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    name: Publish to PyPI
+    runs-on: ubuntu-latest
+    needs: build
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - name: Download build artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+
+  github-release:
+    name: GitHub Release
+    runs-on: ubuntu-latest
+    needs: build
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Download build artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Create GitHub Release
+        uses: softprops/action-gh-release@v2
+        with:
+          generate_release_notes: true
+          files: dist/*

stateloom-0.1.0/.github/workflows/scorecard.yml

@@ -0,0 +1,42 @@
+name: OpenSSF Scorecard
+on:
+  branch_protection_rule:
+  schedule:
+    - cron: "20 7 * * 2"
+  push:
+    branches: ["main"]
+
+permissions: read-all
+
+jobs:
+  analysis:
+    name: Scorecard analysis
+    runs-on: ubuntu-latest
+    permissions:
+      security-events: write
+      id-token: write
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+
+      - name: Run analysis
+        uses: ossf/scorecard-action@v2.4.0
+        with:
+          results_file: results.sarif
+          results_format: sarif
+          publish_results: true
+
+      - name: Upload artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: SARIF file
+          path: results.sarif
+          retention-days: 5
+
+      - name: Upload to code-scanning
+        uses: github/codeql-action/upload-sarif@v3
+        with:
+          sarif_file: results.sarif

stateloom-0.1.0/.github/workflows/security.yml

@@ -0,0 +1,87 @@
+name: Security
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  schedule:
+    - cron: "0 6 * * 1"  # Monday 6am UTC
+  workflow_dispatch:
+
+jobs:
+  bandit:
+    name: Bandit SAST
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Install bandit
+        run: pip install bandit[toml]
+
+      - name: Run bandit
+        run: bandit -c pyproject.toml -r src/ -f json -o bandit-report.json || true
+
+      - name: Upload bandit report
+        uses: actions/upload-artifact@v4
+        with:
+          name: bandit-report
+          path: bandit-report.json
+
+  pip-audit:
+    name: Dependency Audit
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.13"
+
+      - name: Install dependencies
+        run: pip install -e ".[dev]" pip-audit
+
+      - name: Run pip-audit
+        run: pip-audit --desc --format json --output pip-audit-report.json || true
+
+      - name: Upload pip-audit report
+        uses: actions/upload-artifact@v4
+        with:
+          name: pip-audit-report
+          path: pip-audit-report.json
+
+  codeql:
+    name: CodeQL
+    runs-on: ubuntu-latest
+    permissions:
+      security-events: write
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Initialize CodeQL
+        uses: github/codeql-action/init@v3
+        with:
+          languages: python
+
+      - name: Perform CodeQL analysis
+        uses: github/codeql-action/analyze@v3
+
+  secrets-scan:
+    name: Secrets Scan
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: TruffleHog scan
+        uses: trufflesecurity/trufflehog@main
+        with:
+          extra_args: --only-verified

stateloom-0.1.0/.gitignore

@@ -0,0 +1,46 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+# Distribution / packaging
+dist/
+build/
+*.egg-info/
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# Testing
+.pytest_cache/
+.coverage
+coverage.xml
+htmlcov/
+
+# Type checking / linting
+.mypy_cache/
+.ruff_cache/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# Environment
+.env
+.env.*
+
+# Database
+*.db
+*.db-wal
+*.db-shm
+
+# OS
+.DS_Store
+Thumbs.db