datarep 1.0.0__tar.gz

datarep-1.0.0/.github/workflows/publish.yml

@@ -0,0 +1,28 @@
+name: Publish to PyPI
+
+on:
+  push:
+    tags:
+      - "v*"
+  workflow_dispatch:
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+      contents: read
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build package
+        run: |
+          pip install build
+          python -m build
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

datarep-1.0.0/.gitignore

@@ -0,0 +1,15 @@
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+dist/
+build/
+.eggs/
+*.egg
+.venv/
+venv/
+.env
+*.db
+*.key
+.datarep/
+.pytest_cache/

datarep-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 thyself-fyi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

datarep-1.0.0/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.4
+Name: datarep
+Version: 1.0.0
+Summary: A user-authorized agent runtime that acquires data from arbitrary sources and delivers it to consuming applications.
+Project-URL: Homepage, https://thyself-fyi.github.io/datarep-docs/
+Project-URL: Documentation, https://thyself-fyi.github.io/datarep-docs/integration-guide/
+Project-URL: Repository, https://github.com/thyself-fyi/datarep
+Author: thyself-fyi
+License: MIT
+License-File: LICENSE
+Classifier: Development Status :: 4 - Beta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: MacOS
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.10
+Requires-Dist: anthropic>=0.42.0
+Requires-Dist: bcrypt>=4.2.0
+Requires-Dist: click>=8.1.0
+Requires-Dist: cryptography>=44.0.0
+Requires-Dist: fastapi>=0.115.0
+Requires-Dist: httpx>=0.28.0
+Requires-Dist: mcp[cli]>=1.0.0
+Requires-Dist: uvicorn[standard]>=0.34.0
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
+Requires-Dist: pytest-httpx>=0.35.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# datarep
+
+A trusted local service that retrieves data from arbitrary sources on behalf of your application. Your app describes what data it wants — datarep figures out how to get it.
+
+## Quick start
+
+```bash
+pip install -e ".[dev]"
+datarep init
+export ANTHROPIC_API_KEY="sk-ant-..."
+datarep start
+```
+
+Register your app and get an API key:
+
+```bash
+datarep app register my-app
+```
+
+Retrieve data:
+
+```bash
+curl -X POST http://127.0.0.1:7080/get \
+  -H "Authorization: Bearer dr_<your-api-key>" \
+  -H "Content-Type: application/json" \
+  -d '{"source": "my_source", "query": "get recent records"}'
+```
+
+## What it does
+
+Instead of every app building its own integrations, datarep:
+
+- **Manages credentials** — encrypted storage, browser-based OAuth, automatic token refresh
+- **Writes retrieval code** — a Claude agent inspects sources and generates Python code dynamically
+- **Executes in a sandbox** — network and filesystem restrictions via macOS `sandbox-exec`
+- **Caches as recipes** — working code is saved for instant, deterministic replay
+- **Authenticates apps** — per-app API keys with source-level access control and audit logging
+
+Users grant trust once (to datarep). Every consuming app gets safe data access through it.
+
+## Interfaces
+
+| Interface | Use case |
+|-----------|----------|
+| **HTTP API** (`localhost:7080`) | Primary interface for all apps. Bearer token auth. |
+| **MCP server** | Native interface for agentic/LLM-powered apps. |
+| **CLI** (`datarep`) | Setup, source management, debugging. |
+
+## Source types
+
+| Type | Example | Sandbox |
+|------|---------|---------|
+| `local_db` | iMessage `chat.db`, WhatsApp, any SQLite | No network. Read-only DB access. |
+| `rest_api` | Square, Gmail, Quickbooks | Network restricted to source domain. |
+| `local_files` | Photos, documents, exports | No network. Read-only directory access. |
+
+## Integration guide
+
+See **[docs/integration-guide.md](docs/integration-guide.md)** for the full walkthrough: API reference, authentication, handling permissions, MCP setup, recipes, and code examples.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```

datarep-1.0.0/README.md

@@ -0,0 +1,66 @@
+# datarep
+
+A trusted local service that retrieves data from arbitrary sources on behalf of your application. Your app describes what data it wants — datarep figures out how to get it.
+
+## Quick start
+
+```bash
+pip install -e ".[dev]"
+datarep init
+export ANTHROPIC_API_KEY="sk-ant-..."
+datarep start
+```
+
+Register your app and get an API key:
+
+```bash
+datarep app register my-app
+```
+
+Retrieve data:
+
+```bash
+curl -X POST http://127.0.0.1:7080/get \
+  -H "Authorization: Bearer dr_<your-api-key>" \
+  -H "Content-Type: application/json" \
+  -d '{"source": "my_source", "query": "get recent records"}'
+```
+
+## What it does
+
+Instead of every app building its own integrations, datarep:
+
+- **Manages credentials** — encrypted storage, browser-based OAuth, automatic token refresh
+- **Writes retrieval code** — a Claude agent inspects sources and generates Python code dynamically
+- **Executes in a sandbox** — network and filesystem restrictions via macOS `sandbox-exec`
+- **Caches as recipes** — working code is saved for instant, deterministic replay
+- **Authenticates apps** — per-app API keys with source-level access control and audit logging
+
+Users grant trust once (to datarep). Every consuming app gets safe data access through it.
+
+## Interfaces
+
+| Interface | Use case |
+|-----------|----------|
+| **HTTP API** (`localhost:7080`) | Primary interface for all apps. Bearer token auth. |
+| **MCP server** | Native interface for agentic/LLM-powered apps. |
+| **CLI** (`datarep`) | Setup, source management, debugging. |
+
+## Source types
+
+| Type | Example | Sandbox |
+|------|---------|---------|
+| `local_db` | iMessage `chat.db`, WhatsApp, any SQLite | No network. Read-only DB access. |
+| `rest_api` | Square, Gmail, Quickbooks | Network restricted to source domain. |
+| `local_files` | Photos, documents, exports | No network. Read-only directory access. |
+
+## Integration guide
+
+See **[docs/integration-guide.md](docs/integration-guide.md)** for the full walkthrough: API reference, authentication, handling permissions, MCP setup, recipes, and code examples.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```

datarep-1.0.0/datarep/__init__.py

@@ -0,0 +1,148 @@
+"""datarep — a user-authorized agent runtime for delegated data retrieval."""
+
+from __future__ import annotations
+
+__version__ = "1.0.0"
+
+import asyncio
+import json
+from pathlib import Path
+from typing import Any
+
+from datarep.config import ensure_home, get_home
+from datarep.db import get_connection
+from datarep.agent import RetrievalAgent
+from datarep.app_auth import AppAuthManager
+from datarep.credentials import CredentialStore, OAuthFlow
+from datarep.recipes import RecipeStore
+from datarep.registry import SourceRegistry
+from datarep.sync_state import SyncStateManager
+
+
+class DataRep:
+    """High-level Python API for datarep.
+
+    Usage:
+        rep = DataRep()
+        rep.add_source("imessage", source_type="local_db",
+                       config={"path": "~/Library/Messages/chat.db"})
+        result = rep.get("imessage", query="messages from last 7 days")
+    """
+
+    def __init__(self, home: str | Path | None = None):
+        if home:
+            import os
+            os.environ["DATAREP_HOME"] = str(home)
+        ensure_home()
+        self._conn = get_connection()
+        self._registry = SourceRegistry(self._conn)
+        self._cred_store = CredentialStore(self._conn)
+        self._recipe_store = RecipeStore(self._conn)
+        self._sync_state = SyncStateManager(self._conn)
+        self._app_auth = AppAuthManager(self._conn)
+        self._agent: RetrievalAgent | None = None
+
+    def _get_agent(self) -> RetrievalAgent:
+        if self._agent is None:
+            self._agent = RetrievalAgent(
+                self._conn,
+                registry=self._registry,
+                cred_store=self._cred_store,
+                recipe_store=self._recipe_store,
+                sync_state=self._sync_state,
+            )
+        return self._agent
+
+    # --- Source management ---
+
+    def add_source(
+        self,
+        name: str,
+        source_type: str,
+        config: dict[str, Any] | None = None,
+    ) -> dict[str, Any]:
+        return self._registry.add(name, source_type, config)
+
+    def list_sources(self) -> list[dict[str, Any]]:
+        return self._registry.list()
+
+    def remove_source(self, name: str) -> bool:
+        return self._registry.remove(name)
+
+    # --- Credentials ---
+
+    def set_credentials(
+        self,
+        source: str,
+        cred_type: str,
+        data: dict[str, Any],
+        expires_at: str | None = None,
+    ) -> None:
+        self._cred_store.store(source, cred_type, data, expires_at)
+
+    def run_oauth(self, source: str, timeout: int = 120) -> dict[str, Any]:
+        """Run a browser-based OAuth flow for a source."""
+        src = self._registry.get(source)
+        if src is None:
+            raise ValueError(f"Source '{source}' not found.")
+        config = src["config"]
+        flow = OAuthFlow(
+            auth_url=config["auth_url"],
+            token_url=config["token_url"],
+            client_id=config["client_id"],
+            client_secret=config["client_secret"],
+            scopes=config.get("scopes", []),
+        )
+        tokens = flow.run(timeout=timeout)
+        self._cred_store.store(source, "oauth2", {
+            **tokens,
+            "client_id": config["client_id"],
+            "client_secret": config["client_secret"],
+            "token_url": config["token_url"],
+        })
+        return tokens
+
+    # --- Retrieval ---
+
+    def get(self, source: str, query: str) -> dict[str, Any]:
+        """Agent-driven data retrieval."""
+        agent = self._get_agent()
+        return asyncio.run(agent.run(source, query))
+
+    def sync(self, source: str, query: str = "") -> dict[str, Any]:
+        """Incremental sync for a source."""
+        agent = self._get_agent()
+        q = query or f"Incremental sync for {source}"
+        return asyncio.run(agent.run(source, q))
+
+    def run_recipe(self, recipe_id: str) -> dict[str, Any]:
+        """Run a saved recipe (no LLM call)."""
+        agent = self._get_agent()
+        return asyncio.run(agent.run_recipe(recipe_id))
+
+    # --- Recipes ---
+
+    def list_recipes(self, source: str | None = None) -> list[dict[str, Any]]:
+        return self._recipe_store.list(source_name=source)
+
+    def get_recipe(self, recipe_id: str) -> dict[str, Any] | None:
+        return self._recipe_store.get(recipe_id)
+
+    # --- Apps ---
+
+    def register_app(
+        self, name: str, allowed_sources: list[str] | None = None
+    ) -> tuple[str, str]:
+        """Register a consuming app. Returns (app_id, api_key)."""
+        return self._app_auth.register_app(name, allowed_sources)
+
+    def list_apps(self) -> list[dict[str, Any]]:
+        return self._app_auth.list_apps()
+
+    def revoke_app(self, app_id: str) -> bool:
+        return self._app_auth.revoke(app_id)
+
+    # --- Sync state ---
+
+    def get_sync_state(self, source: str) -> dict[str, Any] | None:
+        return self._sync_state.get(source)