citadel-predict 0.1.0__tar.gz

citadel_predict-0.1.0/.gitignore

@@ -0,0 +1,36 @@
+# Virtual Environments
+.venv/
+**/.venv/
+
+# Byte-compiled / cache files
+__pycache__/
+**/*.pyc
+**/*.pyo
+**/*.pyd
+
+# Testing & Linting Caches
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+
+# Node & Frontend
+node_modules/
+**/node_modules/
+frontend/.next/
+frontend/out/
+
+# Environment Variables & Secrets
+.env
+.env.*
+!.env.example
+
+# IDE & Agents
+.agents/
+.gemini/
+.vscode/
+.idea/
+
+# Distribution & Build
+build/
+dist/
+*.egg-info/

citadel_predict-0.1.0/PKG-INFO

@@ -0,0 +1,215 @@
+Metadata-Version: 2.5
+Name: citadel-predict
+Version: 0.1.0
+Summary: Pre-execution LLM token budget and cost prediction client for AI agents
+Author: Citadel Predict Team
+License: MIT
+Keywords: agent,ai,budget,cost-estimation,governance,guardrails,llm,tokens
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Requires-Dist: httpx>=0.24.0
+Provides-Extra: dev
+Requires-Dist: mypy>=1.10.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.4.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# citadel-predict
+
+[![PyPI Version](https://img.shields.io/pypi/v/citadel-predict.svg)](https://pypi.org/project/citadel-predict/)
+[![Python Versions](https://img.shields.io/pypi/pyversions/citadel-predict.svg)](https://pypi.org/project/citadel-predict/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+
+**Pre-execution token budget and cost predictor client for AI agents.**
+
+`citadel-predict` is a lightweight, pure HTTP Python client and CLI for the Citadel Predict API. It allows developers, CI pipelines, and autonomous agent loops to estimate LLM token consumption and cost ranges *before* initiating expensive agent runs.
+
+---
+
+## Installation
+
+```bash
+pip install citadel-predict
+```
+
+---
+
+## Quickstart (3 Lines of Code)
+
+```python
+from citadel_predict import predict_cost
+
+result = predict_cost(
+    task_text="Research competitor pricing across 3 sources and draft report",
+    tools=["web_search", "draft_document"]
+)
+
+print(f"Expected: {result['expected_tokens']:,} tokens (Range: {result['low_tokens']:,} – {result['high_tokens']:,})")
+```
+
+Output:
+```text
+Expected: 3,200 tokens (Range: 1,500 – 5,800)
+```
+
+---
+
+## CLI Usage
+
+`citadel-predict` includes a full-featured CLI for terminal workflows and CI/CD cost checks:
+
+```bash
+# Pretty terminal card output
+citadel-predict --task "Audit repository and write migration guide" --tools list_files,read_document,draft_document
+
+# Scripting / CI mode (JSON output)
+citadel-predict --task "Calculate statistical metrics" --tools calculator --json
+
+# Override API key or URL
+citadel-predict --task "..." --api-key "cp_live_12345" --api-url "https://api.citadel.dev"
+```
+
+### CLI Exit Codes
+- `0`: Success
+- `2`: Validation Error / Bad Request (HTTP 400 / 422 or missing task)
+- `3`: Authentication Failure (HTTP 401)
+- `4`: Rate Limit Exceeded (HTTP 429)
+- `5`: Server Error (HTTP 5xx)
+- `6`: Network / Timeout Error
+
+---
+
+## Real Agent Integration: Pre-Execution Guardrails
+
+Existing agent governance tools (e.g., Portkey, Langfuse, LiteLLM) are **reactive**—they record costs during or after an execution. `citadel-predict` is **predictive**—enabling pre-flight budget checks and dynamic routing before running reasoning loops.
+
+### LangGraph / CrewAI Pre-Flight Cost Guardrail Example
+
+```python
+from typing import TypedDict, List
+from citadel_predict import predict_cost, CitadelError
+
+class AgentState(TypedDict):
+    task: str
+    tools: List[str]
+    budget_tokens: int
+    approved: bool
+
+def pre_flight_budget_guardrail(state: AgentState) -> AgentState:
+    """
+    Evaluates token budget before dispatching tools or multi-agent loops.
+    """
+    try:
+        prediction = predict_cost(
+            task_text=state["task"],
+            tools=state["tools"],
+            model_id="claude-sonnet"
+        )
+    except CitadelError as e:
+        print(f"Cost prediction unavailable: {e}. Falling back to default budget.")
+        return state
+
+    expected = prediction["expected_tokens"]
+    high = prediction["high_tokens"]
+    is_ood = prediction["out_of_distribution"]
+
+    print(f"Pre-flight estimate: ~{expected:,} tokens (Upper bound: {high:,})")
+    if is_ood:
+        print(f"Warning: Out-of-Distribution task ({prediction['ood_reasons']})")
+
+    # Guardrail Policy: Escalate if upper bound exceeds budget
+    if high > state["budget_tokens"]:
+        print(f"[BLOCKED] High-estimate ({high:,}) exceeds budget ({state['budget_tokens']:,})")
+        # In a real agent: switch to smaller model, ask human for approval, or prune tool access
+        state["approved"] = False
+    else:
+        state["approved"] = True
+
+    return state
+
+# Example usage in workflow
+initial_state: AgentState = {
+    "task": "Perform exhaustive market research across 20 industry filings",
+    "tools": ["web_search", "fetch_url", "draft_document"],
+    "budget_tokens": 10000,
+    "approved": False
+}
+
+state = pre_flight_budget_guardrail(initial_state)
+if not state["approved"]:
+    print("Action required: Human-in-the-loop approval or task reformulation needed.")
+```
+
+---
+
+## Authentication & Configuration
+
+The client resolves your API key and base URL according to the following priority:
+
+1. **Explicit argument**: `predict_cost(..., api_key="...", api_url="...")` or CLI `--api-key` / `--api-url`
+2. **Environment variables**: `CITADEL_API_KEY` and `CITADEL_API_URL`
+3. **Configuration file**: `~/.citadel/config.toml`
+
+### Example `~/.citadel/config.toml`
+```toml
+api_key = "cp_live_your_api_key_here"
+api_url = "https://api.citadel.dev"
+```
+
+---
+
+## Error Handling
+
+`citadel-predict` surfaces typed, catchable exceptions:
+
+```python
+from citadel_predict import (
+    predict_cost,
+    CitadelAuthError,
+    CitadelRateLimitError,
+    CitadelValidationError,
+    CitadelServerError,
+    CitadelNetworkError,
+)
+
+try:
+    result = predict_cost("Analyze dataset", tools=["calculator"])
+except CitadelAuthError:
+    # 401: Missing or invalid API key
+    ...
+except CitadelRateLimitError as e:
+    # 429: Rate limited; check e.retry_after
+    print(f"Retry after {e.retry_after} seconds")
+except CitadelValidationError as e:
+    # 422: Input validation bounds exceeded (e.g. task > 4000 chars)
+    ...
+except CitadelNetworkError as e:
+    # Timeout or connection failure
+    ...
+```
+
+---
+
+## Honest Limitations
+
+`citadel-predict` is a thin client wrapping the hosted calibration model. It directly inherits the current system characteristics:
+
+1. **Single-Model Calibration**: Calibration is currently tuned specifically for **Claude Sonnet** (`claude-sonnet`). Future releases will introduce multi-model support via `model_id`.
+2. **Calibration Dataset Scale**: Calibrated on $N=20$ diverse task archetypes across 80 benchmarked runs.
+3. **Synthetic Tool Sizing**: Ground-truth data was collected using deterministic mock tool outputs with representative context expansion. Real-world tools with unbounded payload returns (e.g., massive scraped DOMs) may exhibit higher variance.
+4. **Pre-execution Estimation**: Token predictions represent calibrated statistical ranges $[low, expected, high]$, not runtime guarantees against infinite loops or divergent agent reasoning.
+
+---
+
+## License
+
+MIT

citadel_predict-0.1.0/README.md

@@ -0,0 +1,190 @@
+# citadel-predict
+
+[![PyPI Version](https://img.shields.io/pypi/v/citadel-predict.svg)](https://pypi.org/project/citadel-predict/)
+[![Python Versions](https://img.shields.io/pypi/pyversions/citadel-predict.svg)](https://pypi.org/project/citadel-predict/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+
+**Pre-execution token budget and cost predictor client for AI agents.**
+
+`citadel-predict` is a lightweight, pure HTTP Python client and CLI for the Citadel Predict API. It allows developers, CI pipelines, and autonomous agent loops to estimate LLM token consumption and cost ranges *before* initiating expensive agent runs.
+
+---
+
+## Installation
+
+```bash
+pip install citadel-predict
+```
+
+---
+
+## Quickstart (3 Lines of Code)
+
+```python
+from citadel_predict import predict_cost
+
+result = predict_cost(
+    task_text="Research competitor pricing across 3 sources and draft report",
+    tools=["web_search", "draft_document"]
+)
+
+print(f"Expected: {result['expected_tokens']:,} tokens (Range: {result['low_tokens']:,} – {result['high_tokens']:,})")
+```
+
+Output:
+```text
+Expected: 3,200 tokens (Range: 1,500 – 5,800)
+```
+
+---
+
+## CLI Usage
+
+`citadel-predict` includes a full-featured CLI for terminal workflows and CI/CD cost checks:
+
+```bash
+# Pretty terminal card output
+citadel-predict --task "Audit repository and write migration guide" --tools list_files,read_document,draft_document
+
+# Scripting / CI mode (JSON output)
+citadel-predict --task "Calculate statistical metrics" --tools calculator --json
+
+# Override API key or URL
+citadel-predict --task "..." --api-key "cp_live_12345" --api-url "https://api.citadel.dev"
+```
+
+### CLI Exit Codes
+- `0`: Success
+- `2`: Validation Error / Bad Request (HTTP 400 / 422 or missing task)
+- `3`: Authentication Failure (HTTP 401)
+- `4`: Rate Limit Exceeded (HTTP 429)
+- `5`: Server Error (HTTP 5xx)
+- `6`: Network / Timeout Error
+
+---
+
+## Real Agent Integration: Pre-Execution Guardrails
+
+Existing agent governance tools (e.g., Portkey, Langfuse, LiteLLM) are **reactive**—they record costs during or after an execution. `citadel-predict` is **predictive**—enabling pre-flight budget checks and dynamic routing before running reasoning loops.
+
+### LangGraph / CrewAI Pre-Flight Cost Guardrail Example
+
+```python
+from typing import TypedDict, List
+from citadel_predict import predict_cost, CitadelError
+
+class AgentState(TypedDict):
+    task: str
+    tools: List[str]
+    budget_tokens: int
+    approved: bool
+
+def pre_flight_budget_guardrail(state: AgentState) -> AgentState:
+    """
+    Evaluates token budget before dispatching tools or multi-agent loops.
+    """
+    try:
+        prediction = predict_cost(
+            task_text=state["task"],
+            tools=state["tools"],
+            model_id="claude-sonnet"
+        )
+    except CitadelError as e:
+        print(f"Cost prediction unavailable: {e}. Falling back to default budget.")
+        return state
+
+    expected = prediction["expected_tokens"]
+    high = prediction["high_tokens"]
+    is_ood = prediction["out_of_distribution"]
+
+    print(f"Pre-flight estimate: ~{expected:,} tokens (Upper bound: {high:,})")
+    if is_ood:
+        print(f"Warning: Out-of-Distribution task ({prediction['ood_reasons']})")
+
+    # Guardrail Policy: Escalate if upper bound exceeds budget
+    if high > state["budget_tokens"]:
+        print(f"[BLOCKED] High-estimate ({high:,}) exceeds budget ({state['budget_tokens']:,})")
+        # In a real agent: switch to smaller model, ask human for approval, or prune tool access
+        state["approved"] = False
+    else:
+        state["approved"] = True
+
+    return state
+
+# Example usage in workflow
+initial_state: AgentState = {
+    "task": "Perform exhaustive market research across 20 industry filings",
+    "tools": ["web_search", "fetch_url", "draft_document"],
+    "budget_tokens": 10000,
+    "approved": False
+}
+
+state = pre_flight_budget_guardrail(initial_state)
+if not state["approved"]:
+    print("Action required: Human-in-the-loop approval or task reformulation needed.")
+```
+
+---
+
+## Authentication & Configuration
+
+The client resolves your API key and base URL according to the following priority:
+
+1. **Explicit argument**: `predict_cost(..., api_key="...", api_url="...")` or CLI `--api-key` / `--api-url`
+2. **Environment variables**: `CITADEL_API_KEY` and `CITADEL_API_URL`
+3. **Configuration file**: `~/.citadel/config.toml`
+
+### Example `~/.citadel/config.toml`
+```toml
+api_key = "cp_live_your_api_key_here"
+api_url = "https://api.citadel.dev"
+```
+
+---
+
+## Error Handling
+
+`citadel-predict` surfaces typed, catchable exceptions:
+
+```python
+from citadel_predict import (
+    predict_cost,
+    CitadelAuthError,
+    CitadelRateLimitError,
+    CitadelValidationError,
+    CitadelServerError,
+    CitadelNetworkError,
+)
+
+try:
+    result = predict_cost("Analyze dataset", tools=["calculator"])
+except CitadelAuthError:
+    # 401: Missing or invalid API key
+    ...
+except CitadelRateLimitError as e:
+    # 429: Rate limited; check e.retry_after
+    print(f"Retry after {e.retry_after} seconds")
+except CitadelValidationError as e:
+    # 422: Input validation bounds exceeded (e.g. task > 4000 chars)
+    ...
+except CitadelNetworkError as e:
+    # Timeout or connection failure
+    ...
+```
+
+---
+
+## Honest Limitations
+
+`citadel-predict` is a thin client wrapping the hosted calibration model. It directly inherits the current system characteristics:
+
+1. **Single-Model Calibration**: Calibration is currently tuned specifically for **Claude Sonnet** (`claude-sonnet`). Future releases will introduce multi-model support via `model_id`.
+2. **Calibration Dataset Scale**: Calibrated on $N=20$ diverse task archetypes across 80 benchmarked runs.
+3. **Synthetic Tool Sizing**: Ground-truth data was collected using deterministic mock tool outputs with representative context expansion. Real-world tools with unbounded payload returns (e.g., massive scraped DOMs) may exhibit higher variance.
+4. **Pre-execution Estimation**: Token predictions represent calibrated statistical ranges $[low, expected, high]$, not runtime guarantees against infinite loops or divergent agent reasoning.
+
+---
+
+## License
+
+MIT

citadel_predict-0.1.0/pyproject.toml

@@ -0,0 +1,56 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "citadel-predict"
+version = "0.1.0"
+description = "Pre-execution LLM token budget and cost prediction client for AI agents"
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [
+    { name = "Citadel Predict Team" }
+]
+keywords = [
+    "ai",
+    "agent",
+    "cost-estimation",
+    "tokens",
+    "llm",
+    "budget",
+    "guardrails",
+    "governance"
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = [
+    "httpx>=0.24.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0.0",
+    "pytest-cov>=4.1.0",
+    "ruff>=0.4.0",
+    "mypy>=1.10.0",
+]
+
+[project.scripts]
+citadel-predict = "citadel_predict.cli:main"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/citadel_predict"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]

citadel_predict-0.1.0/src/citadel_predict/__init__.py

@@ -0,0 +1,30 @@
+"""
+citadel-predict: Developer client for AI agent pre-execution cost prediction.
+"""
+
+from .client import CitadelClient, predict_cost
+from .config import resolve_api_key, resolve_api_url
+from .errors import (
+    CitadelAuthError,
+    CitadelBadRequestError,
+    CitadelError,
+    CitadelNetworkError,
+    CitadelRateLimitError,
+    CitadelServerError,
+    CitadelValidationError,
+)
+
+__version__ = "0.1.0"
+__all__ = [
+    "predict_cost",
+    "CitadelClient",
+    "resolve_api_key",
+    "resolve_api_url",
+    "CitadelError",
+    "CitadelAuthError",
+    "CitadelRateLimitError",
+    "CitadelValidationError",
+    "CitadelBadRequestError",
+    "CitadelServerError",
+    "CitadelNetworkError",
+]

citadel_predict-0.1.0/src/citadel_predict/cli.py

@@ -0,0 +1,179 @@
+"""
+Command-line interface for Citadel Predict.
+
+Usage:
+    citadel-predict --task "Research 3 competitors" --tools web_search,draft_document
+"""
+
+import argparse
+import json
+import sys
+from typing import Optional, Sequence
+
+from .client import CitadelClient
+from .errors import (
+    CitadelAuthError,
+    CitadelBadRequestError,
+    CitadelError,
+    CitadelNetworkError,
+    CitadelRateLimitError,
+    CitadelServerError,
+    CitadelValidationError,
+)
+
+EXIT_SUCCESS = 0
+EXIT_GENERAL_ERROR = 1
+EXIT_VALIDATION_ERROR = 2
+EXIT_AUTH_ERROR = 3
+EXIT_RATE_LIMIT_ERROR = 4
+EXIT_SERVER_ERROR = 5
+EXIT_NETWORK_ERROR = 6
+
+
+def parse_args(args: Optional[Sequence[str]] = None) -> argparse.Namespace:
+    parser = argparse.ArgumentParser(
+        prog="citadel-predict",
+        description="Predict AI agent token budgets and costs before execution.",
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
+    )
+    parser.add_argument(
+        "--task",
+        "-t",
+        type=str,
+        help="Task description string for the agent run.",
+    )
+    parser.add_argument(
+        "positional_task",
+        nargs="?",
+        type=str,
+        help="Task description (if --task is not passed).",
+    )
+    parser.add_argument(
+        "--tools",
+        type=str,
+        default="",
+        help="Comma-separated list of tool names (e.g. 'web_search,fetch_url,draft_document').",
+    )
+    parser.add_argument(
+        "--model-id",
+        "-m",
+        type=str,
+        default="claude-sonnet",
+        help="Model calibration identifier.",
+    )
+    parser.add_argument(
+        "--api-key",
+        "-k",
+        type=str,
+        default=None,
+        help="Citadel API key (overrides CITADEL_API_KEY env var and config file).",
+    )
+    parser.add_argument(
+        "--api-url",
+        "-u",
+        type=str,
+        default=None,
+        help="Citadel API base URL (default: http://localhost:8000 or CITADEL_API_URL).",
+    )
+    parser.add_argument(
+        "--timeout",
+        type=float,
+        default=10.0,
+        help="Request timeout in seconds.",
+    )
+    parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output raw JSON instead of formatted text.",
+    )
+    return parser.parse_args(args)
+
+
+def format_pretty_output(result: dict) -> str:
+    lines = [
+        "=" * 50,
+        "  CITADEL PREDICT — TOKEN BUDGET ESTIMATE",
+        "=" * 50,
+        f"Model:           {result.get('model_id')}",
+        f"Expected Tokens: {result.get('expected_tokens', 0):,} tokens",
+        f"Predicted Range: {result.get('low_tokens', 0):,} – {result.get('high_tokens', 0):,} tokens",
+        f"Confidence:      {result.get('confidence', 'unknown').upper()}",
+    ]
+
+    if result.get("out_of_distribution"):
+        lines.append(f"OOD Warning:     YES ({', '.join(result.get('ood_reasons', []))})")
+    else:
+        lines.append("OOD Warning:     No (In-Distribution)")
+
+    factors = result.get("driving_factors", [])
+    if factors:
+        lines.append("Driving Factors:")
+        for factor in factors:
+            lines.append(f"  • {factor}")
+
+    lines.append("=" * 50)
+    return "\n".join(lines)
+
+
+def main(args: Optional[Sequence[str]] = None) -> int:
+    parsed = parse_args(args)
+
+    task = parsed.task or parsed.positional_task
+    if not task:
+        sys.stderr.write("Error: Task description is required. Use --task '...' or pass as argument.\n")
+        return EXIT_VALIDATION_ERROR
+
+    tools = [t.strip() for t in parsed.tools.split(",") if t.strip()] if parsed.tools else []
+
+    try:
+        with CitadelClient(
+            api_key=parsed.api_key,
+            base_url=parsed.api_url,
+            timeout=parsed.timeout,
+        ) as client:
+            result = client.predict(
+                task_text=task,
+                tools=tools,
+                model_id=parsed.model_id,
+            )
+
+        if parsed.json:
+            print(json.dumps(result, indent=2))
+        else:
+            print(format_pretty_output(result))
+        return EXIT_SUCCESS
+
+    except CitadelAuthError as exc:
+        sys.stderr.write(f"Authentication Error: {exc.message}\n")
+        return EXIT_AUTH_ERROR
+
+    except CitadelRateLimitError as exc:
+        msg = f"Rate Limit Error: {exc.message}"
+        if exc.retry_after is not None:
+            msg += f" (Retry after {exc.retry_after}s)"
+        sys.stderr.write(f"{msg}\n")
+        return EXIT_RATE_LIMIT_ERROR
+
+    except (CitadelValidationError, CitadelBadRequestError) as exc:
+        sys.stderr.write(f"Validation Error: {exc.message}\n")
+        return EXIT_VALIDATION_ERROR
+
+    except CitadelServerError as exc:
+        sys.stderr.write(f"Server Error: {exc.message}\n")
+        return EXIT_SERVER_ERROR
+
+    except CitadelNetworkError as exc:
+        sys.stderr.write(f"Network Error: {exc.message}\n")
+        return EXIT_NETWORK_ERROR
+
+    except CitadelError as exc:
+        sys.stderr.write(f"Citadel Error: {exc.message}\n")
+        return EXIT_GENERAL_ERROR
+
+    except Exception as exc:
+        sys.stderr.write(f"Unexpected Error: {exc}\n")
+        return EXIT_GENERAL_ERROR
+
+
+if __name__ == "__main__":
+    sys.exit(main())