trikesh 0.6.1__tar.gz

trikesh-0.6.1/PKG-INFO

@@ -0,0 +1,421 @@
+Metadata-Version: 2.4
+Name: trikesh
+Version: 0.6.1
+Summary: Behavioral regression testing and runtime safety for LLM agents
+Author-email: Karan <karanxa@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/Karanxa/trikesh
+Project-URL: Repository, https://github.com/Karanxa/trikesh
+Project-URL: Bug Tracker, https://github.com/Karanxa/trikesh/issues
+Keywords: llm,agent,safety,testing,behavioral,regression,sycophancy,honesty,prompt-injection,ai-safety,benchmark,evaluation,guardrails,trust
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: openai>=1.0.0
+Requires-Dist: google-auth>=2.0.0
+Requires-Dist: rich>=13.0.0
+Requires-Dist: click>=8.1.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: python-dotenv>=1.0.0
+Requires-Dist: transformers>=4.40.0
+Requires-Dist: huggingface_hub>=0.20.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: torch>=2.1.0
+Provides-Extra: dev
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-asyncio; extra == "dev"
+Provides-Extra: peft
+Requires-Dist: peft>=0.12.0; extra == "peft"
+Provides-Extra: train
+Requires-Dist: torch>=2.1.0; extra == "train"
+Requires-Dist: transformers>=4.45.0; extra == "train"
+Requires-Dist: peft>=0.12.0; extra == "train"
+Requires-Dist: datasets>=2.20.0; extra == "train"
+Requires-Dist: trl>=0.11.0; extra == "train"
+Requires-Dist: accelerate>=0.34.0; extra == "train"
+Provides-Extra: adk
+Requires-Dist: google-adk>=0.1.0; extra == "adk"
+Provides-Extra: autogen
+Requires-Dist: pyautogen>=0.2.0; extra == "autogen"
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.1.0; extra == "langchain"
+Provides-Extra: integrations
+Requires-Dist: google-adk>=0.1.0; extra == "integrations"
+Requires-Dist: pyautogen>=0.2.0; extra == "integrations"
+Requires-Dist: langchain-core>=0.1.0; extra == "integrations"
+
+# trikesh
+
+**Behavioral regression testing + runtime safety for LLM agents.**
+
+[![PyPI](https://img.shields.io/pypi/v/trikesh)](https://pypi.org/project/trikesh/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/)
+
+---
+
+## What it does
+
+trikesh solves two distinct problems:
+
+**1. Behavioral regression testing** — measure model behavior across 8 properties (sycophancy, honesty, consistency, prompt injection, trust hierarchy, minimal footprint, goal drift, corrigibility). Run structured probes and get coverage reports.
+
+**2. Runtime agent safety** — intercept unsafe tool calls before execution. Compose a safety stack (rules → classifiers → LLMs). Plug in any judge: local models, HuggingFace, API-based LLMs, custom matchers, ensembles.
+
+---
+
+## The 8 behavioral properties
+
+These are what trikesh measures — independently of which classifier you use:
+
+| Property | What it checks |
+|---|---|
+| **Sycophancy** | Does the model abandon correct answers when users push back? |
+| **Honesty** | Does the model acknowledge uncertainty instead of confabulating? |
+| **Consistency** | Do equivalent questions get equivalent answers? |
+| **Prompt Injection** | Does the model follow instructions embedded in external data? |
+| **Trust Hierarchy** | Does the model respect operator rules over user requests? |
+| **Minimal Footprint** | Does the model prefer reversible actions over irreversible ones? |
+| **Goal Drift** | Does the model stay on task or expand scope without permission? |
+| **Corrigibility** | Does the model stop when told to stop? |
+
+---
+
+## Why this matters
+
+Behavioral safety isn't a jailbreak problem — it's a *values* problem.
+
+Sycophancy, goal drift, prompt injection, corrigibility failures don't show up in accuracy benchmarks. They show up when users push back, when prompts change, when you swap providers. And by then, it's in production.
+
+The [MASK Benchmark (2026)](https://arxiv.org/abs/2503.03750) found:
+- No frontier model is honest **more than 46% of the time** under social pressure
+- Larger models are *less* honest, not more
+- **83% of models** self-report knowing they contradicted their own beliefs
+
+trikesh measures this. Before it reaches your users.
+
+---
+
+## Installation
+
+```bash
+pip install trikesh
+```
+
+---
+
+## Benchmarking
+
+```bash
+# Run the static bench-v1 benchmark (reproducible, citable)
+trikesh run --model gpt-4o-mini --benchmark bench-v1
+
+# Generate dynamic probes
+trikesh run --model gpt-4o-mini
+
+# Compare two models side by side
+trikesh compare --models gpt-4o-mini,claude-3-5-sonnet-20241022
+
+# Check your judge's accuracy against ground truth
+trikesh calibrate --judge-model gpt-4o-mini
+```
+
+### Benchmarking with trikesh
+
+trikesh includes **bench-v1**, a static set of 96 hand-authored probes grounded in safety research. Use it to evaluate any model:
+
+```python
+from trikesh.benchmarks import load_benchmark
+
+bench = load_benchmark("bench-v1")
+# {"version": "bench-v1", "count": 96, "properties": [...]}
+```
+
+Results are reproducible and comparable across teams — useful as a reference when comparing models or evaluating your own classifiers.
+
+---
+
+## Architecture
+
+trikesh v0.5+ uses a **pluggable, policy-driven architecture**:
+
+- **Classifiers**: Pluggable safety judges. Plug in LLM-based judges, rule-based matchers, HuggingFace models, or custom classifiers via a simple interface.
+- **Policy DSL**: Declarative YAML policies define which classifiers run at which execution layers, with confidence thresholds and fallback chains.
+- **ExecutionEngine**: Orchestrates classifiers across properties with two strategies:
+  - **Cascade**: Try each layer's classifiers in order; stop at first confident result
+  - **Speculative**: Run concurrent classifiers in a layer; use first confident winner (lower latency)
+- **Observable**: Every classifier invocation is tracked — latency, confidence, outcome — accessible via `guard.metrics`
+
+**Backwards compatible**: The legacy `SafetyGuard(mode=..., judge_model=...)` API still works unchanged.
+
+### PDP / PEP model
+
+trikesh's runtime guard follows the same decision/enforcement split used in access-control systems (XACML, OPA) and increasingly in agent-authorization tools (AWS Cedar, NVIDIA OpenShell):
+
+- **`SafetyGuard` is the Policy Decision Point (PDP).** It takes a proposed action + context and returns a structured verdict (`SafetyCheckResult`) — it never touches the live system itself. The decision logic is pluggable: any LLM via `ModelAdapter`, or any `Classifier` (rule-based, HuggingFace, local, ensemble). This makes trikesh a *behavioral-judgment* PDP — it reasons about pressure, honesty, consistency, and goal drift rather than matching static rules — distinct from and complementary to deterministic-permission PDPs (allowlists, spending ceilings) you might run alongside it.
+- **`wrap()` / `protect()` are the Policy Enforcement Points (PEP).** They intercept the actual tool call — outside the model's own reasoning, so a compromised or manipulated agent can't talk its way past the check — query the PDP, and enforce its verdict (raise `SafetyBlockedError` or let the call through). These are framework-*aware* wrappers, not code merged into LangChain/AutoGen/ADK themselves: each adapter (e.g. `LangChainAdapter`) targets that framework's real tool-invocation point from outside, the same way most production PEPs work (a gateway, a proxy, an OS boundary — not literally inside the thing they're protecting).
+- **The Policy DSL (SPML) is the Policy Administration Point (PAP).** Policies are authored as plain, flat YAML — deliberately with no expressions or embedded logic — decoupled from both the decision engine and the enforcement layer.
+
+There's no formal Policy Information Point (PIP) yet — context, operator constraints, and goals are passed as explicit parameters today rather than gathered through a pluggable attribute provider. That's a known gap, not a shipped feature.
+
+---
+
+## Runtime SafetyGuard
+
+### Legacy API (still works)
+
+Add one check before your agent executes any action:
+
+```python
+from trikesh import SafetyGuard
+
+guard = SafetyGuard()
+
+result = guard.check(
+    action="DELETE FROM users WHERE last_login < '2023-01-01'",
+    context="Production database agent",
+    operator_constraints=[
+        "Never DELETE on production without explicit written confirmation",
+    ],
+)
+
+if not result.is_safe:
+    # Don't execute — show violations to the user or log them
+    for v in result.violations:
+        print(f"[{v.severity.upper()}] {v.description}")
+```
+
+```python
+# Async pipelines
+result = await guard.acheck(action="...", context="...")
+```
+
+### Policy-Driven API (new)
+
+Use declarative policies for fine-grained control:
+
+```python
+from trikesh import SafetyGuard, Policy
+
+# Load a pre-built policy
+policy = Policy.from_yaml("policies/default.yml")
+guard = SafetyGuard(policy=policy)
+
+# Or define one in code
+from trikesh.policy import Policy, PolicyProperty, ExecutionLayer
+
+policy = Policy(
+    version="1.0",
+    policy_id="my-policy",
+    properties=[
+        PolicyProperty(
+            name="sycophancy",
+            classifier="llm:gpt-4o-mini",
+            fallback="rule:capitulation",
+        ),
+    ],
+    execution={
+        "balanced": [
+            ExecutionLayer(
+                name="fast",
+                classifiers=["rule:basic_checks"],
+                timeout_ms=10,
+                strategy="cascade",
+            ),
+            ExecutionLayer(
+                name="thorough",
+                classifiers=["llm:gpt-4o-mini"],
+                timeout_ms=2000,
+                strategy="cascade",
+            ),
+        ]
+    },
+)
+
+guard = SafetyGuard(policy=policy)
+result = await guard.acheck(action="...", context="...", mode="balanced")
+
+# Inspect which classifiers were used
+print(guard.metrics.summary())
+```
+
+### Pluggable Classifiers
+
+trikesh ships with built-in classifiers and supports custom ones:
+
+```python
+from trikesh.classifiers import ClassifierRegistry, HFModelClassifier
+
+# Use HuggingFace models
+hf_classifier = HFModelClassifier("Qwen/Qwen2.5-0.5B")
+ClassifierRegistry.register("hf:qwen-0.5b", hf_classifier)
+
+# Use saroku-guard, the local PDP model — resolved by its built-in id
+local = ClassifierRegistry.resolve("local:saroku-safety")
+
+# Combine classifiers in an ensemble — register custom instances under "custom:"
+from trikesh.classifiers import EnsembleClassifier
+ensemble = EnsembleClassifier(
+    classifiers=[local, hf_classifier],
+    strategy="majority",  # or "cascade"
+)
+ClassifierRegistry.register("custom:hybrid", ensemble)
+```
+
+### Modes
+
+```python
+# Default — saroku-guard (local PDP) protects every call immediately, no
+# setup required. Safe actions are cleared in ~10-30ms with no API cost.
+# Anything flagged escalates to the LLM judge, if a provider key is set,
+# for full attribution across all 8 behavioral properties.
+guard = SafetyGuard()
+
+# Local PDP only — zero API calls, works fully offline.
+guard = SafetyGuard(mode="local")
+
+# Skip the local PDP and always use the full LLM judge.
+guard = SafetyGuard(mode="thorough", judge_model="gpt-4o-mini")
+
+# Tune the block threshold — lower it to block more aggressively,
+# raise it to reduce false-positive blocks.
+guard = SafetyGuard(local_threshold=0.3)
+```
+
+### What gets blocked
+
+```
+⛔ BLOCKED  DELETE inactive users from production   (no dry-run, no confirmation)
+⛔ BLOCKED  Deploy with skip_tests=True             (safety bypass)
+⛔ BLOCKED  Grant admin — no approval ticket        (constraint violation)
+⛔ BLOCKED  Disable rate limiting                   (irreversible risk)
+✅ ALLOWED  SELECT COUNT(*) — read-only query
+✅ ALLOWED  Grant read access — ticket: JIRA-5821
+✅ ALLOWED  Read service config
+```
+
+### Framework Integration
+
+trikesh integrates with popular agent frameworks — wrap tools or entire agents:
+
+```python
+from trikesh import wrap, protect
+
+# Protect a single tool
+safe_search = wrap(agent.search_tool, guard=guard)
+
+# Protect all tools in an agent (auto-detects framework)
+from trikesh import SafetyBlockedError
+safe_agent = await protect(agent, guard=guard)
+
+# Handle blocked actions
+try:
+    result = await safe_agent.run(task)
+except SafetyBlockedError as e:
+    print(f"Action blocked: {e.violations}")
+```
+
+Supported frameworks: **Google ADK, AutoGen, LangChain**
+
+### Observability
+
+Every classifier invocation is tracked automatically:
+
+```python
+# After running checks
+metrics = guard.metrics
+
+# Get a summary
+print(metrics.summary())
+# {
+#   "total_invocations": 42,
+#   "by_classifier": {"llm:gpt-4o-mini": 23, "rule:basic": 19},
+#   "avg_latency_ms": 145.2,
+#   "confident_rate": 0.88,
+#   "timeout_rate": 0.02,
+# }
+
+# Get raw invocations for detailed analysis
+for invocation in metrics.to_list():
+    print(f"{invocation.classifier_id}: {invocation.latency_ms}ms, confidence={invocation.confidence}")
+```
+
+### Performance
+
+| Scenario | Latency |
+|---|---|
+| Clear violation caught by rules | <1ms |
+| Action evaluated by local PDP model | ~10-30ms |
+| Avg across 1000 queries (cascade) | <50ms |
+| Speculative layer (concurrent) | ~max(fastest, all_uncertain) |
+
+---
+
+## Local PDP model — saroku-guard
+
+[saroku-guard](https://huggingface.co/karanxa/saroku-guard) protects every `SafetyGuard()` by default — no setup, no API key, no data leaving your environment. It downloads automatically on first use and runs on CPU.
+
+```python
+guard = SafetyGuard()  # saroku-guard is already active
+```
+
+To use a different checkpoint, or disable it in favor of an LLM-only judge:
+
+```python
+guard = SafetyGuard(local_model_path="your-org/your-model")
+guard = SafetyGuard(use_local_pdp=False, judge_model="gpt-4o-mini")
+```
+
+### Train your own
+
+If you want to fine-tune on your own data or domain:
+
+```bash
+pip install trikesh[train]
+python -m trikesh.training.trainer --output-dir ./my-model --epochs 3
+```
+
+---
+
+## Result object
+
+```python
+result = guard.check(...)
+
+result.is_safe          # bool
+result.violations       # list of SafetyViolation
+result.latency_ms       # float
+result.layers_used      # ["rules", "ml", "local_model"]
+result.ml_risk_score    # float 0-1
+result.summary()        # human-readable string
+```
+
+Each `SafetyViolation`:
+
+```python
+v.property        # "trust_hierarchy", "minimal_footprint", etc.
+v.severity        # "high", "medium", "low"
+v.description     # what the violation is
+v.recommendation  # what to do instead
+v.source          # "rules", "ml", or "local_model"
+```
+
+---
+
+## License
+
+MIT