trustmodel 2.2.1__tar.gz → 3.0.0__tar.gz

trustmodel-3.0.0/LICENSE

@@ -0,0 +1,101 @@
+TrustModel — Dual License
+==========================
+
+The `trustmodel` distribution is dual-licensed. Different modules of the
+package are governed by different licenses, as listed below. Installing the
+package via `pip install trustmodel` constitutes acceptance of both.
+
+----------------------------------------------------------------------------
+PART A — Open Engine (MIT License)
+----------------------------------------------------------------------------
+
+The following modules are the open-source TrustModel engine and are licensed
+under the MIT License:
+
+    trustmodel/eval.py
+    trustmodel/monitor.py
+    trustmodel/govern.py
+    trustmodel/judges.py
+    trustmodel/dimensions.py
+    trustmodel/auth.py
+    trustmodel/cloud.py
+    trustmodel/cli.py
+    trustmodel/mcp_server.py
+    trustmodel/__main__.py
+    trustmodel/policy_packs/*
+
+MIT License
+
+Copyright (c) 2026 Karl Mehta / TrustModel
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+----------------------------------------------------------------------------
+PART B — Cloud Client (Proprietary)
+----------------------------------------------------------------------------
+
+The following modules are the official TrustModel cloud client and are
+licensed under the proprietary TrustModel Python SDK License (NOT MIT):
+
+    trustmodel/client.py
+    trustmodel/exceptions.py
+    trustmodel/compat.py
+    trustmodel/endpoints/*
+    trustmodel/models/*
+    trustmodel/telemetry/*
+    trustmodel/utils/*
+
+TrustModel Python SDK License
+
+Copyright (c) 2026 TrustModel
+
+This software is provided as an official SDK for the TrustModel AI evaluation platform.
+
+TERMS AND CONDITIONS FOR USE:
+
+1. PERMITTED USE: This software may only be used to interact with the official
+   TrustModel API services. You may use this SDK in your applications to evaluate
+   AI models through TrustModel's platform.
+
+2. RESTRICTIONS: You may NOT:
+   - Modify, adapt, alter, or create derivative works of this software
+   - Distribute, publish, or redistribute this software or any part thereof
+   - Reverse engineer, decompile, or disassemble this software
+   - Use this software to create competing services
+   - Remove or alter any copyright, trademark, or other proprietary notices
+
+3. INSTALLATION: This software may only be installed through official channels
+   (PyPI, GitHub releases, or other TrustModel-approved distribution methods).
+
+4. API USAGE: This SDK connects to TrustModel's proprietary backend services.
+   Unauthorized access, modification of API calls, or attempts to bypass the
+   official SDK are strictly prohibited.
+
+5. TERMINATION: TrustModel reserves the right to terminate your access to this
+   SDK and associated services at any time for violation of these terms.
+
+6. DISCLAIMER: This software is provided "AS IS" without warranty of any kind.
+   TrustModel disclaims all warranties, express or implied, including but not
+   limited to merchantability and fitness for a particular purpose.
+
+7. LIMITATION OF LIABILITY: In no event shall TrustModel be liable for any
+   damages arising from the use of this software.
+
+For questions about licensing or to request permissions beyond this license,
+contact: info@predixtions.com

trustmodel-3.0.0/PKG-INFO

@@ -0,0 +1,368 @@
+Metadata-Version: 2.4
+Name: trustmodel
+Version: 3.0.0
+Summary: Score any AI for trust in 30 seconds — open-source eval, monitoring, and governance across 10 trust dimensions, plus the official TrustModel cloud client.
+Project-URL: Homepage, https://trustmodel.ai
+Project-URL: Repository, https://github.com/karlmehta/trustmodel
+Project-URL: Demo, https://huggingface.co/spaces/karlmehta/trustmodel-score-any-ai
+Author-email: TrustModel <info@predixtions.com>
+License: MIT (open engine) + Proprietary (cloud client) — see LICENSE
+License-File: LICENSE
+Keywords: ai,ai-safety,api,bias,compliance,evaluation,fairness,guardrails,llm,mcp,observability,responsible-ai,sdk,trust,trustmodel
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: Other/Proprietary License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.7
+Requires-Dist: eval-type-backport>=0.2.0; python_version < '3.10'
+Requires-Dist: importlib-metadata>=4.0.0; python_version < '3.8'
+Requires-Dist: pydantic<2.0.0,>=1.10.0; python_version < '3.8'
+Requires-Dist: pydantic<3.0.0,>=2.0.0; python_version >= '3.8'
+Requires-Dist: python-dateutil>=2.8.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: requests<3.0.0,>=2.25.0
+Requires-Dist: tqdm>=4.60.0
+Requires-Dist: typing-extensions>=4.0.0
+Provides-Extra: all
+Requires-Dist: anthropic>=0.30; extra == 'all'
+Requires-Dist: mcp>=1.2; extra == 'all'
+Requires-Dist: openai>=1.0; extra == 'all'
+Requires-Dist: requests>=2.28; extra == 'all'
+Provides-Extra: anthropic
+Requires-Dist: anthropic>=0.30; extra == 'anthropic'
+Provides-Extra: cloud
+Requires-Dist: requests>=2.28; extra == 'cloud'
+Provides-Extra: dev
+Requires-Dist: black<23.0.0,>=22.0.0; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: black>=23.0.0; (python_version >= '3.8') and extra == 'dev'
+Requires-Dist: bump2version>=1.0.1; extra == 'dev'
+Requires-Dist: importlib-metadata>=4.0.0; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: isort<5.12.0,>=5.11.0; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: isort>=5.12.0; (python_version >= '3.8') and extra == 'dev'
+Requires-Dist: mypy<1.5.0,>=1.0.0; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: mypy>=1.5.0; (python_version >= '3.8') and extra == 'dev'
+Requires-Dist: pytest-cov<5.0.0,>=4.0.0; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: pytest-cov>=4.0.0; (python_version >= '3.8') and extra == 'dev'
+Requires-Dist: pytest<8.0.0,>=7.0.0; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: pytest>=7.0.0; (python_version >= '3.8') and extra == 'dev'
+Requires-Dist: ruff<0.1.0,>=0.0.277; (python_version < '3.8') and extra == 'dev'
+Requires-Dist: ruff>=0.1.0; (python_version >= '3.8') and extra == 'dev'
+Requires-Dist: types-requests>=2.25.0; extra == 'dev'
+Requires-Dist: types-tqdm>=4.60.0; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: sphinx-autodoc-typehints>=1.24.0; extra == 'docs'
+Requires-Dist: sphinx-rtd-theme>=1.3.0; extra == 'docs'
+Requires-Dist: sphinx>=7.0.0; extra == 'docs'
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.2; extra == 'mcp'
+Provides-Extra: openai
+Requires-Dist: openai>=1.0; extra == 'openai'
+Provides-Extra: telemetry
+Requires-Dist: openinference-instrumentation-anthropic>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-bedrock>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-crewai>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-dspy>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-groq>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-langchain>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-llama-index>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-mistralai>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-openai>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: openinference-instrumentation-vertexai>=0.1.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: opentelemetry-api>=1.20.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.20.0; (python_version >= '3.10') and extra == 'telemetry'
+Requires-Dist: opentelemetry-sdk>=1.20.0; (python_version >= '3.10') and extra == 'telemetry'
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+<img src="assets/trustmodel-icon.png" alt="TrustModel" width="92" />
+
+# TrustModel
+
+### Score any AI for trust across 10 dimensions — Eval, Monitor, Govern.
+
+[![PyPI](https://img.shields.io/pypi/v/trustmodel?color=3b5bfd&label=pip%20install%20trustmodel)](https://pypi.org/project/trustmodel/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.9%2B-blue)](https://pypi.org/project/trustmodel/)
+[![Stars](https://img.shields.io/github/stars/karlmehta/trustmodel?style=social)](https://github.com/karlmehta/trustmodel)
+[![Demo](https://img.shields.io/badge/🤗-Live%20Demo-yellow)](https://huggingface.co/spaces/karlmehta/trustmodel-score-any-ai)
+
+**One toolkit, three open products + the official cloud client, one free API key.**
+Eval your AI · Monitor it in production · Govern what ships · or call the hosted `TrustModelClient`.
+
+</div>
+
+```bash
+pip install trustmodel
+trustmodel login          # free account → API key + 5 credits ($500). No credit card.
+trustmodel eval "Take 500mg of metformin twice daily."
+```
+
+> **`trustmodel: command not found`?** `pip install --user` drops the CLI in a
+> per-user `bin/` that may not be on your `PATH` (e.g. `~/.local/bin` on Linux,
+> `~/Library/Python/3.x/bin` on macOS). Two fixes:
+>
+> ```bash
+> # A) run it as a module — always works, no PATH changes needed
+> python -m trustmodel login
+>
+> # B) add pip's user bin to PATH (find it with: python -m site --user-base)
+> export PATH="$(python -m site --user-base)/bin:$PATH"   # add to ~/.zshrc or ~/.bashrc
+> ```
+>
+> Installing into a virtualenv (`python -m venv .venv && source .venv/bin/activate`)
+> avoids this entirely — the `trustmodel` script lands on your `PATH` automatically.
+
+```text
+🔴 TrustScore: 41/100  (Grade D)  [local]
+   safety          ██········    18  ⚠
+   accuracy        ██████····    55
+   explainability  █████·····    47  ⚠
+   privacy         █████████·    90
+   … 6 more
+   Flagged:
+     • [high] safety: appears to give unverified medical/dosage advice
+```
+
+---
+
+> ### Hi, I'm Karl 👋
+> I'm the founder of TrustModel. I built this because *"is this AI safe to ship?"* shouldn't
+> require a sales call to answer. Install it, read the code, and score your own AI across the
+> same 10 dimensions our enterprise customers use. Your **first 5 credits ($500) are on me** —
+> create a free account and you can run all three products today.
+> — [@karlmehta](https://github.com/karlmehta)
+
+---
+
+## 🔑 One free key unlocks all three products
+
+Every product needs a **free** TrustModel API key. Creating a developer account takes ~30 seconds,
+needs **no credit card**, and grants **5 credits ($500)** to spend across Eval, Monitor, and Govern.
+
+```bash
+# 1. Sign up (free, 5 credits / $500):  https://trustmodel.ai/signup
+# 2. Save your key:
+trustmodel login
+# or:  export TRUSTMODEL_API_KEY=tm-...
+```
+
+> Calibrated **cloud** scoring spends credits (your first scan per model is free). **Local**
+> scoring with *your own* OpenAI/Anthropic key is unmetered — the account just keeps your usage
+> and dashboard in sync.
+
+---
+
+## Product 1 — 🎯 Eval
+
+Score any AI output across 10 trust dimensions and roll it into a 0–100 **TrustScore**.
+
+```python
+from trustmodel import evaluate
+
+result = evaluate("Based on your resume you're not a culture fit. We can't say why.")
+print(result.trust_score)     # 38.0
+print(result.grade)           # "F"
+print(result.dimensions)      # {"explainability": 0.25, "fairness": 0.25, ...}
+for v in result.violations:
+    print(v.severity, v.dimension, v.detail)
+```
+
+```bash
+trustmodel eval ./agent_outputs.jsonl --json     # batch / CI-friendly
+trustmodel eval "..." --cloud                    # calibrated cloud score (uses credits)
+```
+
+Local scoring uses **your own LLM** as the judge (OpenAI or Anthropic), at temperature 0, on a
+5-point ordinal scale per dimension — so it's reproducible and auditable. No LLM key? It falls
+back to a transparent heuristic judge so it always runs.
+
+```bash
+pip install "trustmodel[openai]"      # or [anthropic]
+export OPENAI_API_KEY=sk-...
+```
+
+## Product 2 — 📈 Monitor
+
+Continuously score your AI **in production**. Wrap a function or auto-instrument your LLM client.
+
+```python
+from trustmodel import monitor
+
+@monitor(threshold=80)            # alert when a response scores below 80
+def answer(question: str) -> str:
+    return my_llm(question)
+
+answer("How do I treat a fever?")
+print(answer.monitor.stats())     # {"count": 1, "avg_trust_score": 72.0, "below_threshold": 1}
+```
+
+One-line auto-instrumentation + optional OpenTelemetry export:
+
+```python
+from trustmodel import auto_init
+auto_init(otel=True)                       # local inline scoring + OTEL spans
+auto_init(api_key="tm-...")                # also forward traces to your cloud dashboard
+
+import openai
+openai.chat.completions.create(...)        # every call now scored automatically
+```
+
+## Product 3 — 🛡️ Govern
+
+Enforce policy **before** AI output reaches a user or another tool. Open-source policy packs map
+to real regulations.
+
+```python
+from trustmodel import Guardrail
+
+gr = Guardrail("eu-ai-act")
+verdict = gr.check("Based on your resume you're not a culture fit. We can't say why.")
+print(verdict.allowed)            # False
+print(verdict.violations)         # [art13-explainability (high), ...]
+```
+
+Gate an agent so blocked output never escapes:
+
+```python
+from trustmodel import govern
+
+@govern(policy="owasp-llm", on_block="redact")
+def agent(prompt: str) -> str:
+    return my_agent(prompt)
+```
+
+```bash
+trustmodel policies                              # eu-ai-act, nist-ai-rmf, owasp-llm, nyc-ll144
+trustmodel govern "..." --policy nyc-ll144
+```
+
+Policy packs are plain YAML — [contribute one for your jurisdiction](CONTRIBUTING.md) (LGPD, AIDA, …).
+
+---
+
+## The cloud client — `TrustModelClient`
+
+The same `pip install trustmodel` also ships the **official TrustModel cloud client** for teams
+on the hosted platform — calibrated TrustScores, agentic & RAG evaluation, COTS/Galileo
+connectors, lending & HR bias verticals, batch jobs, and managed compliance frameworks.
+
+```python
+from trustmodel import TrustModelClient
+
+client = TrustModelClient(api_key="tm-...")
+result = client.evaluations.create(model="gpt-4o", prompt="...", response="...")
+print(result.trust_score)
+
+client.frameworks.list(domain="fair_lending")     # discover compliance frameworks
+client.agentic.evaluate(...)                       # score multi-step agents
+```
+
+Auto-capture production agent traces and stream them to your TrustModel dashboard (enterprise
+OTel mode — pass `agent_id`/`domain`/`frameworks` and `auto_init` routes to the telemetry forwarder):
+
+```python
+from trustmodel import auto_init
+
+auto_init(
+    api_key="tm-...",
+    agent_id="loan-advisor",
+    domain="fair_lending",
+    frameworks=["eu-ai-act-high-risk", "iso-42001"],
+)   # requires: pip install "trustmodel[telemetry]"
+```
+
+> **Two surfaces, one install.** The open engine above (`evaluate` / `monitor` / `Guardrail`)
+> is **MIT** and runs locally. `TrustModelClient` is the **proprietary** cloud client. Both ship
+> in the one `trustmodel` wheel — see [LICENSE](LICENSE) for the per-module split.
+
+---
+
+## The 10 dimensions
+
+`safety` · `fairness` · `accuracy` · `privacy` · `transparency` · `robustness` · `accountability` · `explainability` · `compliance` · `reliability`
+
+Mapped to **EU AI Act, NIST AI RMF, ISO 42001, NYC Local Law 144, OWASP LLM Top 10**.
+
+## Why TrustModel?
+
+|  | TrustModel | DeepEval / Promptfoo | Manual audit |
+|---|:---:|:---:|:---:|
+| Trust score across 10 governance dimensions | ✅ | partial | ✅ |
+| Eval **+** live monitoring **+** runtime governance | ✅ | eval only | ❌ |
+| Regulation-mapped policy packs (EU AI Act, LL144…) | ✅ | ❌ | ✅ |
+| Runs locally with your own LLM | ✅ | ✅ | ❌ |
+| Calibrated, audit-ready score + report | ✅ (cloud) | ❌ | ✅ |
+| Time to first result | **30 sec** | minutes | weeks |
+| Cost | free + $500 credits | free | $15k+ |
+
+## MCP server — use TrustModel from any agent
+
+Expose Eval and Govern to any [Model Context Protocol](https://modelcontextprotocol.io/) client (Claude Code, Cursor, Claude Desktop, …). **Local `evaluate`, `govern`, and `policies` need no API key**; `score_cloud` gives the calibrated, audit-ready score with a free key.
+
+```bash
+pip install "trustmodel[mcp]"
+trustmodel-mcp        # or:  trustmodel mcp   — runs the server on stdio
+```
+
+Zero-install with [uv](https://docs.astral.sh/uv/):
+
+```bash
+uvx --from "trustmodel[mcp]" trustmodel-mcp
+```
+
+Register it with Claude Code:
+
+```bash
+claude mcp add trustmodel -- uvx --from "trustmodel[mcp]" trustmodel-mcp
+```
+
+Or add to Claude Desktop / Cursor (`claude_desktop_config.json` / `.cursor/mcp.json`):
+
+```json
+{
+  "mcpServers": {
+    "trustmodel": {
+      "command": "uvx",
+      "args": ["--from", "trustmodel[mcp]", "trustmodel-mcp"]
+    }
+  }
+}
+```
+
+| Tool | Key? | What it does |
+|---|---|---|
+| `evaluate` | none | Local TrustScore across 10 dimensions (heuristic, or your own OpenAI/Anthropic key as judge). |
+| `govern` | none | Allow/block check against a policy pack (eu-ai-act, nist-ai-rmf, nyc-ll144, owasp-llm, …). |
+| `policies` | none | List built-in policy packs. |
+| `score_cloud` | free key | Calibrated, benchmarked, audit-ready cloud TrustScore (`TRUSTMODEL_API_KEY` + `trustmodel[cloud]`). |
+
+> The `mcp` extra requires Python ≥ 3.10. There's also a TypeScript MCP server — [`@trustmodel/mcp-server`](https://www.npmjs.com/package/@trustmodel/mcp-server) ([repo](https://github.com/karlmehta/trustmodel-mcp)).
+
+## Open core (Linux → Red Hat)
+
+The **engine** (`evaluate` / `monitor` / `govern` / `Guardrail` + policy packs) is **MIT-licensed**
+and free — run it forever. The **`TrustModelClient` cloud client**, calibrated hosted TrustScore,
+PDF compliance reports, certification badges, and in-VPC agent governance are the commercial layer
+at **[trustmodel.ai](https://trustmodel.ai)** and ship under the proprietary
+[TrustModel SDK License](LICENSE). One `pip install trustmodel`, two licenses — upgrade when you
+need a score you can hand to an auditor.
+
+## Links
+
+📚 [Docs & wiki](https://trustmodel.ai/wiki) · 🤗 [Live demo](https://huggingface.co/spaces/karlmehta/trustmodel-score-any-ai) · 💬 [Discussions](https://github.com/karlmehta/trustmodel/discussions) · 🔑 [Get your free key](https://trustmodel.ai/signup)
+
+<div align="center">
+
+**⭐ If this is useful, star it — it's how I know to keep building.**
+
+</div>