rti-ethics-engine 2.0.1__tar.gz → 3.0.0__tar.gz

{rti_ethics_engine-2.0.1 → rti_ethics_engine-3.0.0}/.gitignore

@@ -14,8 +14,8 @@ dist/
 downloads/
 eggs/
 .eggs/
-lib/
-lib64/
+/lib/
+/lib64/
 parts/
 sdist/
 var/
@@ -205,3 +205,7 @@ cython_debug/
 marimo/_static/
 marimo/_lsp/
 __marimo__/
+
+
+# v3 evidence store (generated)
+backend/data/ethics_evidence.db*

rti_ethics_engine-3.0.0/CHANGELOG.md

@@ -0,0 +1,31 @@
+# Changelog
+
+## 3.0.0
+
+A rewrite around one rule: **no metric is ever defaulted.** A value that cannot be measured is `None`, is listed in
+`result.unmeasured`, and lowers `result.coverage`; below `min_coverage` the verdict is `INSUFFICIENT_EVIDENCE`.
+
+### Added
+- `Evaluator`: interaction in, full 9-dimension evaluation out. Logs to a tamper-evident hash chain.
+- `Interaction` / `Chunk` / `GenResult` evidence types as the only data contract.
+- `DomainProfile` (+ `profiles.rti_profile`): all domain knowledge is configuration; the core is generic.
+- `Auditor`: active audits (security canary attacks, paired-prompt fairness, gold-set performance, membership
+  inference, poisoning detection, counterfactual and explanation stability).
+- `Gatekeeper`: ACTIVE / QUARANTINED / PROBATION state machine, automatic blocking, human re-admission, routing.
+- `Judge`: optional LLM judge (on by default, disable with `judge=None`), structured, cached, self-grading-safe.
+- `GpuMeter`: measured GPU energy and utilisation.
+- `SQLiteStore`: persistent evidence, baselines, audit results, gate state.
+- All 44 formulas implemented as pure functions returning `None` when undefined.
+- Statistical corrections: PSI from baseline quantiles, KS critical value for sample size, Wilson/bootstrap intervals.
+
+### Changed (breaking)
+- Weights are normalised to sum to 1 (they summed to 1.24).
+- Thresholds follow `formulas.md` (SPD 0.1, PSI 0.2, accuracy 0.8, ...), not the 0.5 placeholders of 2.x.
+- Layer-3 explanations are generated from real values instead of canned text.
+- Environmental risk no longer saturates (compute intensity is `2 x params x tokens / output_tokens`).
+- `EthicsEngine.run_protocol` accepts `None` metrics and returns `coverage` / `unmeasured`.
+
+### Removed
+- `build_metrics_from_raw_data`, `build_default_metrics` and the `compute_*_from_data` helpers, which filled in
+  constants (confidence 0.9, TCR 1.0, ALC 1.0, ...). Use `Evaluator` instead.
+- `MetricCalculator` and the dynamic-config loader.

rti_ethics_engine-3.0.0/PKG-INFO

@@ -0,0 +1,213 @@
+Metadata-Version: 2.5
+Name: rti-ethics-engine
+Version: 3.0.0
+Summary: Measured, domain-pluggable ethics monitoring for LLM systems: 9 dimensions, 44 formulas, drift tracking and automatic model gating. No defaulted metrics.
+Project-URL: Homepage, https://github.com/Krishil-Parikh/bazinga_tsec_hacks
+Project-URL: Repository, https://github.com/Krishil-Parikh/bazinga_tsec_hacks
+Project-URL: Issues, https://github.com/Krishil-Parikh/bazinga_tsec_hacks/issues
+Author: RTI Ethics Engine Team
+License: MIT License
+        
+        Copyright (c) 2026 RTI Ethics Engine Team
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: ai-ethics,audit,compliance,drift,eu-ai-act,fairness,guardrails,llm,monitoring,rag,rti
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: scipy>=1.10.0
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: embeddings
+Requires-Dist: sentence-transformers>=2.2; extra == 'embeddings'
+Description-Content-Type: text/markdown
+
+# rti-ethics-engine
+
+**Measured, domain-pluggable ethics monitoring for LLM systems.**
+
+Guardrails judge one request at a time. This package watches a model *over time*: it scores every answer on nine ethical dimensions, tracks drift against a frozen baseline, probes the model actively (prompt injection, fairness, leakage, poisoning), and can **automatically block a model that crosses a threshold** until it is fixed and a human re-admits it.
+
+Built for government chatbots (RTI/FOI) and for any company that serves several LLMs and is accountable for what they say. Nothing in the core is tied to a domain, a vendor or a dataset.
+
+```bash
+pip install rti-ethics-engine                 # numpy + scipy only
+pip install "rti-ethics-engine[embeddings]"   # optional: sentence-transformers embeddings
+```
+
+## The one rule: nothing is defaulted
+
+Every metric is computed from evidence you hand over. If a metric cannot be measured, it is `None`, it is listed under `result.unmeasured`, and it lowers `result.coverage`. The engine will not fill in a "passing" value, and it will not issue a verdict (`INSUFFICIENT_EVIDENCE` instead) until enough of the weight is measured.
+
+## 60-second quick start
+
+```python
+from rti_ethics_engine import Evaluator, Interaction, Chunk, SQLiteStore
+
+ev = Evaluator(store=SQLiteStore("ethics.db"), judge=None)   # judge=None: no LLM judge
+
+ev.begin_request("my-model")                                  # at API ingress (enables audit-log completeness)
+answer = my_model(question)                                   # your code
+out = ev.evaluate(Interaction(
+    query=question, response=answer, model_id="my-model",
+    context=[Chunk(text=c) for c in retrieved_passages],      # RAG context, if any
+    latency_ms=812, input_tokens=420, output_tokens=96,
+))
+
+print(out.result.final_label, out.result.s_llm, out.result.coverage)
+print(out.result.unmeasured)          # what could not be measured, and why that matters
+out.result.print_full_audit()
+```
+
+Only `query` and `response` are required. Every extra field you provide (token log-probs, GPU energy, session ids, retrieved context, ground-truth labels, groups) unlocks more metrics.
+
+## Core objects
+
+| Object | Role |
+|---|---|
+| `Interaction` | One observed request/response. The only data contract. |
+| `DomainProfile` | Everything domain-specific: refusal wording, legal citations, PII patterns, SLA, prices. A generic default ships; `profiles.rti_profile()` is a worked example. |
+| `Evaluator` | Logs an interaction, measures all nine dimensions, runs the 3-layer protocol, returns an `Evaluation`. |
+| `Auditor` (`ev.auditor`) | Active probes: security, fairness pairs, gold-set performance, membership leakage, poisoning detection, explainability stability. |
+| `Gatekeeper` | Per-model ACTIVE / QUARANTINED / PROBATION state machine with human re-admission. |
+| `Judge` | Optional LLM judge for semantic checks. Pluggable, and fully disableable. |
+| `SQLiteStore` | Evidence store with a hash-chained, tamper-evident audit log. |
+| `EthicsEngine` | The pure scoring protocol, if you want to feed it metrics yourself. |
+
+## How each dimension is measured
+
+| Dimension | Metrics | Measured from |
+|---|---|---|
+| Fairness | SPD, DI, EOP, AOD, FNRD, refusal disparity, MI leakage, predictive parity | Decisions by group in live traffic and in paired-prompt audits; worst pairwise gap when there are >2 groups |
+| Drift | PSI, KS, prediction/confidence/embedding drift | Frozen baseline vs recent window; PSI bins from baseline quantiles; KS threshold corrected for sample size |
+| Environmental & cost | Cost, GPU-hours, compute intensity, cost-utility, CO2 | Tokens x price, or measured GPU energy (`GpuMeter`) x tariff; FLOPs ~ 2 x params x tokens |
+| Explainability | Confidence, feature importance, counterfactual distance, explanation stability | Token log-probs (or self-consistency); retrieval attribution (or judge claim support); probe audits |
+| UX | Task completion, frustration, dialog turns, cognitive load, readability | Sessions: reformulation = unresolved; Flesch reading ease |
+| Legal | Lawfulness, consent validity, proportionality, DPIA coverage | Profile citation rules; PII vs personal-data requests; requested-vs-covered items; register of checks that actually ran |
+| Performance | Accuracy, recall, precision, F1, group F1, long-tail accuracy | Your labelled gold set via `Auditor.run_gold` |
+| Monitoring | Audit-log completeness, SLA, policy violations, incident reconstruction | Requests seen vs hash-chain-verified records; latency vs SLA; profile policy rules |
+| Security | Adversarial accuracy, injection success, membership inference, poisoning detection | `Auditor.run_security` (canary attacks + your attacks), membership and poisoning audits |
+
+The 3-layer protocol: **L1** hard gates (PASS/FAIL), **L2** risk in [0, 1] (a failed gate forces 1.0), **L3** forensic evidence generated from the real values. `S_LLM = 1 - sum(w_d * risk_d) / sum(w_d)` over the dimensions that have data. Thresholds and weights are data (`config.METRIC_SPECS`) and can be overridden per profile, JSON file or `RTI_*` environment variables.
+
+## Make it yours: a domain profile
+
+```python
+from rti_ethics_engine import DomainProfile, CategoryRule, Evaluator
+
+bank = DomainProfile(
+    name="retail-banking",
+    refusal_patterns=[r"\bcannot (?:share|disclose|help)\b", r"\bunable to assist\b"],
+    pii_patterns={"iban": r"\b[A-Z]{2}\d{2}[A-Z0-9]{11,30}\b", "email": r"\b[\w.+-]+@[\w-]+\.\w+\b"},
+    required_elements={"disclaimer": r"not financial advice"},
+    category_rules=[CategoryRule("account_data", [r"\bbalance of\b.*\bother customer\b"], expect_refusal=True)],
+    sla_ms=4000,
+    pricing={"gpt-x": (0.0005, 0.0015)},           # USD per 1k tokens (in, out)
+    thresholds={"fairness.spd": 0.05},             # tighten a gate
+    weights={"security": 0.30},                    # re-weight a dimension
+)
+ev = Evaluator(profile=bank, judge=None)
+```
+
+## Active audits
+
+```python
+from rti_ethics_engine import GoldExample, PairedPrompt, default_attacks
+
+def model(query, context=None):                    # your whole system as a callable
+    return answer_with_rag(query, context)
+
+aud = ev.auditor
+aud.run_security("my-model", model, benign=["How do I reset my PIN?"])          # PISR, adversarial accuracy
+aud.run_gold("my-model", model, [GoldExample("Close my account", expected_decision=1)])
+aud.run_fairness_pairs("my-model", model, [PairedPrompt("p1", {"A": "...", "B": "..."}, expected_decision=1)])
+aud.run_membership("my-model", model, members=[...], nonmembers=[...])          # leakage
+aud.run_poisoning(corpus_docs)                                                  # ingestion detector PDR
+```
+
+Fresh audit results (default: younger than 7 days) are folded into every evaluation automatically.
+
+## Gate models in production
+
+```python
+from rti_ethics_engine import Gatekeeper, GatePolicy
+
+gate = Gatekeeper(ev.store, GatePolicy(min_obs=10, block_risk=0.55))
+model = gate.select(["model-a", "model-b"], preferred="model-a")   # routes around blocked models
+...
+out = ev.evaluate(interaction)
+gate.observe(out)                                  # may quarantine automatically
+
+gate.request_readmission("model-a", by="alice")    # -> PROBATION: send shadow traffic
+gate.probation_report("model-a")
+gate.approve("model-a", approver="alice")          # human sign-off; refuses if evidence is not clean
+```
+
+Blocking is automatic; re-admission requires a named human and clean probation evidence. Re-admission thresholds are stricter than blocking thresholds, so a borderline model cannot flap.
+
+## The LLM judge (optional)
+
+Some failures need meaning, not arithmetic: a paraphrased hallucination, a response that contradicts its sources, a legal exemption that does not apply to the question. `Judge` adds those checks with one structured JSON call per interaction at temperature 0, counts claims instead of free-form scoring, never grades its own model, caches by content hash, and returns `None` (not a made-up value) if it fails.
+
+```python
+Evaluator()                                         # default: judge from RTI_JUDGE_* env vars (Ollama, qwen3:8b)
+Evaluator(judge=Judge(my_llm_callable, "judge-model"))
+Evaluator(judge=None)                               # off: deterministic proxies are used where an honest one exists,
+                                                    # judge-only metrics (claim-level faithfulness) are skipped
+```
+
+Environment: `RTI_JUDGE_BACKEND=ollama|openai|none`, `RTI_JUDGE_MODEL`, `RTI_JUDGE_HOST`. Use a judge from a different model family than the models you audit.
+
+## Measured hardware
+
+`GpuMeter` samples `nvidia-smi` during inference and returns `energy_kwh`, `gpu_util`, `power_w`; put them in `Interaction.metadata`. Without a GPU, energy-based metrics stay unmeasured unless you declare `DomainProfile.default_power_w`.
+
+```python
+meter = GpuMeter()
+with meter:
+    text = model(query)
+interaction.metadata.update(meter.result())
+```
+
+## Audit trail
+
+`SQLiteStore` hashes every record into a chain. `store.verify_chain()` detects any edit or deletion; `chain_stats_since()` underlies the audit-log-completeness and incident-reconstruction metrics. `begin_request()` at ingress makes lost records visible.
+
+## Honest limitations
+
+- Fairness needs groups. Supply `Interaction.group`, a `group_fn` proxy in your profile, or run paired-prompt audits.
+- Thresholds are defaults from the reference formulas (`DOCUMENTATION/formulas.md`); tune them for your domain.
+- The LLM judge is an LLM: it can be wrong. Calibrate it against labelled cases before trusting its verdicts.
+- Membership inference and poisoning audits measure the whole system you wrap; interpret them against how sensitive your corpus is.
+- Monitoring does not transfer legal liability. It produces the evidence of reasonable care.
+
+MIT licensed.

rti_ethics_engine-3.0.0/README.md

@@ -0,0 +1,160 @@
+# rti-ethics-engine
+
+**Measured, domain-pluggable ethics monitoring for LLM systems.**
+
+Guardrails judge one request at a time. This package watches a model *over time*: it scores every answer on nine ethical dimensions, tracks drift against a frozen baseline, probes the model actively (prompt injection, fairness, leakage, poisoning), and can **automatically block a model that crosses a threshold** until it is fixed and a human re-admits it.
+
+Built for government chatbots (RTI/FOI) and for any company that serves several LLMs and is accountable for what they say. Nothing in the core is tied to a domain, a vendor or a dataset.
+
+```bash
+pip install rti-ethics-engine                 # numpy + scipy only
+pip install "rti-ethics-engine[embeddings]"   # optional: sentence-transformers embeddings
+```
+
+## The one rule: nothing is defaulted
+
+Every metric is computed from evidence you hand over. If a metric cannot be measured, it is `None`, it is listed under `result.unmeasured`, and it lowers `result.coverage`. The engine will not fill in a "passing" value, and it will not issue a verdict (`INSUFFICIENT_EVIDENCE` instead) until enough of the weight is measured.
+
+## 60-second quick start
+
+```python
+from rti_ethics_engine import Evaluator, Interaction, Chunk, SQLiteStore
+
+ev = Evaluator(store=SQLiteStore("ethics.db"), judge=None)   # judge=None: no LLM judge
+
+ev.begin_request("my-model")                                  # at API ingress (enables audit-log completeness)
+answer = my_model(question)                                   # your code
+out = ev.evaluate(Interaction(
+    query=question, response=answer, model_id="my-model",
+    context=[Chunk(text=c) for c in retrieved_passages],      # RAG context, if any
+    latency_ms=812, input_tokens=420, output_tokens=96,
+))
+
+print(out.result.final_label, out.result.s_llm, out.result.coverage)
+print(out.result.unmeasured)          # what could not be measured, and why that matters
+out.result.print_full_audit()
+```
+
+Only `query` and `response` are required. Every extra field you provide (token log-probs, GPU energy, session ids, retrieved context, ground-truth labels, groups) unlocks more metrics.
+
+## Core objects
+
+| Object | Role |
+|---|---|
+| `Interaction` | One observed request/response. The only data contract. |
+| `DomainProfile` | Everything domain-specific: refusal wording, legal citations, PII patterns, SLA, prices. A generic default ships; `profiles.rti_profile()` is a worked example. |
+| `Evaluator` | Logs an interaction, measures all nine dimensions, runs the 3-layer protocol, returns an `Evaluation`. |
+| `Auditor` (`ev.auditor`) | Active probes: security, fairness pairs, gold-set performance, membership leakage, poisoning detection, explainability stability. |
+| `Gatekeeper` | Per-model ACTIVE / QUARANTINED / PROBATION state machine with human re-admission. |
+| `Judge` | Optional LLM judge for semantic checks. Pluggable, and fully disableable. |
+| `SQLiteStore` | Evidence store with a hash-chained, tamper-evident audit log. |
+| `EthicsEngine` | The pure scoring protocol, if you want to feed it metrics yourself. |
+
+## How each dimension is measured
+
+| Dimension | Metrics | Measured from |
+|---|---|---|
+| Fairness | SPD, DI, EOP, AOD, FNRD, refusal disparity, MI leakage, predictive parity | Decisions by group in live traffic and in paired-prompt audits; worst pairwise gap when there are >2 groups |
+| Drift | PSI, KS, prediction/confidence/embedding drift | Frozen baseline vs recent window; PSI bins from baseline quantiles; KS threshold corrected for sample size |
+| Environmental & cost | Cost, GPU-hours, compute intensity, cost-utility, CO2 | Tokens x price, or measured GPU energy (`GpuMeter`) x tariff; FLOPs ~ 2 x params x tokens |
+| Explainability | Confidence, feature importance, counterfactual distance, explanation stability | Token log-probs (or self-consistency); retrieval attribution (or judge claim support); probe audits |
+| UX | Task completion, frustration, dialog turns, cognitive load, readability | Sessions: reformulation = unresolved; Flesch reading ease |
+| Legal | Lawfulness, consent validity, proportionality, DPIA coverage | Profile citation rules; PII vs personal-data requests; requested-vs-covered items; register of checks that actually ran |
+| Performance | Accuracy, recall, precision, F1, group F1, long-tail accuracy | Your labelled gold set via `Auditor.run_gold` |
+| Monitoring | Audit-log completeness, SLA, policy violations, incident reconstruction | Requests seen vs hash-chain-verified records; latency vs SLA; profile policy rules |
+| Security | Adversarial accuracy, injection success, membership inference, poisoning detection | `Auditor.run_security` (canary attacks + your attacks), membership and poisoning audits |
+
+The 3-layer protocol: **L1** hard gates (PASS/FAIL), **L2** risk in [0, 1] (a failed gate forces 1.0), **L3** forensic evidence generated from the real values. `S_LLM = 1 - sum(w_d * risk_d) / sum(w_d)` over the dimensions that have data. Thresholds and weights are data (`config.METRIC_SPECS`) and can be overridden per profile, JSON file or `RTI_*` environment variables.
+
+## Make it yours: a domain profile
+
+```python
+from rti_ethics_engine import DomainProfile, CategoryRule, Evaluator
+
+bank = DomainProfile(
+    name="retail-banking",
+    refusal_patterns=[r"\bcannot (?:share|disclose|help)\b", r"\bunable to assist\b"],
+    pii_patterns={"iban": r"\b[A-Z]{2}\d{2}[A-Z0-9]{11,30}\b", "email": r"\b[\w.+-]+@[\w-]+\.\w+\b"},
+    required_elements={"disclaimer": r"not financial advice"},
+    category_rules=[CategoryRule("account_data", [r"\bbalance of\b.*\bother customer\b"], expect_refusal=True)],
+    sla_ms=4000,
+    pricing={"gpt-x": (0.0005, 0.0015)},           # USD per 1k tokens (in, out)
+    thresholds={"fairness.spd": 0.05},             # tighten a gate
+    weights={"security": 0.30},                    # re-weight a dimension
+)
+ev = Evaluator(profile=bank, judge=None)
+```
+
+## Active audits
+
+```python
+from rti_ethics_engine import GoldExample, PairedPrompt, default_attacks
+
+def model(query, context=None):                    # your whole system as a callable
+    return answer_with_rag(query, context)
+
+aud = ev.auditor
+aud.run_security("my-model", model, benign=["How do I reset my PIN?"])          # PISR, adversarial accuracy
+aud.run_gold("my-model", model, [GoldExample("Close my account", expected_decision=1)])
+aud.run_fairness_pairs("my-model", model, [PairedPrompt("p1", {"A": "...", "B": "..."}, expected_decision=1)])
+aud.run_membership("my-model", model, members=[...], nonmembers=[...])          # leakage
+aud.run_poisoning(corpus_docs)                                                  # ingestion detector PDR
+```
+
+Fresh audit results (default: younger than 7 days) are folded into every evaluation automatically.
+
+## Gate models in production
+
+```python
+from rti_ethics_engine import Gatekeeper, GatePolicy
+
+gate = Gatekeeper(ev.store, GatePolicy(min_obs=10, block_risk=0.55))
+model = gate.select(["model-a", "model-b"], preferred="model-a")   # routes around blocked models
+...
+out = ev.evaluate(interaction)
+gate.observe(out)                                  # may quarantine automatically
+
+gate.request_readmission("model-a", by="alice")    # -> PROBATION: send shadow traffic
+gate.probation_report("model-a")
+gate.approve("model-a", approver="alice")          # human sign-off; refuses if evidence is not clean
+```
+
+Blocking is automatic; re-admission requires a named human and clean probation evidence. Re-admission thresholds are stricter than blocking thresholds, so a borderline model cannot flap.
+
+## The LLM judge (optional)
+
+Some failures need meaning, not arithmetic: a paraphrased hallucination, a response that contradicts its sources, a legal exemption that does not apply to the question. `Judge` adds those checks with one structured JSON call per interaction at temperature 0, counts claims instead of free-form scoring, never grades its own model, caches by content hash, and returns `None` (not a made-up value) if it fails.
+
+```python
+Evaluator()                                         # default: judge from RTI_JUDGE_* env vars (Ollama, qwen3:8b)
+Evaluator(judge=Judge(my_llm_callable, "judge-model"))
+Evaluator(judge=None)                               # off: deterministic proxies are used where an honest one exists,
+                                                    # judge-only metrics (claim-level faithfulness) are skipped
+```
+
+Environment: `RTI_JUDGE_BACKEND=ollama|openai|none`, `RTI_JUDGE_MODEL`, `RTI_JUDGE_HOST`. Use a judge from a different model family than the models you audit.
+
+## Measured hardware
+
+`GpuMeter` samples `nvidia-smi` during inference and returns `energy_kwh`, `gpu_util`, `power_w`; put them in `Interaction.metadata`. Without a GPU, energy-based metrics stay unmeasured unless you declare `DomainProfile.default_power_w`.
+
+```python
+meter = GpuMeter()
+with meter:
+    text = model(query)
+interaction.metadata.update(meter.result())
+```
+
+## Audit trail
+
+`SQLiteStore` hashes every record into a chain. `store.verify_chain()` detects any edit or deletion; `chain_stats_since()` underlies the audit-log-completeness and incident-reconstruction metrics. `begin_request()` at ingress makes lost records visible.
+
+## Honest limitations
+
+- Fairness needs groups. Supply `Interaction.group`, a `group_fn` proxy in your profile, or run paired-prompt audits.
+- Thresholds are defaults from the reference formulas (`DOCUMENTATION/formulas.md`); tune them for your domain.
+- The LLM judge is an LLM: it can be wrong. Calibrate it against labelled cases before trusting its verdicts.
+- Membership inference and poisoning audits measure the whole system you wrap; interpret them against how sensitive your corpus is.
+- Monitoring does not transfer legal liability. It produces the evidence of reasonable care.
+
+MIT licensed.

{rti_ethics_engine-2.0.1 → rti_ethics_engine-3.0.0}/pyproject.toml

@@ -1,93 +1,74 @@
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"
-
-[project]
-name = "rti-ethics-engine"
-version = "2.0.1"
-description = "India's first granular AI ethics framework for RTI applications (36 metrics, 9 dimensions)"
-readme = "README.md"
-license = {file = "LICENSE"}
-requires-python = ">=3.9"
-authors = [
-    {name = "RTI Ethics Engine Team"}
-]
-keywords = [
-    "ai-ethics",
-    "rti",
-    "india",
-    "government",
-    "compliance",
-    "fairness",
-    "audit",
-    "llm"
-]
-classifiers = [
-    "Development Status :: 5 - Production/Stable",
-    "Intended Audience :: Developers",
-    "Intended Audience :: Science/Research",
-    "License :: OSI Approved :: MIT License",
-    "Operating System :: OS Independent",
-    "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.9",
-    "Programming Language :: Python :: 3.10",
-    "Programming Language :: Python :: 3.11",
-    "Programming Language :: Python :: 3.12",
-    "Topic :: Scientific/Engineering :: Artificial Intelligence",
-    "Topic :: Software Development :: Libraries :: Python Modules"
-]
-dependencies = [
-    "numpy>=1.24.0",
-    "scipy>=1.10.0"
-]
-
-[project.optional-dependencies]
-dev = [
-    "pytest>=7.0",
-    "pytest-cov>=4.0",
-    "black>=23.0",
-    "mypy>=1.0",
-    "ruff>=0.1.0"
-]
-docs = [
-    "sphinx>=5.0",
-    "sphinx-rtd-theme>=1.0"
-]
-
-[project.urls]
-Homepage = "https://github.com/rti-ethics/rti-ethics-engine"
-Documentation = "https://rti-ethics-engine.readthedocs.io"
-Repository = "https://github.com/rti-ethics/rti-ethics-engine"
-Issues = "https://github.com/rti-ethics/rti-ethics-engine/issues"
-
-[tool.hatch.build.targets.wheel]
-packages = ["rti_ethics_engine"]
-include = ["INPUTS_OUTPUTS.md"]
-
-[tool.hatch.build.targets.sdist]
-include = [
-    "/rti_ethics_engine",
-    "/tests",
-    "/README.md",
-    "/INPUTS_OUTPUTS.md",
-    "/LICENSE"
-]
-
-[tool.pytest.ini_options]
-testpaths = ["tests"]
-python_files = ["test_*.py"]
-addopts = "-v --tb=short"
-
-[tool.black]
-line-length = 100
-target-version = ["py39", "py310", "py311"]
-
-[tool.ruff]
-line-length = 100
-select = ["E", "F", "W", "I", "N"]
-
-[tool.mypy]
-python_version = "3.9"
-warn_return_any = true
-warn_unused_configs = true
-disallow_untyped_defs = true
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "rti-ethics-engine"
+version = "3.0.0"
+description = "Measured, domain-pluggable ethics monitoring for LLM systems: 9 dimensions, 44 formulas, drift tracking and automatic model gating. No defaulted metrics."
+readme = "README.md"
+license = {file = "LICENSE"}
+requires-python = ">=3.9"
+authors = [
+    {name = "RTI Ethics Engine Team"}
+]
+keywords = [
+    "ai-ethics",
+    "llm",
+    "monitoring",
+    "drift",
+    "fairness",
+    "guardrails",
+    "audit",
+    "compliance",
+    "eu-ai-act",
+    "rag",
+    "rti"
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries :: Python Modules"
+]
+dependencies = [
+    "numpy>=1.24.0",
+    "scipy>=1.10.0"
+]
+
+[project.optional-dependencies]
+embeddings = ["sentence-transformers>=2.2"]
+dev = [
+    "pytest>=7.0",
+    "pytest-cov>=4.0"
+]
+
+[project.urls]
+Homepage = "https://github.com/Krishil-Parikh/bazinga_tsec_hacks"
+Repository = "https://github.com/Krishil-Parikh/bazinga_tsec_hacks"
+Issues = "https://github.com/Krishil-Parikh/bazinga_tsec_hacks/issues"
+
+[tool.hatch.build.targets.wheel]
+packages = ["rti_ethics_engine"]
+
+[tool.hatch.build.targets.sdist]
+include = [
+    "/rti_ethics_engine",
+    "/tests",
+    "/README.md",
+    "/CHANGELOG.md",
+    "/LICENSE"
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+python_files = ["test_*.py"]
+addopts = "-v --tb=short"

rti_ethics_engine-3.0.0/rti_ethics_engine/__init__.py

@@ -0,0 +1,48 @@
+"""
+RTI Ethics Engine v3 - measured, domain-pluggable ethics monitoring for LLM systems.
+
+Nine dimensions, 44 formulas, every value computed from evidence: nothing is
+defaulted. A metric that cannot be measured is reported as unmeasured (None)
+and lowers coverage instead of silently passing.
+
+Quick start:
+
+    from rti_ethics_engine import Evaluator, Interaction, SQLiteStore
+    from rti_ethics_engine.profiles import rti_profile
+
+    ev = Evaluator(profile=rti_profile(), store=SQLiteStore("ethics.db"))
+    ev.begin_request("llama3")
+    out = ev.evaluate(Interaction(query="...", response="...", model_id="llama3", latency_ms=812))
+    print(out.result.final_label, out.result.s_llm, out.result.coverage)
+
+Run active audits (security, fairness pairs, gold-set performance, ...) with
+ev.auditor, and gate models in production with Gatekeeper. See README.md.
+"""
+
+from .calculators import *  # noqa: F401,F403  (the 44 formulas)
+from .config import (
+    DEFAULT_WEIGHTS, DIMENSION_INFO, DIMENSION_WEIGHTS, METRIC_CATALOG, METRIC_SPECS, RISK_THRESHOLDS,
+    EngineConfig, MetricSpec, build_config,
+)
+from .core import EthicsEngine
+from .evaluator import Evaluation, Evaluator
+from .gatekeeper import ACTIVE, PROBATION, QUARANTINED, GatePolicy, Gatekeeper
+from .judge import Judge, default_judge, ollama_llm, openai_compatible_llm
+from .meters import GpuMeter
+from .probes import (
+    Attack, Auditor, GoldExample, PairedPrompt, PoisonDetector, call_model, default_attacks,
+)
+from .profile import CategoryRule, DomainProfile
+from .results import DimensionResult, EthicsResult, FinalLabel, RiskBand
+from .store import SQLiteStore
+from .textkit import HashingEmbedder, SentenceTransformerEmbedder
+from .types import Chunk, GenResult, Interaction
+
+__version__ = "3.0.0"
+__author__ = "RTI Ethics Engine Team"
+__license__ = "MIT"
+
+
+def evaluate(metrics: dict, profile: DomainProfile = None) -> EthicsResult:
+    """Low-level entry point: run the 3-layer protocol on a ready-made metrics dict."""
+    return EthicsEngine(profile).run_protocol(metrics)