action-sentinel 0.2.0__tar.gz

action_sentinel-0.2.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

action_sentinel-0.2.0/MANIFEST.in

@@ -0,0 +1,5 @@
+# The source distribution carries the files setup.py bundles into the wheel.
+include config/*.yaml
+include examples/rule_packs/*.yaml
+include examples/evaluation_datasets/*.jsonl
+include examples/sample_actions/*.json*

action_sentinel-0.2.0/PKG-INFO

@@ -0,0 +1,384 @@
+Metadata-Version: 2.4
+Name: action-sentinel
+Version: 0.2.0
+Summary: VegaML-powered safety checkpoint for AI agent actions: model risk judgement under deterministic governance.
+Author: Prem Kumar
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/premm25/action-sentinel
+Project-URL: Documentation, https://github.com/premm25/action-sentinel/tree/main/docs
+Project-URL: Changelog, https://github.com/premm25/action-sentinel/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/premm25/action-sentinel/issues
+Keywords: ai-agents,agent-safety,guardrails,governance,human-in-the-loop,vegaml,llm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Security
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.7
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: httpx>=0.27
+Provides-Extra: api
+Requires-Dist: fastapi>=0.110; extra == "api"
+Requires-Dist: uvicorn>=0.27; extra == "api"
+Provides-Extra: dashboard
+Requires-Dist: streamlit>=1.35; extra == "dashboard"
+Requires-Dist: pandas>=2.0; extra == "dashboard"
+Provides-Extra: vegaml
+Requires-Dist: vegaml<0.10,>=0.9; extra == "vegaml"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: fastapi>=0.110; extra == "dev"
+Requires-Dist: uvicorn>=0.27; extra == "dev"
+Requires-Dist: streamlit>=1.35; extra == "dev"
+Dynamic: license-file
+
+# Action Sentinel
+
+[![CI](https://github.com/premm25/action-sentinel/actions/workflows/ci.yml/badge.svg)](https://github.com/premm25/action-sentinel/actions/workflows/ci.yml) [![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/premm25/action-sentinel/blob/main/LICENSE)
+
+**A VegaML-powered safety checkpoint between AI agents and the real world.** Before an agent restarts
+a server, refunds a payment or deletes a database, [VegaML](https://github.com/NandhaKishorM/vegaml) judges
+the risk, Sentinel's rules and governance decide whether it may run, and it can only run with an
+authorization token issued for that exact action.
+
+```
+agent proposes action ──► Sentinel ──► APPROVE  → signed token → executor runs it (exactly once)
+                                   ├─► HUMAN_REVIEW → a person approves, rejects or edits it
+                                   ├─► BLOCK   (policy forbids it, whatever any model says)
+                                   ├─► ABSTAIN (not enough confidence to recommend anything)
+                                   └─► ERROR   (evaluation failed; never treated as approval)
+```
+
+## Why
+
+Agents built on LLMs are increasingly given real tools. The usual safeguard is a model judging
+whether an action is safe, but models are confidently wrong, and their confidence is rarely
+calibrated. Sentinel is built on a few non-negotiable rules:
+
+- **Deterministic policy wins.** A rule that blocks an action can't be outvoted, out-scored or
+  overridden by a model.
+- **Models can only escalate.** High confidence never authorizes anything on its own.
+- **Uncertainty is explicit.** Missing confidence, unknown calibration, abstention and provider
+  failure are first-class states, not defaults.
+- **Humans are in the loop, but checked too.** Reviewer authority levels, no self-review, and every
+  approval or edit is re-validated against the current policy.
+- **Authorization is bound to the exact payload.** Change one field and the token is useless.
+  Replays execute once.
+- **Everything is auditable**, and the pipeline is measurable with a built-in evaluation lab.
+
+## VegaML: the risk model
+
+VegaML reads each proposed action with a frozen Qwen3.5 backbone, and a few-shot head fitted on labelled
+actions rates it safe, needs review or unsafe. Sentinel treats that rating as evidence, not permission.
+
+| on AgentHarm (352 real requests, CPU, 0.8B) | result |
+|---|---|
+| VegaML head accuracy (fitted on 64 examples) | **0.841**, AUROC 0.920 |
+| harmful requests approved, head alone | 24 of 176 |
+| harmful requests approved, **head behind Sentinel** | **0 of 176** |
+| benign requests auto-approved behind Sentinel | 95 of 176 (the rest went to human review) |
+
+```bash
+pip install -e ".[api,vegaml]"
+action-sentinel vega fit --dataset my_actions.jsonl --out heads/my_actions.npz   # fit on your actions
+action-sentinel --config config/sentinel-vegaml.yaml serve                       # rules + VegaML, warmed up
+```
+
+Setup, fitting and operating notes: [`docs/VEGAML.md`](https://github.com/premm25/action-sentinel/blob/main/docs/VEGAML.md). The method and every number:
+[`docs/vegalab/`](https://github.com/premm25/action-sentinel/blob/main/docs/vegalab/RESULTS.md). Without VegaML installed, the default profile runs on the
+rules alone, offline.
+
+## 30-second example
+
+```bash
+pip install -e ".[dev,dashboard]"   # from a clone of this repository
+action-sentinel demo                       # offline end-to-end walkthrough, no model or API key needed
+```
+
+```python
+from action_sentinel.config import load_settings
+from action_sentinel.contracts import ActionProposal
+from action_sentinel.service import Sentinel
+
+sentinel = Sentinel(load_settings())          # config/sentinel.yaml: rule pack + rule-based provider
+
+action = ActionProposal.model_validate({
+    "event_id": "evt-42", "task_id": "t-1", "agent_id": "ops-agent", "agent_version": "1.0",
+    "source_system": "infra-agent", "action_type": "delete_database",
+    "target_resource": "prod/orders", "action_payload": {"database": "orders"},
+    "evidence_refs": [], "proposed_at": "2026-10-14T10:00:00+00:00", "idempotency_key": "evt-42",
+})
+result = sentinel.evaluate(action)
+print(result.decision.status.value, result.decision.reason_codes)
+# BLOCK ['HARD_POLICY_VIOLATION', 'EVIDENCE_INSUFFICIENT', 'RULE_PROD_DATABASE_DELETE']
+```
+
+An approved action returns `result.authorization.token`. The executor accepts only that token
+together with the identical action: `sentinel.execute(token, action)`.
+
+## How it differs from related tools
+
+| | Sentinel | LLM output guardrails | general policy engines |
+|---|---|---|---|
+| Guards | **actions** (tool calls) before execution | model text / structured output | any request, given policy input |
+| Model-based judgement | VegaML few-shot head (or any provider); can only escalate | often central | none |
+| Uncertainty, abstention, disagreement | explicit outcomes | varies | n/a |
+| Human review workflow with re-validation | built in | generally not | not built in |
+| Execution token bound to the exact payload, idempotent | built in | no | no |
+| Evaluation of the governance pipeline | built in (Confidence Lab) | varies | no |
+
+Sentinel complements these tools rather than replacing them: a policy engine can feed it rules,
+and an output guardrail can still check what the agent says.
+
+> **Status:** working prototype, 0.1.0. 279 automated tests, CI on Linux and Windows (Python
+> 3.10–3.14), offline demo. It is
+> **not production-ready** yet; see [Known limitations](#known-limitations) and the
+> [roadmap](#roadmap). Contributions are welcome: see [CONTRIBUTING.md](https://github.com/premm25/action-sentinel/blob/main/CONTRIBUTING.md).
+
+## Architecture
+
+```
+ActionProposal ─► ValidationEngine (rule packs) ──────────────┐  hard violation ⇒ BLOCK, always
+               └► ProviderRunner ─► rules | vegaml-head | vegaml | ollama | mock
+                         │ (single / sequential / parallel, enforced timeout)
+                         ▼
+          UncertaintyEvaluator + DisagreementDetector
+                         ▼
+               GovernanceEngine (fixed precedence) ─► GovernanceDecision
+                    │ APPROVE                    │ HUMAN_REVIEW
+                    ▼                            ▼
+              AuthorizationGrant ◄── ReviewManager (approve / reject / override → re-validate)
+                    ▼
+        ExecutionGateway (re-checks everything) ─► MockExecutionAdapter
+   every step ─► AuditLogger ─► SQLiteStore ─► sanitized JSONL export
+```
+
+Details: [`docs/architecture/ARCHITECTURE.md`](https://github.com/premm25/action-sentinel/blob/main/docs/architecture/ARCHITECTURE.md).
+
+## Prerequisites
+
+- Python 3.10 or newer. Developed and tested on 3.14.6 on Windows 11.
+- Optional: [Ollama](https://ollama.com) with a pulled model (default `llama3.1:8b`) for the LLM provider.
+- Optional: `pip install vegaml` (pulls torch and transformers, about 1.8 GB of weights on first use) for the VegaML provider.
+
+## Installation
+
+From PyPI, into your own project (no clone needed; the settings profiles, rule packs and dataset ship with the package):
+
+```bash
+pip install "action-sentinel[api]"    # add ,vegaml for VegaML; latest main: "action-sentinel[api] @ git+https://github.com/premm25/action-sentinel.git"
+action-sentinel demo                  # works from any directory
+action-sentinel init                  # copies config/ and examples/rule_packs/ here, to edit for your agent
+```
+
+Relative paths in settings resolve against your project first and fall back to the shipped files, so
+`examples/rule_packs/core.yaml` works before and after `init`.
+
+For development on Sentinel itself:
+
+```bash
+python -m venv .venv
+.venv/Scripts/activate            # Windows; on Linux/macOS: source .venv/bin/activate
+pip install -e ".[dev,dashboard]"
+```
+
+## Configuration
+
+`config/sentinel.yaml` holds the rule packs, providers, active providers, evaluation strategy,
+governance thresholds, reviewers and API principals. Environment overrides are listed in
+[`.env.example`](https://github.com/premm25/action-sentinel/blob/main/.env.example): `SENTINEL_CONFIG`, `SENTINEL_DB_PATH`, `SENTINEL_ENV` and
+`SENTINEL_SIGNING_KEY`.
+
+By default only the deterministic `rules` provider is active, so evaluation works without any model.
+To also consult the local LLM, set `active_providers: [rules, ollama]`, or pass
+`--providers rules,ollama` on the CLI.
+
+**Development API keys.** These are public, intended for local use only, and stored in the config as SHA-256 hashes.
+
+| key | principal | roles |
+|---|---|---|
+| `dev-agent-key` | ops-agent | agent |
+| `dev-operator-key` | olivia | reviewer (operator), auditor |
+| `dev-senior-key` | sam | reviewer (senior), auditor, evaluator |
+| `dev-admin-key` | ada | admin |
+| `dev-auditor-key` | auditor | auditor |
+
+## Quick start
+
+```bash
+action-sentinel validate-rules                                 # check the rule packs
+action-sentinel providers                                      # provider health
+action-sentinel evaluate examples/sample_actions/actions.jsonl -v
+action-sentinel reviews list
+action-sentinel reviews approve <review_id> --reviewer olivia --reason "verified"
+action-sentinel demo                                           # full offline end-to-end demonstration
+```
+
+`python -m action_sentinel ...` works the same as `action-sentinel ...`.
+
+## API
+
+```bash
+action-sentinel serve --port 8000          # interactive docs at http://127.0.0.1:8000/docs
+curl -s -X POST http://127.0.0.1:8000/decisions/evaluate -H "X-API-Key: dev-agent-key" \
+     -H "Content-Type: application/json" -d @examples/sample_actions/one.json
+```
+
+Endpoints:
+- `GET /health`, `GET /providers`
+- `POST /decisions/evaluate`, `POST /decisions/assess` (shadow: nothing stored), `GET /decisions`,
+  `GET /decisions/{id}`, `GET /decisions/{id}/events`, `POST /shadow-records`
+- `GET /reviews/pending`, `GET /reviews/{id}`, `POST /reviews/{id}/approve|reject|override`
+- `POST /executions` (Sentinel's adapter runs the action), or `POST /executions/begin` and
+  `POST /executions/{id}/finish` (the agent runs it, as the remote guard does); `POST /authorizations/{id}/revoke`
+- `POST /evaluations/run`, `GET /evaluations`, `GET /evaluations/{run_id}`, `GET /evaluations/{run_id}/report`
+
+## Dashboard
+
+```bash
+action-sentinel dashboard                  # http://127.0.0.1:8501
+action-sentinel demo --db var/sentinel.db  # optional: populate the default database first
+```
+
+The dashboard has six tabs: Overview, Decision Explorer, Confidence Lab, Human Review, Audit
+Explorer and Configuration. It reads the same SQLite database as the API and CLI. Review actions
+require the reviewer's API key.
+
+## Testing
+
+```bash
+pytest                                  # 203 tests, offline (optional live tests are deselected)
+pytest --cov=action_sentinel                   # with coverage
+pytest -m ollama                        # optional: live test against a running Ollama
+```
+
+## Guarding tool functions
+
+Wrap an agent's tools with a decorator instead of building proposals by hand. Shadow mode records
+what Sentinel would have done while changing nothing; enforce mode runs the tool only after the
+execution gateway clears it. Sync and `async def` tools both work, and Sentinel can run in the agent's
+process or as a separate service (`RemoteSentinel`). See [`docs/GUARD.md`](https://github.com/premm25/action-sentinel/blob/main/docs/GUARD.md).
+
+```python
+guard = Guard(sentinel, agent_id="ops-agent", agent_version="1.4.0", source_system="infra-agent", mode="shadow")
+
+@guard.tool("restart_service", target=lambda a: "staging/%s" % a["service"])
+def restart_service(service: str, reason: str): ...
+```
+
+## Evaluation (Confidence Lab)
+
+```bash
+python scripts/build_dataset.py         # regenerate the synthetic dataset (deterministic)
+action-sentinel lab suite                      # offline: rules + simulated providers, test split
+action-sentinel lab suite --model ollama --second ""     # live local LLM (slow on CPU)
+action-sentinel lab run --providers rules,ollama --splits test
+action-sentinel lab calibrate --provider ollama          # ECE on the calibration split
+action-sentinel lab show [run_id]   /   action-sentinel lab export <run_id...> --out report.md
+```
+
+Measured results and their limits are in [`docs/evaluation/RESULTS.md`](https://github.com/premm25/action-sentinel/blob/main/docs/evaluation/RESULTS.md).
+The method is in [`docs/evaluation/METHODOLOGY.md`](https://github.com/premm25/action-sentinel/blob/main/docs/evaluation/METHODOLOGY.md).
+
+## Vega Decision Intelligence Lab (VegaML)
+
+`action_sentinel/vegalab/` makes [VegaML](https://github.com/NandhaKishorM/vegaml) the primary decision
+engine and measures it. VegaML is a typed-decision model: a frozen Qwen3.5 language model reads the
+situation, and a small trained physics engine settles on one of the candidate answers, returning calibrated
+probabilities, a conformal prediction set, an abstain flag and an "unbound" (out-of-distribution) flag.
+It generates no text.
+
+- **Real inference, verified** with `vegaml` 0.9.0, the 0.8B checkpoint on CPU. The lab wraps VegaML's own
+  API and never reimplements its engine. See [`docs/VEGA_MIGRATION.md`](https://github.com/premm25/action-sentinel/blob/main/docs/VEGA_MIGRATION.md).
+- **Typed decision service:** a Python `VegaCore` and `/vega/*` API routes, enabled with `vegalab.enabled`.
+- **Confidence Lab v2:** public-task datasets with provenance and leakage checks; VegaML (adapters off and
+  auto) vs the same frozen backbone used as a plain LM vs a majority baseline; accuracy, calibration,
+  abstention, conformal coverage, paired confidence intervals.
+- **Sentinel integration:** VegaML as a governance provider: `vegaml-head` (the few-shot head, used by
+  `config/sentinel-vegaml.yaml`), and `vegaml-off` / `vegaml-auto` (the engine). Hard rules still win.
+- **Heads you fit yourself:** `action-sentinel vega fit` fits the head on your labelled actions (never on the
+  test split) and saves it. The provider serves it only for the question, framing and checkpoint it was
+  fitted for.
+
+VegaML needs torch (about 3.5 GB RAM for 0.8B on CPU), so it is best installed in its own environment:
+
+```bash
+python -m venv ~/.venvs/vega-lab
+~/.venvs/vega-lab/Scripts/pip install "vegaml==0.9.0" "transformers==5.19.0" torch pyarrow -e ".[dev]"
+action-sentinel vega check --load                     # downloads ~1.8 GB on first use
+action-sentinel vega decide --adapters off --instructions "Which team?" \
+    --state "Invoice overdue, third notice" --option "billing=payments" --option "technical=faults"
+action-sentinel vega data check                       # dataset quality and leakage checks
+action-sentinel vega run --limit 20                   # inference (resumable) -> reports/vegalab/<run>
+action-sentinel vega analyze reports/vegalab/<run>    # metrics.json, summary.csv, REPORT.md
+action-sentinel vega failures reports/vegalab/<run> --kind confident-wrong
+```
+
+GPU runs (0.8B and 4B, 500 test examples per task): [`notebooks/vegalab_kaggle_gpu.ipynb`](https://github.com/premm25/action-sentinel/blob/main/notebooks/vegalab_kaggle_gpu.ipynb).
+Documents: [architecture](https://github.com/premm25/action-sentinel/blob/main/docs/vegalab/ARCHITECTURE.md) · [datasets](https://github.com/premm25/action-sentinel/blob/main/docs/vegalab/DATASETS.md) ·
+[methodology](https://github.com/premm25/action-sentinel/blob/main/docs/vegalab/METHODOLOGY.md) · [results](https://github.com/premm25/action-sentinel/blob/main/docs/vegalab/RESULTS.md).
+
+## Demo
+
+```bash
+action-sentinel demo
+```
+
+The demo runs 12 steps offline and checks the expected result of each one. It exits non-zero if
+any check fails. See [`docs/DEMO_GUIDE.md`](https://github.com/premm25/action-sentinel/blob/main/docs/DEMO_GUIDE.md).
+
+## Known limitations
+
+- Single-process SQLite storage, with no high availability, replication or retention management.
+- The authentication in the dev configuration uses publicly known keys. There is no TLS, OAuth,
+  rate limiting or key rotation.
+- If `SENTINEL_SIGNING_KEY` is unset, each process generates its own key, so a token issued by the
+  API can't be executed through the CLI.
+- The Confidence Lab dataset is synthetic (86 examples, 44 in the test split). Its expected outcomes
+  were written against the same reference policy that the rules enforce, so the rules-only score
+  is circular.
+- The local LLM (`llama3.1:8b`, CPU) was measured at 0.41 accuracy on the test split, and it never
+  chose `unsafe`. On its own it is not a useful classifier for this task as prompted.
+- VegaML results are single CPU runs with the 0.8B checkpoint; there are no GPU or 4B results yet.
+- The mock executor changes nothing real. A real adapter would also need compensation, dry-run and
+  scoping controls.
+
+## Roadmap
+
+1. An MCP proxy, so existing agents can be protected without rewiring. The `@guard` decorator is done,
+   in-process and remote, for sync and async tools ([`docs/GUARD.md`](https://github.com/premm25/action-sentinel/blob/main/docs/GUARD.md)).
+2. PyPI package and Docker image.
+3. A stronger VegaML layer: heads fitted on real labelled agent traffic, GPU and 4B results, automatic
+   calibration profiles.
+4. A public, independently labelled benchmark of risky agent actions, plus a reusable policy library.
+5. Production hardening: Postgres storage, managed signing keys, OIDC, tamper-evident audit, review notifications.
+
+## Documents
+
+- [Architecture](https://github.com/premm25/action-sentinel/blob/main/docs/architecture/ARCHITECTURE.md)
+- [Threat model](https://github.com/premm25/action-sentinel/blob/main/docs/architecture/THREAT_MODEL.md)
+- [Architecture decisions](https://github.com/premm25/action-sentinel/blob/main/docs/decisions/ARCHITECTURE_DECISIONS.md)
+- [Evaluation methodology](https://github.com/premm25/action-sentinel/blob/main/docs/evaluation/METHODOLOGY.md)
+- [Evaluation results](https://github.com/premm25/action-sentinel/blob/main/docs/evaluation/RESULTS.md)
+- [Security](https://github.com/premm25/action-sentinel/blob/main/docs/security/SECURITY.md)
+- [Runbook](https://github.com/premm25/action-sentinel/blob/main/docs/operations/RUNBOOK.md)
+- [Demo guide](https://github.com/premm25/action-sentinel/blob/main/docs/DEMO_GUIDE.md)
+- [Guarding tool functions](https://github.com/premm25/action-sentinel/blob/main/docs/GUARD.md)
+- [Running with VegaML](https://github.com/premm25/action-sentinel/blob/main/docs/VEGAML.md)
+- [Contributing](https://github.com/premm25/action-sentinel/blob/main/CONTRIBUTING.md) · [Security policy](https://github.com/premm25/action-sentinel/blob/main/SECURITY.md) · [Changelog](https://github.com/premm25/action-sentinel/blob/main/CHANGELOG.md)
+
+## License
+
+Apache License 2.0. See [LICENSE](https://github.com/premm25/action-sentinel/blob/main/LICENSE).