agent-release-gates 0.1.0__py3-none-any.whl

agent_release_gates-0.1.0.dist-info/METADATA

@@ -0,0 +1,201 @@
+Metadata-Version: 2.4
+Name: agent-release-gates
+Version: 0.1.0
+Summary: Release-readiness gates for AI agents: replay known incidents, apply policy-as-code gates, and produce ship/warn/block evidence before an agent, prompt, model, or tool-policy change ships.
+Project-URL: Homepage, https://github.com/rosscyking1115/agent-release-gates
+Project-URL: Repository, https://github.com/rosscyking1115/agent-release-gates
+Project-URL: Documentation, https://rosscyking1115.github.io/agent-release-gates/
+Project-URL: Issues, https://github.com/rosscyking1115/agent-release-gates/issues
+Author: rosscyking1115
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent,ai-safety,evaluation,inspect-ai,llm,red-team,release-gate
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.12
+Requires-Dist: pydantic>=2.8.0
+Provides-Extra: api
+Requires-Dist: fastapi>=0.115.0; extra == 'api'
+Requires-Dist: uvicorn[standard]>=0.30.0; extra == 'api'
+Provides-Extra: dashboard
+Requires-Dist: altair>=6.0.0; extra == 'dashboard'
+Requires-Dist: pandas>=2.2.0; extra == 'dashboard'
+Requires-Dist: streamlit>=1.38.0; extra == 'dashboard'
+Description-Content-Type: text/markdown
+
+# Agent Release Safety Gates
+
+A public release-readiness system for testing whether AI-agent workflow changes remain grounded, safe, auditable, and useful under retrieval, refusal, prompt-injection, incident replay, and approval-gated tool-use conditions.
+
+This project is not a clone, assessment, or reverse-engineering attempt of any company's internal AI system. The controlled operations benchmark is synthetic by design; TechQA and WixQA are used separately as public retrieval-validation datasets.
+
+## Live Project
+
+- Public project page: https://rosscyking1115.github.io/agent-release-gates/
+- Full evaluation report (HTML): https://rosscyking1115.github.io/agent-release-gates/evaluation_report.html
+- PDF report: https://rosscyking1115.github.io/agent-release-gates/evaluation_report.pdf
+- Interactive dashboard: run locally (see [Run Locally](#run-locally)) or deploy on
+  Streamlit Cloud — see the [dashboard deployment guide](docs/dashboard.md). A hosted
+  instance must be set to public visibility to be shareable.
+
+## What This Project Does
+
+The project evaluates an AI-agent workflow across five release questions:
+
+- Does the agent retrieve the right evidence and cite it?
+- Does it abstain or refuse when evidence is weak, unsafe, or prompt-injected?
+- Does it require approval before mock side-effecting tool calls?
+- Does it leave enough trace, audit, and monitoring evidence for review?
+- Does it pass incident replay and policy-as-code release gates?
+
+The result is a reproducible evaluation artifact rather than a one-off dashboard: deterministic eval runners, generated reports, CI checks, Dockerized local execution, a Streamlit dashboard, and a GitHub Pages report site.
+
+## Product Direction
+
+**Agent Release Safety Gates** is a release-readiness workflow for replaying known agent incidents, applying policy gates, and producing evidence before a changed agent, prompt, model, or tool policy ships. Its deterministic evaluation benchmark and runners are the evidence layer behind that workflow.
+
+The first module is an Incident Replay Suite that turns redacted synthetic incidents into regression fixtures, replay results, release gates, and incident memos.
+
+To evaluate an external agent, use the public quickstart:
+
+- [Evaluate your agent quickstart](docs/evaluate_your_agent_quickstart.md)
+- [Incident pack schema](docs/incident_pack_schema.md)
+- [Candidate results schema](docs/candidate_results_schema.md)
+
+## Current Evidence Snapshot
+
+| Area | Current result |
+| --- | --- |
+| Controlled benchmark | 358 synthetic golden cases, 60 red-team cases, 180 synthetic operations tickets |
+| Retrieval | 100.00% synthetic retrieval hit rate@3 with local TF-IDF/vector-style retrievers |
+| Public RAG validation | 160 TechQA cases and 80 WixQA cases evaluated separately from the synthetic benchmark |
+| Safety | 90.91% classifier recall, 0 high-severity false negatives in the current challenge set |
+| Agent governance | 100.00% mock side-effect block rate and approval audit rate |
+| Incident replay | 8 seeded synthetic incidents replayed, 100.00% closure rate, 0 replay must-not violations |
+| Intervention study | 3 deterministic safety studies plus public RAG grounding and memory/context studies |
+| Hosted judge calibration | Reviewed OpenAI and Anthropic judge runs with public-safe provider comparison |
+
+These results are engineering evidence over controlled benchmarks. They are not claims of real-world production performance.
+
+## Key Findings
+
+- Safety metrics are not meaningful alone; the lab reports over-review cost, benign auto-blocks, weak-evidence handling, and unsafe misses beside the headline scores.
+- Layered safeguards reduce selected prompt-injection, unsafe-action, and unsafe-request failures in controlled studies while making review burden visible.
+- Public TechQA and WixQA retrieval tracks help test whether the RAG harness works beyond self-contained synthetic data.
+- Public RAG grounding thresholds reduce unsupported answer attempts while making abstention and review cost visible.
+- Memory/context controls reduce polluted-memory following while preserving benign memory usefulness.
+- Goal-conflict arbitration reduces unsafe goal-following while preserving benign task completion.
+- Synthetic operations data remains useful for controlled tests that would be unsafe or impractical to run on confidential real workflows.
+- The next strongest validation step is independent human labelling, followed by broader multi-model comparison.
+
+## What Is Included
+
+- Evaluation runners for retrieval, extraction, safety classification, controlled-agent behavior, and observability.
+- Baseline-vs-intervention studies for instruction hierarchy, action-risk gates, and safety classifier review policy.
+- Public RAG grounding and abstention intervention study over TechQA and WixQA.
+- Memory/context pollution intervention study covering stale, injected, and cross-user memory.
+- Goal-conflict intervention study covering safety, evidence, privacy, and tool-risk arbitration.
+- Incident replay suite with seeded incidents, replay matrix, release gates, regression fixtures, and generated memos.
+- Public benchmark documentation, dataset boundaries, failure taxonomy, and external-review packet.
+- Candidate-results exporters for generic agent logs and LangChain/LangSmith-style traces.
+- Streamlit dashboard for interactive inspection.
+- GitHub Pages report and PDF for public review.
+- CI, Docker, Docker Compose, linting, tests, and deterministic report regeneration.
+
+## Install
+
+Once published to PyPI (see [publishing guide](docs/publishing.md)), the core install
+is lean — `pydantic` only — and gives you the CLI, the Inspect suite, the real-agent
+runner, and the scoring logic. The API and dashboard are opt-in extras:
+
+```bash
+pip install agent-release-gates                # CLI + Inspect suite + scoring
+pip install "agent-release-gates[api]"         # + FastAPI evidence service
+pip install "agent-release-gates[dashboard]"   # + Streamlit dashboard deps
+pip install agent-release-gates inspect_ai     # to run under Inspect
+```
+
+```bash
+agent-safety release-gate                                          # ship / warn / block
+inspect eval agent-release-gates/incident_replay --model openai/gpt-4.1-mini
+```
+
+> Not yet on PyPI — build it yourself with `uv build`, or run from source below.
+
+## Run Locally
+
+```powershell
+uv sync
+uv run python scripts/run_all_evals.py
+# Release gate (installed console command); exits non-zero on a blocking failure.
+uv run agent-safety release-gate --policy config/incident_release_policy.json
+# Interactive dashboard.
+uv run streamlit run streamlit_app.py --server.port 8510
+```
+
+Open `http://localhost:8510`. Run the API and dashboard together with
+`docker compose up --build`, then open `http://localhost:8510` and
+`http://localhost:8000/health`.
+
+Drive a real LLM through the release gate, or run the suite under Inspect:
+
+```powershell
+# Any OpenAI-compatible / self-hosted open model endpoint.
+$env:AGENT_RUNNER_API_KEY = "..."
+uv run python scripts/run_real_agent_replay.py
+
+# Inspect (UK AISI) -- optional peer dependency.
+uv pip install inspect_ai
+inspect eval agent-release-gates/incident_replay --model openai/gpt-4.1-mini
+```
+
+## Verification
+
+```powershell
+uv run ruff check .
+uv run pytest
+uv run python scripts/run_all_evals.py
+uv run agent-safety release-gate --policy config/incident_release_policy.json
+uv run python scripts/build_public_site.py
+docker build -t agent-release-safety-gates:local .
+```
+
+CI runs linting, tests, deterministic report checks, local OpenTelemetry smoke testing, Dockerized collector verification, and Docker build verification.
+
+## Review Materials
+
+- Evaluate your agent quickstart: [docs/evaluate_your_agent_quickstart.md](docs/evaluate_your_agent_quickstart.md)
+- Benchmark card: [docs/benchmark_card.md](docs/benchmark_card.md)
+- Agent safety intervention study: [docs/agent_safety_intervention_study.md](docs/agent_safety_intervention_study.md)
+- RAG grounding intervention report: [reports/rag_grounding_intervention.md](reports/rag_grounding_intervention.md)
+- Memory context intervention report: [reports/memory_context_intervention.md](reports/memory_context_intervention.md)
+- Goal conflict intervention report: [reports/goal_conflict_intervention.md](reports/goal_conflict_intervention.md)
+- Incident pack schema: [docs/incident_pack_schema.md](docs/incident_pack_schema.md)
+- Candidate results schema: [docs/candidate_results_schema.md](docs/candidate_results_schema.md)
+- Incident replay summary: [reports/incident_replay_summary.json](reports/incident_replay_summary.json)
+- Dataset card: [docs/dataset_card.md](docs/dataset_card.md)
+- Failure taxonomy: [docs/failure_taxonomy.md](docs/failure_taxonomy.md)
+- External reviewer handoff pack: [docs/reviewer_handoff_pack.md](docs/reviewer_handoff_pack.md)
+- Technical artifact index: [docs/technical_artifacts.md](docs/technical_artifacts.md)
+- Contribution guide: [CONTRIBUTING.md](CONTRIBUTING.md)
+
+## Current Limitations
+
+- The controlled benchmark is synthetic and still partly templated.
+- Public TechQA and WixQA tracks use compact samples, not the full upstream datasets.
+- Human-review labels are currently simulated workflow labels; independent reviewer labels are prepared but not yet published.
+- Hosted model evidence includes reviewed judge-calibration runs, not a broad multi-model agent comparison.
+- Provider-backed embedding and reranker adapters are prepared, but credentialed hosted results are not claimed until reviewed.
+
+## Roadmap
+
+- Collect independent human labels using the prepared review packet.
+- Add reproducible multi-model comparison across hosted and open-source models.
+- Expand public RAG validation beyond the current compact TechQA and WixQA samples.
+- Add more framework-specific candidate-results exporters for common agent runners.
+- Expand the paper-style intervention report with external reviewer disagreement analysis.
+- Invite external review through issues and contribution guidelines.

agent_release_gates-0.1.0.dist-info/RECORD

@@ -0,0 +1,74 @@
+internal_ai_agent/__init__.py,sha256=5pj6HrvaI6zBb8hDTL-EAC_pcIfa4oInCmxdEdOQ5UY,84
+internal_ai_agent/cli.py,sha256=62KofB0d7ZWpgfmab-zgOAWk1AEPIAURXe80AP45NDo,3983
+internal_ai_agent/io.py,sha256=ubw1DH5DjpEAmgtNPDAN9f6dD1vDJW-D_Rf2g-Tp1Po,772
+internal_ai_agent/agent/__init__.py,sha256=EuPVNLnCG9S5VY8ET2pheIkrlSzWZDwR0P7Y4gVXSUI,43
+internal_ai_agent/agent/schemas.py,sha256=FAVr7YY9_kPddEwBjacwaje8SONDx4yOU_4gkRK5m6E,1174
+internal_ai_agent/agent/tools.py,sha256=wo8ASgW5BtOyUc2V7Jn37n6OgasaQwOtCZATqn7rtPA,2746
+internal_ai_agent/agent/workflow.py,sha256=fx4dDB-sDdfw945h6dm1u9Lr8G8LpWC2nhw5tFm5Sno,8251
+internal_ai_agent/api/__init__.py,sha256=E9WAQukVOuCYKUPDJuT5hOqXc5Gl-uSNuR1eTDUn7Ps,50
+internal_ai_agent/api/main.py,sha256=7XilqOh8Nlghm7mgfssnJbXZTTnnCb4rRrkKkvd48nY,7851
+internal_ai_agent/api/schemas.py,sha256=k-5Cgp77VL3k3MV49KUVZ8lZC8RtBkdAR_AXobBCPgg,1088
+internal_ai_agent/dashboard/__init__.py,sha256=LBhz-cOpyTnJ7Hsjbcp5s2S4tuejkpr2K-Y0hyjs8ck,30
+internal_ai_agent/dashboard/data.py,sha256=NiM2XDfTscRq0rYFdnUGNvgfE27MVA-AS5oRqv-1gLs,65713
+internal_ai_agent/data/__init__.py,sha256=ioR4A1CmQFOKF4Eaur195zQWzMOfYyL-O8PjWKZsE0U,43
+internal_ai_agent/data/synthetic.py,sha256=TgHAQT0LHnplMELYw1mKhPlHc5MtFpwGLQBgcj_dQto,107205
+internal_ai_agent/evals/__init__.py,sha256=9u6VFTKmYlae_jneJrRA-m_Zzc7uT8A5Fse6qPufaYs,62
+internal_ai_agent/evals/agent.py,sha256=qo0g7uSzpXjOvKB6a3Akttq31U4kxjf11on3BdtIh6Q,7721
+internal_ai_agent/evals/candidate_results_export.py,sha256=o_lF5RAhIN3UfzrgRqe2XIW_yXcQlxcrYv4qI-2JeP0,25814
+internal_ai_agent/evals/dataset_profile.py,sha256=KXeJo6bclSwBFCaQeQdPon1Ldh7ppyDp947GuU9n0wY,5598
+internal_ai_agent/evals/external_review.py,sha256=7j76KhQbRG9UUtfseX_KGGbXGFdP74JE2ZWGs3b788A,19390
+internal_ai_agent/evals/extraction.py,sha256=JDeyg0adi63whPPBt0_m7_H4QGB9fHrwnRUYoqEUuhg,2804
+internal_ai_agent/evals/failure_taxonomy.py,sha256=EQAzetI7fyY_qUfFhI3HSunUq0xwgV2yuWrjZabgQwE,8221
+internal_ai_agent/evals/gates.py,sha256=yZEbmpcZ2siuzELue4U0P7n18QD-EHajoMJB3D4HI7U,9203
+internal_ai_agent/evals/goal_conflict_intervention.py,sha256=8fcvUWN-Ii3u_5TMrgrhSa2Wygt94cCitYGEg2xtxvw,14149
+internal_ai_agent/evals/human_calibration.py,sha256=geuaUVjeaqpTmiDAE15nclW5wB3HH7VcsbTmYi00LRk,22506
+internal_ai_agent/evals/incident_replay.py,sha256=AdYTj5qOarYsHINSuhoVVFCTMONcyhppfUsNWg40d1k,83012
+internal_ai_agent/evals/intervention_study.py,sha256=4IrXigSaDr5WHG_4SXPJM5yeB1xuD0lk5IpQdQc8euo,42509
+internal_ai_agent/evals/memory_context_intervention.py,sha256=048raPTWmGuwAlO6ZvES8bSqnN5hTuGSbzYTm8mrLFc,14290
+internal_ai_agent/evals/model_judge.py,sha256=A_8k7u2s2mktAXk-bpS5QeL9ri6Q35h0VKXcLf0K3Xs,11049
+internal_ai_agent/evals/model_judge_promotion.py,sha256=Diy7kXnnzP86iTVnHw_Xh-pcG2ATCZ3pry63fJbr-Bw,3371
+internal_ai_agent/evals/multi_model_comparison.py,sha256=DK6iVbObLjgTkIAGvgYWKFD9mMISQI9Wsx56G0JuK54,14282
+internal_ai_agent/evals/multilingual_safety.py,sha256=0J_1czHj8SBgxXLJ2_ruQvxF42Ec3INg9nr9H5A_MMg,4989
+internal_ai_agent/evals/nist_rmf_mapping.py,sha256=fcTJkLXxp6lD2i2mETyt0CYNpm1ANGo_npSLDj4yF_0,7142
+internal_ai_agent/evals/public_rag_findings.py,sha256=saP0Xx7ul8emsF1IjvRXQK6BjVRvSfaxmFi1fFi1ZPA,9000
+internal_ai_agent/evals/public_rag_model_reranker.py,sha256=EU9I9vpdkFHmLQ5MbxcEoY8nBkGyvCzGm544eMMuOro,10872
+internal_ai_agent/evals/public_rag_reranker.py,sha256=rjF0K-ozj9Ji_3X5zvSQBWDqP6BmvNSfIxuy1mgrXSk,14466
+internal_ai_agent/evals/public_rag_reranking.py,sha256=_cqoNLtcq1NORnoq3Verqgla9ByYsXWunbpEm2m3go0,8269
+internal_ai_agent/evals/rag_grounding_intervention.py,sha256=BIgNza4lhVfBMDM21XzHDGFbB5Q_lSjLinU1CpFa5-w,16593
+internal_ai_agent/evals/runner.py,sha256=2yu__30vplVrsm6S5kVaQAdcMVGfLMAZ0f5RFQ728ik,29377
+internal_ai_agent/evals/safety_classifier.py,sha256=_GZjRQgVavbHCjuy7oKTxRBX4wkMHpnPWu7XXgruhYw,75474
+internal_ai_agent/evals/security.py,sha256=ix9HWX58Nzni7-tAfigN-MbrXoR4-aHJ-mOlEA11Aro,7899
+internal_ai_agent/evals/techqa_public.py,sha256=TWcZDiIQG6BnQFO3JW-Dap7TsS2UMTGRuLl_wo7W_CE,24819
+internal_ai_agent/evals/wixqa_public.py,sha256=a75Y7f21OuBxKsp-VHOi64_A67HgRo0vW7N-sIV10WE,22028
+internal_ai_agent/extraction/__init__.py,sha256=GUXBdvm3lihERH3ShgYWtxFnOy1SGwiqvavygyZbeEc,51
+internal_ai_agent/extraction/schemas.py,sha256=6DXI1dGeHKRZwaeJf6HA8BvIQtdr9kPFNmSXyA_gXuE,732
+internal_ai_agent/extraction/service.py,sha256=ewpJ56CSt3XZTxMsGYgyLdJO-5ftJMYrE6oxhCfe0nc,3155
+internal_ai_agent/inspect_suite/__init__.py,sha256=hVdp13T3YhhJsMZMj1PaYIGPF7slwwNKUcqM8HMzP8M,480
+internal_ai_agent/inspect_suite/_registry.py,sha256=z989a8ETDPafY5Hnyamy-_Y5m-hlOoWXYajpAv1NHWQ,273
+internal_ai_agent/inspect_suite/scorers.py,sha256=Zz4jWjmD84Hi0cZgsuRTmQ6NV_-I11Ebe9X0YZ_Zoz4,1107
+internal_ai_agent/inspect_suite/scoring.py,sha256=qCPai5orU6lNpOdOmlzS2megfsmyUmCTA-GIL4lt-0o,1679
+internal_ai_agent/inspect_suite/tasks.py,sha256=1xQka3YTWhlC6tVRDPrUVUCVkAJ8XE9CCqf23cEYTGA,1148
+internal_ai_agent/observability/__init__.py,sha256=gse8gRKGAmGpXa4UbMI1A0XlSx1jSdmmhGj9Oneqckc,47
+internal_ai_agent/observability/audit.py,sha256=Ecyjsm4AzNIt1wkB_reIbS9ihXvS35tbib_LZODWDjo,1317
+internal_ai_agent/observability/collector.py,sha256=O97N6_XFTWeNXlMFSpYCE3ejwEztEsqQTpAOHwz4zMQ,7789
+internal_ai_agent/observability/collector_deployment.py,sha256=5uaeSVZQTq6UFFGtnx55UCTwyyMeU3-fpIlgOOy96pc,2592
+internal_ai_agent/observability/collector_smoke.py,sha256=3rHaEQAXR3UnHDnyQWLqlv8kHBQM4dEepBiDqVG5vBg,4774
+internal_ai_agent/observability/otel.py,sha256=ylShLmB-NsVYyGhIQsvTe-C6f1DuoHCXQTW07IUdcgk,31422
+internal_ai_agent/observability/trace_index.py,sha256=iBji1AWEE-digjcTXDhXZuUspzCGUyp8guMmhz45SWw,6756
+internal_ai_agent/providers/__init__.py,sha256=5USfd_M83RcGTejgqa7CgKcouNd44pWXn1rcnwGTw5M,78
+internal_ai_agent/providers/agent_runner.py,sha256=m_kbR9Pm65-EYPJE9PNCx9u1a89w0AXSco_HLwAOvVg,9535
+internal_ai_agent/providers/anthropic_judge.py,sha256=Y53oav2M2Vq4LDvNj2kJLBoMke0mrpnvTE6aK4BZQQ4,7812
+internal_ai_agent/providers/openai_embeddings.py,sha256=L968xMSMsAEHOfWGgZgQzF7_rewQ-c0afT6qdgJwcJk,3890
+internal_ai_agent/providers/openai_judge.py,sha256=dAEqWUzkINHb6z8fVvRJpPXGAqpYeQzLsJV4FWL7zdA,7644
+internal_ai_agent/rag/__init__.py,sha256=B82YHSJsXep2C8pLO0F_rPz4yFY4JBxwTtHz-Fvx2yw,50
+internal_ai_agent/rag/baseline.py,sha256=95rJv7SeONuLdqcfYEDHHdRX_OuOAujryzDZ3vP0hWY,38861
+internal_ai_agent/reporting/__init__.py,sha256=H42MMB875Pdhfr56chw7Ewok_lPPoL69Cz6aXekT5f8,60
+internal_ai_agent/reporting/public_report.py,sha256=BTx3jFlBUXaZAyNs2MW6_aeR4LlFqY9I210zSpmAAsU,95585
+internal_ai_agent/security/__init__.py,sha256=XzzoG0N-BvTbuhRu8505p8aM-mwGaqvwyO9m26DzfF0,31
+internal_ai_agent/security/action_safety.py,sha256=uAEwpxgKM5_zV_mIjAm5NZn_1yDL_XcU11v3DkpRLtQ,2938
+internal_ai_agent/security/policy.py,sha256=_l7VCOfH7BxMWrRWtALTxVMBt2UbTmO1vyFyFjgXCzM,6538
+agent_release_gates-0.1.0.dist-info/METADATA,sha256=_IhhkbZAUJ-F4i-SVTV_v5ACEIlgDIVbgF_bODrF9Yc,11494
+agent_release_gates-0.1.0.dist-info/WHEEL,sha256=mffPy8wBnZQn2VnJUU5jE99KsxaSfiyMHV9Yt0aLVxs,87
+agent_release_gates-0.1.0.dist-info/entry_points.txt,sha256=_T_Q2olwdu4cp5tWeNzGuuL01r4IqMAhO-YxYqe6Mmc,138
+agent_release_gates-0.1.0.dist-info/licenses/LICENSE,sha256=AJUwcM_Bzqa4voXHS2MHwr0RZT9U8bxDEOAkKukH6K4,1071
+agent_release_gates-0.1.0.dist-info/RECORD,,

agent_release_gates-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.30.1
+Root-Is-Purelib: true
+Tag: py3-none-any

agent_release_gates-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,5 @@
+[console_scripts]
+agent-safety = internal_ai_agent.cli:main
+
+[inspect_ai]
+agent_release_gates = internal_ai_agent.inspect_suite._registry

agent_release_gates-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 rosscyking1115
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

internal_ai_agent/__init__.py

@@ -0,0 +1,5 @@
+"""Agent Release Safety Gates."""
+
+__all__ = ["__version__"]
+
+__version__ = "0.1.0"

internal_ai_agent/agent/__init__.py

@@ -0,0 +1 @@
+"""Controlled synthetic agent workflow."""

internal_ai_agent/agent/schemas.py

@@ -0,0 +1,47 @@
+from __future__ import annotations
+
+from typing import Any, Literal
+
+from pydantic import BaseModel
+
+from internal_ai_agent.extraction.schemas import RoutingDecision, TicketExtraction
+from internal_ai_agent.observability.audit import AuditEvent
+from internal_ai_agent.security.policy import PolicyDecision
+
+ToolName = Literal[
+    "search_runbook",
+    "extract_ticket",
+    "draft_escalation_note",
+    "route_ticket_mock",
+    "create_followup_task_mock",
+]
+
+ToolType = Literal["read_only", "side_effect"]
+
+
+class ToolDecision(BaseModel):
+    tool_name: ToolName
+    tool_type: ToolType
+    requested: bool
+    requires_approval: bool
+    approval_granted: bool
+    executed: bool
+    valid_schema: bool
+    rationale: str
+    blocked_reason: str | None = None
+    output: dict[str, Any] = {}
+
+
+class AgentRunResult(BaseModel):
+    trace_id: str
+    question: str
+    answer: str
+    citations: list[str]
+    abstained: bool
+    extraction: TicketExtraction | None = None
+    routing: RoutingDecision | None = None
+    tool_decisions: list[ToolDecision]
+    audit_log: list[str]
+    audit_events: list[AuditEvent]
+    monitoring: dict[str, Any]
+    policy: PolicyDecision

internal_ai_agent/agent/tools.py

@@ -0,0 +1,86 @@
+from __future__ import annotations
+
+from typing import Any
+
+from internal_ai_agent.agent.schemas import ToolDecision, ToolName, ToolType
+
+TOOL_TYPES: dict[ToolName, ToolType] = {
+    "search_runbook": "read_only",
+    "extract_ticket": "read_only",
+    "draft_escalation_note": "read_only",
+    "route_ticket_mock": "side_effect",
+    "create_followup_task_mock": "side_effect",
+}
+
+REQUIRED_FIELDS: dict[ToolName, set[str]] = {
+    "search_runbook": {"query"},
+    "extract_ticket": {"text"},
+    "draft_escalation_note": {"ticket_id", "team", "issue_category"},
+    "route_ticket_mock": {"ticket_id", "team"},
+    "create_followup_task_mock": {"ticket_id", "owner", "reason"},
+}
+
+
+def make_tool_decision(
+    tool_name: ToolName,
+    payload: dict[str, Any],
+    *,
+    approval_granted: bool = False,
+    rationale: str,
+) -> ToolDecision:
+    tool_type = TOOL_TYPES[tool_name]
+    valid_schema = _valid_payload(tool_name, payload)
+    requires_approval = tool_type == "side_effect"
+    blocked_reason: str | None = None
+    executed = valid_schema
+
+    if requires_approval and not approval_granted:
+        executed = False
+        blocked_reason = "approval_required"
+    elif not valid_schema:
+        executed = False
+        blocked_reason = "invalid_tool_payload"
+
+    return ToolDecision(
+        tool_name=tool_name,
+        tool_type=tool_type,
+        requested=True,
+        requires_approval=requires_approval,
+        approval_granted=approval_granted,
+        executed=executed,
+        valid_schema=valid_schema,
+        rationale=rationale,
+        blocked_reason=blocked_reason,
+        output=_tool_output(tool_name, payload) if executed else {},
+    )
+
+
+def _valid_payload(tool_name: ToolName, payload: dict[str, Any]) -> bool:
+    required = REQUIRED_FIELDS[tool_name]
+    return all(str(payload.get(field, "")).strip() for field in required)
+
+
+def _tool_output(tool_name: ToolName, payload: dict[str, Any]) -> dict[str, Any]:
+    if tool_name == "route_ticket_mock":
+        return {
+            "status": "mock_routed",
+            "ticket_id": payload["ticket_id"],
+            "team": payload["team"],
+        }
+    if tool_name == "create_followup_task_mock":
+        return {
+            "status": "mock_followup_created",
+            "ticket_id": payload["ticket_id"],
+            "owner": payload["owner"],
+        }
+    if tool_name == "draft_escalation_note":
+        return {
+            "status": "drafted",
+            "note": (
+                f"Draft note for {payload['ticket_id']}: route to {payload['team']} "
+                f"for {payload['issue_category']} review."
+            ),
+        }
+    if tool_name == "extract_ticket":
+        return {"status": "extracted"}
+    return {"status": "searched"}