agentops-cockpit 0.9.5__py3-none-any.whl → 0.9.8__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- agent_ops_cockpit/agent.py +44 -77
- agent_ops_cockpit/cache/semantic_cache.py +10 -21
- agent_ops_cockpit/cli/main.py +105 -153
- agent_ops_cockpit/eval/load_test.py +33 -50
- agent_ops_cockpit/eval/quality_climber.py +88 -93
- agent_ops_cockpit/eval/red_team.py +84 -25
- agent_ops_cockpit/mcp_server.py +26 -93
- agent_ops_cockpit/ops/arch_review.py +221 -147
- agent_ops_cockpit/ops/auditors/base.py +50 -0
- agent_ops_cockpit/ops/auditors/behavioral.py +31 -0
- agent_ops_cockpit/ops/auditors/compliance.py +35 -0
- agent_ops_cockpit/ops/auditors/dependency.py +48 -0
- agent_ops_cockpit/ops/auditors/finops.py +48 -0
- agent_ops_cockpit/ops/auditors/graph.py +49 -0
- agent_ops_cockpit/ops/auditors/pivot.py +51 -0
- agent_ops_cockpit/ops/auditors/reasoning.py +67 -0
- agent_ops_cockpit/ops/auditors/reliability.py +53 -0
- agent_ops_cockpit/ops/auditors/security.py +87 -0
- agent_ops_cockpit/ops/auditors/sme_v12.py +76 -0
- agent_ops_cockpit/ops/auditors/sovereignty.py +74 -0
- agent_ops_cockpit/ops/auditors/sre_a2a.py +179 -0
- agent_ops_cockpit/ops/benchmarker.py +97 -0
- agent_ops_cockpit/ops/cost_optimizer.py +15 -24
- agent_ops_cockpit/ops/discovery.py +214 -0
- agent_ops_cockpit/ops/evidence_bridge.py +30 -63
- agent_ops_cockpit/ops/frameworks.py +124 -1
- agent_ops_cockpit/ops/git_portal.py +74 -0
- agent_ops_cockpit/ops/mcp_hub.py +19 -42
- agent_ops_cockpit/ops/orchestrator.py +477 -277
- agent_ops_cockpit/ops/policy_engine.py +38 -38
- agent_ops_cockpit/ops/reliability.py +121 -52
- agent_ops_cockpit/ops/remediator.py +54 -0
- agent_ops_cockpit/ops/secret_scanner.py +34 -22
- agent_ops_cockpit/ops/swarm.py +17 -27
- agent_ops_cockpit/ops/ui_auditor.py +67 -6
- agent_ops_cockpit/ops/watcher.py +41 -70
- agent_ops_cockpit/ops/watchlist.json +30 -0
- agent_ops_cockpit/optimizer.py +161 -384
- agent_ops_cockpit/tests/test_arch_review.py +6 -6
- agent_ops_cockpit/tests/test_discovery.py +96 -0
- agent_ops_cockpit/tests/test_ops_core.py +56 -0
- agent_ops_cockpit/tests/test_orchestrator_fleet.py +73 -0
- agent_ops_cockpit/tests/test_persona_architect.py +75 -0
- agent_ops_cockpit/tests/test_persona_finops.py +31 -0
- agent_ops_cockpit/tests/test_persona_security.py +55 -0
- agent_ops_cockpit/tests/test_persona_sre.py +43 -0
- agent_ops_cockpit/tests/test_persona_ux.py +42 -0
- agent_ops_cockpit/tests/test_quality_climber.py +2 -2
- agent_ops_cockpit/tests/test_remediator.py +75 -0
- agent_ops_cockpit/tests/test_ui_auditor.py +52 -0
- agentops_cockpit-0.9.8.dist-info/METADATA +172 -0
- agentops_cockpit-0.9.8.dist-info/RECORD +71 -0
- agent_ops_cockpit/tests/test_optimizer.py +0 -68
- agent_ops_cockpit/tests/test_red_team.py +0 -35
- agent_ops_cockpit/tests/test_secret_scanner.py +0 -24
- agentops_cockpit-0.9.5.dist-info/METADATA +0 -246
- agentops_cockpit-0.9.5.dist-info/RECORD +0 -47
- {agentops_cockpit-0.9.5.dist-info → agentops_cockpit-0.9.8.dist-info}/WHEEL +0 -0
- {agentops_cockpit-0.9.5.dist-info → agentops_cockpit-0.9.8.dist-info}/entry_points.txt +0 -0
- {agentops_cockpit-0.9.5.dist-info → agentops_cockpit-0.9.8.dist-info}/licenses/LICENSE +0 -0
|
@@ -0,0 +1,172 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: agentops-cockpit
|
|
3
|
+
Version: 0.9.8
|
|
4
|
+
Summary: Production-grade Agent Operations (AgentOps) Platform
|
|
5
|
+
Project-URL: Homepage, https://github.com/enriquekalven/agent-ops-cockpit
|
|
6
|
+
Project-URL: Bug Tracker, https://github.com/enriquekalven/agent-ops-cockpit/issues
|
|
7
|
+
Author-email: Enrique <enrique@example.com>
|
|
8
|
+
License-File: LICENSE
|
|
9
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Requires-Python: >=3.10
|
|
13
|
+
Requires-Dist: aiohttp>=3.9.0
|
|
14
|
+
Requires-Dist: fastapi>=0.100.0
|
|
15
|
+
Requires-Dist: gitpython>=3.1.0
|
|
16
|
+
Requires-Dist: mcp>=0.1.0
|
|
17
|
+
Requires-Dist: packaging>=23.0
|
|
18
|
+
Requires-Dist: pydantic>=2.0.0
|
|
19
|
+
Requires-Dist: rich>=13.0.0
|
|
20
|
+
Requires-Dist: tenacity>=8.0.0
|
|
21
|
+
Requires-Dist: typer>=0.9.0
|
|
22
|
+
Requires-Dist: uvicorn>=0.20.0
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
|
|
25
|
+
# 🕹️ AgentOps Cockpit
|
|
26
|
+
|
|
27
|
+
<div align="center">
|
|
28
|
+
<img src="public/assets/trinity.png" alt="AgentOps Cockpit Trinity" width="100%" />
|
|
29
|
+
</div>
|
|
30
|
+
|
|
31
|
+
<div align="center">
|
|
32
|
+
<br />
|
|
33
|
+
<a href="https://agent-cockpit.web.app" target="_blank"><strong>🌐 Official Website & Live Demo</strong></a>
|
|
34
|
+
<br /><br />
|
|
35
|
+
<a href="https://deploy.cloud.google.com?repo=https://github.com/enriquekalven/agent-cockpit">
|
|
36
|
+
<img src="https://deploy.cloud.google.com/button.svg" alt="Deploy to Google Cloud" />
|
|
37
|
+
</a>
|
|
38
|
+
<br />
|
|
39
|
+
<br />
|
|
40
|
+
<img src="https://img.shields.io/github/stars/enriquekalven/agent-cockpit?style=for-the-badge&color=ffd700" alt="GitHub Stars" />
|
|
41
|
+
<img src="https://img.shields.io/github/license/enriquekalven/agent-cockpit?style=for-the-badge&color=007bff" alt="License" />
|
|
42
|
+
<img src="https://img.shields.io/badge/Google-Well--Architected-4285F4?style=for-the-badge&logo=google-cloud" alt="Google Well-Architected" />
|
|
43
|
+
<img src="https://img.shields.io/badge/A2A_Standard-Enabled-10b981?style=for-the-badge" alt="A2A Standard" />
|
|
44
|
+
</div>
|
|
45
|
+
|
|
46
|
+
<br />
|
|
47
|
+
|
|
48
|
+
<div align="center">
|
|
49
|
+
<h3>"Infrastructure gives you the pipes. We give you the Intelligence."</h3>
|
|
50
|
+
<p>The developer distribution for building, optimizing, and securing AI agents on Google Cloud.</p>
|
|
51
|
+
</div>
|
|
52
|
+
|
|
53
|
+
---
|
|
54
|
+
|
|
55
|
+
## 📽️ The Mission
|
|
56
|
+
Most AI agent templates stop at a single Python file and an API key. **The AgentOps Cockpit** is for developers moving into production. It provides framework-agnostic governance, safety, and cost guardrails for the entire agentic ecosystem.
|
|
57
|
+
|
|
58
|
+
- **Governance-as-Code**: Audit your agent against [Google Well-Architected](/docs/GOOGLE_ARCHITECTURE.md) best practices with the **Evidence Bridge**—real-time citations for architectural integrity.
|
|
59
|
+
- **SME Persona Audits**: Parallelized review of your codebase by automated [**Principal SMEs**](docs/TECHNICAL_AUDIT_GUIDE.md) across FinOps, SecOps, Architecture, and Quality.
|
|
60
|
+
- **Agentic Trinity**: Dedicated layers for the Engine (Logic), Face (UX), and Cockpit (Ops).
|
|
61
|
+
- **A2A Connectivity**: Implements the [**Agent-to-Agent Transmission Standard**](docs/TECHNICAL_A2A_GUIDE.md) for secure swarm orchestration.
|
|
62
|
+
- **MCP Native**: Registration as a [Model Context Protocol](https://modelcontextprotocol.io) server for 1P/2P/3P tool consumption.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## 🏗️ The Agentic Trinity
|
|
67
|
+
We divide the complexity of production agents into three focused pillars:
|
|
68
|
+
|
|
69
|
+
```mermaid
|
|
70
|
+
graph TD
|
|
71
|
+
subgraph Trinity [The Agentic Trinity 2.0]
|
|
72
|
+
E(The Engine: Reasoning)
|
|
73
|
+
F(The Face: Interface)
|
|
74
|
+
C(The Cockpit: Operations)
|
|
75
|
+
S{Sovereignty & Compliance}
|
|
76
|
+
end
|
|
77
|
+
E <--> C
|
|
78
|
+
F <--> C
|
|
79
|
+
E <--> F
|
|
80
|
+
E -.-> S
|
|
81
|
+
F -.-> S
|
|
82
|
+
C -.-> S
|
|
83
|
+
style Trinity fill:#f8fafc,stroke:#334155,stroke-width:2px
|
|
84
|
+
style S fill:#0ea5e9,color:#fff,stroke:#0284c7
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
- **⚙️ The Engine**: The reasoning core. Built with **ADK**, FastAPI, and Vertex AI.
|
|
88
|
+
- **🎭 The Face**: The user experience. [Adaptive UI surfaces](docs/TECHNICAL_UX_GUIDE.md) and **GenUI** standards via the A2UI spec.
|
|
89
|
+
- **🕹️ The Cockpit**: The operational brain. [Cost control](docs/TECHNICAL_FINOPS_GUIDE.md), semantic caching, shadow routing, and [adversarial audits](docs/TECHNICAL_REDTEAM_GUIDE.md).
|
|
90
|
+
|
|
91
|
+
<div align="center">
|
|
92
|
+
<img src="public/assets/ecosystem.png" alt="Ecosystem Integrations" width="100%" />
|
|
93
|
+
</div>
|
|
94
|
+
|
|
95
|
+
---
|
|
96
|
+
|
|
97
|
+
## 🏛️ v1.3: The "Autonomous Architect" Standard (NEW)
|
|
98
|
+
Evolving from a compliance tool to an autonomous evolution engine. See the [**v1.3 Roadmap**](/docs/ROADMAP_V13.md).
|
|
99
|
+
|
|
100
|
+
- **🚀 Context-Aware Patching**: Moving from templates to LLM-Synthesized PRs that match your project's style.
|
|
101
|
+
- **🧪 Digital Twin Simulation**: `make simulation-run` for high-concurrency "Stress-Reasoning" test. [**Read Guide**](docs/TECHNICAL_ARCH_REVIEW.md).
|
|
102
|
+
- **🧗 Hill Climbing Optimization**: Iterative prompt tuning to reach the Global Peak. [**Read Guide**](docs/TECHNICAL_QUALITY_GUIDE.md).
|
|
103
|
+
- **🌍 Strategic Exit Strategy**: Automated TCO reports and migration plans for moving between cloud providers.
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
## 🚀 Key Innovation: The "Intelligence" Layer
|
|
108
|
+
|
|
109
|
+
### 🛡️ Red Team Auditor (Adversarial SRE)
|
|
110
|
+
Don't wait for your users to find prompt injections. Use the built-in [**Adversarial Evaluator**](docs/TECHNICAL_REDTEAM_GUIDE.md) to launch self-attacks against your agent, testing for PII leaks, instruction overrides, and multilingual jailbreaks.
|
|
111
|
+
|
|
112
|
+
### 🧠 Hive Mind (Semantic Caching)
|
|
113
|
+
**Reduce LLM costs by up to 40%.** The Hive Mind checks for semantically similar queries in 10ms, serving cached answers for common questions without calling the LLM.
|
|
114
|
+
|
|
115
|
+
### 🏛️ Arch Review & Autonomous Evolution
|
|
116
|
+
Every agent in the cockpit is graded against a framework-aware checklist. The Cockpit intelligently detects your stack and runs a tailored [**Architecture Review**](docs/TECHNICAL_ARCH_REVIEW.md). v1.3 introduces **Autonomous Evolution**—the ability to synthesize code fixes directly from audit findings.
|
|
117
|
+
|
|
118
|
+
### 🕹️ MCP Connectivity Hub (Model Context Protocol)
|
|
119
|
+
Stop building one-off tool integrations. The Cockpit provides a unified hub for **MCP Servers**. Connect to 1P/2P/3P tools via the standardized Model Context Protocol for secure, audited tool execution. Start the server with `make mcp-serve`.
|
|
120
|
+
|
|
121
|
+
### 🗄️ Situational Database Audits
|
|
122
|
+
The Cockpit now performs platform-specific performance and security audits for **AlloyDB**, **Pinecone**, **BigQuery**, and **Cloud SQL**.
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
### 🛡️ Advanced Governance & Discovery
|
|
127
|
+
Modern agents don't just live in `agent.py`. The Cockpit uses a centralized **Discovery Engine** to intelligently map your project:
|
|
128
|
+
- **`.gitignore` Compliance**: Zero-noise scanning that respects your project's ignore rules.
|
|
129
|
+
- **Library Isolation**: Intelligently ignores hits in `venv` or `node_modules` to focus on your code.
|
|
130
|
+
- **Config-Driven**: Use `cockpit.yaml` to define custom entry points, exclusions, and CI/CD thresholds.
|
|
131
|
+
|
|
132
|
+
---
|
|
133
|
+
|
|
134
|
+
## ⌨️ Master Command Registry
|
|
135
|
+
|
|
136
|
+
The Cockpit is available as a first-class CLI and a comprehensive Makefile-based operational toolkit.
|
|
137
|
+
|
|
138
|
+
| Registry | Description |
|
|
139
|
+
| :--- | :--- |
|
|
140
|
+
| 🕹️ [**Makefile Commands**](docs/TECHNICAL_COMMANDS_MASTER.md) | Standard local development and orchestration shortcuts. |
|
|
141
|
+
| 🚀 [**UVX Master Guide**](docs/TECHNICAL_UVX_MASTER.md) | Portable, zero-install commands for CI/CD and automation. |
|
|
142
|
+
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
## 🧑💼 Principal SME Persona Approvals
|
|
146
|
+
The Cockpit now features a **Multi-Persona Governance Board**. Every audit result is framed through the lens of a Principal Engineer in that domain:
|
|
147
|
+
* [**🏛️ Architecture**](docs/TECHNICAL_ARCH_REVIEW.md)
|
|
148
|
+
* [**💰 FinOps**](docs/TECHNICAL_FINOPS_GUIDE.md)
|
|
149
|
+
* [**🛡️ Red Team**](docs/TECHNICAL_REDTEAM_GUIDE.md)
|
|
150
|
+
* [**🧗 Quality**](docs/TECHNICAL_QUALITY_GUIDE.md)
|
|
151
|
+
* [**🌐 Infrastructure**](docs/TECHNICAL_INFRA_GUIDE.md)
|
|
152
|
+
* [**🎭 UX/UI**](docs/TECHNICAL_UX_GUIDE.md)
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
## 🚀 1-Click Production Pipeline
|
|
157
|
+
`make deploy-prod` triggers the following lifecycle:
|
|
158
|
+
1. Runs the Quick Safe-Build (`make audit`).
|
|
159
|
+
2. Compiles production frontend assets.
|
|
160
|
+
3. Deploys the Engine to **Google Cloud Run**.
|
|
161
|
+
4. Deploys the Face to **Firebase Hosting**.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## 🤝 Ecosystem & Attribution
|
|
166
|
+
The AgentOps Cockpit is designed to leverage and secure the best-of-breed tools in the Google Cloud ecosystem. We explicitly acknowledge and leverage the excellent work from:
|
|
167
|
+
|
|
168
|
+
* **[GoogleCloudPlatform/agent-starter-pack](https://github.com/GoogleCloudPlatform/agent-starter-pack/)**: We leverage this as a core reference for the **Agent Development Kit (ADK)** patterns and Vertex AI Agent Engine integration.
|
|
169
|
+
* **A2A Standard**: Our implementation follow the Agent-to-Agent Transmission Protocol for swarm intelligence.
|
|
170
|
+
|
|
171
|
+
---
|
|
172
|
+
*Reference: [Google Cloud Architecture Center - Agentic AI Overview](https://docs.cloud.google.com/architecture/agentic-ai-overview)*
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
agent_ops_cockpit/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
2
|
+
agent_ops_cockpit/agent.py,sha256=qJJMQ1GyF_bS-7AEVQcxORPawATXbTYVseEKwcJgCww,4447
|
|
3
|
+
agent_ops_cockpit/cost_control.py,sha256=eO8-3ggK1Kr9iA7S_GURXqUIsDHYyqXF_bBkmCJe_tM,2333
|
|
4
|
+
agent_ops_cockpit/mcp_server.py,sha256=RaFIn0DjyudYheH0yJRkGmOQxMZI3q5VXVrWGoZG10s,4129
|
|
5
|
+
agent_ops_cockpit/optimizer.py,sha256=LCsykerMrlbVvBVTVffo74KD_qh0UaRRYZC61_mmTB4,22327
|
|
6
|
+
agent_ops_cockpit/system_prompt.md,sha256=VlkU4BYDajUoMypkVBRNyeOWre0cUoEgcQHuyVw7TkA,733
|
|
7
|
+
agent_ops_cockpit/cache/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
8
|
+
agent_ops_cockpit/cache/semantic_cache.py,sha256=2ev3ziLXZwBsZcq5slee69W9JqUN8ywANrnLnNrdxkw,1706
|
|
9
|
+
agent_ops_cockpit/cli/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
10
|
+
agent_ops_cockpit/cli/main.py,sha256=oZp-8UZSpEdX2ny7m7NiA0Eyfv2hPTtWe41t6jHCyLQ,12395
|
|
11
|
+
agent_ops_cockpit/eval/__init__.py,sha256=X68nLTYCIbL3U065CSdodzaCTmL94Rf442gV2DoR4E8,23
|
|
12
|
+
agent_ops_cockpit/eval/load_test.py,sha256=FocIP_qho33lHw-3NMXmdUVkgxC8Y_MOEiFtWXD9TKw,3825
|
|
13
|
+
agent_ops_cockpit/eval/quality_climber.py,sha256=kJrAv3-LuelFUInT0A-MzXI5dIBKfYWq0-W9YOXh2uU,6357
|
|
14
|
+
agent_ops_cockpit/eval/red_team.py,sha256=2B1Ipuo_8KMsbDGrjlckkyxcBXySyqECrn556qXxxAs,6837
|
|
15
|
+
agent_ops_cockpit/ops/__init__.py,sha256=YBoDCVs7NvNbjK-kBaFckUTcmd5RBafn0tnsoMR6EFs,22
|
|
16
|
+
agent_ops_cockpit/ops/arch_review.py,sha256=HiL9HVBlJ-JfidAp_Qsv3OCLEKtD5hg62185sK5jaJc,18824
|
|
17
|
+
agent_ops_cockpit/ops/benchmarker.py,sha256=SZ2p-xR1cuh5prPBO9UIjZhh3flFouodz5lkabioCqk,4230
|
|
18
|
+
agent_ops_cockpit/ops/cost_optimizer.py,sha256=c_lGJOmvrtgL4L6Y3Z_ahQHCNkjTBz3g1EhkhFyWqeQ,1655
|
|
19
|
+
agent_ops_cockpit/ops/discovery.py,sha256=m4tlwtOeYeN3TmrevNmZr2dv4z0kbgHZsx8iTTy_BLw,8614
|
|
20
|
+
agent_ops_cockpit/ops/evidence.py,sha256=LRAW57c-2R4ICiMLtc-JA1Tu5dlfO9-VBSUMc3TCLuo,1051
|
|
21
|
+
agent_ops_cockpit/ops/evidence_bridge.py,sha256=C6nT-9KRCjayIAlG_ADDDIubUIa1vQE6rctskAoHDSE,5040
|
|
22
|
+
agent_ops_cockpit/ops/frameworks.py,sha256=ZLSpMjkzu1IeJzWPHR9Lgy_OhvJbGnLmIXyicivx-gw,29089
|
|
23
|
+
agent_ops_cockpit/ops/git_portal.py,sha256=1KSENc_sD1NnD5C9U5aBKpcgffG_CY06BgO_FouYj3Q,2682
|
|
24
|
+
agent_ops_cockpit/ops/mcp_hub.py,sha256=7XvpNJ0HwjBcHRau11UMYOXKbxUPaXgj-WAmdbmsKUw,2855
|
|
25
|
+
agent_ops_cockpit/ops/memory_optimizer.py,sha256=whsKhAuJkEJRa2dxfVeJC_xxwDwKjhx5tnmOmkiKgIQ,1635
|
|
26
|
+
agent_ops_cockpit/ops/orchestrator.py,sha256=TJCaDztI6UoHbbbzhSIoQap8A6ycreYdDSlKF4dnl-U,37173
|
|
27
|
+
agent_ops_cockpit/ops/pii_scrubber.py,sha256=7YiX7tTI-hLCiDXd4vKonOo0byg7kswRn0aInepzrjU,1518
|
|
28
|
+
agent_ops_cockpit/ops/policies.json,sha256=yhQz9MZbSlb8rNXMzVW3qcTOBNjNXB5XqQjR9p_GqnA,568
|
|
29
|
+
agent_ops_cockpit/ops/policy_engine.py,sha256=qGVeINNCP_XIcJhNFZbaWem60qHwvJLkLfI8iSrWq40,4443
|
|
30
|
+
agent_ops_cockpit/ops/reliability.py,sha256=l1wQVT9q0ht9guojB-7IeT4fjBPvKw79Gk_KCxt1jFM,7570
|
|
31
|
+
agent_ops_cockpit/ops/remediator.py,sha256=URox7i--NTuFebBdz6ovEuUlOVHkKQTllsE9NWDItLY,2260
|
|
32
|
+
agent_ops_cockpit/ops/secret_scanner.py,sha256=LgCZ5LmvTmC8LgDkCb15xn4T-a3EkTfcxjvqRq37xXY,3878
|
|
33
|
+
agent_ops_cockpit/ops/swarm.py,sha256=EDcbUjruefwwzgfoo1Htb3btptQpTx2O8-kQi_HldbI,2438
|
|
34
|
+
agent_ops_cockpit/ops/ui_auditor.py,sha256=Ozc5dNCFL1cDZ8BhViK73iP5didLl8HG7lAPWudGfSE,9582
|
|
35
|
+
agent_ops_cockpit/ops/watcher.py,sha256=TT1RrdTN4nSgMsg-pDLm6wQtDg62X_uFAuZpxLq_MNU,5265
|
|
36
|
+
agent_ops_cockpit/ops/watchlist.json,sha256=n-4UO336Ob0ucRP7gt4Yozo00CgmSl_Am0bfw-NBAjY,4310
|
|
37
|
+
agent_ops_cockpit/ops/auditors/base.py,sha256=CrWU0Bmzzm887svila8sYR46R2-Is9tx9VT3IZCnias,1538
|
|
38
|
+
agent_ops_cockpit/ops/auditors/behavioral.py,sha256=FnigfqpPYpZNQGkT31UlIX8d_UjIksG42tAviMIjuJI,1493
|
|
39
|
+
agent_ops_cockpit/ops/auditors/compliance.py,sha256=OUOx16fJbny0wu50oeal-M5mnJ9J2QQb_-D8Er_z4ns,1642
|
|
40
|
+
agent_ops_cockpit/ops/auditors/dependency.py,sha256=T2pvXWgTMWbJjvQfY9Njnjjj7S2f_Rx2X4nVlK8nwz0,2022
|
|
41
|
+
agent_ops_cockpit/ops/auditors/finops.py,sha256=Q7WA_qIlT230FWiFqAbxvu7xUJK0fbUa7hOcRxytzTc,2018
|
|
42
|
+
agent_ops_cockpit/ops/auditors/graph.py,sha256=7sOANjXOclwkZrdmomixRxX7Vjb0rtRtbzqoEOZwHdg,2381
|
|
43
|
+
agent_ops_cockpit/ops/auditors/pivot.py,sha256=IKoycFpywdV1X9g6uNlquBdMoCrA1qYWmaXcNmFpoX8,2686
|
|
44
|
+
agent_ops_cockpit/ops/auditors/reasoning.py,sha256=WDnd-phTgimoQb9VaAlavtZ4oY13WT8B5seGbBaVDAw,3516
|
|
45
|
+
agent_ops_cockpit/ops/auditors/reliability.py,sha256=Gy20FEdWEm3GEMa4e4i698zDD3x-kcDPWiy5y03oLrc,3578
|
|
46
|
+
agent_ops_cockpit/ops/auditors/security.py,sha256=tDgmKPm74NI51d6uJ9ncRw2Csm91ffavAT5m-zoWQT0,4594
|
|
47
|
+
agent_ops_cockpit/ops/auditors/sme_v12.py,sha256=8zQ9a6A7xVq5dzOkxJUoHgDftBZC_YFL20dfm8jKMCs,3809
|
|
48
|
+
agent_ops_cockpit/ops/auditors/sovereignty.py,sha256=Odz8GG3YxkYEyUkdzCmRmBx0dlyBrIpi7khpDlhmhX4,3680
|
|
49
|
+
agent_ops_cockpit/ops/auditors/sre_a2a.py,sha256=IgDfbwKRRcjL_dMfYNN_SM9xcmMArxUJ1znX55XSzO0,9941
|
|
50
|
+
agent_ops_cockpit/shadow/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
51
|
+
agent_ops_cockpit/shadow/router.py,sha256=XB87GEZKONY3SuXpalKFwuzfDEyXRn9wLZqbsyB-f_o,2684
|
|
52
|
+
agent_ops_cockpit/tests/golden_set.json,sha256=dWpYdINQBnZiuhFSxnJfp_oHivPzTAoQf6nuxiPpzao,3446
|
|
53
|
+
agent_ops_cockpit/tests/test_agent.py,sha256=kk9QT2bywvC8bDy8iqQF8mExHXwYRAqPTCPY1MYH_Xo,1232
|
|
54
|
+
agent_ops_cockpit/tests/test_arch_review.py,sha256=J82-hVh0a9Urd7r65EvR3TlmsGBNTXb0GKfyqH_hEb4,1591
|
|
55
|
+
agent_ops_cockpit/tests/test_discovery.py,sha256=eQtPtIfGuCO27R3D_MQuHTgVkR2y-18dQpch_RYgIeo,3808
|
|
56
|
+
agent_ops_cockpit/tests/test_frameworks.py,sha256=8DaiWba6WnAQ6wXj3ZG1nvUxY6y2_PLQOboTk_z4ayc,3075
|
|
57
|
+
agent_ops_cockpit/tests/test_ops_core.py,sha256=lp8B0duhauMR4-HF-Uhr2-z3nXDP042yD6FhaPiIbAk,1955
|
|
58
|
+
agent_ops_cockpit/tests/test_orchestrator_fleet.py,sha256=CPaigPK1OhaZVmMGonTKvmHVtA7VmSzTO_RlGoDZXR0,2499
|
|
59
|
+
agent_ops_cockpit/tests/test_persona_architect.py,sha256=KQdo5-SZ-uy1nvtv3nsm4AgIPdVrUeQ4awcqEE1DjI4,3284
|
|
60
|
+
agent_ops_cockpit/tests/test_persona_finops.py,sha256=GMqxIpKVLOjm0kawxu4HCDV27l6PIOEK98Gb4CeUr04,1420
|
|
61
|
+
agent_ops_cockpit/tests/test_persona_security.py,sha256=wWbAfvCf1x7PyV-zrO7VibZSahBw-WGZA9-L_iezHug,2472
|
|
62
|
+
agent_ops_cockpit/tests/test_persona_sre.py,sha256=WgD_rDA9EKMiBWYfdLjcGRabZzPA6Oxan9M2llWZUnw,1962
|
|
63
|
+
agent_ops_cockpit/tests/test_persona_ux.py,sha256=ut5a8nugErlukc2GmRV-d3u_x75E6nyW82uPh8tQTpk,1634
|
|
64
|
+
agent_ops_cockpit/tests/test_quality_climber.py,sha256=_r52F_61_t0Rw4KbnCJLV47x-8Tih_bwllFv5OwFZIQ,697
|
|
65
|
+
agent_ops_cockpit/tests/test_remediator.py,sha256=q96WTG5q4BxSYSc_-GD1CYiISJcfOLoHunDsmD8WDAQ,2259
|
|
66
|
+
agent_ops_cockpit/tests/test_ui_auditor.py,sha256=OWliV73cfT3e47WRkUc8QbXqFAZCnUKhWw_DPvZlUKs,1973
|
|
67
|
+
agentops_cockpit-0.9.8.dist-info/METADATA,sha256=dBGhGS1RmQ4atUaJoZG9DYijjokeTY2KLDgRqKDJ6VU,8707
|
|
68
|
+
agentops_cockpit-0.9.8.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
|
|
69
|
+
agentops_cockpit-0.9.8.dist-info/entry_points.txt,sha256=suNKteVr6LDBA2FXiepe029Ox6f4yZsbQ1Fy7d_zyzc,162
|
|
70
|
+
agentops_cockpit-0.9.8.dist-info/licenses/LICENSE,sha256=XNJEk4bvf88tBnKqHdGBGi10l9yJWv2yLWPJvvVie1c,1071
|
|
71
|
+
agentops_cockpit-0.9.8.dist-info/RECORD,,
|
|
@@ -1,68 +0,0 @@
|
|
|
1
|
-
from agent_ops_cockpit.optimizer import analyze_code
|
|
2
|
-
|
|
3
|
-
def test_analyze_openai_missing_cache():
|
|
4
|
-
code = "import openai\nclient = openai.OpenAI()"
|
|
5
|
-
issues = analyze_code(code)
|
|
6
|
-
assert any(issue.id == "openai_caching" for issue in issues)
|
|
7
|
-
|
|
8
|
-
def test_analyze_anthropic_missing_orchestrator():
|
|
9
|
-
code = "import anthropic\nclient = anthropic.Anthropic()"
|
|
10
|
-
issues = analyze_code(code)
|
|
11
|
-
assert any(issue.id == "anthropic_orchestration" for issue in issues)
|
|
12
|
-
|
|
13
|
-
def test_analyze_microsoft_missing_workflow():
|
|
14
|
-
code = "from autogen import UserProxyAgent, AssistantAgent"
|
|
15
|
-
issues = analyze_code(code)
|
|
16
|
-
assert any(issue.id == "ms_workflows" for issue in issues)
|
|
17
|
-
|
|
18
|
-
def test_analyze_aws_missing_action_groups():
|
|
19
|
-
code = "import boto3\nbedrock = boto3.client('bedrock-agent-runtime')"
|
|
20
|
-
issues = analyze_code(code)
|
|
21
|
-
assert any(issue.id == "aws_action_groups" for issue in issues)
|
|
22
|
-
|
|
23
|
-
def test_analyze_copilotkit_missing_shared_state():
|
|
24
|
-
code = "import copilotkit\n# Some logic without state sync"
|
|
25
|
-
issues = analyze_code(code)
|
|
26
|
-
assert any(issue.id == "copilot_state" for issue in issues)
|
|
27
|
-
|
|
28
|
-
def test_analyze_model_routing_pro_only():
|
|
29
|
-
code = "model = 'gemini-1.5-pro'"
|
|
30
|
-
issues = analyze_code(code)
|
|
31
|
-
assert any(issue.id == "model_routing" for issue in issues)
|
|
32
|
-
|
|
33
|
-
def test_analyze_missing_semantic_cache():
|
|
34
|
-
code = "def chat(): pass"
|
|
35
|
-
issues = analyze_code(code)
|
|
36
|
-
assert any(issue.id == "semantic_caching" for issue in issues)
|
|
37
|
-
|
|
38
|
-
def test_analyze_context_caching():
|
|
39
|
-
code = '"""' + "A" * 300 + '"""'
|
|
40
|
-
issues = analyze_code(code)
|
|
41
|
-
assert any(issue.id == "context_caching" for issue in issues)
|
|
42
|
-
|
|
43
|
-
def test_analyze_infrastructure_optimizations():
|
|
44
|
-
# Cloud Run
|
|
45
|
-
cr_code = "# Running on Cloud Run"
|
|
46
|
-
cr_issues = analyze_code(cr_code)
|
|
47
|
-
assert any(issue.id == "cr_startup_boost" for issue in cr_issues)
|
|
48
|
-
|
|
49
|
-
# GKE
|
|
50
|
-
gke_code = "# Running on GKE with Kubernetes"
|
|
51
|
-
gke_issues = analyze_code(gke_code)
|
|
52
|
-
assert any(issue.id == "gke_identity" for issue in gke_issues)
|
|
53
|
-
|
|
54
|
-
def test_analyze_language_optimizations():
|
|
55
|
-
# Go
|
|
56
|
-
go_code = "state := make(map[string]int)"
|
|
57
|
-
go_issues = analyze_code(go_code, "main.go")
|
|
58
|
-
assert any(issue.id == "go_concurrency" for issue in go_issues)
|
|
59
|
-
|
|
60
|
-
# NodeJS
|
|
61
|
-
js_code = "import axios from 'axios'"
|
|
62
|
-
js_issues = analyze_code(js_code, "app.ts")
|
|
63
|
-
assert any(issue.id == "node_native_fetch" for issue in js_issues)
|
|
64
|
-
def test_analyze_langgraph_optimizations():
|
|
65
|
-
code = "from langgraph.graph import StateGraph"
|
|
66
|
-
issues = analyze_code(code)
|
|
67
|
-
assert any(issue.id == "langgraph_persistence" for issue in issues)
|
|
68
|
-
assert any(issue.id == "langgraph_recursion" for issue in issues)
|
|
@@ -1,35 +0,0 @@
|
|
|
1
|
-
from typer.testing import CliRunner
|
|
2
|
-
from agent_ops_cockpit.eval.red_team import app
|
|
3
|
-
|
|
4
|
-
runner = CliRunner()
|
|
5
|
-
|
|
6
|
-
def test_red_team_secure_agent(tmp_path):
|
|
7
|
-
# Create a "secure" agent file
|
|
8
|
-
agent_file = tmp_path / "secure_agent.py"
|
|
9
|
-
agent_file.write_text("""
|
|
10
|
-
# Scrubber for PII
|
|
11
|
-
def scrub_pii(text): pass
|
|
12
|
-
# Guardrails and vllm enabled
|
|
13
|
-
# Safety filters enabled
|
|
14
|
-
# Uses proxy for secrets
|
|
15
|
-
# i18n and lang support enabled
|
|
16
|
-
# persona and system_prompt protected
|
|
17
|
-
# Very long agent logic to resist override ... """ + "A" * 600)
|
|
18
|
-
|
|
19
|
-
result = runner.invoke(app, [str(agent_file)])
|
|
20
|
-
assert result.exit_code == 0
|
|
21
|
-
assert "Your agent is production-hardened" in result.stdout
|
|
22
|
-
|
|
23
|
-
def test_red_team_vulnerable_agent(tmp_path):
|
|
24
|
-
# Create a "vulnerable" agent file
|
|
25
|
-
agent_file = tmp_path / "vulnerable_agent.py"
|
|
26
|
-
agent_file.write_text("""
|
|
27
|
-
# Simple agent, no scrub, no safety, secrets in code
|
|
28
|
-
secret = "my-api-key"
|
|
29
|
-
def chat(q): return q
|
|
30
|
-
""")
|
|
31
|
-
|
|
32
|
-
result = runner.invoke(app, [str(agent_file)])
|
|
33
|
-
assert result.exit_code == 1
|
|
34
|
-
assert "BREACH" in result.stdout
|
|
35
|
-
assert "PII Extraction" in result.stdout
|
|
@@ -1,24 +0,0 @@
|
|
|
1
|
-
import re
|
|
2
|
-
from agent_ops_cockpit.ops.secret_scanner import SECRET_PATTERNS
|
|
3
|
-
|
|
4
|
-
def test_google_api_key_pattern():
|
|
5
|
-
key = "AIzaSyD-1234567890abcdefghijklmnopqrstuv"
|
|
6
|
-
assert re.search(SECRET_PATTERNS["Google API Key"], key)
|
|
7
|
-
|
|
8
|
-
def test_aws_key_pattern():
|
|
9
|
-
key = "AKIA1234567890ABCDEF"
|
|
10
|
-
assert re.search(SECRET_PATTERNS["AWS Access Key"], key)
|
|
11
|
-
|
|
12
|
-
def test_bearer_token_pattern():
|
|
13
|
-
token = "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"
|
|
14
|
-
assert re.search(SECRET_PATTERNS["Generic Bearer Token"], token)
|
|
15
|
-
|
|
16
|
-
def test_hardcoded_variable_pattern():
|
|
17
|
-
code1 = 'api_key = "sk-1234567890abcdef"'
|
|
18
|
-
code2 = 'client_secret = "secret-key-123456"'
|
|
19
|
-
assert re.search(SECRET_PATTERNS["Hardcoded API Variable"], code1)
|
|
20
|
-
assert re.search(SECRET_PATTERNS["Hardcoded API Variable"], code2)
|
|
21
|
-
|
|
22
|
-
def test_service_account_pattern():
|
|
23
|
-
json_snippet = '"type": "service_account"'
|
|
24
|
-
assert re.search(SECRET_PATTERNS["GCP Service Account"], json_snippet)
|
|
@@ -1,246 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.4
|
|
2
|
-
Name: agentops-cockpit
|
|
3
|
-
Version: 0.9.5
|
|
4
|
-
Summary: Production-grade Agent Operations (AgentOps) Platform
|
|
5
|
-
Project-URL: Homepage, https://github.com/enriquekalven/agent-ops-cockpit
|
|
6
|
-
Project-URL: Bug Tracker, https://github.com/enriquekalven/agent-ops-cockpit/issues
|
|
7
|
-
Author-email: Enrique <enrique@example.com>
|
|
8
|
-
License-File: LICENSE
|
|
9
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
10
|
-
Classifier: Operating System :: OS Independent
|
|
11
|
-
Classifier: Programming Language :: Python :: 3
|
|
12
|
-
Requires-Python: >=3.10
|
|
13
|
-
Requires-Dist: aiohttp>=3.9.0
|
|
14
|
-
Requires-Dist: fastapi>=0.100.0
|
|
15
|
-
Requires-Dist: gitpython>=3.1.0
|
|
16
|
-
Requires-Dist: mcp>=0.1.0
|
|
17
|
-
Requires-Dist: packaging>=23.0
|
|
18
|
-
Requires-Dist: pydantic>=2.0.0
|
|
19
|
-
Requires-Dist: rich>=13.0.0
|
|
20
|
-
Requires-Dist: tenacity>=8.0.0
|
|
21
|
-
Requires-Dist: typer>=0.9.0
|
|
22
|
-
Requires-Dist: uvicorn>=0.20.0
|
|
23
|
-
Description-Content-Type: text/markdown
|
|
24
|
-
|
|
25
|
-
# 🕹️ AgentOps Cockpit
|
|
26
|
-
|
|
27
|
-
<div align="center">
|
|
28
|
-
<img src="public/assets/trinity.png" alt="AgentOps Cockpit Trinity" width="100%" />
|
|
29
|
-
</div>
|
|
30
|
-
|
|
31
|
-
<div align="center">
|
|
32
|
-
<br />
|
|
33
|
-
<a href="https://agent-cockpit.web.app" target="_blank"><strong>🌐 Official Website & Live Demo</strong></a>
|
|
34
|
-
<br /><br />
|
|
35
|
-
<a href="https://deploy.cloud.google.com?repo=https://github.com/enriquekalven/agent-cockpit">
|
|
36
|
-
<img src="https://deploy.cloud.google.com/button.svg" alt="Deploy to Google Cloud" />
|
|
37
|
-
</a>
|
|
38
|
-
<br />
|
|
39
|
-
<br />
|
|
40
|
-
<img src="https://img.shields.io/github/stars/enriquekalven/agent-cockpit?style=for-the-badge&color=ffd700" alt="GitHub Stars" />
|
|
41
|
-
<img src="https://img.shields.io/github/license/enriquekalven/agent-cockpit?style=for-the-badge&color=007bff" alt="License" />
|
|
42
|
-
<img src="https://img.shields.io/badge/Google-Well--Architected-4285F4?style=for-the-badge&logo=google-cloud" alt="Google Well-Architected" />
|
|
43
|
-
<img src="https://img.shields.io/badge/A2A_Standard-Enabled-10b981?style=for-the-badge" alt="A2A Standard" />
|
|
44
|
-
</div>
|
|
45
|
-
|
|
46
|
-
<br />
|
|
47
|
-
|
|
48
|
-
<div align="center">
|
|
49
|
-
<h3>"Infrastructure gives you the pipes. We give you the Intelligence."</h3>
|
|
50
|
-
<p>The developer distribution for building, optimizing, and securing AI agents on Google Cloud.</p>
|
|
51
|
-
</div>
|
|
52
|
-
|
|
53
|
-
---
|
|
54
|
-
|
|
55
|
-
## 📽️ The Mission
|
|
56
|
-
Most AI agent templates stop at a single Python file and an API key. **The AgentOps Cockpit** is for developers moving into production. It provides framework-agnostic governance, safety, and cost guardrails for the entire agentic ecosystem.
|
|
57
|
-
|
|
58
|
-
- **Governance-as-Code**: Audit your agent against [Google Well-Architected](/docs/google-architecture) best practices with the **Evidence Bridge**—real-time citations for architectural integrity.
|
|
59
|
-
- **SME Persona Audits**: Parallelized review of your codebase by automated "Principal SMEs" across FinOps, SecOps, and Architecture.
|
|
60
|
-
- **Agentic Trinity**: Dedicated layers for the Engine (Logic), Face (UX), and Cockpit (Ops).
|
|
61
|
-
- **A2A Connectivity**: Implements the [Agent-to-Agent Transmission Standard](/A2A_GUIDE.md) for secure swarm orchestration.
|
|
62
|
-
- **MCP Native**: Registration as a [Model Context Protocol](https://modelcontextprotocol.io) server for 1P/2P/3P tool consumption.
|
|
63
|
-
|
|
64
|
-
---
|
|
65
|
-
|
|
66
|
-
## 🏗️ The Agentic Trinity
|
|
67
|
-
We divide the complexity of production agents into three focused pillars:
|
|
68
|
-
|
|
69
|
-
```mermaid
|
|
70
|
-
graph LR
|
|
71
|
-
subgraph Trinity [The Agentic Trinity]
|
|
72
|
-
E(The Engine: Reasoning)
|
|
73
|
-
F(The Face: Interface)
|
|
74
|
-
C(The Cockpit: Operations)
|
|
75
|
-
end
|
|
76
|
-
E <--> C
|
|
77
|
-
F <--> C
|
|
78
|
-
E <--> F
|
|
79
|
-
style Trinity fill:#f9f9f9,stroke:#333,stroke-width:2px
|
|
80
|
-
```
|
|
81
|
-
|
|
82
|
-
- **⚙️ The Engine**: The reasoning core. Built with **ADK**, FastAPI, and Vertex AI.
|
|
83
|
-
- **🎭 The Face**: The user experience. Adaptive UI surfaces and **GenUI** standards via the A2UI spec.
|
|
84
|
-
- **🕹️ The Cockpit**: The operational brain. Cost control, semantic caching, shadow routing, and adversarial audits.
|
|
85
|
-
|
|
86
|
-
<div align="center">
|
|
87
|
-
<img src="public/assets/ecosystem.png" alt="Ecosystem Integrations" width="100%" />
|
|
88
|
-
</div>
|
|
89
|
-
|
|
90
|
-
---
|
|
91
|
-
|
|
92
|
-
## 🌐 Framework Agnostic Governance
|
|
93
|
-
The Cockpit isn't just for ADK. It provides **Best Practices as Code** across all major agentic frameworks:
|
|
94
|
-
|
|
95
|
-
<div align="center">
|
|
96
|
-
<img src="https://img.shields.io/badge/OpenAI_Agentkit-412991?style=for-the-badge&logo=openai" alt="OpenAI Agentkit" />
|
|
97
|
-
<img src="https://img.shields.io/badge/Anthropic_Claude-D97757?style=for-the-badge&logo=anthropic" alt="Anthropic" />
|
|
98
|
-
<img src="https://img.shields.io/badge/Microsoft_AutoGen-0078d4?style=for-the-badge&logo=microsoft" alt="Microsoft" />
|
|
99
|
-
<img src="https://img.shields.io/badge/AWS_Bedrock-FF9900?style=for-the-badge&logo=amazon-aws" alt="AWS" />
|
|
100
|
-
<img src="https://img.shields.io/badge/CopilotKit.ai-6366f1?style=for-the-badge" alt="CopilotKit" />
|
|
101
|
-
<img src="https://img.shields.io/badge/LangChain-1C3C3C?style=for-the-badge" alt="LangChain" />
|
|
102
|
-
<img src="https://img.shields.io/badge/ADK-4285F4?style=for-the-badge&logo=google-cloud" alt="ADK" />
|
|
103
|
-
<img src="public/assets/workflow.png" alt="Operational Workflow" width="100%" />
|
|
104
|
-
</div>
|
|
105
|
-
|
|
106
|
-
## 🛠️ Operational Flow
|
|
107
|
-
|
|
108
|
-
```mermaid
|
|
109
|
-
sequenceDiagram
|
|
110
|
-
participant U as User
|
|
111
|
-
participant C as Cockpit
|
|
112
|
-
participant E as Engine
|
|
113
|
-
participant F as Face
|
|
114
|
-
|
|
115
|
-
U->>C: Prompt / Input
|
|
116
|
-
C->>C: Policy Audit (RFC-307)
|
|
117
|
-
C->>E: Execute Logic / Tools
|
|
118
|
-
E->>C: Action Proposals
|
|
119
|
-
C->>E: Approve (HITL)
|
|
120
|
-
E->>F: GenUI Metadata
|
|
121
|
-
F->>U: Reactive Surface (A2UI)
|
|
122
|
-
```
|
|
123
|
-
|
|
124
|
-
<br />
|
|
125
|
-
|
|
126
|
-
<div align="center">
|
|
127
|
-
<img src="https://img.shields.io/badge/Python-3776AB?style=flat-square&logo=python&logoColor=white" alt="Python" />
|
|
128
|
-
<img src="https://img.shields.io/badge/Go-00ADD8?style=flat-square&logo=go&logoColor=white" alt="Go" />
|
|
129
|
-
<img src="https://img.shields.io/badge/NodeJS-339933?style=flat-square&logo=node.js&logoColor=white" alt="NodeJS" />
|
|
130
|
-
<img src="https://img.shields.io/badge/TypeScript-3178C6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript" />
|
|
131
|
-
<img src="https://img.shields.io/badge/Streamlit-FF4B4B?style=flat-square&logo=streamlit&logoColor=white" alt="Streamlit" />
|
|
132
|
-
<img src="https://img.shields.io/badge/Angular-DD0031?style=flat-square&logo=angular&logoColor=white" alt="Angular" />
|
|
133
|
-
<img src="https://img.shields.io/badge/Lit-324FFF?style=flat-square&logo=lit&logoColor=white" alt="Lit" />
|
|
134
|
-
</div>
|
|
135
|
-
|
|
136
|
-
Whether you are building a swarm in **CrewAI**, a Go-based high-perf engine, or a **Streamlit** dashboard, the Cockpit ensures your agent maps to the **Google Well-Architected Framework**.
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
---
|
|
140
|
-
|
|
141
|
-
## 🚀 Key Innovation: The "Intelligence" Layer
|
|
142
|
-
|
|
143
|
-
### 🛡️ Red Team Auditor (Self-Hacking)
|
|
144
|
-
Don't wait for your users to find prompt injections. Use the built-in Adversarial Evaluator to launch self-attacks against your agent, testing for PII leaks, instruction overrides, and safety filter bypasses.
|
|
145
|
-
|
|
146
|
-
### 🧠 Hive Mind (Semantic Caching)
|
|
147
|
-
**Reduce LLM costs by up to 40%.** The Hive Mind checks for semantically similar queries in 10ms, serving cached answers for common questions without calling the LLM.
|
|
148
|
-
|
|
149
|
-
### 🏛️ Arch Review & Framework Detection
|
|
150
|
-
Every agent in the cockpit is graded against a framework-aware checklist. The Cockpit intelligently detects your stack—**Google ADK**, **OpenAI Agentkit**, **Anthropic Claude**, **Microsoft AutoGen/Semantic Kernel**, **AWS Bedrock Agents**, or **CopilotKit**—and runs a tailored audit against corresponding production standards. Use `make arch-review` to verify your **Governance-as-Code**.
|
|
151
|
-
|
|
152
|
-
### 🕹️ MCP Connectivity Hub (Model Context Protocol)
|
|
153
|
-
Stop building one-off tool integrations. The Cockpit provides a unified hub for **MCP Servers**. Connect to Google Search, Slack, or your internal databases via the standardized Model Context Protocol for secure, audited tool execution. Start the server with `make mcp-serve`.
|
|
154
|
-
|
|
155
|
-
### 🗄️ Situational Database Audits
|
|
156
|
-
The Cockpit now performs platform-specific performance and security audits for:
|
|
157
|
-
- **AlloyDB**: Optimizes for the **Columnar Engine** (100x query speedup).
|
|
158
|
-
- **Pinecone**: Suggests **gRPC** and **Namespace Isolation** for high-perf RAG.
|
|
159
|
-
- **BigQuery**: Suggests **BQ Vector Search** for serverless, cost-effective grounding.
|
|
160
|
-
- **Cloud SQL**: Enforces **IAM-based authentication** via the official Python Connector.
|
|
161
|
-
|
|
162
|
-
### 🧗 Quality Hill Climbing (ADK Evaluation)
|
|
163
|
-
Following **Google ADK Evaluation** best practices, the Cockpit provides an iterative optimization loop. `make quality-baseline` runs your agent against a "Golden Dataset" using **LLM-as-a-Judge** scoring (Response Match & Tool Trajectory), climbing the quality curve until production-grade fidelity is reached.
|
|
164
|
-
|
|
165
|
-
### 🛑 Mandatory Governance Enforcement (NEW)
|
|
166
|
-
The Cockpit now acts as a mandatory gate for production.
|
|
167
|
-
- **Blocking CI/CD**: GitHub Actions now fail if **High Impact** cost issues or **Red Team** security vulnerabilities are detected.
|
|
168
|
-
- **Build-Time Audit**: The `Dockerfile` includes a mandatory `RUN` audit step. If your agent is not "Well-Architected," the container image will fail to build.
|
|
169
|
-
|
|
170
|
-
---
|
|
171
|
-
|
|
172
|
-
## ⌨️ Quick Start
|
|
173
|
-
|
|
174
|
-
The Cockpit is available as a first-class CLI on PyPI.
|
|
175
|
-
|
|
176
|
-
```bash
|
|
177
|
-
# 1. Install the Cockpit globally
|
|
178
|
-
pip install agentops-cockpit
|
|
179
|
-
|
|
180
|
-
# 2. Run Global Audit (Produces unified report)
|
|
181
|
-
agent-ops report --mode quick # ⚡ Quick Safe-Build
|
|
182
|
-
agent-ops report --mode deep # 🚀 Full System Audit
|
|
183
|
-
|
|
184
|
-
# 3. Guardrail Policy Audit (RFC-307)
|
|
185
|
-
agent-ops policy-audit --text "How to make a bomb?"
|
|
186
|
-
|
|
187
|
-
# 4. Global Scaffolding
|
|
188
|
-
agent-ops-cockpit create <name> --ui a2ui
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
### 🔍 Agent Optimizer v2 (Situational Intelligence)
|
|
192
|
-
The Cockpit doesn't just look for generic waste. It now performs **Triple-State Analysis**:
|
|
193
|
-
- **Legacy Workarounds**: Suggests situational fixes for older SDK versions (e.g., manual prompt pruning).
|
|
194
|
-
- **Modernization Paths**: Highlights native performance gains (e.g., 90% cost reduction via Context Caching) available in latest SDKs.
|
|
195
|
-
- **Conflict Guard**: Real-time cross-package validation to prevent architectural deadlocks (e.g., CrewAI vs LangGraph state loops).
|
|
196
|
-
|
|
197
|
-
### ⚡ Quick-Safe Build (12x Faster Loops)
|
|
198
|
-
Development velocity shouldn't sacrifice safety. The new `--quick` mode in the auditor reduces check latency from **1.8s to 0.15s**, providing sub-second feedback while maintaining the integrity of the Conflict Guard and Architecture Review.
|
|
199
|
-
|
|
200
|
-
---
|
|
201
|
-
|
|
202
|
-
### 🧑💼 Principal SME Persona Approvals
|
|
203
|
-
The Cockpit now features a **Multi-Persona Governance Board**. Every audit result is framed through the lens of a Principal Engineer in that domain (Security, Legal, FinOps, UX), ensuring your agent is compliant with organizational standards.
|
|
204
|
-
|
|
205
|
-
### 📄 Export & Reporting
|
|
206
|
-
* **HTML/PDF Export**: Every audit automatically generates `cockpit_report.html`, a premium, printable report ready for PDF export.
|
|
207
|
-
* **Email Reports**: Send audit results directly to stakeholders via the CLI.
|
|
208
|
-
|
|
209
|
-
---
|
|
210
|
-
|
|
211
|
-
## 📊 Local Development
|
|
212
|
-
The Cockpit provides a unified "Mission Control" to evaluate your agents instantly.
|
|
213
|
-
|
|
214
|
-
```bash
|
|
215
|
-
make audit # 🕹️ Run Master Audit (Persona Approved)
|
|
216
|
-
make audit-deep # 🚀 Run Deep Audit (Full SME Verdicts)
|
|
217
|
-
make email-report # 📧 Email the latest result to a stakeholder
|
|
218
|
-
make diagnose # 🩺 Run environment health check
|
|
219
|
-
make optimizer-audit # 🔍 Run Optimizer on specific agent files
|
|
220
|
-
make reliability # 🛡️ Run unit tests and regression suite
|
|
221
|
-
make dev # Start the local Engine + Face stack
|
|
222
|
-
make arch-review # 🏛️ Run the Google Well-Architected design review
|
|
223
|
-
make quality-baseline # 🧗 Run iterative 'Hill Climbing' quality audit
|
|
224
|
-
make red-team # Execute a white-hat security audit
|
|
225
|
-
make deploy-prod # 🚀 1-click deploy to Google Cloud
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
---
|
|
229
|
-
|
|
230
|
-
## 🧭 Roadmap
|
|
231
|
-
- [x] **One-Click GitHub Action**: Automated governance audits on every PR.
|
|
232
|
-
- [x] **Mandatory Build Gates**: Blocking CI/CD and Container audits for production safety.
|
|
233
|
-
- [x] **Multi-Agent Orchestrator**: Standardized A2A Swarm/Coordinator patterns.
|
|
234
|
-
- [ ] **Visual Mission Control**: Real-time cockpit observability dashboard.
|
|
235
|
-
|
|
236
|
-
[View full roadmap →](/ROADMAP.md)
|
|
237
|
-
|
|
238
|
-
---
|
|
239
|
-
|
|
240
|
-
## 🤝 Community
|
|
241
|
-
- **Star this repo** to help us build the future of AgentOps.
|
|
242
|
-
- **Join the Discussion** for patterns on Google Cloud.
|
|
243
|
-
- **Contribute**: Read our [Contributing Guide](/CONTRIBUTING.md).
|
|
244
|
-
|
|
245
|
-
---
|
|
246
|
-
*Reference: [Google Cloud Architecture Center - Agentic AI Overview](https://docs.cloud.google.com/architecture/agentic-ai-overview)*
|