agy-squad 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/full-review/SKILL.md +89 -0
- package/.agents/skills/full-review/references/evidence-intake.md +146 -0
- package/.agents/skills/full-review/references/report-template.md +216 -0
- package/.agents/skills/full-review/references/review-roles.md +91 -0
- package/.agents/skills/pr-review/SKILL.md +60 -0
- package/.agents/skills/pr-review/references/pr-review-standard.md +163 -0
- package/.agents/skills/self-diagnosis/SKILL.md +63 -0
- package/.agents/skills/self-diagnosis/references/execution-and-teardown.md +71 -0
- package/.agents/skills/self-diagnosis/references/optimization-and-review-rubric.md +61 -0
- package/.agents/skills/self-diagnosis/resources/comparative-research-manifest.json +42 -0
- package/.agents/skills/self-diagnosis/resources/self-diagnosis-manifest.json +144 -0
- package/.agents/skills/self-diagnosis/scripts/diagnose.mjs +1971 -0
- package/.agents/skills/swarm-governance/SKILL.md +59 -0
- package/.agents/skills/swarm-governance/references/bundling-and-wire-format.md +88 -0
- package/.agents/skills/swarm-governance/references/intake-and-planning.md +120 -0
- package/.agents/skills/swarm-governance/references/large-swarms.md +39 -0
- package/CHANGELOG.md +16 -0
- package/LICENSE +21 -0
- package/README.md +156 -2
- package/bin/agy-squad.mjs +293 -0
- package/docs/assets/architecture-overview.svg +39 -0
- package/docs/assets/manifest-and-modes.svg +39 -0
- package/docs/assets/role-and-verdict-gate.svg +39 -0
- package/docs/assets/skill-role-workflows.svg +39 -0
- package/docs/assets/swarm-lifecycle.svg +39 -0
- package/docs/benchmarks.md +102 -0
- package/docs/concepts.md +81 -0
- package/docs/core-workflows.md +128 -0
- package/docs/getting-started.md +116 -0
- package/docs/index.md +47 -0
- package/docs/plans/LEDGER.md +24 -0
- package/docs/plans/comparative-harness-architecture-retest.md +1174 -0
- package/docs/plans/comparative-harness-architecture-teamwork.md +1170 -0
- package/docs/plans/comparative-harness-architecture.md +256 -0
- package/docs/plans/native-enforcement-ledger.md +126 -0
- package/docs/plans/pr3-option-b-fixes.md +803 -0
- package/docs/plans/wsl-cli-verification.md +55 -0
- package/docs/reviews/post-pr5-governance-review.md +221 -0
- package/docs/reviews/pr1-code-review-ledger.md +581 -0
- package/docs/reviews/pr1-production-readiness-review.md +764 -0
- package/docs/reviews/pr2-documentation-review-ledger.md +380 -0
- package/docs/reviews/pr3-squad-alignment-full-review.md +1021 -0
- package/docs/reviews/pr4-swarm-governance-pivot-review.md +593 -0
- package/docs/reviews/pr5-full-review-direction.md +330 -0
- package/docs/reviews/pr5-independent-review.md +266 -0
- package/docs/reviews/pr5-layered-swarm-review.md +513 -0
- package/docs/reviews/pr5-phase2-swarm-planner-review.md +470 -0
- package/docs/reviews/pr5-reconciled-review.md +573 -0
- package/docs/reviews/rebuild-wsl-dispatch-review-r2.md +427 -0
- package/docs/reviews/rebuild-wsl-dispatch-review.md +552 -0
- package/docs/roles-and-workflows.md +115 -0
- package/docs/stopping-agents.md +39 -0
- package/docs/tools.md +122 -0
- package/install.ps1 +5 -0
- package/install.sh +4 -0
- package/mcp/briefs.mjs +202 -0
- package/mcp/dispatch.mjs +280 -0
- package/mcp/handoffs.mjs +814 -0
- package/mcp/index.js +322 -0
- package/mcp/planner.mjs +755 -0
- package/mcp/verdicts.mjs +224 -0
- package/package.json +57 -4
- package/plugin.json +20 -0
- package/scripts/install-register.mjs +510 -0
- package/scripts/release-preflight.mjs +142 -0
|
@@ -0,0 +1,256 @@
|
|
|
1
|
+
# Comparative Harness Architecture: General Multi-Agent & Autonomous SWE Frameworks vs. Agy-Squad
|
|
2
|
+
|
|
3
|
+
> [!NOTE]
|
|
4
|
+
> **Historical (pre-Phase 2).** Scores, teardown and `PGID` kill-control criteria, and launcher references in this document describe Agy-Squad at decisions `L1`–`L41` (Phase 1: PTY launcher, managed-session teardown). PR #5 retired the launcher and managed-session teardown (`L17`; `L42`/`L43`). Do not cite these scores as describing the current architecture.
|
|
5
|
+
|
|
6
|
+
**Document Type:** Architectural Blueprint & Comparative Benchmark Synthesis
|
|
7
|
+
**Author:** Harness Comparative Architecture Lead
|
|
8
|
+
**Evaluation Standard:** Agy-Squad PR Review Standard `v1.3.0` & Architectural Decision Ledger (`L1`–`L41`)
|
|
9
|
+
**Target Systems Audited:**
|
|
10
|
+
1. **MetaGPT** (ICLR 2024 Oral — SOP-Driven Pub/Sub Bus)
|
|
11
|
+
2. **CrewAI** (Role-Playing Conversational Teams)
|
|
12
|
+
3. **AutoGen / Swarm** (Conversational Broadcast & State Handoffs)
|
|
13
|
+
4. **LangGraph** (Statechart Directed Acyclic Graphs / Pregel Architecture)
|
|
14
|
+
5. **Devin / Sol** (Cognition AI — Autonomous Sandboxed Cloud Workspace)
|
|
15
|
+
6. **Claude Code** (Anthropic CLI — Interactive Command Execution)
|
|
16
|
+
7. **SWE-agent** (Princeton NLP — Agent-Computer Interface / ACI)
|
|
17
|
+
8. **Google Antigravity Native `/teamwork`** (Google DeepMind — Native Multi-Agent Fleet)
|
|
18
|
+
9. **Agy-Squad + Agy-Context-Saver 1.8.1 (`rtk 0.51.0`)** (Governed Host-Sandboxed Reference Baseline)
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## 1. Executive Summary & Core Architectural Theses
|
|
23
|
+
|
|
24
|
+
Multi-agent coordination and autonomous software engineering (SWE) harnesses face a fundamental systems engineering trilemma between **orchestration expressiveness**, **context window / token efficiency**, and **evidence verifiability**.
|
|
25
|
+
|
|
26
|
+
Most contemporary multi-agent and coding agent frameworks conflate the **Inter-Process Communication (IPC) signaling channel** with the **payload delivery and evidentiary storage channel**. By transmitting full document deliverables, chat transcripts, tool outputs, and execution scratchpads directly across the LLM prompt wire, these frameworks exhibit three systemic failure modes:
|
|
27
|
+
|
|
28
|
+
1. **Quadratic ($O(M^2)$) or Monotonic ($O(N \cdot D)$) Token Context Explosion**: Every conversational exchange, intermediate document revision, or role-playing handoff re-serializes previously generated tokens into downstream prompt contexts, precipitating rapid prompt cache eviction and attention dilution.
|
|
29
|
+
2. **Context Contamination & Hallucination Cascading**: Downstream agents inherit noisy, unstructured scratchpads, unvetted intermediate artifacts, and raw command stdout/stderr, inducing the "lost-in-the-middle" phenomenon and compounding reasoning errors.
|
|
30
|
+
3. **Non-Deterministic Verification & Lifecycle Sprawl**: Output acceptance relies on probabilistic LLM self-critique or regex string matching (`"TERMINATE"`), lacking host-level filesystem isolation, tamper-evident cryptographic checksums, or deterministic lifecycle cleanup contracts.
|
|
31
|
+
|
|
32
|
+
In contrast, **Agy-Squad** implements a decoupled **Pointer-Over-Wire** architecture governed by host-enforced boundaries ([`L1, L2`](file:///home/paragon-ux/Antigravity/Agy-Squad/docs/plans/LEDGER.md#L1-L7)), deterministic manifest compilation ([`mcp/dispatch.mjs`](file:///home/paragon-ux/Antigravity/Agy-Squad/mcp/dispatch.mjs)), isolated filesystem report slots ([`mcp/briefs.mjs`](file:///home/paragon-ux/Antigravity/Agy-Squad/mcp/briefs.mjs)), a strict $\le 500$-byte single-line IPC receipt ceiling ([`mcp/handoffs.mjs`](file:///home/paragon-ux/Antigravity/Agy-Squad/mcp/handoffs.mjs)), and cryptographic SHA-256 digest bundling ([`node mcp/handoffs.mjs bundle / verify`](file:///home/paragon-ux/Antigravity/Agy-Squad/mcp/handoffs.mjs#L140-L240)).
|
|
33
|
+
|
|
34
|
+
In empirical head-to-head benchmarking against conversational fleets ([`benchmarks/teamwork-vs-squad/`](file:///home/paragon-ux/Antigravity/Agy-Squad/benchmarks/teamwork-vs-squad/README.md)), this model demonstrates an **$81.9\%$ reduction in aggregate transcript bytes**, a **$95.0\%$ reduction in parent conversation injected bytes** ($249\text{ B}$ vs. $4,940\text{ B}$), and a **$4.2\times$ wall-clock speedup** ($375\text{s}$ vs. $1,579\text{s}$), with **zero untracked Git files** abandoned in the workspace under the `L41` teardown contract, while maintaining identical $100\%$ ($6/6$) defect recall.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## 2. The 5-Axis Quantitative Evaluation Rubric
|
|
39
|
+
|
|
40
|
+
To evaluate general workflow frameworks and autonomous SWE harnesses on an equal, rigorous foundation, we define the **5-Axis Quantitative Evaluation Rubric**. Each dimension is scored from $1.0$ (Severe Pathological) to $5.0$ (Optimal Reference Baseline), yielding a maximum score of $25.0$.
|
|
41
|
+
|
|
42
|
+
### Axis 1: Token Efficiency & Context Compaction (TE-CC)
|
|
43
|
+
- **1.0 (Pathological):** Unbounded prompt serialization; full chat histories or raw tool/command stdout injected directly into conversation context ($> 15\text{ KB}$ per turn); no AST/regex reduction.
|
|
44
|
+
- **3.0 (Moderate):** Naive head/tail truncation or interactive line scrolling (e.g. SWE-agent ACI); monotonic token growth ($O(N \cdot D)$) across multi-turn handoffs.
|
|
45
|
+
- **5.0 (Optimal):** Transparent AST/regex shell output reduction (`rtk test`, `rtk err`); strict out-of-band pointer receipt ceiling ($\le 500$ B); constant $O(1)$ IPC wire overhead; zero passing-test noise.
|
|
46
|
+
|
|
47
|
+
### Axis 2: Evidence Integrity & Cryptographic Verifiability (EI-CV)
|
|
48
|
+
- **1.0 (Pathological):** Fleeting conversation memory; acceptance via unanchored LLM self-approval or regex termination; no persistent evidence files; hallucinated file citations.
|
|
49
|
+
- **3.0 (Moderate):** Persistent trajectory logs (`.traj`) or cloud session recordings without cryptographic integrity validation; vulnerable to post-execution tampering.
|
|
50
|
+
- **5.0 (Optimal):** Cryptographic SHA-256 digests pairing source files and report artifacts into immutable bundles (`report.sources.json`) verified in $< 25$ ms; 1-ID-per-finding ledgers; strict line-anchored unambiguous verdict enforcement.
|
|
51
|
+
|
|
52
|
+
### Axis 3: Failure Blast Radius & Lifecycle Teardown (BR-LT)
|
|
53
|
+
- **1.0 (Pathological):** Unconstrained working tree pollution; dozens of untracked files abandoned across directories; unbounded subagent sprawl; lingering background processes.
|
|
54
|
+
- **3.0 (Moderate):** Containerized execution with internal root permissions, but heavy host-side telemetry sprawl or cooperative exit cleanup.
|
|
55
|
+
- **5.0 (Optimal):** Deterministic 3-scope teardown (`L41`); atomic post-bundle specialist report pruning; zero leftover untracked Git files; deterministic process group kill controls (`PGID`, PTY teardown).
|
|
56
|
+
|
|
57
|
+
### Axis 4: Orchestration Topology & State Machine (OT-SM)
|
|
58
|
+
- **1.0 (Pathological):** Unbounded conversational chatter; high risk of infinite ping-pong loops; tight status polling loops; lack of structured stage transitions.
|
|
59
|
+
- **3.0 (Moderate):** Role-playing sequential handoffs or broadcast pub/sub without bounded execution stages; cooperative backoff.
|
|
60
|
+
- **5.0 (Optimal):** Declarative, schema-validated acyclic staged dispatch (`dispatch_squad`); decoupled worker slots; event-driven Reactive Wakeup without polling loops; compile-time cycle elimination.
|
|
61
|
+
|
|
62
|
+
### Axis 5: Developer Ergonomics & Operational Determinism (DE-OD)
|
|
63
|
+
- **1.0 (Pathological):** Opaque black-box execution; unpredictable runtime behaviors; non-reproducible prompt drift; heavy external runtime dependencies.
|
|
64
|
+
- **3.0 (Moderate):** Python SDK with extensive configuration boilerplate; complex distributed setup; variable execution paths across identical inputs.
|
|
65
|
+
- **5.0 (Optimal):** Zero-friction CLI launcher (`bin/agy-squad.mjs`); transparent companion tool integration (`rtk`); deterministic output paths; comprehensive offline test harness (`verify-benchmark.mjs`); clear POSIX/WSL2 cross-platform support.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## 3. Comprehensive Comparative Matrix
|
|
70
|
+
|
|
71
|
+
The table below summarizes the quantitative scores across all nine audited harnesses evaluated by Specialist 1 (General Workflow & Graph Harnesses) and Specialist 2 (Autonomous Coding & SWE Harnesses):
|
|
72
|
+
|
|
73
|
+
| Harness / System | Architecture Paradigm | TE-CC | EI-CV | BR-LT | OT-SM | DE-OD | Total (/25) | Composite (/5.0) |
|
|
74
|
+
| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
|
75
|
+
| **AutoGen / Swarm** | Conversational Broadcast | 1.2 | 1.0 | 1.8 | 1.5 | 1.6 | **7.1** | **1.42** |
|
|
76
|
+
| **CrewAI** | Role-Playing Teams | 1.8 | 1.5 | 2.2 | 2.0 | 2.7 | **10.2** | **2.04** |
|
|
77
|
+
| **MetaGPT** | SOP-Driven Pub/Sub Bus | 2.5 | 3.0 | 3.2 | 2.8 | 2.1 | **13.6** | **2.72** |
|
|
78
|
+
| **Claude Code** | Interactive CLI Agent | 2.2 | 2.0 | 3.0 | 2.8 | 3.8 | **13.8** | **2.76** |
|
|
79
|
+
| **SWE-agent** | Agent-Computer Interface (ACI) | 3.2 | 2.5 | 2.8 | 2.5 | 2.8 | **13.8** | **2.76** |
|
|
80
|
+
| **Antigravity `/teamwork`** | Native Multi-Agent Fleet | 2.0 | 2.5 | 1.5 | 3.5 | 4.5 | **14.0** | **2.80** |
|
|
81
|
+
| **Devin / Sol** | Sandboxed Cloud SWE Workspace | 3.8 | 3.5 | 3.6 | 4.8 | 3.3 | **19.0** | **3.80** |
|
|
82
|
+
| **LangGraph** | Statechart DAG (Pregel) | 3.8 | 3.6 | 4.0 | 4.5 | 3.2 | **19.1** | **3.82** |
|
|
83
|
+
| **Agy-Squad + ACS** | **Pointer-Over-Wire + RTK** | **4.9** | **5.0** | **4.9** | **4.8** | **4.9** | **24.5** | **4.90** |
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## 4. Empirical Protocol Comparisons & Architectural Divergences
|
|
88
|
+
|
|
89
|
+
### 4.1 General Workflow & Graph Harnesses (MetaGPT, CrewAI, AutoGen, LangGraph)
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
+---------------------------------------------------------------------------------------------------------------+
|
|
93
|
+
| TAXONOMY OF GENERAL MULTI-AGENT IPC & STATE MODELS |
|
|
94
|
+
+--------------------------+------------------------------+-----------------------------+-----------------------+
|
|
95
|
+
| Framework | Architectural Paradigm | IPC Wire Mechanism | Context Scaling |
|
|
96
|
+
+--------------------------+------------------------------+-----------------------------+-----------------------+
|
|
97
|
+
| MetaGPT | SOP-driven Pub/Sub Bus | Document Serialization | Monotonic: O(N * D) |
|
|
98
|
+
| CrewAI | Role-Playing Teams | Prompt Concatenation/Chat | Quadratic: O(K^2 * T) |
|
|
99
|
+
| AutoGen / Swarm | Conversational Broadcast | Shared Chat History | Quadratic: O(M^2) |
|
|
100
|
+
| LangGraph | Statechart DAG (Pregel) | State Reducer Channel | Linear/Poly: O(M) |
|
|
101
|
+
| Agy-Squad (Pointer-Wire) | Decoupled Out-of-Band Slot | <= 500B Pointer Receipt | Constant: O(1) Wire |
|
|
102
|
+
+--------------------------+------------------------------+-----------------------------+-----------------------+
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
1. **MetaGPT**: Implements an SOP-driven publish-subscribe event bus (`Environment`). While role definitions (`ProductManager`, `Architect`, `Engineer`) structure outputs into schema-validated markdown diagrams, each step re-publishes the full document payload across the bus. This induces monotonic context growth ($O(N \cdot D)$), causing downstream engineers to consume up to $18.5\text{ KB}$ of prompt context per turn.
|
|
106
|
+
2. **CrewAI**: Organizes agents into sequential or hierarchical teams with task delegation. However, delegation transmits the entire task description, agent backstories, and execution memory across the prompt wire. Token growth scales quadratically ($O(K^2 \cdot T)$), frequently triggering context overflow or redundant delegation loops.
|
|
107
|
+
3. **AutoGen / Swarm**: Models collaboration as conversational multi-party turn-taking. Every agent message is appended to a global chat history passed to all participants. Without aggressive custom filtering, context consumption scales at $O(M^2)$ where $M$ is message count. Termination relies on non-deterministic token matching (`"TERMINATE"`), which LLMs frequently omit or hallucinate.
|
|
108
|
+
4. **LangGraph**: Represents multi-agent flows as statecharts executed on the Pregel cyclic graph model. State transitions are governed by reducer channels. While LangGraph provides superior determinism and persistence (checkpointing), naive state definitions append full message histories to state channels, bloating downstream graph nodes.
|
|
109
|
+
|
|
110
|
+
### 4.2 Autonomous SWE & Coding Harnesses (Devin, Claude Code, SWE-agent, Antigravity /teamwork)
|
|
111
|
+
|
|
112
|
+
1. **Devin / Sol**: Executes in dedicated remote cloud microVMs with full root privileges. Devin features deep workspace telemetry, browser automation, and terminal monitoring. However, its execution context is heavily saturated by interactive command telemetry ($> 50\text{ MB}$ session logs), and code review contracts rely on cooperative prompt conventions rather than host-enforced cryptographic proofs.
|
|
113
|
+
2. **Claude Code**: Operates as a local interactive CLI agent. It executes commands directly on the user's host shell. Without token-reduction proxies, running extensive test suites (`npm test`, `pytest`) dumps thousands of lines of passing test output into the context window. It lacks multi-agent staging, cryptographic evidence bundling, and automated scratch cleanup.
|
|
114
|
+
3. **SWE-agent**: Introduced the Agent-Computer Interface (ACI) to bridge LLMs and terminal environments via custom viewing/editing commands (`open_file`, `scroll_down`, `edit_line`). While ACI bounds file inspection to 100-line windows, multi-turn file navigation incurs high step penalties, and the single-agent loop cannot parallelize across complex multi-domain codebases.
|
|
115
|
+
4. **Google Antigravity Native `/teamwork`**: Provides native out-of-the-box subagent orchestration. However, its unconstrained delegation model deploys large conversational swarms (8 subagents for a 2-domain audit), streams progress continuously into parent context (5 wakeups, 4,940 bytes), and relies on semantic LLM reviewers (`reviewer_1`, `auditor_1`) rather than algorithmic verification, while abandoning 41 untracked files ($207.9\text{ KB}$) in `.agents/teamwork/`.
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## 5. Protocol Sequence Diagrams
|
|
120
|
+
|
|
121
|
+
### 5.1 Conventional Conversational / In-Band Swarm (MetaGPT, CrewAI, AutoGen, `/teamwork`)
|
|
122
|
+
|
|
123
|
+
The diagram below illustrates the token inflation and context contamination inherent in conventional in-band multi-agent swarms:
|
|
124
|
+
|
|
125
|
+
```mermaid
|
|
126
|
+
sequenceDiagram
|
|
127
|
+
autonumber
|
|
128
|
+
actor User as Operator / Parent LLM
|
|
129
|
+
participant W1 as Specialist 1 (Domain A)
|
|
130
|
+
participant W2 as Specialist 2 (Domain B)
|
|
131
|
+
participant Rev as Semantic Reviewer LLM
|
|
132
|
+
|
|
133
|
+
User->>W1: Task Prompt + Context (In-Band)
|
|
134
|
+
User->>W2: Task Prompt + Context (In-Band)
|
|
135
|
+
Note over W1,W2: Parallel Execution
|
|
136
|
+
|
|
137
|
+
W1-->>User: Full Evidence Report A (15 KB payload on wire)
|
|
138
|
+
Note over User: Parent Context Flooded (+15 KB)
|
|
139
|
+
|
|
140
|
+
W2-->>User: Full Evidence Report B (18 KB payload on wire)
|
|
141
|
+
Note over User: Parent Context Flooded (+18 KB)
|
|
142
|
+
|
|
143
|
+
User->>Rev: Re-serializes Report A + Report B (33 KB context)
|
|
144
|
+
Note over Rev: High Attention Dilution / Latency (174 steps)
|
|
145
|
+
Rev-->>User: Subjective Assessment (Probabilistic PASS)
|
|
146
|
+
|
|
147
|
+
Note over User,Rev: Filesystem Pollution: 41 untracked files abandoned
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
### 5.2 Agy-Squad Pointer-Over-Wire & Cryptographic Evidence Bundling
|
|
151
|
+
|
|
152
|
+
The diagram below illustrates Agy-Squad's decoupled out-of-band architecture:
|
|
153
|
+
|
|
154
|
+
```mermaid
|
|
155
|
+
sequenceDiagram
|
|
156
|
+
autonumber
|
|
157
|
+
actor User as Parent Session (agy)
|
|
158
|
+
participant Disp as MCP dispatch_squad
|
|
159
|
+
participant S1 as Specialist 1 (Slot A)
|
|
160
|
+
participant S2 as Specialist 2 (Slot B)
|
|
161
|
+
participant Handoff as MCP handoffs.mjs
|
|
162
|
+
participant FS as Workspace Filesystem
|
|
163
|
+
|
|
164
|
+
User->>Disp: dispatch_squad(manifestPath, squadId)
|
|
165
|
+
Disp-->>User: Compiled Subagents Briefs (8.5 ms)
|
|
166
|
+
|
|
167
|
+
User->>S1: invoke_subagent(Brief 1)
|
|
168
|
+
User->>S2: invoke_subagent(Brief 2)
|
|
169
|
+
Note over User: Yields & Waits (Zero Busy-Polling)
|
|
170
|
+
|
|
171
|
+
S1->>FS: Writes slot: scratch/.../specialist-1/report.md
|
|
172
|
+
S1-->>User: send_message("READY scratch/.../report.md — <observation>") (<= 500 B)
|
|
173
|
+
Note over User: Reactive Wakeup 1 (249 B context)
|
|
174
|
+
|
|
175
|
+
S2->>FS: Writes slot: scratch/.../specialist-2/report.md
|
|
176
|
+
S2-->>User: send_message("READY scratch/.../report.md — <observation>") (<= 500 B)
|
|
177
|
+
Note over User: Reactive Wakeup 2 (249 B context)
|
|
178
|
+
|
|
179
|
+
User->>FS: Directly inspects cited source lines
|
|
180
|
+
User->>FS: Writes lead/report.md ("Verdict: PASS")
|
|
181
|
+
|
|
182
|
+
User->>Handoff: bundleReports(record)
|
|
183
|
+
Handoff->>FS: Validates unambiguous PASS, hashes sources + reports
|
|
184
|
+
Handoff->>FS: Writes lead/report.sources.json (SHA-256 Bundle)
|
|
185
|
+
|
|
186
|
+
User->>Handoff: verifyBundle(bundlePath)
|
|
187
|
+
Note over Handoff: Algorithmic Verification (20.0 ms)
|
|
188
|
+
|
|
189
|
+
User->>Handoff: pruneReports(bundlePath, acceptedSha256)
|
|
190
|
+
Handoff->>FS: Deletes loose specialist reports (L41 Teardown)
|
|
191
|
+
Note over FS: Zero Untracked Git Files Remaining
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
---
|
|
195
|
+
|
|
196
|
+
## 6. Empirical Telemetry & Benchmark Findings
|
|
197
|
+
|
|
198
|
+
The following empirical data originates from direct sequential audits conducted on the identical two-domain event-relay fixture on WSL2 Linux ([`benchmarks/teamwork-vs-squad/`](file:///home/paragon-ux/Antigravity/Agy-Squad/benchmarks/teamwork-vs-squad/README.md) and [`benchmarks/teamwork-vs-squad/results/telemetry.json`](file:///home/paragon-ux/Antigravity/Agy-Squad/benchmarks/teamwork-vs-squad/results/telemetry.json)):
|
|
199
|
+
|
|
200
|
+
### 6.1 Head-to-Head Telemetry Comparison
|
|
201
|
+
|
|
202
|
+
| Metric Dimension | Antigravity Native `/teamwork` | Agy-Squad + Agy-Context-Saver | Delta / Architectural Improvement |
|
|
203
|
+
| :--- | :---: | :---: | :---: |
|
|
204
|
+
| **Planted Defect Recall** | `6 / 6` (100%) | `6 / 6` (100%) | **Parity** (Both arms caught all defects) |
|
|
205
|
+
| **Subagents Spawned** | `8` agents | `3` agents | **$62.5\%$ reduction** (3 vs. 8) |
|
|
206
|
+
| **Total Transcript Steps** | `645` steps | `64` steps | **$90.1\%$ reduction** ($10.1\times$ fewer steps) |
|
|
207
|
+
| **Total Assistant LLM Turns** | `321` turns | `26` turns | **$91.9\%$ reduction** ($12.3\times$ fewer turns) |
|
|
208
|
+
| **Total Tool Calls** | `302` calls | `32` calls | **$89.4\%$ reduction** ($9.4\times$ fewer calls) |
|
|
209
|
+
| **Total Transcript Volume** | `817.4 KB` | `148.2 KB` | **$81.9\%$ reduction** (669.2 KB saved) |
|
|
210
|
+
| **Active Wall-Clock Duration** | `1,579s` (26m 19s) | `375s` (6m 15s) | **$76.3\%$ speedup** (**$4.2\times$ faster**) |
|
|
211
|
+
| **Parent Conversation Wakeups** | `5` wakeups | `1` wakeup | **$80.0\%$ reduction** (1 vs. 5 wakeups) |
|
|
212
|
+
| **Parent Injected Context Bytes** | `4,940` bytes | `249` bytes | **$95.0\%$ reduction** (249 B vs. 4,940 B) |
|
|
213
|
+
| **Verification Latency / Overhead** | `174` LLM steps (230 KB) | `20.0 ms` algorithmic | **Four orders of magnitude faster**; zero token cost |
|
|
214
|
+
| **Leftover Untracked Git Files** | `41` files (`207.9 KB`) | `0` files (`0 KB`) | **$100\%$ zero-pollution teardown** (`L41`) |
|
|
215
|
+
|
|
216
|
+
### 6.2 Key Architectural Findings (F1–F5)
|
|
217
|
+
|
|
218
|
+
- **Finding F1: Terminal Context Flooding in Unwrapped Shell Execution:**
|
|
219
|
+
Direct execution of test suites (`npm test`, `pytest`) dumps thousands of lines of passing test output into the context window (~1,860 tokens per run). Agy-Squad transparently routes shell commands through RTK (`rtk test`), stripping passing test boilerplate and preserving only actionable failures. Result: **839.3K tokens saved** ($31.4\%$ net reduction).
|
|
220
|
+
- **Finding F2: Severe Filesystem Pollution in Unconstrained Multi-Agent Swarms:**
|
|
221
|
+
Native `/teamwork` abandoned 41 untracked Markdown, JSON, and briefing files totaling $207.9\text{ KB}$ across `.agents/teamwork/`. Agy-Squad finished the identical benchmark with **`0` untracked files**, enforced by the deterministic `L41` teardown contract.
|
|
222
|
+
- **Finding F3: Parent Wakeup Inflation via Conversational Streaming:**
|
|
223
|
+
Conversational swarms stream status updates and complete reports directly into the parent context, delivering 4,940 bytes across 5 wakeups. Agy-Squad restricts inter-agent signaling to single-line Pointer-Over-Wire receipts ($\le 500$ B), reducing parent context injection by **$95.0\%$** ($249\text{ B}$).
|
|
224
|
+
- **Finding F4: Cryptographic vs. Semantic Verification Scaling:**
|
|
225
|
+
Teamwork deployed two subagents (`reviewer_1` and `auditor_1`) consuming 174 steps and 228.6 KB of transcript to perform subjective semantic checks. Agy-Squad's `mcp/handoffs.mjs verify` computed and checked SHA-256 digests across all source and report files in **`20.0 ms`** at zero token cost.
|
|
226
|
+
- **Finding F5: Non-Deterministic Termination in Free-Form Agent Chat:**
|
|
227
|
+
String-based termination tokens (`"TERMINATE"`) frequently induce either premature truncation (when cited in discussion) or infinite conversational loops (when omitted). Agy-Squad eliminates cyclic termination hazards through compile-time acyclic staging and explicit process exit signals.
|
|
228
|
+
|
|
229
|
+
---
|
|
230
|
+
|
|
231
|
+
## 7. Trade-Off Analysis & Design Contradictions
|
|
232
|
+
|
|
233
|
+
While Agy-Squad achieves state-of-the-art token efficiency and operational determinism, rigorous architectural analysis highlights three genuine design trade-offs:
|
|
234
|
+
|
|
235
|
+
1. **Acyclic Staging vs. Arbitrary Cyclic Expressiveness:**
|
|
236
|
+
- *Trade-Off*: Frameworks like LangGraph support arbitrary cyclic statecharts with dynamic runtime looping. Agy-Squad enforces discrete, acyclic stages (`all_l1` lead stage followed by specialist cohorts).
|
|
237
|
+
- *Architectural Resolution*: Cyclic looping introduces the hazard of unbounded token spend and non-terminating loops. Agy-Squad achieves iterative refinement through operator-supervised sequential cohorts (e.g. initial audit wave followed by targeted re-verification wave) while preserving strict cost predictability.
|
|
238
|
+
2. **Declarative Manifest Scoping vs. Zero-Shot Delegation:**
|
|
239
|
+
- *Trade-Off*: CrewAI and AutoGen allow an LLM to dynamically invent task breakdowns at runtime without an upfront manifest. Agy-Squad requires schema-validated manifests (`manifest.json`).
|
|
240
|
+
- *Architectural Resolution*: Unconstrained dynamic decomposition is the primary cause of role overlapping, file clobbering, and exponential agent sprawl. Agy-Squad provides the zero-subagent profiler (`diagnose.mjs`) and pre-compiled manifests, guaranteeing deterministic execution boundaries.
|
|
241
|
+
3. **Local Host Sandboxing vs. Ephemeral Cloud Containerization:**
|
|
242
|
+
- *Trade-Off*: Devin runs in isolated cloud microVMs with remote snapshots. Agy-Squad runs on POSIX/WSL2 host environments.
|
|
243
|
+
- *Architectural Resolution*: Agy-Squad enforces strict deny-by-default path boundaries (`L21, L40`), CLI sandbox permissions (`L17`), and clean workspace teardowns (`L41`), delivering near-instant local execution without cloud latency, API lock-in, or remote infrastructure hosting costs.
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
## 8. Strategic Recommendations for Future Harness Engineering
|
|
248
|
+
|
|
249
|
+
1. **Context Governance Must Precede Tool Execution:**
|
|
250
|
+
Harnesses must intercept and reduce tool outputs transparently before they reach the model's context window. Post-hoc LLM truncation wastes tokens and corrupts prompt caches.
|
|
251
|
+
2. **Decouple IPC Signaling from Evidentiary Storage:**
|
|
252
|
+
Inter-agent messaging channels should carry strictly bounded pointers ($\le 500$ B). All heavy evidence, logs, and artifacts must reside in designated out-of-band workspace slots.
|
|
253
|
+
3. **Cryptographic Proofs Must Anchor Autonomous Decisions:**
|
|
254
|
+
Automated workflows should not rely on subjective LLM-on-LLM reviews. Source files and deliverables must be cryptographically hashed (SHA-256) into immutable bundles before sign-off.
|
|
255
|
+
4. **Lifecycle Teardown Must Be Host-Enforced:**
|
|
256
|
+
Multi-agent harnesses must incorporate automated, deterministic cleanup contracts (`L41`) to guarantee that temporary audit artifacts, intermediate scratch files, and child processes are purged upon completion.
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
# Native Enforcement Ledger
|
|
2
|
+
|
|
3
|
+
**Status:** Draft v0.1 (scoping ledger; no enforcement is implemented or verified by this document)
|
|
4
|
+
**Date:** 2026-11-13
|
|
5
|
+
**Origin:** PR #5 independent review residuals `PR5-ARCH-05`, `PR5-SCRIPT-02`, `PR5-ARCH-06` ([`docs/reviews/pr5-independent-review.md`](../reviews/pr5-independent-review.md))
|
|
6
|
+
**Supersedes (as a direction):** the retired Docker per-agent container and spawn-gate design (`docs/plans/LEDGER.md` L8–L12, L35, L36, L38; plan text at commit `ee10ebc`)
|
|
7
|
+
|
|
8
|
+
## 1. Purpose
|
|
9
|
+
|
|
10
|
+
Agy-Squad delegates orchestration to Antigravity (native `/swarm`, `invoke_subagent`). What remains the project's own job is **enforcement**: checks that run outside the governed agent's control and that fail closed. This ledger records each enforcement claim, the native mechanism that could provide it, and the pass/fail test that would verify it. It exists so that no enforcement claim is made in docs, skills or PRs before its test has passed.
|
|
11
|
+
|
|
12
|
+
### Principles
|
|
13
|
+
|
|
14
|
+
1. **Enforcement is mechanical.** A check counts only if a script or the host runs it and the governed agent cannot talk its way past it. A model's report is never the evidence.
|
|
15
|
+
2. **Fail closed.** A hook or policy that errors, times out or is absent must deny, or the claim is downgraded.
|
|
16
|
+
3. **A claim needs a falsifiable test.** Each row has a test that can fail. "The API exists" is not "the behavior holds".
|
|
17
|
+
4. **Do not claim propagation without a test.** Hooks and policies on a parent agent do not imply the same controls on subagents or swarm workers.
|
|
18
|
+
5. **The governed agent must not be able to edit its own guard.** If it can write the hook config, the policy file or the audit log, the row is not enforcement.
|
|
19
|
+
|
|
20
|
+
### Status taxonomy
|
|
21
|
+
|
|
22
|
+
| Status | Meaning |
|
|
23
|
+
|---|---|
|
|
24
|
+
| `Unverified` | Claimed or inferred; no test run |
|
|
25
|
+
| `API-verified` | The mechanism exists in installed software (introspected or observed), behavior not tested |
|
|
26
|
+
| `Observed` | Behavior observed once in a real session; no controlled test |
|
|
27
|
+
| `Verified` | Controlled pass/fail test passed and is reproducible (evidence linked) |
|
|
28
|
+
| `Rejected` | Tested or decided against; reason recorded |
|
|
29
|
+
| `Not supported` | The platform does not offer it as of the recorded version |
|
|
30
|
+
|
|
31
|
+
## 2. Evidence base (what is actually known)
|
|
32
|
+
|
|
33
|
+
| ID | Fact | How established | Confidence |
|
|
34
|
+
|---|---|---|---|
|
|
35
|
+
| V-01 | Antigravity `PreToolUse` hooks fire for `run_command`, `read_file`, `schedule`, `manage_task` and can **deny** (and rewrite) calls | Observed repeatedly in the PR #5 review sessions: denied `read_file`, rewrote `run_command` args via the Agy-Context-Saver hook | `Observed` (not a controlled test) |
|
|
36
|
+
| V-02 | The live global hook (`~/.gemini/config/hooks.json`) has a `PreToolUse` matcher limited to `manage_task\|schedule\|run_command\|view_file\|read_file\|read_many_files\|grep_search\|find_by_name\|list_dir\|replace_file_content\|write_to_file`. It does **not** match `swarm`, `post_message`, `read_board` or `invoke_subagent` | Read from `hooks.json` | Verified by inspection |
|
|
37
|
+
| V-03 | `google-antigravity` SDK v0.1.23 installs from PyPI (pre-1.0) | Installed into `scratch/sdk_pkgs` (gitignored) | Verified |
|
|
38
|
+
| V-04 | `LocalAgentConfig` accepts `policies`, `hooks`, `triggers`, `mcp_servers`, `subagents`, `workspaces`, `env`, `budget_config`, `retry_config`, `model`/`models`, `api_key`, `vertex`, `conversation_id` | Introspected signature | `API-verified` |
|
|
39
|
+
| V-05 | `CapabilitiesConfig` includes `enable_subagents`, `allowed_subagents`, `max_subagent_depth`, `enabled_tools`, `disabled_tools`, `run_command_config`, MCP options | Introspected fields | `API-verified` |
|
|
40
|
+
| V-06 | `BudgetConfig` has `max_model_calls`, `max_tool_calls`, `max_input_tokens`, `max_output_tokens`, `max_total_tokens`, `scope` | Introspected fields | `API-verified` |
|
|
41
|
+
| V-07 | Hook types: `PreToolCallDecideHook`, `PostToolCallHook`, `PreTurnHook`, `PostTurnHook`, `StopHook`, `OnToolErrorHook`, `OnSessionStartHook`/`OnSessionEndHook`, `OnInteractionHook`, `OnCompactionHook`. Policy helpers: `allow`, `deny`, `ask_user`, `allow_all`, `deny_all`, `workspace_only`, `safe_defaults`, `confirm_run_command`. `Decision` = `APPROVE`/`ASK_USER`/`DENY`. `HookContext` carries a `parent` | Introspected exports | `API-verified` |
|
|
42
|
+
| V-08 | `BuiltinTools` includes `INVOKE_SUBAGENT`; **no reference to "swarm" exists anywhere in the SDK Python sources** (`grep -rnI swarm` returns empty; raw binary `bin/localharness` matches the byte string `"swarm"`) | `grep -rnI swarm` over installed package | Verified for v0.1.23 |
|
|
43
|
+
| V-09 | `Agent` exposes `chat`, `conversation`, `conversation_id`, `sandbox_status`, `is_started`, `beta` | Introspected | `API-verified` |
|
|
44
|
+
| V-10 | Native swarm: `swarm{action:"status"}` lists each worker's state and model; `add` refuses a model whose quota is exhausted; `kill` deletes a worker's workspace | Local swarm skill and `swarm` tool description | Documented, not tested |
|
|
45
|
+
| V-11 | Live `/swarm` worker capability split (Session `87cb96f4`, 2026-11-14): global `PreToolUse` hooks **do** propagate to all `/swarm` workers (blocking `read_file` and `.system_generated/` shell reads on `audit-code` `a03d8f1e`, `audit-gov` `bf53c412`, `gov-lead` `c33d1552`, `gov-docs-site-reviewer` `fed707af`), while parent `agy-context-saver` MCP tools (`query_transcript`, `read_transcript`) **do not** attach to `/swarm` worker toolsets | Observed across all 9 `/swarm` workers via coordinator `query_transcript` audit | `Observed` (splits NE-10: hook propagation observed, MCP inheritance absent) |
|
|
46
|
+
| W-01 | (Web, secondary sources) tool calls inside spawned subagents may bypass workspace-level hooks unless registered globally or via plugin; blocking a spawn tool with `PreToolUse` may not stop the spawn under some runtime flags (e.g. `--dangerously-skip-permissions`); `PostToolUse` cannot block | One search summary; redirect-only citations | **Unverified. Treat as a risk to test, not a fact.** |
|
|
47
|
+
|
|
48
|
+
## 3. Two enforcement planes
|
|
49
|
+
|
|
50
|
+
The SDK has no swarm support (V-08). That splits enforcement into two planes that do not share a mechanism:
|
|
51
|
+
|
|
52
|
+
| | **Plane A: SDK-hosted agents** | **Plane B: Native app/CLI (`/swarm`, `invoke_subagent`)** |
|
|
53
|
+
|---|---|---|
|
|
54
|
+
| Control surface | Typed `policies`, `hooks`, `budget_config`, `capabilities`, per-agent `mcp_servers` | `hooks.json` (global or `<repo>/.agents/hooks.json`), swarm tool surface, host permissions |
|
|
55
|
+
| Strength | Strong and testable in code | Weaker: matcher-based, propagation to workers unverified (W-01), live hook does not match swarm tools (V-02) |
|
|
56
|
+
| UI / board / @mentions | None (no swarm integration, V-08) | Native Swarm UI, board, graph |
|
|
57
|
+
| Lateral communication | None by construction (each `Agent` is isolated) | Shared board; restriction is a convention unless a hook denies `post_message` |
|
|
58
|
+
| Best use | Contention experiments, CI verification, controlled benchmarks | Interactive multi-agent work where users need the UI to manage and steer |
|
|
59
|
+
|
|
60
|
+
**Consequence:** one design cannot provide both strong typed enforcement and the native Swarm UI today. Rows below name their plane. Choosing a plane per use case is a design decision, not a defect.
|
|
61
|
+
|
|
62
|
+
## 4. Ledger
|
|
63
|
+
|
|
64
|
+
Threat sources: PR #3 evidence (model deleted `FAIL` to evade the verdict parser; `E14`/`E15`), `PR5-ARCH-05` (handoff authorship unattested), `PR5-SCRIPT-02` (health gate trusts the author's ledger), `PR5-ARCH-06` (spawn gate / default-deny), flash-class models gaming benchmarks.
|
|
65
|
+
|
|
66
|
+
| ID | Claim / threat | Plane | Native mechanism | Status | Verification test (must be able to fail) | Blockers / risks | Links |
|
|
67
|
+
|---|---|---|---|---|---|---|---|
|
|
68
|
+
| **NE-01** | Spawn gate: only planned workers may spawn; depth and count bounded | A | `CapabilitiesConfig.allowed_subagents`, `max_subagent_depth`, `enable_subagents=False`; `PreToolCallDecideHook` on `INVOKE_SUBAGENT` comparing the call to a plan hash | `API-verified` | An agent instructed to spawn an unlisted subagent and to exceed the depth gets `DENY`; the spawn does not occur; the denial is logged | Needs credentials (NE-12) | PR5-ARCH-06 |
|
|
69
|
+
| NE-01b | Same, native app/CLI | B | `PreToolUse` hook matching the spawn tools (`invoke_subagent`, `swarm`) | `Unverified` | With a disposable hook that denies `invoke_subagent`, a spawn request is blocked and no child conversation appears | V-02: live matcher excludes these tools; W-01: bypass risk | PR5-ARCH-06 |
|
|
70
|
+
| **NE-02** | Default-deny tool profile per agent (your own tool profile, per worker) | A | `policies=[deny_all(), allow(...)]`, `capabilities.enabled_tools`, per-agent `mcp_servers` | `API-verified` | Two agents with different profiles each list only their own tools; a call to a non-granted tool is denied | NE-12 | PR5-ARCH-04 |
|
|
71
|
+
| **NE-03** | Protected paths: oracle, tests and fixtures cannot be modified by workers | A (primary), B | `DecideHook` on `EDIT_FILE`/`CREATE_FILE`/`RUN_COMMAND` args against a pinned set; post-run SHA-256 verify (existing `agy-squad fingerprint`/`verify`) | `Unverified` (hash verify exists; gating does not) | A worker told to weaken a test is denied, or the post-run hash mismatch is detected and `verify` fails | Shell indirection (`run_command` writing files) must be covered | E14/E15 gaming |
|
|
72
|
+
| **NE-04** | Handoff attestation: who produced which report is recorded outside the model's control | A, B | `PostToolCallHook` appends `(agent id, tool, args hash, result hash, timestamp)` to a log the agent cannot write; bundle consumes the log instead of a hand-assembled `handoffs` list. Plane B: `PostToolUse` on `post_message` | A: `API-verified`; B: `Unverified` | Forge a handoff in the record; `verify` rejects it because the log has no matching entry | B: whether payload exposes member identity; log must be outside agent-writable paths (NE-09) | PR5-ARCH-05 |
|
|
73
|
+
| **NE-05** | Model pinning is verified, not assumed | A, B | A: `model`/`models` (`ModelTarget`) plus readback; B: `swarm{action:"status"}` compared to the plan | `Unverified` | Request a pinned model; assert the readback equals it; request a typo and assert rejection | Whether SDK exposes the resolved model per call; swarm `status` shape unseen | PR5-MCP-01 |
|
|
74
|
+
| **NE-06** | Budget / panic stop | A, B | A: `BudgetConfig.max_*`, `scope`; B: `manage_subagents kill_all`, swarm `kill`, quota refusal on `add` | A: `API-verified`; B: tool surface exists | Set a tiny `max_tool_calls`; the agent stops (or raises) at the limit; document which | Behavior at limit unknown (stop vs raise vs soft warning) | PR5-ARCH-06 |
|
|
75
|
+
| **NE-07** | No lateral communication in contention arms | A (by construction), B | A: isolated `Agent` instances; B: `PreToolUse` deny on `post_message` for worker members; channels | A: `API-verified`; B: `Unverified` | B: a worker attempts to post; it is denied; a control run without the hook posts | Hook must identify the caller as a worker; V-02 | concurrency protocol |
|
|
76
|
+
| **NE-08** | Review/health gate is independent of the change's author | In-repo (not native) | `diagnose.mjs` requires a ledger from a different conversation or a commit not written by the author; `conversation_id` as identity | `Unverified` | Add an author-written ledger newer than the independent one; the gate must still target the independent one or fail | Needs an identity convention for ledgers | PR5-SCRIPT-02 |
|
|
77
|
+
| **NE-09** | The governed agent cannot edit its own guard (hook config, policy files, audit log) | A, B | Host/file permissions; hook config and log outside agent-writable roots; `workspace_only` policy | `Unverified` | On a **disposable copy** of the config, an agent told to disable the hook fails to; the live `hooks.json` must never be used for this test | Precondition for NE-01..NE-07 being enforcement at all | all |
|
|
78
|
+
| **NE-10** | Hooks, policies, and MCP tools propagate to subagents and swarm workers | A, B | A: `HookContext.parent`; B: global `hooks.json`, plugin hooks, MCP inheritance | A: `Unverified`; B: **Split (`Observed`)** — `PreToolUse` hooks propagate to `/swarm` workers (`V-11`), parent MCP servers (`agy-context-saver`) do not | A child agent's tool call hits the parent's deny hook; for B, a worker's `run_command` hits a marker hook and attached MCP list is checked | Requires coordinator transcript index extraction for `/swarm` auditors until MCP propagates | all |
|
|
79
|
+
| NE-11 | SDK-hosted agents appear in the native Swarm UI / board | n/a | None found | `Not supported` (v0.1.23, V-08) | Re-check on SDK upgrades: `grep -rnI swarm` over the package | Pre-1.0 API | direction |
|
|
80
|
+
| NE-12 | SDK agents can run on the maintainer's existing subscriptions | A | `api_key` / `vertex` / `project` / `location` config; unknown whether subscription auth is supported | `Unverified` | Spawn one read-only `Agent` and complete one `chat()` call | **Blocks every Plane A behavioral test.** Credentials are an owner decision | all Plane A rows |
|
|
81
|
+
|
|
82
|
+
## 5. Verification plan (ordered; each spike produces evidence linked back to its row)
|
|
83
|
+
|
|
84
|
+
| Step | Spike | Rows | Needs |
|
|
85
|
+
|---|---|---|---|
|
|
86
|
+
| S0 | Decide credentials for the SDK (API key / Vertex / existing auth) | NE-12 | Owner |
|
|
87
|
+
| S1 | One read-only `Agent`, `deny_all()` plus one `allow`, a `PreToolCallDecideHook` that logs and denies a marked call | NE-02, NE-10 (parent hook) | S0 |
|
|
88
|
+
| S2 | `SubagentConfig` child with a restricted profile; parent hook; assert child tool calls reach the parent's hook | NE-01, NE-10 | S1 |
|
|
89
|
+
| S3 | `BudgetConfig` with tiny limits; record behavior at the limit | NE-06 | S1 |
|
|
90
|
+
| S4 | `PostToolCallHook` writing to a path outside the agent's `workspaces`; forge a handoff and run `verify` | NE-04, NE-09 | S1 |
|
|
91
|
+
| S5 | First verify hook discovery (`<repo>/.agents/hooks.json` vs plugin/global hooks) using a benign marker on `run_command`; then test marker `PreToolUse` on `invoke_subagent` and `post_message` across native subagents and swarm workers | NE-01b, NE-04 (B), NE-07 (B), NE-10 (B) | App session; no live-config edits |
|
|
92
|
+
| S6 | Pinned-model readback via SDK response metadata and `swarm status` | NE-05 | S0 (A), swarm run (B) |
|
|
93
|
+
| S7 | `diagnose.mjs` independent-ledger rule | NE-08 | In-repo change |
|
|
94
|
+
|
|
95
|
+
S5 must not edit or probe the live `~/.gemini/config/hooks.json`. First establish whether Antigravity loads repo-scoped `.agents/hooks.json` at all so a silently ignored file is not mistaken for hook bypass.
|
|
96
|
+
|
|
97
|
+
## 6. Closure criteria
|
|
98
|
+
|
|
99
|
+
- A row moves to `Verified` only with a committed test or a transcript hash and the exact command, plus the date and the SDK/Antigravity version.
|
|
100
|
+
- A row cannot be cited in README, skills or PR descriptions as an enforced guarantee until it is `Verified`. `API-verified` may be cited only as "the platform provides X".
|
|
101
|
+
- PR #5 residuals close like this: `PR5-ARCH-05` ← NE-04 `Verified`; `PR5-SCRIPT-02` ← NE-08 `Verified`; `PR5-ARCH-06` ← NE-01/NE-02/NE-06 `Verified` on at least one plane.
|
|
102
|
+
|
|
103
|
+
## 7. Out of scope and rejected
|
|
104
|
+
|
|
105
|
+
- Docker per-agent containers and the PTY launcher (retired; `L8–L12`, `L17`, `L35`, `L36`, `L38`).
|
|
106
|
+
- Competing with native orchestration (dispatch, board, UI): this ledger covers enforcement only.
|
|
107
|
+
- Any test that disables or probes a live control (the live `hooks.json`, the live Agy-Context-Saver hook, host permissions). Controls are tested on disposable copies only.
|
|
108
|
+
|
|
109
|
+
## 8. Reproduction (evidence for V-03 to V-09)
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
python3 -m venv scratch/sdk_venv
|
|
113
|
+
scratch/sdk_venv/bin/pip install --target scratch/sdk_pkgs "google-antigravity==0.1.23"
|
|
114
|
+
PYTHONPATH=scratch/sdk_pkgs scratch/sdk_venv/bin/python - <<'EOF'
|
|
115
|
+
import inspect, google.antigravity as ag
|
|
116
|
+
import google.antigravity.types as t
|
|
117
|
+
import google.antigravity.hooks as h
|
|
118
|
+
print(inspect.signature(ag.LocalAgentConfig))
|
|
119
|
+
print(list(t.CapabilitiesConfig.model_fields.keys()))
|
|
120
|
+
print(list(t.BudgetConfig.model_fields.keys()))
|
|
121
|
+
print([x for x in dir(h) if x.endswith("Hook")])
|
|
122
|
+
EOF
|
|
123
|
+
grep -rnI swarm scratch/sdk_pkgs/google/antigravity # expect: no output (V-08)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
If the host Python lacks `ensurepip`, point any existing Python 3.11+ interpreter with `pip` (for example `pip install --target scratch/sdk_pkgs "google-antigravity==0.1.23"`) into the gitignored `scratch/` directory to avoid modifying system packages.
|