everything-gemini 2.1.0 → 2.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/BENCHMARK.md +106 -0
- package/CHANGELOG.md +81 -0
- package/PARTNERSHIP.md +100 -0
- package/README.md +63 -6
- package/ROADMAP_AGI_v3.0.md +109 -0
- package/agents/resilience-engineer.md +36 -0
- package/agents/swarm-coordinator.md +35 -0
- package/bin/everything-gemini +950 -12
- package/docs/LAUNCH_KIT_v2.2.0.md +168 -0
- package/docs/LAUNCH_KIT_v2.4.0.md +164 -0
- package/docs/LAUNCH_KIT_v2.5.0.md +187 -0
- package/docs/index.html +20 -10
- package/examples/python-sdk/ephemeral_swarm_orchestration.py +169 -0
- package/examples/python-sdk/resilience_and_telemetry_pipeline.py +183 -0
- package/hooks/drift-detector/hooks.json +16 -0
- package/hooks/drift-detector/scripts/detect-drift.sh +50 -0
- package/hooks/human-gate/hooks.json +16 -0
- package/hooks/human-gate/scripts/human-gate.sh +67 -0
- package/hooks/thinking-budget/hooks.json +16 -0
- package/hooks/thinking-budget/scripts/budget-thinking.sh +56 -0
- package/package.json +1 -1
- package/skills/bigquery-vector-search/SKILL.md +205 -0
- package/skills/cloud-run-agents/SKILL.md +167 -0
- package/skills/context-warmer/SKILL.md +140 -0
- package/skills/cost-oracle-finops/SKILL.md +142 -0
- package/skills/dynamic-thinking-allocation/SKILL.md +105 -0
- package/skills/ebpf-sandbox-enforcer/SKILL.md +101 -0
- package/skills/envoy-ai-gateway/SKILL.md +175 -0
- package/skills/ephemeral-swarm-orchestrator/SKILL.md +124 -0
- package/skills/gemini-live-multimodal/SKILL.md +149 -0
- package/skills/harness3-mutation-testing/SKILL.md +120 -0
- package/skills/opentelemetry-agent-observability/SKILL.md +139 -0
- package/skills/pubsub-event-mesh/SKILL.md +179 -0
- package/skills/quantum-safe-cryptography/SKILL.md +84 -0
- package/skills/spanner-change-streams/SKILL.md +88 -0
- package/skills/spanner-graph-rag/SKILL.md +167 -0
- package/skills/vertex-ai-feature-store/SKILL.md +110 -0
- package/skills/webrtc-agent-gateway/SKILL.md +199 -0
- package/tests/cli.test.js +189 -14
|
@@ -0,0 +1,205 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: bigquery-vector-search
|
|
3
|
+
description: Implements Google BigQuery native vector search, VECTOR_SEARCH SQL statements, ML.GENERATE_TEXT_EMBEDDING, IVF indexing, and BigLake hybrid semantic RAG.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# BigQuery Vector Search & Semantic Analytics
|
|
7
|
+
|
|
8
|
+
Architects, indexes, and queries large-scale vector embeddings directly inside Google BigQuery using native `VECTOR_SEARCH` syntax, Inverted File (IVF) vector indexes, and `ML.GENERATE_TEXT_EMBEDDING` with Gemini models.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. Architecture: BigQuery In-Database Vector Analytics
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
16
|
+
│ Google Cloud BigQuery │
|
|
17
|
+
│ │
|
|
18
|
+
│ ┌────────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │
|
|
19
|
+
│ │ Structured Records │ │ Unstructured │ │ BigLake Object │ │
|
|
20
|
+
│ │ (SQL Tables) │ │ Text & Metadata │ │ Tables (GCS) │ │
|
|
21
|
+
│ └─────────┬──────────┘ └────────┬─────────┘ └────────┬────────┘ │
|
|
22
|
+
│ │ │ │ │
|
|
23
|
+
│ └──────────────────┬───────┴─────────────────────────┘ │
|
|
24
|
+
│ │ │
|
|
25
|
+
│ ▼ │
|
|
26
|
+
│ ┌───────────────────────────────────────────────┐ │
|
|
27
|
+
│ │ ML.GENERATE_TEXT_EMBEDDING (Vertex AI Gemini) │ │
|
|
28
|
+
│ │ 768-dim embeddings stored as ARRAY<FLOAT64> │ │
|
|
29
|
+
│ └───────────────────────┬───────────────────────┘ │
|
|
30
|
+
│ │ │
|
|
31
|
+
│ ▼ │
|
|
32
|
+
│ ┌───────────────────────────────────────────────┐ │
|
|
33
|
+
│ │ IVF Vector Index (Inverted File Partitioning) │ │
|
|
34
|
+
│ │ Distance: COSINE / EUCLIDEAN / DOT_PRODUCT │ │
|
|
35
|
+
│ └───────────────────────┬───────────────────────┘ │
|
|
36
|
+
│ │ │
|
|
37
|
+
│ ▼ │
|
|
38
|
+
│ ┌───────────────────────────────────────────────┐ │
|
|
39
|
+
│ │ VECTOR_SEARCH + Post-Filter SQL Join │ │
|
|
40
|
+
│ │ Sub-second Top-K nearest neighbor retrieval │ │
|
|
41
|
+
│ └───────────────────────────────────────────────┘ │
|
|
42
|
+
└─────────────────────────────────────────────────────────────────────────────┘
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## 2. IVF Vector Indexing
|
|
48
|
+
|
|
49
|
+
BigQuery supports Inverted File Flat (IVF) indexes to partition vector space into Voronoi cells, accelerating approximate nearest neighbor (ANN) retrieval across billions of rows.
|
|
50
|
+
|
|
51
|
+
```sql
|
|
52
|
+
-- 1. Create native vector index on embedding column
|
|
53
|
+
CREATE VECTOR INDEX IF NOT EXISTS knowledge_vector_idx
|
|
54
|
+
ON `my_project.agent_knowledge.document_chunks`(embedding)
|
|
55
|
+
STORING(chunk_id, doc_id, section_title)
|
|
56
|
+
OPTIONS(
|
|
57
|
+
index_type = 'IVF',
|
|
58
|
+
distance_type = 'COSINE',
|
|
59
|
+
ivf_options = '{"num_lists": 1000}'
|
|
60
|
+
);
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
- **`num_lists`**: The number of Voronoi partitions (rule of thumb: $\sqrt{N}$ where $N$ is row count).
|
|
64
|
+
- **`distance_type`**: `COSINE` (recommended for normalized embeddings), `EUCLIDEAN`, or `DOT_PRODUCT`.
|
|
65
|
+
- **`STORING`**: Co-locates frequently queried payload attributes in the vector index leaf nodes, eliminating secondary table lookups.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## 3. In-Database Embeddings Generation with Gemini
|
|
70
|
+
|
|
71
|
+
Connect BigQuery directly to Vertex AI Gemini text-embedding models:
|
|
72
|
+
|
|
73
|
+
```sql
|
|
74
|
+
-- 1. Create connection to Vertex AI Remote Model
|
|
75
|
+
CREATE OR REPLACE MODEL `my_project.agent_knowledge.gemini_embedding_model`
|
|
76
|
+
REMOTE WITH CONNECTION `projects/my_project/locations/us-central1/connections/vertex-ai-conn`
|
|
77
|
+
OPTIONS(
|
|
78
|
+
endpoint = 'text-embedding-004'
|
|
79
|
+
);
|
|
80
|
+
|
|
81
|
+
-- 2. Batch generate embeddings for newly inserted document chunks
|
|
82
|
+
INSERT INTO `my_project.agent_knowledge.document_chunks` (chunk_id, doc_id, content, embedding)
|
|
83
|
+
SELECT
|
|
84
|
+
ml_generate_embedding_result.chunk_id,
|
|
85
|
+
ml_generate_embedding_result.doc_id,
|
|
86
|
+
ml_generate_embedding_result.content,
|
|
87
|
+
ml_generate_embedding_result.text_embedding AS embedding
|
|
88
|
+
FROM ML.GENERATE_TEXT_EMBEDDING(
|
|
89
|
+
MODEL `my_project.agent_knowledge.gemini_embedding_model`,
|
|
90
|
+
(
|
|
91
|
+
SELECT chunk_id, doc_id, content
|
|
92
|
+
FROM `my_project.agent_knowledge.raw_chunks`
|
|
93
|
+
WHERE processed_at IS NULL
|
|
94
|
+
),
|
|
95
|
+
STRUCT(
|
|
96
|
+
'RETRIEVAL_DOCUMENT' AS task_type,
|
|
97
|
+
768 AS output_dimensionality
|
|
98
|
+
)
|
|
99
|
+
) AS ml_generate_embedding_result;
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## 4. Native `VECTOR_SEARCH` Execution
|
|
105
|
+
|
|
106
|
+
Query embeddings directly with sub-second latency and zero external vector database infrastructure:
|
|
107
|
+
|
|
108
|
+
```sql
|
|
109
|
+
-- Top-10 semantic search with structured tenant/date pre-filtering
|
|
110
|
+
SELECT
|
|
111
|
+
base.chunk_id,
|
|
112
|
+
base.doc_id,
|
|
113
|
+
base.content,
|
|
114
|
+
distance
|
|
115
|
+
FROM VECTOR_SEARCH(
|
|
116
|
+
TABLE `my_project.agent_knowledge.document_chunks`,
|
|
117
|
+
'embedding',
|
|
118
|
+
(
|
|
119
|
+
-- Embed user query dynamically using Gemini
|
|
120
|
+
SELECT text_embedding AS query_vector
|
|
121
|
+
FROM ML.GENERATE_TEXT_EMBEDDING(
|
|
122
|
+
MODEL `my_project.agent_knowledge.gemini_embedding_model`,
|
|
123
|
+
(SELECT 'What are the spend limit policies for autonomous subagents?' AS content),
|
|
124
|
+
STRUCT('RETRIEVAL_QUERY' AS task_type, 768 AS output_dimensionality)
|
|
125
|
+
)
|
|
126
|
+
),
|
|
127
|
+
top_k => 10,
|
|
128
|
+
distance_type => 'COSINE',
|
|
129
|
+
options => '{"fraction_lists_to_search": 0.05}'
|
|
130
|
+
);
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
- **`fraction_lists_to_search`**: Controls recall vs. latency tradeoff (0.01 to 0.20). Higher values increase recall at slightly higher compute cost.
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## 5. Hybrid Search (Lexical BM25 + Semantic Vector)
|
|
138
|
+
|
|
139
|
+
Combine BigQuery native search indexes (`SEARCH`) with `VECTOR_SEARCH` using Reciprocal Rank Fusion (RRF):
|
|
140
|
+
|
|
141
|
+
```sql
|
|
142
|
+
WITH vector_matches AS (
|
|
143
|
+
SELECT
|
|
144
|
+
base.chunk_id,
|
|
145
|
+
base.content,
|
|
146
|
+
ROW_NUMBER() OVER (ORDER BY distance ASC) AS vector_rank
|
|
147
|
+
FROM VECTOR_SEARCH(
|
|
148
|
+
TABLE `my_project.agent_knowledge.document_chunks`,
|
|
149
|
+
'embedding',
|
|
150
|
+
(SELECT text_embedding FROM ML.GENERATE_TEXT_EMBEDDING(
|
|
151
|
+
MODEL `my_project.agent_knowledge.gemini_embedding_model`,
|
|
152
|
+
(SELECT 'subagent spend guardrails' AS content),
|
|
153
|
+
STRUCT('RETRIEVAL_QUERY' AS task_type)
|
|
154
|
+
)),
|
|
155
|
+
top_k => 50,
|
|
156
|
+
distance_type => 'COSINE'
|
|
157
|
+
)
|
|
158
|
+
),
|
|
159
|
+
keyword_matches AS (
|
|
160
|
+
SELECT
|
|
161
|
+
chunk_id,
|
|
162
|
+
content,
|
|
163
|
+
ROW_NUMBER() OVER () AS keyword_rank
|
|
164
|
+
FROM `my_project.agent_knowledge.document_chunks`
|
|
165
|
+
WHERE SEARCH(content, 'guardrail spend budget ceiling')
|
|
166
|
+
LIMIT 50
|
|
167
|
+
)
|
|
168
|
+
SELECT
|
|
169
|
+
COALESCE(v.chunk_id, k.chunk_id) AS chunk_id,
|
|
170
|
+
COALESCE(v.content, k.content) AS content,
|
|
171
|
+
(
|
|
172
|
+
COALESCE(1.0 / (60 + v.vector_rank), 0.0) +
|
|
173
|
+
COALESCE(1.0 / (60 + k.keyword_rank), 0.0)
|
|
174
|
+
) AS rrf_score
|
|
175
|
+
FROM vector_matches v
|
|
176
|
+
FULL OUTER JOIN keyword_matches k ON v.chunk_id = k.chunk_id
|
|
177
|
+
ORDER BY rrf_score DESC
|
|
178
|
+
LIMIT 10;
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## 6. Enterprise Best Practices & Anti-Patterns
|
|
184
|
+
|
|
185
|
+
| Category | Recommended Pattern | Anti-Pattern |
|
|
186
|
+
| :--- | :--- | :--- |
|
|
187
|
+
| **Index Choice** | Use IVF with `num_lists` scaled to $\sqrt{N}$ for $N > 100,000$ rows | Querying `ARRAY<FLOAT64>` with brute-force `COSINE_DISTANCE()` |
|
|
188
|
+
| **Payload Storage** | Store lightweight metadata with `STORING(...)` inside index | Joining 10 high-cardinality tables after vector search |
|
|
189
|
+
| **Embedding Types** | Specify `RETRIEVAL_DOCUMENT` during indexing, `RETRIEVAL_QUERY` during search | Using default task types causing asymmetric vector drift |
|
|
190
|
+
| **Partitioning** | Partition and cluster base table on tenant ID or ingestion date | Single unpartitioned table forcing full-table vector scans |
|
|
191
|
+
|
|
192
|
+
---
|
|
193
|
+
|
|
194
|
+
## 7. CLI Runbook (`bq` CLI)
|
|
195
|
+
|
|
196
|
+
```bash
|
|
197
|
+
# 1. Create BigQuery dataset for vector embeddings
|
|
198
|
+
bq mk --dataset --location=us-central1 my_project:agent_knowledge
|
|
199
|
+
|
|
200
|
+
# 2. Check vector index status and coverage
|
|
201
|
+
bq show --format=prettyjson my_project:agent_knowledge.document_chunks | jq '.vectorSearchStatistics'
|
|
202
|
+
|
|
203
|
+
# 3. Dry-run vector query to estimate bytes scanned
|
|
204
|
+
bq query --use_legacy_sql=false --dry_run < vector_query.sql
|
|
205
|
+
```
|
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: cloud-run-agents
|
|
3
|
+
description: Packages, deploys, and scales autonomous Antigravity and Gemini agents on Google Cloud Run services and Cloud Run Jobs with Eventarc triggers and GPU scheduling.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Cloud Run Autonomous Agents
|
|
7
|
+
|
|
8
|
+
Packages, containerizes, and deploys autonomous Google Antigravity & Gemini agents on Google Cloud Run. Covers HTTP agent services, batch-oriented Cloud Run Jobs for recursive Harness² probing, Eventarc event triggers, and NVIDIA L4 GPU acceleration.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. Architecture: Cloud Run Agent Execution Models
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
Google Cloud Run
|
|
16
|
+
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
17
|
+
│ 1. CLOUD RUN SERVICE (Interactive / Webhook Agents) │
|
|
18
|
+
│ • Real-time Gemini Live WebSocket / HTTP streaming │
|
|
19
|
+
│ • Automatic scale-to-zero when idle │
|
|
20
|
+
│ • Up to 60-minute request timeout with WebSockets enabled │
|
|
21
|
+
├─────────────────────────────────────────────────────────────────────────────┤
|
|
22
|
+
│ 2. CLOUD RUN JOBS (Batch / Recursive Probing Agents) │
|
|
23
|
+
│ • Long-running Harness² Propose-Probe-Compose workflows (up to 24 hours) │
|
|
24
|
+
│ • Parallel array task execution for multi-candidate testing │
|
|
25
|
+
│ • Zero HTTP gateway overhead; triggered via Cloud Scheduler / Workflows │
|
|
26
|
+
├─────────────────────────────────────────────────────────────────────────────┤
|
|
27
|
+
│ 3. CLOUD RUN WITH GPU (NVIDIA L4 / T4) │
|
|
28
|
+
│ • Co-locates agent logic with local verification models (vLLM / Triton) │
|
|
29
|
+
│ • Direct hardware acceleration for code analysis & AST parsing │
|
|
30
|
+
└─────────────────────────────────────────────────────────────────────────────┘
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 2. Agent Container Specification (Dockerfile)
|
|
36
|
+
|
|
37
|
+
Production multi-stage Dockerfile bundling the agent runtime, Node CLI, and Python SDK:
|
|
38
|
+
|
|
39
|
+
```dockerfile
|
|
40
|
+
# syntax=docker/dockerfile:1
|
|
41
|
+
FROM python:3.11-slim AS base
|
|
42
|
+
|
|
43
|
+
ENV PYTHONUNBUFFERED=1 \
|
|
44
|
+
DEBIAN_FRONTEND=noninteractive \
|
|
45
|
+
PORT=8080
|
|
46
|
+
|
|
47
|
+
WORKDIR /app
|
|
48
|
+
|
|
49
|
+
# Install system dependencies & Node.js for CLI tools
|
|
50
|
+
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
51
|
+
curl \
|
|
52
|
+
git \
|
|
53
|
+
ca-certificates \
|
|
54
|
+
&& curl -fsSL https://deb.nodesource.com/setup_20.x | bash - \
|
|
55
|
+
&& apt-get install -y nodejs \
|
|
56
|
+
&& rm -rf /var/lib/apt/lists/*
|
|
57
|
+
|
|
58
|
+
# Install everything-gemini CLI globally
|
|
59
|
+
RUN npm install -g everything-gemini
|
|
60
|
+
|
|
61
|
+
# Copy Python requirements & install
|
|
62
|
+
COPY requirements.txt .
|
|
63
|
+
RUN pip install --no-cache-dir -r requirements.txt
|
|
64
|
+
|
|
65
|
+
# Copy agent definitions and source code
|
|
66
|
+
COPY . .
|
|
67
|
+
|
|
68
|
+
# Run as non-root user for enterprise container security
|
|
69
|
+
RUN useradd -m -u 10001 agentuser && chown -R agentuser:agentuser /app
|
|
70
|
+
USER 10001
|
|
71
|
+
|
|
72
|
+
EXPOSE 8080
|
|
73
|
+
ENTRYPOINT ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8080"]
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## 3. Cloud Run Service Deployment (`gcloud`)
|
|
79
|
+
|
|
80
|
+
Deploy an interactive agent service with Workload Identity and automated secret injection:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
gcloud run deploy antigravity-agent-service \
|
|
84
|
+
--image="gcr.io/my_project/antigravity-agent:v2.2.0" \
|
|
85
|
+
--region="us-central1" \
|
|
86
|
+
--platform="managed" \
|
|
87
|
+
--service-account="agent-runner@my_project.iam.gserviceaccount.com" \
|
|
88
|
+
--cpu=4 \
|
|
89
|
+
--memory=8Gi \
|
|
90
|
+
--concurrency=10 \
|
|
91
|
+
--timeout=3600 \
|
|
92
|
+
--min-instances=0 \
|
|
93
|
+
--max-instances=20 \
|
|
94
|
+
--set-env-vars="GEMINI_MODEL=gemini-2.0-flash,MAX_USD_BUDGET=15.00" \
|
|
95
|
+
--set-secrets="GEMINI_API_KEY=gemini-api-key:latest" \
|
|
96
|
+
--vpc-egress=private-ranges-only \
|
|
97
|
+
--ingress=internal-and-cloud-load-balancing
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## 4. Cloud Run Jobs for Harness² Recursive Probing
|
|
103
|
+
|
|
104
|
+
When running test-time recursive optimization (Propose-Probe-Compose) that exceeds standard HTTP request limits, package the run into a Cloud Run Job:
|
|
105
|
+
|
|
106
|
+
```bash
|
|
107
|
+
# 1. Create Cloud Run Job with 3 parallel task instances
|
|
108
|
+
gcloud run jobs create harness2-probe-runner \
|
|
109
|
+
--image="gcr.io/my_project/harness2-runner:v2.2.0" \
|
|
110
|
+
--region="us-central1" \
|
|
111
|
+
--tasks=3 \
|
|
112
|
+
--max-retries=1 \
|
|
113
|
+
--task-timeout=7200s \
|
|
114
|
+
--cpu=8 \
|
|
115
|
+
--memory=16Gi \
|
|
116
|
+
--command="everything-gemini" \
|
|
117
|
+
--args="harness2,run,--bench,local,--max-usd,25,--snapshot,--json"
|
|
118
|
+
|
|
119
|
+
# 2. Trigger asynchronous execution of the probe job
|
|
120
|
+
gcloud run jobs execute harness2-probe-runner --region="us-central1" --wait
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
## 5. Eventarc Trigger Binding (Event-Driven Agent Workflow)
|
|
126
|
+
|
|
127
|
+
Automatically invoke agents whenever new code repositories or error traces land in Cloud Storage:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
gcloud eventarc triggers create trace-ingestion-trigger \
|
|
131
|
+
--location="us-central1" \
|
|
132
|
+
--destination-run-service="antigravity-agent-service" \
|
|
133
|
+
--destination-run-region="us-central1" \
|
|
134
|
+
--destination-run-path="/webhooks/trace-ingest" \
|
|
135
|
+
--event-filters="type=google.cloud.storage.object.v1.finalized" \
|
|
136
|
+
--event-filters="bucket=production-failure-traces" \
|
|
137
|
+
--service-account="eventarc-invoker@my_project.iam.gserviceaccount.com"
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 6. NVIDIA L4 GPU Configuration
|
|
143
|
+
|
|
144
|
+
Accelerate on-container verification models or local embedding calculations:
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
gcloud beta run deploy local-verifier-agent \
|
|
148
|
+
--image="gcr.io/my_project/verifier-agent-l4:latest" \
|
|
149
|
+
--region="us-central1" \
|
|
150
|
+
--cpu=8 \
|
|
151
|
+
--memory=32Gi \
|
|
152
|
+
--gpu=1 \
|
|
153
|
+
--gpu-type=nvidia-l4 \
|
|
154
|
+
--no-cpu-throttling \
|
|
155
|
+
--timeout=1800
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## 7. Enterprise Production Checklist
|
|
161
|
+
|
|
162
|
+
| Category | Requirement | Verification Method |
|
|
163
|
+
| :--- | :--- | :--- |
|
|
164
|
+
| **IAM Security** | Dedicated service account with least-privilege Vertex AI roles | `roles/aiplatform.user` only; no Owner/Editor permissions |
|
|
165
|
+
| **Secret Management** | Mount API keys directly from Google Secret Manager | Use `--set-secrets`; never bake keys into Docker layers |
|
|
166
|
+
| **Budget Enforcement** | Enforce `--set-env-vars="MAX_USD_BUDGET=..."` bound to `budget-guard` | Verification in PreToolUse hook |
|
|
167
|
+
| **Graceful Shutdown** | Handle `SIGTERM` within 10s to flush ephemeral logs to Cloud Storage | Python `signal.signal(signal.SIGTERM, handler)` |
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: context-warmer
|
|
3
|
+
description: Analyzes active repository workspace structure, clusters knowledge items, and pre-warms Gemini prefix context caches (1M-2M+ tokens) before long-running multi-agent invocations.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Gemini Context Warmer (Prefix Cache Orchestration)
|
|
7
|
+
|
|
8
|
+
Automates prefix context cache pre-warming for Google Gemini 2.0 and 3.8 models, maximizing cache hit rates, lowering token expenditure by up to 75%, and slashing time-to-first-token (TTFT) from seconds to milliseconds across long-running Antigravity agent loops.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. Architectural Strategy
|
|
13
|
+
|
|
14
|
+
Gemini Context Caching uses **deterministic prefix matching** over an immutable sequence of static tokens. The `context-warmer` skill structures repository knowledge into an optimal cache hierarchy:
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
18
|
+
│ 1. FROZEN PREFIX (Cached via google-genai client.caches.create) │
|
|
19
|
+
│ ├── System Directives & Security Rules (rules/common/GEMINI.md) │
|
|
20
|
+
│ ├── Multi-Agent Interaction Protocols (rules/common/AGENTS.md) │
|
|
21
|
+
│ ├── Repository Structural Map & Dependency Schemas │
|
|
22
|
+
│ └── Workspace Knowledge Items (<appDataDir>/knowledge/) │
|
|
23
|
+
├─────────────────────────────────────────────────────────────────────────────┤
|
|
24
|
+
│ 2. DYNAMIC TURN SUFFIX (Uncached - Sent with each execution turn) │
|
|
25
|
+
│ ├── Active User Objective & Incremental Task Diff │
|
|
26
|
+
│ └── Recent Execution Logs / Transient Subagent Scratchpad │
|
|
27
|
+
└─────────────────────────────────────────────────────────────────────────────┘
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
---
|
|
31
|
+
|
|
32
|
+
## 2. Thresholds & Cost Efficiency
|
|
33
|
+
|
|
34
|
+
| Metric | Non-Cached Gemini Turn | Context-Warmed Prefix Cache | Efficiency Gain |
|
|
35
|
+
| :--- | :--- | :--- | :--- |
|
|
36
|
+
| **Input Cost** | $0.075 / 1M tokens | $0.01875 / 1M tokens | **75% Cost Reduction** |
|
|
37
|
+
| **TTFT Latency** | 2,800ms - 4,500ms | 280ms - 450ms | **10x Faster First Token** |
|
|
38
|
+
| **Minimum Tokens** | 1 token | 32,768 tokens minimum | Optimal at >100k tokens |
|
|
39
|
+
| **Default TTL** | None (ephemeral) | 3,600s (1 hour, auto-refresh) | Sustained across sessions |
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## 3. Python SDK Cache Warming Implementation
|
|
44
|
+
|
|
45
|
+
Use the official `google-genai` SDK to construct and warm the persistent prefix cache before orchestrating subagents:
|
|
46
|
+
|
|
47
|
+
```python
|
|
48
|
+
import os
|
|
49
|
+
from datetime import timedelta
|
|
50
|
+
from google import genai
|
|
51
|
+
from google.genai import types
|
|
52
|
+
|
|
53
|
+
def prewarm_workspace_cache(
|
|
54
|
+
workspace_root: str,
|
|
55
|
+
model: str = "gemini-2.0-flash",
|
|
56
|
+
ttl_minutes: int = 60
|
|
57
|
+
) -> str:
|
|
58
|
+
"""Scans workspace KIs, builds frozen prefix content, and creates cached content resource."""
|
|
59
|
+
client = genai.Client()
|
|
60
|
+
|
|
61
|
+
# 1. Collate static rules and Knowledge Items (KIs)
|
|
62
|
+
static_documents = []
|
|
63
|
+
rules_path = os.path.join(workspace_root, "rules", "common", "GEMINI.md")
|
|
64
|
+
if os.path.exists(rules_path):
|
|
65
|
+
with open(rules_path, "r", encoding="utf-8") as f:
|
|
66
|
+
static_documents.append(f.read())
|
|
67
|
+
|
|
68
|
+
# 2. Gather repository knowledge items
|
|
69
|
+
ki_root = os.path.expanduser("~/.gemini/knowledge")
|
|
70
|
+
if os.path.exists(ki_root):
|
|
71
|
+
for root, _, files in os.walk(ki_root):
|
|
72
|
+
for file in files:
|
|
73
|
+
if file.endswith((".md", ".json", ".yaml")):
|
|
74
|
+
with open(os.path.join(root, file), "r", encoding="utf-8") as f:
|
|
75
|
+
static_documents.append(f.read())
|
|
76
|
+
|
|
77
|
+
combined_content = "\n\n---\n\n".join(static_documents)
|
|
78
|
+
|
|
79
|
+
# 3. Create frozen context cache
|
|
80
|
+
cache = client.caches.create(
|
|
81
|
+
model=model,
|
|
82
|
+
config=types.CreateCachedContentConfig(
|
|
83
|
+
contents=[types.Content(parts=[types.Part.from_text(combined_content)])],
|
|
84
|
+
display_name="everything-gemini-workspace-cache",
|
|
85
|
+
ttl=f"{ttl_minutes * 60}s",
|
|
86
|
+
)
|
|
87
|
+
)
|
|
88
|
+
|
|
89
|
+
print(f"🔥 Cache Warmed Successfully! Cache Name: {cache.name}")
|
|
90
|
+
print(f"⏱️ TTL: {ttl_minutes} minutes | Expires at: {cache.expire_time}")
|
|
91
|
+
return cache.name
|
|
92
|
+
|
|
93
|
+
def invoke_cached_subagent(cache_name: str, task_prompt: str, model: str = "gemini-2.0-flash"):
|
|
94
|
+
"""Invokes multi-agent query referencing the warmed prefix cache."""
|
|
95
|
+
client = genai.Client()
|
|
96
|
+
response = client.models.generate_content(
|
|
97
|
+
model=model,
|
|
98
|
+
contents=task_prompt,
|
|
99
|
+
config=types.GenerateContentConfig(
|
|
100
|
+
cached_content=cache_name,
|
|
101
|
+
temperature=0.2,
|
|
102
|
+
)
|
|
103
|
+
)
|
|
104
|
+
return response.text
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## 4. TypeScript / Node.js Warming Pattern
|
|
110
|
+
|
|
111
|
+
```typescript
|
|
112
|
+
import { GoogleGenAI } from '@google/genai';
|
|
113
|
+
import * as fs from 'fs';
|
|
114
|
+
import * as path from 'path';
|
|
115
|
+
|
|
116
|
+
export async function warmPrefixContext(repoRoot: string): Promise<string> {
|
|
117
|
+
const ai = new GoogleGenAI();
|
|
118
|
+
const rules = fs.readFileSync(path.join(repoRoot, 'rules', 'common', 'GEMINI.md'), 'utf8');
|
|
119
|
+
|
|
120
|
+
const cache = await ai.caches.create({
|
|
121
|
+
model: 'gemini-2.0-flash',
|
|
122
|
+
config: {
|
|
123
|
+
displayName: 'antigravity-repo-cache',
|
|
124
|
+
ttl: '3600s',
|
|
125
|
+
contents: [{ role: 'user', parts: [{ text: rules }] }]
|
|
126
|
+
}
|
|
127
|
+
});
|
|
128
|
+
|
|
129
|
+
return cache.name;
|
|
130
|
+
}
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## 5. Verification Checklist
|
|
136
|
+
|
|
137
|
+
- [ ] Static prefix exceeds the minimum 32k token threshold.
|
|
138
|
+
- [ ] No variable nonces, random seeds, or dynamic timestamps are embedded in the static prefix.
|
|
139
|
+
- [ ] Subagent invocations specify `cached_content=cache.name`.
|
|
140
|
+
- [ ] TTL refreshed before expiration for long-running recursive optimization loops.
|
|
@@ -0,0 +1,142 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: cost-oracle-finops
|
|
3
|
+
description: Pre-flight predictive token cost modeling, spend budgeting, and pricing tier optimization for Gemini 2.0 Flash, Pro, and Gemini 3.8 extended reasoning agents.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Cost Oracle & FinOps Agent Modeling
|
|
7
|
+
|
|
8
|
+
Implements pre-flight predictive token cost estimation, model pricing tier arbitrage, and autonomous budget forecasting for multi-agent workflows across Google Gemini 2.0 Flash, Gemini 2.0 Pro, and Gemini 3.8 models.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. Architecture: Pre-Flight Financial Oracle
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
16
|
+
│ PRE-FLIGHT COST ORACLE ENGINE │
|
|
17
|
+
│ │
|
|
18
|
+
│ ┌─────────────────────┐ │
|
|
19
|
+
│ │ Proposed Task & │ │
|
|
20
|
+
│ │ Subagent Trajectory │ │
|
|
21
|
+
│ └──────────┬──────────┘ │
|
|
22
|
+
│ │ │
|
|
23
|
+
│ ▼ │
|
|
24
|
+
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
25
|
+
│ │ 1. Token Predictor (Context Caching vs. Dynamic Suffix Heuristics) │ │
|
|
26
|
+
│ │ • System & Skill Frontmatter Tokens │ │
|
|
27
|
+
│ │ • Expected Reasoning Depth (Extended Thinking Tokens) │ │
|
|
28
|
+
│ │ • Tool Call Frequency & Speculative Probing Branches │ │
|
|
29
|
+
│ └──────────────────────────────────┬──────────────────────────────────┘ │
|
|
30
|
+
│ │ │
|
|
31
|
+
│ ▼ │
|
|
32
|
+
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
33
|
+
│ │ 2. FinOps Pricing Engine (Per Model Rates & Discount Tiers) │ │
|
|
34
|
+
│ │ • Gemini 2.0 Flash: $0.075 / 1M input, $0.30 / 1M output │ │
|
|
35
|
+
│ │ • Context Caching: 75% input discount ($0.01875 / 1M input) │ │
|
|
36
|
+
│ │ • Gemini 2.0 Pro: $1.25 / 1M input, $5.00 / 1M output │ │
|
|
37
|
+
│ └──────────────────────────────────┬──────────────────────────────────┘ │
|
|
38
|
+
│ │ │
|
|
39
|
+
│ ▼ │
|
|
40
|
+
│ ┌─────────────────────────────────────────────────────────────────────┐ │
|
|
41
|
+
│ │ 3. Pre-Flight Financial Gating (Pass / Downgrade / Deny) │ │
|
|
42
|
+
│ │ • If Expected Cost > MAX_USD_BUDGET -> Abort or switch to Flash │ │
|
|
43
|
+
│ │ • If Expected Cost <= Budget Ceiling -> Issue Financial Clearance │ │
|
|
44
|
+
│ └─────────────────────────────────────────────────────────────────────┘ │
|
|
45
|
+
└─────────────────────────────────────────────────────────────────────────────┘
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## 2. Mathematical Cost Modeling Formula
|
|
51
|
+
|
|
52
|
+
Let $T_{in}$ be input tokens, $T_{cache}$ be context-cached prefix tokens, $T_{out}$ be output generation tokens, and $T_{think}$ be internal reasoning tokens:
|
|
53
|
+
|
|
54
|
+
$$\text{Cost} = (T_{cache} \times P_{cache}) + ((T_{in} - T_{cache}) \times P_{in}) + ((T_{out} + T_{think}) \times P_{out})$$
|
|
55
|
+
|
|
56
|
+
Where for **Gemini 2.0 Flash**:
|
|
57
|
+
- $P_{in} = \$0.075 / 10^6$
|
|
58
|
+
- $P_{cache} = \$0.01875 / 10^6$
|
|
59
|
+
- $P_{out} = \$0.30 / 10^6$
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## 3. Python Pre-Flight Cost Estimator
|
|
64
|
+
|
|
65
|
+
```python
|
|
66
|
+
from dataclasses import dataclass
|
|
67
|
+
from typing import Dict, Any
|
|
68
|
+
|
|
69
|
+
@dataclass
|
|
70
|
+
class CostEstimate:
|
|
71
|
+
model: str
|
|
72
|
+
expected_input_tokens: int
|
|
73
|
+
cached_prefix_tokens: int
|
|
74
|
+
expected_output_tokens: int
|
|
75
|
+
expected_thinking_tokens: int
|
|
76
|
+
estimated_usd: float
|
|
77
|
+
confidence_interval: tuple[float, float]
|
|
78
|
+
|
|
79
|
+
class FinOpsCostOracle:
|
|
80
|
+
RATES = {
|
|
81
|
+
"gemini-2.0-flash": {
|
|
82
|
+
"input": 0.075 / 1_000_000,
|
|
83
|
+
"cache": 0.01875 / 1_000_000,
|
|
84
|
+
"output": 0.30 / 1_000_000
|
|
85
|
+
},
|
|
86
|
+
"gemini-2.0-pro": {
|
|
87
|
+
"input": 1.25 / 1_000_000,
|
|
88
|
+
"cache": 0.3125 / 1_000_000,
|
|
89
|
+
"output": 5.00 / 1_000_000
|
|
90
|
+
}
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
def estimate_task_cost(
|
|
94
|
+
self,
|
|
95
|
+
model: str,
|
|
96
|
+
prompt_length: int,
|
|
97
|
+
active_skills_count: int,
|
|
98
|
+
estimated_turns: int = 5,
|
|
99
|
+
uses_extended_thinking: bool = False
|
|
100
|
+
) -> CostEstimate:
|
|
101
|
+
rates = self.RATES.get(model, self.RATES["gemini-2.0-flash"])
|
|
102
|
+
|
|
103
|
+
# Heuristics based on empirical benchmarks
|
|
104
|
+
cached_prefix = 32_768 if active_skills_count > 10 else 0
|
|
105
|
+
input_per_turn = prompt_length + (active_skills_count * 250)
|
|
106
|
+
output_per_turn = 800
|
|
107
|
+
thinking_per_turn = 2048 if uses_extended_thinking else 0
|
|
108
|
+
|
|
109
|
+
total_input = input_per_turn * estimated_turns
|
|
110
|
+
total_output = output_per_turn * estimated_turns
|
|
111
|
+
total_thinking = thinking_per_turn * estimated_turns
|
|
112
|
+
|
|
113
|
+
cost = (
|
|
114
|
+
(cached_prefix * rates["cache"]) +
|
|
115
|
+
(max(0, total_input - cached_prefix) * rates["input"]) +
|
|
116
|
+
((total_output + total_thinking) * rates["output"])
|
|
117
|
+
)
|
|
118
|
+
|
|
119
|
+
# 95% Confidence Interval (+/- 25% due to variable branching)
|
|
120
|
+
ci = (round(cost * 0.75, 4), round(cost * 1.35, 4))
|
|
121
|
+
|
|
122
|
+
return CostEstimate(
|
|
123
|
+
model=model,
|
|
124
|
+
expected_input_tokens=total_input,
|
|
125
|
+
cached_prefix_tokens=cached_prefix,
|
|
126
|
+
expected_output_tokens=total_output,
|
|
127
|
+
expected_thinking_tokens=total_thinking,
|
|
128
|
+
estimated_usd=round(cost, 4),
|
|
129
|
+
confidence_interval=ci
|
|
130
|
+
)
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## 4. Enterprise FinOps Best Practices
|
|
136
|
+
|
|
137
|
+
| Strategy | Action | Benefit |
|
|
138
|
+
| :--- | :--- | :--- |
|
|
139
|
+
| **Model Cascading** | Use Gemini 2.0 Flash for verification; reserve Pro for system design | Up to 80% cost reduction |
|
|
140
|
+
| **Prefix Freezing** | Maintain minimum 32,768 tokens in static prefix to qualify for context caching | 75% discount on repeated turns |
|
|
141
|
+
| **Budget Ceiling** | Enforce `--max-usd` via `hooks/budget-guard` | Guarantees zero surprise runaway charges |
|
|
142
|
+
| **Thinking Caps** | Constrain `max_thinking_tokens` on speculative probes | Prevents unbounded internal reasoning loops |
|