everything-gemini 2.1.0 → 2.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. package/BENCHMARK.md +106 -0
  2. package/CHANGELOG.md +81 -0
  3. package/PARTNERSHIP.md +100 -0
  4. package/README.md +63 -6
  5. package/ROADMAP_AGI_v3.0.md +109 -0
  6. package/agents/resilience-engineer.md +36 -0
  7. package/agents/swarm-coordinator.md +35 -0
  8. package/bin/everything-gemini +950 -12
  9. package/docs/LAUNCH_KIT_v2.2.0.md +168 -0
  10. package/docs/LAUNCH_KIT_v2.4.0.md +164 -0
  11. package/docs/LAUNCH_KIT_v2.5.0.md +187 -0
  12. package/docs/index.html +20 -10
  13. package/examples/python-sdk/ephemeral_swarm_orchestration.py +169 -0
  14. package/examples/python-sdk/resilience_and_telemetry_pipeline.py +183 -0
  15. package/hooks/drift-detector/hooks.json +16 -0
  16. package/hooks/drift-detector/scripts/detect-drift.sh +50 -0
  17. package/hooks/human-gate/hooks.json +16 -0
  18. package/hooks/human-gate/scripts/human-gate.sh +67 -0
  19. package/hooks/thinking-budget/hooks.json +16 -0
  20. package/hooks/thinking-budget/scripts/budget-thinking.sh +56 -0
  21. package/package.json +1 -1
  22. package/skills/bigquery-vector-search/SKILL.md +205 -0
  23. package/skills/cloud-run-agents/SKILL.md +167 -0
  24. package/skills/context-warmer/SKILL.md +140 -0
  25. package/skills/cost-oracle-finops/SKILL.md +142 -0
  26. package/skills/dynamic-thinking-allocation/SKILL.md +105 -0
  27. package/skills/ebpf-sandbox-enforcer/SKILL.md +101 -0
  28. package/skills/envoy-ai-gateway/SKILL.md +175 -0
  29. package/skills/ephemeral-swarm-orchestrator/SKILL.md +124 -0
  30. package/skills/gemini-live-multimodal/SKILL.md +149 -0
  31. package/skills/harness3-mutation-testing/SKILL.md +120 -0
  32. package/skills/opentelemetry-agent-observability/SKILL.md +139 -0
  33. package/skills/pubsub-event-mesh/SKILL.md +179 -0
  34. package/skills/quantum-safe-cryptography/SKILL.md +84 -0
  35. package/skills/spanner-change-streams/SKILL.md +88 -0
  36. package/skills/spanner-graph-rag/SKILL.md +167 -0
  37. package/skills/vertex-ai-feature-store/SKILL.md +110 -0
  38. package/skills/webrtc-agent-gateway/SKILL.md +199 -0
  39. package/tests/cli.test.js +189 -14
@@ -0,0 +1,105 @@
1
+ ---
2
+ name: dynamic-thinking-allocation
3
+ description: Profiles cognitive task complexity and dynamically routes between low-latency Gemini Flash and Gemini 3.8 Extended Thinking reasoning budgets.
4
+ ---
5
+
6
+ # Dynamic Thinking Allocation & Extended Reasoning Engine
7
+
8
+ Configures, profiles, and orchestrates dynamic "thinking token" budgets for Google Gemini 3.8 models, automatically determining the optimal cognitive depth between high-speed tool plumbing and deep architectural reasoning.
9
+
10
+ ---
11
+
12
+ ## 1. Architecture: Cognitive Complexity Router
13
+
14
+ ```
15
+ ┌─────────────────────────────────────────────────────────────────────────────┐
16
+ │ DYNAMIC THINKING ALLOCATION ROUTING MATRIX │
17
+ │ │
18
+ │ ┌────────────────────────┐ │
19
+ │ │ Incoming Agent Turn / │ │
20
+ │ │ Architectural Task │ │
21
+ │ └───────────┬────────────┘ │
22
+ │ │ │
23
+ │ ▼ │
24
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
25
+ │ │ Cognitive Profiler (Entropy, Dependency Depth, AST Complexity) │ │
26
+ │ └───────────┬─────────────────────────┬───────────────────────────────┘ │
27
+ │ │ │ │
28
+ │ Score < 30 │ 30-70 │ Score > 70 │
29
+ │ ▼ ▼ ▼
30
+ │ ┌──────────────────────┐ ┌──────────────────────┐ ┌───────────────────┐ │
31
+ │ │ Level 1: FLASH │ │ Level 2: BALANCED │ │ Level 3: DEEP │ │
32
+ │ │ (Thinking: 0 tokens) │ │ (Thinking: 2k-4k) │ │ (Thinking: 8k-24k)│ │
33
+ │ │ • Tool plumbing │ │ • Bug diagnosis │ │ • Distributed ref │ │
34
+ │ │ • CLI invocations │ │ • Schema validation │ │ • Concurrency fix │ │
35
+ │ │ • Log grepping │ │ • Test authoring │ │ • Cryptography │ │
36
+ │ └──────────────────────┘ └──────────────────────┘ └───────────────────┘ │
37
+ │ │ │ │ │
38
+ │ └─────────────────────────┼─────────────────────────┘ │
39
+ │ ▼ │
40
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
41
+ │ │ Model API Execution (Google GenAI SDK / Antigravity Engine) │ │
42
+ │ │ • `config.thinking_config = {"thinking_budget": targetTokens}` │ │
43
+ │ │ • Pre-allocated spend caps monitored via `budget-guard` │ │
44
+ │ └─────────────────────────────────────────────────────────────────────┘ │
45
+ └─────────────────────────────────────────────────────────────────────────────┘
46
+ ```
47
+
48
+ ---
49
+
50
+ ## 2. Dynamic Thinking Budget Allocation Rules
51
+
52
+ | Cognitive Tier | Target Complexity | Thinking Token Budget | Model Target | Primary Use Cases |
53
+ | :--- | :--- | :--- | :--- | :--- |
54
+ | **Flash / None** | AST depth $\le 2$, single file edit | `0` (Off) | Gemini 2.0 Flash | Mechanical formatting, directory listing, shell command dispatch, docstring typo fixes. |
55
+ | **Balanced** | Moderate AST refactor, multi-file unit tests | `2,048` - `4,096` | Gemini 2.5 Pro / Flash | Function signature refactor, API endpoint mocking, data model migration, SQL query optimization. |
56
+ | **Deep Reasoning** | Distributed race conditions, consensus protocols | `8,192` - `16,384` | Gemini 3.8 Thinking | Concurrency bug root-cause analysis, consensus algorithms, eBPF filter generation, memory leak tracing. |
57
+ | **Exhaustive Frontier** | Novel cryptographic implementations, OS kernels | `24,576` (Max) | Gemini 3.8 Thinking (Max) | Post-quantum cryptographic migration, compiler backend optimization, distributed deadlock resolution. |
58
+
59
+ ---
60
+
61
+ ## 3. Implementation with Google GenAI SDK
62
+
63
+ ```python
64
+ from google import genai
65
+ from google.genai import types
66
+
67
+ def configure_thinking_session(task_complexity_score: float, client: genai.Client) -> genai.Chat:
68
+ """Configures dynamic thinking token budget based on task entropy."""
69
+ if task_complexity_score > 0.8:
70
+ # Deep architectural reasoning
71
+ model_name = "gemini-3.8-thinking"
72
+ thinking_budget = 16384
73
+ elif task_complexity_score > 0.4:
74
+ # Balanced analytical tasks
75
+ model_name = "gemini-2.5-pro"
76
+ thinking_budget = 4096
77
+ else:
78
+ # Low-latency tool plumbing
79
+ model_name = "gemini-2.0-flash"
80
+ thinking_budget = 0
81
+
82
+ config = types.GenerateContentConfig(
83
+ temperature=0.2 if thinking_budget > 0 else 0.7,
84
+ thinking_config=types.ThinkingConfig(
85
+ thinking_budget=thinking_budget
86
+ ) if thinking_budget > 0 else None,
87
+ )
88
+
89
+ return client.chats.create(model=model_name, config=config)
90
+ ```
91
+
92
+ ---
93
+
94
+ ## 4. CLI Inspection & Profiling
95
+
96
+ ```bash
97
+ # 1. Profile a task's cognitive complexity and recommend thinking budget
98
+ everything-gemini thinking profile "Resolve distributed Spanner deadlock across 3 microservices"
99
+
100
+ # 2. Benchmark thinking token expenditure in dry-run mode
101
+ everything-gemini thinking budget --level deep --dry-run
102
+
103
+ # 3. Output structured thinking allocation matrix
104
+ everything-gemini thinking --json
105
+ ```
@@ -0,0 +1,101 @@
1
+ ---
2
+ name: ebpf-sandbox-enforcer
3
+ description: Enforces zero-trust rootless micro-VM and eBPF kernel security boundaries for speculative agent tool executions and container sandboxes.
4
+ ---
5
+
6
+ # eBPF Sandbox Enforcer & Zero-Trust Tool Isolation
7
+
8
+ Implements kernel-level zero-trust security isolation, rootless micro-VM sandboxing, Landlock filesystem restriction, and Cilium eBPF network filtering for speculative agent tool executions within Google Antigravity and Everything Gemini workflows.
9
+
10
+ ---
11
+
12
+ ## 1. Architecture: Kernel-Level Agent Sandboxing
13
+
14
+ ```
15
+ ┌─────────────────────────────────────────────────────────────────────────────┐
16
+ │ KERNEL-LEVEL AGENT SECURITY BOUNDARY │
17
+ │ │
18
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
19
+ │ │ Speculative Agent Tool Execution (`run_command`, `write_to_file`) │ │
20
+ │ └──────────────────────────────────┬──────────────────────────────────┘ │
21
+ │ │ │
22
+ │ ▼ │
23
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
24
+ │ │ 1. User-Space Policy Interceptor (`gemini-shield` & OPA Rego) │ │
25
+ │ │ • Validates command AST, blocks `rm -rf`, detects token leakages │ │
26
+ │ └──────────────────────────────────┬──────────────────────────────────┘ │
27
+ │ │ │
28
+ │ ▼ │
29
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
30
+ │ │ 2. Linux Landlock & Seccomp-BPF Syscall Filter │ │
31
+ │ │ • Traps unauthorized `execve`, `ptrace`, `setuid`, and `chroot` │ │
32
+ │ │ • Restricts filesystem I/O strictly to isolated `./scratch/` dir │ │
33
+ │ └──────────────────────────────────┬──────────────────────────────────┘ │
34
+ │ │ │
35
+ │ ▼ │
36
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
37
+ │ │ 3. Cilium eBPF Socket Filter (L3/L4/L7 Network Egress) │ │
38
+ │ │ • Blocks outbound internet access to non-whitelisted IPs │ │
39
+ │ │ • Allows Google Cloud APIs (`*.googleapis.com`) on TLS 443 │ │
40
+ │ │ • Drops all lateral movement probes to internal Kubernetes pods │ │
41
+ │ └─────────────────────────────────────────────────────────────────────┘ │
42
+ └─────────────────────────────────────────────────────────────────────────────┘
43
+ ```
44
+
45
+ ---
46
+
47
+ ## 2. Kernel Enforcement Mechanisms
48
+
49
+ ### A. Landlock Filesystem Sandboxing
50
+ Speculative agent mutations are restricted to the local workspace directory or an ephemeral scratch area. System directories (`/etc`, `/usr`, `/root`, `~/.ssh`) are marked strictly `LANDLOCK_ACCESS_FS_READ` or completely denied:
51
+
52
+ ```c
53
+ // Example Landlock ruleset structure for agent tool sandbox
54
+ #include <linux/landlock.h>
55
+ #include <sys/prctl.h>
56
+
57
+ int enforce_agent_landlock(const char *allowed_dir) {
58
+ int ruleset_fd = landlock_create_ruleset(&ruleset_attr, sizeof(ruleset_attr), 0);
59
+ // Add rule: allowed_dir gets read/write/truncate/execute
60
+ // System directories remain read-only or inaccessible
61
+ prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
62
+ landlock_restrict_self(ruleset_fd, 0);
63
+ return 0;
64
+ }
65
+ ```
66
+
67
+ ### B. Cilium eBPF Network Security Policy (CNP)
68
+ Prevents data exfiltration or reverse shells when the agent runs third-party build scripts or untrusted dependencies:
69
+
70
+ ```yaml
71
+ apiVersion: "cilium.io/v2"
72
+ kind: CiliumNetworkPolicy
73
+ metadata:
74
+ name: "agent-tool-sandbox-egress"
75
+ spec:
76
+ endpointSelector:
77
+ matchLabels:
78
+ role: "autonomous-agent-worker"
79
+ egress:
80
+ - toFQDNs:
81
+ - matchPattern: "*.googleapis.com"
82
+ - matchName: "github.com"
83
+ toPorts:
84
+ - ports:
85
+ - port: "443"
86
+ protocol: TCP
87
+ - toEntities:
88
+ - cluster
89
+ toPorts:
90
+ - ports:
91
+ - port: "53"
92
+ protocol: UDP
93
+ ```
94
+
95
+ ---
96
+
97
+ ## 3. Best Practices & Isolation Checklist
98
+
99
+ 1. **Rootless Execution**: Never execute autonomous agent commands as `root` (UID 0). Always run inside an unprivileged user namespace (`UID 10001`).
100
+ 2. **Ephemeral Disks**: Speculative test runs must target tmpfs or overlayfs volumes that can be unmounted and discarded in $<100\text{ms}$.
101
+ 3. **Audit Telemetry**: Feed eBPF syscall audit logs (`bpftool prog tracelog`) into the Mission Control UI to detect anomalous fork storms or socket scans.
@@ -0,0 +1,175 @@
1
+ ---
2
+ name: envoy-ai-gateway
3
+ description: Configure and deploy Envoy Proxy AI Gateway for Gemini models with token-aware rate limiting, multi-model circuit breakers, active health checks, and zero-trust mTLS termination.
4
+ ---
5
+
6
+ # Envoy Proxy AI Gateway (Multi-Model Traffic Director & Circuit Breaking)
7
+
8
+ Deploys and configures high-performance Envoy Proxy AI Gateways tailored for Google Gemini APIs and autonomous agent clusters. Provides token-aware rate limiting, active health checking, automatic circuit breaking across Gemini Flash, Pro, and Extended Thinking tiers, and zero-trust mTLS proxying.
9
+
10
+ ---
11
+
12
+ ## 1. AI Gateway Architecture
13
+
14
+ ```
15
+ ┌─────────────────────────────────────────────────────────────────────────────┐
16
+ │ ENVOY PROXY AI GATEWAY │
17
+ │ │
18
+ │ Agent Workflows (gRPC / HTTPS) │
19
+ │ │ │
20
+ │ ▼ │
21
+ │ ┌───────────────────────────┐ │
22
+ │ │ Envoy Proxy AI Gateway │◄──── Token-Aware Rate Limiter (Redis) │
23
+ │ │ (Filter Chain & Router) │◄──── W3C Tracecontext Injector │
24
+ │ └─────────────┬─────────────┘ │
25
+ │ │ │
26
+ │ ├── 90% Primary Route ──► Gemini 2.0 Flash (Fast Tool Use) │
27
+ │ │ │
28
+ │ ├── Circuit Broken? ──► Gemini 3.8 Pro (Fallback Tier) │
29
+ │ │ │
30
+ │ └── High Complexity? ──► Gemini 3.8 Thinking (Extended) │
31
+ └─────────────────────────────────────────────────────────────────────────────┘
32
+ ```
33
+
34
+ ---
35
+
36
+ ## 2. Envoy Proxy Configuration (`envoy.yaml`)
37
+
38
+ Below is the production Envoy proxy configuration routing requests to Google Gemini with outlier detection, retry backoff, and local rate limiting:
39
+
40
+ ```yaml
41
+ static_resources:
42
+ listeners:
43
+ - name: gemini_ingress_listener
44
+ address:
45
+ socket_address:
46
+ address: 0.0.0.0
47
+ port_value: 8443
48
+ filter_chains:
49
+ - transport_socket:
50
+ name: envoy.transport_sockets.tls
51
+ typed_config:
52
+ "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
53
+ common_tls_context:
54
+ tls_certificates:
55
+ - certificate_chain: { filename: "/etc/envoy/certs/server.crt" }
56
+ private_key: { filename: "/etc/envoy/certs/server.key" }
57
+ filters:
58
+ - name: envoy.filters.network.http_connection_manager
59
+ typed_config:
60
+ "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
61
+ stat_prefix: ingress_gemini
62
+ route_config:
63
+ name: gemini_api_routes
64
+ virtual_hosts:
65
+ - name: gemini_service
66
+ domains: ["*"]
67
+ routes:
68
+ - match:
69
+ prefix: "/v1beta/models/gemini-2.0-flash"
70
+ route:
71
+ cluster: gemini_api_upstream
72
+ timeout: 10s
73
+ retry_policy:
74
+ retry_on: "5xx,connect-failure,refused-stream,429"
75
+ num_retries: 3
76
+ retry_back_off:
77
+ base_interval: 250ms
78
+ max_interval: 2s
79
+ - match:
80
+ prefix: "/v1beta/models/gemini-3.8-thinking"
81
+ route:
82
+ cluster: gemini_api_upstream
83
+ timeout: 60s
84
+ http_filters:
85
+ - name: envoy.filters.http.local_ratelimit
86
+ typed_config:
87
+ "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
88
+ stat_prefix: http_local_rate_limiter
89
+ token_bucket:
90
+ max_tokens: 1000
91
+ tokens_per_fill: 100
92
+ fill_interval: 1s
93
+ filter_enabled:
94
+ default_value: { numerator: 100, denominator: HUNDRED }
95
+ filter_enforced:
96
+ default_value: { numerator: 100, denominator: HUNDRED }
97
+ - name: envoy.filters.http.router
98
+ typed_config:
99
+ "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
100
+
101
+ clusters:
102
+ - name: gemini_api_upstream
103
+ type: LOGICAL_DNS
104
+ dns_lookup_family: V4_ONLY
105
+ connect_timeout: 2s
106
+ lb_policy: ROUND_ROBIN
107
+ circuit_breakers:
108
+ thresholds:
109
+ - priority: DEFAULT
110
+ max_connections: 500
111
+ max_pending_requests: 100
112
+ max_requests: 1000
113
+ max_retries: 3
114
+ outlier_detection:
115
+ consecutive_5xx: 3
116
+ interval: 10s
117
+ base_ejection_time: 30s
118
+ max_ejection_percent: 50
119
+ load_assignment:
120
+ cluster_name: gemini_api_upstream
121
+ endpoints:
122
+ - lb_endpoints:
123
+ - endpoint:
124
+ address:
125
+ socket_address:
126
+ address: generativelanguage.googleapis.com
127
+ port_value: 443
128
+ transport_socket:
129
+ name: envoy.transport_sockets.tls
130
+ typed_config:
131
+ "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
132
+ sni: generativelanguage.googleapis.com
133
+ ```
134
+
135
+ ---
136
+
137
+ ## 3. Kubernetes Deployment Manifest
138
+
139
+ ```yaml
140
+ apiVersion: apps/v1
141
+ kind: Deployment
142
+ metadata:
143
+ name: envoy-gemini-gateway
144
+ namespace: agent-runtime
145
+ spec:
146
+ replicas: 3
147
+ selector:
148
+ matchLabels:
149
+ app: envoy-gemini-gateway
150
+ template:
151
+ metadata:
152
+ labels:
153
+ app: envoy-gemini-gateway
154
+ spec:
155
+ containers:
156
+ - name: envoy
157
+ image: envoyproxy/envoy:v1.31-latest
158
+ ports:
159
+ - containerPort: 8443
160
+ volumeMounts:
161
+ - name: envoy-config
162
+ mountPath: /etc/envoy
163
+ volumes:
164
+ - name: envoy-config
165
+ configMap:
166
+ name: envoy-gemini-config
167
+ ```
168
+
169
+ ---
170
+
171
+ ## 4. Key Performance Indicators & Metrics
172
+
173
+ - **Ejection Rate (`envoy.cluster.outlier_detection.ejections_enforced`)**: Alerts on sustained API throttling.
174
+ - **Circuit Breakers (`envoy.cluster.circuit_breakers.default.rq_open`)**: Triggers immediate failover to backup model tiers.
175
+ - **P99 Proxy Overhead**: Sub-1.5ms added latency across TLS handshake and routing filters.
@@ -0,0 +1,124 @@
1
+ ---
2
+ name: ephemeral-swarm-orchestrator
3
+ description: Dynamically synthesizes, deploys, and reclaims ephemeral multi-agent swarms for cross-repository parallel refactoring and consensus aggregation.
4
+ ---
5
+
6
+ # Ephemeral Swarm Orchestrator (Swarm Intelligence)
7
+
8
+ Implements dynamic on-demand synthesis, parallel task decomposition, distributed cross-repository execution, consensus aggregation, and lifecycle reclamation of ephemeral micro-agent swarms for Google Gemini and Antigravity.
9
+
10
+ ---
11
+
12
+ ## 1. Architecture: Dynamic Ephemeral Micro-Agent Swarm
13
+
14
+ ```
15
+ ┌─────────────────────────────────────────────────────────────────────────────┐
16
+ │ EPHEMERAL SWARM ORCHESTRATION PIPELINE │
17
+ │ │
18
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
19
+ │ │ 1. Swarm Coordinator & Planner Decomposition │ │
20
+ │ │ • Complex multi-repo goal or massive refactor prompt │ │
21
+ │ │ • Decomposed into DAG of atomic micro-tasks with input contracts │ │
22
+ │ └──────────────────────────────────┬──────────────────────────────────┘ │
23
+ │ │ │
24
+ │ ▼ │
25
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
26
+ │ │ 2. Dynamic Ephemeral Specialist Synthesis │ │
27
+ │ │ • Synthesizes targeted micro-agent definitions in memory │ │
28
+ │ │ • E.g. `spanner-deadlock-tuner`, `grpc-schema-validator` │ │
29
+ │ │ • Mounts minimal progressive skills & scoped tool permissions │ │
30
+ │ └──────────────────────────────────┬──────────────────────────────────┘ │
31
+ │ │ │
32
+ │ ▼ │
33
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
34
+ │ │ 3. Parallel Worker Swarm Execution (Pub/Sub & Cloud Run Jobs) │ │
35
+ │ │ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ │
36
+ │ │ │ Worker #1 (RepoA)│ │ Worker #2 (RepoB)│ │ Worker #3 (Infra)│ │ │
37
+ │ │ └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ │ │
38
+ │ └─────────────┼─────────────────────┼─────────────────────┼───────────┘ │
39
+ │ └─────────────────────┼─────────────────────┘ │
40
+ │ ▼ │
41
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
42
+ │ │ 4. Majority Consensus & Multi-Agent Voting │ │
43
+ │ │ • Aggregates diff proposals across parallel micro-agents │ │
44
+ │ │ • Conflict resolution via Byzantine/majority agreement matrix │ │
45
+ │ └──────────────────────────────────┬──────────────────────────────────┘ │
46
+ │ │ │
47
+ │ ▼ │
48
+ │ ┌─────────────────────────────────────────────────────────────────────┐ │
49
+ │ │ 5. Ephemeral Harness Teardown & Lifecycle Reclamation │ │
50
+ │ │ • Verifies property-based unit & integration tests │ │
51
+ │ │ • Distills learned instincts into persistent memory │ │
52
+ │ │ • Decommissions worker contexts and purges memory state │ │
53
+ │ └─────────────────────────────────────────────────────────────────────┘ │
54
+ └─────────────────────────────────────────────────────────────────────────────┘
55
+ ```
56
+
57
+ ---
58
+
59
+ ## 2. Core Operational Pillars
60
+
61
+ ### A. Dynamic Specialist Synthesis
62
+ Traditional multi-agent frameworks rely on rigid, static subagent registries. The Ephemeral Swarm Orchestrator dynamically constructs targeted agent personalities and tool contracts based on task requirements:
63
+
64
+ ```json
65
+ {
66
+ "swarm_id": "swarm-20261022-spanner-sync",
67
+ "coordinator": "swarm-coordinator",
68
+ "topology": "hierarchical",
69
+ "workers": [
70
+ {
71
+ "id": "worker-schema-evaluator",
72
+ "specialization": "spanner-ddl-auditor",
73
+ "scoped_skills": ["spanner-change-streams", "security-audit"],
74
+ "permitted_tools": ["view_file", "grep_search", "run_command"],
75
+ "token_budget": 8000
76
+ },
77
+ {
78
+ "id": "worker-migration-author",
79
+ "specialization": "liquibase-migration-writer",
80
+ "scoped_skills": ["database-migrator", "tdd-workflow"],
81
+ "permitted_tools": ["write_to_file", "replace_file_content", "run_command"],
82
+ "token_budget": 12000
83
+ }
84
+ ]
85
+ }
86
+ ```
87
+
88
+ ### B. Consensus & Voting Protocol
89
+ When multiple micro-agents generate speculative patches or architectural proposals, the swarm coordinator invokes a multi-agent voting round:
90
+ 1. **P2P Cross-Verification**: Worker 1 reviews Worker 2's AST patch.
91
+ 2. **Deterministic Test Execution**: Candidate diffs are executed against property-based tests in isolated ephemeral directories.
92
+ 3. **Quorum Verification**: A change requires $\ge 67\%$ consensus agreement before promotion into the primary branch.
93
+
94
+ ### C. Resource Reclamation & Zero Leaks
95
+ Upon completion or failure, the swarm executor executes cleanup routines:
96
+ * Terminates background worker processes and subprocess handles.
97
+ * Deletes scratch directories and temporary SQLite / DuckDB scratchpads.
98
+ * Commits learned operational invariants into `.agents/hooks/learned-instincts/`.
99
+
100
+ ---
101
+
102
+ ## 3. CLI Commands
103
+
104
+ ```bash
105
+ # 1. Spawn an ephemeral swarm with 3 parallel workers
106
+ everything-gemini swarm spawn "refactor auth middleware to zero-trust mTLS" --workers 3
107
+
108
+ # 2. Inspect active swarm topology and worker state
109
+ everything-gemini swarm status --json
110
+
111
+ # 3. Force teardown and reclaim all active worker resources
112
+ everything-gemini swarm reclaim --dry-run
113
+ ```
114
+
115
+ ---
116
+
117
+ ## 4. Anti-Patterns & Safety Guardrails
118
+
119
+ | Anti-Pattern | Operational Risk | Swarm Enforced Fix |
120
+ | :--- | :--- | :--- |
121
+ | **Unbounded Worker Spawning** | Exponential token burn and process fork bomb | Hard worker ceiling (`--max-workers 8`) enforced by `budget-guard`. |
122
+ | **Shared State Collisions** | Merge conflicts and dirty git tree states | Each worker operates in an isolated temporary worktree with git rollback snapshots. |
123
+ | **Zombie Subagents** | Orphaned background processes leaking memory | Mandatory timeout watchdog (`--timeout-sec 300`) with auto-reclamation. |
124
+ | **Single-Agent Bias** | Hallucinated code passes without review | Two-agent peer verification required on all synthesized diffs. |
@@ -0,0 +1,149 @@
1
+ ---
2
+ name: gemini-live-multimodal
3
+ description: Implements real-time bidirectional streaming audio and video pair-programming with the Gemini Live API, WebSocket protocols, ephemeral tokens, and voice activity detection.
4
+ ---
5
+
6
+ # Gemini Live Multimodal (Bidirectional Audio/Visual Streaming)
7
+
8
+ Implements real-time, low-latency, bidirectional audio/video interaction with Google Gemini models (Gemini 2.0 Flash and Gemini Live) using WebSockets, Ephemeral Client Tokens, Voice Activity Detection (VAD), and live screen/DOM streaming.
9
+
10
+ ---
11
+
12
+ ## 1. Architecture: Bidirectional Gemini Live API
13
+
14
+ ```
15
+ ┌─────────────────────────────────────────────────────────────────────────────┐
16
+ │ GEMINI LIVE RUNTIME │
17
+ │ │
18
+ │ ┌─────────────────────┐ Audio (PCM 16kHz) ┌─────────────────────────┐ │
19
+ │ │ Developer Audio │──────────────────────►│ Google GenAI Live API │ │
20
+ │ │ (Microphone Input) │ │ (wss://generativelanguage│ │
21
+ │ └─────────────────────┘ │ .googleapis.com/ws) │ │
22
+ │ │ │ │
23
+ │ ┌─────────────────────┐ Frames (JPEG/WebP) │ • Voice Activity Detect │ │
24
+ │ │ Active Screen / IDE │──────────────────────►│ • Extended Reasoning │ │
25
+ │ │ (Visual Context) │ │ • Sub-second TTFT Voice │ │
26
+ │ └─────────────────────┘ └───────────┬─────────────┘ │
27
+ │ │ │
28
+ │ ┌─────────────────────┐ Synthesized Audio (24kHz) │ │
29
+ │ │ Agent Voice Stream │◄──────────────────────────────────┘ │
30
+ │ │ (Speaker Output) │ │
31
+ │ └─────────────────────┘ │
32
+ └─────────────────────────────────────────────────────────────────────────────┘
33
+ ```
34
+
35
+ ---
36
+
37
+ ## 2. Ephemeral Session Token Minting
38
+
39
+ For enterprise browser/IDE security, never expose primary API keys on clients. Mint short-lived ephemeral tokens from your secure backend:
40
+
41
+ ```python
42
+ import os
43
+ from google import genai
44
+ from google.genai import types
45
+
46
+ client = genai.Client()
47
+
48
+ def create_ephemeral_live_session(user_id: str, ttl_seconds: int = 1800) -> str:
49
+ """Mints an ephemeral session token with pre-configured system instructions."""
50
+ session = client.aio.live.create_session_token(
51
+ model="gemini-2.0-flash-exp",
52
+ config=types.LiveConnectConfig(
53
+ response_modalities=[types.LiveModality.AUDIO],
54
+ system_instruction=types.Content(
55
+ parts=[types.Part.from_text(
56
+ "You are an expert pair-programming agent inside Antigravity. "
57
+ "Analyze screen frames and audio simultaneously. Be concise and actionable."
58
+ )]
59
+ ),
60
+ speech_config=types.SpeechConfig(
61
+ voice_config=types.VoiceConfig(
62
+ prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Aoede")
63
+ )
64
+ )
65
+ )
66
+ )
67
+ return session.token
68
+ ```
69
+
70
+ ---
71
+
72
+ ## 3. Bidirectional WebSocket Audio Streaming (Python Client)
73
+
74
+ Connects to the Gemini Live WebSocket endpoint and streams audio bi-directionally:
75
+
76
+ ```python
77
+ import asyncio
78
+ import websockets
79
+ import json
80
+
81
+ LIVE_WS_URL = "wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1alpha.GenerativeService.BidiGenerateContent"
82
+
83
+ async def stream_live_pair_programming(api_key: str):
84
+ url = f"{LIVE_WS_URL}?key={api_key}"
85
+
86
+ async with websockets.connect(url) as ws:
87
+ # 1. Send initial session handshake
88
+ setup_message = {
89
+ "setup": {
90
+ "model": "models/gemini-2.0-flash-exp",
91
+ "generation_config": {
92
+ "response_modalities": ["AUDIO"],
93
+ "speech_config": {
94
+ "voice_config": {
95
+ "prebuilt_voice_config": {"voice_name": "Puck"}
96
+ }
97
+ }
98
+ }
99
+ }
100
+ }
101
+ await ws.send(json.dumps(setup_message))
102
+
103
+ # 2. Receive live audio chunks & emit to speaker
104
+ async for raw_msg in ws:
105
+ response = json.loads(raw_msg)
106
+ server_content = response.get("serverContent", {})
107
+ model_turn = server_content.get("modelTurn", {})
108
+
109
+ for part in model_turn.get("parts", []):
110
+ inline_data = part.get("inlineData", {})
111
+ if inline_data.get("mimeType", "").startswith("audio/"):
112
+ pcm_chunk = inline_data.get("data")
113
+ # Send to audio output buffer
114
+ ```
115
+
116
+ ---
117
+
118
+ ## 4. Visual Screen Frame Injection
119
+
120
+ Inject captured screen frames at 1 frame per second to ground audio conversation in visual code context:
121
+
122
+ ```python
123
+ import base64
124
+
125
+ async def send_screen_frame(ws, jpeg_bytes: bytes):
126
+ """Encodes a screen capture and sends it as a visual context turn."""
127
+ frame_payload = {
128
+ "realtime_input": {
129
+ "media_chunks": [
130
+ {
131
+ "mime_type": "image/jpeg",
132
+ "data": base64.b64encode(jpeg_bytes).decode("utf-8")
133
+ }
134
+ ]
135
+ }
136
+ }
137
+ await ws.send(json.dumps(frame_payload))
138
+ ```
139
+
140
+ ---
141
+
142
+ ## 5. Enterprise Best Practices & Anti-Patterns
143
+
144
+ | Category | Recommended Pattern | Anti-Pattern |
145
+ | :--- | :--- | :--- |
146
+ | **Authentication** | Ephemeral tokens with 30m TTL minted by backend | Hardcoding permanent API keys in frontend bundles |
147
+ | **Audio Format** | PCM 16-bit little-endian at 16,000Hz mono | Compressed MP3 with variable bitrates causing VAD jitter |
148
+ | **Video Cadence** | Throttled keyframes at 1 FPS or on viewport change | 60 FPS full video ingestion saturating network bandwidth |
149
+ | **Voice Activity** | Enable built-in server-side Voice Activity Detection | Client-side manual push-to-talk creating unnatural latency |