genkit-plugin-google-cloud 0.3.2__tar.gz → 0.5.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- genkit_plugin_google_cloud-0.5.0/PARITY_ANALYSIS.md +344 -0
- {genkit_plugin_google_cloud-0.3.2 → genkit_plugin_google_cloud-0.5.0}/PKG-INFO +10 -2
- genkit_plugin_google_cloud-0.5.0/pyproject.toml +74 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/__init__.py +155 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/__init__.py +74 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/action.py +124 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/engagement.py +170 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/feature.py +186 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/generate.py +605 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/metrics.py +246 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/path.py +157 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/tracing.py +969 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud/telemetry/utils.py +217 -0
- genkit_plugin_google_cloud-0.5.0/src/genkit/py.typed +0 -0
- genkit_plugin_google_cloud-0.5.0/tests/tracing_test.py +341 -0
- genkit_plugin_google_cloud-0.3.2/pyproject.toml +0 -37
- genkit_plugin_google_cloud-0.3.2/src/genkit/plugins/google_cloud/__init__.py +0 -30
- genkit_plugin_google_cloud-0.3.2/src/genkit/plugins/google_cloud/telemetry/tracing.py +0 -118
- {genkit_plugin_google_cloud-0.3.2 → genkit_plugin_google_cloud-0.5.0}/.gitignore +0 -0
- {genkit_plugin_google_cloud-0.3.2 → genkit_plugin_google_cloud-0.5.0}/LICENSE +0 -0
- {genkit_plugin_google_cloud-0.3.2 → genkit_plugin_google_cloud-0.5.0}/README.md +0 -0
- {genkit_plugin_google_cloud-0.3.2/src/genkit → genkit_plugin_google_cloud-0.5.0/src/genkit/plugins/google_cloud}/py.typed +0 -0
|
@@ -0,0 +1,344 @@
|
|
|
1
|
+
# GCP Telemetry Parity Analysis
|
|
2
|
+
|
|
3
|
+
This document provides a comprehensive cross-language parity analysis of the Genkit GCP telemetry implementations across JavaScript, Go, and Python, verified against official Google Cloud documentation.
|
|
4
|
+
|
|
5
|
+
## Summary
|
|
6
|
+
|
|
7
|
+
| Category | JS | Go | Python | Status |
|
|
8
|
+
|----------|----|----|--------|--------|
|
|
9
|
+
| Configuration Options | ✅ | ✅ | ✅ | **PARITY** |
|
|
10
|
+
| Metrics (names, types) | ✅ | ✅ | ✅ | **PARITY** |
|
|
11
|
+
| Metric Dimensions | ✅ | ✅ | ✅ | **PARITY** (fixed) |
|
|
12
|
+
| Log Formats | ✅ | ✅ | ✅ | **PARITY** |
|
|
13
|
+
| Span Attributes | ✅ | ✅ | ✅ | **PARITY** |
|
|
14
|
+
| Error Handling | ✅ | ✅ | ✅ | **PARITY** |
|
|
15
|
+
| Constants/Limits | ✅ | ✅ | ✅ | **PARITY** (fixed) |
|
|
16
|
+
|
|
17
|
+
***
|
|
18
|
+
|
|
19
|
+
## 1. Configuration Options Comparison
|
|
20
|
+
|
|
21
|
+
### Main Configuration
|
|
22
|
+
|
|
23
|
+
| Option | JS | Go | Python | GCP Docs | Notes |
|
|
24
|
+
|--------|----|----|--------|----------|-------|
|
|
25
|
+
| `projectId` | ✅ | ✅ | ✅ | ✅ | All support auto-detection |
|
|
26
|
+
| `credentials` | ✅ | ✅ | ✅ | ✅ | ADC fallback |
|
|
27
|
+
| `sampler` | ✅ | ✅ | ✅ | ✅ | OpenTelemetry sampler |
|
|
28
|
+
| `disableMetrics` | ✅ | ✅ | ✅ | N/A | - |
|
|
29
|
+
| `disableTraces` | ✅ | ✅ | ✅ | N/A | - |
|
|
30
|
+
| `disableLoggingInputAndOutput` | ✅ (inverted) | ✅ (inverted) | ✅ (`log_input_and_output`) | N/A | Python uses positive flag |
|
|
31
|
+
| `forceDevExport` | ✅ | ✅ | ✅ | N/A | - |
|
|
32
|
+
| `metricExportIntervalMillis` | ✅ | ✅ | ✅ | ✅ (min 5s) | All enforce 5000ms min |
|
|
33
|
+
| `metricExportTimeoutMillis` | ✅ | ✅ | ✅ | N/A | - |
|
|
34
|
+
| `autoInstrumentation` | ✅ | ❌ | ❌ | N/A | JS-specific |
|
|
35
|
+
| `instrumentations` | ✅ | ❌ | ❌ | N/A | JS-specific |
|
|
36
|
+
|
|
37
|
+
### Project ID Resolution Order
|
|
38
|
+
|
|
39
|
+
| Priority | JS | Go | Python | Notes |
|
|
40
|
+
|----------|----|----|--------|-------|
|
|
41
|
+
| 1 | Explicit param | Explicit param | Explicit param | ✅ All match |
|
|
42
|
+
| 2 | - | `FIREBASE_PROJECT_ID` | `FIREBASE_PROJECT_ID` | ⚠️ JS missing |
|
|
43
|
+
| 3 | - | `GOOGLE_CLOUD_PROJECT` | `GOOGLE_CLOUD_PROJECT` | ⚠️ JS missing |
|
|
44
|
+
| 4 | - | `GCLOUD_PROJECT` | `GCLOUD_PROJECT` | ⚠️ JS missing |
|
|
45
|
+
| 5 | ADC | Credentials | Credentials dict | ✅ All match |
|
|
46
|
+
|
|
47
|
+
**Action Required:** JS should add env var resolution to match Go/Python.
|
|
48
|
+
|
|
49
|
+
***
|
|
50
|
+
|
|
51
|
+
## 2. Metrics Comparison
|
|
52
|
+
|
|
53
|
+
### Generate Metrics
|
|
54
|
+
|
|
55
|
+
| Metric | JS | Go | Python | GCP Docs |
|
|
56
|
+
|--------|----|----|--------|----------|
|
|
57
|
+
| `genkit/ai/generate/requests` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
58
|
+
| `genkit/ai/generate/latency` | ✅ Histogram | ✅ Histogram | ✅ Histogram | ✅ |
|
|
59
|
+
| `genkit/ai/generate/input/tokens` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
60
|
+
| `genkit/ai/generate/input/characters` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
61
|
+
| `genkit/ai/generate/input/images` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
62
|
+
| `genkit/ai/generate/input/videos` | ❌ | ✅ Counter | ✅ Counter | ✅ |
|
|
63
|
+
| `genkit/ai/generate/input/audio` | ❌ | ✅ Counter | ✅ Counter | ✅ |
|
|
64
|
+
| `genkit/ai/generate/output/tokens` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
65
|
+
| `genkit/ai/generate/output/characters` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
66
|
+
| `genkit/ai/generate/output/images` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
67
|
+
| `genkit/ai/generate/output/videos` | ❌ | ✅ Counter | ✅ Counter | ✅ |
|
|
68
|
+
| `genkit/ai/generate/output/audio` | ❌ | ✅ Counter | ✅ Counter | ✅ |
|
|
69
|
+
| `genkit/ai/generate/thinking/tokens` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ |
|
|
70
|
+
|
|
71
|
+
**Gap Found:** JS is missing video and audio metrics that Go and Python have.
|
|
72
|
+
|
|
73
|
+
### Feature Metrics
|
|
74
|
+
|
|
75
|
+
| Metric | JS | Go | Python | Status |
|
|
76
|
+
|--------|----|----|--------|--------|
|
|
77
|
+
| `genkit/feature/requests` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ PARITY |
|
|
78
|
+
| `genkit/feature/latency` | ✅ Histogram | ✅ Histogram | ✅ Histogram | ✅ PARITY |
|
|
79
|
+
|
|
80
|
+
### Path Metrics
|
|
81
|
+
|
|
82
|
+
| Metric | JS | Go | Python | Status |
|
|
83
|
+
|--------|----|----|--------|--------|
|
|
84
|
+
| `genkit/feature/path/requests` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ PARITY |
|
|
85
|
+
| `genkit/feature/path/latency` | ✅ Histogram | ✅ Histogram | ✅ Histogram | ✅ PARITY |
|
|
86
|
+
|
|
87
|
+
### Engagement Metrics
|
|
88
|
+
|
|
89
|
+
| Metric | JS | Go | Python | Status |
|
|
90
|
+
|--------|----|----|--------|--------|
|
|
91
|
+
| `genkit/engagement/feedback` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ PARITY |
|
|
92
|
+
| `genkit/engagement/acceptance` | ✅ Counter | ✅ Counter | ✅ Counter | ✅ PARITY |
|
|
93
|
+
|
|
94
|
+
***
|
|
95
|
+
|
|
96
|
+
## 3. Metric Dimensions Comparison
|
|
97
|
+
|
|
98
|
+
### Generate Metric Dimensions
|
|
99
|
+
|
|
100
|
+
| Dimension | JS | Go | Python | Notes |
|
|
101
|
+
|-----------|----|----|--------|-------|
|
|
102
|
+
| `modelName` | ✅ (1024 chars) | ✅ (1024 chars) | ✅ (1024 chars) | ✅ PARITY (fixed) |
|
|
103
|
+
| `featureName` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
104
|
+
| `path` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
105
|
+
| `status` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
106
|
+
| `error` | ✅ (on failure) | ✅ (on failure) | ✅ (on failure) | ✅ PARITY |
|
|
107
|
+
| `source` | `"ts"` | `"go"` | `"py"` | ✅ Correctly different |
|
|
108
|
+
| `sourceVersion` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
109
|
+
|
|
110
|
+
### Feature Metric Dimensions
|
|
111
|
+
|
|
112
|
+
| Dimension | JS | Go | Python | Notes |
|
|
113
|
+
|-----------|----|----|--------|-------|
|
|
114
|
+
| `name` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
115
|
+
| `status` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
116
|
+
| `error` | ✅ (on failure) | ✅ (on failure) | ✅ (on failure) | ✅ PARITY |
|
|
117
|
+
| `source` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
118
|
+
| `sourceVersion` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
119
|
+
|
|
120
|
+
### Path Metric Dimensions
|
|
121
|
+
|
|
122
|
+
| Dimension | JS | Go | Python | Notes |
|
|
123
|
+
|-----------|----|----|--------|-------|
|
|
124
|
+
| `featureName` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
125
|
+
| `status` | ✅ (always "failure") | ✅ | ✅ | ✅ PARITY |
|
|
126
|
+
| `error` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
127
|
+
| `path` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
128
|
+
| `source` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
129
|
+
| `sourceVersion` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
130
|
+
|
|
131
|
+
### Engagement Dimensions
|
|
132
|
+
|
|
133
|
+
| Dimension | JS | Go | Python | Notes |
|
|
134
|
+
|-----------|----|----|--------|-------|
|
|
135
|
+
| `name` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
136
|
+
| `value` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
137
|
+
| `hasText` (feedback) | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
138
|
+
| `source` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
139
|
+
| `sourceVersion` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
140
|
+
|
|
141
|
+
***
|
|
142
|
+
|
|
143
|
+
## 4. Constants and Limits Comparison
|
|
144
|
+
|
|
145
|
+
### Content Limits
|
|
146
|
+
|
|
147
|
+
| Constant | JS | Go | Python | GCP Docs | Notes |
|
|
148
|
+
|----------|----|----|--------|----------|-------|
|
|
149
|
+
| Max log content | 128,000 | 128,000 | 128,000 | N/A | ✅ PARITY |
|
|
150
|
+
| Max path length | 4,096 | 4,096 | 4,096 | N/A | ✅ PARITY |
|
|
151
|
+
| Error name truncation | 1,024 | - | 1,024 | N/A | ✅ PARITY |
|
|
152
|
+
| Error message truncation | 4,096 | - | 4,096 | N/A | ✅ PARITY |
|
|
153
|
+
| Error stack truncation | 32,768 | - | 32,768 | N/A | ✅ PARITY |
|
|
154
|
+
| Metric dimension max | 256 | - | 256 | ✅ (256) | ✅ PARITY |
|
|
155
|
+
| Model name truncation | 1,024 | 1,024 | 256 | N/A | ⚠️ Python shorter |
|
|
156
|
+
|
|
157
|
+
### Timing Constants
|
|
158
|
+
|
|
159
|
+
| Constant | JS | Go | Python | GCP Docs | Notes |
|
|
160
|
+
|----------|----|----|--------|----------|-------|
|
|
161
|
+
| Min metric interval | 5,000ms | 5,000ms | 5,000ms | ✅ 5,000ms | ✅ PARITY |
|
|
162
|
+
| Dev metric interval | 5,000ms | 5,000ms | 5,000ms | N/A | ✅ PARITY |
|
|
163
|
+
| Prod metric interval | - | 300,000ms | 300,000ms | N/A | JS uses custom |
|
|
164
|
+
| Default metric interval | - | - | 60,000ms | N/A | Python specific |
|
|
165
|
+
| Start time adjustment | 1ms | - | 1ms | N/A | ✅ PARITY |
|
|
166
|
+
|
|
167
|
+
***
|
|
168
|
+
|
|
169
|
+
## 5. Span Attributes Comparison
|
|
170
|
+
|
|
171
|
+
### Input Attributes (Read)
|
|
172
|
+
|
|
173
|
+
| Attribute | JS | Go | Python | Notes |
|
|
174
|
+
|-----------|----|----|--------|-------|
|
|
175
|
+
| `genkit:type` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
176
|
+
| `genkit:metadata:subtype` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
177
|
+
| `genkit:isRoot` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
178
|
+
| `genkit:name` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
179
|
+
| `genkit:path` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
180
|
+
| `genkit:input` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
181
|
+
| `genkit:output` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
182
|
+
| `genkit:state` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
183
|
+
| `genkit:isFailureSource` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
184
|
+
| `genkit:sessionId` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
185
|
+
| `genkit:threadName` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
186
|
+
| `genkit:metadata:flow:name` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
187
|
+
| `genkit:metadata:feedbackValue` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
188
|
+
| `genkit:metadata:textFeedback` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
189
|
+
| `genkit:metadata:acceptanceValue` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
190
|
+
|
|
191
|
+
### Output Attributes (Written)
|
|
192
|
+
|
|
193
|
+
| Attribute | JS | Go | Python | Notes |
|
|
194
|
+
|-----------|----|----|--------|-------|
|
|
195
|
+
| `genkit:input` → `<redacted>` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
196
|
+
| `genkit:output` → `<redacted>` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
197
|
+
| `/http/status_code` = "599" | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
198
|
+
| `genkit:failedSpan` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
199
|
+
| `genkit:failedPath` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
200
|
+
| `genkit:feature` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
201
|
+
| `genkit:model` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
202
|
+
| `genkit:rootState` | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
203
|
+
| Label normalization (`:` → `/`) | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
204
|
+
|
|
205
|
+
***
|
|
206
|
+
|
|
207
|
+
## 6. Log Message Format Comparison
|
|
208
|
+
|
|
209
|
+
### Generate Logs
|
|
210
|
+
|
|
211
|
+
| Log Type | JS Format | Go Format | Python Format | Status |
|
|
212
|
+
|----------|-----------|-----------|---------------|--------|
|
|
213
|
+
| Config | `Config[{path}, {model}]` | `[genkit] Config[{path}, {model}]` | `Config[{path}, {model}]` | ⚠️ Go prefix |
|
|
214
|
+
| Input | `Input[{path}, {model}] (part X of Y in message M of N)` | Same | Same | ✅ PARITY |
|
|
215
|
+
| Output | `Output[{path}, {model}] (part X of Y)` | Same | Same | ✅ PARITY |
|
|
216
|
+
|
|
217
|
+
### Feature Logs
|
|
218
|
+
|
|
219
|
+
| Log Type | JS Format | Go Format | Python Format | Status |
|
|
220
|
+
|----------|-----------|-----------|---------------|--------|
|
|
221
|
+
| Input | `Input[{path}, {name}]` | `[genkit] Input[...]` | `Input[{path}, {name}]` | ⚠️ Go prefix |
|
|
222
|
+
| Output | `Output[{path}, {name}]` | `[genkit] Output[...]` | `Output[{path}, {name}]` | ⚠️ Go prefix |
|
|
223
|
+
|
|
224
|
+
### Error Logs
|
|
225
|
+
|
|
226
|
+
| Log Type | JS Format | Go Format | Python Format | Status |
|
|
227
|
+
|----------|-----------|-----------|---------------|--------|
|
|
228
|
+
| Error | `Error[{path}, {error}]` | `[genkit] Error[...]` | `Error[{path}, {error}]` | ⚠️ Go prefix |
|
|
229
|
+
|
|
230
|
+
### Engagement Logs
|
|
231
|
+
|
|
232
|
+
| Log Type | JS Format | Go Format | Python Format | Status |
|
|
233
|
+
|----------|-----------|-----------|---------------|--------|
|
|
234
|
+
| Feedback | `UserFeedback[{name}]` | `[genkit] UserFeedback[...]` | `UserFeedback[{name}]` | ⚠️ Go prefix |
|
|
235
|
+
| Acceptance | `UserAcceptance[{name}]` | `[genkit] UserAcceptance[...]` | `UserAcceptance[{name}]` | ⚠️ Go prefix |
|
|
236
|
+
|
|
237
|
+
**Note:** Go adds `[genkit]` prefix to all logs. This is acceptable variation for log filtering.
|
|
238
|
+
|
|
239
|
+
***
|
|
240
|
+
|
|
241
|
+
## 7. GCP Log Correlation Attributes
|
|
242
|
+
|
|
243
|
+
Per [Cloud Logging documentation](https://cloud.google.com/logging/docs/structured-logging):
|
|
244
|
+
|
|
245
|
+
| Attribute | JS | Go | Python | GCP Docs | Status |
|
|
246
|
+
|-----------|----|----|--------|----------|--------|
|
|
247
|
+
| `logging.googleapis.com/trace` | ✅ | ✅ | ✅ | ✅ Required | ✅ PARITY |
|
|
248
|
+
| `logging.googleapis.com/spanId` | ✅ | ✅ | ✅ | ✅ Required | ✅ PARITY |
|
|
249
|
+
| `logging.googleapis.com/trace_sampled` | ✅ | ✅ | ✅ | ✅ Required | ✅ PARITY |
|
|
250
|
+
|
|
251
|
+
Format: `projects/{PROJECT_ID}/traces/{TRACE_ID}`
|
|
252
|
+
|
|
253
|
+
***
|
|
254
|
+
|
|
255
|
+
## 8. IAM Roles Required
|
|
256
|
+
|
|
257
|
+
Per GCP documentation:
|
|
258
|
+
|
|
259
|
+
| Service | Role | JS | Go | Python | GCP Docs |
|
|
260
|
+
|---------|------|----|----|--------|----------|
|
|
261
|
+
| Cloud Trace | `roles/cloudtrace.agent` | ✅ | ✅ | ✅ | ✅ |
|
|
262
|
+
| Cloud Monitoring | `roles/monitoring.metricWriter` | ✅ | ✅ | ✅ | ✅ |
|
|
263
|
+
| Cloud Monitoring | `roles/telemetry.metricsWriter` | - | - | ✅ | ✅ |
|
|
264
|
+
| Cloud Logging | `roles/logging.logWriter` | ✅ | - | - | ✅ |
|
|
265
|
+
|
|
266
|
+
***
|
|
267
|
+
|
|
268
|
+
## 9. Telemetry Dispatch Logic Comparison
|
|
269
|
+
|
|
270
|
+
| Condition | JS | Go | Python | Status |
|
|
271
|
+
|-----------|----|----|--------|--------|
|
|
272
|
+
| All genkit spans → paths.tick() | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
273
|
+
| isRoot → features.tick() | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
274
|
+
| isRoot → set rootState | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
275
|
+
| action + model (non-root) → generate.tick() | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
276
|
+
| action/flow/flowStep/util (non-root) → action.tick() | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
277
|
+
| userEngagement → engagement.tick() | ✅ | ✅ | ✅ | ✅ PARITY |
|
|
278
|
+
|
|
279
|
+
***
|
|
280
|
+
|
|
281
|
+
## 10. Issues Found and Recommendations
|
|
282
|
+
|
|
283
|
+
### High Priority
|
|
284
|
+
|
|
285
|
+
1. **~~Python: Model name truncation too short~~** ✅ FIXED
|
|
286
|
+
* \~~Current: 256 chars~~
|
|
287
|
+
* \~~Should be: 1024 chars (matching JS/Go)~~
|
|
288
|
+
* \~~File: `generate.py`~~
|
|
289
|
+
* **Status:** Fixed - now uses 1024 chars for modelName dimension
|
|
290
|
+
|
|
291
|
+
### Medium Priority
|
|
292
|
+
|
|
293
|
+
2. **JS: Missing video/audio metrics**
|
|
294
|
+
* Missing: `input/videos`, `input/audio`, `output/videos`, `output/audio`
|
|
295
|
+
* Go and Python have these metrics
|
|
296
|
+
|
|
297
|
+
3. **JS: Missing env var project ID resolution**
|
|
298
|
+
* Should add: `FIREBASE_PROJECT_ID`, `GOOGLE_CLOUD_PROJECT`, `GCLOUD_PROJECT`
|
|
299
|
+
|
|
300
|
+
### Low Priority (Acceptable Variations)
|
|
301
|
+
|
|
302
|
+
4. **Go: Log message prefix**
|
|
303
|
+
* Go adds `[genkit]` prefix to all logs
|
|
304
|
+
* Acceptable for filtering purposes
|
|
305
|
+
|
|
306
|
+
5. **Python: Positive flag for I/O logging**
|
|
307
|
+
* Python: `log_input_and_output=True` enables logging
|
|
308
|
+
* JS/Go: `disableLoggingInputAndOutput=False` enables logging
|
|
309
|
+
* Both achieve same result, Python's is more intuitive
|
|
310
|
+
|
|
311
|
+
***
|
|
312
|
+
|
|
313
|
+
## 11. GCP Documentation References
|
|
314
|
+
|
|
315
|
+
* Cloud Trace Overview: https://cloud.google.com/trace/docs
|
|
316
|
+
* Cloud Trace IAM: https://cloud.google.com/trace/docs/iam
|
|
317
|
+
* Cloud Monitoring Overview: https://cloud.google.com/monitoring/docs
|
|
318
|
+
* Cloud Monitoring Quotas: https://cloud.google.com/monitoring/quotas
|
|
319
|
+
* Cloud Logging Structured: https://cloud.google.com/logging/docs/structured-logging
|
|
320
|
+
* Log-Trace Correlation: https://cloud.google.com/trace/docs/trace-log-integration
|
|
321
|
+
* Metric Naming: https://cloud.google.com/monitoring/api/v3/naming-conventions
|
|
322
|
+
* Custom Metrics: https://cloud.google.com/monitoring/custom-metrics
|
|
323
|
+
* OpenTelemetry GCP: https://google-cloud-opentelemetry.readthedocs.io/
|
|
324
|
+
|
|
325
|
+
***
|
|
326
|
+
|
|
327
|
+
## 12. Verification Checklist
|
|
328
|
+
|
|
329
|
+
* \[x] All metric names match across implementations
|
|
330
|
+
* \[x] All metric types (Counter/Histogram) match
|
|
331
|
+
* \[x] Span attributes read/written match
|
|
332
|
+
* \[x] Log correlation attributes follow GCP spec
|
|
333
|
+
* \[x] Minimum metric interval enforced (5000ms)
|
|
334
|
+
* \[x] PII redaction implemented
|
|
335
|
+
* \[x] Error span marking (`/http/status_code: 599`)
|
|
336
|
+
* \[x] Label normalization (`:` → `/`)
|
|
337
|
+
* \[x] Start time adjustment for DELTA→CUMULATIVE
|
|
338
|
+
* \[x] Model name truncation (Python fixed to 1024 chars)
|
|
339
|
+
* \[ ] Video/audio metrics (JS needs addition - tracked separately)
|
|
340
|
+
|
|
341
|
+
***
|
|
342
|
+
|
|
343
|
+
*Last updated: 2026-01-28*
|
|
344
|
+
*Analyzed versions: JS (latest), Go (latest), Python (latest)*
|
|
@@ -1,10 +1,15 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: genkit-plugin-google-cloud
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.5.0
|
|
4
4
|
Summary: Genkit Google Cloud Plugin
|
|
5
|
+
Project-URL: Bug Tracker, https://github.com/firebase/genkit/issues
|
|
6
|
+
Project-URL: Documentation, https://firebase.google.com/docs/genkit
|
|
7
|
+
Project-URL: Homepage, https://github.com/firebase/genkit
|
|
8
|
+
Project-URL: Repository, https://github.com/firebase/genkit/tree/main/py
|
|
5
9
|
Author: Google
|
|
6
|
-
License: Apache-2.0
|
|
10
|
+
License-Expression: Apache-2.0
|
|
7
11
|
License-File: LICENSE
|
|
12
|
+
Keywords: ai,artificial-intelligence,cloud-trace,gcp,generative-ai,genkit,google-cloud,llm,machine-learning,telemetry
|
|
8
13
|
Classifier: Development Status :: 3 - Alpha
|
|
9
14
|
Classifier: Environment :: Console
|
|
10
15
|
Classifier: Environment :: Web Environment
|
|
@@ -17,10 +22,13 @@ Classifier: Programming Language :: Python :: 3.10
|
|
|
17
22
|
Classifier: Programming Language :: Python :: 3.11
|
|
18
23
|
Classifier: Programming Language :: Python :: 3.12
|
|
19
24
|
Classifier: Programming Language :: Python :: 3.13
|
|
25
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
20
26
|
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
21
27
|
Classifier: Topic :: Software Development :: Libraries
|
|
28
|
+
Classifier: Typing :: Typed
|
|
22
29
|
Requires-Python: >=3.10
|
|
23
30
|
Requires-Dist: genkit
|
|
31
|
+
Requires-Dist: opentelemetry-exporter-gcp-monitoring>=1.9.0
|
|
24
32
|
Requires-Dist: opentelemetry-exporter-gcp-trace>=1.9.0
|
|
25
33
|
Requires-Dist: strenum>=0.4.15; python_version < '3.11'
|
|
26
34
|
Description-Content-Type: text/markdown
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Copyright 2025 Google LLC
|
|
2
|
+
#
|
|
3
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
4
|
+
# you may not use this file except in compliance with the License.
|
|
5
|
+
# You may obtain a copy of the License at
|
|
6
|
+
#
|
|
7
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
8
|
+
#
|
|
9
|
+
# Unless required by applicable law or agreed to in writing, software
|
|
10
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
11
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
12
|
+
# See the License for the specific language governing permissions and
|
|
13
|
+
# limitations under the License.
|
|
14
|
+
#
|
|
15
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
16
|
+
|
|
17
|
+
[project]
|
|
18
|
+
authors = [{ name = "Google" }]
|
|
19
|
+
classifiers = [
|
|
20
|
+
"Development Status :: 3 - Alpha",
|
|
21
|
+
"Environment :: Console",
|
|
22
|
+
"Environment :: Web Environment",
|
|
23
|
+
"Intended Audience :: Developers",
|
|
24
|
+
"Operating System :: OS Independent",
|
|
25
|
+
"Programming Language :: Python",
|
|
26
|
+
"Programming Language :: Python :: 3 :: Only",
|
|
27
|
+
"Programming Language :: Python :: 3.10",
|
|
28
|
+
"Programming Language :: Python :: 3.11",
|
|
29
|
+
"Programming Language :: Python :: 3.12",
|
|
30
|
+
"Programming Language :: Python :: 3.13",
|
|
31
|
+
"Programming Language :: Python :: 3.14",
|
|
32
|
+
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
|
33
|
+
"Topic :: Software Development :: Libraries",
|
|
34
|
+
"Typing :: Typed",
|
|
35
|
+
"License :: OSI Approved :: Apache Software License",
|
|
36
|
+
]
|
|
37
|
+
dependencies = [
|
|
38
|
+
"genkit",
|
|
39
|
+
"opentelemetry-exporter-gcp-trace>=1.9.0",
|
|
40
|
+
"opentelemetry-exporter-gcp-monitoring>=1.9.0",
|
|
41
|
+
"strenum>=0.4.15; python_version < '3.11'",
|
|
42
|
+
]
|
|
43
|
+
description = "Genkit Google Cloud Plugin"
|
|
44
|
+
keywords = [
|
|
45
|
+
"genkit",
|
|
46
|
+
"ai",
|
|
47
|
+
"llm",
|
|
48
|
+
"machine-learning",
|
|
49
|
+
"artificial-intelligence",
|
|
50
|
+
"generative-ai",
|
|
51
|
+
"google-cloud",
|
|
52
|
+
"gcp",
|
|
53
|
+
"cloud-trace",
|
|
54
|
+
"telemetry",
|
|
55
|
+
]
|
|
56
|
+
license = "Apache-2.0"
|
|
57
|
+
name = "genkit-plugin-google-cloud"
|
|
58
|
+
readme = "README.md"
|
|
59
|
+
requires-python = ">=3.10"
|
|
60
|
+
version = "0.5.0"
|
|
61
|
+
|
|
62
|
+
[project.urls]
|
|
63
|
+
"Bug Tracker" = "https://github.com/firebase/genkit/issues"
|
|
64
|
+
"Documentation" = "https://firebase.google.com/docs/genkit"
|
|
65
|
+
"Homepage" = "https://github.com/firebase/genkit"
|
|
66
|
+
"Repository" = "https://github.com/firebase/genkit/tree/main/py"
|
|
67
|
+
|
|
68
|
+
[build-system]
|
|
69
|
+
build-backend = "hatchling.build"
|
|
70
|
+
requires = ["hatchling"]
|
|
71
|
+
|
|
72
|
+
[tool.hatch.build.targets.wheel]
|
|
73
|
+
only-include = ["src/genkit/plugins/google_cloud"]
|
|
74
|
+
sources = ["src"]
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
# Copyright 2025 Google LLC
|
|
2
|
+
#
|
|
3
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
4
|
+
# you may not use this file except in compliance with the License.
|
|
5
|
+
# You may obtain a copy of the License at
|
|
6
|
+
#
|
|
7
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
|
8
|
+
#
|
|
9
|
+
# Unless required by applicable law or agreed to in writing, software
|
|
10
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
11
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
12
|
+
# See the License for the specific language governing permissions and
|
|
13
|
+
# limitations under the License.
|
|
14
|
+
#
|
|
15
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
"""Google Cloud Plugin for Genkit.
|
|
19
|
+
|
|
20
|
+
This plugin provides Google Cloud observability integration for Genkit,
|
|
21
|
+
enabling telemetry export to Cloud Trace and Cloud Monitoring.
|
|
22
|
+
|
|
23
|
+
Key Concepts (ELI5)::
|
|
24
|
+
|
|
25
|
+
┌─────────────────────┬────────────────────────────────────────────────────┐
|
|
26
|
+
│ Concept │ ELI5 Explanation │
|
|
27
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
28
|
+
│ Telemetry │ Data about how your app is running. Like a │
|
|
29
|
+
│ │ fitness tracker for your code. │
|
|
30
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
31
|
+
│ Cloud Trace │ Shows the path requests take through your app. │
|
|
32
|
+
│ │ Like GPS tracking for your API calls. │
|
|
33
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
34
|
+
│ Cloud Monitoring │ Graphs and alerts for your app's health. │
|
|
35
|
+
│ │ Like a heart rate monitor dashboard. │
|
|
36
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
37
|
+
│ Span │ One step in a request's journey. Like one │
|
|
38
|
+
│ │ leg of a relay race. │
|
|
39
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
40
|
+
│ Trace │ All spans for one request connected together. │
|
|
41
|
+
│ │ The complete story of one API call. │
|
|
42
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
43
|
+
│ Metrics │ Numbers that describe your app (requests/sec, │
|
|
44
|
+
│ │ error rate, latency). Like a report card. │
|
|
45
|
+
├─────────────────────┼────────────────────────────────────────────────────┤
|
|
46
|
+
│ PII Redaction │ Hiding sensitive data in traces. Like blurring │
|
|
47
|
+
│ │ faces in photos before sharing. │
|
|
48
|
+
└─────────────────────┴────────────────────────────────────────────────────┘
|
|
49
|
+
|
|
50
|
+
Data Flow::
|
|
51
|
+
|
|
52
|
+
┌─────────────────────────────────────────────────────────────────────────┐
|
|
53
|
+
│ HOW TELEMETRY FLOWS TO GOOGLE CLOUD │
|
|
54
|
+
│ │
|
|
55
|
+
│ Your Genkit App │
|
|
56
|
+
│ │ │
|
|
57
|
+
│ │ (1) App runs flows, calls models, uses tools │
|
|
58
|
+
│ ▼ │
|
|
59
|
+
│ ┌─────────────────┐ │
|
|
60
|
+
│ │ OpenTelemetry │ Automatically creates spans for each │
|
|
61
|
+
│ │ SDK │ operation (you don't write this code!) │
|
|
62
|
+
│ └────────┬────────┘ │
|
|
63
|
+
│ │ │
|
|
64
|
+
│ │ (2) Spans collected and processed │
|
|
65
|
+
│ ▼ │
|
|
66
|
+
│ ┌─────────────────┐ │
|
|
67
|
+
│ │ GCP Exporters │ • Redact PII (input/output) │
|
|
68
|
+
│ │ │ • Add error markers │
|
|
69
|
+
│ │ │ • Batch for efficiency │
|
|
70
|
+
│ └────────┬────────┘ │
|
|
71
|
+
│ │ │
|
|
72
|
+
│ │ (3) HTTPS to Google Cloud │
|
|
73
|
+
│ ▼ │
|
|
74
|
+
│ ════════════════════════════════════════════════════ │
|
|
75
|
+
│ │ Internet │
|
|
76
|
+
│ ▼ │
|
|
77
|
+
│ ┌─────────────────────────────────────────────────────┐ │
|
|
78
|
+
│ │ Google Cloud Console │ │
|
|
79
|
+
│ │ ┌──────────────┐ ┌──────────────┐ │ │
|
|
80
|
+
│ │ │ Cloud Trace │ │ Cloud │ │ │
|
|
81
|
+
│ │ │ (waterfall │ │ Monitoring │ │ │
|
|
82
|
+
│ │ │ diagrams) │ │ (dashboards) │ │ │
|
|
83
|
+
│ │ └──────────────┘ └──────────────┘ │ │
|
|
84
|
+
│ └─────────────────────────────────────────────────────┘ │
|
|
85
|
+
└─────────────────────────────────────────────────────────────────────────┘
|
|
86
|
+
|
|
87
|
+
Architecture Overview::
|
|
88
|
+
|
|
89
|
+
┌─────────────────────────────────────────────────────────────────────────┐
|
|
90
|
+
│ Google Cloud Plugin │
|
|
91
|
+
├─────────────────────────────────────────────────────────────────────────┤
|
|
92
|
+
│ Plugin Entry Point (__init__.py) │
|
|
93
|
+
│ └── add_gcp_telemetry() - Enable Cloud Trace/Monitoring export │
|
|
94
|
+
├─────────────────────────────────────────────────────────────────────────┤
|
|
95
|
+
│ telemetry/__init__.py - Telemetry Module │
|
|
96
|
+
│ └── Re-exports from submodules │
|
|
97
|
+
├─────────────────────────────────────────────────────────────────────────┤
|
|
98
|
+
│ telemetry/tracing.py - Distributed Tracing │
|
|
99
|
+
│ ├── Cloud Trace exporter configuration │
|
|
100
|
+
│ └── OpenTelemetry integration │
|
|
101
|
+
├─────────────────────────────────────────────────────────────────────────┤
|
|
102
|
+
│ telemetry/metrics.py - Metrics Collection │
|
|
103
|
+
│ ├── Cloud Monitoring exporter │
|
|
104
|
+
│ └── Custom Genkit metrics │
|
|
105
|
+
├─────────────────────────────────────────────────────────────────────────┤
|
|
106
|
+
│ telemetry/action.py - Action Instrumentation │
|
|
107
|
+
│ └── Automatic span creation for Genkit actions │
|
|
108
|
+
└─────────────────────────────────────────────────────────────────────────┘
|
|
109
|
+
|
|
110
|
+
┌─────────────────────────────────────────────────────────────────────────┐
|
|
111
|
+
│ Telemetry Data Flow │
|
|
112
|
+
│ │
|
|
113
|
+
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────┐ │
|
|
114
|
+
│ │ Genkit App │───►│ OpenTelemetry│───►│ Google Cloud │ │
|
|
115
|
+
│ │ (actions, │ │ SDK │ │ (Trace, Monitoring) │ │
|
|
116
|
+
│ │ flows) │ └──────────────┘ └──────────────────────┘ │
|
|
117
|
+
│ └──────────────┘ │
|
|
118
|
+
└─────────────────────────────────────────────────────────────────────────┘
|
|
119
|
+
|
|
120
|
+
Example:
|
|
121
|
+
```python
|
|
122
|
+
from genkit.plugins.google_cloud import add_gcp_telemetry
|
|
123
|
+
|
|
124
|
+
# Enable telemetry export to Google Cloud
|
|
125
|
+
add_gcp_telemetry()
|
|
126
|
+
|
|
127
|
+
# Traces and metrics are now exported to:
|
|
128
|
+
# - Cloud Trace (distributed tracing)
|
|
129
|
+
# - Cloud Monitoring (metrics)
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
Caveats:
|
|
133
|
+
- Requires Google Cloud credentials (ADC or explicit)
|
|
134
|
+
- Telemetry is disabled by default in development mode (GENKIT_ENV=dev)
|
|
135
|
+
- Requires opentelemetry and google-cloud-* packages
|
|
136
|
+
|
|
137
|
+
See Also:
|
|
138
|
+
- Cloud Trace: https://cloud.google.com/trace
|
|
139
|
+
- Cloud Monitoring: https://cloud.google.com/monitoring
|
|
140
|
+
- Genkit documentation: https://genkit.dev/
|
|
141
|
+
"""
|
|
142
|
+
|
|
143
|
+
from .telemetry import add_gcp_telemetry
|
|
144
|
+
|
|
145
|
+
|
|
146
|
+
def package_name() -> str:
|
|
147
|
+
"""Get the package name for the Google Cloud plugin.
|
|
148
|
+
|
|
149
|
+
Returns:
|
|
150
|
+
The fully qualified package name as a string.
|
|
151
|
+
"""
|
|
152
|
+
return 'genkit.plugins.google_cloud'
|
|
153
|
+
|
|
154
|
+
|
|
155
|
+
__all__ = ['add_gcp_telemetry', 'package_name']
|