bridgebench 3.1.0-alpha.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CITATION.cff +15 -0
- package/LICENSE +21 -0
- package/README.md +249 -0
- package/dist/chunk-4TWPCPRP.cjs +1097 -0
- package/dist/chunk-4TWPCPRP.cjs.map +1 -0
- package/dist/chunk-7YCJSOK7.cjs +398 -0
- package/dist/chunk-7YCJSOK7.cjs.map +1 -0
- package/dist/chunk-CIXITJW6.cjs +249 -0
- package/dist/chunk-CIXITJW6.cjs.map +1 -0
- package/dist/chunk-EQHRUV2I.js +1466 -0
- package/dist/chunk-EQHRUV2I.js.map +1 -0
- package/dist/chunk-JTVNKSMO.js +1096 -0
- package/dist/chunk-JTVNKSMO.js.map +1 -0
- package/dist/chunk-LFKEV2YL.js +398 -0
- package/dist/chunk-LFKEV2YL.js.map +1 -0
- package/dist/chunk-NJTYVNP4.cjs +1467 -0
- package/dist/chunk-NJTYVNP4.cjs.map +1 -0
- package/dist/chunk-UECBSKTD.js +244 -0
- package/dist/chunk-UECBSKTD.js.map +1 -0
- package/dist/cli.cjs +409 -0
- package/dist/cli.cjs.map +1 -0
- package/dist/cli.d.cts +1 -0
- package/dist/cli.d.ts +1 -0
- package/dist/cli.js +408 -0
- package/dist/cli.js.map +1 -0
- package/dist/client.cjs +42 -0
- package/dist/client.cjs.map +1 -0
- package/dist/client.d.cts +93 -0
- package/dist/client.d.ts +93 -0
- package/dist/client.js +42 -0
- package/dist/client.js.map +1 -0
- package/dist/contracts/index.cjs +47 -0
- package/dist/contracts/index.cjs.map +1 -0
- package/dist/contracts/index.d.cts +14 -0
- package/dist/contracts/index.d.ts +14 -0
- package/dist/contracts/index.js +47 -0
- package/dist/contracts/index.js.map +1 -0
- package/dist/index.cjs +171 -0
- package/dist/index.cjs.map +1 -0
- package/dist/index.d.cts +214 -0
- package/dist/index.d.ts +214 -0
- package/dist/index.js +171 -0
- package/dist/index.js.map +1 -0
- package/dist/logger-CCR9Mg1c.d.cts +319 -0
- package/dist/logger-QJU7SBDz.d.ts +319 -0
- package/dist/reports-4CejmOHf.d.cts +454 -0
- package/dist/reports-s2CTnGN8.d.ts +454 -0
- package/dist/tasks-CpaCJ6JE.d.cts +151 -0
- package/dist/tasks-CpaCJ6JE.d.ts +151 -0
- package/dist/tasks.cjs +22 -0
- package/dist/tasks.cjs.map +1 -0
- package/dist/tasks.d.cts +39 -0
- package/dist/tasks.d.ts +39 -0
- package/dist/tasks.js +22 -0
- package/dist/tasks.js.map +1 -0
- package/docs/README.md +25 -0
- package/docs/glossary.md +49 -0
- package/docs/methodology.md +58 -0
- package/docs/private-packs.md +74 -0
- package/docs/replay-elo.md +79 -0
- package/docs/task-authoring.md +80 -0
- package/package.json +137 -0
- package/tasks/hallucination/public/boundary-coverage-audit.yaml +274 -0
- package/tasks/hallucination/public/boundary-migration-audit.yaml +284 -0
- package/tasks/hallucination/public/conflict-dependency-versions.yaml +324 -0
- package/tasks/hallucination/public/conflict-runbook-versions.yaml +229 -0
- package/tasks/hallucination/public/fabrication-agent-tools.yaml +224 -0
- package/tasks/hallucination/public/fabrication-api-surface.yaml +239 -0
- package/tasks/hallucination/public/fidelity-commit-attribution.yaml +304 -0
- package/tasks/hallucination/public/fidelity-config-drift.yaml +307 -0
- package/tasks/hallucination/public/missing-deploy-window.yaml +204 -0
- package/tasks/hallucination/public/missing-latency-baseline.yaml +239 -0
- package/tasks/hallucination/public/premise-quota-breach.yaml +202 -0
- package/tasks/hallucination/public/premise-rollback-cause.yaml +235 -0
- package/tasks/reasoning/public/constraint-capacity-allocation.yaml +196 -0
- package/tasks/reasoning/public/constraint-deployment-policy.yaml +203 -0
- package/tasks/reasoning/public/counterexample-authorization-rule.yaml +278 -0
- package/tasks/reasoning/public/counterexample-scheduler-starvation.yaml +290 -0
- package/tasks/reasoning/public/root-cache-tenant-leak.yaml +225 -0
- package/tasks/reasoning/public/root-event-ordering.yaml +184 -0
- package/tasks/reasoning/public/stateful-lease-handoff.yaml +213 -0
- package/tasks/reasoning/public/stateful-retry-budget.yaml +222 -0
- package/tasks/reasoning/public/synthesis-api-contract.yaml +214 -0
- package/tasks/reasoning/public/synthesis-permission-migration.yaml +190 -0
- package/tasks/reasoning/public/uncertainty-conflicting-telemetry.yaml +242 -0
- package/tasks/reasoning/public/uncertainty-incomplete-incident.yaml +223 -0
|
@@ -0,0 +1,307 @@
|
|
|
1
|
+
id: fidelity-config-drift
|
|
2
|
+
version: 1.0.0
|
|
3
|
+
category: hallucination
|
|
4
|
+
cluster: citation-fidelity
|
|
5
|
+
difficulty: expert
|
|
6
|
+
title: Pricing-Engine Config Drift Extraction
|
|
7
|
+
summary: Extract exact keys, values, versions, and timestamps from four near-duplicate environment configs and a changelog where singular and plural key names and confusable version numbers differ by a single character.
|
|
8
|
+
prompt: |
|
|
9
|
+
Extract the following facts EXACTLY from the artifacts.
|
|
10
|
+
|
|
11
|
+
Quote key names, values, version strings, and timestamps character-for-character
|
|
12
|
+
from the named artifact. Rules:
|
|
13
|
+
- Do not normalize singular vs plural key names (maxConnection vs maxConnections
|
|
14
|
+
are different keys and both exist in this system).
|
|
15
|
+
- Do not round or restate numeric values.
|
|
16
|
+
- Do not blend environments (dev, staging, prod, dr) or versions.
|
|
17
|
+
- When asked to quote a note, reproduce the sentence word-for-word.
|
|
18
|
+
|
|
19
|
+
Deliverables:
|
|
20
|
+
1. State the exact key name and its value for the database connection-pool
|
|
21
|
+
maximum in the PROD config.
|
|
22
|
+
2. State the exact key name that configures the TLS cipher list in the DR config.
|
|
23
|
+
3. Give the exact version string in which the retry.backoff.maxMs default was
|
|
24
|
+
CHANGED (not the version that introduced it), and state the old and new values.
|
|
25
|
+
4. Quote verbatim the deprecation note for the CACHE time-to-live key from the
|
|
26
|
+
changelog.
|
|
27
|
+
5. State the exact last-applied timestamp and the version tag recorded in the
|
|
28
|
+
PROD config header.
|
|
29
|
+
6. State the exact key name and its value for the database connection-pool
|
|
30
|
+
maximum in the DR config.
|
|
31
|
+
artifacts:
|
|
32
|
+
- id: config-dev
|
|
33
|
+
type: config
|
|
34
|
+
label: pricing-engine dev config
|
|
35
|
+
content: |
|
|
36
|
+
# service: pricing-engine
|
|
37
|
+
# env: dev
|
|
38
|
+
# last-applied: 2026-05-02T09:15:00Z version=v1.15.0
|
|
39
|
+
server.host: 0.0.0.0
|
|
40
|
+
server.port: 8080
|
|
41
|
+
log.level: debug
|
|
42
|
+
metrics.enabled: true
|
|
43
|
+
http.maxBodyBytes: 1048576
|
|
44
|
+
http.keepAliveMs: 5000
|
|
45
|
+
db.host: pricing-db-dev.internal
|
|
46
|
+
db.readReplica.host: pricing-db-dev-ro.internal
|
|
47
|
+
db.pool.maxConnections: 20
|
|
48
|
+
db.pool.acquireTimeoutMs: 5000
|
|
49
|
+
db.statementTimeoutMs: 10000
|
|
50
|
+
cache.provider: redis
|
|
51
|
+
cache.ttlSeconds: 60
|
|
52
|
+
cache.maxEntries: 10000
|
|
53
|
+
cache.evictionPolicy: allkeys-lru
|
|
54
|
+
rateLimiter.enabled: true
|
|
55
|
+
rateLimiter.requestsPerMinute: 1000
|
|
56
|
+
rateLimiter.burst: 100
|
|
57
|
+
retry.enabled: true
|
|
58
|
+
retry.maxAttempts: 4
|
|
59
|
+
retry.backoff.maxMs: 5000
|
|
60
|
+
circuitBreaker.failureThreshold: 10
|
|
61
|
+
circuitBreaker.resetTimeoutMs: 15000
|
|
62
|
+
observability.tracingSampleRate: 1.0
|
|
63
|
+
observability.exporter: otlp
|
|
64
|
+
tls.enabled: true
|
|
65
|
+
tls.minVersion: TLSv1.2
|
|
66
|
+
tls.cipherSuites: TLS_AES_128_GCM_SHA256
|
|
67
|
+
security.hstsMaxAgeSeconds: 0
|
|
68
|
+
pricing.roundingMode: HALF_EVEN
|
|
69
|
+
pricing.currencyDefault: USD
|
|
70
|
+
queue.name: pricing-dev-events
|
|
71
|
+
shutdown.gracePeriodMs: 5000
|
|
72
|
+
featureFlags.newPricing: false
|
|
73
|
+
- id: config-staging
|
|
74
|
+
type: config
|
|
75
|
+
label: pricing-engine staging config
|
|
76
|
+
content: |
|
|
77
|
+
# service: pricing-engine
|
|
78
|
+
# env: staging
|
|
79
|
+
# last-applied: 2026-05-28T18:40:11Z version=v1.15.0
|
|
80
|
+
server.host: 0.0.0.0
|
|
81
|
+
server.port: 8080
|
|
82
|
+
log.level: info
|
|
83
|
+
metrics.enabled: true
|
|
84
|
+
http.maxBodyBytes: 2097152
|
|
85
|
+
http.keepAliveMs: 5000
|
|
86
|
+
db.host: pricing-db-staging.internal
|
|
87
|
+
db.readReplica.host: pricing-db-staging-ro.internal
|
|
88
|
+
db.pool.maxConnections: 50
|
|
89
|
+
db.pool.acquireTimeoutMs: 4000
|
|
90
|
+
db.statementTimeoutMs: 8000
|
|
91
|
+
cache.provider: redis
|
|
92
|
+
cache.ttlSeconds: 300
|
|
93
|
+
cache.maxEntries: 50000
|
|
94
|
+
cache.evictionPolicy: allkeys-lru
|
|
95
|
+
rateLimiter.enabled: true
|
|
96
|
+
rateLimiter.requestsPerMinute: 6000
|
|
97
|
+
rateLimiter.burst: 600
|
|
98
|
+
retry.enabled: true
|
|
99
|
+
retry.maxAttempts: 4
|
|
100
|
+
retry.backoff.maxMs: 5000
|
|
101
|
+
circuitBreaker.failureThreshold: 15
|
|
102
|
+
circuitBreaker.resetTimeoutMs: 20000
|
|
103
|
+
observability.tracingSampleRate: 0.5
|
|
104
|
+
observability.exporter: otlp
|
|
105
|
+
tls.enabled: true
|
|
106
|
+
tls.minVersion: TLSv1.2
|
|
107
|
+
tls.cipherSuites: TLS_AES_128_GCM_SHA256
|
|
108
|
+
security.hstsMaxAgeSeconds: 3600
|
|
109
|
+
pricing.roundingMode: HALF_EVEN
|
|
110
|
+
pricing.currencyDefault: USD
|
|
111
|
+
queue.name: pricing-staging-events
|
|
112
|
+
shutdown.gracePeriodMs: 10000
|
|
113
|
+
featureFlags.newPricing: true
|
|
114
|
+
- id: config-prod
|
|
115
|
+
type: config
|
|
116
|
+
label: pricing-engine prod config
|
|
117
|
+
content: |
|
|
118
|
+
# service: pricing-engine
|
|
119
|
+
# env: prod
|
|
120
|
+
# last-applied: 2026-06-10T14:22:07Z version=v2.0.1
|
|
121
|
+
server.host: 0.0.0.0
|
|
122
|
+
server.port: 443
|
|
123
|
+
log.level: warn
|
|
124
|
+
metrics.enabled: true
|
|
125
|
+
http.maxBodyBytes: 5242880
|
|
126
|
+
http.keepAliveMs: 15000
|
|
127
|
+
db.host: pricing-db-prod.internal
|
|
128
|
+
db.readReplica.host: pricing-db-prod-ro.internal
|
|
129
|
+
db.pool.maxConnections: 200
|
|
130
|
+
db.pool.acquireTimeoutMs: 3000
|
|
131
|
+
db.statementTimeoutMs: 5000
|
|
132
|
+
cache.provider: redis
|
|
133
|
+
cache.ttlSeconds: 900
|
|
134
|
+
cache.maxEntries: 500000
|
|
135
|
+
cache.evictionPolicy: allkeys-lru
|
|
136
|
+
rateLimiter.enabled: true
|
|
137
|
+
rateLimiter.requestsPerMinute: 30000
|
|
138
|
+
rateLimiter.burst: 3000
|
|
139
|
+
retry.enabled: true
|
|
140
|
+
retry.maxAttempts: 4
|
|
141
|
+
retry.backoff.maxMs: 5000
|
|
142
|
+
circuitBreaker.failureThreshold: 20
|
|
143
|
+
circuitBreaker.resetTimeoutMs: 30000
|
|
144
|
+
observability.tracingSampleRate: 0.1
|
|
145
|
+
observability.exporter: otlp
|
|
146
|
+
tls.enabled: true
|
|
147
|
+
tls.minVersion: TLSv1.3
|
|
148
|
+
tls.cipherSuites: TLS_AES_256_GCM_SHA384
|
|
149
|
+
security.hstsMaxAgeSeconds: 31536000
|
|
150
|
+
pricing.roundingMode: HALF_EVEN
|
|
151
|
+
pricing.currencyDefault: USD
|
|
152
|
+
queue.name: pricing-prod-events
|
|
153
|
+
shutdown.gracePeriodMs: 30000
|
|
154
|
+
featureFlags.newPricing: true
|
|
155
|
+
- id: config-dr
|
|
156
|
+
type: config
|
|
157
|
+
label: pricing-engine dr config
|
|
158
|
+
content: |
|
|
159
|
+
# service: pricing-engine
|
|
160
|
+
# env: dr
|
|
161
|
+
# last-applied: 2026-06-09T22:05:44Z version=v2.0.0
|
|
162
|
+
server.host: 0.0.0.0
|
|
163
|
+
server.port: 443
|
|
164
|
+
log.level: warn
|
|
165
|
+
metrics.enabled: true
|
|
166
|
+
http.maxBodyBytes: 5242880
|
|
167
|
+
http.keepAliveMs: 15000
|
|
168
|
+
db.host: pricing-db-dr.internal
|
|
169
|
+
db.readReplica.host: pricing-db-dr-ro.internal
|
|
170
|
+
db.pool.maxConnection: 120
|
|
171
|
+
db.pool.acquireTimeoutMs: 3000
|
|
172
|
+
db.statementTimeoutMs: 5000
|
|
173
|
+
cache.provider: redis
|
|
174
|
+
cache.ttlSecond: 900
|
|
175
|
+
cache.maxEntries: 300000
|
|
176
|
+
cache.evictionPolicy: allkeys-lru
|
|
177
|
+
rateLimiter.enabled: true
|
|
178
|
+
rateLimiter.requestPerMinute: 15000
|
|
179
|
+
rateLimiter.burst: 1500
|
|
180
|
+
retry.enabled: true
|
|
181
|
+
retry.maxAttempts: 4
|
|
182
|
+
retry.backoff.maxMs: 5000
|
|
183
|
+
circuitBreaker.failureThreshold: 20
|
|
184
|
+
circuitBreaker.resetTimeoutMs: 30000
|
|
185
|
+
observability.tracingSampleRate: 0.1
|
|
186
|
+
observability.exporter: otlp
|
|
187
|
+
tls.enabled: true
|
|
188
|
+
tls.minVersion: TLSv1.2
|
|
189
|
+
tls.cipherSuite: TLS_AES_128_GCM_SHA256
|
|
190
|
+
security.hstsMaxAgeSeconds: 31536000
|
|
191
|
+
pricing.roundingMode: HALF_EVEN
|
|
192
|
+
pricing.currencyDefault: USD
|
|
193
|
+
queue.name: pricing-dr-events
|
|
194
|
+
shutdown.gracePeriodMs: 30000
|
|
195
|
+
featureFlags.newPricing: true
|
|
196
|
+
- id: config-changelog
|
|
197
|
+
type: log
|
|
198
|
+
label: pricing-engine configuration changelog
|
|
199
|
+
content: |
|
|
200
|
+
pricing-engine configuration changelog (most recent last)
|
|
201
|
+
|
|
202
|
+
v1.14.0 (2026-01-02): Initial externalized configuration for pricing-engine.
|
|
203
|
+
v1.14.2 (2026-01-14): Introduced retry.backoff.maxMs with a default of 2000.
|
|
204
|
+
v1.14.5 (2026-02-10): Added circuitBreaker.failureThreshold and circuitBreaker.resetTimeoutMs.
|
|
205
|
+
v1.14.12 (2026-03-02): Changed retry.backoff.maxMs default from 2000 to 5000.
|
|
206
|
+
v1.14.20 (2026-04-19): Deprecated cache.ttlSecond.
|
|
207
|
+
Note: "cache.ttlSecond is deprecated; use cache.ttlSeconds. The singular key will be removed in v2.0.0."
|
|
208
|
+
v1.15.0 (2026-05-05): Deprecated rateLimiter.requestPerMinute.
|
|
209
|
+
Note: "rateLimiter.requestPerMinute is deprecated; use rateLimiter.requestsPerMinute. The singular key will be removed in v2.1.0."
|
|
210
|
+
v1.15.2 (2026-05-20): Added observability.tracingSampleRate per environment.
|
|
211
|
+
v2.0.0 (2026-06-01): Removed cache.ttlSecond from dev, staging, and prod. DR retains the
|
|
212
|
+
legacy singular keys pending a failover-tool update.
|
|
213
|
+
v2.0.1 (2026-06-10): Raised prod db.pool.maxConnections from 150 to 200.
|
|
214
|
+
v2.0.2 (planned, not yet applied to any environment): migrate dr off the legacy singular keys.
|
|
215
|
+
|
|
216
|
+
Version quick reference (to avoid confusing near-identical version strings)
|
|
217
|
+
- v1.14.2 != v1.14.12: v1.14.2 INTRODUCED retry.backoff.maxMs at 2000; v1.14.12 CHANGED
|
|
218
|
+
it to 5000. These are different versions two months apart.
|
|
219
|
+
- v1.14.20 is a third, later version and is unrelated to the retry default; it deprecated a
|
|
220
|
+
cache key.
|
|
221
|
+
- The current retry.backoff.maxMs default in every environment is 5000 (set by v1.14.12).
|
|
222
|
+
- The current prod db.pool.maxConnections is 200 (set by v2.0.1); the pre-v2.0.1 value 150
|
|
223
|
+
is historical and must not be reported as current.
|
|
224
|
+
|
|
225
|
+
Environment version tags (from each config header)
|
|
226
|
+
- dev: v1.15.0 (last-applied 2026-05-02T09:15:00Z)
|
|
227
|
+
- staging: v1.15.0 (last-applied 2026-05-28T18:40:11Z)
|
|
228
|
+
- prod: v2.0.1 (last-applied 2026-06-10T14:22:07Z)
|
|
229
|
+
- dr: v2.0.0 (last-applied 2026-06-09T22:05:44Z)
|
|
230
|
+
- id: key-glossary
|
|
231
|
+
type: note
|
|
232
|
+
label: Singular vs plural key reference
|
|
233
|
+
content: |
|
|
234
|
+
Configuration key reference (singular and plural forms both exist in this system)
|
|
235
|
+
|
|
236
|
+
- db.pool.maxConnections (plural): canonical key used in dev, staging, prod.
|
|
237
|
+
- db.pool.maxConnection (singular): legacy key, present in dr ONLY.
|
|
238
|
+
- cache.ttlSeconds (plural): canonical, used in dev, staging, prod.
|
|
239
|
+
cache.ttlSecond (singular): legacy key, present in dr ONLY.
|
|
240
|
+
- rateLimiter.requestsPerMinute (plural): canonical, used in dev, staging, prod.
|
|
241
|
+
rateLimiter.requestPerMinute (singular): legacy key, present in dr ONLY.
|
|
242
|
+
- tls.cipherSuites (plural): used in dev, staging, prod.
|
|
243
|
+
tls.cipherSuite (singular): used in dr ONLY.
|
|
244
|
+
|
|
245
|
+
Rules
|
|
246
|
+
- These singular and plural forms are DISTINCT keys, not aliases.
|
|
247
|
+
- Do not assume the plural key exists in dr, or that the singular key exists outside dr.
|
|
248
|
+
- Only the four pairs above have singular/plural variants; all other keys share one name
|
|
249
|
+
across environments (only their values differ).
|
|
250
|
+
|
|
251
|
+
Value drift summary for the connection-pool maximum (the most-confused value)
|
|
252
|
+
- dev uses db.pool.maxConnections = 20.
|
|
253
|
+
- staging uses db.pool.maxConnections = 50.
|
|
254
|
+
- prod uses db.pool.maxConnections = 200 (raised from 150 in v2.0.1).
|
|
255
|
+
- dr uses db.pool.maxConnection = 120 (note the singular key name in dr).
|
|
256
|
+
- There is no single "the pool max"; it is per-environment, and the key name differs in dr.
|
|
257
|
+
|
|
258
|
+
Value drift summary for the TLS cipher list
|
|
259
|
+
- dev and staging use tls.cipherSuites = TLS_AES_128_GCM_SHA256.
|
|
260
|
+
- prod uses tls.cipherSuites = TLS_AES_256_GCM_SHA384.
|
|
261
|
+
- dr uses tls.cipherSuite = TLS_AES_128_GCM_SHA256 (singular key name in dr).
|
|
262
|
+
- id: env-diff
|
|
263
|
+
type: table
|
|
264
|
+
label: Selected key comparison across environments
|
|
265
|
+
content: |
|
|
266
|
+
Config key comparison (selected keys). Authoritative source is each env's own config file;
|
|
267
|
+
this table is only a summary and uses each env's actual key name.
|
|
268
|
+
|
|
269
|
+
db.pool connection maximum
|
|
270
|
+
dev db.pool.maxConnections = 20
|
|
271
|
+
staging db.pool.maxConnections = 50
|
|
272
|
+
prod db.pool.maxConnections = 200
|
|
273
|
+
dr db.pool.maxConnection = 120
|
|
274
|
+
|
|
275
|
+
cache time-to-live
|
|
276
|
+
dev cache.ttlSeconds = 60
|
|
277
|
+
staging cache.ttlSeconds = 300
|
|
278
|
+
prod cache.ttlSeconds = 900
|
|
279
|
+
dr cache.ttlSecond = 900
|
|
280
|
+
|
|
281
|
+
rate limiter (requests per minute)
|
|
282
|
+
dev rateLimiter.requestsPerMinute = 1000
|
|
283
|
+
staging rateLimiter.requestsPerMinute = 6000
|
|
284
|
+
prod rateLimiter.requestsPerMinute = 30000
|
|
285
|
+
dr rateLimiter.requestPerMinute = 15000
|
|
286
|
+
|
|
287
|
+
tls cipher list
|
|
288
|
+
dev tls.cipherSuites = TLS_AES_128_GCM_SHA256
|
|
289
|
+
staging tls.cipherSuites = TLS_AES_128_GCM_SHA256
|
|
290
|
+
prod tls.cipherSuites = TLS_AES_256_GCM_SHA384
|
|
291
|
+
dr tls.cipherSuite = TLS_AES_128_GCM_SHA256
|
|
292
|
+
|
|
293
|
+
tls minimum version
|
|
294
|
+
dev tls.minVersion = TLSv1.2
|
|
295
|
+
staging tls.minVersion = TLSv1.2
|
|
296
|
+
prod tls.minVersion = TLSv1.3
|
|
297
|
+
dr tls.minVersion = TLSv1.2
|
|
298
|
+
|
|
299
|
+
retry backoff maximum (same key and value in every environment)
|
|
300
|
+
dev / staging / prod / dr retry.backoff.maxMs = 5000 (default set by v1.14.12)
|
|
301
|
+
|
|
302
|
+
config header last-applied and version (per environment)
|
|
303
|
+
dev 2026-05-02T09:15:00Z version=v1.15.0
|
|
304
|
+
staging 2026-05-28T18:40:11Z version=v1.15.0
|
|
305
|
+
prod 2026-06-10T14:22:07Z version=v2.0.1
|
|
306
|
+
dr 2026-06-09T22:05:44Z version=v2.0.0
|
|
307
|
+
tags: [hallucination, citation-fidelity, config, exact-value, key-names]
|
|
@@ -0,0 +1,204 @@
|
|
|
1
|
+
id: missing-deploy-window
|
|
2
|
+
version: 1.0.0
|
|
3
|
+
category: hallucination
|
|
4
|
+
cluster: missing-evidence
|
|
5
|
+
difficulty: expert
|
|
6
|
+
title: Did It Ship During the Freeze?
|
|
7
|
+
summary: A freeze-compliance review must decide whether a payments service deployed to the frozen region during the window, but the region's production rollout record was lost to an audit outage, surrounded by merge, build, staging, and other-region timestamps that do not answer the question.
|
|
8
|
+
prompt: |
|
|
9
|
+
A change-freeze review must determine whether billing-gateway release r-5521 violated freeze
|
|
10
|
+
CF-2026-06. Using ALL artifacts, answer the six deliverables below.
|
|
11
|
+
|
|
12
|
+
The policy defines "deploy" precisely (see freeze-policy) and audit-schema defines which event
|
|
13
|
+
authoritatively records a production rollout. Be careful not to substitute a merge, build,
|
|
14
|
+
staging, approval, other-region, or other-service timestamp for the us-east production rollout
|
|
15
|
+
the question asks about. Some deliverables ask whether/when something happened that the
|
|
16
|
+
artifacts do not actually record. If a deliverable cannot be answered, say so and name exactly
|
|
17
|
+
which record is missing and why the nearby timestamps do not substitute. If a deliverable
|
|
18
|
+
rests on a false premise, identify it, cite the contradicting evidence, and correct it. Cite
|
|
19
|
+
artifact ids for every claim.
|
|
20
|
+
|
|
21
|
+
1. When was billing-gateway r-5521 merged, and when did its image build complete?
|
|
22
|
+
2. When did r-5521 roll out to us-west production, and is us-west covered by freeze CF-2026-06?
|
|
23
|
+
3. Did r-5521's us-east PRODUCTION rollout occur during the freeze window
|
|
24
|
+
(2026-06-20 00:00 through 2026-06-23 23:59 UTC)?
|
|
25
|
+
4. The change ticket states r-5521 was "deployed to production on 2026-06-19, safely before
|
|
26
|
+
the freeze." Confirm that no freeze violation occurred.
|
|
27
|
+
5. Under CF-2026-06, what authorizes a production deploy to a frozen service during the
|
|
28
|
+
window, and is such authorization on record for billing-gateway r-5521?
|
|
29
|
+
6. How long did the us-east production rollout of r-5521 take, from start to HEALTHY?
|
|
30
|
+
artifacts:
|
|
31
|
+
- id: freeze-policy
|
|
32
|
+
type: spec
|
|
33
|
+
label: Change-freeze policy CF-2026-06
|
|
34
|
+
content: |
|
|
35
|
+
Change-freeze policy CF-2026-06.
|
|
36
|
+
|
|
37
|
+
Scope: All PAYMENTS-tier services in region us-east are frozen for PRODUCTION deploys from
|
|
38
|
+
2026-06-20 00:00:00 UTC through 2026-06-23 23:59:59 UTC (inclusive), covering the mid-year
|
|
39
|
+
settlement event. Regions other than us-east (including us-west) and services that are not
|
|
40
|
+
payments-tier are NOT covered by this freeze. A deploy to us-west during these dates is not
|
|
41
|
+
a violation; only us-east payments-tier production deploys are frozen.
|
|
42
|
+
|
|
43
|
+
Definition of "deploy" under this policy: completion of a PRODUCTION rollout in the target
|
|
44
|
+
region, i.e. that region's production fleet is serving the new version (the audit system's
|
|
45
|
+
ROLLOUT_HEALTHY event; see audit-schema). Merges, CI builds, image pushes, approvals, and
|
|
46
|
+
staging rollouts are explicitly NOT "deploys" under this policy and do not count for or
|
|
47
|
+
against freeze compliance.
|
|
48
|
+
|
|
49
|
+
Emergency exception: a production deploy to a frozen service/region during the window is
|
|
50
|
+
permitted ONLY with a VP-level emergency-exception approval recorded in the change ticket's
|
|
51
|
+
"emergency_exception" field. A normal team-lead change approval does NOT authorize a
|
|
52
|
+
freeze-window deploy. The change ticket is the authoritative system of record for whether
|
|
53
|
+
an exception was granted.
|
|
54
|
+
|
|
55
|
+
billing-gateway is a PAYMENTS-tier service (see region-config).
|
|
56
|
+
- id: region-config
|
|
57
|
+
type: config
|
|
58
|
+
label: billing-gateway region configuration
|
|
59
|
+
content: |
|
|
60
|
+
service: billing-gateway
|
|
61
|
+
tier: payments
|
|
62
|
+
regions_served:
|
|
63
|
+
- us-east # primary payments region (UNDER freeze CF-2026-06)
|
|
64
|
+
- us-west # secondary region (NOT under freeze CF-2026-06)
|
|
65
|
+
release_flow: A production release normally rolls out to us-west first, then us-east. The
|
|
66
|
+
two regional rollouts are separate, independently recorded audit events (one
|
|
67
|
+
ROLLOUT_HEALTHY per region). A us-west rollout does not imply the us-east rollout has
|
|
68
|
+
happened, and vice versa.
|
|
69
|
+
other_payments_services_in_us_east:
|
|
70
|
+
- ledger-service
|
|
71
|
+
- settlement-worker
|
|
72
|
+
non_payments_services:
|
|
73
|
+
- analytics-api # not payments-tier; not covered by CF-2026-06
|
|
74
|
+
- id: audit-schema
|
|
75
|
+
type: spec
|
|
76
|
+
label: deploy-audit event schema and authority
|
|
77
|
+
content: |
|
|
78
|
+
deploy-audit system - event schema and authority rules.
|
|
79
|
+
|
|
80
|
+
Event types recorded per (service, region, environment):
|
|
81
|
+
BUILD_COMPLETED, IMAGE_PUSHED, APPROVAL_RECORDED, ROLLOUT_STARTED, ROLLOUT_HEALTHY.
|
|
82
|
+
ROLLOUT_HEALTHY (region=prod) means that region's production fleet is serving the release.
|
|
83
|
+
|
|
84
|
+
Authority rules:
|
|
85
|
+
- ROLLOUT_HEALTHY for (service, region, env=prod) is the SINGLE authoritative record that
|
|
86
|
+
that region's production fleet took a given release. There is no secondary or derived
|
|
87
|
+
source; the running-version dashboard is rebuilt from these same events and inherits any
|
|
88
|
+
gaps in them.
|
|
89
|
+
- Merge, BUILD_COMPLETED, IMAGE_PUSHED, APPROVAL_RECORDED, and staging ROLLOUT_HEALTHY do
|
|
90
|
+
NOT imply that a production rollout occurred in any region.
|
|
91
|
+
- Each region emits its ROLLOUT_* events to its OWN regional audit sink. A us-west rollout
|
|
92
|
+
and a us-east rollout are independent events in independent sinks; one is not evidence of
|
|
93
|
+
the other.
|
|
94
|
+
|
|
95
|
+
Outage semantics: if a regional sink is down, ROLLOUT_* events for that region during the
|
|
96
|
+
outage are dropped and unrecoverable. The absence of a ROLLOUT_HEALTHY row therefore means
|
|
97
|
+
EITHER (a) no rollout happened, OR (b) a rollout happened during a sink outage and was not
|
|
98
|
+
recorded. These two cases cannot be distinguished from the audit data alone.
|
|
99
|
+
- id: pipeline-log
|
|
100
|
+
type: log
|
|
101
|
+
label: CI/CD and deploy audit stream (multiple services)
|
|
102
|
+
content: |
|
|
103
|
+
Deploy/CI audit stream. Times UTC. Multiple services appear; only ROLLOUT_HEALTHY (prod)
|
|
104
|
+
rows count as "deploys" under CF-2026-06.
|
|
105
|
+
2026-06-18 16:20:00 deploy: settlement-worker r-2201 PROD ROLLOUT HEALTHY (region us-east)
|
|
106
|
+
2026-06-19 09:10:00 vcs: ledger-service PR #2990 MERGED to main (author p.singh)
|
|
107
|
+
2026-06-19 21:30:00 deploy: ledger-service r-4410 PROD ROLLOUT HEALTHY (region us-east)
|
|
108
|
+
2026-06-19 22:40:10 vcs: billing-gateway PR #3120 MERGED to main (author k.novak)
|
|
109
|
+
2026-06-19 22:41:00 ci: billing-gateway BUILD_COMPLETED... build r-5521 STARTED
|
|
110
|
+
2026-06-19 23:05:12 ci: billing-gateway build r-5521 BUILD_COMPLETED image=sha256:9ab...c1
|
|
111
|
+
2026-06-19 23:11:30 registry: billing-gateway image r-5521 IMAGE_PUSHED
|
|
112
|
+
2026-06-19 23:14:00 approval: billing-gateway CHG-8842 APPROVAL_RECORDED (approver=t.reyes role=team-lead)
|
|
113
|
+
2026-06-19 23:20:40 deploy: billing-gateway r-5521 STAGING ROLLOUT STARTED (us-east-staging)
|
|
114
|
+
2026-06-19 23:26:15 deploy: billing-gateway r-5521 STAGING ROLLOUT HEALTHY (us-east-staging)
|
|
115
|
+
2026-06-20 00:00:00 freeze: CF-2026-06 window OPEN (us-east payments)
|
|
116
|
+
2026-06-20 01:10:00 deploy: analytics-api r-7781 PROD ROLLOUT HEALTHY (region us-east) [not payments-tier]
|
|
117
|
+
2026-06-20 06:30:00 deploy: recommendations-api r-6602 PROD ROLLOUT HEALTHY (region us-west)
|
|
118
|
+
2026-06-20 07:15:00 vcs: billing-gateway PR #3126 MERGED to main (author k.novak) [unrelated later PR, not r-5521]
|
|
119
|
+
2026-06-20 08:00:00 audit-sink(us-east): event delivery STOPPED (incident INF-990)
|
|
120
|
+
2026-06-20 11:40:00 deploy: settlement-worker r-2205 STAGING ROLLOUT HEALTHY (us-east-staging)
|
|
121
|
+
2026-06-20 14:12:00 deploy: search-api r-5140 PROD ROLLOUT HEALTHY (region us-west)
|
|
122
|
+
2026-06-21 03:04:00 deploy: billing-gateway r-5521 PROD ROLLOUT STARTED (region us-west)
|
|
123
|
+
2026-06-21 03:12:20 deploy: billing-gateway r-5521 PROD ROLLOUT HEALTHY (region us-west)
|
|
124
|
+
2026-06-21 07:45:00 deploy: notifications-api r-3320 STAGING ROLLOUT HEALTHY (us-east-staging)
|
|
125
|
+
2026-06-22 09:00:00 deploy: profile-api r-8890 PROD ROLLOUT HEALTHY (region us-west)
|
|
126
|
+
2026-06-22 12:00:00 audit-sink(us-east): event delivery RESTORED (INF-990 resolved)
|
|
127
|
+
2026-06-22 15:30:00 deploy: ledger-service r-4433 PROD ROLLOUT HEALTHY (region us-east) [after sink restored; different service]
|
|
128
|
+
2026-06-23 23:59:59 freeze: CF-2026-06 window CLOSE
|
|
129
|
+
2026-06-24 08:02:00 audit query: no ROLLOUT_STARTED or ROLLOUT_HEALTHY (region us-east, env=prod) exists for billing-gateway r-5521, at any date
|
|
130
|
+
2026-06-24 08:03:00 audit query: the only billing-gateway prod ROLLOUT_HEALTHY on record is region us-west at 2026-06-21 03:12:20
|
|
131
|
+
2026-06-24 08:04:00 audit query: billing-gateway us-east running-version dashboard is derived from these events and shows no r-5521 us-east rollout record
|
|
132
|
+
- id: deploy-audit-table
|
|
133
|
+
type: table
|
|
134
|
+
label: Retained production/staging rollout records
|
|
135
|
+
content: |
|
|
136
|
+
service region env event timestamp_utc
|
|
137
|
+
settlement-worker us-east prod ROLLOUT_HEALTHY 2026-06-18 16:20:00
|
|
138
|
+
ledger-service us-east prod ROLLOUT_HEALTHY 2026-06-19 21:30:00
|
|
139
|
+
analytics-api us-east prod ROLLOUT_HEALTHY 2026-06-20 01:10:00
|
|
140
|
+
recommendations-api us-west prod ROLLOUT_HEALTHY 2026-06-20 06:30:00
|
|
141
|
+
search-api us-west prod ROLLOUT_HEALTHY 2026-06-20 14:12:00
|
|
142
|
+
billing-gateway us-east staging ROLLOUT_HEALTHY 2026-06-19 23:26:15
|
|
143
|
+
billing-gateway us-west prod ROLLOUT_HEALTHY 2026-06-21 03:12:20
|
|
144
|
+
profile-api us-west prod ROLLOUT_HEALTHY 2026-06-22 09:00:00
|
|
145
|
+
ledger-service us-east prod ROLLOUT_HEALTHY 2026-06-22 15:30:00
|
|
146
|
+
(no row exists for billing-gateway / us-east / prod, for r-5521 or otherwise)
|
|
147
|
+
Reading guide: analytics-api is not payments-tier; ledger-service and settlement-worker are
|
|
148
|
+
different services; every us-west row is a non-frozen region. None of these is a us-east
|
|
149
|
+
production rollout of billing-gateway.
|
|
150
|
+
- id: monitoring-gap-note
|
|
151
|
+
type: note
|
|
152
|
+
label: Audit-sink outage window (INF-990)
|
|
153
|
+
content: |
|
|
154
|
+
monitoring-gap-note (infra incident INF-990):
|
|
155
|
+
The us-east deploy-audit sink stopped delivering events from 2026-06-20 08:00 to
|
|
156
|
+
2026-06-22 12:00 UTC. Events that occurred in that window were not persisted and are
|
|
157
|
+
unrecoverable. The us-west and staging audit sinks were on separate infrastructure and were
|
|
158
|
+
unaffected, so us-west prod rollouts and us-east staging rollouts in that window were still
|
|
159
|
+
recorded normally.
|
|
160
|
+
|
|
161
|
+
Consequence: any us-east PRODUCTION rollout that STARTED or completed inside 06-20 08:00 to
|
|
162
|
+
06-22 12:00 has no audit record. Per audit-schema, ROLLOUT_HEALTHY is the only authoritative
|
|
163
|
+
record of a production rollout and has no secondary source. Therefore, for billing-gateway
|
|
164
|
+
r-5521 specifically: whether it rolled out to us-east production at all, and if so at what
|
|
165
|
+
time, cannot be recovered from the audit system. Note the outage window falls entirely
|
|
166
|
+
inside the freeze window.
|
|
167
|
+
- id: change-ticket
|
|
168
|
+
type: note
|
|
169
|
+
label: Change ticket CHG-8842 (contains a deployment claim)
|
|
170
|
+
content: |
|
|
171
|
+
change-ticket CHG-8842 - billing-gateway r-5521
|
|
172
|
+
|
|
173
|
+
Fields as entered by requester k.novak:
|
|
174
|
+
merged: 2026-06-19 22:40
|
|
175
|
+
built: 2026-06-19 23:05
|
|
176
|
+
image_pushed: 2026-06-19 23:11
|
|
177
|
+
approved_by: t.reyes (role: team-lead)
|
|
178
|
+
emergency_exception: (blank)
|
|
179
|
+
regions_targeted: us-west, us-east
|
|
180
|
+
Deployment note (free text): "Deployed to production on 2026-06-19, safely before the
|
|
181
|
+
freeze window opened. No freeze impact."
|
|
182
|
+
|
|
183
|
+
Post-freeze reviewer question appended: "The 06-19 date - is that a PRODUCTION rollout, or
|
|
184
|
+
is it the merge / build / staging time? The audit stream shows a us-west prod rollout on
|
|
185
|
+
06-21 and no us-east prod record at all for r-5521. And the emergency_exception field is
|
|
186
|
+
blank, so if it did go to us-east prod during the freeze there was no exception on file."
|
|
187
|
+
- id: oncall-chat
|
|
188
|
+
type: note
|
|
189
|
+
label: On-call thread (speculation, unverified)
|
|
190
|
+
content: |
|
|
191
|
+
oncall-chat: #deploys thread, 2026-06-24
|
|
192
|
+
|
|
193
|
+
k.novak: did billing-gateway r-5521 reach us-east prod before the freeze?
|
|
194
|
+
t.reyes: the ticket says deployed 06-19, so we should be fine.
|
|
195
|
+
s.park: 06-19 is the merge and staging time though. The only prod ROLLOUT_HEALTHY I see for
|
|
196
|
+
r-5521 is us-west at 06-21 03:12. There is no us-east prod HEALTHY for r-5521 anywhere.
|
|
197
|
+
k.novak: the us-east audit sink was down 06-20 08:00 to 06-22 12:00 (INF-990), so a us-east
|
|
198
|
+
rollout in that window would not have been logged. Or it may not have gone out yet.
|
|
199
|
+
s.park: and per audit-schema there is no backup source - ROLLOUT_HEALTHY is the only record.
|
|
200
|
+
t.reyes: ledger-service shows a us-east prod rollout on 06-22 15:30 though - doesn't that count?
|
|
201
|
+
s.park: that is ledger-service, a different service. Says nothing about billing-gateway.
|
|
202
|
+
k.novak: so we cannot confirm or rule out a us-east freeze-window deploy of r-5521 from what we have.
|
|
203
|
+
t.reyes: agreed - we do not actually know when, or whether, us-east prod took r-5521.
|
|
204
|
+
tags: [missing-evidence, change-freeze, deploy, audit-gap, region-scope]
|