bridgebench 3.1.0-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (86) hide show
  1. package/CITATION.cff +15 -0
  2. package/LICENSE +21 -0
  3. package/README.md +249 -0
  4. package/dist/chunk-4TWPCPRP.cjs +1097 -0
  5. package/dist/chunk-4TWPCPRP.cjs.map +1 -0
  6. package/dist/chunk-7YCJSOK7.cjs +398 -0
  7. package/dist/chunk-7YCJSOK7.cjs.map +1 -0
  8. package/dist/chunk-CIXITJW6.cjs +249 -0
  9. package/dist/chunk-CIXITJW6.cjs.map +1 -0
  10. package/dist/chunk-EQHRUV2I.js +1466 -0
  11. package/dist/chunk-EQHRUV2I.js.map +1 -0
  12. package/dist/chunk-JTVNKSMO.js +1096 -0
  13. package/dist/chunk-JTVNKSMO.js.map +1 -0
  14. package/dist/chunk-LFKEV2YL.js +398 -0
  15. package/dist/chunk-LFKEV2YL.js.map +1 -0
  16. package/dist/chunk-NJTYVNP4.cjs +1467 -0
  17. package/dist/chunk-NJTYVNP4.cjs.map +1 -0
  18. package/dist/chunk-UECBSKTD.js +244 -0
  19. package/dist/chunk-UECBSKTD.js.map +1 -0
  20. package/dist/cli.cjs +409 -0
  21. package/dist/cli.cjs.map +1 -0
  22. package/dist/cli.d.cts +1 -0
  23. package/dist/cli.d.ts +1 -0
  24. package/dist/cli.js +408 -0
  25. package/dist/cli.js.map +1 -0
  26. package/dist/client.cjs +42 -0
  27. package/dist/client.cjs.map +1 -0
  28. package/dist/client.d.cts +93 -0
  29. package/dist/client.d.ts +93 -0
  30. package/dist/client.js +42 -0
  31. package/dist/client.js.map +1 -0
  32. package/dist/contracts/index.cjs +47 -0
  33. package/dist/contracts/index.cjs.map +1 -0
  34. package/dist/contracts/index.d.cts +14 -0
  35. package/dist/contracts/index.d.ts +14 -0
  36. package/dist/contracts/index.js +47 -0
  37. package/dist/contracts/index.js.map +1 -0
  38. package/dist/index.cjs +171 -0
  39. package/dist/index.cjs.map +1 -0
  40. package/dist/index.d.cts +214 -0
  41. package/dist/index.d.ts +214 -0
  42. package/dist/index.js +171 -0
  43. package/dist/index.js.map +1 -0
  44. package/dist/logger-CCR9Mg1c.d.cts +319 -0
  45. package/dist/logger-QJU7SBDz.d.ts +319 -0
  46. package/dist/reports-4CejmOHf.d.cts +454 -0
  47. package/dist/reports-s2CTnGN8.d.ts +454 -0
  48. package/dist/tasks-CpaCJ6JE.d.cts +151 -0
  49. package/dist/tasks-CpaCJ6JE.d.ts +151 -0
  50. package/dist/tasks.cjs +22 -0
  51. package/dist/tasks.cjs.map +1 -0
  52. package/dist/tasks.d.cts +39 -0
  53. package/dist/tasks.d.ts +39 -0
  54. package/dist/tasks.js +22 -0
  55. package/dist/tasks.js.map +1 -0
  56. package/docs/README.md +25 -0
  57. package/docs/glossary.md +49 -0
  58. package/docs/methodology.md +58 -0
  59. package/docs/private-packs.md +74 -0
  60. package/docs/replay-elo.md +79 -0
  61. package/docs/task-authoring.md +80 -0
  62. package/package.json +137 -0
  63. package/tasks/hallucination/public/boundary-coverage-audit.yaml +274 -0
  64. package/tasks/hallucination/public/boundary-migration-audit.yaml +284 -0
  65. package/tasks/hallucination/public/conflict-dependency-versions.yaml +324 -0
  66. package/tasks/hallucination/public/conflict-runbook-versions.yaml +229 -0
  67. package/tasks/hallucination/public/fabrication-agent-tools.yaml +224 -0
  68. package/tasks/hallucination/public/fabrication-api-surface.yaml +239 -0
  69. package/tasks/hallucination/public/fidelity-commit-attribution.yaml +304 -0
  70. package/tasks/hallucination/public/fidelity-config-drift.yaml +307 -0
  71. package/tasks/hallucination/public/missing-deploy-window.yaml +204 -0
  72. package/tasks/hallucination/public/missing-latency-baseline.yaml +239 -0
  73. package/tasks/hallucination/public/premise-quota-breach.yaml +202 -0
  74. package/tasks/hallucination/public/premise-rollback-cause.yaml +235 -0
  75. package/tasks/reasoning/public/constraint-capacity-allocation.yaml +196 -0
  76. package/tasks/reasoning/public/constraint-deployment-policy.yaml +203 -0
  77. package/tasks/reasoning/public/counterexample-authorization-rule.yaml +278 -0
  78. package/tasks/reasoning/public/counterexample-scheduler-starvation.yaml +290 -0
  79. package/tasks/reasoning/public/root-cache-tenant-leak.yaml +225 -0
  80. package/tasks/reasoning/public/root-event-ordering.yaml +184 -0
  81. package/tasks/reasoning/public/stateful-lease-handoff.yaml +213 -0
  82. package/tasks/reasoning/public/stateful-retry-budget.yaml +222 -0
  83. package/tasks/reasoning/public/synthesis-api-contract.yaml +214 -0
  84. package/tasks/reasoning/public/synthesis-permission-migration.yaml +190 -0
  85. package/tasks/reasoning/public/uncertainty-conflicting-telemetry.yaml +242 -0
  86. package/tasks/reasoning/public/uncertainty-incomplete-incident.yaml +223 -0
@@ -0,0 +1,222 @@
1
+ id: stateful-retry-budget
2
+ version: 2.0.0
3
+ category: reasoning
4
+ cluster: stateful-execution
5
+ difficulty: expert
6
+ title: Shared Retry Budget Across Five Concurrent Requests With Cancellation
7
+ summary: >-
8
+ Trace five concurrent requests through a retry state machine that share one token-bucket retry
9
+ budget, with deterministic backoff and jitter, per-request deadlines, cancellation, and three
10
+ distinct failure modes, then evaluate a proposed increase to the per-request attempt cap.
11
+ prompt: |
12
+ A client library runs five requests (R1..R5) concurrently against one upstream. They share a single
13
+ retry budget. The state machine and its transition precedence are in [retry-state-machine]; the
14
+ dispatcher that implements it is [dispatcher-code]; numeric parameters are in [retry-config]; status
15
+ codes are classified by [status-classification]; retry start times use the deterministic offsets in
16
+ [jitter-table]; the raw global-ordered event stream is [event-log]. [ops-note] is an engineer's
17
+ post-incident summary. [proposed-change] is a config edit under review.
18
+
19
+ All times are milliseconds from an origin. The budget is shared: every request's retries draw from
20
+ and credit the same bucket, so you must process budget-affecting events in strict time order. Answer
21
+ every deliverable; the log alone does not state outcomes — you must apply the state machine.
22
+
23
+ 1. Terminal state. For each of R1..R5 give the terminal state. Where a request FAILED, give the
24
+ specific reason (max-attempts, budget-exhausted, deadline, or non-retryable).
25
+ 2. Call accounting. Give the total number of upstream calls, the per-request call count, and the
26
+ exact start time (ms) of R2's third attempt.
27
+ 3. Budget. Give the exact final value of the shared retry budget after all events.
28
+ 4. Failure modes. Exactly three requests do not end in SUCCESS. Identify which one ends by
29
+ max-attempts, which by budget-exhaustion, and which by cancellation — and justify each against
30
+ the precedence order in [retry-state-machine].
31
+ 5. Cancellation and the stale timer. For R4: does attempt 3 begin? State what happens when the timer
32
+ scheduled for R4 fires at t=610. Then, using [event-log], identify every claim in [ops-note] that
33
+ is false and give the correct fact for each.
34
+ 6. Proposed change. Under [proposed-change] (max_attempts_per_request 3 -> 5), state which requests'
35
+ terminal outcomes change, which requests' failure reason changes, whether the total upstream call
36
+ count changes, and explain why the change does or does not improve the success rate.
37
+ artifacts:
38
+ - id: retry-state-machine
39
+ type: spec
40
+ label: Retry state machine and precedence
41
+ content: |
42
+ DEFINITIONS
43
+ - attempt: one call issued to the upstream. attempts_started counts attempts issued so far for
44
+ a request; when the response for attempt N arrives, attempts_started == N.
45
+ - RETRY_WAIT: the request is idle, holding a scheduled timer for its next attempt.
46
+ - budget: the shared token bucket in [retry-config]; there is exactly one, common to all
47
+ requests.
48
+
49
+ States: PENDING -> IN_FLIGHT -> one of { SUCCESS, RETRY_WAIT, FAILED, CANCELLED,
50
+ DEADLINE_EXCEEDED }. attempt 1 starts at admission (PENDING -> IN_FLIGHT).
51
+
52
+ On a response classified retryable (see [status-classification]), evaluate IN THIS ORDER and act
53
+ on the FIRST matching rule (later rules are not consulted once one fires):
54
+ (1) if attempts_started >= max_attempts_per_request -> terminal FAILED, reason max-attempts.
55
+ (2) else if now >= (admission_time + per_request_deadline_ms) -> terminal DEADLINE_EXCEEDED.
56
+ (3) else if budget < min_to_retry -> terminal FAILED, reason budget-exhausted.
57
+ (4) else schedule attempt N+1 at time (response_time + backoff_gap(N) + jitter); set
58
+ budget := budget - retry_cost; state RETRY_WAIT.
59
+ backoff_gap(N) = backoff_base_ms * backoff_multiplier^(N-1): after attempt 1 = 100 ms, after
60
+ attempt 2 = 200 ms. jitter is taken from [jitter-table] for (request, N).
61
+
62
+ On a response classified success -> terminal SUCCESS; budget := min(max, budget + success_credit).
63
+ On a response classified terminal-fail (non-retryable) -> terminal FAILED, reason non-retryable.
64
+
65
+ Cancellation (caller-initiated) is immediately terminal CANCELLED. It cancels any scheduled retry
66
+ timer. A timer that fires for an already-terminal request is IGNORED and starts no attempt. A
67
+ response that arrives for an attempt after the request is terminal is IGNORED. retry_cost spent
68
+ on a scheduled-but-not-yet-started retry is NOT refunded on cancellation.
69
+
70
+ Deadline reached while RETRY_WAIT or IN_FLIGHT -> terminal DEADLINE_EXCEEDED. Terminal states are
71
+ final and never re-enter the machine.
72
+
73
+ ILLUSTRATIVE ONLY (not one of R1..R5): a request admitted at t=1000 (deadline 1500) whose attempt
74
+ 1 returns 500 at t=1040, with budget 4.0 at that instant: rule (1) no (1 < 3), rule (2) no
75
+ (1040 < 1500), rule (3) no (4.0 >= 1.0), so rule (4) fires — schedule attempt 2 at
76
+ 1040 + 100 + jitter and set budget := 3.0. This example uses invented numbers to show the
77
+ precedence; do not fold it into the R1..R5 accounting.
78
+ - id: dispatcher-code
79
+ type: code
80
+ label: Shared dispatcher pseudocode
81
+ content: |
82
+ // One dispatcher serves all requests. `budget` is a single shared variable (the token bucket).
83
+ // Events are delivered in nondecreasing time order (see [event-log]).
84
+
85
+ budget = config.retry_budget.initial // 8.0, cap config.retry_budget.max (8.0)
86
+
87
+ function onResponse(req, attemptN, status, nowMs):
88
+ if req.terminal: return // late response for a finished request: ignore
89
+ cls = classify(status) // see [status-classification]
90
+ if cls == "success":
91
+ req.finish(SUCCESS)
92
+ budget = min(config.max, budget + config.success_credit) // +0.5
93
+ return
94
+ if cls == "terminal-fail":
95
+ req.finish(FAILED, reason = "non-retryable"); return
96
+ // cls == "retryable" -> precedence order:
97
+ if attemptN >= config.max_attempts_per_request:
98
+ req.finish(FAILED, reason = "max-attempts"); return // rule (1)
99
+ if nowMs >= req.admittedAt + config.per_request_deadline_ms:
100
+ req.finish(DEADLINE_EXCEEDED); return // rule (2)
101
+ if budget < config.min_to_retry:
102
+ req.finish(FAILED, reason = "budget-exhausted"); return // rule (3)
103
+ gap = config.backoff_base_ms * (config.backoff_multiplier ** (attemptN - 1))
104
+ startAt = nowMs + gap + jitter(req, attemptN) // [jitter-table]
105
+ budget = budget - config.retry_cost // rule (4): -1.0, not refunded
106
+ req.scheduleAttempt(attemptN + 1, startAt)
107
+
108
+ function onCancel(req, nowMs):
109
+ if req.terminal: return
110
+ req.cancelTimer(); req.finish(CANCELLED)
111
+
112
+ function onTimer(req, attemptN):
113
+ if req.terminal: return // stale timer for a finished request: ignore
114
+ req.startAttempt(attemptN)
115
+ - id: retry-config
116
+ type: config
117
+ label: Retry parameters
118
+ content: |
119
+ max_attempts_per_request: 3
120
+ backoff_base_ms: 100
121
+ backoff_multiplier: 2 # gap after attempt N = base * multiplier^(N-1)
122
+ per_request_deadline_ms: 500 # absolute deadline = admission_time + 500
123
+ timer_resolution_ms: 1 # timers fire at their scheduled ms
124
+ upstream: "quote-svc" # single upstream shared by all requests
125
+ retry_budget:
126
+ initial: 8.0 # bucket starts full
127
+ max: 8.0 # credits are capped here
128
+ retry_cost: 1.0 # subtracted when a retry is scheduled (rule 4)
129
+ success_credit: 0.5 # added on SUCCESS, capped at max
130
+ min_to_retry: 1.0 # a retry requires budget >= this value
131
+ - id: status-classification
132
+ type: table
133
+ label: Response classification
134
+ content: |
135
+ status | class
136
+ 200, 201, 204 | success
137
+ 400, 401, 403, 404, 422 | terminal-fail (non-retryable)
138
+ 429, 500, 502, 503 | retryable
139
+ TIMEOUT | retryable
140
+ note: only the statuses that actually occur in [event-log] matter for the answer; the rest are
141
+ listed so the classification is unambiguous. TIMEOUT is a transport timeout of a single attempt,
142
+ not the per-request deadline.
143
+ - id: jitter-table
144
+ type: table
145
+ label: Deterministic retry jitter
146
+ content: |
147
+ request | attempt N | jitter_ms
148
+ R1 | 1 | 12
149
+ R1 | 2 | 0
150
+ R2 | 1 | 20
151
+ R2 | 2 | 15
152
+ R3 | 1 | 8
153
+ R3 | 2 | 25
154
+ R4 | 1 | 5
155
+ R4 | 2 | 10
156
+ R5 | 1 | 18
157
+ R5 | 2 | 0
158
+ note: jitter for (request, N) is added to attempt N+1's start. No entry is needed for attempt 3's
159
+ successor because max attempts is 3.
160
+ - id: event-log
161
+ type: log
162
+ label: Global-ordered event stream (raw; outcomes not annotated)
163
+ content: |
164
+ t_ms | request | phase | status/detail | latency_ms | endpoint
165
+ 0 | R1 | admitted | deadline 500 | - | -
166
+ 0 | R1 | attempt 1 start | - | - | quote-svc
167
+ 30 | R1 | attempt 1 response | 503 | 30 | quote-svc
168
+ 50 | R2 | admitted | deadline 550 | - | -
169
+ 50 | R2 | attempt 1 start | - | - | quote-svc
170
+ 90 | R2 | attempt 1 response | TIMEOUT | 40 | quote-svc
171
+ 100 | R3 | admitted | deadline 600 | - | -
172
+ 100 | R3 | attempt 1 start | - | - | quote-svc
173
+ 142 | R1 | attempt 2 start | - | - | quote-svc
174
+ 150 | R3 | attempt 1 response | 503 | 50 | quote-svc
175
+ 175 | R1 | attempt 2 response | 200 | 33 | quote-svc
176
+ 200 | R4 | admitted | deadline 700 | - | -
177
+ 200 | R4 | attempt 1 start | - | - | quote-svc
178
+ 210 | R2 | attempt 2 start | - | - | quote-svc
179
+ 250 | R4 | attempt 1 response | 500 | 50 | quote-svc
180
+ 258 | R3 | attempt 2 start | - | - | quote-svc
181
+ 260 | R2 | attempt 2 response | 429 | 50 | quote-svc
182
+ 300 | R3 | attempt 2 response | TIMEOUT | 42 | quote-svc
183
+ 310 | R5 | admitted | deadline 810 | - | -
184
+ 310 | R5 | attempt 1 start | - | - | quote-svc
185
+ 340 | R5 | attempt 1 response | 429 | 30 | quote-svc
186
+ 355 | R4 | attempt 2 start | - | - | quote-svc
187
+ 400 | R4 | attempt 2 response | 503 | 45 | quote-svc
188
+ 458 | R5 | attempt 2 start | - | - | quote-svc
189
+ 475 | R2 | attempt 3 start | - | - | quote-svc
190
+ 480 | R4 | caller cancel | - | - | -
191
+ 500 | R5 | attempt 2 response | TIMEOUT | 42 | quote-svc
192
+ 520 | R2 | attempt 3 response | 503 | 45 | quote-svc
193
+ 525 | R3 | attempt 3 start | - | - | quote-svc
194
+ 560 | R3 | attempt 3 response | 200 | 35 | quote-svc
195
+ 610 | R4 | retry timer fires | for attempt 3 | - | -
196
+ - id: ops-note
197
+ type: note
198
+ label: Post-incident summary (unverified)
199
+ content: |
200
+ On-call notes, filed after the incident (NOT independently verified):
201
+ - R2 exceeded its request deadline and timed out.
202
+ - R5 failed after using all 3 of its attempts.
203
+ - R4 completed its third attempt at t=610.
204
+ - Raising max_attempts_per_request to 5 will recover both R2 and R5, so the fix is a one-line
205
+ config bump.
206
+ Proposed remediation attached to the note: "Bump max attempts to 5; keep everything else the
207
+ same." Treat every line above as a claim to be checked against [event-log], not as fact.
208
+ - id: proposed-change
209
+ type: diff
210
+ label: Proposed configuration change
211
+ content: |
212
+ --- retry-config (current)
213
+ +++ retry-config (proposed)
214
+ - max_attempts_per_request: 3
215
+ + max_attempts_per_request: 5 # "let struggling requests try harder to succeed"
216
+ backoff_base_ms: 100 # unchanged
217
+ retry_budget:
218
+ initial: 8.0 # unchanged
219
+ success_credit: 0.5 # unchanged
220
+ min_to_retry: 1.0 # unchanged
221
+ Only max_attempts_per_request changes; the shared budget parameters are untouched.
222
+ tags: [state-machine, retries, token-bucket, backoff, cancellation, concurrency]
@@ -0,0 +1,214 @@
1
+ id: synthesis-api-contract
2
+ version: 2.0.0
3
+ category: reasoning
4
+ cluster: multi-artifact-synthesis
5
+ difficulty: expert
6
+ title: Legacy API Clients Lose Exactly Their Renamed Fields After a Uniform Projection Allowlist
7
+ summary: A shared profile handler serves both a frozen v4 contract and a current v5 contract through a per-route gateway pipeline. After a security ticket added a response-field allowlist to every route, v4 clients silently lose three fields while keeping two, all with HTTP 200. Reconcile the contracts, pipeline, handler, logs, and field observations to find the one misconfiguration, reject the decoys, and specify the minimal fix scoped to the affected route.
8
+ prompt: |
9
+ v4 clients (mobile and a partner batch job) started getting HTTP 200 profile responses that are
10
+ missing required fields, while v5 clients are fine. Using only the artifacts, produce:
11
+
12
+ 1. FAILURE SURFACE. State exactly which fields are missing, on which route, for which clients, and
13
+ which fields still arrive. Cite client-logs and the field-observation-table.
14
+ 2. ROOT CAUSE. Explain the single misconfiguration that drops those fields and why it drops exactly
15
+ those and not the others. Cite gateway-pipeline, both contracts, and change-ticket.
16
+ 3. AVATAR DECOY. Explain why the recent avatar-object rollout (PROJ-799 in change-ticket) is not the
17
+ cause, even though avatar_url is among the missing fields.
18
+ 4. UPGRADE DECOY. Explain why "ship a mobile/partner client update" is not an acceptable resolution,
19
+ reconciling the dates in deprecation-policy.
20
+ 5. MINIMAL FIX. Give the smallest correction, scoped to a single route, that restores the v4 fields
21
+ without changing v5 behavior and without discarding the security intent of the allowlist.
22
+ 6. VERIFICATION. State the exact field set and value formats a correct v4 profile response must
23
+ contain, and confirm the v5 response must remain unchanged. Cite v4-contract and v5-contract.
24
+
25
+ Exactly one resolution is defensible per deliverable; it is fully determined by the artifacts.
26
+ artifacts:
27
+ - id: v5-contract
28
+ type: spec
29
+ label: v5 profile response contract (current)
30
+ content: |
31
+ GET /api/v5/profile -> 200 application/json
32
+ Required fields:
33
+ id string (uuid)
34
+ display_name string # renamed from v4 user_name
35
+ avatar object { url: string, blurhash: string } # replaced v4 avatar_url string
36
+ created_at string (ISO-8601 UTC) # replaced v4 integer "created"
37
+ plan enum "pro" | "ultra" # renamed values from v4 "plus" | "max"
38
+ The handler emits this shape natively; the v5 route performs no field renaming.
39
+
40
+ Canonical example (what a conforming v5 response looks like):
41
+ {
42
+ "id": "7b3f...e21",
43
+ "display_name": "Dana Okoro",
44
+ "avatar": { "url": "https://cdn.example/av/7b3f.png", "blurhash": "L6Pj0^" },
45
+ "created_at": "2025-02-11T09:30:00Z",
46
+ "plan": "pro"
47
+ }
48
+ Consumers: web-v5 (current web app) and any client that migrated to v5. These read the object
49
+ avatar and the ISO created_at directly. No v5 client depends on the legacy names.
50
+ Field notes:
51
+ - display_name is always present (never null); the handler falls back to the account handle.
52
+ - avatar.url is always a fully-qualified https URL; blurhash is a short placeholder string.
53
+ - id: v4-contract
54
+ type: spec
55
+ label: v4 profile response contract (frozen, deprecation window)
56
+ content: |
57
+ GET /api/v4/profile -> 200 application/json
58
+ Required fields (byte-stable; clients hard-fail if any are absent):
59
+ id string (uuid) # identical to v5
60
+ user_name string # v5 display_name
61
+ avatar_url string (url) # v5 avatar.url
62
+ created integer (epoch seconds) # v5 created_at
63
+ plan enum "plus" | "max" # v5 "pro" -> "plus", "ultra" -> "max"
64
+ Note: only user_name, avatar_url, and created require a NAME change from the v5 shape.
65
+ id keeps its name; plan keeps its name (only its enum values are remapped).
66
+
67
+ Canonical example (what a conforming v4 response MUST look like for the same user):
68
+ {
69
+ "id": "7b3f...e21",
70
+ "user_name": "Dana Okoro",
71
+ "avatar_url": "https://cdn.example/av/7b3f.png",
72
+ "created": 1739266200,
73
+ "plan": "plus"
74
+ }
75
+ Consumers: mobile-v4 (shipped app binaries in the field) and partner-batch (a nightly partner
76
+ integration job). Both hard-validate the response against this schema and abort on any missing
77
+ required field, surfacing a user-visible error rather than degrading gracefully.
78
+ Field notes:
79
+ - created is the integer epoch-seconds form of the same instant v5 encodes as created_at ISO.
80
+ - "plus"/"max" are the ONLY plan values v4 clients accept; "pro"/"ultra" would be rejected too.
81
+ - id: deprecation-policy
82
+ type: note
83
+ label: API deprecation policy
84
+ content: |
85
+ Incident date: 2026-07-10.
86
+ Policy: v4 is deprecated but SUPPORTED through 2026-12-31. Until that date the server MUST keep
87
+ v4 responses byte-compatible with the frozen v4-contract above. Rules:
88
+ - Server-side compatibility is mandatory, not best-effort.
89
+ - Clients on v4 cannot be compelled to upgrade before the window closes.
90
+ - A change that makes a v4 response non-conforming is a Sev-2 regression, not an accepted cost
91
+ of shipping v5.
92
+ Forcing a client release inside the window (mobile store update, partner redeploy, etc.) is a
93
+ policy violation and is not an accepted remediation. As of 2026-07-10 the window is OPEN
94
+ (roughly six months remain).
95
+ - id: gateway-pipeline
96
+ type: config
97
+ label: gateway per-route response pipeline
98
+ content: |
99
+ # gateway/response-pipeline.yaml
100
+ # Each route runs an ORDERED list of steps, top to bottom, over the handler's JSON output.
101
+ # Step reference:
102
+ # rename { from_field: to_field } renames keys
103
+ # mapValues { field: { from_value: to } } remaps enum values in place
104
+ # coerce { field: type } converts a value's type/shape
105
+ # project { allow: [fields...] } DROPS any field not listed in `allow`
106
+ routes:
107
+ /api/v5/profile:
108
+ enabled: true
109
+ pipeline:
110
+ - project: { allow: [id, display_name, avatar, created_at, plan] }
111
+
112
+ /api/v4/profile:
113
+ enabled: true
114
+ pipeline:
115
+ - rename: { display_name: user_name, avatar: avatar_url, created_at: created }
116
+ - mapValues: { plan: { pro: plus, ultra: max } }
117
+ - coerce: { avatar_url: url_string, created: epoch_seconds }
118
+ - project: { allow: [id, display_name, avatar, created_at, plan] }
119
+ # NOTE: `project` filters on the field names present AT THAT POINT in the pipeline.
120
+ # Order matters: steps run top to bottom. A field dropped by `project` is absent from the
121
+ # response body entirely (not null); the client sees a shorter object.
122
+ - id: profile-handler
123
+ type: code
124
+ label: shared profile handler
125
+ content: |
126
+ // profile/handler.ts — ONE implementation serves BOTH /api/v4/profile and /api/v5/profile.
127
+ // Per-route gateway pipelines (see gateway-pipeline) adapt this output to each contract.
128
+ export async function getProfile(userId: string) {
129
+ const u = await db.users.findById(userId);
130
+ return {
131
+ id: u.id,
132
+ display_name: u.displayName, // v5-native field names
133
+ avatar: { url: u.avatarUrl, blurhash: u.blurhash },
134
+ created_at: u.createdAt.toISOString(), // ISO-8601 string
135
+ plan: normalizePlan(u.plan), // returns "pro" | "ultra"
136
+ };
137
+ }
138
+
139
+ function normalizePlan(raw: string): 'pro' | 'ultra' {
140
+ // Legacy stored values are normalized up to the v5 enum.
141
+ if (raw === 'plus' || raw === 'pro') return 'pro';
142
+ if (raw === 'max' || raw === 'ultra') return 'ultra';
143
+ return 'pro';
144
+ }
145
+ // The handler ALWAYS emits the v5 shape. It has no per-version branching.
146
+ - id: change-ticket
147
+ type: note
148
+ label: recent change tickets
149
+ content: |
150
+ PROJ-812 (merged 2 days before the incident): "Add a response-field allowlist (project step) to
151
+ ALL profile routes to prevent accidental field leakage." Implementation notes on the ticket:
152
+ - A single allowlist was written using the v5 field names.
153
+ - It was appended as the LAST step of every route's pipeline, including /api/v4/profile.
154
+ - No route-specific field names were used; the reviewer asked "does v4 rename before this?"
155
+ and the thread was never resolved before merge.
156
+ This is the only change to the v4 pipeline in the last 30 days.
157
+
158
+ PROJ-799 (merged 3 weeks before the incident): "Roll out v5 avatar as an object { url, blurhash }."
159
+ This changed the handler's avatar from a string to an object. v5 clients read avatar.url; the v4
160
+ route already had a `coerce` step to flatten avatar back to a url string. No client complaints
161
+ were filed when PROJ-799 shipped, and v4 traffic was healthy for the following two weeks.
162
+
163
+ Timeline:
164
+ - day -21: PROJ-799 merged and deployed. v4 and v5 both healthy afterward.
165
+ - day -14 to day -2: no profile-pipeline changes; v4 error rate ~0.
166
+ - day -2: PROJ-812 merged and deployed. v4 error rate begins climbing on the next deploy.
167
+ - day 0: incident opened; mobile-v4 and partner-batch failing on missing required fields.
168
+ The v4 failures start immediately after PROJ-812, not after PROJ-799.
169
+ - id: client-logs
170
+ type: log
171
+ label: client request outcomes and a captured v4 body
172
+ content: |
173
+ web-v5 GET /api/v5/profile 200 keys=[id, display_name, avatar, created_at, plan] parsed OK
174
+ mobile-v4 GET /api/v4/profile 200 keys=[id, plan] ERROR missing required: user_name, avatar_url, created
175
+ mobile-v4 GET /api/v4/profile 200 keys=[id, plan] ERROR missing required: user_name, avatar_url, created
176
+ partner-batch GET /api/v4/profile 200 keys=[id, plan] ERROR missing required: user_name, avatar_url, created
177
+ web-v5 GET /api/v5/profile 200 keys=[id, display_name, avatar, created_at, plan] parsed OK
178
+ partner-batch GET /api/v4/profile 200 keys=[id, plan] ERROR missing required: user_name, avatar_url, created
179
+
180
+ # Captured /api/v4/profile response body during the incident (same user as the contract examples):
181
+ # { "id": "7b3f...e21", "plan": "plus" }
182
+ # Observations: the body is HTTP 200; `plan` is present AND correctly remapped to "plus"
183
+ # (so the mapValues step ran); id is present; the three renamed fields are gone.
184
+
185
+ # Captured /api/v5/profile response body during the same window (for contrast):
186
+ # { "id": "7b3f...e21", "display_name": "Dana Okoro",
187
+ # "avatar": { "url": "https://cdn.example/av/7b3f.png", "blurhash": "L6Pj0^" },
188
+ # "created_at": "2025-02-11T09:30:00Z", "plan": "pro" }
189
+ # The v5 body is complete and conforming; only the v4 route sheds fields.
190
+ - id: field-observation-table
191
+ type: table
192
+ label: observed field presence by route
193
+ content: |
194
+ v4 response field maps from (v5) requires rename? required by v4? present for v4 clients?
195
+ id id no yes yes
196
+ user_name display_name yes yes NO
197
+ avatar_url avatar.url yes yes NO
198
+ created created_at yes yes NO
199
+ plan plan no (value remap) yes yes (value = "plus")
200
+
201
+ value-format check on the fields that DID arrive for v4 clients:
202
+ id -> uuid string, correct
203
+ plan -> "plus", correctly remapped from "pro" (mapValues ran successfully)
204
+
205
+ counts:
206
+ v4 required fields: 5 (id, user_name, avatar_url, created, plan)
207
+ v4 fields present in response: 2 (id, plan)
208
+ v4 fields missing: 3 (user_name, avatar_url, created)
209
+ of the 3 missing, all 3 are the fields that require a name change from v5
210
+ of the 2 present, neither requires a name change from v5
211
+ # Fingerprint: every field that needs a NAME change is absent; the two fields whose name is
212
+ # unchanged across versions (id, plan) are present. v5 clients receive the full v5 shape.
213
+ # (All rows describe /api/v4/profile responses observed during the incident.)
214
+ tags: [api, compatibility, versioning, gateway, multi-artifact-synthesis]
@@ -0,0 +1,190 @@
1
+ id: synthesis-permission-migration
2
+ version: 2.0.0
3
+ category: reasoning
4
+ cluster: multi-artifact-synthesis
5
+ difficulty: expert
6
+ title: Explicit-Grant Lockout Where the Backfill Ran but the Read Model Was Never Rebuilt
7
+ summary: A role-to-grants authorization migration locked out every legacy editor and viewer while admins kept working. The obvious story (backfill never ran, or enforcement shipped first) is contradicted by the artifacts because the backfill completed and the source grants exist. Reconcile both models, the authz check, the rollout plan, and the audit evidence to find the real gap, reject three decoys, and specify a recovery that restores access without broadening it.
8
+ prompt: |
9
+ After cutover to explicit grants, legacy editors and viewers are denied document.read while admins
10
+ are allowed. On-call's first guess was "enforcement shipped before the backfill." Using only the
11
+ artifacts, produce:
12
+
13
+ 1. ROOT CAUSE. Name the single gap and cite the exact artifacts and lines that prove it. Explain
14
+ why grants exist yet authorization still denies.
15
+ 2. REFUTE THE OBVIOUS STORY. Explain, with citations, why "the backfill never ran" and "enforcement
16
+ shipped before the backfill" are NOT the operative cause here.
17
+ 3. ADMIN VS EVERYONE ELSE. Explain why admin-01 is allowed while editor-17 and viewer-04 are denied,
18
+ even though all three have rows in the grants source table.
19
+ 4. DEAD-FLAG DECOY. Explain why `useLegacyRoleFallback` cannot rescue the locked-out accounts.
20
+ 5. BLAST RADIUS. State exactly which subjects are locked out and which are safe, tied to how each was
21
+ provisioned, and which operations are affected.
22
+ 6. MINIMAL RECOVERY. Give the smallest fix that restores access without granting anyone more than
23
+ their legacy role implied and without making role names authoritative again. State what must NOT
24
+ be done.
25
+
26
+ Exactly one resolution is defensible per deliverable; it is fully determined by the artifacts.
27
+ artifacts:
28
+ - id: legacy-model
29
+ type: spec
30
+ label: legacy role model (pre-migration)
31
+ content: |
32
+ Legacy role model (in force until cutover):
33
+ editor -> implicit document.read AND document.write
34
+ viewer -> implicit document.read
35
+ admin -> implicit all permissions
36
+ Authorization was derived directly from role membership; a check looked up the caller's role
37
+ and expanded it to the implied permission set at request time. There was no grants table, and
38
+ no per-subject permission rows existed anywhere. Role assignments live in a legacy table
39
+ `role_assignments(subject_id, role)`, which the backfill reads (see backfill-script).
40
+
41
+ Population at cutover (from role_assignments):
42
+ editors: 1812 (each should end with document.read + document.write = 3624 grant rows)
43
+ viewers: 455 (each should end with document.read = 455 grant rows)
44
+ Expected backfill output: 3624 + 455 = 4079 grant rows for editors and viewers (all via direct SQL).
45
+ Separately, 6 admins plus service accounts were provisioned through the grants API; those account
46
+ for the 41 rows currently visible in grant_view and are included in the 4120-row grants total.
47
+ The intent of the migration is to preserve EXACTLY these permissions as explicit grants and
48
+ nothing more (viewers must not gain write; editors must not gain admin).
49
+ - id: grant-model
50
+ type: spec
51
+ label: new explicit-grant model
52
+ content: |
53
+ New authorization model:
54
+ - All permissions are explicit rows in the `grants` table: (subject, permission, scope).
55
+ - The authorization service NEVER reads `grants` directly. It reads ONLY `grant_view`, a
56
+ materialized read model derived from `grants`.
57
+ - `grant_view` is maintained by a projector that reacts to writes made THROUGH the grants API.
58
+ A grants-API write updates `grants` AND emits a change event the projector consumes to
59
+ upsert `grant_view`.
60
+ - Direct SQL writes to `grants` do NOT emit a change event, so they do NOT appear in
61
+ `grant_view` until a projector rebuild (full re-projection from `grants`) is run.
62
+ - Role names (editor, viewer, admin) are retained for display only and confer no permissions.
63
+
64
+ Schema:
65
+ CREATE TABLE grants (subject text, permission text, scope text); -- source of truth
66
+ CREATE MATERIALIZED VIEW-LIKE grant_view (subject, permission, scope); -- projector output
67
+ Example rows AFTER a correct grants-API write for admin-01 (both tables in sync):
68
+ grants: (admin-01, document.read, global)
69
+ grant_view: (admin-01, document.read, global)
70
+ Example rows AFTER the direct-SQL backfill for editor-17 (source only, view missing):
71
+ grants: (editor-17, document.read, global), (editor-17, document.write, global)
72
+ grant_view: <no rows>
73
+ The projector's only trigger is the grants-API change event. A `rebuild` command re-projects the
74
+ entire `grants` table into `grant_view` from scratch; it is the supported way to reconcile rows
75
+ that were inserted out-of-band. Until it runs, out-of-band rows are invisible to authorization.
76
+ - id: authz-check
77
+ type: code
78
+ label: authorization check (enforcement, current)
79
+ content: |
80
+ // authz-service/src/check.ts (enforcement.mode = strict)
81
+ export async function can(subject, permission, scope) {
82
+ const rows = await db.query(
83
+ `SELECT 1 FROM grant_view
84
+ WHERE subject = $1 AND permission = $2 AND scope IN ('global', $3)
85
+ LIMIT 1`,
86
+ [subject, permission, scope],
87
+ );
88
+ return rows.length > 0; // deny if the read model has no matching grant
89
+ }
90
+
91
+ // Called by the document service on every read/write:
92
+ // if (!await can(user.id, 'document.read', doc.workspaceId)) throw Forbidden();
93
+ // NOTE: this query targets grant_view (the projected read model), NEVER the grants source
94
+ // table. There is no role lookup and no fallback path; an empty grant_view result = deny.
95
+ - id: deploy-plan
96
+ type: table
97
+ label: migration deployment plan and status
98
+ content: |
99
+ step action via owner status
100
+ 1 create grants table migration platform complete
101
+ 2 deploy grant_view projector (grants -> grant_view) deploy platform complete
102
+ 3 deploy enforcement (authz reads grant_view only) deploy authz complete
103
+ 4 bootstrap admin grants grants API authz complete
104
+ 5 backfill grants from legacy roles direct SQL data-eng complete
105
+ 6 rebuild grant_view from backfilled grants rows projector rerun data-eng NOT STARTED (scheduled)
106
+ # Steps 1-5 are all COMPLETE. Step 6 (the read-model rebuild after the direct-SQL backfill)
107
+ # was scheduled as a separate job and has not run. It is the only incomplete step.
108
+ - id: backfill-script
109
+ type: code
110
+ label: backfill script (deploy step 5)
111
+ content: |
112
+ // scripts/backfill-grants.ts (ran to completion in step 5)
113
+ // Reads legacy role_assignments and materializes explicit grants.
114
+ let inserted = 0;
115
+ for (const a of await legacy.roleAssignments()) { // SELECT subject_id, role FROM role_assignments
116
+ const perms =
117
+ a.role === 'editor' ? ['document.read', 'document.write'] :
118
+ a.role === 'viewer' ? ['document.read'] : [];
119
+ for (const p of perms) {
120
+ // Direct INSERT into the SOURCE table via the DB connection. This does NOT go through the
121
+ // grants API, so no change event is emitted and the grant_view projector is never notified.
122
+ await db.query(
123
+ `INSERT INTO grants (subject, permission, scope) VALUES ($1, $2, 'global')`,
124
+ [a.subjectId, p],
125
+ );
126
+ inserted++;
127
+ }
128
+ }
129
+ console.log(`backfill complete: inserted ${inserted} grants for legacy editors/viewers, 0 errors`);
130
+ // The script never calls grantsApi.create(); it writes the source table directly for speed.
131
+ - id: authz-audit
132
+ type: log
133
+ label: authorization decisions and ops reconciliation queries
134
+ content: |
135
+ # authorization path (reads grant_view) — sampled after cutover
136
+ editor-17 action=document.read source=grant_view rows=0 -> DENY role=editor
137
+ editor-17 action=document.write source=grant_view rows=0 -> DENY role=editor
138
+ editor-31 action=document.write source=grant_view rows=0 -> DENY role=editor
139
+ viewer-04 action=document.read source=grant_view rows=0 -> DENY role=viewer
140
+ admin-01 action=document.read source=grant_view rows=1 -> ALLOW role=admin
141
+ svc-88 action=document.read source=grant_view rows=1 -> ALLOW role=none
142
+
143
+ # ops reconciliation queries (run manually; NOT part of the authz path)
144
+ ops SELECT count(*) FROM grants WHERE subject='editor-17' -> 2 # read + write present in SOURCE
145
+ ops SELECT count(*) FROM grant_view WHERE subject='editor-17' -> 0 # but absent from the VIEW
146
+ ops SELECT count(*) FROM grants WHERE subject='editor-31' -> 2
147
+ ops SELECT count(*) FROM grant_view WHERE subject='editor-31' -> 0
148
+ ops SELECT count(*) FROM grants WHERE subject='viewer-04' -> 1
149
+ ops SELECT count(*) FROM grant_view WHERE subject='viewer-04' -> 0
150
+ ops SELECT count(*) FROM grants WHERE subject='admin-01' -> 1
151
+ ops SELECT count(*) FROM grant_view WHERE subject='admin-01' -> 1 # admin present in BOTH
152
+ ops SELECT count(*) FROM grants WHERE subject='svc-88' -> 3
153
+ ops SELECT count(*) FROM grant_view WHERE subject='svc-88' -> 3 # API-provisioned; in BOTH
154
+ # Aggregate: SELECT count(*) FROM grants -> 4120 ; SELECT count(*) FROM grant_view -> 41
155
+ # The 41 rows in grant_view are exactly the admin/service subjects provisioned via the grants API.
156
+ - id: migration-ticket
157
+ type: note
158
+ label: migration ticket and on-call notes
159
+ content: |
160
+ SEC-4400 Migration to explicit grants.
161
+ - Backfill (step 5) completed successfully: grants inserted for every legacy editor and viewer
162
+ (see the ops counts in authz-audit and store-state-table). The `grants` source table now
163
+ holds 4120 rows; only 41 are visible in `grant_view`.
164
+ - enforcement.mode = strict. A flag `useLegacyRoleFallback` still exists in the OLD gateway
165
+ config, but it was REMOVED from the authorization service during the enforcement deploy and
166
+ is now ignored; it can no longer re-derive permissions from role names. Setting it true has
167
+ no effect on `can()`.
168
+ - Rollback of enforcement to permissive was considered; it would allow ALL actions (fail-open),
169
+ so it is only acceptable as short-term containment, not as the fix.
170
+ - On-call first hypothesis: "enforcement shipped before the backfill (deploy-order inversion)."
171
+ Recorded at 02:10 before the ops queries returned. Second note at 02:40: "backfill IS done and
172
+ grants exist — deny persists anyway; revisit against deploy-plan and the source-vs-view counts."
173
+ - id: store-state-table
174
+ type: table
175
+ label: source-vs-view grant reconciliation
176
+ content: |
177
+ subject role grants_source_rows grant_view_rows provisioned_via authz_result
178
+ editor-17 editor 2 (read, write) 0 backfill (SQL) DENY
179
+ editor-31 editor 2 (read, write) 0 backfill (SQL) DENY
180
+ viewer-04 viewer 1 (read) 0 backfill (SQL) DENY
181
+ viewer-77 viewer 1 (read) 0 backfill (SQL) DENY
182
+ admin-01 admin 1 (all) 1 grants API (boot) ALLOW
183
+ admin-02 admin 1 (all) 1 grants API (boot) ALLOW
184
+ svc-88 none 3 (scoped) 3 grants API (new) ALLOW
185
+ # Every subject provisioned via direct-SQL backfill has source rows but 0 view rows and is denied.
186
+ # Every subject provisioned through the grants API has matching source and view rows and is allowed.
187
+ # svc-88 was created via the grants API after cutover and works, isolating the projector path.
188
+ # The ONLY variable that predicts DENY is provisioned_via = backfill (SQL); role is not predictive
189
+ # (both admins and editors have source rows; only the API-written ones reached grant_view).
190
+ tags: [authorization, migration, read-model, projection, multi-artifact-synthesis]