@evolvingmachines/sdk 0.0.51 → 0.0.52-launch-round-1.20260803.cb1be5b

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,4050 @@
1
+ openapi: 3.1.0
2
+
3
+ info:
4
+ title: Evolve Evaluations API
5
+ version: 2.0.0
6
+ summary: Hosted evaluations — datasets, jobs, trials, agents.
7
+ description: |
8
+ The single source of truth for the renamed hosted-evaluations surface.
9
+
10
+ ## Vocabulary
11
+ Four nouns, used everywhere and without exception:
12
+ - **dataset** — a named, versioned set of tasks.
13
+ - **job** — one run: datasets x agents x attempts.
14
+ - **trial** — one attempt of one task by one agent. Trial ids are globally
15
+ addressable: no route requires the job id to reach a trial.
16
+ - **agent** — the thing that attempts a task (a harness plus a model).
17
+ Registered private agents live under `/api/agents`.
18
+
19
+ Wire fields are `snake_case`, except the individually frozen historical
20
+ keys named in the deviations below — nothing else is exempt.
21
+
22
+ ## Documented deviations (kept deliberately)
23
+ - **Status enums.** `Job.status` (6 members) and `Trial.status` (8 members)
24
+ are load-bearing: they drive cancel, watch exit codes, resume and regrade
25
+ eligibility, and the `?status=` filters. `verifier_result` and
26
+ `exception_info` are published alongside, never instead.
27
+ - **Cursor pagination.** Every collection is `{items, nextCursor, hasMore}`
28
+ with opaque keyset cursors — these three keys are frozen verbatim.
29
+ Offset paging duplicates and skips rows on a live, growing collection.
30
+ - **Idempotency.** `Idempotency-Key` on every job-creating POST, surfaced
31
+ as `idempotent_replay` on the job body. A hosted double-run
32
+ double-charges a customer.
33
+ - **SSE events.** `GET /api/jobs/{jobId}/events` is the progress display
34
+ for a run that executes on someone else's machine. Event *type* names
35
+ are frozen; event payload fields speak this document's vocabulary.
36
+ - **Money.** Spend enforcement is ours: `max_trial_spend_usd` (the only
37
+ enforced budget — a per-trial cap minted onto the trial's gateway key),
38
+ `worst_case_spend_usd` (stated, never left to the caller to multiply),
39
+ and the live lower-bound pair `live_spent_usd` / `live_spend_at`.
40
+ Measured spend travels as `cost_usd` (job: `stats.cost_usd`; trial:
41
+ `agent_result.cost_usd`).
42
+ - **Typed error catalog.** A closed 52-code union; every error response is
43
+ one envelope: `{code, message, param, details, retryAfterSec,
44
+ request_id, requestId}` (envelope keys frozen verbatim; `requestId` is
45
+ the legacy spelling of `request_id`, identical value).
46
+ - **Request correlation.** Every response — success and error — carries an
47
+ `X-Request-Id` header: the value of a client-sent `X-Request-Id`
48
+ (printable ASCII, at most 128 chars) echoed back, else a server-minted
49
+ `req_<32 hex>`. Error envelopes repeat it as `request_id`.
50
+ - **Two frozen camelCase field keys.** `trials.byStatus` (the trial-status
51
+ tally on job bodies) and `taskMatrix` (on the compare response) keep
52
+ their exact historical spellings, same as the pagination and error
53
+ envelope keys above. These four spots are the complete list of
54
+ non-`snake_case` keys on the wire.
55
+ - **Cancel and stop.** `POST /api/jobs/{jobId}/cancel` and
56
+ `POST /api/trials/stop` exist because a hosted run cannot be Ctrl-C'd.
57
+ - **Resume semantics.** `POST /api/jobs/{jobId}/resume` creates a NEW
58
+ linked job (`source_jobs`), keeping the original immutable and
59
+ separately citable. It never mutates the source job.
60
+ - **Compare cap.** `GET /api/jobs/compare` takes 2–10 ids. Bounded, not
61
+ unbounded: one GET must not become an unbounded server-side fan-out.
62
+
63
+ ## Wave sequencing
64
+ Anything marked `x-wave: 2` belongs to the second build wave: today the
65
+ `trajectory` artifact selector (not yet served) and the API-key
66
+ management pair (served, not yet in the SDKs). Everything unmarked is
67
+ live end to end.
68
+
69
+ ## Deliberately absent (wave 3, not yet specified)
70
+ The rename map's additive rows are new capabilities, not renames, and are
71
+ intentionally NOT in this document: job sharing
72
+ (`GET /api/jobs/{jobId}/shares`, `POST /api/jobs/{jobId}/share`), copying
73
+ (`POST /api/jobs/{jobId}/copy`, `POST /api/trials/{trialId}/copy`), job
74
+ deletion (`DELETE /api/jobs/{jobId}`), dataset visibility
75
+ (`POST /api/datasets/{name}/visibility` — public/private toggling arrives
76
+ with the registry/sharing model), and the leaderboard family. They
77
+ will be specified when they are designed; their absence here is a
78
+ decision, not an omission.
79
+
80
+ servers:
81
+ - url: /
82
+ description: The platform origin. All paths are origin-relative.
83
+
84
+ security:
85
+ - apiKey: []
86
+
87
+ tags:
88
+ - name: jobs
89
+ description: Create, read, watch, cancel, resume, regrade, download, compare.
90
+ - name: trials
91
+ description: Globally addressable trials — read, trace, regrade, stop.
92
+ - name: datasets
93
+ description: Catalog, publish (async import), package download, activation.
94
+ - name: agents
95
+ description: Bring-your-own agents (private to their owner).
96
+ - name: meta
97
+ description: Public capability document.
98
+ - name: auth
99
+ description: Key and identity surface.
100
+
101
+ paths:
102
+ # ===========================================================================
103
+ # Jobs
104
+ # ===========================================================================
105
+
106
+ /api/jobs:
107
+ post:
108
+ tags: [jobs]
109
+ operationId: createJob
110
+ summary: Start a job
111
+ description: |
112
+ Starts a job over one or more catalog datasets. Accepted (202) as soon
113
+ as the trials are recorded; execution is asynchronous — follow
114
+ `GET /api/jobs/{jobId}/events` or poll `GET /api/jobs/{jobId}`.
115
+
116
+ Datasets are a LIST with per-dataset glob filters (`task_names`,
117
+ `exclude_task_names`, `n_tasks`); each selector filters only its own
118
+ dataset, and the job's trials are the union across the list. The same
119
+ dataset name twice is refused (`invalid_input`). Every agent must name
120
+ a model; the server applies no model default.
121
+
122
+ The response is the job body plus `job_id`, an additive alias of `id`
123
+ that only this verb answers with — see JobCreated.
124
+
125
+ Send `Idempotency-Key` to make retries safe: the same key with the
126
+ same resolved request replays the original job (200,
127
+ `idempotent_replay: true`); the same key with a different request is
128
+ refused with `idempotency_key_reused` (409). Dataset references are
129
+ fingerprinted RESOLVED — a bare name and the currently active version
130
+ fingerprint alike while that version is active. The dataset list is
131
+ fingerprinted IN ORDER: `datasets` echoes the request order back, so
132
+ the same selectors reordered are a different request and a shared key
133
+ across the two orders is refused rather than silently replayed.
134
+ parameters:
135
+ - $ref: '#/components/parameters/IdempotencyKey'
136
+ requestBody:
137
+ required: true
138
+ content:
139
+ application/json:
140
+ schema:
141
+ $ref: '#/components/schemas/JobCreate'
142
+ responses:
143
+ '202':
144
+ description: Job created and queued.
145
+ content:
146
+ application/json:
147
+ schema:
148
+ $ref: '#/components/schemas/JobCreated'
149
+ '200':
150
+ description: Idempotent replay of an existing identical request.
151
+ content:
152
+ application/json:
153
+ schema:
154
+ $ref: '#/components/schemas/JobCreated'
155
+ '400':
156
+ $ref: '#/components/responses/BadRequest'
157
+ '401':
158
+ $ref: '#/components/responses/Unauthorized'
159
+ '402':
160
+ description: Out of credits (`insufficient_credits`).
161
+ content:
162
+ application/json:
163
+ schema:
164
+ $ref: '#/components/schemas/Error'
165
+ '404':
166
+ $ref: '#/components/responses/NotFound'
167
+ '409':
168
+ $ref: '#/components/responses/Conflict'
169
+ '422':
170
+ description: >
171
+ Job too large (`job_too_large`) or a task refused by the chosen
172
+ provider (`provider_unsupported`).
173
+ content:
174
+ application/json:
175
+ schema:
176
+ $ref: '#/components/schemas/Error'
177
+ '429':
178
+ $ref: '#/components/responses/TooManyRequests'
179
+ '500':
180
+ $ref: '#/components/responses/InternalError'
181
+ get:
182
+ tags: [jobs]
183
+ operationId: listJobs
184
+ summary: List jobs
185
+ description: Newest first. Every filter is server-side.
186
+ parameters:
187
+ - $ref: '#/components/parameters/Limit'
188
+ - $ref: '#/components/parameters/Cursor'
189
+ - name: status
190
+ in: query
191
+ description: Filter by job status (repeatable).
192
+ schema:
193
+ type: array
194
+ items:
195
+ $ref: '#/components/schemas/JobStatus'
196
+ explode: true
197
+ - name: agent
198
+ in: query
199
+ description: Filter by agent name (repeatable).
200
+ schema:
201
+ type: array
202
+ items:
203
+ type: string
204
+ explode: true
205
+ - name: model
206
+ in: query
207
+ description: Filter by model name (repeatable).
208
+ schema:
209
+ type: array
210
+ items:
211
+ type: string
212
+ explode: true
213
+ - name: sandbox_provider
214
+ in: query
215
+ description: Filter by sandbox provider (repeatable).
216
+ schema:
217
+ type: array
218
+ items:
219
+ $ref: '#/components/schemas/SandboxProvider'
220
+ explode: true
221
+ - name: dataset
222
+ in: query
223
+ description: Filter to jobs that ran a given dataset name.
224
+ schema:
225
+ type: string
226
+ - name: is_regrade
227
+ in: query
228
+ description: >
229
+ true returns only regrade jobs; false only non-regrade jobs.
230
+ Replaces the retired standalone regrade listing — a regrade IS a
231
+ job.
232
+ schema:
233
+ type: boolean
234
+ - name: search
235
+ in: query
236
+ description: Free-text filter over job name and dataset names.
237
+ schema:
238
+ type: string
239
+ responses:
240
+ '200':
241
+ description: One page of jobs.
242
+ content:
243
+ application/json:
244
+ schema:
245
+ $ref: '#/components/schemas/JobPage'
246
+ '400':
247
+ $ref: '#/components/responses/BadRequest'
248
+ '401':
249
+ $ref: '#/components/responses/Unauthorized'
250
+ '429':
251
+ $ref: '#/components/responses/TooManyRequests'
252
+ '500':
253
+ $ref: '#/components/responses/InternalError'
254
+
255
+ /api/jobs/compare:
256
+ get:
257
+ tags: [jobs]
258
+ operationId: compareJobs
259
+ summary: Compare jobs
260
+ description: |
261
+ Side-by-side comparison of 2–10 owned jobs: per-job aggregates plus a
262
+ per-task matrix. Matrix cells report `MIXED` when a cell's trials
263
+ disagree in status and `MISSING` when a job has no trials for a task;
264
+ disagreement rows sort first.
265
+ parameters:
266
+ - name: ids
267
+ in: query
268
+ required: true
269
+ description: Comma-separated job ids (2–10).
270
+ schema:
271
+ type: string
272
+ examples: ["9f2c1a7e-...,4b8d0e21-..."]
273
+ responses:
274
+ '200':
275
+ description: Comparison payload.
276
+ content:
277
+ application/json:
278
+ schema:
279
+ $ref: '#/components/schemas/CompareResponse'
280
+ '400':
281
+ $ref: '#/components/responses/BadRequest'
282
+ '401':
283
+ $ref: '#/components/responses/Unauthorized'
284
+ '404':
285
+ $ref: '#/components/responses/NotFound'
286
+ '429':
287
+ $ref: '#/components/responses/TooManyRequests'
288
+ '500':
289
+ $ref: '#/components/responses/InternalError'
290
+
291
+ /api/jobs/{jobId}:
292
+ get:
293
+ tags: [jobs]
294
+ operationId: getJob
295
+ summary: Get one job
296
+ parameters:
297
+ - $ref: '#/components/parameters/JobId'
298
+ responses:
299
+ '200':
300
+ description: The job.
301
+ content:
302
+ application/json:
303
+ schema:
304
+ $ref: '#/components/schemas/Job'
305
+ '401':
306
+ $ref: '#/components/responses/Unauthorized'
307
+ '404':
308
+ $ref: '#/components/responses/NotFound'
309
+ '429':
310
+ $ref: '#/components/responses/TooManyRequests'
311
+ '500':
312
+ $ref: '#/components/responses/InternalError'
313
+
314
+ /api/jobs/{jobId}/cancel:
315
+ post:
316
+ tags: [jobs]
317
+ operationId: cancelJob
318
+ summary: Cancel a job
319
+ description: |
320
+ Requests cancellation: QUEUED/RUNNING moves to CANCELLING (compare-and-
321
+ set with retries), queued trials are cancelled outright, and in-flight
322
+ trials are STOPPED — sandboxes killed, spend read honestly — through
323
+ the same per-trial machinery as `POST /api/trials/stop` before the job
324
+ settles CANCELLED. Idempotent — calling it on a job already CANCELLING
325
+ or terminal returns the current body.
326
+ parameters:
327
+ - $ref: '#/components/parameters/JobId'
328
+ responses:
329
+ '200':
330
+ description: The job after the cancel request.
331
+ content:
332
+ application/json:
333
+ schema:
334
+ $ref: '#/components/schemas/Job'
335
+ '401':
336
+ $ref: '#/components/responses/Unauthorized'
337
+ '404':
338
+ $ref: '#/components/responses/NotFound'
339
+ '409':
340
+ description: Lost a concurrent status race after retries (`concurrent_update`).
341
+ content:
342
+ application/json:
343
+ schema:
344
+ $ref: '#/components/schemas/Error'
345
+ '429':
346
+ $ref: '#/components/responses/TooManyRequests'
347
+ '500':
348
+ $ref: '#/components/responses/InternalError'
349
+
350
+ /api/jobs/{jobId}/events:
351
+ get:
352
+ tags: [jobs]
353
+ operationId: watchJob
354
+ summary: Job event stream (SSE)
355
+ description: |
356
+ `text/event-stream` of the job's lifecycle. Each SSE message has
357
+ `id` = the event's monotonic `seq`, `event` = the event type, and
358
+ `data` = the JSON payload for that type.
359
+
360
+ Event types (frozen): `job.created`, `job.running`, `job.cancelling`,
361
+ `job.cancelled`, `job.completed`, `job.failed` (reserved — declared
362
+ terminal, no server path emits it today), `trial.running`,
363
+ `trial.scoring`, `trial.spend`, `trial.settled`.
364
+
365
+ Resume with the standard `Last-Event-ID` header or `?after=<seq>` —
366
+ both mean "events strictly after this sequence number". Heartbeat
367
+ comments (`:`) are sent every 15s; clients must skip comment lines.
368
+ parameters:
369
+ - $ref: '#/components/parameters/JobId'
370
+ - name: after
371
+ in: query
372
+ description: Resume position — deliver events with seq strictly greater.
373
+ schema:
374
+ type: integer
375
+ minimum: 0
376
+ - name: Last-Event-ID
377
+ in: header
378
+ description: Standard SSE resume header; same meaning as `?after=`.
379
+ schema:
380
+ type: string
381
+ responses:
382
+ '200':
383
+ description: The event stream. Ends after the terminal job event.
384
+ content:
385
+ text/event-stream:
386
+ schema:
387
+ $ref: '#/components/schemas/JobEvent'
388
+ '400':
389
+ description: Bad resume position (`invalid_after`).
390
+ content:
391
+ application/json:
392
+ schema:
393
+ $ref: '#/components/schemas/Error'
394
+ '401':
395
+ $ref: '#/components/responses/Unauthorized'
396
+ '404':
397
+ $ref: '#/components/responses/NotFound'
398
+ '429':
399
+ $ref: '#/components/responses/TooManyRequests'
400
+ '500':
401
+ $ref: '#/components/responses/InternalError'
402
+
403
+ /api/jobs/{jobId}/download:
404
+ get:
405
+ tags: [jobs]
406
+ operationId: downloadJob
407
+ summary: Download a job's results archive
408
+ description: |
409
+ A gzipped archive of the job in the standard results layout
410
+ (`config.json`, `result.json`, per-trial `config.json`/`result.json`).
411
+ Only terminal jobs are downloadable. The archive is deterministic:
412
+ sorted keys at every depth, no generation timestamp, zero gzip mtime.
413
+ Task environment identity only — instructions, verifier commands, and
414
+ test files never leave the server.
415
+ parameters:
416
+ - $ref: '#/components/parameters/JobId'
417
+ responses:
418
+ '200':
419
+ description: The archive.
420
+ content:
421
+ application/gzip:
422
+ schema:
423
+ type: string
424
+ format: binary
425
+ '401':
426
+ $ref: '#/components/responses/Unauthorized'
427
+ '404':
428
+ $ref: '#/components/responses/NotFound'
429
+ '409':
430
+ description: Job is not terminal (`job_not_terminal`).
431
+ content:
432
+ application/json:
433
+ schema:
434
+ $ref: '#/components/schemas/Error'
435
+ '429':
436
+ $ref: '#/components/responses/TooManyRequests'
437
+ '500':
438
+ $ref: '#/components/responses/InternalError'
439
+
440
+ /api/jobs/{jobId}/resume:
441
+ post:
442
+ tags: [jobs]
443
+ operationId: resumeJob
444
+ summary: Resume a job (new linked job over its failed and stopped trials)
445
+ description: |
446
+ Creates a NEW job holding fresh trials for the source job's failed
447
+ and stopped work, linked back via `source_jobs` (`action: "resume"`).
448
+ The source job is never mutated — the original run stays immutable and
449
+ separately citable.
450
+
451
+ `filter_error_types` selects which failures to resume by their
452
+ `exception_info.exception_type`; omitted, the default set is
453
+ `["ScoringError", "InfrastructureError", "IncompleteTrialError"]`
454
+ plus stopped trials (settled CANCELLED, exception type
455
+ `CancelledError`) and any still-QUEUED trials of a cancelled source.
456
+ The source job must be terminal.
457
+
458
+ Supports `Idempotency-Key` with a fingerprint namespaced to this
459
+ route, so a create key can never replay a resume.
460
+ parameters:
461
+ - $ref: '#/components/parameters/JobId'
462
+ - $ref: '#/components/parameters/IdempotencyKey'
463
+ requestBody:
464
+ required: false
465
+ content:
466
+ application/json:
467
+ schema:
468
+ $ref: '#/components/schemas/ResumeRequest'
469
+ responses:
470
+ '202':
471
+ description: The new linked job.
472
+ content:
473
+ application/json:
474
+ schema:
475
+ $ref: '#/components/schemas/Job'
476
+ '200':
477
+ description: Idempotent replay of an identical resume request.
478
+ content:
479
+ application/json:
480
+ schema:
481
+ $ref: '#/components/schemas/Job'
482
+ '400':
483
+ $ref: '#/components/responses/BadRequest'
484
+ '401':
485
+ $ref: '#/components/responses/Unauthorized'
486
+ '402':
487
+ description: Out of credits (`insufficient_credits`).
488
+ content:
489
+ application/json:
490
+ schema:
491
+ $ref: '#/components/schemas/Error'
492
+ '404':
493
+ $ref: '#/components/responses/NotFound'
494
+ '409':
495
+ description: >
496
+ Source not terminal (`job_not_terminal`), nothing to resume
497
+ (`no_failed_trials`), or `idempotency_key_reused`.
498
+ content:
499
+ application/json:
500
+ schema:
501
+ $ref: '#/components/schemas/Error'
502
+ '429':
503
+ $ref: '#/components/responses/TooManyRequests'
504
+ '500':
505
+ $ref: '#/components/responses/InternalError'
506
+
507
+ /api/jobs/{jobId}/regrade:
508
+ post:
509
+ tags: [jobs]
510
+ operationId: regradeJob
511
+ summary: Regrade a job (verifier-only re-run)
512
+ description: |
513
+ Re-runs verification over a terminal job's trials. **The response is a
514
+ Job**: a regrade is an ordinary job whose `source_jobs` records
515
+ `action: "regrade"` and whose `is_regrade` is true. View it with
516
+ `GET /api/jobs/{jobId}`; there is no separate regrade resource.
517
+ The source job and its trials are immutable.
518
+
519
+ The optional filter narrows which trials are regraded. Decision order:
520
+ 404 (unknown job) → 409 `job_not_terminal` → 409 `no_regradable_trials`
521
+ → 202.
522
+ parameters:
523
+ - $ref: '#/components/parameters/JobId'
524
+ requestBody:
525
+ required: false
526
+ content:
527
+ application/json:
528
+ schema:
529
+ $ref: '#/components/schemas/RegradeRequest'
530
+ responses:
531
+ '202':
532
+ description: The regrade job.
533
+ content:
534
+ application/json:
535
+ schema:
536
+ $ref: '#/components/schemas/Job'
537
+ '400':
538
+ $ref: '#/components/responses/BadRequest'
539
+ '401':
540
+ $ref: '#/components/responses/Unauthorized'
541
+ '404':
542
+ $ref: '#/components/responses/NotFound'
543
+ '409':
544
+ description: >
545
+ `job_not_terminal`, `no_regradable_trials`, or
546
+ `regrade_source_ineligible` (a pre-cutover job with no inputs
547
+ bundle can never be regraded).
548
+ content:
549
+ application/json:
550
+ schema:
551
+ $ref: '#/components/schemas/Error'
552
+ '429':
553
+ $ref: '#/components/responses/TooManyRequests'
554
+ '500':
555
+ $ref: '#/components/responses/InternalError'
556
+
557
+ /api/jobs/{jobId}/trials:
558
+ get:
559
+ tags: [jobs]
560
+ operationId: listJobTrials
561
+ summary: List a job's trials
562
+ description: |
563
+ One page of the job's trials. List rows truncate
564
+ `exception_info.exception_message` to 2000 characters; the trial
565
+ detail route returns it untruncated. Reading a page refreshes the
566
+ live-spend lower bound on in-flight trials.
567
+ parameters:
568
+ - $ref: '#/components/parameters/JobId'
569
+ - $ref: '#/components/parameters/Limit'
570
+ - $ref: '#/components/parameters/Cursor'
571
+ - name: status
572
+ in: query
573
+ description: Filter by trial status — repeatable and/or comma-separated.
574
+ schema:
575
+ type: array
576
+ items:
577
+ $ref: '#/components/schemas/TrialStatus'
578
+ explode: true
579
+ - name: task_name
580
+ in: query
581
+ description: Filter to one task's trials.
582
+ schema:
583
+ type: string
584
+ - name: dataset
585
+ in: query
586
+ description: >
587
+ Filter to one dataset's trials — exact match on the trial's
588
+ `source`. A name the job does not span is an ordinary empty
589
+ selection.
590
+ schema:
591
+ type: string
592
+ - name: failed_only
593
+ in: query
594
+ description: Only trials in a failure status.
595
+ schema:
596
+ type: boolean
597
+ responses:
598
+ '200':
599
+ description: One page of trials.
600
+ content:
601
+ application/json:
602
+ schema:
603
+ $ref: '#/components/schemas/TrialPage'
604
+ '400':
605
+ $ref: '#/components/responses/BadRequest'
606
+ '401':
607
+ $ref: '#/components/responses/Unauthorized'
608
+ '404':
609
+ $ref: '#/components/responses/NotFound'
610
+ '429':
611
+ $ref: '#/components/responses/TooManyRequests'
612
+ '500':
613
+ $ref: '#/components/responses/InternalError'
614
+
615
+ /api/jobs/{jobId}/tasks:
616
+ get:
617
+ tags: [jobs]
618
+ operationId: listJobTasks
619
+ summary: Per-task rollup of a job
620
+ description: |
621
+ One row per distinct task: its trial-status histogram, mean reward
622
+ over SCORED trials, and measured cost. Sits between the job body and
623
+ the trial list so a caller need not fetch every trial to see which
624
+ tasks are dragging.
625
+ parameters:
626
+ - $ref: '#/components/parameters/JobId'
627
+ - $ref: '#/components/parameters/Limit'
628
+ - $ref: '#/components/parameters/Cursor'
629
+ responses:
630
+ '200':
631
+ description: One page of task rollups.
632
+ content:
633
+ application/json:
634
+ schema:
635
+ $ref: '#/components/schemas/JobTaskRollupPage'
636
+ '400':
637
+ $ref: '#/components/responses/BadRequest'
638
+ '401':
639
+ $ref: '#/components/responses/Unauthorized'
640
+ '404':
641
+ $ref: '#/components/responses/NotFound'
642
+ '429':
643
+ $ref: '#/components/responses/TooManyRequests'
644
+ '500':
645
+ $ref: '#/components/responses/InternalError'
646
+
647
+ # ===========================================================================
648
+ # Trials (globally addressable)
649
+ # ===========================================================================
650
+
651
+ /api/trials/{trialId}:
652
+ get:
653
+ tags: [trials]
654
+ operationId: getTrial
655
+ summary: Get one trial
656
+ description: >
657
+ A trial id is globally addressable — no job id in the path. The body
658
+ carries `job_id` for the reverse pointer. Detail responses return
659
+ `exception_info` untruncated.
660
+ parameters:
661
+ - $ref: '#/components/parameters/TrialId'
662
+ responses:
663
+ '200':
664
+ description: The trial.
665
+ content:
666
+ application/json:
667
+ schema:
668
+ $ref: '#/components/schemas/Trial'
669
+ '401':
670
+ $ref: '#/components/responses/Unauthorized'
671
+ '404':
672
+ $ref: '#/components/responses/NotFound'
673
+ '429':
674
+ $ref: '#/components/responses/TooManyRequests'
675
+ '500':
676
+ $ref: '#/components/responses/InternalError'
677
+
678
+ /api/trials/{trialId}/trace:
679
+ get:
680
+ tags: [trials]
681
+ operationId: getTrialTrace
682
+ summary: Trial trace and artifacts
683
+ description: |
684
+ Without `stream` (or with `stream=trace-parsed`, the same answer by
685
+ its vocabulary name): one page of parsed trace events, paged by
686
+ `cursor` (a sequence number) and `limit` (default 200, max 1000).
687
+
688
+ With `stream`: raw artifacts instead of parsed events —
689
+ - `verifier`, `trace-stdout`, `trace-stderr`, `trajectory` (wave 2):
690
+ the log text as JSON (`{"log": string | null}`; null means this
691
+ trial never stored that log — a normal answer, never an error).
692
+ With `format=log` a present log is delivered as a `text/plain`
693
+ attachment; an absent one still answers the JSON `{"log": null}`.
694
+ - `agent-home`: the agent home tree as JSON
695
+ (`{"files": {"<sandbox-path>": "<text>", ...} | null}`; null means
696
+ nothing was collected). With `format=tgz` a present tree is
697
+ delivered as a gzipped tarball; an absent one still answers the
698
+ JSON `{"files": null}`.
699
+ parameters:
700
+ - $ref: '#/components/parameters/TrialId'
701
+ - name: cursor
702
+ in: query
703
+ description: Sequence position to page from (parsed-event mode).
704
+ schema:
705
+ type: string
706
+ - name: limit
707
+ in: query
708
+ description: Page size (parsed-event mode). Default 200, max 1000.
709
+ schema:
710
+ type: integer
711
+ minimum: 1
712
+ maximum: 1000
713
+ default: 200
714
+ - name: stream
715
+ in: query
716
+ description: >
717
+ Raw artifact selector. `trajectory` is in the vocabulary ahead of
718
+ its server wave: until that wave lands the route answers
719
+ not-found for it, reported as the API error it is.
720
+ schema:
721
+ type: string
722
+ enum:
723
+ - trace-parsed # the parsed event trace — same answer as omitting
724
+ - verifier
725
+ - trace-stdout
726
+ - trace-stderr
727
+ - trajectory # x-wave: 2 — normalized-trajectory artifact slot
728
+ - agent-home
729
+ - name: format
730
+ in: query
731
+ description: >
732
+ `log` (text/plain, log selectors only) or `tgz` (gzipped tarball,
733
+ agent-home only).
734
+ schema:
735
+ type: string
736
+ enum: [log, tgz]
737
+ responses:
738
+ '200':
739
+ description: Trace events, or the selected artifact.
740
+ content:
741
+ application/json:
742
+ schema:
743
+ oneOf:
744
+ - $ref: '#/components/schemas/TraceEventPage'
745
+ - type: object
746
+ description: >
747
+ Log selector JSON form; `log` is null when this trial
748
+ never stored that log — a normal answer, never an error.
749
+ required: [log]
750
+ properties:
751
+ log:
752
+ type: [string, 'null']
753
+ - type: object
754
+ description: >
755
+ Agent-home JSON form — sandbox path to file text; `files`
756
+ is null when nothing was collected, which no other shape
757
+ can express.
758
+ required: [files]
759
+ properties:
760
+ files:
761
+ type: [object, 'null']
762
+ additionalProperties:
763
+ type: string
764
+ text/plain:
765
+ schema:
766
+ type: string
767
+ description: '`format=log` delivery of a log selector.'
768
+ application/gzip:
769
+ schema:
770
+ type: string
771
+ format: binary
772
+ description: '`format=tgz` delivery of agent-home.'
773
+ '400':
774
+ $ref: '#/components/responses/BadRequest'
775
+ '401':
776
+ $ref: '#/components/responses/Unauthorized'
777
+ '404':
778
+ $ref: '#/components/responses/NotFound'
779
+ '429':
780
+ $ref: '#/components/responses/TooManyRequests'
781
+ '500':
782
+ $ref: '#/components/responses/InternalError'
783
+
784
+ /api/trials/{trialId}/regrade:
785
+ post:
786
+ tags: [trials]
787
+ operationId: regradeTrial
788
+ summary: Regrade one trial
789
+ description: |
790
+ Verifier-only re-run of a single trial. **The response is a Job** — a
791
+ one-trial regrade job with `source_jobs` recording the provenance.
792
+ The source trial is immutable.
793
+ parameters:
794
+ - $ref: '#/components/parameters/TrialId'
795
+ responses:
796
+ '202':
797
+ description: The regrade job.
798
+ content:
799
+ application/json:
800
+ schema:
801
+ $ref: '#/components/schemas/Job'
802
+ '401':
803
+ $ref: '#/components/responses/Unauthorized'
804
+ '404':
805
+ $ref: '#/components/responses/NotFound'
806
+ '409':
807
+ description: >
808
+ `job_not_terminal`, `no_regradable_trials`, or
809
+ `regrade_source_ineligible`.
810
+ content:
811
+ application/json:
812
+ schema:
813
+ $ref: '#/components/schemas/Error'
814
+ '429':
815
+ $ref: '#/components/responses/TooManyRequests'
816
+ '500':
817
+ $ref: '#/components/responses/InternalError'
818
+
819
+ /api/trials/stop:
820
+ post:
821
+ tags: [trials]
822
+ operationId: stopTrials
823
+ summary: Stop selected running trials
824
+ description: |
825
+ Stops individual in-flight trials without cancelling their job: each
826
+ trial's sandbox is killed and the trial is settled with its spend read
827
+ from the gateway. Only the caller's own trials; ids belonging to
828
+ someone else are reported in `not_found` (existence is never leaked).
829
+ Idempotent — already-terminal trials are reported as such and left
830
+ untouched.
831
+ requestBody:
832
+ required: true
833
+ content:
834
+ application/json:
835
+ schema:
836
+ $ref: '#/components/schemas/StopRequest'
837
+ responses:
838
+ '200':
839
+ description: Per-trial outcome of the stop request.
840
+ content:
841
+ application/json:
842
+ schema:
843
+ $ref: '#/components/schemas/StopResponse'
844
+ '400':
845
+ $ref: '#/components/responses/BadRequest'
846
+ '401':
847
+ $ref: '#/components/responses/Unauthorized'
848
+ '429':
849
+ $ref: '#/components/responses/TooManyRequests'
850
+ '500':
851
+ $ref: '#/components/responses/InternalError'
852
+
853
+ # ===========================================================================
854
+ # Datasets
855
+ # ===========================================================================
856
+
857
+ /api/datasets:
858
+ get:
859
+ tags: [datasets]
860
+ operationId: listDatasets
861
+ summary: List datasets
862
+ description: >
863
+ The catalog: public datasets plus the caller's own private ones.
864
+ Another user's private dataset is invisible, never 403. A dataset
865
+ whose every version is FAILED is not listed either — no partial
866
+ corpus ever exists, and that includes the catalog view. The name
867
+ still answers `GET /api/datasets/{name}` (and DELETE), and the
868
+ failed import stays readable at its import id, so nothing about the
869
+ failure is hidden; re-publishing the same name simply retries it.
870
+ parameters:
871
+ - $ref: '#/components/parameters/Limit'
872
+ - $ref: '#/components/parameters/Cursor'
873
+ - name: search
874
+ in: query
875
+ description: Free-text filter over name and description.
876
+ schema:
877
+ type: string
878
+ responses:
879
+ '200':
880
+ description: One page of datasets (summary fields only).
881
+ content:
882
+ application/json:
883
+ schema:
884
+ $ref: '#/components/schemas/DatasetPage'
885
+ '400':
886
+ $ref: '#/components/responses/BadRequest'
887
+ '401':
888
+ $ref: '#/components/responses/Unauthorized'
889
+ '429':
890
+ $ref: '#/components/responses/TooManyRequests'
891
+ '500':
892
+ $ref: '#/components/responses/InternalError'
893
+
894
+ /api/datasets/{name}:
895
+ get:
896
+ tags: [datasets]
897
+ operationId: getDataset
898
+ summary: Get one dataset
899
+ description: >
900
+ Detail view: all versions, the active version, upstream status, and
901
+ one page of the selected version's tasks (`limit`/`cursor` page the
902
+ task list; `version` selects which version's tasks are listed).
903
+ parameters:
904
+ - $ref: '#/components/parameters/DatasetName'
905
+ - name: version
906
+ in: query
907
+ description: Version whose tasks to list; default the active version.
908
+ schema:
909
+ type: string
910
+ - $ref: '#/components/parameters/Limit'
911
+ - $ref: '#/components/parameters/Cursor'
912
+ responses:
913
+ '200':
914
+ description: The dataset.
915
+ content:
916
+ application/json:
917
+ schema:
918
+ $ref: '#/components/schemas/Dataset'
919
+ '400':
920
+ $ref: '#/components/responses/BadRequest'
921
+ '401':
922
+ $ref: '#/components/responses/Unauthorized'
923
+ '404':
924
+ $ref: '#/components/responses/NotFound'
925
+ '429':
926
+ $ref: '#/components/responses/TooManyRequests'
927
+ '500':
928
+ $ref: '#/components/responses/InternalError'
929
+ patch:
930
+ tags: [datasets]
931
+ operationId: updateDataset
932
+ summary: Update dataset settings
933
+ description: |
934
+ The only settable field is `upstream_auto_import`: automatically
935
+ import a new version when the dataset's upstream git ref moves.
936
+ Refused with `upstream_not_watchable` (409) when the dataset has no
937
+ moving git ref to follow, and 403 on platform-curated datasets.
938
+ parameters:
939
+ - $ref: '#/components/parameters/DatasetName'
940
+ requestBody:
941
+ required: true
942
+ content:
943
+ application/json:
944
+ schema:
945
+ $ref: '#/components/schemas/DatasetPatch'
946
+ responses:
947
+ '200':
948
+ description: The updated dataset.
949
+ content:
950
+ application/json:
951
+ schema:
952
+ $ref: '#/components/schemas/Dataset'
953
+ '400':
954
+ $ref: '#/components/responses/BadRequest'
955
+ '401':
956
+ $ref: '#/components/responses/Unauthorized'
957
+ '403':
958
+ description: Platform-curated dataset (`dataset_not_owned`).
959
+ content:
960
+ application/json:
961
+ schema:
962
+ $ref: '#/components/schemas/Error'
963
+ '404':
964
+ $ref: '#/components/responses/NotFound'
965
+ '409':
966
+ description: Nothing to watch (`upstream_not_watchable`).
967
+ content:
968
+ application/json:
969
+ schema:
970
+ $ref: '#/components/schemas/Error'
971
+ '429':
972
+ $ref: '#/components/responses/TooManyRequests'
973
+ '500':
974
+ $ref: '#/components/responses/InternalError'
975
+ delete:
976
+ tags: [datasets]
977
+ operationId: deleteDataset
978
+ summary: Delete an owned dataset
979
+ parameters:
980
+ - $ref: '#/components/parameters/DatasetName'
981
+ responses:
982
+ '204':
983
+ description: Deleted.
984
+ '401':
985
+ $ref: '#/components/responses/Unauthorized'
986
+ '403':
987
+ description: Not the owner (`dataset_not_owned`).
988
+ content:
989
+ application/json:
990
+ schema:
991
+ $ref: '#/components/schemas/Error'
992
+ '404':
993
+ $ref: '#/components/responses/NotFound'
994
+ '409':
995
+ description: >
996
+ In use by jobs (`dataset_in_use`); `details` names the blocking
997
+ job ids.
998
+ content:
999
+ application/json:
1000
+ schema:
1001
+ $ref: '#/components/schemas/Error'
1002
+ '429':
1003
+ $ref: '#/components/responses/TooManyRequests'
1004
+ '500':
1005
+ $ref: '#/components/responses/InternalError'
1006
+
1007
+ /api/datasets/{name}/download:
1008
+ get:
1009
+ tags: [datasets]
1010
+ operationId: downloadDataset
1011
+ summary: Download a dataset version's source package
1012
+ description: |
1013
+ The original uploaded corpus tarball for one of the caller's own
1014
+ dataset versions (owner only). The server verifies the stored sha256
1015
+ before streaming and echoes it in `x-package-sha256`; clients should
1016
+ verify the digest and the Content-Length.
1017
+ parameters:
1018
+ - $ref: '#/components/parameters/DatasetName'
1019
+ - name: version
1020
+ in: query
1021
+ description: Version to download; default the active version.
1022
+ schema:
1023
+ type: string
1024
+ responses:
1025
+ '200':
1026
+ description: The gzipped source package.
1027
+ headers:
1028
+ x-package-sha256:
1029
+ description: Hex sha256 of the exact bytes streamed.
1030
+ schema:
1031
+ type: string
1032
+ pattern: '^[a-f0-9]{64}$'
1033
+ content:
1034
+ application/gzip:
1035
+ schema:
1036
+ type: string
1037
+ format: binary
1038
+ '401':
1039
+ $ref: '#/components/responses/Unauthorized'
1040
+ '404':
1041
+ $ref: '#/components/responses/NotFound'
1042
+ '409':
1043
+ description: >
1044
+ Package unavailable: `package_not_retained`, `package_corrupt`,
1045
+ or `package_missing`.
1046
+ content:
1047
+ application/json:
1048
+ schema:
1049
+ $ref: '#/components/schemas/Error'
1050
+ '429':
1051
+ $ref: '#/components/responses/TooManyRequests'
1052
+ '500':
1053
+ $ref: '#/components/responses/InternalError'
1054
+
1055
+ /api/datasets/{name}/versions/{version}/activate:
1056
+ post:
1057
+ tags: [datasets]
1058
+ operationId: activateDatasetVersion
1059
+ summary: Activate a dataset version
1060
+ description: |
1061
+ Promotes a READY (or VALIDATING-complete) version to the dataset's
1062
+ active version, making bare-name job references resolve to it.
1063
+
1064
+ You usually do not need this. A version you publish is activated for
1065
+ you the moment its activation gate passes — VALIDATING becomes READY
1066
+ and the dataset's default points at it, with no call. What this verb
1067
+ is for is choosing a DIFFERENT default: rolling back to an older
1068
+ READY version, or re-pointing at one auto-activation moved past.
1069
+
1070
+ The proof it requires is the same activation gate, which the worker
1071
+ schedules automatically after a hosted import: the gold solution
1072
+ scores exactly 1.0 through the real execution path, and a do-nothing
1073
+ agent does not. While the gate is scheduled or running the verb
1074
+ answers 202 `gate_running` (a healthy in-progress state, not an
1075
+ error); once it lands, activation either promotes (200) or explains
1076
+ (409 `version_not_ready`, with the gate's failure detail in
1077
+ `details.gate_failure`). Refused with `version_not_activatable` when
1078
+ the version can never be activated (for example, no reference
1079
+ solutions were archived — see the import's `warnings`).
1080
+
1081
+ Idempotent: activating the version that is already active rewrites
1082
+ the same pointer and answers 200.
1083
+ parameters:
1084
+ - $ref: '#/components/parameters/DatasetName'
1085
+ - name: version
1086
+ in: path
1087
+ required: true
1088
+ schema:
1089
+ type: string
1090
+ responses:
1091
+ '200':
1092
+ description: The dataset with its new active version.
1093
+ content:
1094
+ application/json:
1095
+ schema:
1096
+ $ref: '#/components/schemas/Dataset'
1097
+ '202':
1098
+ description: >
1099
+ Not yet — the activation gate is scheduled or running for this
1100
+ version. Nothing is wrong and nothing is asked of the caller:
1101
+ poll the dataset (each version carries `gate`). A gate that
1102
+ PASSES activates the version itself, so there is normally
1103
+ nothing left to call.
1104
+ content:
1105
+ application/json:
1106
+ schema:
1107
+ $ref: '#/components/schemas/GateRunning'
1108
+ '401':
1109
+ $ref: '#/components/responses/Unauthorized'
1110
+ '403':
1111
+ description: Not the owner (`dataset_not_owned`).
1112
+ content:
1113
+ application/json:
1114
+ schema:
1115
+ $ref: '#/components/schemas/Error'
1116
+ '404':
1117
+ $ref: '#/components/responses/NotFound'
1118
+ '409':
1119
+ description: '`version_not_ready` or `version_not_activatable`.'
1120
+ content:
1121
+ application/json:
1122
+ schema:
1123
+ $ref: '#/components/schemas/Error'
1124
+ '429':
1125
+ $ref: '#/components/responses/TooManyRequests'
1126
+ '500':
1127
+ $ref: '#/components/responses/InternalError'
1128
+
1129
+ /api/datasets/publish:
1130
+ post:
1131
+ tags: [datasets]
1132
+ operationId: publishDataset
1133
+ summary: Publish a dataset version (async import)
1134
+ description: |
1135
+ Starts a server-side import that creates or extends a catalog dataset
1136
+ with one new immutable version. Multipart only. Source is exactly one
1137
+ of: a git repository (`git_url` + pinned `git_ref` — https only, an
1138
+ unpinned import is not reproducible) or an uploaded gzipped tarball
1139
+ (`archive`).
1140
+
1141
+ The import is asynchronous: the 202 returns a DatasetImport to poll at
1142
+ `GET /api/datasets/imports/{importId}`. Name collisions are checked
1143
+ before the upload lands; oversized uploads are fast-rejected on
1144
+ Content-Length (413). The name `imports` is reserved (it is a literal
1145
+ path segment under `/api/datasets`) and is refused like any other
1146
+ invalid name.
1147
+
1148
+ Once the import COMPLETES, the worker automatically schedules the
1149
+ activation gate (the version's `gate` field tracks it: PENDING ->
1150
+ RUNNING -> PASSED/FAILED). When it PASSES, the version becomes READY
1151
+ and the dataset's active version on its own — publish, poll until
1152
+ `gate.status` is PASSED, and run the dataset by bare name. A version
1153
+ with no archived reference solutions gets no gate — its import
1154
+ `warnings` say so, and activation is refused
1155
+ `version_not_activatable`.
1156
+ requestBody:
1157
+ required: true
1158
+ content:
1159
+ multipart/form-data:
1160
+ schema:
1161
+ $ref: '#/components/schemas/PublishRequest'
1162
+ responses:
1163
+ '202':
1164
+ description: The import job.
1165
+ content:
1166
+ application/json:
1167
+ schema:
1168
+ $ref: '#/components/schemas/DatasetImport'
1169
+ '400':
1170
+ $ref: '#/components/responses/BadRequest'
1171
+ '401':
1172
+ $ref: '#/components/responses/Unauthorized'
1173
+ '409':
1174
+ description: '`dataset_name_taken` or `dataset_version_not_found` conflicts.'
1175
+ content:
1176
+ application/json:
1177
+ schema:
1178
+ $ref: '#/components/schemas/Error'
1179
+ '413':
1180
+ description: Archive over the size cap (`import_too_large`).
1181
+ content:
1182
+ application/json:
1183
+ schema:
1184
+ $ref: '#/components/schemas/Error'
1185
+ '429':
1186
+ $ref: '#/components/responses/TooManyRequests'
1187
+ '500':
1188
+ $ref: '#/components/responses/InternalError'
1189
+
1190
+ /api/datasets/imports:
1191
+ get:
1192
+ tags: [datasets]
1193
+ operationId: listDatasetImports
1194
+ summary: List the caller's imports
1195
+ parameters:
1196
+ - $ref: '#/components/parameters/Limit'
1197
+ - $ref: '#/components/parameters/Cursor'
1198
+ - name: status
1199
+ in: query
1200
+ schema:
1201
+ $ref: '#/components/schemas/DatasetImportStatus'
1202
+ - name: dataset
1203
+ in: query
1204
+ description: Filter to imports of one dataset name.
1205
+ schema:
1206
+ type: string
1207
+ responses:
1208
+ '200':
1209
+ description: One page of imports.
1210
+ content:
1211
+ application/json:
1212
+ schema:
1213
+ $ref: '#/components/schemas/DatasetImportPage'
1214
+ '400':
1215
+ $ref: '#/components/responses/BadRequest'
1216
+ '401':
1217
+ $ref: '#/components/responses/Unauthorized'
1218
+ '429':
1219
+ $ref: '#/components/responses/TooManyRequests'
1220
+ '500':
1221
+ $ref: '#/components/responses/InternalError'
1222
+
1223
+ /api/datasets/imports/{importId}:
1224
+ get:
1225
+ tags: [datasets]
1226
+ operationId: getDatasetImport
1227
+ summary: Poll one import
1228
+ description: >
1229
+ Imports of public datasets are pollable by anyone; private ones are
1230
+ owner-only and read 404 (never 403) for anyone else.
1231
+ parameters:
1232
+ - name: importId
1233
+ in: path
1234
+ required: true
1235
+ schema:
1236
+ type: string
1237
+ responses:
1238
+ '200':
1239
+ description: The import.
1240
+ content:
1241
+ application/json:
1242
+ schema:
1243
+ $ref: '#/components/schemas/DatasetImport'
1244
+ '401':
1245
+ $ref: '#/components/responses/Unauthorized'
1246
+ '404':
1247
+ $ref: '#/components/responses/NotFound'
1248
+ '429':
1249
+ $ref: '#/components/responses/TooManyRequests'
1250
+ '500':
1251
+ $ref: '#/components/responses/InternalError'
1252
+
1253
+ # ===========================================================================
1254
+ # Agents (bring-your-own)
1255
+ # ===========================================================================
1256
+
1257
+ /api/agents:
1258
+ post:
1259
+ tags: [agents]
1260
+ operationId: registerAgent
1261
+ summary: Register a private agent
1262
+ description: |
1263
+ Registers a bring-your-own agent. Once registered, its `name` is
1264
+ usable in `agents[].name` on job creation exactly like a built-in.
1265
+ Source is exactly one of: an `install_script` (runs in a throwaway
1266
+ builder sandbox with internet and ZERO secrets; must leave executables
1267
+ in `$PREFIX/bin`) or an uploaded gzipped tarball (`archive`).
1268
+ Multipart only — nothing travels in the query string.
1269
+
1270
+ Registration validates SHAPE only and never executes the install
1271
+ script: the bundle is built by the first trial that names the agent,
1272
+ and a failing build settles that trial as an infrastructure error
1273
+ with the builder's stdout/stderr stored on it (readable through the
1274
+ trial's `trace-stdout` / `trace-stderr` streams). Line endings in
1275
+ `install_script` and `run_command` are normalized CRLF -> LF on
1276
+ receipt — multipart transports rewrite text-field newlines to CRLF,
1277
+ which is never meaningful in shell text.
1278
+ requestBody:
1279
+ required: true
1280
+ content:
1281
+ multipart/form-data:
1282
+ schema:
1283
+ $ref: '#/components/schemas/AgentRegistration'
1284
+ responses:
1285
+ '201':
1286
+ description: The registered agent.
1287
+ content:
1288
+ application/json:
1289
+ schema:
1290
+ $ref: '#/components/schemas/Agent'
1291
+ '400':
1292
+ $ref: '#/components/responses/BadRequest'
1293
+ '401':
1294
+ $ref: '#/components/responses/Unauthorized'
1295
+ '409':
1296
+ description: >
1297
+ `agent_name_taken`, `agent_name_reserved`, `agent_source_conflict`,
1298
+ or `agent_limit_reached`.
1299
+ content:
1300
+ application/json:
1301
+ schema:
1302
+ $ref: '#/components/schemas/Error'
1303
+ '413':
1304
+ description: Upload over the size cap (`agent_too_large`).
1305
+ content:
1306
+ application/json:
1307
+ schema:
1308
+ $ref: '#/components/schemas/Error'
1309
+ '429':
1310
+ $ref: '#/components/responses/TooManyRequests'
1311
+ '500':
1312
+ $ref: '#/components/responses/InternalError'
1313
+ get:
1314
+ tags: [agents]
1315
+ operationId: listAgents
1316
+ summary: List the caller's registered agents
1317
+ parameters:
1318
+ - $ref: '#/components/parameters/Limit'
1319
+ - $ref: '#/components/parameters/Cursor'
1320
+ responses:
1321
+ '200':
1322
+ description: One page of agents.
1323
+ content:
1324
+ application/json:
1325
+ schema:
1326
+ $ref: '#/components/schemas/AgentPage'
1327
+ '400':
1328
+ $ref: '#/components/responses/BadRequest'
1329
+ '401':
1330
+ $ref: '#/components/responses/Unauthorized'
1331
+ '429':
1332
+ $ref: '#/components/responses/TooManyRequests'
1333
+ '500':
1334
+ $ref: '#/components/responses/InternalError'
1335
+
1336
+ /api/agents/{name}:
1337
+ get:
1338
+ tags: [agents]
1339
+ operationId: getAgent
1340
+ summary: Get one registered agent
1341
+ description: >
1342
+ Private to its owner — another user's agent name reads
1343
+ `agent_not_found`, never a permission error.
1344
+ parameters:
1345
+ - $ref: '#/components/parameters/AgentName'
1346
+ responses:
1347
+ '200':
1348
+ description: The agent.
1349
+ content:
1350
+ application/json:
1351
+ schema:
1352
+ $ref: '#/components/schemas/Agent'
1353
+ '401':
1354
+ $ref: '#/components/responses/Unauthorized'
1355
+ '404':
1356
+ $ref: '#/components/responses/NotFound'
1357
+ '429':
1358
+ $ref: '#/components/responses/TooManyRequests'
1359
+ '500':
1360
+ $ref: '#/components/responses/InternalError'
1361
+ put:
1362
+ tags: [agents]
1363
+ operationId: upsertAgent
1364
+ summary: Create or replace a registered agent
1365
+ parameters:
1366
+ - $ref: '#/components/parameters/AgentName'
1367
+ requestBody:
1368
+ required: true
1369
+ content:
1370
+ multipart/form-data:
1371
+ schema:
1372
+ $ref: '#/components/schemas/AgentRegistration'
1373
+ responses:
1374
+ '200':
1375
+ description: The agent after the upsert.
1376
+ content:
1377
+ application/json:
1378
+ schema:
1379
+ $ref: '#/components/schemas/Agent'
1380
+ '400':
1381
+ $ref: '#/components/responses/BadRequest'
1382
+ '401':
1383
+ $ref: '#/components/responses/Unauthorized'
1384
+ '409':
1385
+ description: '`agent_name_reserved` or `agent_source_conflict`.'
1386
+ content:
1387
+ application/json:
1388
+ schema:
1389
+ $ref: '#/components/schemas/Error'
1390
+ '413':
1391
+ description: Upload over the size cap (`agent_too_large`).
1392
+ content:
1393
+ application/json:
1394
+ schema:
1395
+ $ref: '#/components/schemas/Error'
1396
+ '429':
1397
+ $ref: '#/components/responses/TooManyRequests'
1398
+ '500':
1399
+ $ref: '#/components/responses/InternalError'
1400
+ delete:
1401
+ tags: [agents]
1402
+ operationId: deleteAgent
1403
+ summary: Delete a registered agent
1404
+ parameters:
1405
+ - $ref: '#/components/parameters/AgentName'
1406
+ responses:
1407
+ '204':
1408
+ description: Deleted.
1409
+ '401':
1410
+ $ref: '#/components/responses/Unauthorized'
1411
+ '404':
1412
+ $ref: '#/components/responses/NotFound'
1413
+ '429':
1414
+ $ref: '#/components/responses/TooManyRequests'
1415
+ '500':
1416
+ $ref: '#/components/responses/InternalError'
1417
+
1418
+ # ===========================================================================
1419
+ # Meta
1420
+ # ===========================================================================
1421
+
1422
+ /api/meta:
1423
+ get:
1424
+ tags: [meta]
1425
+ operationId: getMeta
1426
+ summary: Capability document (no auth)
1427
+ security: []
1428
+ description: >
1429
+ Everything a client would otherwise hardcode: agents (built-in and
1430
+ registration rules), sandbox providers with sizing ceilings and
1431
+ refusals, platform constraints, network modes, status vocabularies
1432
+ with their terminal members, limits, and the closed error-code list.
1433
+ Public and cacheable — weak ETag, 304 support, `max-age=300`.
1434
+ parameters:
1435
+ - name: If-None-Match
1436
+ in: header
1437
+ schema:
1438
+ type: string
1439
+ responses:
1440
+ '200':
1441
+ description: The capability document.
1442
+ headers:
1443
+ ETag:
1444
+ schema:
1445
+ type: string
1446
+ Cache-Control:
1447
+ schema:
1448
+ type: string
1449
+ content:
1450
+ application/json:
1451
+ schema:
1452
+ $ref: '#/components/schemas/CapabilityDocument'
1453
+ '304':
1454
+ description: Not modified.
1455
+ '500':
1456
+ $ref: '#/components/responses/InternalError'
1457
+
1458
+ # ===========================================================================
1459
+ # Auth
1460
+ # ===========================================================================
1461
+
1462
+ /api/auth/status:
1463
+ get:
1464
+ tags: [auth]
1465
+ operationId: getAuthStatus
1466
+ summary: Who am I
1467
+ description: >
1468
+ Identifies the caller and the API key used, so `auth status` in
1469
+ clients has a server answer.
1470
+ responses:
1471
+ '200':
1472
+ description: The caller's identity.
1473
+ content:
1474
+ application/json:
1475
+ schema:
1476
+ $ref: '#/components/schemas/AuthStatus'
1477
+ '401':
1478
+ $ref: '#/components/responses/Unauthorized'
1479
+ '429':
1480
+ $ref: '#/components/responses/TooManyRequests'
1481
+ '500':
1482
+ $ref: '#/components/responses/InternalError'
1483
+
1484
+ /api/auth/keys:
1485
+ get:
1486
+ tags: [auth]
1487
+ operationId: listApiKeys
1488
+ summary: List the caller's API keys
1489
+ x-wave: 2
1490
+ description: Secrets are never returned.
1491
+ parameters:
1492
+ - $ref: '#/components/parameters/Limit'
1493
+ - $ref: '#/components/parameters/Cursor'
1494
+ responses:
1495
+ '200':
1496
+ description: One page of key descriptors.
1497
+ content:
1498
+ application/json:
1499
+ schema:
1500
+ $ref: '#/components/schemas/ApiKeyPage'
1501
+ '400':
1502
+ $ref: '#/components/responses/BadRequest'
1503
+ '401':
1504
+ $ref: '#/components/responses/Unauthorized'
1505
+ '429':
1506
+ $ref: '#/components/responses/TooManyRequests'
1507
+ '500':
1508
+ $ref: '#/components/responses/InternalError'
1509
+
1510
+ /api/auth/keys/{keyId}:
1511
+ delete:
1512
+ tags: [auth]
1513
+ operationId: revokeApiKey
1514
+ summary: Revoke an API key
1515
+ x-wave: 2
1516
+ description: Revocation is immediate.
1517
+ parameters:
1518
+ - name: keyId
1519
+ in: path
1520
+ required: true
1521
+ schema:
1522
+ type: string
1523
+ responses:
1524
+ '204':
1525
+ description: Revoked.
1526
+ '401':
1527
+ $ref: '#/components/responses/Unauthorized'
1528
+ '404':
1529
+ $ref: '#/components/responses/NotFound'
1530
+ '429':
1531
+ $ref: '#/components/responses/TooManyRequests'
1532
+ '500':
1533
+ $ref: '#/components/responses/InternalError'
1534
+
1535
+ # =============================================================================
1536
+ # Components
1537
+ # =============================================================================
1538
+
1539
+ components:
1540
+ securitySchemes:
1541
+ apiKey:
1542
+ type: http
1543
+ scheme: bearer
1544
+ description: >
1545
+ `Authorization: Bearer <API key>`. Required on everything except
1546
+ `GET /api/meta`.
1547
+
1548
+ parameters:
1549
+ JobId:
1550
+ name: jobId
1551
+ in: path
1552
+ required: true
1553
+ schema:
1554
+ type: string
1555
+ format: uuid
1556
+ TrialId:
1557
+ name: trialId
1558
+ in: path
1559
+ required: true
1560
+ schema:
1561
+ type: string
1562
+ format: uuid
1563
+ DatasetName:
1564
+ name: name
1565
+ in: path
1566
+ required: true
1567
+ description: >
1568
+ A dataset's catalog name. `imports` is a reserved name — it is a
1569
+ literal path segment under `/api/datasets` — and publish refuses it,
1570
+ so `/api/datasets/imports/{importId}` can never collide with a
1571
+ dataset route.
1572
+ schema:
1573
+ type: string
1574
+ AgentName:
1575
+ name: name
1576
+ in: path
1577
+ required: true
1578
+ schema:
1579
+ type: string
1580
+ Limit:
1581
+ name: limit
1582
+ in: query
1583
+ description: Page size. Collection default 50, max 200 (dataset tasks 200/500).
1584
+ schema:
1585
+ type: integer
1586
+ minimum: 1
1587
+ Cursor:
1588
+ name: cursor
1589
+ in: query
1590
+ description: >
1591
+ Opaque keyset cursor from a previous page's `nextCursor`. Never
1592
+ constructed by the client.
1593
+ schema:
1594
+ type: string
1595
+ IdempotencyKey:
1596
+ name: Idempotency-Key
1597
+ in: header
1598
+ required: false
1599
+ description: |
1600
+ Client-chosen key that makes the POST safe to retry. The same key
1601
+ with the same resolved request returns the original resource (200,
1602
+ `idempotent_replay: true`); the same key with a different request is
1603
+ refused with `idempotency_key_reused` (409). Keys are namespaced per
1604
+ route and per caller.
1605
+ schema:
1606
+ type: string
1607
+ maxLength: 200
1608
+
1609
+ responses:
1610
+ BadRequest:
1611
+ description: >
1612
+ Malformed request — `invalid_json`, `invalid_input`, `invalid_limit`,
1613
+ `invalid_status`, `invalid_cursor`, `invalid_format`, `invalid_ids`,
1614
+ `invalid_multipart`, `invalid_archive`, `unknown_task_names`, or
1615
+ `no_tasks`.
1616
+ content:
1617
+ application/json:
1618
+ schema:
1619
+ $ref: '#/components/schemas/Error'
1620
+ Unauthorized:
1621
+ description: >
1622
+ `missing_authorization`, `invalid_api_key`, or (503-adjacent, still
1623
+ this envelope) `credential_service_unavailable`.
1624
+ content:
1625
+ application/json:
1626
+ schema:
1627
+ $ref: '#/components/schemas/Error'
1628
+ NotFound:
1629
+ description: >
1630
+ The addressed resource does not exist or is not visible to the
1631
+ caller. Existence is never leaked: someone else's private resource
1632
+ reads 404, not 403.
1633
+ content:
1634
+ application/json:
1635
+ schema:
1636
+ $ref: '#/components/schemas/Error'
1637
+ Conflict:
1638
+ description: State conflict; the `code` names which law was hit.
1639
+ content:
1640
+ application/json:
1641
+ schema:
1642
+ $ref: '#/components/schemas/Error'
1643
+ TooManyRequests:
1644
+ description: '`rate_limited`. Honor `Retry-After`.'
1645
+ headers:
1646
+ Retry-After:
1647
+ schema:
1648
+ type: integer
1649
+ X-RateLimit-Limit:
1650
+ schema:
1651
+ type: integer
1652
+ X-RateLimit-Remaining:
1653
+ schema:
1654
+ type: integer
1655
+ X-RateLimit-Reset:
1656
+ schema:
1657
+ type: integer
1658
+ description: Epoch seconds when the window resets.
1659
+ content:
1660
+ application/json:
1661
+ schema:
1662
+ $ref: '#/components/schemas/Error'
1663
+ InternalError:
1664
+ description: '`internal_error`; carries `requestId` for support.'
1665
+ content:
1666
+ application/json:
1667
+ schema:
1668
+ $ref: '#/components/schemas/Error'
1669
+
1670
+ schemas:
1671
+ # -------------------------------------------------------------------------
1672
+ # Errors
1673
+ # -------------------------------------------------------------------------
1674
+
1675
+ ErrorCode:
1676
+ type: string
1677
+ description: |
1678
+ The closed error-code union (52 members). Published at runtime by
1679
+ `GET /api/meta` (`error_codes`); no route ever answers with a code
1680
+ outside this list.
1681
+ enum:
1682
+ # Auth + throttling
1683
+ - missing_authorization
1684
+ - invalid_api_key
1685
+ - credential_service_unavailable
1686
+ - rate_limited
1687
+ - insufficient_credits
1688
+ # Request shape
1689
+ - invalid_json
1690
+ - invalid_input
1691
+ - invalid_limit
1692
+ - invalid_status
1693
+ - invalid_cursor
1694
+ - invalid_after
1695
+ - invalid_format
1696
+ - invalid_ids
1697
+ - invalid_multipart
1698
+ - idempotency_key_reused
1699
+ # Datasets
1700
+ - dataset_not_found
1701
+ - dataset_version_not_found
1702
+ - dataset_name_taken
1703
+ - dataset_in_use
1704
+ - dataset_not_owned
1705
+ - upstream_not_watchable
1706
+ - no_active_version
1707
+ - version_not_ready
1708
+ - version_not_activatable
1709
+ - unknown_task_names
1710
+ - no_tasks
1711
+ # Registered agents
1712
+ - agent_not_found
1713
+ - agent_name_taken
1714
+ - agent_name_reserved
1715
+ - agent_invalid_name
1716
+ - agent_source_required
1717
+ - agent_source_conflict
1718
+ - agent_invalid_env
1719
+ - agent_too_large
1720
+ - agent_limit_reached
1721
+ # Jobs + trials
1722
+ - agent_version_not_found
1723
+ - job_too_large
1724
+ - provider_unsupported
1725
+ - job_not_found
1726
+ - job_not_terminal
1727
+ - no_failed_trials
1728
+ - trial_not_found
1729
+ - concurrent_update
1730
+ # Regrade
1731
+ - regrade_source_ineligible
1732
+ - no_regradable_trials
1733
+ # Imports
1734
+ - import_not_found
1735
+ - import_too_large
1736
+ - invalid_archive
1737
+ # Retained source package (owner-only download)
1738
+ - package_not_retained
1739
+ - package_corrupt
1740
+ - package_missing
1741
+ # Fallback
1742
+ - internal_error
1743
+
1744
+ Error:
1745
+ type: object
1746
+ description: |
1747
+ The ONE error envelope, under the top-level key `error`. Envelope keys
1748
+ are frozen verbatim (documented deviation). `details` is open — each
1749
+ code carries the machine-readable shape its own refusal needs (a list
1750
+ of task names, a provider and per-task reasons, a ceiling and the
1751
+ value that breached it, the blocking job ids).
1752
+ required: [error]
1753
+ properties:
1754
+ error:
1755
+ type: object
1756
+ required: [code, message]
1757
+ properties:
1758
+ code:
1759
+ $ref: '#/components/schemas/ErrorCode'
1760
+ message:
1761
+ type: string
1762
+ description: Human-readable; never parse it — parse `code`.
1763
+ param:
1764
+ type: [string, 'null']
1765
+ description: The input field the caller got wrong, in the caller's vocabulary.
1766
+ details:
1767
+ type: [object, 'null']
1768
+ additionalProperties: true
1769
+ retryAfterSec:
1770
+ type: [integer, 'null']
1771
+ description: Present on retryable refusals; mirrors the Retry-After header.
1772
+ request_id:
1773
+ type: string
1774
+ description: |
1775
+ The per-request correlation id — identical to the response's
1776
+ `X-Request-Id` header (every response on this surface carries
1777
+ one; a client-sent `X-Request-Id` is echoed, otherwise the
1778
+ server mints `req_<32 hex>`). Quote it in support requests.
1779
+ requestId:
1780
+ type: string
1781
+ description: Legacy spelling of `request_id`, identical value.
1782
+
1783
+ # -------------------------------------------------------------------------
1784
+ # Shared enums and small shapes
1785
+ # -------------------------------------------------------------------------
1786
+
1787
+ JobStatus:
1788
+ type: string
1789
+ description: >
1790
+ Job lifecycle (kept deviation — load-bearing for cancel, watch exit,
1791
+ download gating, and resume/regrade eligibility). Terminal: COMPLETED,
1792
+ CANCELLED, FAILED.
1793
+ enum: [QUEUED, RUNNING, CANCELLING, COMPLETED, CANCELLED, FAILED]
1794
+
1795
+ TrialStatus:
1796
+ type: string
1797
+ description: |
1798
+ Trial lifecycle (kept deviation). The status law: a valid reward
1799
+ (including 0) = SCORED; a verifier crash or out-of-domain reward =
1800
+ SCORING_ERROR (never a fabricated zero); INFRASTRUCTURE_ERROR = the
1801
+ trial was lost before a result was recorded; INDETERMINATE = the
1802
+ platform cannot tell whether the trial completed. Terminal: SCORED,
1803
+ SCORING_ERROR, INFRASTRUCTURE_ERROR, INDETERMINATE, CANCELLED.
1804
+ enum:
1805
+ - QUEUED
1806
+ - RUNNING
1807
+ - SCORING
1808
+ - SCORED
1809
+ - SCORING_ERROR
1810
+ - INFRASTRUCTURE_ERROR
1811
+ - INDETERMINATE
1812
+ - CANCELLED
1813
+
1814
+ SandboxProvider:
1815
+ type: string
1816
+ description: Where trials execute. Spelled identically on every surface.
1817
+ enum: [e2b, daytona, modal]
1818
+
1819
+ SpendSource:
1820
+ type: string
1821
+ description: >
1822
+ Which lane a settled trial's `agent_result.cost_usd` is in. `measured`
1823
+ is final. The other two are PRE-CONFIRMATION lanes a freshly settled
1824
+ trial commonly holds for a few minutes, served verbatim because the
1825
+ lane is the only signal that the figure is still provisional:
1826
+ `measured_provisional` is an honest floor read inside the gateway's
1827
+ async spend flush (it can only be raised), and `assumed_cap` means
1828
+ nothing was measured yet — it holds a zero placeholder, never an
1829
+ invented figure. A deferred pass confirms or raises both from the
1830
+ gateway's spend logs.
1831
+ enum: [measured, measured_provisional, assumed_cap]
1832
+
1833
+ VerifierEnvironmentMode:
1834
+ type: string
1835
+ description: >
1836
+ Where the verifier ran — inside the agent's environment (shared) or
1837
+ in a separate one.
1838
+ enum: [shared, separate]
1839
+
1840
+ AttemptPhase:
1841
+ type: string
1842
+ description: >
1843
+ Which step a RUNNING trial is in, so a polling caller can tell a slow
1844
+ build from a slow agent — RUNNING alone cannot.
1845
+ enum: [prepare, build, boot, install, agent, verify, persist]
1846
+
1847
+ TrialStatusTally:
1848
+ type: object
1849
+ description: >
1850
+ The one "how many" shape: a total plus a histogram over EVERY trial
1851
+ status, zeros included, so a UI draws its status bar off the response
1852
+ instead of hardcoding the enum.
1853
+ required: [total, byStatus]
1854
+ properties:
1855
+ total:
1856
+ type: integer
1857
+ minimum: 0
1858
+ byStatus:
1859
+ type: object
1860
+ description: One integer per TrialStatus member, zeros included.
1861
+ required:
1862
+ - QUEUED
1863
+ - RUNNING
1864
+ - SCORING
1865
+ - SCORED
1866
+ - SCORING_ERROR
1867
+ - INFRASTRUCTURE_ERROR
1868
+ - INDETERMINATE
1869
+ - CANCELLED
1870
+ properties:
1871
+ QUEUED: { type: integer, minimum: 0 }
1872
+ RUNNING: { type: integer, minimum: 0 }
1873
+ SCORING: { type: integer, minimum: 0 }
1874
+ SCORED: { type: integer, minimum: 0 }
1875
+ SCORING_ERROR: { type: integer, minimum: 0 }
1876
+ INFRASTRUCTURE_ERROR: { type: integer, minimum: 0 }
1877
+ INDETERMINATE: { type: integer, minimum: 0 }
1878
+ CANCELLED: { type: integer, minimum: 0 }
1879
+
1880
+ # -------------------------------------------------------------------------
1881
+ # Job — config side
1882
+ # -------------------------------------------------------------------------
1883
+
1884
+ DatasetRef:
1885
+ type: object
1886
+ description: A resolved dataset reference as echoed on job bodies.
1887
+ required: [name, version]
1888
+ properties:
1889
+ name:
1890
+ type: string
1891
+ version:
1892
+ type: string
1893
+
1894
+ DatasetSelector:
1895
+ type: object
1896
+ description: |
1897
+ One dataset a job runs, with per-dataset task filters. `task_names`
1898
+ and `exclude_task_names` are glob patterns; `n_tasks` caps the task
1899
+ count AFTER filtering. A bare `name` resolves to the active version
1900
+ (`no_active_version` when none).
1901
+ required: [name]
1902
+ properties:
1903
+ name:
1904
+ type: string
1905
+ description: Catalog dataset name.
1906
+ version:
1907
+ type: string
1908
+ description: Pin a version; omitted, the active version is used.
1909
+ task_names:
1910
+ type: array
1911
+ description: Include filter — glob patterns over task names.
1912
+ items:
1913
+ type: string
1914
+ exclude_task_names:
1915
+ type: array
1916
+ description: Exclude filter — glob patterns over task names.
1917
+ items:
1918
+ type: string
1919
+ n_tasks:
1920
+ type: integer
1921
+ minimum: 1
1922
+ description: Cap the task count after filters are applied.
1923
+
1924
+ AgentArmInput:
1925
+ type: object
1926
+ description: >
1927
+ One agent arm of a job: an agent (built-in or registered) plus a
1928
+ model. A model is always required; the server applies no default.
1929
+ required: [name, model_name]
1930
+ properties:
1931
+ name:
1932
+ type: string
1933
+ description: Agent name — a built-in or one registered under /api/agents.
1934
+ model_name:
1935
+ type: string
1936
+ version:
1937
+ type: string
1938
+ description: >
1939
+ Pin an agent version; omitted, the platform resolves the latest
1940
+ supported (`agent_version_not_found` when a pin cannot resolve).
1941
+ reasoning_effort:
1942
+ type: string
1943
+ description: >
1944
+ Platform extension — declared effort, part of arm identity.
1945
+ Accepted values published by /api/meta; omitted, the runner
1946
+ default applies.
1947
+
1948
+ AgentArm:
1949
+ type: object
1950
+ description: One agent arm as echoed on job bodies.
1951
+ required: [name, model_name, version, reasoning_effort]
1952
+ properties:
1953
+ name:
1954
+ type: string
1955
+ model_name:
1956
+ type: string
1957
+ version:
1958
+ type: [string, 'null']
1959
+ description: The requested pin; null when the run took the latest.
1960
+ reasoning_effort:
1961
+ type: [string, 'null']
1962
+ description: Declared effort; null when the run took the runner default.
1963
+
1964
+ SourceJob:
1965
+ type: object
1966
+ description: >
1967
+ Provenance of a derived job. `action: regrade` = verifier-only re-run
1968
+ of the source; `action: resume` (platform extension) = new job over
1969
+ the source's failed and stopped trials. `type` is always `hub` on
1970
+ this hosted surface.
1971
+ required: [action, type, job_id]
1972
+ properties:
1973
+ action:
1974
+ type: string
1975
+ enum: [regrade, resume]
1976
+ type:
1977
+ type: string
1978
+ enum: [hub]
1979
+ job_id:
1980
+ type: string
1981
+ format: uuid
1982
+
1983
+ JobCreate:
1984
+ type: object
1985
+ description: The job-creation body.
1986
+ required: [datasets, agents]
1987
+ properties:
1988
+ job_name:
1989
+ type: string
1990
+ maxLength: 200
1991
+ description: >
1992
+ User-facing label; server-generated when omitted. A LABEL, not an
1993
+ identity: names are not unique, and two jobs may carry the same
1994
+ one (Harbor's job name is a local directory and resumes in place;
1995
+ a hosted registry keeps `id` as the only identity). Reuse a name
1996
+ freely across runs; to prevent accidental double submission use
1997
+ Idempotency-Key, which replays by request, not by name.
1998
+ datasets:
1999
+ type: array
2000
+ minItems: 1
2001
+ items:
2002
+ $ref: '#/components/schemas/DatasetSelector'
2003
+ agents:
2004
+ type: array
2005
+ minItems: 1
2006
+ maxItems: 8
2007
+ items:
2008
+ $ref: '#/components/schemas/AgentArmInput'
2009
+ n_attempts:
2010
+ type: integer
2011
+ minimum: 1
2012
+ maximum: 100
2013
+ default: 1
2014
+ description: Attempts per task per agent arm.
2015
+ n_concurrent_trials:
2016
+ type: integer
2017
+ minimum: 1
2018
+ maximum: 16
2019
+ default: 4
2020
+ description: Parallel trials across the job.
2021
+ max_trial_spend_usd:
2022
+ type: number
2023
+ minimum: 0
2024
+ description: |
2025
+ Per-trial spend cap in USD, minted onto each trial's gateway key —
2026
+ the platform's ONLY spend enforcement (there is no job-wide
2027
+ budget). Default published by /api/meta (200 unless the operator
2028
+ tuned it).
2029
+ sandbox_provider:
2030
+ $ref: '#/components/schemas/SandboxProvider'
2031
+ agent_env:
2032
+ type: object
2033
+ additionalProperties:
2034
+ type: string
2035
+ description: >
2036
+ Env injected into every agent run — a pass-through slot: the
2037
+ client sends it verbatim and the server owns acceptance (refused
2038
+ where unsupported, never silently dropped).
2039
+ verifier_env:
2040
+ type: object
2041
+ additionalProperties:
2042
+ type: string
2043
+ description: >
2044
+ Env injected into every verifier run — same pass-through
2045
+ contract.
2046
+
2047
+ ResumeRequest:
2048
+ type: object
2049
+ description: Body of POST /api/jobs/{jobId}/resume.
2050
+ properties:
2051
+ filter_error_types:
2052
+ type: array
2053
+ description: |
2054
+ Which failures to resume, matched against
2055
+ `exception_info.exception_type`. Omitted, the default set is
2056
+ `["ScoringError", "InfrastructureError", "IncompleteTrialError"]`
2057
+ plus stopped trials (settled CANCELLED, exception type
2058
+ `CancelledError`) and still-QUEUED trials of a cancelled source.
2059
+ items:
2060
+ type: string
2061
+
2062
+ RegradeRequest:
2063
+ type: object
2064
+ description: >
2065
+ Optional filter narrowing which trials a job-level regrade re-runs.
2066
+ Omitted, every regradable trial is regraded.
2067
+ properties:
2068
+ statuses:
2069
+ type: array
2070
+ items:
2071
+ $ref: '#/components/schemas/TrialStatus'
2072
+ task_name:
2073
+ type: string
2074
+ description: Restrict to one task's trials.
2075
+
2076
+ # -------------------------------------------------------------------------
2077
+ # Job — result side
2078
+ # -------------------------------------------------------------------------
2079
+
2080
+ AgentDatasetStats:
2081
+ type: object
2082
+ description: |
2083
+ Per-(agent, model, dataset) statistics. The evals key format is
2084
+ `{agent}__{model}__{dataset}` — the dataset ref is always the LAST
2085
+ `__` segment, which is where Harbor-compatible readers recover it —
2086
+ with the platform extension of an `__{effort}` segment inserted
2087
+ BEFORE the dataset when a declared reasoning effort is part of the
2088
+ arm identity: `{agent}__{model}__{effort}__{dataset}`.
2089
+ properties:
2090
+ n_trials:
2091
+ type: integer
2092
+ minimum: 0
2093
+ description: Trials that produced a rewards map — rewarded, not merely settled.
2094
+ n_errors:
2095
+ type: integer
2096
+ minimum: 0
2097
+ description: >
2098
+ Trials carrying `exception_info` — indeterminate and cancelled
2099
+ included.
2100
+ metrics:
2101
+ type: array
2102
+ description: >
2103
+ Metric results (a mean entry per arm today: the primary reward
2104
+ averaged over EVERY trial of the group, unrewarded trials
2105
+ counting 0); open objects so new metric types need no wire
2106
+ change.
2107
+ items:
2108
+ type: object
2109
+ additionalProperties: true
2110
+ pass_at_k:
2111
+ type: object
2112
+ description: >
2113
+ pass@k slot — keys are k as strings, values in [0,1]. Present and
2114
+ empty until the platform computes it; the slot exists so adding
2115
+ the statistic is not a wire change.
2116
+ additionalProperties:
2117
+ type: number
2118
+ reward_stats:
2119
+ type: object
2120
+ description: 'reward key -> reward value -> trial identifiers.'
2121
+ additionalProperties:
2122
+ type: object
2123
+ additionalProperties:
2124
+ type: array
2125
+ items:
2126
+ type: string
2127
+ exception_stats:
2128
+ type: object
2129
+ description: 'exception type -> trial identifiers.'
2130
+ additionalProperties:
2131
+ type: array
2132
+ items:
2133
+ type: string
2134
+
2135
+ JobStats:
2136
+ type: object
2137
+ description: |
2138
+ Aggregate statistics of a job. Progress counters, token totals, and
2139
+ measured cost. The `n_*` counters are CUMULATIVE, Harbor-style:
2140
+ errored trials are a subset of completed, cancelled a subset of
2141
+ errored — a cancelled trial counts in all three. The disjoint
2142
+ per-status breakdown rides `Job.trials.byStatus`.
2143
+ `cost_usd` is what the trials actually spent so far — reporting,
2144
+ never a gate (enforcement is the per-trial cap).
2145
+ properties:
2146
+ n_completed_trials:
2147
+ type: integer
2148
+ minimum: 0
2149
+ description: >
2150
+ Cumulative: every trial that produced a result — errored and
2151
+ cancelled included, never the scored subset alone.
2152
+ n_errored_trials:
2153
+ type: integer
2154
+ minimum: 0
2155
+ description: >
2156
+ Cumulative: every completed trial carrying `exception_info`,
2157
+ cancelled included. A subset of `n_completed_trials`.
2158
+ n_running_trials:
2159
+ type: integer
2160
+ minimum: 0
2161
+ n_pending_trials:
2162
+ type: integer
2163
+ minimum: 0
2164
+ n_cancelled_trials:
2165
+ type: integer
2166
+ minimum: 0
2167
+ description: A subset of `n_errored_trials`.
2168
+ n_retries:
2169
+ type: integer
2170
+ minimum: 0
2171
+ evals:
2172
+ type: object
2173
+ description: >
2174
+ Keyed `{agent}__{model}__{dataset}` — dataset ref last, with the
2175
+ optional effort segment inserted before it.
2176
+ additionalProperties:
2177
+ $ref: '#/components/schemas/AgentDatasetStats'
2178
+ n_input_tokens:
2179
+ type: [integer, 'null']
2180
+ description: Total input tokens (cache included); null until recorded.
2181
+ n_cache_tokens:
2182
+ type: [integer, 'null']
2183
+ n_output_tokens:
2184
+ type: [integer, 'null']
2185
+ cost_usd:
2186
+ type: [number, 'null']
2187
+ description: Measured spend across settled trials; null before any settled.
2188
+
2189
+ JobFailure:
2190
+ type: object
2191
+ description: >
2192
+ Why a job FAILED — deliberately NOT under the key `error`, which on
2193
+ this surface always means "this request failed". `if (body.error)
2194
+ throw` stays correct on a healthy 200 read of a failed job.
2195
+ required: [code, message]
2196
+ properties:
2197
+ code:
2198
+ type: string
2199
+ description: '`job_execution_failed` when the runner recorded no code.'
2200
+ message:
2201
+ type: string
2202
+
2203
+ Job:
2204
+ type: object
2205
+ description: |
2206
+ THE job body — the same shape from create, get, list items, cancel,
2207
+ resume, and regrade responses; no field appears on some responses and
2208
+ not others. The create verb answers with JobCreated, which is this
2209
+ shape plus one aliased id and nothing else.
2210
+ required:
2211
+ - id
2212
+ - job_name
2213
+ - status
2214
+ - datasets
2215
+ - agents
2216
+ - n_attempts
2217
+ - n_concurrent_trials
2218
+ - max_trial_spend_usd
2219
+ - worst_case_spend_usd
2220
+ - sandbox_provider
2221
+ - counts
2222
+ - n_total_trials
2223
+ - trials
2224
+ - stats
2225
+ - failure
2226
+ - source_jobs
2227
+ - is_regrade
2228
+ - idempotent_replay
2229
+ - started_at
2230
+ - updated_at
2231
+ - finished_at
2232
+ properties:
2233
+ id:
2234
+ type: string
2235
+ format: uuid
2236
+ job_name:
2237
+ type: string
2238
+ description: User-facing label.
2239
+ status:
2240
+ $ref: '#/components/schemas/JobStatus'
2241
+ datasets:
2242
+ type: array
2243
+ description: The resolved dataset references this job ran.
2244
+ items:
2245
+ $ref: '#/components/schemas/DatasetRef'
2246
+ agents:
2247
+ type: array
2248
+ items:
2249
+ $ref: '#/components/schemas/AgentArm'
2250
+ n_attempts:
2251
+ type: integer
2252
+ minimum: 1
2253
+ n_concurrent_trials:
2254
+ type: integer
2255
+ minimum: 1
2256
+ max_trial_spend_usd:
2257
+ type: number
2258
+ description: The resolved per-trial cap every trial key was minted with.
2259
+ worst_case_spend_usd:
2260
+ type: number
2261
+ description: |
2262
+ The most this job can cost: every trial spending its whole cap.
2263
+ Stated outright — the per-trial cap is the only enforcement, so
2264
+ the product is the number someone approving a 500-trial run
2265
+ actually needs to see.
2266
+ sandbox_provider:
2267
+ $ref: '#/components/schemas/SandboxProvider'
2268
+ counts:
2269
+ type: object
2270
+ description: Entity cardinality only — things with no status of their own.
2271
+ required: [agents, tasks]
2272
+ properties:
2273
+ agents:
2274
+ type: integer
2275
+ minimum: 0
2276
+ tasks:
2277
+ type: integer
2278
+ minimum: 0
2279
+ n_total_trials:
2280
+ type: integer
2281
+ minimum: 0
2282
+ trials:
2283
+ $ref: '#/components/schemas/TrialStatusTally'
2284
+ description: >
2285
+ Kept deviation — the zeros-included 8-status histogram, published
2286
+ alongside the coarser counters in `stats`.
2287
+ stats:
2288
+ $ref: '#/components/schemas/JobStats'
2289
+ failure:
2290
+ oneOf:
2291
+ - $ref: '#/components/schemas/JobFailure'
2292
+ - type: 'null'
2293
+ source_jobs:
2294
+ type: array
2295
+ description: Empty for an original job.
2296
+ items:
2297
+ $ref: '#/components/schemas/SourceJob'
2298
+ is_regrade:
2299
+ type: boolean
2300
+ description: 'Derived: any source_jobs entry with action "regrade".'
2301
+ idempotent_replay:
2302
+ type: boolean
2303
+ description: True only on a response that replayed an existing job for an Idempotency-Key.
2304
+ started_at:
2305
+ type: string
2306
+ format: date-time
2307
+ updated_at:
2308
+ type: string
2309
+ format: date-time
2310
+ finished_at:
2311
+ type: [string, 'null']
2312
+ format: date-time
2313
+ description: Null while the job is live.
2314
+
2315
+ JobCreated:
2316
+ description: |
2317
+ The CREATE verb's body: the whole Job shape plus `job_id`, an additive
2318
+ alias of `id`.
2319
+
2320
+ The alias exists for one reason and lives in one place. Harbor's
2321
+ hosted submit reads the new job's id under the key `job_id` and aborts
2322
+ the submit outright when that key is absent, so the verb an
2323
+ unmodified Harbor CLI starts a run with answers to both names. Every
2324
+ other job response — get, list, cancel, resume, regrade — carries `id`
2325
+ alone, so no reader has to learn a second spelling to follow a job.
2326
+
2327
+ Both of this verb's success bodies carry it: a create and the
2328
+ idempotent replay of that same create (their submit sends an
2329
+ `Idempotency-Key` on every attempt, so a retry lands on the replay).
2330
+ required: [job_id]
2331
+ allOf:
2332
+ - $ref: '#/components/schemas/Job'
2333
+ - type: object
2334
+ properties:
2335
+ job_id:
2336
+ type: string
2337
+ format: uuid
2338
+ description: Alias of `id`, identical value. Create responses only.
2339
+
2340
+ JobPage:
2341
+ type: object
2342
+ description: Cursor page of jobs (envelope keys frozen verbatim).
2343
+ required: [items, nextCursor, hasMore]
2344
+ properties:
2345
+ items:
2346
+ type: array
2347
+ items:
2348
+ $ref: '#/components/schemas/Job'
2349
+ nextCursor:
2350
+ type: [string, 'null']
2351
+ description: Pass back as `?cursor=`; null means no next page.
2352
+ hasMore:
2353
+ type: boolean
2354
+
2355
+ # -------------------------------------------------------------------------
2356
+ # Trial
2357
+ # -------------------------------------------------------------------------
2358
+
2359
+ TimingInfo:
2360
+ type: object
2361
+ description: >
2362
+ A phase's wall-clock as a start/stop pair (never a duration).
2363
+ Either bound is null while the phase has not reached it.
2364
+ properties:
2365
+ started_at:
2366
+ type: [string, 'null']
2367
+ format: date-time
2368
+ finished_at:
2369
+ type: [string, 'null']
2370
+ format: date-time
2371
+
2372
+ ModelInfo:
2373
+ type: object
2374
+ required: [name]
2375
+ properties:
2376
+ name:
2377
+ type: string
2378
+ provider:
2379
+ type: [string, 'null']
2380
+ description: Null means "not specified", never "unknown provider".
2381
+
2382
+ AgentInfo:
2383
+ type: object
2384
+ description: |
2385
+ The agent that ran a trial. `version` is the version actually
2386
+ RESOLVED and used (null until resolved) — the requested pin lives on
2387
+ the job's `agents[].version`. `reasoning_effort` is the platform's
2388
+ arm-identity extension.
2389
+ required: [name, version, model_info]
2390
+ properties:
2391
+ name:
2392
+ type: string
2393
+ version:
2394
+ type: [string, 'null']
2395
+ model_info:
2396
+ $ref: '#/components/schemas/ModelInfo'
2397
+ reasoning_effort:
2398
+ type: [string, 'null']
2399
+
2400
+ AgentResult:
2401
+ type: object
2402
+ description: |
2403
+ What the agent phase produced and consumed. `n_input_tokens` includes
2404
+ cache tokens. `cost_usd` is the settled spend (see `spend_source` on
2405
+ the trial for which lane it came from, and whether it is final).
2406
+ `metadata` carries
2407
+ open per-run detail: the harness bundle digest and runtime, the
2408
+ network mode the trial ran under and where that decision came from,
2409
+ and any harness-reported usage detail.
2410
+ properties:
2411
+ n_input_tokens:
2412
+ type: [integer, 'null']
2413
+ n_cache_tokens:
2414
+ type: [integer, 'null']
2415
+ n_output_tokens:
2416
+ type: [integer, 'null']
2417
+ cost_usd:
2418
+ type: [number, 'null']
2419
+ description: Null until the trial has executed; null never means $0.
2420
+ rollout_details:
2421
+ type: [array, 'null']
2422
+ description: Reserved for token-level rollout detail; null today.
2423
+ items:
2424
+ type: object
2425
+ additionalProperties: true
2426
+ metadata:
2427
+ type: [object, 'null']
2428
+ additionalProperties: true
2429
+
2430
+ VerifierResult:
2431
+ type: object
2432
+ description: |
2433
+ The verifier's rewards map. The primary-reward convention: the value
2434
+ under the key `"reward"`; else, when exactly one key exists, that
2435
+ value; else no primary reward. Zero is a reward.
2436
+ properties:
2437
+ rewards:
2438
+ type: [object, 'null']
2439
+ additionalProperties:
2440
+ type: number
2441
+
2442
+ ExceptionInfo:
2443
+ type: object
2444
+ description: |
2445
+ Why a trial failed, when it did. `exception_type` is one of the
2446
+ platform's stable failure names (`ScoringError`,
2447
+ `InfrastructureError`, `CancelledError`, `IncompleteTrialError`) —
2448
+ but filter with `Trial.status`, which is the primary key for failure
2449
+ classes; this is the detail.
2450
+ required: [exception_type, exception_message, occurred_at]
2451
+ properties:
2452
+ exception_type:
2453
+ type: string
2454
+ exception_message:
2455
+ type: string
2456
+ description: Truncated to 2000 chars on list rows; full on the detail route.
2457
+ exception_traceback:
2458
+ type: string
2459
+ description: Empty when the platform recorded no traceback.
2460
+ occurred_at:
2461
+ type: string
2462
+ format: date-time
2463
+
2464
+ StepResult:
2465
+ type: object
2466
+ description: >
2467
+ Placeholder for multi-step tasks. Always null on trials today —
2468
+ declared so multi-step lands without a wire change.
2469
+ properties:
2470
+ step_name:
2471
+ type: string
2472
+ agent_result:
2473
+ oneOf:
2474
+ - $ref: '#/components/schemas/AgentResult'
2475
+ - type: 'null'
2476
+ verifier_result:
2477
+ oneOf:
2478
+ - $ref: '#/components/schemas/VerifierResult'
2479
+ - type: 'null'
2480
+ exception_info:
2481
+ oneOf:
2482
+ - $ref: '#/components/schemas/ExceptionInfo'
2483
+ - type: 'null'
2484
+ agent_execution:
2485
+ oneOf:
2486
+ - $ref: '#/components/schemas/TimingInfo'
2487
+ - type: 'null'
2488
+ verifier:
2489
+ oneOf:
2490
+ - $ref: '#/components/schemas/TimingInfo'
2491
+ - type: 'null'
2492
+
2493
+ Trial:
2494
+ type: object
2495
+ description: |
2496
+ The ONE public trial shape, shared verbatim by list rows and the
2497
+ detail route (detail returns `exception_info.exception_message`
2498
+ untruncated — the only documented difference).
2499
+
2500
+ Execution facts (`sandbox_provider`, `verifier_environment_mode`,
2501
+ `agent_result.cost_usd`, `spend_source`) are null until the trial has
2502
+ actually executed: a QUEUED or CANCELLED trial never ran, so null
2503
+ means "did not run" and never zero.
2504
+ required:
2505
+ - id
2506
+ - job_id
2507
+ - task_name
2508
+ - source
2509
+ - agent_info
2510
+ - attempt
2511
+ - status
2512
+ - reward
2513
+ - verifier_result
2514
+ - exception_info
2515
+ - started_at
2516
+ - finished_at
2517
+ properties:
2518
+ id:
2519
+ type: string
2520
+ format: uuid
2521
+ job_id:
2522
+ type: string
2523
+ format: uuid
2524
+ task_name:
2525
+ type: string
2526
+ source:
2527
+ type: string
2528
+ description: The dataset this trial's task came from.
2529
+ agent_info:
2530
+ $ref: '#/components/schemas/AgentInfo'
2531
+ attempt:
2532
+ type: integer
2533
+ minimum: 1
2534
+ description: 'Attempt index within the arm (1..n_attempts).'
2535
+ status:
2536
+ $ref: '#/components/schemas/TrialStatus'
2537
+ reward:
2538
+ type: [number, 'null']
2539
+ description: >
2540
+ Convenience primary reward derived from `verifier_result.rewards`
2541
+ by the primary-reward convention. Zero is a reward; null means
2542
+ the trial did not score.
2543
+ verifier_result:
2544
+ oneOf:
2545
+ - $ref: '#/components/schemas/VerifierResult'
2546
+ - type: 'null'
2547
+ exception_info:
2548
+ oneOf:
2549
+ - $ref: '#/components/schemas/ExceptionInfo'
2550
+ - type: 'null'
2551
+ agent_result:
2552
+ oneOf:
2553
+ - $ref: '#/components/schemas/AgentResult'
2554
+ - type: 'null'
2555
+ environment_setup:
2556
+ oneOf:
2557
+ - $ref: '#/components/schemas/TimingInfo'
2558
+ - type: 'null'
2559
+ agent_setup:
2560
+ oneOf:
2561
+ - $ref: '#/components/schemas/TimingInfo'
2562
+ - type: 'null'
2563
+ agent_execution:
2564
+ oneOf:
2565
+ - $ref: '#/components/schemas/TimingInfo'
2566
+ - type: 'null'
2567
+ verifier:
2568
+ oneOf:
2569
+ - $ref: '#/components/schemas/TimingInfo'
2570
+ - type: 'null'
2571
+ step_results:
2572
+ type: [array, 'null']
2573
+ description: Multi-step placeholder; null today.
2574
+ items:
2575
+ $ref: '#/components/schemas/StepResult'
2576
+ spend_source:
2577
+ oneOf:
2578
+ - $ref: '#/components/schemas/SpendSource'
2579
+ - type: 'null'
2580
+ live_spent_usd:
2581
+ type: [number, 'null']
2582
+ description: |
2583
+ A mid-run LOWER BOUND on spend, never the trial's cost. Only ever
2584
+ climbs while the trial runs, and is CLEARED when the trial
2585
+ settles — on a terminal trial read `agent_result.cost_usd` and
2586
+ `spend_source`; those are the settled truth. Null is "no reading
2587
+ yet", never $0.
2588
+ live_spend_at:
2589
+ type: [string, 'null']
2590
+ format: date-time
2591
+ description: When that reading was taken — show its age, never the figure alone.
2592
+ max_trial_spend_usd:
2593
+ type: [number, 'null']
2594
+ description: >
2595
+ The cap THIS trial's gateway key carried — history, which can
2596
+ differ from the job's current cap for rows settled before a
2597
+ change.
2598
+ sandbox_provider:
2599
+ oneOf:
2600
+ - $ref: '#/components/schemas/SandboxProvider'
2601
+ - type: 'null'
2602
+ sandbox_id:
2603
+ type: [string, 'null']
2604
+ description: Provider id of the box the agent executed in; null when none booted.
2605
+ verifier_sandbox_id:
2606
+ type: [string, 'null']
2607
+ description: The separate verifier box; null in shared mode or when never reached.
2608
+ verifier_environment_mode:
2609
+ oneOf:
2610
+ - $ref: '#/components/schemas/VerifierEnvironmentMode'
2611
+ - type: 'null'
2612
+ attempt_phase:
2613
+ oneOf:
2614
+ - $ref: '#/components/schemas/AttemptPhase'
2615
+ - type: 'null'
2616
+ session_ref:
2617
+ type: [string, 'null']
2618
+ description: Reference to the agent session/trace, when recorded.
2619
+ started_at:
2620
+ type: [string, 'null']
2621
+ format: date-time
2622
+ finished_at:
2623
+ type: [string, 'null']
2624
+ format: date-time
2625
+
2626
+ TrialPage:
2627
+ type: object
2628
+ required: [items, nextCursor, hasMore]
2629
+ properties:
2630
+ items:
2631
+ type: array
2632
+ items:
2633
+ $ref: '#/components/schemas/Trial'
2634
+ nextCursor:
2635
+ type: [string, 'null']
2636
+ hasMore:
2637
+ type: boolean
2638
+
2639
+ StopRequest:
2640
+ type: object
2641
+ required: [trial_ids]
2642
+ properties:
2643
+ trial_ids:
2644
+ type: array
2645
+ minItems: 1
2646
+ maxItems: 100
2647
+ items:
2648
+ type: string
2649
+ format: uuid
2650
+
2651
+ StopResponse:
2652
+ type: object
2653
+ description: Per-trial outcome; every requested id appears in exactly one list.
2654
+ required: [stopped, already_terminal, not_found]
2655
+ properties:
2656
+ stopped:
2657
+ type: array
2658
+ description: Trials killed and settled by this request, with their settled rows.
2659
+ items:
2660
+ $ref: '#/components/schemas/Trial'
2661
+ already_terminal:
2662
+ type: array
2663
+ description: Ids that were already terminal; untouched.
2664
+ items:
2665
+ type: string
2666
+ format: uuid
2667
+ not_found:
2668
+ type: array
2669
+ description: Ids that do not exist or are not the caller's.
2670
+ items:
2671
+ type: string
2672
+ format: uuid
2673
+
2674
+ JobTaskRollup:
2675
+ type: object
2676
+ description: One task's rollup within a job (wave 2).
2677
+ required: [task_name, source, trials, mean_reward, cost_usd]
2678
+ properties:
2679
+ task_name:
2680
+ type: string
2681
+ source:
2682
+ type: string
2683
+ description: The dataset the task came from.
2684
+ trials:
2685
+ $ref: '#/components/schemas/TrialStatusTally'
2686
+ mean_reward:
2687
+ type: [number, 'null']
2688
+ description: Mean over SCORED trials only; null when none. Zero is a reward.
2689
+ cost_usd:
2690
+ type: [number, 'null']
2691
+ description: Measured spend across the task's settled trials.
2692
+
2693
+ JobTaskRollupPage:
2694
+ type: object
2695
+ required: [items, nextCursor, hasMore]
2696
+ properties:
2697
+ items:
2698
+ type: array
2699
+ items:
2700
+ $ref: '#/components/schemas/JobTaskRollup'
2701
+ nextCursor:
2702
+ type: [string, 'null']
2703
+ hasMore:
2704
+ type: boolean
2705
+
2706
+ TraceEvent:
2707
+ type: object
2708
+ description: >
2709
+ One parsed trace event of a trial's transcript. `seq` orders the
2710
+ stream and is the paging cursor. `data` is the harness-native payload,
2711
+ deliberately open.
2712
+ required: [seq, type]
2713
+ properties:
2714
+ seq:
2715
+ type: integer
2716
+ type:
2717
+ type: string
2718
+ timestamp:
2719
+ type: [string, 'null']
2720
+ format: date-time
2721
+ data:
2722
+ type: object
2723
+ additionalProperties: true
2724
+ additionalProperties: true
2725
+
2726
+ TraceEventPage:
2727
+ type: object
2728
+ required: [items, nextCursor, hasMore]
2729
+ properties:
2730
+ items:
2731
+ type: array
2732
+ items:
2733
+ $ref: '#/components/schemas/TraceEvent'
2734
+ nextCursor:
2735
+ type: [string, 'null']
2736
+ hasMore:
2737
+ type: boolean
2738
+
2739
+ # -------------------------------------------------------------------------
2740
+ # Compare
2741
+ # -------------------------------------------------------------------------
2742
+
2743
+ CompareCoverage:
2744
+ type: object
2745
+ description: How many of the trials behind an aggregate were SCORED.
2746
+ required: [scored, total]
2747
+ properties:
2748
+ scored:
2749
+ type: integer
2750
+ minimum: 0
2751
+ total:
2752
+ type: integer
2753
+ minimum: 0
2754
+
2755
+ CompareCell:
2756
+ type: object
2757
+ description: |
2758
+ One (task, job) cell. `status` is a TrialStatus when every trial in
2759
+ the cell shares it, `MIXED` when they differ, `MISSING` when the job
2760
+ has no trials for the task.
2761
+ required: [job_id, status, mean_reward, coverage]
2762
+ properties:
2763
+ job_id:
2764
+ type: string
2765
+ format: uuid
2766
+ status:
2767
+ type: string
2768
+ mean_reward:
2769
+ type: [number, 'null']
2770
+ coverage:
2771
+ $ref: '#/components/schemas/CompareCoverage'
2772
+
2773
+ CompareTaskRow:
2774
+ type: object
2775
+ required: [task_name, disagreement, cells]
2776
+ properties:
2777
+ task_name:
2778
+ type: string
2779
+ disagreement:
2780
+ type: boolean
2781
+ description: True when the jobs' cells differ in status or reward.
2782
+ cells:
2783
+ type: array
2784
+ items:
2785
+ $ref: '#/components/schemas/CompareCell'
2786
+
2787
+ CompareJobAggregate:
2788
+ type: object
2789
+ required: [id, datasets, status, mean_reward, coverage, cost_usd, agents, started_at]
2790
+ properties:
2791
+ id:
2792
+ type: string
2793
+ format: uuid
2794
+ datasets:
2795
+ type: array
2796
+ items:
2797
+ $ref: '#/components/schemas/DatasetRef'
2798
+ status:
2799
+ $ref: '#/components/schemas/JobStatus'
2800
+ mean_reward:
2801
+ type: [number, 'null']
2802
+ description: Mean over SCORED trials only; null when none. Zero is a reward.
2803
+ coverage:
2804
+ $ref: '#/components/schemas/CompareCoverage'
2805
+ cost_usd:
2806
+ type: number
2807
+ agents:
2808
+ type: array
2809
+ items:
2810
+ $ref: '#/components/schemas/AgentArm'
2811
+ started_at:
2812
+ type: string
2813
+ format: date-time
2814
+
2815
+ CompareResponse:
2816
+ type: object
2817
+ required: [jobs, taskMatrix]
2818
+ properties:
2819
+ jobs:
2820
+ type: array
2821
+ items:
2822
+ $ref: '#/components/schemas/CompareJobAggregate'
2823
+ taskMatrix:
2824
+ type: array
2825
+ description: Disagreement rows sort first.
2826
+ items:
2827
+ $ref: '#/components/schemas/CompareTaskRow'
2828
+
2829
+ # -------------------------------------------------------------------------
2830
+ # SSE events
2831
+ # -------------------------------------------------------------------------
2832
+
2833
+ JobEvent:
2834
+ description: |
2835
+ One server-sent event. On the wire, `seq` rides as the SSE `id` field
2836
+ and `type` as the SSE `event` field; the JSON body is `data`. Payload
2837
+ fields speak this document's vocabulary.
2838
+
2839
+ The union is discriminated on `type` and ONLY on `type`: several
2840
+ event types carry identically shaped payloads (`job.running` and
2841
+ `job.completed` are both `{job_id}`), so payload shape can never
2842
+ route a validator — the `type` constant on each variant does.
2843
+ oneOf:
2844
+ - $ref: '#/components/schemas/JobCreatedEvent'
2845
+ - $ref: '#/components/schemas/JobRunningEvent'
2846
+ - $ref: '#/components/schemas/JobCancellingEvent'
2847
+ - $ref: '#/components/schemas/JobCancelledEvent'
2848
+ - $ref: '#/components/schemas/JobCompletedEvent'
2849
+ - $ref: '#/components/schemas/JobFailedEvent'
2850
+ - $ref: '#/components/schemas/TrialRunningEvent'
2851
+ - $ref: '#/components/schemas/TrialScoringEvent'
2852
+ - $ref: '#/components/schemas/TrialSpendEvent'
2853
+ - $ref: '#/components/schemas/TrialSettledEvent'
2854
+ discriminator:
2855
+ propertyName: type
2856
+ mapping:
2857
+ job.created: '#/components/schemas/JobCreatedEvent'
2858
+ job.running: '#/components/schemas/JobRunningEvent'
2859
+ job.cancelling: '#/components/schemas/JobCancellingEvent'
2860
+ job.cancelled: '#/components/schemas/JobCancelledEvent'
2861
+ job.completed: '#/components/schemas/JobCompletedEvent'
2862
+ job.failed: '#/components/schemas/JobFailedEvent'
2863
+ trial.running: '#/components/schemas/TrialRunningEvent'
2864
+ trial.scoring: '#/components/schemas/TrialScoringEvent'
2865
+ trial.spend: '#/components/schemas/TrialSpendEvent'
2866
+ trial.settled: '#/components/schemas/TrialSettledEvent'
2867
+
2868
+ EventSeq:
2869
+ type: integer
2870
+ minimum: 0
2871
+ description: Monotonic sequence number — the resume position.
2872
+
2873
+ JobCreatedEvent:
2874
+ type: object
2875
+ required: [seq, type, data]
2876
+ properties:
2877
+ seq:
2878
+ $ref: '#/components/schemas/EventSeq'
2879
+ type:
2880
+ type: string
2881
+ const: job.created
2882
+ data:
2883
+ $ref: '#/components/schemas/JobCreatedData'
2884
+
2885
+ JobRunningEvent:
2886
+ type: object
2887
+ required: [seq, type, data]
2888
+ properties:
2889
+ seq:
2890
+ $ref: '#/components/schemas/EventSeq'
2891
+ type:
2892
+ type: string
2893
+ const: job.running
2894
+ data:
2895
+ $ref: '#/components/schemas/JobRunningData'
2896
+
2897
+ JobCancellingEvent:
2898
+ type: object
2899
+ required: [seq, type, data]
2900
+ properties:
2901
+ seq:
2902
+ $ref: '#/components/schemas/EventSeq'
2903
+ type:
2904
+ type: string
2905
+ const: job.cancelling
2906
+ data:
2907
+ $ref: '#/components/schemas/JobCancellingData'
2908
+
2909
+ JobCancelledEvent:
2910
+ type: object
2911
+ required: [seq, type, data]
2912
+ properties:
2913
+ seq:
2914
+ $ref: '#/components/schemas/EventSeq'
2915
+ type:
2916
+ type: string
2917
+ const: job.cancelled
2918
+ data:
2919
+ $ref: '#/components/schemas/JobCancelledData'
2920
+
2921
+ JobCompletedEvent:
2922
+ type: object
2923
+ required: [seq, type, data]
2924
+ properties:
2925
+ seq:
2926
+ $ref: '#/components/schemas/EventSeq'
2927
+ type:
2928
+ type: string
2929
+ const: job.completed
2930
+ data:
2931
+ $ref: '#/components/schemas/JobCompletedData'
2932
+
2933
+ JobFailedEvent:
2934
+ type: object
2935
+ required: [seq, type, data]
2936
+ properties:
2937
+ seq:
2938
+ $ref: '#/components/schemas/EventSeq'
2939
+ type:
2940
+ type: string
2941
+ const: job.failed
2942
+ data:
2943
+ $ref: '#/components/schemas/JobFailedData'
2944
+
2945
+ TrialRunningEvent:
2946
+ type: object
2947
+ required: [seq, type, data]
2948
+ properties:
2949
+ seq:
2950
+ $ref: '#/components/schemas/EventSeq'
2951
+ type:
2952
+ type: string
2953
+ const: trial.running
2954
+ data:
2955
+ $ref: '#/components/schemas/TrialRunningData'
2956
+
2957
+ TrialScoringEvent:
2958
+ type: object
2959
+ required: [seq, type, data]
2960
+ properties:
2961
+ seq:
2962
+ $ref: '#/components/schemas/EventSeq'
2963
+ type:
2964
+ type: string
2965
+ const: trial.scoring
2966
+ data:
2967
+ $ref: '#/components/schemas/TrialScoringData'
2968
+
2969
+ TrialSpendEvent:
2970
+ type: object
2971
+ required: [seq, type, data]
2972
+ properties:
2973
+ seq:
2974
+ $ref: '#/components/schemas/EventSeq'
2975
+ type:
2976
+ type: string
2977
+ const: trial.spend
2978
+ data:
2979
+ $ref: '#/components/schemas/TrialSpendData'
2980
+
2981
+ TrialSettledEvent:
2982
+ type: object
2983
+ required: [seq, type, data]
2984
+ properties:
2985
+ seq:
2986
+ $ref: '#/components/schemas/EventSeq'
2987
+ type:
2988
+ type: string
2989
+ const: trial.settled
2990
+ data:
2991
+ $ref: '#/components/schemas/TrialSettledData'
2992
+
2993
+ JobCreatedData:
2994
+ type: object
2995
+ description: >
2996
+ The job's resolved creation inputs, echoed so a watcher that joined
2997
+ late knows what it is watching.
2998
+ required:
2999
+ - datasets
3000
+ - task_count
3001
+ - agents
3002
+ - n_attempts
3003
+ - n_concurrent_trials
3004
+ - max_trial_spend_usd
3005
+ - sandbox_provider
3006
+ - trial_count
3007
+ properties:
3008
+ datasets:
3009
+ type: array
3010
+ items:
3011
+ $ref: '#/components/schemas/DatasetRef'
3012
+ task_count:
3013
+ type: integer
3014
+ agents:
3015
+ type: array
3016
+ items:
3017
+ $ref: '#/components/schemas/AgentArm'
3018
+ n_attempts:
3019
+ type: integer
3020
+ n_concurrent_trials:
3021
+ type: integer
3022
+ max_trial_spend_usd:
3023
+ type: number
3024
+ sandbox_provider:
3025
+ $ref: '#/components/schemas/SandboxProvider'
3026
+ trial_count:
3027
+ type: integer
3028
+
3029
+ JobRunningData:
3030
+ type: object
3031
+ required: [job_id]
3032
+ properties:
3033
+ job_id:
3034
+ type: string
3035
+ format: uuid
3036
+
3037
+ JobCancellingData:
3038
+ type: object
3039
+ required: [job_id, cancelled_trials, active_trials]
3040
+ properties:
3041
+ job_id:
3042
+ type: string
3043
+ format: uuid
3044
+ cancelled_trials:
3045
+ type: integer
3046
+ description: Queued trials cancelled outright by the request.
3047
+ active_trials:
3048
+ type: integer
3049
+ description: Trials still in flight, winding down before the job settles.
3050
+
3051
+ JobCancelledData:
3052
+ type: object
3053
+ required: [job_id, cancelled_trials]
3054
+ properties:
3055
+ job_id:
3056
+ type: string
3057
+ format: uuid
3058
+ cancelled_trials:
3059
+ type: integer
3060
+ description: Total queued trials cancelled across the request and the settle.
3061
+
3062
+ JobCompletedData:
3063
+ type: object
3064
+ required: [job_id]
3065
+ properties:
3066
+ job_id:
3067
+ type: string
3068
+ format: uuid
3069
+
3070
+ JobFailedData:
3071
+ type: object
3072
+ description: >
3073
+ The job settled FAILED. Terminal — the stream closes after this
3074
+ event. The event type is declared and reserved (FAILED is a real job
3075
+ status) but no server path emits it today; the payload is fixed here
3076
+ so a client written now parses it when it first appears.
3077
+ required: [job_id]
3078
+ properties:
3079
+ job_id:
3080
+ type: string
3081
+ format: uuid
3082
+
3083
+ TrialRunningData:
3084
+ type: object
3085
+ required: [trial_id, task_name]
3086
+ properties:
3087
+ trial_id:
3088
+ type: string
3089
+ format: uuid
3090
+ task_name:
3091
+ type: string
3092
+
3093
+ TrialScoringData:
3094
+ type: object
3095
+ required: [trial_id]
3096
+ properties:
3097
+ trial_id:
3098
+ type: string
3099
+ format: uuid
3100
+ captured_bytes:
3101
+ type: integer
3102
+ description: Bytes of agent stdout retained for the failure detail.
3103
+
3104
+ TrialSpendData:
3105
+ type: object
3106
+ description: >
3107
+ A mid-run spend sample landed on a still-live trial. Emitted only
3108
+ when the reading actually updated a RUNNING/SCORING row. The token
3109
+ sums come from the same ledger aggregation that produced the money
3110
+ figure, present only when the sample carried them — an older event
3111
+ replays without them.
3112
+ required: [trial_id, task_name, live_spent_usd]
3113
+ properties:
3114
+ trial_id:
3115
+ type: string
3116
+ format: uuid
3117
+ task_name:
3118
+ type: string
3119
+ live_spent_usd:
3120
+ type: number
3121
+ description: The same lagging lower bound as Trial.live_spent_usd.
3122
+ n_input_tokens:
3123
+ type: integer
3124
+ description: Input tokens so far; includes cache tokens.
3125
+ n_cache_tokens:
3126
+ type: integer
3127
+ description: Cached input tokens so far (a subset of n_input_tokens).
3128
+ n_output_tokens:
3129
+ type: integer
3130
+ description: Output tokens so far.
3131
+
3132
+ TrialSettledData:
3133
+ type: object
3134
+ description: >
3135
+ A trial reached a terminal status. `reward` is present only on the
3136
+ scored path; `exception_type` only on a failure; `attempt_phase`
3137
+ appears when the settle happened mid-phase (worker death), which is
3138
+ exactly when knowing the phase is worth having.
3139
+ required: [trial_id, task_name, status]
3140
+ properties:
3141
+ trial_id:
3142
+ type: string
3143
+ format: uuid
3144
+ task_name:
3145
+ type: string
3146
+ status:
3147
+ $ref: '#/components/schemas/TrialStatus'
3148
+ reward:
3149
+ type: [number, 'null']
3150
+ exception_type:
3151
+ type: string
3152
+ attempt_phase:
3153
+ oneOf:
3154
+ - $ref: '#/components/schemas/AttemptPhase'
3155
+ - type: 'null'
3156
+
3157
+ # -------------------------------------------------------------------------
3158
+ # Datasets
3159
+ # -------------------------------------------------------------------------
3160
+
3161
+ DatasetVersionState:
3162
+ type: string
3163
+ description: |
3164
+ Terminal: READY, FAILED, ARCHIVED.
3165
+
3166
+ A hosted publish walks IMPORTING -> BUILDING -> VALIDATING on its own,
3167
+ and VALIDATING -> READY happens on its own too: the activation gate
3168
+ promotes the version it proves, and points the dataset's default at
3169
+ it. The activate verb is not part of that path — it exists to move
3170
+ the default to a different version (an older READY one, or a newer
3171
+ one), never to finish a publish.
3172
+
3173
+ FAILED is reached two ways, both terminal and both explained by the
3174
+ version's `gate`/import failure detail: the import itself failed, or
3175
+ the activation gate reached a terminal FAILED verdict — the version
3176
+ then moves VALIDATING -> FAILED (it never sits VALIDATING forever),
3177
+ with `gate.failure` naming the failing tasks and their causes.
3178
+ Re-publishing the same name@version starts the walk over.
3179
+ enum: [DRAFT, IMPORTING, BUILDING, VALIDATING, READY, FAILED, ARCHIVED]
3180
+
3181
+ DatasetVersion:
3182
+ type: object
3183
+ description: One immutable version of a dataset — one shape on every surface.
3184
+ required: [version, state, created_at, task_count, gate]
3185
+ properties:
3186
+ version:
3187
+ type: string
3188
+ state:
3189
+ $ref: '#/components/schemas/DatasetVersionState'
3190
+ created_at:
3191
+ type: string
3192
+ format: date-time
3193
+ task_count:
3194
+ type: integer
3195
+ minimum: 0
3196
+ gate:
3197
+ oneOf:
3198
+ - $ref: '#/components/schemas/ActivationGate'
3199
+ - type: 'null'
3200
+ description: >
3201
+ The activation gate's schedule state for this version. Null when
3202
+ no gate was ever scheduled — a version imported before the gate
3203
+ scheduler existed, a platform-seeded one, or one whose import
3204
+ archived no reference solutions (see the import's `warnings`).
3205
+
3206
+ ActivationGateStatus:
3207
+ type: string
3208
+ description: |
3209
+ Where the automatic activation gate stands for a version. The gate is
3210
+ the oracle-conformance run the worker schedules once a hosted import
3211
+ completes: the version's held-out reference (gold) solution must
3212
+ score exactly 1.0 through the real execution path, and a do-nothing
3213
+ agent must not. PENDING = scheduled, waiting for a worker; RUNNING =
3214
+ proving tasks now (per-task verdicts land on the task list as they
3215
+ finish); PASSED = every task is activation-eligible, and the version
3216
+ was promoted to READY and made the dataset's active version by the
3217
+ gate itself — except on a platform-curated dataset, whose default is
3218
+ an operator setting, so its versions sit PASSED until an operator
3219
+ promotes them; FAILED = at least one task is not — `failure` names
3220
+ them and the dataset's active version is left alone.
3221
+ enum: [PENDING, RUNNING, PASSED, FAILED]
3222
+
3223
+ ActivationGate:
3224
+ type: object
3225
+ description: >
3226
+ A version's activation-gate schedule state. Watch `status` move
3227
+ PENDING -> RUNNING -> PASSED/FAILED after a publish; the state that
3228
+ used to read as "VALIDATING forever" is this object saying where the
3229
+ proof stands.
3230
+ required: [status, attempts, failure]
3231
+ properties:
3232
+ status:
3233
+ $ref: '#/components/schemas/ActivationGateStatus'
3234
+ attempts:
3235
+ type: integer
3236
+ minimum: 0
3237
+ description: >
3238
+ Times a worker has taken this gate. A run lost to a dead worker
3239
+ is re-queued and RESUMES (verdicts already landed are kept),
3240
+ bounded before the gate fails with `gate_lease_lost`.
3241
+ failure:
3242
+ oneOf:
3243
+ - $ref: '#/components/schemas/ActivationGateFailure'
3244
+ - type: 'null'
3245
+ description: Why the gate FAILED; null unless status is FAILED.
3246
+
3247
+ ActivationGateFailure:
3248
+ type: object
3249
+ description: >
3250
+ The gate's structured failure. `failed_tasks` names each ineligible
3251
+ task with the gate's own reasons (a gold solution that never scored
3252
+ 1.0, a no-op that did, a solution that could not be resolved), so
3253
+ nothing has to be parsed back out of the message.
3254
+ required: [code, message]
3255
+ properties:
3256
+ code:
3257
+ type: string
3258
+ description: '`gate_failed`, or `gate_lease_lost` after repeated worker loss.'
3259
+ message:
3260
+ type: string
3261
+ failed_tasks:
3262
+ type: array
3263
+ description: >
3264
+ The ineligible tasks (first 25; when truncated,
3265
+ `failed_task_count` carries the true total).
3266
+ items:
3267
+ type: object
3268
+ required: [task_name, outcome, reasons]
3269
+ properties:
3270
+ task_name:
3271
+ type: string
3272
+ outcome:
3273
+ type: string
3274
+ description: FAIL, or ERROR (inconclusive — no usable score).
3275
+ reasons:
3276
+ type: array
3277
+ items:
3278
+ type: string
3279
+ failed_task_count:
3280
+ type: integer
3281
+ minimum: 0
3282
+
3283
+ GateRunning:
3284
+ type: object
3285
+ description: |
3286
+ The 202 body of the activation verb while the gate is still proving
3287
+ the version. Deliberately NOT the error envelope: an in-progress
3288
+ gate is a healthy state, and `error` on this surface always means
3289
+ "this request failed".
3290
+ required: [code, message, gate]
3291
+ properties:
3292
+ code:
3293
+ type: string
3294
+ enum: [gate_running]
3295
+ message:
3296
+ type: string
3297
+ gate:
3298
+ type: object
3299
+ description: Gate progress at the moment of the call.
3300
+ required: [status, tasks, unverified, ineligible]
3301
+ properties:
3302
+ status:
3303
+ $ref: '#/components/schemas/ActivationGateStatus'
3304
+ tasks:
3305
+ type: integer
3306
+ minimum: 0
3307
+ unverified:
3308
+ type: integer
3309
+ minimum: 0
3310
+ description: Tasks the gate has not yet produced a verdict for.
3311
+ ineligible:
3312
+ type: integer
3313
+ minimum: 0
3314
+ description: Tasks whose verdict so far is not activation-eligible.
3315
+
3316
+ TaskProviderVerdict:
3317
+ type: object
3318
+ description: >
3319
+ One provider's verdict for a task: runnable there, or refused with
3320
+ the limitation named. Advisory for planning — creating a job whose
3321
+ tasks include one refused on the chosen provider is rejected with the
3322
+ same reason, so nothing is ever spent on a trial that cannot execute.
3323
+ required: [ok]
3324
+ properties:
3325
+ ok:
3326
+ type: boolean
3327
+ reason:
3328
+ type: string
3329
+ description: Present when ok is false.
3330
+
3331
+ Task:
3332
+ type: object
3333
+ description: >
3334
+ Public task fields only — instructions, environments, and tests never
3335
+ leave the server.
3336
+ required: [task_name, agent_timeout_sec, verifier_timeout_sec, providers, gate]
3337
+ properties:
3338
+ task_name:
3339
+ type: string
3340
+ agent_timeout_sec:
3341
+ type: number
3342
+ verifier_timeout_sec:
3343
+ type: number
3344
+ providers:
3345
+ type: object
3346
+ description: Verdict per sandbox provider.
3347
+ additionalProperties:
3348
+ $ref: '#/components/schemas/TaskProviderVerdict'
3349
+ gate:
3350
+ oneOf:
3351
+ - $ref: '#/components/schemas/TaskGate'
3352
+ - type: 'null'
3353
+ description: >
3354
+ This task's activation-gate verdict; null until the gate has run
3355
+ it. The per-task half of the version's `gate` — while the gate is
3356
+ RUNNING, verdicts appear here as they land.
3357
+
3358
+ TaskGate:
3359
+ type: object
3360
+ description: >
3361
+ One task's activation-gate verdict — the public subset. The full
3362
+ stored verdict carries oracle diagnostics that stay internal, like
3363
+ the environment specs beside it.
3364
+ required: [outcome, flaky, reasons, ran_at]
3365
+ properties:
3366
+ outcome:
3367
+ type: string
3368
+ description: >
3369
+ PASS; FLAKY (gold passed only on a retry — still eligible under
3370
+ the operator default); FAIL (definitive: gold never scored 1.0,
3371
+ or a do-nothing agent did); ERROR (inconclusive — no usable
3372
+ score, e.g. no archived solution to run).
3373
+ flaky:
3374
+ type: boolean
3375
+ reasons:
3376
+ type: array
3377
+ items:
3378
+ type: string
3379
+ description: Human-readable; empty on PASS.
3380
+ ran_at:
3381
+ type: [string, 'null']
3382
+ format: date-time
3383
+
3384
+ TaskPage:
3385
+ type: object
3386
+ required: [items, nextCursor, hasMore]
3387
+ properties:
3388
+ items:
3389
+ type: array
3390
+ items:
3391
+ $ref: '#/components/schemas/Task'
3392
+ nextCursor:
3393
+ type: [string, 'null']
3394
+ hasMore:
3395
+ type: boolean
3396
+
3397
+ UpstreamStatus:
3398
+ type: object
3399
+ description: |
3400
+ Where the dataset's git source points now versus what its active
3401
+ version was built from — the data behind a "new version available"
3402
+ badge. The whole object is null on a dataset with nothing to watch;
3403
+ null is never "up to date". Nothing here imports anything — a new
3404
+ version is always a row you create (or `auto_import` creates).
3405
+ required: [ref, current_commit, latest_commit, moved, behind_by, checked_at, error, auto_import]
3406
+ properties:
3407
+ ref:
3408
+ type: string
3409
+ description: The ref the active version was imported from.
3410
+ current_commit:
3411
+ type: string
3412
+ latest_commit:
3413
+ type: [string, 'null']
3414
+ description: Where the ref points upstream now; null when the last check failed.
3415
+ moved:
3416
+ type: boolean
3417
+ description: True when upstream has moved off the built-from commit. Branch on this.
3418
+ behind_by:
3419
+ type: [integer, 'null']
3420
+ description: Reserved; always null today.
3421
+ checked_at:
3422
+ type: [string, 'null']
3423
+ format: date-time
3424
+ error:
3425
+ type: [string, 'null']
3426
+ description: Why the last check failed. Show "could not check", not "up to date".
3427
+ auto_import:
3428
+ type: boolean
3429
+ description: Whether a moved upstream automatically imports a new version.
3430
+
3431
+ Dataset:
3432
+ type: object
3433
+ description: >
3434
+ A dataset in the catalog. List responses carry the summary fields;
3435
+ the detail route additionally populates `versions`,
3436
+ `selected_version`, `tasks`, `created_at`, and `updated_at`.
3437
+ required: [name, title, description, active_version, upstream]
3438
+ properties:
3439
+ name:
3440
+ type: string
3441
+ title:
3442
+ type: [string, 'null']
3443
+ description:
3444
+ type: [string, 'null']
3445
+ active_version:
3446
+ oneOf:
3447
+ - $ref: '#/components/schemas/DatasetVersion'
3448
+ - type: 'null'
3449
+ description: Null when no version is active (bare-name job refs refuse).
3450
+ versions:
3451
+ type: array
3452
+ description: All versions, newest first (detail only).
3453
+ items:
3454
+ $ref: '#/components/schemas/DatasetVersion'
3455
+ selected_version:
3456
+ oneOf:
3457
+ - $ref: '#/components/schemas/DatasetVersion'
3458
+ - type: 'null'
3459
+ description: The version whose tasks are listed below (detail only).
3460
+ tasks:
3461
+ $ref: '#/components/schemas/TaskPage'
3462
+ description: One page of the selected version's tasks (detail only).
3463
+ upstream:
3464
+ oneOf:
3465
+ - $ref: '#/components/schemas/UpstreamStatus'
3466
+ - type: 'null'
3467
+ created_at:
3468
+ type: string
3469
+ format: date-time
3470
+ updated_at:
3471
+ type: string
3472
+ format: date-time
3473
+
3474
+ DatasetPage:
3475
+ type: object
3476
+ required: [items, nextCursor, hasMore]
3477
+ properties:
3478
+ items:
3479
+ type: array
3480
+ items:
3481
+ $ref: '#/components/schemas/Dataset'
3482
+ nextCursor:
3483
+ type: [string, 'null']
3484
+ hasMore:
3485
+ type: boolean
3486
+
3487
+ DatasetPatch:
3488
+ type: object
3489
+ description: The only settable dataset field.
3490
+ required: [upstream_auto_import]
3491
+ properties:
3492
+ upstream_auto_import:
3493
+ type: boolean
3494
+
3495
+ PublishRequest:
3496
+ type: object
3497
+ description: >
3498
+ Multipart body of POST /api/datasets/publish. Exactly one source:
3499
+ `git_url` + `git_ref`, or `archive`.
3500
+ required: [name, version]
3501
+ properties:
3502
+ name:
3503
+ type: string
3504
+ description: Catalog dataset name the version lands under (created or extended).
3505
+ version:
3506
+ type: string
3507
+ description: Version label for the new immutable version.
3508
+ git_url:
3509
+ type: string
3510
+ description: https:// repository URL. For a private repo, put a token in the URL.
3511
+ git_ref:
3512
+ type: string
3513
+ description: Pinned branch, tag, or commit. Required with git_url.
3514
+ archive:
3515
+ type: string
3516
+ format: binary
3517
+ description: Gzipped tarball of a standard-layout corpus directory.
3518
+
3519
+ DatasetImportStatus:
3520
+ type: string
3521
+ description: >
3522
+ Same four words a job uses, deliberately. Terminal: COMPLETED (the
3523
+ corpus landed as a dataset version; runnable once its activation gate
3524
+ passes, which also activates it) and FAILED.
3525
+ enum: [QUEUED, RUNNING, COMPLETED, FAILED]
3526
+
3527
+ DatasetImportFailure:
3528
+ type: object
3529
+ description: Structured failure detail for a FAILED import.
3530
+ required: [code, message]
3531
+ properties:
3532
+ code:
3533
+ type: string
3534
+ description: '`import_failed` when none was recorded.'
3535
+ message:
3536
+ type: string
3537
+ failures:
3538
+ type: array
3539
+ description: Per-task parse/validation failures, when the corpus was reachable.
3540
+ items:
3541
+ type: object
3542
+ required: [task_name, error]
3543
+ properties:
3544
+ task_name:
3545
+ type: string
3546
+ error:
3547
+ type: string
3548
+
3549
+ ImportWarning:
3550
+ type: object
3551
+ description: |
3552
+ Non-fatal but consequential import outcomes. A version whose warnings
3553
+ include `no_solutions_archived` cannot be activated through this API
3554
+ (`version_not_activatable`).
3555
+ required: [code]
3556
+ properties:
3557
+ code:
3558
+ type: string
3559
+ enum:
3560
+ - solutions_archiving_disabled
3561
+ - no_solutions_archived
3562
+ - partial_solutions_archived
3563
+ message:
3564
+ type: string
3565
+
3566
+ DatasetImport:
3567
+ type: object
3568
+ description: >
3569
+ An asynchronous publish. Self-describing: every response names the
3570
+ dataset@version being imported, so a caller can render the row it
3571
+ just created without a follow-up read.
3572
+ required: [id, status, name, version, failure, warnings]
3573
+ properties:
3574
+ id:
3575
+ type: string
3576
+ status:
3577
+ $ref: '#/components/schemas/DatasetImportStatus'
3578
+ name:
3579
+ type: string
3580
+ description: Catalog dataset name the import creates or extends.
3581
+ version:
3582
+ type: string
3583
+ failure:
3584
+ oneOf:
3585
+ - $ref: '#/components/schemas/DatasetImportFailure'
3586
+ - type: 'null'
3587
+ description: >
3588
+ Why the import FAILED; null otherwise. Named `failure`, never
3589
+ `error` — see JobFailure.
3590
+ warnings:
3591
+ type: array
3592
+ items:
3593
+ $ref: '#/components/schemas/ImportWarning'
3594
+ task_count:
3595
+ type: integer
3596
+ description: Number of tasks parsed, once counted.
3597
+ created_at:
3598
+ type: string
3599
+ format: date-time
3600
+ updated_at:
3601
+ type: string
3602
+ format: date-time
3603
+
3604
+ DatasetImportPage:
3605
+ type: object
3606
+ required: [items, nextCursor, hasMore]
3607
+ properties:
3608
+ items:
3609
+ type: array
3610
+ items:
3611
+ $ref: '#/components/schemas/DatasetImport'
3612
+ nextCursor:
3613
+ type: [string, 'null']
3614
+ hasMore:
3615
+ type: boolean
3616
+
3617
+ # -------------------------------------------------------------------------
3618
+ # Registered agents
3619
+ # -------------------------------------------------------------------------
3620
+
3621
+ Agent:
3622
+ type: object
3623
+ description: A private agent registered by the caller.
3624
+ required: [name, source, run_command, env, created_at, updated_at]
3625
+ properties:
3626
+ name:
3627
+ type: string
3628
+ description: The name to put in job `agents[].name`.
3629
+ source:
3630
+ type: string
3631
+ enum: [install_script, tarball]
3632
+ description: How the executables were produced. Echoed, never guessed.
3633
+ run_command:
3634
+ type: string
3635
+ description: Run headless with `sh -c` at the task working directory.
3636
+ env:
3637
+ type: object
3638
+ description: >
3639
+ Caller-declared env injected at RUN time only; may not override
3640
+ the run contract's own keys (`agent_invalid_env` at
3641
+ registration).
3642
+ additionalProperties:
3643
+ type: string
3644
+ created_at:
3645
+ type: string
3646
+ format: date-time
3647
+ updated_at:
3648
+ type: string
3649
+ format: date-time
3650
+
3651
+ AgentRegistration:
3652
+ type: object
3653
+ description: >
3654
+ Multipart registration body. Exactly one source: `install_script` or
3655
+ `archive`.
3656
+ required: [name, run_command]
3657
+ properties:
3658
+ name:
3659
+ type: string
3660
+ run_command:
3661
+ type: string
3662
+ env:
3663
+ type: string
3664
+ description: JSON object of env entries, as a multipart text part.
3665
+ install_script:
3666
+ type: string
3667
+ description: >
3668
+ The script itself (not a path). Runs in a throwaway builder
3669
+ sandbox with internet and zero secrets; must leave executables in
3670
+ `$PREFIX/bin`.
3671
+ archive:
3672
+ type: string
3673
+ format: binary
3674
+ description: Gzipped tarball of the agent directory. Same build rules.
3675
+
3676
+ AgentPage:
3677
+ type: object
3678
+ required: [items, nextCursor, hasMore]
3679
+ properties:
3680
+ items:
3681
+ type: array
3682
+ items:
3683
+ $ref: '#/components/schemas/Agent'
3684
+ nextCursor:
3685
+ type: [string, 'null']
3686
+ hasMore:
3687
+ type: boolean
3688
+
3689
+ # -------------------------------------------------------------------------
3690
+ # Meta
3691
+ # -------------------------------------------------------------------------
3692
+
3693
+ StatusVocabulary:
3694
+ type: object
3695
+ description: One status vocabulary and its terminal members.
3696
+ required: [values, terminal]
3697
+ properties:
3698
+ values:
3699
+ type: array
3700
+ items:
3701
+ type: string
3702
+ terminal:
3703
+ type: array
3704
+ items:
3705
+ type: string
3706
+
3707
+ AgentCapability:
3708
+ type: object
3709
+ description: One built-in agent's declared capabilities.
3710
+ required: [name, effort_support, version_pinnable]
3711
+ properties:
3712
+ name:
3713
+ type: string
3714
+ effort_support:
3715
+ type: boolean
3716
+ version_pinnable:
3717
+ type: boolean
3718
+ latest_version:
3719
+ type: [string, 'null']
3720
+
3721
+ ProviderCapability:
3722
+ type: object
3723
+ required: [name, default, sizing, refuses]
3724
+ properties:
3725
+ name:
3726
+ type: string
3727
+ default:
3728
+ type: boolean
3729
+ sizing:
3730
+ type: object
3731
+ required: [max_cpus, max_memory_mb, max_storage_mb, storage]
3732
+ properties:
3733
+ max_cpus:
3734
+ type: integer
3735
+ max_memory_mb:
3736
+ type: integer
3737
+ max_storage_mb:
3738
+ type: integer
3739
+ storage:
3740
+ type: string
3741
+ enum: [sized, fixed]
3742
+ refuses:
3743
+ type: array
3744
+ items:
3745
+ type: object
3746
+ required: [capability, reason]
3747
+ properties:
3748
+ capability:
3749
+ type: string
3750
+ reason:
3751
+ type: string
3752
+
3753
+ ManagedProviderCapability:
3754
+ type: object
3755
+ description: |
3756
+ One managed sandbox door and whether this deployment serves it.
3757
+ `configured` is derived from operator config presence only — it says
3758
+ nothing about whether the pass-through behind the door is deployed or
3759
+ the credential is valid. `agent_sessions` answers the separate
3760
+ question of whether the door can carry a full SDK agent session.
3761
+ required:
3762
+ - name
3763
+ - configured
3764
+ - requires_config
3765
+ - missing_config
3766
+ - agent_sessions
3767
+ - agent_sessions_reason
3768
+ properties:
3769
+ name:
3770
+ type: string
3771
+ configured:
3772
+ type: boolean
3773
+ requires_config:
3774
+ type: array
3775
+ items:
3776
+ type: string
3777
+ missing_config:
3778
+ description: The subset of `requires_config` missing right now — empty when configured.
3779
+ type: array
3780
+ items:
3781
+ type: string
3782
+ agent_sessions:
3783
+ type: boolean
3784
+ agent_sessions_reason:
3785
+ description: Why not, when `agent_sessions` is false. Null otherwise.
3786
+ type: [string, 'null']
3787
+
3788
+ CapabilityDocument:
3789
+ type: object
3790
+ description: |
3791
+ Everything a client would otherwise hardcode. Public and cacheable —
3792
+ no API key needed, so a signed-out page can populate its own agent
3793
+ picker. `schema_version` bumps when a FIELD changes meaning, never
3794
+ when a value changes.
3795
+ required:
3796
+ - schema_version
3797
+ - agents
3798
+ - agent_registration
3799
+ - sandbox_providers
3800
+ - managed_providers
3801
+ - platform_constraints
3802
+ - network_modes
3803
+ - statuses
3804
+ - limits
3805
+ - import_warning_codes
3806
+ - error_codes
3807
+ properties:
3808
+ schema_version:
3809
+ type: integer
3810
+ agents:
3811
+ type: array
3812
+ description: Built-in agents and their declared capabilities.
3813
+ items:
3814
+ $ref: '#/components/schemas/AgentCapability'
3815
+ agent_registration:
3816
+ type: object
3817
+ description: Rules a bring-your-own agent registration must satisfy.
3818
+ properties:
3819
+ name_pattern:
3820
+ type: string
3821
+ max_name_length:
3822
+ type: integer
3823
+ max_run_command_length:
3824
+ type: integer
3825
+ max_install_script_length:
3826
+ type: integer
3827
+ max_env_entries:
3828
+ type: integer
3829
+ max_per_user:
3830
+ type: integer
3831
+ max_upload_bytes:
3832
+ type: integer
3833
+ reserved_names:
3834
+ type: array
3835
+ description: Built-in names a registration may not reuse.
3836
+ items:
3837
+ type: string
3838
+ reserved_env_keys:
3839
+ type: array
3840
+ description: Env keys the platform owns; declaring one is refused.
3841
+ items:
3842
+ type: string
3843
+ sandbox_providers:
3844
+ type: array
3845
+ items:
3846
+ $ref: '#/components/schemas/ProviderCapability'
3847
+ managed_providers:
3848
+ type: array
3849
+ description: |
3850
+ The managed doors this deployment serves — a different question
3851
+ from `sandbox_providers`, which is about the eval lane. A managed
3852
+ sandbox is one the caller drives directly holding nothing but an
3853
+ Evolve key.
3854
+ items:
3855
+ $ref: '#/components/schemas/ManagedProviderCapability'
3856
+ platform_constraints:
3857
+ type: array
3858
+ description: Constraints that hold on EVERY provider.
3859
+ items:
3860
+ type: object
3861
+ required: [capability, reason]
3862
+ properties:
3863
+ capability:
3864
+ type: string
3865
+ reason:
3866
+ type: string
3867
+ network_modes:
3868
+ type: array
3869
+ items:
3870
+ type: string
3871
+ statuses:
3872
+ type: object
3873
+ description: Every status vocabulary on the surface, with terminal members.
3874
+ required: [job, trial, import, dataset_version]
3875
+ properties:
3876
+ job:
3877
+ $ref: '#/components/schemas/StatusVocabulary'
3878
+ trial:
3879
+ $ref: '#/components/schemas/StatusVocabulary'
3880
+ import:
3881
+ $ref: '#/components/schemas/StatusVocabulary'
3882
+ dataset_version:
3883
+ $ref: '#/components/schemas/StatusVocabulary'
3884
+ limits:
3885
+ type: object
3886
+ properties:
3887
+ job:
3888
+ type: object
3889
+ properties:
3890
+ max_n_attempts:
3891
+ type: integer
3892
+ max_agents:
3893
+ type: integer
3894
+ max_trials:
3895
+ type: integer
3896
+ n_concurrent_trials:
3897
+ type: object
3898
+ properties:
3899
+ default:
3900
+ type: integer
3901
+ max:
3902
+ type: integer
3903
+ default_max_trial_spend_usd:
3904
+ type: number
3905
+ default_sandbox_provider:
3906
+ type: string
3907
+ default_sizing:
3908
+ type: object
3909
+ properties:
3910
+ cpus:
3911
+ type: integer
3912
+ memory_mb:
3913
+ type: integer
3914
+ storage_mb:
3915
+ type: integer
3916
+ model_required:
3917
+ type: boolean
3918
+ description: Every agent must name a model; the server applies no default.
3919
+ default_agent_timeout_sec:
3920
+ type: number
3921
+ default_verifier_timeout_sec:
3922
+ type: number
3923
+ reasoning_efforts:
3924
+ type: array
3925
+ items:
3926
+ type: string
3927
+ default_reasoning_effort:
3928
+ type: string
3929
+ max_job_name_length:
3930
+ type: integer
3931
+ description: Longest accepted `job_name` (JobCreate mirrors it as maxLength).
3932
+ compare:
3933
+ type: object
3934
+ properties:
3935
+ min_ids:
3936
+ type: integer
3937
+ max_ids:
3938
+ type: integer
3939
+ stop:
3940
+ type: object
3941
+ properties:
3942
+ max_trial_ids:
3943
+ type: integer
3944
+ description: Most trials one stop request may name (StopRequest mirrors it as maxItems).
3945
+ pagination:
3946
+ type: object
3947
+ properties:
3948
+ collections:
3949
+ type: object
3950
+ properties:
3951
+ default:
3952
+ type: integer
3953
+ max:
3954
+ type: integer
3955
+ dataset_tasks:
3956
+ type: object
3957
+ properties:
3958
+ default:
3959
+ type: integer
3960
+ max:
3961
+ type: integer
3962
+ trace_events:
3963
+ type: object
3964
+ properties:
3965
+ default:
3966
+ type: integer
3967
+ max:
3968
+ type: integer
3969
+ uploads:
3970
+ type: object
3971
+ properties:
3972
+ dataset_archive_bytes:
3973
+ type: integer
3974
+ agent_tarball_bytes:
3975
+ type: integer
3976
+ dataset_names:
3977
+ type: object
3978
+ description: >
3979
+ Name constraints for published datasets. Beyond the pattern,
3980
+ `imports` is a reserved name (a literal path segment under
3981
+ `/api/datasets`).
3982
+ properties:
3983
+ pattern:
3984
+ type: string
3985
+ max_name_length:
3986
+ type: integer
3987
+ max_version_length:
3988
+ type: integer
3989
+ max_git_url_length:
3990
+ type: integer
3991
+ max_git_ref_length:
3992
+ type: integer
3993
+ max_items_named_in_error_message:
3994
+ type: integer
3995
+ description: How many items an error MESSAGE names before "and N more".
3996
+ import_warning_codes:
3997
+ type: array
3998
+ items:
3999
+ type: string
4000
+ error_codes:
4001
+ type: array
4002
+ description: The closed error-code union, enumerated at runtime.
4003
+ items:
4004
+ type: string
4005
+
4006
+ # -------------------------------------------------------------------------
4007
+ # Auth
4008
+ # -------------------------------------------------------------------------
4009
+
4010
+ AuthStatus:
4011
+ type: object
4012
+ description: The caller's identity and the key this request authenticated with.
4013
+ required: [user_id, key]
4014
+ properties:
4015
+ user_id:
4016
+ type: string
4017
+ email:
4018
+ type: [string, 'null']
4019
+ key:
4020
+ $ref: '#/components/schemas/ApiKey'
4021
+
4022
+ ApiKey:
4023
+ type: object
4024
+ description: A key descriptor. The secret is never returned.
4025
+ required: [id, created_at]
4026
+ properties:
4027
+ id:
4028
+ type: string
4029
+ label:
4030
+ type: [string, 'null']
4031
+ created_at:
4032
+ type: [string, 'null']
4033
+ format: date-time
4034
+ description: Null when the credential store predates creation stamping.
4035
+ last_used_at:
4036
+ type: [string, 'null']
4037
+ format: date-time
4038
+
4039
+ ApiKeyPage:
4040
+ type: object
4041
+ required: [items, nextCursor, hasMore]
4042
+ properties:
4043
+ items:
4044
+ type: array
4045
+ items:
4046
+ $ref: '#/components/schemas/ApiKey'
4047
+ nextCursor:
4048
+ type: [string, 'null']
4049
+ hasMore:
4050
+ type: boolean