@evolvingmachines/sdk 0.0.50 → 0.0.52-launch-round-1.20260803.37a56fe

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,4031 @@
1
+ openapi: 3.1.0
2
+
3
+ info:
4
+ title: Evolve Evaluations API
5
+ version: 2.0.0
6
+ summary: Hosted evaluations — datasets, jobs, trials, agents.
7
+ description: |
8
+ The single source of truth for the renamed hosted-evaluations surface.
9
+
10
+ ## Vocabulary
11
+ Four nouns, used everywhere and without exception:
12
+ - **dataset** — a named, versioned set of tasks.
13
+ - **job** — one run: datasets x agents x attempts.
14
+ - **trial** — one attempt of one task by one agent. Trial ids are globally
15
+ addressable: no route requires the job id to reach a trial.
16
+ - **agent** — the thing that attempts a task (a harness plus a model).
17
+ Registered private agents live under `/api/agents`.
18
+
19
+ Wire fields are `snake_case`, except the individually frozen historical
20
+ keys named in the deviations below — nothing else is exempt.
21
+
22
+ ## Documented deviations (kept deliberately)
23
+ - **Status enums.** `Job.status` (6 members) and `Trial.status` (8 members)
24
+ are load-bearing: they drive cancel, watch exit codes, resume and regrade
25
+ eligibility, and the `?status=` filters. `verifier_result` and
26
+ `exception_info` are published alongside, never instead.
27
+ - **Cursor pagination.** Every collection is `{items, nextCursor, hasMore}`
28
+ with opaque keyset cursors — these three keys are frozen verbatim.
29
+ Offset paging duplicates and skips rows on a live, growing collection.
30
+ - **Idempotency.** `Idempotency-Key` on every job-creating POST, surfaced
31
+ as `idempotent_replay` on the job body. A hosted double-run
32
+ double-charges a customer.
33
+ - **SSE events.** `GET /api/jobs/{jobId}/events` is the progress display
34
+ for a run that executes on someone else's machine. Event *type* names
35
+ are frozen; event payload fields speak this document's vocabulary.
36
+ - **Money.** Spend enforcement is ours: `max_trial_spend_usd` (the only
37
+ enforced budget — a per-trial cap minted onto the trial's gateway key),
38
+ `worst_case_spend_usd` (stated, never left to the caller to multiply),
39
+ and the live lower-bound pair `live_spent_usd` / `live_spend_at`.
40
+ Measured spend travels as `cost_usd` (job: `stats.cost_usd`; trial:
41
+ `agent_result.cost_usd`).
42
+ - **Typed error catalog.** A closed 52-code union; every error response is
43
+ one envelope: `{code, message, param, details, retryAfterSec, requestId}`
44
+ (envelope keys frozen verbatim).
45
+ - **Two frozen camelCase field keys.** `trials.byStatus` (the trial-status
46
+ tally on job bodies) and `taskMatrix` (on the compare response) keep
47
+ their exact historical spellings, same as the pagination and error
48
+ envelope keys above. These four spots are the complete list of
49
+ non-`snake_case` keys on the wire.
50
+ - **Cancel and stop.** `POST /api/jobs/{jobId}/cancel` and
51
+ `POST /api/trials/stop` exist because a hosted run cannot be Ctrl-C'd.
52
+ - **Resume semantics.** `POST /api/jobs/{jobId}/resume` creates a NEW
53
+ linked job (`source_jobs`), keeping the original immutable and
54
+ separately citable. It never mutates the source job.
55
+ - **Compare cap.** `GET /api/jobs/compare` takes 2–10 ids. Bounded, not
56
+ unbounded: one GET must not become an unbounded server-side fan-out.
57
+
58
+ ## Wave sequencing
59
+ Anything marked `x-wave: 2` belongs to the second build wave: today the
60
+ `trajectory` artifact selector (not yet served) and the API-key
61
+ management pair (served, not yet in the SDKs). Everything unmarked is
62
+ live end to end.
63
+
64
+ ## Deliberately absent (wave 3, not yet specified)
65
+ The rename map's additive rows are new capabilities, not renames, and are
66
+ intentionally NOT in this document: job sharing
67
+ (`GET /api/jobs/{jobId}/shares`, `POST /api/jobs/{jobId}/share`), copying
68
+ (`POST /api/jobs/{jobId}/copy`, `POST /api/trials/{trialId}/copy`), job
69
+ deletion (`DELETE /api/jobs/{jobId}`), dataset visibility
70
+ (`POST /api/datasets/{name}/visibility` — public/private toggling arrives
71
+ with the registry/sharing model), and the leaderboard family. They
72
+ will be specified when they are designed; their absence here is a
73
+ decision, not an omission.
74
+
75
+ servers:
76
+ - url: /
77
+ description: The platform origin. All paths are origin-relative.
78
+
79
+ security:
80
+ - apiKey: []
81
+
82
+ tags:
83
+ - name: jobs
84
+ description: Create, read, watch, cancel, resume, regrade, download, compare.
85
+ - name: trials
86
+ description: Globally addressable trials — read, trace, regrade, stop.
87
+ - name: datasets
88
+ description: Catalog, publish (async import), package download, activation.
89
+ - name: agents
90
+ description: Bring-your-own agents (private to their owner).
91
+ - name: meta
92
+ description: Public capability document.
93
+ - name: auth
94
+ description: Key and identity surface.
95
+
96
+ paths:
97
+ # ===========================================================================
98
+ # Jobs
99
+ # ===========================================================================
100
+
101
+ /api/jobs:
102
+ post:
103
+ tags: [jobs]
104
+ operationId: createJob
105
+ summary: Start a job
106
+ description: |
107
+ Starts a job over one or more catalog datasets. Accepted (202) as soon
108
+ as the trials are recorded; execution is asynchronous — follow
109
+ `GET /api/jobs/{jobId}/events` or poll `GET /api/jobs/{jobId}`.
110
+
111
+ Datasets are a LIST with per-dataset glob filters (`task_names`,
112
+ `exclude_task_names`, `n_tasks`); each selector filters only its own
113
+ dataset, and the job's trials are the union across the list. The same
114
+ dataset name twice is refused (`invalid_input`). Every agent must name
115
+ a model; the server applies no model default.
116
+
117
+ The response is the job body plus `job_id`, an additive alias of `id`
118
+ that only this verb answers with — see JobCreated.
119
+
120
+ Send `Idempotency-Key` to make retries safe: the same key with the
121
+ same resolved request replays the original job (200,
122
+ `idempotent_replay: true`); the same key with a different request is
123
+ refused with `idempotency_key_reused` (409). Dataset references are
124
+ fingerprinted RESOLVED — a bare name and the currently active version
125
+ fingerprint alike while that version is active. The dataset list is
126
+ fingerprinted IN ORDER: `datasets` echoes the request order back, so
127
+ the same selectors reordered are a different request and a shared key
128
+ across the two orders is refused rather than silently replayed.
129
+ parameters:
130
+ - $ref: '#/components/parameters/IdempotencyKey'
131
+ requestBody:
132
+ required: true
133
+ content:
134
+ application/json:
135
+ schema:
136
+ $ref: '#/components/schemas/JobCreate'
137
+ responses:
138
+ '202':
139
+ description: Job created and queued.
140
+ content:
141
+ application/json:
142
+ schema:
143
+ $ref: '#/components/schemas/JobCreated'
144
+ '200':
145
+ description: Idempotent replay of an existing identical request.
146
+ content:
147
+ application/json:
148
+ schema:
149
+ $ref: '#/components/schemas/JobCreated'
150
+ '400':
151
+ $ref: '#/components/responses/BadRequest'
152
+ '401':
153
+ $ref: '#/components/responses/Unauthorized'
154
+ '402':
155
+ description: Out of credits (`insufficient_credits`).
156
+ content:
157
+ application/json:
158
+ schema:
159
+ $ref: '#/components/schemas/Error'
160
+ '404':
161
+ $ref: '#/components/responses/NotFound'
162
+ '409':
163
+ $ref: '#/components/responses/Conflict'
164
+ '422':
165
+ description: >
166
+ Job too large (`job_too_large`) or a task refused by the chosen
167
+ provider (`provider_unsupported`).
168
+ content:
169
+ application/json:
170
+ schema:
171
+ $ref: '#/components/schemas/Error'
172
+ '429':
173
+ $ref: '#/components/responses/TooManyRequests'
174
+ '500':
175
+ $ref: '#/components/responses/InternalError'
176
+ get:
177
+ tags: [jobs]
178
+ operationId: listJobs
179
+ summary: List jobs
180
+ description: Newest first. Every filter is server-side.
181
+ parameters:
182
+ - $ref: '#/components/parameters/Limit'
183
+ - $ref: '#/components/parameters/Cursor'
184
+ - name: status
185
+ in: query
186
+ description: Filter by job status (repeatable).
187
+ schema:
188
+ type: array
189
+ items:
190
+ $ref: '#/components/schemas/JobStatus'
191
+ explode: true
192
+ - name: agent
193
+ in: query
194
+ description: Filter by agent name (repeatable).
195
+ schema:
196
+ type: array
197
+ items:
198
+ type: string
199
+ explode: true
200
+ - name: model
201
+ in: query
202
+ description: Filter by model name (repeatable).
203
+ schema:
204
+ type: array
205
+ items:
206
+ type: string
207
+ explode: true
208
+ - name: sandbox_provider
209
+ in: query
210
+ description: Filter by sandbox provider (repeatable).
211
+ schema:
212
+ type: array
213
+ items:
214
+ $ref: '#/components/schemas/SandboxProvider'
215
+ explode: true
216
+ - name: dataset
217
+ in: query
218
+ description: Filter to jobs that ran a given dataset name.
219
+ schema:
220
+ type: string
221
+ - name: is_regrade
222
+ in: query
223
+ description: >
224
+ true returns only regrade jobs; false only non-regrade jobs.
225
+ Replaces the retired standalone regrade listing — a regrade IS a
226
+ job.
227
+ schema:
228
+ type: boolean
229
+ - name: search
230
+ in: query
231
+ description: Free-text filter over job name and dataset names.
232
+ schema:
233
+ type: string
234
+ responses:
235
+ '200':
236
+ description: One page of jobs.
237
+ content:
238
+ application/json:
239
+ schema:
240
+ $ref: '#/components/schemas/JobPage'
241
+ '400':
242
+ $ref: '#/components/responses/BadRequest'
243
+ '401':
244
+ $ref: '#/components/responses/Unauthorized'
245
+ '429':
246
+ $ref: '#/components/responses/TooManyRequests'
247
+ '500':
248
+ $ref: '#/components/responses/InternalError'
249
+
250
+ /api/jobs/compare:
251
+ get:
252
+ tags: [jobs]
253
+ operationId: compareJobs
254
+ summary: Compare jobs
255
+ description: |
256
+ Side-by-side comparison of 2–10 owned jobs: per-job aggregates plus a
257
+ per-task matrix. Matrix cells report `MIXED` when a cell's trials
258
+ disagree in status and `MISSING` when a job has no trials for a task;
259
+ disagreement rows sort first.
260
+ parameters:
261
+ - name: ids
262
+ in: query
263
+ required: true
264
+ description: Comma-separated job ids (2–10).
265
+ schema:
266
+ type: string
267
+ examples: ["9f2c1a7e-...,4b8d0e21-..."]
268
+ responses:
269
+ '200':
270
+ description: Comparison payload.
271
+ content:
272
+ application/json:
273
+ schema:
274
+ $ref: '#/components/schemas/CompareResponse'
275
+ '400':
276
+ $ref: '#/components/responses/BadRequest'
277
+ '401':
278
+ $ref: '#/components/responses/Unauthorized'
279
+ '404':
280
+ $ref: '#/components/responses/NotFound'
281
+ '429':
282
+ $ref: '#/components/responses/TooManyRequests'
283
+ '500':
284
+ $ref: '#/components/responses/InternalError'
285
+
286
+ /api/jobs/{jobId}:
287
+ get:
288
+ tags: [jobs]
289
+ operationId: getJob
290
+ summary: Get one job
291
+ parameters:
292
+ - $ref: '#/components/parameters/JobId'
293
+ responses:
294
+ '200':
295
+ description: The job.
296
+ content:
297
+ application/json:
298
+ schema:
299
+ $ref: '#/components/schemas/Job'
300
+ '401':
301
+ $ref: '#/components/responses/Unauthorized'
302
+ '404':
303
+ $ref: '#/components/responses/NotFound'
304
+ '429':
305
+ $ref: '#/components/responses/TooManyRequests'
306
+ '500':
307
+ $ref: '#/components/responses/InternalError'
308
+
309
+ /api/jobs/{jobId}/cancel:
310
+ post:
311
+ tags: [jobs]
312
+ operationId: cancelJob
313
+ summary: Cancel a job
314
+ description: |
315
+ Requests cancellation: QUEUED/RUNNING moves to CANCELLING (compare-and-
316
+ set with retries), queued trials are cancelled outright, and in-flight
317
+ trials are STOPPED — sandboxes killed, spend read honestly — through
318
+ the same per-trial machinery as `POST /api/trials/stop` before the job
319
+ settles CANCELLED. Idempotent — calling it on a job already CANCELLING
320
+ or terminal returns the current body.
321
+ parameters:
322
+ - $ref: '#/components/parameters/JobId'
323
+ responses:
324
+ '200':
325
+ description: The job after the cancel request.
326
+ content:
327
+ application/json:
328
+ schema:
329
+ $ref: '#/components/schemas/Job'
330
+ '401':
331
+ $ref: '#/components/responses/Unauthorized'
332
+ '404':
333
+ $ref: '#/components/responses/NotFound'
334
+ '409':
335
+ description: Lost a concurrent status race after retries (`concurrent_update`).
336
+ content:
337
+ application/json:
338
+ schema:
339
+ $ref: '#/components/schemas/Error'
340
+ '429':
341
+ $ref: '#/components/responses/TooManyRequests'
342
+ '500':
343
+ $ref: '#/components/responses/InternalError'
344
+
345
+ /api/jobs/{jobId}/events:
346
+ get:
347
+ tags: [jobs]
348
+ operationId: watchJob
349
+ summary: Job event stream (SSE)
350
+ description: |
351
+ `text/event-stream` of the job's lifecycle. Each SSE message has
352
+ `id` = the event's monotonic `seq`, `event` = the event type, and
353
+ `data` = the JSON payload for that type.
354
+
355
+ Event types (frozen): `job.created`, `job.running`, `job.cancelling`,
356
+ `job.cancelled`, `job.completed`, `job.failed` (reserved — declared
357
+ terminal, no server path emits it today), `trial.running`,
358
+ `trial.scoring`, `trial.spend`, `trial.settled`.
359
+
360
+ Resume with the standard `Last-Event-ID` header or `?after=<seq>` —
361
+ both mean "events strictly after this sequence number". Heartbeat
362
+ comments (`:`) are sent every 15s; clients must skip comment lines.
363
+ parameters:
364
+ - $ref: '#/components/parameters/JobId'
365
+ - name: after
366
+ in: query
367
+ description: Resume position — deliver events with seq strictly greater.
368
+ schema:
369
+ type: integer
370
+ minimum: 0
371
+ - name: Last-Event-ID
372
+ in: header
373
+ description: Standard SSE resume header; same meaning as `?after=`.
374
+ schema:
375
+ type: string
376
+ responses:
377
+ '200':
378
+ description: The event stream. Ends after the terminal job event.
379
+ content:
380
+ text/event-stream:
381
+ schema:
382
+ $ref: '#/components/schemas/JobEvent'
383
+ '400':
384
+ description: Bad resume position (`invalid_after`).
385
+ content:
386
+ application/json:
387
+ schema:
388
+ $ref: '#/components/schemas/Error'
389
+ '401':
390
+ $ref: '#/components/responses/Unauthorized'
391
+ '404':
392
+ $ref: '#/components/responses/NotFound'
393
+ '429':
394
+ $ref: '#/components/responses/TooManyRequests'
395
+ '500':
396
+ $ref: '#/components/responses/InternalError'
397
+
398
+ /api/jobs/{jobId}/download:
399
+ get:
400
+ tags: [jobs]
401
+ operationId: downloadJob
402
+ summary: Download a job's results archive
403
+ description: |
404
+ A gzipped archive of the job in the standard results layout
405
+ (`config.json`, `result.json`, per-trial `config.json`/`result.json`).
406
+ Only terminal jobs are downloadable. The archive is deterministic:
407
+ sorted keys at every depth, no generation timestamp, zero gzip mtime.
408
+ Task environment identity only — instructions, verifier commands, and
409
+ test files never leave the server.
410
+ parameters:
411
+ - $ref: '#/components/parameters/JobId'
412
+ responses:
413
+ '200':
414
+ description: The archive.
415
+ content:
416
+ application/gzip:
417
+ schema:
418
+ type: string
419
+ format: binary
420
+ '401':
421
+ $ref: '#/components/responses/Unauthorized'
422
+ '404':
423
+ $ref: '#/components/responses/NotFound'
424
+ '409':
425
+ description: Job is not terminal (`job_not_terminal`).
426
+ content:
427
+ application/json:
428
+ schema:
429
+ $ref: '#/components/schemas/Error'
430
+ '429':
431
+ $ref: '#/components/responses/TooManyRequests'
432
+ '500':
433
+ $ref: '#/components/responses/InternalError'
434
+
435
+ /api/jobs/{jobId}/resume:
436
+ post:
437
+ tags: [jobs]
438
+ operationId: resumeJob
439
+ summary: Resume a job (new linked job over its failed and stopped trials)
440
+ description: |
441
+ Creates a NEW job holding fresh trials for the source job's failed
442
+ and stopped work, linked back via `source_jobs` (`action: "resume"`).
443
+ The source job is never mutated — the original run stays immutable and
444
+ separately citable.
445
+
446
+ `filter_error_types` selects which failures to resume by their
447
+ `exception_info.exception_type`; omitted, the default set is
448
+ `["ScoringError", "InfrastructureError", "IncompleteTrialError"]`
449
+ plus stopped trials (settled CANCELLED, exception type
450
+ `CancelledError`) and any still-QUEUED trials of a cancelled source.
451
+ The source job must be terminal.
452
+
453
+ Supports `Idempotency-Key` with a fingerprint namespaced to this
454
+ route, so a create key can never replay a resume.
455
+ parameters:
456
+ - $ref: '#/components/parameters/JobId'
457
+ - $ref: '#/components/parameters/IdempotencyKey'
458
+ requestBody:
459
+ required: false
460
+ content:
461
+ application/json:
462
+ schema:
463
+ $ref: '#/components/schemas/ResumeRequest'
464
+ responses:
465
+ '202':
466
+ description: The new linked job.
467
+ content:
468
+ application/json:
469
+ schema:
470
+ $ref: '#/components/schemas/Job'
471
+ '200':
472
+ description: Idempotent replay of an identical resume request.
473
+ content:
474
+ application/json:
475
+ schema:
476
+ $ref: '#/components/schemas/Job'
477
+ '400':
478
+ $ref: '#/components/responses/BadRequest'
479
+ '401':
480
+ $ref: '#/components/responses/Unauthorized'
481
+ '402':
482
+ description: Out of credits (`insufficient_credits`).
483
+ content:
484
+ application/json:
485
+ schema:
486
+ $ref: '#/components/schemas/Error'
487
+ '404':
488
+ $ref: '#/components/responses/NotFound'
489
+ '409':
490
+ description: >
491
+ Source not terminal (`job_not_terminal`), nothing to resume
492
+ (`no_failed_trials`), or `idempotency_key_reused`.
493
+ content:
494
+ application/json:
495
+ schema:
496
+ $ref: '#/components/schemas/Error'
497
+ '429':
498
+ $ref: '#/components/responses/TooManyRequests'
499
+ '500':
500
+ $ref: '#/components/responses/InternalError'
501
+
502
+ /api/jobs/{jobId}/regrade:
503
+ post:
504
+ tags: [jobs]
505
+ operationId: regradeJob
506
+ summary: Regrade a job (verifier-only re-run)
507
+ description: |
508
+ Re-runs verification over a terminal job's trials. **The response is a
509
+ Job**: a regrade is an ordinary job whose `source_jobs` records
510
+ `action: "regrade"` and whose `is_regrade` is true. View it with
511
+ `GET /api/jobs/{jobId}`; there is no separate regrade resource.
512
+ The source job and its trials are immutable.
513
+
514
+ The optional filter narrows which trials are regraded. Decision order:
515
+ 404 (unknown job) → 409 `job_not_terminal` → 409 `no_regradable_trials`
516
+ → 202.
517
+ parameters:
518
+ - $ref: '#/components/parameters/JobId'
519
+ requestBody:
520
+ required: false
521
+ content:
522
+ application/json:
523
+ schema:
524
+ $ref: '#/components/schemas/RegradeRequest'
525
+ responses:
526
+ '202':
527
+ description: The regrade job.
528
+ content:
529
+ application/json:
530
+ schema:
531
+ $ref: '#/components/schemas/Job'
532
+ '400':
533
+ $ref: '#/components/responses/BadRequest'
534
+ '401':
535
+ $ref: '#/components/responses/Unauthorized'
536
+ '404':
537
+ $ref: '#/components/responses/NotFound'
538
+ '409':
539
+ description: >
540
+ `job_not_terminal`, `no_regradable_trials`, or
541
+ `regrade_source_ineligible` (a pre-cutover job with no inputs
542
+ bundle can never be regraded).
543
+ content:
544
+ application/json:
545
+ schema:
546
+ $ref: '#/components/schemas/Error'
547
+ '429':
548
+ $ref: '#/components/responses/TooManyRequests'
549
+ '500':
550
+ $ref: '#/components/responses/InternalError'
551
+
552
+ /api/jobs/{jobId}/trials:
553
+ get:
554
+ tags: [jobs]
555
+ operationId: listJobTrials
556
+ summary: List a job's trials
557
+ description: |
558
+ One page of the job's trials. List rows truncate
559
+ `exception_info.exception_message` to 2000 characters; the trial
560
+ detail route returns it untruncated. Reading a page refreshes the
561
+ live-spend lower bound on in-flight trials.
562
+ parameters:
563
+ - $ref: '#/components/parameters/JobId'
564
+ - $ref: '#/components/parameters/Limit'
565
+ - $ref: '#/components/parameters/Cursor'
566
+ - name: status
567
+ in: query
568
+ description: Filter by trial status — repeatable and/or comma-separated.
569
+ schema:
570
+ type: array
571
+ items:
572
+ $ref: '#/components/schemas/TrialStatus'
573
+ explode: true
574
+ - name: task_name
575
+ in: query
576
+ description: Filter to one task's trials.
577
+ schema:
578
+ type: string
579
+ - name: dataset
580
+ in: query
581
+ description: >
582
+ Filter to one dataset's trials — exact match on the trial's
583
+ `source`. A name the job does not span is an ordinary empty
584
+ selection.
585
+ schema:
586
+ type: string
587
+ - name: failed_only
588
+ in: query
589
+ description: Only trials in a failure status.
590
+ schema:
591
+ type: boolean
592
+ responses:
593
+ '200':
594
+ description: One page of trials.
595
+ content:
596
+ application/json:
597
+ schema:
598
+ $ref: '#/components/schemas/TrialPage'
599
+ '400':
600
+ $ref: '#/components/responses/BadRequest'
601
+ '401':
602
+ $ref: '#/components/responses/Unauthorized'
603
+ '404':
604
+ $ref: '#/components/responses/NotFound'
605
+ '429':
606
+ $ref: '#/components/responses/TooManyRequests'
607
+ '500':
608
+ $ref: '#/components/responses/InternalError'
609
+
610
+ /api/jobs/{jobId}/tasks:
611
+ get:
612
+ tags: [jobs]
613
+ operationId: listJobTasks
614
+ summary: Per-task rollup of a job
615
+ description: |
616
+ One row per distinct task: its trial-status histogram, mean reward
617
+ over SCORED trials, and measured cost. Sits between the job body and
618
+ the trial list so a caller need not fetch every trial to see which
619
+ tasks are dragging.
620
+ parameters:
621
+ - $ref: '#/components/parameters/JobId'
622
+ - $ref: '#/components/parameters/Limit'
623
+ - $ref: '#/components/parameters/Cursor'
624
+ responses:
625
+ '200':
626
+ description: One page of task rollups.
627
+ content:
628
+ application/json:
629
+ schema:
630
+ $ref: '#/components/schemas/JobTaskRollupPage'
631
+ '400':
632
+ $ref: '#/components/responses/BadRequest'
633
+ '401':
634
+ $ref: '#/components/responses/Unauthorized'
635
+ '404':
636
+ $ref: '#/components/responses/NotFound'
637
+ '429':
638
+ $ref: '#/components/responses/TooManyRequests'
639
+ '500':
640
+ $ref: '#/components/responses/InternalError'
641
+
642
+ # ===========================================================================
643
+ # Trials (globally addressable)
644
+ # ===========================================================================
645
+
646
+ /api/trials/{trialId}:
647
+ get:
648
+ tags: [trials]
649
+ operationId: getTrial
650
+ summary: Get one trial
651
+ description: >
652
+ A trial id is globally addressable — no job id in the path. The body
653
+ carries `job_id` for the reverse pointer. Detail responses return
654
+ `exception_info` untruncated.
655
+ parameters:
656
+ - $ref: '#/components/parameters/TrialId'
657
+ responses:
658
+ '200':
659
+ description: The trial.
660
+ content:
661
+ application/json:
662
+ schema:
663
+ $ref: '#/components/schemas/Trial'
664
+ '401':
665
+ $ref: '#/components/responses/Unauthorized'
666
+ '404':
667
+ $ref: '#/components/responses/NotFound'
668
+ '429':
669
+ $ref: '#/components/responses/TooManyRequests'
670
+ '500':
671
+ $ref: '#/components/responses/InternalError'
672
+
673
+ /api/trials/{trialId}/trace:
674
+ get:
675
+ tags: [trials]
676
+ operationId: getTrialTrace
677
+ summary: Trial trace and artifacts
678
+ description: |
679
+ Without `stream` (or with `stream=trace-parsed`, the same answer by
680
+ its vocabulary name): one page of parsed trace events, paged by
681
+ `cursor` (a sequence number) and `limit` (default 200, max 1000).
682
+
683
+ With `stream`: raw artifacts instead of parsed events —
684
+ - `verifier`, `trace-stdout`, `trace-stderr`, `trajectory` (wave 2):
685
+ the log text as JSON (`{"log": string | null}`; null means this
686
+ trial never stored that log — a normal answer, never an error).
687
+ With `format=log` a present log is delivered as a `text/plain`
688
+ attachment; an absent one still answers the JSON `{"log": null}`.
689
+ - `agent-home`: the agent home tree as JSON
690
+ (`{"files": {"<sandbox-path>": "<text>", ...} | null}`; null means
691
+ nothing was collected). With `format=tgz` a present tree is
692
+ delivered as a gzipped tarball; an absent one still answers the
693
+ JSON `{"files": null}`.
694
+ parameters:
695
+ - $ref: '#/components/parameters/TrialId'
696
+ - name: cursor
697
+ in: query
698
+ description: Sequence position to page from (parsed-event mode).
699
+ schema:
700
+ type: string
701
+ - name: limit
702
+ in: query
703
+ description: Page size (parsed-event mode). Default 200, max 1000.
704
+ schema:
705
+ type: integer
706
+ minimum: 1
707
+ maximum: 1000
708
+ default: 200
709
+ - name: stream
710
+ in: query
711
+ description: >
712
+ Raw artifact selector. `trajectory` is in the vocabulary ahead of
713
+ its server wave: until that wave lands the route answers
714
+ not-found for it, reported as the API error it is.
715
+ schema:
716
+ type: string
717
+ enum:
718
+ - trace-parsed # the parsed event trace — same answer as omitting
719
+ - verifier
720
+ - trace-stdout
721
+ - trace-stderr
722
+ - trajectory # x-wave: 2 — normalized-trajectory artifact slot
723
+ - agent-home
724
+ - name: format
725
+ in: query
726
+ description: >
727
+ `log` (text/plain, log selectors only) or `tgz` (gzipped tarball,
728
+ agent-home only).
729
+ schema:
730
+ type: string
731
+ enum: [log, tgz]
732
+ responses:
733
+ '200':
734
+ description: Trace events, or the selected artifact.
735
+ content:
736
+ application/json:
737
+ schema:
738
+ oneOf:
739
+ - $ref: '#/components/schemas/TraceEventPage'
740
+ - type: object
741
+ description: >
742
+ Log selector JSON form; `log` is null when this trial
743
+ never stored that log — a normal answer, never an error.
744
+ required: [log]
745
+ properties:
746
+ log:
747
+ type: [string, 'null']
748
+ - type: object
749
+ description: >
750
+ Agent-home JSON form — sandbox path to file text; `files`
751
+ is null when nothing was collected, which no other shape
752
+ can express.
753
+ required: [files]
754
+ properties:
755
+ files:
756
+ type: [object, 'null']
757
+ additionalProperties:
758
+ type: string
759
+ text/plain:
760
+ schema:
761
+ type: string
762
+ description: '`format=log` delivery of a log selector.'
763
+ application/gzip:
764
+ schema:
765
+ type: string
766
+ format: binary
767
+ description: '`format=tgz` delivery of agent-home.'
768
+ '400':
769
+ $ref: '#/components/responses/BadRequest'
770
+ '401':
771
+ $ref: '#/components/responses/Unauthorized'
772
+ '404':
773
+ $ref: '#/components/responses/NotFound'
774
+ '429':
775
+ $ref: '#/components/responses/TooManyRequests'
776
+ '500':
777
+ $ref: '#/components/responses/InternalError'
778
+
779
+ /api/trials/{trialId}/regrade:
780
+ post:
781
+ tags: [trials]
782
+ operationId: regradeTrial
783
+ summary: Regrade one trial
784
+ description: |
785
+ Verifier-only re-run of a single trial. **The response is a Job** — a
786
+ one-trial regrade job with `source_jobs` recording the provenance.
787
+ The source trial is immutable.
788
+ parameters:
789
+ - $ref: '#/components/parameters/TrialId'
790
+ responses:
791
+ '202':
792
+ description: The regrade job.
793
+ content:
794
+ application/json:
795
+ schema:
796
+ $ref: '#/components/schemas/Job'
797
+ '401':
798
+ $ref: '#/components/responses/Unauthorized'
799
+ '404':
800
+ $ref: '#/components/responses/NotFound'
801
+ '409':
802
+ description: >
803
+ `job_not_terminal`, `no_regradable_trials`, or
804
+ `regrade_source_ineligible`.
805
+ content:
806
+ application/json:
807
+ schema:
808
+ $ref: '#/components/schemas/Error'
809
+ '429':
810
+ $ref: '#/components/responses/TooManyRequests'
811
+ '500':
812
+ $ref: '#/components/responses/InternalError'
813
+
814
+ /api/trials/stop:
815
+ post:
816
+ tags: [trials]
817
+ operationId: stopTrials
818
+ summary: Stop selected running trials
819
+ description: |
820
+ Stops individual in-flight trials without cancelling their job: each
821
+ trial's sandbox is killed and the trial is settled with its spend read
822
+ from the gateway. Only the caller's own trials; ids belonging to
823
+ someone else are reported in `not_found` (existence is never leaked).
824
+ Idempotent — already-terminal trials are reported as such and left
825
+ untouched.
826
+ requestBody:
827
+ required: true
828
+ content:
829
+ application/json:
830
+ schema:
831
+ $ref: '#/components/schemas/StopRequest'
832
+ responses:
833
+ '200':
834
+ description: Per-trial outcome of the stop request.
835
+ content:
836
+ application/json:
837
+ schema:
838
+ $ref: '#/components/schemas/StopResponse'
839
+ '400':
840
+ $ref: '#/components/responses/BadRequest'
841
+ '401':
842
+ $ref: '#/components/responses/Unauthorized'
843
+ '429':
844
+ $ref: '#/components/responses/TooManyRequests'
845
+ '500':
846
+ $ref: '#/components/responses/InternalError'
847
+
848
+ # ===========================================================================
849
+ # Datasets
850
+ # ===========================================================================
851
+
852
+ /api/datasets:
853
+ get:
854
+ tags: [datasets]
855
+ operationId: listDatasets
856
+ summary: List datasets
857
+ description: >
858
+ The catalog: public datasets plus the caller's own private ones.
859
+ Another user's private dataset is invisible, never 403. A dataset
860
+ whose every version is FAILED is not listed either — no partial
861
+ corpus ever exists, and that includes the catalog view. The name
862
+ still answers `GET /api/datasets/{name}` (and DELETE), and the
863
+ failed import stays readable at its import id, so nothing about the
864
+ failure is hidden; re-publishing the same name simply retries it.
865
+ parameters:
866
+ - $ref: '#/components/parameters/Limit'
867
+ - $ref: '#/components/parameters/Cursor'
868
+ - name: search
869
+ in: query
870
+ description: Free-text filter over name and description.
871
+ schema:
872
+ type: string
873
+ responses:
874
+ '200':
875
+ description: One page of datasets (summary fields only).
876
+ content:
877
+ application/json:
878
+ schema:
879
+ $ref: '#/components/schemas/DatasetPage'
880
+ '400':
881
+ $ref: '#/components/responses/BadRequest'
882
+ '401':
883
+ $ref: '#/components/responses/Unauthorized'
884
+ '429':
885
+ $ref: '#/components/responses/TooManyRequests'
886
+ '500':
887
+ $ref: '#/components/responses/InternalError'
888
+
889
+ /api/datasets/{name}:
890
+ get:
891
+ tags: [datasets]
892
+ operationId: getDataset
893
+ summary: Get one dataset
894
+ description: >
895
+ Detail view: all versions, the active version, upstream status, and
896
+ one page of the selected version's tasks (`limit`/`cursor` page the
897
+ task list; `version` selects which version's tasks are listed).
898
+ parameters:
899
+ - $ref: '#/components/parameters/DatasetName'
900
+ - name: version
901
+ in: query
902
+ description: Version whose tasks to list; default the active version.
903
+ schema:
904
+ type: string
905
+ - $ref: '#/components/parameters/Limit'
906
+ - $ref: '#/components/parameters/Cursor'
907
+ responses:
908
+ '200':
909
+ description: The dataset.
910
+ content:
911
+ application/json:
912
+ schema:
913
+ $ref: '#/components/schemas/Dataset'
914
+ '400':
915
+ $ref: '#/components/responses/BadRequest'
916
+ '401':
917
+ $ref: '#/components/responses/Unauthorized'
918
+ '404':
919
+ $ref: '#/components/responses/NotFound'
920
+ '429':
921
+ $ref: '#/components/responses/TooManyRequests'
922
+ '500':
923
+ $ref: '#/components/responses/InternalError'
924
+ patch:
925
+ tags: [datasets]
926
+ operationId: updateDataset
927
+ summary: Update dataset settings
928
+ description: |
929
+ The only settable field is `upstream_auto_import`: automatically
930
+ import a new version when the dataset's upstream git ref moves.
931
+ Refused with `upstream_not_watchable` (409) when the dataset has no
932
+ moving git ref to follow, and 403 on platform-curated datasets.
933
+ parameters:
934
+ - $ref: '#/components/parameters/DatasetName'
935
+ requestBody:
936
+ required: true
937
+ content:
938
+ application/json:
939
+ schema:
940
+ $ref: '#/components/schemas/DatasetPatch'
941
+ responses:
942
+ '200':
943
+ description: The updated dataset.
944
+ content:
945
+ application/json:
946
+ schema:
947
+ $ref: '#/components/schemas/Dataset'
948
+ '400':
949
+ $ref: '#/components/responses/BadRequest'
950
+ '401':
951
+ $ref: '#/components/responses/Unauthorized'
952
+ '403':
953
+ description: Platform-curated dataset (`dataset_not_owned`).
954
+ content:
955
+ application/json:
956
+ schema:
957
+ $ref: '#/components/schemas/Error'
958
+ '404':
959
+ $ref: '#/components/responses/NotFound'
960
+ '409':
961
+ description: Nothing to watch (`upstream_not_watchable`).
962
+ content:
963
+ application/json:
964
+ schema:
965
+ $ref: '#/components/schemas/Error'
966
+ '429':
967
+ $ref: '#/components/responses/TooManyRequests'
968
+ '500':
969
+ $ref: '#/components/responses/InternalError'
970
+ delete:
971
+ tags: [datasets]
972
+ operationId: deleteDataset
973
+ summary: Delete an owned dataset
974
+ parameters:
975
+ - $ref: '#/components/parameters/DatasetName'
976
+ responses:
977
+ '204':
978
+ description: Deleted.
979
+ '401':
980
+ $ref: '#/components/responses/Unauthorized'
981
+ '403':
982
+ description: Not the owner (`dataset_not_owned`).
983
+ content:
984
+ application/json:
985
+ schema:
986
+ $ref: '#/components/schemas/Error'
987
+ '404':
988
+ $ref: '#/components/responses/NotFound'
989
+ '409':
990
+ description: >
991
+ In use by jobs (`dataset_in_use`); `details` names the blocking
992
+ job ids.
993
+ content:
994
+ application/json:
995
+ schema:
996
+ $ref: '#/components/schemas/Error'
997
+ '429':
998
+ $ref: '#/components/responses/TooManyRequests'
999
+ '500':
1000
+ $ref: '#/components/responses/InternalError'
1001
+
1002
+ /api/datasets/{name}/download:
1003
+ get:
1004
+ tags: [datasets]
1005
+ operationId: downloadDataset
1006
+ summary: Download a dataset version's source package
1007
+ description: |
1008
+ The original uploaded corpus tarball for one of the caller's own
1009
+ dataset versions (owner only). The server verifies the stored sha256
1010
+ before streaming and echoes it in `x-package-sha256`; clients should
1011
+ verify the digest and the Content-Length.
1012
+ parameters:
1013
+ - $ref: '#/components/parameters/DatasetName'
1014
+ - name: version
1015
+ in: query
1016
+ description: Version to download; default the active version.
1017
+ schema:
1018
+ type: string
1019
+ responses:
1020
+ '200':
1021
+ description: The gzipped source package.
1022
+ headers:
1023
+ x-package-sha256:
1024
+ description: Hex sha256 of the exact bytes streamed.
1025
+ schema:
1026
+ type: string
1027
+ pattern: '^[a-f0-9]{64}$'
1028
+ content:
1029
+ application/gzip:
1030
+ schema:
1031
+ type: string
1032
+ format: binary
1033
+ '401':
1034
+ $ref: '#/components/responses/Unauthorized'
1035
+ '404':
1036
+ $ref: '#/components/responses/NotFound'
1037
+ '409':
1038
+ description: >
1039
+ Package unavailable: `package_not_retained`, `package_corrupt`,
1040
+ or `package_missing`.
1041
+ content:
1042
+ application/json:
1043
+ schema:
1044
+ $ref: '#/components/schemas/Error'
1045
+ '429':
1046
+ $ref: '#/components/responses/TooManyRequests'
1047
+ '500':
1048
+ $ref: '#/components/responses/InternalError'
1049
+
1050
+ /api/datasets/{name}/versions/{version}/activate:
1051
+ post:
1052
+ tags: [datasets]
1053
+ operationId: activateDatasetVersion
1054
+ summary: Activate a dataset version
1055
+ description: |
1056
+ Promotes a READY (or VALIDATING-complete) version to the dataset's
1057
+ active version, making bare-name job references resolve to it.
1058
+
1059
+ You usually do not need this. A version you publish is activated for
1060
+ you the moment its activation gate passes — VALIDATING becomes READY
1061
+ and the dataset's default points at it, with no call. What this verb
1062
+ is for is choosing a DIFFERENT default: rolling back to an older
1063
+ READY version, or re-pointing at one auto-activation moved past.
1064
+
1065
+ The proof it requires is the same activation gate, which the worker
1066
+ schedules automatically after a hosted import: the gold solution
1067
+ scores exactly 1.0 through the real execution path, and a do-nothing
1068
+ agent does not. While the gate is scheduled or running the verb
1069
+ answers 202 `gate_running` (a healthy in-progress state, not an
1070
+ error); once it lands, activation either promotes (200) or explains
1071
+ (409 `version_not_ready`, with the gate's failure detail in
1072
+ `details.gate_failure`). Refused with `version_not_activatable` when
1073
+ the version can never be activated (for example, no reference
1074
+ solutions were archived — see the import's `warnings`).
1075
+
1076
+ Idempotent: activating the version that is already active rewrites
1077
+ the same pointer and answers 200.
1078
+ parameters:
1079
+ - $ref: '#/components/parameters/DatasetName'
1080
+ - name: version
1081
+ in: path
1082
+ required: true
1083
+ schema:
1084
+ type: string
1085
+ responses:
1086
+ '200':
1087
+ description: The dataset with its new active version.
1088
+ content:
1089
+ application/json:
1090
+ schema:
1091
+ $ref: '#/components/schemas/Dataset'
1092
+ '202':
1093
+ description: >
1094
+ Not yet — the activation gate is scheduled or running for this
1095
+ version. Nothing is wrong and nothing is asked of the caller:
1096
+ poll the dataset (each version carries `gate`). A gate that
1097
+ PASSES activates the version itself, so there is normally
1098
+ nothing left to call.
1099
+ content:
1100
+ application/json:
1101
+ schema:
1102
+ $ref: '#/components/schemas/GateRunning'
1103
+ '401':
1104
+ $ref: '#/components/responses/Unauthorized'
1105
+ '403':
1106
+ description: Not the owner (`dataset_not_owned`).
1107
+ content:
1108
+ application/json:
1109
+ schema:
1110
+ $ref: '#/components/schemas/Error'
1111
+ '404':
1112
+ $ref: '#/components/responses/NotFound'
1113
+ '409':
1114
+ description: '`version_not_ready` or `version_not_activatable`.'
1115
+ content:
1116
+ application/json:
1117
+ schema:
1118
+ $ref: '#/components/schemas/Error'
1119
+ '429':
1120
+ $ref: '#/components/responses/TooManyRequests'
1121
+ '500':
1122
+ $ref: '#/components/responses/InternalError'
1123
+
1124
+ /api/datasets/publish:
1125
+ post:
1126
+ tags: [datasets]
1127
+ operationId: publishDataset
1128
+ summary: Publish a dataset version (async import)
1129
+ description: |
1130
+ Starts a server-side import that creates or extends a catalog dataset
1131
+ with one new immutable version. Multipart only. Source is exactly one
1132
+ of: a git repository (`git_url` + pinned `git_ref` — https only, an
1133
+ unpinned import is not reproducible) or an uploaded gzipped tarball
1134
+ (`archive`).
1135
+
1136
+ The import is asynchronous: the 202 returns a DatasetImport to poll at
1137
+ `GET /api/datasets/imports/{importId}`. Name collisions are checked
1138
+ before the upload lands; oversized uploads are fast-rejected on
1139
+ Content-Length (413). The name `imports` is reserved (it is a literal
1140
+ path segment under `/api/datasets`) and is refused like any other
1141
+ invalid name.
1142
+
1143
+ Once the import COMPLETES, the worker automatically schedules the
1144
+ activation gate (the version's `gate` field tracks it: PENDING ->
1145
+ RUNNING -> PASSED/FAILED). When it PASSES, the version becomes READY
1146
+ and the dataset's active version on its own — publish, poll until
1147
+ `gate.status` is PASSED, and run the dataset by bare name. A version
1148
+ with no archived reference solutions gets no gate — its import
1149
+ `warnings` say so, and activation is refused
1150
+ `version_not_activatable`.
1151
+ requestBody:
1152
+ required: true
1153
+ content:
1154
+ multipart/form-data:
1155
+ schema:
1156
+ $ref: '#/components/schemas/PublishRequest'
1157
+ responses:
1158
+ '202':
1159
+ description: The import job.
1160
+ content:
1161
+ application/json:
1162
+ schema:
1163
+ $ref: '#/components/schemas/DatasetImport'
1164
+ '400':
1165
+ $ref: '#/components/responses/BadRequest'
1166
+ '401':
1167
+ $ref: '#/components/responses/Unauthorized'
1168
+ '409':
1169
+ description: '`dataset_name_taken` or `dataset_version_not_found` conflicts.'
1170
+ content:
1171
+ application/json:
1172
+ schema:
1173
+ $ref: '#/components/schemas/Error'
1174
+ '413':
1175
+ description: Archive over the size cap (`import_too_large`).
1176
+ content:
1177
+ application/json:
1178
+ schema:
1179
+ $ref: '#/components/schemas/Error'
1180
+ '429':
1181
+ $ref: '#/components/responses/TooManyRequests'
1182
+ '500':
1183
+ $ref: '#/components/responses/InternalError'
1184
+
1185
+ /api/datasets/imports:
1186
+ get:
1187
+ tags: [datasets]
1188
+ operationId: listDatasetImports
1189
+ summary: List the caller's imports
1190
+ parameters:
1191
+ - $ref: '#/components/parameters/Limit'
1192
+ - $ref: '#/components/parameters/Cursor'
1193
+ - name: status
1194
+ in: query
1195
+ schema:
1196
+ $ref: '#/components/schemas/DatasetImportStatus'
1197
+ - name: dataset
1198
+ in: query
1199
+ description: Filter to imports of one dataset name.
1200
+ schema:
1201
+ type: string
1202
+ responses:
1203
+ '200':
1204
+ description: One page of imports.
1205
+ content:
1206
+ application/json:
1207
+ schema:
1208
+ $ref: '#/components/schemas/DatasetImportPage'
1209
+ '400':
1210
+ $ref: '#/components/responses/BadRequest'
1211
+ '401':
1212
+ $ref: '#/components/responses/Unauthorized'
1213
+ '429':
1214
+ $ref: '#/components/responses/TooManyRequests'
1215
+ '500':
1216
+ $ref: '#/components/responses/InternalError'
1217
+
1218
+ /api/datasets/imports/{importId}:
1219
+ get:
1220
+ tags: [datasets]
1221
+ operationId: getDatasetImport
1222
+ summary: Poll one import
1223
+ description: >
1224
+ Imports of public datasets are pollable by anyone; private ones are
1225
+ owner-only and read 404 (never 403) for anyone else.
1226
+ parameters:
1227
+ - name: importId
1228
+ in: path
1229
+ required: true
1230
+ schema:
1231
+ type: string
1232
+ responses:
1233
+ '200':
1234
+ description: The import.
1235
+ content:
1236
+ application/json:
1237
+ schema:
1238
+ $ref: '#/components/schemas/DatasetImport'
1239
+ '401':
1240
+ $ref: '#/components/responses/Unauthorized'
1241
+ '404':
1242
+ $ref: '#/components/responses/NotFound'
1243
+ '429':
1244
+ $ref: '#/components/responses/TooManyRequests'
1245
+ '500':
1246
+ $ref: '#/components/responses/InternalError'
1247
+
1248
+ # ===========================================================================
1249
+ # Agents (bring-your-own)
1250
+ # ===========================================================================
1251
+
1252
+ /api/agents:
1253
+ post:
1254
+ tags: [agents]
1255
+ operationId: registerAgent
1256
+ summary: Register a private agent
1257
+ description: |
1258
+ Registers a bring-your-own agent. Once registered, its `name` is
1259
+ usable in `agents[].name` on job creation exactly like a built-in.
1260
+ Source is exactly one of: an `install_script` (runs in a throwaway
1261
+ builder sandbox with internet and ZERO secrets; must leave executables
1262
+ in `$PREFIX/bin`) or an uploaded gzipped tarball (`archive`).
1263
+ Multipart only — nothing travels in the query string.
1264
+
1265
+ Registration validates SHAPE only and never executes the install
1266
+ script: the bundle is built by the first trial that names the agent,
1267
+ and a failing build settles that trial as an infrastructure error
1268
+ with the builder's stdout/stderr stored on it (readable through the
1269
+ trial's `trace-stdout` / `trace-stderr` streams). Line endings in
1270
+ `install_script` and `run_command` are normalized CRLF -> LF on
1271
+ receipt — multipart transports rewrite text-field newlines to CRLF,
1272
+ which is never meaningful in shell text.
1273
+ requestBody:
1274
+ required: true
1275
+ content:
1276
+ multipart/form-data:
1277
+ schema:
1278
+ $ref: '#/components/schemas/AgentRegistration'
1279
+ responses:
1280
+ '201':
1281
+ description: The registered agent.
1282
+ content:
1283
+ application/json:
1284
+ schema:
1285
+ $ref: '#/components/schemas/Agent'
1286
+ '400':
1287
+ $ref: '#/components/responses/BadRequest'
1288
+ '401':
1289
+ $ref: '#/components/responses/Unauthorized'
1290
+ '409':
1291
+ description: >
1292
+ `agent_name_taken`, `agent_name_reserved`, `agent_source_conflict`,
1293
+ or `agent_limit_reached`.
1294
+ content:
1295
+ application/json:
1296
+ schema:
1297
+ $ref: '#/components/schemas/Error'
1298
+ '413':
1299
+ description: Upload over the size cap (`agent_too_large`).
1300
+ content:
1301
+ application/json:
1302
+ schema:
1303
+ $ref: '#/components/schemas/Error'
1304
+ '429':
1305
+ $ref: '#/components/responses/TooManyRequests'
1306
+ '500':
1307
+ $ref: '#/components/responses/InternalError'
1308
+ get:
1309
+ tags: [agents]
1310
+ operationId: listAgents
1311
+ summary: List the caller's registered agents
1312
+ parameters:
1313
+ - $ref: '#/components/parameters/Limit'
1314
+ - $ref: '#/components/parameters/Cursor'
1315
+ responses:
1316
+ '200':
1317
+ description: One page of agents.
1318
+ content:
1319
+ application/json:
1320
+ schema:
1321
+ $ref: '#/components/schemas/AgentPage'
1322
+ '400':
1323
+ $ref: '#/components/responses/BadRequest'
1324
+ '401':
1325
+ $ref: '#/components/responses/Unauthorized'
1326
+ '429':
1327
+ $ref: '#/components/responses/TooManyRequests'
1328
+ '500':
1329
+ $ref: '#/components/responses/InternalError'
1330
+
1331
+ /api/agents/{name}:
1332
+ get:
1333
+ tags: [agents]
1334
+ operationId: getAgent
1335
+ summary: Get one registered agent
1336
+ description: >
1337
+ Private to its owner — another user's agent name reads
1338
+ `agent_not_found`, never a permission error.
1339
+ parameters:
1340
+ - $ref: '#/components/parameters/AgentName'
1341
+ responses:
1342
+ '200':
1343
+ description: The agent.
1344
+ content:
1345
+ application/json:
1346
+ schema:
1347
+ $ref: '#/components/schemas/Agent'
1348
+ '401':
1349
+ $ref: '#/components/responses/Unauthorized'
1350
+ '404':
1351
+ $ref: '#/components/responses/NotFound'
1352
+ '429':
1353
+ $ref: '#/components/responses/TooManyRequests'
1354
+ '500':
1355
+ $ref: '#/components/responses/InternalError'
1356
+ put:
1357
+ tags: [agents]
1358
+ operationId: upsertAgent
1359
+ summary: Create or replace a registered agent
1360
+ parameters:
1361
+ - $ref: '#/components/parameters/AgentName'
1362
+ requestBody:
1363
+ required: true
1364
+ content:
1365
+ multipart/form-data:
1366
+ schema:
1367
+ $ref: '#/components/schemas/AgentRegistration'
1368
+ responses:
1369
+ '200':
1370
+ description: The agent after the upsert.
1371
+ content:
1372
+ application/json:
1373
+ schema:
1374
+ $ref: '#/components/schemas/Agent'
1375
+ '400':
1376
+ $ref: '#/components/responses/BadRequest'
1377
+ '401':
1378
+ $ref: '#/components/responses/Unauthorized'
1379
+ '409':
1380
+ description: '`agent_name_reserved` or `agent_source_conflict`.'
1381
+ content:
1382
+ application/json:
1383
+ schema:
1384
+ $ref: '#/components/schemas/Error'
1385
+ '413':
1386
+ description: Upload over the size cap (`agent_too_large`).
1387
+ content:
1388
+ application/json:
1389
+ schema:
1390
+ $ref: '#/components/schemas/Error'
1391
+ '429':
1392
+ $ref: '#/components/responses/TooManyRequests'
1393
+ '500':
1394
+ $ref: '#/components/responses/InternalError'
1395
+ delete:
1396
+ tags: [agents]
1397
+ operationId: deleteAgent
1398
+ summary: Delete a registered agent
1399
+ parameters:
1400
+ - $ref: '#/components/parameters/AgentName'
1401
+ responses:
1402
+ '204':
1403
+ description: Deleted.
1404
+ '401':
1405
+ $ref: '#/components/responses/Unauthorized'
1406
+ '404':
1407
+ $ref: '#/components/responses/NotFound'
1408
+ '429':
1409
+ $ref: '#/components/responses/TooManyRequests'
1410
+ '500':
1411
+ $ref: '#/components/responses/InternalError'
1412
+
1413
+ # ===========================================================================
1414
+ # Meta
1415
+ # ===========================================================================
1416
+
1417
+ /api/meta:
1418
+ get:
1419
+ tags: [meta]
1420
+ operationId: getMeta
1421
+ summary: Capability document (no auth)
1422
+ security: []
1423
+ description: >
1424
+ Everything a client would otherwise hardcode: agents (built-in and
1425
+ registration rules), sandbox providers with sizing ceilings and
1426
+ refusals, platform constraints, network modes, status vocabularies
1427
+ with their terminal members, limits, and the closed error-code list.
1428
+ Public and cacheable — weak ETag, 304 support, `max-age=300`.
1429
+ parameters:
1430
+ - name: If-None-Match
1431
+ in: header
1432
+ schema:
1433
+ type: string
1434
+ responses:
1435
+ '200':
1436
+ description: The capability document.
1437
+ headers:
1438
+ ETag:
1439
+ schema:
1440
+ type: string
1441
+ Cache-Control:
1442
+ schema:
1443
+ type: string
1444
+ content:
1445
+ application/json:
1446
+ schema:
1447
+ $ref: '#/components/schemas/CapabilityDocument'
1448
+ '304':
1449
+ description: Not modified.
1450
+ '500':
1451
+ $ref: '#/components/responses/InternalError'
1452
+
1453
+ # ===========================================================================
1454
+ # Auth
1455
+ # ===========================================================================
1456
+
1457
+ /api/auth/status:
1458
+ get:
1459
+ tags: [auth]
1460
+ operationId: getAuthStatus
1461
+ summary: Who am I
1462
+ description: >
1463
+ Identifies the caller and the API key used, so `auth status` in
1464
+ clients has a server answer.
1465
+ responses:
1466
+ '200':
1467
+ description: The caller's identity.
1468
+ content:
1469
+ application/json:
1470
+ schema:
1471
+ $ref: '#/components/schemas/AuthStatus'
1472
+ '401':
1473
+ $ref: '#/components/responses/Unauthorized'
1474
+ '429':
1475
+ $ref: '#/components/responses/TooManyRequests'
1476
+ '500':
1477
+ $ref: '#/components/responses/InternalError'
1478
+
1479
+ /api/auth/keys:
1480
+ get:
1481
+ tags: [auth]
1482
+ operationId: listApiKeys
1483
+ summary: List the caller's API keys
1484
+ x-wave: 2
1485
+ description: Secrets are never returned.
1486
+ parameters:
1487
+ - $ref: '#/components/parameters/Limit'
1488
+ - $ref: '#/components/parameters/Cursor'
1489
+ responses:
1490
+ '200':
1491
+ description: One page of key descriptors.
1492
+ content:
1493
+ application/json:
1494
+ schema:
1495
+ $ref: '#/components/schemas/ApiKeyPage'
1496
+ '400':
1497
+ $ref: '#/components/responses/BadRequest'
1498
+ '401':
1499
+ $ref: '#/components/responses/Unauthorized'
1500
+ '429':
1501
+ $ref: '#/components/responses/TooManyRequests'
1502
+ '500':
1503
+ $ref: '#/components/responses/InternalError'
1504
+
1505
+ /api/auth/keys/{keyId}:
1506
+ delete:
1507
+ tags: [auth]
1508
+ operationId: revokeApiKey
1509
+ summary: Revoke an API key
1510
+ x-wave: 2
1511
+ description: Revocation is immediate.
1512
+ parameters:
1513
+ - name: keyId
1514
+ in: path
1515
+ required: true
1516
+ schema:
1517
+ type: string
1518
+ responses:
1519
+ '204':
1520
+ description: Revoked.
1521
+ '401':
1522
+ $ref: '#/components/responses/Unauthorized'
1523
+ '404':
1524
+ $ref: '#/components/responses/NotFound'
1525
+ '429':
1526
+ $ref: '#/components/responses/TooManyRequests'
1527
+ '500':
1528
+ $ref: '#/components/responses/InternalError'
1529
+
1530
+ # =============================================================================
1531
+ # Components
1532
+ # =============================================================================
1533
+
1534
+ components:
1535
+ securitySchemes:
1536
+ apiKey:
1537
+ type: http
1538
+ scheme: bearer
1539
+ description: >
1540
+ `Authorization: Bearer <API key>`. Required on everything except
1541
+ `GET /api/meta`.
1542
+
1543
+ parameters:
1544
+ JobId:
1545
+ name: jobId
1546
+ in: path
1547
+ required: true
1548
+ schema:
1549
+ type: string
1550
+ format: uuid
1551
+ TrialId:
1552
+ name: trialId
1553
+ in: path
1554
+ required: true
1555
+ schema:
1556
+ type: string
1557
+ format: uuid
1558
+ DatasetName:
1559
+ name: name
1560
+ in: path
1561
+ required: true
1562
+ description: >
1563
+ A dataset's catalog name. `imports` is a reserved name — it is a
1564
+ literal path segment under `/api/datasets` — and publish refuses it,
1565
+ so `/api/datasets/imports/{importId}` can never collide with a
1566
+ dataset route.
1567
+ schema:
1568
+ type: string
1569
+ AgentName:
1570
+ name: name
1571
+ in: path
1572
+ required: true
1573
+ schema:
1574
+ type: string
1575
+ Limit:
1576
+ name: limit
1577
+ in: query
1578
+ description: Page size. Collection default 50, max 200 (dataset tasks 200/500).
1579
+ schema:
1580
+ type: integer
1581
+ minimum: 1
1582
+ Cursor:
1583
+ name: cursor
1584
+ in: query
1585
+ description: >
1586
+ Opaque keyset cursor from a previous page's `nextCursor`. Never
1587
+ constructed by the client.
1588
+ schema:
1589
+ type: string
1590
+ IdempotencyKey:
1591
+ name: Idempotency-Key
1592
+ in: header
1593
+ required: false
1594
+ description: |
1595
+ Client-chosen key that makes the POST safe to retry. The same key
1596
+ with the same resolved request returns the original resource (200,
1597
+ `idempotent_replay: true`); the same key with a different request is
1598
+ refused with `idempotency_key_reused` (409). Keys are namespaced per
1599
+ route and per caller.
1600
+ schema:
1601
+ type: string
1602
+ maxLength: 200
1603
+
1604
+ responses:
1605
+ BadRequest:
1606
+ description: >
1607
+ Malformed request — `invalid_json`, `invalid_input`, `invalid_limit`,
1608
+ `invalid_status`, `invalid_cursor`, `invalid_format`, `invalid_ids`,
1609
+ `invalid_multipart`, `invalid_archive`, `unknown_task_names`, or
1610
+ `no_tasks`.
1611
+ content:
1612
+ application/json:
1613
+ schema:
1614
+ $ref: '#/components/schemas/Error'
1615
+ Unauthorized:
1616
+ description: >
1617
+ `missing_authorization`, `invalid_api_key`, or (503-adjacent, still
1618
+ this envelope) `credential_service_unavailable`.
1619
+ content:
1620
+ application/json:
1621
+ schema:
1622
+ $ref: '#/components/schemas/Error'
1623
+ NotFound:
1624
+ description: >
1625
+ The addressed resource does not exist or is not visible to the
1626
+ caller. Existence is never leaked: someone else's private resource
1627
+ reads 404, not 403.
1628
+ content:
1629
+ application/json:
1630
+ schema:
1631
+ $ref: '#/components/schemas/Error'
1632
+ Conflict:
1633
+ description: State conflict; the `code` names which law was hit.
1634
+ content:
1635
+ application/json:
1636
+ schema:
1637
+ $ref: '#/components/schemas/Error'
1638
+ TooManyRequests:
1639
+ description: '`rate_limited`. Honor `Retry-After`.'
1640
+ headers:
1641
+ Retry-After:
1642
+ schema:
1643
+ type: integer
1644
+ X-RateLimit-Limit:
1645
+ schema:
1646
+ type: integer
1647
+ X-RateLimit-Remaining:
1648
+ schema:
1649
+ type: integer
1650
+ X-RateLimit-Reset:
1651
+ schema:
1652
+ type: integer
1653
+ description: Epoch seconds when the window resets.
1654
+ content:
1655
+ application/json:
1656
+ schema:
1657
+ $ref: '#/components/schemas/Error'
1658
+ InternalError:
1659
+ description: '`internal_error`; carries `requestId` for support.'
1660
+ content:
1661
+ application/json:
1662
+ schema:
1663
+ $ref: '#/components/schemas/Error'
1664
+
1665
+ schemas:
1666
+ # -------------------------------------------------------------------------
1667
+ # Errors
1668
+ # -------------------------------------------------------------------------
1669
+
1670
+ ErrorCode:
1671
+ type: string
1672
+ description: |
1673
+ The closed error-code union (52 members). Published at runtime by
1674
+ `GET /api/meta` (`error_codes`); no route ever answers with a code
1675
+ outside this list.
1676
+ enum:
1677
+ # Auth + throttling
1678
+ - missing_authorization
1679
+ - invalid_api_key
1680
+ - credential_service_unavailable
1681
+ - rate_limited
1682
+ - insufficient_credits
1683
+ # Request shape
1684
+ - invalid_json
1685
+ - invalid_input
1686
+ - invalid_limit
1687
+ - invalid_status
1688
+ - invalid_cursor
1689
+ - invalid_after
1690
+ - invalid_format
1691
+ - invalid_ids
1692
+ - invalid_multipart
1693
+ - idempotency_key_reused
1694
+ # Datasets
1695
+ - dataset_not_found
1696
+ - dataset_version_not_found
1697
+ - dataset_name_taken
1698
+ - dataset_in_use
1699
+ - dataset_not_owned
1700
+ - upstream_not_watchable
1701
+ - no_active_version
1702
+ - version_not_ready
1703
+ - version_not_activatable
1704
+ - unknown_task_names
1705
+ - no_tasks
1706
+ # Registered agents
1707
+ - agent_not_found
1708
+ - agent_name_taken
1709
+ - agent_name_reserved
1710
+ - agent_invalid_name
1711
+ - agent_source_required
1712
+ - agent_source_conflict
1713
+ - agent_invalid_env
1714
+ - agent_too_large
1715
+ - agent_limit_reached
1716
+ # Jobs + trials
1717
+ - agent_version_not_found
1718
+ - job_too_large
1719
+ - provider_unsupported
1720
+ - job_not_found
1721
+ - job_not_terminal
1722
+ - no_failed_trials
1723
+ - trial_not_found
1724
+ - concurrent_update
1725
+ # Regrade
1726
+ - regrade_source_ineligible
1727
+ - no_regradable_trials
1728
+ # Imports
1729
+ - import_not_found
1730
+ - import_too_large
1731
+ - invalid_archive
1732
+ # Retained source package (owner-only download)
1733
+ - package_not_retained
1734
+ - package_corrupt
1735
+ - package_missing
1736
+ # Fallback
1737
+ - internal_error
1738
+
1739
+ Error:
1740
+ type: object
1741
+ description: |
1742
+ The ONE error envelope, under the top-level key `error`. Envelope keys
1743
+ are frozen verbatim (documented deviation). `details` is open — each
1744
+ code carries the machine-readable shape its own refusal needs (a list
1745
+ of task names, a provider and per-task reasons, a ceiling and the
1746
+ value that breached it, the blocking job ids).
1747
+ required: [error]
1748
+ properties:
1749
+ error:
1750
+ type: object
1751
+ required: [code, message]
1752
+ properties:
1753
+ code:
1754
+ $ref: '#/components/schemas/ErrorCode'
1755
+ message:
1756
+ type: string
1757
+ description: Human-readable; never parse it — parse `code`.
1758
+ param:
1759
+ type: [string, 'null']
1760
+ description: The input field the caller got wrong, in the caller's vocabulary.
1761
+ details:
1762
+ type: [object, 'null']
1763
+ additionalProperties: true
1764
+ retryAfterSec:
1765
+ type: [integer, 'null']
1766
+ description: Present on retryable refusals; mirrors the Retry-After header.
1767
+ requestId:
1768
+ type: string
1769
+ description: Correlates with server logs; quote it in support requests.
1770
+
1771
+ # -------------------------------------------------------------------------
1772
+ # Shared enums and small shapes
1773
+ # -------------------------------------------------------------------------
1774
+
1775
+ JobStatus:
1776
+ type: string
1777
+ description: >
1778
+ Job lifecycle (kept deviation — load-bearing for cancel, watch exit,
1779
+ download gating, and resume/regrade eligibility). Terminal: COMPLETED,
1780
+ CANCELLED, FAILED.
1781
+ enum: [QUEUED, RUNNING, CANCELLING, COMPLETED, CANCELLED, FAILED]
1782
+
1783
+ TrialStatus:
1784
+ type: string
1785
+ description: |
1786
+ Trial lifecycle (kept deviation). The status law: a valid reward
1787
+ (including 0) = SCORED; a verifier crash or out-of-domain reward =
1788
+ SCORING_ERROR (never a fabricated zero); INFRASTRUCTURE_ERROR = the
1789
+ trial was lost before a result was recorded; INDETERMINATE = the
1790
+ platform cannot tell whether the trial completed. Terminal: SCORED,
1791
+ SCORING_ERROR, INFRASTRUCTURE_ERROR, INDETERMINATE, CANCELLED.
1792
+ enum:
1793
+ - QUEUED
1794
+ - RUNNING
1795
+ - SCORING
1796
+ - SCORED
1797
+ - SCORING_ERROR
1798
+ - INFRASTRUCTURE_ERROR
1799
+ - INDETERMINATE
1800
+ - CANCELLED
1801
+
1802
+ SandboxProvider:
1803
+ type: string
1804
+ description: Where trials execute. Spelled identically on every surface.
1805
+ enum: [e2b, daytona, modal]
1806
+
1807
+ SpendSource:
1808
+ type: string
1809
+ description: >
1810
+ Which lane a settled trial's `agent_result.cost_usd` is in. `measured`
1811
+ is final. The other two are PRE-CONFIRMATION lanes a freshly settled
1812
+ trial commonly holds for a few minutes, served verbatim because the
1813
+ lane is the only signal that the figure is still provisional:
1814
+ `measured_provisional` is an honest floor read inside the gateway's
1815
+ async spend flush (it can only be raised), and `assumed_cap` means
1816
+ nothing was measured yet — it holds a zero placeholder, never an
1817
+ invented figure. A deferred pass confirms or raises both from the
1818
+ gateway's spend logs.
1819
+ enum: [measured, measured_provisional, assumed_cap]
1820
+
1821
+ VerifierEnvironmentMode:
1822
+ type: string
1823
+ description: >
1824
+ Where the verifier ran — inside the agent's environment (shared) or
1825
+ in a separate one.
1826
+ enum: [shared, separate]
1827
+
1828
+ AttemptPhase:
1829
+ type: string
1830
+ description: >
1831
+ Which step a RUNNING trial is in, so a polling caller can tell a slow
1832
+ build from a slow agent — RUNNING alone cannot.
1833
+ enum: [prepare, build, boot, install, agent, verify, persist]
1834
+
1835
+ TrialStatusTally:
1836
+ type: object
1837
+ description: >
1838
+ The one "how many" shape: a total plus a histogram over EVERY trial
1839
+ status, zeros included, so a UI draws its status bar off the response
1840
+ instead of hardcoding the enum.
1841
+ required: [total, byStatus]
1842
+ properties:
1843
+ total:
1844
+ type: integer
1845
+ minimum: 0
1846
+ byStatus:
1847
+ type: object
1848
+ description: One integer per TrialStatus member, zeros included.
1849
+ required:
1850
+ - QUEUED
1851
+ - RUNNING
1852
+ - SCORING
1853
+ - SCORED
1854
+ - SCORING_ERROR
1855
+ - INFRASTRUCTURE_ERROR
1856
+ - INDETERMINATE
1857
+ - CANCELLED
1858
+ properties:
1859
+ QUEUED: { type: integer, minimum: 0 }
1860
+ RUNNING: { type: integer, minimum: 0 }
1861
+ SCORING: { type: integer, minimum: 0 }
1862
+ SCORED: { type: integer, minimum: 0 }
1863
+ SCORING_ERROR: { type: integer, minimum: 0 }
1864
+ INFRASTRUCTURE_ERROR: { type: integer, minimum: 0 }
1865
+ INDETERMINATE: { type: integer, minimum: 0 }
1866
+ CANCELLED: { type: integer, minimum: 0 }
1867
+
1868
+ # -------------------------------------------------------------------------
1869
+ # Job — config side
1870
+ # -------------------------------------------------------------------------
1871
+
1872
+ DatasetRef:
1873
+ type: object
1874
+ description: A resolved dataset reference as echoed on job bodies.
1875
+ required: [name, version]
1876
+ properties:
1877
+ name:
1878
+ type: string
1879
+ version:
1880
+ type: string
1881
+
1882
+ DatasetSelector:
1883
+ type: object
1884
+ description: |
1885
+ One dataset a job runs, with per-dataset task filters. `task_names`
1886
+ and `exclude_task_names` are glob patterns; `n_tasks` caps the task
1887
+ count AFTER filtering. A bare `name` resolves to the active version
1888
+ (`no_active_version` when none).
1889
+ required: [name]
1890
+ properties:
1891
+ name:
1892
+ type: string
1893
+ description: Catalog dataset name.
1894
+ version:
1895
+ type: string
1896
+ description: Pin a version; omitted, the active version is used.
1897
+ task_names:
1898
+ type: array
1899
+ description: Include filter — glob patterns over task names.
1900
+ items:
1901
+ type: string
1902
+ exclude_task_names:
1903
+ type: array
1904
+ description: Exclude filter — glob patterns over task names.
1905
+ items:
1906
+ type: string
1907
+ n_tasks:
1908
+ type: integer
1909
+ minimum: 1
1910
+ description: Cap the task count after filters are applied.
1911
+
1912
+ AgentArmInput:
1913
+ type: object
1914
+ description: >
1915
+ One agent arm of a job: an agent (built-in or registered) plus a
1916
+ model. A model is always required; the server applies no default.
1917
+ required: [name, model_name]
1918
+ properties:
1919
+ name:
1920
+ type: string
1921
+ description: Agent name — a built-in or one registered under /api/agents.
1922
+ model_name:
1923
+ type: string
1924
+ version:
1925
+ type: string
1926
+ description: >
1927
+ Pin an agent version; omitted, the platform resolves the latest
1928
+ supported (`agent_version_not_found` when a pin cannot resolve).
1929
+ reasoning_effort:
1930
+ type: string
1931
+ description: >
1932
+ Platform extension — declared effort, part of arm identity.
1933
+ Accepted values published by /api/meta; omitted, the runner
1934
+ default applies.
1935
+
1936
+ AgentArm:
1937
+ type: object
1938
+ description: One agent arm as echoed on job bodies.
1939
+ required: [name, model_name, version, reasoning_effort]
1940
+ properties:
1941
+ name:
1942
+ type: string
1943
+ model_name:
1944
+ type: string
1945
+ version:
1946
+ type: [string, 'null']
1947
+ description: The requested pin; null when the run took the latest.
1948
+ reasoning_effort:
1949
+ type: [string, 'null']
1950
+ description: Declared effort; null when the run took the runner default.
1951
+
1952
+ SourceJob:
1953
+ type: object
1954
+ description: >
1955
+ Provenance of a derived job. `action: regrade` = verifier-only re-run
1956
+ of the source; `action: resume` (platform extension) = new job over
1957
+ the source's failed and stopped trials. `type` is always `hub` on
1958
+ this hosted surface.
1959
+ required: [action, type, job_id]
1960
+ properties:
1961
+ action:
1962
+ type: string
1963
+ enum: [regrade, resume]
1964
+ type:
1965
+ type: string
1966
+ enum: [hub]
1967
+ job_id:
1968
+ type: string
1969
+ format: uuid
1970
+
1971
+ JobCreate:
1972
+ type: object
1973
+ description: The job-creation body.
1974
+ required: [datasets, agents]
1975
+ properties:
1976
+ job_name:
1977
+ type: string
1978
+ maxLength: 200
1979
+ description: >
1980
+ User-facing label; server-generated when omitted. A LABEL, not an
1981
+ identity: names are not unique, and two jobs may carry the same
1982
+ one (Harbor's job name is a local directory and resumes in place;
1983
+ a hosted registry keeps `id` as the only identity). Reuse a name
1984
+ freely across runs; to prevent accidental double submission use
1985
+ Idempotency-Key, which replays by request, not by name.
1986
+ datasets:
1987
+ type: array
1988
+ minItems: 1
1989
+ items:
1990
+ $ref: '#/components/schemas/DatasetSelector'
1991
+ agents:
1992
+ type: array
1993
+ minItems: 1
1994
+ maxItems: 8
1995
+ items:
1996
+ $ref: '#/components/schemas/AgentArmInput'
1997
+ n_attempts:
1998
+ type: integer
1999
+ minimum: 1
2000
+ maximum: 100
2001
+ default: 1
2002
+ description: Attempts per task per agent arm.
2003
+ n_concurrent_trials:
2004
+ type: integer
2005
+ minimum: 1
2006
+ maximum: 16
2007
+ default: 4
2008
+ description: Parallel trials across the job.
2009
+ max_trial_spend_usd:
2010
+ type: number
2011
+ minimum: 0
2012
+ description: |
2013
+ Per-trial spend cap in USD, minted onto each trial's gateway key —
2014
+ the platform's ONLY spend enforcement (there is no job-wide
2015
+ budget). Default published by /api/meta (200 unless the operator
2016
+ tuned it).
2017
+ sandbox_provider:
2018
+ $ref: '#/components/schemas/SandboxProvider'
2019
+ agent_env:
2020
+ type: object
2021
+ additionalProperties:
2022
+ type: string
2023
+ description: >
2024
+ Env injected into every agent run — a pass-through slot: the
2025
+ client sends it verbatim and the server owns acceptance (refused
2026
+ where unsupported, never silently dropped).
2027
+ verifier_env:
2028
+ type: object
2029
+ additionalProperties:
2030
+ type: string
2031
+ description: >
2032
+ Env injected into every verifier run — same pass-through
2033
+ contract.
2034
+
2035
+ ResumeRequest:
2036
+ type: object
2037
+ description: Body of POST /api/jobs/{jobId}/resume.
2038
+ properties:
2039
+ filter_error_types:
2040
+ type: array
2041
+ description: |
2042
+ Which failures to resume, matched against
2043
+ `exception_info.exception_type`. Omitted, the default set is
2044
+ `["ScoringError", "InfrastructureError", "IncompleteTrialError"]`
2045
+ plus stopped trials (settled CANCELLED, exception type
2046
+ `CancelledError`) and still-QUEUED trials of a cancelled source.
2047
+ items:
2048
+ type: string
2049
+
2050
+ RegradeRequest:
2051
+ type: object
2052
+ description: >
2053
+ Optional filter narrowing which trials a job-level regrade re-runs.
2054
+ Omitted, every regradable trial is regraded.
2055
+ properties:
2056
+ statuses:
2057
+ type: array
2058
+ items:
2059
+ $ref: '#/components/schemas/TrialStatus'
2060
+ task_name:
2061
+ type: string
2062
+ description: Restrict to one task's trials.
2063
+
2064
+ # -------------------------------------------------------------------------
2065
+ # Job — result side
2066
+ # -------------------------------------------------------------------------
2067
+
2068
+ AgentDatasetStats:
2069
+ type: object
2070
+ description: |
2071
+ Per-(agent, model, dataset) statistics. The evals key format is
2072
+ `{agent}__{model}__{dataset}` — the dataset ref is always the LAST
2073
+ `__` segment, which is where Harbor-compatible readers recover it —
2074
+ with the platform extension of an `__{effort}` segment inserted
2075
+ BEFORE the dataset when a declared reasoning effort is part of the
2076
+ arm identity: `{agent}__{model}__{effort}__{dataset}`.
2077
+ properties:
2078
+ n_trials:
2079
+ type: integer
2080
+ minimum: 0
2081
+ description: Trials that produced a rewards map — rewarded, not merely settled.
2082
+ n_errors:
2083
+ type: integer
2084
+ minimum: 0
2085
+ description: >
2086
+ Trials carrying `exception_info` — indeterminate and cancelled
2087
+ included.
2088
+ metrics:
2089
+ type: array
2090
+ description: >
2091
+ Metric results (a mean entry per arm today: the primary reward
2092
+ averaged over EVERY trial of the group, unrewarded trials
2093
+ counting 0); open objects so new metric types need no wire
2094
+ change.
2095
+ items:
2096
+ type: object
2097
+ additionalProperties: true
2098
+ pass_at_k:
2099
+ type: object
2100
+ description: >
2101
+ pass@k slot — keys are k as strings, values in [0,1]. Present and
2102
+ empty until the platform computes it; the slot exists so adding
2103
+ the statistic is not a wire change.
2104
+ additionalProperties:
2105
+ type: number
2106
+ reward_stats:
2107
+ type: object
2108
+ description: 'reward key -> reward value -> trial identifiers.'
2109
+ additionalProperties:
2110
+ type: object
2111
+ additionalProperties:
2112
+ type: array
2113
+ items:
2114
+ type: string
2115
+ exception_stats:
2116
+ type: object
2117
+ description: 'exception type -> trial identifiers.'
2118
+ additionalProperties:
2119
+ type: array
2120
+ items:
2121
+ type: string
2122
+
2123
+ JobStats:
2124
+ type: object
2125
+ description: |
2126
+ Aggregate statistics of a job. Progress counters, token totals, and
2127
+ measured cost. The `n_*` counters are CUMULATIVE, Harbor-style:
2128
+ errored trials are a subset of completed, cancelled a subset of
2129
+ errored — a cancelled trial counts in all three. The disjoint
2130
+ per-status breakdown rides `Job.trials.byStatus`.
2131
+ `cost_usd` is what the trials actually spent so far — reporting,
2132
+ never a gate (enforcement is the per-trial cap).
2133
+ properties:
2134
+ n_completed_trials:
2135
+ type: integer
2136
+ minimum: 0
2137
+ description: >
2138
+ Cumulative: every trial that produced a result — errored and
2139
+ cancelled included, never the scored subset alone.
2140
+ n_errored_trials:
2141
+ type: integer
2142
+ minimum: 0
2143
+ description: >
2144
+ Cumulative: every completed trial carrying `exception_info`,
2145
+ cancelled included. A subset of `n_completed_trials`.
2146
+ n_running_trials:
2147
+ type: integer
2148
+ minimum: 0
2149
+ n_pending_trials:
2150
+ type: integer
2151
+ minimum: 0
2152
+ n_cancelled_trials:
2153
+ type: integer
2154
+ minimum: 0
2155
+ description: A subset of `n_errored_trials`.
2156
+ n_retries:
2157
+ type: integer
2158
+ minimum: 0
2159
+ evals:
2160
+ type: object
2161
+ description: >
2162
+ Keyed `{agent}__{model}__{dataset}` — dataset ref last, with the
2163
+ optional effort segment inserted before it.
2164
+ additionalProperties:
2165
+ $ref: '#/components/schemas/AgentDatasetStats'
2166
+ n_input_tokens:
2167
+ type: [integer, 'null']
2168
+ description: Total input tokens (cache included); null until recorded.
2169
+ n_cache_tokens:
2170
+ type: [integer, 'null']
2171
+ n_output_tokens:
2172
+ type: [integer, 'null']
2173
+ cost_usd:
2174
+ type: [number, 'null']
2175
+ description: Measured spend across settled trials; null before any settled.
2176
+
2177
+ JobFailure:
2178
+ type: object
2179
+ description: >
2180
+ Why a job FAILED — deliberately NOT under the key `error`, which on
2181
+ this surface always means "this request failed". `if (body.error)
2182
+ throw` stays correct on a healthy 200 read of a failed job.
2183
+ required: [code, message]
2184
+ properties:
2185
+ code:
2186
+ type: string
2187
+ description: '`job_execution_failed` when the runner recorded no code.'
2188
+ message:
2189
+ type: string
2190
+
2191
+ Job:
2192
+ type: object
2193
+ description: |
2194
+ THE job body — the same shape from create, get, list items, cancel,
2195
+ resume, and regrade responses; no field appears on some responses and
2196
+ not others. The create verb answers with JobCreated, which is this
2197
+ shape plus one aliased id and nothing else.
2198
+ required:
2199
+ - id
2200
+ - job_name
2201
+ - status
2202
+ - datasets
2203
+ - agents
2204
+ - n_attempts
2205
+ - n_concurrent_trials
2206
+ - max_trial_spend_usd
2207
+ - worst_case_spend_usd
2208
+ - sandbox_provider
2209
+ - counts
2210
+ - n_total_trials
2211
+ - trials
2212
+ - stats
2213
+ - failure
2214
+ - source_jobs
2215
+ - is_regrade
2216
+ - idempotent_replay
2217
+ - started_at
2218
+ - updated_at
2219
+ - finished_at
2220
+ properties:
2221
+ id:
2222
+ type: string
2223
+ format: uuid
2224
+ job_name:
2225
+ type: string
2226
+ description: User-facing label.
2227
+ status:
2228
+ $ref: '#/components/schemas/JobStatus'
2229
+ datasets:
2230
+ type: array
2231
+ description: The resolved dataset references this job ran.
2232
+ items:
2233
+ $ref: '#/components/schemas/DatasetRef'
2234
+ agents:
2235
+ type: array
2236
+ items:
2237
+ $ref: '#/components/schemas/AgentArm'
2238
+ n_attempts:
2239
+ type: integer
2240
+ minimum: 1
2241
+ n_concurrent_trials:
2242
+ type: integer
2243
+ minimum: 1
2244
+ max_trial_spend_usd:
2245
+ type: number
2246
+ description: The resolved per-trial cap every trial key was minted with.
2247
+ worst_case_spend_usd:
2248
+ type: number
2249
+ description: |
2250
+ The most this job can cost: every trial spending its whole cap.
2251
+ Stated outright — the per-trial cap is the only enforcement, so
2252
+ the product is the number someone approving a 500-trial run
2253
+ actually needs to see.
2254
+ sandbox_provider:
2255
+ $ref: '#/components/schemas/SandboxProvider'
2256
+ counts:
2257
+ type: object
2258
+ description: Entity cardinality only — things with no status of their own.
2259
+ required: [agents, tasks]
2260
+ properties:
2261
+ agents:
2262
+ type: integer
2263
+ minimum: 0
2264
+ tasks:
2265
+ type: integer
2266
+ minimum: 0
2267
+ n_total_trials:
2268
+ type: integer
2269
+ minimum: 0
2270
+ trials:
2271
+ $ref: '#/components/schemas/TrialStatusTally'
2272
+ description: >
2273
+ Kept deviation — the zeros-included 8-status histogram, published
2274
+ alongside the coarser counters in `stats`.
2275
+ stats:
2276
+ $ref: '#/components/schemas/JobStats'
2277
+ failure:
2278
+ oneOf:
2279
+ - $ref: '#/components/schemas/JobFailure'
2280
+ - type: 'null'
2281
+ source_jobs:
2282
+ type: array
2283
+ description: Empty for an original job.
2284
+ items:
2285
+ $ref: '#/components/schemas/SourceJob'
2286
+ is_regrade:
2287
+ type: boolean
2288
+ description: 'Derived: any source_jobs entry with action "regrade".'
2289
+ idempotent_replay:
2290
+ type: boolean
2291
+ description: True only on a response that replayed an existing job for an Idempotency-Key.
2292
+ started_at:
2293
+ type: string
2294
+ format: date-time
2295
+ updated_at:
2296
+ type: string
2297
+ format: date-time
2298
+ finished_at:
2299
+ type: [string, 'null']
2300
+ format: date-time
2301
+ description: Null while the job is live.
2302
+
2303
+ JobCreated:
2304
+ description: |
2305
+ The CREATE verb's body: the whole Job shape plus `job_id`, an additive
2306
+ alias of `id`.
2307
+
2308
+ The alias exists for one reason and lives in one place. Harbor's
2309
+ hosted submit reads the new job's id under the key `job_id` and aborts
2310
+ the submit outright when that key is absent, so the verb an
2311
+ unmodified Harbor CLI starts a run with answers to both names. Every
2312
+ other job response — get, list, cancel, resume, regrade — carries `id`
2313
+ alone, so no reader has to learn a second spelling to follow a job.
2314
+
2315
+ Both of this verb's success bodies carry it: a create and the
2316
+ idempotent replay of that same create (their submit sends an
2317
+ `Idempotency-Key` on every attempt, so a retry lands on the replay).
2318
+ required: [job_id]
2319
+ allOf:
2320
+ - $ref: '#/components/schemas/Job'
2321
+ - type: object
2322
+ properties:
2323
+ job_id:
2324
+ type: string
2325
+ format: uuid
2326
+ description: Alias of `id`, identical value. Create responses only.
2327
+
2328
+ JobPage:
2329
+ type: object
2330
+ description: Cursor page of jobs (envelope keys frozen verbatim).
2331
+ required: [items, nextCursor, hasMore]
2332
+ properties:
2333
+ items:
2334
+ type: array
2335
+ items:
2336
+ $ref: '#/components/schemas/Job'
2337
+ nextCursor:
2338
+ type: [string, 'null']
2339
+ description: Pass back as `?cursor=`; null means no next page.
2340
+ hasMore:
2341
+ type: boolean
2342
+
2343
+ # -------------------------------------------------------------------------
2344
+ # Trial
2345
+ # -------------------------------------------------------------------------
2346
+
2347
+ TimingInfo:
2348
+ type: object
2349
+ description: >
2350
+ A phase's wall-clock as a start/stop pair (never a duration).
2351
+ Either bound is null while the phase has not reached it.
2352
+ properties:
2353
+ started_at:
2354
+ type: [string, 'null']
2355
+ format: date-time
2356
+ finished_at:
2357
+ type: [string, 'null']
2358
+ format: date-time
2359
+
2360
+ ModelInfo:
2361
+ type: object
2362
+ required: [name]
2363
+ properties:
2364
+ name:
2365
+ type: string
2366
+ provider:
2367
+ type: [string, 'null']
2368
+ description: Null means "not specified", never "unknown provider".
2369
+
2370
+ AgentInfo:
2371
+ type: object
2372
+ description: |
2373
+ The agent that ran a trial. `version` is the version actually
2374
+ RESOLVED and used (null until resolved) — the requested pin lives on
2375
+ the job's `agents[].version`. `reasoning_effort` is the platform's
2376
+ arm-identity extension.
2377
+ required: [name, version, model_info]
2378
+ properties:
2379
+ name:
2380
+ type: string
2381
+ version:
2382
+ type: [string, 'null']
2383
+ model_info:
2384
+ $ref: '#/components/schemas/ModelInfo'
2385
+ reasoning_effort:
2386
+ type: [string, 'null']
2387
+
2388
+ AgentResult:
2389
+ type: object
2390
+ description: |
2391
+ What the agent phase produced and consumed. `n_input_tokens` includes
2392
+ cache tokens. `cost_usd` is the settled spend (see `spend_source` on
2393
+ the trial for which lane it came from, and whether it is final).
2394
+ `metadata` carries
2395
+ open per-run detail: the harness bundle digest and runtime, the
2396
+ network mode the trial ran under and where that decision came from,
2397
+ and any harness-reported usage detail.
2398
+ properties:
2399
+ n_input_tokens:
2400
+ type: [integer, 'null']
2401
+ n_cache_tokens:
2402
+ type: [integer, 'null']
2403
+ n_output_tokens:
2404
+ type: [integer, 'null']
2405
+ cost_usd:
2406
+ type: [number, 'null']
2407
+ description: Null until the trial has executed; null never means $0.
2408
+ rollout_details:
2409
+ type: [array, 'null']
2410
+ description: Reserved for token-level rollout detail; null today.
2411
+ items:
2412
+ type: object
2413
+ additionalProperties: true
2414
+ metadata:
2415
+ type: [object, 'null']
2416
+ additionalProperties: true
2417
+
2418
+ VerifierResult:
2419
+ type: object
2420
+ description: |
2421
+ The verifier's rewards map. The primary-reward convention: the value
2422
+ under the key `"reward"`; else, when exactly one key exists, that
2423
+ value; else no primary reward. Zero is a reward.
2424
+ properties:
2425
+ rewards:
2426
+ type: [object, 'null']
2427
+ additionalProperties:
2428
+ type: number
2429
+
2430
+ ExceptionInfo:
2431
+ type: object
2432
+ description: |
2433
+ Why a trial failed, when it did. `exception_type` is one of the
2434
+ platform's stable failure names (`ScoringError`,
2435
+ `InfrastructureError`, `CancelledError`, `IncompleteTrialError`) —
2436
+ but filter with `Trial.status`, which is the primary key for failure
2437
+ classes; this is the detail.
2438
+ required: [exception_type, exception_message, occurred_at]
2439
+ properties:
2440
+ exception_type:
2441
+ type: string
2442
+ exception_message:
2443
+ type: string
2444
+ description: Truncated to 2000 chars on list rows; full on the detail route.
2445
+ exception_traceback:
2446
+ type: string
2447
+ description: Empty when the platform recorded no traceback.
2448
+ occurred_at:
2449
+ type: string
2450
+ format: date-time
2451
+
2452
+ StepResult:
2453
+ type: object
2454
+ description: >
2455
+ Placeholder for multi-step tasks. Always null on trials today —
2456
+ declared so multi-step lands without a wire change.
2457
+ properties:
2458
+ step_name:
2459
+ type: string
2460
+ agent_result:
2461
+ oneOf:
2462
+ - $ref: '#/components/schemas/AgentResult'
2463
+ - type: 'null'
2464
+ verifier_result:
2465
+ oneOf:
2466
+ - $ref: '#/components/schemas/VerifierResult'
2467
+ - type: 'null'
2468
+ exception_info:
2469
+ oneOf:
2470
+ - $ref: '#/components/schemas/ExceptionInfo'
2471
+ - type: 'null'
2472
+ agent_execution:
2473
+ oneOf:
2474
+ - $ref: '#/components/schemas/TimingInfo'
2475
+ - type: 'null'
2476
+ verifier:
2477
+ oneOf:
2478
+ - $ref: '#/components/schemas/TimingInfo'
2479
+ - type: 'null'
2480
+
2481
+ Trial:
2482
+ type: object
2483
+ description: |
2484
+ The ONE public trial shape, shared verbatim by list rows and the
2485
+ detail route (detail returns `exception_info.exception_message`
2486
+ untruncated — the only documented difference).
2487
+
2488
+ Execution facts (`sandbox_provider`, `verifier_environment_mode`,
2489
+ `agent_result.cost_usd`, `spend_source`) are null until the trial has
2490
+ actually executed: a QUEUED or CANCELLED trial never ran, so null
2491
+ means "did not run" and never zero.
2492
+ required:
2493
+ - id
2494
+ - job_id
2495
+ - task_name
2496
+ - source
2497
+ - agent_info
2498
+ - attempt
2499
+ - status
2500
+ - reward
2501
+ - verifier_result
2502
+ - exception_info
2503
+ - started_at
2504
+ - finished_at
2505
+ properties:
2506
+ id:
2507
+ type: string
2508
+ format: uuid
2509
+ job_id:
2510
+ type: string
2511
+ format: uuid
2512
+ task_name:
2513
+ type: string
2514
+ source:
2515
+ type: string
2516
+ description: The dataset this trial's task came from.
2517
+ agent_info:
2518
+ $ref: '#/components/schemas/AgentInfo'
2519
+ attempt:
2520
+ type: integer
2521
+ minimum: 1
2522
+ description: 'Attempt index within the arm (1..n_attempts).'
2523
+ status:
2524
+ $ref: '#/components/schemas/TrialStatus'
2525
+ reward:
2526
+ type: [number, 'null']
2527
+ description: >
2528
+ Convenience primary reward derived from `verifier_result.rewards`
2529
+ by the primary-reward convention. Zero is a reward; null means
2530
+ the trial did not score.
2531
+ verifier_result:
2532
+ oneOf:
2533
+ - $ref: '#/components/schemas/VerifierResult'
2534
+ - type: 'null'
2535
+ exception_info:
2536
+ oneOf:
2537
+ - $ref: '#/components/schemas/ExceptionInfo'
2538
+ - type: 'null'
2539
+ agent_result:
2540
+ oneOf:
2541
+ - $ref: '#/components/schemas/AgentResult'
2542
+ - type: 'null'
2543
+ environment_setup:
2544
+ oneOf:
2545
+ - $ref: '#/components/schemas/TimingInfo'
2546
+ - type: 'null'
2547
+ agent_setup:
2548
+ oneOf:
2549
+ - $ref: '#/components/schemas/TimingInfo'
2550
+ - type: 'null'
2551
+ agent_execution:
2552
+ oneOf:
2553
+ - $ref: '#/components/schemas/TimingInfo'
2554
+ - type: 'null'
2555
+ verifier:
2556
+ oneOf:
2557
+ - $ref: '#/components/schemas/TimingInfo'
2558
+ - type: 'null'
2559
+ step_results:
2560
+ type: [array, 'null']
2561
+ description: Multi-step placeholder; null today.
2562
+ items:
2563
+ $ref: '#/components/schemas/StepResult'
2564
+ spend_source:
2565
+ oneOf:
2566
+ - $ref: '#/components/schemas/SpendSource'
2567
+ - type: 'null'
2568
+ live_spent_usd:
2569
+ type: [number, 'null']
2570
+ description: |
2571
+ A mid-run LOWER BOUND on spend, never the trial's cost. Only ever
2572
+ climbs while the trial runs, and is CLEARED when the trial
2573
+ settles — on a terminal trial read `agent_result.cost_usd` and
2574
+ `spend_source`; those are the settled truth. Null is "no reading
2575
+ yet", never $0.
2576
+ live_spend_at:
2577
+ type: [string, 'null']
2578
+ format: date-time
2579
+ description: When that reading was taken — show its age, never the figure alone.
2580
+ max_trial_spend_usd:
2581
+ type: [number, 'null']
2582
+ description: >
2583
+ The cap THIS trial's gateway key carried — history, which can
2584
+ differ from the job's current cap for rows settled before a
2585
+ change.
2586
+ sandbox_provider:
2587
+ oneOf:
2588
+ - $ref: '#/components/schemas/SandboxProvider'
2589
+ - type: 'null'
2590
+ sandbox_id:
2591
+ type: [string, 'null']
2592
+ description: Provider id of the box the agent executed in; null when none booted.
2593
+ verifier_sandbox_id:
2594
+ type: [string, 'null']
2595
+ description: The separate verifier box; null in shared mode or when never reached.
2596
+ verifier_environment_mode:
2597
+ oneOf:
2598
+ - $ref: '#/components/schemas/VerifierEnvironmentMode'
2599
+ - type: 'null'
2600
+ attempt_phase:
2601
+ oneOf:
2602
+ - $ref: '#/components/schemas/AttemptPhase'
2603
+ - type: 'null'
2604
+ session_ref:
2605
+ type: [string, 'null']
2606
+ description: Reference to the agent session/trace, when recorded.
2607
+ started_at:
2608
+ type: [string, 'null']
2609
+ format: date-time
2610
+ finished_at:
2611
+ type: [string, 'null']
2612
+ format: date-time
2613
+
2614
+ TrialPage:
2615
+ type: object
2616
+ required: [items, nextCursor, hasMore]
2617
+ properties:
2618
+ items:
2619
+ type: array
2620
+ items:
2621
+ $ref: '#/components/schemas/Trial'
2622
+ nextCursor:
2623
+ type: [string, 'null']
2624
+ hasMore:
2625
+ type: boolean
2626
+
2627
+ StopRequest:
2628
+ type: object
2629
+ required: [trial_ids]
2630
+ properties:
2631
+ trial_ids:
2632
+ type: array
2633
+ minItems: 1
2634
+ maxItems: 100
2635
+ items:
2636
+ type: string
2637
+ format: uuid
2638
+
2639
+ StopResponse:
2640
+ type: object
2641
+ description: Per-trial outcome; every requested id appears in exactly one list.
2642
+ required: [stopped, already_terminal, not_found]
2643
+ properties:
2644
+ stopped:
2645
+ type: array
2646
+ description: Trials killed and settled by this request, with their settled rows.
2647
+ items:
2648
+ $ref: '#/components/schemas/Trial'
2649
+ already_terminal:
2650
+ type: array
2651
+ description: Ids that were already terminal; untouched.
2652
+ items:
2653
+ type: string
2654
+ format: uuid
2655
+ not_found:
2656
+ type: array
2657
+ description: Ids that do not exist or are not the caller's.
2658
+ items:
2659
+ type: string
2660
+ format: uuid
2661
+
2662
+ JobTaskRollup:
2663
+ type: object
2664
+ description: One task's rollup within a job (wave 2).
2665
+ required: [task_name, source, trials, mean_reward, cost_usd]
2666
+ properties:
2667
+ task_name:
2668
+ type: string
2669
+ source:
2670
+ type: string
2671
+ description: The dataset the task came from.
2672
+ trials:
2673
+ $ref: '#/components/schemas/TrialStatusTally'
2674
+ mean_reward:
2675
+ type: [number, 'null']
2676
+ description: Mean over SCORED trials only; null when none. Zero is a reward.
2677
+ cost_usd:
2678
+ type: [number, 'null']
2679
+ description: Measured spend across the task's settled trials.
2680
+
2681
+ JobTaskRollupPage:
2682
+ type: object
2683
+ required: [items, nextCursor, hasMore]
2684
+ properties:
2685
+ items:
2686
+ type: array
2687
+ items:
2688
+ $ref: '#/components/schemas/JobTaskRollup'
2689
+ nextCursor:
2690
+ type: [string, 'null']
2691
+ hasMore:
2692
+ type: boolean
2693
+
2694
+ TraceEvent:
2695
+ type: object
2696
+ description: >
2697
+ One parsed trace event of a trial's transcript. `seq` orders the
2698
+ stream and is the paging cursor. `data` is the harness-native payload,
2699
+ deliberately open.
2700
+ required: [seq, type]
2701
+ properties:
2702
+ seq:
2703
+ type: integer
2704
+ type:
2705
+ type: string
2706
+ timestamp:
2707
+ type: [string, 'null']
2708
+ format: date-time
2709
+ data:
2710
+ type: object
2711
+ additionalProperties: true
2712
+ additionalProperties: true
2713
+
2714
+ TraceEventPage:
2715
+ type: object
2716
+ required: [items, nextCursor, hasMore]
2717
+ properties:
2718
+ items:
2719
+ type: array
2720
+ items:
2721
+ $ref: '#/components/schemas/TraceEvent'
2722
+ nextCursor:
2723
+ type: [string, 'null']
2724
+ hasMore:
2725
+ type: boolean
2726
+
2727
+ # -------------------------------------------------------------------------
2728
+ # Compare
2729
+ # -------------------------------------------------------------------------
2730
+
2731
+ CompareCoverage:
2732
+ type: object
2733
+ description: How many of the trials behind an aggregate were SCORED.
2734
+ required: [scored, total]
2735
+ properties:
2736
+ scored:
2737
+ type: integer
2738
+ minimum: 0
2739
+ total:
2740
+ type: integer
2741
+ minimum: 0
2742
+
2743
+ CompareCell:
2744
+ type: object
2745
+ description: |
2746
+ One (task, job) cell. `status` is a TrialStatus when every trial in
2747
+ the cell shares it, `MIXED` when they differ, `MISSING` when the job
2748
+ has no trials for the task.
2749
+ required: [job_id, status, mean_reward, coverage]
2750
+ properties:
2751
+ job_id:
2752
+ type: string
2753
+ format: uuid
2754
+ status:
2755
+ type: string
2756
+ mean_reward:
2757
+ type: [number, 'null']
2758
+ coverage:
2759
+ $ref: '#/components/schemas/CompareCoverage'
2760
+
2761
+ CompareTaskRow:
2762
+ type: object
2763
+ required: [task_name, disagreement, cells]
2764
+ properties:
2765
+ task_name:
2766
+ type: string
2767
+ disagreement:
2768
+ type: boolean
2769
+ description: True when the jobs' cells differ in status or reward.
2770
+ cells:
2771
+ type: array
2772
+ items:
2773
+ $ref: '#/components/schemas/CompareCell'
2774
+
2775
+ CompareJobAggregate:
2776
+ type: object
2777
+ required: [id, datasets, status, mean_reward, coverage, cost_usd, agents, started_at]
2778
+ properties:
2779
+ id:
2780
+ type: string
2781
+ format: uuid
2782
+ datasets:
2783
+ type: array
2784
+ items:
2785
+ $ref: '#/components/schemas/DatasetRef'
2786
+ status:
2787
+ $ref: '#/components/schemas/JobStatus'
2788
+ mean_reward:
2789
+ type: [number, 'null']
2790
+ description: Mean over SCORED trials only; null when none. Zero is a reward.
2791
+ coverage:
2792
+ $ref: '#/components/schemas/CompareCoverage'
2793
+ cost_usd:
2794
+ type: number
2795
+ agents:
2796
+ type: array
2797
+ items:
2798
+ $ref: '#/components/schemas/AgentArm'
2799
+ started_at:
2800
+ type: string
2801
+ format: date-time
2802
+
2803
+ CompareResponse:
2804
+ type: object
2805
+ required: [jobs, taskMatrix]
2806
+ properties:
2807
+ jobs:
2808
+ type: array
2809
+ items:
2810
+ $ref: '#/components/schemas/CompareJobAggregate'
2811
+ taskMatrix:
2812
+ type: array
2813
+ description: Disagreement rows sort first.
2814
+ items:
2815
+ $ref: '#/components/schemas/CompareTaskRow'
2816
+
2817
+ # -------------------------------------------------------------------------
2818
+ # SSE events
2819
+ # -------------------------------------------------------------------------
2820
+
2821
+ JobEvent:
2822
+ description: |
2823
+ One server-sent event. On the wire, `seq` rides as the SSE `id` field
2824
+ and `type` as the SSE `event` field; the JSON body is `data`. Payload
2825
+ fields speak this document's vocabulary.
2826
+
2827
+ The union is discriminated on `type` and ONLY on `type`: several
2828
+ event types carry identically shaped payloads (`job.running` and
2829
+ `job.completed` are both `{job_id}`), so payload shape can never
2830
+ route a validator — the `type` constant on each variant does.
2831
+ oneOf:
2832
+ - $ref: '#/components/schemas/JobCreatedEvent'
2833
+ - $ref: '#/components/schemas/JobRunningEvent'
2834
+ - $ref: '#/components/schemas/JobCancellingEvent'
2835
+ - $ref: '#/components/schemas/JobCancelledEvent'
2836
+ - $ref: '#/components/schemas/JobCompletedEvent'
2837
+ - $ref: '#/components/schemas/JobFailedEvent'
2838
+ - $ref: '#/components/schemas/TrialRunningEvent'
2839
+ - $ref: '#/components/schemas/TrialScoringEvent'
2840
+ - $ref: '#/components/schemas/TrialSpendEvent'
2841
+ - $ref: '#/components/schemas/TrialSettledEvent'
2842
+ discriminator:
2843
+ propertyName: type
2844
+ mapping:
2845
+ job.created: '#/components/schemas/JobCreatedEvent'
2846
+ job.running: '#/components/schemas/JobRunningEvent'
2847
+ job.cancelling: '#/components/schemas/JobCancellingEvent'
2848
+ job.cancelled: '#/components/schemas/JobCancelledEvent'
2849
+ job.completed: '#/components/schemas/JobCompletedEvent'
2850
+ job.failed: '#/components/schemas/JobFailedEvent'
2851
+ trial.running: '#/components/schemas/TrialRunningEvent'
2852
+ trial.scoring: '#/components/schemas/TrialScoringEvent'
2853
+ trial.spend: '#/components/schemas/TrialSpendEvent'
2854
+ trial.settled: '#/components/schemas/TrialSettledEvent'
2855
+
2856
+ EventSeq:
2857
+ type: integer
2858
+ minimum: 0
2859
+ description: Monotonic sequence number — the resume position.
2860
+
2861
+ JobCreatedEvent:
2862
+ type: object
2863
+ required: [seq, type, data]
2864
+ properties:
2865
+ seq:
2866
+ $ref: '#/components/schemas/EventSeq'
2867
+ type:
2868
+ type: string
2869
+ const: job.created
2870
+ data:
2871
+ $ref: '#/components/schemas/JobCreatedData'
2872
+
2873
+ JobRunningEvent:
2874
+ type: object
2875
+ required: [seq, type, data]
2876
+ properties:
2877
+ seq:
2878
+ $ref: '#/components/schemas/EventSeq'
2879
+ type:
2880
+ type: string
2881
+ const: job.running
2882
+ data:
2883
+ $ref: '#/components/schemas/JobRunningData'
2884
+
2885
+ JobCancellingEvent:
2886
+ type: object
2887
+ required: [seq, type, data]
2888
+ properties:
2889
+ seq:
2890
+ $ref: '#/components/schemas/EventSeq'
2891
+ type:
2892
+ type: string
2893
+ const: job.cancelling
2894
+ data:
2895
+ $ref: '#/components/schemas/JobCancellingData'
2896
+
2897
+ JobCancelledEvent:
2898
+ type: object
2899
+ required: [seq, type, data]
2900
+ properties:
2901
+ seq:
2902
+ $ref: '#/components/schemas/EventSeq'
2903
+ type:
2904
+ type: string
2905
+ const: job.cancelled
2906
+ data:
2907
+ $ref: '#/components/schemas/JobCancelledData'
2908
+
2909
+ JobCompletedEvent:
2910
+ type: object
2911
+ required: [seq, type, data]
2912
+ properties:
2913
+ seq:
2914
+ $ref: '#/components/schemas/EventSeq'
2915
+ type:
2916
+ type: string
2917
+ const: job.completed
2918
+ data:
2919
+ $ref: '#/components/schemas/JobCompletedData'
2920
+
2921
+ JobFailedEvent:
2922
+ type: object
2923
+ required: [seq, type, data]
2924
+ properties:
2925
+ seq:
2926
+ $ref: '#/components/schemas/EventSeq'
2927
+ type:
2928
+ type: string
2929
+ const: job.failed
2930
+ data:
2931
+ $ref: '#/components/schemas/JobFailedData'
2932
+
2933
+ TrialRunningEvent:
2934
+ type: object
2935
+ required: [seq, type, data]
2936
+ properties:
2937
+ seq:
2938
+ $ref: '#/components/schemas/EventSeq'
2939
+ type:
2940
+ type: string
2941
+ const: trial.running
2942
+ data:
2943
+ $ref: '#/components/schemas/TrialRunningData'
2944
+
2945
+ TrialScoringEvent:
2946
+ type: object
2947
+ required: [seq, type, data]
2948
+ properties:
2949
+ seq:
2950
+ $ref: '#/components/schemas/EventSeq'
2951
+ type:
2952
+ type: string
2953
+ const: trial.scoring
2954
+ data:
2955
+ $ref: '#/components/schemas/TrialScoringData'
2956
+
2957
+ TrialSpendEvent:
2958
+ type: object
2959
+ required: [seq, type, data]
2960
+ properties:
2961
+ seq:
2962
+ $ref: '#/components/schemas/EventSeq'
2963
+ type:
2964
+ type: string
2965
+ const: trial.spend
2966
+ data:
2967
+ $ref: '#/components/schemas/TrialSpendData'
2968
+
2969
+ TrialSettledEvent:
2970
+ type: object
2971
+ required: [seq, type, data]
2972
+ properties:
2973
+ seq:
2974
+ $ref: '#/components/schemas/EventSeq'
2975
+ type:
2976
+ type: string
2977
+ const: trial.settled
2978
+ data:
2979
+ $ref: '#/components/schemas/TrialSettledData'
2980
+
2981
+ JobCreatedData:
2982
+ type: object
2983
+ description: >
2984
+ The job's resolved creation inputs, echoed so a watcher that joined
2985
+ late knows what it is watching.
2986
+ required:
2987
+ - datasets
2988
+ - task_count
2989
+ - agents
2990
+ - n_attempts
2991
+ - n_concurrent_trials
2992
+ - max_trial_spend_usd
2993
+ - sandbox_provider
2994
+ - trial_count
2995
+ properties:
2996
+ datasets:
2997
+ type: array
2998
+ items:
2999
+ $ref: '#/components/schemas/DatasetRef'
3000
+ task_count:
3001
+ type: integer
3002
+ agents:
3003
+ type: array
3004
+ items:
3005
+ $ref: '#/components/schemas/AgentArm'
3006
+ n_attempts:
3007
+ type: integer
3008
+ n_concurrent_trials:
3009
+ type: integer
3010
+ max_trial_spend_usd:
3011
+ type: number
3012
+ sandbox_provider:
3013
+ $ref: '#/components/schemas/SandboxProvider'
3014
+ trial_count:
3015
+ type: integer
3016
+
3017
+ JobRunningData:
3018
+ type: object
3019
+ required: [job_id]
3020
+ properties:
3021
+ job_id:
3022
+ type: string
3023
+ format: uuid
3024
+
3025
+ JobCancellingData:
3026
+ type: object
3027
+ required: [job_id, cancelled_trials, active_trials]
3028
+ properties:
3029
+ job_id:
3030
+ type: string
3031
+ format: uuid
3032
+ cancelled_trials:
3033
+ type: integer
3034
+ description: Queued trials cancelled outright by the request.
3035
+ active_trials:
3036
+ type: integer
3037
+ description: Trials still in flight, winding down before the job settles.
3038
+
3039
+ JobCancelledData:
3040
+ type: object
3041
+ required: [job_id, cancelled_trials]
3042
+ properties:
3043
+ job_id:
3044
+ type: string
3045
+ format: uuid
3046
+ cancelled_trials:
3047
+ type: integer
3048
+ description: Total queued trials cancelled across the request and the settle.
3049
+
3050
+ JobCompletedData:
3051
+ type: object
3052
+ required: [job_id]
3053
+ properties:
3054
+ job_id:
3055
+ type: string
3056
+ format: uuid
3057
+
3058
+ JobFailedData:
3059
+ type: object
3060
+ description: >
3061
+ The job settled FAILED. Terminal — the stream closes after this
3062
+ event. The event type is declared and reserved (FAILED is a real job
3063
+ status) but no server path emits it today; the payload is fixed here
3064
+ so a client written now parses it when it first appears.
3065
+ required: [job_id]
3066
+ properties:
3067
+ job_id:
3068
+ type: string
3069
+ format: uuid
3070
+
3071
+ TrialRunningData:
3072
+ type: object
3073
+ required: [trial_id, task_name]
3074
+ properties:
3075
+ trial_id:
3076
+ type: string
3077
+ format: uuid
3078
+ task_name:
3079
+ type: string
3080
+
3081
+ TrialScoringData:
3082
+ type: object
3083
+ required: [trial_id]
3084
+ properties:
3085
+ trial_id:
3086
+ type: string
3087
+ format: uuid
3088
+ captured_bytes:
3089
+ type: integer
3090
+ description: Bytes of agent stdout retained for the failure detail.
3091
+
3092
+ TrialSpendData:
3093
+ type: object
3094
+ description: >
3095
+ A mid-run spend sample landed on a still-live trial. Emitted only
3096
+ when the reading actually updated a RUNNING/SCORING row. The token
3097
+ sums come from the same ledger aggregation that produced the money
3098
+ figure, present only when the sample carried them — an older event
3099
+ replays without them.
3100
+ required: [trial_id, task_name, live_spent_usd]
3101
+ properties:
3102
+ trial_id:
3103
+ type: string
3104
+ format: uuid
3105
+ task_name:
3106
+ type: string
3107
+ live_spent_usd:
3108
+ type: number
3109
+ description: The same lagging lower bound as Trial.live_spent_usd.
3110
+ n_input_tokens:
3111
+ type: integer
3112
+ description: Input tokens so far; includes cache tokens.
3113
+ n_cache_tokens:
3114
+ type: integer
3115
+ description: Cached input tokens so far (a subset of n_input_tokens).
3116
+ n_output_tokens:
3117
+ type: integer
3118
+ description: Output tokens so far.
3119
+
3120
+ TrialSettledData:
3121
+ type: object
3122
+ description: >
3123
+ A trial reached a terminal status. `reward` is present only on the
3124
+ scored path; `exception_type` only on a failure; `attempt_phase`
3125
+ appears when the settle happened mid-phase (worker death), which is
3126
+ exactly when knowing the phase is worth having.
3127
+ required: [trial_id, task_name, status]
3128
+ properties:
3129
+ trial_id:
3130
+ type: string
3131
+ format: uuid
3132
+ task_name:
3133
+ type: string
3134
+ status:
3135
+ $ref: '#/components/schemas/TrialStatus'
3136
+ reward:
3137
+ type: [number, 'null']
3138
+ exception_type:
3139
+ type: string
3140
+ attempt_phase:
3141
+ oneOf:
3142
+ - $ref: '#/components/schemas/AttemptPhase'
3143
+ - type: 'null'
3144
+
3145
+ # -------------------------------------------------------------------------
3146
+ # Datasets
3147
+ # -------------------------------------------------------------------------
3148
+
3149
+ DatasetVersionState:
3150
+ type: string
3151
+ description: |
3152
+ Terminal: READY, FAILED, ARCHIVED.
3153
+
3154
+ A hosted publish walks IMPORTING -> BUILDING -> VALIDATING on its own,
3155
+ and VALIDATING -> READY happens on its own too: the activation gate
3156
+ promotes the version it proves, and points the dataset's default at
3157
+ it. The activate verb is not part of that path — it exists to move
3158
+ the default to a different version (an older READY one, or a newer
3159
+ one), never to finish a publish.
3160
+ enum: [DRAFT, IMPORTING, BUILDING, VALIDATING, READY, FAILED, ARCHIVED]
3161
+
3162
+ DatasetVersion:
3163
+ type: object
3164
+ description: One immutable version of a dataset — one shape on every surface.
3165
+ required: [version, state, created_at, task_count, gate]
3166
+ properties:
3167
+ version:
3168
+ type: string
3169
+ state:
3170
+ $ref: '#/components/schemas/DatasetVersionState'
3171
+ created_at:
3172
+ type: string
3173
+ format: date-time
3174
+ task_count:
3175
+ type: integer
3176
+ minimum: 0
3177
+ gate:
3178
+ oneOf:
3179
+ - $ref: '#/components/schemas/ActivationGate'
3180
+ - type: 'null'
3181
+ description: >
3182
+ The activation gate's schedule state for this version. Null when
3183
+ no gate was ever scheduled — a version imported before the gate
3184
+ scheduler existed, a platform-seeded one, or one whose import
3185
+ archived no reference solutions (see the import's `warnings`).
3186
+
3187
+ ActivationGateStatus:
3188
+ type: string
3189
+ description: |
3190
+ Where the automatic activation gate stands for a version. The gate is
3191
+ the oracle-conformance run the worker schedules once a hosted import
3192
+ completes: the version's held-out reference (gold) solution must
3193
+ score exactly 1.0 through the real execution path, and a do-nothing
3194
+ agent must not. PENDING = scheduled, waiting for a worker; RUNNING =
3195
+ proving tasks now (per-task verdicts land on the task list as they
3196
+ finish); PASSED = every task is activation-eligible, and the version
3197
+ was promoted to READY and made the dataset's active version by the
3198
+ gate itself — except on a platform-curated dataset, whose default is
3199
+ an operator setting, so its versions sit PASSED until an operator
3200
+ promotes them; FAILED = at least one task is not — `failure` names
3201
+ them and the dataset's active version is left alone.
3202
+ enum: [PENDING, RUNNING, PASSED, FAILED]
3203
+
3204
+ ActivationGate:
3205
+ type: object
3206
+ description: >
3207
+ A version's activation-gate schedule state. Watch `status` move
3208
+ PENDING -> RUNNING -> PASSED/FAILED after a publish; the state that
3209
+ used to read as "VALIDATING forever" is this object saying where the
3210
+ proof stands.
3211
+ required: [status, attempts, failure]
3212
+ properties:
3213
+ status:
3214
+ $ref: '#/components/schemas/ActivationGateStatus'
3215
+ attempts:
3216
+ type: integer
3217
+ minimum: 0
3218
+ description: >
3219
+ Times a worker has taken this gate. A run lost to a dead worker
3220
+ is re-queued and RESUMES (verdicts already landed are kept),
3221
+ bounded before the gate fails with `gate_lease_lost`.
3222
+ failure:
3223
+ oneOf:
3224
+ - $ref: '#/components/schemas/ActivationGateFailure'
3225
+ - type: 'null'
3226
+ description: Why the gate FAILED; null unless status is FAILED.
3227
+
3228
+ ActivationGateFailure:
3229
+ type: object
3230
+ description: >
3231
+ The gate's structured failure. `failed_tasks` names each ineligible
3232
+ task with the gate's own reasons (a gold solution that never scored
3233
+ 1.0, a no-op that did, a solution that could not be resolved), so
3234
+ nothing has to be parsed back out of the message.
3235
+ required: [code, message]
3236
+ properties:
3237
+ code:
3238
+ type: string
3239
+ description: '`gate_failed`, or `gate_lease_lost` after repeated worker loss.'
3240
+ message:
3241
+ type: string
3242
+ failed_tasks:
3243
+ type: array
3244
+ description: >
3245
+ The ineligible tasks (first 25; when truncated,
3246
+ `failed_task_count` carries the true total).
3247
+ items:
3248
+ type: object
3249
+ required: [task_name, outcome, reasons]
3250
+ properties:
3251
+ task_name:
3252
+ type: string
3253
+ outcome:
3254
+ type: string
3255
+ description: FAIL, or ERROR (inconclusive — no usable score).
3256
+ reasons:
3257
+ type: array
3258
+ items:
3259
+ type: string
3260
+ failed_task_count:
3261
+ type: integer
3262
+ minimum: 0
3263
+
3264
+ GateRunning:
3265
+ type: object
3266
+ description: |
3267
+ The 202 body of the activation verb while the gate is still proving
3268
+ the version. Deliberately NOT the error envelope: an in-progress
3269
+ gate is a healthy state, and `error` on this surface always means
3270
+ "this request failed".
3271
+ required: [code, message, gate]
3272
+ properties:
3273
+ code:
3274
+ type: string
3275
+ enum: [gate_running]
3276
+ message:
3277
+ type: string
3278
+ gate:
3279
+ type: object
3280
+ description: Gate progress at the moment of the call.
3281
+ required: [status, tasks, unverified, ineligible]
3282
+ properties:
3283
+ status:
3284
+ $ref: '#/components/schemas/ActivationGateStatus'
3285
+ tasks:
3286
+ type: integer
3287
+ minimum: 0
3288
+ unverified:
3289
+ type: integer
3290
+ minimum: 0
3291
+ description: Tasks the gate has not yet produced a verdict for.
3292
+ ineligible:
3293
+ type: integer
3294
+ minimum: 0
3295
+ description: Tasks whose verdict so far is not activation-eligible.
3296
+
3297
+ TaskProviderVerdict:
3298
+ type: object
3299
+ description: >
3300
+ One provider's verdict for a task: runnable there, or refused with
3301
+ the limitation named. Advisory for planning — creating a job whose
3302
+ tasks include one refused on the chosen provider is rejected with the
3303
+ same reason, so nothing is ever spent on a trial that cannot execute.
3304
+ required: [ok]
3305
+ properties:
3306
+ ok:
3307
+ type: boolean
3308
+ reason:
3309
+ type: string
3310
+ description: Present when ok is false.
3311
+
3312
+ Task:
3313
+ type: object
3314
+ description: >
3315
+ Public task fields only — instructions, environments, and tests never
3316
+ leave the server.
3317
+ required: [task_name, agent_timeout_sec, verifier_timeout_sec, providers, gate]
3318
+ properties:
3319
+ task_name:
3320
+ type: string
3321
+ agent_timeout_sec:
3322
+ type: number
3323
+ verifier_timeout_sec:
3324
+ type: number
3325
+ providers:
3326
+ type: object
3327
+ description: Verdict per sandbox provider.
3328
+ additionalProperties:
3329
+ $ref: '#/components/schemas/TaskProviderVerdict'
3330
+ gate:
3331
+ oneOf:
3332
+ - $ref: '#/components/schemas/TaskGate'
3333
+ - type: 'null'
3334
+ description: >
3335
+ This task's activation-gate verdict; null until the gate has run
3336
+ it. The per-task half of the version's `gate` — while the gate is
3337
+ RUNNING, verdicts appear here as they land.
3338
+
3339
+ TaskGate:
3340
+ type: object
3341
+ description: >
3342
+ One task's activation-gate verdict — the public subset. The full
3343
+ stored verdict carries oracle diagnostics that stay internal, like
3344
+ the environment specs beside it.
3345
+ required: [outcome, flaky, reasons, ran_at]
3346
+ properties:
3347
+ outcome:
3348
+ type: string
3349
+ description: >
3350
+ PASS; FLAKY (gold passed only on a retry — still eligible under
3351
+ the operator default); FAIL (definitive: gold never scored 1.0,
3352
+ or a do-nothing agent did); ERROR (inconclusive — no usable
3353
+ score, e.g. no archived solution to run).
3354
+ flaky:
3355
+ type: boolean
3356
+ reasons:
3357
+ type: array
3358
+ items:
3359
+ type: string
3360
+ description: Human-readable; empty on PASS.
3361
+ ran_at:
3362
+ type: [string, 'null']
3363
+ format: date-time
3364
+
3365
+ TaskPage:
3366
+ type: object
3367
+ required: [items, nextCursor, hasMore]
3368
+ properties:
3369
+ items:
3370
+ type: array
3371
+ items:
3372
+ $ref: '#/components/schemas/Task'
3373
+ nextCursor:
3374
+ type: [string, 'null']
3375
+ hasMore:
3376
+ type: boolean
3377
+
3378
+ UpstreamStatus:
3379
+ type: object
3380
+ description: |
3381
+ Where the dataset's git source points now versus what its active
3382
+ version was built from — the data behind a "new version available"
3383
+ badge. The whole object is null on a dataset with nothing to watch;
3384
+ null is never "up to date". Nothing here imports anything — a new
3385
+ version is always a row you create (or `auto_import` creates).
3386
+ required: [ref, current_commit, latest_commit, moved, behind_by, checked_at, error, auto_import]
3387
+ properties:
3388
+ ref:
3389
+ type: string
3390
+ description: The ref the active version was imported from.
3391
+ current_commit:
3392
+ type: string
3393
+ latest_commit:
3394
+ type: [string, 'null']
3395
+ description: Where the ref points upstream now; null when the last check failed.
3396
+ moved:
3397
+ type: boolean
3398
+ description: True when upstream has moved off the built-from commit. Branch on this.
3399
+ behind_by:
3400
+ type: [integer, 'null']
3401
+ description: Reserved; always null today.
3402
+ checked_at:
3403
+ type: [string, 'null']
3404
+ format: date-time
3405
+ error:
3406
+ type: [string, 'null']
3407
+ description: Why the last check failed. Show "could not check", not "up to date".
3408
+ auto_import:
3409
+ type: boolean
3410
+ description: Whether a moved upstream automatically imports a new version.
3411
+
3412
+ Dataset:
3413
+ type: object
3414
+ description: >
3415
+ A dataset in the catalog. List responses carry the summary fields;
3416
+ the detail route additionally populates `versions`,
3417
+ `selected_version`, `tasks`, `created_at`, and `updated_at`.
3418
+ required: [name, title, description, active_version, upstream]
3419
+ properties:
3420
+ name:
3421
+ type: string
3422
+ title:
3423
+ type: [string, 'null']
3424
+ description:
3425
+ type: [string, 'null']
3426
+ active_version:
3427
+ oneOf:
3428
+ - $ref: '#/components/schemas/DatasetVersion'
3429
+ - type: 'null'
3430
+ description: Null when no version is active (bare-name job refs refuse).
3431
+ versions:
3432
+ type: array
3433
+ description: All versions, newest first (detail only).
3434
+ items:
3435
+ $ref: '#/components/schemas/DatasetVersion'
3436
+ selected_version:
3437
+ oneOf:
3438
+ - $ref: '#/components/schemas/DatasetVersion'
3439
+ - type: 'null'
3440
+ description: The version whose tasks are listed below (detail only).
3441
+ tasks:
3442
+ $ref: '#/components/schemas/TaskPage'
3443
+ description: One page of the selected version's tasks (detail only).
3444
+ upstream:
3445
+ oneOf:
3446
+ - $ref: '#/components/schemas/UpstreamStatus'
3447
+ - type: 'null'
3448
+ created_at:
3449
+ type: string
3450
+ format: date-time
3451
+ updated_at:
3452
+ type: string
3453
+ format: date-time
3454
+
3455
+ DatasetPage:
3456
+ type: object
3457
+ required: [items, nextCursor, hasMore]
3458
+ properties:
3459
+ items:
3460
+ type: array
3461
+ items:
3462
+ $ref: '#/components/schemas/Dataset'
3463
+ nextCursor:
3464
+ type: [string, 'null']
3465
+ hasMore:
3466
+ type: boolean
3467
+
3468
+ DatasetPatch:
3469
+ type: object
3470
+ description: The only settable dataset field.
3471
+ required: [upstream_auto_import]
3472
+ properties:
3473
+ upstream_auto_import:
3474
+ type: boolean
3475
+
3476
+ PublishRequest:
3477
+ type: object
3478
+ description: >
3479
+ Multipart body of POST /api/datasets/publish. Exactly one source:
3480
+ `git_url` + `git_ref`, or `archive`.
3481
+ required: [name, version]
3482
+ properties:
3483
+ name:
3484
+ type: string
3485
+ description: Catalog dataset name the version lands under (created or extended).
3486
+ version:
3487
+ type: string
3488
+ description: Version label for the new immutable version.
3489
+ git_url:
3490
+ type: string
3491
+ description: https:// repository URL. For a private repo, put a token in the URL.
3492
+ git_ref:
3493
+ type: string
3494
+ description: Pinned branch, tag, or commit. Required with git_url.
3495
+ archive:
3496
+ type: string
3497
+ format: binary
3498
+ description: Gzipped tarball of a standard-layout corpus directory.
3499
+
3500
+ DatasetImportStatus:
3501
+ type: string
3502
+ description: >
3503
+ Same four words a job uses, deliberately. Terminal: COMPLETED (the
3504
+ corpus landed as a dataset version; runnable once its activation gate
3505
+ passes, which also activates it) and FAILED.
3506
+ enum: [QUEUED, RUNNING, COMPLETED, FAILED]
3507
+
3508
+ DatasetImportFailure:
3509
+ type: object
3510
+ description: Structured failure detail for a FAILED import.
3511
+ required: [code, message]
3512
+ properties:
3513
+ code:
3514
+ type: string
3515
+ description: '`import_failed` when none was recorded.'
3516
+ message:
3517
+ type: string
3518
+ failures:
3519
+ type: array
3520
+ description: Per-task parse/validation failures, when the corpus was reachable.
3521
+ items:
3522
+ type: object
3523
+ required: [task_name, error]
3524
+ properties:
3525
+ task_name:
3526
+ type: string
3527
+ error:
3528
+ type: string
3529
+
3530
+ ImportWarning:
3531
+ type: object
3532
+ description: |
3533
+ Non-fatal but consequential import outcomes. A version whose warnings
3534
+ include `no_solutions_archived` cannot be activated through this API
3535
+ (`version_not_activatable`).
3536
+ required: [code]
3537
+ properties:
3538
+ code:
3539
+ type: string
3540
+ enum:
3541
+ - solutions_archiving_disabled
3542
+ - no_solutions_archived
3543
+ - partial_solutions_archived
3544
+ message:
3545
+ type: string
3546
+
3547
+ DatasetImport:
3548
+ type: object
3549
+ description: >
3550
+ An asynchronous publish. Self-describing: every response names the
3551
+ dataset@version being imported, so a caller can render the row it
3552
+ just created without a follow-up read.
3553
+ required: [id, status, name, version, failure, warnings]
3554
+ properties:
3555
+ id:
3556
+ type: string
3557
+ status:
3558
+ $ref: '#/components/schemas/DatasetImportStatus'
3559
+ name:
3560
+ type: string
3561
+ description: Catalog dataset name the import creates or extends.
3562
+ version:
3563
+ type: string
3564
+ failure:
3565
+ oneOf:
3566
+ - $ref: '#/components/schemas/DatasetImportFailure'
3567
+ - type: 'null'
3568
+ description: >
3569
+ Why the import FAILED; null otherwise. Named `failure`, never
3570
+ `error` — see JobFailure.
3571
+ warnings:
3572
+ type: array
3573
+ items:
3574
+ $ref: '#/components/schemas/ImportWarning'
3575
+ task_count:
3576
+ type: integer
3577
+ description: Number of tasks parsed, once counted.
3578
+ created_at:
3579
+ type: string
3580
+ format: date-time
3581
+ updated_at:
3582
+ type: string
3583
+ format: date-time
3584
+
3585
+ DatasetImportPage:
3586
+ type: object
3587
+ required: [items, nextCursor, hasMore]
3588
+ properties:
3589
+ items:
3590
+ type: array
3591
+ items:
3592
+ $ref: '#/components/schemas/DatasetImport'
3593
+ nextCursor:
3594
+ type: [string, 'null']
3595
+ hasMore:
3596
+ type: boolean
3597
+
3598
+ # -------------------------------------------------------------------------
3599
+ # Registered agents
3600
+ # -------------------------------------------------------------------------
3601
+
3602
+ Agent:
3603
+ type: object
3604
+ description: A private agent registered by the caller.
3605
+ required: [name, source, run_command, env, created_at, updated_at]
3606
+ properties:
3607
+ name:
3608
+ type: string
3609
+ description: The name to put in job `agents[].name`.
3610
+ source:
3611
+ type: string
3612
+ enum: [install_script, tarball]
3613
+ description: How the executables were produced. Echoed, never guessed.
3614
+ run_command:
3615
+ type: string
3616
+ description: Run headless with `sh -c` at the task working directory.
3617
+ env:
3618
+ type: object
3619
+ description: >
3620
+ Caller-declared env injected at RUN time only; may not override
3621
+ the run contract's own keys (`agent_invalid_env` at
3622
+ registration).
3623
+ additionalProperties:
3624
+ type: string
3625
+ created_at:
3626
+ type: string
3627
+ format: date-time
3628
+ updated_at:
3629
+ type: string
3630
+ format: date-time
3631
+
3632
+ AgentRegistration:
3633
+ type: object
3634
+ description: >
3635
+ Multipart registration body. Exactly one source: `install_script` or
3636
+ `archive`.
3637
+ required: [name, run_command]
3638
+ properties:
3639
+ name:
3640
+ type: string
3641
+ run_command:
3642
+ type: string
3643
+ env:
3644
+ type: string
3645
+ description: JSON object of env entries, as a multipart text part.
3646
+ install_script:
3647
+ type: string
3648
+ description: >
3649
+ The script itself (not a path). Runs in a throwaway builder
3650
+ sandbox with internet and zero secrets; must leave executables in
3651
+ `$PREFIX/bin`.
3652
+ archive:
3653
+ type: string
3654
+ format: binary
3655
+ description: Gzipped tarball of the agent directory. Same build rules.
3656
+
3657
+ AgentPage:
3658
+ type: object
3659
+ required: [items, nextCursor, hasMore]
3660
+ properties:
3661
+ items:
3662
+ type: array
3663
+ items:
3664
+ $ref: '#/components/schemas/Agent'
3665
+ nextCursor:
3666
+ type: [string, 'null']
3667
+ hasMore:
3668
+ type: boolean
3669
+
3670
+ # -------------------------------------------------------------------------
3671
+ # Meta
3672
+ # -------------------------------------------------------------------------
3673
+
3674
+ StatusVocabulary:
3675
+ type: object
3676
+ description: One status vocabulary and its terminal members.
3677
+ required: [values, terminal]
3678
+ properties:
3679
+ values:
3680
+ type: array
3681
+ items:
3682
+ type: string
3683
+ terminal:
3684
+ type: array
3685
+ items:
3686
+ type: string
3687
+
3688
+ AgentCapability:
3689
+ type: object
3690
+ description: One built-in agent's declared capabilities.
3691
+ required: [name, effort_support, version_pinnable]
3692
+ properties:
3693
+ name:
3694
+ type: string
3695
+ effort_support:
3696
+ type: boolean
3697
+ version_pinnable:
3698
+ type: boolean
3699
+ latest_version:
3700
+ type: [string, 'null']
3701
+
3702
+ ProviderCapability:
3703
+ type: object
3704
+ required: [name, default, sizing, refuses]
3705
+ properties:
3706
+ name:
3707
+ type: string
3708
+ default:
3709
+ type: boolean
3710
+ sizing:
3711
+ type: object
3712
+ required: [max_cpus, max_memory_mb, max_storage_mb, storage]
3713
+ properties:
3714
+ max_cpus:
3715
+ type: integer
3716
+ max_memory_mb:
3717
+ type: integer
3718
+ max_storage_mb:
3719
+ type: integer
3720
+ storage:
3721
+ type: string
3722
+ enum: [sized, fixed]
3723
+ refuses:
3724
+ type: array
3725
+ items:
3726
+ type: object
3727
+ required: [capability, reason]
3728
+ properties:
3729
+ capability:
3730
+ type: string
3731
+ reason:
3732
+ type: string
3733
+
3734
+ ManagedProviderCapability:
3735
+ type: object
3736
+ description: |
3737
+ One managed sandbox door and whether this deployment serves it.
3738
+ `configured` is derived from operator config presence only — it says
3739
+ nothing about whether the pass-through behind the door is deployed or
3740
+ the credential is valid. `agent_sessions` answers the separate
3741
+ question of whether the door can carry a full SDK agent session.
3742
+ required:
3743
+ - name
3744
+ - configured
3745
+ - requires_config
3746
+ - missing_config
3747
+ - agent_sessions
3748
+ - agent_sessions_reason
3749
+ properties:
3750
+ name:
3751
+ type: string
3752
+ configured:
3753
+ type: boolean
3754
+ requires_config:
3755
+ type: array
3756
+ items:
3757
+ type: string
3758
+ missing_config:
3759
+ description: The subset of `requires_config` missing right now — empty when configured.
3760
+ type: array
3761
+ items:
3762
+ type: string
3763
+ agent_sessions:
3764
+ type: boolean
3765
+ agent_sessions_reason:
3766
+ description: Why not, when `agent_sessions` is false. Null otherwise.
3767
+ type: [string, 'null']
3768
+
3769
+ CapabilityDocument:
3770
+ type: object
3771
+ description: |
3772
+ Everything a client would otherwise hardcode. Public and cacheable —
3773
+ no API key needed, so a signed-out page can populate its own agent
3774
+ picker. `schema_version` bumps when a FIELD changes meaning, never
3775
+ when a value changes.
3776
+ required:
3777
+ - schema_version
3778
+ - agents
3779
+ - agent_registration
3780
+ - sandbox_providers
3781
+ - managed_providers
3782
+ - platform_constraints
3783
+ - network_modes
3784
+ - statuses
3785
+ - limits
3786
+ - import_warning_codes
3787
+ - error_codes
3788
+ properties:
3789
+ schema_version:
3790
+ type: integer
3791
+ agents:
3792
+ type: array
3793
+ description: Built-in agents and their declared capabilities.
3794
+ items:
3795
+ $ref: '#/components/schemas/AgentCapability'
3796
+ agent_registration:
3797
+ type: object
3798
+ description: Rules a bring-your-own agent registration must satisfy.
3799
+ properties:
3800
+ name_pattern:
3801
+ type: string
3802
+ max_name_length:
3803
+ type: integer
3804
+ max_run_command_length:
3805
+ type: integer
3806
+ max_install_script_length:
3807
+ type: integer
3808
+ max_env_entries:
3809
+ type: integer
3810
+ max_per_user:
3811
+ type: integer
3812
+ max_upload_bytes:
3813
+ type: integer
3814
+ reserved_names:
3815
+ type: array
3816
+ description: Built-in names a registration may not reuse.
3817
+ items:
3818
+ type: string
3819
+ reserved_env_keys:
3820
+ type: array
3821
+ description: Env keys the platform owns; declaring one is refused.
3822
+ items:
3823
+ type: string
3824
+ sandbox_providers:
3825
+ type: array
3826
+ items:
3827
+ $ref: '#/components/schemas/ProviderCapability'
3828
+ managed_providers:
3829
+ type: array
3830
+ description: |
3831
+ The managed doors this deployment serves — a different question
3832
+ from `sandbox_providers`, which is about the eval lane. A managed
3833
+ sandbox is one the caller drives directly holding nothing but an
3834
+ Evolve key.
3835
+ items:
3836
+ $ref: '#/components/schemas/ManagedProviderCapability'
3837
+ platform_constraints:
3838
+ type: array
3839
+ description: Constraints that hold on EVERY provider.
3840
+ items:
3841
+ type: object
3842
+ required: [capability, reason]
3843
+ properties:
3844
+ capability:
3845
+ type: string
3846
+ reason:
3847
+ type: string
3848
+ network_modes:
3849
+ type: array
3850
+ items:
3851
+ type: string
3852
+ statuses:
3853
+ type: object
3854
+ description: Every status vocabulary on the surface, with terminal members.
3855
+ required: [job, trial, import, dataset_version]
3856
+ properties:
3857
+ job:
3858
+ $ref: '#/components/schemas/StatusVocabulary'
3859
+ trial:
3860
+ $ref: '#/components/schemas/StatusVocabulary'
3861
+ import:
3862
+ $ref: '#/components/schemas/StatusVocabulary'
3863
+ dataset_version:
3864
+ $ref: '#/components/schemas/StatusVocabulary'
3865
+ limits:
3866
+ type: object
3867
+ properties:
3868
+ job:
3869
+ type: object
3870
+ properties:
3871
+ max_n_attempts:
3872
+ type: integer
3873
+ max_agents:
3874
+ type: integer
3875
+ max_trials:
3876
+ type: integer
3877
+ n_concurrent_trials:
3878
+ type: object
3879
+ properties:
3880
+ default:
3881
+ type: integer
3882
+ max:
3883
+ type: integer
3884
+ default_max_trial_spend_usd:
3885
+ type: number
3886
+ default_sandbox_provider:
3887
+ type: string
3888
+ default_sizing:
3889
+ type: object
3890
+ properties:
3891
+ cpus:
3892
+ type: integer
3893
+ memory_mb:
3894
+ type: integer
3895
+ storage_mb:
3896
+ type: integer
3897
+ model_required:
3898
+ type: boolean
3899
+ description: Every agent must name a model; the server applies no default.
3900
+ default_agent_timeout_sec:
3901
+ type: number
3902
+ default_verifier_timeout_sec:
3903
+ type: number
3904
+ reasoning_efforts:
3905
+ type: array
3906
+ items:
3907
+ type: string
3908
+ default_reasoning_effort:
3909
+ type: string
3910
+ max_job_name_length:
3911
+ type: integer
3912
+ description: Longest accepted `job_name` (JobCreate mirrors it as maxLength).
3913
+ compare:
3914
+ type: object
3915
+ properties:
3916
+ min_ids:
3917
+ type: integer
3918
+ max_ids:
3919
+ type: integer
3920
+ stop:
3921
+ type: object
3922
+ properties:
3923
+ max_trial_ids:
3924
+ type: integer
3925
+ description: Most trials one stop request may name (StopRequest mirrors it as maxItems).
3926
+ pagination:
3927
+ type: object
3928
+ properties:
3929
+ collections:
3930
+ type: object
3931
+ properties:
3932
+ default:
3933
+ type: integer
3934
+ max:
3935
+ type: integer
3936
+ dataset_tasks:
3937
+ type: object
3938
+ properties:
3939
+ default:
3940
+ type: integer
3941
+ max:
3942
+ type: integer
3943
+ trace_events:
3944
+ type: object
3945
+ properties:
3946
+ default:
3947
+ type: integer
3948
+ max:
3949
+ type: integer
3950
+ uploads:
3951
+ type: object
3952
+ properties:
3953
+ dataset_archive_bytes:
3954
+ type: integer
3955
+ agent_tarball_bytes:
3956
+ type: integer
3957
+ dataset_names:
3958
+ type: object
3959
+ description: >
3960
+ Name constraints for published datasets. Beyond the pattern,
3961
+ `imports` is a reserved name (a literal path segment under
3962
+ `/api/datasets`).
3963
+ properties:
3964
+ pattern:
3965
+ type: string
3966
+ max_name_length:
3967
+ type: integer
3968
+ max_version_length:
3969
+ type: integer
3970
+ max_git_url_length:
3971
+ type: integer
3972
+ max_git_ref_length:
3973
+ type: integer
3974
+ max_items_named_in_error_message:
3975
+ type: integer
3976
+ description: How many items an error MESSAGE names before "and N more".
3977
+ import_warning_codes:
3978
+ type: array
3979
+ items:
3980
+ type: string
3981
+ error_codes:
3982
+ type: array
3983
+ description: The closed error-code union, enumerated at runtime.
3984
+ items:
3985
+ type: string
3986
+
3987
+ # -------------------------------------------------------------------------
3988
+ # Auth
3989
+ # -------------------------------------------------------------------------
3990
+
3991
+ AuthStatus:
3992
+ type: object
3993
+ description: The caller's identity and the key this request authenticated with.
3994
+ required: [user_id, key]
3995
+ properties:
3996
+ user_id:
3997
+ type: string
3998
+ email:
3999
+ type: [string, 'null']
4000
+ key:
4001
+ $ref: '#/components/schemas/ApiKey'
4002
+
4003
+ ApiKey:
4004
+ type: object
4005
+ description: A key descriptor. The secret is never returned.
4006
+ required: [id, created_at]
4007
+ properties:
4008
+ id:
4009
+ type: string
4010
+ label:
4011
+ type: [string, 'null']
4012
+ created_at:
4013
+ type: [string, 'null']
4014
+ format: date-time
4015
+ description: Null when the credential store predates creation stamping.
4016
+ last_used_at:
4017
+ type: [string, 'null']
4018
+ format: date-time
4019
+
4020
+ ApiKeyPage:
4021
+ type: object
4022
+ required: [items, nextCursor, hasMore]
4023
+ properties:
4024
+ items:
4025
+ type: array
4026
+ items:
4027
+ $ref: '#/components/schemas/ApiKey'
4028
+ nextCursor:
4029
+ type: [string, 'null']
4030
+ hasMore:
4031
+ type: boolean