@bjoernrochel/sevro 0.1.0-rc.2 → 0.1.0-rc.21

package/docs/evidence/local-http-74.md

@@ -0,0 +1,104 @@
+# Local HTTP isolation validation for #74
+
+Observed on 2026-10-14, macOS arm64, Bun 1.3.13, Node 24.13.0 and Codex CLI
+0.162.1. This record addresses [issue #74](https://github.com/BjRo/sevro/issues/74).
+The supported operator interface is the runtime
+[local HTTP option](../runtime-v1.md#local-http-test-servers).
+
+## Host boundary
+
+The installed `codex sandbox` command ran the same wildcard Node HTTP listener
+and loopback client under four generated permission profiles:
+
+| Configuration                                         | Observed result                        |
+| ----------------------------------------------------- | -------------------------------------- |
+| Command networking disabled                           | `listen()` fails with `EPERM`          |
+| Disabled networking with `allow_local_binding = true` | `listen()` fails with `EPERM`          |
+| Enabled managed proxy with local binding              | HTTP round trip succeeds               |
+| Enabled networking without managed proxy              | HTTP round trip and public TCP succeed |
+
+With the managed proxy and local binding, a direct TCP connection to
+`1.1.1.1:443` instead fails with `EPERM`; its unproxied control connects. The
+regression fixture uses the documentation-only `192.0.2.1` address and requires
+an explicit denial rather than treating a timeout as evidence. The macOS shell
+boundary separately permits binding and loopback traffic while denying other
+outbound IP traffic. Source/authentication/configuration denials remain part of
+the candidate's real native preflight.
+
+The implementation does not use an outer macOS sandbox around Codex: the
+native Codex sandbox cannot nest inside `sandbox-exec`. Codex's managed proxy
+and its native local-binding rules establish that boundary. This selection can
+reach existing loopback services and does not establish per-trial network
+privacy on macOS. Native DNS permissions remain Codex-owned.
+
+## Original upstream outcomes
+
+Both fresh evaluations used Darrow's `fix/pr257-validation` checkout at
+`1e6759c4349567f897a167a588b3714095889e86` with its existing unpublished fixture
+repairs, the Sevro source override, and a replacement runtime profile containing
+`network.localHttp: true`. Candidate route: `gpt-6.1-sol/medium`, Codex
+`app-server`, passive, N=1, jobs=1, threshold=1, timeout 1,200,000 ms. Semantic
+route: `gpt-6-luna/medium`; these cases use shell graders. No Darrow case or
+integration test was removed or edited for these runs.
+
+| Case                               | Hidden contract | Configured upstream check                                                                  |
+| ---------------------------------- | --------------- | ------------------------------------------------------------------------------------------ |
+| `orchestration-oss-express-links`  | Passed, exit 0  | `npx mocha --reporter dot test/res.links.js`: exit 0                                       |
+| `orchestration-oss-requests-proxy` | Passed, exit 0  | `python -m pytest -q tests/test_requests.py -k 'proxy or send' --disable-warnings`: exit 0 |
+
+The native Express candidate also ran
+`node node_modules/mocha/bin/_mocha --reporter dot test/res.links.js`: **8
+passing**, exit 0. Its `npx` invocation attempted the public npm registry and
+was denied by the native proxy; `npm test` exited 255 without test output. The
+direct Mocha run and shell-grader outcome remain distinct from those failures.
+
+The two exact commands quoted in #74 were additionally rerun through Sevro's
+isolated shell grader against the original retained rc.11 candidates:
+
+```sh
+node node_modules/mocha/bin/_mocha --reporter dot test/res.links.js
+.git/fixture-python/bin/python -m pytest tests/test_requests.py -q -k '(session or redirect or proxy or mixed_case or ALTERNATIVE or prepared or requests_are_updated_each_time) and not proxy_error'
+```
+
+Express: **8 passing**, exit 0. Requests: **77 passed, 2 failed, 229 deselected**,
+exit 1, with no socket-binding setup errors. Both Requests failures ran their
+local servers: `test_pyopenssl_redirect` encounters an expired pytest-httpbin
+certificate; `test_redirecting_to_bad_url` receives an HTTP 500 from the newer
+Werkzeug fixture instead of the expected invalid-URL redirect. The fresh native
+candidate reproduced both against the original `HEAD:requests/sessions.py`.
+Neither failure was waived in this exact-command rerun. A supplemental native
+selection excluding them passed 126 tests; it does not replace the full result.
+
+Fresh run IDs are `acba33c5-974c-4ab5-ba6f-5d9e947560dc` (Express) and
+`a3868fae-bca0-4e29-a995-09d749bbe367` (Requests). Local retained evidence is
+under Darrow's `evals/results/sevro-74-orchestration-oss-{express-links,requests-proxy}`;
+the exact-command outputs are in `/private/tmp/sevro-74-original-upstream`.
+Raw native evidence remains local. These model runs used the implementation
+snapshot rooted at Sevro `f66e6667f28c956c41aae096bd11581f81c18a40`, build digest
+`a4b11528f46daafa541ff672aad815a1183ca0615543eb587ae79e268680566b`, before final
+helper cleanup and documentation. Final-source integration and CI evidence
+must be assessed separately.
+
+## Regression development
+
+Observed test-first slices, in order:
+
+1. Red — `lean-ctx -c 'bun test tests/runtime-cli.test.ts -t "CLI snapshots local HTTP"'`:
+   the new runtime field is refused with exit 64.
+   Green — `lean-ctx -c 'bun test tests/runtime-cli.test.ts -t "CLI snapshots local HTTP"'`:
+   1 passed, including retained policy and changed evaluation identity.
+2. Red — `lean-ctx -c 'bun test tests/shell-grader.test.ts -t "shell local HTTP"'`:
+   the real listener reports `bind-error:EPERM`, exit 1.
+   Green — `lean-ctx -c 'bun test tests/shell-grader.test.ts -t "shell local HTTP"'`:
+   1 passed, with real HTTP round trip and outbound TCP denial.
+3. Red — `lean-ctx -c 'bun test tests/runtime-host.test.ts -t "Codex candidate local HTTP"'`:
+   the unchanged native command policy returns `unavailable` for the socket probe.
+   Green — `lean-ctx -c 'bun test tests/runtime-host.test.ts -t "Codex candidate local HTTP"'`:
+   1 passed through the installed Codex sandbox.
+
+The affected six-file integration selection passed **296 tests**, skipped three
+platform-specific tests, and failed none. Additional semantic/advisory native
+probes passed and confirm the opt-in leaves those roles disabled.
+`bun run check:typescript --fast` passed. Full acceptance is owned by the exact
+published commit's GitHub `TypeScript quality` gate; these local observations
+do not establish Ubuntu acceptance or replace that gate.

package/docs/evidence/runtime-startup-30/README.md

@@ -0,0 +1,185 @@
+# Runtime startup measurements for issue 30
+
+The matched synthetic native-host benchmark reduced candidate setup plus the
+controlled participant from 14.37–14.92 seconds to 6.20–6.66 seconds. The response-only
+grading host call, including its separate native startup, fell from 13.49–15.03
+seconds to 5.45–5.65 seconds. These are runner-boundary measurements with synthetic
+model responses, not measurements of API latency or live model quality.
+
+## Reproduction and conditions
+
+Run the same maintained [benchmark harness](../../../scripts/benchmark-runtime-startup.ts)
+against an unchanged checkout of `c205482b25d4d52a395c70434f2b392f6acfb279` and this
+candidate, sequentially on one machine:
+
+```sh
+bun scripts/benchmark-runtime-startup.ts /absolute/baseline /tmp/before.json
+bun scripts/benchmark-runtime-startup.ts /absolute/candidate /tmp/after.json
+```
+
+The retained [before](before.json) and [after](after.json) observations used
+macOS arm64, Bun 1.3.13, Codex CLI 0.162.1, the same synthetic plugin/response case,
+`synthetic-candidate` and `synthetic-semantic` model labels, and `low` effort.
+There were three candidate/semantic repetitions in each process, with a 32 MiB,
+2,049-entry public dependency tree. Every role received a fresh private home and
+native plugin installation. The first candidate used a cold verified-snapshot
+cache; repetitions two and three reused the policy. OS filesystem caches were
+uncontrolled and are not described as cold. Baseline ran before candidate;
+order effects and three repetitions limit generalization.
+
+The harness wraps the same production functions and process boundaries without
+changing their arguments or timeout. Only external model output is replaced by a
+fixed event stream. The native plugin installer and `codex sandbox` remain real.
+This fixture does not claim live skill invocation; separate native integration
+coverage and consumer smoke own that evidence. Timings overlap: runtime and
+sandbox phases are inside candidate/grading-host totals, and plugin commands are
+inside plugin installation. Do not sum both levels. The `grading` measurement surrounds the semantic host
+call with its response-only policy; it excludes engine fixture preparation,
+verdict parsing, and extension bookkeeping.
+
+## Observed timings
+
+All values below are milliseconds, in repetition order.
+
+| Boundary                                     | Before                   | After                 |
+| -------------------------------------------- | ------------------------ | --------------------- |
+| Configuration fingerprint (once)             | 262                      | 299                   |
+| Candidate runtime preparation                | 1,969 / 1,357 / 1,273    | 721 / 431 / 443       |
+| Candidate hook dependency preparation        | 1,226 / 1,465 / 1,235    | No invocation         |
+| Fresh native plugin installation             | 385 / 141 / 143          | 188 / 141 / 143       |
+| Candidate sandbox probes, summed per role    | 10,938 / 11,032 / 11,622 | 5,490 / 5,404 / 5,448 |
+| Controlled participant process               | 27 / 29 / 39             | 27 / 27 / 24          |
+| Candidate total                              | 14,925 / 14,369 / 14,666 | 6,659 / 6,199 / 6,257 |
+| Semantic runtime preparation                 | 1,174 / 1,236 / 2,262    | Under 1 each          |
+| Semantic hook dependency preparation         | 1,105 / 1,087 / 1,363    | No invocation         |
+| Semantic sandbox probes, summed per role     | 10,855 / 10,803 / 11,032 | 5,548 / 5,360 / 5,365 |
+| Controlled semantic response process         | 28 / 44 / 27             | 29 / 28 / 28          |
+| Grading host total, including native startup | 13,495 / 13,534 / 15,030 | 5,647 / 5,454 / 5,460 |
+
+The probe count fell from two to one per fresh role. Candidate dependency
+materialization reuses immutable verified bytes after its first capture, and
+response-only grading no longer materializes implicit candidate dependencies.
+Configuration hashing remains necessary for identity. Fresh-install warm timings
+were essentially unchanged; no plugin-install speedup is claimed. Participant
+and semantic outputs are synthetic controls, so no API/model saving is claimed.
+
+## Startup deadline diagnosis
+
+A separate authenticated Darrow baseline using Codex 0.162.1 and `gpt-6-luna`
+with medium effort failed before participant execution: the first sandbox probe
+hit its 10-second deadline at 10,004.8 ms. A seedless control retained the same
+tool declarations and also timed out at 10,004.2 ms. With only the diagnostic
+probe deadline increased to 60 seconds, both original full-policy probes passed
+at 10,649.1 and 10,690.4 ms. Participant execution was deliberately withheld in
+that diagnostic. A minimal-policy control without tool grants or seeds passed
+at 5,372.8 and 5,681.1 ms.
+
+Raw diagnostic records are retained for the [original timeout](baseline-timeout.jsonl),
+[seedless timeout](seedless-timeout.jsonl), [full policy with diagnostic deadline](full-policy-diagnostic.jsonl),
+and [minimal policy](minimal-policy-diagnostic.jsonl). The maintained
+[diagnostic preload](../../../scripts/diagnose-codex-startup.ts) records bounded
+phase timing/errors and withholds app-server participant execution. To reproduce
+against an explicit checkout, set `SEVRO_DIAGNOSTIC_SOURCE_ROOT` and
+`SEVRO_DIAGNOSTIC_PHASE_FILE`, then use a CLI wrapper running:
+
+```sh
+bun --preload /absolute/candidate/scripts/diagnose-codex-startup.ts /absolute/baseline/src/cli.ts "$@" --runner-checkout-root /absolute/baseline
+```
+
+Point Darrow's `SEVRO_PACKAGE_BIN` at that wrapper and run:
+
+```sh
+bun evals/runner/run.ts --harness codex --owner-evaluation passive --skill create-commit --case create-commit-conventional-format --model gpt-6-luna --effort medium --trials 1 --jobs 1 --threshold 1 --results-root /absolute/outside-project-results --output /absolute/selection.json
+```
+
+For the deadline control, additionally set `SEVRO_DIAGNOSTIC_TIMEOUT_MS=60000`.
+For runtime controls, append `-- --runtime-config-file /absolute/runtime.json`:
+seedless keeps the original policy except `runtime.seedDirectories=[]`; minimal
+keeps only `format`, `nativeTranscripts: true`, and `hooks.nativeGoal: true`.
+The original diagnostic used the same wrappers with fixed paths; the retained
+preload makes those paths explicit through environment variables.
+
+These one-off controls locate the failure in native sandbox startup and show
+that dependency copying is not necessary for the timeout. Declared tool grants
+increase observed startup on this machine. They do not establish the exact
+native CLI mechanism or a general policy-size scaling law. The product now uses
+a bounded 60-second preflight; the measured saving comes from eliminating the
+second invocation and repeated preparation, not from increasing that deadline.
+The unavailable baseline participant, activation, and grading evidence remain
+unavailable rather than being assigned successful outcomes.
+
+## Snapshot and copy-on-write assessment
+
+The production snapshot owns verified bytes only in runner memory, keyed by the
+policy's seed descriptor. Candidate, peer, checker, and hook writable files never
+alias it. First capture must match the configured digest; later source changes
+do not change that snapshot. Regression coverage changes candidate files and the
+operator source between trials and verifies that the next trial still receives
+the original bytes. Executable modes, internal-link dereferencing, empty
+directories, source protection, offline environment, and role separation remain
+checked.
+
+The benchmark also exercised `COPYFILE_FICLONE_FORCE` on a 32 MiB file. It
+succeeded in about one millisecond in both runs, and changing the destination
+left the source length unchanged. This establishes local copy-on-write support,
+not portable availability or an immutable directory snapshot. The current
+in-memory snapshot deliberately avoids a persistent cache path and its cleanup,
+source-mutation, and cross-process trust requirements. Native clone reuse is not
+implemented; ordinary materialization works on filesystems without cloning.
+Retained policies retain their verified bytes until garbage collection, up to
+the existing per-seed bounds. This memory tradeoff should be measured separately
+for substantially larger dependency sets.
+
+## Native installation reuse assessment
+
+Every actual native installation in both benchmark runs produced configuration
+containing its private home and fixture workspace paths, as retained in the
+`installation-assets` observations. Native install receipts bind an
+`installedPath` inside that particular home's plugin cache; production verifies
+it before accepting explicit invocation. Authorized hook wrappers also bind the
+private runtime, sandbox profile, and execution receipt directory.
+
+A separate [native cache-only reuse experiment](plugin-template-assessment.json)
+installed a local synthetic plugin into a fresh credential-free home. The native
+registration wrote `source_type = "local"` and the original absolute marketplace
+path; the receipt contained the original private `installedPath`. After copying
+only `plugins/cache` into a second fresh home, `codex plugin list --json` returned
+empty `installed` and `available` arrays. To reproduce, use fresh `HOME` and
+`CODEX_HOME`, run `codex plugin marketplace add /absolute/market --json` and
+`codex plugin add probe@sevro-assessment --json`, copy `plugins/cache` to another
+fresh home, and list there. Paths in the raw record use `<assessment>` for the
+throwaway root; no credential file was present.
+
+An entire installed-home template therefore cannot be replayed into a new trial
+without rewriting and reverifying those identities, isolating credentials and
+configuration, and obtaining evidence for the new installed paths. Reusing
+just package files would still require native registration and receipt checks.
+For the measured fixture, warm fresh installation costs about 140 ms versus
+roughly 5.4 seconds for one sandbox startup. No template reuse is implemented.
+Fresh authentication/configuration, native registration and plugin installation,
+receipt validation, hook projection, the combined preflight, and native
+invocation evidence remain per-request work. Fresh-install integration coverage
+is retained.
+
+## Authenticated consumer smoke
+
+The [bounded consumer summary](consumer-smoke.json) records Darrow revision
+`34606ad9e1952b7f4e9c56befa446c18e1a2eb5d`, with no tracked consumer edits. The
+candidate's explicit `tdd-explicit-invocation` case passed execution, shell and
+semantic grading, and complete native explicit-invocation activation. Candidate
+and semantic routes used Codex 0.162.1, `gpt-6-luna`, medium effort, one trial,
+one job, and threshold one:
+
+```sh
+SEVRO_CHECKOUT=/absolute/candidate bun evals/runner/run.ts --harness codex --owner-evaluation passive --skill tdd --case tdd-explicit-invocation --model gpt-6-luna --effort medium --semantic-check-model gpt-6-luna --semantic-check-effort medium --trials 1 --jobs 1 --threshold 1 --results-root /absolute/outside-project-results --output /absolute/selection.json
+```
+
+The earlier implicit `create-commit-conventional-format` smoke completed
+execution and grading on the candidate, including its three shell checks, but
+failed the expected HEAD-change assertion. Its fixture mounted the backend under
+`.agents` without plugin manifests required by the launcher; the participant
+refused the commit. Activation stayed unavailable. The unchanged baseline had
+failed that case before participant execution at sandbox preflight. These
+outcomes are retained honestly; neither the fixture failure nor the later
+single-trial success establishes a matched live latency comparison or repeated
+model reliability. Private native transcripts and credentials remain outside Git.

package/docs/evidence/runtime-startup-30/after.json

@@ -0,0 +1,319 @@
+{
+  "format": "sevro.runtime-startup-benchmark.v1",
+  "sourceRoot": "/Users/bjro/Sources/sevro/.worktrees/perf/30-runtime-startup-overhead",
+  "platform": "darwin",
+  "arch": "arm64",
+  "bun": "1.3.13",
+  "codex": "codex-cli 0.162.1",
+  "case": "synthetic-plugin-response",
+  "model": "synthetic-candidate / synthetic-semantic",
+  "effort": "low",
+  "seed": {
+    "entries": 2049,
+    "bytes": 33554432
+  },
+  "repetitions": 3,
+  "conditions": "Fresh role homes each invocation; first verified snapshot cold, later repetitions warm; OS filesystem caches uncontrolled; no live API/model calls",
+  "rows": [
+    {
+      "repetition": 0,
+      "role": "candidate",
+      "phase": "configuration",
+      "milliseconds": 298.645833,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "runtime.candidate",
+      "milliseconds": 721.155541,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "plugin-command",
+      "milliseconds": 113.721,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "plugin-command",
+      "milliseconds": 70.76491699999997,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "installation-assets",
+      "files": [
+        "cache"
+      ],
+      "configurationContainsPrivateHome": true,
+      "configurationContainsWorkspace": true
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "plugin-install",
+      "milliseconds": 187.57354199999986,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "sandbox-probe",
+      "milliseconds": 5489.6383749999995,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "participant-execution",
+      "milliseconds": 27.323707999999897,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "candidate",
+      "phase": "candidate-total",
+      "milliseconds": 6659.420667,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "semantic",
+      "phase": "runtime.semantic",
+      "milliseconds": 0.3450830000001588,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "semantic",
+      "phase": "plugin-install",
+      "milliseconds": 0.005458999999973457,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "semantic",
+      "phase": "sandbox-probe",
+      "milliseconds": 5548.128959000001,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "semantic",
+      "phase": "participant-execution",
+      "milliseconds": 28.713165999999546,
+      "status": "completed"
+    },
+    {
+      "repetition": 1,
+      "role": "semantic",
+      "phase": "grading",
+      "milliseconds": 5646.800708000001,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "runtime.candidate",
+      "milliseconds": 430.6864170000008,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "plugin-command",
+      "milliseconds": 73.67158399999971,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "plugin-command",
+      "milliseconds": 65.96320800000103,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "installation-assets",
+      "files": [
+        "cache"
+      ],
+      "configurationContainsPrivateHome": true,
+      "configurationContainsWorkspace": true
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "plugin-install",
+      "milliseconds": 141.00866599999972,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "sandbox-probe",
+      "milliseconds": 5404.434915999998,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "participant-execution",
+      "milliseconds": 26.502958000000945,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "candidate",
+      "phase": "candidate-total",
+      "milliseconds": 6199.1395840000005,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "semantic",
+      "phase": "runtime.semantic",
+      "milliseconds": 0.314249999999447,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "semantic",
+      "phase": "plugin-install",
+      "milliseconds": 0.005916999998589745,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "semantic",
+      "phase": "sandbox-probe",
+      "milliseconds": 5359.928749999999,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "semantic",
+      "phase": "participant-execution",
+      "milliseconds": 27.60699999999997,
+      "status": "completed"
+    },
+    {
+      "repetition": 2,
+      "role": "semantic",
+      "phase": "grading",
+      "milliseconds": 5454.279749999998,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "runtime.candidate",
+      "milliseconds": 442.62854199999856,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "plugin-command",
+      "milliseconds": 72.3325420000001,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "plugin-command",
+      "milliseconds": 68.99258300000292,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "installation-assets",
+      "files": [
+        "cache"
+      ],
+      "configurationContainsPrivateHome": true,
+      "configurationContainsWorkspace": true
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "plugin-install",
+      "milliseconds": 142.90399999999863,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "sandbox-probe",
+      "milliseconds": 5447.905040999998,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "participant-execution",
+      "milliseconds": 23.8626669999976,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "candidate",
+      "phase": "candidate-total",
+      "milliseconds": 6256.5165830000005,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "semantic",
+      "phase": "runtime.semantic",
+      "milliseconds": 0.3507499999977881,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "semantic",
+      "phase": "plugin-install",
+      "milliseconds": 0.01341600000159815,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "semantic",
+      "phase": "sandbox-probe",
+      "milliseconds": 5365.139791000001,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "semantic",
+      "phase": "participant-execution",
+      "milliseconds": 28.228125000001455,
+      "status": "completed"
+    },
+    {
+      "repetition": 3,
+      "role": "semantic",
+      "phase": "grading",
+      "milliseconds": 5460.212874999997,
+      "status": "completed"
+    },
+    {
+      "repetition": 4,
+      "role": "semantic",
+      "phase": "cow-copy",
+      "milliseconds": 1.0158329999976559,
+      "status": "completed"
+    },
+    {
+      "phase": "cow-isolation",
+      "sourceUnchanged": true
+    }
+  ]
+}

package/docs/evidence/runtime-startup-30/baseline-timeout.jsonl

@@ -0,0 +1,2 @@
+{"phase":"sandbox-probe","durationMs":10004.7745,"error":"Codex run timed out"}
+{"phase":"host-failure","error":"Codex run timed out"}