@bjoernrochel/sevro 0.1.0-rc.8 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CONTRIBUTING.md +52 -6
- package/README.md +12 -7
- package/dist/examples/basic/graded.json +13 -0
- package/dist/examples/basic/host.js +9 -0
- package/dist/examples/basic/host.js.map +1 -0
- package/dist/examples/basic/prompt-only.json +7 -0
- package/dist/examples/recipes/candidate.js +23 -0
- package/dist/examples/recipes/candidate.js.map +1 -0
- package/dist/examples/recipes/cases/coding-change.json +21 -0
- package/dist/examples/recipes/cases/host-tools.json +21 -0
- package/dist/examples/recipes/cases/mise.json +21 -0
- package/dist/examples/recipes/cases/node.json +21 -0
- package/dist/examples/recipes/cases/python-uv.json +21 -0
- package/dist/examples/recipes/cases/semantic.json +17 -0
- package/dist/examples/recipes/coding-change/check.js +16 -0
- package/dist/examples/recipes/coding-change/check.js.map +1 -0
- package/dist/examples/recipes/executable.js +20 -0
- package/dist/examples/recipes/executable.js.map +1 -0
- package/dist/examples/recipes/extension-artifacts.js +56 -0
- package/dist/examples/recipes/extension-artifacts.js.map +1 -0
- package/dist/examples/recipes/extension-methods.js +167 -0
- package/dist/examples/recipes/extension-methods.js.map +1 -0
- package/dist/examples/recipes/extension.js +28 -0
- package/dist/examples/recipes/extension.js.map +1 -0
- package/dist/examples/recipes/modules.js +9 -0
- package/dist/examples/recipes/modules.js.map +1 -0
- package/dist/examples/recipes/repository-skill/.agents/skills/startup-guide/SKILL.md +7 -0
- package/dist/examples/recipes/repository-skill/.claude/skills/startup-guide/SKILL.md +7 -0
- package/dist/examples/recipes/repository-skill/README.md +3 -0
- package/dist/examples/recipes/run-inputs.js +144 -0
- package/dist/examples/recipes/run-inputs.js.map +1 -0
- package/dist/examples/recipes/run-options.js +103 -0
- package/dist/examples/recipes/run-options.js.map +1 -0
- package/dist/examples/recipes/run.js +62 -0
- package/dist/examples/recipes/run.js.map +1 -0
- package/dist/examples/recipes/runtime/empty.json +1 -0
- package/dist/examples/recipes/runtime/goals.json +1 -0
- package/dist/examples/recipes/runtime/mise.json +6 -0
- package/dist/examples/recipes/runtime/path.json +1 -0
- package/dist/examples/recipes/runtime/plugins.json +4 -0
- package/dist/examples/recipes/runtime/python-uv.json +10 -0
- package/dist/examples/recipes/semantic-judge.js +27 -0
- package/dist/examples/recipes/semantic-judge.js.map +1 -0
- package/dist/examples/templates/coding/adapter.js +14 -0
- package/dist/examples/templates/coding/adapter.js.map +1 -0
- package/dist/examples/templates/coding/case.json +15 -0
- package/dist/examples/templates/repository-skill/.agents/skills/startup-guide/SKILL.md +6 -0
- package/dist/examples/templates/repository-skill/extension-data.js +61 -0
- package/dist/examples/templates/repository-skill/extension-data.js.map +1 -0
- package/dist/examples/templates/repository-skill/extension.js +29 -0
- package/dist/examples/templates/repository-skill/extension.js.map +1 -0
- package/dist/examples/templates/response/adapter.js +13 -0
- package/dist/examples/templates/response/adapter.js.map +1 -0
- package/dist/examples/templates/response/case.json +13 -0
- package/dist/src/advisory.js +86 -0
- package/dist/src/advisory.js.map +1 -0
- package/dist/src/candidate-artifacts.js +139 -0
- package/dist/src/candidate-artifacts.js.map +1 -0
- package/dist/src/candidate-codex-state.js +110 -0
- package/dist/src/candidate-codex-state.js.map +1 -0
- package/dist/src/case-validation.js +92 -0
- package/dist/src/case-validation.js.map +1 -0
- package/dist/src/cli/cancellation.js +21 -0
- package/dist/src/cli/cancellation.js.map +1 -0
- package/dist/src/cli/command.js +60 -0
- package/dist/src/cli/command.js.map +1 -0
- package/dist/src/cli/completion.js +14 -0
- package/dist/src/cli/completion.js.map +1 -0
- package/dist/src/cli/demo.js +45 -0
- package/dist/src/cli/demo.js.map +1 -0
- package/dist/src/cli/evaluation.js +40 -0
- package/dist/src/cli/evaluation.js.map +1 -0
- package/dist/src/cli/extension.js +54 -0
- package/dist/src/cli/extension.js.map +1 -0
- package/dist/src/cli/help.js +190 -0
- package/dist/src/cli/help.js.map +1 -0
- package/dist/src/cli/hosts.js +33 -0
- package/dist/src/cli/hosts.js.map +1 -0
- package/dist/src/cli/init-settings.js +29 -0
- package/dist/src/cli/init-settings.js.map +1 -0
- package/dist/src/cli/init.js +82 -0
- package/dist/src/cli/init.js.map +1 -0
- package/dist/src/cli/invocation.js +431 -0
- package/dist/src/cli/invocation.js.map +1 -0
- package/dist/src/cli/loading.js +110 -0
- package/dist/src/cli/loading.js.map +1 -0
- package/dist/src/cli/preparation.js +88 -0
- package/dist/src/cli/preparation.js.map +1 -0
- package/dist/src/cli/presentation.js +61 -0
- package/dist/src/cli/presentation.js.map +1 -0
- package/dist/src/cli/progress.js +5 -0
- package/dist/src/cli/progress.js.map +1 -0
- package/dist/src/cli/run-manifest.js +108 -0
- package/dist/src/cli/run-manifest.js.map +1 -0
- package/dist/src/cli/run-options.js +367 -0
- package/dist/src/cli/run-options.js.map +1 -0
- package/dist/src/cli/run-overrides.js +51 -0
- package/dist/src/cli/run-overrides.js.map +1 -0
- package/dist/src/cli/settings.js +127 -0
- package/dist/src/cli/settings.js.map +1 -0
- package/dist/src/cli/summary.js +28 -0
- package/dist/src/cli/summary.js.map +1 -0
- package/dist/src/cli.js +20 -0
- package/dist/src/cli.js.map +1 -0
- package/dist/src/contract-types.js +2 -0
- package/dist/src/contract-types.js.map +1 -0
- package/dist/src/engine.js +4 -0
- package/dist/src/engine.js.map +1 -0
- package/dist/src/evaluation/advisory.js +85 -0
- package/dist/src/evaluation/advisory.js.map +1 -0
- package/dist/src/evaluation/check-plan.js +292 -0
- package/dist/src/evaluation/check-plan.js.map +1 -0
- package/dist/src/evaluation/collection.js +61 -0
- package/dist/src/evaluation/collection.js.map +1 -0
- package/dist/src/evaluation/context.js +2 -0
- package/dist/src/evaluation/context.js.map +1 -0
- package/dist/src/evaluation/grading-host.js +28 -0
- package/dist/src/evaluation/grading-host.js.map +1 -0
- package/dist/src/evaluation/grading.js +154 -0
- package/dist/src/evaluation/grading.js.map +1 -0
- package/dist/src/evaluation/invocation-plan.js +312 -0
- package/dist/src/evaluation/invocation-plan.js.map +1 -0
- package/dist/src/evaluation/plan.js +376 -0
- package/dist/src/evaluation/plan.js.map +1 -0
- package/dist/src/evaluation/progress.js +14 -0
- package/dist/src/evaluation/progress.js.map +1 -0
- package/dist/src/evaluation/resources.js +159 -0
- package/dist/src/evaluation/resources.js.map +1 -0
- package/dist/src/evaluation/run.js +148 -0
- package/dist/src/evaluation/run.js.map +1 -0
- package/dist/src/evaluation/scheduler.js +27 -0
- package/dist/src/evaluation/scheduler.js.map +1 -0
- package/dist/src/evaluation/semantic.js +183 -0
- package/dist/src/evaluation/semantic.js.map +1 -0
- package/dist/src/evaluation/trial.js +593 -0
- package/dist/src/evaluation/trial.js.map +1 -0
- package/dist/src/evaluation/types.js +20 -0
- package/dist/src/evaluation/types.js.map +1 -0
- package/dist/src/evidence/host-results.js +67 -0
- package/dist/src/evidence/host-results.js.map +1 -0
- package/dist/src/evidence/retained-artifacts.js +17 -0
- package/dist/src/evidence/retained-artifacts.js.map +1 -0
- package/dist/src/evidence/trial-writer.js +131 -0
- package/dist/src/evidence/trial-writer.js.map +1 -0
- package/dist/src/evidence/usage.js +14 -0
- package/dist/src/evidence/usage.js.map +1 -0
- package/dist/src/executable.js +20 -0
- package/dist/src/executable.js.map +1 -0
- package/dist/src/extension-client.js +212 -0
- package/dist/src/extension-client.js.map +1 -0
- package/dist/src/extension-session.js +236 -0
- package/dist/src/extension-session.js.map +1 -0
- package/dist/src/fixtures/advisory.js +206 -0
- package/dist/src/fixtures/advisory.js.map +1 -0
- package/dist/src/fixtures/bin.js +41 -0
- package/dist/src/fixtures/bin.js.map +1 -0
- package/dist/src/fixtures/cleanup.js +43 -0
- package/dist/src/fixtures/cleanup.js.map +1 -0
- package/dist/src/fixtures/create.js +94 -0
- package/dist/src/fixtures/create.js.map +1 -0
- package/dist/src/fixtures/generated.js +181 -0
- package/dist/src/fixtures/generated.js.map +1 -0
- package/dist/src/fixtures/git-hooks.js +16 -0
- package/dist/src/fixtures/git-hooks.js.map +1 -0
- package/dist/src/fixtures/preparation.js +201 -0
- package/dist/src/fixtures/preparation.js.map +1 -0
- package/dist/src/fixtures/repository.js +255 -0
- package/dist/src/fixtures/repository.js.map +1 -0
- package/dist/src/fixtures/setup.js +175 -0
- package/dist/src/fixtures/setup.js.map +1 -0
- package/dist/src/fixtures/tools.js +57 -0
- package/dist/src/fixtures/tools.js.map +1 -0
- package/dist/src/fixtures/workspace-state.js +155 -0
- package/dist/src/fixtures/workspace-state.js.map +1 -0
- package/dist/src/generated/cli-result.cjs +2 -0
- package/dist/src/generated/extension.cjs +2 -0
- package/dist/src/generated/report.cjs +2 -0
- package/dist/src/generated/run-evidence.cjs +2 -0
- package/dist/src/generated/runtime.cjs +2 -0
- package/dist/src/git.js +72 -0
- package/dist/src/git.js.map +1 -0
- package/dist/src/graders/artifact.js +68 -0
- package/dist/src/graders/artifact.js.map +1 -0
- package/dist/src/graders/git-head.js +112 -0
- package/dist/src/graders/git-head.js.map +1 -0
- package/dist/src/graders/output.js +345 -0
- package/dist/src/graders/output.js.map +1 -0
- package/dist/src/graders/semantic.js +132 -0
- package/dist/src/graders/semantic.js.map +1 -0
- package/dist/src/graders/shell.js +321 -0
- package/dist/src/graders/shell.js.map +1 -0
- package/dist/src/host-diagnostic-redaction.js +52 -0
- package/dist/src/host-diagnostic-redaction.js.map +1 -0
- package/dist/src/host-diagnostics.js +248 -0
- package/dist/src/host-diagnostics.js.map +1 -0
- package/dist/src/hosts/claude-continuation.js +89 -0
- package/dist/src/hosts/claude-continuation.js.map +1 -0
- package/dist/src/hosts/claude-credential.js +121 -0
- package/dist/src/hosts/claude-credential.js.map +1 -0
- package/dist/src/hosts/claude-events.js +136 -0
- package/dist/src/hosts/claude-events.js.map +1 -0
- package/dist/src/hosts/claude-keychain.js +100 -0
- package/dist/src/hosts/claude-keychain.js.map +1 -0
- package/dist/src/hosts/claude-login-recovery.js +67 -0
- package/dist/src/hosts/claude-login-recovery.js.map +1 -0
- package/dist/src/hosts/claude-login-refresh.js +146 -0
- package/dist/src/hosts/claude-login-refresh.js.map +1 -0
- package/dist/src/hosts/claude-login-store.js +124 -0
- package/dist/src/hosts/claude-login-store.js.map +1 -0
- package/dist/src/hosts/claude-login.js +74 -0
- package/dist/src/hosts/claude-login.js.map +1 -0
- package/dist/src/hosts/claude-native-goal.js +138 -0
- package/dist/src/hosts/claude-native-goal.js.map +1 -0
- package/dist/src/hosts/claude-process.js +10 -0
- package/dist/src/hosts/claude-process.js.map +1 -0
- package/dist/src/hosts/claude-refresh-lock.js +76 -0
- package/dist/src/hosts/claude-refresh-lock.js.map +1 -0
- package/dist/src/hosts/claude-repository-invocation.js +229 -0
- package/dist/src/hosts/claude-repository-invocation.js.map +1 -0
- package/dist/src/hosts/claude-settings.js +156 -0
- package/dist/src/hosts/claude-settings.js.map +1 -0
- package/dist/src/hosts/claude-skill-activations.js +131 -0
- package/dist/src/hosts/claude-skill-activations.js.map +1 -0
- package/dist/src/hosts/claude-state.js +237 -0
- package/dist/src/hosts/claude-state.js.map +1 -0
- package/dist/src/hosts/claude-tool-calls.js +150 -0
- package/dist/src/hosts/claude-tool-calls.js.map +1 -0
- package/dist/src/hosts/claude.js +373 -0
- package/dist/src/hosts/claude.js.map +1 -0
- package/dist/src/hosts/codex-app-server.js +764 -0
- package/dist/src/hosts/codex-app-server.js.map +1 -0
- package/dist/src/hosts/codex-configuration.js +91 -0
- package/dist/src/hosts/codex-configuration.js.map +1 -0
- package/dist/src/hosts/codex-events.js +104 -0
- package/dist/src/hosts/codex-events.js.map +1 -0
- package/dist/src/hosts/codex-exec-controls.js +434 -0
- package/dist/src/hosts/codex-exec-controls.js.map +1 -0
- package/dist/src/hosts/codex-invocation.js +54 -0
- package/dist/src/hosts/codex-invocation.js.map +1 -0
- package/dist/src/hosts/codex-marketplace.js +185 -0
- package/dist/src/hosts/codex-marketplace.js.map +1 -0
- package/dist/src/hosts/codex-native-calls.js +516 -0
- package/dist/src/hosts/codex-native-calls.js.map +1 -0
- package/dist/src/hosts/codex-process.js +10 -0
- package/dist/src/hosts/codex-process.js.map +1 -0
- package/dist/src/hosts/codex-profile.js +206 -0
- package/dist/src/hosts/codex-profile.js.map +1 -0
- package/dist/src/hosts/codex-session-files.js +89 -0
- package/dist/src/hosts/codex-session-files.js.map +1 -0
- package/dist/src/hosts/codex-state.js +310 -0
- package/dist/src/hosts/codex-state.js.map +1 -0
- package/dist/src/hosts/codex.js +527 -0
- package/dist/src/hosts/codex.js.map +1 -0
- package/dist/src/hosts/isolation-common.js +43 -0
- package/dist/src/hosts/isolation-common.js.map +1 -0
- package/dist/src/hosts/isolation-roots.js +64 -0
- package/dist/src/hosts/isolation-roots.js.map +1 -0
- package/dist/src/hosts/isolation.js +12 -0
- package/dist/src/hosts/isolation.js.map +1 -0
- package/dist/src/hosts/linux-sandbox.js +127 -0
- package/dist/src/hosts/linux-sandbox.js.map +1 -0
- package/dist/src/hosts/mac-sandbox.js +137 -0
- package/dist/src/hosts/mac-sandbox.js.map +1 -0
- package/dist/src/hosts/native-controls.js +151 -0
- package/dist/src/hosts/native-controls.js.map +1 -0
- package/dist/src/hosts/native-request.js +33 -0
- package/dist/src/hosts/native-request.js.map +1 -0
- package/dist/src/hosts/process-lifecycle.js +167 -0
- package/dist/src/hosts/process-lifecycle.js.map +1 -0
- package/dist/src/hosts/protected-worktrees.js +81 -0
- package/dist/src/hosts/protected-worktrees.js.map +1 -0
- package/dist/src/hosts/skill-hook-catalog.js +100 -0
- package/dist/src/hosts/skill-hook-catalog.js.map +1 -0
- package/dist/src/hosts/skill-hook-handler.js +228 -0
- package/dist/src/hosts/skill-hook-handler.js.map +1 -0
- package/dist/src/hosts/skill-hooks.js +423 -0
- package/dist/src/hosts/skill-hooks.js.map +1 -0
- package/dist/src/hosts/workspace-fingerprint.js +72 -0
- package/dist/src/hosts/workspace-fingerprint.js.map +1 -0
- package/dist/src/identity.js +167 -0
- package/dist/src/identity.js.map +1 -0
- package/dist/src/instrumentation.js +83 -0
- package/dist/src/instrumentation.js.map +1 -0
- package/dist/src/native-transcript-state.js +85 -0
- package/dist/src/native-transcript-state.js.map +1 -0
- package/dist/src/native-transcript-view.js +65 -0
- package/dist/src/native-transcript-view.js.map +1 -0
- package/dist/src/native-transcripts.js +208 -0
- package/dist/src/native-transcripts.js.map +1 -0
- package/dist/src/paths.js +13 -0
- package/dist/src/paths.js.map +1 -0
- package/dist/src/process-lifecycle.js +101 -0
- package/dist/src/process-lifecycle.js.map +1 -0
- package/dist/src/provenance.js +263 -0
- package/dist/src/provenance.js.map +1 -0
- package/dist/src/prune.js +228 -0
- package/dist/src/prune.js.map +1 -0
- package/dist/src/report.js +276 -0
- package/dist/src/report.js.map +1 -0
- package/dist/src/resolved-case.js +29 -0
- package/dist/src/resolved-case.js.map +1 -0
- package/dist/src/results-lock.js +40 -0
- package/dist/src/results-lock.js.map +1 -0
- package/dist/src/results.js +127 -0
- package/dist/src/results.js.map +1 -0
- package/dist/src/run-owner.js +283 -0
- package/dist/src/run-owner.js.map +1 -0
- package/dist/src/runner-paths.js +11 -0
- package/dist/src/runner-paths.js.map +1 -0
- package/dist/src/runtime/config.js +323 -0
- package/dist/src/runtime/config.js.map +1 -0
- package/dist/src/runtime/discovery-apple.js +65 -0
- package/dist/src/runtime/discovery-apple.js.map +1 -0
- package/dist/src/runtime/discovery-providers.js +37 -0
- package/dist/src/runtime/discovery-providers.js.map +1 -0
- package/dist/src/runtime/discovery.js +40 -0
- package/dist/src/runtime/discovery.js.map +1 -0
- package/dist/src/runtime/hook-processes.js +81 -0
- package/dist/src/runtime/hook-processes.js.map +1 -0
- package/dist/src/runtime/hooks.js +304 -0
- package/dist/src/runtime/hooks.js.map +1 -0
- package/dist/src/runtime/paths.js +244 -0
- package/dist/src/runtime/paths.js.map +1 -0
- package/dist/src/runtime/query.js +76 -0
- package/dist/src/runtime/query.js.map +1 -0
- package/dist/src/runtime/seeds.js +98 -0
- package/dist/src/runtime/seeds.js.map +1 -0
- package/dist/src/runtime/shared-cache.js +81 -0
- package/dist/src/runtime/shared-cache.js.map +1 -0
- package/dist/src/runtime/state.js +106 -0
- package/dist/src/runtime/state.js.map +1 -0
- package/dist/src/runtime-process.js +148 -0
- package/dist/src/runtime-process.js.map +1 -0
- package/dist/src/schema-types.js +2 -0
- package/dist/src/schema-types.js.map +1 -0
- package/dist/src/schema.js +16 -0
- package/dist/src/schema.js.map +1 -0
- package/dist/src/shell.js +5 -0
- package/dist/src/shell.js.map +1 -0
- package/dist/src/sqlite-lock.js +18 -0
- package/dist/src/sqlite-lock.js.map +1 -0
- package/dist/src/storage.js +21 -0
- package/dist/src/storage.js.map +1 -0
- package/dist/src/transcript-retention.js +109 -0
- package/dist/src/transcript-retention.js.map +1 -0
- package/dist/src/value-guards.js +26 -0
- package/dist/src/value-guards.js.map +1 -0
- package/docs/README.md +19 -15
- package/docs/architecture.md +141 -17
- package/docs/assets/README.md +5 -4
- package/docs/builtin-graders-v1.md +9 -2
- package/docs/codex-host.md +155 -57
- package/docs/contracts.md +2 -1
- package/docs/creating-cases.md +10 -4
- package/docs/development-cli.md +197 -24
- package/docs/documentation-quality.md +16 -5
- package/docs/evaluation-workflow.md +110 -0
- package/docs/extension-protocol-v1.md +26 -3
- package/docs/getting-started.md +29 -7
- package/docs/guide-evaluation.md +34 -2
- package/docs/identity-v1.md +29 -3
- package/docs/installing.md +35 -17
- package/docs/licensing.md +2 -2
- package/docs/native-controls.md +76 -5
- package/docs/native-hosts.md +197 -8
- package/docs/reading-results.md +102 -0
- package/docs/recipes.md +490 -0
- package/docs/releases.md +48 -13
- package/docs/report-v1.md +29 -1
- package/docs/results-v1.md +122 -4
- package/docs/run-settings-v1.md +147 -0
- package/docs/runtime-v1.md +255 -22
- package/docs/specs/repository-guide.md +1 -1
- package/docs/troubleshooting.md +12 -12
- package/docs/typescript-quality.md +106 -26
- package/examples/recipes/candidate.ts +26 -0
- package/examples/recipes/cases/coding-change.json +21 -0
- package/examples/recipes/cases/host-tools.json +21 -0
- package/examples/recipes/cases/mise.json +21 -0
- package/examples/recipes/cases/node.json +21 -0
- package/examples/recipes/cases/python-uv.json +21 -0
- package/examples/recipes/cases/semantic.json +17 -0
- package/examples/recipes/coding-change/check.ts +23 -0
- package/examples/recipes/executable.ts +19 -0
- package/examples/recipes/extension-artifacts.ts +65 -0
- package/examples/recipes/extension-methods.ts +174 -0
- package/examples/recipes/extension.ts +38 -0
- package/examples/recipes/modules.ts +9 -0
- package/examples/recipes/repository-skill/.agents/skills/startup-guide/SKILL.md +7 -0
- package/examples/recipes/repository-skill/.claude/skills/startup-guide/SKILL.md +7 -0
- package/examples/recipes/repository-skill/README.md +3 -0
- package/examples/recipes/run-inputs.ts +183 -0
- package/examples/recipes/run-options.ts +132 -0
- package/examples/recipes/run.ts +77 -0
- package/examples/recipes/runtime/empty.json +1 -0
- package/examples/recipes/runtime/goals.json +1 -0
- package/examples/recipes/runtime/mise.json +6 -0
- package/examples/recipes/runtime/path.json +1 -0
- package/examples/recipes/runtime/plugins.json +4 -0
- package/examples/recipes/runtime/python-uv.json +10 -0
- package/examples/recipes/semantic-judge.ts +29 -0
- package/examples/templates/coding/adapter.ts +17 -0
- package/examples/templates/coding/case.json +15 -0
- package/examples/templates/repository-skill/.agents/skills/startup-guide/SKILL.md +6 -0
- package/examples/templates/repository-skill/extension-data.ts +64 -0
- package/examples/templates/repository-skill/extension.ts +32 -0
- package/examples/templates/response/adapter.ts +12 -0
- package/examples/templates/response/case.json +13 -0
- package/package.json +23 -9
- package/schemas/report-v1.schema.json +9 -0
- package/schemas/run-evidence-v1.schema.json +2 -1
- package/schemas/runtime-v1.schema.json +43 -7
- package/src/advisory.ts +2 -2
- package/src/candidate-artifacts.ts +42 -11
- package/src/candidate-codex-state.ts +185 -0
- package/src/case-validation.ts +73 -50
- package/src/cli/cancellation.ts +20 -0
- package/src/cli/command.ts +87 -0
- package/src/cli/completion.ts +15 -0
- package/src/cli/demo.ts +61 -0
- package/src/cli/evaluation.ts +55 -0
- package/src/cli/extension.ts +79 -0
- package/src/cli/help.ts +199 -0
- package/src/cli/hosts.ts +55 -0
- package/src/cli/init-settings.ts +31 -0
- package/src/cli/init.ts +112 -0
- package/src/{cli-invocation.ts → cli/invocation.ts} +215 -218
- package/src/cli/loading.ts +151 -0
- package/src/cli/preparation.ts +154 -0
- package/src/cli/presentation.ts +93 -0
- package/src/cli/progress.ts +6 -0
- package/src/cli/run-manifest.ts +128 -0
- package/src/cli/run-options.ts +403 -0
- package/src/cli/run-overrides.ts +69 -0
- package/src/cli/settings.ts +154 -0
- package/src/cli/summary.ts +43 -0
- package/src/cli.ts +14 -592
- package/src/contract-types.ts +94 -0
- package/src/engine.ts +9 -5
- package/src/evaluation/advisory.ts +150 -0
- package/src/{evaluation-check-plan.ts → evaluation/check-plan.ts} +103 -67
- package/src/evaluation/collection.ts +88 -0
- package/src/evaluation/context.ts +10 -0
- package/src/{evaluation-grading-host.ts → evaluation/grading-host.ts} +5 -5
- package/src/evaluation/grading.ts +235 -0
- package/src/{evaluation-invocation-plan.ts → evaluation/invocation-plan.ts} +147 -58
- package/src/{evaluation-plan.ts → evaluation/plan.ts} +208 -141
- package/src/evaluation/progress.ts +19 -0
- package/src/evaluation/resources.ts +241 -0
- package/src/evaluation/run.ts +206 -0
- package/src/{evaluation-semantic.ts → evaluation/semantic.ts} +77 -87
- package/src/evaluation/trial.ts +800 -0
- package/src/{evaluation-types.ts → evaluation/types.ts} +79 -29
- package/src/evidence/host-results.ts +85 -0
- package/src/evidence/retained-artifacts.ts +18 -0
- package/src/evidence/trial-writer.ts +203 -0
- package/src/evidence/usage.ts +17 -0
- package/src/executable.ts +19 -0
- package/src/extension-client.ts +40 -108
- package/src/extension-session.ts +24 -48
- package/src/{advisory-fixture.ts → fixtures/advisory.ts} +12 -72
- package/src/{fixture-bin.ts → fixtures/bin.ts} +1 -1
- package/src/{fixture-cleanup.ts → fixtures/cleanup.ts} +13 -7
- package/src/fixtures/create.ts +183 -0
- package/src/{generated-fixture.ts → fixtures/generated.ts} +10 -53
- package/src/{git-hooks.ts → fixtures/git-hooks.ts} +1 -1
- package/src/{preparation.ts → fixtures/preparation.ts} +6 -16
- package/src/{repository-fixture.ts → fixtures/repository.ts} +30 -64
- package/src/{fixture-setup.ts → fixtures/setup.ts} +36 -53
- package/src/{fixture-tools.ts → fixtures/tools.ts} +1 -1
- package/src/{workspace-state.ts → fixtures/workspace-state.ts} +5 -5
- package/src/generated/cli-result.d.cts +2 -1
- package/src/generated/extension.d.cts +2 -1
- package/src/generated/report.cjs +1 -1
- package/src/generated/report.d.cts +2 -1
- package/src/generated/run-evidence.cjs +1 -1
- package/src/generated/run-evidence.d.cts +2 -1
- package/src/generated/runtime.cjs +1 -1
- package/src/generated/runtime.d.cts +2 -1
- package/src/git.ts +94 -0
- package/src/graders/git-head.ts +31 -85
- package/src/graders/shell.ts +54 -71
- package/src/host-diagnostic-redaction.ts +52 -0
- package/src/host-diagnostics.ts +331 -0
- package/src/hosts/claude-continuation.ts +57 -40
- package/src/hosts/claude-credential.ts +160 -63
- package/src/hosts/claude-events.ts +34 -18
- package/src/hosts/claude-keychain.ts +79 -21
- package/src/hosts/claude-login-recovery.ts +85 -0
- package/src/hosts/claude-login-refresh.ts +179 -0
- package/src/hosts/claude-login-store.ts +162 -0
- package/src/hosts/claude-login.ts +107 -0
- package/src/hosts/claude-native-goal.ts +15 -12
- package/src/hosts/claude-process.ts +10 -50
- package/src/hosts/claude-refresh-lock.ts +82 -0
- package/src/hosts/claude-repository-invocation.ts +9 -34
- package/src/hosts/claude-settings.ts +46 -9
- package/src/hosts/claude-skill-activations.ts +169 -0
- package/src/hosts/claude-state.ts +28 -15
- package/src/hosts/claude-tool-calls.ts +13 -10
- package/src/hosts/claude.ts +121 -81
- package/src/hosts/codex-app-server.ts +78 -52
- package/src/hosts/codex-configuration.ts +5 -10
- package/src/hosts/codex-exec-controls.ts +516 -0
- package/src/hosts/codex-marketplace.ts +12 -10
- package/src/hosts/codex-native-calls.ts +15 -122
- package/src/hosts/codex-process.ts +10 -183
- package/src/hosts/codex-profile.ts +36 -21
- package/src/hosts/codex-session-files.ts +1 -9
- package/src/hosts/codex-state.ts +134 -18
- package/src/hosts/codex.ts +210 -183
- package/src/hosts/isolation-common.ts +8 -11
- package/src/hosts/isolation-roots.ts +5 -4
- package/src/hosts/linux-sandbox.ts +15 -2
- package/src/hosts/mac-sandbox.ts +87 -18
- package/src/hosts/native-controls.ts +72 -22
- package/src/hosts/native-request.ts +55 -0
- package/src/hosts/process-lifecycle.ts +215 -49
- package/src/hosts/protected-worktrees.ts +3 -2
- package/src/hosts/skill-hook-catalog.ts +131 -0
- package/src/hosts/skill-hook-handler.ts +374 -0
- package/src/hosts/skill-hooks.ts +585 -0
- package/src/instrumentation.ts +7 -8
- package/src/native-transcript-state.ts +22 -3
- package/src/native-transcript-view.ts +52 -13
- package/src/native-transcripts.ts +26 -11
- package/src/paths.ts +22 -0
- package/src/process-lifecycle.ts +150 -0
- package/src/provenance.ts +34 -13
- package/src/prune.ts +300 -0
- package/src/report.ts +63 -36
- package/src/resolved-case.ts +1 -1
- package/src/results-lock.ts +42 -0
- package/src/results.ts +14 -18
- package/src/run-owner.ts +15 -9
- package/src/runner-paths.ts +21 -0
- package/src/{runtime-config.ts → runtime/config.ts} +160 -46
- package/src/{runtime-discovery-apple.ts → runtime/discovery-apple.ts} +3 -3
- package/src/{runtime-discovery-providers.ts → runtime/discovery-providers.ts} +3 -9
- package/src/{runtime-discovery.ts → runtime/discovery.ts} +4 -4
- package/src/{runtime-hook-processes.ts → runtime/hook-processes.ts} +3 -2
- package/src/{runtime-hooks.ts → runtime/hooks.ts} +40 -28
- package/src/{runtime-paths.ts → runtime/paths.ts} +5 -19
- package/src/{runtime-query.ts → runtime/query.ts} +5 -5
- package/src/{runtime-seeds.ts → runtime/seeds.ts} +3 -3
- package/src/runtime/shared-cache.ts +130 -0
- package/src/{runtime-state.ts → runtime/state.ts} +28 -11
- package/src/runtime-process.ts +194 -0
- package/src/schema-types.ts +73 -161
- package/src/shell.ts +4 -0
- package/src/sqlite-lock.ts +25 -0
- package/src/transcript-retention.ts +146 -0
- package/src/value-guards.ts +22 -0
- package/docs/assets/sevro-logo.png +0 -0
- package/docs/assets/sevro-symbol.svg +0 -1
- package/docs/assets/sevro-wordmark.svg +0 -1
- package/docs/cache-ownership-validation.md +0 -49
- package/docs/claude-continuation-validation.md +0 -47
- package/docs/concurrency-validation.md +0 -86
- package/docs/configuration-root-validation.md +0 -70
- package/docs/engine-slice.md +0 -114
- package/docs/evidence/runtime-startup-30/README.md +0 -185
- package/docs/evidence/runtime-startup-30/after.json +0 -319
- package/docs/evidence/runtime-startup-30/baseline-timeout.jsonl +0 -2
- package/docs/evidence/runtime-startup-30/before.json +0 -403
- package/docs/evidence/runtime-startup-30/consumer-smoke.json +0 -63
- package/docs/evidence/runtime-startup-30/full-policy-diagnostic.jsonl +0 -3
- package/docs/evidence/runtime-startup-30/minimal-policy-diagnostic.jsonl +0 -3
- package/docs/evidence/runtime-startup-30/plugin-template-assessment.json +0 -20
- package/docs/evidence/runtime-startup-30/seedless-timeout.jsonl +0 -2
- package/docs/evidence.md +0 -17
- package/docs/fixture-cleanup-validation.md +0 -73
- package/docs/guide-verification.json +0 -239
- package/src/evaluation-context.ts +0 -259
- package/src/evaluation-fixture.ts +0 -302
- package/src/evaluation-run.ts +0 -200
- package/src/evaluation-trial.ts +0 -1016
- package/src/hosts/claude-nested-skills.ts +0 -272
- package/src/hosts/codex-command-output.ts +0 -510
- package/src/hosts/codex-executor-command.ts +0 -291
- package/src/hosts/codex-skill-reads.ts +0 -783
- /package/src/{trial-scheduler.ts → evaluation/scheduler.ts} +0 -0
|
@@ -1,403 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"format": "sevro.runtime-startup-benchmark.v1",
|
|
3
|
-
"sourceRoot": "/Users/bjro/Sources/sevro",
|
|
4
|
-
"platform": "darwin",
|
|
5
|
-
"arch": "arm64",
|
|
6
|
-
"bun": "1.3.13",
|
|
7
|
-
"codex": "codex-cli 0.162.1",
|
|
8
|
-
"case": "synthetic-plugin-response",
|
|
9
|
-
"model": "synthetic-candidate / synthetic-semantic",
|
|
10
|
-
"effort": "low",
|
|
11
|
-
"seed": {
|
|
12
|
-
"entries": 2049,
|
|
13
|
-
"bytes": 33554432
|
|
14
|
-
},
|
|
15
|
-
"repetitions": 3,
|
|
16
|
-
"conditions": "Fresh role homes each invocation; first verified snapshot cold, later repetitions warm; OS filesystem caches uncontrolled; no live API/model calls",
|
|
17
|
-
"rows": [
|
|
18
|
-
{
|
|
19
|
-
"repetition": 0,
|
|
20
|
-
"role": "candidate",
|
|
21
|
-
"phase": "configuration",
|
|
22
|
-
"milliseconds": 261.92554200000006,
|
|
23
|
-
"status": "completed"
|
|
24
|
-
},
|
|
25
|
-
{
|
|
26
|
-
"repetition": 1,
|
|
27
|
-
"role": "candidate",
|
|
28
|
-
"phase": "runtime.candidate",
|
|
29
|
-
"milliseconds": 1969.3474580000002,
|
|
30
|
-
"status": "completed"
|
|
31
|
-
},
|
|
32
|
-
{
|
|
33
|
-
"repetition": 1,
|
|
34
|
-
"role": "candidate",
|
|
35
|
-
"phase": "plugin-command",
|
|
36
|
-
"milliseconds": 300.13179200000013,
|
|
37
|
-
"status": "completed"
|
|
38
|
-
},
|
|
39
|
-
{
|
|
40
|
-
"repetition": 1,
|
|
41
|
-
"role": "candidate",
|
|
42
|
-
"phase": "plugin-command",
|
|
43
|
-
"milliseconds": 81.15104199999996,
|
|
44
|
-
"status": "completed"
|
|
45
|
-
},
|
|
46
|
-
{
|
|
47
|
-
"repetition": 1,
|
|
48
|
-
"role": "candidate",
|
|
49
|
-
"phase": "installation-assets",
|
|
50
|
-
"files": [
|
|
51
|
-
"cache"
|
|
52
|
-
],
|
|
53
|
-
"configurationContainsPrivateHome": true,
|
|
54
|
-
"configurationContainsWorkspace": true
|
|
55
|
-
},
|
|
56
|
-
{
|
|
57
|
-
"repetition": 1,
|
|
58
|
-
"role": "candidate",
|
|
59
|
-
"phase": "plugin-install",
|
|
60
|
-
"milliseconds": 385.09974999999986,
|
|
61
|
-
"status": "completed"
|
|
62
|
-
},
|
|
63
|
-
{
|
|
64
|
-
"repetition": 1,
|
|
65
|
-
"role": "candidate",
|
|
66
|
-
"phase": "runtime.hooks",
|
|
67
|
-
"milliseconds": 1225.7079159999994,
|
|
68
|
-
"status": "completed"
|
|
69
|
-
},
|
|
70
|
-
{
|
|
71
|
-
"repetition": 1,
|
|
72
|
-
"role": "candidate",
|
|
73
|
-
"phase": "sandbox-probe",
|
|
74
|
-
"milliseconds": 5528.486166999999,
|
|
75
|
-
"status": "completed"
|
|
76
|
-
},
|
|
77
|
-
{
|
|
78
|
-
"repetition": 1,
|
|
79
|
-
"role": "candidate",
|
|
80
|
-
"phase": "sandbox-probe",
|
|
81
|
-
"milliseconds": 5409.168874999999,
|
|
82
|
-
"status": "completed"
|
|
83
|
-
},
|
|
84
|
-
{
|
|
85
|
-
"repetition": 1,
|
|
86
|
-
"role": "candidate",
|
|
87
|
-
"phase": "participant-execution",
|
|
88
|
-
"milliseconds": 27.37004100000013,
|
|
89
|
-
"status": "completed"
|
|
90
|
-
},
|
|
91
|
-
{
|
|
92
|
-
"repetition": 1,
|
|
93
|
-
"role": "candidate",
|
|
94
|
-
"phase": "candidate-total",
|
|
95
|
-
"milliseconds": 14924.6165,
|
|
96
|
-
"status": "completed"
|
|
97
|
-
},
|
|
98
|
-
{
|
|
99
|
-
"repetition": 1,
|
|
100
|
-
"role": "semantic",
|
|
101
|
-
"phase": "runtime.semantic",
|
|
102
|
-
"milliseconds": 1174.1918750000004,
|
|
103
|
-
"status": "completed"
|
|
104
|
-
},
|
|
105
|
-
{
|
|
106
|
-
"repetition": 1,
|
|
107
|
-
"role": "semantic",
|
|
108
|
-
"phase": "plugin-install",
|
|
109
|
-
"milliseconds": 0.014666000002762303,
|
|
110
|
-
"status": "completed"
|
|
111
|
-
},
|
|
112
|
-
{
|
|
113
|
-
"repetition": 1,
|
|
114
|
-
"role": "semantic",
|
|
115
|
-
"phase": "runtime.hooks",
|
|
116
|
-
"milliseconds": 1104.8465419999993,
|
|
117
|
-
"status": "completed"
|
|
118
|
-
},
|
|
119
|
-
{
|
|
120
|
-
"repetition": 1,
|
|
121
|
-
"role": "semantic",
|
|
122
|
-
"phase": "sandbox-probe",
|
|
123
|
-
"milliseconds": 5485.405999999999,
|
|
124
|
-
"status": "completed"
|
|
125
|
-
},
|
|
126
|
-
{
|
|
127
|
-
"repetition": 1,
|
|
128
|
-
"role": "semantic",
|
|
129
|
-
"phase": "sandbox-probe",
|
|
130
|
-
"milliseconds": 5369.393208999998,
|
|
131
|
-
"status": "completed"
|
|
132
|
-
},
|
|
133
|
-
{
|
|
134
|
-
"repetition": 1,
|
|
135
|
-
"role": "semantic",
|
|
136
|
-
"phase": "participant-execution",
|
|
137
|
-
"milliseconds": 27.99250000000029,
|
|
138
|
-
"status": "completed"
|
|
139
|
-
},
|
|
140
|
-
{
|
|
141
|
-
"repetition": 1,
|
|
142
|
-
"role": "semantic",
|
|
143
|
-
"phase": "grading",
|
|
144
|
-
"milliseconds": 13494.667042000001,
|
|
145
|
-
"status": "completed"
|
|
146
|
-
},
|
|
147
|
-
{
|
|
148
|
-
"repetition": 2,
|
|
149
|
-
"role": "candidate",
|
|
150
|
-
"phase": "runtime.candidate",
|
|
151
|
-
"milliseconds": 1357.1145419999993,
|
|
152
|
-
"status": "completed"
|
|
153
|
-
},
|
|
154
|
-
{
|
|
155
|
-
"repetition": 2,
|
|
156
|
-
"role": "candidate",
|
|
157
|
-
"phase": "plugin-command",
|
|
158
|
-
"milliseconds": 72.52966600000218,
|
|
159
|
-
"status": "completed"
|
|
160
|
-
},
|
|
161
|
-
{
|
|
162
|
-
"repetition": 2,
|
|
163
|
-
"role": "candidate",
|
|
164
|
-
"phase": "plugin-command",
|
|
165
|
-
"milliseconds": 67.09337500000038,
|
|
166
|
-
"status": "completed"
|
|
167
|
-
},
|
|
168
|
-
{
|
|
169
|
-
"repetition": 2,
|
|
170
|
-
"role": "candidate",
|
|
171
|
-
"phase": "installation-assets",
|
|
172
|
-
"files": [
|
|
173
|
-
"cache"
|
|
174
|
-
],
|
|
175
|
-
"configurationContainsPrivateHome": true,
|
|
176
|
-
"configurationContainsWorkspace": true
|
|
177
|
-
},
|
|
178
|
-
{
|
|
179
|
-
"repetition": 2,
|
|
180
|
-
"role": "candidate",
|
|
181
|
-
"phase": "plugin-install",
|
|
182
|
-
"milliseconds": 141.13416599999982,
|
|
183
|
-
"status": "completed"
|
|
184
|
-
},
|
|
185
|
-
{
|
|
186
|
-
"repetition": 2,
|
|
187
|
-
"role": "candidate",
|
|
188
|
-
"phase": "runtime.hooks",
|
|
189
|
-
"milliseconds": 1464.9581660000003,
|
|
190
|
-
"status": "completed"
|
|
191
|
-
},
|
|
192
|
-
{
|
|
193
|
-
"repetition": 2,
|
|
194
|
-
"role": "candidate",
|
|
195
|
-
"phase": "sandbox-probe",
|
|
196
|
-
"milliseconds": 5689.016333,
|
|
197
|
-
"status": "completed"
|
|
198
|
-
},
|
|
199
|
-
{
|
|
200
|
-
"repetition": 2,
|
|
201
|
-
"role": "candidate",
|
|
202
|
-
"phase": "sandbox-probe",
|
|
203
|
-
"milliseconds": 5342.566417000002,
|
|
204
|
-
"status": "completed"
|
|
205
|
-
},
|
|
206
|
-
{
|
|
207
|
-
"repetition": 2,
|
|
208
|
-
"role": "candidate",
|
|
209
|
-
"phase": "participant-execution",
|
|
210
|
-
"milliseconds": 28.610333000004175,
|
|
211
|
-
"status": "completed"
|
|
212
|
-
},
|
|
213
|
-
{
|
|
214
|
-
"repetition": 2,
|
|
215
|
-
"role": "candidate",
|
|
216
|
-
"phase": "candidate-total",
|
|
217
|
-
"milliseconds": 14369.195708,
|
|
218
|
-
"status": "completed"
|
|
219
|
-
},
|
|
220
|
-
{
|
|
221
|
-
"repetition": 2,
|
|
222
|
-
"role": "semantic",
|
|
223
|
-
"phase": "runtime.semantic",
|
|
224
|
-
"milliseconds": 1235.6402080000043,
|
|
225
|
-
"status": "completed"
|
|
226
|
-
},
|
|
227
|
-
{
|
|
228
|
-
"repetition": 2,
|
|
229
|
-
"role": "semantic",
|
|
230
|
-
"phase": "plugin-install",
|
|
231
|
-
"milliseconds": 0.014084000002185348,
|
|
232
|
-
"status": "completed"
|
|
233
|
-
},
|
|
234
|
-
{
|
|
235
|
-
"repetition": 2,
|
|
236
|
-
"role": "semantic",
|
|
237
|
-
"phase": "runtime.hooks",
|
|
238
|
-
"milliseconds": 1087.1219169999968,
|
|
239
|
-
"status": "completed"
|
|
240
|
-
},
|
|
241
|
-
{
|
|
242
|
-
"repetition": 2,
|
|
243
|
-
"role": "semantic",
|
|
244
|
-
"phase": "sandbox-probe",
|
|
245
|
-
"milliseconds": 5409.527834,
|
|
246
|
-
"status": "completed"
|
|
247
|
-
},
|
|
248
|
-
{
|
|
249
|
-
"repetition": 2,
|
|
250
|
-
"role": "semantic",
|
|
251
|
-
"phase": "sandbox-probe",
|
|
252
|
-
"milliseconds": 5393.348124999997,
|
|
253
|
-
"status": "completed"
|
|
254
|
-
},
|
|
255
|
-
{
|
|
256
|
-
"repetition": 2,
|
|
257
|
-
"role": "semantic",
|
|
258
|
-
"phase": "participant-execution",
|
|
259
|
-
"milliseconds": 44.382833999996365,
|
|
260
|
-
"status": "completed"
|
|
261
|
-
},
|
|
262
|
-
{
|
|
263
|
-
"repetition": 2,
|
|
264
|
-
"role": "semantic",
|
|
265
|
-
"phase": "grading",
|
|
266
|
-
"milliseconds": 13533.550041999995,
|
|
267
|
-
"status": "completed"
|
|
268
|
-
},
|
|
269
|
-
{
|
|
270
|
-
"repetition": 3,
|
|
271
|
-
"role": "candidate",
|
|
272
|
-
"phase": "runtime.candidate",
|
|
273
|
-
"milliseconds": 1273.4362919999985,
|
|
274
|
-
"status": "completed"
|
|
275
|
-
},
|
|
276
|
-
{
|
|
277
|
-
"repetition": 3,
|
|
278
|
-
"role": "candidate",
|
|
279
|
-
"phase": "plugin-command",
|
|
280
|
-
"milliseconds": 75.75125000000116,
|
|
281
|
-
"status": "completed"
|
|
282
|
-
},
|
|
283
|
-
{
|
|
284
|
-
"repetition": 3,
|
|
285
|
-
"role": "candidate",
|
|
286
|
-
"phase": "plugin-command",
|
|
287
|
-
"milliseconds": 65.14912499999627,
|
|
288
|
-
"status": "completed"
|
|
289
|
-
},
|
|
290
|
-
{
|
|
291
|
-
"repetition": 3,
|
|
292
|
-
"role": "candidate",
|
|
293
|
-
"phase": "installation-assets",
|
|
294
|
-
"files": [
|
|
295
|
-
"cache"
|
|
296
|
-
],
|
|
297
|
-
"configurationContainsPrivateHome": true,
|
|
298
|
-
"configurationContainsWorkspace": true
|
|
299
|
-
},
|
|
300
|
-
{
|
|
301
|
-
"repetition": 3,
|
|
302
|
-
"role": "candidate",
|
|
303
|
-
"phase": "plugin-install",
|
|
304
|
-
"milliseconds": 143.153999999995,
|
|
305
|
-
"status": "completed"
|
|
306
|
-
},
|
|
307
|
-
{
|
|
308
|
-
"repetition": 3,
|
|
309
|
-
"role": "candidate",
|
|
310
|
-
"phase": "runtime.hooks",
|
|
311
|
-
"milliseconds": 1234.849000000002,
|
|
312
|
-
"status": "completed"
|
|
313
|
-
},
|
|
314
|
-
{
|
|
315
|
-
"repetition": 3,
|
|
316
|
-
"role": "candidate",
|
|
317
|
-
"phase": "sandbox-probe",
|
|
318
|
-
"milliseconds": 5916.286457999995,
|
|
319
|
-
"status": "completed"
|
|
320
|
-
},
|
|
321
|
-
{
|
|
322
|
-
"repetition": 3,
|
|
323
|
-
"role": "candidate",
|
|
324
|
-
"phase": "sandbox-probe",
|
|
325
|
-
"milliseconds": 5706.173332999999,
|
|
326
|
-
"status": "completed"
|
|
327
|
-
},
|
|
328
|
-
{
|
|
329
|
-
"repetition": 3,
|
|
330
|
-
"role": "candidate",
|
|
331
|
-
"phase": "participant-execution",
|
|
332
|
-
"milliseconds": 38.983124999998836,
|
|
333
|
-
"status": "completed"
|
|
334
|
-
},
|
|
335
|
-
{
|
|
336
|
-
"repetition": 3,
|
|
337
|
-
"role": "candidate",
|
|
338
|
-
"phase": "candidate-total",
|
|
339
|
-
"milliseconds": 14666.299250000004,
|
|
340
|
-
"status": "completed"
|
|
341
|
-
},
|
|
342
|
-
{
|
|
343
|
-
"repetition": 3,
|
|
344
|
-
"role": "semantic",
|
|
345
|
-
"phase": "runtime.semantic",
|
|
346
|
-
"milliseconds": 2261.5522090000013,
|
|
347
|
-
"status": "completed"
|
|
348
|
-
},
|
|
349
|
-
{
|
|
350
|
-
"repetition": 3,
|
|
351
|
-
"role": "semantic",
|
|
352
|
-
"phase": "plugin-install",
|
|
353
|
-
"milliseconds": 0.016417000006185845,
|
|
354
|
-
"status": "completed"
|
|
355
|
-
},
|
|
356
|
-
{
|
|
357
|
-
"repetition": 3,
|
|
358
|
-
"role": "semantic",
|
|
359
|
-
"phase": "runtime.hooks",
|
|
360
|
-
"milliseconds": 1363.224583999996,
|
|
361
|
-
"status": "completed"
|
|
362
|
-
},
|
|
363
|
-
{
|
|
364
|
-
"repetition": 3,
|
|
365
|
-
"role": "semantic",
|
|
366
|
-
"phase": "sandbox-probe",
|
|
367
|
-
"milliseconds": 5592.143207999994,
|
|
368
|
-
"status": "completed"
|
|
369
|
-
},
|
|
370
|
-
{
|
|
371
|
-
"repetition": 3,
|
|
372
|
-
"role": "semantic",
|
|
373
|
-
"phase": "sandbox-probe",
|
|
374
|
-
"milliseconds": 5439.415540999995,
|
|
375
|
-
"status": "completed"
|
|
376
|
-
},
|
|
377
|
-
{
|
|
378
|
-
"repetition": 3,
|
|
379
|
-
"role": "semantic",
|
|
380
|
-
"phase": "participant-execution",
|
|
381
|
-
"milliseconds": 27.27187499999127,
|
|
382
|
-
"status": "completed"
|
|
383
|
-
},
|
|
384
|
-
{
|
|
385
|
-
"repetition": 3,
|
|
386
|
-
"role": "semantic",
|
|
387
|
-
"phase": "grading",
|
|
388
|
-
"milliseconds": 15030.493917,
|
|
389
|
-
"status": "completed"
|
|
390
|
-
},
|
|
391
|
-
{
|
|
392
|
-
"repetition": 4,
|
|
393
|
-
"role": "semantic",
|
|
394
|
-
"phase": "cow-copy",
|
|
395
|
-
"milliseconds": 1.0108749999926658,
|
|
396
|
-
"status": "completed"
|
|
397
|
-
},
|
|
398
|
-
{
|
|
399
|
-
"phase": "cow-isolation",
|
|
400
|
-
"sourceUnchanged": true
|
|
401
|
-
}
|
|
402
|
-
]
|
|
403
|
-
}
|
|
@@ -1,63 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"format": "sevro.consumer-smoke.v1",
|
|
3
|
-
"date": "2026-10-11",
|
|
4
|
-
"consumer": {
|
|
5
|
-
"repository": "https://github.com/BjRo/darrow",
|
|
6
|
-
"revision": "34606ad9e1952b7f4e9c56befa446c18e1a2eb5d",
|
|
7
|
-
"trackedChanges": false
|
|
8
|
-
},
|
|
9
|
-
"environment": {
|
|
10
|
-
"platform": "darwin",
|
|
11
|
-
"arch": "arm64",
|
|
12
|
-
"bun": "1.3.13",
|
|
13
|
-
"codex": "0.162.1",
|
|
14
|
-
"candidateModel": "gpt-6-luna",
|
|
15
|
-
"candidateEffort": "medium",
|
|
16
|
-
"semanticModel": "gpt-6-luna",
|
|
17
|
-
"semanticEffort": "medium"
|
|
18
|
-
},
|
|
19
|
-
"sampling": {
|
|
20
|
-
"trials": 1,
|
|
21
|
-
"jobs": 1,
|
|
22
|
-
"threshold": 1
|
|
23
|
-
},
|
|
24
|
-
"runs": [
|
|
25
|
-
{
|
|
26
|
-
"runner": "unchanged c205482b25d4d52a395c70434f2b392f6acfb279",
|
|
27
|
-
"case": "create-commit-conventional-format",
|
|
28
|
-
"execution": "failed",
|
|
29
|
-
"task": "not_assessed",
|
|
30
|
-
"activation": "unavailable",
|
|
31
|
-
"participantReached": false,
|
|
32
|
-
"diagnostic": "First sandbox preflight exceeded its 10-second deadline."
|
|
33
|
-
},
|
|
34
|
-
{
|
|
35
|
-
"runner": "issue-30 candidate",
|
|
36
|
-
"case": "create-commit-conventional-format",
|
|
37
|
-
"execution": "completed",
|
|
38
|
-
"grading": "completed",
|
|
39
|
-
"task": "failed",
|
|
40
|
-
"shellChecksPassed": 3,
|
|
41
|
-
"activation": "unavailable",
|
|
42
|
-
"limitation": "Implicit repository fixture omitted plugin manifests required by its backend launcher. Candidate correctly refused to create the requested commit; head-change assertion failed."
|
|
43
|
-
},
|
|
44
|
-
{
|
|
45
|
-
"runner": "issue-30 candidate",
|
|
46
|
-
"case": "tdd-explicit-invocation",
|
|
47
|
-
"execution": "completed",
|
|
48
|
-
"grading": "completed",
|
|
49
|
-
"task": "passed",
|
|
50
|
-
"shellCheck": "passed",
|
|
51
|
-
"semanticCheck": "passed",
|
|
52
|
-
"activation": "passed",
|
|
53
|
-
"activationCompleteness": "complete",
|
|
54
|
-
"activationEvidence": "Native Codex explicit-invocation receipt"
|
|
55
|
-
}
|
|
56
|
-
],
|
|
57
|
-
"limits": [
|
|
58
|
-
"Only one authenticated trial per listed run.",
|
|
59
|
-
"Different cases are not a matched live performance comparison.",
|
|
60
|
-
"No live API/model latency saving is inferred.",
|
|
61
|
-
"Private transcripts, prompts, tool arguments, and credentials are omitted."
|
|
62
|
-
]
|
|
63
|
-
}
|
|
@@ -1,20 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"codex": "codex-cli 0.162.1",
|
|
3
|
-
"registered": {
|
|
4
|
-
"code": 0,
|
|
5
|
-
"out": "{\n \"marketplaceName\": \"sevro-assessment\",\n \"installedRoot\": \"<assessment>/market\",\n \"alreadyAdded\": false\n}\n",
|
|
6
|
-
"err": ""
|
|
7
|
-
},
|
|
8
|
-
"installed": {
|
|
9
|
-
"code": 0,
|
|
10
|
-
"out": "{\n \"pluginId\": \"probe@sevro-assessment\",\n \"name\": \"probe\",\n \"marketplaceName\": \"sevro-assessment\",\n \"version\": \"1.0.0\",\n \"installedPath\": \"<assessment>/home/plugins/cache/sevro-assessment/probe/1.0.0\",\n \"authPolicy\": \"ON_INSTALL\"\n}\n",
|
|
11
|
-
"err": ""
|
|
12
|
-
},
|
|
13
|
-
"configuration": "[marketplaces.sevro-assessment]\nsource_type = \"local\"\nsource = \"<assessment>/market\"\n\n[plugins.\"probe@sevro-assessment\"]\nenabled = true\n",
|
|
14
|
-
"cacheOnlyFreshHomeList": {
|
|
15
|
-
"code": 0,
|
|
16
|
-
"out": "{\n \"installed\": [],\n \"available\": []\n}\n",
|
|
17
|
-
"err": ""
|
|
18
|
-
},
|
|
19
|
-
"conditions": "Local synthetic plugin; fresh homes without credentials; installed files copied to a second home; no registration copied"
|
|
20
|
-
}
|
package/docs/evidence.md
DELETED
|
@@ -1,17 +0,0 @@
|
|
|
1
|
-
# Historical validation evidence
|
|
2
|
-
|
|
3
|
-
These records preserve specific observations and limits. They are not current
|
|
4
|
-
public contracts, fresh acceptance runs, or proof of broader platform support.
|
|
5
|
-
Keep their observations intact:
|
|
6
|
-
|
|
7
|
-
- [Cache ownership](cache-ownership-validation.md)
|
|
8
|
-
- [Claude continuation](claude-continuation-validation.md)
|
|
9
|
-
- [Concurrency](concurrency-validation.md)
|
|
10
|
-
- [Configuration roots](configuration-root-validation.md)
|
|
11
|
-
- [Fixture cleanup](fixture-cleanup-validation.md)
|
|
12
|
-
|
|
13
|
-
The [first engine slice](engine-slice.md) also records bounded implementation
|
|
14
|
-
context. [Current contracts](contracts.md) and [architecture](architecture.md)
|
|
15
|
-
describe present interfaces. Coverage discovery belongs to [issue #1](https://github.com/BjRo/sevro/issues/1);
|
|
16
|
-
targeted prototypes do not prove full-suite coverage.
|
|
17
|
-
Return to the [hub](README.md).
|
|
@@ -1,73 +0,0 @@
|
|
|
1
|
-
# Permission-locked fixture cleanup
|
|
2
|
-
|
|
3
|
-
These historical observations predate the current
|
|
4
|
-
[workspace retention policy](results-v1.md).
|
|
5
|
-
|
|
6
|
-
The Darrow extraction comparison exposed a cleanup difference: its legacy
|
|
7
|
-
fixture destroyer removes permission-locked directories, while Sevro retained
|
|
8
|
-
them after a completed trial. Sevro now restores owner access inside completed
|
|
9
|
-
candidate workspaces when needed. It clears their contents after trial evidence
|
|
10
|
-
is retained, keeps empty reservations until active peers drain, and then removes
|
|
11
|
-
the completed roots. Directory traversal uses link metadata and skips symbolic
|
|
12
|
-
links; external targets retain their permissions and contents.
|
|
13
|
-
|
|
14
|
-
## Public CLI regression slices
|
|
15
|
-
|
|
16
|
-
The public seam is `sevro run --case-file ... --adapter-module ...`, as documented
|
|
17
|
-
in [Development CLI](development-cli.md). Each controlled adapter uses real
|
|
18
|
-
filesystem permissions and returns its workspace path as a bounded observation.
|
|
19
|
-
The test reads retained public evidence and verifies that the workspace is gone.
|
|
20
|
-
A preliminary internal-engine test was diagnostic only; it is not part of this
|
|
21
|
-
TDD trace and was removed before implementation.
|
|
22
|
-
|
|
23
|
-
Red — `bun test tests/cli.test.ts --test-name-pattern 'CLI removes a permission-locked candidate directory'`: completed evidence existed, but the mode-000 child directory kept the workspace present.
|
|
24
|
-
|
|
25
|
-
Green — `bun test tests/cli.test.ts --test-name-pattern 'CLI removes a permission-locked candidate directory'`: the workspace was removed and retained evidence still agreed with the CLI result.
|
|
26
|
-
|
|
27
|
-
Red — `bun test tests/cli.test.ts --test-name-pattern 'permission-locked candidate directory \(root\)'`: a mode-000 workspace root remained present after completion.
|
|
28
|
-
|
|
29
|
-
Green — `bun test tests/cli.test.ts --test-name-pattern 'permission-locked candidate directory \(root\)'`: the workspace root was removed after evidence retention.
|
|
30
|
-
|
|
31
|
-
Red — `bun test tests/cli.test.ts --test-name-pattern 'permission-locked candidate directory \(readable root\)'`: the readable mode-500 root lacked write access and remained present.
|
|
32
|
-
|
|
33
|
-
Green — `bun test tests/cli.test.ts --test-name-pattern 'permission-locked candidate directory \(readable root\)'`: restoring owner write access allowed cleanup; the external link target's mode and bytes were unchanged.
|
|
34
|
-
|
|
35
|
-
Each final example also links to an external directory with a read-only file.
|
|
36
|
-
All three check that its directory mode, file mode, and file contents stay intact.
|
|
37
|
-
These are deterministic filesystem and lifecycle checks, with no live model call.
|
|
38
|
-
|
|
39
|
-
## Final source gates
|
|
40
|
-
|
|
41
|
-
`bun test` passed all 243 tests and 1399 assertions across 38 files in 93.83
|
|
42
|
-
seconds. The gate includes parallel admission, peer isolation, cancellation,
|
|
43
|
-
failed persistence, extension lifecycle, and generated/repository fixture tests.
|
|
44
|
-
`bun run typecheck` and `bun run format:check` pass.
|
|
45
|
-
The full log and subsequent installed-archive checks are retained under
|
|
46
|
-
`/Users/bjro/.darrow/issue95-fixture-cleanup/`. The tests ran on macOS with Bun
|
|
47
|
-
1.3.13; they do not establish another platform or native model route.
|
|
48
|
-
|
|
49
|
-
## Canonical workspace follow-up
|
|
50
|
-
|
|
51
|
-
Darrow's private fixture builder also checked that its returned path was
|
|
52
|
-
canonical. An initial diagnostic assertion in the cleanup examples exposed
|
|
53
|
-
Sevro's logical temporary-directory alias; those diagnostic additions were
|
|
54
|
-
removed before the isolated regression. The public canonical-workspace test
|
|
55
|
-
records the operating system's `realpath` through a controlled host and compares
|
|
56
|
-
it with the received workspace in retained evidence. The full source gate above
|
|
57
|
-
predates this follow-up.
|
|
58
|
-
|
|
59
|
-
The public CLI contract was clarified before implementation. Candidate and
|
|
60
|
-
semantic workspace allocations now resolve the new directory through `realpath`
|
|
61
|
-
before reservation, preparation, or host execution.
|
|
62
|
-
|
|
63
|
-
Red — `bun test tests/cli.test.ts --test-name-pattern 'CLI supplies a canonical candidate workspace'`:
|
|
64
|
-
exit `1`; the received `/var/...` path differed from the operating system's
|
|
65
|
-
`/private/var/...` path. One test failed with four assertions in 222 milliseconds.
|
|
66
|
-
|
|
67
|
-
Green — `bun test tests/cli.test.ts --test-name-pattern 'CLI supplies a canonical candidate workspace'`:
|
|
68
|
-
exit `0`; one test and four assertions passed in 228 milliseconds after the fix.
|
|
69
|
-
|
|
70
|
-
The final `bun test` gate passed 244 tests and 1403 assertions across 38 files in
|
|
71
|
-
94.77 seconds. Typecheck and formatting pass. The source log is retained at
|
|
72
|
-
`/Users/bjro/.darrow/issue95-fixture-mounting/sevro-full-gate.log`. These remain
|
|
73
|
-
deterministic macOS/Bun checks, without a live model or additional platform claim.
|