@zhivex-ai/harness 1.0.0 → 1.1.0-rc.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +109 -1
- package/README.md +52 -306
- package/SECURITY.md +1 -1
- package/SUPPORT.md +7 -3
- package/benchmarks/README.md +3 -3
- package/contracts/client-protocol-v1.schema.json +341 -0
- package/contracts/public-api.json +100 -1
- package/contracts/stable-api-signatures.json +14 -14
- package/dist/approvals/approval-diff.d.ts +45 -0
- package/dist/approvals/approval-history.d.ts +38 -0
- package/dist/approvals/approval-preview.d.ts +26 -0
- package/dist/cli/arguments.d.ts +103 -0
- package/dist/cli/changes.d.ts +2 -0
- package/dist/cli/cli-credentials.d.ts +38 -0
- package/dist/cli/cli-help.d.ts +3 -0
- package/dist/{cli-options.d.ts → cli/cli-options.d.ts} +1 -1
- package/dist/{cli-profiles.d.ts → cli/cli-profiles.d.ts} +1 -1
- package/dist/cli/configured-harness.d.ts +12 -0
- package/dist/cli/console/console-commands.d.ts +6 -0
- package/dist/{console-context.d.ts → cli/console/console-context.d.ts} +7 -3
- package/dist/cli/console/console-input.d.ts +68 -0
- package/dist/cli/console/console-model-catalog.d.ts +25 -0
- package/dist/cli/console/console-navigation.d.ts +27 -0
- package/dist/cli/console/console-presentation.d.ts +18 -0
- package/dist/cli/console/console-welcome.d.ts +13 -0
- package/dist/cli/console.d.ts +2 -0
- package/dist/cli/doctor.d.ts +50 -0
- package/dist/cli/errors.d.ts +7 -0
- package/dist/cli/help-text.d.ts +2 -0
- package/dist/cli/presentation.d.ts +23 -0
- package/dist/cli/resume-metadata.d.ts +12 -0
- package/dist/cli/routing.d.ts +6 -0
- package/dist/cli/run-commands.d.ts +4 -0
- package/dist/cli/run-document.d.ts +91 -0
- package/dist/cli/run-management.d.ts +2 -0
- package/dist/cli/service-client-cli.d.ts +2 -0
- package/dist/cli/session-management.d.ts +8 -0
- package/dist/cli/setup.d.ts +16 -0
- package/dist/cli/state.d.ts +2 -0
- package/dist/cli/terminal/terminal-continuation.d.ts +3 -0
- package/dist/{terminal-markdown.d.ts → cli/terminal/terminal-markdown.d.ts} +1 -0
- package/dist/{terminal-ui.d.ts → cli/terminal/terminal-ui.d.ts} +7 -0
- package/dist/cli/terminal/tool-activity.d.ts +19 -0
- package/dist/cli.d.ts +33 -238
- package/dist/cli.js +16049 -12066
- package/dist/client/adapter.d.ts +4 -0
- package/dist/client/index.d.ts +15 -0
- package/dist/client/local-service.d.ts +34 -0
- package/dist/client/protocol.d.ts +205 -0
- package/dist/client/service-events.d.ts +50 -0
- package/dist/{context-engineering.d.ts → context/context-engineering.d.ts} +2 -2
- package/dist/{execution-environment.d.ts → execution/execution-environment.d.ts} +8 -4
- package/dist/hostile-repository-demo.js +2025 -897
- package/dist/index.d.ts +48 -41
- package/dist/index.js +7541 -5906
- package/dist/internal/desktop/persistence.d.ts +14 -0
- package/dist/internal/desktop/protocol.d.ts +8 -0
- package/dist/internal/desktop/providers.d.ts +6 -0
- package/dist/internal/desktop/runtime.d.ts +7 -0
- package/dist/{operations.d.ts → persistence/operations.d.ts} +24 -2
- package/dist/persistence/persistence-secrets.d.ts +2 -0
- package/dist/persistence/recorded-workspace.d.ts +4 -0
- package/dist/{sessions.d.ts → persistence/sessions.d.ts} +3 -0
- package/dist/persistence/sqlite-access.d.ts +26 -0
- package/dist/{sqlite-database.d.ts → persistence/sqlite-database.d.ts} +2 -0
- package/dist/{state-backup.d.ts → persistence/state-backup.d.ts} +9 -1
- package/dist/{capabilities.d.ts → providers/capabilities.d.ts} +1 -1
- package/dist/{routing.d.ts → providers/routing.d.ts} +1 -1
- package/dist/{config.d.ts → runtime/config.d.ts} +1 -1
- package/dist/{harness.d.ts → runtime/harness.d.ts} +12 -135
- package/dist/{orchestration.d.ts → runtime/orchestration.d.ts} +3 -0
- package/dist/{runtime-policy.d.ts → runtime/runtime-policy.d.ts} +1 -1
- package/dist/runtime/usage-ledger.d.ts +84 -0
- package/dist/service-cli.d.ts +2 -0
- package/dist/service-cli.js +10899 -0
- package/dist/tools/execution.d.ts +47 -0
- package/dist/tools/shared.d.ts +67 -0
- package/dist/tools/workspace.d.ts +85 -0
- package/dist/verify-historical-migrations.js +5786 -5265
- package/dist/{workspace.d.ts → workspace/workspace.d.ts} +22 -0
- package/dist/zhx.js +16049 -12066
- package/docs/CLI.md +151 -30
- package/docs/CLIENT_PROTOCOL.md +212 -0
- package/docs/CREDENTIALS.md +59 -0
- package/docs/DURABLE_OPERATIONS.md +2 -2
- package/docs/FIRST_USE.md +82 -0
- package/docs/FIRST_USE_EXAMPLE.md +40 -0
- package/docs/HOSTILE_REPOSITORY_DEMO.md +1 -1
- package/docs/LOCAL_SERVICE.md +142 -0
- package/docs/README.md +24 -23
- package/docs/THREAT_MODEL.md +1 -1
- package/docs/TIME_TO_SAFE_FIX.md +2 -2
- package/docs/USAGE.md +181 -0
- package/evaluations/daily-cli-cohort.json +33 -0
- package/evaluations/representative-assembly-matrix.json +28 -1
- package/examples/README.md +26 -0
- package/examples/first-use/greeting.test.ts +3 -0
- package/examples/first-use/greeting.ts +1 -0
- package/examples/first-use/package.json +7 -0
- package/package.json +39 -11
- package/ROADMAP.md +0 -414
- package/benchmarks/baselines/context-navigation-2026-09-09.json +0 -94
- package/benchmarks/baselines/efficiency-2026-09-08.json +0 -233
- package/benchmarks/baselines/external-final-2026-09-08.json +0 -588
- package/benchmarks/baselines/external-intermediate-2026-09-08.json +0 -116
- package/benchmarks/baselines/external-matched-openai-2026-09-08.json +0 -1922
- package/benchmarks/baselines/external-matched-qwen-2026-09-08.json +0 -1706
- package/benchmarks/baselines/external-pilot-2026-09-08.json +0 -439
- package/benchmarks/baselines/external-qwen-flash-2026-09-08.json +0 -1570
- package/benchmarks/baselines/external-qwen-remediation-2026-09-09.json +0 -1509
- package/benchmarks/baselines/external-qwen-sdk-114-progress-2026-09-09.json +0 -2509
- package/benchmarks/baselines/external-sdk-upgrade-2026-09-08.json +0 -564
- package/benchmarks/baselines/external-verifier-recovery-2026-09-09.json +0 -794
- package/benchmarks/baselines/full-harness-review-2026-09-10.json +0 -134
- package/benchmarks/baselines/harness-audit-2026-09-09.json +0 -136
- package/benchmarks/baselines/harness-remediation-2026-09-09.json +0 -160
- package/benchmarks/baselines/oci-remediation-2026-09-10.json +0 -535
- package/benchmarks/baselines/rc14-holdout-2026-09-10.json +0 -854
- package/benchmarks/baselines/repair-architecture-audit-2026-09-09.json +0 -75
- package/benchmarks/baselines/time-to-safe-fix-live-expanded-2026-08-21.json +0 -186
- package/benchmarks/baselines/time-to-safe-fix-live-smoke-2026-08-21.json +0 -186
- package/dist/console-input.d.ts +0 -28
- package/docs/GA_READINESS.md +0 -29
- package/docs/GEMINI_1_0_DECISION.md +0 -9
- package/docs/LIVE_CERTIFICATION.md +0 -107
- package/docs/RC13_SECURITY_REVIEW.md +0 -28
- package/docs/RC14_DELEGATED_SECURITY_DECISION.md +0 -80
- package/docs/RC14_SECURITY_REVIEW.md +0 -40
- package/docs/RELEASE.md +0 -96
- package/docs/ROLLBACK.md +0 -9
- package/docs/SECURITY_REVIEW_EVIDENCE.md +0 -17
- package/docs/ga-readiness.json +0 -366
- package/docs/reports/EFFICIENCY_REVIEW.md +0 -156
- package/docs/reports/EXTERNAL_COMPARISON.md +0 -434
- package/docs/reports/FULL_HARNESS_REVIEW_2026-09-10.md +0 -200
- package/docs/reports/GA_BUDGET_REMEDIATION_2026-09-19.md +0 -38
- package/docs/reports/GA_REPAIR_RECOVERY_2026-09-19.md +0 -700
- package/docs/reports/GA_VALIDATION_2026-09-19.md +0 -111
- package/docs/reports/HARNESS_ARCHITECTURE_REVIEW_2026-09-09.md +0 -202
- package/docs/reports/HARNESS_AUDIT_2026-09-09.md +0 -306
- package/docs/reports/HARNESS_REMEDIATION_2026-09-09.md +0 -74
- package/docs/reports/HARNESS_REMEDIATION_2026-09-10.md +0 -90
- package/docs/reports/MODEL_COMPARISON_2026-09-09.md +0 -184
- package/docs/reports/QWEN_REMEDIATION_COMPARISON_2026-09-09.md +0 -142
- package/docs/reports/RC14_VALIDATION_2026-09-10.md +0 -45
- package/docs/reports/README.md +0 -26
- package/docs/reports/evidence/ga-budget-remediation-2026-09-19.json +0 -133
- package/docs/reports/evidence/ga-catalog-ablation-2026-09-19.json +0 -699
- package/docs/reports/evidence/ga-catalog-ablation-plan-2026-09-19.json +0 -40
- package/docs/reports/evidence/ga-catalog-observed-2026-09-19.json +0 -985
- package/docs/reports/evidence/ga-catalog-observed-plan-2026-09-19.json +0 -28
- package/docs/reports/evidence/ga-catalog-observed-status-2026-09-19.json +0 -15
- package/docs/reports/evidence/ga-complete-local-check-2026-09-19.json +0 -47
- package/docs/reports/evidence/ga-completion-reminder-2026-09-19.json +0 -22
- package/docs/reports/evidence/ga-fresh-cohort-plan-2026-09-19.json +0 -54
- package/docs/reports/evidence/ga-fresh-cohort-preparation-2026-09-19.json +0 -242
- package/docs/reports/evidence/ga-fresh-cohort-result-2026-09-19.json +0 -4318
- package/docs/reports/evidence/ga-local-validation-followup-2026-09-19.json +0 -66
- package/docs/reports/evidence/ga-node-consumers-2026-09-19.json +0 -31
- package/docs/reports/evidence/ga-package-content-validation-2026-09-19.json +0 -22
- package/docs/reports/evidence/ga-plan-completion-2026-09-19.json +0 -39
- package/docs/reports/evidence/ga-plan-reserve-2026-09-19.json +0 -573
- package/docs/reports/evidence/ga-plan-reserve-plan-2026-09-19.json +0 -24
- package/docs/reports/evidence/ga-plan-transition-2026-09-19.json +0 -19
- package/docs/reports/evidence/ga-private-verifier-plan-2026-09-19.json +0 -24
- package/docs/reports/evidence/ga-private-verifier-root-cause-2026-09-19.json +0 -41
- package/docs/reports/evidence/ga-published-sdk-2026-09-19.json +0 -35
- package/docs/reports/evidence/ga-python-checkout-imports-2026-09-19.json +0 -95
- package/docs/reports/evidence/ga-python-origin-delivery-2026-09-19.json +0 -211
- package/docs/reports/evidence/ga-python-origin-delivery-plan-2026-09-19.json +0 -27
- package/docs/reports/evidence/ga-qwen-patched-package-2026-09-19.json +0 -1092
- package/docs/reports/evidence/ga-qwen-patched-package-plan-2026-09-19.json +0 -18
- package/docs/reports/evidence/ga-qwen-published-2026-09-19.json +0 -571
- package/docs/reports/evidence/ga-qwen-published-plan-2026-09-19.json +0 -18
- package/docs/reports/evidence/ga-qwen-tool-choice-2026-09-19.json +0 -85
- package/docs/reports/evidence/ga-qwen-tool-choice-patched-2026-09-19.json +0 -85
- package/docs/reports/evidence/ga-recovery-development-2026-09-19.json +0 -1066
- package/docs/reports/evidence/ga-recovery-graded-development-2026-09-19.json +0 -1313
- package/docs/reports/evidence/ga-recovery-graded-v2-2026-09-19.json +0 -1151
- package/docs/reports/evidence/ga-recovery-v3-2026-09-19.json +0 -3943
- package/docs/reports/evidence/ga-recovery-v4-2026-09-19.json +0 -1055
- package/docs/reports/evidence/ga-recovery-v5-2026-09-19.json +0 -1245
- package/docs/reports/evidence/ga-recovery-v6-2026-09-19.json +0 -699
- package/docs/reports/evidence/ga-recovery-v7-2026-09-19.json +0 -854
- package/docs/reports/evidence/ga-recovery-v8-2026-09-19.json +0 -596
- package/docs/reports/evidence/ga-required-delivery-2026-09-19.json +0 -25
- package/docs/reports/evidence/ga-required-delivery-live-2026-09-19.json +0 -1337
- package/docs/reports/evidence/ga-required-delivery-live-plan-2026-09-19.json +0 -25
- package/docs/reports/evidence/ga-retained-candidate-grading-2026-09-19.json +0 -30
- package/docs/reports/evidence/ga-tool-selection-live-2026-09-19.json +0 -559
- package/docs/reports/evidence/ga-tool-selection-live-plan-2026-09-19.json +0 -28
- package/docs/reports/evidence/ga-tool-selection-recovery-2026-09-19.json +0 -27
- package/docs/reports/evidence/ga-validation-2026-09-19.json +0 -236
- package/docs/reports/evidence/ga-verifier-discovery-live-2026-09-19.json +0 -1369
- package/docs/reports/evidence/ga-verifier-discovery-plan-2026-09-19.json +0 -29
- package/docs/reports/evidence/ga-verifier-failure-identification-2026-09-19.json +0 -30
- package/docs/reports/evidence/ga-verifier-feedback-2026-09-19.json +0 -65
- package/docs/reports/evidence/ga-verifier-hints-live-2026-09-19.json +0 -1297
- package/docs/reports/evidence/ga-verifier-hints-plan-2026-09-19.json +0 -27
- package/docs/reports/evidence/ga-work-boundary-context-analysis-2026-09-19.json +0 -113
- package/docs/reports/evidence/ga-work-boundary-live-2026-09-19.json +0 -301
- package/docs/reports/evidence/ga-work-boundary-live-plan-2026-09-19.json +0 -84
- package/docs/reports/evidence/ga-work-budget-planning-2026-09-19.json +0 -32
- package/docs/reports/evidence/ga-working-context-2026-09-19.json +0 -22
- package/docs/reports/evidence/rc14-delegated-security-review-2026-09-20.json +0 -81
- package/docs/reports/evidence/rc14-ga-gate-2026-09-20.json +0 -18
- package/docs/reports/evidence/rc14-representative-2026-09-20.json +0 -509
- package/docs/reports/evidence/rc14-security-regressions-2026-09-20.json +0 -64
- package/docs/reports/evidence/security-current-2026-09-19.json +0 -91
- package/docs/reports/evidence/security-remediation-2026-09-19.json +0 -97
- /package/dist/{cli-stream.d.ts → cli/cli-stream.d.ts} +0 -0
- /package/dist/{json-contracts.d.ts → client/json-contracts.d.ts} +0 -0
- /package/dist/{compaction.d.ts → context/compaction.d.ts} +0 -0
- /package/dist/{context-metrics.d.ts → context/context-metrics.d.ts} +0 -0
- /package/dist/{task-memory.d.ts → context/task-memory.d.ts} +0 -0
- /package/dist/{package-manager.d.ts → execution/package-manager.d.ts} +0 -0
- /package/dist/{process-runtime.d.ts → execution/process-runtime.d.ts} +0 -0
- /package/dist/{mcp.d.ts → integrations/mcp.d.ts} +0 -0
- /package/dist/{state-directory.d.ts → persistence/state-directory.d.ts} +0 -0
- /package/dist/{providers.d.ts → providers/providers.d.ts} +0 -0
- /package/dist/{errors.d.ts → runtime/errors.d.ts} +0 -0
- /package/dist/{model-budget.d.ts → runtime/model-budget.d.ts} +0 -0
- /package/dist/{repair-controller.d.ts → runtime/repair-controller.d.ts} +0 -0
- /package/dist/{repair-progress.d.ts → runtime/repair-progress.d.ts} +0 -0
- /package/dist/{repair-verifier.d.ts → runtime/repair-verifier.d.ts} +0 -0
- /package/dist/{run-interruption.d.ts → runtime/run-interruption.d.ts} +0 -0
- /package/dist/{runtime-checkpoints.d.ts → runtime/runtime-checkpoints.d.ts} +0 -0
- /package/dist/{runtime-diagnostics.d.ts → runtime/runtime-diagnostics.d.ts} +0 -0
- /package/dist/{time-to-safe-fix.d.ts → runtime/time-to-safe-fix.d.ts} +0 -0
- /package/dist/{tool-registry.d.ts → tools/tool-registry.d.ts} +0 -0
- /package/dist/{bounded-reads.d.ts → workspace/bounded-reads.d.ts} +0 -0
- /package/dist/{change-envelope.d.ts → workspace/change-envelope.d.ts} +0 -0
- /package/dist/{edit-contracts.d.ts → workspace/edit-contracts.d.ts} +0 -0
- /package/dist/{file-security.d.ts → workspace/file-security.d.ts} +0 -0
- /package/dist/{replacement-edits.d.ts → workspace/replacement-edits.d.ts} +0 -0
- /package/dist/{workspace-mutation-lock.d.ts → workspace/workspace-mutation-lock.d.ts} +0 -0
package/docs/ga-readiness.json
DELETED
|
@@ -1,366 +0,0 @@
|
|
|
1
|
-
{
|
|
2
|
-
"schemaVersion": 1,
|
|
3
|
-
"targetVersion": "1.0.0",
|
|
4
|
-
"phase": "ready",
|
|
5
|
-
"evidenceMaxAgeDays": 30,
|
|
6
|
-
"contractManifest": "contracts/public-api.json",
|
|
7
|
-
"stabilityPolicy": "docs/STABILITY.md",
|
|
8
|
-
"supportMatrix": "docs/support-matrix.json",
|
|
9
|
-
"securityControls": "contracts/security-controls.json",
|
|
10
|
-
"threatModel": "docs/THREAT_MODEL.md",
|
|
11
|
-
"rollbackPolicy": "docs/ROLLBACK.md",
|
|
12
|
-
"deprecationPolicy": "docs/DEPRECATIONS.md",
|
|
13
|
-
"representativeEvaluation": "evaluations/representative-matrix.json",
|
|
14
|
-
"migrationGuarantee": {
|
|
15
|
-
"configSchemas": [
|
|
16
|
-
4,
|
|
17
|
-
5
|
|
18
|
-
],
|
|
19
|
-
"operationsSchema": 1,
|
|
20
|
-
"sessionSchema": 1,
|
|
21
|
-
"pausedApprovalPolicy": "original-artifact-only",
|
|
22
|
-
"historicalFixtures": "passed",
|
|
23
|
-
"historicalFixtureEvidence": [
|
|
24
|
-
".github/workflows/ci.yml",
|
|
25
|
-
"fixtures/migrations/0.10.0.json",
|
|
26
|
-
"fixtures/migrations/0.10.0.sqlite",
|
|
27
|
-
"fixtures/migrations/0.11.1.json",
|
|
28
|
-
"fixtures/migrations/0.11.1.sqlite",
|
|
29
|
-
"scripts/verify-historical-migrations.ts"
|
|
30
|
-
]
|
|
31
|
-
},
|
|
32
|
-
"securityReview": {
|
|
33
|
-
"status": "passed",
|
|
34
|
-
"criticalFindings": 0,
|
|
35
|
-
"highFindings": 0,
|
|
36
|
-
"observedAt": "2026-09-20T13:25:48.476Z",
|
|
37
|
-
"evidence": "security-reviews/rc14-review.json"
|
|
38
|
-
},
|
|
39
|
-
"releaseCandidates": [
|
|
40
|
-
{
|
|
41
|
-
"version": "1.0.0-rc.1",
|
|
42
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/34",
|
|
43
|
-
"status": "failed-gates",
|
|
44
|
-
"channel": "next",
|
|
45
|
-
"tag": "v1.0.0-rc.1",
|
|
46
|
-
"contractBreakingDefects": null,
|
|
47
|
-
"publishedAt": null,
|
|
48
|
-
"sourceCommit": "08d27d97b572f1197e66a568ba0148335e020b95",
|
|
49
|
-
"artifactSha512": "sha512-LoXIvlxphjeOG0pS8s5kwiSUCUuoC3OsK2PkqO5ENSlLwpewuIJ1HGgOXKeIptjr0e5jJTMlOvAWPYtqD2vocw==",
|
|
50
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32674617898",
|
|
51
|
-
"provenance": "not-published",
|
|
52
|
-
"liveCertification": "failed-release-bound-run",
|
|
53
|
-
"observedAt": "2026-08-23T23:54:37Z",
|
|
54
|
-
"failedGates": [
|
|
55
|
-
"live-provider-certification",
|
|
56
|
-
"representative-evaluation"
|
|
57
|
-
]
|
|
58
|
-
},
|
|
59
|
-
{
|
|
60
|
-
"version": "1.0.0-rc.2",
|
|
61
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/35",
|
|
62
|
-
"status": "failed-gates",
|
|
63
|
-
"channel": "next",
|
|
64
|
-
"tag": "v1.0.0-rc.2",
|
|
65
|
-
"contractBreakingDefects": null,
|
|
66
|
-
"publishedAt": null,
|
|
67
|
-
"sourceCommit": "57f465bdb44da4093152195c6bf33f9a2c3c47c2",
|
|
68
|
-
"artifactSha512": "sha512-qKAQj7ptM/5mv7NuzVea4dhl8M3GQbRNAbd+bb58Pxhbf7HxKBULoFNG5RGBrTvTeQcNP+tsNJSeuLAn0EkKrg==",
|
|
69
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32676949946",
|
|
70
|
-
"provenance": "not-published",
|
|
71
|
-
"liveCertification": "passed-release-bound-run",
|
|
72
|
-
"observedAt": "2026-08-24T01:22:46Z",
|
|
73
|
-
"failedGates": [
|
|
74
|
-
"representative-evaluation"
|
|
75
|
-
]
|
|
76
|
-
},
|
|
77
|
-
{
|
|
78
|
-
"version": "1.0.0-rc.3",
|
|
79
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/39",
|
|
80
|
-
"status": "failed-gates",
|
|
81
|
-
"channel": "next",
|
|
82
|
-
"tag": "v1.0.0-rc.3",
|
|
83
|
-
"contractBreakingDefects": null,
|
|
84
|
-
"publishedAt": null,
|
|
85
|
-
"sourceCommit": "d906859f74e573be37575c8fa5a953fbabd6268c",
|
|
86
|
-
"artifactSha512": "sha512-XyrEco/p6Bf86PezraBHbnLT0gqXRqygMDSu9ViBrSz1iRwAhAtFg/mzTaTScd1WkRXMIQfFOg3uTVGSNgsR3Q==",
|
|
87
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32725445245",
|
|
88
|
-
"provenance": "not-published",
|
|
89
|
-
"liveCertification": "passed-release-bound-run",
|
|
90
|
-
"observedAt": "2026-08-24T12:31:41Z",
|
|
91
|
-
"failedGates": [
|
|
92
|
-
"representative-evaluation"
|
|
93
|
-
]
|
|
94
|
-
},
|
|
95
|
-
{
|
|
96
|
-
"version": "1.0.0-rc.4",
|
|
97
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/41",
|
|
98
|
-
"status": "failed-gates",
|
|
99
|
-
"channel": "next",
|
|
100
|
-
"tag": "v1.0.0-rc.4",
|
|
101
|
-
"contractBreakingDefects": null,
|
|
102
|
-
"publishedAt": null,
|
|
103
|
-
"sourceCommit": "05ae8e899647207695556c896ef7a8025abcdce4",
|
|
104
|
-
"artifactSha512": "sha512-zpg7yxeQDn2nAws/AsrPd4C79IkXHm4vxWOEiGeVGiil04OcVad/pSn5kCp5nfUiZj0MlygLovBzV83MeD55GQ==",
|
|
105
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32738506085",
|
|
106
|
-
"provenance": "not-published",
|
|
107
|
-
"liveCertification": "passed-release-bound-run",
|
|
108
|
-
"observedAt": "2026-08-24T14:45:46Z",
|
|
109
|
-
"failedGates": [
|
|
110
|
-
"representative-evaluation"
|
|
111
|
-
]
|
|
112
|
-
},
|
|
113
|
-
{
|
|
114
|
-
"version": "1.0.0-rc.5",
|
|
115
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/44",
|
|
116
|
-
"status": "failed-gates",
|
|
117
|
-
"channel": "next",
|
|
118
|
-
"tag": "v1.0.0-rc.5",
|
|
119
|
-
"contractBreakingDefects": null,
|
|
120
|
-
"publishedAt": null,
|
|
121
|
-
"sourceCommit": "a515e87d6f898684e77dbfe415c4f8481844ca81",
|
|
122
|
-
"artifactSha512": "sha512-pCx5y+jH8J00b4+gn44y3jQ0h+elhDDNCemh0LXKNUaGumtYLUPGsonmoo9xhVwLb0sS4XomiacTHM2hSgI1EQ==",
|
|
123
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32792078857",
|
|
124
|
-
"provenance": "not-published",
|
|
125
|
-
"liveCertification": "passed-release-bound-run",
|
|
126
|
-
"observedAt": "2026-08-25T00:26:56Z",
|
|
127
|
-
"failedGates": [
|
|
128
|
-
"representative-evaluation"
|
|
129
|
-
]
|
|
130
|
-
},
|
|
131
|
-
{
|
|
132
|
-
"version": "1.0.0-rc.6",
|
|
133
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/46",
|
|
134
|
-
"status": "failed-gates",
|
|
135
|
-
"channel": "next",
|
|
136
|
-
"tag": "v1.0.0-rc.6",
|
|
137
|
-
"contractBreakingDefects": null,
|
|
138
|
-
"publishedAt": null,
|
|
139
|
-
"sourceCommit": "2301f5c5a2040a8319252fad2a7b0f7c9914b0ae",
|
|
140
|
-
"artifactSha512": "sha512-+dsxQBisN4S99WI5dUmiXIctGsPxvydrTS/Ri63kb6+MkoSjV0oCP0xc/G9Gxw3J4AIbqhxIV0COJvzaJmJLDg==",
|
|
141
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32907208889",
|
|
142
|
-
"provenance": "not-published",
|
|
143
|
-
"liveCertification": "passed-release-bound-run",
|
|
144
|
-
"observedAt": "2026-08-25T23:07:35Z",
|
|
145
|
-
"failedGates": [
|
|
146
|
-
"representative-evaluation"
|
|
147
|
-
]
|
|
148
|
-
},
|
|
149
|
-
{
|
|
150
|
-
"version": "1.0.0-rc.7",
|
|
151
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/31",
|
|
152
|
-
"status": "passed",
|
|
153
|
-
"channel": "next",
|
|
154
|
-
"tag": "v1.0.0-rc.7",
|
|
155
|
-
"contractBreakingDefects": 0,
|
|
156
|
-
"publishedAt": "2026-08-25T23:41:47.980Z",
|
|
157
|
-
"sourceCommit": "4caa83a8011e09abda58b66c4b1d3f65d860b0cf",
|
|
158
|
-
"artifactSha512": "sha512-lXZhlDQR92SrrHZj/H5cirKIKqhiAEaQD03R7+RvJdalhbYDpQHx6TqXJFtSGUXAwGkONCfftOxNRJGrqA/FFw==",
|
|
159
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/32910606768",
|
|
160
|
-
"provenance": "verified",
|
|
161
|
-
"liveCertification": "passed-release-bound-run"
|
|
162
|
-
},
|
|
163
|
-
{
|
|
164
|
-
"version": "1.0.0-rc.8",
|
|
165
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/31",
|
|
166
|
-
"status": "failed-gates",
|
|
167
|
-
"channel": "next",
|
|
168
|
-
"tag": "v1.0.0-rc.8",
|
|
169
|
-
"contractBreakingDefects": null,
|
|
170
|
-
"publishedAt": null,
|
|
171
|
-
"sourceCommit": "db1c6917e23be73c2b4c83ed7762dc676d39f038",
|
|
172
|
-
"artifactSha512": "sha512-SaVLNFzrR2BKEyI11N7Dhx4nDrbt+3AQbD5BVaO4hex5umGP5STpyawSbk742GNUZ/MhqlzhBiS4se2DDpkhRg==",
|
|
173
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/33558133048",
|
|
174
|
-
"provenance": "not-published",
|
|
175
|
-
"liveCertification": "not-run",
|
|
176
|
-
"observedAt": "2026-09-01T20:55:44Z",
|
|
177
|
-
"failedGates": [
|
|
178
|
-
"artifact-binding"
|
|
179
|
-
]
|
|
180
|
-
},
|
|
181
|
-
{
|
|
182
|
-
"version": "1.0.0-rc.9",
|
|
183
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/58",
|
|
184
|
-
"status": "failed-gates",
|
|
185
|
-
"channel": "next",
|
|
186
|
-
"tag": "v1.0.0-rc.9",
|
|
187
|
-
"contractBreakingDefects": null,
|
|
188
|
-
"publishedAt": null,
|
|
189
|
-
"sourceCommit": "545b247c7b33d62c64c3c13e72faca632bae10b6",
|
|
190
|
-
"artifactSha512": null,
|
|
191
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/33562430984",
|
|
192
|
-
"provenance": "not-published",
|
|
193
|
-
"liveCertification": "not-run",
|
|
194
|
-
"observedAt": "2026-09-01T21:42:13Z",
|
|
195
|
-
"failedGates": [
|
|
196
|
-
"deterministic-validation"
|
|
197
|
-
]
|
|
198
|
-
},
|
|
199
|
-
{
|
|
200
|
-
"version": "1.0.0-rc.10",
|
|
201
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/58",
|
|
202
|
-
"status": "failed-gates",
|
|
203
|
-
"channel": "next",
|
|
204
|
-
"tag": "v1.0.0-rc.10",
|
|
205
|
-
"contractBreakingDefects": null,
|
|
206
|
-
"publishedAt": null,
|
|
207
|
-
"sourceCommit": "a6cb74e99f9f449592187663d53a40f4f7a186db",
|
|
208
|
-
"artifactSha512": "sha512-UheGnKl0z6U8To2r/IeFvj27ZVynGE8OnSu9IKPNk2zcCzVi2s5nfNLqXie8Ly+U1BblLs4dt/26tJKlV4GSbA==",
|
|
209
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/33564054752",
|
|
210
|
-
"provenance": "not-published",
|
|
211
|
-
"liveCertification": "failed-release-bound-run",
|
|
212
|
-
"observedAt": "2026-09-01T22:17:10Z",
|
|
213
|
-
"failedGates": [
|
|
214
|
-
"live-provider-certification",
|
|
215
|
-
"representative-evaluation"
|
|
216
|
-
]
|
|
217
|
-
},
|
|
218
|
-
{
|
|
219
|
-
"version": "1.0.0-rc.11",
|
|
220
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/60",
|
|
221
|
-
"status": "passed",
|
|
222
|
-
"channel": "next",
|
|
223
|
-
"tag": "v1.0.0-rc.11",
|
|
224
|
-
"contractBreakingDefects": 0,
|
|
225
|
-
"publishedAt": "2026-09-02T05:37:17.346Z",
|
|
226
|
-
"sourceCommit": "0deb3e3f103eac5b4ccf38c585d8149b8472a356",
|
|
227
|
-
"artifactSha512": "sha512-U/wCcN0wWpfZdyDu8zOkrU6abTnHigGa6hEt9ivSxrc8q/ev8oIvKMszzt4B9P4sO6s6naAUQz5jruASNpWNTA==",
|
|
228
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/33592828351",
|
|
229
|
-
"provenance": "verified",
|
|
230
|
-
"liveCertification": "passed-release-bound-run"
|
|
231
|
-
},
|
|
232
|
-
{
|
|
233
|
-
"version": "1.0.0-rc.12",
|
|
234
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/32",
|
|
235
|
-
"status": "passed",
|
|
236
|
-
"channel": "next",
|
|
237
|
-
"tag": "v1.0.0-rc.12",
|
|
238
|
-
"contractBreakingDefects": 0,
|
|
239
|
-
"publishedAt": "2026-09-02T23:54:47.041Z",
|
|
240
|
-
"sourceCommit": "eb288bd3a0ef76dfbe630939d64e679437703bcb",
|
|
241
|
-
"artifactSha512": "sha512-Qpo7W9VmGvKmnX08Zr4NK15AWUuHRoMaYA9f0iAksGKxeQ3gGP2ntLv3xSwth1uJnknoT1XMMS7CtTFdy5i/eg==",
|
|
242
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/33695876968",
|
|
243
|
-
"provenance": "verified",
|
|
244
|
-
"liveCertification": "passed-release-bound-run"
|
|
245
|
-
},
|
|
246
|
-
{
|
|
247
|
-
"version": "1.0.0-rc.13",
|
|
248
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/32",
|
|
249
|
-
"status": "passed",
|
|
250
|
-
"channel": "next",
|
|
251
|
-
"tag": "v1.0.0-rc.13",
|
|
252
|
-
"contractBreakingDefects": 0,
|
|
253
|
-
"publishedAt": "2026-09-08T18:50:18.941Z",
|
|
254
|
-
"sourceCommit": "d6e310ce12fd3ee74db5bd40451f5af70056e3ce",
|
|
255
|
-
"artifactSha512": "sha512-EhKcjH44ZvRS1Y4POhs2xnPCmw0IFQo1trKf1gusYaFMkLthwXND0+ZEkfzG25JFCCf7kfOAZabDp35SoPnubw==",
|
|
256
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/34263591864",
|
|
257
|
-
"provenance": "verified",
|
|
258
|
-
"liveCertification": "passed-release-bound-run"
|
|
259
|
-
},
|
|
260
|
-
{
|
|
261
|
-
"version": "1.0.0-rc.14",
|
|
262
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/31",
|
|
263
|
-
"status": "passed",
|
|
264
|
-
"channel": "next",
|
|
265
|
-
"tag": "v1.0.0-rc.14",
|
|
266
|
-
"contractBreakingDefects": 0,
|
|
267
|
-
"publishedAt": "2026-09-20T12:47:20.607Z",
|
|
268
|
-
"sourceCommit": "d1c5abcc6d1fa482c211c4327c86f592663d559b",
|
|
269
|
-
"artifactSha512": "sha512-UMu4JFUNs/kOTJETZSJMMvVYlo+a4MKG6PoSvE313QxMEqK1z420AXVgPOGJ8jj2jp6mQud/Rvc7+ixKyH/4og==",
|
|
270
|
-
"workflowUrl": "https://github.com/Zhivex/zhivex-harness/actions/runs/35510484383",
|
|
271
|
-
"provenance": "verified",
|
|
272
|
-
"liveCertification": "passed-release-bound-run"
|
|
273
|
-
}
|
|
274
|
-
],
|
|
275
|
-
"blockers": [
|
|
276
|
-
{
|
|
277
|
-
"id": "public-json-contracts",
|
|
278
|
-
"status": "closed",
|
|
279
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/26",
|
|
280
|
-
"evidence": [
|
|
281
|
-
"contracts/public-api.json",
|
|
282
|
-
"fixtures/contracts/v1/doctor.json",
|
|
283
|
-
"fixtures/contracts/v1/init.json",
|
|
284
|
-
"fixtures/contracts/v1/observational-documents.json",
|
|
285
|
-
"fixtures/contracts/v1/providers.json",
|
|
286
|
-
"fixtures/contracts/v1/run-result.json",
|
|
287
|
-
"tests/json-contracts.test.ts"
|
|
288
|
-
]
|
|
289
|
-
},
|
|
290
|
-
{
|
|
291
|
-
"id": "cli-option-contract",
|
|
292
|
-
"status": "closed",
|
|
293
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/27",
|
|
294
|
-
"evidence": [
|
|
295
|
-
"scripts/package-smoke.ts",
|
|
296
|
-
"src/cli-options.ts",
|
|
297
|
-
"tests/cli-options.test.ts"
|
|
298
|
-
]
|
|
299
|
-
},
|
|
300
|
-
{
|
|
301
|
-
"id": "complete-error-taxonomy",
|
|
302
|
-
"status": "closed",
|
|
303
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/28",
|
|
304
|
-
"evidence": [
|
|
305
|
-
"scripts/package-smoke.ts",
|
|
306
|
-
"src/errors.ts",
|
|
307
|
-
"tests/errors.test.ts"
|
|
308
|
-
]
|
|
309
|
-
},
|
|
310
|
-
{
|
|
311
|
-
"id": "historical-migration-fixtures",
|
|
312
|
-
"status": "closed",
|
|
313
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/29",
|
|
314
|
-
"evidence": [
|
|
315
|
-
".github/workflows/ci.yml",
|
|
316
|
-
"fixtures/migrations/0.10.0.json",
|
|
317
|
-
"fixtures/migrations/0.10.0.sqlite",
|
|
318
|
-
"fixtures/migrations/0.11.1.json",
|
|
319
|
-
"fixtures/migrations/0.11.1.sqlite",
|
|
320
|
-
"scripts/verify-historical-migrations.ts"
|
|
321
|
-
]
|
|
322
|
-
},
|
|
323
|
-
{
|
|
324
|
-
"id": "consistent-state-backup",
|
|
325
|
-
"status": "closed",
|
|
326
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/30",
|
|
327
|
-
"evidence": [
|
|
328
|
-
"docs/CLI.md",
|
|
329
|
-
"src/state-backup.ts",
|
|
330
|
-
"tests/state-backup.test.ts"
|
|
331
|
-
]
|
|
332
|
-
},
|
|
333
|
-
{
|
|
334
|
-
"id": "representative-evaluation-evidence",
|
|
335
|
-
"status": "closed",
|
|
336
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/31",
|
|
337
|
-
"evidence": [
|
|
338
|
-
"docs/LIVE_CERTIFICATION.md",
|
|
339
|
-
"evaluations/representative-matrix.json"
|
|
340
|
-
]
|
|
341
|
-
},
|
|
342
|
-
{
|
|
343
|
-
"id": "security-review",
|
|
344
|
-
"status": "closed",
|
|
345
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/32",
|
|
346
|
-
"evidence": [
|
|
347
|
-
"docs/RC14_DELEGATED_SECURITY_DECISION.md",
|
|
348
|
-
"security-reviews/rc14-review.json"
|
|
349
|
-
]
|
|
350
|
-
},
|
|
351
|
-
{
|
|
352
|
-
"id": "gemini-ga-decision",
|
|
353
|
-
"status": "closed",
|
|
354
|
-
"issue": "https://github.com/Zhivex/zhivex-harness/issues/33",
|
|
355
|
-
"evidence": [
|
|
356
|
-
".github/workflows/release.yml",
|
|
357
|
-
"docs/GA_READINESS.md",
|
|
358
|
-
"docs/GEMINI_1_0_DECISION.md",
|
|
359
|
-
"docs/LIVE_CERTIFICATION.md",
|
|
360
|
-
"docs/support-matrix.json",
|
|
361
|
-
"evaluations/representative-matrix.json",
|
|
362
|
-
"src/providers.ts"
|
|
363
|
-
]
|
|
364
|
-
}
|
|
365
|
-
]
|
|
366
|
-
}
|
|
@@ -1,156 +0,0 @@
|
|
|
1
|
-
# Efficiency implementation evidence — 2026-09-08
|
|
2
|
-
|
|
3
|
-
> Historical snapshot: versions, findings and measurements below describe the recorded run, not the current checkout. See the [report index](README.md) for follow-up work and the [documentation index](../README.md) for maintained guides.
|
|
4
|
-
|
|
5
|
-
## SDK patch integration
|
|
6
|
-
|
|
7
|
-
The measured checkout pinned published Core 1.13.0, Agents 1.3.1 and OpenAI 0.11.2,
|
|
8
|
-
including the Core override and lockfile. The isolated three-response diagnostic
|
|
9
|
-
now reports 300 input tokens for 300 measured tokens. Consumer regressions exercise
|
|
10
|
-
persisted and unpersisted streams, compactor usage, per-step budgets, real approval
|
|
11
|
-
resumes and synthetic assistant serialization through mocked Responses HTTP.
|
|
12
|
-
|
|
13
|
-
The local usage-accounting correction and compacted-history transport projection
|
|
14
|
-
have been removed. The SDK preserves complete interaction groups and assistant
|
|
15
|
-
roles; the harness no longer drops orphan observations or changes summary roles.
|
|
16
|
-
The new runtime identity requires older paused runs to use their matching artifact.
|
|
17
|
-
Those checks are installed-package evidence. The subsequent live
|
|
18
|
-
[repeated-cohort comparison](./EXTERNAL_COMPARISON.md#sdk-upgrade-reevaluation--2026-09-08)
|
|
19
|
-
resolved 2/5 cases for Zhivex and 3/5 for mini-SWE-agent, with 19.4% fewer Zhivex
|
|
20
|
-
input tokens than its previous run. It does not establish general superiority.
|
|
21
|
-
The sections below describe the earlier development evidence and task creation.
|
|
22
|
-
|
|
23
|
-
Upgrade validation passed: 442 tests, documentation and contract checks, typecheck,
|
|
24
|
-
migrations, deterministic evaluations and local benchmark checks. MCP loopback
|
|
25
|
-
checks required execution outside the sandbox; both MCP smokes, real OCI execution
|
|
26
|
-
and installed-package smoke passed with that permission. The isolated SDK diagnostic
|
|
27
|
-
also passed with the published dependency versions and no provider calls.
|
|
28
|
-
|
|
29
|
-
## Follow-up: command selection and user corrections
|
|
30
|
-
|
|
31
|
-
Configured OCI executable allowlists now appear as enums in all four command
|
|
32
|
-
tool schemas, including batch and verified-edit transactions. Descriptions expose
|
|
33
|
-
the configured choices and the Python module invocation for pytest when Python is
|
|
34
|
-
allowed. This does not broaden process permissions or guarantee module installation;
|
|
35
|
-
the execution session still enforces policy and exact approval. Calls to the public
|
|
36
|
-
factory without a configuration retain the legacy schema and direct the model to
|
|
37
|
-
`environment_status`.
|
|
38
|
-
|
|
39
|
-
Compaction strategy `bounded-evidence-v2` retains three bounded user corrections
|
|
40
|
-
separately from assistant excerpts. A regression runs five compactions with twenty
|
|
41
|
-
assistant messages between each, preserving the correction inside the same 4,000
|
|
42
|
-
character summary ceiling. This is deterministic continuity evidence, not a new
|
|
43
|
-
external benchmark result. Previous paused runs require their matching artifact.
|
|
44
|
-
|
|
45
|
-
The SDK team backlog in Notion's Zhivex area contains these verified-created tasks:
|
|
46
|
-
|
|
47
|
-
- [Persisted stream usage duplication](https://app.notion.com/p/3d5777b104f6810a8e46df7186bf5e21):
|
|
48
|
-
reproducible offline using `bun --no-env-file run scripts/diagnose-sdk-compaction.ts`.
|
|
49
|
-
Core 1.11.0 reports 500 input tokens for three measured responses of 100 when a
|
|
50
|
-
store and compaction are enabled. Without a store the isolated control reports 300.
|
|
51
|
-
- [Synthetic assistant serialization](https://app.notion.com/p/3d5777b104f6818db026dea3738f9b5b):
|
|
52
|
-
OpenAI 0.10.0 transport contract issue; the earlier harness projected summaries
|
|
53
|
-
into user context.
|
|
54
|
-
- [Approval/tool group boundaries](https://app.notion.com/p/3d5777b104f6814b80f2d46269c9453c):
|
|
55
|
-
code inspection and transport regression; the SDK task explicitly requires an
|
|
56
|
-
integrated approval/compaction reproduction before implementing an upstream fix.
|
|
57
|
-
|
|
58
|
-
At creation all three were tagged SDK-Typescript, priority Alta, status Sin empezar.
|
|
59
|
-
Their SDK fixes are now integrated as described above; this historical record does
|
|
60
|
-
not assert their current Notion workflow status.
|
|
61
|
-
|
|
62
|
-
Follow-up validation: 429 tests passed, plus documentation, stable contracts,
|
|
63
|
-
typecheck, migrations, deterministic evaluations and local benchmark checks.
|
|
64
|
-
The sandbox blocked the loopback MCP listener; MCP interoperability (controlled
|
|
65
|
-
and official SDK), real Docker OCI and installed-package smoke subsequently passed
|
|
66
|
-
with the required local execution permission. The isolated SDK diagnostic was also
|
|
67
|
-
typechecked and executed without provider calls. No new external score is claimed.
|
|
68
|
-
|
|
69
|
-
Implemented on top of `0a7b69dddaa70a54b6b5b4097691c8112d4d76eb` as an
|
|
70
|
-
uncommitted development change. Compact source-report digests and aggregate
|
|
71
|
-
measurements are recorded in [the baseline](../../benchmarks/baselines/efficiency-2026-09-08.json).
|
|
72
|
-
Full reports remain local under `results/efficiency/2026-09-08/`.
|
|
73
|
-
|
|
74
|
-
| Measurement (p50) | Before | After |
|
|
75
|
-
| --- | ---: | ---: |
|
|
76
|
-
| Three searches, 10,000 small files | 1,136.79 ms | 492.41 ms |
|
|
77
|
-
| `searchMany`, 10,000 small files | 941.46 ms | 232.24 ms |
|
|
78
|
-
| Digest inventory, 10,000 small files | 970.63 ms | 245.74 ms |
|
|
79
|
-
| Reused topology inventory, 100 nested groups / 1,000 files | 27.61 ms | 10.32 ms |
|
|
80
|
-
| OCI session acquisition, 1,000 source files | 417.04 ms | 193.16 ms |
|
|
81
|
-
|
|
82
|
-
Three repetitions per measurement, Bun 1.4.0 on local macOS ARM64, filesystem
|
|
83
|
-
cache not flushed. These are sequential development observations, not randomized
|
|
84
|
-
paired trials. The nested and OCI before/after measurements isolate later stages:
|
|
85
|
-
both already include concurrent content reads; OCI after additionally includes
|
|
86
|
-
batched index freshness and snapshot construction. No model latency or human
|
|
87
|
-
approval time is represented by these workspace numbers. No p99 claim is warranted.
|
|
88
|
-
|
|
89
|
-
Validation: full `bun run check` passed, including 414 tests, historical migrations,
|
|
90
|
-
deterministic evaluations, workspace/safe-fix benchmarks, MCP interoperability,
|
|
91
|
-
real Docker 29.7.2 OCI smoke, and the installed package smoke. Tests exercise batch
|
|
92
|
-
failure draining, early termination, deduplication, aggregate limits, symlinks,
|
|
93
|
-
Unicode pagination, changed file contents, repeated compaction and redaction.
|
|
94
|
-
The Stable declaration snapshot update reflects only TypeScript's reordered
|
|
95
|
-
`fulfilled`/`rejected` enum members, checked against declarations emitted from the
|
|
96
|
-
base checkout. It does not add or remove a public API member.
|
|
97
|
-
|
|
98
|
-
Reproduce workspace scale with `bun run benchmark:workspace:scale`, continuity with
|
|
99
|
-
`bun run evaluate:continuity`, and OCI with `bun run benchmark:oci`. See the
|
|
100
|
-
[benchmark guide](../../benchmarks/README.md) for limits and competitive methodology.
|
|
101
|
-
|
|
102
|
-
Remaining research: persistent content indexing, semantic plans/constraint retention,
|
|
103
|
-
automatic model selection and cross-tool concurrency. Those mechanisms were not
|
|
104
|
-
introduced by this patch. External held-out agent evaluations still require a
|
|
105
|
-
selected competing implementation, dataset and matched runtime/model budgets; the
|
|
106
|
-
existing external-driver interface is available but no competitor was executed.
|
|
107
|
-
|
|
108
|
-
## External-pilot follow-up
|
|
109
|
-
|
|
110
|
-
The follow-up adds exact approved replacements to avoid generating full files,
|
|
111
|
-
reduces default read/search volume, preserves the objective through SDK-formatted
|
|
112
|
-
compaction, and fixes model-transport context at compaction boundaries. OCI's
|
|
113
|
-
inventory uses the snapshot byte budget independently of the model's file-read
|
|
114
|
-
limit, allowing large unchanged fixture files. The authority inventory and enforced
|
|
115
|
-
OCI tool list both include the new replacement tool.
|
|
116
|
-
|
|
117
|
-
One previously failing development task produced an officially accepted repair
|
|
118
|
-
with two compacted continuations after these fixes. That run subsequently exhausted
|
|
119
|
-
its input budget after already importing the patch; the frozen comparison ends on
|
|
120
|
-
the approved import receipt. This development result is not holdout evidence.
|
|
121
|
-
See [the external comparison](EXTERNAL_COMPARISON.md) for the independent selection.
|
|
122
|
-
|
|
123
|
-
## Qwen repair follow-up
|
|
124
|
-
|
|
125
|
-
The subsequent implementation adds exact-file batched searches, character-bounded
|
|
126
|
-
read output, earlier benchmark compaction, remaining-budget reminders and required
|
|
127
|
-
verification before native benchmark import. Command telemetry retains typed exit
|
|
128
|
-
codes and timeouts without raw logs. Qwen adapter 0.11.4 incorporates the upstream
|
|
129
|
-
streamed-usage fix. Validation passed 446 tests and the MCP, OCI and installed
|
|
130
|
-
package checks.
|
|
131
|
-
|
|
132
|
-
The repeated five-case comparison using `qwen3.8-flash` with thinking disabled
|
|
133
|
-
resolved 0/5 for Zhivex and 2/5 for mini-SWE-agent. Native input usage was 424,720
|
|
134
|
-
tokens: three attempts reached the budget, one failed argument validation and one
|
|
135
|
-
failed final verification (exit 4), preventing import. This is no demonstrated
|
|
136
|
-
competitive improvement. Model and harness changes are confounded, and the cases
|
|
137
|
-
are known development data. See the [full result and limitations](EXTERNAL_COMPARISON.md#qwen-flash-repair-reevaluation--2026-09-08).
|
|
138
|
-
|
|
139
|
-
## Navigation retention follow-up — 2026-09-09
|
|
140
|
-
|
|
141
|
-
The harness compactor previously ignored nested file references in batched search
|
|
142
|
-
and read outputs. Strategy `bounded-evidence-v3` now retains up to eight validated,
|
|
143
|
-
deduplicated path/digest/line references within the existing total summary budget.
|
|
144
|
-
New observations with a changed digest replace older locations for that file.
|
|
145
|
-
Current source must still be read before editing; historical locations do not
|
|
146
|
-
authorize edits or stand in for source content.
|
|
147
|
-
|
|
148
|
-
The [deterministic control](../../benchmarks/baselines/context-navigation-2026-09-09.json)
|
|
149
|
-
retains two locations through six successive compactions, compared with zero in
|
|
150
|
-
the archived v2 implementation. On this small fixture the summary grows from 225
|
|
151
|
-
to 553 characters, still below the 4,000-character cap. This is improved information
|
|
152
|
-
retention, not measured token savings. The intended benefit is avoiding repository
|
|
153
|
-
rediscovery after compaction; a fixed-model live comparison is still needed to
|
|
154
|
-
measure that benefit. Regression tests use actual workspace batched outputs and
|
|
155
|
-
cover redaction, stale references, clipped lines, hostile recalled metadata and
|
|
156
|
-
small summary budgets. The local suite passes 456 tests.
|