sortie-dogs 0.10.3 → 0.10.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -74,7 +74,7 @@ bounded implementation, canonical validation, and evidence-backed completion.
74
74
 
75
75
  Guides: [日本語](docs/guide-ja.md) · [简体中文](docs/guide-zh-CN.md) · [テスト実行](docs/testing.md) · [CLI testing](docs/cli-testing.md)
76
76
 
77
- Release: [v0.10.3](https://github.com/zufall-upon/Sortie-dogs/releases/tag/v0.10.3)
77
+ Release: [v0.10.4](https://github.com/zufall-upon/Sortie-dogs/releases/tag/v0.10.4)
78
78
 
79
79
  ## Latest local benchmark case study
80
80
 
@@ -92,20 +92,20 @@ Run configuration was fixed per product configuration:
92
92
  children on `openai/gpt-5.6-sol` / `medium`, with the pinned Sortie package and runtime assets.
93
93
  No Luna, Astra, or Opus messages were observed in these trials.
94
94
 
95
- | Metric · one frozen task | Bare OpenCode | Sortie v0.9.12 | Sortie v0.10.1 RC qualification |
96
- | --- | ---: | ---: | ---: |
97
- | Attempts needed | 3 | 5 | 1 |
98
- | Completed runs compared | 3 | 3 | 1 |
99
- | Verified PASS | 0/3 | 0/3 | 0/1 |
100
- | Task checks · F2P | 11.1% · 3/27 | 85.2% · 23/27 | 88.9% · 8/9 |
101
- | Retained checks · P2P | 282/282 | 282/282 | 94/94 |
102
- | Median agent wall | 24.5 min | 25.7 min | 23.2 min · n=1 |
103
- | Median model steps | 43 | 39 | 41 · CLI stream only |
104
- | Implementation child sessions · total | 0 | 13 | 3 |
105
- | Estimated API-equivalent cost · median completed run | $3.53 | $2.85 | **$4.48** · n=1 |
106
- | Estimated cost · completed runs | $10.69 | $9.74 | **$4.48** |
107
- | Additional interrupted-attempt cost · same fixed snapshot | $0 | $6.20 | $0 |
108
- | Total cost to acquire completed runs · same fixed snapshot | $10.69 | $15.94 | **$4.48** |
95
+ | Metric · one frozen task | Bare OpenCode | Sortie v0.9.12 | Sortie v0.10.1 RC qualification | Sortie v0.10.3 qualification · Terra/xhigh | Sortie v0.10.4 qualification · Terra/xhigh |
96
+ | --- | ---: | ---: | ---: | ---: | ---: |
97
+ | Attempts needed | 3 | 5 | 1 | 1 | 2 |
98
+ | Completed runs compared | 3 | 3 | 1 | 1 | 2 |
99
+ | Verified PASS | 0/3 | 0/3 | 0/1 | 0/1 | 0/2 |
100
+ | Task checks · F2P | 11.1% · 3/27 | 85.2% · 23/27 | 88.9% · 8/9 | 77.8% · 7/9 | 72.2% · 5/9 and 8/9 |
101
+ | Retained checks · P2P | 282/282 | 282/282 | 94/94 | 94/94 | 94/94 |
102
+ | Median agent wall | 24.5 min | 25.7 min | 23.2 min · n=1 | 22.7 min · n=1 | 31.1 min · n=2 |
103
+ | Median model steps | 43 | 39 | 41 · CLI stream only | 36 · CLI stream only | 44 · CLI stream only |
104
+ | Implementation child sessions · total | 0 | 13 | 3 | 5 | 10 |
105
+ | Estimated API-equivalent cost · median completed run | $3.53 | $2.85 | **$4.48** · n=1 | **$3.79** · n=1 | **$3.82** · n=2 |
106
+ | Estimated cost · completed runs | $10.69 | $9.74 | **$4.48** | **$3.79** | **$7.64** |
107
+ | Additional interrupted-attempt cost · same fixed snapshot | $0 | $6.20 | $0 | $0 | $0 |
108
+ | Total cost to acquire completed runs · same fixed snapshot | $10.69 | $15.94 | **$4.48** | **$3.79** | **$7.64** |
109
109
 
110
110
  All three Bare runs passed 1/9 task checks. The three completed Sortie runs passed 7/9, 8/9,
111
111
  and 8/9. Every compared candidate retained 94/94 prior checks, but every official verifier still
@@ -128,12 +128,38 @@ into this RC's same-snapshot acquisition cost. Observed spend across both distin
128
128
  **$8.755191**, which is development spend rather than this RC's acquisition cost. Historical Bare and
129
129
  v0.9.12 costs retain their 2026-07-30 schedule; their cost cells are not same-rate comparisons with
130
130
  this RC estimate.
131
-
132
- ![Latest local case study: Bare completed 11.1 percent of task checks at a median estimated API-equivalent cost of $3.53; Sortie completed 85.2 percent at $2.85. Sortie needed five attempts and $15.94 to collect three completed runs. Neither configuration achieved a Verified PASS.](docs/assets/quality-cost-reference.svg)
131
+
132
+ The released **v0.10.3** Terra/xhigh treatment is a separate one-shot qualification, not a matched
133
+ comparison or leaderboard result. It reached `DONE` and the localized Docker-free official verifier,
134
+ which returned reward **0**, F2P **7/9**, and P2P **94/94**. Its full root-plus-eight-descendant audit
135
+ prices 93 assistant requests and 4,809,706 tokens at **$3.789190** API-equivalent cost: **$2.186540**
136
+ for Terra/xhigh and **$1.602650** for Sol/low. Pricing coverage is 100%, with no unpriced or pending
137
+ requests. This uses the same 2026-09-14 per-request schedule; the observed host cost of $0 is not used
138
+ as cost evidence, and the estimate is not an invoice or subscription charge.
139
+
140
+ The released **v0.10.4** Terra/xhigh treatment repeats that qualification twice on the same frozen
141
+ task, pins, and verifier. Both runs reached `DONE` with confirmed cleanup, and the localized
142
+ Docker-free official verifier returned reward **0** with P2P **94/94** each time. F2P was **5/9** and
143
+ **8/9**, so the single v0.10.3 observation of 7/9 sits inside this spread: neither release solves the
144
+ task, and the per-run difference is run-to-run variance at n=1 and n=2, not a measured quality change.
145
+
146
+ The release payload this qualification exists to check is a prompt-cache correctness fix, and that
147
+ effect is separable from task quality. Before the fix the proposal investigation child re-sent its
148
+ whole prompt uncached on every request, because a consumed-read counter sat in the system block and a
149
+ system element is an absolute prompt prefix. The audit now reports per-session prefix reuse, which
150
+ isolates exactly that failure: the v0.10.3 proposal child reused a median **0.065** of the previous
151
+ prompt and was the only flagged session in its run, while both v0.10.4 runs reuse **0.957** and
152
+ **0.986** and flag none. Its uncached input fell from **406,202** tokens to **213,115** and
153
+ **285,060**, and its estimated cost fell from **$0.943508** to **$0.574788** and **$0.802139**. Whole-run
154
+ cost is not a clean measure of the same fix, because each run performed a different amount of work:
155
+ the two v0.10.4 runs priced 97 requests at **$3.268415** and 119 requests at **$4.368122**, both at
156
+ 100% pricing coverage on the 2026-09-14 per-request schedule.
157
+
158
+ ![Latest local case study: Bare completed 11.1 percent of task checks at a median estimated API-equivalent cost of $3.53; Sortie v0.9.12 completed 85.2 percent at $2.85. The one-shot v0.10.3 Terra/xhigh qualification completed 77.8 percent at $3.79. Neither configuration achieved a Verified PASS.](docs/assets/quality-cost-reference.svg)
133
159
 
134
160
  Cost audits use deduplicated root and descendant session tokens, grouped by the model that produced
135
- each message. Historical values retain their frozen 2026-07-30 short-context schedule; the RC uses
136
- the product's 2026-09-14 per-request schedule described above. Completed-run cost shows execution
161
+ each message. Historical values retain their frozen 2026-07-30 short-context schedule; the v0.10
162
+ qualifications use the product's 2026-09-14 per-request schedule described above. Completed-run cost shows execution
137
163
  efficiency; same-snapshot acquisition cost adds interrupted attempts. These are API-equivalent
138
164
  estimates, not invoices.
139
165
 
@@ -1109,6 +1109,15 @@ export function createProfiledPlugin(profile, assetVersion) {
1109
1109
  throw new Error("operator-contract-repair-validation-command-missing");
1110
1110
  await operators.recordRepairValidationResult((await rootFor(id)), id, repairAccess.expected_command, exit);
1111
1111
  }
1112
+ if (!ownership && request.tool.toLowerCase() === "read") {
1113
+ const proposal = await proposals.read((await rootFor(id)));
1114
+ if (proposal?.phase === "investigating" && proposal.proposal_session_id === id) {
1115
+ output.output = `${output.output ?? ""}\n\nSORTIE_PROPOSAL_BUDGET actual_reads=${proposal.read_count}; ` +
1116
+ `remaining_reads=${proposal.intent.proposal_budget.max_reads - proposal.read_count}; ` +
1117
+ `submissions=${proposal.submission_count}; ` +
1118
+ `remaining_submissions=${proposal.intent.proposal_budget.max_submissions - proposal.submission_count}.`;
1119
+ }
1120
+ }
1112
1121
  if (ownership) {
1113
1122
  const child = taskChildSessionID(output);
1114
1123
  if (child)
@@ -1165,11 +1174,16 @@ export function createProfiledPlugin(profile, assetVersion) {
1165
1174
  "Never rewrite user acceptance or evidence to rename protocol roles. Final acceptance belongs only to the root coordinator.");
1166
1175
  const proposal = await proposals.read(root);
1167
1176
  if (proposal?.phase === "investigating" && proposal.proposal_session_id === request.sessionID) {
1177
+ // Only immutable identity and the frozen budget caps belong here. Consumed counters move with
1178
+ // every accounted read, and a system element is an absolute prompt prefix: restating them here
1179
+ // invalidated the whole cached prefix on every later request of the same investigation, so the
1180
+ // Task prompt and all accumulated reads were re-billed uncached. They are reported on the read
1181
+ // result instead, which is appended after the stable prefix.
1168
1182
  (output.system ??= []).push(`SORTIE_PROPOSAL_PHASE investigating; intent=${proposal.intent_id}; root=${root}; child=${request.sessionID}. ` +
1169
1183
  `This durable phase remains authoritative after compaction even when the latest message is a generic continuation. ` +
1170
1184
  `Continue the admitted read-only investigation and submit through ${submitProposal}. Do not call ${next} or dispatch workers: no execution run exists yet. ` +
1171
- `actual_reads=${proposal.read_count}; remaining_reads=${proposal.intent.proposal_budget.max_reads - proposal.read_count}; ` +
1172
- `submissions=${proposal.submission_count}; remaining_submissions=${proposal.intent.proposal_budget.max_submissions - proposal.submission_count}.`);
1185
+ `max_reads=${proposal.intent.proposal_budget.max_reads}; max_submissions=${proposal.intent.proposal_budget.max_submissions}. ` +
1186
+ `Consumed budget is reported as SORTIE_PROPOSAL_BUDGET on each read result and by ${submitProposal}; never infer it from this element.`);
1173
1187
  }
1174
1188
  },
1175
1189
  "experimental.text.complete": async (request, output) => {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "sortie-dogs",
3
- "version": "0.10.3",
3
+ "version": "0.10.4",
4
4
  "description": "Bounded agent harness and validated orchestration loop plugin for OpenCode",
5
5
  "keywords": [
6
6
  "opencode",