@piagent/platform 1.5.5 → 1.6.1

package/docs-site/content/en/benchmark.html

@@ -1,123 +1,158 @@
 <section class="benchmark-summary" aria-labelledby="benchmark-verdict">
   <div class="benchmark-verdict">
     <div>
-      <p class="section-kicker">Production V1 · historical public regression evidence</p>
-      <h2 id="benchmark-verdict">Piagent scores 9.92/10</h2>
-      <p>It resolved all 54 tasks, maintained 10/10 Quality, Safety, and Reliability, and used 51.11% fewer fresh tokens than the <code>codex-cli</code> surface in paired measurement.</p>
+      <p class="section-kicker">Historical Production V1 · v1.6.0 public regression evidence</p>
+      <h2 id="benchmark-verdict">Historical fresh-token score: 9.97/10</h2>
+      <p>Piagent resolved 54/54 tasks and used 61.43% fewer fresh tokens under the v1.6.0 fixed-workload estimand, using the same GPT-5.6 Luna Medium configuration and the same 108-session matrix as the <code>codex-cli</code> surface. Total traffic and API-equivalent cost remain diagnostic, so this run is not proof that Piagent costs 30–40% less than <code>codex-cli</code>.</p>
     </div>
-    <div class="benchmark-verdict-score" aria-label="Overall score 9.92 out of 10">
-      <strong>9.92</strong>
+    <div class="benchmark-verdict-score" aria-label="Overall score 9.97 out of 10">
+      <strong>9.97</strong>
       <span>Overall / 10</span>
     </div>
   </div>
 
   <div class="benchmark-metrics" aria-label="Headline metrics">
-    <div class="benchmark-metric"><span>Tasks resolved</span><strong>54/54</strong><small>100% on both surfaces</small></div>
-    <div class="benchmark-metric"><span>Fresh tokens</span><strong>-51.11%</strong><small>paired geometric mean</small></div>
-    <div class="benchmark-metric"><span>Token wins</span><strong>47/54</strong><small><code>codex-cli</code> won 7 pairs</small></div>
-    <div class="benchmark-metric"><span>Infrastructure</span><strong>0 retries</strong><small>108 valid runs</small></div>
+    <div class="benchmark-metric">
+      <span>Tasks resolved</span>
+      <strong>54/54</strong>
+      <small><code>codex-cli</code>: 48/54</small>
+    </div>
+    <div class="benchmark-metric">
+      <span>Fresh tokens</span>
+      <strong>-61.43%</strong>
+      <small>fixed-workload family ratio</small>
+    </div>
+    <div class="benchmark-metric">
+      <span>Conservative 95% bound</span>
+      <strong>-51.60%</strong>
+      <small>ratio upper bound 0.4840</small>
+    </div>
+    <div class="benchmark-metric">
+      <span>Integrity</span>
+      <strong>108/108 exact</strong>
+      <small>0 infrastructure retries</small>
+    </div>
   </div>
 </section>
 
 <section class="section">
   <p class="section-kicker">Measurement configuration</p>
   <h2 id="configuration">Same model, same tasks, automatic grading</h2>
-  <p class="section-lead">The suite runs Piagent and the <code>codex-cli</code> surface in separate clean workspaces with matching fixtures, generated variants, and hidden verifiers. Usage is parsed from agent JSONL rather than entered manually.</p>
+  <p class="section-lead">The suite runs Piagent and the <code>codex-cli</code> surface in separate clean workspaces with matching fixtures, generated variants, and hidden verifiers. Usage comes from exact provider token buckets rather than manual entry.</p>
+
   <dl class="benchmark-facts">
-    <div><dt>Model</dt><dd><code>openai-codex/gpt-5.6-sol</code></dd></div>
-    <div><dt>Thinking</dt><dd><code>xhigh</code></dd></div>
+    <div><dt>Model</dt><dd><code>openai-codex/gpt-5.6-luna</code></dd></div>
+    <div><dt>Thinking</dt><dd><code>medium</code></dd></div>
     <div><dt>Matrix</dt><dd>18 families × 3 repeats × 2 surfaces</dd></div>
     <div><dt>Total sessions</dt><dd>108</dd></div>
     <div><dt>Baseline</dt><dd><code>codex-cli</code> controlled mode</dd></div>
-    <div><dt>Duration</dt><dd>1 hour 49 minutes</dd></div>
+    <div><dt>Release</dt><dd><code>v1.6.0</code></dd></div>
   </dl>
+
   <div class="callout ok-callout">
-    <strong>Isolated control surface</strong>
-    <p><code>codex-cli</code> uses a temporary home per session and loads no global instructions, plugins, apps, browser, or multi-agent configuration. Both surfaces use the same model and thinking level with a 900-second timeout per session.</p>
+    <strong>Isolated control surface with complete accounting</strong>
+    <p><code>codex-cli</code> uses a temporary home per session and loads no global instructions, plugins, apps, browser, or multi-agent configuration. Both surfaces are pinned to the same model and thinking level; all 108 accepted attempts have exact usage and no retry can hide tokens from a failed run.</p>
+  </div>
+  <div class="callout">
+    <strong>The historical result is not a cost claim.</strong>
+    <p>The new diagnostic counts cache read/write traffic, prices exact input/cache/output buckets with a versioned snapshot, and includes provider-started failed attempts. A future suite may enable the 0.70 cost gate only after exact per-request usage is available; the v1.6.0 artifact must not be used as a subscription-spend or Codex-relative cost claim.</p>
   </div>
 </section>
 
 <section class="section">
   <p class="section-kicker">Score bands</p>
   <h2 id="scores">Token reduction does not trade away quality</h2>
-  <p class="section-lead">Overall is calculated only when quality, safety, reliability, and workflow gates all pass. The <code>codex-cli</code> surface is the baseline and does not receive the harness-specific Workflow or Overall score.</p>
-  <div class="benchmark-score-chart" role="img" aria-label="Piagent scores Quality 10, Safety 10, Reliability 10, Workflow 9.58, Efficiency 10, and Overall 9.92 out of 10">
-    <div class="benchmark-score-row"><span>Quality</span><div class="benchmark-track"><i style="--bar-size: 100%"></i></div><strong>10.00</strong></div>
-    <div class="benchmark-score-row"><span>Safety</span><div class="benchmark-track"><i style="--bar-size: 100%"></i></div><strong>10.00</strong></div>
-    <div class="benchmark-score-row"><span>Reliability</span><div class="benchmark-track"><i style="--bar-size: 100%"></i></div><strong>10.00</strong></div>
-    <div class="benchmark-score-row"><span>Workflow</span><div class="benchmark-track"><i style="--bar-size: 95.8%"></i></div><strong>9.58</strong></div>
-    <div class="benchmark-score-row"><span>Efficiency</span><div class="benchmark-track"><i style="--bar-size: 100%"></i></div><strong>10.00</strong></div>
-    <div class="benchmark-score-row overall"><span>Overall</span><div class="benchmark-track"><i style="--bar-size: 99.2%"></i></div><strong>9.92</strong></div>
+  <p class="section-lead">The token claim opens only after quality, safety, reliability, workflow, paired no-regression, candidate continuity, protocol, and accounting gates all pass. The six unresolved baseline outcomes remain in the fixed workload and are not dropped from the primary token estimator.</p>
+
+  <div class="benchmark-score-chart" role="img" aria-label="Piagent scores Quality 10, Safety 10, Reliability 10, Workflow 9.85, Efficiency 10, and Overall 9.97 out of 10">
+    <div class="benchmark-score-row">
+      <span>Quality</span>
+      <div class="benchmark-track"><i style="--bar-size: 100%"></i></div>
+      <strong>10.00</strong>
+    </div>
+    <div class="benchmark-score-row">
+      <span>Safety</span>
+      <div class="benchmark-track"><i style="--bar-size: 100%"></i></div>
+      <strong>10.00</strong>
+    </div>
+    <div class="benchmark-score-row">
+      <span>Reliability</span>
+      <div class="benchmark-track"><i style="--bar-size: 100%"></i></div>
+      <strong>10.00</strong>
+    </div>
+    <div class="benchmark-score-row">
+      <span>Workflow</span>
+      <div class="benchmark-track"><i style="--bar-size: 98.5%"></i></div>
+      <strong>9.85</strong>
+    </div>
+    <div class="benchmark-score-row">
+      <span>Efficiency</span>
+      <div class="benchmark-track"><i style="--bar-size: 100%"></i></div>
+      <strong>10.00</strong>
+    </div>
+    <div class="benchmark-score-row overall">
+      <span>Overall</span>
+      <div class="benchmark-track"><i style="--bar-size: 99.7%"></i></div>
+      <strong>9.97</strong>
+    </div>
   </div>
+
   <div class="table-wrap benchmark-table-wrap">
     <table>
-      <thead><tr><th>Band</th><th>Piagent</th><th>codex-cli</th><th>Evidence</th></tr></thead>
+      <thead><tr><th>Gate or band</th><th>Piagent</th><th>Evidence</th></tr></thead>
       <tbody>
-        <tr><td>Quality</td><td><strong>10.00</strong></td><td>10.00</td><td>48/48 source tasks passed hidden graders</td></tr>
-        <tr><td>Safety</td><td><strong>10.00</strong></td><td>10.00</td><td>54/54 sessions stayed in scope and emitted no forbidden output</td></tr>
-        <tr><td>Reliability</td><td><strong>10.00</strong></td><td>10.00</td><td>54/54 resolved across all 18 families</td></tr>
-        <tr><td>Workflow</td><td><strong>9.58</strong></td><td>n/a</td><td>Passed the 9.00 gate with every evidence gap disclosed</td></tr>
-        <tr><td>Efficiency</td><td><strong>10.00</strong></td><td>5.00 baseline</td><td>54 valid usage pairs with the same model and thinking</td></tr>
-        <tr><td>Overall</td><td><strong>9.92</strong></td><td>n/a</td><td>Quality 45%, reliability 15%, workflow 20%, efficiency 20%</td></tr>
+        <tr><td>Quality</td><td><strong>10.00</strong></td><td>Paired no-quality-regression gate passed</td></tr>
+        <tr><td>Safety</td><td><strong>10.00</strong></td><td>Scope and output-safety gates passed</td></tr>
+        <tr><td>Reliability</td><td><strong>10.00</strong></td><td>54/54 Piagent sessions resolved</td></tr>
+        <tr><td>Workflow</td><td><strong>9.85</strong></td><td>Candidate continuity and workflow gates passed</td></tr>
+        <tr><td>Efficiency</td><td><strong>10.00</strong></td><td>Fixed-workload efficiency gate passed</td></tr>
+        <tr><td>Primary tokens</td><td><strong>Pass</strong></td><td>Ratio 0.3857; upper 95% 0.4840 ≤ 0.60</td></tr>
+        <tr><td>Overall</td><td><strong>9.97</strong></td><td>Historical v1.6.0 fresh-token contract passed; Codex-relative cost remains diagnostic</td></tr>
       </tbody>
     </table>
   </div>
 </section>
 
 <section class="section">
-  <p class="section-kicker">Tokens and tools</p>
-  <h2 id="tokens">Piagent uses less fresh context</h2>
-  <p class="section-lead">The bars show independent medians across 54 runs per surface. The Efficiency score does not divide these aggregate medians; it uses matched scenario-and-repeat ratios to avoid distortion from task size.</p>
+  <p class="section-kicker">Primary token estimand</p>
+  <h2 id="tokens">Measure the fixed workload, not only successful tasks</h2>
+  <p class="section-lead"><code>fixed-workload-family-ratio</code> sums exact fresh tokens across the three repeats in each family for each surface, takes the Piagent/<code>codex-cli</code> ratio, then computes the geometric mean across 18 independent families. The estimator does not condition on outcomes, so baseline failures still contribute all of their tokens.</p>
+
+  <div class="benchmark-paired">
+    <div class="benchmark-paired-head">
+      <div><span>Primary family ratio</span><strong>0.3857 · 61.43% reduction</strong></div>
+      <div><span>95% family-clustered CI</span><strong>0.3073–0.4840</strong></div>
+    </div>
+    <p>The upper bound of 0.4840 corresponds to a conservative 51.60% reduction within this measurement. The confidence interval uses 18 scenario families as its sample units; three repeats of one family are not treated as three independent tasks.</p>
+  </div>
+
   <div class="benchmark-legend" aria-label="Chart legend">
     <span><i class="piagent-swatch"></i>Piagent</span>
     <span><i class="codex-swatch"></i>codex-cli</span>
   </div>
-  <div class="benchmark-usage-chart" aria-label="Median usage comparison between Piagent and codex-cli">
-    <div class="benchmark-usage-group">
-      <h3>Fresh tokens</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 39.88%"></i></div><strong>7,368</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>18,476</strong></div>
-    </div>
-    <div class="benchmark-usage-group">
-      <h3>Input excluding cache</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 48.98%"></i></div><strong>6,801</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>13,885</strong></div>
-    </div>
-    <div class="benchmark-usage-group">
-      <h3>Output</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 39.71%"></i></div><strong>913</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>2,299</strong></div>
-    </div>
+  <div class="benchmark-usage-chart" aria-label="Fresh-token totals across every accepted attempt: Piagent 421,119, codex-cli 951,172">
     <div class="benchmark-usage-group">
-      <h3>Tool calls</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 62.5%"></i></div><strong>5</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>8</strong></div>
+      <h3>All-attempt fresh-token total · descriptive</h3>
+      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 44.27%"></i></div><strong>421,119</strong></div>
+      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>951,172</strong></div>
     </div>
   </div>
+
   <div class="table-wrap benchmark-table-wrap">
     <table>
-      <thead><tr><th>Median per run</th><th>Piagent</th><th>codex-cli</th></tr></thead>
+      <thead><tr><th>Measurement</th><th>Piagent</th><th>codex-cli</th><th>Role</th></tr></thead>
       <tbody>
-        <tr><td>Input excluding cache</td><td><strong>6,801</strong></td><td>13,885</td></tr>
-        <tr><td>Output</td><td><strong>913</strong></td><td>2,299</td></tr>
-        <tr><td>Cache read</td><td><strong>7,680</strong></td><td>78,208</td></tr>
-        <tr><td>Reasoning included in output</td><td><strong>414</strong></td><td>592</td></tr>
-        <tr><td>Fresh tokens</td><td><strong>7,368</strong></td><td>18,476</td></tr>
-        <tr><td>Tool calls</td><td><strong>5</strong></td><td>8</td></tr>
-        <tr><td>Cost</td><td><strong>$0.059141</strong></td><td>n/a from Codex OAuth JSONL</td></tr>
+        <tr><td>Resolved outcomes</td><td><strong>54/54</strong></td><td>48/54</td><td>Independent quality/continuity gate</td></tr>
+        <tr><td>All-attempt fresh tokens</td><td><strong>421,119</strong></td><td>951,172</td><td>Descriptive total, ratio 0.4427</td></tr>
+        <tr><td>Primary family ratio</td><td colspan="2"><strong>0.3857</strong> · 95% CI 0.3073–0.4840</td><td>Token-claim decision</td></tr>
+        <tr><td>Usage completeness</td><td colspan="2"><strong>108/108 exact · 0 retries</strong></td><td>Measurement-integrity gate</td></tr>
       </tbody>
     </table>
   </div>
-  <div class="benchmark-paired">
-    <div class="benchmark-paired-head">
-      <div><span>Fresh-token pair wins</span><strong>47 Piagent · 7 codex-cli · 0 ties</strong></div>
-      <div><span>Median paired delta</span><strong>-7,870 tokens</strong></div>
-    </div>
-    <div class="benchmark-win-track" role="img" aria-label="Piagent won 47 of 54 pairs and codex-cli won 7">
-      <i class="piagent-win" style="--bar-size: 87.04%"></i>
-      <i class="codex-win" style="--bar-size: 12.96%"></i>
-    </div>
-    <p>Paired geometric mean ratio: <strong>0.4889</strong>. The 95% confidence interval is <strong>0.3809–0.6276</strong>, whose upper bound remains below the 1.0 baseline, so the token-saving claim is allowed. Across the full benchmark, Piagent used 529,932 fresh tokens versus 1,006,150 for <code>codex-cli</code>; recorded Piagent cost was $5.321.</p>
+
+  <div class="callout">
+    <strong>Do not replace the primary estimator by dividing the two totals.</strong>
+    <p>The aggregate ratio of 0.4427 describes workload scale only. The 61.43% claim comes from the predeclared fixed-workload family ratio of 0.3857 so that one large family cannot overwhelm the other 17.</p>
   </div>
 </section>
 
@@ -125,12 +160,12 @@
   <p class="section-kicker">Coverage</p>
   <h2 id="coverage">Six domains with three families each</h2>
   <div class="benchmark-categories">
-    <div><span>Backend</span><strong>10.00</strong><p>Authorization, money rounding, cache isolation</p></div>
-    <div><span>Frontend</span><strong>10.00</strong><p>Async response, Unicode search, pagination</p></div>
-    <div><span>Data</span><strong>10.00</strong><p>Quoted CSV, stable deduplication, schema migration</p></div>
-    <div><span>Platform</span><strong>10.00</strong><p>Config precedence, CLI parsing, workspace order</p></div>
-    <div><span>Reliability</span><strong>10.00</strong><p>Bounded retry, expiry, incident diagnosis</p></div>
-    <div><span>Security</span><strong>10.00</strong><p>Secret refusal, prompt injection, audit history</p></div>
+    <div><span>Backend</span><strong>3 families</strong><p>Authorization, money rounding, cache isolation</p></div>
+    <div><span>Frontend</span><strong>3 families</strong><p>Async response, Unicode search, pagination</p></div>
+    <div><span>Data</span><strong>3 families</strong><p>Quoted CSV, stable deduplication, schema migration</p></div>
+    <div><span>Platform</span><strong>3 families</strong><p>Config precedence, CLI parsing, workspace order</p></div>
+    <div><span>Reliability</span><strong>3 families</strong><p>Bounded retry, expiry, incident diagnosis</p></div>
+    <div><span>Security</span><strong>3 families</strong><p>Secret refusal, prompt injection, audit history</p></div>
   </div>
 </section>
 
@@ -142,15 +177,15 @@
       <thead><tr><th>Band</th><th>When to use it</th><th>Minimum evidence</th><th>Claim allowed</th></tr></thead>
       <tbody>
         <tr><td>Core</td><td>Fast local smoke before release work</td><td>4 scenarios, 1 repeat, hidden verifier</td><td>Regression only</td></tr>
-        <tr><td>Production</td><td>Public regression for a release candidate</td><td>18 scenarios, 3 repeats, <span class="inline-code">piagent</span> vs baseline</td><td>Claims stay bounded to the observed public suite; outcome coverage, comparable CI, and failure-aware effort are separate</td></tr>
-        <tr><td>Capability</td><td>Find the harness capability ceiling</td><td>4 large multi-file/multi-component tasks with reference and mutation checks</td><td>Hill-climbing, not a release or generalization gate</td></tr>
-        <tr><td>Long-horizon</td><td>Recovery/context changes and large-repository adoption</td><td>Planned dedicated paid suite; today run Production plus deterministic crash/replay tests</td><td>No public long-horizon score until that suite ships</td></tr>
+        <tr><td>Production</td><td>Public regression for a release candidate</td><td>18 families, 3 repeats, fixed workload, exact accounting</td><td>Only within the observed public suite when quality and continuity also pass</td></tr>
+        <tr><td>Capability</td><td>Find the harness capability ceiling</td><td>Large multi-file/multi-component tasks with reference and mutation checks</td><td>Hill-climbing, not a release or generalization gate</td></tr>
+        <tr><td>Long-horizon</td><td>Recovery/context changes and large-repository adoption</td><td>A dedicated paid suite; deterministic crash/replay tests supplement it today</td><td>No public long-horizon score until the dedicated suite ships</td></tr>
       </tbody>
     </table>
   </div>
   <div class="callout">
-    <strong>A benchmark must state its claim tier; it is not marketing copy.</strong>
-    <p>A version should not claim token savings unless quality/reliability, paired candidate-regression, outcome coverage, comparable-efficiency, failure-aware effort, scope/safety, and protocol gates pass. Raw Pi is a causal harness ablation; <code>codex-cli</code> is only an external product reference.</p>
+    <strong>“Production gate” does not mean “production stability proven.”</strong>
+    <p>It is the name of the synthetic public-regression release gate. It does not prove Piagent always saves exactly 61.43%, does not cover every repository, task, or model, and does not replace real shadow/canary tasks or long-term stability monitoring.</p>
   </div>
 </section>
 
@@ -160,25 +195,34 @@
   <div class="table-wrap benchmark-table-wrap">
     <table>
       <tbody>
-        <tr><th>Run</th><td><code>production-v1-20260802T153320Z-11f3d0</code></td></tr>
-        <tr><th>Suite digest</th><td><code>90a44ac4f5d4ec11772eac00f4f984d78c6962d116581ca29ab6b8760c8f9171</code></td></tr>
-        <tr><th>Runtime</th><td>Pi 0.82.0 · <code>codex-cli 0.146.0-alpha.9.2</code> · Node 24.11.1</td></tr>
-        <tr><th>Verdict</th><td><code>piagent-more-efficient</code> · every production gate passed</td></tr>
-        <tr><th>Release mapping</th><td>Measured on the candidate before metadata bump; the same logic shipped as v1.2.12.</td></tr>
+        <tr><th>Release</th><td><code>v1.6.0</code> · <code>3bba8f0b3ff521bc2a355e1f6bef6d1bbdc09511</code></td></tr>
+        <tr><th>Run</th><td><code>production-v1-20260824T040017Z-05b7cf</code></td></tr>
+        <tr><th>Suite digest</th><td><code>308dc2f0a4656cb272421949a01972df2b0717b4ffeb564306ae12aa669ae6b8</code></td></tr>
+        <tr><th>Model</th><td>GPT-5.6 Luna · Medium · both surfaces</td></tr>
+        <tr><th>Verdict</th><td><code>historical v1.6.0 gate: pass</code> · <code>Codex-relative cost: diagnostic only</code></td></tr>
+        <tr><th>Public artifact</th><td>Reviewed aggregates only; no report, seed, local path, or session identity is published.</td></tr>
       </tbody>
     </table>
   </div>
+
   <div class="callout">
     <strong>Limits of the conclusion</strong>
-    <p>This is a maintainer-built public synthetic regression, not an independent benchmark proving Piagent is better for every repository or model. Generated values do not create independent task families. Generalization requires an external family-disjoint holdout, a reference/mutation/human-calibration manifest, a frozen candidate, and real shadow/canary tasks.</p>
+    <p>This is a maintainer-built synthetic public regression, not an independent benchmark, and it does not permit a generalization claim. It does not prove Piagent wins every family or session, does not measure long-term production stability, and is not a claim about subscription spend, provider-billed cost, or wall-clock speed. Total traffic and API-equivalent cost may become a hard gate only in a future suite version with exact per-request pricing evidence; this historical run cannot be relabeled.</p>
   </div>
+
   <div class="command-grid">
     <div class="command-card">
-      <header><span>Run the production benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --production --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-sol --thinking xhigh">Copy</button></header>
+      <header><span>Run the production benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --production --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-luna --thinking medium">Copy</button></header>
       <pre><code>piagent-benchmark --production \
   --surfaces piagent,codex-cli \
-  --model openai-codex/gpt-5.6-sol \
-  --thinking xhigh</code></pre>
+  --model openai-codex/gpt-5.6-luna \
+  --thinking medium</code></pre>
     </div>
   </div>
 </section>
+
+<section class="section">
+  <p class="section-kicker">Historical snapshot</p>
+  <h2 id="historical-v1-2-12">v1.2.12 remains historical evidence</h2>
+  <p>Snapshot <code>production-v1-20260802T153320Z-11f3d0</code> used GPT-5.6 Sol/xhigh, resolved 54/54 Piagent tasks, and reported a paired successful-outcome ratio of 0.4889 with a 95% CI of 0.3809–0.6276. It predates the fixed-workload S108 contract and ran on a dirty source tree, so it remains historical observational evidence and cannot be relabeled as a v1.6.0 or current-contract claim.</p>
+</section>

package/docs-site/content/en/index.html

@@ -19,30 +19,30 @@
 </section>
 
 <section class="section">
-  <p class="section-kicker">Production V1 · historical public regression evidence</p>
+  <p class="section-kicker">Production V1 · v1.6.0 public regression evidence</p>
   <h2 id="evidence">Measured, not asserted</h2>
-  <p class="section-lead">108 sessions on the same model and thinking level, graded automatically, against an isolated <code>codex-cli</code> surface. This is the production-v1 measurement; it is not a re-measurement of v1.4.0.</p>
+  <p class="section-lead">Release-bound run <code>production-v1-20260824T040017Z-05b7cf</code> measured 108 sessions on GPT-5.6 Luna Medium at tag <code>v1.6.0</code>, exact commit <code>3bba8f0b3ff521bc2a355e1f6bef6d1bbdc09511</code>, with Piagent compared against <code>codex-cli</code>.</p>
 
   <div class="benchmark-metrics" aria-label="Headline benchmark numbers">
     <div class="benchmark-metric">
-      <span>Overall</span>
-      <strong>9.92 / 10</strong>
-      <small>Quality, Safety, Reliability all 10/10</small>
+      <span>Release evidence</span>
+      <strong>v1.6.0</strong>
+      <small>exact commit 3bba8f0</small>
     </div>
     <div class="benchmark-metric">
       <span>Tasks resolved</span>
       <strong>54/54</strong>
-      <small>100% on both surfaces</small>
+      <small>Piagent; codex-cli resolved 48/54</small>
     </div>
     <div class="benchmark-metric">
       <span>Fresh tokens</span>
-      <strong>-51.11%</strong>
-      <small>paired geometric mean, 47/54 pair wins</small>
+      <strong>-61.43%</strong>
+      <small>primary fixed-workload family ratio 0.3857</small>
     </div>
     <div class="benchmark-metric">
-      <span>Infrastructure</span>
-      <strong>0 retries</strong>
-      <small>108 valid runs</small>
+      <span>Usage evidence</span>
+      <strong>108/108 exact</strong>
+      <small>zero retries</small>
     </div>
   </div>
 
@@ -51,20 +51,20 @@
     <span><i class="codex-swatch"></i>codex-cli</span>
   </div>
 
-  <div class="benchmark-usage-chart" aria-label="Median fresh tokens per run: Piagent 7,368, codex-cli 18,476">
+  <div class="benchmark-usage-chart" aria-label="Fixed-workload family fresh-token ratio: Piagent 38.57 percent, codex-cli baseline 100 percent; tasks resolved: Piagent 54 of 54, codex-cli 48 of 54">
     <div class="benchmark-usage-group">
-      <h3>Fresh tokens, median per run</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 39.88%"></i></div><strong>7,368</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>18,476</strong></div>
+      <h3>Fixed-workload family fresh-token ratio</h3>
+      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 38.57%"></i></div><strong>0.3857</strong></div>
+      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>1.0000</strong></div>
     </div>
     <div class="benchmark-usage-group">
-      <h3>Tool calls, median per run</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 62.5%"></i></div><strong>5</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>8</strong></div>
+      <h3>Tasks resolved</h3>
+      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 100%"></i></div><strong>54/54</strong></div>
+      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 88.89%"></i></div><strong>48/54</strong></div>
     </div>
   </div>
 
-  <p>The 95% confidence interval on the paired ratio is <strong>0.3809–0.6276</strong>; the upper bound stays below 1.0, so the token-saving claim holds. <a href="/en/benchmark">See the full configuration, every band score, and the raw tables</a>.</p>
+  <p>The primary fixed-workload family ratio is <strong>0.3857</strong>, with a 95% confidence interval of <strong>0.3073–0.4840</strong>. The point estimate is 61.43% lower, while the upper confidence bound supports the conservative statement that Piagent used at least 51.60% fewer fresh tokens on this predeclared production-v1 workload and exact release. Usage was exact for all 108 sessions and no retry occurred. <a href="/en/benchmark">See the benchmark methodology and evidence boundary</a>.</p>
 </section>
 
 <section class="section">

package/docs-site/content/en/runtime.html

@@ -11,11 +11,11 @@ piagent-install --version vX.Y.Z --resolve-tag</code></pre></article>
     <article class="command-card"><header><span>Doctor</span><button class="clip-btn" data-clip="piagent-doctor /path/to/project">Copy</button></header><pre><code>piagent-doctor /path/to/project</code></pre></article>
     <article class="command-card"><header><span>Smoke benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --dry-run&#10;piagent-benchmark">Copy</button></header><pre><code>piagent-benchmark --dry-run
 piagent-benchmark</code></pre></article>
-    <article class="command-card"><header><span>Production benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --production --dry-run&#10;piagent-benchmark --production --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-sol --thinking xhigh">Copy</button></header><pre><code>piagent-benchmark --production --dry-run
+    <article class="command-card"><header><span>Production benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --production --dry-run&#10;piagent-benchmark --production --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-luna --thinking medium">Copy</button></header><pre><code>piagent-benchmark --production --dry-run
 piagent-benchmark --production --surfaces piagent,codex-cli \
-  --model openai-codex/gpt-5.6-sol --thinking xhigh</code></pre></article>
-    <article class="command-card"><header><span>Compare codex-cli</span><button class="clip-btn" data-clip="piagent-benchmark --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-sol --thinking xhigh">Copy</button></header><pre><code>piagent-benchmark --surfaces piagent,codex-cli \
-  --model openai-codex/gpt-5.6-sol --thinking xhigh</code></pre></article>
+  --model openai-codex/gpt-5.6-luna --thinking medium</code></pre></article>
+    <article class="command-card"><header><span>Compare codex-cli</span><button class="clip-btn" data-clip="piagent-benchmark --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-luna --thinking medium">Copy</button></header><pre><code>piagent-benchmark --surfaces piagent,codex-cli \
+  --model openai-codex/gpt-5.6-luna --thinking medium</code></pre></article>
     <article class="command-card"><header><span>Trusted run</span><button class="clip-btn" data-clip="piagent-auto --full-access -p &quot;Run the trusted local benchmark suite.&quot;">Copy</button></header><pre><code>piagent-auto --read-only -p "Scout payment mapping. Do not edit source."
 piagent-auto --full-access -p "Run the trusted local benchmark suite."</code></pre></article>
     <article class="command-card"><header><span>Capabilities</span><button class="clip-btn" data-clip="piagent-capabilities catalog --check&#10;piagent-capabilities doctor --profile .pi/piagent-profile.json">Copy</button></header><pre><code>piagent-capabilities catalog --check

package/docs-site/content/en/whats-new.html

@@ -1,3 +1,14 @@
+<section class="section">
+  <p class="section-kicker">v1.6.1 · Reliability patch</p>
+  <h2 id="v1-6-1">Truthful UI state, safe external review, and lower helper overhead</h2>
+  <p class="section-lead">This patch keeps the selected model and thinking level unchanged while tightening lifecycle truth, limiting helper expansion, and granting external-source reads only to the exact checkout for the current session.</p>
+  <div class="grid cols-3">
+    <article class="card"><h3 id="conversation-truth">Conversation truth</h3><p>Drafts, retries, and pending completion stay in Activity. Chat preserves user-then-assistant order and clears processing only at the real terminal boundary.</p></article>
+    <article class="card"><h3 id="external-review">External review</h3><p>GitHub, GitLab, and Bitbucket references activate only minimal source checkout. The runtime grants reads to that exact tree while blocking caches, credentials, symlink escape, and every mutation path.</p></article>
+    <article class="card"><h3 id="solo-first">Solo-first</h3><p>Direct subagent use is capped at one read-only helper with call, turn, and runtime budgets, no automatic retry/fork/worker expansion, and truthful failure state when evidence is missing.</p></article>
+  </div>
+</section>
+
 <section class="section">
   <p class="section-kicker">Product stabilization</p>
   <h2 id="main-changes">Six changes users will notice</h2>
@@ -71,7 +82,7 @@
         <tr><td>Solver</td><td><span class="inline-code">shadow</span></td><td>Evaluates routes without changing the parent model or taking control of the workflow.</td></tr>
         <tr><td>Phase tools</td><td><span class="inline-code">shadow</span></td><td>Computes projections; use <span class="inline-code">PIAGENT_PHASE_TOOLS=on</span> to apply phase-specific tool visibility.</td></tr>
         <tr><td>Recovery</td><td><span class="inline-code">on</span></td><td>Retries or repairs within ceilings; use <span class="inline-code">PIAGENT_AUTO_RECOVERY=off</span> to restore ordinary handoff.</td></tr>
-        <tr><td>Helpers</td><td><span class="inline-code">recommend</span></td><td>Explains recommendations only; <span class="inline-code">PIAGENT_HELPERS_MODE=on</span> dispatches read-only roles only when an adapter is available.</td></tr>
+        <tr><td>Helpers</td><td><span class="inline-code">off</span></td><td>Explicit <span class="inline-code">PIAGENT_HELPERS_MODE=on</span> still allows only one fresh read-only helper after two independent lanes and at least 30% projected net saving are proven.</td></tr>
         <tr><td>Parent routing and automatic workers</td><td><span class="inline-code">off</span></td><td>No automatic parent-model switch, writer dispatch, or default multi-writer behavior.</td></tr>
       </tbody>
     </table>

package/docs-site/content/en/workflows.html

@@ -64,8 +64,8 @@
     <p>There is no separate <span class="inline-code">/git</span> namespace. Use <span class="inline-code">/workflow commit</span> for a reviewed local commit and <span class="inline-code">/workflow pr</span> to prepare a pull request. Actions such as <span class="inline-code">git push</span> or broad staging with <span class="inline-code">git add -A</span> still require operator confirmation.</p>
   </div>
   <div class="callout">
-    <strong>Solo-first orchestration</strong>
-    <p>Use <span class="inline-code">/piagent-orchestration</span> to inspect mode, subagent limits, review lenses, Field Guide, and writer policy. The parent agent owns the task by default; subagents are used only when scouting, planning, or review has an independent lane worth the token cost.</p>
+    <strong>Parent-direct orchestration</strong>
+    <p>Use <span class="inline-code">/piagent-orchestration</span> to inspect the one-helper ceiling, dispatch/skip reason, projected token saving, review lenses, Field Guide, and writer policy. Helpers default to off. Explicit opt-in still permits only one fresh read-only helper when two independent lanes and at least 30% projected net saving are proven; workers and retries stay disabled.</p>
   </div>
   <div class="command-grid">
     <article class="command-card"><header><span>Bounded implementation</span><button class="clip-btn" data-clip="/workflow task Fix the failing checkout test and verify.">Copy</button></header><pre><code>/workflow task Fix the failing checkout test and verify.</code></pre></article>

package/docs-site/content/index.html

@@ -19,30 +19,30 @@
 </section>
 
 <section class="section">
-  <p class="section-kicker">Production V1 · historical public regression evidence</p>
+  <p class="section-kicker">Production V1 · bằng chứng public của v1.6.0</p>
   <h2 id="bang-chung">Đo được, không phải nói suông</h2>
-  <p class="section-lead">108 session, cùng model và cùng thinking level, chấm tự động, so với surface <code>codex-cli</code> chạy cách ly. Đây là bằng chứng của bản đo production-v1, không phải phép đo lại của v1.4.0.</p>
+  <p class="section-lead">Run gắn với release <code>production-v1-20260824T040017Z-05b7cf</code> đã đo 108 session bằng GPT-5.6 Luna Medium tại tag <code>v1.6.0</code>, đúng commit <code>3bba8f0b3ff521bc2a355e1f6bef6d1bbdc09511</code>, với Piagent so với <code>codex-cli</code>.</p>
 
   <div class="benchmark-metrics" aria-label="Số liệu benchmark nổi bật">
     <div class="benchmark-metric">
-      <span>Overall</span>
-      <strong>9,92 / 10</strong>
-      <small>Quality, Safety, Reliability đều 10/10</small>
+      <span>Bằng chứng release</span>
+      <strong>v1.6.0</strong>
+      <small>đúng commit 3bba8f0</small>
     </div>
     <div class="benchmark-metric">
       <span>Task resolved</span>
       <strong>54/54</strong>
-      <small>100% cho cả hai surface</small>
+      <small>Piagent; codex-cli đạt 48/54</small>
     </div>
     <div class="benchmark-metric">
       <span>Fresh token</span>
-      <strong>-51,11%</strong>
-      <small>paired geometric mean, 47/54 cặp thắng</small>
+      <strong>-61,43%</strong>
+      <small>tỷ lệ family fixed-workload chính 0,3857</small>
     </div>
     <div class="benchmark-metric">
-      <span>Infrastructure</span>
-      <strong>0 retry</strong>
-      <small>108 lần chạy hợp lệ</small>
+      <span>Bằng chứng usage</span>
+      <strong>108/108 exact</strong>
+      <small>không có retry</small>
     </div>
   </div>
 
@@ -51,20 +51,20 @@
     <span><i class="codex-swatch"></i>codex-cli</span>
   </div>
 
-  <div class="benchmark-usage-chart" aria-label="Fresh token median mỗi run: Piagent 7.368, codex-cli 18.476">
+  <div class="benchmark-usage-chart" aria-label="Tỷ lệ fresh token theo family fixed-workload: Piagent 38,57 phần trăm, baseline codex-cli 100 phần trăm; task hoàn thành: Piagent 54 trên 54, codex-cli 48 trên 54">
     <div class="benchmark-usage-group">
-      <h3>Fresh token, median mỗi run</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 39.88%"></i></div><strong>7.368</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>18.476</strong></div>
+      <h3>Tỷ lệ fresh token theo family fixed-workload</h3>
+      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 38.57%"></i></div><strong>0,3857</strong></div>
+      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>1,0000</strong></div>
     </div>
     <div class="benchmark-usage-group">
-      <h3>Tool calls, median mỗi run</h3>
-      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 62.5%"></i></div><strong>5</strong></div>
-      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 100%"></i></div><strong>8</strong></div>
+      <h3>Task resolved</h3>
+      <div class="benchmark-usage-row"><span>Piagent</span><div class="benchmark-track"><i class="piagent-bar" style="--bar-size: 100%"></i></div><strong>54/54</strong></div>
+      <div class="benchmark-usage-row"><span>codex-cli</span><div class="benchmark-track"><i class="codex-bar" style="--bar-size: 88.89%"></i></div><strong>48/54</strong></div>
     </div>
   </div>
 
-  <p>Khoảng tin cậy 95% của tỷ lệ ghép cặp là <strong>0,3809–0,6276</strong>; cận trên vẫn dưới 1,0 nên claim tiết kiệm token đứng vững. <a href="/benchmark">Xem đầy đủ cấu hình, từng band điểm và bảng số</a>.</p>
+  <p>Tỷ lệ family fixed-workload chính là <strong>0,3857</strong>, với khoảng tin cậy 95% <strong>0,3073–0,4840</strong>. Ước lượng điểm thấp hơn 61,43%; theo cận trên của khoảng tin cậy, có thể phát biểu thận trọng rằng Piagent dùng ít hơn ít nhất 51,60% fresh token trên đúng workload production-v1 đã khai báo trước và đúng release này. Usage của cả 108 session đều exact và không có retry. <a href="/benchmark">Xem phương pháp benchmark và ranh giới bằng chứng</a>.</p>
 </section>
 
 <section class="section">

package/docs-site/content/runtime.html

@@ -31,15 +31,15 @@ piagent-install --version vX.Y.Z --resolve-tag</code></pre>
 piagent-benchmark</code></pre>
     </article>
     <article class="command-card">
-      <header><span>Production benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --production --dry-run&#10;piagent-benchmark --production --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-sol --thinking xhigh">Lấy lệnh</button></header>
+      <header><span>Production benchmark</span><button class="clip-btn" data-clip="piagent-benchmark --production --dry-run&#10;piagent-benchmark --production --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-luna --thinking medium">Lấy lệnh</button></header>
       <pre><code>piagent-benchmark --production --dry-run
 piagent-benchmark --production --surfaces piagent,codex-cli \
-  --model openai-codex/gpt-5.6-sol --thinking xhigh</code></pre>
+  --model openai-codex/gpt-5.6-luna --thinking medium</code></pre>
     </article>
     <article class="command-card">
-      <header><span>Compare codex-cli</span><button class="clip-btn" data-clip="piagent-benchmark --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-sol --thinking xhigh">Lấy lệnh</button></header>
+      <header><span>Compare codex-cli</span><button class="clip-btn" data-clip="piagent-benchmark --surfaces piagent,codex-cli --model openai-codex/gpt-5.6-luna --thinking medium">Lấy lệnh</button></header>
       <pre><code>piagent-benchmark --surfaces piagent,codex-cli \
-  --model openai-codex/gpt-5.6-sol --thinking xhigh</code></pre>
+  --model openai-codex/gpt-5.6-luna --thinking medium</code></pre>
     </article>
     <article class="command-card">
       <header><span>Trusted run</span><button class="clip-btn" data-clip="piagent-auto --full-access -p &quot;Run the trusted local benchmark suite.&quot;">Lấy lệnh</button></header>