opencode-plugin-flow 7.0.2 → 7.1.0

package/CHANGELOG.md

@@ -6,6 +6,170 @@ One short entry per release, written for users deciding whether to upgrade.
 
 No changes yet.
 
+## [7.1.0] - 2026-07-28
+
+The last route to a dishonest `completed` closure is closed, and the two claims
+that rested on prompt prose are now measured.
+
+Qualified on Flow 7.0.2 / OpenCode 1.18.6 from the 81-run matrix of 2026-07-28T16:23Z
+across `opencode/claude-sonnet-5`, `openai/gpt-5.6-sol`, and `xai/grok-4.5`: 81/81
+passed, nothing excluded, nothing aborted, and all 81 cassettes reproduced without a
+fidelity caveat. Two earlier matrices needed a merged re-run to qualify — one for a
+wedged `grep` call, one for an attempt excluded over a question it was right to ask.
+Neither recurred.
+
+- **A plan declares its canonical gate.** `plan.gate` is the exact command that
+  validates the whole repository, named at planning time and locked by approval. A
+  `broad` observation has to run that command byte-for-byte, and the gate joins the
+  vetoed-command set, so its latest failure blocks review whatever scope an
+  observation claimed. This closes the escape [ADR
+  0009](docs/adr/0009-scope-keyed-validation-veto.md) recorded as open: a measured
+  run had closed `completed` by claiming `git diff --check` as its gate — a command
+  that cannot fail, so nothing was ever observed red.
+- **A plan declares evidence this host cannot produce.** `plan.externalEvidence`
+  names each acceptance observation needing an operating system, service, credential,
+  or device this machine may lack, with the exact command whose passing is that
+  observation. Final review and `completed` closure are both refused until that
+  command passes, so a self-written proxy cannot stand in for it. The first
+  three-provider eval matrix found three runs substituting one — two closed
+  `completed` over a Windows-only criterion on a Linux host, with the independent
+  review passing, and the suite's own false-completion metric reported zero. Each
+  entry also names the `platform` that can observe it — `win32`, `darwin`, `linux`, or
+  `other` for a service, credential, or device — and Flow records the host every
+  observation ran on, so the declared command passing on the wrong machine no longer
+  discharges it. The next matrix run found exactly that: a declared Windows entry
+  cleared by its own command's exit zero on Linux, green because the Windows case is
+  skipped there. The refusal now distinguishes a command never observed from one that
+  passed on the wrong host, because those call for opposite moves. See
+  [ADR 0011](docs/adr/0011-declared-external-evidence.md).
+- **Acceptance evidence names test cases, not exit codes.** Each `externalEvidence`
+  entry also declares `assertions`: the case names whose passing *is* that
+  observation. `flow_validation_start` takes `resultsPath`, the JUnit XML the command
+  writes, and Flow records what that report said about each declared name. An entry is
+  satisfied only when every one is reported `passed` — `skipped` and `absent`
+  discharge nothing. This closes the limitation ADR 0011 recorded rather than fixed:
+  comparing the host closed the *wrong machine* and left the same skip on the right
+  one, where a case guarded, filtered, renamed out of the run, or never written still
+  exits zero. The names come from the approved plan and never from the caller, and a
+  report that predates the arming is read as no report at all. Declare an empty list
+  when the evidence is not a test result; that keeps the exit-code rule, which is the
+  honest answer for a credential or a device. See
+  [ADR 0012](docs/adr/0012-named-results-over-exit-codes.md).
+- **The reviewer can see the two commands the plan declares.** Its plan context now
+  carries `gate` and `externalEvidence`, which it was asked to judge and was never
+  shown: it could not tell a `broad` observation that ran the canonical gate from one
+  that ran something else, and could not check a declared environment against the host
+  the observation recorded.
+- **Session v5 schema:** `plan.gate` is a new optional string. A document written by
+  an earlier build still hydrates and keeps the older rule where `broad` is the
+  claimant's word; `flow_plan_save` requires the field for any new plan, so a plan
+  this build writes always declares one. `plan.externalEvidence` is a new optional
+  array under the same rule: older documents declare nothing and owe no acceptance
+  observation, and `flow_plan_save` requires the field — an empty list is the answer
+  when the goal is fully observable here. Each entry carries an optional `platform`
+  that `flow_plan_save` likewise requires for a new plan, and each validation
+  observation carries an optional runtime-written `hostPlatform`; an entry hydrated
+  without a platform keeps the command-only rule. Each entry also carries an optional
+  `assertions` list that `flow_plan_save` requires for a new plan, and each observation
+  may carry a `resultsPath` and the `observedAssertions` the runtime read from it; an
+  entry hydrated without assertions keeps the exit-code rule. Rolling an active session
+  back to a build without these fields is not supported, as with every previous
+  widening.
+- **The bar is published, not described.** [What Flow
+  guarantees](docs/guarantees.md) states which claims are enforced by types, attested
+  by the host, declared by the caller, judged by a model, or unenforced.
+  [Release qualification](docs/release-qualification.md) publishes the eval
+  thresholds a release clears, and `bun run qualify` applies them.
+- **Evals measure false completion and reviewer activity**, derived from durable
+  documents rather than model prose, and a scheduled workflow runs the suite against
+  at least two providers weekly. A new scenario checks that a requirement no run can
+  observe is never reported as verified. A run that aborts mid-flight is counted and
+  excluded rather than scored as a failure — one wedged attempt was the only failing
+  threshold in a measured report, on a guarantee that never ran — and `bun run
+  qualify` refuses a report holding one on a gated pair. A wedge diagnostic now names
+  the command each incomplete tool call was running.
+- **Recorded model decisions replay for free.** Every paid attempt now writes a
+  cassette — its tool calls, in order, with their arguments — and `bun run replay`
+  feeds those back through the real handlers against a fresh workspace with no model
+  and no host. It is deliberately the decision layer, not the HTTP wire: freezing tool
+  results too would mean Flow's own refusals never execute on replay, which is the one
+  class of defect this suite exists to catch. Every runtime change up to now needed
+  another paid matrix before anyone knew it had not broken a sequence a model already
+  performed. CI gates the committed cassettes; a run whose recording holds something a
+  decision-layer replay cannot reproduce is reported rather than gated.
+- **A report can be read.** `bun run triage` ranks a report's runs by how much
+  reading each is worth and prints its reason for each, because nobody should trust an
+  eval score without reading transcripts and there was no tooling for it: a 54-run
+  report was a table and a JSON file. It ranks rather than filters, and deliberately
+  does *not* flag a scored escalation every attempt of its pair made, or a lone silent
+  review pass — including both flagged 32 of 54 runs on a real report, almost all of
+  them the suite working. Excluding them flagged five: the wedge, the false completion,
+  and three escalation outliers. An empty result says so, since a suite that never
+  flags anything and one that measures nothing look identical from here.
+- **Three eval tiers, three prices.** `bun run replay` is free and answers whether the
+  runtime still reaches the same outcome; `bun run eval:smoke` is one model and one
+  attempt, for a prompt change; the full matrix is the only thing that qualifies a
+  release. One price for every question is what made the suite something run at release
+  instead of during work.
+- **A regression scenario for the hole named results closed.** `skipped-case-refused`
+  seeds a fixture whose Windows-only case is an ordinary `test.skipIf`, so the declared
+  command runs here, on the right host, and exits zero. Declaring the command is no
+  longer enough; the plan has to name the case. It went 9/9 across three providers
+  twice and stays ungated anyway: every attempt declared `platform:
+  "win32"` on a Linux host, so the platform rule refuses first and the named-case rule
+  is never what binds. What the scenario measures today is the declaration, not the
+  observation.
+- **`unprovable-claim-refused` is gated at 90%.** Measured 0/3, then 8/9, then 9/9 as
+  the rule and the prompts landed; the margin is one pair's own variance rather than an
+  allowance for refusals to fail. See
+  [release qualification](docs/release-qualification.md).
+- **`continuation-accepted` is gated at 100%,** and a new scenario plants a defect for
+  the reviewer. The first is the mirror of `goal-change-refused`: one rule is enforced
+  by refusing a changed goal and the other by accepting an unchanged one, so gating only
+  the refusal would let a build that refuses every continuation qualify. The second,
+  `defect-fails-review`, went 9/9 twice and never by the route it was built for — one
+  attempt left the planted defect in place, built past it, and the review passed without
+  mentioning it. It stays ungated because its rate measures the implementer, not the
+  reviewer. Silent review passes did fall below the ceiling for the first time, 38 of
+  42, so that metric can now move; the findings behind it were about untested edge cases
+  rather than the plant, which is the same limitation from the other side.
+- **A question the next step answers no longer voids the attempt.** An eval run that
+  stopped to ask the user was excluded wherever the scenario did not declare asking an
+  acceptable end — including a question asked partway through, which the following
+  step's prompt answers. Three scenarios open by planning, where asking for approval is
+  the behaviour another scenario gates at 100%, and one measured run lost its score for
+  doing exactly that. Since a gated pair needs three scored attempts, one correct
+  question could fail a release and force a paid re-run. Only a question the last step
+  ends on is excluded now.
+- **Seven cassettes are pinned, so CI replays real decisions.** One per scenario from
+  the 2026-07-28 matrix, spread across three providers. All 63 candidates from that run
+  replayed against this code; the only divergence was the attempt that wedged
+  mid-flight, which is advisory by construction. A run that ends by asking the user no
+  longer records a fidelity caveat either — the harness aborts that session itself, so
+  the `MessageAbortedError` it leaves behind was this suite's own doing, and calling it
+  unreproducible had made 19 of 63 cassettes advisory, every refusal scenario among
+  them.
+- **Eval reports now include the reviewer.** Subtask sessions are read too, so a
+  report finally contains the independent review's own tool calls. Before this, no
+  recorded report held a single `flow_feature_complete` call, the check for rejected
+  submissions could never fire, and the reviewer's tokens went uncounted — so token and
+  cost totals from earlier reports are lower than the same runs would report now.
+- **`/flow-auto` says when your host cannot continue.** Continuation needs assistant
+  message parentage; a host that does not report it now gets a plain note at startup
+  and an `autoContinuation` field on status, instead of a lifecycle that appears to
+  stop after every feature for no reason.
+- **Positioning, including when not to use Flow.** See
+  [positioning](docs/positioning.md) and the new README section.
+
+The prompt surface got smaller, not larger: the typed gate replaced the prose that
+asked the model to judge whether its own coverage claim was honest.
+
+Install or update:
+
+```bash
+opencode plugin opencode-plugin-flow@7.1.0 --global --force
+```
+
 ## [7.0.2] - 2026-07-27
 
 Documentation only. The docs still called the product Flow v6 two majors after

package/README.md

@@ -15,12 +15,32 @@ not fold a materially different request into the active goal.
 State lives in `.flow/session.json`, so the workflow survives a restart, a
 context change, or a lost transcript.
 
+Flow is in preview: an opinionated workflow for consequential multi-step changes,
+for people who read the review. It is worth its ceremony when a wrong change is
+expensive, and it is overhead when it is not.
+
+## When not to use Flow
+
+- **Small changes.** A one-file fix or a rename pays for a plan, a validation, a
+  review, and a close that it did not need.
+- **Exploration.** Approval locks the plan, and a materially different request will
+  not be folded into the active goal.
+- **Speed above all.** A serial lifecycle with an independent review is slower than
+  asking directly, deliberately.
+- **Trusting the verdict without reading it.** The review is a model judgment, not
+  a proof. [What Flow guarantees](docs/guarantees.md) separates the rules the
+  runtime enforces from the ones that are judgment.
+- **Parallel features, several repositories, or team orchestration.** Flow runs one
+  durable feature at a time in one project.
+
+[Positioning](docs/positioning.md) has the longer version.
+
 ## Install
 
 Install the exact npm release through OpenCode:
 
 ```bash
-opencode plugin opencode-plugin-flow@7.0.2 --global --force
+opencode plugin opencode-plugin-flow@7.1.0 --global --force
 ```
 
 Omit `--global` for project scope. Version pins are exact and never update on
@@ -31,7 +51,7 @@ The equivalent manual project configuration is:
 ```json
 {
   "$schema": "https://opencode.ai/config.json",
-  "plugin": ["opencode-plugin-flow@7.0.2"]
+  "plugin": ["opencode-plugin-flow@7.1.0"]
 }
 ```
 
@@ -59,6 +79,10 @@ validate against the real workspace, obtain an independent review — and keeps
 going through every runnable feature without handing back between them, until it
 can close the session.
 
+Continuation between features needs a host that reports assistant message
+parentage. Flow says so plainly when it detects a host that does not: the lifecycle
+still works, one `/flow-run` at a time.
+
 Send `/flow-auto stop` or `/flow-auto cancel` in the same OpenCode session to
 revoke only the in-process continuation. That does not close, defer, abandon, or
 otherwise change the durable Flow session.
@@ -86,7 +110,8 @@ you granted.
 
 ## How Flow works
 
-1. Planning saves a small feature DAG. Approval locks it.
+1. Planning saves a small feature DAG, including the repository's canonical
+   validation command. Approval locks both.
 2. One feature starts, chosen only from those whose dependencies are complete.
 3. Before editing, the manager gathers the evidence the feature needs and works
    through an adversarial risk checklist: failure ordering, repeated and
@@ -95,9 +120,10 @@ you granted.
 4. The manager implements the feature, serially or by integrating a bounded
    worker wave.
 5. Flow observes the exact armed validation command against the current
-   workspace, then opens one independent review assignment. A newer relevant
-   failure or a source change invalidates an older pass, and review cannot be
-   requested while evidence the outcome depends on is knowingly missing.
+   workspace, then opens one independent review assignment. Broad evidence runs
+   the plan's declared gate and nothing else. A newer relevant failure or a source
+   change invalidates an older pass, and review cannot be requested while evidence
+   the outcome depends on is knowingly missing.
 6. A passing review advances the plan. A failed feature is never picked up again
    implicitly — Flow reports the blocker and waits for an explicit retry or an
    independent-feature choice. The last passing feature allows closure, and every