blastproof 0.12.0 → 0.12.1

package/README.md

@@ -141,7 +141,9 @@ Write steps that end in an observable result — text on the page, a count, a st
 - fill the note field                           # cannot run
 ```
 
-The agent is **forbidden from inventing values** — one it types must come from the step, from the page, or from an `{{env.*}}` placeholder. So the second step is not merely vague, it is impossible, and the run discovers that a minute in, with a failure reason about page state that reads as though your application is broken.
+The agent is **forbidden from inventing values** — one it types must come from the step, from the page, or from an `{{env.*}}` placeholder. But that rule lives in a prompt, and a prompt instructs rather than enforces. Run against a real model, the second step does not fail: the agent makes a value up, fills it, and the step **passes**. `fill the note field` produced "This is a test note." on two runs and "This is a new note" on a third.
+
+That is worse than a failure. The test goes green having verified a value nobody wrote, differing between runs — a passing check over an unspecified input. Closing the gap in the runner is [#57](https://github.com/hamc/blastproof/issues/57); until then, this warning is what stands between you and a green test that means nothing.
 
 `run` warns about it first, on every path, before launching a browser or asking for a key:
 

package/dist/cli.js

@@ -1280,7 +1280,7 @@ Rules:
 - Prefer the journey the changed files touch over a generic tour of the page. The changed files tell you which part of the page matters.
 - **The test starts at the application's base URL, not at this route.** Begin with a step that navigates to the route and says what should be visible once it loads \u2014 "navigate to /support and verify the heading "Contact support" is shown". Without it the run opens the home page and every later step looks for controls that are not there.
 - **Every step says what it should produce.** Name what must be true once the step has been carried out, not the action alone: "submit the support form and verify the confirmation page shows the ticket number", never "submit the support form". A step that names an action without an outcome asks the runner to judge whether something happened while looking at the page that succeeding produces \u2014 a submitted form comes back empty, a redirect moves the URL \u2014 and that is the shape behind several real failures.
-- **A step that enters a value writes the value.** "fill the subject field with Order not received", never "enter a subject". The runner is forbidden from inventing values, so a step that does not supply one cannot be carried out.
+- **A step that enters a value writes the value.** "fill the subject field with Order not received", never "enter a subject". The runner is forbidden from inventing values, but a step that supplies none does not fail \u2014 the value gets made up, the step passes, and the test then verifies something nobody wrote.
 - If a step needs a credential or any secret, write it as a placeholder like {{env.TEST_PASSWORD}}. Never write a real or invented password, token or key.
 - Keep the whole test to a handful of steps: one journey, not an exhaustive suite.`;
 }
@@ -2346,7 +2346,7 @@ function printAuthoring(result, cwd) {
     console.error(`    \u2192 ${suggestValueClause(step)}`);
   }
   console.error(
-    "The runner is forbidden from inventing values, so a step that supplies none cannot be carried out."
+    "The runner is forbidden from inventing values, but only by instruction: in practice the model supplies one anyway, the step passes, and the test verifies a value nobody wrote \u2014 a different one on each run."
   );
   console.error("Steps are inspected in English only; steps in other languages are not checked.");
 }
@@ -2893,7 +2893,7 @@ function parsePositiveNumber(flag) {
   };
 }
 var program = new Command();
-program.name("blastproof").description("Open-source AI testing agent: plain-English YAML tests executed agentically on a real browser.").version("0.12.0");
+program.name("blastproof").description("Open-source AI testing agent: plain-English YAML tests executed agentically on a real browser.").version("0.12.1");
 program.command("init").description("Scaffold .blastproof/ (config, tests, sample tests) in the current directory").action(async () => {
   try {
     const result = await initProject(process.cwd());