blastproof 0.12.0 → 0.13.0

package/README.md

@@ -141,7 +141,9 @@ Write steps that end in an observable result — text on the page, a count, a st
 - fill the note field                           # cannot run
 ```
 
-The agent is **forbidden from inventing values** — one it types must come from the step, from the page, or from an `{{env.*}}` placeholder. So the second step is not merely vague, it is impossible, and the run discovers that a minute in, with a failure reason about page state that reads as though your application is broken.
+The agent is **forbidden from inventing values** — one it types must come from the step, from the page, or from an `{{env.*}}` placeholder. But that rule lives in a prompt, and a prompt instructs rather than enforces. Run against a real model, the second step does not fail: the agent makes a value up, fills it, and the step **passes**. `fill the note field` produced "This is a test note." on two runs and "This is a new note" on a third.
+
+That is worse than a failure. The test goes green having verified a value nobody wrote, differing between runs — a passing check over an unspecified input. Closing the gap in the runner is [#57](https://github.com/hamc/blastproof/issues/57); until then, this warning is what stands between you and a green test that means nothing.
 
 `run` warns about it first, on every path, before launching a browser or asking for a key:
 
@@ -196,6 +198,8 @@ ignore:
   - "**/*.md"
 ```
 
+The key is the file glob and the value is the routes it can affect — the opposite way round from a test file's own `routes:`, which is a plain list of the routes that test covers. Inverted, the map matches nothing at all and every run reports a diff that affected no page, so `blastproof` refuses a map written that way rather than running green against it.
+
 Every changed file lands in one of three buckets: it matches `routes:` and contributes them, matches `ignore:` and is knowingly irrelevant, or **matches neither — nobody has said what it affects**. `--fail-on-unmapped` blocks on that third case, naming the files and both ways to resolve them.
 
 **Nothing is ignored by default**, on purpose: a default that guesses on your behalf would hide the first files worth thinking about. The flag is additive — a run can meet `--min-score` and still be blocked here, because "the tests I ran passed" and "something changed that nobody classified" are different claims.

package/dist/cli.js

@@ -49,6 +49,9 @@ browser:
 # max_retries_per_step: 3
 
 # Impact hints mapping file globs to routes (used by \`blastproof run --impacted\`).
+# Read each line as "if this file changed, these pages are at risk" \u2014 the key is the
+# file glob, the value is the routes. Not the other way round: a test file's own
+# \`routes:\` is a list of routes, and this one is a map keyed by source path.
 routes:
   "src/auth/**": ["/login"]
   "src/cart/**": ["/cart", "/checkout"]
@@ -988,11 +991,33 @@ var authSchema = z.object({
     });
   }
 });
+var ROUTE_SHAPED = /^\/[^*?]*$/;
+var FILE_SHAPED = /[*?]|\.[a-z0-9]{1,8}$/i;
+function findInvertedRouteEntries(routes) {
+  const inverted = [];
+  for (const [key, values] of Object.entries(routes)) {
+    if (!ROUTE_SHAPED.test(key)) continue;
+    const file = values.find((value) => FILE_SHAPED.test(value));
+    if (file !== void 0) inverted.push({ key, file });
+  }
+  return inverted;
+}
 var configSchema = z.object({
   base_url: z.string().url(),
   llm: llmSchema.default({}),
   browser: browserSchema.default({}),
-  routes: z.record(z.array(z.string())).optional(),
+  routes: z.record(z.array(z.string())).superRefine((value, ctx) => {
+    const inverted = findInvertedRouteEntries(value);
+    const first = inverted[0];
+    if (!first) return;
+    const others = inverted.length > 1 ? ` (and ${inverted.length - 1} more)` : "";
+    ctx.addIssue({
+      code: z.ZodIssueCode.custom,
+      message: `is the wrong way round${others} \u2014 the key is the file glob, the value is the routes it affects.
+      found:    "${first.key}": ["${first.file}"]
+      expected: "${first.file}": ["${first.key}"]`
+    });
+  }).optional(),
   /** Globs for files knowingly irrelevant to any route (docs, CI config, licences). */
   ignore: z.array(z.string()).optional(),
   /**
@@ -1280,7 +1305,7 @@ Rules:
 - Prefer the journey the changed files touch over a generic tour of the page. The changed files tell you which part of the page matters.
 - **The test starts at the application's base URL, not at this route.** Begin with a step that navigates to the route and says what should be visible once it loads \u2014 "navigate to /support and verify the heading "Contact support" is shown". Without it the run opens the home page and every later step looks for controls that are not there.
 - **Every step says what it should produce.** Name what must be true once the step has been carried out, not the action alone: "submit the support form and verify the confirmation page shows the ticket number", never "submit the support form". A step that names an action without an outcome asks the runner to judge whether something happened while looking at the page that succeeding produces \u2014 a submitted form comes back empty, a redirect moves the URL \u2014 and that is the shape behind several real failures.
-- **A step that enters a value writes the value.** "fill the subject field with Order not received", never "enter a subject". The runner is forbidden from inventing values, so a step that does not supply one cannot be carried out.
+- **A step that enters a value writes the value.** "fill the subject field with Order not received", never "enter a subject". The runner is forbidden from inventing values, but a step that supplies none does not fail \u2014 the value gets made up, the step passes, and the test then verifies something nobody wrote.
 - If a step needs a credential or any secret, write it as a placeholder like {{env.TEST_PASSWORD}}. Never write a real or invented password, token or key.
 - Keep the whole test to a handful of steps: one journey, not an exhaustive suite.`;
 }
@@ -2346,7 +2371,7 @@ function printAuthoring(result, cwd) {
     console.error(`    \u2192 ${suggestValueClause(step)}`);
   }
   console.error(
-    "The runner is forbidden from inventing values, so a step that supplies none cannot be carried out."
+    "The runner is forbidden from inventing values, but only by instruction: in practice the model supplies one anyway, the step passes, and the test verifies a value nobody wrote \u2014 a different one on each run."
   );
   console.error("Steps are inspected in English only; steps in other languages are not checked.");
 }
@@ -2893,7 +2918,7 @@ function parsePositiveNumber(flag) {
   };
 }
 var program = new Command();
-program.name("blastproof").description("Open-source AI testing agent: plain-English YAML tests executed agentically on a real browser.").version("0.12.0");
+program.name("blastproof").description("Open-source AI testing agent: plain-English YAML tests executed agentically on a real browser.").version("0.13.0");
 program.command("init").description("Scaffold .blastproof/ (config, tests, sample tests) in the current directory").action(async () => {
   try {
     const result = await initProject(process.cwd());