pi-aia-asf 0.9.2 → 0.11.0

package/CHANGELOG.md

@@ -7,11 +7,118 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.11.0] - 2026-10-12
+
+### Added
+
+- (describe changes for 0.11.0)
+
+
+### Added
+
+- **Browser verification now follows what a change TOUCHES, and happens in the
+  build (06b Rule 9, strengthened in place)**. The rule already existed, but
+  every formulation keyed on *"if the deliverable has a UI"* — which an agent
+  editing one helper in an existing app reads as not engaged, and "I only
+  changed a helper" is exactly the change that reaches a rendered surface
+  unnoticed. The trigger is now **can this change reach a rendered surface**,
+  with an explicit touched list: pages, routes, components, forms, templates;
+  markup, styling, copy, formatting, truncation, empty and error states;
+  client-side state, routing and the fetching a view depends on; **an API,
+  serializer or query that a UI consumes** (a server-side diff is not exempt
+  for living in `src/server/`); auth/permission paths that gate what a user
+  sees; user-triggered uploads and downloads.
+- **Moved from final verification into the Phase 6 loop.** "Must be verified"
+  was Phase 7 language, which makes the gate the first time anyone has looked —
+  by then the cause is buried in the whole change. The touched interface is now
+  opened in the browser **before committing that change**, while the context is
+  warm and the defect is one edit wide (test-first step 5).
+- **Same safety inversion as Rule 19**: uncertain whether the change reaches a
+  surface? Then it is touched. "It's only a backend change" is not an argument;
+  naming the render path it cannot affect is. Uncertainty costs a browser run,
+  never a skipped one.
+- **Closes a hole Rule 19 opened**: nothing previously stopped a subset run
+  from excluding the browser/E2E suite for the surface just changed. For a
+  touched surface that suite is now **never excludable** — selection may drop
+  unrelated suites, never the one that looks at what you just changed.
+- **Evidence, not assertion.** A run is reported with the page/URL opened, the
+  action taken and what was asserted — or `n/a: no rendered surface touched,
+  because <reason>`. Bare "browser tested" is named as an assertion rather than
+  evidence, and an unavailable `pi-aia-browser` must be said out loud instead of
+  reading as a passed check. Backed by the Rule 10 checkbox, the `/asf verify`
+  checklist item, four new anti-patterns, and `test/test-browser-touched.mjs`
+  (42 assertions, 6 negative controls). Rule 9 keeps its number, its five steps
+  and its silent-async paragraph — strengthened, not duplicated.
+
+## [0.10.0] - 2026-10-07
+
+### Added
+
+- **06b Rule 19 — test selection for the inner loop (P25)**. Rule 10's
+  "full test suite green (not a subset)" is a **gate**, and it stays exactly
+  where it is: the run before a commit is always the full suite. What Rule 19
+  relaxes is only the **intermediate red-green-refactor runs inside a single
+  change** — the runs on code you are still editing. The rule is the user's
+  own: **run a partial, impact-area subset only when certain that is
+  warranted; if not certain, or the blast radius *might* be larger, always run
+  full. Default is full** — so the burden of proof sits on the subset, and the
+  cheap direction to be wrong in is also the safe one.
+- **"Certain" is evidence, not a feeling.** A subset run must be preceded by a
+  stated **impact statement** — changed files, tests selected, and *why nothing
+  else can be reached*. Restating the file list does not qualify; no statement
+  means not certain, which means full. On top of that, a **mechanical
+  disqualifier list** removes judgment wherever "might be bigger" is the
+  honest answer: shared/global state, dependencies and lockfile, build/test/CI
+  config or the harness itself, schema/migrations/shared fixtures, public
+  contracts and shared types, cross-module refactor/rename/move/delete — and
+  **any changed file with no entry in the test-impact map**, so a project that
+  has not built a map always runs full. The relaxation only ever applies where
+  the mapping is known, never where it is guessed.
+- **Deliberately not built**: the drafted coin toss and `iteration % N`
+  counter are gone. An LLM cannot roll a dice — asked to flip one it picks the
+  answer that skips the slow run — and an undefined iteration boundary means
+  the trigger silently never fires. A subset streak also cannot survive a
+  commit, because every commit is full-gated. Below a measured ~90s full-suite
+  runtime the assessment is skipped entirely and the suite is just run.
+- **Wired in**: `06b-testing-qa.md` Rule 10's checkbox now says in place that
+  the gate never moves; `06-implementation.md` test-first steps 3 and 4 name
+  the loop/gate split; `SKILL.md` gains two anti-patterns (a subset run with no
+  impact statement, and a subset run passed off as the Rule 10 checkbox) and
+  the index count moves to 19 rules. Covered by
+  `test/test-test-selection.mjs` (26 assertions), which includes a regression
+  guard that **fails if the coin-toss or iteration-counter design ever returns**.
+
 ## [0.9.2] - 2026-10-04
 
 ### Added
 
-- (describe changes for 0.9.2)
+- **`references/06i-completeness.md` — the completeness standard (P24, mandatory)**.
+  Newer models increasingly ship work that *looks* finished: a function whose body is
+  a placeholder, a screen populated with invented records, a report citing a limit
+  nobody checked. Green tests do not catch it — the tests get written against the
+  stub, so the fake looks verified. The reference states the core rule (**ship the
+  real thing, or say out loud what is not real**), the banned patterns (`TODO` /
+  `not implemented` / `coming soon` on a user-reachable path, fake or seed data the
+  user reads as their own, invented facts stated as checked, silently deferred scope,
+  built-but-never-wired, mocks in production paths, config nothing sets), the
+  exceptions that must be **declared, never silent** (the user asked for a scaffold,
+  an approved deferral, a fake at a test seam), how to prove it is real (exercise each
+  part through the entry point the user will use and name that action in the delivery
+  report), and a completeness DoD.
+- **Wired in, not bolted on**: Phase 6 gains a mandatory read block and discipline
+  rule 11; Phase 7 gains the completeness DoD check; the anti-pattern list gains six
+  entries (stub wearing a name, fake data the user reads as their own, invented facts
+  stated as fact, quietly deferred scope, stub presented as complete because its own
+  tests pass, unexplained deliberate stub); the reference index marks 06i as always
+  read. Proportionality holds — small work reads only the core rule and the DoD.
+
+### Requires
+
+- The enforcement side lives in **pi-vigilant 0.6.2**, which scans the files a
+  session changed for stub / fabricated-data / fact-hole markers at final
+  verification and puts the `file:line` evidence in front of the agent. The rule and
+  the check are the same proposal: the agent must finish the work or state the
+  discrepancy.
 
 
 ## [0.9.1] - 2026-10-03

package/index.ts

@@ -44,12 +44,12 @@ interface AsfState {
 /** Definition-of-done checks (references/06b-testing-qa.md, Rule 10). */
 const QA_CHECKLIST: Array<{ key: string; label: string }> = [
   { key: "build", label: "Typecheck/build passes" },
-  { key: "tests", label: "Full test suite green (not a subset)" },
+  { key: "tests", label: "Full test suite green (not a subset) — the gate; Rule 19 never relaxes this checkbox" },
   { key: "regression", label: "Regression test added for every bug fixed this cycle" },
   { key: "triggers", label: "Triggers AND non-triggers tested for conditional behavior" },
   { key: "artifact", label: "Shipped artifact inspected (npm pack file list) + clean-room install" },
   { key: "observable", label: "Observable end state verified as a user would experience it" },
-  { key: "browser", label: "Web surfaces exercised through a real browser (n/a if none)" },
+  { key: "browser", label: "Every interface the change TOUCHES exercised through a real browser during the build (06b Rule 9) — page + action + assertion named; n/a with reason if none" },
   { key: "specs", label: "Every MUST spec 'met' with concrete evidence" },
   { key: "trace", label: "Spec-to-code traceability: every met spec has outcome → codePath → test" },
   { key: "code-health", label: "Code Health Gate passed (or enabled:false / gate.mode:warn with the report shown)" },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-aia-asf",
-  "version": "0.9.2",
+  "version": "0.11.0",
   "description": "Ai Applied Agentic Software Factory — codifies the full software development flow: intake, research, spec capture, adversarial analysis, planning with approval gates, test-first implementation, and release. Requires pi-vigilant, pi-smart-web-search, pi-smart-fetch, and pi-aia-browser.",
   "keywords": [
     "pi-package",

package/skills/aia-asf/SKILL.md

@@ -229,7 +229,7 @@ Execute the task list milestone by milestone. Discipline rules:
 2. **Codebase isolation**: work strictly inside the project's own codebase. Do NOT edit files in other repos, global config, or unrelated directories — **unless the user explicitly instructs otherwise**. If a change would touch another codebase, stop and ask.
 3. **No scope creep**: if something new is discovered that changes specs, capture it, ask the user, and update the plan before implementing.
 4. **Descriptive commits**: `git commit -m "type: specific description of what and why"` (e.g. `fix: verify specs before rotation`). No vague messages, no placeholders.
-5. **Browser testing — MANDATORY for any web interface**: if the deliverable has a UI/website/web app, test it through `pi-aia-browser` (`browser_init`, `browser_navigate`, `browser_click`, `browser_type`, `browser_screenshot`, `browser_dom`, …) to replicate the user's real experience — not just curl/API checks. Verify: loads, key user journeys, responsive behavior, console errors. **Silent async paths included**: ingest through the real flow and wait for the enrichment to land (06b Rule 9).
+5. **Browser testing — MANDATORY for every interface the change TOUCHES, in the build (06b Rule 9)**: the trigger is **can this change reach a rendered surface**, not whether the project "has a UI" — a page, route, component, form, markup, styling, copy, formatting, client-side state, or an API that a UI consumes all count. Exercise it through `pi-aia-browser` (`browser_init`, `browser_navigate`, `browser_click`, `browser_type`, `browser_screenshot`, `browser_dom`, …) **before committing that change**, not at final verification — not just curl/API checks. Verify: loads, key user journeys, responsive behavior, console errors. **Uncertain whether it reaches a surface? Then it is touched** — "only a backend change" is not an argument, naming the render path it cannot affect is. **Silent async paths included**: ingest through the real flow and wait for the enrichment to land (06b Rule 9). Report the page opened, the action taken and what was asserted, or `n/a` with a reason; if the browser tool is unavailable, say so rather than letting it read as passed.
 6. **Bounded waits — ALWAYS (06b Rule 15)**: **every** tool call or command that waits on the system gets an explicit, reasonable **hard timeout** (`timeout N …` or the tool's `timeout` parameter) — never run unbounded. State the expected duration first and size the bound to it with headroom (a 30s suite gets ~120s, not 3000s). This matters most **during testing**, where harnesses, servers, daemons and browsers hang on ports, locks and prompts. **Absolute ceiling: no single wait may exceed 30 min (1800s).** The ceiling is a backstop, never a substitute for the real per-call bound. When it is hit: kill the wait and diagnose what it is blocking on — that is a bug signal, not patience. The ceiling may be **waived only for a single run**, stated explicitly beforehand with a reason and a finite larger bound; the waiver **does not persist** and resets immediately after that run. A 50,000-second hang is a failure to investigate.
 7. **Let pi-vigilant do its job**: it will auto-continue after premature stops and verify specs at settle. When it asks for `update_spec_status` with evidence, do it.
 8. **CHANGELOG discipline**: every user-visible change gets a CHANGELOG entry describing exactly what changed (no placeholder text).
@@ -281,6 +281,10 @@ conflicts). Record the review + triage in the delivery report.
 - ❌ Skipping research for new projects
 - ❌ Editing other codebases without explicit permission
 - ❌ Verifying a UI only via API/curl — browser testing is mandatory
+- ❌ Treating browser testing as about "projects that have a UI" — the trigger is whether **this change** can reach a rendered surface (06b Rule 9)
+- ❌ Deferring browser verification of a touched interface to final verification — open it in the build, before committing that change (06b Rule 9)
+- ❌ "It's only a backend change" with no render path named — if uncertain whether it reaches a surface, browser-test it (06b Rule 9)
+- ❌ Reporting "browser tested" without the page, action and assertion — that is an assertion, not evidence; and a missing browser tool must be said out loud, never read as passed (06b Rule 9)
 - ❌ Vague commits or CHANGELOG placeholders
 - ❌ Declaring done while specs are still `open`
 - ❌ Publishing anything beyond the configured release policy — no `.asf-release.json` means the default: always get the user's explicit go-ahead
@@ -324,6 +328,8 @@ conflicts). Record the review + triage in the delivery report.
 - ❌ Shipping a delivery summary without the report fields: spec status + evidence, test results, skipped checks, not-tested register (06b Rule 17)
 - ❌ Letting an escaped defect teach nothing — ask which gate should have caught it and close the gap (06b Rule 18)
 - ❌ Self-modifying the ASF's own rules during a task run — standards change only via the review process (P22)
+- ❌ Running a subset of tests inside the loop without a stated impact statement, or when uncertain whether the blast radius is larger — default is full (06b Rule 19)
+- ❌ Treating a subset run as satisfying the Rule 10 full-suite checkbox — the commit/DoD gate is always a real full run (06b Rule 19)
 
 ## Reference index (read on demand — never load all)
 
@@ -338,7 +344,7 @@ reads at most 06b + 06c; large work reads the ones its phase calls for.
 | `04-adversarial.md` | adversarial checklist (spec/architecture/security lenses) | Phase 4 (large) |
 | `05-plan.md` | PLAN.md template + ADRs + quality scenarios + ATAM-lite + views | Phase 5 (large) |
 | `06-implementation.md` | coding discipline, M4/M5, drift self-correction (P20) | Phase 6 |
-| `06b-testing-qa.md` | **mandatory** testing & QA standard (18 rules + DoD) | before writing tests (always) |
+| `06b-testing-qa.md` | **mandatory** testing & QA standard (19 rules + DoD) | before writing tests (always) |
 | `06c-code-quality.md` | **mandatory** modularity standard (9 rules, deep modules, smells) | before structuring code |
 | `06d-delegation.md` | intercom & subagents + independent reviewer gate (P19) | before delegating; Phase 7 review (large) |
 | `06e-code-health.md` | Code Health Gate (+ depth, supply-chain, secrets metrics) | Gate 8 (large) |

package/skills/aia-asf/references/06-implementation.md

@@ -15,8 +15,15 @@
 
 1. Write the failing test for the next behavior
 2. Implement the minimum to pass
-3. Run the suite; refactor; re-run
-4. Only commit green
+3. Run the suite; refactor; re-run — steps 3's intermediate runs (not the
+   commit-gate run) may use a selected subset per **06b Rule 19**, and only
+   when certain the subset is warranted; default is full
+4. Only commit green — the pre-commit run is always the FULL suite (06b
+   Rule 10), never a subset, regardless of what ran in step 3
+5. Browser-verify every interface the change **touches** through
+   `pi-aia-browser` **before committing that change** (06b Rule 9) — not at
+   final verification. If you cannot argue the change provably cannot reach a
+   rendered surface, it counts as touched and gets a browser run.
 
 For web interfaces, browser tests via pi-aia-browser are part of the test suite, not an afterthought.
 
@@ -27,9 +34,13 @@ For web interfaces, browser tests via pi-aia-browser are part of the test suite,
 - If a legitimate need to touch another codebase arises (shared lib, dependency fix): **stop and ask the user**.
 - Exception: the user explicitly instructs otherwise — then note the instruction and proceed.
 
-## Browser testing (MANDATORY for web interfaces)
+## Browser testing (MANDATORY for every touched interface, in the build)
 
-Any deliverable with a UI/web surface must be verified through the browser to replicate the user's real experience:
+The trigger is **what the change touches**, not whether the project has a UI.
+If the change can reach a rendered surface — page, route, component, form,
+markup, styling, copy, formatting, client-side state, or an API a UI consumes —
+it gets a browser run, **during Phase 6 while building**, before that change is
+committed. Uncertain whether it reaches one? Then it is touched (06b Rule 9).
 
 1. `browser_init` — start the browser
 2. `browser_navigate` to the app URL (local dev server or deployed)
@@ -40,6 +51,11 @@ Any deliverable with a UI/web surface must be verified through the browser to re
 
 API-only verification is NOT sufficient for anything a human would see or click.
 
+Report it as evidence, not assertion: the page/URL opened, the action taken,
+what was asserted — or `n/a: no rendered surface touched, because <reason>`.
+If `pi-aia-browser` is unavailable, say so and name what you did instead; a
+missing tool must never read as a passed check.
+
 ## Commit messages
 
 `type: specific description of what and why`

package/skills/aia-asf/references/06b-testing-qa.md

@@ -117,7 +117,7 @@ two-sided. Half a test suite hides half the bugs.
 - Check terminal states are reachable, and no state can block its own exit.
 - Test repeat/idempotent invocations and out-of-order calls.
 
-## Rule 9 — Browser testing is mandatory for any web surface
+## Rule 9 — Browser-test every interface you TOUCH, during the build
 
 API/curl checks do not replicate what a human sees. Via `pi-aia-browser`:
 
@@ -129,6 +129,44 @@ API/curl checks do not replicate what a human sees. Via `pi-aia-browser`:
 
 A 200 response with a blank or broken page is a **failure**.
 
+**The trigger is what the change TOUCHES, not whether the project "has a UI".**
+A deliverable-level trigger lets a small edit slide: "I only changed a helper"
+is exactly the change that reaches a rendered surface unnoticed. So the
+question is never *does this project have a UI* — it is **can this change reach
+one**. Anything below counts as touched, and must be exercised through the
+browser:
+
+- a page, route, component, widget, form, or template
+- anything that changes what is rendered: markup, styling, copy, formatting,
+  date/number/currency rendering, truncation, empty and error states
+- client-side state, routing, data fetching the view depends on
+- an API, serializer or query **that a UI consumes** (server-side changes are
+  not exempt just because the diff is in `src/server/`)
+- auth/permission paths that gate what a user sees, and uploads/downloads
+  a user triggers from a page
+
+**When uncertain, browser-test it — same inversion as Rule 19.** If you cannot
+argue that the change provably cannot reach a rendered surface, treat it as
+touched. "It's only a backend change" is not an argument; naming the render
+path it cannot affect is. The cheap direction to be wrong in must be the safe
+one, so uncertainty costs a browser run, never a skipped one.
+
+**This happens in the Phase 6 build loop, per change — not at final
+verification.** Open the touched interface in the browser *before committing
+that change*, while the context is warm and the defect is one edit wide.
+Deferring it to Phase 7 means the gate is the first time anyone has looked, by
+which point the cause is buried in the whole change. If the loop run was a
+Rule 19 subset, the browser/E2E suite for a touched surface is **never
+excludable** — selection may drop unrelated suites, never the one that looks at
+what you just changed.
+
+**Evidence, not assertion.** A browser run is reported with what was actually
+done: the page/URL opened, the action taken, and what was asserted — or
+explicitly `n/a: no rendered surface touched, because <reason>`. "Browser
+tested" with no page and no action named is not evidence (Rule 11). If
+`pi-aia-browser` is unavailable, **say so** and name what you did instead;
+never let a missing tool read as a passed check.
+
 **Silent async paths are the priority.** Fire-and-forget enrichment (an async
 budgeted analysis that fills in a card title after render) fails silently — the
 page renders, nobody notices the enrichment never landed. For any async/silent
@@ -139,12 +177,12 @@ never arrives, the wiring is dead — that is a failed test.
 ## Rule 10 — Definition of done (all must hold)
 
 - [ ] Typecheck/build passes
-- [ ] Full test suite green (not a subset)
+- [ ] Full test suite green (not a subset) — **the gate never moves: this checkbox always means a full run, even when Rule 19 permitted subset runs earlier in the loop that led here**
 - [ ] Regression test added for every bug fixed this cycle
 - [ ] Triggers **and** non-triggers tested for conditional behavior
 - [ ] Shipped artifact inspected (`npm pack` file list) and installed clean-room
 - [ ] Observable end state verified as a user would experience it
-- [ ] Web surfaces exercised through a real browser
+- [ ] Web surfaces exercised through a real browser **in the build, for every interface the change touches** (Rule 9) — report names the page/URL opened, the action taken and what was asserted; `n/a` stated with a reason if nothing rendered is touched
 - [ ] Every MUST spec `met` with concrete evidence (`update_spec_status`)
 - [ ] **Spec-to-code traceability: every `met` spec carries `trace` (outcome → codePath → testFile + assertion); `/asf verify` mechanically validates it (large work)**
 - [ ] **Code Health Gate passed (or `enabled: false` / `gate.mode: warn` with the report shown)**
@@ -325,3 +363,57 @@ production incident), the fix is not the end — the **loop** is:
 > **Real failure:** a package shipped with a broken `files` allowlist; the
 > fix landed but the *gap* ("no test inspects the packaged artifact") was not
 > closed until Rule 1 was added. The loop is what prevents the next one.
+
+## Rule 19 — Test selection for the inner loop (P25; the DoD gate never moves)
+
+Rule 10's full-suite checkbox is a **gate**, not a loop constraint. The gate
+never relaxes. What this rule relaxes is the **intermediate** runs inside one
+change's own red-green-refactor cycle (06-implementation "Test-first" steps
+1-3) — the runs that happen *before* the commit-gate run, on code you are
+still editing.
+
+**The rule, in the user's own words: run a partial (impact-area) subset only
+when certain that this is warranted; if not certain, or if the blast radius
+might be greater than the impact area, always run full. Default is full.**
+
+- **Burden of proof is on the subset, not on the full run.** Silence,
+  uncertainty, or "probably fine" all mean full. A subset run requires an
+  affirmative, checkable case.
+- **A subset run requires a stated impact statement** in the transcript,
+  before the run: the changed files, the tests selected because of them, and
+  *why nothing else can be reached*. A file list alone is not an impact
+  statement — restating the diff is not an argument for why the blast radius
+  stops there. No statement, or a statement that does not argue the
+  boundary → **not certain → full.**
+- **Certainty is impossible by definition (no judgment call — always full)**
+  when the changed footprint includes any of:
+  - shared/global state, a singleton, a cache, env/config, a feature flag
+  - `package.json`, the lockfile, or any dependency
+  - build, test, lint, or CI config, or the test harness itself
+  - a DB schema, a migration, or a shared fixture
+  - a public API/contract, CLI args, or a wire/shared type
+  - a cross-module refactor, a rename, a file move, or a deletion
+  - **any changed file with no entry in the project's test-impact map** — a
+    project that has not built a map always runs full; the relaxation only
+    ever applies where the mapping is known, never where it is guessed.
+- **The ~90s-and-under guard**: if the full suite's own measured runtime is
+  small enough that an assessment costs more than just running it, skip the
+  assessment and run full — state the measured number once rather than
+  re-litigating it every iteration.
+- **Reporting stays honest (Rule 11/17) no matter which path was taken.** A
+  subset run is reported as a subset run — "ran N/M suites (impact-area: X);
+  full suite not run this iteration" — never as "suite green". The Rule 10
+  checkbox is only ever satisfied by an actual full run, shown in this cycle,
+  regardless of how many subset runs preceded it.
+- **Never an escape from a failing test.** Selection decides *which* suites
+  run; it never decides whether a failing test counts. A failing test is
+  fixed or deleted (Rule 16), never excluded from the run that would have
+  caught it.
+- **If a defect escapes that a full run would have caught**, that is Rule 18:
+  the fix must say so, and the certainty bar for that class of change tightens
+  (added to the "certainty is impossible" list above if it recurs).
+
+This does not change Rule 10, Rule 11, or the commit-gate workflow: a project
+that commits after every green run already runs full before every commit, so
+Rule 19 only ever saves the intermediate runs inside a single change — it
+never reduces what runs before anything is committed.