bmad-method-test-architecture-enterprise 1.22.1 → 1.22.2

package/.claude-plugin/marketplace.json

@@ -31,7 +31,7 @@
       "name": "bmad-method-test-architecture-enterprise",
       "source": "./",
       "description": "Master Test Architect module for quality strategy, test automation, CI/CD quality gates, and structured testing education. Part of the BMad Method ecosystem.",
-      "version": "1.22.1",
+      "version": "1.22.2",
       "author": {
         "name": "Murat K Ozcan (TEA Creator) & Brian (BMad) Madison"
       },

package/CHANGELOG.md

@@ -7,6 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.22.2] - 2026-08-14
+
+### Added
+
+- `evidence-integrity.md` knowledge fragment (core tier). Covers the two ways a suite reports a result it did not earn: a check that cannot fail (soft assertions carrying an outcome, `continue-on-error` on the test step, a runner manifest naming a subset of the suite, an assertion whose meaning differs per platform) and a diagnostic with no could-not-measure state, where a missing tool is reported as a failed condition. Also covers verifying a framework property exists and behaves before relying on it, emitting cross-boundary verdicts from the side that can observe them, stating local-versus-CI environment asymmetry, and resolving environment-dependent values before anything derives from them.
+- `mobile-ci-device-lab.md` knowledge fragment (specialized tier). Leads with the build-artifact decision: device flows run against a release-shaped or development build, never a prebuilt development shell such as Expo Go, whose absent native modules make deep links, notifications, and payments unassertable and whose launch path exists only in CI. Also covers `reactivecircus/android-emulator-runner` behavior that breaks jobs silently (the `script:` input is split on newlines and each line runs as its own `sh -c`; hardware inputs are appended to `config.ini` on every invocation, outside the AVD-creation guard, which makes the emulator reject the cached snapshot at boot), AVD snapshot cache keying, runner version pinning with a resolved-version assertion, dev-server reachability over `adb reverse` with a manifest health check that sends the client's own headers, the run-directory artifact layout, and driver-port collisions under sharding.
+- Criteria registry row `C7` (CRITICAL, applicability-gated to Maestro flows): a flow whose only assertion about its destination state carries `optional: true`, or whose sole outcome assertion follows a command the target platform does not implement, so nothing in the flow could have changed the screen. The registry now carries 32 rows, 4 of them mobile-specific.
+
+### Changed
+
+- `maestro-flows.md` gains a section on commands whose behavior does not match their name: `back` is documented for Android and Web only and the iOS driver's implementation is empty, so it no-ops while reporting COMPLETED; `hideKeyboard` on Android is documented as identical to `back`, which dismisses an open React Native modal; `index:` counts currently-rendered matches rather than list items, so it drifts under virtualization; `point:` is a sanctioned escape hatch only for elements absent from the accessibility tree. Adds the waiting facts that decide flow design (default assertion timeout, no wait-for-app-ready command, `waitForAnimationToEnd` succeeding on timeout, `retry` around a journey being an anti-pattern), plus matching anti-pattern and checklist rows.
+- `mobile-test-strategy.md` now makes the build artifact the first CI decision, and corrects the failure-diagnosis order: read per-step statuses and the hierarchy dump captured at failure, and treat the failure screenshot with suspicion because it is taken after teardown.
+- `*ci` mobile pipeline guidance now specifies the build artifact, the one-line emulator `script:` form, split AVD cache restore/save, hardware inputs on the creation step only, an asserted runner version, and run-directory artifact resolution.
+- `*automate` mobile generation requires every outcome assertion to be able to fail and every cross-platform command to be documented for both platforms or split by `runFlow: when: platform:`.
+
 ## [1.22.1] - 2026-08-13
 
 ### Removed

package/cli/lib/changed-tests.js

@@ -98,7 +98,7 @@ function resetExtraTestPatterns() {
  *
  * Code extensions are the general gate, so docs/example.spec.md is documentation,
  * not a test. Maestro is the one deliberate exception: the criteria registry
- * carries mobile rows (C4, H1, H3, H4, H9, M8, L8), so a declarative YAML flow is
+ * carries mobile rows (C4, C7, H1, H3, H4, H9, M8, L8), so a declarative YAML flow is
  * a format the ledger CAN score and it is recognized natively rather than needing
  * --test-glob. The exception is scoped to YAML under a Maestro directory (or a
  * *.flow.yaml anywhere) and excludes Maestro's own workspace config, which is

package/docs/explanation/knowledge-base-system.md

@@ -20,7 +20,7 @@ network-first,Network-First Safeguards,"Intercept-before-navigate workflow, HAR
 test-quality,Test Quality Definition of Done,"Execution limits, isolation rules, green criteria","quality,definition-of-done,tests",core,knowledge/test-quality.md
 ```
 
-`tier` is one of `core`, `extended`, or `specialized`. The 54 fragments split 20 / 19 / 15 across those tiers.
+`tier` is one of `core`, `extended`, or `specialized`. The 56 fragments split 21 / 19 / 16 across those tiers.
 
 The agent-level `resources/` directory is the reference catalog. Workflow skills also carry their own `resources/tea-index.csv` and `resources/knowledge/` directories. That duplication is intentional: workflow step frontmatter resolves `knowledgeIndex: './resources/tea-index.csv'` from `{skill-root}`, which keeps each workflow skill modular and self-contained.
 
@@ -150,7 +150,7 @@ test('job completion', async ({ apiRequest, recurse }) => {
 
 ## Related
 
-- [Knowledge Base Index](/docs/reference/knowledge-base.md) - all 54 fragments, categorized
+- [Knowledge Base Index](/docs/reference/knowledge-base.md) - all 56 fragments, categorized
 - [Testing as Engineering](/docs/explanation/testing-as-engineering.md) - the context engineering argument
 - [Test Quality Standards](/docs/explanation/test-quality-standards.md) - what `test-quality.md` encodes
 - [Network-First Patterns](/docs/explanation/network-first-patterns.md) - what `network-first.md` encodes

package/docs/explanation/verification-architecture.md

@@ -87,7 +87,7 @@ TEA's execution coverage is uneven, and the uneven part is deliberate to state r
 
 ### The self-audit
 
-Of TEA's 54 knowledge fragments, 38 name Playwright or Cypress. Two cover mobile. None covers a backend test framework, and the knowledge index has no row tagged for pytest, JUnit, Go test, xUnit, or RSpec.
+Of TEA's 56 knowledge fragments, 38 name Playwright or Cypress. Three cover mobile. None covers a backend test framework, and the knowledge index has no row tagged for pytest, JUnit, Go test, xUnit, or RSpec.
 
 That is the real shape of the gap, and it is uneven in a specific way. Web and mobile are knowledge-backed: TEA carries curated patterns for both, so what it generates follows a pattern library rather than the model's improvisation. Backend languages are not. TEA can decide to scaffold pytest, and does, but it works from the conventions named inline in the workflow step and from the project it can see. The reasoning that selects pytest is as rigorous for a Python service as for a React Native app. The execution guidance behind it is not yet equivalent.
 

package/docs/glossary/index.md

@@ -135,7 +135,7 @@ Terminology reference for Test Architect (TEA).
 | **Epic-Level Test Design**   | Test planning per epic (Phase 4) focusing on risk assessment, priorities, and coverage strategy for that specific epic.                                                                                                                                            |
 | **Fixture Architecture**     | Pattern of building pure functions first, then wrapping in framework-specific fixtures for testability, reusability, and composition.                                                                                                                              |
 | **Gate Decision**            | Go/no-go decision for release with four outcomes: PASS ✅ (ready), CONCERNS ⚠️ (proceed with mitigation), FAIL ❌ (blocked), WAIVED ⏭️ (approved despite issues).                                                                                                  |
-| **Knowledge Fragment**       | Individual markdown file in TEA's knowledge base covering a specific testing pattern or practice (54 fragments total).                                                                                                                                             |
+| **Knowledge Fragment**       | Individual markdown file in TEA's knowledge base covering a specific testing pattern or practice (56 fragments total).                                                                                                                                             |
 | **Network-First Pattern**    | Testing pattern that waits for actual network responses instead of fixed timeouts to avoid race conditions and flakiness.                                                                                                                                          |
 | **NFR Evidence Audit**       | Validation of non-functional requirement evidence (security, performance, reliability, maintainability) against defined thresholds.                                                                                                                                |
 | **No TEA**                   | Engagement model 1. Skip all TEA workflows and keep the team's existing testing approach. A valid choice when that approach already works.                                                                                                                         |

package/docs/how-to/workflows/teach-me-testing.md

@@ -32,7 +32,7 @@ Use TEA's `teach-me-testing` workflow (TEA Academy) to learn testing progressive
 4. **Test Design (60 min)** - Risk assessment and coverage planning workflow
 5. **ATDD & Automate (60 min)** - TDD red-green approach, test generation
 6. **Quality & Trace (45 min)** - Test review (5 dimensions), coverage traceability
-7. **Advanced Patterns (ongoing)** - Explore 54 knowledge fragments on-demand
+7. **Advanced Patterns (ongoing)** - Explore 56 knowledge fragments on-demand
 
 ### What You'll Gain
 
@@ -125,7 +125,7 @@ Skip fundamentals, focus on:
 
 - Session 3: TEA architecture patterns
 - Session 4: Test Design workflow
-- Session 7: Advanced patterns (54 knowledge fragments)
+- Session 7: Advanced patterns (56 knowledge fragments)
 
 **Time commitment:** 3-4 hours (highly targeted)
 
@@ -205,12 +205,12 @@ Skip fundamentals, focus on:
 
 **Topics:**
 
-- Menu-driven exploration of 54 knowledge fragments
+- Menu-driven exploration of 56 knowledge fragments
 - 5 categories: Testing Patterns, Playwright Utils, Configuration & Governance, Quality Frameworks, Auth & Security
 - Deep-dive into specific patterns as needed
 - GitHub links for browsing source
 
-**Resources:** All 54 knowledge fragments
+**Resources:** All 56 knowledge fragments
 **GitHub:** [Knowledge Base Repository](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/tree/main/src/agents/bmad-tea/resources/knowledge)
 
 ## Completion Summary
@@ -228,7 +228,7 @@ Complete all 7 sessions to receive your TEA Academy completion summary with:
 1. **Set aside dedicated time** - Each session requires focus (30-90 min)
 2. **Take your own notes** - Session notes are generated, but add personal insights
 3. **Apply immediately** - Practice concepts on your current project
-4. **Explore fragments** - Session 7 has 54 fragments for deep-dive topics
+4. **Explore fragments** - Session 7 has 56 fragments for deep-dive topics
 5. **Share with team** - Help others learn by sharing your experience
 6. **Don't rush** - Learning takes time, pause and resume as needed
 

package/docs/reference/commands.md

@@ -78,7 +78,7 @@ To ship your own workflow, package it as custom content and attach it to `bmad-t
 4. Test Design (60 min) - Risk assessment, coverage planning
 5. ATDD & Automate (60 min) - TDD red-green, test generation
 6. Quality & Trace (45 min) - Test review, traceability, metrics
-7. Advanced Patterns (ongoing) - 54 knowledge fragments exploration
+7. Advanced Patterns (ongoing) - 56 knowledge fragments exploration
 
 **Features:**
 

package/docs/reference/execution-targets.md

@@ -93,8 +93,8 @@ All shipped templates are written around a Node and browser toolchain and are ad
 
 Published deliberately, so an evaluation does not have to discover them:
 
-- **No knowledge fragments exist for any backend test framework.** 38 of 54 fragments name Playwright or Cypress and 2 cover mobile; zero cover pytest, JUnit, Go test, xUnit, or RSpec. Generation for those stacks relies on inline workflow conventions rather than a curated pattern library.
-- **Review criteria are still mostly JavaScript-shaped.** The registry carries 31 rows: 7 are portable across languages, 3 are mobile-specific, and the rest key on browser, Testing Library, Vitest, or Pact constructs. Scores on non-JavaScript, non-Maestro suites are directionally useful and not comparable to scores on a JavaScript suite.
+- **No knowledge fragments exist for any backend test framework.** 38 of 56 fragments name Playwright or Cypress and 3 cover mobile; zero cover pytest, JUnit, Go test, xUnit, or RSpec. Generation for those stacks relies on inline workflow conventions rather than a curated pattern library.
+- **Review criteria are still mostly JavaScript-shaped.** The registry carries 32 rows: 7 are portable across languages, 4 are mobile-specific, and the rest key on browser, Testing Library, Vitest, or Pact constructs. Scores on non-JavaScript, non-Maestro suites are directionally useful and not comparable to scores on a JavaScript suite.
 - **Rust is declared but incomplete.** See the generation table above.
 - **CI templates are Node-first.** See the CI section above.
 

package/docs/reference/knowledge-base.md

@@ -1,11 +1,11 @@
 ---
 title: 'TEA Knowledge Base Index'
-description: Complete index of TEA's 54 knowledge fragments for context engineering
+description: Complete index of TEA's 56 knowledge fragments for context engineering
 ---
 
 # TEA Knowledge Base Index
 
-TEA loads domain standards into context from 54 knowledge fragments, selected per workflow by the `tea-index.csv` manifest. Why that beats prompting, and how loading is wired: [Knowledge Base System](/docs/explanation/knowledge-base-system.md).
+TEA loads domain standards into context from 56 knowledge fragments, selected per workflow by the `tea-index.csv` manifest. Why that beats prompting, and how loading is wired: [Knowledge Base System](/docs/explanation/knowledge-base-system.md).
 
 This page indexes every row of that manifest. Each entry is named by its manifest `id`, which differs from the file name for a few fragments; the link resolves to the file. The `Tier` column is the manifest's own `tier` value and decides when the fragment loads (see [Loading tiers](#loading-tiers)).
 
@@ -73,19 +73,22 @@ CI/CD patterns, burn-in testing, and selective test execution.
 
 Test quality standards, test level selection, TDD patterns, and the generation-safety gate.
 
-| Fragment                                                                                                                                                                      | Tier     | Description                                                                                                                | Key Topics                           |
-| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ |
-| [test-quality](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-quality.md)                   | core     | Execution limits, isolation rules, green criteria                                                                          | DoD, best practices, anti-patterns   |
-| [test-levels](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-levels-framework.md)           | core     | Guidelines for choosing unit, integration, or end-to-end coverage                                                          | Test pyramid, level selection        |
-| [test-priorities](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-priorities-matrix.md)      | core     | P0–P3 criteria, coverage targets, execution ordering                                                                       | Prioritization, risk-based testing   |
-| [test-healing-patterns](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-healing-patterns.md) | core     | Common failure patterns and automated fixes                                                                                | Debugging, healing, fixes            |
-| [confidence-gate](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/confidence-gate.md)             | core     | 1-10 confidence score with a stop-and-ask rule below threshold, so the agent declares unknowns instead of fabricating them | Agent safety, generation, governance |
-| [component-tdd](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/component-tdd.md)                 | extended | Red→green→refactor workflow, provider isolation                                                                            | TDD, component testing               |
+| Fragment                                                                                                                                                                      | Tier     | Description                                                                                                                    | Key Topics                                |
+| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- |
+| [test-quality](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-quality.md)                   | core     | Execution limits, isolation rules, green criteria                                                                              | DoD, best practices, anti-patterns        |
+| [test-levels](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-levels-framework.md)           | core     | Guidelines for choosing unit, integration, or end-to-end coverage                                                              | Test pyramid, level selection             |
+| [test-priorities](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-priorities-matrix.md)      | core     | P0–P3 criteria, coverage targets, execution ordering                                                                           | Prioritization, risk-based testing        |
+| [test-healing-patterns](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/test-healing-patterns.md) | core     | Common failure patterns and automated fixes                                                                                    | Debugging, healing, fixes                 |
+| [confidence-gate](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/confidence-gate.md)             | core     | 1-10 confidence score with a stop-and-ask rule below threshold, so the agent declares unknowns instead of fabricating them     | Agent safety, generation, governance      |
+| [evidence-integrity](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/evidence-integrity.md)       | core     | Checks that cannot fail, three-state diagnostics, verifying framework properties before use, and stating environment asymmetry | Falsifiability, hollow green, diagnostics |
+| [component-tdd](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/component-tdd.md)                 | extended | Red→green→refactor workflow, provider isolation                                                                                | TDD, component testing                    |
 
 **Used in:** `test-design`, `atdd`, `automate`, `test-review`, `trace`
 
 `confidence-gate` covers selectors, endpoints, risk classification, fixtures, schemas, and data factories. Any generation step that cannot establish a value from the repo records it as an unknown rather than inventing it.
 
+`evidence-integrity` covers the other half of the same problem: a check that cannot go red, and a diagnostic that reports a verdict it could not measure. Both produce green with nothing behind it, which is why the review registry's CRITICAL rows exist.
+
 ---
 
 ### Risk & Gates
@@ -175,10 +178,11 @@ Delivery-side testing for asynchronous, eventually-consistent webhook flows usin
 
 Maestro device flows and the level discipline that decides what becomes a flow at all. Loaded when `test_stack_type` is `mobile` or when the review set contains a Maestro flow (`.yaml`/`.yml` under `maestro/` or `.maestro/`, or `*.flow.yaml` or `*.flow.yml`).
 
-| Fragment                                                                                                                                                                    | Tier        | Description                                                                                                     | Key Topics                                          |
-| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
-| [mobile-test-strategy](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/mobile-test-strategy.md) | specialized | Mobile test level framework, what belongs in a device flow, mobile risk categories, device matrix, CI shape     | Levels, risk, device matrix, permissions, lifecycle |
-| [maestro-flows](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/maestro-flows.md)               | specialized | Flow structure, selector hierarchy, `clearState` isolation, synchronization without sleeps, subflow composition | Maestro, selectors, isolation, anti-patterns        |
+| Fragment                                                                                                                                                                    | Tier        | Description                                                                                                                                                | Key Topics                                            |
+| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
+| [mobile-test-strategy](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/mobile-test-strategy.md) | specialized | Mobile test level framework, what belongs in a device flow, mobile risk categories, device matrix, CI shape                                                | Levels, risk, device matrix, permissions, lifecycle   |
+| [maestro-flows](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/maestro-flows.md)               | specialized | Flow structure, selector hierarchy, `clearState` isolation, synchronization without sleeps, subflow composition, command semantics that differ by platform | Maestro, selectors, isolation, anti-patterns          |
+| [mobile-ci-device-lab](https://github.com/bmad-code-org/bmad-method-test-architecture-enterprise/blob/main/src/agents/bmad-tea/resources/knowledge/mobile-ci-device-lab.md) | specialized | Build artifact selection, emulator snapshot caching, runner version pinning, dev-server reachability, artifact layout and failure diagnosis, sharding      | Build artifact, emulator, caching, pinning, artifacts |
 
 **Used in:** `framework`, `automate`, `atdd`, `test-design`, `test-review`, `ci` (when `test_stack_type` is `mobile` or a Maestro flow is present)
 
@@ -222,7 +226,7 @@ The package's remaining fragments are indexed under the category that matches wh
 
 **Location:** `src/agents/bmad-tea/resources/tea-index.csv`
 
-**Fragment location:** `src/agents/bmad-tea/resources/knowledge/` (all 54 fragments in a single directory)
+**Fragment location:** `src/agents/bmad-tea/resources/knowledge/` (all 56 fragments in a single directory)
 
 **Structure:**
 

package/docs/reference/troubleshooting.md

@@ -137,8 +137,8 @@ tea_capability_probe: true
 **Cause**: `tea-index.csv` is missing or truncated, or fragment files are missing.
 
 ```bash
-wc -l < _bmad/tea/agents/bmad-tea/resources/tea-index.csv   # 55 (header + 54 fragments)
-ls _bmad/tea/agents/bmad-tea/resources/knowledge/*.md | wc -l   # 54
+wc -l < _bmad/tea/agents/bmad-tea/resources/tea-index.csv   # 57 (header + 56 fragments)
+ls _bmad/tea/agents/bmad-tea/resources/knowledge/*.md | wc -l   # 56
 head -1 _bmad/tea/agents/bmad-tea/resources/tea-index.csv
 # id,name,description,tags,tier,fragment_file
 

package/docs/tutorials/learn-testing-tea-academy.md

@@ -60,8 +60,8 @@ Run the workflow again at any time. It detects the progress file, shows your das
 
 ## Next Steps
 
-You have completed one session of seven. The rest cover core concepts, architecture patterns, test design, ATDD and automate, quality and trace, and a menu-driven tour of the 54 knowledge fragments.
+You have completed one session of seven. The rest cover core concepts, architecture patterns, test design, ATDD and automate, quality and trace, and a menu-driven tour of the 56 knowledge fragments.
 
 - [How to Learn Testing with TEA Academy](/docs/how-to/workflows/teach-me-testing.md) for the full session list, learning paths by experience, role customization, and troubleshooting
 - [Getting Started with Test Architect](/docs/tutorials/tea-lite-quickstart.md) to generate and run real tests in 30 minutes
-- [Knowledge Base](/docs/reference/knowledge-base.md) for the 54 fragments
+- [Knowledge Base](/docs/reference/knowledge-base.md) for the 56 fragments

package/package.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://json.schemastore.org/package.json",
   "name": "bmad-method-test-architecture-enterprise",
-  "version": "1.22.1",
+  "version": "1.22.2",
   "description": "Master Test Architect for quality strategy, test automation, and release gates",
   "keywords": [
     "bmad",

package/src/agents/bmad-tea/resources/knowledge/evidence-integrity.md

@@ -0,0 +1,205 @@
+# Evidence Integrity
+
+## Principle
+
+A suite lies in two ways. A test that **cannot fail** reports coverage it does not have, and a diagnostic that **could not measure** reports a verdict it did not earn. Both produce green that means nothing, and the second one is worse, because a false negative sends the investigation somewhere wrong and every conclusion downstream inherits the error. Every check needs a way to fail. Every probe needs three states (pass, fail, and could-not-measure) and has to observe the thing it reports on rather than a proxy that correlates with it.
+
+## Rationale
+
+**The Problem**: Suites are scored by their result, so pressure runs one direction. An assertion that never fires, a step marked `continue-on-error`, a runner manifest that names three of eighteen files, a probe that reports "unreachable" when the tool it needed was missing: each converts an unknown into a green. Nothing in a CI summary distinguishes a passing check from a check that had no way to fail, and nothing distinguishes "the device could not reach the host" from "the command I used to ask was not installed."
+
+**The Solution**: Treat falsifiability as a property to verify, the same as any other. For every check, name the input that would turn it red; if you cannot, the check is decoration. For every diagnostic, separate the measurement from the verdict, make the absence of a measurement its own reported state rather than a silent fail verdict, and name what else could have made it pass.
+
+**Why This Matters**:
+
+- Green means the behavior works instead of meaning the harness ran
+- A failing diagnostic points at the real fault instead of at whichever tool was missing
+- Root-cause work stops compounding on unearned verdicts
+- Gate decisions (PASS / CONCERNS / FAIL) rest on evidence that would have shown the defect
+
+## Pattern Examples
+
+### Example 1: The Check That Cannot Fail
+
+**Context**: Four shapes found live in one suite that reported success on every run.
+
+**Implementation**:
+
+```yaml
+# ❌ Shape 1: an optional assertion downstream of a command that is a no-op here.
+# `back` is Android and Web only; on iOS it does nothing and still reports COMPLETED.
+# The assertion then cannot fail either, because `optional: true` swallows the miss.
+# Neither half can go red, and this pair asserted a Home-screen label from a screen
+# the flow never visits.
+- back
+- assertVisible:
+    text: 'Home'
+    optional: true
+
+# ✅ Falsifiable: branch the platform, then assert without an escape hatch
+- runFlow:
+    when:
+      platform: Android
+    commands:
+      - back
+- runFlow:
+    when:
+      platform: iOS
+    commands:
+      - tapOn:
+          id: 'nav_back_button'
+- assertVisible:
+    id: 'home_screen_root'
+```
+
+```yaml
+# ❌ Shape 2: the job that runs the tests cannot turn the build red
+- name: E2E flows
+  continue-on-error: true
+  run: maestro test maestro/
+
+# ✅ continue-on-error belongs on artifact collection, never on the test step
+- name: E2E flows
+  run: maestro test maestro/
+- name: Upload artifacts
+  if: always()
+  uses: actions/upload-artifact@v4
+```
+
+```yaml
+# ❌ Shape 3: the manifest names 3 of the 18 flows in the directory.
+# The suite is green because fifteen files never ran.
+flows:
+  - login.yaml
+  - checkout.yaml
+  - profile.yaml
+
+# ✅ Include by pattern, exclude by exception, and assert the executed count
+flows:
+  - '*.yaml'
+```
+
+**Shape 4** has no snippet, because the step looks correct: an assertion passes on iOS because the element is still in the hierarchy behind a presented modal, and fails on Android where the modal replaces the hierarchy. Same assertion, different meaning per platform. Any assertion whose truth depends on how a platform composes its view tree needs its own per-platform expectation, not one shared line.
+
+**Key points**:
+
+- Name the input that would turn each check red. If none exists, the check is decoration.
+- `optional: true`, `continue-on-error`, a partial manifest, and a soft assertion are the four common ways a result stops being falsifiable.
+- **When you make a hollow check falsifiable and it goes red, the red is the finding.** It is a defect that was always there and is now visible. Reporting it as a regression you introduced is the wrong read and usually gets the fix reverted.
+
+### Example 2: Diagnostics Need a Could-Not-Measure State
+
+**Context**: A probe checking whether a device can reach a host. The tool it invokes is not installed on that device.
+
+**Implementation**:
+
+```bash
+# ❌ Two states only. A missing binary is indistinguishable from a real failure,
+# and this reported "device cannot reach the network" three times in one session
+# while the network was fine.
+if adb shell "wget -q -O - http://10.0.2.2:8081/status"; then
+  echo "PASS: device reached the dev server"
+else
+  echo "FAIL: device cannot reach the dev server"
+fi
+
+# ✅ Three states. Establish the instrument before trusting the reading.
+if ! adb shell 'command -v curl >/dev/null 2>&1'; then
+  echo "COULD-NOT-MEASURE: no HTTP client on the device; reachability unknown"
+  exit 77
+fi
+if adb shell "curl -fsS http://127.0.0.1:8081/status >/dev/null 2>&1"; then
+  echo "PASS: device reached the dev server"
+else
+  echo "FAIL: device reached the network stack and the request did not succeed"
+fi
+```
+
+**A probe fails in two directions, and both produce a confident wrong answer.** The block above is the strict failure: a missing instrument read as a false condition. The permissive failure is subtler and harder to catch, because it produces a green. The same investigation later probed reachability by having the device open a TCP connection to a port that `adb reverse` had mapped. With a reverse mapping in place the device always has a local listener on that port, so the connect succeeds whether or not anything on the host is behind it. The probe measured that a mapping existed and reported that a service was reachable.
+
+**Rule**: a probe must observe the thing it claims to observe, never a proxy that merely correlates with it. Ask what else could make this check pass. Here the fix is the same as Example 4: open a throwaway listener inside the test process and assert that the process itself accepted a socket.
+
+**Key points**:
+
+- A non-zero exit means "the command failed," which is not the same claim as "the condition is false"
+- A zero exit means "the command succeeded," which is not the same claim as "the condition is true"
+- Reserve a distinct exit code and a distinct log word for could-not-measure so it never reads as a fail
+- Apply this to every derived verdict, including the ones a harness prints as a convenience line. A convenience line is quoted later as evidence.
+
+### Example 3: Verify the Property Exists, Then Verify It Behaves
+
+**Context**: Two separate failures, one about existence and one about behavior.
+
+- **Existence**: a config key was invented from a plausible name and committed. The runner rejected the file on a parse error and 18 flows never executed. Nothing in the suite name suggested the cause.
+- **Behavior**: a comment claimed a command was "a left-edge swipe on iOS." The implementation is an empty method that reports success. The comment propagated into other files and into a second session's reasoning before anyone opened the source.
+
+**Rule**: before using a framework property, confirm it exists in the version you pin, from the docs or the shipped artifact. Before writing a comment that asserts **why** something works, confirm the mechanism from the docs or the source. A comment stating a mechanism is a claim with the same evidentiary standing as an assertion, and it is more dangerous, because nothing tests it.
+
+Corollary for reviewers: "X is not supported / is platform-specific / only works on Y" needs a citation. One session asserted a flag was GNU-only when the platform's own manual documents it.
+
+### Example 4: Take the Verdict on the Side That Can Prove It
+
+**Context**: A host-side reachability check used to argue that a device could reach a service.
+
+The host and the device are different network namespaces. A host that resolves and connects proves the host's route and nothing about the guest's. Move the assertion to the side whose route is in question: open a throwaway listener on the host, have the device connect to it, and let the verdict be "this process observed the socket." Structure every environment claim so the proving party is the one that emits the result.
+
+### Example 5: State the Environment Asymmetry Before Arguing From Local to CI
+
+**Context**: A local pass used as evidence about a CI failure.
+
+Write the asymmetry down whenever a local result enters a CI argument:
+
+| Axis          | Local                                      | CI                                 |
+| ------------- | ------------------------------------------ | ---------------------------------- |
+| OS / arch     | macOS, arm64                               | Linux, x86_64                      |
+| Platform ver. | newest API level                           | two levels older                   |
+| Image variant | vendor image with store services signed in | plain AOSP-style image, no account |
+| Acceleration  | native hypervisor                          | KVM, may be unavailable            |
+| Provisioning  | long-lived machine                         | fresh runner every job             |
+
+A local pass proves the application path. It proves nothing about acceleration, snapshot restore, `PATH` handling, or an image variant the local machine never runs. Naming the axes converts "it works on my machine" from an argument into a scoped fact.
+
+### Example 6: Resolve Environment-Dependent Values Before Anything Derives From Them
+
+**Context**: A harness that probed which host address the device could reach, and ran the probe after that address had already been baked into the built artifact.
+
+The probe would have reported the right answer and changed nothing, and the app would have loaded over one route while its API calls went over another. Order of operations is part of correctness in a harness: resolve every environment-dependent value first, then derive. If a value is discovered after its consumers are built, the discovery is telemetry rather than configuration.
+
+## Anti-Patterns
+
+| Anti-pattern                                            | Why it fails                                                                 | Fix                                                                     |
+| ------------------------------------------------------- | ---------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
+| Assertion with a soft or optional modifier as default   | Cannot go red; reports coverage that does not exist                          | Reserve softness for genuinely optional UI, and assert the outcome hard |
+| `continue-on-error` on the test step                    | The suite cannot fail the build                                              | Put it on artifact collection only; use `if: always()` for uploads      |
+| Runner manifest listing a subset of the suite           | Files silently never run; the count is the only clue                         | Include by pattern; assert the executed count against the file count    |
+| Missing tool reported as a failed condition             | Sends the investigation at the wrong subsystem                               | Three-state probes; distinct exit code for could-not-measure            |
+| Probe observing a proxy that correlates with the target | Passes for a reason unrelated to the claim, and a green is never re-examined | Ask what else could make this pass; observe the thing itself            |
+| Verdict emitted by the side that cannot observe it      | Proves the wrong namespace                                                   | Move the assertion to the party whose route or state is in question     |
+| Comment asserting a mechanism with no source read       | Propagates into other files and into other people's reasoning                | Cite the doc or source line, or omit the mechanism                      |
+| Local result used as a CI argument, asymmetry unstated  | Hides the axes that actually differ                                          | Tabulate the differing axes with the claim                              |
+| Environment probe running after its consumers           | The answer arrives too late to configure anything                            | Resolve environment-dependent values first, then derive                 |
+| Reverting a newly-red check as a regression             | Restores the hollow green and loses the finding                              | Treat the red as the pre-existing defect it exposed                     |
+
+## Evidence Integrity Checklist
+
+- [ ] **Every check is falsifiable**: for each assertion, the input that turns it red is nameable
+- [ ] **No soft assertion by default**: optional modifiers only on genuinely optional UI, with a comment
+- [ ] **No `continue-on-error` on a test step**: only on artifact collection
+- [ ] **Executed count reconciled**: the number of tests that ran matches the number of test files discovered
+- [ ] **Platform-divergent assertions split**: no single assertion whose meaning depends on how a platform composes its view tree
+- [ ] **Probes are three-state**: pass, fail, and could-not-measure, with distinct exit codes
+- [ ] **Probes observe their own claim**: no proxy that merely correlates with the condition being reported
+- [ ] **Instruments verified before readings**: the probe confirms its tool exists before interpreting its result
+- [ ] **Framework properties verified**: every key, flag, and command confirmed against the pinned version's docs or artifact
+- [ ] **Mechanism comments cited**: any comment claiming why something works names its source
+- [ ] **Verdicts emitted by the proving party**: cross-boundary claims asserted on the side that can observe them
+- [ ] **Environment asymmetry stated**: local-versus-CI arguments list the differing axes
+- [ ] **Resolution precedes derivation**: environment-dependent values resolved before any consumer is built
+
+## Integration Points
+
+- **Used in workflows**: `*test-review` (the CRITICAL rows exist to catch checks that cannot fail), `*ci` (gate wiring and artifact steps), `*nfr-assess` (a measurement that could not be taken is CONCERNS, never PASS), `*trace` (coverage claims), `*automate` and `*atdd` (generated checks must be falsifiable)
+- **Related fragments**: `confidence-gate.md` (do not fabricate the artifact in the first place), `test-quality.md` (determinism and isolation), `risk-governance.md` (what a gate decision may rest on), `mobile-ci-device-lab.md` (where these failures concentrate on mobile)
+- **Tools**: any CI summary, the runner's own executed-test count, exit codes
+
+_Source: TEA quality-gate standards; hollow-green and false-negative diagnostic patterns observed in a live mobile CI investigation_