genat-mcp 2.3.3 → 2.3.5

package/README.md

@@ -6,6 +6,10 @@ Full setup (n8n workflow, services, usage) is in the [project README](../README.
 
 **For teams:** See [GENAT_OVERVIEW.md](GENAT_OVERVIEW.md) for a concise, presentation-ready overview. Suitable as a source for NotebookLM or other AI tools to create slide decks.
 
+**Coverage claims:** Teams can copy [examples/coverage-manifest.example.yaml](examples/coverage-manifest.example.yaml) into their own repo (e.g. as `coverage-manifest.yaml`) and adjust it to document which standards, axe tags, workflows, and verification steps apply to their deployment. The manifest does not replace manual accessibility review; it aligns documented claims with the analyzer defaults and GenAT workflow behavior.
+
+**CI (Phase D):** The parent repo includes [scripts/ci/a11y-analyzer-scan.mjs](../scripts/ci/a11y-analyzer-scan.mjs) and a [GitHub Actions example](../templates/github-actions/a11y-analyzer-url.example.yml) to save analyzer JSON and gate on violation impact—useful even when GenAT is not invoked.
+
 ## Install
 
 **If the package is published to npm** (otherwise you get 404):
@@ -162,6 +166,7 @@ If the global binary is not on PATH, use the full path, e.g. `node $(npm root -g
 - **Node.js** 20+
 - **n8n** with a GenAT workflow imported and activated. Use [genat-accessibility-tests-with-login-jira.json](../workflows/genat-accessibility-tests-with-login-jira.json) (path `webhook-test/webhook-genat-login-jira`) for JIRA support, or [genat-accessibility-tests-with-login.json](../workflows/genat-accessibility-tests-with-login.json) (path `webhook-test/webhook-genat-login`). Set **N8N_WEBHOOK_URL** to the workflow's webhook URL.
 - **Optional v2 pipeline (Python / pytest-bdd quality):** Import [GenAT Accessibility Tests (with Login + JIRA) v2.json](../workflows/GenAT%20Accessibility%20Tests%20(with%20Login%20+%20JIRA)%20v2.json) and set **N8N_WEBHOOK_URL** to `…/webhook-test/webhook-genat-login-jira-v2`. v2 adds stricter pytest-bdd + Python rules (no nested markdown in code, layout hints, fixture-aware steps) and can run alongside the original workflow. The original v1 workflow is unchanged.
+- **Optional v3 pipeline (a11y contract validation):** Import [GenAT Accessibility Tests (with Login + JIRA) v3.json](../workflows/GenAT%20Accessibility%20Tests%20(with%20Login%20+%20JIRA)%20v3.json) and set **N8N_WEBHOOK_URL** to `…/webhook-test/webhook-genat-login-jira-v3`. Same as v2, plus a terminal **A11y contract validator** node that sets **`a11yContractOk`** (boolean) and **`a11yContractErrors`** (string[]) on the webhook JSON. The MCP also exposes **`_a11yContractOk`** / **`_a11yContractErrors`**. Optional JSON body **`failOnA11yContract`: true** fails the execution when the contract check fails (default is warn-only: still returns generated code). To rebuild v3 from an updated v2 in this repo, run `node scripts/build-genat-v3-workflow.mjs` from the **n8n_playwright_tests** root (regenerates **`webhookId`**; re-import or update the webhook in n8n if needed).
 - **DOM Analyzer** and **Accessibility Analyzer** services running (see main README: `npm run services` from repo root)
 
 ## Standalone script: run-genat.mjs
@@ -212,7 +217,7 @@ When creating run-genat.mjs: Use webhook-genat-login, NOT webhook-genat. Prefer
 
 - **url** (string): Page URL to analyze for accessibility.
 - **parentProjectFolder** (string): Path to the project root (TypeScript, JavaScript, or Python Playwright).
-- **writeToProject** (boolean, optional): If `true`, write generated files under `tests/accessibility/` by default; for **pytest-bdd** Python projects, files go under `features/` / `step_defs/` (or `tests/features`, `steps`, etc.) when those directories exist.
+- **writeToProject** (boolean, optional): If `true`, write generated files under `tests/accessibility/` by default; for **pytest-bdd** Python projects, files go under `features/` / `step_defs/` (or `tests/features`, `steps`, etc.) when those directories exist. New projects get **`step_defs/test_accessibility_steps.py`** when `accessibility_steps.py` is not already present.
 - **maxAssertions** (number, optional): Override default assertion count for complex pages (e.g. `25`).
 - **scopeToRegions** (string[], optional): Limit tests to specific regions (e.g. `["header", "main", "table"]`).
 - **analyzeStates** (boolean, optional): If `true`, run axe in expanded dropdown/combobox states and include state-specific violations. Increases analysis time.
@@ -220,12 +225,13 @@ When creating run-genat.mjs: Use webhook-genat-login, NOT webhook-genat. Prefer
 - **loginPassword** (string, optional): Login password. Overrides .env when provided. Requires loginUsername.
 - **loginUrl** (string, optional): Login page URL if different from target URL.
 - **jiraNumber** (string, optional): JIRA issue key (e.g. PROJ-123). When provided, the workflow fetches the issue and injects acceptance criteria into the AI prompt.
+- **failOnA11yContract** (boolean, optional): When **N8N_WEBHOOK_URL** points at the **v3** JIRA workflow (`webhook-genat-login-jira-v3`), set `true` to make n8n error if the **A11y contract validator** fails (default is warn-only: JSON still returns generated files with `a11yContractOk` / `a11yContractErrors`).
 
 Login credentials: provide in the prompt (loginUsername, loginPassword) or in project `.env` (TEST_USER, TEST_PASSWORD, LOGIN_USER, LOGIN_PASSWORD). Tool params override .env. For sensitive environments, prefer .env (gitignored) since credentials in the prompt are visible in chat history.
 
 Framework detection: language (TypeScript vs JavaScript vs Python), BDD (Cucumber, pytest-bdd, Behave), and Page Object pattern (e.g. `pages/` directory or `*Page.ts`, `*Page.js`, `*_page.py` files). Returns JSON with `testCode`, `featureFile`, `stepDefCode`, `pageObjectCode` (and `_written` paths when writing).
 
-**Response fields for calling applications:** When `stepDefCode` is present, the MCP may set **`_stepDefIncomplete`** (boolean) and **`_stepDefIncompleteReason`** (string) using a lightweight heuristic (unclosed strings/delimiters, line continuation at EOF). Treat `true` as a signal to warn the user, retry generation, or block merge—do not assume the step file is runnable. Other optional fields from the workflow include **`usedLogin`** and **`loginReason`** when using login-capable workflows.
+**Response fields for calling applications:** When `stepDefCode` is present, the MCP may set **`_stepDefIncomplete`** (boolean) and **`_stepDefIncompleteReason`** (string) using a lightweight heuristic (unclosed strings/delimiters, line continuation at EOF). Treat `true` as a signal to warn the user, retry generation, or block merge—do not assume the step file is runnable. Other optional fields from the workflow include **`usedLogin`** and **`loginReason`** when using login-capable workflows. With the **v3** JIRA workflow, responses may include **`a11yContractOk`** / **`a11yContractErrors`** and mirrored **`_a11yContractOk`** / **`_a11yContractErrors`** (static checks for `withTags` / `AxeBuilder` on TS/JS and axe-playwright-python usage on Python).
 
 Generated tests include keyboard accessibility checks (Tab order, focus) by default. For complex pages, the workflow uses page complexity (regions, tables, widgets) to scale assertion count.
 
@@ -234,10 +240,15 @@ Generated tests include keyboard accessibility checks (Tab order, focus) by defa
 GenAT does not guarantee that every Gherkin line has an implemented step. For **pytest-bdd** projects, the calling application or CI can:
 
 1. Parse `*.feature` step lines (after `Given`/`When`/`Then`/`And`/`But`).
-2. Confirm `accessibility_steps.py` (or your step module) defines matching `@given`/`@when`/`@then` patterns (or run `pytest --collect-only` / a dry run).
+2. Confirm `test_accessibility_steps.py` or `accessibility_steps.py` (your step module) defines matching `@given`/`@when`/`@then` patterns (or run `pytest --collect-only` / a dry run).
 
 Keep this as an optional pre-merge check alongside `_stepDefIncomplete`.
 
+### pytest-bdd discovery and axe-playwright-python
+
+- **Pytest** collects `test_*.py` by default. GenAT now prefers writing **`step_defs/test_accessibility_steps.py`** for new projects; if **`accessibility_steps.py`** already exists, that path is used. Ensure **`@scenario`** test functions are in a collected module (often the same file when named `test_*.py`) or import step defs from **`conftest.py`** / **`pytest_plugins`**.
+- **axe-playwright-python (sync):** `from axe_playwright_python.sync_playwright import Axe`; `axe = Axe()`; `results = axe.run(page)`; read **`results.response`** (axe-core JSON) and **`results.violations_count`**. Do not invent other APIs—see [axe-playwright-python docs](https://pamelafox.github.io/axe-playwright-python/).
+
 ## Sample prompts
 
 See [prompts.md](prompts.md) for a full list of prompts including specific parent folders, login, JIRA, and combined options.

package/detect-framework.js

@@ -77,15 +77,21 @@ function collectProjectLayoutHints(root, scriptType, bddFramework) {
   }
 
   if (existsSync(join(root, 'step_defs'))) {
-    parts.push('Put step definitions under step_defs/ (e.g. accessibility_steps.py)');
+    parts.push(
+      'Put step definitions under step_defs/; prefer test_accessibility_steps.py (test_*.py) so pytest discovers @scenario tests, or keep accessibility_steps.py and import that module from conftest.py'
+    );
   } else if (existsSync(join(root, 'steps'))) {
-    parts.push('Put step definitions under steps/');
+    parts.push('Put step definitions under steps/; prefer test_*.py naming for pytest collection or import via conftest.py');
   } else if (existsSync(join(root, 'tests', 'step_defs'))) {
-    parts.push('Put step definitions under tests/step_defs/');
+    parts.push('Put step definitions under tests/step_defs/; prefer test_accessibility_steps.py or conftest import');
   } else {
-    parts.push('Prefer step_defs/ for pytest-bdd step modules');
+    parts.push('Prefer step_defs/ for pytest-bdd; use test_accessibility_steps.py for pytest discovery of scenario tests');
   }
 
+  parts.push(
+    'Pytest collects test_*.py by default; ensure BDD scenario functions (@scenario) live in a collected module or are pulled in via conftest/pytest_plugins'
+  );
+
   const conftestPaths = walkWithPaths(root, (f) => f === 'conftest.py');
   const fixtureNames = new Set();
   for (const p of conftestPaths.slice(0, 3)) {

package/index.js

@@ -59,7 +59,7 @@ function insecureHttpsFetch(url, { method = 'GET', headers = {}, body, signal })
 const server = new McpServer(
   {
     name: 'GenAT',
-    version: '2.3.3',
+    version: '2.3.5',
   },
   {
     capabilities: {
@@ -122,6 +122,12 @@ server.registerTool(
         .string()
         .optional()
         .describe('JIRA issue key (e.g. PROJ-123, RB-40039). Extract from prompts like "JIRA RB-40039" or "JIRA ticket PROJ-123". Pass only the key (e.g. RB-40039), not "JIRA RB-40039".'),
+      failOnA11yContract: z
+        .boolean()
+        .optional()
+        .describe(
+          'v3 workflow only (webhook-genat-login-jira-v3): if true, n8n errors when static a11y contract checks fail on generated code. Default false: response still includes a11yContractOk / a11yContractErrors.'
+        ),
     },
   },
   async ({
@@ -137,6 +143,7 @@ server.registerTool(
     requiresAuth,
     forceLogin,
     jiraNumber,
+    failOnA11yContract,
   }) => {
     if (!url || typeof url !== 'string') {
       return {
@@ -205,6 +212,7 @@ server.registerTool(
       ...(requiresAuth === true && { requiresAuth: true }),
       ...(forceLogin === true && { forceLogin: true }),
       ...(jiraNumber && jiraNumber.trim() && { jiraNumber: jiraNumber.trim() }),
+      ...(failOnA11yContract === true && { failOnA11yContract: true }),
     };
 
     const useInsecureTls = N8N_INSECURE_TLS && N8N_WEBHOOK_URL.startsWith('https://');
@@ -290,6 +298,13 @@ server.registerTool(
       if (inc.reason) data._stepDefIncompleteReason = inc.reason;
     }
 
+    if (typeof data.a11yContractOk === 'boolean') {
+      data._a11yContractOk = data.a11yContractOk;
+    }
+    if (Array.isArray(data.a11yContractErrors)) {
+      data._a11yContractErrors = data.a11yContractErrors;
+    }
+
     if (writeToProject && resolvedPath && data) {
       if (data.bddFramework == null) data.bddFramework = framework.bddFramework;
       try {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "genat-mcp",
-  "version": "2.3.3",
+  "version": "2.3.5",
   "mcpName": "io.github.asokans@oclc.org/genat",
   "description": "MCP server GenAT: generate accessibility tests via n8n workflow (url + project folder)",
   "type": "module",

package/pytest-bdd-paths.js

@@ -1,11 +1,29 @@
 /**
  * Resolve pytest-bdd feature / step / spec paths from project layout (same rules as write-files).
+ * Step file: prefer legacy accessibility_steps.py if present; otherwise test_accessibility_steps.py
+ * so pytest discovers scenario tests (test_*.py).
  */
 import { existsSync } from 'fs';
 import { join } from 'path';
 
 const DEFAULT_DIR = 'tests/accessibility';
 
+const LEGACY_STEP = 'accessibility_steps.py';
+const TEST_STEP = 'test_accessibility_steps.py';
+
+/**
+ * @param {string} root
+ * @param {string[]} dirSegments - e.g. ['step_defs']
+ * @returns {string} relative path to step file
+ */
+function pickStepFileUnderDir(root, dirSegments) {
+  const legacy = join(...dirSegments, LEGACY_STEP);
+  const testNamed = join(...dirSegments, TEST_STEP);
+  if (existsSync(join(root, legacy))) return legacy;
+  if (existsSync(join(root, testNamed))) return testNamed;
+  return testNamed;
+}
+
 /**
  * @param {string} root
  * @returns {{ featureRel: string, stepRel: string, specRel: string }}
@@ -17,11 +35,11 @@ export function resolvePytestBddPaths(root) {
   else featureRel = join('features', 'accessibility.feature');
 
   let stepRel;
-  if (existsSync(join(root, 'step_defs'))) stepRel = join('step_defs', 'accessibility_steps.py');
-  else if (existsSync(join(root, 'steps'))) stepRel = join('steps', 'accessibility_steps.py');
-  else if (existsSync(join(root, 'tests', 'step_defs'))) stepRel = join('tests', 'step_defs', 'accessibility_steps.py');
-  else if (existsSync(join(root, 'tests', 'steps'))) stepRel = join('tests', 'steps', 'accessibility_steps.py');
-  else stepRel = join('step_defs', 'accessibility_steps.py');
+  if (existsSync(join(root, 'step_defs'))) stepRel = pickStepFileUnderDir(root, ['step_defs']);
+  else if (existsSync(join(root, 'steps'))) stepRel = pickStepFileUnderDir(root, ['steps']);
+  else if (existsSync(join(root, 'tests', 'step_defs'))) stepRel = pickStepFileUnderDir(root, ['tests', 'step_defs']);
+  else if (existsSync(join(root, 'tests', 'steps'))) stepRel = pickStepFileUnderDir(root, ['tests', 'steps']);
+  else stepRel = join('step_defs', TEST_STEP);
 
   const specRel = join(DEFAULT_DIR, 'accessibility_test.py');
   return { featureRel, stepRel, specRel };