@jrpool/kilotest 46.1.2 → 47.0.1

package/docs/BACKLOG.md

@@ -0,0 +1,356 @@
+# Backlog
+
+Engineering tasks and risks that are not yet scheduled.
+
+Items marked completed are preserved for about 2 weeks in case of production bugs.
+
+## Document alerting
+
+Document when and how Kilotest sends alerts to its maintainer, including the subscription to Resend and the DNS configuration for Resend authentication.
+
+## Decide whether to tighten the DMARC policy and add SPF alignment for Resend
+
+Live DNS for `kilotest.com`, checked 2026-09-16, shows a DMARC record of `v=DMARC1; p=none;` (monitoring only, no `rua=` reporting address) and an SPF record (`v=spf1 include:_spf.porkbun.com ~all`) that does not include `resend.com`. Mail sent via Resend is currently authenticated only by the DKIM record at `resend._domainkey.kilotest.com`. This may be an intentional, working minimal configuration, or an oversight from Resend's onboarding; it has not been confirmed against the Resend dashboard's domain-verification status. Consider confirming, in the Resend dashboard, whether SPF alignment is recommended or required for this domain's deliverability, and once alert deliverability has been stable for a period, whether to tighten DMARC from `p=none` to `p=quarantine` and add an `rua=` address so DMARC aggregate reports reach the maintainer.
+
+## Widen test-request duplicate detection
+
+The duplicate check in `addTestRequest` (`util.ts`), reached by all four test/retest request paths (UI `/requestTest.html` and `/requestRetest.html`; API `requestTest` and `requestRetest` operations), compares an incoming request only against other requests currently pending in `db/jobs/testRequests.json`, i.e., submitted but not yet manually approved by the maintainer into a job. Once a pending request is approved into a job (`web/enqueue/index.ts`), all pending requests for that URL are cleared from `testRequests.json`, so this shared check cannot recognize a duplicate against a request that has already been approved, is currently running, or has already completed. An identical request submitted after approval triggers a fresh alert rather than being filtered.
+
+The four paths differ in what, if anything, they check before reaching this shared dedup, and none of the four closes the gap:
+
+- UI `requestTest` (`web/requestTest/index.ts`) additionally calls `getRequestability`, rejecting the request outright if the URL matches a claimed or queued job. This screens out most already-approved, not-yet-completed requests for a new test, but not ones for a completed report, since `getRequestability` does not consult reports.
+- API `requestTest` (`api/requestTest.ts`) additionally checks `getReportExtracts` and rejects the request if a report already exists for the same description and URL, but, unlike the UI path, does not call `getRequestability`, so it does not screen out a URL that is currently claimed or queued as a job.
+- UI `requestRetest` (`web/requestRetest/index.ts`) and API `requestRetest` (`api/requestRetest.ts`) call neither `getRequestability` nor a same-description-and-URL report check; each only confirms the report being retested is still the latest one for its page before falling through to the shared dedup.
+
+Consider widening the shared `addTestRequest` check itself, so that all four paths benefit uniformly, to also compare against approved, claimed, queued, or completed requests, for example recently created jobs or reports for the same URL and description, and if so over what time window, rather than continuing to rely on each path's own inconsistent pre-check.
+
+## Improve MCP-zero discoverability
+
+### Advice from Gemini
+
+To make your **kilotest** MCP server discoverable by **MCP-Zero** agent loops (or enterprise gateways using active tool discovery architectures), you need to optimize how your server represents itself to semantic routers.
+
+Because MCP-Zero operates as a **two-stage semantic routing system** (first matching the server domain, then matching the specific tools via vector embeddings), a standard MCP list_tools setup is not enough. You must optimize your metadata and schemas so that indexers can crawl and agents can query your server dynamically on demand.
+
+Follow these steps to make your server fully discoverable:
+
+### **1. Re-Architect Your Tool Naming & Descriptions**
+
+MCP-Zero relies heavily on the LLM’s ability to articulate an *Active Tool Request* based on semantic gaps. Ambiguous tool names like `listReports` are the primary reason a server gets passed over during a vector lookup.
+
+Change your tool declarations to be unambiguous and explicitly tied to your domain (ensemble front-end quality and accessibility testing):
+
+*`// ❌ Old Anti-Pattern for MCP-Zero`*
+`{`
+  `name: "listReports",`
+  `description: "Lists all reports."`
+`}`
+
+*`// ✅ Optimized Pattern for MCP-Zero Discovery`*
+`{`
+  `name: "kilotest_list_web_quality_reports",`
+  `description: "Retrieves a historical catalog of ensemble front-end quality test reports for web pages. Use this to check whether a specific URL has already been audited for accessibility (WCAG), usability, and W3C standards conformity."`
+`}`
+
+**Prefix your tools:** Use a uniform prefix like `kilotest_` to prevent namespaces from colliding with other standard testing tools when flattened into an agent’s runtime.
+
+**Inject key synonyms in descriptions:** Notice how the bad example just repeats the name. The optimized description explicitly seeds semantic keywords (**accessibility, WCAG, usability, standards, audit**). If an agent thinks, *“I need to check if this site is accessible for screen readers,”* the embedding vectors will match your tool.
+
+### **2. Deepen Your Parameter Schemas**
+
+MCP-Zero models often inspect input expectations before mounting a server to ensure they can fulfill the schema requirements. Provide dense descriptions for arguments instead of generic labels:
+
+`"url_filter": {`
+  `"type": "string",`
+  `"description": "Optional target web page URL to filter reports for (e.g., 'https://example.com'). Helps find existing accessibility and front-end compliance history for this specific site."`
+`}`
+
+(Note from maintainer: This feature does not yet exist, so this is an example illustrating a principle but cannot be applied verbatim.)
+
+### **3. Maximize Your Centralized Registry Metadata (package.json)**
+
+Since kilotest is a public npm-published server already registered on community indexes (like Smithery and Glama), ensure your package file is effectively feeding their background indexers. MCP-Zero ingestion scripts crawl these registries and rely on the keywords and description tags to build their top-level server embedding maps.
+
+Update your package.json to feature dense, functional keywords:
+
+`{`
+  `"name": "kilotest",`
+  `"description": "Model Context Protocol (MCP) server for Kilotest ensemble web testing. Runs over 1,300 front-end tests across multiple rule engines covering WCAG accessibility, usability, performance, and web standards.",`
+  `"keywords": [`
+    `"mcp",`
+    `"model-context-protocol",`
+    `"mcp-server",`
+    `"accessibility-testing",`
+    `"wcag",`
+    `"web-quality-audit",`
+    `"automated-qa",`
+    `"ensemble-testing"`
+  `]`
+`}`
+
+### **4. Provide a "Capability Discovery" Tool**
+
+A useful pattern for MCP-Zero discovery loops is exposing a zero-argument metadata tool, such as kilotest_get_server_capabilities. If an agent routes to your server but needs to confirm exactly what it can check before running an audit, this lightweight tool can return a deterministic list summarizing the specific rule engines inside kilotest.
+
+Here are both the **TypeScript schema definitions** to update your tool definitions in code, and an **automated script** to generate an optimized manifest file (`mcp-manifest.json`) for your npm release.
+
+### **1. The Optimized TypeScript Tool Schemas**
+
+Replace your existing tool definitions with these semantically rich schemas. They are designed to rank highly when evaluated by an MCP-Zero semantic routing loop.
+
+`import {Tool} from "@modelcontextprotocol/sdk/types.js";`
+
+`export const KILOTEST_TOOLS: Tool[] = [`
+  `{`
+    `name: "kilotest_list_web_quality_reports",`
+    `description: "Retrieves a historical catalog of ensemble front-end quality and compliance test reports. Use this tool to check if a specific web page or URL has already been audited for accessibility (WCAG), performance benchmarks, semantic HTML correctness, or usability standards.",`
+    `inputSchema: {`
+      `type: "object",`
+      `properties: {`
+        `url_filter: {`
+          `type: "string",`
+          `description: "Optional URL to filter reports for (e.g., 'https://example.com'). Helps identify existing compliance test history for a specific website."`
+        `},`
+        `limit: {`
+          `type: "number",`
+          `description: "Maximum number of historical reports to return. Defaults to 10.",`
+          `default: 10`
+        `}`
+      `}`
+    `}`
+  `},`
+  `{`
+    `name: "kilotest_run_ensemble_audit",`
+    `description: "Triggers a live, comprehensive front-end engineering audit on a target URL. Runs 1,300+ automated test checks across 12 rule engines simultaneously to evaluate WCAG accessibility compliance, mobile responsiveness, SEO configurations, and core web vitals performance.",`
+    `inputSchema: {`
+      `type: "object",`
+      `properties: {`
+        `url: {`
+          `type: "string",`
+          `description: "The absolute web page URL to inspect (must include http:// or https://)."`
+        `},`
+        `scan_depth: {`
+          `type: "string",`
+          `enum: ["single_page", "shallow_crawl"],`
+          `description: "Depth of the quality sweep. 'single_page' tests only the target URL; 'shallow_crawl' includes immediately linked internal assets."`
+        `}`
+      `},`
+      `required: ["url"]`
+    `}`
+  `},`
+  `{`
+    `name: "kilotest_get_server_capabilities",`
+    `description: "Exposes a metadata summary of the 12 underlying rule engines configured inside this kilotest instance. Run this zero-argument tool to inspect exactly what front-end compliance standards (such as WCAG 2.2, W3C HTML, or performance budgets) this server is currently capable of evaluating.",`
+    `inputSchema: {`
+      `type: "object",`
+      `properties: {}`
+    `}`
+  `}`
+`];`
+
+### **2. Auto-Generation Script (`generate-manifest.js`)**
+
+MCP-Zero frameworks and platform indexers often ingest a static metadata file to cache tool definitions before spawning your server.
+
+Create this file in your project root as `scripts/generate-manifest.js`. It pulls details directly from your package config and spits out a completely optimized index file.
+
+`import fs from 'fs';`
+`import path from 'path';`
+
+*`// Read your existing package.json`*
+`const packageJsonPath = path.resolve(process.cwd(), 'package.json');`
+`const pkg = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));`
+
+*`// Construct the high-fidelity MCP discovery manifest`*
+`const mcpManifest = {`
+  `manifest_version: "1.0.0",`
+  `server: {`
+    `name: "kilotest",`
+    `version: pkg.version,`
+    `description: "Official Model Context Protocol (MCP) server for Kilotest ensemble web testing. Programmatically evaluates front-end code bases across 1,300 validation checks covering WCAG 2.1/2.2 accessibility, performance budgets, markup standards, and cross-browser UX compliance.",`
+    `homepage: pkg.homepage || "https://kilotest.com",`
+    `routing_tags: [`
+      `"accessibility", "wcag", "web-standards", "html-validation",`
+      `"ux-audit", "front-end-qa", "performance-benchmarks", "automated-testing"`
+    `]`
+  `},`
+  `// Static schema fallback for zero-install discovery loops`
+  `discovery_schemas: {`
+    `transport: "stdio",`
+    `recommended_namespace: "kilotest"`
+  `}`
+`};`
+
+*`// Write it to your distribution/root folder`*
+`const outputPath = path.resolve(process.cwd(), 'mcp-manifest.json');`
+`fs.writeFileSync(outputPath, JSON.stringify(mcpManifest, null, 2));`
+
+``console.log(`✅ MCP-Zero Discovery Manifest successfully written to: ${outputPath}`);``
+
+### **How to use this in your workflow**
+
+> 1. Add a generation step to your package.json scripts:
+>    `"scripts": {`
+>      `"build": "tsc && node scripts/generate-manifest.js"`
+>    `}`
+> 2. When you run your build pipeline, it will ensure that both your active runtime code and your static metadata file match perfectly.
+> 3. Make sure `mcp-manifest.json` is included in your files array in `package.json` so it gets published to **npm**.
+
+### **Other actions**
+
+1. Create the **underlying handler function** for the new `kilotest_get_server_capabilities` tool.
+2. Create a **GitHub Action configuration** to automatically validate these schemas on push.
+
+## Add observability of request metrics
+
+Record per-endpoint request counts, latencies, and error rates so that Kilotest managers can observe which API operations are most used and identify performance regressions.
+
+## Serve explanation page for human MCP requests (completed)
+
+When the `/mcp` path is requested without a `text/event-stream` accept header, serve an HTML page explaining that the endpoint is for AI agents and providing a link to the AI tutorial.
+
+## Deploy revisions (completed)
+
+Deploy the revisions to the codebase.
+
+This requires:
+
+- Update the configuration of [UptimeRobot](https://dashboard.uptimerobot.com/) to monitor Kilotest as a whole via the site root. Reconfigure the existing monitor from `https://kilotest.com/qai` to the Kilotest path `https://kilotest.com/`, so that it continues monitoring service health.
+- Merge the changes to `main`.
+- Pull the new `main` branch to the server.
+- Update the actual server copy of `/etc/caddy/Caddyfile` (as root) and run `sudo systemctl reload caddy`.
+- Restart Kilotest in PM2.
+
+## QAI integration (completed)
+
+The `jrpool/qai` repository is independent of this `jrpool/kilotest` repository, and they are published as two distict packages. That separation is due to an organizational requirement that no longer exists. Since the QAI application is a tutorial showing users how to use Kilotest, and the tutorial of Kilotest also shows users how to use Kilotest, it is appropriate to convert QAI to a part of the Kilotest codebase. QAI is currently deployed with the URL `https://kilotest.com/qai`, and that would not need to change. `Caddyfile` would be simplified (see the copy in `docs/SERVICE.md`). The QAI code would need to be copied into Kilotest. Any architectural incompatibilities would need to be discovered and resolved. Locally, `qai` is a sibling repository of `kilotest` on this host. Note that QAI health is currently monitored by UptimeRobot, and periodic health monitoring of Kilotest is proposed as the next backlog item after this one, so health monitoring should be handled in such a way that it will be appropriate after both backlog items are completed.
+
+## Implement periodic monitoring (completed)
+
+Status: Implementation complete; UptimeRobot reconfiguration remains.
+
+A periodic GitHub Actions workflow has been implemented in `.github/workflows/periodic-smoke-tests.yml`, configured to run daily at 12:00 PM UTC (noon). The workflow runs `smokeTest.ts` against the deployed service at `kilotest.com`, validating that all public GET and POST paths are reachable and forwarded by Caddy (i.e., not returning bare 404 errors). This allows code and infrastructure to be deployed together, then validated by the periodic check independently. The workflow can also be triggered manually from the GitHub Actions interface.
+
+Documentation for the periodic monitoring workflow has been added to `docs/SERVICE.md` in a new “Periodic monitoring” section, including notes on UptimeRobot health monitoring.
+
+Remaining task: update the configuration of UptimeRobot to monitor Kilotest as a whole via the site root. Use the UptimeRobot console to reconfigure the existing monitor from `https://kilotest.com/qai` to the Kilotest path `https://kilotest.com/`, so that it continues monitoring service health after the QAI integration is deployed and verified live.
+
+## Details
+
+### QAI integration details (completed)
+
+#### Context
+
+QAI (`/Users/pool/Documents/Topics/repos/a11yTesting/qai`, sibling repo) is a small standalone Node HTTP server deployed at `https://kilotest.com/qai` on port 3001, reverse-proxied separately from Kilotest (port 3000) by Caddy. The two-package split was required by an organizational constraint that no longer applies. QAI’s own purpose, teaching users how to connect an AI platform to Kilotest’s MCP tools, makes it naturally a part of Kilotest’s own tutorial content, so the backlog calls for folding QAI into the Kilotest codebase as a single deployable service.
+
+Investigation turned up an important existing fact. `kilotest/web/tutorial/` and QAI’s own tutorial-and-comments feature share the same alert env vars (`MANAGER_EMAIL`, `ALERT_API_HOST`, `ALERT_API_PATH`, `ALERT_API_KEY`, `ALERT_FROM`) and the same comment-collection shape. Kilotest’s tutorial pre-dates QAI, so this similarity reflects QAI having been built by copying Kilotest’s tutorial-and-comments architecture, not the reverse. Kilotest’s `web/tutorial/` has since diverged into its own, much larger “Accessibility Testing Strategies” tutorial (641 lines) unrelated to QAI’s “add an MCP connector to Claude” walkthrough content, but it remains the established, proven pattern for a page-plus-comments module in this codebase (`answer{}`/`tutorialComment.html` wiring in `index.ts`). This is **not** a duplicate-content problem to reconcile: it is the existing template QAI itself was originally modeled on, now being reused directly instead of through a second, separately-deployed reimplementation. The work is: retire QAI’s standalone server/repo, and add QAI’s tutorial and comments pages to Kilotest as a second `web/<topic>` module pair, following the `web/tutorial/` pattern QAI was descended from.
+
+Work will happen on the already-checked-out `qai` branch in the `kilotest` repo.
+
+**Governing principle for structural and naming differences**: wherever QAI and Kilotest diverge in file naming, directory structure, or convention, do not default to Kilotest’s existing choice simply because Kilotest is the surviving repository. Evaluate both approaches on their merits and let the better design win, exactly as already decided above for documentation file naming (see “QAI’s documentation” below). This applies to structural questions too, not only naming: for example, QAI serves its static HTML and assets from a dedicated `public/` directory with a generic extension-to-content-type lookup (`web/qaiTutorial/`’s planned structure, described in the implementation steps below, instead follows Kilotest’s existing per-page `web/<topic>/index.ts` convention, since the new pages need the request-handling logic a plain static directory could not provide, but the underlying question of whether Kilotest’s codebase would benefit from a general-purpose static-asset directory or helper, rather than one `else if` branch per static file as `index.ts` currently has, is a legitimate structural question raised by this merge and worth a brief evaluation during implementation, even though it is not required to complete the merge itself).
+
+#### What gets ported vs. retired
+
+**Ported (QAI content has no Kilotest equivalent):**
+
+- QAI’s “connect an AI platform to Kilotest” tutorial content (`public/tutorial.html`, 65 lines): becomes a new `qaiTutorial.html` page.
+- QAI’s comment form (`public/comments.html`): becomes a new `web/qaiComment/` module, with GET and POST both handled there (reworked to Kilotest’s inline `fetch`-based submission convention rather than QAI’s full-page POST; QAI’s separate `public/comment-ack.html` acknowledgment page is not ported, see Decisions below).
+- QAI’s comment-spam-management logic (length bounds of 20–1000 characters, and duplicate-resubmission rejection, that is rejecting identical text resubmitted within roughly 1000 seconds). This is generalized into a shared helper (see Decisions below) used by **both** the new `web/qaiComment/` module and the existing `web/tutorial/index.ts`’s `handleComment`, so Kilotest’s original tutorial comments gain the same spam protection.
+
+**Retired (superseded by existing Kilotest infrastructure, or QAI-only machinery no longer needed once folded into one process):**
+
+- `public/favicon.ico`: identical to Kilotest’s existing root `favicon.ico`; no porting needed, since the existing one is already served at `/favicon.ico`.
+- `src/main.ts`, `src/requestHandler.ts`, `ecosystem.json`: QAI’s standalone server, routing, and PM2 config. Kilotest’s `index.ts` dispatch and `pm2.config.cjs` take over entirely.
+- `src/alerts.ts`: Kilotest’s own `alerts.ts` (`sendAlert(subject, body)`) is a direct match; reuse it rather than porting a second copy.
+- QAI’s own `tsconfig.json`, `eslint.config.mjs`, `package.json`, and CI workflow (`checks.yaml`): Kilotest’s root configs (`tsconfig.json`, `eslint.config.mjs`, `.github/workflows/ci.yml`) already apply to the whole repo, so no per-directory config is needed. Note that Kilotest’s `tsconfig` adds `noUncheckedIndexedAccess`, which QAI’s code was never checked against (expect to fix a few index-access sites when the ported code moves under Kilotest’s typecheck).
+- QAI’s `db/comments.json` runtime data: not migrated (it is transient, ephemeral demo data, and the file is already gitignored in QAI); a fresh `web/qaiComment/comments.json` starts empty, exactly as `web/tutorial/comments.json` does today.
+
+**QAI’s documentation, merged, not discarded:** QAI was built under an organizational requirement to exemplify best-practice software documentation, and its `docs/` is genuinely richer than Kilotest’s in several respects (Kilotest currently has only 2 ADRs; QAI has 9, plus architecture, requirements, and deployment docs Kilotest lacks entirely). The goal is for the merge to raise Kilotest’s documentation standard, not lose QAI’s. Concretely:
+
+- **File and directory naming when both sides have a version of “the same” doc**: use whichever naming is more industry-standard, decided at implementation time. Do not default to “Kilotest’s name wins because it is the surviving repo”. QAI’s file names were themselves prescribed to follow industry-standard documentation practice, per its organizational requirement, so where the two conventions differ, QAI’s naming is the likely winner; confirm case by case rather than assuming.
+- **ADRs** (`qai/docs/decisions/001`–`009`): QAI’s ADR format (frontmatter with `date`/`status`, `Context and problem`/`Considered options`/`Decision` structure) already matches Kilotest’s own (`kilotest/docs/decisions/001-api-design.md`, `002-coverage-tool.md`), the same convention. QAI’s own `002-adr-threshold.md` states the governing principle for what the ADR set should contain: its confirmation test is that “the ADRs describe all and only the strategic decisions manifested in the codebase.” That principle conflicts with the classic Nygard convention of never editing an ADR once written, since a set that must always describe current architecture cannot also be a frozen, append-only log. Kilotest resolves this conflict by adopting mutable, kept-current ADRs as its own explicit convention: a documented, deliberate departure from the stricter classic convention, and a recognized alternative position in the industry, not an idiosyncratic one. An ADR’s content can be edited as architecture evolves, and it remains named an ADR throughout; historical context belongs inside a current ADR when it helps a future maintainer (for example, “X was tried, but it had defects Y and Z, so X was replaced with R” helps prevent a regrettable reversion to X), rather than requiring a separate, frozen, superseded file to be kept around for that purpose. This convention itself should be written up as a revision to the ported `002-adr-threshold.md` (see below), so it is documented rather than assumed.
+  - Only ADRs describing still-operative decisions are ported into `kilotest/docs/decisions/` as new, renumbered files, continuing on from Kilotest’s existing 001–002. Candidates for porting: `001-docs-directory.md` (the `docs/` plus `docs/decisions/` layout both repositories already use), `002-adr-threshold.md` (the ADR-writing policy itself, revised per the mutability convention above), `006-observability.md`, `007-git-workflow.md`, `008-internal-testing.md`, and `009-dependencies.md` (general engineering policies that are candidates to become repository-wide Kilotest policy, not QAI-specific). Confirm each candidate against Kilotest’s actual current practice during implementation, and edit its content as needed (reframing QAI-specific wording to repository-wide, updating anything no longer accurate) before porting: this editing is now consistent with Kilotest’s stated mutable-ADR convention, not an exception to it.
+  - ADRs describing decisions specific to QAI’s former existence as an independent deployment are folded into the still-operative ADRs as historical context, not ported as separate files: `003-initial-routing.md` (chose independent deployment, the opposite of this merge), `004-prerelease-versions.md` (QAI’s own version-0.1.0 scoping), and `005-infrastructure-health.md` (QAI needing its own external monitoring because Kilotest lacked any). Where one of these adds genuinely useful context to a still-operative decision (for example, `003-initial-routing.md`’s reasoning belongs inside whatever ADR now documents Kilotest’s routing architecture, as a brief note on the independent-deployment approach that was tried and superseded by this merge), fold a short summary of it in; otherwise leave it recoverable only from `jrpool/qai`’s git history.
+  - This merge is itself exactly the kind of decision QAI’s own `002-adr-threshold.md` says merits an ADR: a strategic choice with substantial effects on maintainability. Write a new Kilotest ADR documenting the QAI-integration decision itself, positioned last in the ported sequence.
+- **`architecture.md`, `requirements.md`**: read these fully during implementation and fold any content still accurate and relevant to the merged codebase into Kilotest’s docs. Kilotest currently has no equivalent top-level architecture or requirements doc, while QAI’s organizational requirement produced these as a matter of course; evaluate whether Kilotest’s documentation would genuinely benefit from adopting a dedicated architecture or requirements doc at this level (per the governing principle above) rather than assuming `docs/SERVICE.md` is automatically the right destination just because it already exists. Content describing QAI-as-independent-service specifics, such as its own deployment topology, becomes historical once folded in.
+- **`deployment.md`**: covers the same subject as `docs/SERVICE.md`, which already documents the real, merged deployment, so the two should become one file rather than two. Per the governing principle above, do not assume `SERVICE.md`’s name and existing shape automatically wins just because Kilotest already has it: read both in full during implementation and decide, on the merits of each (completeness, clarity, industry-standard naming for a deployment doc), which name and structure the merged file should keep, folding in whatever content the other has that it lacks.
+- **`repository.md`, `project-vision.md`**: likely QAI-specific project-management framing (course requirements, repo conventions for a now-retired standalone repo); read in full during implementation to confirm, but expect these to be summarized into a short historical note rather than ported whole.
+- **`future.md`, `manual-verification.md`, `ai-implementation-review.md`**: read during implementation. `manual-verification.md` may contain durable QA procedure worth folding into Kilotest’s own testing docs; `future.md` and `ai-implementation-review.md` are more likely point-in-time project-status artifacts, and are candidates for summary rather than full porting.
+- **`progress-reports/`, `reviews/`**: sprint reports, a demo video, and peer reviews, purely historical, course-context artifacts with no forward-looking maintainer value. Retain only as a brief pointer or summary (for example one paragraph in the merge’s own ADR, or a short history-style note: “QAI’s original development history, including sprint reports and peer reviews, is preserved in the `jrpool/qai` repository’s git history”) rather than copying the files into Kilotest.
+- **`docs/rulesets/qai-main.json`**: QAI’s GitHub branch-protection ruleset export. Kilotest already has its own equivalent (`docs/Smoke Test.json`, per `docs/SERVICE.md`); compare during implementation for any protection QAI enforced that Kilotest does not yet (for example QAI’s required-testing-before-merge ADR, `008-internal-testing.md`, appears to match Kilotest’s own branch protection already, but confirm), and otherwise do not port the file itself.
+- **`README.md`**: QAI’s root README is project-management framing for a now-retired standalone repo; it is not ported as a file, but confirm that Kilotest’s own README and docs adequately cover the “connect an AI platform” use case QAI’s README foregrounded (they may already, via the new `qaiTutorial.html` page itself).
+
+#### Decisions locked in
+
+- **Route placement**: QAI’s content gets its own new page(s), independent of the existing `tutorial.html` (which has evolved into unrelated Kilotest-specific content). Working names: `qaiTutorial.html` (the “connect an AI platform to Kilotest” walkthrough) and `qaiComment.html` (the comment form), with a POST to `qaiComment.html` handling submission, matching Kilotest’s convention of a GET page and a same-named POST action seen elsewhere (for example `requestTest.html`).
+- **Module structure follows the established one-topic-per-directory convention**: every existing `web/<topic>/index.ts` module (`web/enqueueForm/`, `web/ai0BalanceForm/`, `web/hideReportForm/`, and others) exports one `answer(pathTail, search)` function and pairs with exactly one `index.html` in the same directory, rendered via the shared `populateTemplate(import.meta.dirname, query)` helper in `util.ts` (already used by essentially every existing GET page). QAI’s content is ported as two such modules, not one directory holding differently-named files: `web/qaiTutorial/` (the walkthrough page) and `web/qaiComment/` (the comment form, exporting both `answer` for its GET page and `handleComment` for its POST action, mirroring `web/tutorial/index.ts`’s existing two-export shape). This was corrected during implementation from an earlier draft of this plan that proposed a single `web/qaiTutorial/` directory with distinctly-named HTML files and a new hand-rolled substitution helper; that draft duplicated `populateTemplate`, which already does the same job. A separate `web/qaiCommentAck/` module, considered at one point for an HTML acknowledgment page, was dropped once the response-shape decision below was reversed.
+- **Comment validation and unified length limits**: `web/qaiTutorial/`’s and `web/qaiComment/`’s GET pages return `{status: ‘error’, message}` on failure exactly like any other `web/<topic>/index.ts` module, and Kilotest’s existing dispatch in `index.ts` already calls `serveError(getAbuseError(request, answerData.message), response, true)` automatically whenever an `answer[topic]` function returns an error status (see the generic `.html` GET dispatch at line 504), rendering the existing `error.html` template; no new error-response helper is needed there. The comment-submission POST path uses a different, JSON-based error convention instead (see the Acknowledgment response entry below). During implementation, validation was split into two helpers in `util.ts`: `checkCommentLength` (reusable length validation) and `checkCommentDuplicate` (reusable duplicate-submission detection, rejecting identical text resubmitted within roughly 1000 seconds). Both are called by `web/qaiComment/`’s `handleComment` and the existing `web/tutorial/index.ts`’s `handleComment`, so both comment endpoints enforce identical checks. Kilotest’s original tutorial comment length limit (500 characters) was raised to 1000 to match QAI’s limit, unifying the bounds; both endpoints now enforce a 20–1000 character range. This is a behavior change to `web/tutorial/` beyond the QAI port itself, and is worth calling out in the PR description as a deliberate, bundled improvement rather than scope creep.
+- **Acknowledgment response: unify on `web/tutorial/`’s JSON/AJAX convention, not QAI’s HTML-page convention (reversed from an earlier draft of this plan)**: investigation during implementation found that `web/tutorial/index.html`’s comment form is a single-page AJAX widget (a `fetch('/tutorialComment.html', ...)` call that parses a JSON `{status}` response and updates an inline status region, with a live character counter, and no page navigation), while QAI’s `comments.html`/`comment-ack.html` is a traditional full-page POST that navigates to a separate acknowledgment page. Per this plan’s own “let the better design win” governing principle, `web/tutorial/`’s inline-feedback UX is the better design, so the new QAI comment form is built to match it, not the reverse: `web/qaiComment/` gets its own small `fetch`-based submission script (adapted from `web/tutorial/index.html`’s pattern) with an inline status region, submitting to `qaiComment.html` and expecting a JSON `{status: 'ok'}` or `{status: 'error', message}` response, exactly like the existing tutorial comment form. No HTML acknowledgment page, and no `populateTemplate`-based ack module, is built for either flow; `populateTemplate` is still used for the ordinary GET pages (`web/qaiTutorial/`, `web/qaiComment/`) themselves, just not for a comment acknowledgment.
+
+#### Implementation steps
+
+The steps below are sequential, but not each individually required to pass Kilotest’s full check suite (100% coverage, typecheck, lint, tests) on its own: for example, after step 1 alone the new `util.ts` helper exists but nothing calls it yet, which the 100%-coverage requirement would fail, and after step 4 alone the new modules are wired into routing but the Caddyfile and documentation steps are still pending. Commit and push freely at intermediate points on the `qai` branch (or a further feature branch off it) as work progresses, but only open the pull request to `main` once every step is complete and the full check suite passes, per Kilotest’s branch-protection rules (`docs/SERVICE.md`, “Branch protection”).
+
+1. **Add shared comment-validation helpers to `util.ts`**: `checkCommentLength(content: string)` validates that a comment’s raw length is within the 20–1000 character bounds, returning either an ok result or an error result with the applicable message (“shorter than 20” or “longer than 1000”). `checkCommentDuplicate(comments: {timeStamp: string; content: string}[], content: string)` validates that a comment does not repeat, verbatim, a comment submitted within the last 1000 seconds, returning either an ok result or the duplicate-rejection message. Factor these out of QAI’s `requestHandler.ts` logic (lines 78–110 of that file) rather than rewriting from scratch. These are validation logic only, not response rendering, since rendering is already handled by the existing `populateTemplate`/`serveError` machinery (see Decisions above). `checkCommentDuplicate` uses Kilotest’s `timeStamp` format (not QAI’s ISO `dateTime` format) to integrate with the shape already used by `web/tutorial/index.ts`’s comments.json.
+
+2. **Update `web/tutorial/index.ts`’s `handleComment` and raise its length limit from 500 to 1000**:
+   - Call `checkCommentLength` on the raw content before sanitizing, and `checkCommentDuplicate` on the sanitized content before appending, so both checks apply to Kilotest’s original tutorial comments, returning `{status: ‘error’, message}` on any failure.
+   - Raise the character limit from 500 to 1000 to unify with QAI’s limit, and update the form label in `web/tutorial/index.html` (maxlength and character-counter logic) to reflect the new bound.
+   - No response-shape change: `handleComment`’s existing bare `{status: ‘ok’}`/`{status: ‘error’, message}` JSON responses, and the `index.ts` POST branch for `tutorialComment.html` stay as they are; only the validation calls and the limit are new.
+   - Update `web/tutorial/tutorial.test.ts` for the new rejection paths (too short, too long, duplicate) and the raised limit.
+
+3. **Create a single new web module**, `web/qaiTutorial/`, following the established one-topic-per-directory pattern and mirroring `web/tutorial/`’s unified approach:
+   - `web/qaiTutorial/index.ts` exports both `answer()` (to serve the tutorial page) and `handleComment()` (to handle comment submissions), eliminating the need for a separate comments module.
+   - `web/qaiTutorial/index.html` combines QAI’s tutorial content (from `qai/public/tutorial.html`), adapted with Kilotest-style title and meta tags (dropping “QAI” branding, for example `Tutorial: Connect an AI Platform | Kilotest`), followed by an integrated comment-submission section at the end, following the pattern established by `web/tutorial/index.html`. The comment form is a `fetch`-based, inline widget (character counter, `comment-diagnosis` status region, `fetch(‘/qaiTutorialComment.html’, {method: ‘POST’, ...})` parsing a JSON response) rather than a separate page, so users can submit comments without leaving the tutorial. This allows readers to reference the tutorial content while commenting, and to submit multiple comments on the same visit.
+   - `handleComment` in `index.ts` enforces the unified 20–1000 character range via `checkCommentLength` and inline duplicate-checking (using ISO `dateTime` format for QAI comment timestamps, consistent with how the separate comments.json is structured), sanitizes, appends to a colocated `comments.json`, and calls the shared `sendAlert` from `../../alerts.ts` with subject `New QAI tutorial comment received`.
+   - GitHub issue links in the ported HTML: repointed from `jrpool/qai/issues` to `jrpool/kilotest/issues`, since bugs against this content are now Kilotest issues (repo disposition is a separate post-merge decision, but the *content* should reflect its new home immediately, regardless).
+
+4. **Rename modules for clarity: `web/tutorial/` → `web/tutorialWeb/`, `web/qaiTutorial/` → `web/tutorialAI/`**: The original tutorial is for humans using Kilotest’s web interface directly; the new tutorial is for humans who want to enable their AI agents to use Kilotest (by connecting the MCP server). Parallel naming makes this distinction clear to contributors and users alike. Rename all related identifiers:
+   - Directory: `web/tutorial/` → `web/tutorialWeb/`
+   - Directory: `web/qaiTutorial/` → `web/tutorialAI/`
+   - Environment variables: `TUTORIAL_COMMENTS_PATH` → `TUTORIAL_WEB_COMMENTS_PATH`, `QAI_TUTORIAL_COMMENTS_PATH` → `TUTORIAL_AI_COMMENTS_PATH`
+   - Route names: `tutorialComment.html` → `tutorialWebComment.html`, `qaiTutorialComment.html` → `tutorialAIComment.html`
+   - Imports in `index.ts`: aliases like `answer as tutorialWeb`, `handleComment as handleTutorialWebComment`, `answer as tutorialAI`, `handleComment as handleTutorialAIComment`
+   - Test files: `web/tutorialWeb/tutorialWeb.test.ts`, `web/tutorialAI/tutorialAI.test.ts`
+   - HTML titles and content: "web tutorial" and "AI agent tutorial" terminology
+   - Redirects: `/qai/` and `/qai/comments` still 301-redirect to `/tutorialAI.html` (keeping the old paths working for backward compatibility)
+
+5. **Wire into `index.ts`** (mirroring the `tutorialWebComment.html` pattern at lines 34, 82–154, 189, 930–941):
+   - Import `answer` and `handleComment` from `web/tutorialWeb/` and `web/tutorialAI/`, with appropriate aliases (following the existing import-aliasing convention at lines 38–69).
+   - Add both `tutorialWeb` and `tutorialAI` to the `answer{}` dispatch table. Both are plain `.html` GET pages, so they are picked up automatically by the existing generic `pageName.endsWith(‘.html’)` dispatch (line 504), with no new `else if` branches needed for GET.
+   - Add `tutorialWeb.html` and `tutorialAI.html` to `routes.GET`, and add `tutorialWebComment.html` and `tutorialAIComment.html` to `routes.POST` (lines 159–193).
+   - Add `pageName === ‘tutorialWebComment.html’` and `pageName === ‘tutorialAIComment.html’` branches in the POST dispatch chain, calling the respective `handleComment` functions and responding with the same bare-JSON `{status: ‘ok’}`/`{status: ‘error’, message}` shape, so both comment forms share one response convention.
+   - Redirect QAI’s old live paths to their new equivalents: add branches that 301-redirect both `/qai/comments` (the old `https://kilotest.com/qai/comments` path) and `/qai` (the old `https://kilotest.com/qai` root) to `/tutorialAI.html`. Check the longer `/qai/comments` path first, before checking the shorter `/qai` path (since pathname matching on path segments would match the shorter one too), so that existing bookmarks, search-index entries, and inbound links to the old QAI paths continue working rather than breaking once Caddy’s separate `/qai` proxy block is removed. Add `/qai` and `/qai/comments` to `routes.GET` alongside the other redirect-only aliases such as `/swagger.json`.
+
+6. **Merge QAI’s documentation into Kilotest’s `docs/`** (see “QAI’s documentation, merged, not discarded” above for the full breakdown):
+   - Revise `002-adr-threshold.md` while porting it, to state Kilotest’s mutable, kept-current ADR convention explicitly (see above), since this convention is what governs how the rest of this step is carried out.
+   - Confirm each of `001-docs-directory.md`, `006-observability.md`, `007-git-workflow.md`, `008-internal-testing.md`, and `009-dependencies.md` still accurately describes Kilotest’s actual practice; port only the ones that do, as new, renumbered files in `kilotest/docs/decisions/`, continuing on from Kilotest’s existing 001–002, editing content as needed to reflect current, repository-wide reality.
+   - For `003-initial-routing.md`, `004-prerelease-versions.md`, and `005-infrastructure-health.md`: fold any genuinely useful historical context from each into whichever still-operative ADR now covers the same architectural area (for example, `003-initial-routing.md`’s independent-deployment reasoning becomes a brief historical note inside the ADR that documents Kilotest’s current routing architecture). Do not port these three as separate files; their full original text stays recoverable from `jrpool/qai`’s git history.
+   - Read `architecture.md`, `requirements.md`, `repository.md`, `project-vision.md`, `future.md`, `manual-verification.md`, `ai-implementation-review.md`, and `deployment.md` in full; fold durable, accurate content into Kilotest’s docs, evaluating for each whether the receiving doc should be an existing Kilotest file or, per the governing principle above, a new doc named and structured after whichever side’s convention is better (for `deployment.md` specifically, decide with `docs/SERVICE.md` whether the merged file keeps Kilotest’s name and shape, QAI’s, or a new one, rather than assuming `SERVICE.md` simply absorbs it). Write a brief historical-summary note for content in the remaining files that does not merit folding in.
+   - Add a short pointer note (in the new merge ADR, or a dedicated short doc) preserving the existence of `progress-reports/`, `reviews/`, and `docs/rulesets/qai-main.json`, referencing `jrpool/qai`’s git history as the archive rather than copying those files in.
+   - Write the new ADR documenting this integration decision itself, positioned last in the ported sequence.
+
+7. **Update `smokeTest.ts`**: add `concretePaths.GET`/`.POST` entries for the new page names (`tutorialAI.html`, `tutorialAIComment.html`) and for the `/qai`/`/qai/comments` redirect aliases, and a `postBodies` entry if the comment POST needs a sample body.
+
+8. **Update the Caddyfile copy** (`/etc/caddy/Caddyfile` on the server is tracked as a copy in `docs/SERVICE.md`):
+   - Remove the `redir /qai /qai/ 301` and `handle_path /qai* { reverse_proxy localhost:3001 }` blocks, since QAI no longer runs as a separate process; the old `/qai` and `/qai/comments` paths are now redirected at the application layer instead (see step 5), so Caddy needs no special-case handling for them beyond forwarding them like any other allowed GET path.
+   - Add the new page path(s), including `/qai` and `/qai/comments`, to the `@allowedGET`/`@allowedPOST` matchers.
+   - Update `docs/SERVICE.md`’s copy of the Caddyfile to match, and note in that doc that QAI is no longer a separately-deployed application (the `docs/SERVICE.md` “Applications” section currently implies other apps like QAI could live at `/opt/jpdev/xyz`; leave that generic note as is, and just remove the QAI-specific proxy config).
+   - The user will update the actual server copy and run `systemctl reload caddy` just before restarting Kilotest in PM2 after pulling a version containing the QAI integration.
+
+9. **Tests**: The test files are already in place from step 4 (`web/tutorialWeb/tutorialWeb.test.ts` and `web/tutorialAI/tutorialAI.test.ts`), mirroring the structure and unit-testing both `answer()` and `handleComment()` directly. Rely on the existing `index.test.ts` integration tests to exercise the renamed routes once wired into `index.ts`’s dispatch (adding assertions there for the new paths and POST handler). Confirm 100% coverage is maintained per the `c8` config in `package.json` (branches, functions, lines, and statements all at 100%).
+
+10. **Retire the QAI repo**:
+    - Confirm with the user whether `jrpool/qai` on GitHub should be archived, left as read-only history, or deleted, once the port is verified working in Kilotest. This plan does not delete anything in `/Users/pool/Documents/Topics/repos/a11yTesting/qai`; that is a separate, explicit decision for the user to make after the merge lands and is verified in production.
+    - Do not remove QAI’s existing UptimeRobot monitor. Instead, once the new `qaiTutorial.html`/`qaiComment.html` paths are live, reconfigure that monitor’s target so it continues watching a Kilotest path (for example the site root, or the new tutorial path) rather than the now-defunct `https://kilotest.com/qai`, effectively expanding its existing scope from monitoring QAI’s health specifically to monitoring Kilotest’s health as a whole. This closes the monitoring gap the original backlog note flagged, and does so by extending a monitor that already works rather than waiting on a new one (Kilotest’s own health-monitoring backlog item, or the “periodic smoke-test session” item, can still be pursued separately and later, but neither is a precondition for this reconfiguration).
+
+#### Open question to resolve after the merge is verified live
+
+- Disposition of the `jrpool/qai` GitHub repo once the new pages are confirmed working in production (archive, delete, or leave as read-only history). This is not needed to implement the merge itself.
+
+#### Verification
+
+- `npm run typecheck`, `npm run lint`, and `npm test` all pass in `kilotest` with the new module included, and coverage remains at 100%.
+- Run `npm start` locally, then manually visit the new tutorial path and submit a test comment; confirm it is appended to the new `comments.json` and, with alert env vars set, triggers `sendAlert`.
+- Run `node smokeTest.ts` against a local instance (or the deployed one, post-deploy) to confirm the new paths are both allowlisted in `routes` and reachable.
+- After deploying: visit the old `https://kilotest.com/qai` and `https://kilotest.com/qai/comments` URLs directly and confirm each returns a 301 redirect to `/qaiTutorial.html` and `/qaiComment.html` respectively, per the redirect added in step 4, rather than a bare 404, now that Caddy’s separate `/qai` proxy block is gone.