@jrpool/kilotest 47.0.1 → 49.0.0

package/docs/BACKLOG.md

@@ -2,31 +2,223 @@
 
 Engineering tasks and risks that are not yet scheduled.
 
-Items marked completed are preserved for about 2 weeks in case of production bugs.
+Items marked completed or not adopted are preserved for about 2 weeks in case of production bugs.
 
-## Document alerting
+## Make validation locations consistent
 
-Document when and how Kilotest sends alerts to its maintainer, including the subscription to Resend and the DNS configuration for Resend authentication.
+Inventory where request validations occur, e.g. in `index.ts` versus in the modules to which it routes requests. Find any arbitrary inconsistencies in the locations. If any are found, decide where it is most parsimonious and maintainable for the validation to take place and standardize on that.
 
-## Decide whether to tighten the DMARC policy and add SPF alignment for Resend
+## Protect hidden report list (completed)
 
-Live DNS for `kilotest.com`, checked 2026-09-16, shows a DMARC record of `v=DMARC1; p=none;` (monitoring only, no `rua=` reporting address) and an SPF record (`v=spf1 include:_spf.porkbun.com ~all`) that does not include `resend.com`. Mail sent via Resend is currently authenticated only by the DKIM record at `resend._domainkey.kilotest.com`. This may be an intentional, working minimal configuration, or an oversight from Resend's onboarding; it has not been confirmed against the Resend dashboard's domain-verification status. Consider confirming, in the Resend dashboard, whether SPF alignment is recommended or required for this domain's deliverability, and once alert deliverability has been stable for a period, whether to tighten DMARC from `p=none` to `p=quarantine` and add an `rua=` address so DMARC aggregate reports reach the maintainer.
+### Problem
 
-## Widen test-request duplicate detection
+`unhideReportForm.html` currently serves the list of hidden reports, including the names of the pages they report on, in response to a plain `GET` request, with no authorization check. Only its `POST` (the unhide action itself) currently checks `authCode`. This leaks the names of hidden pages to anyone who requests the URL, which is the information this item protects; `hideReportForm.html` (which lists non-hidden reports) is not in scope.
 
-The duplicate check in `addTestRequest` (`util.ts`), reached by all four test/retest request paths (UI `/requestTest.html` and `/requestRetest.html`; API `requestTest` and `requestRetest` operations), compares an incoming request only against other requests currently pending in `db/jobs/testRequests.json`, i.e., submitted but not yet manually approved by the maintainer into a job. Once a pending request is approved into a job (`web/enqueue/index.ts`), all pending requests for that URL are cleared from `testRequests.json`, so this shared check cannot recognize a duplicate against a request that has already been approved, is currently running, or has already completed. An identical request submitted after approval triggers a fresh alert rather than being filtered.
+### Adopted solution: 2 successive forms
 
-The four paths differ in what, if anything, they check before reaching this shared dedup, and none of the four closes the gap:
+This is a new pattern, with no precedent elsewhere in the codebase (the closest existing pattern, a single self-submitting form per manager page, is described under “Addition… addition… clear counts checkbox” above, but that pattern always leaves its `GET` publicly viewable, which is exactly what must not happen here).
 
-- UI `requestTest` (`web/requestTest/index.ts`) additionally calls `getRequestability`, rejecting the request outright if the URL matches a claimed or queued job. This screens out most already-approved, not-yet-completed requests for a new test, but not ones for a completed report, since `getRequestability` does not consult reports.
-- API `requestTest` (`api/requestTest.ts`) additionally checks `getReportExtracts` and rejects the request if a report already exists for the same description and URL, but, unlike the UI path, does not call `getRequestability`, so it does not screen out a URL that is currently claimed or queued as a job.
-- UI `requestRetest` (`web/requestRetest/index.ts`) and API `requestRetest` (`api/requestRetest.ts`) call neither `getRequestability` nor a same-description-and-URL report check; each only confirms the report being retested is still the latest one for its page before falling through to the shared dedup.
+- **Form 1**: a new page, `web/showHiddenReportsForm/` (`showHiddenReportsForm.html`). `GET` renders a bare form with one text input (authorization code) and a submit button, `POST`-ing to itself. It has no other content, and does not itself reveal anything about hidden reports. On a submitted `POST`, it validates the authorization code:
+  - **Invalid code**: serve the error page (`status: 'error'`, message “Invalid request”, deliberately vague so as not to confirm to an attacker that the authorization code specifically was wrong), matching the now-standardized `authCode`-failure pattern used by every manager-facing form and action page in the codebase (`hideReportForm`, `unhideReportForm`, `ai0BalanceForm`, `metrics`, `pruneReportsForm`, `rewindReportsForm`, `expungeReportsForm`, `enqueue`, `reannotate`, `renewWCAG`): self-submitting forms in this codebase serve the generic error page on an invalid code, not a re-rendering of themselves with an inline error.
+  - **Valid code**: respond with Form 2 (`unhideReportForm`’s rendered content: the current hidden-reports list, plus its own authorization-code input and submit button), freshly rendered.
+- **Form 2**: `unhideReportForm.html`, unchanged in its own `POST` (unhide) behavior, including serving the error page on an invalid code exactly as it does today, but its `GET` route is removed entirely. It is reachable only as the response to a valid Form 1 submission, never by direct `GET`. It remains self-submitting in the existing sense: each `POST` (submitted with the authorization code and the report to unhide) re-serves the form with the updated hidden-reports list and the code input, so the manager can unhide further reports without returning to Form 1. Because a self-submitting form must accept `POST`, `unhideReportForm.html` keeps a `POST` route; only its `GET` route is removed.
 
-Consider widening the shared `addTestRequest` check itself, so that all four paths benefit uniformly, to also compare against approved, claimed, queued, or completed requests, for example recently created jobs or reports for the same URL and description, and if so over what time window, rather than continuing to rely on each path's own inconsistent pre-check.
+Both `showHiddenReportsForm` and `unhideReportForm` are added to (or, for `unhideReportForm`, already are part of) the `managerPages` set (`index.ts`), so both are counted under `managerActivity` in the metrics table rather than `pageViews`, consistent with how every other manager route is already counted, regardless of `GET` or `POST`.
+
+### Manage-page link
+
+The “What can a manager do?” link on `manage.html` that currently points at the hidden-reports flow is reworded from wording that implies “declassify” to “View the list of experimental reports”, and repointed from `unhideReportForm.html` to `showHiddenReportsForm.html`.
+
+## Expand observability (completed)
+
+### Observability context
+
+Kilotest currently:
+
+- sends email alerts to a manager on certain events that are captured internally or (by UptimeRobot) externally.
+- records test and retest requests, tutorial comments, and feature requests in files.
+- creates log files containing significant details.
+- provides public data about the tests that it has performed on web pages. That is the essential value proposition of Kilotest.
+
+Some additional metrics would be useful for the maintainer. Currently, the maintainer has no easy method for answering questions such as:
+
+- Is anybody using the Kilotest web UI?
+- Do any AI agents use the Kilotest MCP server?
+- If Kilotest is used, what classes of requests are made, with what frequencies?
+
+Conceptually, the metrics of most potential value reveal how, how much, how often, and by how diverse a set of programmatic and web users the capabilities of Kilotest are interrogated and used, and how well or poorly the resources (memory, storage, CPU, etc.) deployed on the Kilotest server satisfy the demands placed on them. It is not obvious what those metrics should be. Whatever they are, it is also not obvious how they should be collected, recorded, and exposed.
+
+It is not realistic to expect these questions to be answered precisely in a plan, but a plan can define a small initial observability feature set that can be implemented, experienced, and revised and expanded over time.
+
+### Findings from investigation of current state (2026-09-19)
+
+- **Logging**: There is no application-level logging subsystem. Kilotest writes only plain-text, one-line messages via `console.log`/`console.error`/`console.warn` (in `index.ts`, `alerts.ts`, `balances.ts`, `serve.ts`, `checkinfo.ts`, `api/util.ts`, and a few `web/*/index.ts` modules), which PM2 captures into its own stdout/stderr log files. `docs/SERVICE.md` defers rotation and retention entirely to PM2’s defaults. There is no per-request log line for ordinary traffic; the closest thing is `getAbuseError` (`index.ts`), which captures IP, method, and reason, but only for rejected or suspicious requests, and only logs it, does not persist it. “Creates log files containing significant details” (as stated above) describes PM2’s captured console output, not a bespoke logging system.
+- **Alerting**: `alerts.ts`’s `sendAlert(subject, body)` and its 10 current trigger conditions are already fully documented in `docs/SERVICE.md` (“Alerting” section). No gaps found there; this observability plan does not touch alerting.
+- **Request recording**: Test/retest requests (`db/jobs/testRequests.json`) and tutorial comments (`db/comments/tutorialWeb.json`, `db/comments/tutorialAI.json`) are recorded as plain JSON, keyed or listed with a `timeStamp` field but no IP, user-agent, or other client-identifying data. Feature requests (`api/requestFeature.ts`) are, despite a response message claiming they are “logged”, not persisted to any file at all; they exist only as the body of a `sendAlert` email. This is a pre-existing inconsistency, not something this plan needs to fix, but the metrics feature described below does not depend on fixing it.
+- **MCP server usage**: `mcp.ts`’s `handleMCP` creates a new, stateless `McpServer` and `StreamableHTTPServerTransport` per POST request (no session ID), registers all 8 tools, and hands off to the SDK. There is no instrumentation of which tool was called, how often, or by what client. The MCP transport, being stateless, exposes no stable per-caller identifier; any usage counting can distinguish tools called from one another, but not one calling agent from another, without a new signal.
+- **Web UI usage**: `index.ts` dispatches routes through a simple glob-pattern allowlist with no middleware layer; there is no existing request counter, timing, or per-route instrumentation of any kind.
+- **Public reports feature** (`listReports`, `getReport`, `listIssues`, `listViolators`, `listDiagnoses`, both as MCP tools and as web/API routes): this is Kilotest’s core value proposition, serving test results publicly, and is functionally distinct from the maintainer-facing usage metrics this plan addresses. Out of scope here, except that its routes and tools are among those a usage counter would count.
+- **Deployment constraints**: the production host is a 1 vCPU, 2GB RAM Vultr VM running a single PM2 instance (`max_memory_restart: 500M`), already tuned with `zram` swap because memory is tight. There is no existing metrics agent, APM, or observability dependency of any kind (`package.json` has only `@modelcontextprotocol/sdk`, `dotenv`, `testaro-issues`, and `zod` as runtime dependencies). The project is maintained by one person and enforces 100% test coverage (branches, functions, lines, statements) via `c8` on all application code. These constraints rule out any heavyweight metrics stack (Prometheus, OpenTelemetry, a queryable time-series database, etc.) as a starting point; a small initial feature set should reuse Kilotest’s existing plain-JSON-file convention rather than add a new class of dependency or infrastructure.
+- **Existing smoke-test short-circuit (correction to the finding below, discovered during implementation of step 1)**: `smokeTest.ts` tags its own web and API requests with an `x-kilotest-smoke: 1` header (`sendRequest`, used for every GET/POST path in `concretePaths`). This header is already read, in `index.ts`’s `handleRequest` (around line 411), and any request carrying it receives an immediate, perfunctory `{}` response before any dispatch, route handler, or `handleMCP` call executes, application-wide, for both GET and POST, on every path including `/mcp` (tested at `index.test.ts`, “GET/POST with x-kilotest-smoke header returns a perfunctory 200”). Consequently, no smoke-test-tagged request ever reaches the web-page dispatch, `api/*` handlers, or MCP tool-call handling that this plan’s counting sites live in, regardless of whether the counting code itself checks the header. The “Excluding smoke-test traffic” decision and the data-model note below, originally written on the mistaken premise that this header was unread and that counting code needed to check it, are superseded: no exclusion logic is needed in `recordMetric`, and `sendMCPRequest` does not need the header added, since the smoke test’s own MCP probes are a deliberate, separate case that already exercises the real `handleMCP` path (the header is intentionally omitted there today, per that function’s own comment, to test unmodified request handling; adding it would defeat that purpose without helping metrics, since metrics exclusion turns out not to require it).
+
+### Decisions
+
+- **Internal (in-process) instrumentation, not an external/proxy-level one, and not a hybrid of both, at least for now**: an alternative was considered where some or all counting happens outside Kilotest, at the Caddy reverse-proxy layer (Caddy’s built-in `log` directive can write a structured, rotated access log per request: path, method, status, latency, remote IP, User-Agent), rather than by modifying application code. That alternative was rejected, not merely deferred, for this initial feature set:
+  - It cannot answer the most important of the three questions this plan exists to answer, “what MCP tools are being used, and how often.” Kilotest funnels all 8 MCP tools through a single `POST /mcp` endpoint, with the tool name carried inside the JSON-RPC request body; a proxy operating on HTTP method and path alone cannot see it without parsing the application’s own protocol, at which point it is no longer a boundary-external solution. Since this question is the important one, a solution that cannot answer it is not viable as the sole mechanism, so internal instrumentation is necessary regardless of what else is decided.
+  - Given that internal instrumentation is therefore required for MCP tool counting, a further, separate question is whether an external layer should be added on top of it, to count web-page views and API operations instead of (or in addition to) doing so internally. Decided against, for now: a dual internal-plus-external solution is more complex to build, reason about, and maintain than a single internal one, and it would require writing and maintaining configuration at the host/Caddy level (tracked only as a documentation copy in `docs/SERVICE.md`, not exercised by any test or CI check), rather than in version-controlled, type-checked, unit-tested application code.
+  - An external solution’s main distinguishing benefit is visibility into requester-identifying detail (remote IP, User-Agent, and similar client facts) that internal code does not currently capture. That detail is not currently useful to the maintainer, and capturing it would sit in tension with Kilotest’s current privacy posture of not tracking who is using the service or making requests. If that posture changes later (for example, if user accounts are introduced), requester identity would become directly available inside the application anyway, which removes the main motivating reason to add an external layer at all. This can be revisited if and when that changes.
+  - Conclusion: build one internal mechanism, applied uniformly (see the next decision), and treat resource-level and requester-identity observability, the areas where an external or host-level layer would have concrete advantages, as explicitly out of scope for this iteration (see Explicitly deferred, below), not as a gap this plan tries to partially fill with a second mechanism.
+- **One shared utility function, called at every counted site, not ad hoc calls per call site**: once metrics recording is being added to the codebase at all, every location that needs to record an event calls the same utility function, rather than each of the three call sites (web-page dispatch, MCP tool dispatch, API dispatch) growing its own inline increment logic. This keeps the increment logic and any future change to what gets recorded in one place, so a successor maintainer only needs to understand one function to understand how metrics recording works everywhere it is used, rather than auditing each call site separately for consistency. (The function does not need to implement smoke-test exclusion; see the correction above, smoke-test traffic never reaches any counted call site in the first place.)
+- **Storage**: a single new file, `db/metrics.json`, following the same plain-JSON-file convention as `testRequests.json` and the comments files. It holds simple counters (see Data model below), read into memory, incremented, and written back to disk synchronously on each `recordMetric` call, matching the existing write-on-every-mutation pattern used by `deleteTestRequests` for `testRequests.json`, rather than an interval-based buffering scheme (which has no precedent in this codebase and would need its own flush-timing and shutdown-handling logic, neither of which any existing Kilotest code implements). This is acceptable because metrics writes are small, infrequent relative to a typical request’s own I/O, and not on any latency-sensitive path. This avoids both the unbounded growth of an append-only per-event log (which would need its own retention policy from day one) and the data loss that a purely in-memory counter would suffer on every PM2 restart or deploy.
+- **Access**: a new page, `/metrics.html`, built as a `web/metrics/` module following the established one-topic-per-directory convention (`answer()` exported, rendered via the existing `populateTemplate` helper), displaying the counts in a simple HTML table, with no new delivery mechanism (email digesting via `alerts.ts` was considered and deferred; it can be layered on later, reusing the same `db/metrics.json` data, if the maintainer finds a page alone insufficient). It is gated by the existing `authCode === process.env.AUTH_CODE` convention already used by the codebase’s other manager-power pages (`web/reannotate/`, `web/hideReportForm/`, `web/unhideReportForm/`, `web/enqueue/`), so it lives among the “what can a manager do?” capabilities rather than being publicly viewable, at least initially. Metrics data is not treated as more sensitive than the data already exposed behind (or, for a page like `hideReportForm.html`, partly without) this same check, such as which reports are hidden or which rules are unclassified; the gate reflects that this starts as a manager power and may later be made public, not a claim of special sensitivity. Should the maintainer later decide to expose it publicly, that is a one-line change (dropping the `authCode` check from `web/metrics/index.ts`), not a rearchitecture, since the counters and their storage are unrelated to who is allowed to view them.
+
+### Data model (initial)
+
+`db/metrics.json` holds, at minimum:
+
+- **Web UI page views**: a count per page name (the same page names already used as dispatch keys in `index.ts`’s `answer{}` table), incremented once per successful GET of a `.html` page.
+- **MCP tool calls**: a count per tool name (the 8 names already registered in `mcp.ts`), incremented once per successful tool invocation.
+- **API operation calls**: a count per operation, for the `api/*.ts` handlers reached outside the MCP transport, since those are a third, distinct traffic class alongside web-UI and MCP usage.
+- **A `since` timestamp**: when the current counting period began, so the maintainer can interpret the counts as “since this date” rather than assuming they cover all of Kilotest’s history; reset only if the file is intentionally cleared.
+
+Smoke-test traffic never reaches any counted call site (see the corrected finding above), so no separate exclusion logic is needed for the numbers to reflect real usage only, not CI-driven smoke-test traffic against the live service.
+
+This does not attempt to capture request-level detail (no per-event timestamps, no IP, no user-agent), a deliberate scope limit: it answers “is this used, how much, and what kinds of requests” at the frequency level asked for in the backlog item, not “by whom” (MCP’s stateless transport has no stable caller identity to record) or “when, precisely” (an append-only event log with its own retention story is a natural second iteration if the maintainer finds category counts insufficient once seen).
+
+Resource metrics (memory, storage, CPU) are explicitly deferred to a later iteration: PM2 and the host OS already expose these (`pm2 monit`, `free`, `df`), so an initial feature set should first confirm the usage-frequency counters are worth having before building a second, separate mechanism for resource metrics.
+
+### Observability implementation steps
+
+The steps below are sequential and, except for step 7, interdependent: for example, after step 1 alone `recordMetric` exists but no call site uses it yet, which the 100%-coverage requirement would fail, and after step 3 alone `web/metrics/` exists but is not reachable through `index.ts`’s dispatch, so nothing exercises it. Only step 7 (documentation) is a true no-op on functionality and could land as its own trailing commit without affecting `npm test`. Commit and push freely at intermediate points on the `observability` branch as work progresses, to avoid losing work, but only open the pull request to `main` once every step is complete and the full check suite (100% coverage, typecheck, lint, tests) passes, per Kilotest’s branch-protection rules (`docs/SERVICE.md`, “Branch protection”).
+
+1. Add a small counters module (e.g. functions in `util.ts` or a new `metrics.ts`) that loads `db/metrics.json` at startup (creating it with zeroed counts if absent, following the `getTestRequests`-style ENOENT-creates-default pattern), and exposes one shared utility function, e.g. `recordMetric(category, name)`, that increments `category.name` and writes the file. No smoke-test check is needed here (see the corrected finding above).
+2. Call `recordMetric` from the three call sites identified above: the generic `.html` GET dispatch in `index.ts` (page views), `mcp.ts`’s tool-call handling (one call per tool, at the point each tool’s handler is invoked), and the `api/*.ts` dispatch (operation calls). Each site passes its own category and name but otherwise calls the same function the same way; no call site implements its own increment logic.
+3. Add `web/metrics/` (`index.ts` exporting `answer()`, `index.html`) rendering the current counts and `since` timestamp as an HTML table, following `web/listTopIssues/`’s pattern for an aggregate-data page, gated by the same `authCode === process.env.AUTH_CODE` check used in `web/reannotateForm/` and `web/reannotate/` (a query-string `authCode` param on the GET request, checked before returning the populated page rather than an error).
+4. Wire `metrics.html` into `index.ts`’s `answer{}` table and `routes.GET`, and add the path to the Caddyfile allowlist (repository copy and `docs/SERVICE.md`’s copy) and to `smokeTest.ts`’s `concretePaths.GET`, per the existing “add a new route” checklist in `docs/SERVICE.md`. No special handling is needed for the smoke test’s own visit to `/metrics.html`: like every other page, it is intercepted by the existing `x-kilotest-smoke` short-circuit in `handleRequest` before `answer.metrics` (and therefore `recordMetric`) ever runs.
+5. Implement the `authCode` gate in `web/metrics/index.ts` per the Access decision above: return `{status: 'error', message: 'Invalid authorization code'}` when the submitted `authCode` query-string parameter does not equal `process.env.AUTH_CODE`, matching the existing pattern in `web/reannotateForm/index.ts`. Note the gate’s location (one check, in one file) for a future change to make the page public, so that revisiting this decision later is a small, easily found edit.
+6. Tests: unit-test `recordMetric` directly (increment, load-on-missing-file) and separately confirm each of the three call sites actually calls it (so a future refactor that bypasses the shared function is caught), plus test `web/metrics/`’s `answer()` (both the authCode gate and the rendered table), and extend `index.test.ts` to cover the new route, maintaining the required 100% coverage.
+7. Document the new page and its data model briefly in `docs/SERVICE.md`, alongside the existing “Alerting” and “Health monitoring” sections, so this becomes part of the same operational-documentation set.
+
+### Observability verification
+
+- `npm run typecheck`, `npm run lint`, and `npm test` pass with 100% coverage maintained.
+- Locally, exercise a few web pages, MCP tool calls, and API operations, then load `/metrics.html` and confirm the counts increment as expected and persist across a process restart (i.e., the write-on-every-call and load-on-startup behavior both work).
+- `node smokeTest.ts` confirms `/metrics.html` is both allowlisted and reachable, and, run against a local instance, confirms the counts in `db/metrics.json` do not change as a result of the smoke test’s own requests (web, API, and MCP), since all of them are intercepted by the existing `x-kilotest-smoke` short-circuit before reaching any counted call site.
+
+### Explicitly deferred (candidates for a future, revised iteration once this is experienced in practice)
+
+- Resource metrics (memory, storage, CPU headroom on the host).
+- Per-event time-series data (e.g., usage trends over time, not just cumulative counts since a reset).
+- Any notion of caller/client identity for MCP usage (would require a new signal, since the current transport is stateless and session-less).
+- A periodic email digest of the metrics, reusing `alerts.ts`, if a page alone proves insufficient for the maintainer’s workflow.
+
+### Revision: separate manager-page activity from real usage (2026-09-19, after initial deployment)
+
+After the initial 7 steps were deployed, the maintainer observed a concrete problem while manually testing the feature in a browser: the maintainer’s own clicks (visiting `/manage.html`, then `/metrics.html`) were themselves being counted as `pageViews`, indistinguishable from real usage. The `x-kilotest-smoke` exclusion (see the corrected finding above) only ever excluded automated smoke-test traffic, not a human maintainer manually operating the browser, so this is a real, separate gap the initial plan did not address.
+
+**Considered and rejected: a cookie set on successful `authCode` verification.** The idea was that submitting a valid `authCode` anywhere already proves the requester is the maintainer, so that moment could set a long-lived cookie, and `recordMetric`’s call sites could check for it and skip counting. This was rejected, not merely deferred, for a structural reason surfaced during design: the very first request of any testing session necessarily happens *before* the cookie exists (a cookie cannot be set retroactively on the request that earns it), so at least one page-view per session would always leak into the counts regardless of where the cookie-setting logic lived. Retroactive correction (decrementing the bootstrap hit once the cookie is issued) was considered and rejected as disproportionate complexity (tracking session state before identity is proven) for a small, shrinking rounding error. A cookie would also have raised a second question, whether it should also suppress the `authCode` prompt on future manager-page visits (a lightweight session-authentication feature): that broader feature was explicitly declined as out of scope for this fix, since it touches every `authCode` check site in the codebase and deserves its own design, not one riding in on a metrics correction.
+
+**Adopted instead: exclude manager pages from `pageViews` by identity, not by session state.** The insight is that manager pages are not the traffic the backlog item’s three questions (is anybody using the web UI, do AI agents use MCP, what request classes occur) are actually about; they are the maintainer operating Kilotest, structurally analogous to how MCP tool calls are agent-driven and web pages are (assumed) human-driven. Excluding them by page name, a static, known set, sidesteps the cookie/session/bootstrap problem entirely: there is no “first request before identity is proven” concept when the exclusion is by identity of the *page*, not identity of the *requester*. The maintainer can also now navigate directly to `/metrics.html?authCode=...` (bookmarked or typed) to authenticate before doing any other browsing, avoiding even the `/manage.html` hit, though this is a workflow suggestion, not a requirement the mechanism depends on.
+
+**The excluded set**: every page linked from `/manage.html` (`enqueueForm.html`, `reannotateForm.html`, `pruneReportsForm.html`, `rewindReportsForm.html`, `expungeReportsForm.html`, `hideReportForm.html`, `unhideReportForm.html`, `ai0BalanceForm.html`, `renewWCAGForm.html`, `metrics.html`), plus the POST-only action pages some of them submit to (`requestAction.html`, which performs the `enqueue` action; `reannotate.html`; `renewWCAG.html`). The self-submitting Form pages (`pruneReportsForm.html` and similarly-shaped others) have only one page for both viewing and acting, so they need no separate action-page entry. `manage.html` itself is also excluded, since it is purely a manager-facing directory of these other pages.
+
+**New data model: a `managerActivity` category, tracking success and failure separately.** A single combined count per manager page (matching how `pageViews`/`apiOperations` work today) was considered and rejected: the actual value of a separate table, once manager pages are pulled out of the main counts anyway, is to make repeated *failed* `authCode` attempts visible as a signal of suspected abuse, distinct from the maintainer’s own successful use. A combined count would hide exactly that signal. `db/metrics.json`’s `managerActivity` category is therefore `Record<string, {ok: number; error: number}>`, keyed by page name, incremented on the outcome actually returned by that page’s `answer()` call (or, for the three POST action pages, by the same success/failure branching `index.ts` already uses to choose between serving the answer page and calling `serveError`).
+
+**New instrumentation required, not just re-categorization.** Because `requestAction.html`, `reannotate.html`, and `renewWCAG.html` are POST-only and were never wired to `recordMetric` in the original 7 steps (step 2 only instrumented the generic `.html` **GET** dispatch, `mcp.ts`’s tool handlers, and `/api/*`), adding them to `managerActivity` is new instrumentation, not a relabeling of existing counts. This was weighed against deferring the three POST action pages and only re-categorizing the already-tracked GET Form pages; instrumenting all of them was chosen for completeness, since an incomplete manager-activity table (views but not the actual submissions and their outcomes) would defeat much of the point of a suspected-abuse signal.
+
+#### Revised implementation steps
+
+1. In `util.ts`, extend the `Metrics` type with `managerActivity: Record<string, {ok: number; error: number}>`, initialized empty alongside the other categories in `getMetrics`’s ENOENT branch. Extend `recordMetric`’s `category` parameter type to include `'managerActivity'`, and give it a way to record an outcome for that category (for example, an optional third `outcome: 'ok' | 'error'` parameter, used only by `managerActivity` calls, defaulting to incrementing a plain count for the other two categories as today).
+2. Define the manager-page name set (the list above) once, in one place (for example an exported `const managerPages` in `util.ts` or `index.ts`), so it is not duplicated across call sites and stays easy to extend if `manage.html` grows new links.
+3. In `index.ts`’s generic `.html` GET dispatch (where `recordMetric('pageViews', topic)` was added in the original step 2), branch on membership in the manager-page set: if `topic` is a manager page, call `recordMetric('managerActivity', topic, 'ok')` instead of recording a `pageViews` hit (the GET dispatch only reaches this call on `answerData.status === 'ok'`, so only the success side applies here; a manager Form page’s own internal validation errors, such as `hideReportForm.html`’s “Invalid authorization code”, already route through the existing `status: 'error'` branch and `serveError`, which is the natural place to add the `error`-outcome call for self-submitting Form pages).
+4. Add `recordMetric('managerActivity', 'requestAction.html', outcome)`, `recordMetric('managerActivity', 'reannotate.html', outcome)`, and `recordMetric('managerActivity', 'renewWCAG.html', outcome)` calls to their respective POST branches in `index.ts`, on both the success path (matching `answerData.status === 'ok'`) and the failure path (the invalid-request / `serveError` branch), so both outcomes are captured for these three previously-uncounted pages.
+5. Update `web/metrics/index.ts` to render a fourth table, “Manager page activity”, with columns for page name, successful count, and failed count, sourced from `metrics.managerActivity`. Keep the existing `pageViews`, `mcpToolCalls`, and `apiOperations` tables as they are, now containing only non-manager traffic as a side effect of step 3.
+6. Update tests: `util.test.ts` for the extended `recordMetric` signature and `managerActivity` counting; `index.test.ts` to confirm a manager page (for example `hideReportForm.html`) increments `managerActivity` rather than `pageViews`, and that both outcomes are recorded for at least one POST action page (for example `renewWCAG.html`, both a valid- and an invalid-`authCode` submission); `web/metrics/metrics.test.ts` for the new table’s rendering.
+7. Document the `managerActivity` category and the manager-page exclusion rationale in `docs/SERVICE.md`’s “Usage metrics” section, alongside the existing description of `pageViews`, `mcpToolCalls`, and `apiOperations`.
+
+#### Explicitly deferred (from this revision)
+
+- Any cookie- or session-based mechanism (rejected above, not merely deferred, given the structural bootstrap problem and the separate, larger session-authentication question it would raise).
+- Alerting on `managerActivity` failure spikes (for example via `sendAlert`, matching how other suspected-abuse-adjacent conditions already alert): worth considering once the table has been observed in practice, but not needed for this initial correction.
 
-## Improve MCP-zero discoverability
+### Correction: the manager-page split did not solve the maintainer’s actual problem (2026-09-19, same day)
 
-### Advice from Gemini
+After the revision above was deployed, the maintainer identified that it missed the mission: the maintainer tests *all* URLs manually in a browser, not only manager-specific ones, so ordinary page views (`tutorialWeb.html`, `listReports.html`, and so on) were still being counted as real usage during manual testing sessions, exactly the original complaint. The manager-page/`managerActivity` split is still worth keeping, since it usefully separates a distinct traffic class and surfaces a suspected-abuse signal (failed `authCode` attempts), but it was never a substitute for excluding the maintainer’s own browsing generally, and this revision’s prior write-up incorrectly treated it as though it were. The cookie-based approach, rejected above for the narrower “does this fully solve the metrics-page bootstrap problem” question, is reconsidered here for the broader, actual goal.
+
+**Revisiting the cookie’s bootstrap gap**: the earlier rejection reasoned that a cookie can never exclude the very request that earns it, so it can never be a *complete* solution to unwanted counting from the metrics page specifically. That reasoning still holds, but it was answering the wrong question: a mechanism that excludes an entire subsequent testing session, at the cost of one first request still being counted, is a large improvement over excluding nothing, which is what the manager-page split alone achieves for ordinary pages. The bootstrap gap is accepted as a small, known, one-time-per-session cost, as originally proposed, and no longer treated as disqualifying.
+
+**Mechanism**: submitting a valid `authCode` on `/metrics.html` sets a cookie whose value is a SHA-256 hash of `process.env.AUTH_CODE` (via Node’s built-in `crypto`, no new dependency), not the raw code itself, so the secret is never placed in a long-lived browser cookie. `recordMetric`’s `pageViews` and `apiOperations` call sites in `index.ts` (both already have the request in scope) recompute the same hash and skip recording when the request’s cookie matches. MCP tool calls are not covered (`mcp.ts`’s handlers do not currently receive the raw request, and a maintainer manually driving MCP tools via a browser is a rare case not worth the signature changes this would require); this is an explicit, narrower scope than full coverage, left for a future iteration if it proves worth doing.
+
+**Why a hash, not a fixed cookie value or the raw `authCode`**: Kilotest’s source is public. A fixed, guessable cookie value (for example a literal `excluded=1`) would let any visitor read the source, set that cookie in their own browser, and exclude themselves from metrics with no effort and no knowledge of `AUTH_CODE`, defeating the feature’s purpose of producing trustworthy counts. Hashing `AUTH_CODE` ensures only someone who has already proven they know the real code (by submitting it correctly) can derive the cookie value, while never exposing the code itself in a cookie a browser (or anyone with access to that browser) could read back out.
+
+**New mechanism needed for `web/metrics/index.ts` to ask `index.ts` to set a cookie**: `answer()` functions return data; they do not touch the `IncomingMessage`/`ServerResponse` objects directly, and `index.ts` alone decides how to translate an `AnswerData` result into an actual HTTP response. To preserve that separation, `AnswerData` gains an optional `setCookie?: {name: string; value: string; maxAgeSeconds: number}` field, which `index.ts`’s `.html` GET dispatch applies via `response.setHeader('Set-Cookie', ...)` when present, right alongside where it already serves `answerData.answerPage`. This is a small, generic extension, not metrics-specific, reusable by any future page that needs to set a cookie.
+
+**Cookie lifetime**: 30 days (`Max-Age=2592000`), long enough that the maintainer rarely needs to resubmit `authCode` across sporadic testing sessions spread over weeks, while still naturally expiring rather than being effectively permanent.
+
+**Two further, separate gaps identified during this correction, both to be fixed alongside the cookie work**:
+
+- A `metrics` row currently appears in both the `pageViews` table and the `managerActivity` table on a server that had previously counted `/metrics.html` under `pageViews` (before the manager-page-split revision existed) and has since accumulated new `managerActivity` counts for the same page. This is stale historical data, not a live code defect: `pageViews.metrics` will not grow further, since `/metrics.html` is now always excluded from it. No code change is needed; the maintainer can manually clear the stale `pageViews.metrics` entry from `db/metrics.json` if the split view is wanted immediately, or let it remain as a historical artifact from before the split.
+- Visits to the home page (`/` and `/index.html`) are not recorded at all, in any category. This is a genuine gap distinct from the manager-page and cookie issues: the home page is served by a separate, raw static-file branch in `index.ts` (`pathname === '/' || pathname === '/index.html'`) that predates, and is not reached by, the generic `.html` GET dispatch where `recordMetric('pageViews', topic)` lives. Since the home page is plausibly the single most important page for answering “is anybody using the Kilotest web UI?”, this branch needs its own `recordMetric('pageViews', 'index')` call, consistent with the cookie-exclusion check applying there too.
+
+#### Corrected implementation steps
+
+1. Add a `getExclusionCookieValue()` helper (in `util.ts`) that computes `crypto.createHash('sha256').update(process.env.AUTH_CODE ?? '').digest('hex')`, used both to set and to verify the cookie, so the two sides can never drift out of sync with each other.
+2. Extend the shared `AnswerData` type (`index.ts`) with an optional `setCookie?: {name: string; value: string; maxAgeSeconds: number}` field. In the `.html` GET dispatch, when `answerData.setCookie` is present, call `response.setHeader('Set-Cookie', ...)` (name, value, `Max-Age`, `Path=/`, `HttpOnly`, `SameSite=Strict`) before serving the answer page.
+3. In `web/metrics/index.ts`, on a successful `authCode` check, include `setCookie: {name: 'kilotestExclude', value: getExclusionCookieValue(), maxAgeSeconds: 2592000}` in the returned `AnswerData`.
+4. Add a small helper (`index.ts` or `util.ts`) that reads `request.headers.cookie`, parses it minimally (no new dependency), and returns whether the exclusion cookie is present with the expected value.
+5. In the `.html` GET dispatch’s success branch, skip the `recordMetric('pageViews', topic)` call (but still serve the page normally) when the exclusion check passes; manager-page (`managerActivity`) recording is unaffected, since it exists for a different purpose (a suspected-abuse signal) and should continue regardless of the maintainer’s own cookie.
+6. Apply the same exclusion check to the `/api/*` GET and POST branches’ `recordMetric('apiOperations', ...)` calls.
+7. Add `recordMetric('pageViews', 'index')` (with the same exclusion check) to the home-page branch (`pathname === '/' || pathname === '/index.html'`) in `index.ts`, closing the separate home-page gap identified above.
+8. Tests: unit-test `getExclusionCookieValue` (deterministic for a given `AUTH_CODE`, changes if `AUTH_CODE` changes); test that submitting a valid `authCode` on `/metrics.html` returns a `Set-Cookie` header with the expected attributes; test that a subsequent request carrying that cookie does not increment `pageViews` or `apiOperations` for an ordinary page, while a request without it does; test that the home page now increments `pageViews.index`.
+9. Document the cookie mechanism, its 30-day lifetime, and the home-page fix in `docs/SERVICE.md`’s “Usage metrics” section.
+
+#### Explicitly out of scope (from this correction)
+
+- MCP tool-call exclusion via the cookie (would require threading the request/cookie header into `mcp.ts`’s handlers; deferred as a separable, lower-priority change).
+- Automatically migrating or clearing the stale `pageViews.metrics` entry described above; left as a manual, one-time cleanup for the maintainer if wanted.
+
+### Addition: a “clear counts” checkbox on `/metrics.html`, and a related bug fix affecting 6 pages (2026-09-20)
+
+The maintainer asked for a checkbox on `/metrics.html`, labeled “Clear counts”, making the page dual-use: viewing the metrics, or clearing them and then viewing the (now empty) result as confirmation. Checking it resets all four categories (`pageViews`, `mcpToolCalls`, `apiOperations`, `managerActivity`) and `since` (to the moment of clearing), a full reset chosen over leaving `managerActivity` untouched, since clearing usage counts and reviewing a suspected-abuse log were judged to be one action, not two, for this feature.
+
+**A pre-existing, unrelated bug surfaced while designing this**: `web/metrics/index.ts`’s new form (like several other manager pages) defaulted to a GET submission, and while checking that choice against the codebase’s convention, the maintainer confirmed the actual intent has always been that all form submissions are `POST` requests, matching `web/enqueueForm/index.html`, `web/reannotateForm/`, `web/renewWCAGForm/index.html`, and `web/requestTestForm/index.html`, which already do this correctly. Five existing pages had omitted `method="post"` and were submitting via GET instead: `ai0BalanceForm.html`, `expungeReportsForm.html`, `hideReportForm.html`, `pruneReportsForm.html`, and `rewindReportsForm.html`. This is fixed alongside the new `metrics.html` form, rather than adding a sixth instance of the same bug.
+
+**Architecture chosen for the fix**: unlike `enqueueForm.html`/`reannotateForm.html`, whose submissions POST to a separate, differently-named action page (`requestAction.html`, `reannotate.html`), all 6 affected pages self-submit to their own exact page name (for example `hideReportForm.html` posts to `/hideReportForm.html`). Splitting them into Form/Action page pairs, to match the other split, was considered and rejected as unnecessarily large: it would introduce 6 new page names and routes where none are needed. Instead, each page keeps its single URL and single `answer(pathTail, search)` function; `index.ts` gains a new POST dispatch branch, shared by all 6 (`selfSubmittingManagerPages`, a subset of the existing `managerPages` set), that reconstructs a query-string-shaped value from the POST body via `new URLSearchParams(postData).toString()` and calls the same `answer[topic]` function GET already calls, so each page’s internal parameter-reading code needs no change for this part of the fix.
+
+**A second bug found by manual testing, not caught by any test**: after wiring the new POST branch, `GET /metrics.html?authCode=...&clearCounts=on` still cleared the counts, because adding a POST path never removed the pre-existing GET path’s ability to process the exact same query-string parameters; the two paths simply both worked. Discovered by hand (`curl`), not by the test suite, since every added test happened to test either GET-with-no-params or POST-with-params, never GET-with-params. This is the actual crux of “make all submissions POST”: the point is not merely that a POST option exists, but that GET must no longer be able to submit at all.
+
+**Fix considered and rejected**: stripping the query string before calling `answer[topic]` on GET (in `index.ts`, one line, for pages in `selfSubmittingManagerPages`), leaving all 6 pages’ own code untouched. Rejected because it fixes the symptom via an implicit invariant a future maintainer editing one of these pages would have no way to discover: the function still contains a working “if this param is present, process the submission” branch, and nothing in it says that branch is unreachable via GET. A future refactor of `index.ts`’s dispatch that reintroduces query-string forwarding, for an unrelated reason, would silently reopen the exact same hole, with no test protecting against it beyond luck.
+
+**Fix adopted**: each of the 6 `answer()` functions gains an explicit method parameter (`answer(pathTail, search, method)`), and their existing “if the action parameter is present” checks become “if `method === 'POST'`”, so a GET request can never trigger the action regardless of what is in its query string, and the rule is a visible condition in the code itself rather than a property of what the caller happens to pass in. `index.ts` passes `'GET'` or `'POST'` at each of the two call sites (the generic `.html` GET dispatch, and the new self-submitting-pages POST branch).
+
+#### Implementation steps for this addition
+
+1. Add `clearMetrics()` to `util.ts`: resets all four categories to empty and `since` to `getNowStamp()`, using the same `metricsLock`, returning the fresh `Metrics` object so the caller can render it without a second read.
+2. In `web/metrics/index.ts`: read a new `clearCounts` checkbox param; if present (and `authCode` valid), call `clearMetrics()` instead of `getMetrics()`, and prepend a “Counts cleared.” confirmation line to the rendered body. Add the checkbox to the initial (no-`authCode`-yet) form.
+3. Fix the method-omission bug: add `method="post"` to the 5 pre-existing forms’ HTML (`ai0BalanceForm`, `expungeReportsForm`, `hideReportForm`, `pruneReportsForm`, `rewindReportsForm`) and to the new `metrics.html` form.
+4. In `index.ts`: add a `selfSubmittingManagerPages` set (the 6 page topics above); add the 6 pages’ exact paths to `routes.POST`; add a new POST-dispatch branch, shared by all 6, calling `answer[topic](pathTail, search, 'POST')` with a reconstructed `search` from the POST body, recording `managerActivity` on both outcomes; widen the existing `.html` GET dispatch’s POST-exclusion guard (`!isPathAllowed('POST', pathname)`) so pages in `selfSubmittingManagerPages` are not excluded from the GET path just because they are now also POST-allowed.
+5. Give each of the 6 `answer()` functions a third `method` parameter, and change their internal “action parameter present” checks to “`method === 'POST'`”, so GET can never trigger the action; update `index.ts`’s two call sites (GET dispatch, new POST branch) to pass the method.
+6. Update the Caddyfile (repository doc copy in `docs/SERVICE.md`, and the real server copy) and `smokeTest.ts`’s `concretePaths.POST`/`postBodies` to add the 6 new POST paths, per the existing “add a new route” checklist.
+7. Tests: add POST-path integration tests for all 6 pages (at least one success and one authCode-failure case each) in `index.test.ts`; add a regression test confirming a GET request carrying the same params a POST would use does **not** trigger the action (the exact bug found by manual testing); add `web/metrics/metrics.test.ts` coverage for the checkbox’s presence, the clear behavior, and the confirmation message; each of the 6 pages’ own unit test files need no change, since their exported `answer()` behavior for a given `method` argument is what is now tested, not a new behavior.
+8. Document the `method="post"` fix and the clear-counts checkbox in `docs/SERVICE.md`’s “Usage metrics” section.
+
+## Improve MCP-zero discoverability (not adopted)
+
+The MCP-Zero protocol for tool discovery and selection has not yet been widely adopted but appears to have gained substantial traction. The following review (by Gemini) asserts that Kilotest is not as well prepared to be discovered and used by MCP-Zero agents as it could be.
+
+Get a second opinion on the following review. Is it prudent for Kilotest to make itself more MCP-Zero-friendly? If so, is the expansion of identifiers from short IDs to descriptions-as-names a necessary part of that improvement? If it is, can the identifier expansion be limited to the externally visible interface, so that it does not creep into the internal implementation? If the proposed revision is prudent, plan it.
+
+### Second opinion (obtained 2026-09-19, not adopted)
+
+The review does not hold up under verification against the actual codebase and against MCP-Zero itself, so its proposal was not adopted.
+
+- **The review’s “before” code quotes are fabricated.** It presents `listReports` as having the description “Lists all reports.”, but no such string exists anywhere in the repo; the real tool (`mcp.ts`, lines 56 to 58) already has a substantive description (“Provide basics about all available reports.”) plus annotations such as `title` and `readOnlyHint`. Its “optimized” replacement tool, `kilotest_list_web_quality_reports` with a `url_filter`/`limit` schema, corresponds to no real tool: `listReports` takes no input at all, and the review’s own footnote admits that schema “does not yet exist”. The proposed `kilotest_run_ensemble_audit` also matches no real tool. These are invented strawmen, not an accurate before/after of Kilotest’s code.
+- **The review misrepresents what MCP-Zero is.** MCP-Zero (arXiv 2506.01056) is a single 2025 research paper, not an adopted protocol, and there is no evidence of the “substantial traction” the review claims. It describes a client-side technique: the agent generates a request, and a routing layer embeds it against whatever names, descriptions, and schemas a server already exposes. Nothing in the paper asks server authors to add a manifest file, adopt name prefixes, or restructure schemas. `mcp-manifest.json` is a real but unrelated third-party convention (mcp-manifest.dev) that the review conflates with MCP-Zero to manufacture a concrete-sounding deliverable.
+- **One claim does check out.** Kilotest is genuinely registered on Smithery and Glama (`README.md`, lines 60 to 61; `docs/AI-TOOLS.md`, lines 46 to 47), so that part of the review is accurate, though it is not evidence for the MCP-Zero framing built around it.
+
+**Conclusion:** it is not prudent to restructure Kilotest’s MCP tools on the basis of this review, since there is no verified mechanism by which the proposed changes would improve discoverability under MCP-Zero specifically. The identifier-expansion question (and whether it could be confined to the externally visible interface without creeping into the internal implementation) is therefore moot; it was not reached. Any future, independently justified improvements to tool naming, descriptions, or package metadata should be evaluated on their own merits (for example, human and LLM legibility in MCP clients generally), not as compliance with a research paper that does not impose such requirements.
+
+### Introduction
 
 To make your **kilotest** MCP server discoverable by **MCP-Zero** agent loops (or enterprise gateways using active tool discovery architectures), you need to optimize how your server represents itself to semantic routers.
 
@@ -201,9 +393,29 @@ Create this file in your project root as `scripts/generate-manifest.js`. It pull
 1. Create the **underlying handler function** for the new `kilotest_get_server_capabilities` tool.
 2. Create a **GitHub Action configuration** to automatically validate these schemas on push.
 
-## Add observability of request metrics
+## Widen test-request duplicate detection (completed)
+
+The duplicate check in `addTestRequest` (`util.ts`), reached by all four test/retest request paths (UI `/requestTest.html` and `/requestRetest.html`; API `requestTest` and `requestRetest` operations), compares an incoming request only against other requests currently pending in `db/jobs/testRequests.json`, i.e., submitted but not yet manually approved by the maintainer into a job. Once a pending request is approved into a job (`web/enqueue/index.ts`), all pending requests for that URL are cleared from `testRequests.json`, so this shared check cannot recognize a duplicate against a request that has already been approved, is currently running, or has already completed. An identical request submitted after approval triggers a fresh alert rather than being filtered.
+
+The four paths differ in what, if anything, they check before reaching this shared dedup, and none of the four closes the gap:
+
+- UI `requestTest` (`web/requestTest/index.ts`) additionally calls `getRequestability`, rejecting the request outright if the URL matches a claimed or queued job. This screens out most already-approved, not-yet-completed requests for a new test, but not ones for a completed report, since `getRequestability` does not consult reports.
+- API `requestTest` (`api/requestTest.ts`) additionally checks `getReportExtracts` and rejects the request if a report already exists for the same description and URL, but, unlike the UI path, does not call `getRequestability`, so it does not screen out a URL that is currently claimed or queued as a job.
+- UI `requestRetest` (`web/requestRetest/index.ts`) and API `requestRetest` (`api/requestRetest.ts`) call neither `getRequestability` nor a same-description-and-URL report check; each only confirms the report being retested is still the latest one for its page before falling through to the shared dedup.
+
+Consider widening the shared `addTestRequest` check itself, so that all four paths benefit uniformly, to also compare against approved, claimed, queued, or completed requests, for example recently created jobs or reports for the same URL and description, and if so over what time window, rather than continuing to rely on each path's own inconsistent pre-check.
+
+## Tighten the DMARC policy after adding aggregate reporting (revised and completed)
 
-Record per-endpoint request counts, latencies, and error rates so that Kilotest managers can observe which API operations are most used and identify performance regressions.
+Resend mail is already fully authenticated and passes DMARC; what remains undecided is reporting and enforcement, not SPF alignment. Confirmed on 2026-09-17 by querying DNS live and by inspecting the full headers of a received alert: SPF passes against a Resend-configured MAIL FROM on `send.kilotest.com` (MX `feedback-smtp.us-east-1.amazonses.com` and TXT `v=spf1 include:amazonses.com ~all`, the pair created by Resend’s “Enable SPF” option), which aligns with the `kilotest.com` From domain under relaxed alignment, and DKIM passes with an aligned `d=kilotest.com` signature (`s=resend`, key published at `resend._domainkey.kilotest.com`). The captured message showed `dmarc=pass` at both the Porkbun forwarding hop and the final Zoho mailbox; Porkbun’s SRS rewrite makes post-forward SPF unaligned, so DKIM is what carries DMARC across the forward. `ALERT_FROM` is `info@kilotest.com`, matching the DMARC domain, and `MANAGER_EMAIL` is the same Porkbun-forwarded address, which terminates at `pool@jpdev.pro` on Zoho.
+
+The maintainer sends no mail as `@kilotest.com` (human mail is sent from `pool@jpdev.pro`), so Resend alerts are currently the domain’s only legitimate sending stream, and tightening DMARC risks no known sender. If transactional mail is added later it will also use `info@kilotest.com`; any provider other than Resend would need its own SPF and DKIM alignment configured before deployment under an enforced policy. The root-domain SPF record has no Resend mechanism and needs none: SPF is evaluated against the MAIL FROM domain, which for Resend is `send.kilotest.com`, not the root domain.
+
+Consider: (1) adding `rua=mailto:info@kilotest.com` to the `_dmarc.kilotest.com` record while still at `p=none` (aggregate reports are zipped XML and reach Zoho through the existing forward); (2) after a few weeks of reports confirm that no unexpected stream is failing, moving to `p=quarantine`, optionally with a `pct=` ramp, and later to `p=reject`; (3) reflecting any DNS change in `docs/SERVICE.md` and its DNS CSV.
+
+## Document alerting (completed)
+
+Document when and how Kilotest sends alerts to its maintainer, including the subscription to Resend and the DNS configuration for Resend authentication.
 
 ## Serve explanation page for human MCP requests (completed)
 

package/docs/SERVICE.md

@@ -145,6 +145,26 @@ The DNS records for `kilotest.com` are configured as follows. In each case, `TTL
    - **Host**: _acme-challenge.kilotest.com
    - **Answers**: validation tokens provided by Porkbun ACME client
 
+6. **TXT** (SPF for Porkbun-handled mail)
+   - **Host**: kilotest.com
+   - **Answer**: `v=spf1 include:_spf.porkbun.com ~all`
+
+7. **MX and TXT** (Resend MAIL FROM domain; provides SPF alignment for Resend mail)
+   - **Host**: send.kilotest.com
+   - **Answers**: MX `feedback-smtp.us-east-1.amazonses.com` (priority 10); TXT `v=spf1 include:amazonses.com ~all`
+
+8. **TXT** (Resend DKIM public key)
+   - **Host**: resend._domainkey.kilotest.com
+   - **Answer**: an RSA public key in `p=...` form, issued by Resend
+
+9. **TXT** (DMARC policy)
+   - **Host**: _dmarc.kilotest.com
+   - **Answer**: `v=DMARC1; p=none;`
+
+10. **TXT** (site verification)
+    - **Host**: kilotest.com
+    - **Answers**: `google-site-verification=...` and `smithery-verification=...` tokens issued by Google and Smithery
+
 The DNS configuration as CSV:
 
 ```csv
@@ -153,7 +173,13 @@ kilotest.com,kilotest.com,A,149.28.208.106,3600,0
 kilotest.com,www.kilotest.com,CNAME,kilotest.com,3600,0
 kilotest.com,kilotest.com,MX,fwd1.porkbun.com,3600,10
 kilotest.com,kilotest.com,MX,fwd2.porkbun.com,3600,20
+kilotest.com,send.kilotest.com,MX,feedback-smtp.us-east-1.amazonses.com,3600,10
 kilotest.com,kilotest.com,TXT,"v=spf1 include:_spf.porkbun.com ~all",3600,0
+kilotest.com,send.kilotest.com,TXT,"v=spf1 include:amazonses.com ~all",3600,0
+kilotest.com,resend._domainkey.kilotest.com,TXT,"p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDWjPkYLQ+xYcmMRODdXoXl7QTXK5TkhNQXjl5HcPvk1D5A5eqtjuw17lgZOQy1wiUq8BGId+Iq3BWO32kYTJo+lWMQJ8/Me4M0xQ0qaYPmtTJ9cQ8oyrFUuheu7T0MqTYEwYRbHwE74WkQSQTHnCx0Uy98pcXqOaucA3n1v3YkjQIDAQAB",3600,0
+kilotest.com,_dmarc.kilotest.com,TXT,"v=DMARC1; p=none;",3600,0
+kilotest.com,kilotest.com,TXT,google-site-verification=EnNe2chUyUaV645ENd3c6pHs1PZ_Zb3u5516HSirvrA,3600,0
+kilotest.com,kilotest.com,TXT,smithery-verification=6a3a2327c26bc478be748041f340e14fa50c4386eea3b2692eeffda703b8b3f9,3600,0
 kilotest.com,_acme-challenge.kilotest.com,TXT,GIzAHOFW416fHp7cuTkg4gvJDsyuPZvsPlSsnyViFLQ,3600,0
 kilotest.com,_acme-challenge.kilotest.com,TXT,qQdIIiUSC76PvYuku-AxPsRPY-fJV7T7i3b8fimlFjU,3600,0
 ```
@@ -216,7 +242,7 @@ kilotest.com {
   }
   @allowedPOST {
     method POST
-    path /mcp /requestRetest.html/* /requestTest.html /requestAction.html /reannotate.html /renewWCAG.html /api/* /tutorialWebComment.html /tutorialAIComment.html /worker/job /worker/report
+    path /mcp /requestRetest.html/* /requestTest.html /requestAction.html /reannotate.html /renewWCAG.html /ai0BalanceForm.html /expungeReportsForm.html /hideReportForm.html /metrics.html /pruneReportsForm.html /rewindReportsForm.html /showHiddenReportsForm.html /unhideReportForm.html /api/* /tutorialWebComment.html /tutorialAIComment.html /worker/job /worker/report
   }
   # Respond to OPTIONS requests.
   @allowedOPTIONS method OPTIONS
@@ -356,7 +382,7 @@ Kilotest uses [Resend](https://resend.com/) as its transactional email delivery
 
 ### DNS configuration for Resend
 
-Resend authenticates outgoing mail using DKIM, and mail receivers additionally consult SPF and DMARC records for the sending domain. The `kilotest.com` DNS zone (managed at [porkbun.com](https://porkbun.com/); see the DNS Configuration section above for the full record set) currently contains the following records relevant to Resend, confirmed live on 2026-09-16 by querying DNS directly:
+Resend authenticates outgoing mail using DKIM, and mail receivers additionally consult SPF and DMARC records for the sending domain. The `kilotest.com` DNS zone (managed at [porkbun.com](https://porkbun.com/); see the DNS Configuration section above for the full record set) currently contains the following records relevant to Resend, confirmed live on 2026-09-17 by querying DNS directly and by inspecting the full headers of a delivered alert message:
 
 1. **DKIM (DomainKeys Identified Mail)**
    - **Record type**: TXT (not a CNAME)
@@ -364,10 +390,10 @@ Resend authenticates outgoing mail using DKIM, and mail receivers additionally c
    - **Value**: an RSA public key in `p=...` form, issued by Resend for this domain.
 
 2. **SPF (Sender Policy Framework)**
-   - **Record type**: TXT
-   - **Host**: `kilotest.com` (the root domain)
-   - **Value**: `v=spf1 include:_spf.porkbun.com ~all`
-   - This record does **not** include `include:resend.com` or any other Resend-related mechanism. Whether Resend's deliverability for this domain depends on that inclusion, or relies on DKIM alignment alone under the current relaxed DMARC policy (see below), has not been confirmed; this is worth confirming against the Resend dashboard's domain-verification status rather than assuming either way.
+   - **Record types**: MX and TXT
+   - **Host**: `send.kilotest.com` (a dedicated MAIL FROM subdomain, not the root domain)
+   - **Values**: MX `feedback-smtp.us-east-1.amazonses.com` (priority 10) and TXT `v=spf1 include:amazonses.com ~all`
+   - This pair is what Resend’s “Enable SPF” option creates. It makes Resend send with a MAIL FROM on `send.kilotest.com`, which aligns with the `kilotest.com` From domain under relaxed DMARC alignment. Because Resend sends through Amazon SES, the include target is `amazonses.com`, not `resend.com`. The separate root-domain TXT record `v=spf1 include:_spf.porkbun.com ~all` authorizes Porkbun’s servers for mail sent as `@kilotest.com` through Porkbun; SPF is evaluated against the MAIL FROM domain, so the root record plays no role in Resend authentication and needs no Resend mechanism.
 
 3. **DMARC (Domain-based Message Authentication, Reporting and Conformance)**
    - **Record type**: TXT
@@ -375,7 +401,7 @@ Resend authenticates outgoing mail using DKIM, and mail receivers additionally c
    - **Value**: `v=DMARC1; p=none;`
    - The `p=none` policy is monitoring-only: it asks receivers to take no special action on mail that fails alignment, rather than quarantining or rejecting it. No `rua=` reporting address is configured, so no aggregate DMARC reports are being sent anywhere.
 
-These records, along with the MX records already documented above, are the complete set of email-related DNS configuration for `kilotest.com` as of the date given above. Any future change to Resend's configuration (for example, rotating the DKIM key, or tightening the DMARC policy from `p=none` to `p=quarantine` once deliverability is confirmed stable) should be reflected here and in the DNS Configuration section's CSV.
+A delivered alert message inspected on 2026-09-17 confirms this configuration end to end: `spf=pass` with `smtp.mailfrom` on `send.kilotest.com`, `dkim=pass` with `d=kilotest.com` (`s=resend`), and `dmarc=pass` at both the Porkbun forwarding hop and the final mailbox. Any future change to Resend’s configuration (for example, rotating the DKIM key, adding a `rua=` reporting address to the DMARC record, or tightening the DMARC policy from `p=none` to `p=quarantine`) should be reflected here and in the DNS Configuration section’s CSV.
 
 ### Setting environment variables in production
 
@@ -410,6 +436,57 @@ To change any of the five alert variables:
 3. Check the maintainer's email spam folder; some email providers filter unfamiliar senders.
 4. Verify DMARC, SPF, and DKIM records are correctly configured.
 
+## Usage metrics
+
+Kilotest records basic counts of how it is used, so the maintainer can answer questions such as whether the web UI or the MCP server is being used, and what classes of requests are made, without needing external tooling.
+
+### What is recorded
+
+`recordMetric(category, name)`, in `util.ts`, increments a count for a `(category, name)` pair and writes the result to `db/metrics.json`. It is called from three places:
+
+- The generic `.html` GET dispatch in `index.ts`, once per successfully served web page, recorded under the `pageViews` category by page name (for example `tutorialWeb`, `listReports`), unless the page is a manager-only page (see "Manager page activity" below), in which case it is recorded there instead.
+- The home page (`/` and `/index.html`), served by its own branch in `index.ts` rather than through the generic `.html` dispatch, recorded under `pageViews` as `index`.
+- Each of the 8 MCP tool handlers in `mcp.ts`, once per successful tool call, recorded under the `mcpToolCalls` category by tool name (for example `listReports`, `requestTest`).
+- The `/api/*` GET and POST service branches in `index.ts`, once per call, recorded under the `apiOperations` category by operation name.
+
+`db/metrics.json` also stores a `since` time stamp, set when the file is first created, so the counts can be read as "since this date" rather than assumed to cover Kilotest's entire history. `getMetrics()` backfills any category absent from an existing `db/metrics.json` (for example one written before a category such as `managerActivity` existed), so an older file remains readable rather than causing every subsequent request to error.
+
+#### Manager page activity
+
+Every page linked from `/manage.html` (`enqueueForm.html`, `reannotateForm.html`, `pruneReportsForm.html`, `rewindReportsForm.html`, `expungeReportsForm.html`, `hideReportForm.html`, `showHiddenReportsForm.html`, `unhideReportForm.html`, `ai0BalanceForm.html`, `renewWCAGForm.html`, `metrics.html`), plus the POST-only action pages some of them submit to (`requestAction.html`, `reannotate.html`, `renewWCAG.html`), is excluded from `pageViews` and recorded instead under a separate `managerActivity` category, keyed by page name, with `ok` and `error` counts tracked separately.
+
+This exclusion exists because manager pages reflect the maintainer operating Kilotest, not the usage the other three categories are meant to reveal. Tracking `ok` and `error` outcomes separately, rather than a single combined count, makes a spike of failed `authCode` submissions against a manager page visible as a possible sign of an attempted attack, a signal a combined count would hide.
+
+#### Excluding the maintainer's own manual testing
+
+The manager-page exclusion above only keeps the maintainer's use of manager-only pages out of `pageViews`; it does not, by itself, keep the maintainer's manual testing of *ordinary* pages (for example clicking through `tutorialWeb.html` or `listReports.html` after a deployment) from being counted as real usage. A separate mechanism addresses that: submitting a valid `authCode` on `/metrics.html` sets a cookie (`kilotestExclude`), valid for 30 days, whose value is a SHA-256 hash of `AUTH_CODE` rather than the code itself, so the secret is never placed in a long-lived browser cookie. Every subsequent request from that browser that carries a matching cookie is excluded from `pageViews` and `apiOperations` (not from `managerActivity`, which should keep counting regardless, since it exists to surface suspected abuse, including from the maintainer's own browser if its cookie were ever compromised).
+
+A fixed or guessable cookie value was deliberately avoided: Kilotest's source is public, so anyone could otherwise read it and set the same cookie in their own browser to exclude themselves from metrics at no cost, defeating the feature. Hashing `AUTH_CODE` means only someone who has already proven they know the real code can derive the correct cookie value.
+
+This mechanism has one known, accepted limitation: the very first request of a testing session, the one that submits the `authCode` and earns the cookie, is necessarily made before the cookie exists and so cannot itself be excluded (visible today as the `metrics` page's own `managerActivity` count, which is unaffected, since that page is a manager page regardless). This is treated as a small, one-time-per-session cost rather than something worth building retroactive correction for.
+
+MCP tool calls are not currently covered by this cookie, since `mcp.ts`'s handlers do not receive the raw request needed to read it, and a maintainer manually driving MCP tools from a browser is a rare case; this is an explicit, narrower scope, left for a future iteration if it proves worth extending.
+
+Requests from the periodic smoke test (`smokeTest.ts`) are not recorded: `index.ts`'s `handleRequest` intercepts any request bearing the `x-kilotest-smoke` header before any dispatch, route handler, or MCP tool call runs, so smoke-test traffic never reaches a `recordMetric` call site in the first place.
+
+This is a small, initial feature set, not a complete observability solution: it counts requests by class and frequency, but does not record resource metrics (memory, storage, CPU; already available via `pm2 monit`, `free`, and `df`), per-event time-series data, or any caller identity (Kilotest's MCP transport is stateless and has no stable per-caller identifier).
+
+### Viewing the metrics
+
+The counts are displayed at `/metrics.html`, linked from `/manage.html`, as four tables: web page views, MCP tool calls, API operation calls, and manager page activity (successful and failed counts per manager page). Like the other self-submitting manager-power pages that display a form on GET (`/hideReportForm.html`, `/showHiddenReportsForm.html`, `/ai0BalanceForm.html`), a GET request always displays a form requesting an authorization code, since the maintainer following the `/manage.html` link has had no earlier opportunity to supply one; only a POST request (the form's own submission) either shows the counts, if the code matches the `AUTH_CODE` environment variable, or reports an error if it does not. This makes the data a manager-only capability for now, not because it is considered more sensitive than other manager-only data, but because it may later be made public, and starting restricted keeps that as a small, easily found reversal (dropping the `authCode` check in `web/metrics/index.ts`) rather than a rearchitecture.
+
+A GET request's query string is never processed as a submission, even if it happens to contain a valid `authCode` (for example typed or pasted into the address bar): all 7 self-submitting manager pages (`ai0BalanceForm.html`, `expungeReportsForm.html`, `hideReportForm.html`, `metrics.html`, `pruneReportsForm.html`, `rewindReportsForm.html`, `showHiddenReportsForm.html`, plus `unhideReportForm.html`) take an explicit HTTP method argument and only act on a submission when that method is POST, so the same query-string parameters that a POST body carries are inert on GET. This closes a gap found during development: 5 of these pages had omitted `method="post"` from their HTML forms and were submitting via GET, and even after adding a POST option, the pages continued to also accept the identical action via GET until this method check was added.
+
+Every manager-facing form and action page (the 7 self-submitting pages above, plus `enqueue.ts`, `reannotate.ts`, and `renewWCAG.ts`, the separate action pages that `enqueueForm.html`, `reannotateForm.html`, and `renewWCAGForm.html` submit to) reports the same generic message, "Invalid request", for any failed submission on its own POST, whether the cause is an invalid `authCode` or something else about the request. This is deliberate: naming the authorization code specifically would confirm to an attacker which part of a guessed submission was wrong, information a generic message withholds. The real reason is still available to the maintainer, both in the server log (`serveError` always logs the full error object) and, for authCode failures specifically, as a count in `/metrics.html`'s manager page activity table.
+
+### Protecting the hidden report list
+
+`unhideReportForm.html` lists every hidden report by the name of the page it reports on, so that a manager can choose one to unhide, but that same list is also the information a hidden report is meant to keep out of public view. To avoid disclosing it to anyone who simply requests the URL, `unhideReportForm.html` has no GET route at all; `index.ts`'s generic `.html` GET dispatch excludes it explicitly via a `noDirectGetPages` set, even though its `answer()` function is otherwise a self-submitting manager page like the others. It is reachable only as the response to a valid submission of a separate gate, `showHiddenReportsForm.html`: a bare form requesting an authorization code, with no other content, whose own GET route discloses nothing about hidden reports. On a valid code, `showHiddenReportsForm`'s POST handler renders `unhideReportForm`'s own page (the hidden-report list and its unhide form) as its response; on an invalid code, it returns the same generic error response as any other manager page's failed `authCode` check. `unhideReportForm.html` keeps its own POST route, since it remains self-submitting: each unhide action re-serves the form with the updated list, so a manager can unhide several reports in one sitting without returning to `showHiddenReportsForm.html` each time. The `/manage.html` link to this flow points at `showHiddenReportsForm.html` and reads "View the list of experimental reports", naming neither "hidden" nor "declassify", so that the link text itself does not hint that a restricted list exists.
+
+#### Clearing the counts
+
+`/metrics.html`'s form includes a "Clear counts" checkbox. Submitting the form with it checked (and a valid `authCode`) resets all four categories, and the `since` time stamp, to empty, then renders the (now empty) result as confirmation, alongside a "Counts cleared." message.
+
 ## Performance
 
 The Cloud Compute host, in initial testing, took about 2.5 as long to process an example job as an Apple M2 Pro MacBook Pro with 16GB of memory. After tuning, the ratio was reduced to about 1.7.