@jrpool/kilotest 52.0.0 → 57.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.devin/config.local.json +4 -1
- package/.github/workflows/ci.yml +3 -0
- package/README.md +1 -1
- package/api/awaitTest.test.ts +114 -0
- package/api/awaitTest.ts +108 -0
- package/api/listReports.ts +2 -2
- package/api/orderNewTest.test.ts +213 -0
- package/api/orderNewTest.ts +145 -0
- package/api/orderRetest.test.ts +179 -0
- package/api/orderRetest.ts +119 -0
- package/api/requestFeature.test.ts +36 -0
- package/api/requestFeature.ts +26 -4
- package/api/{requestTest.test.ts → requestNewTest.test.ts} +57 -23
- package/api/{requestTest.ts → requestNewTest.ts} +31 -36
- package/api/requestRetest.test.ts +24 -6
- package/api/requestRetest.ts +10 -29
- package/api/routes.test.ts +7 -4
- package/api/routes.ts +45 -7
- package/api/schemas.ts +120 -10
- package/api/util.ts +59 -2
- package/api/version.ts +6 -2
- package/capability.md +9 -6
- package/docs/AI-TOOLS.md +3 -3
- package/docs/SERVICE.md +19 -8
- package/env.example +39 -0
- package/index.test.ts +146 -24
- package/index.ts +129 -36
- package/lhm.plugin.json +1 -1
- package/linkValidation.test.ts +2 -2
- package/llms-full.txt +35 -4
- package/llms.txt +6 -3
- package/mcp.test.ts +109 -12
- package/mcp.ts +146 -26
- package/memory/MEMORY.md +1 -0
- package/memory/plan_issue_111_instant_test.md +198 -0
- package/openapi.yaml +456 -9
- package/package.json +2 -1
- package/routes.test.ts +1 -1
- package/routing.test.ts +2 -2
- package/scripts/generate-openapi.ts +1 -1
- package/server.json +1 -1
- package/sitemap.xml +1 -1
- package/smokeTest.ts +2 -2
- package/util.test.ts +486 -2
- package/util.ts +276 -45
- package/web/listReports/index.html +1 -1
- package/web/listReports/listReports.test.ts +1 -1
- package/web/{requestTest → requestNewTest}/index.ts +10 -1
- package/web/{requestTest/requestTest.test.ts → requestNewTest/requestNewTest.test.ts} +18 -2
- package/web/{requestTestForm → requestNewTestForm}/index.html +1 -1
- package/web/{requestTestForm/requestTestForm.test.ts → requestNewTestForm/requestNewTestForm.test.ts} +6 -6
- package/web/requestRetest/index.ts +9 -0
- package/web/requestRetest/requestRetest.test.ts +14 -0
- package/web/tutorialAI/index.ts +25 -3
- package/web/tutorialAI/tutorialAI.test.ts +32 -0
- package/web/tutorialWeb/index.ts +25 -3
- package/web/tutorialWeb/tutorialWeb.test.ts +32 -0
- /package/web/{requestTest → requestNewTest}/index.html +0 -0
- /package/web/{requestTestForm → requestNewTestForm}/index.ts +0 -0
package/capability.md
CHANGED
|
@@ -2,11 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
## Capability
|
|
4
4
|
|
|
5
|
-
Web-page front-end quality audit. Kilotest tests
|
|
5
|
+
Web-page front-end quality audit. Kilotest tests web pages for front-end quality, namely accessibility, usability, and standards conformity, and makes structured findings available for retrieval.
|
|
6
6
|
|
|
7
7
|
## Purpose
|
|
8
8
|
|
|
9
|
-
Language models and AI platforms often need accurate, page-specific data about the front-end quality of a web page to answer user questions. Rule engines, such as `axe-core`, `QualWeb`, and `WAVE`, provide such data, but with large coverage gaps. Orchestrating and reconciling numerous complementary rule engines would be difficult. Kilotest solves that problem by testing pages with an ensemble of numerous independent rule engines, making their results comparable, saving the results as JSON reports, mining the reports for facts at selectable levels of detail.
|
|
9
|
+
Language models and AI platforms often need accurate, page-specific data about the front-end quality of a web page to answer user questions. Rule engines, such as `axe-core`, `QualWeb`, and `WAVE`, provide such data, but with large coverage gaps. Orchestrating and reconciling numerous complementary rule engines would be difficult. Kilotest solves that problem by testing pages with an ensemble of numerous independent rule engines, making their results comparable, saving the results as JSON reports, and mining the reports for facts at selectable levels of detail.
|
|
10
10
|
|
|
11
11
|
## Interfaces
|
|
12
12
|
|
|
@@ -26,17 +26,20 @@ A list of available reports.
|
|
|
26
26
|
|
|
27
27
|
## Subsequent inputs
|
|
28
28
|
|
|
29
|
-
- If there is no report about the wanted page, a request
|
|
30
|
-
- If there is a report about the wanted page, requests for findings from the report at successive levels of detail.
|
|
29
|
+
- If there is no report about the wanted page, a request or order for a new test, with a page description, its URL, and a reason for testing it.
|
|
30
|
+
- If there is already a report about the wanted page, requests for findings from the report at successive levels of detail, or, if the report is deemed obsolete, a request or order for a retest, with a reason for retesting it.
|
|
31
|
+
- Complaints and suggestions about features of Kilotest.
|
|
31
32
|
|
|
32
33
|
## Subsequent outputs
|
|
33
34
|
|
|
34
|
-
- From a request
|
|
35
|
+
- From a **request** for a new test or a retest: if valid, confirmation of receipt. If the request is manager-approved and executed, the list of available reports will include the new report, typically in 1 hour to 1 day.
|
|
36
|
+
- From an **order** for a new test or a retest: if valid, confirmation of receipt and immediate approval or rejection. If the request is approved, the output will include the identifier of the forthcoming report and two methods for getting the report: (1) allowing time (2 to 4 minutes) for completion of the report and then requesting it, and (2) making a request whose response will await the report and then link to it. Orders are enabled in the MCP and API interfaces only.
|
|
35
37
|
- From a request for findings: Details about issues reported, related WCAG standards, what rules were violated, and which DOM elements violated them.
|
|
38
|
+
- From complaints and suggestions about features: acknowledgment of receipt and of the manager being notified.
|
|
36
39
|
|
|
37
40
|
## Side effects
|
|
38
41
|
|
|
39
|
-
Kilotest reads the specified
|
|
42
|
+
Kilotest reads the specified web page during testing. It does not modify the target site.
|
|
40
43
|
|
|
41
44
|
## Cost
|
|
42
45
|
|
package/docs/AI-TOOLS.md
CHANGED
|
@@ -42,7 +42,7 @@ The external features that support the use of Kilotest as a collection of AI too
|
|
|
42
42
|
- A [pull request](https://github.com/public-apis/public-apis/pull/6346/changes) to add Kilotest to the list of public APIs in the `public-apis` repository.
|
|
43
43
|
- An [issue](https://github.com/APIs-guru/openapi-directory/issues/2677) to add Kilotest to `openapi-directory`.
|
|
44
44
|
- A [pull request](https://github.com/w3c/wai-evaluation-tools-list/pull/1153) to add Kilotest to the WAI evaluation tools list.
|
|
45
|
-
- Registration of Kilotest as an active server in the [official MCP Registry](https://registry.modelcontextprotocol.io
|
|
45
|
+
- Registration of Kilotest as an active server in the [official MCP Registry](https://registry.modelcontextprotocol.io) (`io.github.jrpool/kilotest`), maintained by the [Model Context Protocol](https://modelcontextprotocol.io/) project. The [PulseMCP](https://www.pulsemcp.com/servers) directory and the VS Code MCP gallery both crawl this registry, so Kilotest is discoverable through them with no separate submission.
|
|
46
46
|
- Registration of Kilotest with the [Smithery](https://smithery.ai/servers/pool/kilotest) MCP server registry.
|
|
47
47
|
- Registration of Kilotest with the [Glama](https://glama.ai/mcp/connectors/com.kilotest/kilotest) MCP server registry, including a claim of ownership of the MCP connector by the maintainer via a `TXT` DNS record for `_glama-claim.kilotest.com`, with the value `glama_claim_gVV4HrPdcXr9TEfj1p3icEUBJ3_OVJGX`.
|
|
48
48
|
- Registration of Kilotest with [Awesome MCP Servers](https://www.tensorblock.co/mcp/servers/github-jrpool-kilotest-f518ba36).
|
|
@@ -107,7 +107,7 @@ Other future work:
|
|
|
107
107
|
Capability: web-page-quality-audit
|
|
108
108
|
|
|
109
109
|
Purpose:
|
|
110
|
-
Test a
|
|
110
|
+
Test a web page for accessibility,
|
|
111
111
|
usability, HTML and web-standards problems.
|
|
112
112
|
|
|
113
113
|
Interfaces:
|
|
@@ -123,7 +123,7 @@ Other future work:
|
|
|
123
123
|
rule-engine diagnoses, WCAG mappings
|
|
124
124
|
|
|
125
125
|
Side effects:
|
|
126
|
-
Reads the specified
|
|
126
|
+
Reads the specified web page.
|
|
127
127
|
Does not modify the target site.
|
|
128
128
|
|
|
129
129
|
Cost:
|
package/docs/SERVICE.md
CHANGED
|
@@ -270,7 +270,7 @@ kilotest.com {
|
|
|
270
270
|
}
|
|
271
271
|
@allowedPOST {
|
|
272
272
|
method POST
|
|
273
|
-
path /mcp /requestRetest.html/* /
|
|
273
|
+
path /mcp /requestRetest.html/* /requestNewTest.html /requestAction.html /reannotate.html /renewWCAG.html /ai0BalanceForm.html /deleteNotesForm.html /expungeReportsForm.html /hideReportForm.html /metrics.html /pruneReportsForm.html /rewindReportsForm.html /showHiddenReportsForm.html /unhideReportForm.html /api/* /tutorialWebComment.html /tutorialAIComment.html /worker/job /worker/report
|
|
274
274
|
}
|
|
275
275
|
# Respond to OPTIONS requests.
|
|
276
276
|
@allowedOPTIONS method OPTIONS
|
|
@@ -395,15 +395,13 @@ Alerts are sent to the maintainer in the following conditions:
|
|
|
395
395
|
|
|
396
396
|
5. **Requested full report unavailable**: When a user requests a full report via the API but the report cannot be retrieved (e.g., the report file is missing from disk), an alert is sent with subject `"Kilotest: requested full report unavailable"`, including details on which report was requested and why it could not be retrieved.
|
|
397
397
|
|
|
398
|
-
6. **
|
|
398
|
+
6. **Request for a test or retest received**: When a user submits a request to test or retest a page, whether through the web interface (`/requestNewTest.html` or `/requestRetest.html`) or the API (the `requestNewTest` or `requestRetest` operation), an alert is sent with subject `"Kilotest: request for a new test awaits approval"` or `"Kilotest: request for a retest awaits approval"`, including the target URL, description, and the user's stated reason for the request. Both the UI and API paths funnel through the same underlying request-queuing function, so the subject and the rest of this behavior are identical regardless of which path the request arrived through. A request is treated as a duplicate, and no alert is sent for it, only if a request already pending in `db/jobs/testRequests.json`, i.e., submitted but not yet manually approved by the maintainer into an actual job, for the same URL has the identical description; this check ignores the stated reason and the test-versus-retest distinction, and is not limited to a recent time window, it compares against every request still pending approval for that URL. Once the maintainer approves a pending request (turning it into a job via `/enqueue.html`), all pending requests for that URL are cleared, so this check cannot detect a duplicate against a request that has already been approved, is currently running, or has already completed; an identical request submitted after approval alerts again.
|
|
399
399
|
|
|
400
|
-
7. **
|
|
400
|
+
7. **MCP feature request received**: When a user submits a feature request via the MCP server's `requestFeature` tool, an alert is sent with subject `"MCP feature request received"`, including the text of the feature request, and the request is also appended to `db/featureRequests.json`, the same way tutorial comments are appended to their own files (see item 1), so that a manager can review or delete it later via `/deleteNotesForm.html`.
|
|
401
401
|
|
|
402
|
-
8. **
|
|
402
|
+
8. **WAVE balance low**: When Kilotest detects that the account balance for the WAVE API service (used for accessibility testing) has fallen below a threshold, an alert is sent with subject `"Kilotest: WAVE balance low"`, indicating the number of credits remaining.
|
|
403
403
|
|
|
404
|
-
9. **
|
|
405
|
-
|
|
406
|
-
10. **AI service balance low**: When Kilotest detects that the account balance for an AI service account (such as Anthropic) has fallen below a threshold, an alert is sent with subject `"Kilotest: AI service [N] balance low"`, indicating the approximate remaining balance in dollars and the estimated cost per job.
|
|
404
|
+
9. **AI service balance low**: When Kilotest detects that the account balance for an AI service account (such as Anthropic) has fallen below a threshold, an alert is sent with subject `"Kilotest: AI service [N] balance low"`, indicating the approximate remaining balance in dollars and the estimated cost per job.
|
|
407
405
|
|
|
408
406
|
### Alert configuration
|
|
409
407
|
|
|
@@ -417,6 +415,19 @@ Alerts are sent via the `sendAlert()` function in [alerts.ts](../alerts.ts), whi
|
|
|
417
415
|
|
|
418
416
|
If any of these five variables is missing or empty at alert time, the alert is not sent. Instead, a warning message is logged to the console, and the application continues normal operation. The warning includes the alert subject and body so the maintainer can manually review what would have been sent.
|
|
419
417
|
|
|
418
|
+
### Inbox size limits
|
|
419
|
+
|
|
420
|
+
Four submission types (test requests and retest requests together, web tutorial comments, AI tutorial comments, and MCP feature requests) are each held in a pending "inbox" (`db/jobs/testRequests.json`, `db/comments/tutorialWeb.json`, `db/comments/tutorialAI.json`, and `db/featureRequests.json` respectively) awaiting manual review by the maintainer. Each inbox has a maximum size; once reached, a new, distinct submission is rejected (with a message suggesting the requester instead post to the [GitHub issues page](https://github.com/jrpool/kilotest/issues) or email `info@kilotest.com`) rather than added, so neither the inbox file nor the manager alert emails it generates can grow without bound (see GitHub issue #3, abuse type 3, repeated/automated submissions). This is modeled on a full mailbox rejecting new mail, rather than on a submission rate limit: it is the backlog size itself, not how quickly it grows, that costs the maintainer review time and risks unbounded storage. Every alert sent for an accepted submission reports the resulting inbox count against its cap, so a maintainer returning after an absence can immediately see how close an inbox is to full.
|
|
421
|
+
|
|
422
|
+
Each cap defaults to 20 and can be set independently via an environment variable:
|
|
423
|
+
|
|
424
|
+
1. **`TEST_REQUEST_QUEUE_MAX`**: The maximum number of test and retest requests awaiting approval at once, summed across all requested URLs.
|
|
425
|
+
2. **`TUTORIAL_WEB_COMMENTS_MAX`**: The maximum number of comments held on the web tutorial page (`/tutorialWeb.html`).
|
|
426
|
+
3. **`TUTORIAL_AI_COMMENTS_MAX`**: The maximum number of comments held on the AI agent tutorial page (`/tutorialAI.html`).
|
|
427
|
+
4. **`FEATURE_REQUESTS_MAX`**: The maximum number of feature requests held via the MCP server's `requestFeature` tool.
|
|
428
|
+
|
|
429
|
+
Each variable, if set, must be a non-negative integer; if unset, empty, or not a non-negative integer, the default of 20 is used instead. A value of `0` means no limit: that inbox is never treated as full, regardless of size. Unlike the alert-configuration variables above, these are not required: an unset value never disables the limit, it only selects the default size.
|
|
430
|
+
|
|
420
431
|
### Resend integration
|
|
421
432
|
|
|
422
433
|
Kilotest uses [Resend](https://resend.com/) as its transactional email delivery service. Resend is configured through the environment variables above.
|
|
@@ -489,7 +500,7 @@ Kilotest records basic counts of how it is used, so the maintainer can answer qu
|
|
|
489
500
|
|
|
490
501
|
- The generic `.html` GET dispatch in `index.ts`, once per successfully served web page, recorded under the `pageViews` category by page name (for example `tutorialWeb`, `listReports`), unless the page is a manager-only page (see "Manager page activity" below), in which case it is recorded there instead.
|
|
491
502
|
- The home page (`/` and `/index.html`), served by its own branch in `index.ts` rather than through the generic `.html` dispatch, recorded under `pageViews` as `index`.
|
|
492
|
-
- Each of the
|
|
503
|
+
- Each of the 9 MCP tool handlers in `mcp.ts`, once per successful tool call, recorded under the `mcpToolCalls` category by tool name (for example `listReports`, `requestNewTest`).
|
|
493
504
|
- The `/api/*` GET and POST service branches in `index.ts`, once per call, recorded under the `apiOperations` category by operation name.
|
|
494
505
|
|
|
495
506
|
`db/metrics.json` also stores a `since` time stamp, set when the file is first created, so the counts can be read as "since this date" rather than assumed to cover Kilotest's entire history. `getMetrics()` backfills any category absent from an existing `db/metrics.json` (for example one written before a category such as `managerActivity` existed), so an older file remains readable rather than causing every subsequent request to error.
|
package/env.example
CHANGED
|
@@ -2,6 +2,12 @@
|
|
|
2
2
|
ENVIRONMENT=__placeholder__
|
|
3
3
|
# http (local or reverse-proxied server) or https.
|
|
4
4
|
PROTOCOL=__placeholder__
|
|
5
|
+
# Port the server listens on (defaults to 3000).
|
|
6
|
+
PORT=__placeholder__
|
|
7
|
+
# Path of the private key file. Required only when PROTOCOL is https.
|
|
8
|
+
KEY=__placeholder__
|
|
9
|
+
# Path of the certificate file. Required only when PROTOCOL is https.
|
|
10
|
+
CERT=__placeholder__
|
|
5
11
|
# 3-character password for manager operations.
|
|
6
12
|
AUTH_CODE=__placeholder__
|
|
7
13
|
# ID of each Testaro worker and, for each, its secret and published name.
|
|
@@ -23,6 +29,39 @@ DB_DIR=__placeholder__
|
|
|
23
29
|
# since such a worker could otherwise be induced to fetch a target it has no legitimate reason
|
|
24
30
|
# to reach, such as its own host’s cloud-metadata service.
|
|
25
31
|
ALLOW_INTERNAL_TARGETS=__placeholder__
|
|
32
|
+
# Path of the web tutorial comments file (defaults to db/comments/tutorialWeb.json).
|
|
33
|
+
TUTORIAL_WEB_COMMENTS_PATH=__placeholder__
|
|
34
|
+
# Path of the AI tutorial comments file (defaults to db/comments/tutorialAI.json).
|
|
35
|
+
TUTORIAL_AI_COMMENTS_PATH=__placeholder__
|
|
36
|
+
# Path of the feature requests file (defaults to db/featureRequests.json).
|
|
37
|
+
FEATURE_REQUESTS_PATH=__placeholder__
|
|
38
|
+
|
|
39
|
+
# Inbox size limits: each defaults to 20 if unset, empty, or not a non-negative integer.
|
|
40
|
+
# A value of 0 means no limit.
|
|
41
|
+
|
|
42
|
+
# Maximum number of test/retest requests awaiting approval at once, across all URLs.
|
|
43
|
+
TEST_REQUEST_QUEUE_MAX=__placeholder__
|
|
44
|
+
# Maximum number of comments held on the web tutorial page.
|
|
45
|
+
TUTORIAL_WEB_COMMENTS_MAX=__placeholder__
|
|
46
|
+
# Maximum number of comments held on the AI agent tutorial page.
|
|
47
|
+
TUTORIAL_AI_COMMENTS_MAX=__placeholder__
|
|
48
|
+
# Maximum number of feature requests held via the MCP server's requestFeature tool.
|
|
49
|
+
FEATURE_REQUESTS_MAX=__placeholder__
|
|
50
|
+
# Maximum number of jobs allowed in the job queue (db/jobs/queue) at once. Shared by every
|
|
51
|
+
# path that enqueues a job: manager approval of a pending test/retest request, and the
|
|
52
|
+
# orderNewTest/orderRetest MCP tools, which enqueue directly.
|
|
53
|
+
JOB_QUEUE_MAX=__placeholder__
|
|
54
|
+
|
|
55
|
+
# Timing for the awaitTest MCP tool's wait on a job ordered via orderNewTest/orderRetest.
|
|
56
|
+
# Defaults: 2000ms poll interval, 300000ms (5 minutes) maximum wait. The maximum wait
|
|
57
|
+
# deliberately exceeds the "occasionally up to 4 minutes" callers are told to expect, to
|
|
58
|
+
# leave margin so a job that is genuinely still running is not mistaken for one that
|
|
59
|
+
# never will finish. Lower values are useful for local testing.
|
|
60
|
+
|
|
61
|
+
# Milliseconds between polls while awaitTest waits for a job to complete.
|
|
62
|
+
AWAIT_JOB_POLL_MS=__placeholder__
|
|
63
|
+
# Maximum milliseconds awaitTest waits before giving up.
|
|
64
|
+
AWAIT_JOB_TIMEOUT_MS=__placeholder__
|
|
26
65
|
|
|
27
66
|
# Alert configuration:
|
|
28
67
|
|
package/index.test.ts
CHANGED
|
@@ -24,7 +24,7 @@ process.env.TESTARO_WORKERS = JSON.stringify({
|
|
|
24
24
|
// isAllowedTarget performs. Tests of the check itself (below) override this per test.
|
|
25
25
|
process.env.ALLOW_INTERNAL_TARGETS = 'true';
|
|
26
26
|
// Blank the alert configuration unconditionally, so this file sends no real alert
|
|
27
|
-
// emails (e.g. from the
|
|
27
|
+
// emails (e.g. from the requestNewTest/requestRetest/tutorial-comment tests below) even
|
|
28
28
|
// when run directly (e.g. `npx tsx --test index.test.ts`) rather than via `npm test`,
|
|
29
29
|
// which normally guards against this by --require-ing test/setup.ts first. dotenv does
|
|
30
30
|
// not override an already-set env var, so setting these here, before index.ts is ever
|
|
@@ -459,8 +459,8 @@ test('GET /forbidden-path returns an invalid GET request error', async () => {
|
|
|
459
459
|
|
|
460
460
|
// TESTS: POST routes
|
|
461
461
|
|
|
462
|
-
test('POST /
|
|
463
|
-
const res = await formRequest('POST', '/
|
|
462
|
+
test('POST /requestNewTest.html with valid data returns an HTML page', async () => {
|
|
463
|
+
const res = await formRequest('POST', '/requestNewTest.html', {
|
|
464
464
|
description: `Unique Test Page ${uniqueStamp}`,
|
|
465
465
|
url: `https://example.com/unique-${uniqueStamp}`,
|
|
466
466
|
why: 'Because accessibility matters'
|
|
@@ -469,8 +469,8 @@ test('POST /requestTest.html with valid data returns an HTML page', async () =>
|
|
|
469
469
|
assert.ok(res.headers['content-type'].includes('text/html'));
|
|
470
470
|
});
|
|
471
471
|
|
|
472
|
-
test('POST /
|
|
473
|
-
const res = await formRequest('POST', '/
|
|
472
|
+
test('POST /requestNewTest.html with an already-tested URL returns an error', async () => {
|
|
473
|
+
const res = await formRequest('POST', '/requestNewTest.html', {
|
|
474
474
|
description: 'Mixed Outcomes Page',
|
|
475
475
|
url: 'https://example.com/mixed',
|
|
476
476
|
why: 'Because accessibility matters'
|
|
@@ -479,8 +479,8 @@ test('POST /requestTest.html with an already-tested URL returns an error', async
|
|
|
479
479
|
assert.ok(res.body.includes('already been tested'));
|
|
480
480
|
});
|
|
481
481
|
|
|
482
|
-
test('POST /
|
|
483
|
-
const res = await formRequest('POST', '/
|
|
482
|
+
test('POST /requestNewTest.html with invalid data returns an error', async () => {
|
|
483
|
+
const res = await formRequest('POST', '/requestNewTest.html', {
|
|
484
484
|
description: '',
|
|
485
485
|
url: 'not-a-url',
|
|
486
486
|
why: ''
|
|
@@ -489,8 +489,8 @@ test('POST /requestTest.html with invalid data returns an error', async () => {
|
|
|
489
489
|
assert.ok(res.body.includes('Invalid test request'));
|
|
490
490
|
});
|
|
491
491
|
|
|
492
|
-
test('POST /
|
|
493
|
-
const res = await request('POST', '/
|
|
492
|
+
test('POST /requestNewTest.html with an unreadable body returns an error', async () => {
|
|
493
|
+
const res = await request('POST', '/requestNewTest.html', 'rawbody', {'content-type': 'text/plain'});
|
|
494
494
|
assert.equal(res.statusCode, 400);
|
|
495
495
|
assert.ok(res.body.includes('Unreadable request body'));
|
|
496
496
|
});
|
|
@@ -513,8 +513,8 @@ test('POST /requestRetest.html/invalid/invalid with invalid data returns an erro
|
|
|
513
513
|
assert.ok(res.body.includes('Invalid retest request'));
|
|
514
514
|
});
|
|
515
515
|
|
|
516
|
-
test('POST /api/
|
|
517
|
-
const res = await request('POST', '/api/
|
|
516
|
+
test('POST /api/requestNewTest with valid JSON returns a JSON response', {timeout: 500}, async () => {
|
|
517
|
+
const res = await request('POST', '/api/requestNewTest', {
|
|
518
518
|
description: `API Test Page ${uniqueStamp}`,
|
|
519
519
|
URL: `https://example.com/api-test-${uniqueStamp}`,
|
|
520
520
|
reason: 'Because accessibility matters'
|
|
@@ -544,6 +544,45 @@ test('POST /api/requestFeature with valid JSON returns a JSON response and recor
|
|
|
544
544
|
assert.equal(await getMetricCount('apiOperations', 'requestFeature'), countBefore + 1);
|
|
545
545
|
});
|
|
546
546
|
|
|
547
|
+
test('POST /api/orderNewTest with valid JSON enqueues a job directly and records an API-operation metric', {timeout: 500}, async () => {
|
|
548
|
+
const countBefore = await getMetricCount('apiOperations', 'orderNewTest');
|
|
549
|
+
const res = await request('POST', '/api/orderNewTest', {
|
|
550
|
+
description: `API Ordered Page ${uniqueStamp}`,
|
|
551
|
+
URL: `https://example.com/api-ordered-${uniqueStamp}`,
|
|
552
|
+
reason: 'Because accessibility matters here'
|
|
553
|
+
});
|
|
554
|
+
assert.equal(res.statusCode, 200);
|
|
555
|
+
const body = jsonBody(res);
|
|
556
|
+
assert.ok(body['tool name'] || body['response content']);
|
|
557
|
+
assert.equal(await getMetricCount('apiOperations', 'orderNewTest'), countBefore + 1);
|
|
558
|
+
const reportIdentifier = body['response content']['disposition of your order']['report identifier'];
|
|
559
|
+
assert.ok(reportIdentifier);
|
|
560
|
+
await fs.unlink(path.join(fixtureDBDir, 'jobs', 'queue', `${reportIdentifier.timeStamp}-${reportIdentifier.jobID}.json`));
|
|
561
|
+
});
|
|
562
|
+
|
|
563
|
+
test('POST /api/orderRetest/260202T0000/new with valid JSON enqueues a job directly and records an API-operation metric', async () => {
|
|
564
|
+
const countBefore = await getMetricCount('apiOperations', 'orderRetest');
|
|
565
|
+
const res = await request('POST', '/api/orderRetest/260202T0000/new', {
|
|
566
|
+
reason: 'Because the report is obsolete and needs refreshing'
|
|
567
|
+
});
|
|
568
|
+
assert.equal(res.statusCode, 200);
|
|
569
|
+
const body = jsonBody(res);
|
|
570
|
+
assert.ok(body['tool name'] || body['response content']);
|
|
571
|
+
assert.equal(await getMetricCount('apiOperations', 'orderRetest'), countBefore + 1);
|
|
572
|
+
const reportIdentifier = body['response content']['disposition of your order']['report identifier'];
|
|
573
|
+
assert.ok(reportIdentifier);
|
|
574
|
+
await fs.unlink(path.join(fixtureDBDir, 'jobs', 'queue', `${reportIdentifier.timeStamp}-${reportIdentifier.jobID}.json`));
|
|
575
|
+
});
|
|
576
|
+
|
|
577
|
+
test('POST /api/awaitTest/260101T0000/mix with an already-completed report records an API-operation metric', async () => {
|
|
578
|
+
const countBefore = await getMetricCount('apiOperations', 'awaitTest');
|
|
579
|
+
const res = await request('POST', '/api/awaitTest/260101T0000/mix', {});
|
|
580
|
+
assert.equal(res.statusCode, 200);
|
|
581
|
+
const body = jsonBody(res);
|
|
582
|
+
assert.equal(body['response content']['disposition of your wait'].outcome, 'completed');
|
|
583
|
+
assert.equal(await getMetricCount('apiOperations', 'awaitTest'), countBefore + 1);
|
|
584
|
+
});
|
|
585
|
+
|
|
547
586
|
test('POST /api/invalidService returns an error', async () => {
|
|
548
587
|
const res = await request('POST', '/api/invalidService', {data: 'test'});
|
|
549
588
|
assert.equal(res.statusCode, 400);
|
|
@@ -697,7 +736,7 @@ test('POST /requestAction.html with invalid auth code returns an error and recor
|
|
|
697
736
|
});
|
|
698
737
|
|
|
699
738
|
test('POST /requestAction.html with valid auth code and rejection (no description) returns HTML and records a managerActivity success', async () => {
|
|
700
|
-
await formRequest('POST', '/
|
|
739
|
+
await formRequest('POST', '/requestNewTest.html', {
|
|
701
740
|
description: `Reject Test Page ${uniqueStamp}`,
|
|
702
741
|
url: `https://example.com/reject-${uniqueStamp}`,
|
|
703
742
|
why: 'Because accessibility matters'
|
|
@@ -740,7 +779,7 @@ test('GET /fullReport.json/260101T0007/hid returns an error for an unavailable r
|
|
|
740
779
|
|
|
741
780
|
test('POST /requestAction.html with valid auth code and approval returns HTML', {timeout: 500}, async () => {
|
|
742
781
|
await fs.writeFile(testRequestsPath, '{}\n');
|
|
743
|
-
await formRequest('POST', '/
|
|
782
|
+
await formRequest('POST', '/requestNewTest.html', {
|
|
744
783
|
description: `Approval Test Page ${uniqueStamp}`,
|
|
745
784
|
url: `https://example.com/approval-${uniqueStamp}`,
|
|
746
785
|
why: 'Because accessibility matters'
|
|
@@ -899,8 +938,40 @@ test('GET /targets.html redirects to /listReports.html', async () => {
|
|
|
899
938
|
assert.equal(res.headers['location'], '/listReports.html');
|
|
900
939
|
});
|
|
901
940
|
|
|
902
|
-
test('
|
|
903
|
-
const res = await
|
|
941
|
+
test('GET /requestTest.html redirects to /requestNewTest.html', async () => {
|
|
942
|
+
const res = await request('GET', '/requestTest.html', {}, {followRedirects: false});
|
|
943
|
+
assert.equal(res.statusCode, 301);
|
|
944
|
+
assert.equal(res.headers['location'], '/requestNewTest.html');
|
|
945
|
+
});
|
|
946
|
+
|
|
947
|
+
test('GET /requestTestForm.html redirects to /requestNewTestForm.html', async () => {
|
|
948
|
+
const res = await request('GET', '/requestTestForm.html', {}, {followRedirects: false});
|
|
949
|
+
assert.equal(res.statusCode, 301);
|
|
950
|
+
assert.equal(res.headers['location'], '/requestNewTestForm.html');
|
|
951
|
+
});
|
|
952
|
+
|
|
953
|
+
test('POST /api/requestTest redirects (308, preserving method) to /api/requestNewTest', async () => {
|
|
954
|
+
const res = await request('POST', '/api/requestTest', {
|
|
955
|
+
description: `Old Path Page ${uniqueStamp}`,
|
|
956
|
+
URL: `https://example.com/old-path-${uniqueStamp}`,
|
|
957
|
+
reason: 'Because accessibility matters'
|
|
958
|
+
}, {followRedirects: false});
|
|
959
|
+
assert.equal(res.statusCode, 308);
|
|
960
|
+
assert.equal(res.headers['location'], '/api/requestNewTest');
|
|
961
|
+
});
|
|
962
|
+
|
|
963
|
+
test('POST /api/orderTest redirects (308, preserving method) to /api/orderNewTest', async () => {
|
|
964
|
+
const res = await request('POST', '/api/orderTest', {
|
|
965
|
+
description: `Old Path Ordered Page ${uniqueStamp}`,
|
|
966
|
+
URL: `https://example.com/old-path-ordered-${uniqueStamp}`,
|
|
967
|
+
reason: 'Because accessibility matters'
|
|
968
|
+
}, {followRedirects: false});
|
|
969
|
+
assert.equal(res.statusCode, 308);
|
|
970
|
+
assert.equal(res.headers['location'], '/api/orderNewTest');
|
|
971
|
+
});
|
|
972
|
+
|
|
973
|
+
test('POST /requestNewTest.html with non-https URL returns an error', async () => {
|
|
974
|
+
const res = await formRequest('POST', '/requestNewTest.html', {
|
|
904
975
|
description: 'Test Page',
|
|
905
976
|
url: 'http://example.com',
|
|
906
977
|
why: 'Because accessibility matters'
|
|
@@ -909,10 +980,10 @@ test('POST /requestTest.html with non-https URL returns an error', async () => {
|
|
|
909
980
|
assert.ok(res.body.includes('Invalid test request'));
|
|
910
981
|
});
|
|
911
982
|
|
|
912
|
-
test('POST /
|
|
983
|
+
test('POST /requestNewTest.html with a private-address URL returns an error unless internal targets are allowed', async () => {
|
|
913
984
|
delete process.env.ALLOW_INTERNAL_TARGETS;
|
|
914
985
|
try {
|
|
915
|
-
const res = await formRequest('POST', '/
|
|
986
|
+
const res = await formRequest('POST', '/requestNewTest.html', {
|
|
916
987
|
description: 'Internal Page',
|
|
917
988
|
url: 'https://192.168.1.1/page',
|
|
918
989
|
why: 'Because accessibility matters'
|
|
@@ -1166,6 +1237,57 @@ test('POST /worker/job with a queued job assigns it to the worker', {timeout: 50
|
|
|
1166
1237
|
await fs.unlink(path.join(claimedDir, jobFile)).catch(() => {});
|
|
1167
1238
|
});
|
|
1168
1239
|
|
|
1240
|
+
test('POST /worker/job with a queued job whose host now redirects to a disallowed target rejects it without assigning it', {timeout: 500}, async (t) => {
|
|
1241
|
+
delete process.env.ALLOW_INTERNAL_TARGETS;
|
|
1242
|
+
// Clean up claimed, queue, and failed directories.
|
|
1243
|
+
const claimedDir = path.join(fixtureDBDir, 'jobs', 'claimed');
|
|
1244
|
+
const queueDir = path.join(fixtureDBDir, 'jobs', 'queue');
|
|
1245
|
+
const failedDir = path.join(fixtureDBDir, 'jobs', 'failed');
|
|
1246
|
+
for (const dir of [claimedDir, queueDir]) {
|
|
1247
|
+
await fs.mkdir(dir, {recursive: true});
|
|
1248
|
+
const files = await fs.readdir(dir).catch(() => []);
|
|
1249
|
+
for (const file of files) {
|
|
1250
|
+
await fs.unlink(path.join(dir, file)).catch(() => {});
|
|
1251
|
+
}
|
|
1252
|
+
}
|
|
1253
|
+
const jobFile = '260101T0000-mix.json';
|
|
1254
|
+
const failedJobPath = path.join(failedDir, jobFile);
|
|
1255
|
+
await fs.unlink(failedJobPath).catch(() => {});
|
|
1256
|
+
// The job's host still resolves publicly, but by the time it is dequeued for
|
|
1257
|
+
// assignment it has been reconfigured (e.g. by an attacker) to redirect to a
|
|
1258
|
+
// private address, which only a fetch-based check, not a DNS-only one, can see.
|
|
1259
|
+
t.mock.method(globalThis, 'fetch', async () => ({url: 'https://192.168.1.1/test'}) as Response);
|
|
1260
|
+
try {
|
|
1261
|
+
await fs.writeFile(path.join(queueDir, jobFile), JSON.stringify({
|
|
1262
|
+
id: '260101T0000-mix',
|
|
1263
|
+
target: {what: 'Test Page', url: 'https://example.com/test'},
|
|
1264
|
+
sources: {}
|
|
1265
|
+
}));
|
|
1266
|
+
const auth = Buffer.from('worker1:secret1').toString('base64');
|
|
1267
|
+
const res = await request('POST', '/worker/job', {}, {
|
|
1268
|
+
authorization: `Basic ${auth}`
|
|
1269
|
+
});
|
|
1270
|
+
// The worker should be told there is no job, not given the disallowed job.
|
|
1271
|
+
assert.equal(res.statusCode, 200);
|
|
1272
|
+
const body = jsonBody(res);
|
|
1273
|
+
assert.equal(body.id, undefined);
|
|
1274
|
+
// Wait for the async rename to complete.
|
|
1275
|
+
await new Promise<void>(resolve => setTimeout(resolve, 100));
|
|
1276
|
+
// The job should have been moved from queue to failed, never to claimed.
|
|
1277
|
+
const queueExists = await fs.access(path.join(queueDir, jobFile)).then(() => true).catch(() => false);
|
|
1278
|
+
assert.equal(queueExists, false, 'Job should be removed from queue');
|
|
1279
|
+
const claimedExists = await fs.access(path.join(claimedDir, jobFile)).then(() => true).catch(() => false);
|
|
1280
|
+
assert.equal(claimedExists, false, 'Job should not be moved to claimed directory');
|
|
1281
|
+
const failedExists = await fs.access(failedJobPath).then(() => true).catch(() => false);
|
|
1282
|
+
assert.ok(failedExists, 'Job should be moved to the failed directory');
|
|
1283
|
+
} finally {
|
|
1284
|
+
t.mock.reset();
|
|
1285
|
+
process.env.ALLOW_INTERNAL_TARGETS = 'true';
|
|
1286
|
+
// Clean up.
|
|
1287
|
+
await fs.unlink(failedJobPath).catch(() => {});
|
|
1288
|
+
}
|
|
1289
|
+
});
|
|
1290
|
+
|
|
1169
1291
|
// TESTS: worker/report valid submission
|
|
1170
1292
|
|
|
1171
1293
|
test('POST /worker/report with valid authentication and valid claimed job processes the report', {timeout: 500}, async () => {
|
|
@@ -1319,14 +1441,14 @@ test('GET /style.css with a read error returns an error page', async () => {
|
|
|
1319
1441
|
}
|
|
1320
1442
|
});
|
|
1321
1443
|
|
|
1322
|
-
//
|
|
1444
|
+
// requestNewTest.html and requestRetest.html are POST-only routes (see the
|
|
1323
1445
|
// `routes.POST` list above). They also match the generic '*.html*' GET pattern,
|
|
1324
1446
|
// but the generic `pageName.endsWith('.html')` branch of the GET dispatcher
|
|
1325
1447
|
// excludes any path that is POST-only, since such a path's answer handler
|
|
1326
1448
|
// expects POST's argument list and performs no GET-appropriate rendering.
|
|
1327
1449
|
|
|
1328
|
-
test('GET /
|
|
1329
|
-
const res = await request('GET', '/
|
|
1450
|
+
test('GET /requestNewTest.html without arguments is rejected as an invalid GET request', async () => {
|
|
1451
|
+
const res = await request('GET', '/requestNewTest.html');
|
|
1330
1452
|
assert.equal(res.statusCode, 400);
|
|
1331
1453
|
});
|
|
1332
1454
|
|
|
@@ -1680,18 +1802,18 @@ test('POST /mcp with a valid initialize request returns server info via SSE', as
|
|
|
1680
1802
|
|
|
1681
1803
|
// TESTS: answer error branches
|
|
1682
1804
|
|
|
1683
|
-
test('POST /
|
|
1805
|
+
test('POST /requestNewTest.html with valid format but duplicate URL returns an answer error', {timeout: 500}, async () => {
|
|
1684
1806
|
await fs.writeFile(testRequestsPath, '{}\n');
|
|
1685
1807
|
// First request to create the request. processTestRequest treats a request as a
|
|
1686
1808
|
// duplicate only when description, URL, AND reason all match an existing pending
|
|
1687
1809
|
// request, so the reason must be identical between the two requests here.
|
|
1688
|
-
await formRequest('POST', '/
|
|
1810
|
+
await formRequest('POST', '/requestNewTest.html', {
|
|
1689
1811
|
description: `Dup Test Page ${uniqueStamp}`,
|
|
1690
1812
|
url: `https://example.com/dup-${uniqueStamp}`,
|
|
1691
1813
|
why: 'Because accessibility matters'
|
|
1692
1814
|
});
|
|
1693
1815
|
// Second, identical request should get a duplicate error.
|
|
1694
|
-
const res = await formRequest('POST', '/
|
|
1816
|
+
const res = await formRequest('POST', '/requestNewTest.html', {
|
|
1695
1817
|
description: `Dup Test Page ${uniqueStamp}`,
|
|
1696
1818
|
url: `https://example.com/dup-${uniqueStamp}`,
|
|
1697
1819
|
why: 'Because accessibility matters'
|
|
@@ -1719,7 +1841,7 @@ test('POST /requestRetest.html with valid format but duplicate retest returns an
|
|
|
1719
1841
|
test('POST /requestAction.html with valid auth code approves the same target twice without error', {timeout: 500}, async () => {
|
|
1720
1842
|
await fs.writeFile(testRequestsPath, '{}\n');
|
|
1721
1843
|
// Create a request.
|
|
1722
|
-
await formRequest('POST', '/
|
|
1844
|
+
await formRequest('POST', '/requestNewTest.html', {
|
|
1723
1845
|
description: `Action Dup Page ${uniqueStamp}`,
|
|
1724
1846
|
url: `https://example.com/action-dup-${uniqueStamp}`,
|
|
1725
1847
|
why: 'Because accessibility matters'
|