@jrpool/kilotest 51.1.0 → 55.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -36,7 +36,10 @@
36
36
  "Exec(git mv)",
37
37
  "Exec(perl)",
38
38
  "Exec(ssh)",
39
- "Exec(scp)"
39
+ "Exec(scp)",
40
+ "Exec(sqlite3)",
41
+ "Exec(npm update)",
42
+ "Exec(npm ls)"
40
43
  ]
41
44
  }
42
45
  }
@@ -12,16 +12,19 @@ jobs:
12
12
 
13
13
  steps:
14
14
  - name: Checkout code
15
- uses: actions/checkout@v4
15
+ uses: actions/checkout@v5
16
16
 
17
17
  - name: Set Node.js version
18
- uses: actions/setup-node@v4
18
+ uses: actions/setup-node@v5
19
19
  with:
20
20
  node-version: '>=24.16.0'
21
21
 
22
22
  - name: Install dependencies
23
23
  run: npm ci
24
24
 
25
+ - name: Run typecheck
26
+ run: npm run typecheck
27
+
25
28
  - name: Run lint
26
29
  run: npm run lint
27
30
 
@@ -14,10 +14,10 @@ jobs:
14
14
 
15
15
  steps:
16
16
  - name: Checkout code
17
- uses: actions/checkout@v4
17
+ uses: actions/checkout@v5
18
18
 
19
19
  - name: Set Node.js version
20
- uses: actions/setup-node@v4
20
+ uses: actions/setup-node@v5
21
21
  with:
22
22
  node-version: '>=24.16.0'
23
23
 
@@ -19,7 +19,7 @@ jobs:
19
19
  uses: actions/checkout@v7
20
20
 
21
21
  - name: Set Node.js version
22
- uses: actions/setup-node@v4
22
+ uses: actions/setup-node@v5
23
23
  with:
24
24
  node-version: '>=24.16.0'
25
25
 
package/README.md CHANGED
@@ -10,7 +10,7 @@ An ensemble testing and reporting service for front-end web quality
10
10
 
11
11
  ## Features
12
12
 
13
- This application uses an ensemble of 12 rule engines to test public web pages for front-end quality (i.e. accessibility, usability, and standards conformity).
13
+ This application uses an ensemble of 12 rule engines to test web pages for front-end quality (i.e. accessibility, usability, and standards conformity).
14
14
 
15
15
  The testing paradigm employed by Kilotest is discussed in these papers:
16
16
 
@@ -20,11 +20,11 @@ The testing paradigm employed by Kilotest is discussed in these papers:
20
20
 
21
21
  Kilotest acts as a web server for human users, an MCP server for AI platforms, and an API for programmatic access.
22
22
 
23
- Testing for Kilotest is performed by one or more testing agents that obtain jobs from Kilotest, perform them, and send reports of the results back to Kilotest. Those agents are instances of the [Testaro](https://www.npmjs.com/package/testaro) package.
23
+ Testing for Kilotest is performed by one or more workers (testing agents) that obtain jobs from Kilotest, perform them, and send reports of the results back to Kilotest. Those agents are instances of the [Testaro](https://www.npmjs.com/package/testaro) package. Thus, the pages that Kilotest tests need not be on the public Internet. They need only be reachable by the Testaro workers that do the testing for Kilotest. So, one or more Testaro workers running inside a private intranet can test internal pages, as well as any external ones reachable from inside the intranet.
24
24
 
25
- An active production instance of Kilotest may require multiple testing agents to handle the load, because testing one web page typically takes about 3 minutes and agents test only one page at a time.
25
+ An active production instance of Kilotest may require multiple Testaro workers to handle the load, because testing one web page typically takes about 3 minutes and workers test only one page at a time.
26
26
 
27
- ## Getting started locally with 1 testing worker
27
+ ## Getting started locally with 1 worker
28
28
 
29
29
  ### Installation
30
30
 
@@ -36,7 +36,7 @@ In the steps below, hosts `T` and `K` may be the same host or two different host
36
36
  1. Clone the Kilotest repository into a new directory on host `K`.
37
37
  1. In that directory, install the Kilotest dependencies: `npm install`.
38
38
  1. Copy the `env.testaro` file from the `kilotest` directory to `.env` in the `testaro` directory and replace the `__placeholder__` values in `.env` with actual values.
39
- 1. Copy the `env.example` file in the `kilotest` directory to a new `.env` file in the same directory and replace the `__placeholder__` values in `.env` with actual values. `TESTARO_WORKERS` is a JSON object mapping each Testaro worker's ID to an object with its `secret` and its `name`; the ID and secret for a worker must match the `NETWATCH_URL_WORKER_ID` and `NETWATCH_URL_AUTH` values in that worker's `.env`. A worker authenticates each request with an HTTP Basic `Authorization` header (`id:secret`, base64-encoded); as of this writing that header is not yet sent by the published Testaro package, so this scheme requires a matching update to Testaro's `netWatch.js` request code (only to send the header; no change to Testaro's handling of jobs or reports is needed). The ID is a credential and is never published; the `name` is written by Kilotest into a job's and report's `sources.worker` property, which Testaro does not read or alter, so it is a distinct property from `sources.agent` (Testaro's own, untrusted, self-reported label, which Kilotest ignores). Choose a `name` that is safe to appear in every report Kilotest serves.
39
+ 1. Copy the `env.example` file in the `kilotest` directory to a new `.env` file in the same directory and replace the `__placeholder__` values in `.env` with actual values. `TESTARO_WORKERS` is a JSON object mapping each Testaro worker's ID to an object with its `secret` and its `name`; the ID and secret for a worker must match the `NETWATCH_URL_WORKER_ID` and `NETWATCH_URL_AUTH` values in that worker's `.env`. A worker authenticates each request with an HTTP Basic `Authorization` header (`id:secret`, base64-encoded); as of this writing that header is not yet sent by the published Testaro package, so this scheme requires a matching update to Testaro's `netWatch.js` request code (only to send the header; no change to Testaro's handling of jobs or reports is needed). The ID is a credential and is never published; the `name` is written by Kilotest into a job's and report's `sources.worker` property, which Testaro does not read or alter, so it is a distinct property from `sources.agent` (Testaro's own, untrusted, self-reported label, which Kilotest ignores). Choose a `name` that is safe to appear in every report Kilotest serves. `ALLOW_INTERNAL_TARGETS` controls whether this deployment can be induced to test pages at private, loopback, or link-local addresses; see [`SERVICE.md`](docs/SERVICE.md#testing-target-restrictions) before changing it from its default.
40
40
 
41
41
  ### Usage
42
42
 
@@ -81,9 +81,45 @@ test('requestFeature accepts a non-empty feature request and notifies the manage
81
81
  assert.ok(logged.some(line =>
82
82
  line.startsWith('WARNING (Kilotest: MCP feature request received)')
83
83
  && line.includes('Add a dark mode toggle')
84
+ // The alert reports the resulting queue size, so a maintainer who has been away can
85
+ // see at a glance how close the queue is to full.
86
+ && line.includes('Feature requests now awaiting review: 1 of 20')
84
87
  ));
85
88
  });
86
89
 
90
+ test('requestFeature rejects a request with a fallback channel when the queue is full', async () => {
91
+ // Fill featureRequests.json with 20 distinct requests, reaching the cap.
92
+ const filler = Array.from({length: 20}, (_, i) => ({
93
+ timeStamp: '260101T0000', content: `Filler feature request number ${i}, long enough to pass the length check.`
94
+ }));
95
+ await fs.writeFile(process.env.FEATURE_REQUESTS_PATH!, JSON.stringify(filler));
96
+ const body = await response(['One feature request too many, long enough to pass the length check.']);
97
+ const details = body['response content']['details about your request'] as any;
98
+ assert.ok(details.error.includes('too many feature requests are awaiting review right now.'));
99
+ assert.ok(details.error.includes('https://github.com/jrpool/kilotest/issues'));
100
+ assert.ok(details.error.includes('info@kilotest.com'));
101
+ const featureRequests = JSON.parse(await fs.readFile(process.env.FEATURE_REQUESTS_PATH!, 'utf8'));
102
+ // The rejected request was not recorded.
103
+ assert.equal(featureRequests.length, 20);
104
+ });
105
+
106
+ test('requestFeature never rejects for a full inbox when FEATURE_REQUESTS_MAX is 0 (no limit)', async () => {
107
+ process.env.FEATURE_REQUESTS_MAX = '0';
108
+ try {
109
+ // Fill featureRequests.json past the default cap.
110
+ const filler = Array.from({length: 25}, (_, i) => ({
111
+ timeStamp: '260101T0000', content: `Filler feature request number ${i}, long enough to pass the length check.`
112
+ }));
113
+ await fs.writeFile(process.env.FEATURE_REQUESTS_PATH!, JSON.stringify(filler));
114
+ const body = await response(['One more feature request, long enough to pass the length check.']);
115
+ const details = body['response content']['details about your request'] as any;
116
+ assert.equal(details.error, undefined);
117
+ }
118
+ finally {
119
+ delete process.env.FEATURE_REQUESTS_MAX;
120
+ }
121
+ });
122
+
87
123
  test('requestFeature includes tool name and metadata', async () => {
88
124
  const body = await response(['A sufficiently long feature description']);
89
125
  assert.equal(body['tool name'], 'requestFeature');
@@ -8,7 +8,7 @@
8
8
  import {z} from 'zod';
9
9
  import {getResponseMetadata, getThisHost, getToolsFacts} from './util.ts';
10
10
  import {sendAlert} from '../alerts.ts';
11
- import {checkCommentDuplicate, checkLength, getJSON, getNowStamp} from '../util.ts';
11
+ import {checkCommentDuplicate, checkLength, describeMax, getEnvMax, getJSON, getNowStamp} from '../util.ts';
12
12
  import {requestFeatureResponseSchema} from './schemas.ts';
13
13
  import fs from 'node:fs/promises';
14
14
  import path from 'node:path';
@@ -21,6 +21,13 @@ type ResponseContent = z.infer<typeof requestFeatureResponseSchema>['response co
21
21
  // FUNCTIONS (helpers)
22
22
 
23
23
  const getFeatureRequestsPath = () => process.env.FEATURE_REQUESTS_PATH || path.join(import.meta.dirname, '../db/featureRequests.json');
24
+ // Maximum number of feature requests held at once. Once reached, a new, distinct request
25
+ // is rejected rather than stored, so the feature-requests file and the manager alert
26
+ // emails it generates cannot grow without bound (modeled on a full mailbox rejecting new
27
+ // mail; see GitHub issue #3, abuse type 3). Configurable via FEATURE_REQUESTS_MAX, since
28
+ // different Kilotest deployments' maintainers may want a different size; a value of 0
29
+ // means no limit.
30
+ const getFeatureRequestsMax = () => getEnvMax('FEATURE_REQUESTS_MAX', 20);
24
31
 
25
32
  // FUNCTIONS
26
33
 
@@ -61,7 +68,18 @@ export const response = async (args: string[]) => {
61
68
  }
62
69
  };
63
70
  }
64
- // Otherwise, i.e. if it is not a duplicate:
71
+ // Otherwise, if the feature requests already on file have reached the cap (a cap of 0
72
+ // means no limit, so the queue is never full):
73
+ else if (getFeatureRequestsMax() > 0 && featureRequests.length >= getFeatureRequestsMax()) {
74
+ responseContent = {
75
+ 'details about your request': {
76
+ error: 'request invalid: too many feature requests are awaiting review right now. ' +
77
+ 'Please try again later, or post your request at https://github.com/jrpool/kilotest/issues ' +
78
+ 'or email info@kilotest.com.'
79
+ }
80
+ };
81
+ }
82
+ // Otherwise, i.e. if it is not a duplicate and the queue is not full:
65
83
  else {
66
84
  // Record the feature request alongside the existing ones.
67
85
  featureRequests.push({
@@ -70,8 +88,12 @@ export const response = async (args: string[]) => {
70
88
  });
71
89
  await fs.mkdir(path.dirname(featureRequestsPath), {recursive: true});
72
90
  await fs.writeFile(featureRequestsPath, getJSON(featureRequests));
73
- // Notify the manager.
74
- await sendAlert('Kilotest: MCP feature request received', feature);
91
+ // Notify the manager, including the resulting count, so a maintainer who has been
92
+ // away sees at a glance how urgently the requests need review.
93
+ await sendAlert(
94
+ 'Kilotest: MCP feature request received',
95
+ `${feature}\nFeature requests now awaiting review: ${featureRequests.length} of ${describeMax(getFeatureRequestsMax())}`
96
+ );
75
97
  // Add the disposition to the response content.
76
98
  responseContent = {
77
99
  'details about your request': {
@@ -173,5 +173,23 @@ test('requestRetest accepts a valid retest request for the latest report of a pa
173
173
  line.startsWith('WARNING (Kilotest: new retest request awaits approval)')
174
174
  && line.includes('Mixed Outcomes Page')
175
175
  && line.includes('https://example.com/mixed')
176
+ // The alert reports the resulting queue size, so a maintainer who has been away can
177
+ // see at a glance how close the queue is to full.
178
+ && line.includes('Requests now awaiting approval: 1 of 20')
176
179
  ));
177
180
  });
181
+
182
+ test('requestRetest rejects a request with a fallback channel when the queue is full', async () => {
183
+ // Fill testRequests.json with 20 pending requests, all for one URL, to reach the cap.
184
+ const filler = Array.from({length: 20}, (_, i) => ({
185
+ timeStamp: '260101T0000', description: `Filler Page ${i}`, reason: 'Filler reason.'
186
+ }));
187
+ await fs.writeFile(testRequestsPath, JSON.stringify({'https://example.com/filler': filler}));
188
+ const body = await response(['260101T0003', 'ret', 'A reason that is long enough.']);
189
+ const details = body['response content']['details about your request'] as any;
190
+ assert.equal(details.error, undefined);
191
+ const disposition = body['response content']['disposition of your request'] as any;
192
+ assert.ok(disposition['what happens next'].includes('too many requests are awaiting approval right now.'));
193
+ assert.ok(disposition['what happens next'].includes('https://github.com/jrpool/kilotest/issues'));
194
+ assert.ok(disposition['what happens next'].includes('info@kilotest.com'));
195
+ });
@@ -79,6 +79,10 @@ export const response = async (args: string[]) => {
79
79
  else if (requestResult === 'superseded') {
80
80
  failureReason = 'a later report about a page with the same description is available.'
81
81
  }
82
+ else if (requestResult === 'queueFull') {
83
+ failureReason = 'too many requests are awaiting approval right now. Please try again later, ' +
84
+ 'or post your request at https://github.com/jrpool/kilotest/issues or email info@kilotest.com.'
85
+ }
82
86
  else {
83
87
  failureReason = 'an identical request is already awaiting approval.'
84
88
  }
@@ -17,6 +17,10 @@ import {fixtureDBDir} from '../test/dbFixture.ts';
17
17
  for (const key of ['MANAGER_EMAIL', 'ALERT_API_HOST', 'ALERT_API_PATH', 'ALERT_API_KEY', 'ALERT_FROM']) {
18
18
  process.env[key] = '';
19
19
  }
20
+ // Allow internal targets by default, so tests that submit an ordinary https://example.com/...
21
+ // URL do not depend on real DNS/network access to pass the resolution check that
22
+ // isAllowedTarget performs. The test of the check itself (below) overrides this.
23
+ process.env.ALLOW_INTERNAL_TARGETS = 'true';
20
24
 
21
25
  // SETUP AND TEARDOWN
22
26
 
@@ -77,6 +81,34 @@ test('requestTest rejects a syntactically invalid URL with the correct length',
77
81
  assert.ok(!logged.some(line => line.includes('new test request awaits approval')));
78
82
  });
79
83
 
84
+ test('requestTest rejects a private-address URL unless internal targets are allowed', async () => {
85
+ delete process.env.ALLOW_INTERNAL_TARGETS;
86
+ try {
87
+ const body = await response(['Internal Page', 'https://192.168.1.1/page', 'A reason that is long enough.']);
88
+ const details = body['response content']['details about your request'] as any;
89
+ assert.ok(details.error.includes('does not resolve to a page that this deployment can test'));
90
+ assert.ok(!logged.some(line => line.includes('new test request awaits approval')));
91
+ } finally {
92
+ process.env.ALLOW_INTERNAL_TARGETS = 'true';
93
+ }
94
+ });
95
+
96
+ test('requestTest rejects a URL that redirects to a disallowed target, and alerts a manager', async (t) => {
97
+ delete process.env.ALLOW_INTERNAL_TARGETS;
98
+ t.mock.method(globalThis, 'fetch', async () => ({url: 'https://10.0.0.5/page'}) as Response);
99
+ try {
100
+ const body = await response(['Redirecting Page', 'https://example.com/redirector', 'A reason that is long enough.']);
101
+ const details = body['response content']['details about your request'] as any;
102
+ assert.ok(details.error.includes('does not resolve to a page that this deployment can test'));
103
+ assert.ok(logged.some(line =>
104
+ line.includes('WARNING (Kilotest: request rejected for redirecting to a disallowed target)')
105
+ && line.includes('https://example.com/redirector')
106
+ ));
107
+ } finally {
108
+ process.env.ALLOW_INTERNAL_TARGETS = 'true';
109
+ }
110
+ });
111
+
80
112
  test('requestTest rejects an already-tested page', async () => {
81
113
  const body = await response(['Mixed Outcomes Page', 'https://example.com/mixed', 'A reason that is long enough.']);
82
114
  const details = body['response content']['details about your request'] as any;
@@ -149,6 +181,9 @@ test('requestTest accepts a valid new page request', async () => {
149
181
  line.startsWith('WARNING (Kilotest: new test request awaits approval)')
150
182
  && line.includes('Brand New Page')
151
183
  && line.includes('https://example.com/brandnew')
184
+ // The alert reports the resulting queue size, so a maintainer who has been away can
185
+ // see at a glance how close the queue is to full.
186
+ && line.includes('Requests now awaiting approval: 1 of 20')
152
187
  ));
153
188
  });
154
189
 
@@ -158,3 +193,18 @@ test('requestTest includes disposition information for a valid request', async (
158
193
  assert.ok(disposition);
159
194
  assert.ok(disposition['what happens next']);
160
195
  });
196
+
197
+ test('requestTest rejects a request with a fallback channel when the queue is full', async () => {
198
+ // Fill testRequests.json with 20 pending requests, all for one URL, to reach the cap.
199
+ const filler = Array.from({length: 20}, (_, i) => ({
200
+ timeStamp: '260101T0000', description: `Filler Page ${i}`, reason: 'Filler reason.'
201
+ }));
202
+ await fs.writeFile(testRequestsPath, JSON.stringify({'https://example.com/filler': filler}));
203
+ const body = await response(['One Too Many Page', 'https://example.com/one-too-many', 'A reason that is long enough.']);
204
+ const details = body['response content']['details about your request'] as any;
205
+ assert.equal(details.error, undefined);
206
+ const disposition = body['response content']['disposition of your request'] as any;
207
+ assert.ok(disposition['what happens next'].includes('too many requests are awaiting approval right now.'));
208
+ assert.ok(disposition['what happens next'].includes('https://github.com/jrpool/kilotest/issues'));
209
+ assert.ok(disposition['what happens next'].includes('info@kilotest.com'));
210
+ });
@@ -7,8 +7,9 @@
7
7
 
8
8
  import {z} from 'zod';
9
9
  import {getResponseMetadata, getThisHost, getToolsFacts} from './util.ts';
10
- import {checkLength, isURL, processTestRequest} from '../util.ts';
10
+ import {checkLength, isAllowedRedirectTarget, isAllowedTarget, isURL, processTestRequest} from '../util.ts';
11
11
  import {requestTestResponseSchema} from './schemas.ts';
12
+ import {sendAlert} from '../alerts.ts';
12
13
 
13
14
  // TYPES
14
15
 
@@ -43,6 +44,25 @@ export const response = async (args: string[]) => {
43
44
  error: 'request invalid: you specified an invalid URL for the page'
44
45
  };
45
46
  }
47
+ // Otherwise, i.e. if the URL does not resolve to an allowed target:
48
+ else if (!(await isAllowedTarget(url))) {
49
+ requestDetails = {
50
+ error: 'request invalid: the URL you specified does not resolve to a page that this deployment can test'
51
+ };
52
+ }
53
+ // Otherwise, i.e. if the URL redirects to a target this deployment does not allow:
54
+ else if (!(await isAllowedRedirectTarget(url))) {
55
+ requestDetails = {
56
+ error: 'request invalid: the URL you specified does not resolve to a page that this deployment can test'
57
+ };
58
+ // Alert a manager, since a redirect to a disallowed target on a request that
59
+ // itself named an allowed hostname suggests an attempted or accidental SSRF,
60
+ // rather than an ordinary mistake by a legitimate submitter.
61
+ await sendAlert(
62
+ 'Kilotest: request rejected for redirecting to a disallowed target',
63
+ `A request to test ${url} (described as "${description}") was rejected because it redirects to a target this deployment does not allow (e.g. a private, loopback, or link-local address).`
64
+ );
65
+ }
46
66
  // Otherwise, i.e. if the description and URL are valid:
47
67
  else {
48
68
  // Process the request.
@@ -76,6 +96,10 @@ export const response = async (args: string[]) => {
76
96
  else if (requestResult === 'retest') {
77
97
  failureReason = 'a report about a page with the same description and URL is available.'
78
98
  }
99
+ else if (requestResult === 'queueFull') {
100
+ failureReason = 'too many requests are awaiting approval right now. Please try again later, ' +
101
+ 'or post your request at https://github.com/jrpool/kilotest/issues or email info@kilotest.com.'
102
+ }
79
103
  else {
80
104
  failureReason = 'an identical request is already awaiting approval.'
81
105
  }
package/capability.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## Capability
4
4
 
5
- Web-page front-end quality audit. Kilotest tests public web pages for front-end quality, namely accessibility, usability, and standards conformity, and makes structured findings available for retrieval.
5
+ Web-page front-end quality audit. Kilotest tests web pages for front-end quality, namely accessibility, usability, and standards conformity, and makes structured findings available for retrieval.
6
6
 
7
7
  ## Purpose
8
8
 
@@ -36,7 +36,7 @@ A list of available reports.
36
36
 
37
37
  ## Side effects
38
38
 
39
- Kilotest reads the specified public web page during testing. It does not modify the target site. Testing is asynchronous and may take up to approximately one day before reports are available for newly requested pages.
39
+ Kilotest reads the specified web page during testing. It does not modify the target site. Testing is asynchronous and may take up to approximately one day before reports are available for newly requested pages.
40
40
 
41
41
  ## Cost
42
42
 
package/docs/AI-TOOLS.md CHANGED
@@ -42,7 +42,7 @@ The external features that support the use of Kilotest as a collection of AI too
42
42
  - A [pull request](https://github.com/public-apis/public-apis/pull/6346/changes) to add Kilotest to the list of public APIs in the `public-apis` repository.
43
43
  - An [issue](https://github.com/APIs-guru/openapi-directory/issues/2677) to add Kilotest to `openapi-directory`.
44
44
  - A [pull request](https://github.com/w3c/wai-evaluation-tools-list/pull/1153) to add Kilotest to the WAI evaluation tools list.
45
- - Registration of Kilotest as an active server in the [official MCP Registry](https://registry.modelcontextprotocol.io/v0/servers?search=kilotest) (`io.github.jrpool/kilotest`), maintained by the [Model Context Protocol](https://modelcontextprotocol.io/) project. The [PulseMCP](https://www.pulsemcp.com/servers) directory and the VS Code MCP gallery both crawl this registry, so Kilotest is discoverable through them with no separate submission.
45
+ - Registration of Kilotest as an active server in the [official MCP Registry](https://registry.modelcontextprotocol.io) (`io.github.jrpool/kilotest`), maintained by the [Model Context Protocol](https://modelcontextprotocol.io/) project. The [PulseMCP](https://www.pulsemcp.com/servers) directory and the VS Code MCP gallery both crawl this registry, so Kilotest is discoverable through them with no separate submission.
46
46
  - Registration of Kilotest with the [Smithery](https://smithery.ai/servers/pool/kilotest) MCP server registry.
47
47
  - Registration of Kilotest with the [Glama](https://glama.ai/mcp/connectors/com.kilotest/kilotest) MCP server registry, including a claim of ownership of the MCP connector by the maintainer via a `TXT` DNS record for `_glama-claim.kilotest.com`, with the value `glama_claim_gVV4HrPdcXr9TEfj1p3icEUBJ3_OVJGX`.
48
48
  - Registration of Kilotest with [Awesome MCP Servers](https://www.tensorblock.co/mcp/servers/github-jrpool-kilotest-f518ba36).
@@ -107,7 +107,7 @@ Other future work:
107
107
  Capability: web-page-quality-audit
108
108
 
109
109
  Purpose:
110
- Test a public web page for accessibility,
110
+ Test a web page for accessibility,
111
111
  usability, HTML and web-standards problems.
112
112
 
113
113
  Interfaces:
@@ -123,7 +123,7 @@ Other future work:
123
123
  rule-engine diagnoses, WCAG mappings
124
124
 
125
125
  Side effects:
126
- Reads the specified public web page.
126
+ Reads the specified web page.
127
127
  Does not modify the target site.
128
128
 
129
129
  Cost:
package/docs/SERVICE.md CHANGED
@@ -270,7 +270,7 @@ kilotest.com {
270
270
  }
271
271
  @allowedPOST {
272
272
  method POST
273
- path /mcp /requestRetest.html/* /requestTest.html /requestAction.html /reannotate.html /renewWCAG.html /ai0BalanceForm.html /expungeReportsForm.html /hideReportForm.html /metrics.html /pruneReportsForm.html /rewindReportsForm.html /showHiddenReportsForm.html /unhideReportForm.html /api/* /tutorialWebComment.html /tutorialAIComment.html /worker/job /worker/report
273
+ path /mcp /requestRetest.html/* /requestTest.html /requestAction.html /reannotate.html /renewWCAG.html /ai0BalanceForm.html /deleteNotesForm.html /expungeReportsForm.html /hideReportForm.html /metrics.html /pruneReportsForm.html /rewindReportsForm.html /showHiddenReportsForm.html /unhideReportForm.html /api/* /tutorialWebComment.html /tutorialAIComment.html /worker/job /worker/report
274
274
  }
275
275
  # Respond to OPTIONS requests.
276
276
  @allowedOPTIONS method OPTIONS
@@ -417,6 +417,19 @@ Alerts are sent via the `sendAlert()` function in [alerts.ts](../alerts.ts), whi
417
417
 
418
418
  If any of these five variables is missing or empty at alert time, the alert is not sent. Instead, a warning message is logged to the console, and the application continues normal operation. The warning includes the alert subject and body so the maintainer can manually review what would have been sent.
419
419
 
420
+ ### Inbox size limits
421
+
422
+ Four submission types (test requests and retest requests together, web tutorial comments, AI tutorial comments, and MCP feature requests) are each held in a pending "inbox" (`db/jobs/testRequests.json`, `db/comments/tutorialWeb.json`, `db/comments/tutorialAI.json`, and `db/featureRequests.json` respectively) awaiting manual review by the maintainer. Each inbox has a maximum size; once reached, a new, distinct submission is rejected (with a message suggesting the requester instead post to the [GitHub issues page](https://github.com/jrpool/kilotest/issues) or email `info@kilotest.com`) rather than added, so neither the inbox file nor the manager alert emails it generates can grow without bound (see GitHub issue #3, abuse type 3, repeated/automated submissions). This is modeled on a full mailbox rejecting new mail, rather than on a submission rate limit: it is the backlog size itself, not how quickly it grows, that costs the maintainer review time and risks unbounded storage. Every alert sent for an accepted submission reports the resulting inbox count against its cap, so a maintainer returning after an absence can immediately see how close an inbox is to full.
423
+
424
+ Each cap defaults to 20 and can be set independently via an environment variable:
425
+
426
+ 1. **`TEST_REQUEST_QUEUE_MAX`**: The maximum number of test and retest requests awaiting approval at once, summed across all requested URLs.
427
+ 2. **`TUTORIAL_WEB_COMMENTS_MAX`**: The maximum number of comments held on the web tutorial page (`/tutorialWeb.html`).
428
+ 3. **`TUTORIAL_AI_COMMENTS_MAX`**: The maximum number of comments held on the AI agent tutorial page (`/tutorialAI.html`).
429
+ 4. **`FEATURE_REQUESTS_MAX`**: The maximum number of feature requests held via the MCP server's `requestFeature` tool.
430
+
431
+ Each variable, if set, must be a non-negative integer; if unset, empty, or not a non-negative integer, the default of 20 is used instead. A value of `0` means no limit: that inbox is never treated as full, regardless of size. Unlike the alert-configuration variables above, these are not required: an unset value never disables the limit, it only selects the default size.
432
+
420
433
  ### Resend integration
421
434
 
422
435
  Kilotest uses [Resend](https://resend.com/) as its transactional email delivery service. Resend is configured through the environment variables above.
@@ -489,7 +502,7 @@ Kilotest records basic counts of how it is used, so the maintainer can answer qu
489
502
 
490
503
  - The generic `.html` GET dispatch in `index.ts`, once per successfully served web page, recorded under the `pageViews` category by page name (for example `tutorialWeb`, `listReports`), unless the page is a manager-only page (see "Manager page activity" below), in which case it is recorded there instead.
491
504
  - The home page (`/` and `/index.html`), served by its own branch in `index.ts` rather than through the generic `.html` dispatch, recorded under `pageViews` as `index`.
492
- - Each of the 8 MCP tool handlers in `mcp.ts`, once per successful tool call, recorded under the `mcpToolCalls` category by tool name (for example `listReports`, `requestTest`).
505
+ - Each of the 9 MCP tool handlers in `mcp.ts`, once per successful tool call, recorded under the `mcpToolCalls` category by tool name (for example `listReports`, `requestTest`).
493
506
  - The `/api/*` GET and POST service branches in `index.ts`, once per call, recorded under the `apiOperations` category by operation name.
494
507
 
495
508
  `db/metrics.json` also stores a `since` time stamp, set when the file is first created, so the counts can be read as "since this date" rather than assumed to cover Kilotest's entire history. `getMetrics()` backfills any category absent from an existing `db/metrics.json` (for example one written before a category such as `managerActivity` existed), so an older file remains readable rather than causing every subsequent request to error.
@@ -595,6 +608,14 @@ If restoration becomes necessary, the files can be copied in the opposite direct
595
608
 
596
609
  After that the maintainer can inspect the original and restored directories and delete or move report files as needed. After that the `db/reportsExtract.json` file should be deleted, so it will be regenerated when next needed.
597
610
 
611
+ ### Testing target restrictions
612
+
613
+ A page submitted for testing need not be publicly accessible; it need only be reachable from the Testaro worker assigned the job. Without a restriction on which addresses a worker may be sent to, an arbitrary requester could induce a worker to fetch a target it has no legitimate reason to reach, such as [a private or link-local address](https://developer.mozilla.org/en-US/docs/Glossary/Private_network) on the network local to that worker, including a metadata service run by a cloud host. This class of attack is known as [server-side request forgery (SSRF)](https://developer.mozilla.org/en-US/docs/Web/Security/Attacks/SSRF).
614
+
615
+ Kilotest guards against this with the `ALLOW_INTERNAL_TARGETS` environment variable (documented in `env.example`). Left unset, a deployment restricts testing to addresses that resolve to the public address space, checked both when a URL is submitted and again against the actually visited URL of each report before that report is stored (since a redirect could otherwise route a worker to a disallowed target after submission-time validation passed). Setting it to `true` opts a deployment in to testing private, loopback, and link-local addresses, and removes this protection entirely for that Kilotest instance.
616
+
617
+ Setting `ALLOW_INTERNAL_TARGETS` to `true` is not safe merely because every Testaro worker able to poll that Kilotest instance is confined to a private network. A report on an internal page is stored and served by Kilotest itself, so if that Kilotest instance is reachable from the public Internet, anyone able to reach it, not only someone with access to the private network, can read the content of that report, including page text and, depending on job configuration, a screenshot or the HTML source of the page. Setting this variable to `true` is appropriate only for a Kilotest instance that is itself kept off the public Internet, or otherwise restricts who can request and retrieve reports, in addition to confining every one of its Testaro workers to the private network being tested.
618
+
598
619
  ### Possible future Testaro integration
599
620
 
600
621
  Kilotest uses Testaro to run jobs. In previous versions of Kilotest, Testaro was a dependency. It is currently not a dependency. Instead, Testaro instances are installed on one or more other hosts, and each instance polls Kilotest to ask for jobs to run.
package/env.example CHANGED
@@ -2,6 +2,12 @@
2
2
  ENVIRONMENT=__placeholder__
3
3
  # http (local or reverse-proxied server) or https.
4
4
  PROTOCOL=__placeholder__
5
+ # Port the server listens on (defaults to 3000).
6
+ PORT=__placeholder__
7
+ # Path of the private key file. Required only when PROTOCOL is https.
8
+ KEY=__placeholder__
9
+ # Path of the certificate file. Required only when PROTOCOL is https.
10
+ CERT=__placeholder__
5
11
  # 3-character password for manager operations.
6
12
  AUTH_CODE=__placeholder__
7
13
  # ID of each Testaro worker and, for each, its secret and published name.
@@ -15,6 +21,32 @@ DEPLOYED_KILOTEST_HOST=https://kilotest.com
15
21
  # Path of the data directory (defaults to the db subdirectory of the project root). Tests set
16
22
  # this to a fixture directory so they never read or write the real db.
17
23
  DB_DIR=__placeholder__
24
+ # Whether this deployment allows testing pages at private, loopback, or link-local addresses
25
+ # (true) or restricts testing to publicly routable addresses (any other value or omission).
26
+ # Set this to true only when every Testaro worker that can poll this Kilotest instance is
27
+ # intentionally deployed on a private network to test that network’s own internal pages;
28
+ # leave it unset for a deployment whose Testaro workers accept jobs from arbitrary requesters,
29
+ # since such a worker could otherwise be induced to fetch a target it has no legitimate reason
30
+ # to reach, such as its own host’s cloud-metadata service.
31
+ ALLOW_INTERNAL_TARGETS=__placeholder__
32
+ # Path of the web tutorial comments file (defaults to db/comments/tutorialWeb.json).
33
+ TUTORIAL_WEB_COMMENTS_PATH=__placeholder__
34
+ # Path of the AI tutorial comments file (defaults to db/comments/tutorialAI.json).
35
+ TUTORIAL_AI_COMMENTS_PATH=__placeholder__
36
+ # Path of the feature requests file (defaults to db/featureRequests.json).
37
+ FEATURE_REQUESTS_PATH=__placeholder__
38
+
39
+ # Inbox size limits: each defaults to 20 if unset, empty, or not a non-negative integer.
40
+ # A value of 0 means no limit.
41
+
42
+ # Maximum number of test/retest requests awaiting approval at once, across all URLs.
43
+ TEST_REQUEST_QUEUE_MAX=__placeholder__
44
+ # Maximum number of comments held on the web tutorial page.
45
+ TUTORIAL_WEB_COMMENTS_MAX=__placeholder__
46
+ # Maximum number of comments held on the AI agent tutorial page.
47
+ TUTORIAL_AI_COMMENTS_MAX=__placeholder__
48
+ # Maximum number of feature requests held via the MCP server's requestFeature tool.
49
+ FEATURE_REQUESTS_MAX=__placeholder__
18
50
 
19
51
  # Alert configuration:
20
52