@jrpool/kilotest 24.0.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.local.json +9 -0
- package/DEVELOPMENT.md +5860 -0
- package/LICENSE +21 -0
- package/README.md +44 -0
- package/SERVICE.md +268 -0
- package/aceconfig.js +28 -0
- package/ai0BalanceForm/index.html +30 -0
- package/ai0BalanceForm/index.js +79 -0
- package/alerts.js +73 -0
- package/diagnoses/index.html +46 -0
- package/diagnoses/index.js +140 -0
- package/env.example +21 -0
- package/env.testaro +17 -0
- package/error.html +18 -0
- package/eslint.config.mjs +53 -0
- package/favicon.ico +0 -0
- package/index.html +39 -0
- package/index.js +639 -0
- package/issues/index.html +20 -0
- package/issues/index.js +173 -0
- package/job.json +100 -0
- package/manage/index.html +32 -0
- package/manage/index.js +22 -0
- package/package.json +38 -0
- package/pm2.config.js +15 -0
- package/reannotate/index.html +19 -0
- package/reannotate/index.js +39 -0
- package/reannotateForm/index.html +29 -0
- package/reannotateForm/index.js +114 -0
- package/recActionForm/index.html +33 -0
- package/recActionForm/index.js +49 -0
- package/reportHideForm/index.html +29 -0
- package/reportHideForm/index.js +89 -0
- package/reportIssue/index.html +38 -0
- package/reportIssue/index.js +181 -0
- package/reportIssues/index.html +47 -0
- package/reportIssues/index.js +259 -0
- package/reportUnhideForm/index.html +29 -0
- package/reportUnhideForm/index.js +89 -0
- package/reportsExpungeForm/index.html +29 -0
- package/reportsExpungeForm/index.js +105 -0
- package/reportsPruneForm/index.html +29 -0
- package/reportsPruneForm/index.js +105 -0
- package/reportsRewindForm/index.html +29 -0
- package/reportsRewindForm/index.js +105 -0
- package/retestRec/index.html +23 -0
- package/retestRec/index.js +19 -0
- package/retestRecForm/index.html +27 -0
- package/retestRecForm/index.js +36 -0
- package/rules/index.html +28 -0
- package/rules/index.js +71 -0
- package/style.css +196 -0
- package/targets/index.html +37 -0
- package/targets/index.js +170 -0
- package/testOrder/index.html +23 -0
- package/testOrder/index.js +62 -0
- package/testRec/index.html +23 -0
- package/testRec/index.js +25 -0
- package/testRecForm/index.html +34 -0
- package/testRecForm/index.js +22 -0
- package/tutorial/images/newsletter-form.png +0 -0
- package/tutorial/index.html +796 -0
- package/tutorial/index.js +53 -0
- package/util.js +686 -0
- package/wcagMap.json +102 -0
- package/wcagRenew/index.html +19 -0
- package/wcagRenew/index.js +70 -0
- package/wcagRenewForm/index.html +25 -0
- package/wcagRenewForm/index.js +22 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Jonathan Pool
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Kilotest
|
|
2
|
+
|
|
3
|
+
An ensemble testing service with a focus on accessibility
|
|
4
|
+
|
|
5
|
+
## Features
|
|
6
|
+
|
|
7
|
+
This application uses an ensemble of 10 tools to test public web pages for standards conformity, usability, and accessibility.
|
|
8
|
+
|
|
9
|
+
The testing paradigm employed by Kilotest is discussed in these papers:
|
|
10
|
+
|
|
11
|
+
- [How to run a thousand accessibility tests](https://medium.com/cvs-health-tech-blog/how-to-run-a-thousand-accessibility-tests-63692ad120c3)
|
|
12
|
+
- [Testaro: Efficient Ensemble Testing for Web Accessibility](https://arxiv.org/abs/2309.10167)
|
|
13
|
+
- [Accessibility Metatesting: Comparing Nine Testing Tools](https://arxiv.org/abs/2304.07591)
|
|
14
|
+
|
|
15
|
+
Kilotest acts as a server with human users (such as you) and with one or more testing agents that obtain jobs from Kilotest and do the actual testing. Those agents are instances of the [Testaro](https://www.npmjs.com/package/testaro) package.
|
|
16
|
+
|
|
17
|
+
An active production instance of Kilotest may require multiple testing agents to handle the load, because testing one web page typically takes about 3 minutes and agents test only one page at a time.
|
|
18
|
+
|
|
19
|
+
## Getting started locally with 1 testing agent
|
|
20
|
+
|
|
21
|
+
### Installation
|
|
22
|
+
|
|
23
|
+
In the steps below, hosts `T` and `K` may be the same host or two different hosts. Host `T` can be a Debian stable, Ubuntu LTS, Windows, or macOS host. Host `K` can be any server host that can run the latest LTS version of Node.js. If hosts `T` and `K` differ, then they must be open to `https` traffic and host `K` must permit `https` requests from host `T`.
|
|
24
|
+
|
|
25
|
+
1. Clone the [Testaro project](https://github.com/jrpool/testaro) into a new directory on host `T`.
|
|
26
|
+
1. In that directory, install the Testaro dependencies: `npm install`.
|
|
27
|
+
1. Update the Testaro dependencies and rebuild: `npm run deps`.
|
|
28
|
+
1. Clone the Kilotest repository into a new directory on host `K`.
|
|
29
|
+
1. In that directory, install the Kilotest dependencies: `npm install`.
|
|
30
|
+
1. Copy the `env.testaro` file from the `kilotest` directory to `.env` in the `testaro` directory and replace the `__placeholder__` values in `.env` with actual values.
|
|
31
|
+
1. Copy the `env.example` file in the `kilotest` directory to a new `.env` file in the same directory and replace the `__placeholder__` values in `.env` with actual values.
|
|
32
|
+
|
|
33
|
+
### Usage
|
|
34
|
+
|
|
35
|
+
1. In the `testaro` directory, make Testaro start listening for jobs: `node call netWatch true nn true`, where `nn` is the number of seconds to wait between checks for new jobs.
|
|
36
|
+
1. In the `kilotest` directory, start the Kilotest service: `node index`.
|
|
37
|
+
|
|
38
|
+
### Contributing
|
|
39
|
+
|
|
40
|
+
Contributions are welcome! You can use GitHub issues to initiate discussions and propose changes. If you want to contribute code, please fork the repository and create a pull request.
|
|
41
|
+
|
|
42
|
+
## Making Kilotest a service
|
|
43
|
+
|
|
44
|
+
See the `SERVICE.md` file for instructions on how to make Kilotest a service.
|
package/SERVICE.md
ADDED
|
@@ -0,0 +1,268 @@
|
|
|
1
|
+
# Service
|
|
2
|
+
|
|
3
|
+
This document describes one deployment of Kilotest as a web service.
|
|
4
|
+
|
|
5
|
+
## Host
|
|
6
|
+
|
|
7
|
+
The host of the service is a [Vultr](https://www.vultr.com) Cloud Compute High Frequency virtual machine named `jpdev` with the IPV4 address 149.28.208.106. The host has 1 vCPU, 2GB of RAM, and 64GB NVMe of storage.
|
|
8
|
+
|
|
9
|
+
The server operating system is Ubuntu LTS 22.04.
|
|
10
|
+
|
|
11
|
+
The server has a `sudo`-capable non-root user named `linuxuser`, which is the owner of the project directory and files.
|
|
12
|
+
|
|
13
|
+
Connection to the host is made with `ssh linuxuser@kilotest.com`. Periodically the server requires password authentication in addition to public key authentication. The password is available in the server details on the Vultr web console, after the administrator logs in via GitHub.
|
|
14
|
+
|
|
15
|
+
## Applications
|
|
16
|
+
|
|
17
|
+
Kilotest is installed at `/opt/jpdev/kilotest` on the server.
|
|
18
|
+
|
|
19
|
+
Any other application `xyz` can be installed at `/opt/jpdev/xyz` on the server.
|
|
20
|
+
|
|
21
|
+
## Process management
|
|
22
|
+
|
|
23
|
+
Kilotest is managed with [PM2](https://pm2.keymetrics.io) on the server (not on the local development host). The PM2 configuration is specified in the repository as `pm2.config.js`:
|
|
24
|
+
|
|
25
|
+
```javascript
|
|
26
|
+
module.exports = {
|
|
27
|
+
apps: [
|
|
28
|
+
{
|
|
29
|
+
name: 'kilotest',
|
|
30
|
+
script: 'index.js',
|
|
31
|
+
instances: 1,
|
|
32
|
+
autorestart: true,
|
|
33
|
+
watch: false,
|
|
34
|
+
max_memory_restart: '500M',
|
|
35
|
+
env: {
|
|
36
|
+
NODE_ENV: 'production',
|
|
37
|
+
BASE_PATH: '/',
|
|
38
|
+
DEMO_SSE_DELAY_MS: '100'
|
|
39
|
+
}
|
|
40
|
+
}
|
|
41
|
+
]
|
|
42
|
+
};
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
When the PM2 configuration or environment is changed, restart PM2 with:
|
|
46
|
+
|
|
47
|
+
```text
|
|
48
|
+
pm2 restart kilotest --time --update-env
|
|
49
|
+
pm2 save
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
The server configuration has been tuned for improved performance. The file edited for this purpose is `/etc/default/zramswap`:
|
|
53
|
+
|
|
54
|
+
```properties
|
|
55
|
+
# Compression algorithm selection
|
|
56
|
+
# speed: lz4 > zstd > lzo
|
|
57
|
+
# compression: zstd > lzo > lz4
|
|
58
|
+
# This is not inclusive of all that is available in latest kernels
|
|
59
|
+
# See /sys/block/zram0/comp_algorithm (when zram module is loaded) to see
|
|
60
|
+
# what is currently set and available for your kernel[1]
|
|
61
|
+
# [1] https://github.com/torvalds/linux/blob/master/Documentation/blockdev/zram.txt#L86
|
|
62
|
+
ALGO=zstd
|
|
63
|
+
|
|
64
|
+
# Specifies the amount of RAM that should be used for zram
|
|
65
|
+
# based on a percentage the total amount of available memory
|
|
66
|
+
# This takes precedence and overrides SIZE below
|
|
67
|
+
PERCENT=75
|
|
68
|
+
|
|
69
|
+
# Specifies a static amount of RAM that should be used for
|
|
70
|
+
# the ZRAM devices, this is in MiB
|
|
71
|
+
#SIZE=256
|
|
72
|
+
|
|
73
|
+
# Specifies the priority for the swap devices. See swapon(2)
|
|
74
|
+
# for more details. Higher number = higher priority
|
|
75
|
+
# This should probably be higher than hdd/ssd swaps.
|
|
76
|
+
#PRIORITY=100
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
This enables `zram` and decreases the amount of disk swapping.
|
|
80
|
+
|
|
81
|
+
## Keepalive
|
|
82
|
+
|
|
83
|
+
The SSH configuration on the local client is customized so that connections will be kept alive longer than the default. This customization is performed in the `~/.ssh/config` file:
|
|
84
|
+
|
|
85
|
+
```ssh-config
|
|
86
|
+
Host kilotest.com 149.28.208.106
|
|
87
|
+
HostName 149.28.208.106
|
|
88
|
+
User linuxuser
|
|
89
|
+
IdentityFile ~/.ssh/id_ed25519
|
|
90
|
+
IdentitiesOnly yes
|
|
91
|
+
ServerAliveInterval 60
|
|
92
|
+
ServerAliveCountMax 3
|
|
93
|
+
TCPKeepAlive yes
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The corresponding server file `/etc/ssh/sshd_config` needs no customization.
|
|
97
|
+
|
|
98
|
+
## Internet domain
|
|
99
|
+
|
|
100
|
+
The domain name `kilotest.com` is registered with [Porkbun](https://porkbun.com) for use by this application.
|
|
101
|
+
|
|
102
|
+
## DNS Configuration
|
|
103
|
+
|
|
104
|
+
### kilotest.com
|
|
105
|
+
|
|
106
|
+
The DNS records for `kilotest.com` are configured as follows. In each case, `TTL` is set to 3600 (1 hour). (The Porkbun interface uses `kilotest.com` as a name where some others use `@`.)
|
|
107
|
+
|
|
108
|
+
1. **A**
|
|
109
|
+
- **Host**: kilotest.com
|
|
110
|
+
- **Answer**: 149.28.208.106
|
|
111
|
+
|
|
112
|
+
2. **AAAA** (if IPv6 is used)
|
|
113
|
+
- **Host**: @
|
|
114
|
+
- **Answer**: IPv6 address
|
|
115
|
+
|
|
116
|
+
3. **CNAME**
|
|
117
|
+
- **Host**: www
|
|
118
|
+
- **Answer**: kilotest.com
|
|
119
|
+
|
|
120
|
+
4. **MX** (for email forwarding)
|
|
121
|
+
- **Host**: kilotest.com
|
|
122
|
+
- **Answer**: fwd1.porkbun.com or fwd2.porkbun.com
|
|
123
|
+
- **Priority**: 10 or 20, respectively
|
|
124
|
+
|
|
125
|
+
5. **TXT** (domain validation)
|
|
126
|
+
- **Host**: _acme-challenge.kilotest.com
|
|
127
|
+
- **Answers**: validation tokens provided by Porkbun ACME client
|
|
128
|
+
|
|
129
|
+
The DNS configuration as CSV:
|
|
130
|
+
|
|
131
|
+
```csv
|
|
132
|
+
DOMAIN,HOST,TYPE,ANSWER,TTL,PRIO
|
|
133
|
+
kilotest.com,kilotest.com,A,149.28.208.106,3600,0
|
|
134
|
+
kilotest.com,www.kilotest.com,CNAME,kilotest.com,3600,0
|
|
135
|
+
kilotest.com,kilotest.com,MX,fwd1.porkbun.com,3600,10
|
|
136
|
+
kilotest.com,kilotest.com,MX,fwd2.porkbun.com,3600,20
|
|
137
|
+
kilotest.com,kilotest.com,TXT,"v=spf1 include:_spf.porkbun.com ~all",3600,0
|
|
138
|
+
kilotest.com,_acme-challenge.kilotest.com,TXT,GIzAHOFW416fHp7cuTkg4gvJDsyuPZvsPlSsnyViFLQ,3600,0
|
|
139
|
+
kilotest.com,_acme-challenge.kilotest.com,TXT,qQdIIiUSC76PvYuku-AxPsRPY-fJV7T7i3b8fimlFjU,3600,0
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Kilotest service
|
|
143
|
+
|
|
144
|
+
Reliance on the default Vultr DNS resolvers has caused erratic failures. Therefore, the service has been configured to use specific DNS resolvers.
|
|
145
|
+
|
|
146
|
+
These commands disabled `cloud-init` network management:
|
|
147
|
+
|
|
148
|
+
```bash
|
|
149
|
+
sudo mkdir -p /etc/cloud/cloud.cfg.d
|
|
150
|
+
printf "network: {config: disabled}\n" | sudo tee /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
The following `/etc/netplan/01-dns.yaml` file with `root` as owner and group was created and given permissions 600. It custom-configures network management and specifies the DNS resolvers of APNIC Research and Development and Google LLC.
|
|
154
|
+
|
|
155
|
+
```yaml
|
|
156
|
+
network:
|
|
157
|
+
version: 2
|
|
158
|
+
ethernets:
|
|
159
|
+
enp1s0:
|
|
160
|
+
dhcp4: true
|
|
161
|
+
dhcp6: true
|
|
162
|
+
nameservers:
|
|
163
|
+
addresses:
|
|
164
|
+
- 1.1.1.1
|
|
165
|
+
- 8.8.8.8
|
|
166
|
+
- 2606:4700:4700::1111
|
|
167
|
+
- 2001:4860:4860::8888
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
These commands applied and verified the configuration:
|
|
171
|
+
|
|
172
|
+
```bash
|
|
173
|
+
sudo netplan generate
|
|
174
|
+
sudo netplan apply
|
|
175
|
+
resolvectl status
|
|
176
|
+
dig +short A example.com
|
|
177
|
+
curl -sv https://example.com/ -o /dev/null
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
## Request management
|
|
181
|
+
|
|
182
|
+
Requests to `https://kilotest.com` are received on port 443 and processed by [Caddy](https://caddyserver.com/), which forwards them via HTTP to the application at `localhost:3000`. Caddy manages, provisions, and renews a TLS certificate via Let’s Encrypt. Any request to `http://kilotest.com` is received on port 80, and Caddy redirects it to an `https` request. Caddy forwards `https` requests to the [local server](http://localhost:3000), where it is processed by the Kilotest service.
|
|
183
|
+
|
|
184
|
+
The Caddy configuration is maintained and tracked in `/etc/caddy/Caddyfile`:
|
|
185
|
+
|
|
186
|
+
```javascript
|
|
187
|
+
kilotest.com {
|
|
188
|
+
# Enable Zstandard and Gzip compression of responses.
|
|
189
|
+
encode zstd gzip
|
|
190
|
+
# Reverse proxy all requests to localhost:3000.
|
|
191
|
+
reverse_proxy localhost:3000 {
|
|
192
|
+
# Improve SSE latency.
|
|
193
|
+
flush_interval -1
|
|
194
|
+
}
|
|
195
|
+
}
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
This configuration prevents granular reporting by Testaro agents from being buffered, so the updates reach the browser without delay.
|
|
199
|
+
|
|
200
|
+
## Version management
|
|
201
|
+
|
|
202
|
+
When a new version of the `kilotest` package has been published, the service can be updated as follows:
|
|
203
|
+
|
|
204
|
+
1. Connect to the server: `ssh linuxuser@kilotest.com`
|
|
205
|
+
1. Navigate to the package root: `cd /opt/jpdev/kilotest`
|
|
206
|
+
1. Discard any locally altered lockfile: `git stash`
|
|
207
|
+
1. Delete the record of that discard: `git stash drop`
|
|
208
|
+
1. Fetch and merge the new version: `git pull`
|
|
209
|
+
1. Update the dependencies: `npm update`
|
|
210
|
+
1. Update the Playwright browsers: `npx playwright install`
|
|
211
|
+
1. Restart the service: `pm2 restart kilotest`
|
|
212
|
+
|
|
213
|
+
## Performance
|
|
214
|
+
|
|
215
|
+
The Cloud Compute host, in initial testing, took about 2.5 as long to process an example job as an Apple M2 Pro MacBook Pro with 16GB of memory. After tuning, the ratio was reduced to about 1.7.
|
|
216
|
+
|
|
217
|
+
The performances of several alternative host architectures were evaluated, and it was decided to migrate the initial host to a High Frequency compute instance. Any subsequent migration is straightforward if the destination root volume is at least as large as the originating one (64GB). The process is:
|
|
218
|
+
|
|
219
|
+
- Create a snapshot of `jpdev`.
|
|
220
|
+
- Deploy the new host using the snapshot.
|
|
221
|
+
|
|
222
|
+
To evaluate a new host before deciding to keep it, do this after deploying it:
|
|
223
|
+
|
|
224
|
+
1. Connect to it via SSH (`ssh linuxuser@<IP address>`).
|
|
225
|
+
1. Add a firewall rule permitting `http` connections to port 3000 (`sudo ufw allow 3000/tcp`).
|
|
226
|
+
1. Navigate with a browser to `http://<IP address>:3000/`.
|
|
227
|
+
1. Request the sample job.
|
|
228
|
+
1. Observe the elapsed time reported in the result basics.
|
|
229
|
+
|
|
230
|
+
Experimentation revealed that a high-frequency instance could decrease the elapsed-time ratio (compared with the MacBook Pro) to 1.3 to 1.5, and a dedicated instance could decrease it further to about 1.2. Swapping was found eliminated with 4GB of RAM, but that elimination had no significant impact on elapsed time.
|
|
231
|
+
|
|
232
|
+
## Security
|
|
233
|
+
|
|
234
|
+
### Possible future Testaro integration
|
|
235
|
+
|
|
236
|
+
Kilotest uses Testaro to run jobs. In previous versions of Kilotest, Testaro was a dependency. It is currently not a dependenc. Instead, Testaro instances are installed on one or more other hosts, and each instance polls Kilotest to ask for jobs to run.
|
|
237
|
+
|
|
238
|
+
In case Testaro again becomes a dependency of Kilotest, the notes below on security issues will be useful.
|
|
239
|
+
|
|
240
|
+
### Browser privileges
|
|
241
|
+
|
|
242
|
+
Testaro uses Playwright to launch and control headless browsers, often `chromium`. Those browsers navigate to web pages that are tested by the tools that Testaro integrates. The `qualWeb` tool launches its own browser via Puppeteer as a dependency to perform its tests.
|
|
243
|
+
|
|
244
|
+
When either Playwright or Puppeteer launches a `chromium` browser, in most environments it is [sandboxed](https://www.geeksforgeeks.org/ethical-hacking/what-is-browser-sandboxing/). Sandboxing is a security feature that prevents the browser from accessing potentially unsafe system resources. But in the Ubuntu Linux operating system that was installed on the Vultr Cloud Compute host a sandboxed browser requires an [unprivileged user namespace](https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces), and when Ubuntu was installed its configuration disallowed such namespaces. The file `/etc/sysctl.d/99-kilotest-userns.conf` with the content `kernel.apparmor_restrict_unprivileged_userns = 1` prohibited unprivileged user namespaces and thereby made sandboxed browsers unlaunchable.
|
|
245
|
+
|
|
246
|
+
### Potential modification
|
|
247
|
+
|
|
248
|
+
One modification to cope with this prohibition on the Vultr Cloud Compute host would be to configure Playwright and Puppeteer to launch `chromium` non-sandboxed. In both cases, launch arguments `'--no-sandbox'` and `'--disable-setuid-sandbox'` are available to specify this.
|
|
249
|
+
|
|
250
|
+
- For Playwright, `'--no-sandbox'` and `'--disable-setuid-sandbox'` would be added to the arguments of `browserOptionArgs.push` in the Testaro `run.js` file.
|
|
251
|
+
- For the `qualWeb` tool, this would be done in the Testaro `tests/qualweb.js` file, where the `qualWeb.start` method is called with an options argument. Its `args` array property would include `'--no-sandbox'` and `'--disable-setuid-sandbox'`.
|
|
252
|
+
- The `ibm` tool, too, could launch a Puppeteer `chromium` browser, if page content instead of a Playwright page were passed to the `accessibilityChecker.getCompliance` method, or if the implementation of the tool were changed in the future. For anticipation of such a case, the Testaro `aceconfig.js` file, of which a copy has been created at the root of the Kilotest project, would be modified. That file defines a `module.exports` object with a `puppeteerArgs` property, and, `--no-sandbox` and `--disable-setuid-sandbox` would be added to its array value.
|
|
253
|
+
|
|
254
|
+
### Current modification
|
|
255
|
+
|
|
256
|
+
However, non-sandboxed browsers are less secure than sandboxed ones, particularly when there is no restriction on who can use the service and what web pages they can test with it. Such restrictions are currently in place, but still permit testing of arbitrary web pages and may be relaxed in the future. Therefore, the potential modification described above would introduce nontrivial risk. An alternative solution was adopted instead. In it, the `chromium` configuration was left unchanged.
|
|
257
|
+
|
|
258
|
+
This solution required configuring the operating system of the Vultr Cloud Compute host to permit a sandboxed browser to be launched. This reconfiguration was performed with:
|
|
259
|
+
|
|
260
|
+
```bash
|
|
261
|
+
sudo sysctl -w kernel.unprivileged_userns_clone=1
|
|
262
|
+
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
|
|
263
|
+
sudo tee /etc/sysctl.d/99-kilotest-userns.conf >/dev/null <<'EOF'
|
|
264
|
+
kernel.unprivileged_userns_clone = 1
|
|
265
|
+
kernel.apparmor_restrict_unprivileged_userns = 0
|
|
266
|
+
EOF
|
|
267
|
+
sudo sysctl --system
|
|
268
|
+
```
|
package/aceconfig.js
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
/*
|
|
2
|
+
© 2021–2025 CVS Health and/or one of its affiliates. All rights reserved.
|
|
3
|
+
© 2025 Jonathan Robert Pool.
|
|
4
|
+
|
|
5
|
+
Licensed under the MIT License. See LICENSE file at the project root or
|
|
6
|
+
https://opensource.org/license/mit/ for details.
|
|
7
|
+
|
|
8
|
+
SPDX-License-Identifier: MIT
|
|
9
|
+
*/
|
|
10
|
+
|
|
11
|
+
/*
|
|
12
|
+
aceconfig
|
|
13
|
+
Configuration for the ibm tool.
|
|
14
|
+
*/
|
|
15
|
+
|
|
16
|
+
const os = require('os');
|
|
17
|
+
|
|
18
|
+
const tmpDir = os.tmpdir();
|
|
19
|
+
|
|
20
|
+
module.exports = {
|
|
21
|
+
reportLevels: [
|
|
22
|
+
'violation',
|
|
23
|
+
'recommendation'
|
|
24
|
+
],
|
|
25
|
+
cacheFolder: tmpDir,
|
|
26
|
+
outputFolder: tmpDir,
|
|
27
|
+
puppeteerArgs: ['--no-sandbox', '--disable-setuid-sandbox']
|
|
28
|
+
};
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html lang="en-US">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8">
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
6
|
+
<meta name="publisher" content="Jonathan Robert Pool">
|
|
7
|
+
<meta name="creator" content="Jonathan Robert Pool">
|
|
8
|
+
<meta name="keywords" content="report,accessibility,a11y">
|
|
9
|
+
<title>Record AI service 0 balance | Kilotest</title>
|
|
10
|
+
<link rel="icon" href="/favicon.ico">
|
|
11
|
+
<link rel="stylesheet" href="/style.css">
|
|
12
|
+
</head>
|
|
13
|
+
<body>
|
|
14
|
+
<main>
|
|
15
|
+
<h1><a href="/">Kilotest</a>: Record AI service 0 balance</h1>
|
|
16
|
+
<p>__oldBalance__ recorded AI service 0 balance.</p>
|
|
17
|
+
<p>The current AI service 0 balance is available on the <a href="https://platform.claude.com/settings/billing">Claude Platform Billing page</a>.</p>
|
|
18
|
+
<form action="/ai0BalanceForm.html">
|
|
19
|
+
<p><label>
|
|
20
|
+
Balance (in $):
|
|
21
|
+
<input type="number" size="5" min="0" max="99.99" step="0.01" name="newBalance" required>
|
|
22
|
+
</label></p>
|
|
23
|
+
<p><label>
|
|
24
|
+
Authorization code: <input size="3" minLength="3" maxlength="3" name="authCode" required>
|
|
25
|
+
</label></p>
|
|
26
|
+
<p><button type="submit">Submit</button></p>
|
|
27
|
+
</form>
|
|
28
|
+
</main>
|
|
29
|
+
</body>
|
|
30
|
+
</html>
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
/*
|
|
2
|
+
index.js
|
|
3
|
+
Serves a form for deleting superseded reports.
|
|
4
|
+
*/
|
|
5
|
+
|
|
6
|
+
// IMPORTS
|
|
7
|
+
|
|
8
|
+
const fs = require('fs/promises');
|
|
9
|
+
const path = require('path');
|
|
10
|
+
const {getJSON} = require('../util');
|
|
11
|
+
|
|
12
|
+
// CONSTANTS
|
|
13
|
+
|
|
14
|
+
const balancePath = path.join(__dirname, '../ai0Balance.json');
|
|
15
|
+
|
|
16
|
+
// FUNCTIONS
|
|
17
|
+
|
|
18
|
+
// Returns a form for recording the AI service 0 balance.
|
|
19
|
+
exports.answer = async (_, search) => {
|
|
20
|
+
const searchParams = new URLSearchParams(search);
|
|
21
|
+
const authCode = searchParams?.get('authCode');
|
|
22
|
+
const newBalanceString = searchParams?.get('newBalance');
|
|
23
|
+
let oldBalance;
|
|
24
|
+
// If the form displayed itself:
|
|
25
|
+
if (newBalanceString) {
|
|
26
|
+
// If the authorization code is valid:
|
|
27
|
+
if (authCode === process.env.AUTH_CODE) {
|
|
28
|
+
const newBalance = Number.parseFloat(newBalanceString);
|
|
29
|
+
// If the new balance is valid:
|
|
30
|
+
if (
|
|
31
|
+
typeof newBalance === 'number'
|
|
32
|
+
&& newBalance >= 0
|
|
33
|
+
&& newBalance < 100
|
|
34
|
+
&& Number.isInteger(100 * newBalance)
|
|
35
|
+
) {
|
|
36
|
+
const balanceData = {
|
|
37
|
+
balance: newBalance
|
|
38
|
+
}
|
|
39
|
+
// Record it.
|
|
40
|
+
await fs.writeFile(balancePath, getJSON(balanceData));
|
|
41
|
+
// Make it the old balance.
|
|
42
|
+
oldBalance = `$${balanceData.balance} is the`;
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
// Otherwise, i.e. if the authorization code is invalid:
|
|
46
|
+
else {
|
|
47
|
+
// Report the error.
|
|
48
|
+
return {
|
|
49
|
+
status: 'error',
|
|
50
|
+
error: 'Invalid authorization code'
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
// Otherwise, i.e. if the form has not been displayed by itself:
|
|
55
|
+
else {
|
|
56
|
+
// Get the current balance from the balance file.
|
|
57
|
+
try {
|
|
58
|
+
const balanceDataJSON = await fs.readFile(balancePath, 'utf8');
|
|
59
|
+
const balanceData = JSON.parse(balanceDataJSON);
|
|
60
|
+
oldBalance = `$${balanceData.balance} is the`;
|
|
61
|
+
}
|
|
62
|
+
// If the balance file does not exist or is invalid:
|
|
63
|
+
catch (error) {
|
|
64
|
+
oldBalance = 'There is no';
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
const query = {oldBalance};
|
|
68
|
+
// Get the order form template.
|
|
69
|
+
let answerPage = await fs.readFile(path.join(__dirname, 'index.html'), 'utf8');
|
|
70
|
+
// Replace its placeholders.
|
|
71
|
+
Object.keys(query).forEach(param => {
|
|
72
|
+
answerPage = answerPage.replace(new RegExp(`__${param}__`, 'g'), query[param]);
|
|
73
|
+
});
|
|
74
|
+
// Return the populated page.
|
|
75
|
+
return {
|
|
76
|
+
status: 'ok',
|
|
77
|
+
answerPage
|
|
78
|
+
};
|
|
79
|
+
};
|
package/alerts.js
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
/*
|
|
2
|
+
alerts.js
|
|
3
|
+
Sends alert emails to a Kilotest manager when required.
|
|
4
|
+
*/
|
|
5
|
+
|
|
6
|
+
// IMPORTS
|
|
7
|
+
|
|
8
|
+
const https = require('https');
|
|
9
|
+
|
|
10
|
+
// CONSTANTS
|
|
11
|
+
|
|
12
|
+
// Alert configuration.
|
|
13
|
+
const MANAGER_EMAIL = process.env.MANAGER_EMAIL;
|
|
14
|
+
const ALERT_API_HOST = process.env.ALERT_API_HOST;
|
|
15
|
+
const ALERT_API_PATH = process.env.ALERT_API_PATH;
|
|
16
|
+
const ALERT_API_KEY = process.env.ALERT_API_KEY;
|
|
17
|
+
const ALERT_FROM = process.env.ALERT_FROM;
|
|
18
|
+
|
|
19
|
+
// FUNCTIONS
|
|
20
|
+
|
|
21
|
+
// Sends an email alert to a manager.
|
|
22
|
+
exports.sendAlert = (subject, body) => new Promise((resolve, reject) => {
|
|
23
|
+
// If the alert configuration is complete:
|
|
24
|
+
if (MANAGER_EMAIL && ALERT_API_HOST && ALERT_API_PATH && ALERT_API_KEY && ALERT_FROM) {
|
|
25
|
+
const payload = JSON.stringify({
|
|
26
|
+
from: ALERT_FROM,
|
|
27
|
+
to: [MANAGER_EMAIL],
|
|
28
|
+
subject,
|
|
29
|
+
text: body
|
|
30
|
+
});
|
|
31
|
+
const req = https.request({
|
|
32
|
+
hostname: ALERT_API_HOST,
|
|
33
|
+
path: ALERT_API_PATH,
|
|
34
|
+
method: 'POST',
|
|
35
|
+
headers: {
|
|
36
|
+
'authorization': `Bearer ${ALERT_API_KEY}`,
|
|
37
|
+
'content-type': 'application/json',
|
|
38
|
+
'content-length': Buffer.byteLength(payload)
|
|
39
|
+
}
|
|
40
|
+
}, res => {
|
|
41
|
+
let data = '';
|
|
42
|
+
res.on('data', chunk => { data += chunk; });
|
|
43
|
+
res.on('end', () => {
|
|
44
|
+
if (res.statusCode >= 200 && res.statusCode < 300) {
|
|
45
|
+
console.log(`Alert sent (${subject})`);
|
|
46
|
+
}
|
|
47
|
+
else {
|
|
48
|
+
console.log(`ERROR: Alert API responded ${res.statusCode}: ${data}`);
|
|
49
|
+
}
|
|
50
|
+
resolve();
|
|
51
|
+
});
|
|
52
|
+
});
|
|
53
|
+
req.on('error', error => {
|
|
54
|
+
console.log(`ERROR: Alert API error: ${error.message}`);
|
|
55
|
+
resolve();
|
|
56
|
+
});
|
|
57
|
+
req.setTimeout(10000, () => {
|
|
58
|
+
req.destroy();
|
|
59
|
+
console.log('ERROR: Alert API timeout');
|
|
60
|
+
resolve();
|
|
61
|
+
});
|
|
62
|
+
req.write(payload);
|
|
63
|
+
req.end();
|
|
64
|
+
}
|
|
65
|
+
// Otherwise, i.e. if the alert configuration is incomplete:
|
|
66
|
+
else {
|
|
67
|
+
// Report this.
|
|
68
|
+
console.log(
|
|
69
|
+
`WARNING (${subject}): ${body}\nERROR: Alert configuration incomplete, so no alert sent`
|
|
70
|
+
);
|
|
71
|
+
resolve();
|
|
72
|
+
}
|
|
73
|
+
});
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
<!DOCTYPE html>
|
|
2
|
+
<html lang="en-US">
|
|
3
|
+
<head>
|
|
4
|
+
<meta charset="utf-8">
|
|
5
|
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
6
|
+
<meta name="publisher" content="Jonathan Robert Pool">
|
|
7
|
+
<meta name="creator" content="Jonathan Robert Pool">
|
|
8
|
+
<meta name="keywords" content="report,accessibility,a11y">
|
|
9
|
+
<title>HTML element __catalogIndex__ of __target__ for __issue__ issue | Kilotest</title>
|
|
10
|
+
<link rel="icon" href="/favicon.ico">
|
|
11
|
+
<link rel="stylesheet" href="/style.css">
|
|
12
|
+
</head>
|
|
13
|
+
<body>
|
|
14
|
+
<main>
|
|
15
|
+
<h1><a href="/">Kilotest</a>: Diagnoses of <q>__issue__</q> violation by HTML element __catalogIndex__ on __target__ page</h1>
|
|
16
|
+
<h2>Basics</h2>
|
|
17
|
+
<h3>About the __target__ page</h3>
|
|
18
|
+
<ul>
|
|
19
|
+
<li>URL: __urlLink__</li>
|
|
20
|
+
<li>__testInfo__</li>
|
|
21
|
+
</ul>
|
|
22
|
+
<h3>About HTML element __catalogIndex__</h3>
|
|
23
|
+
<ul>
|
|
24
|
+
<li>Tag name: <code>__tagName__</code></li>
|
|
25
|
+
<li>Text: __text__</li>
|
|
26
|
+
<li>Start tag: <code>__startTag__</code></li>
|
|
27
|
+
<li>XPath: <code>__pathID__</code></li>
|
|
28
|
+
<li>Bounding box: __box__</li>
|
|
29
|
+
</ul>
|
|
30
|
+
<ul class="nav">
|
|
31
|
+
__takeMeThere__
|
|
32
|
+
</ul>
|
|
33
|
+
<h3>About the <q>__issue__</q> issue</h3>
|
|
34
|
+
<ul>
|
|
35
|
+
<li>Why it matters: __why__</li>
|
|
36
|
+
<li>Priority: __priority__</li>
|
|
37
|
+
<li>Related WCAG standard: __wcag__</li>
|
|
38
|
+
</ul>
|
|
39
|
+
<h2>Diagnoses</h2>
|
|
40
|
+
<p>Here is how tools diagnose the <q>__issue__</q> issue for HTML element __catalogIndex__ of the __target__ page.</p>
|
|
41
|
+
<ol>
|
|
42
|
+
__diagnoses__
|
|
43
|
+
</ol>
|
|
44
|
+
</main>
|
|
45
|
+
</body>
|
|
46
|
+
</html>
|