artel-qa-lite 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +36 -0
- package/README.md +851 -0
- package/bin/artel.js +23 -0
- package/package.json +42 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
Artel QA Lite — Free Use License
|
|
2
|
+
Copyright (c) 2026 Metantel. All rights reserved.
|
|
3
|
+
|
|
4
|
+
1. GRANT. Metantel grants you a free, non-exclusive, non-transferable, revocable
|
|
5
|
+
licence to install and run the Artel QA Lite software (the "Software"), in
|
|
6
|
+
binary form as distributed by Metantel, to test web applications that you
|
|
7
|
+
own or are authorised to test.
|
|
8
|
+
|
|
9
|
+
2. WHAT YOU MAY NOT DO. You may not: (a) sell, sublicense, rent or lease the
|
|
10
|
+
Software; (b) reverse engineer, decompile or disassemble it except where the
|
|
11
|
+
law forbids this restriction; (c) remove or alter any notice in it;
|
|
12
|
+
(d) use it to test systems you are not authorised to test; (e) use it to
|
|
13
|
+
build a competing product.
|
|
14
|
+
|
|
15
|
+
3. YOUR DATA. The Software runs on your machine. It does not transmit your
|
|
16
|
+
source code, credentials or test results to Metantel. Inspectors run through
|
|
17
|
+
your own Claude Code subscription under Anthropic's terms; Metantel neither
|
|
18
|
+
sees nor bills that traffic.
|
|
19
|
+
|
|
20
|
+
4. NO WARRANTY. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
21
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
|
22
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.
|
|
23
|
+
A verdict the Software files is evidence for a person to weigh, not a
|
|
24
|
+
guarantee about your application.
|
|
25
|
+
|
|
26
|
+
5. LIMITATION OF LIABILITY. IN NO EVENT SHALL METANTEL BE LIABLE FOR ANY CLAIM,
|
|
27
|
+
DAMAGES OR OTHER LIABILITY ARISING FROM THE USE OF THE SOFTWARE.
|
|
28
|
+
|
|
29
|
+
6. TERMINATION. This licence ends automatically if you breach it. Metantel may
|
|
30
|
+
release future versions under different terms; this licence covers the
|
|
31
|
+
version it ships with.
|
|
32
|
+
|
|
33
|
+
7. THIRD-PARTY COMPONENTS. The Software includes open-source components under
|
|
34
|
+
their own licences; see THIRD-PARTY.md in the distribution.
|
|
35
|
+
|
|
36
|
+
Questions: hello@metantel.com
|
package/README.md
ADDED
|
@@ -0,0 +1,851 @@
|
|
|
1
|
+
<!-- Generated from README.md and docs/MANUAL.md by scripts/readme.mjs — edit those, not this. -->
|
|
2
|
+
# Artel QA Lite
|
|
3
|
+
|
|
4
|
+
**Agentic QA for web applications.** It reads your source code, tests the running application in a
|
|
5
|
+
real browser, and hands you a test plan and a report a QA lead can act on — with the evidence behind
|
|
6
|
+
every verdict.
|
|
7
|
+
|
|
8
|
+

|
|
9
|
+

|
|
10
|
+

|
|
11
|
+

|
|
12
|
+
|
|
13
|
+
<!-- install:start -->
|
|
14
|
+
## Install
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
npm install -g artel-qa-lite
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Then check the machine — it proves Claude Code and a browser actually work, and spends nothing:
|
|
21
|
+
|
|
22
|
+
```sh
|
|
23
|
+
artel doctor
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
This is Artel QA Lite installed as `artel`, the same product as
|
|
27
|
+
[metantel-cli](https://www.npmjs.com/package/metantel-cli): every `metantel` command on this page
|
|
28
|
+
works as `artel` — `artel plan --open`, `artel run --dry-run`.
|
|
29
|
+
<!-- install:end -->
|
|
30
|
+
|
|
31
|
+
### Requirements
|
|
32
|
+
|
|
33
|
+
| | |
|
|
34
|
+
|---|---|
|
|
35
|
+
| **Node.js** | 20 or newer — used only to install and launch; the product is a native binary |
|
|
36
|
+
| **Claude Code** | [installed](https://code.claude.com/docs/en/setup) and signed in. Inspectors run on your own subscription |
|
|
37
|
+
| **A Chromium** | Google Chrome, Chromium, or `npx playwright install chromium chromium-headless-shell` |
|
|
38
|
+
| **System** | macOS 12+ (Apple Silicon or Intel) · Linux x64 / arm64 (glibc) · Windows 10/11 x64 |
|
|
39
|
+
|
|
40
|
+
npm installs one prebuilt binary for your platform. There is no postinstall script and nothing is
|
|
41
|
+
downloaded at install or at run time.
|
|
42
|
+
|
|
43
|
+
### Other ways to install
|
|
44
|
+
|
|
45
|
+
```sh
|
|
46
|
+
npx metantel-cli doctor # try it without installing
|
|
47
|
+
npm install --save-dev metantel-cli # pin it per project, run with npx metantel
|
|
48
|
+
npm install -g metantel-cli@latest # upgrade
|
|
49
|
+
npm uninstall -g metantel-cli # remove (your .metantel/ folders stay)
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Quick start
|
|
55
|
+
|
|
56
|
+
From the root of your application's repository, with the application running:
|
|
57
|
+
|
|
58
|
+
```sh
|
|
59
|
+
metantel init # writes .metantel/config.json — set app.url and auth
|
|
60
|
+
metantel plan --open # the test plan, generated from your source — free
|
|
61
|
+
metantel run --dry-run # the whole pipeline, nothing dispatched — free
|
|
62
|
+
metantel run --budget 5 # a real run, bounded to $5 of your Claude subscription
|
|
63
|
+
metantel report --open # the report (also written when the run finishes)
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
A minimal `.metantel/config.json`:
|
|
67
|
+
|
|
68
|
+
```json
|
|
69
|
+
{
|
|
70
|
+
"app": { "url": "http://localhost:3000", "name": "My application" },
|
|
71
|
+
"codebase": { "path": "." },
|
|
72
|
+
"auth": { "at": "/login", "username": "qa@example.com", "password": "${QA_PASSWORD}" }
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
`${QA_PASSWORD}` is read from the environment, so the password never has to be in the file.
|
|
77
|
+
`metantel help` lists every command; `metantel <command> --help` gives its options.
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## What you get
|
|
82
|
+
|
|
83
|
+
**A test plan you did not write.** `metantel plan` reads the repository — routes, views, components,
|
|
84
|
+
forms, translation catalogues — and writes `test-plan.html`: one scenario per thing a user can do,
|
|
85
|
+
with the page, how to get there, preconditions, procedure and pass condition. On a 200-page HR
|
|
86
|
+
system that is 151 scenarios covering 1,027 controls, generated in under a minute.
|
|
87
|
+
|
|
88
|
+
**A walk before a word is spent.** Before any inspector starts, the harness itself opens every page
|
|
89
|
+
it found, signs in, records which controls are really there, what reveals the hidden ones (a tab, an
|
|
90
|
+
"Add" button, a `?tab=` link), which pages redirect and which are missing — and hands each inspector
|
|
91
|
+
those facts.
|
|
92
|
+
|
|
93
|
+
**Verdicts the model does not get to write.** An inspector says what it concludes and why; the
|
|
94
|
+
harness measured what actually happened — the screen before and after, every request that left the
|
|
95
|
+
browser, what the page announced. If the inspector says "saved" and nothing left the browser, the
|
|
96
|
+
verdict is **refuted**, and the report says so in that sentence.
|
|
97
|
+
|
|
98
|
+
**A report for a QA lead who has never seen us.** `report.html` in every run: findings with severity,
|
|
99
|
+
where, expected, observed, **steps to reproduce**, evidence, confidence and the source file;
|
|
100
|
+
coverage by area; every control not reached with the reason in plain words; one test case per
|
|
101
|
+
control. Every number in it is derived from the run's own case file.
|
|
102
|
+
|
|
103
|
+
```
|
|
104
|
+
Reached 22% Assessed 14% Passed 103 Findings 13 (12 high · 1 low)
|
|
105
|
+
|
|
106
|
+
F-001 · High · Number Of Employees: nothing on the page changed and nothing left the browser
|
|
107
|
+
Where /admin/viewOrganizationGeneralInformation — Number Of Employees (Input field)
|
|
108
|
+
Expected Accepts input, shows it, and keeps it when the form is saved
|
|
109
|
+
Observed Entering a value had no effect the run could see
|
|
110
|
+
Evidence Screen: nothing on the screen changed after the action
|
|
111
|
+
Network: nothing left the browser — no request was made
|
|
112
|
+
Confidence Medium — judged on what the screen showed and what left the browser
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
**Reached** means somebody stood in front of the control. **Assessed** means a pass or a fail with
|
|
116
|
+
evidence behind it. Both err low, and the report names the reason for every control not reached.
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## What it works on
|
|
121
|
+
|
|
122
|
+
Identification reads the source, with readers for Next.js, Nuxt, SvelteKit, React Router, Vue Router,
|
|
123
|
+
Angular, Django, Flask/FastAPI, Rails, Laravel, Symfony, Spring, ASP.NET, Express/Nest, Go routers
|
|
124
|
+
(chi, gin, echo, gorilla, `net/http`) and static HTML — and a floor under all of them: the running
|
|
125
|
+
application's own links. Translation catalogues in six formats, JSX in `.js`, Pug, Python form
|
|
126
|
+
classes and .NET display names are read so labels are the words a person sees.
|
|
127
|
+
|
|
128
|
+
The browser side is framework-neutral: it recognises a focusable `div` as the dropdown it is, names a
|
|
129
|
+
field by the label a person reads, clicks styled checkboxes through their label, operates custom
|
|
130
|
+
dropdowns and autocompletes (OXD, Ant, MUI, PrimeNG, Element, Fomantic…), and generates a valid file
|
|
131
|
+
of whatever type a file input accepts.
|
|
132
|
+
|
|
133
|
+
---
|
|
134
|
+
|
|
135
|
+
## Private by construction
|
|
136
|
+
|
|
137
|
+
It runs entirely on your machine. Your source, credentials and results never leave it. The binary has
|
|
138
|
+
no TLS library in it, so it cannot speak HTTPS to anyone; nothing calls a model we bill you for, and
|
|
139
|
+
nothing phones home. An inspector is a model reading pages it does not control, so it is fenced:
|
|
140
|
+
|
|
141
|
+
- **no shell, no file access, no web fetch** — only the browser tools this binary serves
|
|
142
|
+
- **only your application's origin** (plus any `safety.allowed_origins` you list); `file:`,
|
|
143
|
+
`javascript:` and foreign sites are refused
|
|
144
|
+
- **no page script** unless you allow it, and a verdict filed after script ran is marked
|
|
145
|
+
- **no uploads from your disk** — the harness generates test files matching each file input
|
|
146
|
+
- **your password is masked** in every brief, report and log
|
|
147
|
+
- the console binds `127.0.0.1` and refuses cross-site writes
|
|
148
|
+
|
|
149
|
+
**Cost, measured:** about $0.04–0.07 of your Claude subscription per control assessed. `plan`,
|
|
150
|
+
`identify`, `sweep` and `run --dry-run` are free.
|
|
151
|
+
|
|
152
|
+
---
|
|
153
|
+
|
|
154
|
+
## Known limits
|
|
155
|
+
|
|
156
|
+
- OAuth / SSO / 2FA sign-in is not supported yet; password and passkey (WebAuthn) are.
|
|
157
|
+
- Pages that need a record to exist (an employee, a repository) are reached once an inspector has
|
|
158
|
+
created one; the first walk reports them as needing that.
|
|
159
|
+
- Coverage numbers are honest, which means lower than a tool that counts "visited".
|
|
160
|
+
|
|
161
|
+
---
|
|
162
|
+
|
|
163
|
+
## The bigger picture
|
|
164
|
+
|
|
165
|
+
Artel QA Lite is the free, local edition of **Artel**, Metantel's QA platform. The full suite adds a
|
|
166
|
+
database channel as a third independent opinion, cross-run regression gates, per-role accounts, and
|
|
167
|
+
the Datalake Intelligence Platform behind it. Everything Lite files is an open, append-only case file
|
|
168
|
+
you keep. [metantel.com](https://metantel.com)
|
|
169
|
+
|
|
170
|
+
**Licence:** free to use on applications you are authorised to test — see the LICENSE file in this
|
|
171
|
+
package. **Security reports:** security@metantel.com. **Everything else:** hello@metantel.com.
|
|
172
|
+
|
|
173
|
+
The complete [user manual](#user-manual) covers configuration, every command, reading the report,
|
|
174
|
+
security, troubleshooting and FAQ.
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## User manual
|
|
179
|
+
|
|
180
|
+
*Version 0.1 · for the `metantel` command line*
|
|
181
|
+
|
|
182
|
+
Artel QA Lite reads your application's source code, drives the running application in a real
|
|
183
|
+
browser, and writes a report a QA lead can act on. This manual covers everything a user needs:
|
|
184
|
+
installing it, configuring a project, running it, reading what it produces, and what it will and
|
|
185
|
+
will not do on your machine.
|
|
186
|
+
|
|
187
|
+
---
|
|
188
|
+
|
|
189
|
+
### Contents
|
|
190
|
+
|
|
191
|
+
1. [How it works, in one page](#1-how-it-works-in-one-page)
|
|
192
|
+
2. [Requirements](#2-requirements)
|
|
193
|
+
3. [Install](#3-install)
|
|
194
|
+
4. [Set up a project](#4-set-up-a-project)
|
|
195
|
+
5. [The configuration file](#5-the-configuration-file)
|
|
196
|
+
6. [Your first session](#6-your-first-session)
|
|
197
|
+
7. [Commands](#7-commands)
|
|
198
|
+
8. [The test plan](#8-the-test-plan)
|
|
199
|
+
9. [Running](#9-running)
|
|
200
|
+
10. [The console](#10-the-console)
|
|
201
|
+
11. [The report](#11-the-report)
|
|
202
|
+
12. [Files a run leaves behind](#12-files-a-run-leaves-behind)
|
|
203
|
+
13. [Cost and how to bound it](#13-cost-and-how-to-bound-it)
|
|
204
|
+
14. [Security and privacy](#14-security-and-privacy)
|
|
205
|
+
15. [Environment variables](#15-environment-variables)
|
|
206
|
+
16. [Troubleshooting](#16-troubleshooting)
|
|
207
|
+
17. [Frequently asked questions](#17-frequently-asked-questions)
|
|
208
|
+
18. [Glossary](#18-glossary)
|
|
209
|
+
|
|
210
|
+
---
|
|
211
|
+
|
|
212
|
+
### 1. How it works, in one page
|
|
213
|
+
|
|
214
|
+
There are three stages, and they are deliberately separate.
|
|
215
|
+
|
|
216
|
+
**Identify.** The source is read — routes, views, components, forms, translation catalogues — and
|
|
217
|
+
every control a user could act on is listed with the page it renders on: every field, button, link
|
|
218
|
+
and piece of displayed text. This is the *denominator*: the number everything else is measured
|
|
219
|
+
against. It comes from your code, not from what a crawler happened to bump into.
|
|
220
|
+
|
|
221
|
+
**Walk, then inspect.** Before anything is billed, the harness itself opens every page it found,
|
|
222
|
+
signs in, and records which controls are really there, what reveals the hidden ones, which pages
|
|
223
|
+
redirect and which are missing. Then *inspectors* — Claude Code sessions on your own subscription,
|
|
224
|
+
working in a real browser — take the pages in bundles, exercise each control the way a user would,
|
|
225
|
+
and say what they conclude.
|
|
226
|
+
|
|
227
|
+
**Adjudicate.** An inspector never writes its own verdict. While it worked, the harness measured
|
|
228
|
+
what actually happened: the screen before and after, every request that left the browser, what the
|
|
229
|
+
page announced. The verdict is made from those measurements. If an inspector says "saved" and
|
|
230
|
+
nothing left the browser, the verdict is *refuted*, and the report says so in that sentence.
|
|
231
|
+
|
|
232
|
+
Everything lands in an append-only case file on your disk, and the report is derived from it.
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
### 2. Requirements
|
|
237
|
+
|
|
238
|
+
| | |
|
|
239
|
+
|---|---|
|
|
240
|
+
| **Operating system** | macOS 12+ (Apple Silicon or Intel), Linux x64/arm64 (glibc 2.35+, e.g. Ubuntu 22.04 / Debian 12), Windows 10/11 x64 |
|
|
241
|
+
| **Node.js** | 20 or newer — only to install and launch; the product is a native binary |
|
|
242
|
+
| **Claude Code** | installed and signed in (`claude --version` works). Inspectors run through it, on your subscription |
|
|
243
|
+
| **A Chromium** | Google Chrome, Chromium, or a Playwright browser (`npx playwright install chromium chromium-headless-shell`) |
|
|
244
|
+
| **The application** | running and reachable from this machine, with a test account |
|
|
245
|
+
| **The source** | a checkout of the application's repository on this machine |
|
|
246
|
+
|
|
247
|
+
Nothing is downloaded on install or at run time. `metantel doctor` tells you what is missing.
|
|
248
|
+
|
|
249
|
+
---
|
|
250
|
+
|
|
251
|
+
### 3. Install
|
|
252
|
+
|
|
253
|
+
```sh
|
|
254
|
+
npm install -g metantel-cli
|
|
255
|
+
metantel --version
|
|
256
|
+
metantel doctor
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
npm installs the prebuilt binary for your platform (`@metantel/cli-<platform>`, pulled in
|
|
260
|
+
automatically) and a small launcher that runs it. There is no postinstall script and nothing is
|
|
261
|
+
downloaded at install or at run time. The same product is also published as `artel-qa-lite`, which
|
|
262
|
+
installs the command as `artel`; every `metantel` command in this manual works as `artel`.
|
|
263
|
+
|
|
264
|
+
`doctor` starts Claude Code, launches a browser, drives a page and reads it back, and runs every
|
|
265
|
+
rule in the binary. It ends with `ok ready` or a list of what would stop a run.
|
|
266
|
+
|
|
267
|
+
**Other ways to install**
|
|
268
|
+
|
|
269
|
+
| | |
|
|
270
|
+
|---|---|
|
|
271
|
+
| try it once | `npx metantel-cli doctor` |
|
|
272
|
+
| pin it per project | `npm install --save-dev metantel-cli`, then `npx metantel run` |
|
|
273
|
+
| in CI | `npm install -g metantel-cli@0.1.0` — pin the version, and do not pass `--omit=optional` (the binary *is* the optional dependency) |
|
|
274
|
+
| upgrade | `npm install -g metantel-cli@latest`. Projects and runs are unaffected |
|
|
275
|
+
| uninstall | `npm uninstall -g metantel-cli`. Your `.metantel/` directories stay; delete them if you want the runs gone |
|
|
276
|
+
|
|
277
|
+
**Platforms built:** macOS (Apple Silicon and Intel), Linux x64 and arm64 (glibc — Debian, Ubuntu,
|
|
278
|
+
Fedora, RHEL; not Alpine/musl), Windows x64. On anything else the launcher says so by name.
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
### 4. Set up a project
|
|
283
|
+
|
|
284
|
+
From the root of the application's repository:
|
|
285
|
+
|
|
286
|
+
```sh
|
|
287
|
+
metantel init
|
|
288
|
+
```
|
|
289
|
+
|
|
290
|
+
This writes `.metantel/config.json` with a starter configuration and adds `.metantel/runs/` to
|
|
291
|
+
`.gitignore` (runs contain screenshots and case files; they are not source). Edit the file: at
|
|
292
|
+
minimum `app.url`, and `auth` if the application has a sign-in.
|
|
293
|
+
|
|
294
|
+
Keep `.metantel/config.json` in version control if your team shares the setup; keep the password
|
|
295
|
+
out of it with `${VAR}` — see the next section.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
299
|
+
### 5. The configuration file
|
|
300
|
+
|
|
301
|
+
`.metantel/config.json`, found from the current directory upwards. `metantel.json`, `artel.json`
|
|
302
|
+
and `.artel/config.json` are also recognised, and every command takes an explicit path.
|
|
303
|
+
|
|
304
|
+
```json
|
|
305
|
+
{
|
|
306
|
+
"app": {
|
|
307
|
+
"url": "http://localhost:3000",
|
|
308
|
+
"name": "My application",
|
|
309
|
+
"plugins": []
|
|
310
|
+
},
|
|
311
|
+
"codebase": {
|
|
312
|
+
"path": "."
|
|
313
|
+
},
|
|
314
|
+
"auth": {
|
|
315
|
+
"at": "/login",
|
|
316
|
+
"username": "qa@example.com",
|
|
317
|
+
"password": "${QA_PASSWORD}"
|
|
318
|
+
},
|
|
319
|
+
"safety": {
|
|
320
|
+
"allowed_origins": [],
|
|
321
|
+
"page_script": false,
|
|
322
|
+
"upload_dirs": []
|
|
323
|
+
},
|
|
324
|
+
"accounts": {
|
|
325
|
+
"use": { "default": "admin" },
|
|
326
|
+
"catalogue": [],
|
|
327
|
+
"states": []
|
|
328
|
+
}
|
|
329
|
+
}
|
|
330
|
+
```
|
|
331
|
+
|
|
332
|
+
#### `app`
|
|
333
|
+
|
|
334
|
+
| field | |
|
|
335
|
+
|---|---|
|
|
336
|
+
| `url` | The running application. Every route is resolved against it. It must be up before a run starts. |
|
|
337
|
+
| `name` | Used in reports and the console. |
|
|
338
|
+
| `plugins` | Optional. The extension slugs this deployment actually carries (for applications with an app store or plugin directory). Controls inside an extension you have not installed are then withheld with a reason instead of counted as failures. Leave it out and nothing is withheld — the safe direction. |
|
|
339
|
+
|
|
340
|
+
#### `codebase`
|
|
341
|
+
|
|
342
|
+
| field | |
|
|
343
|
+
|---|---|
|
|
344
|
+
| `path` | The application's source. Relative paths are relative to the project (the directory holding `.metantel/`). Without it there is no denominator. |
|
|
345
|
+
| `mapFile` | Optional override: a pre-built control map. Normally the map is built from `path`. |
|
|
346
|
+
|
|
347
|
+
#### `auth`
|
|
348
|
+
|
|
349
|
+
The harness signs in **itself**, once per inspector, before the inspector's first turn. Inspectors
|
|
350
|
+
are never given the password and are told not to invent one.
|
|
351
|
+
|
|
352
|
+
| field | |
|
|
353
|
+
|---|---|
|
|
354
|
+
| `at` | The sign-in page, as a path or a full URL. Optional — many applications redirect there anyway. |
|
|
355
|
+
| `username` / `email` | Two names for the same field; use whichever the form calls it. |
|
|
356
|
+
| `password` | The password. Write `"${QA_PASSWORD}"` to read it from the environment; a reference to an unset variable is an error at start, not a failed sign-in later. |
|
|
357
|
+
| `passkey` | `true` for WebAuthn sign-in; the harness installs a virtual authenticator. Or `{ "rp_id": "…", "transport": "internal" }`. |
|
|
358
|
+
| `note` | Free text shown to inspectors ("the dashboard loads after login; dismiss the tour"). |
|
|
359
|
+
|
|
360
|
+
Password and passkey sign-in are supported. OAuth, SSO and 2FA are not yet; an application behind
|
|
361
|
+
them cannot be reached past the front door.
|
|
362
|
+
|
|
363
|
+
#### `safety`
|
|
364
|
+
|
|
365
|
+
Every inspector is a model reading pages it does not control, and a page can carry instructions.
|
|
366
|
+
These fences are closed by default. Open one only when you know why.
|
|
367
|
+
|
|
368
|
+
| field | default | |
|
|
369
|
+
|---|---|---|
|
|
370
|
+
| `allowed_origins` | `[]` | Origins an inspector may open besides the application's own and its sign-in page — an identity provider, a payment page, a documentation site the app links to. `"https://idp.example.com"`. |
|
|
371
|
+
| `page_script` | `false` | Allows `browser_evaluate` and `browser_run_code_unsafe`. Off, the tools are not even offered. On, every verdict filed after script ran is marked. |
|
|
372
|
+
| `upload_dirs` | `[]` | Directories an inspector may upload real files from. Off, the harness generates a valid test file of the type each file input accepts. |
|
|
373
|
+
|
|
374
|
+
#### `accounts`
|
|
375
|
+
|
|
376
|
+
Optional. For applications where different controls need different roles or states. `catalogue`
|
|
377
|
+
lists the accounts your own provisioning publishes; `states` lists named states an account can be
|
|
378
|
+
in; `use.default` names the one to start with. When empty, the single `auth` account is used for
|
|
379
|
+
everything.
|
|
380
|
+
|
|
381
|
+
#### `.metantel/prefs.json`
|
|
382
|
+
|
|
383
|
+
Run preferences, written by `metantel init` defaults and by the console's form. The command line
|
|
384
|
+
overrides them.
|
|
385
|
+
|
|
386
|
+
```json
|
|
387
|
+
{ "headed": false, "lanes": 5, "max_waves": 10, "target": 70, "budget_usd": null, "model": null, "max_turns": 90 }
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
#### `.metantel/priority.json`
|
|
391
|
+
|
|
392
|
+
What to test first. Written from the console or by hand.
|
|
393
|
+
|
|
394
|
+
```json
|
|
395
|
+
{ "routes": ["/checkout", "/billing"], "controls": [], "tiers": ["write"], "note": "release blocker on checkout" }
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
---
|
|
399
|
+
|
|
400
|
+
### 6. Your first session
|
|
401
|
+
|
|
402
|
+
A sensible first hour, spending nothing until the last step.
|
|
403
|
+
|
|
404
|
+
```sh
|
|
405
|
+
metantel doctor # everything this machine needs, proven
|
|
406
|
+
metantel init # then edit .metantel/config.json
|
|
407
|
+
metantel identify # what the source declares: controls, routes, how many were placed
|
|
408
|
+
metantel plan --open # the test plan, as a page
|
|
409
|
+
metantel sweep # the harness walks the application: what is really on each page
|
|
410
|
+
metantel run --dry-run # the whole pipeline, briefs written, nothing dispatched
|
|
411
|
+
metantel run --budget 5 # the first real run, bounded to five dollars of your subscription
|
|
412
|
+
metantel report --open # the report (it was also written when the run finished)
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
Read `identify` before trusting anything else. It prints how many controls were found, how many
|
|
416
|
+
were placed on a page and how, and which routing conventions were recognised. A low placement
|
|
417
|
+
percentage means the report's denominator is a sliver of the application — fix that first (usually
|
|
418
|
+
by pointing `codebase.path` at the right directory).
|
|
419
|
+
|
|
420
|
+
---
|
|
421
|
+
|
|
422
|
+
### 7. Commands
|
|
423
|
+
|
|
424
|
+
Every command accepts `--help` (`metantel run --help`), and `metantel help` lists them all. Flags
|
|
425
|
+
take their value either way — `--budget 5` or `--budget=5`. Paths in `[config]` are optional; the
|
|
426
|
+
config is otherwise found from the current directory upwards. A mistyped command is answered with
|
|
427
|
+
the one that was probably meant.
|
|
428
|
+
|
|
429
|
+
Exit codes: `0` the command did what was asked — a run that files findings still exits `0`, the
|
|
430
|
+
findings are in the report; `1` a `selftest` rule failed; `2` it could not do what was asked — a
|
|
431
|
+
missing config, an unknown flag, a machine `doctor` says is not ready.
|
|
432
|
+
|
|
433
|
+
#### `metantel init`
|
|
434
|
+
Writes a starter `.metantel/config.json` and adds `.metantel/runs/` to `.gitignore`. Refuses to
|
|
435
|
+
overwrite an existing file.
|
|
436
|
+
|
|
437
|
+
#### `metantel doctor`
|
|
438
|
+
Checks Claude Code, the headless browser, the windowed browser (two windows at once), every rule in
|
|
439
|
+
the binary, and where runs will be written. Exit code is non-zero if anything would stop a run.
|
|
440
|
+
|
|
441
|
+
#### `metantel selftest`
|
|
442
|
+
Runs every rule in the binary offline, in about a second. No browser, no network.
|
|
443
|
+
|
|
444
|
+
#### `metantel version` · `metantel --version`
|
|
445
|
+
`--version` prints one line (`metantel 0.1.0`) for scripts; `version` adds the platform the binary
|
|
446
|
+
was built for.
|
|
447
|
+
|
|
448
|
+
#### `metantel where [config]`
|
|
449
|
+
Prints where this project's runs, caches and preferences live, before anything is written.
|
|
450
|
+
|
|
451
|
+
#### `metantel identify [path]` — `--samples --json --out FILE`
|
|
452
|
+
Identification only: no application, browser or model. Prints what the repository declares, how
|
|
453
|
+
many controls were named, placed and catalogued, and why not. `--samples` prints unnamed controls
|
|
454
|
+
as found; `--json` prints machine-readable counts and the route templates; `--out` writes the
|
|
455
|
+
catalogue.
|
|
456
|
+
|
|
457
|
+
#### `metantel clusters [path]` — `--json`
|
|
458
|
+
The solution-cluster catalogue: the general rules and the per-stack rules identification applies,
|
|
459
|
+
what each solves, and which apply to this repository.
|
|
460
|
+
|
|
461
|
+
#### `metantel plan [config]` — `--open --config PATH`
|
|
462
|
+
Writes `.metantel/test-plan.html` and `test-plan.md`: one scenario per goal (page, how to get
|
|
463
|
+
there, preconditions, procedure, pass condition), plus the control checks outside any scenario.
|
|
464
|
+
Scenarios come from `metantel goals`; without goals the plan lists control checks only.
|
|
465
|
+
|
|
466
|
+
#### `metantel goals [config]` — `--model M --route R --offline --refresh --fill --link --relink`
|
|
467
|
+
Reads the codebase into *goals*: the things a user must be able to do on each page, with
|
|
468
|
+
preconditions. This is the one command that uses a stronger model by default (Sonnet), once per
|
|
469
|
+
codebase, cached against the source. `--refresh` re-reads everything; `--fill` re-reads only routes
|
|
470
|
+
that got nothing; `--link` works out which goals must be done before which.
|
|
471
|
+
|
|
472
|
+
#### `metantel sweep [config]` — `--workers N --headed --json --route R`
|
|
473
|
+
The harness walks every catalogued page without a model: which controls are on their page, what
|
|
474
|
+
reveals the hidden ones, which pages redirect or are missing, which framework and design system drew
|
|
475
|
+
the page. Every run does this first; run it alone to see the facts. `--route` limits it to routes
|
|
476
|
+
containing a string.
|
|
477
|
+
|
|
478
|
+
#### `metantel run [config]`
|
|
479
|
+
The main command. See [Running](#9-running).
|
|
480
|
+
|
|
481
|
+
| flag | default | |
|
|
482
|
+
|---|---|---|
|
|
483
|
+
| `--lanes N` | 5 | inspectors working at once |
|
|
484
|
+
| `--budget USD` | none | stop when this much of your subscription is spent |
|
|
485
|
+
| `--target P` | 70 | stop when this % of controls is assessed |
|
|
486
|
+
| `--max-waves N` | 10 | ceiling on waves |
|
|
487
|
+
| `--model M` | Haiku | the inspector model |
|
|
488
|
+
| `--escalate-model M` | Sonnet | the model for a job's last attempt (`none` to disable) |
|
|
489
|
+
| `--max-attempts N` | 3 | sessions a job may take before it is parked with its reason |
|
|
490
|
+
| `--session-size N` | 24 | controls one inspector session takes on, a page at a time |
|
|
491
|
+
| `--max-turns N` | 90 | how long one inspector session may work |
|
|
492
|
+
| `--headed` | off | watch in a browser window |
|
|
493
|
+
| `--headless` | on | force headless whatever `prefs.json` says |
|
|
494
|
+
| `--dry-run` | off | identify, walk, write the briefs, dispatch nothing |
|
|
495
|
+
| `--auto` | off | never stop to ask between waves (required without a terminal) |
|
|
496
|
+
| `--no-sweep` / `--resweep` | | skip the harness's walk, or force a fresh one |
|
|
497
|
+
|
|
498
|
+
#### `metantel ui [config]` — `--port N --no-open`
|
|
499
|
+
The local console. Binds `127.0.0.1` only. See [The console](#10-the-console).
|
|
500
|
+
|
|
501
|
+
#### `metantel report [config]` — `--run ID --open --config PATH`
|
|
502
|
+
Writes `report.html` and `report.md` into a run (the latest by default). Also done automatically
|
|
503
|
+
when a run finishes.
|
|
504
|
+
|
|
505
|
+
#### `metantel mcp --lane ID --run ID --config PATH --store PATH`
|
|
506
|
+
Serves the browser tool surface over MCP on stdio. Claude Code spawns this for each inspector; you
|
|
507
|
+
can also attach it to your own agent.
|
|
508
|
+
|
|
509
|
+
---
|
|
510
|
+
|
|
511
|
+
### 8. The test plan
|
|
512
|
+
|
|
513
|
+
`metantel plan` produces a numbered plan before anything is spent:
|
|
514
|
+
|
|
515
|
+
```
|
|
516
|
+
TP-002 · Register a new OpenID Connect login provider
|
|
517
|
+
Page /admin/addAuthProvider
|
|
518
|
+
How to get there Admin > Configuration > OpenID Connect > Add
|
|
519
|
+
Preconditions Logged in as an Admin user; a provider name not already in use
|
|
520
|
+
Procedure Fill in Name, Url, Client Id and Client Secret with valid values and submit
|
|
521
|
+
Pass when The save completes without error and the new provider is listed afterward
|
|
522
|
+
Controls 4
|
|
523
|
+
```
|
|
524
|
+
|
|
525
|
+
Scenarios are grouped by area of the application. Below them, *control checks outside a scenario*
|
|
526
|
+
lists the controls no scenario names — they are still tested one by one in a run.
|
|
527
|
+
|
|
528
|
+
The plan is what a run will attempt. The report says what it found. Reviewing the plan with the
|
|
529
|
+
team before the first run is the cheapest correction you can make.
|
|
530
|
+
|
|
531
|
+
---
|
|
532
|
+
|
|
533
|
+
### 9. Running
|
|
534
|
+
|
|
535
|
+
#### What happens
|
|
536
|
+
|
|
537
|
+
1. **Identify** — from cache when the source has not changed.
|
|
538
|
+
2. **Walk** — the harness opens every page (reused if walked in the last 12 hours; `--resweep`
|
|
539
|
+
forces it).
|
|
540
|
+
3. **Board** — controls are cut into jobs; goals decide what must be done before what.
|
|
541
|
+
4. **Waves** — inspector sessions are dispatched in parallel, each taking a bundle of jobs on one or
|
|
542
|
+
two pages. A session that finishes takes the next bundle. A wave ends when nothing is left to
|
|
543
|
+
hand out.
|
|
544
|
+
5. **Between waves** — pages that needed a record to exist (an employee, a repository) are walked
|
|
545
|
+
again once an inspector has created one and they come onto the board.
|
|
546
|
+
6. **Stop** — at the target, the budget, the wave ceiling, or when a wave converts nothing.
|
|
547
|
+
7. **Report** — written into the run directory.
|
|
548
|
+
|
|
549
|
+
#### What you see
|
|
550
|
+
|
|
551
|
+
```
|
|
552
|
+
Artel QA Lite run r-20260913163000
|
|
553
|
+
· config …/.metantel/config.json (found here)
|
|
554
|
+
· app http://34.62.61.254/web/index.php
|
|
555
|
+
· map 1297 controls, from cache
|
|
556
|
+
· render 190 route(s) declared (symfony 190), 182 resolved to a view
|
|
557
|
+
· routes 96.5% of 1297 controls placed on a page
|
|
558
|
+
· catalogue 1031 controls on 170 routes
|
|
559
|
+
· sweep walked the application — 223 of 1031 controls seen on their page (21.6%)
|
|
560
|
+
· goals 227 across 94 route(s)
|
|
561
|
+
board: 290 jobs, 290 open
|
|
562
|
+
inspector-0: session 1 — 3 job(s), 27 control(s) on /admin/viewOrganizationGeneralInformation
|
|
563
|
+
…
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
Each session prints what it settled and what it cost. Press Ctrl-C at any time: everything filed so
|
|
567
|
+
far is on disk, and the report can be written from it with `metantel report`.
|
|
568
|
+
|
|
569
|
+
#### Re-running
|
|
570
|
+
|
|
571
|
+
Each run is its own case file with its own verdicts. What carries over between runs is knowledge,
|
|
572
|
+
not results: where controls were found and what revealed them (`ui-map.json`), the walk of the
|
|
573
|
+
application (reused for 12 hours), the goals (until the source changes), and the identification
|
|
574
|
+
cache. A second run is therefore faster and reaches more, and its numbers are its own. Use
|
|
575
|
+
`--budget` and `--target` to size each run; `metantel report --run <id>` reads any past one.
|
|
576
|
+
|
|
577
|
+
---
|
|
578
|
+
|
|
579
|
+
### 10. The console
|
|
580
|
+
|
|
581
|
+
```sh
|
|
582
|
+
metantel ui
|
|
583
|
+
```
|
|
584
|
+
|
|
585
|
+
Opens `http://127.0.0.1:<port>` in your browser: the stages of the current run animated from real
|
|
586
|
+
events, the board, findings as they are filed, coverage, and controls to start a run with the
|
|
587
|
+
preferences shown, set a priority, or stop a run the console started.
|
|
588
|
+
|
|
589
|
+
The console is local only. It binds the loopback address, refuses requests from other origins and
|
|
590
|
+
hosts, and accepts writes only from its own page. It never computes coverage itself; it shows the
|
|
591
|
+
numbers the run published.
|
|
592
|
+
|
|
593
|
+
---
|
|
594
|
+
|
|
595
|
+
### 11. The report
|
|
596
|
+
|
|
597
|
+
Every run writes `report.html` (self-contained, prints well) and `report.md` (paste into a ticket).
|
|
598
|
+
|
|
599
|
+
#### At a glance
|
|
600
|
+
|
|
601
|
+
| number | meaning |
|
|
602
|
+
|---|---|
|
|
603
|
+
| **Controls** | every field, button, link and label the source declares — the denominator |
|
|
604
|
+
| **Reached** | somebody stood in front of it: an inspector, or the pre-run walk |
|
|
605
|
+
| **Assessed** | a pass or a fail with evidence behind it |
|
|
606
|
+
| **Passed** | behaved as its label says |
|
|
607
|
+
| **Findings** | defects, with severity |
|
|
608
|
+
|
|
609
|
+
Both percentages err low. A control the harness could not reach counts against it.
|
|
610
|
+
|
|
611
|
+
#### Findings
|
|
612
|
+
|
|
613
|
+
Each finding carries: an id (`F-001`), severity (High / Medium / Low), a title in plain words,
|
|
614
|
+
**Where** (page and control), **Expected**, **Observed**, **To reproduce** (the steps the inspector
|
|
615
|
+
took, in a tester's words), **Evidence** (what the screen showed, what left the browser, screenshots
|
|
616
|
+
when taken), **Confidence** (and which channels it rests on), the inspector's note, and the source
|
|
617
|
+
file the control comes from.
|
|
618
|
+
|
|
619
|
+
Severity: High for a field or button that does not do its job or a save the server accepted
|
|
620
|
+
without changing anything; Medium for a link or displayed text; Low for a control that only partly
|
|
621
|
+
works.
|
|
622
|
+
|
|
623
|
+
#### Results, per control
|
|
624
|
+
|
|
625
|
+
Every internal verdict maps to one word a QA lead reads:
|
|
626
|
+
|
|
627
|
+
| result | meaning |
|
|
628
|
+
|---|---|
|
|
629
|
+
| **Pass** | it did what its label says, and the network agrees |
|
|
630
|
+
| **Pass (screen only)** | the screen moved and the network was silent — fine for a client-side control, a flag for a save |
|
|
631
|
+
| **Fail** | it did not — the screen said otherwise, the server disagreed, or nothing left the browser |
|
|
632
|
+
| **Inconclusive** | the evidence does not settle it |
|
|
633
|
+
| **Blocked** | the inspector could not put the application into the state the control needs |
|
|
634
|
+
| **Not shown** | the inspector opened the page and did not find it |
|
|
635
|
+
| **Not tested** | nobody exercised it |
|
|
636
|
+
| **Seen, not tested** | the pre-run walk saw it on its page; no inspector reached it |
|
|
637
|
+
| **Not a control** | declared in the source, not a control on the page — excluded from every count |
|
|
638
|
+
|
|
639
|
+
#### Not reached, and why
|
|
640
|
+
|
|
641
|
+
A table of every control nobody reached, grouped by reason: the page needs a record this account
|
|
642
|
+
does not have; the page is not served by this deployment (404); the page sends this account back to
|
|
643
|
+
sign-in; hidden until a state; not on the page in its default state; the run stopped first. This is
|
|
644
|
+
the list to read when a number looks low.
|
|
645
|
+
|
|
646
|
+
#### Test cases
|
|
647
|
+
|
|
648
|
+
One case per control, grouped by page, with its result and the inspector's steps. Expand a page to
|
|
649
|
+
read them.
|
|
650
|
+
|
|
651
|
+
#### Method
|
|
652
|
+
|
|
653
|
+
How the run was conducted, what was detected at runtime (framework, design system), which account
|
|
654
|
+
was used, and whether a database was attached.
|
|
655
|
+
|
|
656
|
+
---
|
|
657
|
+
|
|
658
|
+
### 12. Files a run leaves behind
|
|
659
|
+
|
|
660
|
+
Everything is under the project's `.metantel/`. Nothing is written anywhere else in your repository.
|
|
661
|
+
|
|
662
|
+
```
|
|
663
|
+
.metantel/
|
|
664
|
+
config.json your configuration
|
|
665
|
+
prefs.json run preferences
|
|
666
|
+
priority.json what to test first
|
|
667
|
+
goals.json the goals read from the source (cached)
|
|
668
|
+
sweep.json the last walk of the application
|
|
669
|
+
ui-map.json where controls were found, across runs — makes the next run cheaper
|
|
670
|
+
test-plan.html/.md the plan
|
|
671
|
+
runs/<run-id>/
|
|
672
|
+
report.html/.md the report
|
|
673
|
+
summary.json numbers, cost, why it stopped
|
|
674
|
+
triage.json ranked findings; accusations with no evidence held back
|
|
675
|
+
catalog.json the denominator this run used
|
|
676
|
+
board.json the jobs
|
|
677
|
+
ledger.jsonl one row per verdict, append-only — the record of truth
|
|
678
|
+
actions.jsonl every browser action, in a tester's words
|
|
679
|
+
calls.jsonl every tool call, attributed to a lane and a control
|
|
680
|
+
progress.jsonl sessions, timings, spend
|
|
681
|
+
brief-*.md what each inspector session was told
|
|
682
|
+
evidence/ screenshots
|
|
683
|
+
uploads/ the test files the harness generated for file inputs
|
|
684
|
+
```
|
|
685
|
+
|
|
686
|
+
`ledger.jsonl` is append-only and the report is derived from it. If a report and a ledger ever
|
|
687
|
+
disagree, the ledger is right.
|
|
688
|
+
|
|
689
|
+
---
|
|
690
|
+
|
|
691
|
+
### 13. Cost and how to bound it
|
|
692
|
+
|
|
693
|
+
Inspectors run through **your** Claude Code subscription. Measured on real applications: about
|
|
694
|
+
$0.04–0.07 per control assessed with the default model. Identification, the plan, the walk and
|
|
695
|
+
`--dry-run` cost nothing.
|
|
696
|
+
|
|
697
|
+
- `--budget USD` stops the run at that spend. The console shows spend as it happens.
|
|
698
|
+
- `--target P` stops once that share of controls is assessed.
|
|
699
|
+
- `--lanes` trades speed for concurrent spend; five is a sensible default.
|
|
700
|
+
- `metantel goals` uses Sonnet once per codebase (about $9 on a 200-page application) and is cached
|
|
701
|
+
until the source changes.
|
|
702
|
+
- Every summary line in the console and in `summary.json` says what was spent and which model
|
|
703
|
+
answered.
|
|
704
|
+
|
|
705
|
+
---
|
|
706
|
+
|
|
707
|
+
### 14. Security and privacy
|
|
708
|
+
|
|
709
|
+
**What leaves your machine.** Only the prompts and tool results exchanged with Claude through Claude
|
|
710
|
+
Code, under your subscription and Anthropic's terms. Metantel receives nothing: no source, no
|
|
711
|
+
credentials, no results, no telemetry. The binary contains no TLS library and cannot make an HTTPS
|
|
712
|
+
connection to anyone.
|
|
713
|
+
|
|
714
|
+
**What an inspector can do.** Only what the browser tools this binary serves allow:
|
|
715
|
+
|
|
716
|
+
- **No shell, no file access, no web fetch.** Claude Code's built-in tools are removed for
|
|
717
|
+
inspectors.
|
|
718
|
+
- **Only your application.** An inspector may open the application's own origin, its sign-in page,
|
|
719
|
+
and `safety.allowed_origins`. `file:`, `javascript:`, `data:`, `chrome:` and foreign sites are
|
|
720
|
+
refused. A page that says "open this link" is not an instruction it can follow.
|
|
721
|
+
- **No page script** unless `safety.page_script` is on; a verdict filed after script ran is marked.
|
|
722
|
+
- **No files from your machine.** File inputs receive a valid generated test file
|
|
723
|
+
(`artel-test-file.<ext>`) of the type the input accepts. `safety.upload_dirs` opens a directory
|
|
724
|
+
deliberately.
|
|
725
|
+
- **Credentials.** The harness signs in; inspectors are never given the password and are told not
|
|
726
|
+
to invent one. The configured password is masked in every brief, report, plan and log, even when
|
|
727
|
+
the application's own repository quotes it (demo credentials in a CI file, say).
|
|
728
|
+
|
|
729
|
+
**The console** binds `127.0.0.1`, refuses cross-origin requests and non-loopback hosts, and takes
|
|
730
|
+
writes only from its own page.
|
|
731
|
+
|
|
732
|
+
**What the run does to your application.** Inspectors create, edit and press things — that is the
|
|
733
|
+
job. Point them at a staging or test deployment with a test account, not at production. Installer
|
|
734
|
+
and upgrade routes are withheld from inspectors automatically.
|
|
735
|
+
|
|
736
|
+
**Reporting a security issue:** security@metantel.com.
|
|
737
|
+
|
|
738
|
+
---
|
|
739
|
+
|
|
740
|
+
### 15. Environment variables
|
|
741
|
+
|
|
742
|
+
| variable | |
|
|
743
|
+
|---|---|
|
|
744
|
+
| `METANTEL_CHROMIUM` | path to a Chrome/Chromium binary to use instead of searching |
|
|
745
|
+
| `PLAYWRIGHT_BROWSERS_PATH` | a Playwright browser directory to search |
|
|
746
|
+
| `METANTEL_CLAUDE_BIN` | the Claude Code executable, if not `claude` on your PATH |
|
|
747
|
+
| `METANTEL_BIN` | (launcher) run a specific `metantel` binary — for a build of your own |
|
|
748
|
+
| `${NAME}` in config | any of `app.url`, `auth.username`, `auth.email`, `auth.password` may reference the environment |
|
|
749
|
+
|
|
750
|
+
---
|
|
751
|
+
|
|
752
|
+
### 16. Troubleshooting
|
|
753
|
+
|
|
754
|
+
**`metantel: the binary for <platform> … is not installed`.** npm skipped the optional dependency
|
|
755
|
+
that carries the binary — usually because the install ran with `--omit=optional` / `--no-optional`,
|
|
756
|
+
or `node_modules` was copied from another operating system (a Mac's into a Linux container).
|
|
757
|
+
Reinstall on the machine that will run it: `npm install -g metantel-cli`.
|
|
758
|
+
|
|
759
|
+
**`metantel: there is no build for <platform>`** or **`this Linux uses musl libc`.** Builds exist for
|
|
760
|
+
macOS, glibc Linux (x64, arm64) and Windows x64. In a container, use a glibc base image such as
|
|
761
|
+
`node:20-bookworm-slim` rather than Alpine.
|
|
762
|
+
|
|
763
|
+
**`metantel doctor` says `x claude`.** Claude Code is not on your PATH or not signed in. Install it,
|
|
764
|
+
run `claude` once interactively to sign in, or set `METANTEL_CLAUDE_BIN`.
|
|
765
|
+
|
|
766
|
+
**`x browser … no Chromium found`.** Install one: `npx playwright install chromium
|
|
767
|
+
chromium-headless-shell`, or point `METANTEL_CHROMIUM` at Chrome. Nothing is downloaded for you.
|
|
768
|
+
|
|
769
|
+
**`nothing to test` / very low placement in `identify`.** `codebase.path` points at the wrong
|
|
770
|
+
directory, or the application uses a routing convention identification does not read yet. `metantel
|
|
771
|
+
identify` names the conventions it recognised; `metantel sweep` and a run still work from the
|
|
772
|
+
application's own links, with a smaller denominator.
|
|
773
|
+
|
|
774
|
+
**Every page comes back `signed-out` or `blocked`.** The front door did not open. Check `auth.at`,
|
|
775
|
+
the username field name (`username` vs `email`), and that the test account works in a normal
|
|
776
|
+
browser. `metantel sweep --route login --headed` shows what the harness sees.
|
|
777
|
+
|
|
778
|
+
**A run stopped after one wave.** Without a terminal (CI, the console) pass `--auto`; otherwise the
|
|
779
|
+
question "buy another wave?" reads end-of-file and stops. The console always passes it.
|
|
780
|
+
|
|
781
|
+
**`the application did not serve a single page to the sweep`.** The application is down or
|
|
782
|
+
unreachable from this machine. Nothing was saved; start the application and rerun.
|
|
783
|
+
|
|
784
|
+
**Rate limits.** The run reads Claude Code's own rate-limit events and waits; the console shows it.
|
|
785
|
+
Long stalls on a laptop are usually the machine sleeping — the run keeps a wake lock while on mains
|
|
786
|
+
power.
|
|
787
|
+
|
|
788
|
+
**A finding looks wrong.** Open the control's row in `ledger.jsonl`: every verdict carries the
|
|
789
|
+
evidence it rests on, what the inspector claimed, and whether the harness overruled it. The report
|
|
790
|
+
holds back accusations with nothing behind them and counts them separately.
|
|
791
|
+
|
|
792
|
+
**Windows.** Use PowerShell or Git Bash. Paths in `config.json` may use forward slashes.
|
|
793
|
+
|
|
794
|
+
---
|
|
795
|
+
|
|
796
|
+
### 17. Frequently asked questions
|
|
797
|
+
|
|
798
|
+
**Does it need access to my database?** No. Lite verifies on the screen and the network; a write is
|
|
799
|
+
proven by the server accepting it and the application reading it back. A database channel is part of
|
|
800
|
+
the full Artel suite.
|
|
801
|
+
|
|
802
|
+
**Can it test an application behind Google / Okta / SSO?** Not yet. Password and passkey sign-in are
|
|
803
|
+
supported. If an internal identity provider is on a separate origin, add it to
|
|
804
|
+
`safety.allowed_origins` — but the sign-in itself must be password or passkey.
|
|
805
|
+
|
|
806
|
+
**Will it make changes to my application's data?** Yes, in the test deployment you point it at:
|
|
807
|
+
creating, editing and saving things is how a control is tested. Use a disposable environment and
|
|
808
|
+
account.
|
|
809
|
+
|
|
810
|
+
**Can I run it in CI?** Yes: `metantel run --auto --budget 10 --dry-run` in a pipeline is free and
|
|
811
|
+
proves the setup; drop `--dry-run` to run for real. Exit codes are non-zero when the run could not
|
|
812
|
+
start. Linux x64/arm64 binaries are published for this; use a glibc image, pin the version, and keep
|
|
813
|
+
optional dependencies on.
|
|
814
|
+
|
|
815
|
+
**Does it support my framework?** Identification reads the source of most web stacks (see *What it
|
|
816
|
+
works on* in the README). Where a convention is unknown, the application's own links are used. The browser side is
|
|
817
|
+
framework-neutral.
|
|
818
|
+
|
|
819
|
+
**Why is coverage lower than tool X reports?** Because *reached* and *assessed* here mean what they
|
|
820
|
+
say. A control counts only when someone stood in front of it, and only counts as assessed with a
|
|
821
|
+
pass or fail and evidence. Visiting a page is not testing it.
|
|
822
|
+
|
|
823
|
+
**Where does the model run?** Through Claude Code, on your subscription. Choose the model with
|
|
824
|
+
`--model`; the default is the cheapest capable one.
|
|
825
|
+
|
|
826
|
+
**Can I see what an inspector was told?** Yes — `brief-*.md` in the run directory, one per session,
|
|
827
|
+
with the configured password masked.
|
|
828
|
+
|
|
829
|
+
---
|
|
830
|
+
|
|
831
|
+
### 18. Glossary
|
|
832
|
+
|
|
833
|
+
| term | |
|
|
834
|
+
|---|---|
|
|
835
|
+
| **control** | anything a user can act on or must be able to see: a field, button, link, checkbox, dropdown, or displayed text |
|
|
836
|
+
| **catalogue** | the list of controls with the page each renders on — the denominator |
|
|
837
|
+
| **route / page** | a URL the application serves; `[name]` marks a parameter (`/orders/[id]`) |
|
|
838
|
+
| **walk (sweep)** | the harness opening every page without a model, before a run |
|
|
839
|
+
| **goal** | something a user must be able to do on a page, with its preconditions, read from the source |
|
|
840
|
+
| **brief** | what an inspector session is told: the pages, the controls, the facts from the walk |
|
|
841
|
+
| **inspector / lane** | one Claude Code session driving the browser through the harness's tools |
|
|
842
|
+
| **wave** | one round of dispatching sessions until nothing is left to hand out |
|
|
843
|
+
| **verdict** | the harness's ruling on one control, from measured evidence; the inspector's claim is recorded beside it |
|
|
844
|
+
| **assessed** | a pass or fail with evidence; not "visited", not "attempted" |
|
|
845
|
+
| **reached** | an inspector or the walk stood in front of the control |
|
|
846
|
+
| **finding** | a verdict that says the application is broken, ranked by severity |
|
|
847
|
+
| **case file** | the run directory: ledger, evidence, briefs, report |
|
|
848
|
+
|
|
849
|
+
---
|
|
850
|
+
|
|
851
|
+
*Artel QA Lite is the free, local edition of Artel by Metantel. hello@metantel.com*
|
package/bin/artel.js
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// artel — Artel QA Lite under its product name. Hands everything to the metantel-cli launcher, so
|
|
3
|
+
// there is exactly one place that decides which binary runs.
|
|
4
|
+
import { spawn } from 'node:child_process';
|
|
5
|
+
import { createRequire } from 'node:module';
|
|
6
|
+
|
|
7
|
+
const require = createRequire(import.meta.url);
|
|
8
|
+
let launcher;
|
|
9
|
+
try {
|
|
10
|
+
launcher = require.resolve('metantel-cli/bin/metantel.js');
|
|
11
|
+
} catch {
|
|
12
|
+
console.error('artel: metantel-cli is not installed alongside artel-qa-lite. Reinstall: npm i -g artel-qa-lite');
|
|
13
|
+
process.exit(2);
|
|
14
|
+
}
|
|
15
|
+
const child = spawn(process.execPath, [launcher, ...process.argv.slice(2)], { stdio: 'inherit' });
|
|
16
|
+
for (const sig of ['SIGINT', 'SIGTERM', 'SIGHUP']) {
|
|
17
|
+
process.on(sig, () => { if (sig !== 'SIGINT' && child.exitCode === null) child.kill(sig); });
|
|
18
|
+
}
|
|
19
|
+
child.on('error', (err) => { console.error(`artel: ${err.message}`); process.exit(2); });
|
|
20
|
+
child.on('exit', (code, signal) => {
|
|
21
|
+
if (signal) { process.removeAllListeners(signal); process.kill(process.pid, signal); return; }
|
|
22
|
+
process.exit(code ?? 1);
|
|
23
|
+
});
|
package/package.json
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "artel-qa-lite",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Artel QA Lite — agentic QA for web applications, installed as `artel`. The same product as metantel-cli: reads your source, tests in a real browser, writes a report a QA lead can act on.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"artel": "bin/artel.js"
|
|
8
|
+
},
|
|
9
|
+
"engines": {
|
|
10
|
+
"node": ">=20"
|
|
11
|
+
},
|
|
12
|
+
"files": [
|
|
13
|
+
"bin/",
|
|
14
|
+
"README.md",
|
|
15
|
+
"LICENSE"
|
|
16
|
+
],
|
|
17
|
+
"dependencies": {
|
|
18
|
+
"metantel-cli": "0.1.0"
|
|
19
|
+
},
|
|
20
|
+
"keywords": [
|
|
21
|
+
"qa",
|
|
22
|
+
"testing",
|
|
23
|
+
"test-automation",
|
|
24
|
+
"e2e",
|
|
25
|
+
"browser-testing",
|
|
26
|
+
"test-plan",
|
|
27
|
+
"test-report",
|
|
28
|
+
"ai-agents",
|
|
29
|
+
"claude-code",
|
|
30
|
+
"artel",
|
|
31
|
+
"metantel"
|
|
32
|
+
],
|
|
33
|
+
"homepage": "https://metantel.com",
|
|
34
|
+
"bugs": {
|
|
35
|
+
"email": "hello@metantel.com"
|
|
36
|
+
},
|
|
37
|
+
"author": "Metantel <hello@metantel.com>",
|
|
38
|
+
"license": "SEE LICENSE IN LICENSE",
|
|
39
|
+
"publishConfig": {
|
|
40
|
+
"access": "public"
|
|
41
|
+
}
|
|
42
|
+
}
|