squally-mcp 0.6.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +321 -321
- package/dist/openapi/v1.json +33 -58
- package/dist/tools.js +4 -3
- package/package.json +63 -63
package/README.md
CHANGED
|
@@ -1,321 +1,321 @@
|
|
|
1
|
-
# squally-mcp
|
|
2
|
-
|
|
3
|
-
A local [MCP](https://modelcontextprotocol.io) server that lets a coding agent
|
|
4
|
-
read your [Squally](https://app.squally.dev) data: which runs happened, how
|
|
5
|
-
stable each test is, and why a particular test failed.
|
|
6
|
-
|
|
7
|
-
It runs on your machine over stdio and talks to Squally's read API over HTTPS.
|
|
8
|
-
Point Claude Code, Codex, Cursor or Claude Desktop at it and ask "why is
|
|
9
|
-
`checkout.spec.ts` failing on main?".
|
|
10
|
-
|
|
11
|
-
## Read-only, and structurally so
|
|
12
|
-
|
|
13
|
-
The server issues **HTTP GETs and nothing else**. There is no code path that
|
|
14
|
-
writes, and every tool is annotated `readOnlyHint: true`, so a client can run
|
|
15
|
-
them without asking you to confirm each one.
|
|
16
|
-
|
|
17
|
-
It also cannot reach past your own organization: the key is scoped to one
|
|
18
|
-
organization, and scoping, plan checks and secret exclusion are all enforced
|
|
19
|
-
by the API, not by this client.
|
|
20
|
-
|
|
21
|
-
## What an agent can see
|
|
22
|
-
|
|
23
|
-
Worth knowing before you hand a key to an assistant. Through this server an
|
|
24
|
-
agent can read:
|
|
25
|
-
|
|
26
|
-
- your **projects
|
|
27
|
-
- your **runs**, CI and local — commit SHA and message, branch, author, pull
|
|
28
|
-
request, job key, a link to the CI run, timing, pass/fail counts;
|
|
29
|
-
- **per-test results** for a run, including which shard ran what;
|
|
30
|
-
- for a failing attempt: the **error message and stack**, the **code snippet**
|
|
31
|
-
— **verbatim source code from your test file** — and the **
|
|
32
|
-
prompt
|
|
33
|
-
application at the moment of failure** (whatever was on screen, truncated to
|
|
34
|
-
3000 characters);
|
|
35
|
-
- **per-test metrics**: how often each test passed on the first try, passed
|
|
36
|
-
only after a retry, or failed, and how much time retries and failures cost;
|
|
37
|
-
- **error signatures**: failures grouped across runs and branches.
|
|
38
|
-
|
|
39
|
-
It cannot see screenshots, videos or traces — artifacts are deliberately out of
|
|
40
|
-
scope — and it cannot see any key, secret or webhook URL.
|
|
41
|
-
|
|
42
|
-
The key reaches **every project in the organization**. Create one per machine so
|
|
43
|
-
a single revocation does not lock out everything, and revoke it under
|
|
44
|
-
Organization › Read keys when the machine is retired.
|
|
45
|
-
|
|
46
|
-
## Setup
|
|
47
|
-
|
|
48
|
-
You need an **organization read key** (`sqly_ro_…`): Squally → Organization ›
|
|
49
|
-
Read keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
50
|
-
project ingest key your Playwright reporter uses — this server refuses that one
|
|
51
|
-
by name.
|
|
52
|
-
|
|
53
|
-
Requires **Node 22 or newer**. Read keys are available from the **Standard**
|
|
54
|
-
plan.
|
|
55
|
-
|
|
56
|
-
### Claude Code
|
|
57
|
-
|
|
58
|
-
```bash
|
|
59
|
-
claude mcp add squally --scope user \
|
|
60
|
-
--env SQUALLY_API_KEY=sqly_ro_your_key_here \
|
|
61
|
-
-- npx -y squally-mcp
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
### Codex
|
|
65
|
-
|
|
66
|
-
`~/.codex/config.toml`:
|
|
67
|
-
|
|
68
|
-
```toml
|
|
69
|
-
[mcp_servers.squally]
|
|
70
|
-
command = "npx"
|
|
71
|
-
args = ["-y", "squally-mcp"]
|
|
72
|
-
env = { SQUALLY_API_KEY = "sqly_ro_your_key_here" }
|
|
73
|
-
```
|
|
74
|
-
|
|
75
|
-
### Cursor
|
|
76
|
-
|
|
77
|
-
`~/.cursor/mcp.json` (or `.cursor/mcp.json` in a project):
|
|
78
|
-
|
|
79
|
-
```json
|
|
80
|
-
{
|
|
81
|
-
"mcpServers": {
|
|
82
|
-
"squally": {
|
|
83
|
-
"command": "npx",
|
|
84
|
-
"args": ["-y", "squally-mcp"],
|
|
85
|
-
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
86
|
-
}
|
|
87
|
-
}
|
|
88
|
-
}
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
### Claude Desktop
|
|
92
|
-
|
|
93
|
-
`claude_desktop_config.json` — macOS:
|
|
94
|
-
`~/Library/Application Support/Claude/`, Windows: `%APPDATA%\Claude\` — then
|
|
95
|
-
restart the app:
|
|
96
|
-
|
|
97
|
-
```json
|
|
98
|
-
{
|
|
99
|
-
"mcpServers": {
|
|
100
|
-
"squally": {
|
|
101
|
-
"command": "npx",
|
|
102
|
-
"args": ["-y", "squally-mcp"],
|
|
103
|
-
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
104
|
-
}
|
|
105
|
-
}
|
|
106
|
-
}
|
|
107
|
-
```
|
|
108
|
-
|
|
109
|
-
## Environment
|
|
110
|
-
|
|
111
|
-
| Variable | Required | Meaning |
|
|
112
|
-
|---|---|---|
|
|
113
|
-
| `SQUALLY_API_KEY` | yes | Organization read key, `sqly_ro_…`. Missing or an ingest key by mistake, and the server exits with one line saying so. |
|
|
114
|
-
| `SQUALLY_API_URL` | no | Base URL, default `https://app.squally.dev`. A trailing `/` or `/api/v1` is tolerated. Useful against a local Squally: `http://localhost:3000`. |
|
|
115
|
-
| `SQUALLY_USE_SYSTEM_CA` | no | `0` keeps Node's bundled certificates only. |
|
|
116
|
-
|
|
117
|
-
squally-mcp trusts your operating system's certificate store, like your browser does; set SQUALLY_USE_SYSTEM_CA=0 to use Node's bundled certificates only.
|
|
118
|
-
|
|
119
|
-
## Troubleshooting
|
|
120
|
-
|
|
121
|
-
### Claude Desktop
|
|
122
|
-
|
|
123
|
-
- **The server does not start in time.** The first start through `npx`
|
|
124
|
-
downloads the package, which can take longer than Claude Desktop's 60-second
|
|
125
|
-
start-up limit. Install it once beforehand with `npm install -g squally-mcp`;
|
|
126
|
-
`npx -y squally-mcp` then runs that copy without a download. Update it with
|
|
127
|
-
the same command.
|
|
128
|
-
- **Where the log is.** The Microsoft Store build of Claude Desktop writes it to
|
|
129
|
-
`%LOCALAPPDATA%\Claude\logs\mcp-server-squally.log`. Its first lines say which
|
|
130
|
-
Node runs the server and whether it trusts the system's certificate store;
|
|
131
|
-
a request that never arrives is logged there with its cause.
|
|
132
|
-
- **Edit the config only while the app is closed.** Quit Claude Desktop
|
|
133
|
-
completely, including from the system tray, before changing
|
|
134
|
-
`claude_desktop_config.json`.
|
|
135
|
-
|
|
136
|
-
## The tools
|
|
137
|
-
|
|
138
|
-
| Tool | What it answers | Cost |
|
|
139
|
-
|---|---|---|
|
|
140
|
-
| `squally-list-projects` | Which projects exist. **Start here** — every other tool needs a `projectId` from it. | cheap |
|
|
141
|
-
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA, status or source. Counters only. | cheap |
|
|
142
|
-
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
143
|
-
| `squally-debug-failure` | Every attempt of one test in one run: error, stack,
|
|
144
|
-
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
145
|
-
| `squally-list-tests` | Every test with a CI run in the period, ranked by stability, flaky rate or failures. | cheap — two queries, paged |
|
|
146
|
-
| `squally-list-errors` | Error signatures in a period: what keeps failing, grouped, and how its runs ended. | cheap |
|
|
147
|
-
|
|
148
|
-
For a single test use `squally-get-test-metrics`, not `squally-list-tests` —
|
|
149
|
-
the tool descriptions say so, and the server repeats it in its instructions.
|
|
150
|
-
|
|
151
|
-
### Runs
|
|
152
|
-
|
|
153
|
-
`squally-find-run` lists every run, CI and local alike; `source` (`all` by
|
|
154
|
-
default, `ci` or `local`) narrows it, and each run says whether it was
|
|
155
|
-
`local`. A run's `status` is the dashboard's word: `passed`, `failed`,
|
|
156
|
-
`cancelled`, `incomplete` (a shard timed out, was stopped by the monthly
|
|
157
|
-
result limit or never reported) or `running` - and `?status=` filters by
|
|
158
|
-
any of them. A cancelled run is `cancelled`; `cancellation` says how, when
|
|
159
|
-
and by whom. A sharded run is one row, its counters over all its shards with
|
|
160
|
-
every test once, and the tests a cancel cut off count nowhere.
|
|
161
|
-
|
|
162
|
-
Each run also says whether it is `settled` - final and counted; an
|
|
163
|
-
`incomplete` run that is not settled still waits for a shard within the run
|
|
164
|
-
timeout - and, from squally-reporter 0.11.0, which job of its CI run it is
|
|
165
|
-
(`jobKey`, from `SQUALLY_CI_JOB_KEY`; each job is its own run), which
|
|
166
|
-
`attempt`, which run it re-ran (`rerunOf`) and where the CI run is
|
|
167
|
-
(`ciRunUrl`; for GitHub Actions also for older reporters when the project
|
|
168
|
-
has one linked repository).
|
|
169
|
-
|
|
170
|
-
### What the numbers mean
|
|
171
|
-
|
|
172
|
-
The two test tools count **finished CI runs** in the period (7, 14, 30 or 90
|
|
173
|
-
days; 30 by default), on every branch unless you name one. A run finishes when
|
|
174
|
-
all its shards have - or, once the run timeout has passed, with the results its
|
|
175
|
-
shards reported, like a cancelled run; a shard stopped by the monthly result
|
|
176
|
-
limit counts like one that timed out. A sharded run counts once, placed in the period
|
|
177
|
-
by its first shard's start; a run that was a mass failure as a whole is left
|
|
178
|
-
out. Local runs count only
|
|
179
|
-
when you ask for them: `source` is `ci` (the default), `local` or `all`, and
|
|
180
|
-
the response's `population` names which runs it counted. Per test:
|
|
181
|
-
|
|
182
|
-
- **stability** — runs that passed on the first try ÷ runs;
|
|
183
|
-
- **flakyRate** — runs that passed only after a retry ÷ runs;
|
|
184
|
-
- **failureRate** — failed runs ÷ runs.
|
|
185
|
-
|
|
186
|
-
Each is a fraction from 0 to 1, and `null` when the test has no run in the
|
|
187
|
-
period. **There is no verdict**: nothing calls a test flaky, broken or
|
|
188
|
-
healthy. The tools return the numbers; the agent — or you — judges them.
|
|
189
|
-
|
|
190
|
-
### Example: `squally-list-tests`
|
|
191
|
-
|
|
192
|
-
```json
|
|
193
|
-
{ "projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15", "perPage": 3 }
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
The three least stable tests of the last 30 days (abridged):
|
|
197
|
-
|
|
198
|
-
```json
|
|
199
|
-
{
|
|
200
|
-
"population": "ci_groups_finished_not_mass_failure",
|
|
201
|
-
"days": 30,
|
|
202
|
-
"sort": "stability",
|
|
203
|
-
"total": 409,
|
|
204
|
-
"items": [
|
|
205
|
-
{
|
|
206
|
-
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
207
|
-
"filePath": "tests/checkout.spec.ts",
|
|
208
|
-
"runs": 8,
|
|
209
|
-
"stability": 0.5,
|
|
210
|
-
"flakyRate": 0.5,
|
|
211
|
-
"failureRate": 0,
|
|
212
|
-
"timeLostMs": 7361,
|
|
213
|
-
"topBranch": { "branch": "main", "kind": "retries", "count": 3 }
|
|
214
|
-
}
|
|
215
|
-
]
|
|
216
|
-
}
|
|
217
|
-
```
|
|
218
|
-
|
|
219
|
-
### Example: `squally-get-test-metrics`
|
|
220
|
-
|
|
221
|
-
```json
|
|
222
|
-
{
|
|
223
|
-
"projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15",
|
|
224
|
-
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code"
|
|
225
|
-
}
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
The same numbers as that test's row, for one test (abridged):
|
|
229
|
-
|
|
230
|
-
```json
|
|
231
|
-
{
|
|
232
|
-
"days": 30,
|
|
233
|
-
"branch": null,
|
|
234
|
-
"runs": 8,
|
|
235
|
-
"stableRuns": 4,
|
|
236
|
-
"flakyRuns": 4,
|
|
237
|
-
"failedRuns": 0,
|
|
238
|
-
"stability": 0.5,
|
|
239
|
-
"flakyRate": 0.5,
|
|
240
|
-
"failureRate": 0,
|
|
241
|
-
"branches": ["feature/saved-carts", "main"]
|
|
242
|
-
}
|
|
243
|
-
```
|
|
244
|
-
|
|
245
|
-
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
246
|
-
files, the tool answers with the `filePath` values to retry with; a test with
|
|
247
|
-
results but no counted run in the period answers `runs: 0` and `null`
|
|
248
|
-
rates — an answer, not an error.
|
|
249
|
-
|
|
250
|
-
## Development
|
|
251
|
-
|
|
252
|
-
```bash
|
|
253
|
-
npm install
|
|
254
|
-
npm run build # tsc, then copy the vendored OpenAPI document into dist/
|
|
255
|
-
npm test # builds, then runs the suite
|
|
256
|
-
npm run vendor:openapi # re-fetch the API document; review the diff, then re-pin
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
The tool schemas are **derived from Squally's published OpenAPI document**,
|
|
260
|
-
vendored at `src/openapi/v1.json` — an input schema is the operation's
|
|
261
|
-
parameters, an output schema is its 200 response. Nothing is transcribed by
|
|
262
|
-
hand, so the two cannot drift. `test/drift.test.js` compares the vendored copy
|
|
263
|
-
against the live document and fails when an operation or parameter has moved;
|
|
264
|
-
it skips loudly when offline, and while the live API is still an older version
|
|
265
|
-
than the vendored one — a release that has not been deployed yet.
|
|
266
|
-
|
|
267
|
-
## Releasing
|
|
268
|
-
|
|
269
|
-
Publishing is **staged**: CI uploads the tarball, a maintainer approves it, and
|
|
270
|
-
only then is the version installable. `npm stage publish` never asks for 2FA,
|
|
271
|
-
which is what makes it usable from a workflow; the 2FA prompt moves to the
|
|
272
|
-
approval step, so a compromised workflow can stage a version but cannot put one
|
|
273
|
-
in front of users. Direct `npm publish` is not permitted for this package's
|
|
274
|
-
trusted publisher, so this is the only route.
|
|
275
|
-
|
|
276
|
-
> **0.1.0 is the exception.** It was published by hand with `npm publish`
|
|
277
|
-
> before the trusted publisher existed, so it carries no provenance
|
|
278
|
-
> attestation. Every version from 0.1.1 on goes through the steps below.
|
|
279
|
-
|
|
280
|
-
1. **Bump and tag.** The lockfile belongs to the release commit, so let `npm
|
|
281
|
-
version` write both and commit them together:
|
|
282
|
-
|
|
283
|
-
```bash
|
|
284
|
-
npm install # only if package.json changed by hand
|
|
285
|
-
npm version patch # or minor / major - writes package.json + lockfile, makes the tag
|
|
286
|
-
git push --follow-tags
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
2. **The tag starts the workflow.** `.github/workflows/publish.yml` runs on
|
|
290
|
-
`v*`: it tests, checks that the tag and `package.json` agree, and runs
|
|
291
|
-
|
|
292
|
-
```bash
|
|
293
|
-
npm stage publish --provenance --access public
|
|
294
|
-
```
|
|
295
|
-
|
|
296
|
-
through npm's trusted publishing (OIDC) — no npm token in GitHub secrets.
|
|
297
|
-
`--provenance` attaches a signed statement linking the tarball to that
|
|
298
|
-
workflow run and commit.
|
|
299
|
-
|
|
300
|
-
3. **Approve the staged version.** On
|
|
301
|
-
[npmjs.com/package/squally-mcp](https://www.npmjs.com/package/squally-mcp) →
|
|
302
|
-
**Staged Packages** → **Approve**, which asks for 2FA. Or from a terminal:
|
|
303
|
-
|
|
304
|
-
```bash
|
|
305
|
-
npm stage list squally-mcp
|
|
306
|
-
npm stage view <stage-id> # what is in the tarball
|
|
307
|
-
npm stage approve <stage-id> # also asks for 2FA
|
|
308
|
-
npm stage reject <stage-id> # if something is wrong
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
Until this step the version exists in the registry but installs nothing.
|
|
312
|
-
|
|
313
|
-
**Requirements for step 3 on your machine:** npm **11.15.0 or later** and Node
|
|
314
|
-
**22.14.0 or higher** ([npm docs](https://docs.npmjs.com/staged-publishing/)).
|
|
315
|
-
Node 22 still bundles npm 10.9.9, so check `npm --version` rather than assuming
|
|
316
|
-
your Node version brought a new enough npm. The workflow uses Node 24 for the
|
|
317
|
-
same reason and fails with that requirement if a runner ever ships an older npm.
|
|
318
|
-
|
|
319
|
-
## License
|
|
320
|
-
|
|
321
|
-
MIT
|
|
1
|
+
# squally-mcp
|
|
2
|
+
|
|
3
|
+
A local [MCP](https://modelcontextprotocol.io) server that lets a coding agent
|
|
4
|
+
read your [Squally](https://app.squally.dev) data: which runs happened, how
|
|
5
|
+
stable each test is, and why a particular test failed.
|
|
6
|
+
|
|
7
|
+
It runs on your machine over stdio and talks to Squally's read API over HTTPS.
|
|
8
|
+
Point Claude Code, Codex, Cursor or Claude Desktop at it and ask "why is
|
|
9
|
+
`checkout.spec.ts` failing on main?".
|
|
10
|
+
|
|
11
|
+
## Read-only, and structurally so
|
|
12
|
+
|
|
13
|
+
The server issues **HTTP GETs and nothing else**. There is no code path that
|
|
14
|
+
writes, and every tool is annotated `readOnlyHint: true`, so a client can run
|
|
15
|
+
them without asking you to confirm each one.
|
|
16
|
+
|
|
17
|
+
It also cannot reach past your own organization: the key is scoped to one
|
|
18
|
+
organization, and scoping, plan checks and secret exclusion are all enforced
|
|
19
|
+
by the API, not by this client.
|
|
20
|
+
|
|
21
|
+
## What an agent can see
|
|
22
|
+
|
|
23
|
+
Worth knowing before you hand a key to an assistant. Through this server an
|
|
24
|
+
agent can read:
|
|
25
|
+
|
|
26
|
+
- your **projects** and their names;
|
|
27
|
+
- your **runs**, CI and local — commit SHA and message, branch, author, pull
|
|
28
|
+
request, job key, a link to the CI run, timing, pass/fail counts;
|
|
29
|
+
- **per-test results** for a run, including which shard ran what;
|
|
30
|
+
- for a failing attempt: the **error message and stack**, the **code snippet**
|
|
31
|
+
— **verbatim source code from your test file** — and the **full-context
|
|
32
|
+
prompt** (the text behind Copy the full context in Squally), which carries the same snippet plus an **ARIA snapshot of your
|
|
33
|
+
application at the moment of failure** (whatever was on screen, truncated to
|
|
34
|
+
3000 characters);
|
|
35
|
+
- **per-test metrics**: how often each test passed on the first try, passed
|
|
36
|
+
only after a retry, or failed, and how much time retries and failures cost;
|
|
37
|
+
- **error signatures**: failures grouped across runs and branches.
|
|
38
|
+
|
|
39
|
+
It cannot see screenshots, videos or traces — artifacts are deliberately out of
|
|
40
|
+
scope — and it cannot see any key, secret or webhook URL.
|
|
41
|
+
|
|
42
|
+
The key reaches **every project in the organization**. Create one per machine so
|
|
43
|
+
a single revocation does not lock out everything, and revoke it under
|
|
44
|
+
Organization › Read keys when the machine is retired.
|
|
45
|
+
|
|
46
|
+
## Setup
|
|
47
|
+
|
|
48
|
+
You need an **organization read key** (`sqly_ro_…`): Squally → Organization ›
|
|
49
|
+
Read keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
50
|
+
project ingest key your Playwright reporter uses — this server refuses that one
|
|
51
|
+
by name.
|
|
52
|
+
|
|
53
|
+
Requires **Node 22 or newer**. Read keys are available from the **Standard**
|
|
54
|
+
plan.
|
|
55
|
+
|
|
56
|
+
### Claude Code
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
claude mcp add squally --scope user \
|
|
60
|
+
--env SQUALLY_API_KEY=sqly_ro_your_key_here \
|
|
61
|
+
-- npx -y squally-mcp
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### Codex
|
|
65
|
+
|
|
66
|
+
`~/.codex/config.toml`:
|
|
67
|
+
|
|
68
|
+
```toml
|
|
69
|
+
[mcp_servers.squally]
|
|
70
|
+
command = "npx"
|
|
71
|
+
args = ["-y", "squally-mcp"]
|
|
72
|
+
env = { SQUALLY_API_KEY = "sqly_ro_your_key_here" }
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Cursor
|
|
76
|
+
|
|
77
|
+
`~/.cursor/mcp.json` (or `.cursor/mcp.json` in a project):
|
|
78
|
+
|
|
79
|
+
```json
|
|
80
|
+
{
|
|
81
|
+
"mcpServers": {
|
|
82
|
+
"squally": {
|
|
83
|
+
"command": "npx",
|
|
84
|
+
"args": ["-y", "squally-mcp"],
|
|
85
|
+
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
### Claude Desktop
|
|
92
|
+
|
|
93
|
+
`claude_desktop_config.json` — macOS:
|
|
94
|
+
`~/Library/Application Support/Claude/`, Windows: `%APPDATA%\Claude\` — then
|
|
95
|
+
restart the app:
|
|
96
|
+
|
|
97
|
+
```json
|
|
98
|
+
{
|
|
99
|
+
"mcpServers": {
|
|
100
|
+
"squally": {
|
|
101
|
+
"command": "npx",
|
|
102
|
+
"args": ["-y", "squally-mcp"],
|
|
103
|
+
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Environment
|
|
110
|
+
|
|
111
|
+
| Variable | Required | Meaning |
|
|
112
|
+
|---|---|---|
|
|
113
|
+
| `SQUALLY_API_KEY` | yes | Organization read key, `sqly_ro_…`. Missing or an ingest key by mistake, and the server exits with one line saying so. |
|
|
114
|
+
| `SQUALLY_API_URL` | no | Base URL, default `https://app.squally.dev`. A trailing `/` or `/api/v1` is tolerated. Useful against a local Squally: `http://localhost:3000`. |
|
|
115
|
+
| `SQUALLY_USE_SYSTEM_CA` | no | `0` keeps Node's bundled certificates only. |
|
|
116
|
+
|
|
117
|
+
squally-mcp trusts your operating system's certificate store, like your browser does; set SQUALLY_USE_SYSTEM_CA=0 to use Node's bundled certificates only.
|
|
118
|
+
|
|
119
|
+
## Troubleshooting
|
|
120
|
+
|
|
121
|
+
### Claude Desktop
|
|
122
|
+
|
|
123
|
+
- **The server does not start in time.** The first start through `npx`
|
|
124
|
+
downloads the package, which can take longer than Claude Desktop's 60-second
|
|
125
|
+
start-up limit. Install it once beforehand with `npm install -g squally-mcp`;
|
|
126
|
+
`npx -y squally-mcp` then runs that copy without a download. Update it with
|
|
127
|
+
the same command.
|
|
128
|
+
- **Where the log is.** The Microsoft Store build of Claude Desktop writes it to
|
|
129
|
+
`%LOCALAPPDATA%\Claude\logs\mcp-server-squally.log`. Its first lines say which
|
|
130
|
+
Node runs the server and whether it trusts the system's certificate store;
|
|
131
|
+
a request that never arrives is logged there with its cause.
|
|
132
|
+
- **Edit the config only while the app is closed.** Quit Claude Desktop
|
|
133
|
+
completely, including from the system tray, before changing
|
|
134
|
+
`claude_desktop_config.json`.
|
|
135
|
+
|
|
136
|
+
## The tools
|
|
137
|
+
|
|
138
|
+
| Tool | What it answers | Cost |
|
|
139
|
+
|---|---|---|
|
|
140
|
+
| `squally-list-projects` | Which projects exist. **Start here** — every other tool needs a `projectId` from it. | cheap |
|
|
141
|
+
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA, status or source. Counters only. | cheap |
|
|
142
|
+
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
143
|
+
| `squally-debug-failure` | Every attempt of one test in one run: error, stack, full-context prompt. | cheap |
|
|
144
|
+
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
145
|
+
| `squally-list-tests` | Every test with a CI run in the period, ranked by stability, flaky rate or failures. | cheap — two queries, paged |
|
|
146
|
+
| `squally-list-errors` | Error signatures in a period: what keeps failing, grouped, and how its runs ended. | cheap |
|
|
147
|
+
|
|
148
|
+
For a single test use `squally-get-test-metrics`, not `squally-list-tests` —
|
|
149
|
+
the tool descriptions say so, and the server repeats it in its instructions.
|
|
150
|
+
|
|
151
|
+
### Runs
|
|
152
|
+
|
|
153
|
+
`squally-find-run` lists every run, CI and local alike; `source` (`all` by
|
|
154
|
+
default, `ci` or `local`) narrows it, and each run says whether it was
|
|
155
|
+
`local`. A run's `status` is the dashboard's word: `passed`, `failed`,
|
|
156
|
+
`cancelled`, `incomplete` (a shard timed out, was stopped by the monthly
|
|
157
|
+
result limit or never reported) or `running` - and `?status=` filters by
|
|
158
|
+
any of them. A cancelled run is `cancelled`; `cancellation` says how, when
|
|
159
|
+
and by whom. A sharded run is one row, its counters over all its shards with
|
|
160
|
+
every test once, and the tests a cancel cut off count nowhere.
|
|
161
|
+
|
|
162
|
+
Each run also says whether it is `settled` - final and counted; an
|
|
163
|
+
`incomplete` run that is not settled still waits for a shard within the run
|
|
164
|
+
timeout - and, from squally-reporter 0.11.0, which job of its CI run it is
|
|
165
|
+
(`jobKey`, from `SQUALLY_CI_JOB_KEY`; each job is its own run), which
|
|
166
|
+
`attempt`, which run it re-ran (`rerunOf`) and where the CI run is
|
|
167
|
+
(`ciRunUrl`; for GitHub Actions also for older reporters when the project
|
|
168
|
+
has one linked repository).
|
|
169
|
+
|
|
170
|
+
### What the numbers mean
|
|
171
|
+
|
|
172
|
+
The two test tools count **finished CI runs** in the period (7, 14, 30 or 90
|
|
173
|
+
days; 30 by default), on every branch unless you name one. A run finishes when
|
|
174
|
+
all its shards have - or, once the run timeout has passed, with the results its
|
|
175
|
+
shards reported, like a cancelled run; a shard stopped by the monthly result
|
|
176
|
+
limit counts like one that timed out. A sharded run counts once, placed in the period
|
|
177
|
+
by its first shard's start; a run that was a mass failure as a whole is left
|
|
178
|
+
out. Local runs count only
|
|
179
|
+
when you ask for them: `source` is `ci` (the default), `local` or `all`, and
|
|
180
|
+
the response's `population` names which runs it counted. Per test:
|
|
181
|
+
|
|
182
|
+
- **stability** — runs that passed on the first try ÷ runs;
|
|
183
|
+
- **flakyRate** — runs that passed only after a retry ÷ runs;
|
|
184
|
+
- **failureRate** — failed runs ÷ runs.
|
|
185
|
+
|
|
186
|
+
Each is a fraction from 0 to 1, and `null` when the test has no run in the
|
|
187
|
+
period. **There is no verdict**: nothing calls a test flaky, broken or
|
|
188
|
+
healthy. The tools return the numbers; the agent — or you — judges them.
|
|
189
|
+
|
|
190
|
+
### Example: `squally-list-tests`
|
|
191
|
+
|
|
192
|
+
```json
|
|
193
|
+
{ "projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15", "perPage": 3 }
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
The three least stable tests of the last 30 days (abridged):
|
|
197
|
+
|
|
198
|
+
```json
|
|
199
|
+
{
|
|
200
|
+
"population": "ci_groups_finished_not_mass_failure",
|
|
201
|
+
"days": 30,
|
|
202
|
+
"sort": "stability",
|
|
203
|
+
"total": 409,
|
|
204
|
+
"items": [
|
|
205
|
+
{
|
|
206
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
207
|
+
"filePath": "tests/checkout.spec.ts",
|
|
208
|
+
"runs": 8,
|
|
209
|
+
"stability": 0.5,
|
|
210
|
+
"flakyRate": 0.5,
|
|
211
|
+
"failureRate": 0,
|
|
212
|
+
"timeLostMs": 7361,
|
|
213
|
+
"topBranch": { "branch": "main", "kind": "retries", "count": 3 }
|
|
214
|
+
}
|
|
215
|
+
]
|
|
216
|
+
}
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
### Example: `squally-get-test-metrics`
|
|
220
|
+
|
|
221
|
+
```json
|
|
222
|
+
{
|
|
223
|
+
"projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15",
|
|
224
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code"
|
|
225
|
+
}
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
The same numbers as that test's row, for one test (abridged):
|
|
229
|
+
|
|
230
|
+
```json
|
|
231
|
+
{
|
|
232
|
+
"days": 30,
|
|
233
|
+
"branch": null,
|
|
234
|
+
"runs": 8,
|
|
235
|
+
"stableRuns": 4,
|
|
236
|
+
"flakyRuns": 4,
|
|
237
|
+
"failedRuns": 0,
|
|
238
|
+
"stability": 0.5,
|
|
239
|
+
"flakyRate": 0.5,
|
|
240
|
+
"failureRate": 0,
|
|
241
|
+
"branches": ["feature/saved-carts", "main"]
|
|
242
|
+
}
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
246
|
+
files, the tool answers with the `filePath` values to retry with; a test with
|
|
247
|
+
results but no counted run in the period answers `runs: 0` and `null`
|
|
248
|
+
rates — an answer, not an error.
|
|
249
|
+
|
|
250
|
+
## Development
|
|
251
|
+
|
|
252
|
+
```bash
|
|
253
|
+
npm install
|
|
254
|
+
npm run build # tsc, then copy the vendored OpenAPI document into dist/
|
|
255
|
+
npm test # builds, then runs the suite
|
|
256
|
+
npm run vendor:openapi # re-fetch the API document; review the diff, then re-pin
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
The tool schemas are **derived from Squally's published OpenAPI document**,
|
|
260
|
+
vendored at `src/openapi/v1.json` — an input schema is the operation's
|
|
261
|
+
parameters, an output schema is its 200 response. Nothing is transcribed by
|
|
262
|
+
hand, so the two cannot drift. `test/drift.test.js` compares the vendored copy
|
|
263
|
+
against the live document and fails when an operation or parameter has moved;
|
|
264
|
+
it skips loudly when offline, and while the live API is still an older version
|
|
265
|
+
than the vendored one — a release that has not been deployed yet.
|
|
266
|
+
|
|
267
|
+
## Releasing
|
|
268
|
+
|
|
269
|
+
Publishing is **staged**: CI uploads the tarball, a maintainer approves it, and
|
|
270
|
+
only then is the version installable. `npm stage publish` never asks for 2FA,
|
|
271
|
+
which is what makes it usable from a workflow; the 2FA prompt moves to the
|
|
272
|
+
approval step, so a compromised workflow can stage a version but cannot put one
|
|
273
|
+
in front of users. Direct `npm publish` is not permitted for this package's
|
|
274
|
+
trusted publisher, so this is the only route.
|
|
275
|
+
|
|
276
|
+
> **0.1.0 is the exception.** It was published by hand with `npm publish`
|
|
277
|
+
> before the trusted publisher existed, so it carries no provenance
|
|
278
|
+
> attestation. Every version from 0.1.1 on goes through the steps below.
|
|
279
|
+
|
|
280
|
+
1. **Bump and tag.** The lockfile belongs to the release commit, so let `npm
|
|
281
|
+
version` write both and commit them together:
|
|
282
|
+
|
|
283
|
+
```bash
|
|
284
|
+
npm install # only if package.json changed by hand
|
|
285
|
+
npm version patch # or minor / major - writes package.json + lockfile, makes the tag
|
|
286
|
+
git push --follow-tags
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
2. **The tag starts the workflow.** `.github/workflows/publish.yml` runs on
|
|
290
|
+
`v*`: it tests, checks that the tag and `package.json` agree, and runs
|
|
291
|
+
|
|
292
|
+
```bash
|
|
293
|
+
npm stage publish --provenance --access public
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
through npm's trusted publishing (OIDC) — no npm token in GitHub secrets.
|
|
297
|
+
`--provenance` attaches a signed statement linking the tarball to that
|
|
298
|
+
workflow run and commit.
|
|
299
|
+
|
|
300
|
+
3. **Approve the staged version.** On
|
|
301
|
+
[npmjs.com/package/squally-mcp](https://www.npmjs.com/package/squally-mcp) →
|
|
302
|
+
**Staged Packages** → **Approve**, which asks for 2FA. Or from a terminal:
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
npm stage list squally-mcp
|
|
306
|
+
npm stage view <stage-id> # what is in the tarball
|
|
307
|
+
npm stage approve <stage-id> # also asks for 2FA
|
|
308
|
+
npm stage reject <stage-id> # if something is wrong
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Until this step the version exists in the registry but installs nothing.
|
|
312
|
+
|
|
313
|
+
**Requirements for step 3 on your machine:** npm **11.15.0 or later** and Node
|
|
314
|
+
**22.14.0 or higher** ([npm docs](https://docs.npmjs.com/staged-publishing/)).
|
|
315
|
+
Node 22 still bundles npm 10.9.9, so check `npm --version` rather than assuming
|
|
316
|
+
your Node version brought a new enough npm. The workflow uses Node 24 for the
|
|
317
|
+
same reason and fails with that requirement if a runner ever ships an older npm.
|
|
318
|
+
|
|
319
|
+
## License
|
|
320
|
+
|
|
321
|
+
MIT
|
package/dist/openapi/v1.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema",
|
|
4
4
|
"info": {
|
|
5
5
|
"title": "Squally Read API",
|
|
6
|
-
"version": "1.0.0-beta.
|
|
6
|
+
"version": "1.0.0-beta.17",
|
|
7
7
|
"description": "Read access to a Squally organization's test results, for scripts, CI jobs and AI agents.\n\n**Authentication.** Every request carries an organization read key as `Authorization: Bearer sqly_ro_…`. A read key is organization-wide (every project), created by an organization admin in Organization › Read keys, and expires after 30, 60, 90, 180 days as chosen at creation. There is no renewal: create a new key and revoke the old one. Project ingest keys (`sqly_…`) are refused with `wrong_key_type`.\n\n**Expiry warning.** From 7 days before a key expires, every authenticated response carries `Squally-Key-Expires-At` with the expiry instant (ISO-8601).\n\n**Rate limit.** Currently 60 requests per 60 seconds per key, counted per `Authorization` header - a read key has its own budget, shared with nothing else. Over the limit the edge answers 429 without a JSON body.\n\n**Retries.** 429: wait 60 seconds, then retry. The edge counts requests per key in 60-second windows and sends no Retry-After header. 5xx: retry with exponential backoff, starting at 1 second and capped at 30 seconds. 400, 401, 403, 404: never retry - the same request gets the same answer.\n\n**Plans.** The read API is part of the Standard and Pro plans. A key of an organization on Free is answered 403 `tier_too_low`; keys are not revoked on a downgrade and work again after an upgrade.\n\n**Errors.** Every error is `{\"error\": \"<sentence>\", \"code\": \"<slug>\"}`. The sentence may be reworded; the slug may not. An id that does not exist and an id that belongs to another organization get the same 404, byte for byte.\n\n**Test names in paths.** Test names are Playwright title paths and often contain `/`: URL-encode them, including `/` as `%2F`.\n\n**Sharded runs** are returned as one logical run: /runs lists one row per logical run, and the run routes answer for all shards whichever shard's id you pass.\n\n**Stability.** v1, beta. The API may change until the beta ends; every change is listed in the changelog (docs/openapi/CHANGELOG.md)."
|
|
8
8
|
},
|
|
9
9
|
"servers": [
|
|
@@ -173,13 +173,11 @@
|
|
|
173
173
|
{
|
|
174
174
|
"id": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15",
|
|
175
175
|
"name": "storefront-web",
|
|
176
|
-
"stableBranch": "main",
|
|
177
176
|
"createdAt": "2026-06-02T09:14:00.000Z"
|
|
178
177
|
},
|
|
179
178
|
{
|
|
180
179
|
"id": "a8d4e1c7-6f20-4b93-8e5a-1c7d3b9f0e42",
|
|
181
180
|
"name": "admin-console",
|
|
182
|
-
"stableBranch": "develop",
|
|
183
181
|
"createdAt": "2026-07-18T13:40:00.000Z"
|
|
184
182
|
}
|
|
185
183
|
]
|
|
@@ -482,7 +480,6 @@
|
|
|
482
480
|
"from": "2026-08-25T08:00:00.000Z",
|
|
483
481
|
"to": "2026-09-24T08:00:00.000Z"
|
|
484
482
|
},
|
|
485
|
-
"stableBranch": "main",
|
|
486
483
|
"runs": [
|
|
487
484
|
{
|
|
488
485
|
"id": "b1e0c6d2-7a43-4f19-8d5e-0c3a9e6f2b71",
|
|
@@ -3136,7 +3133,7 @@
|
|
|
3136
3133
|
"get": {
|
|
3137
3134
|
"operationId": "listErrors",
|
|
3138
3135
|
"summary": "List errors",
|
|
3139
|
-
"description": "List errors. The project's errors in a period - failures grouped by fingerprint - most frequent first, over the finished runs of the source (see population), all shards of a run one run. Every count covers the period; firstSeen is the first occurrence still stored, not bounded by it. Moderate: one aggregate query.",
|
|
3136
|
+
"description": "List errors. The project's errors in a period - failures grouped by fingerprint - most frequent first: by occurrences, then runs, then tests, ties by fingerprint, so a page never changes order between two requests over the same data. Over the finished runs of the source (see population), all shards of a run one run. Every count covers the period; firstSeen is the first occurrence still stored, not bounded by it. Moderate: one aggregate query, paged in the database.",
|
|
3140
3137
|
"tags": [
|
|
3141
3138
|
"Errors"
|
|
3142
3139
|
],
|
|
@@ -3895,14 +3892,11 @@
|
|
|
3895
3892
|
"name": "branch",
|
|
3896
3893
|
"in": "query",
|
|
3897
3894
|
"required": false,
|
|
3898
|
-
"description": "
|
|
3895
|
+
"description": "Exact branch name: every figure counts that branch's runs. Every branch when absent or all (since 1.0.0-beta.17; it took stable or all, default stable, before - stable is now a 400).",
|
|
3899
3896
|
"schema": {
|
|
3900
3897
|
"type": "string",
|
|
3901
|
-
"
|
|
3902
|
-
|
|
3903
|
-
"all"
|
|
3904
|
-
],
|
|
3905
|
-
"default": "stable"
|
|
3898
|
+
"minLength": 1,
|
|
3899
|
+
"maxLength": 200
|
|
3906
3900
|
}
|
|
3907
3901
|
},
|
|
3908
3902
|
{
|
|
@@ -3944,8 +3938,7 @@
|
|
|
3944
3938
|
"from": "2026-08-25T08:00:00.000Z",
|
|
3945
3939
|
"to": "2026-09-24T08:00:00.000Z"
|
|
3946
3940
|
},
|
|
3947
|
-
"
|
|
3948
|
-
"stableBranch": "main",
|
|
3941
|
+
"branch": "main",
|
|
3949
3942
|
"lastRun": {
|
|
3950
3943
|
"id": "b1e0c6d2-7a43-4f19-8d5e-0c3a9e6f2b71",
|
|
3951
3944
|
"runNumber": 1284,
|
|
@@ -3957,8 +3950,7 @@
|
|
|
3957
3950
|
"passed": 404,
|
|
3958
3951
|
"flaky": 4,
|
|
3959
3952
|
"failed": 0,
|
|
3960
|
-
"skipped": 4
|
|
3961
|
-
"isFallbackBranch": false
|
|
3953
|
+
"skipped": 4
|
|
3962
3954
|
},
|
|
3963
3955
|
"runsInWindow": 11,
|
|
3964
3956
|
"unfinishedRuns": 1,
|
|
@@ -4244,7 +4236,7 @@
|
|
|
4244
4236
|
"get": {
|
|
4245
4237
|
"operationId": "getSettings",
|
|
4246
4238
|
"summary": "Get settings",
|
|
4247
|
-
"description": "Get the project configuration.
|
|
4239
|
+
"description": "Get the project configuration. The plan's retention, Slack events, PR comments. Configuration only - never a key, token or webhook. Cheap.",
|
|
4248
4240
|
"tags": [
|
|
4249
4241
|
"Project"
|
|
4250
4242
|
],
|
|
@@ -4278,7 +4270,6 @@
|
|
|
4278
4270
|
"name": "storefront-web"
|
|
4279
4271
|
},
|
|
4280
4272
|
"settings": {
|
|
4281
|
-
"stableBranch": "main",
|
|
4282
4273
|
"retentionDays": 60,
|
|
4283
4274
|
"slack": {
|
|
4284
4275
|
"configured": true,
|
|
@@ -4510,9 +4501,6 @@
|
|
|
4510
4501
|
"name": {
|
|
4511
4502
|
"type": "string"
|
|
4512
4503
|
},
|
|
4513
|
-
"stableBranch": {
|
|
4514
|
-
"type": "string"
|
|
4515
|
-
},
|
|
4516
4504
|
"createdAt": {
|
|
4517
4505
|
"type": "string",
|
|
4518
4506
|
"format": "date-time"
|
|
@@ -4521,7 +4509,6 @@
|
|
|
4521
4509
|
"required": [
|
|
4522
4510
|
"id",
|
|
4523
4511
|
"name",
|
|
4524
|
-
"stableBranch",
|
|
4525
4512
|
"createdAt"
|
|
4526
4513
|
]
|
|
4527
4514
|
}
|
|
@@ -4570,9 +4557,6 @@
|
|
|
4570
4557
|
"to"
|
|
4571
4558
|
]
|
|
4572
4559
|
},
|
|
4573
|
-
"stableBranch": {
|
|
4574
|
-
"type": "string"
|
|
4575
|
-
},
|
|
4576
4560
|
"runs": {
|
|
4577
4561
|
"type": "array",
|
|
4578
4562
|
"items": {
|
|
@@ -4635,7 +4619,7 @@
|
|
|
4635
4619
|
"incomplete",
|
|
4636
4620
|
"running"
|
|
4637
4621
|
],
|
|
4638
|
-
"description": "How the run ended, the dashboard's own word: cancelled - the run was cancelled (for a sharded run, any shard recorded a cancel; cancellation says how and when); incomplete - a shard timed out, was stopped by the monthly result limit or never reported; running - a shard is still running; failed - every shard finished and a test failed; passed - every shard finished and none failed. In that order: the first that applies. Since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel and a cancelled boolean."
|
|
4622
|
+
"description": "How the run ended, the dashboard's own word: cancelled - the run was cancelled (for a sharded run, any shard recorded a cancel; cancellation says how and when); incomplete - a shard timed out, was stopped by the monthly result limit, or never reported by the time the run is overdue by the project's run timeout, counted from its latest shard's start; running - a shard is still running, or the run still waits for a shard within that time; failed - every shard finished and a test failed; passed - every shard finished and none failed. In that order: the first that applies. Since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel and a cancelled boolean. Waiting for a shard is running since 2026-10-20 - before it, incomplete."
|
|
4639
4623
|
},
|
|
4640
4624
|
"cancellation": {
|
|
4641
4625
|
"anyOf": [
|
|
@@ -4688,7 +4672,7 @@
|
|
|
4688
4672
|
},
|
|
4689
4673
|
"settled": {
|
|
4690
4674
|
"type": "boolean",
|
|
4691
|
-
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while
|
|
4675
|
+
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while the run still waits for a shard within the run time limit - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
4692
4676
|
},
|
|
4693
4677
|
"jobKey": {
|
|
4694
4678
|
"anyOf": [
|
|
@@ -4886,7 +4870,6 @@
|
|
|
4886
4870
|
"required": [
|
|
4887
4871
|
"project",
|
|
4888
4872
|
"window",
|
|
4889
|
-
"stableBranch",
|
|
4890
4873
|
"runs",
|
|
4891
4874
|
"total",
|
|
4892
4875
|
"nextCursor",
|
|
@@ -5011,7 +4994,7 @@
|
|
|
5011
4994
|
"incomplete",
|
|
5012
4995
|
"running"
|
|
5013
4996
|
],
|
|
5014
|
-
"description": "How the run ended, the dashboard's own word: cancelled - the run was cancelled (for a sharded run, any shard recorded a cancel; cancellation says how and when); incomplete - a shard timed out, was stopped by the monthly result limit or never reported; running - a shard is still running; failed - every shard finished and a test failed; passed - every shard finished and none failed. In that order: the first that applies. Since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel and a cancelled boolean."
|
|
4997
|
+
"description": "How the run ended, the dashboard's own word: cancelled - the run was cancelled (for a sharded run, any shard recorded a cancel; cancellation says how and when); incomplete - a shard timed out, was stopped by the monthly result limit, or never reported by the time the run is overdue by the project's run timeout, counted from its latest shard's start; running - a shard is still running, or the run still waits for a shard within that time; failed - every shard finished and a test failed; passed - every shard finished and none failed. In that order: the first that applies. Since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel and a cancelled boolean. Waiting for a shard is running since 2026-10-20 - before it, incomplete."
|
|
5015
4998
|
},
|
|
5016
4999
|
"cancellation": {
|
|
5017
5000
|
"anyOf": [
|
|
@@ -5064,7 +5047,7 @@
|
|
|
5064
5047
|
},
|
|
5065
5048
|
"settled": {
|
|
5066
5049
|
"type": "boolean",
|
|
5067
|
-
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while
|
|
5050
|
+
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while the run still waits for a shard within the run time limit - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
5068
5051
|
},
|
|
5069
5052
|
"jobKey": {
|
|
5070
5053
|
"anyOf": [
|
|
@@ -5628,7 +5611,7 @@
|
|
|
5628
5611
|
"type": "null"
|
|
5629
5612
|
}
|
|
5630
5613
|
],
|
|
5631
|
-
"description": "A prompt for an assistant that has the repository open, byte-identical to the Copy
|
|
5614
|
+
"description": "A prompt for an assistant that has the repository open, byte-identical to the text that Copy the full context puts on the clipboard (Debug with AI on the attempt page). null for an attempt without a real failure (a pass, a test.fail() marker, no error text)."
|
|
5632
5615
|
},
|
|
5633
5616
|
"shard": {
|
|
5634
5617
|
"anyOf": [
|
|
@@ -6923,15 +6906,16 @@
|
|
|
6923
6906
|
"to"
|
|
6924
6907
|
]
|
|
6925
6908
|
},
|
|
6926
|
-
"
|
|
6927
|
-
"
|
|
6928
|
-
|
|
6929
|
-
|
|
6930
|
-
|
|
6931
|
-
|
|
6932
|
-
|
|
6933
|
-
|
|
6934
|
-
|
|
6909
|
+
"branch": {
|
|
6910
|
+
"anyOf": [
|
|
6911
|
+
{
|
|
6912
|
+
"type": "string"
|
|
6913
|
+
},
|
|
6914
|
+
{
|
|
6915
|
+
"type": "null"
|
|
6916
|
+
}
|
|
6917
|
+
],
|
|
6918
|
+
"description": "The branch every figure counts - the branch parameter - or null for every branch (since 1.0.0-beta.17; branchScope and stableBranch until then)."
|
|
6935
6919
|
},
|
|
6936
6920
|
"lastRun": {
|
|
6937
6921
|
"anyOf": [
|
|
@@ -7000,9 +6984,6 @@
|
|
|
7000
6984
|
"skipped": {
|
|
7001
6985
|
"type": "integer",
|
|
7002
6986
|
"description": "Since 1.0.0-beta.7."
|
|
7003
|
-
},
|
|
7004
|
-
"isFallbackBranch": {
|
|
7005
|
-
"type": "boolean"
|
|
7006
6987
|
}
|
|
7007
6988
|
},
|
|
7008
6989
|
"required": [
|
|
@@ -7016,23 +6997,22 @@
|
|
|
7016
6997
|
"passed",
|
|
7017
6998
|
"flaky",
|
|
7018
6999
|
"failed",
|
|
7019
|
-
"skipped"
|
|
7020
|
-
"isFallbackBranch"
|
|
7000
|
+
"skipped"
|
|
7021
7001
|
]
|
|
7022
7002
|
},
|
|
7023
7003
|
{
|
|
7024
7004
|
"type": "null"
|
|
7025
7005
|
}
|
|
7026
7006
|
],
|
|
7027
|
-
"description": "The newest finished run of the source on
|
|
7007
|
+
"description": "The newest finished run of the source on the branch, whatever the window (see the source parameter and testMetrics.population); null when there is none. No fallback to another branch since 1.0.0-beta.17, and no isFallbackBranch."
|
|
7028
7008
|
},
|
|
7029
7009
|
"runsInWindow": {
|
|
7030
7010
|
"type": "integer",
|
|
7031
|
-
"description": "Finished runs of the source in the window on
|
|
7011
|
+
"description": "Finished runs of the source in the window on the branch (see the source parameter and testMetrics.population), each counted once however many shards it had, by the start of its first shard; mass-failure, cancelled and incomplete runs included (incomplete - finished by the run timeout - since 1.0.0-beta.10). The runs windowStability is counted over."
|
|
7032
7012
|
},
|
|
7033
7013
|
"unfinishedRuns": {
|
|
7034
7014
|
"type": "integer",
|
|
7035
|
-
"description": "Runs of the source in the window on
|
|
7015
|
+
"description": "Runs of the source in the window on the branch that have not finished - a shard still running, or one timed out, stopped by the monthly result limit or never reported while the run is not yet overdue (an overdue one is finished and counted since 1.0.0-beta.10; a limit-stopped one since 1.0.0-beta.11). In no other figure (since 1.0.0-beta.7; of the source since 1.0.0-beta.8)."
|
|
7036
7016
|
},
|
|
7037
7017
|
"suiteSize": {
|
|
7038
7018
|
"type": "integer",
|
|
@@ -7047,7 +7027,7 @@
|
|
|
7047
7027
|
"type": "null"
|
|
7048
7028
|
}
|
|
7049
7029
|
],
|
|
7050
|
-
"description": "Suite stability, 0-100: of the final test results in the window's finished runs of the source on
|
|
7030
|
+
"description": "Suite stability, 0-100: of the final test results in the window's finished runs of the source on the branch (skipped results left out), the share that passed on the first try, rounded. Mass-failure runs count (since 1.0.0-beta.7). A result that passed only after a retry counts against it. null when no such result exists."
|
|
7051
7031
|
},
|
|
7052
7032
|
"timeLost": {
|
|
7053
7033
|
"type": "object",
|
|
@@ -7078,7 +7058,7 @@
|
|
|
7078
7058
|
"wastedAttempts",
|
|
7079
7059
|
"wallClockMs"
|
|
7080
7060
|
],
|
|
7081
|
-
"description": "Time spent on attempts a later retry superseded, in the window's finished runs of the source on
|
|
7061
|
+
"description": "Time spent on attempts a later retry superseded, in the window's finished runs of the source on the branch; wallClockMs is the time those runs used - each shard's own duration, summed - and lostPercent is lostMs of it."
|
|
7082
7062
|
},
|
|
7083
7063
|
"failingTests": {
|
|
7084
7064
|
"type": "object",
|
|
@@ -7094,7 +7074,7 @@
|
|
|
7094
7074
|
"failing",
|
|
7095
7075
|
"testsRun"
|
|
7096
7076
|
],
|
|
7097
|
-
"description": "Tests whose final attempt failed in at least one of the window's finished runs of the source on
|
|
7077
|
+
"description": "Tests whose final attempt failed in at least one of the window's finished runs of the source on the branch, out of the tests those runs reported (a skipped test counts as reported)."
|
|
7098
7078
|
},
|
|
7099
7079
|
"testMetrics": {
|
|
7100
7080
|
"type": "object",
|
|
@@ -7132,7 +7112,7 @@
|
|
|
7132
7112
|
"failingTests",
|
|
7133
7113
|
"tests"
|
|
7134
7114
|
],
|
|
7135
|
-
"description": "The dashboard's Flaky tests and Failing tests tiles and their denominator, counted like /tests on
|
|
7115
|
+
"description": "The dashboard's Flaky tests and Failing tests tiles and their denominator, counted like /tests on the branch (since 1.0.0-beta.7; every branch before). Mass-failure runs are left out here, as in every per-test metric."
|
|
7136
7116
|
},
|
|
7137
7117
|
"slowestTests": {
|
|
7138
7118
|
"type": "array",
|
|
@@ -7230,14 +7210,13 @@
|
|
|
7230
7210
|
"durationMs"
|
|
7231
7211
|
]
|
|
7232
7212
|
},
|
|
7233
|
-
"description": "The last 30 finished runs of the source on
|
|
7213
|
+
"description": "The last 30 finished runs of the source on the branch, whatever the window, oldest first - one point per run, passed counting first-try passes, durationMs from its first shard's start to its last shard's finish. Cancelled and incomplete runs are left out (incomplete since 1.0.0-beta.10)."
|
|
7234
7214
|
}
|
|
7235
7215
|
},
|
|
7236
7216
|
"required": [
|
|
7237
7217
|
"project",
|
|
7238
7218
|
"window",
|
|
7239
|
-
"
|
|
7240
|
-
"stableBranch",
|
|
7219
|
+
"branch",
|
|
7241
7220
|
"lastRun",
|
|
7242
7221
|
"runsInWindow",
|
|
7243
7222
|
"unfinishedRuns",
|
|
@@ -7272,9 +7251,6 @@
|
|
|
7272
7251
|
"settings": {
|
|
7273
7252
|
"type": "object",
|
|
7274
7253
|
"properties": {
|
|
7275
|
-
"stableBranch": {
|
|
7276
|
-
"type": "string"
|
|
7277
|
-
},
|
|
7278
7254
|
"retentionDays": {
|
|
7279
7255
|
"type": "integer"
|
|
7280
7256
|
},
|
|
@@ -7309,7 +7285,6 @@
|
|
|
7309
7285
|
}
|
|
7310
7286
|
},
|
|
7311
7287
|
"required": [
|
|
7312
|
-
"stableBranch",
|
|
7313
7288
|
"retentionDays",
|
|
7314
7289
|
"slack",
|
|
7315
7290
|
"github"
|
package/dist/tools.js
CHANGED
|
@@ -51,9 +51,10 @@ export const TOOLS = [
|
|
|
51
51
|
title: "Debug a failing test",
|
|
52
52
|
operationId: "getRunTestAttempts",
|
|
53
53
|
description: "Every attempt of one test in one run, oldest first, each with the error, the stack and " +
|
|
54
|
-
"the
|
|
54
|
+
"the full-context prompt (the same text Copy the full context puts on the clipboard in " +
|
|
55
|
+
"Squally's Debug with AI panel). This is the " +
|
|
55
56
|
"tool for 'why did this test fail'. If the test name is ambiguous, pass filePath.",
|
|
56
|
-
answers: "Every attempt of one test in one run: error, stack,
|
|
57
|
+
answers: "Every attempt of one test in one run: error, stack, full-context prompt.",
|
|
57
58
|
cost: "cheap",
|
|
58
59
|
},
|
|
59
60
|
{
|
|
@@ -91,7 +92,7 @@ export const TOOLS = [
|
|
|
91
92
|
},
|
|
92
93
|
{
|
|
93
94
|
name: "squally-list-errors",
|
|
94
|
-
title: "List
|
|
95
|
+
title: "List errors",
|
|
95
96
|
operationId: "listErrors",
|
|
96
97
|
description: "The project's error signatures in a period: failures grouped by fingerprint, with how " +
|
|
97
98
|
"often and how recently each occurred. Answers 'is this failure one instance of something " +
|
package/package.json
CHANGED
|
@@ -1,63 +1,63 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "squally-mcp",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Local MCP server for Squally — read-only access to your Playwright test runs, flaky tests and failures.",
|
|
5
|
-
"license": "MIT",
|
|
6
|
-
"author": "Squally",
|
|
7
|
-
"homepage": "https://docs.squally.dev/agents/mcp-server/",
|
|
8
|
-
"repository": {
|
|
9
|
-
"type": "git",
|
|
10
|
-
"url": "git+https://github.com/squallydev-admin/squally-mcp.git"
|
|
11
|
-
},
|
|
12
|
-
"bugs": {
|
|
13
|
-
"url": "https://github.com/squallydev-admin/squally-mcp/issues"
|
|
14
|
-
},
|
|
15
|
-
"keywords": [
|
|
16
|
-
"mcp",
|
|
17
|
-
"modelcontextprotocol",
|
|
18
|
-
"playwright",
|
|
19
|
-
"flaky-tests",
|
|
20
|
-
"squally"
|
|
21
|
-
],
|
|
22
|
-
"type": "module",
|
|
23
|
-
"bin": {
|
|
24
|
-
"squally-mcp": "dist/index.js"
|
|
25
|
-
},
|
|
26
|
-
"main": "dist/index.js",
|
|
27
|
-
"types": "dist/index.d.ts",
|
|
28
|
-
"exports": {
|
|
29
|
-
".": {
|
|
30
|
-
"types": "./dist/index.d.ts",
|
|
31
|
-
"default": "./dist/index.js"
|
|
32
|
-
},
|
|
33
|
-
"./tools": {
|
|
34
|
-
"types": "./dist/tools.d.ts",
|
|
35
|
-
"default": "./dist/tools.js"
|
|
36
|
-
},
|
|
37
|
-
"./instructions": {
|
|
38
|
-
"types": "./dist/instructions.d.ts",
|
|
39
|
-
"default": "./dist/instructions.js"
|
|
40
|
-
},
|
|
41
|
-
"./package.json": "./package.json"
|
|
42
|
-
},
|
|
43
|
-
"files": [
|
|
44
|
-
"dist"
|
|
45
|
-
],
|
|
46
|
-
"engines": {
|
|
47
|
-
"node": ">=22"
|
|
48
|
-
},
|
|
49
|
-
"scripts": {
|
|
50
|
-
"build": "tsc && node scripts/copy-assets.mjs",
|
|
51
|
-
"vendor:openapi": "node scripts/vendor-openapi.mjs",
|
|
52
|
-
"test": "npm run build && node --test \"test/**/*.test.js\"",
|
|
53
|
-
"prepublishOnly": "npm run build"
|
|
54
|
-
},
|
|
55
|
-
"dependencies": {
|
|
56
|
-
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
57
|
-
"zod": "^4.6.5"
|
|
58
|
-
},
|
|
59
|
-
"devDependencies": {
|
|
60
|
-
"@types/node": "^22.10.0",
|
|
61
|
-
"typescript": "^5.6.0"
|
|
62
|
-
}
|
|
63
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"name": "squally-mcp",
|
|
3
|
+
"version": "0.7.0",
|
|
4
|
+
"description": "Local MCP server for Squally — read-only access to your Playwright test runs, flaky tests and failures.",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"author": "Squally",
|
|
7
|
+
"homepage": "https://docs.squally.dev/agents/mcp-server/",
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/squallydev-admin/squally-mcp.git"
|
|
11
|
+
},
|
|
12
|
+
"bugs": {
|
|
13
|
+
"url": "https://github.com/squallydev-admin/squally-mcp/issues"
|
|
14
|
+
},
|
|
15
|
+
"keywords": [
|
|
16
|
+
"mcp",
|
|
17
|
+
"modelcontextprotocol",
|
|
18
|
+
"playwright",
|
|
19
|
+
"flaky-tests",
|
|
20
|
+
"squally"
|
|
21
|
+
],
|
|
22
|
+
"type": "module",
|
|
23
|
+
"bin": {
|
|
24
|
+
"squally-mcp": "dist/index.js"
|
|
25
|
+
},
|
|
26
|
+
"main": "dist/index.js",
|
|
27
|
+
"types": "dist/index.d.ts",
|
|
28
|
+
"exports": {
|
|
29
|
+
".": {
|
|
30
|
+
"types": "./dist/index.d.ts",
|
|
31
|
+
"default": "./dist/index.js"
|
|
32
|
+
},
|
|
33
|
+
"./tools": {
|
|
34
|
+
"types": "./dist/tools.d.ts",
|
|
35
|
+
"default": "./dist/tools.js"
|
|
36
|
+
},
|
|
37
|
+
"./instructions": {
|
|
38
|
+
"types": "./dist/instructions.d.ts",
|
|
39
|
+
"default": "./dist/instructions.js"
|
|
40
|
+
},
|
|
41
|
+
"./package.json": "./package.json"
|
|
42
|
+
},
|
|
43
|
+
"files": [
|
|
44
|
+
"dist"
|
|
45
|
+
],
|
|
46
|
+
"engines": {
|
|
47
|
+
"node": ">=22"
|
|
48
|
+
},
|
|
49
|
+
"scripts": {
|
|
50
|
+
"build": "tsc && node scripts/copy-assets.mjs",
|
|
51
|
+
"vendor:openapi": "node scripts/vendor-openapi.mjs",
|
|
52
|
+
"test": "npm run build && node --test \"test/**/*.test.js\"",
|
|
53
|
+
"prepublishOnly": "npm run build"
|
|
54
|
+
},
|
|
55
|
+
"dependencies": {
|
|
56
|
+
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
57
|
+
"zod": "^4.6.5"
|
|
58
|
+
},
|
|
59
|
+
"devDependencies": {
|
|
60
|
+
"@types/node": "^22.10.0",
|
|
61
|
+
"typescript": "^5.6.0"
|
|
62
|
+
}
|
|
63
|
+
}
|