squally-mcp 0.7.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +321 -321
- package/dist/instructions.js +6 -3
- package/dist/openapi/v1.json +21 -21
- package/dist/tools.js +6 -6
- package/package.json +63 -63
package/README.md
CHANGED
|
@@ -1,321 +1,321 @@
|
|
|
1
|
-
# squally-mcp
|
|
2
|
-
|
|
3
|
-
A local [MCP](https://modelcontextprotocol.io) server that lets a coding agent
|
|
4
|
-
read your [Squally](https://app.squally.dev) data: which runs happened, how
|
|
5
|
-
stable each test is, and why a particular test failed.
|
|
6
|
-
|
|
7
|
-
It runs on your machine over stdio and talks to Squally's read API over HTTPS.
|
|
8
|
-
Point Claude Code, Codex, Cursor or Claude Desktop at it and ask "why is
|
|
9
|
-
`checkout.spec.ts` failing on main?".
|
|
10
|
-
|
|
11
|
-
## Read-only, and structurally so
|
|
12
|
-
|
|
13
|
-
The server issues **HTTP GETs and nothing else**. There is no code path that
|
|
14
|
-
writes, and every tool is annotated `readOnlyHint: true`, so a client can run
|
|
15
|
-
them without asking you to confirm each one.
|
|
16
|
-
|
|
17
|
-
It also cannot reach past your own organization: the key is scoped to one
|
|
18
|
-
organization, and scoping, plan checks and secret exclusion are all enforced
|
|
19
|
-
by the API, not by this client.
|
|
20
|
-
|
|
21
|
-
## What an agent can see
|
|
22
|
-
|
|
23
|
-
Worth knowing before you hand a key to an assistant. Through this server an
|
|
24
|
-
agent can read:
|
|
25
|
-
|
|
26
|
-
- your **projects** and their names;
|
|
27
|
-
- your **runs**, CI and local — commit SHA and message, branch, author, pull
|
|
28
|
-
request, job key, a link to the CI run, timing, pass/fail counts;
|
|
29
|
-
- **per-test results** for a run, including which shard ran what;
|
|
30
|
-
- for a failing attempt: the **error message and stack**, the **code snippet**
|
|
31
|
-
— **verbatim source code from your test file** — and the **full-context
|
|
32
|
-
prompt** (the text behind Copy the full context in Squally), which carries the same snippet plus an **ARIA snapshot of your
|
|
33
|
-
application at the moment of failure** (whatever was on screen, truncated to
|
|
34
|
-
3000 characters);
|
|
35
|
-
- **per-test metrics**: how often each test passed on the first try, passed
|
|
36
|
-
only after a retry, or failed, and how much time retries and failures cost;
|
|
37
|
-
- **error signatures**: failures grouped across runs and branches.
|
|
38
|
-
|
|
39
|
-
It cannot see screenshots, videos or traces — artifacts are deliberately out of
|
|
40
|
-
scope — and it cannot see any key, secret or webhook URL.
|
|
41
|
-
|
|
42
|
-
The key reaches **every project in the organization**. Create one per machine so
|
|
43
|
-
a single revocation does not lock out everything, and revoke it under
|
|
44
|
-
Organization › Read keys when the machine is retired.
|
|
45
|
-
|
|
46
|
-
## Setup
|
|
47
|
-
|
|
48
|
-
You need an **organization read key** (`sqly_ro_…`): Squally → Organization ›
|
|
49
|
-
Read keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
50
|
-
project ingest key your Playwright reporter uses — this server refuses that one
|
|
51
|
-
by name.
|
|
52
|
-
|
|
53
|
-
Requires **Node 22 or newer**. Read keys are available from the **Standard**
|
|
54
|
-
plan.
|
|
55
|
-
|
|
56
|
-
### Claude Code
|
|
57
|
-
|
|
58
|
-
```bash
|
|
59
|
-
claude mcp add squally --scope user \
|
|
60
|
-
--env SQUALLY_API_KEY=sqly_ro_your_key_here \
|
|
61
|
-
-- npx -y squally-mcp
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
### Codex
|
|
65
|
-
|
|
66
|
-
`~/.codex/config.toml`:
|
|
67
|
-
|
|
68
|
-
```toml
|
|
69
|
-
[mcp_servers.squally]
|
|
70
|
-
command = "npx"
|
|
71
|
-
args = ["-y", "squally-mcp"]
|
|
72
|
-
env = { SQUALLY_API_KEY = "sqly_ro_your_key_here" }
|
|
73
|
-
```
|
|
74
|
-
|
|
75
|
-
### Cursor
|
|
76
|
-
|
|
77
|
-
`~/.cursor/mcp.json` (or `.cursor/mcp.json` in a project):
|
|
78
|
-
|
|
79
|
-
```json
|
|
80
|
-
{
|
|
81
|
-
"mcpServers": {
|
|
82
|
-
"squally": {
|
|
83
|
-
"command": "npx",
|
|
84
|
-
"args": ["-y", "squally-mcp"],
|
|
85
|
-
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
86
|
-
}
|
|
87
|
-
}
|
|
88
|
-
}
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
### Claude Desktop
|
|
92
|
-
|
|
93
|
-
`claude_desktop_config.json` — macOS:
|
|
94
|
-
`~/Library/Application Support/Claude/`, Windows: `%APPDATA%\Claude\` — then
|
|
95
|
-
restart the app:
|
|
96
|
-
|
|
97
|
-
```json
|
|
98
|
-
{
|
|
99
|
-
"mcpServers": {
|
|
100
|
-
"squally": {
|
|
101
|
-
"command": "npx",
|
|
102
|
-
"args": ["-y", "squally-mcp"],
|
|
103
|
-
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
104
|
-
}
|
|
105
|
-
}
|
|
106
|
-
}
|
|
107
|
-
```
|
|
108
|
-
|
|
109
|
-
## Environment
|
|
110
|
-
|
|
111
|
-
| Variable | Required | Meaning |
|
|
112
|
-
|---|---|---|
|
|
113
|
-
| `SQUALLY_API_KEY` | yes | Organization read key, `sqly_ro_…`. Missing or an ingest key by mistake, and the server exits with one line saying so. |
|
|
114
|
-
| `SQUALLY_API_URL` | no | Base URL, default `https://app.squally.dev`. A trailing `/` or `/api/v1` is tolerated. Useful against a local Squally: `http://localhost:3000`. |
|
|
115
|
-
| `SQUALLY_USE_SYSTEM_CA` | no | `0` keeps Node's bundled certificates only. |
|
|
116
|
-
|
|
117
|
-
squally-mcp trusts your operating system's certificate store, like your browser does; set SQUALLY_USE_SYSTEM_CA=0 to use Node's bundled certificates only.
|
|
118
|
-
|
|
119
|
-
## Troubleshooting
|
|
120
|
-
|
|
121
|
-
### Claude Desktop
|
|
122
|
-
|
|
123
|
-
- **The server does not start in time.** The first start through `npx`
|
|
124
|
-
downloads the package, which can take longer than Claude Desktop's 60-second
|
|
125
|
-
start-up limit. Install it once beforehand with `npm install -g squally-mcp`;
|
|
126
|
-
`npx -y squally-mcp` then runs that copy without a download. Update it with
|
|
127
|
-
the same command.
|
|
128
|
-
- **Where the log is.** The Microsoft Store build of Claude Desktop writes it to
|
|
129
|
-
`%LOCALAPPDATA%\Claude\logs\mcp-server-squally.log`. Its first lines say which
|
|
130
|
-
Node runs the server and whether it trusts the system's certificate store;
|
|
131
|
-
a request that never arrives is logged there with its cause.
|
|
132
|
-
- **Edit the config only while the app is closed.** Quit Claude Desktop
|
|
133
|
-
completely, including from the system tray, before changing
|
|
134
|
-
`claude_desktop_config.json`.
|
|
135
|
-
|
|
136
|
-
## The tools
|
|
137
|
-
|
|
138
|
-
| Tool | What it answers | Cost |
|
|
139
|
-
|---|---|---|
|
|
140
|
-
| `squally-list-projects` | Which projects exist. **Start here** — every other tool needs a `projectId` from it. | cheap |
|
|
141
|
-
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA, status or source. Counters only. | cheap |
|
|
142
|
-
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
143
|
-
| `squally-debug-failure` | Every attempt of one test in one run: error, stack, full-context prompt. | cheap |
|
|
144
|
-
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
145
|
-
| `squally-list-tests` | Every test with a
|
|
146
|
-
| `squally-list-errors` | Error signatures in a period: what keeps failing, grouped, and how its runs ended. | cheap |
|
|
147
|
-
|
|
148
|
-
For a single test use `squally-get-test-metrics`, not `squally-list-tests` —
|
|
149
|
-
the tool descriptions say so, and the server repeats it in its instructions.
|
|
150
|
-
|
|
151
|
-
### Runs
|
|
152
|
-
|
|
153
|
-
`squally-find-run` lists every run, CI and local alike; `source` (`all` by
|
|
154
|
-
default, `ci` or `local`) narrows it, and each run says whether it was
|
|
155
|
-
`local`. A run's `status` is the dashboard's word: `passed`, `failed`,
|
|
156
|
-
`cancelled`, `incomplete` (a shard timed out, was stopped by the monthly
|
|
157
|
-
result limit or never reported) or `running` - and `?status=` filters by
|
|
158
|
-
any of them. A cancelled run is `cancelled`; `cancellation` says how, when
|
|
159
|
-
and by whom. A sharded run is one row, its counters over all its shards with
|
|
160
|
-
every test once, and the tests a cancel cut off count nowhere.
|
|
161
|
-
|
|
162
|
-
Each run also says whether it is `settled` - final and counted; an
|
|
163
|
-
`incomplete` run that is not settled still waits for a shard within the run
|
|
164
|
-
timeout - and, from squally-reporter 0.11.0, which job of its CI run it is
|
|
165
|
-
(`jobKey`, from `SQUALLY_CI_JOB_KEY`; each job is its own run), which
|
|
166
|
-
`attempt`, which run it re-ran (`rerunOf`) and where the CI run is
|
|
167
|
-
(`ciRunUrl`; for GitHub Actions also for older reporters when the project
|
|
168
|
-
has one linked repository).
|
|
169
|
-
|
|
170
|
-
### What the numbers mean
|
|
171
|
-
|
|
172
|
-
The two test tools count **finished CI
|
|
173
|
-
days; 30 by default), on every branch unless you name one. A run finishes when
|
|
174
|
-
all its shards have - or, once the run timeout has passed, with the results its
|
|
175
|
-
shards reported, like a cancelled run; a shard stopped by the monthly result
|
|
176
|
-
limit counts like one that timed out. A sharded run counts once, placed in the period
|
|
177
|
-
by its first shard's start; a run that was a mass failure as a whole is left
|
|
178
|
-
out.
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
- **stability** — runs that passed on the first try ÷ runs;
|
|
183
|
-
- **flakyRate** — runs that passed only after a retry ÷ runs;
|
|
184
|
-
- **failureRate** — failed runs ÷ runs.
|
|
185
|
-
|
|
186
|
-
Each is a fraction from 0 to 1, and `null` when the test has no run in the
|
|
187
|
-
period. **There is no verdict**: nothing calls a test flaky, broken or
|
|
188
|
-
healthy. The tools return the numbers; the agent — or you — judges them.
|
|
189
|
-
|
|
190
|
-
### Example: `squally-list-tests`
|
|
191
|
-
|
|
192
|
-
```json
|
|
193
|
-
{ "projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15", "perPage": 3 }
|
|
194
|
-
```
|
|
195
|
-
|
|
196
|
-
The three least stable tests of the last 30 days (abridged):
|
|
197
|
-
|
|
198
|
-
```json
|
|
199
|
-
{
|
|
200
|
-
"population": "ci_groups_finished_not_mass_failure",
|
|
201
|
-
"days": 30,
|
|
202
|
-
"sort": "stability",
|
|
203
|
-
"total": 409,
|
|
204
|
-
"items": [
|
|
205
|
-
{
|
|
206
|
-
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
207
|
-
"filePath": "tests/checkout.spec.ts",
|
|
208
|
-
"runs": 8,
|
|
209
|
-
"stability": 0.5,
|
|
210
|
-
"flakyRate": 0.5,
|
|
211
|
-
"failureRate": 0,
|
|
212
|
-
"timeLostMs": 7361,
|
|
213
|
-
"topBranch": { "branch": "main", "kind": "retries", "count": 3 }
|
|
214
|
-
}
|
|
215
|
-
]
|
|
216
|
-
}
|
|
217
|
-
```
|
|
218
|
-
|
|
219
|
-
### Example: `squally-get-test-metrics`
|
|
220
|
-
|
|
221
|
-
```json
|
|
222
|
-
{
|
|
223
|
-
"projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15",
|
|
224
|
-
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code"
|
|
225
|
-
}
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
The same numbers as that test's row, for one test (abridged):
|
|
229
|
-
|
|
230
|
-
```json
|
|
231
|
-
{
|
|
232
|
-
"days": 30,
|
|
233
|
-
"branch": null,
|
|
234
|
-
"runs": 8,
|
|
235
|
-
"stableRuns": 4,
|
|
236
|
-
"flakyRuns": 4,
|
|
237
|
-
"failedRuns": 0,
|
|
238
|
-
"stability": 0.5,
|
|
239
|
-
"flakyRate": 0.5,
|
|
240
|
-
"failureRate": 0,
|
|
241
|
-
"branches": ["feature/saved-carts", "main"]
|
|
242
|
-
}
|
|
243
|
-
```
|
|
244
|
-
|
|
245
|
-
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
246
|
-
files, the tool answers with the `filePath` values to retry with; a test with
|
|
247
|
-
results but no counted run in the period answers `runs: 0` and `null`
|
|
248
|
-
rates — an answer, not an error.
|
|
249
|
-
|
|
250
|
-
## Development
|
|
251
|
-
|
|
252
|
-
```bash
|
|
253
|
-
npm install
|
|
254
|
-
npm run build # tsc, then copy the vendored OpenAPI document into dist/
|
|
255
|
-
npm test # builds, then runs the suite
|
|
256
|
-
npm run vendor:openapi # re-fetch the API document; review the diff, then re-pin
|
|
257
|
-
```
|
|
258
|
-
|
|
259
|
-
The tool schemas are **derived from Squally's published OpenAPI document**,
|
|
260
|
-
vendored at `src/openapi/v1.json` — an input schema is the operation's
|
|
261
|
-
parameters, an output schema is its 200 response. Nothing is transcribed by
|
|
262
|
-
hand, so the two cannot drift. `test/drift.test.js` compares the vendored copy
|
|
263
|
-
against the live document and fails when an operation or parameter has moved;
|
|
264
|
-
it skips loudly when offline, and while the live API is still an older version
|
|
265
|
-
than the vendored one — a release that has not been deployed yet.
|
|
266
|
-
|
|
267
|
-
## Releasing
|
|
268
|
-
|
|
269
|
-
Publishing is **staged**: CI uploads the tarball, a maintainer approves it, and
|
|
270
|
-
only then is the version installable. `npm stage publish` never asks for 2FA,
|
|
271
|
-
which is what makes it usable from a workflow; the 2FA prompt moves to the
|
|
272
|
-
approval step, so a compromised workflow can stage a version but cannot put one
|
|
273
|
-
in front of users. Direct `npm publish` is not permitted for this package's
|
|
274
|
-
trusted publisher, so this is the only route.
|
|
275
|
-
|
|
276
|
-
> **0.1.0 is the exception.** It was published by hand with `npm publish`
|
|
277
|
-
> before the trusted publisher existed, so it carries no provenance
|
|
278
|
-
> attestation. Every version from 0.1.1 on goes through the steps below.
|
|
279
|
-
|
|
280
|
-
1. **Bump and tag.** The lockfile belongs to the release commit, so let `npm
|
|
281
|
-
version` write both and commit them together:
|
|
282
|
-
|
|
283
|
-
```bash
|
|
284
|
-
npm install # only if package.json changed by hand
|
|
285
|
-
npm version patch # or minor / major - writes package.json + lockfile, makes the tag
|
|
286
|
-
git push --follow-tags
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
2. **The tag starts the workflow.** `.github/workflows/publish.yml` runs on
|
|
290
|
-
`v*`: it tests, checks that the tag and `package.json` agree, and runs
|
|
291
|
-
|
|
292
|
-
```bash
|
|
293
|
-
npm stage publish --provenance --access public
|
|
294
|
-
```
|
|
295
|
-
|
|
296
|
-
through npm's trusted publishing (OIDC) — no npm token in GitHub secrets.
|
|
297
|
-
`--provenance` attaches a signed statement linking the tarball to that
|
|
298
|
-
workflow run and commit.
|
|
299
|
-
|
|
300
|
-
3. **Approve the staged version.** On
|
|
301
|
-
[npmjs.com/package/squally-mcp](https://www.npmjs.com/package/squally-mcp) →
|
|
302
|
-
**Staged Packages** → **Approve**, which asks for 2FA. Or from a terminal:
|
|
303
|
-
|
|
304
|
-
```bash
|
|
305
|
-
npm stage list squally-mcp
|
|
306
|
-
npm stage view <stage-id> # what is in the tarball
|
|
307
|
-
npm stage approve <stage-id> # also asks for 2FA
|
|
308
|
-
npm stage reject <stage-id> # if something is wrong
|
|
309
|
-
```
|
|
310
|
-
|
|
311
|
-
Until this step the version exists in the registry but installs nothing.
|
|
312
|
-
|
|
313
|
-
**Requirements for step 3 on your machine:** npm **11.15.0 or later** and Node
|
|
314
|
-
**22.14.0 or higher** ([npm docs](https://docs.npmjs.com/staged-publishing/)).
|
|
315
|
-
Node 22 still bundles npm 10.9.9, so check `npm --version` rather than assuming
|
|
316
|
-
your Node version brought a new enough npm. The workflow uses Node 24 for the
|
|
317
|
-
same reason and fails with that requirement if a runner ever ships an older npm.
|
|
318
|
-
|
|
319
|
-
## License
|
|
320
|
-
|
|
321
|
-
MIT
|
|
1
|
+
# squally-mcp
|
|
2
|
+
|
|
3
|
+
A local [MCP](https://modelcontextprotocol.io) server that lets a coding agent
|
|
4
|
+
read your [Squally](https://app.squally.dev) data: which runs happened, how
|
|
5
|
+
stable each test is, and why a particular test failed.
|
|
6
|
+
|
|
7
|
+
It runs on your machine over stdio and talks to Squally's read API over HTTPS.
|
|
8
|
+
Point Claude Code, Codex, Cursor or Claude Desktop at it and ask "why is
|
|
9
|
+
`checkout.spec.ts` failing on main?".
|
|
10
|
+
|
|
11
|
+
## Read-only, and structurally so
|
|
12
|
+
|
|
13
|
+
The server issues **HTTP GETs and nothing else**. There is no code path that
|
|
14
|
+
writes, and every tool is annotated `readOnlyHint: true`, so a client can run
|
|
15
|
+
them without asking you to confirm each one.
|
|
16
|
+
|
|
17
|
+
It also cannot reach past your own organization: the key is scoped to one
|
|
18
|
+
organization, and scoping, plan checks and secret exclusion are all enforced
|
|
19
|
+
by the API, not by this client.
|
|
20
|
+
|
|
21
|
+
## What an agent can see
|
|
22
|
+
|
|
23
|
+
Worth knowing before you hand a key to an assistant. Through this server an
|
|
24
|
+
agent can read:
|
|
25
|
+
|
|
26
|
+
- your **projects** and their names;
|
|
27
|
+
- your **runs**, CI and local — commit SHA and message, branch, author, pull
|
|
28
|
+
request, job key, a link to the CI run, timing, pass/fail counts;
|
|
29
|
+
- **per-test results** for a run, including which shard ran what;
|
|
30
|
+
- for a failing attempt: the **error message and stack**, the **code snippet**
|
|
31
|
+
— **verbatim source code from your test file** — and the **full-context
|
|
32
|
+
prompt** (the text behind Copy the full context in Squally), which carries the same snippet plus an **ARIA snapshot of your
|
|
33
|
+
application at the moment of failure** (whatever was on screen, truncated to
|
|
34
|
+
3000 characters);
|
|
35
|
+
- **per-test metrics**: how often each test passed on the first try, passed
|
|
36
|
+
only after a retry, or failed, and how much time retries and failures cost;
|
|
37
|
+
- **error signatures**: failures grouped across runs and branches.
|
|
38
|
+
|
|
39
|
+
It cannot see screenshots, videos or traces — artifacts are deliberately out of
|
|
40
|
+
scope — and it cannot see any key, secret or webhook URL.
|
|
41
|
+
|
|
42
|
+
The key reaches **every project in the organization**. Create one per machine so
|
|
43
|
+
a single revocation does not lock out everything, and revoke it under
|
|
44
|
+
Organization › Read keys when the machine is retired.
|
|
45
|
+
|
|
46
|
+
## Setup
|
|
47
|
+
|
|
48
|
+
You need an **organization read key** (`sqly_ro_…`): Squally → Organization ›
|
|
49
|
+
Read keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
50
|
+
project ingest key your Playwright reporter uses — this server refuses that one
|
|
51
|
+
by name.
|
|
52
|
+
|
|
53
|
+
Requires **Node 22 or newer**. Read keys are available from the **Standard**
|
|
54
|
+
plan.
|
|
55
|
+
|
|
56
|
+
### Claude Code
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
claude mcp add squally --scope user \
|
|
60
|
+
--env SQUALLY_API_KEY=sqly_ro_your_key_here \
|
|
61
|
+
-- npx -y squally-mcp
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### Codex
|
|
65
|
+
|
|
66
|
+
`~/.codex/config.toml`:
|
|
67
|
+
|
|
68
|
+
```toml
|
|
69
|
+
[mcp_servers.squally]
|
|
70
|
+
command = "npx"
|
|
71
|
+
args = ["-y", "squally-mcp"]
|
|
72
|
+
env = { SQUALLY_API_KEY = "sqly_ro_your_key_here" }
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Cursor
|
|
76
|
+
|
|
77
|
+
`~/.cursor/mcp.json` (or `.cursor/mcp.json` in a project):
|
|
78
|
+
|
|
79
|
+
```json
|
|
80
|
+
{
|
|
81
|
+
"mcpServers": {
|
|
82
|
+
"squally": {
|
|
83
|
+
"command": "npx",
|
|
84
|
+
"args": ["-y", "squally-mcp"],
|
|
85
|
+
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
### Claude Desktop
|
|
92
|
+
|
|
93
|
+
`claude_desktop_config.json` — macOS:
|
|
94
|
+
`~/Library/Application Support/Claude/`, Windows: `%APPDATA%\Claude\` — then
|
|
95
|
+
restart the app:
|
|
96
|
+
|
|
97
|
+
```json
|
|
98
|
+
{
|
|
99
|
+
"mcpServers": {
|
|
100
|
+
"squally": {
|
|
101
|
+
"command": "npx",
|
|
102
|
+
"args": ["-y", "squally-mcp"],
|
|
103
|
+
"env": { "SQUALLY_API_KEY": "sqly_ro_your_key_here" }
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
}
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
## Environment
|
|
110
|
+
|
|
111
|
+
| Variable | Required | Meaning |
|
|
112
|
+
|---|---|---|
|
|
113
|
+
| `SQUALLY_API_KEY` | yes | Organization read key, `sqly_ro_…`. Missing or an ingest key by mistake, and the server exits with one line saying so. |
|
|
114
|
+
| `SQUALLY_API_URL` | no | Base URL, default `https://app.squally.dev`. A trailing `/` or `/api/v1` is tolerated. Useful against a local Squally: `http://localhost:3000`. |
|
|
115
|
+
| `SQUALLY_USE_SYSTEM_CA` | no | `0` keeps Node's bundled certificates only. |
|
|
116
|
+
|
|
117
|
+
squally-mcp trusts your operating system's certificate store, like your browser does; set SQUALLY_USE_SYSTEM_CA=0 to use Node's bundled certificates only.
|
|
118
|
+
|
|
119
|
+
## Troubleshooting
|
|
120
|
+
|
|
121
|
+
### Claude Desktop
|
|
122
|
+
|
|
123
|
+
- **The server does not start in time.** The first start through `npx`
|
|
124
|
+
downloads the package, which can take longer than Claude Desktop's 60-second
|
|
125
|
+
start-up limit. Install it once beforehand with `npm install -g squally-mcp`;
|
|
126
|
+
`npx -y squally-mcp` then runs that copy without a download. Update it with
|
|
127
|
+
the same command.
|
|
128
|
+
- **Where the log is.** The Microsoft Store build of Claude Desktop writes it to
|
|
129
|
+
`%LOCALAPPDATA%\Claude\logs\mcp-server-squally.log`. Its first lines say which
|
|
130
|
+
Node runs the server and whether it trusts the system's certificate store;
|
|
131
|
+
a request that never arrives is logged there with its cause.
|
|
132
|
+
- **Edit the config only while the app is closed.** Quit Claude Desktop
|
|
133
|
+
completely, including from the system tray, before changing
|
|
134
|
+
`claude_desktop_config.json`.
|
|
135
|
+
|
|
136
|
+
## The tools
|
|
137
|
+
|
|
138
|
+
| Tool | What it answers | Cost |
|
|
139
|
+
|---|---|---|
|
|
140
|
+
| `squally-list-projects` | Which projects exist. **Start here** — every other tool needs a `projectId` from it. | cheap |
|
|
141
|
+
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA, status or source. Counters only. | cheap |
|
|
142
|
+
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
143
|
+
| `squally-debug-failure` | Every attempt of one test in one run: error, stack, full-context prompt. | cheap |
|
|
144
|
+
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
145
|
+
| `squally-list-tests` | Every test with a run in the period, ranked by stability, flaky rate or failures. | cheap — two queries, paged |
|
|
146
|
+
| `squally-list-errors` | Error signatures in a period: what keeps failing, grouped, and how its runs ended. | cheap |
|
|
147
|
+
|
|
148
|
+
For a single test use `squally-get-test-metrics`, not `squally-list-tests` —
|
|
149
|
+
the tool descriptions say so, and the server repeats it in its instructions.
|
|
150
|
+
|
|
151
|
+
### Runs
|
|
152
|
+
|
|
153
|
+
`squally-find-run` lists every run, CI and local alike; `source` (`all` by
|
|
154
|
+
default, `ci` or `local`) narrows it, and each run says whether it was
|
|
155
|
+
`local`. A run's `status` is the dashboard's word: `passed`, `failed`,
|
|
156
|
+
`cancelled`, `incomplete` (a shard timed out, was stopped by the monthly
|
|
157
|
+
result limit or never reported) or `running` - and `?status=` filters by
|
|
158
|
+
any of them. A cancelled run is `cancelled`; `cancellation` says how, when
|
|
159
|
+
and by whom. A sharded run is one row, its counters over all its shards with
|
|
160
|
+
every test once, and the tests a cancel cut off count nowhere.
|
|
161
|
+
|
|
162
|
+
Each run also says whether it is `settled` - final and counted; an
|
|
163
|
+
`incomplete` run that is not settled still waits for a shard within the run
|
|
164
|
+
timeout - and, from squally-reporter 0.11.0, which job of its CI run it is
|
|
165
|
+
(`jobKey`, from `SQUALLY_CI_JOB_KEY`; each job is its own run), which
|
|
166
|
+
`attempt`, which run it re-ran (`rerunOf`) and where the CI run is
|
|
167
|
+
(`ciRunUrl`; for GitHub Actions also for older reporters when the project
|
|
168
|
+
has one linked repository).
|
|
169
|
+
|
|
170
|
+
### What the numbers mean
|
|
171
|
+
|
|
172
|
+
The two test tools count **finished runs, CI and local,** in the period (7, 14, 30 or 90
|
|
173
|
+
days; 30 by default), on every branch unless you name one. A run finishes when
|
|
174
|
+
all its shards have - or, once the run timeout has passed, with the results its
|
|
175
|
+
shards reported, like a cancelled run; a shard stopped by the monthly result
|
|
176
|
+
limit counts like one that timed out. A sharded run counts once, placed in the period
|
|
177
|
+
by its first shard's start; a run that was a mass failure as a whole is left
|
|
178
|
+
out. `source` is `all` (the default since read API 1.0.0-beta.18; `ci`
|
|
179
|
+
before), `ci` or `local` - send `source=ci` for CI runs only - and the
|
|
180
|
+
response's `population` names which runs it counted. Per test:
|
|
181
|
+
|
|
182
|
+
- **stability** — runs that passed on the first try ÷ runs;
|
|
183
|
+
- **flakyRate** — runs that passed only after a retry ÷ runs;
|
|
184
|
+
- **failureRate** — failed runs ÷ runs.
|
|
185
|
+
|
|
186
|
+
Each is a fraction from 0 to 1, and `null` when the test has no run in the
|
|
187
|
+
period. **There is no verdict**: nothing calls a test flaky, broken or
|
|
188
|
+
healthy. The tools return the numbers; the agent — or you — judges them.
|
|
189
|
+
|
|
190
|
+
### Example: `squally-list-tests`
|
|
191
|
+
|
|
192
|
+
```json
|
|
193
|
+
{ "projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15", "perPage": 3 }
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
The three least stable tests of the last 30 days (abridged):
|
|
197
|
+
|
|
198
|
+
```json
|
|
199
|
+
{
|
|
200
|
+
"population": "ci_groups_finished_not_mass_failure",
|
|
201
|
+
"days": 30,
|
|
202
|
+
"sort": "stability",
|
|
203
|
+
"total": 409,
|
|
204
|
+
"items": [
|
|
205
|
+
{
|
|
206
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
207
|
+
"filePath": "tests/checkout.spec.ts",
|
|
208
|
+
"runs": 8,
|
|
209
|
+
"stability": 0.5,
|
|
210
|
+
"flakyRate": 0.5,
|
|
211
|
+
"failureRate": 0,
|
|
212
|
+
"timeLostMs": 7361,
|
|
213
|
+
"topBranch": { "branch": "main", "kind": "retries", "count": 3 }
|
|
214
|
+
}
|
|
215
|
+
]
|
|
216
|
+
}
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
### Example: `squally-get-test-metrics`
|
|
220
|
+
|
|
221
|
+
```json
|
|
222
|
+
{
|
|
223
|
+
"projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15",
|
|
224
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code"
|
|
225
|
+
}
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
The same numbers as that test's row, for one test (abridged):
|
|
229
|
+
|
|
230
|
+
```json
|
|
231
|
+
{
|
|
232
|
+
"days": 30,
|
|
233
|
+
"branch": null,
|
|
234
|
+
"runs": 8,
|
|
235
|
+
"stableRuns": 4,
|
|
236
|
+
"flakyRuns": 4,
|
|
237
|
+
"failedRuns": 0,
|
|
238
|
+
"stability": 0.5,
|
|
239
|
+
"flakyRate": 0.5,
|
|
240
|
+
"failureRate": 0,
|
|
241
|
+
"branches": ["feature/saved-carts", "main"]
|
|
242
|
+
}
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
246
|
+
files, the tool answers with the `filePath` values to retry with; a test with
|
|
247
|
+
results but no counted run in the period answers `runs: 0` and `null`
|
|
248
|
+
rates — an answer, not an error.
|
|
249
|
+
|
|
250
|
+
## Development
|
|
251
|
+
|
|
252
|
+
```bash
|
|
253
|
+
npm install
|
|
254
|
+
npm run build # tsc, then copy the vendored OpenAPI document into dist/
|
|
255
|
+
npm test # builds, then runs the suite
|
|
256
|
+
npm run vendor:openapi # re-fetch the API document; review the diff, then re-pin
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
The tool schemas are **derived from Squally's published OpenAPI document**,
|
|
260
|
+
vendored at `src/openapi/v1.json` — an input schema is the operation's
|
|
261
|
+
parameters, an output schema is its 200 response. Nothing is transcribed by
|
|
262
|
+
hand, so the two cannot drift. `test/drift.test.js` compares the vendored copy
|
|
263
|
+
against the live document and fails when an operation or parameter has moved;
|
|
264
|
+
it skips loudly when offline, and while the live API is still an older version
|
|
265
|
+
than the vendored one — a release that has not been deployed yet.
|
|
266
|
+
|
|
267
|
+
## Releasing
|
|
268
|
+
|
|
269
|
+
Publishing is **staged**: CI uploads the tarball, a maintainer approves it, and
|
|
270
|
+
only then is the version installable. `npm stage publish` never asks for 2FA,
|
|
271
|
+
which is what makes it usable from a workflow; the 2FA prompt moves to the
|
|
272
|
+
approval step, so a compromised workflow can stage a version but cannot put one
|
|
273
|
+
in front of users. Direct `npm publish` is not permitted for this package's
|
|
274
|
+
trusted publisher, so this is the only route.
|
|
275
|
+
|
|
276
|
+
> **0.1.0 is the exception.** It was published by hand with `npm publish`
|
|
277
|
+
> before the trusted publisher existed, so it carries no provenance
|
|
278
|
+
> attestation. Every version from 0.1.1 on goes through the steps below.
|
|
279
|
+
|
|
280
|
+
1. **Bump and tag.** The lockfile belongs to the release commit, so let `npm
|
|
281
|
+
version` write both and commit them together:
|
|
282
|
+
|
|
283
|
+
```bash
|
|
284
|
+
npm install # only if package.json changed by hand
|
|
285
|
+
npm version patch # or minor / major - writes package.json + lockfile, makes the tag
|
|
286
|
+
git push --follow-tags
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
2. **The tag starts the workflow.** `.github/workflows/publish.yml` runs on
|
|
290
|
+
`v*`: it tests, checks that the tag and `package.json` agree, and runs
|
|
291
|
+
|
|
292
|
+
```bash
|
|
293
|
+
npm stage publish --provenance --access public
|
|
294
|
+
```
|
|
295
|
+
|
|
296
|
+
through npm's trusted publishing (OIDC) — no npm token in GitHub secrets.
|
|
297
|
+
`--provenance` attaches a signed statement linking the tarball to that
|
|
298
|
+
workflow run and commit.
|
|
299
|
+
|
|
300
|
+
3. **Approve the staged version.** On
|
|
301
|
+
[npmjs.com/package/squally-mcp](https://www.npmjs.com/package/squally-mcp) →
|
|
302
|
+
**Staged Packages** → **Approve**, which asks for 2FA. Or from a terminal:
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
npm stage list squally-mcp
|
|
306
|
+
npm stage view <stage-id> # what is in the tarball
|
|
307
|
+
npm stage approve <stage-id> # also asks for 2FA
|
|
308
|
+
npm stage reject <stage-id> # if something is wrong
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
Until this step the version exists in the registry but installs nothing.
|
|
312
|
+
|
|
313
|
+
**Requirements for step 3 on your machine:** npm **11.15.0 or later** and Node
|
|
314
|
+
**22.14.0 or higher** ([npm docs](https://docs.npmjs.com/staged-publishing/)).
|
|
315
|
+
Node 22 still bundles npm 10.9.9, so check `npm --version` rather than assuming
|
|
316
|
+
your Node version brought a new enough npm. The workflow uses Node 24 for the
|
|
317
|
+
same reason and fails with that requirement if a runner ever ships an older npm.
|
|
318
|
+
|
|
319
|
+
## License
|
|
320
|
+
|
|
321
|
+
MIT
|
package/dist/instructions.js
CHANGED
|
@@ -23,6 +23,9 @@
|
|
|
23
23
|
//
|
|
24
24
|
// THE SOURCE SINCE 0.4.0 (API 1.0.0-beta.8): CI runs by default; local runs
|
|
25
25
|
// count only when the agent asks for them - they were never counted before.
|
|
26
|
+
// ALL BY DEFAULT SINCE 0.8.0 (API 1.0.0-beta.18, 23.10.2026): CI and local
|
|
27
|
+
// runs both count unless the agent sends source=ci or source=local - the
|
|
28
|
+
// API's default moved, and the sentence follows it.
|
|
26
29
|
//
|
|
27
30
|
// THE RUN TIMEOUT (API 1.0.0-beta.10, since 0.5.0): a run whose shard died or
|
|
28
31
|
// never started counts once the run timeout has passed, with the shards that
|
|
@@ -50,9 +53,9 @@
|
|
|
50
53
|
export const INSTRUCTIONS = [
|
|
51
54
|
"Start with squally-list-projects; every other tool needs a projectId from it.",
|
|
52
55
|
"For one test, use squally-get-test-metrics, not squally-list-tests.",
|
|
53
|
-
"Test metrics count the finished
|
|
54
|
-
"without runs that were a mass failure as a whole; pass source=
|
|
55
|
-
"to count
|
|
56
|
+
"Test metrics count the finished runs of the period, CI and local - a sharded run counts once -",
|
|
57
|
+
"without runs that were a mass failure as a whole; pass source=ci or source=local",
|
|
58
|
+
"to count only one of them. A run still in progress counts once all its shards have",
|
|
56
59
|
"finished, or once the run timeout has passed, with the results its shards reported -",
|
|
57
60
|
"also when a shard was stopped by the monthly result limit.",
|
|
58
61
|
"stability = runs that passed on the",
|
package/dist/openapi/v1.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema",
|
|
4
4
|
"info": {
|
|
5
5
|
"title": "Squally Read API",
|
|
6
|
-
"version": "1.0.0-beta.
|
|
6
|
+
"version": "1.0.0-beta.18",
|
|
7
7
|
"description": "Read access to a Squally organization's test results, for scripts, CI jobs and AI agents.\n\n**Authentication.** Every request carries an organization read key as `Authorization: Bearer sqly_ro_…`. A read key is organization-wide (every project), created by an organization admin in Organization › Read keys, and expires after 30, 60, 90, 180 days as chosen at creation. There is no renewal: create a new key and revoke the old one. Project ingest keys (`sqly_…`) are refused with `wrong_key_type`.\n\n**Expiry warning.** From 7 days before a key expires, every authenticated response carries `Squally-Key-Expires-At` with the expiry instant (ISO-8601).\n\n**Rate limit.** Currently 60 requests per 60 seconds per key, counted per `Authorization` header - a read key has its own budget, shared with nothing else. Over the limit the edge answers 429 without a JSON body.\n\n**Retries.** 429: wait 60 seconds, then retry. The edge counts requests per key in 60-second windows and sends no Retry-After header. 5xx: retry with exponential backoff, starting at 1 second and capped at 30 seconds. 400, 401, 403, 404: never retry - the same request gets the same answer.\n\n**Plans.** The read API is part of the Standard and Pro plans. A key of an organization on Free is answered 403 `tier_too_low`; keys are not revoked on a downgrade and work again after an upgrade.\n\n**Errors.** Every error is `{\"error\": \"<sentence>\", \"code\": \"<slug>\"}`. The sentence may be reworded; the slug may not. An id that does not exist and an id that belongs to another organization get the same 404, byte for byte.\n\n**Test names in paths.** Test names are Playwright title paths and often contain `/`: URL-encode them, including `/` as `%2F`.\n\n**Sharded runs** are returned as one logical run: /runs lists one row per logical run, and the run routes answer for all shards whichever shard's id you pass.\n\n**Stability.** v1, beta. The API may change until the beta ends; every change is listed in the changelog (docs/openapi/CHANGELOG.md)."
|
|
8
8
|
},
|
|
9
9
|
"servers": [
|
|
@@ -1710,7 +1710,7 @@
|
|
|
1710
1710
|
"get": {
|
|
1711
1711
|
"operationId": "listTests",
|
|
1712
1712
|
"summary": "List tests",
|
|
1713
|
-
"description": "Every test with a run of the source (
|
|
1713
|
+
"description": "Every test with a run of the source (all by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Counts finished runs of the source - all by default - without mass-failure runs, on every branch unless branch names one (see population). Moderate: aggregate queries over the period's results of the source, paged in the database; use the metrics route for one test.",
|
|
1714
1714
|
"tags": [
|
|
1715
1715
|
"Tests"
|
|
1716
1716
|
],
|
|
@@ -1766,7 +1766,7 @@
|
|
|
1766
1766
|
"name": "source",
|
|
1767
1767
|
"in": "query",
|
|
1768
1768
|
"required": false,
|
|
1769
|
-
"description": "Which runs count, by where they ran: ci - runs a shard reported a CI provider for
|
|
1769
|
+
"description": "Which runs count, by where they ran: all - every run (the default since 1.0.0-beta.18; ci before); ci - runs a shard reported a CI provider for; local - runs no shard did. The response's population names the one counted.",
|
|
1770
1770
|
"schema": {
|
|
1771
1771
|
"type": "string",
|
|
1772
1772
|
"enum": [
|
|
@@ -1774,7 +1774,7 @@
|
|
|
1774
1774
|
"local",
|
|
1775
1775
|
"all"
|
|
1776
1776
|
],
|
|
1777
|
-
"default": "
|
|
1777
|
+
"default": "all"
|
|
1778
1778
|
}
|
|
1779
1779
|
},
|
|
1780
1780
|
{
|
|
@@ -2397,7 +2397,7 @@
|
|
|
2397
2397
|
"name": "source",
|
|
2398
2398
|
"in": "query",
|
|
2399
2399
|
"required": false,
|
|
2400
|
-
"description": "Which runs count, by where they ran: ci - runs a shard reported a CI provider for
|
|
2400
|
+
"description": "Which runs count, by where they ran: all - every run (the default since 1.0.0-beta.18; ci before); ci - runs a shard reported a CI provider for; local - runs no shard did. The response's population names the one counted.",
|
|
2401
2401
|
"schema": {
|
|
2402
2402
|
"type": "string",
|
|
2403
2403
|
"enum": [
|
|
@@ -2405,7 +2405,7 @@
|
|
|
2405
2405
|
"local",
|
|
2406
2406
|
"all"
|
|
2407
2407
|
],
|
|
2408
|
-
"default": "
|
|
2408
|
+
"default": "all"
|
|
2409
2409
|
}
|
|
2410
2410
|
}
|
|
2411
2411
|
],
|
|
@@ -2812,7 +2812,7 @@
|
|
|
2812
2812
|
"name": "source",
|
|
2813
2813
|
"in": "query",
|
|
2814
2814
|
"required": false,
|
|
2815
|
-
"description": "Which runs count, by where they ran: ci - runs a shard reported a CI provider for
|
|
2815
|
+
"description": "Which runs count, by where they ran: all - every run (the default since 1.0.0-beta.18; ci before); ci - runs a shard reported a CI provider for; local - runs no shard did. The response's population names the one counted.",
|
|
2816
2816
|
"schema": {
|
|
2817
2817
|
"type": "string",
|
|
2818
2818
|
"enum": [
|
|
@@ -2820,7 +2820,7 @@
|
|
|
2820
2820
|
"local",
|
|
2821
2821
|
"all"
|
|
2822
2822
|
],
|
|
2823
|
-
"default": "
|
|
2823
|
+
"default": "all"
|
|
2824
2824
|
}
|
|
2825
2825
|
}
|
|
2826
2826
|
],
|
|
@@ -3167,7 +3167,7 @@
|
|
|
3167
3167
|
"name": "source",
|
|
3168
3168
|
"in": "query",
|
|
3169
3169
|
"required": false,
|
|
3170
|
-
"description": "Which runs count, by where they ran: ci - runs a shard reported a CI provider for
|
|
3170
|
+
"description": "Which runs count, by where they ran: all - every run (the default since 1.0.0-beta.18; ci before); ci - runs a shard reported a CI provider for; local - runs no shard did. The response's population names the one counted.",
|
|
3171
3171
|
"schema": {
|
|
3172
3172
|
"type": "string",
|
|
3173
3173
|
"enum": [
|
|
@@ -3175,7 +3175,7 @@
|
|
|
3175
3175
|
"local",
|
|
3176
3176
|
"all"
|
|
3177
3177
|
],
|
|
3178
|
-
"default": "
|
|
3178
|
+
"default": "all"
|
|
3179
3179
|
}
|
|
3180
3180
|
},
|
|
3181
3181
|
{
|
|
@@ -3536,7 +3536,7 @@
|
|
|
3536
3536
|
"name": "source",
|
|
3537
3537
|
"in": "query",
|
|
3538
3538
|
"required": false,
|
|
3539
|
-
"description": "Which runs count, by where they ran: ci - runs a shard reported a CI provider for
|
|
3539
|
+
"description": "Which runs count, by where they ran: all - every run (the default since 1.0.0-beta.18; ci before); ci - runs a shard reported a CI provider for; local - runs no shard did. The response's population names the one counted.",
|
|
3540
3540
|
"schema": {
|
|
3541
3541
|
"type": "string",
|
|
3542
3542
|
"enum": [
|
|
@@ -3544,7 +3544,7 @@
|
|
|
3544
3544
|
"local",
|
|
3545
3545
|
"all"
|
|
3546
3546
|
],
|
|
3547
|
-
"default": "
|
|
3547
|
+
"default": "all"
|
|
3548
3548
|
}
|
|
3549
3549
|
}
|
|
3550
3550
|
],
|
|
@@ -3858,7 +3858,7 @@
|
|
|
3858
3858
|
"get": {
|
|
3859
3859
|
"operationId": "getOverview",
|
|
3860
3860
|
"summary": "Get dashboard figures",
|
|
3861
|
-
"description": "The dashboard for a period: last run, suite stability, time lost, the two test-metric tiles (testMetrics), slowest tests, time sinks, recent runs. Every figure counts finished runs of the source -
|
|
3861
|
+
"description": "The dashboard for a period: last run, suite stability, time lost, the two test-metric tiles (testMetrics), slowest tests, time sinks, recent runs. Every figure counts finished runs of the source - all by default - and testMetrics.population names it. Moderate: aggregate queries over the period's finished runs, run in the database - no result rows are loaded.",
|
|
3862
3862
|
"tags": [
|
|
3863
3863
|
"Project"
|
|
3864
3864
|
],
|
|
@@ -3903,7 +3903,7 @@
|
|
|
3903
3903
|
"name": "source",
|
|
3904
3904
|
"in": "query",
|
|
3905
3905
|
"required": false,
|
|
3906
|
-
"description": "Which runs count, by where they ran: ci - runs a shard reported a CI provider for
|
|
3906
|
+
"description": "Which runs count, by where they ran: all - every run (the default since 1.0.0-beta.18; ci before); ci - runs a shard reported a CI provider for; local - runs no shard did. The response's population names the one counted.",
|
|
3907
3907
|
"schema": {
|
|
3908
3908
|
"type": "string",
|
|
3909
3909
|
"enum": [
|
|
@@ -3911,7 +3911,7 @@
|
|
|
3911
3911
|
"local",
|
|
3912
3912
|
"all"
|
|
3913
3913
|
],
|
|
3914
|
-
"default": "
|
|
3914
|
+
"default": "all"
|
|
3915
3915
|
}
|
|
3916
3916
|
}
|
|
3917
3917
|
],
|
|
@@ -5721,7 +5721,7 @@
|
|
|
5721
5721
|
"local_groups_finished_not_mass_failure",
|
|
5722
5722
|
"all_groups_finished_not_mass_failure"
|
|
5723
5723
|
],
|
|
5724
|
-
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider)
|
|
5724
|
+
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider); local_ - runs no shard reported a CI provider for; all_ - both, the default since 1.0.0-beta.18 (ci_ before it; since 1.0.0-beta.8; before it, ci_groups_finished_not_mass_failure was the only value, and since 1.0.0-beta.6 - it was ci_completed_not_excluded, which counted every completed shard on its own)."
|
|
5725
5725
|
},
|
|
5726
5726
|
"days": {
|
|
5727
5727
|
"type": "integer",
|
|
@@ -6026,7 +6026,7 @@
|
|
|
6026
6026
|
"local_groups_finished_not_mass_failure",
|
|
6027
6027
|
"all_groups_finished_not_mass_failure"
|
|
6028
6028
|
],
|
|
6029
|
-
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider)
|
|
6029
|
+
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider); local_ - runs no shard reported a CI provider for; all_ - both, the default since 1.0.0-beta.18 (ci_ before it; since 1.0.0-beta.8; before it, ci_groups_finished_not_mass_failure was the only value, and since 1.0.0-beta.6 - it was ci_completed_not_excluded, which counted every completed shard on its own)."
|
|
6030
6030
|
},
|
|
6031
6031
|
"days": {
|
|
6032
6032
|
"type": "integer",
|
|
@@ -6334,7 +6334,7 @@
|
|
|
6334
6334
|
},
|
|
6335
6335
|
"counted": {
|
|
6336
6336
|
"type": "boolean",
|
|
6337
|
-
"description": "Whether the run is in the metrics' population (see population on the metrics endpoints): a run of the source parameter's kind -
|
|
6337
|
+
"description": "Whether the run is in the metrics' population (see population on the metrics endpoints): a run of the source parameter's kind - all by default since 1.0.0-beta.18, CI before - that finished (all its shards, or overdue with every shard that started completed, timed out or stopped by the monthly result limit - since 1.0.0-beta.10, the limit since 1.0.0-beta.11), not a mass failure, and the test not just skipped by a cancellation (since 1.0.0-beta.6; the source since 1.0.0-beta.8)."
|
|
6338
6338
|
},
|
|
6339
6339
|
"branch": {
|
|
6340
6340
|
"anyOf": [
|
|
@@ -6454,7 +6454,7 @@
|
|
|
6454
6454
|
"local_groups_finished",
|
|
6455
6455
|
"all_groups_finished"
|
|
6456
6456
|
],
|
|
6457
|
-
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount). A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider)
|
|
6457
|
+
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount). A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider); local_ - runs no shard reported a CI provider for; all_ - both, the default since 1.0.0-beta.18 (ci_ before it; since 1.0.0-beta.9)."
|
|
6458
6458
|
},
|
|
6459
6459
|
"window": {
|
|
6460
6460
|
"type": "object",
|
|
@@ -6631,7 +6631,7 @@
|
|
|
6631
6631
|
"local_groups_finished",
|
|
6632
6632
|
"all_groups_finished"
|
|
6633
6633
|
],
|
|
6634
|
-
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount). A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider)
|
|
6634
|
+
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount). A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider); local_ - runs no shard reported a CI provider for; all_ - both, the default since 1.0.0-beta.18 (ci_ before it; since 1.0.0-beta.9)."
|
|
6635
6635
|
},
|
|
6636
6636
|
"window": {
|
|
6637
6637
|
"type": "object",
|
|
@@ -7086,7 +7086,7 @@
|
|
|
7086
7086
|
"local_groups_finished_not_mass_failure",
|
|
7087
7087
|
"all_groups_finished_not_mass_failure"
|
|
7088
7088
|
],
|
|
7089
|
-
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider)
|
|
7089
|
+
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider); local_ - runs no shard reported a CI provider for; all_ - both, the default since 1.0.0-beta.18 (ci_ before it; since 1.0.0-beta.8; before it, ci_groups_finished_not_mass_failure was the only value, and since 1.0.0-beta.6 - it was ci_completed_not_excluded, which counted every completed shard on its own)."
|
|
7090
7090
|
},
|
|
7091
7091
|
"days": {
|
|
7092
7092
|
"type": "integer",
|
package/dist/tools.js
CHANGED
|
@@ -66,9 +66,9 @@ export const TOOLS = [
|
|
|
66
66
|
"runs / runs), time lost to retries and failures, its last 20 runs (each with the run " +
|
|
67
67
|
"id squally-find-run gives; runs it skipped included, as result \"skipped\" - shown, " +
|
|
68
68
|
"never counted in runs or any rate) and the branches it ran on. Counts the finished " +
|
|
69
|
-
"
|
|
70
|
-
"without runs that were a mass failure as a whole; source=
|
|
71
|
-
"
|
|
69
|
+
"runs of the period, CI and local - a sharded run counts once, by its first shard's start - " +
|
|
70
|
+
"without runs that were a mass failure as a whole; source=ci or source=local counts " +
|
|
71
|
+
"only those instead (population names which). A run still in progress counts " +
|
|
72
72
|
"once all its shards have finished, or once the run timeout has passed, with the " +
|
|
73
73
|
"results its shards reported - also when a shard was stopped by the monthly result " +
|
|
74
74
|
"limit. There " +
|
|
@@ -82,12 +82,12 @@ export const TOOLS = [
|
|
|
82
82
|
name: "squally-list-tests",
|
|
83
83
|
title: "List tests",
|
|
84
84
|
operationId: "listTests",
|
|
85
|
-
description: "Every test with a counted
|
|
85
|
+
description: "Every test with a counted run in the period, CI and local (source=ci or source=local narrows it), with the same numbers per test as " +
|
|
86
86
|
"squally-get-test-metrics - runs, stability, flakyRate, failureRate, time lost - ranked " +
|
|
87
87
|
"worst first by the chosen sort (default: lowest stability). There is no verdict and no " +
|
|
88
88
|
"status - the tool labels no test; you judge the numbers. Moderate - two queries over " +
|
|
89
89
|
"the period, paged; for a single test use squally-get-test-metrics.",
|
|
90
|
-
answers: "Every test with a
|
|
90
|
+
answers: "Every test with a run in the period, ranked by stability, flaky rate or failures.",
|
|
91
91
|
cost: "cheap",
|
|
92
92
|
},
|
|
93
93
|
{
|
|
@@ -97,7 +97,7 @@ export const TOOLS = [
|
|
|
97
97
|
description: "The project's error signatures in a period: failures grouped by fingerprint, with how " +
|
|
98
98
|
"often and how recently each occurred. Answers 'is this failure one instance of something " +
|
|
99
99
|
"that keeps happening' - a grouping no agent can compute from the repository alone. " +
|
|
100
|
-
"Counts finished CI
|
|
100
|
+
"Counts finished runs, CI and local, by default (source=ci or source=local for one of them), a sharded run " +
|
|
101
101
|
"once, mass-failure runs included; failedCount and flakyCount say how the runs ended " +
|
|
102
102
|
"for the tests the error hit.",
|
|
103
103
|
answers: "Error signatures in a period: what keeps failing, grouped, and how its runs ended.",
|
package/package.json
CHANGED
|
@@ -1,63 +1,63 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "squally-mcp",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Local MCP server for Squally — read-only access to your Playwright test runs, flaky tests and failures.",
|
|
5
|
-
"license": "MIT",
|
|
6
|
-
"author": "Squally",
|
|
7
|
-
"homepage": "https://docs.squally.dev/agents/mcp-server/",
|
|
8
|
-
"repository": {
|
|
9
|
-
"type": "git",
|
|
10
|
-
"url": "git+https://github.com/squallydev-admin/squally-mcp.git"
|
|
11
|
-
},
|
|
12
|
-
"bugs": {
|
|
13
|
-
"url": "https://github.com/squallydev-admin/squally-mcp/issues"
|
|
14
|
-
},
|
|
15
|
-
"keywords": [
|
|
16
|
-
"mcp",
|
|
17
|
-
"modelcontextprotocol",
|
|
18
|
-
"playwright",
|
|
19
|
-
"flaky-tests",
|
|
20
|
-
"squally"
|
|
21
|
-
],
|
|
22
|
-
"type": "module",
|
|
23
|
-
"bin": {
|
|
24
|
-
"squally-mcp": "dist/index.js"
|
|
25
|
-
},
|
|
26
|
-
"main": "dist/index.js",
|
|
27
|
-
"types": "dist/index.d.ts",
|
|
28
|
-
"exports": {
|
|
29
|
-
".": {
|
|
30
|
-
"types": "./dist/index.d.ts",
|
|
31
|
-
"default": "./dist/index.js"
|
|
32
|
-
},
|
|
33
|
-
"./tools": {
|
|
34
|
-
"types": "./dist/tools.d.ts",
|
|
35
|
-
"default": "./dist/tools.js"
|
|
36
|
-
},
|
|
37
|
-
"./instructions": {
|
|
38
|
-
"types": "./dist/instructions.d.ts",
|
|
39
|
-
"default": "./dist/instructions.js"
|
|
40
|
-
},
|
|
41
|
-
"./package.json": "./package.json"
|
|
42
|
-
},
|
|
43
|
-
"files": [
|
|
44
|
-
"dist"
|
|
45
|
-
],
|
|
46
|
-
"engines": {
|
|
47
|
-
"node": ">=22"
|
|
48
|
-
},
|
|
49
|
-
"scripts": {
|
|
50
|
-
"build": "tsc && node scripts/copy-assets.mjs",
|
|
51
|
-
"vendor:openapi": "node scripts/vendor-openapi.mjs",
|
|
52
|
-
"test": "npm run build && node --test \"test/**/*.test.js\"",
|
|
53
|
-
"prepublishOnly": "npm run build"
|
|
54
|
-
},
|
|
55
|
-
"dependencies": {
|
|
56
|
-
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
57
|
-
"zod": "^4.6.5"
|
|
58
|
-
},
|
|
59
|
-
"devDependencies": {
|
|
60
|
-
"@types/node": "^22.10.0",
|
|
61
|
-
"typescript": "^5.6.0"
|
|
62
|
-
}
|
|
63
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"name": "squally-mcp",
|
|
3
|
+
"version": "0.8.0",
|
|
4
|
+
"description": "Local MCP server for Squally — read-only access to your Playwright test runs, flaky tests and failures.",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"author": "Squally",
|
|
7
|
+
"homepage": "https://docs.squally.dev/agents/mcp-server/",
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/squallydev-admin/squally-mcp.git"
|
|
11
|
+
},
|
|
12
|
+
"bugs": {
|
|
13
|
+
"url": "https://github.com/squallydev-admin/squally-mcp/issues"
|
|
14
|
+
},
|
|
15
|
+
"keywords": [
|
|
16
|
+
"mcp",
|
|
17
|
+
"modelcontextprotocol",
|
|
18
|
+
"playwright",
|
|
19
|
+
"flaky-tests",
|
|
20
|
+
"squally"
|
|
21
|
+
],
|
|
22
|
+
"type": "module",
|
|
23
|
+
"bin": {
|
|
24
|
+
"squally-mcp": "dist/index.js"
|
|
25
|
+
},
|
|
26
|
+
"main": "dist/index.js",
|
|
27
|
+
"types": "dist/index.d.ts",
|
|
28
|
+
"exports": {
|
|
29
|
+
".": {
|
|
30
|
+
"types": "./dist/index.d.ts",
|
|
31
|
+
"default": "./dist/index.js"
|
|
32
|
+
},
|
|
33
|
+
"./tools": {
|
|
34
|
+
"types": "./dist/tools.d.ts",
|
|
35
|
+
"default": "./dist/tools.js"
|
|
36
|
+
},
|
|
37
|
+
"./instructions": {
|
|
38
|
+
"types": "./dist/instructions.d.ts",
|
|
39
|
+
"default": "./dist/instructions.js"
|
|
40
|
+
},
|
|
41
|
+
"./package.json": "./package.json"
|
|
42
|
+
},
|
|
43
|
+
"files": [
|
|
44
|
+
"dist"
|
|
45
|
+
],
|
|
46
|
+
"engines": {
|
|
47
|
+
"node": ">=22"
|
|
48
|
+
},
|
|
49
|
+
"scripts": {
|
|
50
|
+
"build": "tsc && node scripts/copy-assets.mjs",
|
|
51
|
+
"vendor:openapi": "node scripts/vendor-openapi.mjs",
|
|
52
|
+
"test": "npm run build && node --test \"test/**/*.test.js\"",
|
|
53
|
+
"prepublishOnly": "npm run build"
|
|
54
|
+
},
|
|
55
|
+
"dependencies": {
|
|
56
|
+
"@modelcontextprotocol/sdk": "^1.30.1",
|
|
57
|
+
"zod": "^4.6.5"
|
|
58
|
+
},
|
|
59
|
+
"devDependencies": {
|
|
60
|
+
"@types/node": "^22.10.0",
|
|
61
|
+
"typescript": "^5.6.0"
|
|
62
|
+
}
|
|
63
|
+
}
|