squally-mcp 0.1.4 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +84 -10
- package/dist/api.js +17 -2
- package/dist/instructions.js +18 -7
- package/dist/openapi/v1.json +978 -280
- package/dist/openapi.d.ts +14 -8
- package/dist/openapi.js +22 -9
- package/dist/result.d.ts +12 -0
- package/dist/result.js +26 -0
- package/dist/tools.d.ts +6 -3
- package/dist/tools.js +21 -15
- package/dist/validate.js +6 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# squally-mcp
|
|
2
2
|
|
|
3
3
|
A local [MCP](https://modelcontextprotocol.io) server that lets a coding agent
|
|
4
|
-
read your [Squally](https://app.squally.dev) data: which CI runs happened,
|
|
5
|
-
|
|
4
|
+
read your [Squally](https://app.squally.dev) data: which CI runs happened, how
|
|
5
|
+
stable each test is, and why a particular test failed.
|
|
6
6
|
|
|
7
7
|
It runs on your machine over stdio and talks to Squally's read API over HTTPS.
|
|
8
8
|
Point Claude Code, Codex, Cursor or Claude Desktop at it and ask "why is
|
|
@@ -32,7 +32,8 @@ agent can read:
|
|
|
32
32
|
prompt**, which carries the same snippet plus an **ARIA snapshot of your
|
|
33
33
|
application at the moment of failure** (whatever was on screen, truncated to
|
|
34
34
|
3000 characters);
|
|
35
|
-
- **
|
|
35
|
+
- **per-test metrics**: how often each test passed on the first try, passed
|
|
36
|
+
only after a retry, or failed, and how much time retries and failures cost;
|
|
36
37
|
- **error signatures**: failures grouped across runs and branches.
|
|
37
38
|
|
|
38
39
|
It cannot see screenshots, videos or traces — artifacts are deliberately out of
|
|
@@ -140,14 +141,86 @@ squally-mcp trusts your operating system's certificate store, like your browser
|
|
|
140
141
|
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA or status. Counters only. | cheap |
|
|
141
142
|
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
142
143
|
| `squally-debug-failure` | Every attempt of one test in one run: error, stack, Copy-for-AI prompt. | cheap |
|
|
143
|
-
| `squally-get-test-
|
|
144
|
-
| `squally-list-
|
|
144
|
+
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
145
|
+
| `squally-list-tests` | Every test with a CI run in the period, ranked by stability, flaky rate or failures. | cheap — two queries, paged |
|
|
145
146
|
| `squally-list-errors` | Error signatures in a period: what keeps failing, grouped. | cheap |
|
|
146
147
|
|
|
147
|
-
For a single test use `squally-get-test-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
148
|
+
For a single test use `squally-get-test-metrics`, not `squally-list-tests` —
|
|
149
|
+
the tool descriptions say so, and the server repeats it in its instructions.
|
|
150
|
+
|
|
151
|
+
### What the numbers mean
|
|
152
|
+
|
|
153
|
+
The two test tools count **completed CI runs** in the period (7, 14, 30 or 90
|
|
154
|
+
days; 14 by default), on every branch unless you name one — without runs where
|
|
155
|
+
most of the suite failed at once. Local runs never count. Per test:
|
|
156
|
+
|
|
157
|
+
- **stability** — runs that passed on the first try ÷ runs;
|
|
158
|
+
- **flakyRate** — runs that passed only after a retry ÷ runs;
|
|
159
|
+
- **failureRate** — failed runs ÷ runs.
|
|
160
|
+
|
|
161
|
+
Each is a fraction from 0 to 1, and `null` when the test has no run in the
|
|
162
|
+
period. **There is no verdict**: nothing calls a test flaky, broken or
|
|
163
|
+
healthy. The tools return the numbers; the agent — or you — judges them.
|
|
164
|
+
|
|
165
|
+
### Example: `squally-list-tests`
|
|
166
|
+
|
|
167
|
+
```json
|
|
168
|
+
{ "projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15", "perPage": 3 }
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
The three least stable tests of the last 14 days (abridged):
|
|
172
|
+
|
|
173
|
+
```json
|
|
174
|
+
{
|
|
175
|
+
"population": "ci_completed_not_excluded",
|
|
176
|
+
"days": 14,
|
|
177
|
+
"sort": "stability",
|
|
178
|
+
"total": 409,
|
|
179
|
+
"items": [
|
|
180
|
+
{
|
|
181
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
182
|
+
"filePath": "tests/checkout.spec.ts",
|
|
183
|
+
"runs": 8,
|
|
184
|
+
"stability": 0.5,
|
|
185
|
+
"flakyRate": 0.5,
|
|
186
|
+
"failureRate": 0,
|
|
187
|
+
"timeLostMs": 7361,
|
|
188
|
+
"topBranch": { "branch": "main", "kind": "retries", "count": 3 }
|
|
189
|
+
}
|
|
190
|
+
]
|
|
191
|
+
}
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
### Example: `squally-get-test-metrics`
|
|
195
|
+
|
|
196
|
+
```json
|
|
197
|
+
{
|
|
198
|
+
"projectId": "3f7c2a91-5b8e-4d0a-9c61-2e4b7f0d8a15",
|
|
199
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code"
|
|
200
|
+
}
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
The same numbers as that test's row, for one test (abridged):
|
|
204
|
+
|
|
205
|
+
```json
|
|
206
|
+
{
|
|
207
|
+
"days": 14,
|
|
208
|
+
"branch": null,
|
|
209
|
+
"runs": 8,
|
|
210
|
+
"stableRuns": 4,
|
|
211
|
+
"flakyRuns": 4,
|
|
212
|
+
"failedRuns": 0,
|
|
213
|
+
"stability": 0.5,
|
|
214
|
+
"flakyRate": 0.5,
|
|
215
|
+
"failureRate": 0,
|
|
216
|
+
"branches": ["feature/saved-carts", "main"]
|
|
217
|
+
}
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
221
|
+
files, the tool answers with the `filePath` values to retry with; a test with
|
|
222
|
+
results but no completed CI run in the period answers `runs: 0` and `null`
|
|
223
|
+
rates — an answer, not an error.
|
|
151
224
|
|
|
152
225
|
## Development
|
|
153
226
|
|
|
@@ -163,7 +236,8 @@ vendored at `src/openapi/v1.json` — an input schema is the operation's
|
|
|
163
236
|
parameters, an output schema is its 200 response. Nothing is transcribed by
|
|
164
237
|
hand, so the two cannot drift. `test/drift.test.js` compares the vendored copy
|
|
165
238
|
against the live document and fails when an operation or parameter has moved;
|
|
166
|
-
it skips loudly when offline
|
|
239
|
+
it skips loudly when offline, and while the live API is still an older version
|
|
240
|
+
than the vendored one — a release that has not been deployed yet.
|
|
167
241
|
|
|
168
242
|
## Releasing
|
|
169
243
|
|
package/dist/api.js
CHANGED
|
@@ -1,6 +1,18 @@
|
|
|
1
1
|
import { errorCauses } from "./diagnostics.js";
|
|
2
2
|
/** Set by the API from 7 days before the key expires (OpenAPI components.headers). */
|
|
3
3
|
export const KEY_EXPIRY_HEADER = "Squally-Key-Expires-At";
|
|
4
|
+
/**
|
|
5
|
+
* Query parameters for which an EMPTY STRING is a value, not an absence.
|
|
6
|
+
*
|
|
7
|
+
* filePath: "" names a test with no file, and the API tells the two apart
|
|
8
|
+
* (`?filePath=` present and empty vs. not sent - squally-app's test-scoped
|
|
9
|
+
* routes read `searchParams.get("filePath")`, so "" and null differ). Until
|
|
10
|
+
* 0.2.0 this client dropped the empty value like any other, so a tool
|
|
11
|
+
* description that promised "an empty value means a test with no file"
|
|
12
|
+
* silently asked about the name alone - and got ambiguous_test back for the
|
|
13
|
+
* very case the value exists to settle.
|
|
14
|
+
*/
|
|
15
|
+
const EMPTY_IS_A_VALUE = new Set(["filePath"]);
|
|
4
16
|
/**
|
|
5
17
|
* Builds the request URL.
|
|
6
18
|
*
|
|
@@ -26,8 +38,11 @@ export function buildUrl(apiBase, operation, args) {
|
|
|
26
38
|
const value = args[parameter.name];
|
|
27
39
|
// Omitted rather than sent empty: the API is strict about query values and
|
|
28
40
|
// answers invalid_parameter rather than falling back to a default, so an
|
|
29
|
-
// empty string would turn "not specified" into a 400.
|
|
30
|
-
|
|
41
|
+
// empty string would turn "not specified" into a 400. Except where the
|
|
42
|
+
// empty string IS the value (EMPTY_IS_A_VALUE).
|
|
43
|
+
if (value === undefined || value === null)
|
|
44
|
+
continue;
|
|
45
|
+
if (value === "" && !EMPTY_IS_A_VALUE.has(parameter.name))
|
|
31
46
|
continue;
|
|
32
47
|
query.set(parameter.name, String(value));
|
|
33
48
|
}
|
package/dist/instructions.js
CHANGED
|
@@ -1,16 +1,27 @@
|
|
|
1
1
|
// The server's `instructions` - what a client puts in front of the model once,
|
|
2
2
|
// before any tool is called (decided 24.09.).
|
|
3
3
|
//
|
|
4
|
-
//
|
|
5
|
-
//
|
|
6
|
-
//
|
|
7
|
-
//
|
|
4
|
+
// Each sentence prevents one concrete mistake: starting without a projectId
|
|
5
|
+
// (which every other tool needs), listing the whole suite to ask about one
|
|
6
|
+
// test, reading a rate without knowing what it divides by or which runs it
|
|
7
|
+
// counts, reading a verdict into numbers that carry none, guessing at an
|
|
8
|
+
// ambiguous test name instead of passing filePath, and ignoring the action the
|
|
9
|
+
// API already put in the error.
|
|
10
|
+
//
|
|
11
|
+
// NO VERDICT SINCE 0.2.0 (API 1.0.0-beta.2): squally-app removed the engine
|
|
12
|
+
// that called tests flaky, broken or healthy (25.09.2026). The metrics are
|
|
13
|
+
// fractions of runs and nothing more; saying so here stops a model from
|
|
14
|
+
// presenting a rate as Squally's judgement.
|
|
8
15
|
//
|
|
9
16
|
// Exported as `squally-mcp/instructions` (package.json), so it stays a plain
|
|
10
17
|
// constant - test/exports.test.js imports it through the package name.
|
|
11
18
|
export const INSTRUCTIONS = [
|
|
12
19
|
"Start with squally-list-projects; every other tool needs a projectId from it.",
|
|
13
|
-
"For one test, use squally-get-test-
|
|
14
|
-
"
|
|
15
|
-
"
|
|
20
|
+
"For one test, use squally-get-test-metrics, not squally-list-tests.",
|
|
21
|
+
"Test metrics count completed CI runs in the period, without runs where most of the",
|
|
22
|
+
"suite failed at once; local runs never count. stability = runs that passed on the",
|
|
23
|
+
"first try / runs; flakyRate = runs that passed only after a retry / runs;",
|
|
24
|
+
"failureRate = failed runs / runs. There is no verdict: the tools return numbers,",
|
|
25
|
+
"and you judge them. If a test name is ambiguous, repeat with filePath from the",
|
|
26
|
+
"error. Errors carry a code and an action; follow the action.",
|
|
16
27
|
].join(" ");
|