squally-mcp 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -13
- package/dist/config.js +2 -2
- package/dist/instructions.js +30 -5
- package/dist/openapi/v1.json +261 -76
- package/dist/result.js +1 -1
- package/dist/tools.js +21 -11
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# squally-mcp
|
|
2
2
|
|
|
3
3
|
A local [MCP](https://modelcontextprotocol.io) server that lets a coding agent
|
|
4
|
-
read your [Squally](https://app.squally.dev) data: which
|
|
4
|
+
read your [Squally](https://app.squally.dev) data: which runs happened, how
|
|
5
5
|
stable each test is, and why a particular test failed.
|
|
6
6
|
|
|
7
7
|
It runs on your machine over stdio and talks to Squally's read API over HTTPS.
|
|
@@ -24,8 +24,8 @@ Worth knowing before you hand a key to an assistant. Through this server an
|
|
|
24
24
|
agent can read:
|
|
25
25
|
|
|
26
26
|
- your **projects**, their names and stable branches;
|
|
27
|
-
- your **CI
|
|
28
|
-
timing, pass/fail counts;
|
|
27
|
+
- your **runs**, CI and local — commit SHA and message, branch, author, pull
|
|
28
|
+
request, job key, a link to the CI run, timing, pass/fail counts;
|
|
29
29
|
- **per-test results** for a run, including which shard ran what;
|
|
30
30
|
- for a failing attempt: the **error message and stack**, the **code snippet**
|
|
31
31
|
— **verbatim source code from your test file** — and the **Copy-for-AI
|
|
@@ -40,13 +40,13 @@ It cannot see screenshots, videos or traces — artifacts are deliberately out o
|
|
|
40
40
|
scope — and it cannot see any key, secret or webhook URL.
|
|
41
41
|
|
|
42
42
|
The key reaches **every project in the organization**. Create one per machine so
|
|
43
|
-
a single revocation does not lock out everything, and revoke it
|
|
44
|
-
|
|
43
|
+
a single revocation does not lock out everything, and revoke it under
|
|
44
|
+
Organization › Read keys when the machine is retired.
|
|
45
45
|
|
|
46
46
|
## Setup
|
|
47
47
|
|
|
48
|
-
You need an **organization read key** (`sqly_ro_…`): Squally →
|
|
49
|
-
keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
48
|
+
You need an **organization read key** (`sqly_ro_…`): Squally → Organization ›
|
|
49
|
+
Read keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
50
50
|
project ingest key your Playwright reporter uses — this server refuses that one
|
|
51
51
|
by name.
|
|
52
52
|
|
|
@@ -138,7 +138,7 @@ squally-mcp trusts your operating system's certificate store, like your browser
|
|
|
138
138
|
| Tool | What it answers | Cost |
|
|
139
139
|
|---|---|---|
|
|
140
140
|
| `squally-list-projects` | Which projects exist. **Start here** — every other tool needs a `projectId` from it. | cheap |
|
|
141
|
-
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA or
|
|
141
|
+
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA, status or source. Counters only. | cheap |
|
|
142
142
|
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
143
143
|
| `squally-debug-failure` | Every attempt of one test in one run: error, stack, Copy-for-AI prompt. | cheap |
|
|
144
144
|
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
@@ -148,12 +148,34 @@ squally-mcp trusts your operating system's certificate store, like your browser
|
|
|
148
148
|
For a single test use `squally-get-test-metrics`, not `squally-list-tests` —
|
|
149
149
|
the tool descriptions say so, and the server repeats it in its instructions.
|
|
150
150
|
|
|
151
|
+
### Runs
|
|
152
|
+
|
|
153
|
+
`squally-find-run` lists every run, CI and local alike; `source` (`all` by
|
|
154
|
+
default, `ci` or `local`) narrows it, and each run says whether it was
|
|
155
|
+
`local`. A run's `status` is the dashboard's word: `passed`, `failed`,
|
|
156
|
+
`cancelled`, `incomplete` (a shard timed out, was stopped by the monthly
|
|
157
|
+
result limit or never reported) or `running` - and `?status=` filters by
|
|
158
|
+
any of them. A cancelled run is `cancelled`; `cancellation` says how, when
|
|
159
|
+
and by whom. A sharded run is one row, its counters over all its shards with
|
|
160
|
+
every test once, and the tests a cancel cut off count nowhere.
|
|
161
|
+
|
|
162
|
+
Each run also says whether it is `settled` - final and counted; an
|
|
163
|
+
`incomplete` run that is not settled still waits for a shard within the run
|
|
164
|
+
timeout - and, from squally-reporter 0.11.0, which job of its CI run it is
|
|
165
|
+
(`jobKey`, from `SQUALLY_CI_JOB_KEY`; each job is its own run), which
|
|
166
|
+
`attempt`, which run it re-ran (`rerunOf`) and where the CI run is
|
|
167
|
+
(`ciRunUrl`; for GitHub Actions also for older reporters when the project
|
|
168
|
+
has one linked repository).
|
|
169
|
+
|
|
151
170
|
### What the numbers mean
|
|
152
171
|
|
|
153
|
-
The two test tools count **CI runs
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
172
|
+
The two test tools count **finished CI runs** in the period (7, 14, 30 or 90
|
|
173
|
+
days; 30 by default), on every branch unless you name one. A run finishes when
|
|
174
|
+
all its shards have - or, once the run timeout has passed, with the results its
|
|
175
|
+
shards reported, like a cancelled run; a shard stopped by the monthly result
|
|
176
|
+
limit counts like one that timed out. A sharded run counts once, placed in the period
|
|
177
|
+
by its first shard's start; a run that was a mass failure as a whole is left
|
|
178
|
+
out. Local runs count only
|
|
157
179
|
when you ask for them: `source` is `ci` (the default), `local` or `all`, and
|
|
158
180
|
the response's `population` names which runs it counted. Per test:
|
|
159
181
|
|
|
@@ -222,7 +244,7 @@ The same numbers as that test's row, for one test (abridged):
|
|
|
222
244
|
|
|
223
245
|
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
224
246
|
files, the tool answers with the `filePath` values to retry with; a test with
|
|
225
|
-
results but no counted
|
|
247
|
+
results but no counted run in the period answers `runs: 0` and `null`
|
|
226
248
|
rates — an answer, not an error.
|
|
227
249
|
|
|
228
250
|
## Development
|
package/dist/config.js
CHANGED
|
@@ -55,7 +55,7 @@ export function resolveConfig(env, version) {
|
|
|
55
55
|
return {
|
|
56
56
|
ok: false,
|
|
57
57
|
message: "SQUALLY_API_KEY is not set. Create an organization read key in Squally under " +
|
|
58
|
-
"
|
|
58
|
+
"Organization -> Read keys and set it in this server's environment.",
|
|
59
59
|
};
|
|
60
60
|
}
|
|
61
61
|
// The order matters: an ingest key DOES start with sqly_, so the read-key
|
|
@@ -65,7 +65,7 @@ export function resolveConfig(env, version) {
|
|
|
65
65
|
return {
|
|
66
66
|
ok: false,
|
|
67
67
|
message: "SQUALLY_API_KEY looks like a project ingest key; the MCP server needs an " +
|
|
68
|
-
"organization read key (
|
|
68
|
+
"organization read key (Organization -> Read keys). A read key starts with " +
|
|
69
69
|
`${READ_KEY_PREFIX} and is created separately from the ingest key your ` +
|
|
70
70
|
"Playwright reporter uses.",
|
|
71
71
|
};
|
package/dist/instructions.js
CHANGED
|
@@ -24,18 +24,43 @@
|
|
|
24
24
|
// THE SOURCE SINCE 0.4.0 (API 1.0.0-beta.8): CI runs by default; local runs
|
|
25
25
|
// count only when the agent asks for them - they were never counted before.
|
|
26
26
|
//
|
|
27
|
+
// THE RUN TIMEOUT (API 1.0.0-beta.10, since 0.5.0): a run whose shard died or
|
|
28
|
+
// never started counts once the run timeout has passed, with the shards that
|
|
29
|
+
// finished - "whose shards all finished" stopped being the whole rule.
|
|
30
|
+
//
|
|
31
|
+
// THE MONTHLY RESULT LIMIT (API 1.0.0-beta.11, since 0.5.0): a shard the limit
|
|
32
|
+
// stopped is treated like a timed-out one, so the sentence names it - and says
|
|
33
|
+
// "the results its shards reported", since such a shard did not finish.
|
|
34
|
+
//
|
|
35
|
+
// ONE RUN STATUS SINCE 0.5.0 (API 1.0.0-beta.14): a run's status is the
|
|
36
|
+
// dashboard's own word - passed, failed, cancelled, incomplete or running -
|
|
37
|
+
// and the `cancelled` boolean is gone (`cancellation` says how and when). Said
|
|
38
|
+
// here because a model that knew the old shape looks for a flag that no longer
|
|
39
|
+
// exists, or reads "cancelled" as an unknown value.
|
|
40
|
+
//
|
|
41
|
+
// SETTLED AND THE JOB KEY SINCE 0.6.0 (API 1.0.0-beta.15): a run that is not
|
|
42
|
+
// settled can still change - an incomplete one may still be waiting for a
|
|
43
|
+
// shard - and one CI run can be several Squally runs, one per job key. Said
|
|
44
|
+
// here because the newest run is the one an agent asks about, and an
|
|
45
|
+
// unsettled "incomplete" read as final, or two runs of one commit read as a
|
|
46
|
+
// duplicate, is the mistake.
|
|
47
|
+
//
|
|
27
48
|
// Exported as `squally-mcp/instructions` (package.json), so it stays a plain
|
|
28
49
|
// constant - test/exports.test.js imports it through the package name.
|
|
29
50
|
export const INSTRUCTIONS = [
|
|
30
51
|
"Start with squally-list-projects; every other tool needs a projectId from it.",
|
|
31
52
|
"For one test, use squally-get-test-metrics, not squally-list-tests.",
|
|
32
|
-
"Test metrics count the CI runs of the period
|
|
33
|
-
"
|
|
34
|
-
"
|
|
35
|
-
"
|
|
53
|
+
"Test metrics count the finished CI runs of the period - a sharded run counts once -",
|
|
54
|
+
"without runs that were a mass failure as a whole; pass source=local or source=all",
|
|
55
|
+
"to count local runs or both. A run still in progress counts once all its shards have",
|
|
56
|
+
"finished, or once the run timeout has passed, with the results its shards reported -",
|
|
57
|
+
"also when a shard was stopped by the monthly result limit.",
|
|
36
58
|
"stability = runs that passed on the",
|
|
37
59
|
"first try / runs; flakyRate = runs that passed only after a retry / runs;",
|
|
38
60
|
"failureRate = failed runs / runs. There is no verdict: the tools return numbers,",
|
|
39
61
|
"and you judge them. If a test name is ambiguous, repeat with filePath from the",
|
|
40
|
-
"error.
|
|
62
|
+
"error. A run's status is passed, failed, cancelled, incomplete or running;",
|
|
63
|
+
"a run that is not settled can still change. Each job of a CI run that sets a",
|
|
64
|
+
"job key is its own run (jobKey).",
|
|
65
|
+
"Errors carry a code and an action; follow the action.",
|
|
41
66
|
].join(" ");
|
package/dist/openapi/v1.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema",
|
|
4
4
|
"info": {
|
|
5
5
|
"title": "Squally Read API",
|
|
6
|
-
"version": "1.0.0-beta.
|
|
6
|
+
"version": "1.0.0-beta.15",
|
|
7
7
|
"description": "Read access to a Squally organization's test results, for scripts, CI jobs and AI agents.\n\n**Authentication.** Every request carries an organization read key as `Authorization: Bearer sqly_ro_…`. A read key is organization-wide (every project), created by an organization admin in Organization › Read keys, and expires after 30, 60, 90, 180 days as chosen at creation. There is no renewal: create a new key and revoke the old one. Project ingest keys (`sqly_…`) are refused with `wrong_key_type`.\n\n**Expiry warning.** From 7 days before a key expires, every authenticated response carries `Squally-Key-Expires-At` with the expiry instant (ISO-8601).\n\n**Rate limit.** Currently 60 requests per 60 seconds per key, counted per `Authorization` header - a read key has its own budget, shared with nothing else. Over the limit the edge answers 429 without a JSON body.\n\n**Retries.** 429: wait 60 seconds, then retry. The edge counts requests per key in 60-second windows and sends no Retry-After header. 5xx: retry with exponential backoff, starting at 1 second and capped at 30 seconds. 400, 401, 403, 404: never retry - the same request gets the same answer.\n\n**Plans.** The read API is part of the Standard and Pro plans. A key of an organization on Free is answered 403 `tier_too_low`; keys are not revoked on a downgrade and work again after an upgrade.\n\n**Errors.** Every error is `{\"error\": \"<sentence>\", \"code\": \"<slug>\"}`. The sentence may be reworded; the slug may not. An id that does not exist and an id that belongs to another organization get the same 404, byte for byte.\n\n**Test names in paths.** Test names are Playwright title paths and often contain `/`: URL-encode them, including `/` as `%2F`.\n\n**Sharded runs** are returned as one logical run: /runs lists one row per logical run, and the run routes answer for all shards whichever shard's id you pass.\n\n**Stability.** v1, beta. The API may change until the beta ends; every change is listed in the changelog (docs/openapi/CHANGELOG.md)."
|
|
8
8
|
},
|
|
9
9
|
"servers": [
|
|
@@ -337,7 +337,7 @@
|
|
|
337
337
|
"get": {
|
|
338
338
|
"operationId": "listRuns",
|
|
339
339
|
"summary": "List runs",
|
|
340
|
-
"description": "Runs in a period, newest first - one row per logical run (a sharded CI run is one row). Filter by branch, commit SHA prefix or
|
|
340
|
+
"description": "Runs in a period, newest first - one row per logical run (a sharded CI run is one row; each job of a CI run that sets a job key is its own run, see jobKey), placed in the period and ordered by the start of its first shard (since 1.0.0-beta.13). CI and local runs both, unless source narrows it. Filter by branch, commit SHA prefix, status or source; page with the cursors. Cheap to moderate: bounded by the period's runs.",
|
|
341
341
|
"tags": [
|
|
342
342
|
"Runs"
|
|
343
343
|
],
|
|
@@ -392,15 +392,33 @@
|
|
|
392
392
|
"name": "status",
|
|
393
393
|
"in": "query",
|
|
394
394
|
"required": false,
|
|
395
|
-
"description": "Only runs with this
|
|
395
|
+
"description": "Only runs with this status - the status field's values. A cancelled run matches cancelled only (since 1.0.0-beta.14; passed and failed were the only values before it, and a cancelled run matched neither).",
|
|
396
396
|
"schema": {
|
|
397
397
|
"type": "string",
|
|
398
398
|
"enum": [
|
|
399
399
|
"passed",
|
|
400
|
-
"failed"
|
|
400
|
+
"failed",
|
|
401
|
+
"cancelled",
|
|
402
|
+
"incomplete",
|
|
403
|
+
"running"
|
|
401
404
|
]
|
|
402
405
|
}
|
|
403
406
|
},
|
|
407
|
+
{
|
|
408
|
+
"name": "source",
|
|
409
|
+
"in": "query",
|
|
410
|
+
"required": false,
|
|
411
|
+
"description": "Which runs are listed, by where they ran: all - every run (the default); ci - runs a shard reported a CI provider for; local - runs no shard did (each run's local field). Since 1.0.0-beta.14.",
|
|
412
|
+
"schema": {
|
|
413
|
+
"type": "string",
|
|
414
|
+
"enum": [
|
|
415
|
+
"ci",
|
|
416
|
+
"local",
|
|
417
|
+
"all"
|
|
418
|
+
],
|
|
419
|
+
"default": "all"
|
|
420
|
+
}
|
|
421
|
+
},
|
|
404
422
|
{
|
|
405
423
|
"name": "perPage",
|
|
406
424
|
"in": "query",
|
|
@@ -474,8 +492,13 @@
|
|
|
474
492
|
"testedRevision": null,
|
|
475
493
|
"pullRequestNumber": null,
|
|
476
494
|
"status": "passed",
|
|
477
|
-
"cancelled": false,
|
|
478
495
|
"cancellation": null,
|
|
496
|
+
"local": false,
|
|
497
|
+
"settled": true,
|
|
498
|
+
"jobKey": null,
|
|
499
|
+
"attempt": 1,
|
|
500
|
+
"rerunOf": null,
|
|
501
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18144270113/attempts/1",
|
|
479
502
|
"startedAt": "2026-09-24T07:31:12.000Z",
|
|
480
503
|
"finishedAt": "2026-09-24T07:39:47.000Z",
|
|
481
504
|
"durationMs": 515000,
|
|
@@ -499,8 +522,13 @@
|
|
|
499
522
|
"testedRevision": "c4f7a1e9b3d5c8e2f0a6b4d9e1c7f3a5b8d2e0f6",
|
|
500
523
|
"pullRequestNumber": 318,
|
|
501
524
|
"status": "failed",
|
|
502
|
-
"cancelled": false,
|
|
503
525
|
"cancellation": null,
|
|
526
|
+
"local": false,
|
|
527
|
+
"settled": true,
|
|
528
|
+
"jobKey": null,
|
|
529
|
+
"attempt": 1,
|
|
530
|
+
"rerunOf": null,
|
|
531
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18127766204/attempts/1",
|
|
504
532
|
"startedAt": "2026-09-23T16:05:40.000Z",
|
|
505
533
|
"finishedAt": "2026-09-23T16:12:02.000Z",
|
|
506
534
|
"durationMs": 382000,
|
|
@@ -520,13 +548,18 @@
|
|
|
520
548
|
"commitSha": "5f1d9b3e7a2c6e0f8b4d2a9c7e1f5b3d8a6c0e4f",
|
|
521
549
|
"testedRevision": null,
|
|
522
550
|
"pullRequestNumber": 318,
|
|
523
|
-
"status": "
|
|
524
|
-
"cancelled": true,
|
|
551
|
+
"status": "cancelled",
|
|
525
552
|
"cancellation": {
|
|
526
553
|
"source": "reporter",
|
|
527
554
|
"at": "2026-09-23T15:58:31.000Z",
|
|
528
555
|
"byUserId": null
|
|
529
556
|
},
|
|
557
|
+
"local": false,
|
|
558
|
+
"settled": true,
|
|
559
|
+
"jobKey": null,
|
|
560
|
+
"attempt": 1,
|
|
561
|
+
"rerunOf": null,
|
|
562
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18127598830/attempts/1",
|
|
530
563
|
"startedAt": "2026-09-23T15:55:02.000Z",
|
|
531
564
|
"finishedAt": "2026-09-23T15:58:31.000Z",
|
|
532
565
|
"durationMs": 209000,
|
|
@@ -831,8 +864,13 @@
|
|
|
831
864
|
"pullRequestTitle": null,
|
|
832
865
|
"ciProvider": "github-actions",
|
|
833
866
|
"status": "passed",
|
|
834
|
-
"cancelled": false,
|
|
835
867
|
"cancellation": null,
|
|
868
|
+
"local": false,
|
|
869
|
+
"settled": true,
|
|
870
|
+
"jobKey": null,
|
|
871
|
+
"attempt": 1,
|
|
872
|
+
"rerunOf": null,
|
|
873
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18144270113/attempts/1",
|
|
836
874
|
"startedAt": "2026-09-24T07:31:12.000Z",
|
|
837
875
|
"finishedAt": "2026-09-24T07:39:47.000Z",
|
|
838
876
|
"durationMs": 515000,
|
|
@@ -1675,7 +1713,7 @@
|
|
|
1675
1713
|
"get": {
|
|
1676
1714
|
"operationId": "listTests",
|
|
1677
1715
|
"summary": "List tests",
|
|
1678
|
-
"description": "Every test with a run of the source (CI by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Counts
|
|
1716
|
+
"description": "Every test with a run of the source (CI by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Counts finished runs of the source - CI by default - without mass-failure runs, on every branch unless branch names one (see population). Moderate: aggregate queries over the period's results of the source, paged in the database; use the metrics route for one test.",
|
|
1679
1717
|
"tags": [
|
|
1680
1718
|
"Tests"
|
|
1681
1719
|
],
|
|
@@ -3823,7 +3861,7 @@
|
|
|
3823
3861
|
"get": {
|
|
3824
3862
|
"operationId": "getOverview",
|
|
3825
3863
|
"summary": "Get dashboard figures",
|
|
3826
|
-
"description": "The dashboard for a period: last run, suite stability, time lost, the two test-metric tiles (testMetrics), slowest tests, time sinks, recent runs. Every figure counts finished runs of the source - CI by default - and testMetrics.population names it. Moderate:
|
|
3864
|
+
"description": "The dashboard for a period: last run, suite stability, time lost, the two test-metric tiles (testMetrics), slowest tests, time sinks, recent runs. Every figure counts finished runs of the source - CI by default - and testMetrics.population names it. Moderate: aggregate queries over the period's finished runs, run in the database - no result rows are loaded.",
|
|
3827
3865
|
"tags": [
|
|
3828
3866
|
"Project"
|
|
3829
3867
|
],
|
|
@@ -3939,9 +3977,9 @@
|
|
|
3939
3977
|
"testMetrics": {
|
|
3940
3978
|
"population": "ci_groups_finished_not_mass_failure",
|
|
3941
3979
|
"days": 30,
|
|
3942
|
-
"
|
|
3980
|
+
"flakyTests": 7,
|
|
3943
3981
|
"failingTests": 2,
|
|
3944
|
-
"
|
|
3982
|
+
"tests": 409
|
|
3945
3983
|
},
|
|
3946
3984
|
"slowestTests": [
|
|
3947
3985
|
{
|
|
@@ -4589,19 +4627,15 @@
|
|
|
4589
4627
|
]
|
|
4590
4628
|
},
|
|
4591
4629
|
"status": {
|
|
4592
|
-
"
|
|
4593
|
-
|
|
4594
|
-
|
|
4595
|
-
|
|
4596
|
-
|
|
4597
|
-
|
|
4598
|
-
|
|
4630
|
+
"type": "string",
|
|
4631
|
+
"enum": [
|
|
4632
|
+
"passed",
|
|
4633
|
+
"failed",
|
|
4634
|
+
"cancelled",
|
|
4635
|
+
"incomplete",
|
|
4636
|
+
"running"
|
|
4599
4637
|
],
|
|
4600
|
-
"description": "
|
|
4601
|
-
},
|
|
4602
|
-
"cancelled": {
|
|
4603
|
-
"type": "boolean",
|
|
4604
|
-
"description": "True when the run was cancelled - for a sharded run, when any shard recorded a cancel. A cancelled run keeps its status (the outcome of what it reported before the stop) and never matches ?status=passed or ?status=failed."
|
|
4638
|
+
"description": "How the run ended, the dashboard's own word: cancelled - the run was cancelled (for a sharded run, any shard recorded a cancel; cancellation says how and when); incomplete - a shard timed out, was stopped by the monthly result limit or never reported; running - a shard is still running; failed - every shard finished and a test failed; passed - every shard finished and none failed. In that order: the first that applies. Since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel and a cancelled boolean."
|
|
4605
4639
|
},
|
|
4606
4640
|
"cancellation": {
|
|
4607
4641
|
"anyOf": [
|
|
@@ -4646,7 +4680,72 @@
|
|
|
4646
4680
|
"type": "null"
|
|
4647
4681
|
}
|
|
4648
4682
|
],
|
|
4649
|
-
"description": "How and when the run was cancelled; null when it was not."
|
|
4683
|
+
"description": "How and when the run was cancelled; null when it was not - set exactly when status is cancelled."
|
|
4684
|
+
},
|
|
4685
|
+
"local": {
|
|
4686
|
+
"type": "boolean",
|
|
4687
|
+
"description": "No shard of the run reported a CI provider: a local run (since 1.0.0-beta.14). GET /runs lists both kinds unless source says otherwise."
|
|
4688
|
+
},
|
|
4689
|
+
"settled": {
|
|
4690
|
+
"type": "boolean",
|
|
4691
|
+
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while an incomplete run still waits for a shard within the run time limit - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
4692
|
+
},
|
|
4693
|
+
"jobKey": {
|
|
4694
|
+
"anyOf": [
|
|
4695
|
+
{
|
|
4696
|
+
"type": "string"
|
|
4697
|
+
},
|
|
4698
|
+
{
|
|
4699
|
+
"type": "null"
|
|
4700
|
+
}
|
|
4701
|
+
],
|
|
4702
|
+
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check, Squally · <key>. null when the job set none. Since 1.0.0-beta.15."
|
|
4703
|
+
},
|
|
4704
|
+
"attempt": {
|
|
4705
|
+
"anyOf": [
|
|
4706
|
+
{
|
|
4707
|
+
"type": "integer"
|
|
4708
|
+
},
|
|
4709
|
+
{
|
|
4710
|
+
"type": "null"
|
|
4711
|
+
}
|
|
4712
|
+
],
|
|
4713
|
+
"description": "Which attempt of its CI run this is - GitHub Actions' run attempt, 2 after Re-run failed jobs. null when the reporter did not say and the run id did not show it. Since 1.0.0-beta.15."
|
|
4714
|
+
},
|
|
4715
|
+
"rerunOf": {
|
|
4716
|
+
"anyOf": [
|
|
4717
|
+
{
|
|
4718
|
+
"type": "object",
|
|
4719
|
+
"properties": {
|
|
4720
|
+
"id": {
|
|
4721
|
+
"type": "string",
|
|
4722
|
+
"description": "The earlier attempt's id - the one GET /runs/{runId} takes."
|
|
4723
|
+
},
|
|
4724
|
+
"runNumber": {
|
|
4725
|
+
"type": "integer"
|
|
4726
|
+
}
|
|
4727
|
+
},
|
|
4728
|
+
"required": [
|
|
4729
|
+
"id",
|
|
4730
|
+
"runNumber"
|
|
4731
|
+
]
|
|
4732
|
+
},
|
|
4733
|
+
{
|
|
4734
|
+
"type": "null"
|
|
4735
|
+
}
|
|
4736
|
+
],
|
|
4737
|
+
"description": "The run this one re-ran: the latest earlier attempt of the same CI run (and job key) that reported to Squally. null for a first attempt, when the earlier attempt never reported, or once retention deleted it. Since 1.0.0-beta.15."
|
|
4738
|
+
},
|
|
4739
|
+
"ciRunUrl": {
|
|
4740
|
+
"anyOf": [
|
|
4741
|
+
{
|
|
4742
|
+
"type": "string"
|
|
4743
|
+
},
|
|
4744
|
+
{
|
|
4745
|
+
"type": "null"
|
|
4746
|
+
}
|
|
4747
|
+
],
|
|
4748
|
+
"description": "The CI run's page - the run page's View in CI link: the URL the reporter sent (squally-reporter 0.11.0; GitHub Actions links the attempt), else for GitHub Actions one built from the project's linked repository when it has exactly one. https only; null when neither applies. Since 1.0.0-beta.15."
|
|
4650
4749
|
},
|
|
4651
4750
|
"startedAt": {
|
|
4652
4751
|
"type": "string",
|
|
@@ -4678,7 +4777,8 @@
|
|
|
4678
4777
|
"type": "object",
|
|
4679
4778
|
"properties": {
|
|
4680
4779
|
"total": {
|
|
4681
|
-
"type": "integer"
|
|
4780
|
+
"type": "integer",
|
|
4781
|
+
"description": "Tests in the run, every test once however many shards reported it - its final attempt across all of them decides (highest retry, then the lowest shard, then the first repetition). total = passed + failed + skipped + flaky. Since 1.0.0-beta.12; until then each shard's count was summed, so a test reported from two shards counted twice. In a cancelled run the tests the cancel interrupted - reported as skipped - count nowhere, so skipped there counts no test (since 1.0.0-beta.13); the run's tests list still has them."
|
|
4682
4782
|
},
|
|
4683
4783
|
"passed": {
|
|
4684
4784
|
"type": "integer"
|
|
@@ -4708,7 +4808,8 @@
|
|
|
4708
4808
|
"type": "object",
|
|
4709
4809
|
"properties": {
|
|
4710
4810
|
"received": {
|
|
4711
|
-
"type": "integer"
|
|
4811
|
+
"type": "integer",
|
|
4812
|
+
"description": "How many shards reported."
|
|
4712
4813
|
},
|
|
4713
4814
|
"expected": {
|
|
4714
4815
|
"anyOf": [
|
|
@@ -4718,7 +4819,8 @@
|
|
|
4718
4819
|
{
|
|
4719
4820
|
"type": "null"
|
|
4720
4821
|
}
|
|
4721
|
-
]
|
|
4822
|
+
],
|
|
4823
|
+
"description": "How many shards the run waits for: the shard total the CI run declared - for a re-run, the shards it re-ran (those that had not passed before it; the others are not run again and do not report). null when no shard declared a total. The run is complete when every expected shard has completed."
|
|
4722
4824
|
}
|
|
4723
4825
|
},
|
|
4724
4826
|
"required": [
|
|
@@ -4740,8 +4842,13 @@
|
|
|
4740
4842
|
"testedRevision",
|
|
4741
4843
|
"pullRequestNumber",
|
|
4742
4844
|
"status",
|
|
4743
|
-
"cancelled",
|
|
4744
4845
|
"cancellation",
|
|
4846
|
+
"local",
|
|
4847
|
+
"settled",
|
|
4848
|
+
"jobKey",
|
|
4849
|
+
"attempt",
|
|
4850
|
+
"rerunOf",
|
|
4851
|
+
"ciRunUrl",
|
|
4745
4852
|
"startedAt",
|
|
4746
4853
|
"finishedAt",
|
|
4747
4854
|
"durationMs",
|
|
@@ -4897,11 +5004,14 @@
|
|
|
4897
5004
|
},
|
|
4898
5005
|
"status": {
|
|
4899
5006
|
"type": "string",
|
|
4900
|
-
"
|
|
4901
|
-
|
|
4902
|
-
|
|
4903
|
-
|
|
4904
|
-
|
|
5007
|
+
"enum": [
|
|
5008
|
+
"passed",
|
|
5009
|
+
"failed",
|
|
5010
|
+
"cancelled",
|
|
5011
|
+
"incomplete",
|
|
5012
|
+
"running"
|
|
5013
|
+
],
|
|
5014
|
+
"description": "How the run ended, the dashboard's own word: cancelled - the run was cancelled (for a sharded run, any shard recorded a cancel; cancellation says how and when); incomplete - a shard timed out, was stopped by the monthly result limit or never reported; running - a shard is still running; failed - every shard finished and a test failed; passed - every shard finished and none failed. In that order: the first that applies. Since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel and a cancelled boolean."
|
|
4905
5015
|
},
|
|
4906
5016
|
"cancellation": {
|
|
4907
5017
|
"anyOf": [
|
|
@@ -4946,7 +5056,72 @@
|
|
|
4946
5056
|
"type": "null"
|
|
4947
5057
|
}
|
|
4948
5058
|
],
|
|
4949
|
-
"description": "How and when the run was cancelled; null when it was not."
|
|
5059
|
+
"description": "How and when the run was cancelled; null when it was not - set exactly when status is cancelled."
|
|
5060
|
+
},
|
|
5061
|
+
"local": {
|
|
5062
|
+
"type": "boolean",
|
|
5063
|
+
"description": "No shard of the run reported a CI provider: a local run (since 1.0.0-beta.14). GET /runs lists both kinds unless source says otherwise."
|
|
5064
|
+
},
|
|
5065
|
+
"settled": {
|
|
5066
|
+
"type": "boolean",
|
|
5067
|
+
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while an incomplete run still waits for a shard within the run time limit - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
5068
|
+
},
|
|
5069
|
+
"jobKey": {
|
|
5070
|
+
"anyOf": [
|
|
5071
|
+
{
|
|
5072
|
+
"type": "string"
|
|
5073
|
+
},
|
|
5074
|
+
{
|
|
5075
|
+
"type": "null"
|
|
5076
|
+
}
|
|
5077
|
+
],
|
|
5078
|
+
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check, Squally · <key>. null when the job set none. Since 1.0.0-beta.15."
|
|
5079
|
+
},
|
|
5080
|
+
"attempt": {
|
|
5081
|
+
"anyOf": [
|
|
5082
|
+
{
|
|
5083
|
+
"type": "integer"
|
|
5084
|
+
},
|
|
5085
|
+
{
|
|
5086
|
+
"type": "null"
|
|
5087
|
+
}
|
|
5088
|
+
],
|
|
5089
|
+
"description": "Which attempt of its CI run this is - GitHub Actions' run attempt, 2 after Re-run failed jobs. null when the reporter did not say and the run id did not show it. Since 1.0.0-beta.15."
|
|
5090
|
+
},
|
|
5091
|
+
"rerunOf": {
|
|
5092
|
+
"anyOf": [
|
|
5093
|
+
{
|
|
5094
|
+
"type": "object",
|
|
5095
|
+
"properties": {
|
|
5096
|
+
"id": {
|
|
5097
|
+
"type": "string",
|
|
5098
|
+
"description": "The earlier attempt's id - the one GET /runs/{runId} takes."
|
|
5099
|
+
},
|
|
5100
|
+
"runNumber": {
|
|
5101
|
+
"type": "integer"
|
|
5102
|
+
}
|
|
5103
|
+
},
|
|
5104
|
+
"required": [
|
|
5105
|
+
"id",
|
|
5106
|
+
"runNumber"
|
|
5107
|
+
]
|
|
5108
|
+
},
|
|
5109
|
+
{
|
|
5110
|
+
"type": "null"
|
|
5111
|
+
}
|
|
5112
|
+
],
|
|
5113
|
+
"description": "The run this one re-ran: the latest earlier attempt of the same CI run (and job key) that reported to Squally. null for a first attempt, when the earlier attempt never reported, or once retention deleted it. Since 1.0.0-beta.15."
|
|
5114
|
+
},
|
|
5115
|
+
"ciRunUrl": {
|
|
5116
|
+
"anyOf": [
|
|
5117
|
+
{
|
|
5118
|
+
"type": "string"
|
|
5119
|
+
},
|
|
5120
|
+
{
|
|
5121
|
+
"type": "null"
|
|
5122
|
+
}
|
|
5123
|
+
],
|
|
5124
|
+
"description": "The CI run's page - the run page's View in CI link: the URL the reporter sent (squally-reporter 0.11.0; GitHub Actions links the attempt), else for GitHub Actions one built from the project's linked repository when it has exactly one. https only; null when neither applies. Since 1.0.0-beta.15."
|
|
4950
5125
|
},
|
|
4951
5126
|
"startedAt": {
|
|
4952
5127
|
"type": "string",
|
|
@@ -4987,7 +5162,8 @@
|
|
|
4987
5162
|
{
|
|
4988
5163
|
"type": "null"
|
|
4989
5164
|
}
|
|
4990
|
-
]
|
|
5165
|
+
],
|
|
5166
|
+
"description": "How many shards the run waits for: the shard total the CI run declared - for a re-run, the shards it re-ran (those that had not passed before it; the others are not run again and do not report). null when no shard declared a total. The run is complete when every expected shard has completed."
|
|
4991
5167
|
},
|
|
4992
5168
|
"received": {
|
|
4993
5169
|
"type": "array",
|
|
@@ -5011,14 +5187,15 @@
|
|
|
5011
5187
|
"type": "string"
|
|
5012
5188
|
},
|
|
5013
5189
|
"status": {
|
|
5014
|
-
"
|
|
5015
|
-
|
|
5016
|
-
|
|
5017
|
-
|
|
5018
|
-
|
|
5019
|
-
|
|
5020
|
-
|
|
5021
|
-
]
|
|
5190
|
+
"type": "string",
|
|
5191
|
+
"enum": [
|
|
5192
|
+
"passed",
|
|
5193
|
+
"failed",
|
|
5194
|
+
"cancelled",
|
|
5195
|
+
"incomplete",
|
|
5196
|
+
"running"
|
|
5197
|
+
],
|
|
5198
|
+
"description": "What this shard did, in the run status's words: cancelled when it recorded a cancel; else running or incomplete by its state; else failed when a test failed, passed otherwise. The shards panel's own rule (since 1.0.0-beta.14 - it was the shard's raw outcome, null until it finished)."
|
|
5022
5199
|
}
|
|
5023
5200
|
},
|
|
5024
5201
|
"required": [
|
|
@@ -5053,8 +5230,13 @@
|
|
|
5053
5230
|
"pullRequestTitle",
|
|
5054
5231
|
"ciProvider",
|
|
5055
5232
|
"status",
|
|
5056
|
-
"cancelled",
|
|
5057
5233
|
"cancellation",
|
|
5234
|
+
"local",
|
|
5235
|
+
"settled",
|
|
5236
|
+
"jobKey",
|
|
5237
|
+
"attempt",
|
|
5238
|
+
"rerunOf",
|
|
5239
|
+
"ciRunUrl",
|
|
5058
5240
|
"startedAt",
|
|
5059
5241
|
"finishedAt",
|
|
5060
5242
|
"durationMs",
|
|
@@ -5065,7 +5247,8 @@
|
|
|
5065
5247
|
"type": "object",
|
|
5066
5248
|
"properties": {
|
|
5067
5249
|
"total": {
|
|
5068
|
-
"type": "integer"
|
|
5250
|
+
"type": "integer",
|
|
5251
|
+
"description": "Tests in the run, every test once however many shards reported it - its final attempt across all of them decides (highest retry, then the lowest shard, then the first repetition). total = passed + failed + skipped + flaky. Since 1.0.0-beta.12; until then each shard's count was summed, so a test reported from two shards counted twice. In a cancelled run the tests the cancel interrupted - reported as skipped - count nowhere, so skipped there counts no test (since 1.0.0-beta.13); the run's tests list still has them."
|
|
5069
5252
|
},
|
|
5070
5253
|
"passed": {
|
|
5071
5254
|
"type": "integer"
|
|
@@ -5111,7 +5294,8 @@
|
|
|
5111
5294
|
"type": "string"
|
|
5112
5295
|
},
|
|
5113
5296
|
"attempts": {
|
|
5114
|
-
"type": "integer"
|
|
5297
|
+
"type": "integer",
|
|
5298
|
+
"description": "Every attempt of the test in the run, across all its shards (since 1.0.0-beta.12; per shard before)."
|
|
5115
5299
|
},
|
|
5116
5300
|
"durationMs": {
|
|
5117
5301
|
"anyOf": [
|
|
@@ -5202,7 +5386,8 @@
|
|
|
5202
5386
|
{
|
|
5203
5387
|
"type": "null"
|
|
5204
5388
|
}
|
|
5205
|
-
]
|
|
5389
|
+
],
|
|
5390
|
+
"description": "The shard the test's final attempt ran in; null for an unsharded run. One row per test since 1.0.0-beta.12 - a test reported from two shards was one row per shard before."
|
|
5206
5391
|
}
|
|
5207
5392
|
},
|
|
5208
5393
|
"required": [
|
|
@@ -5472,7 +5657,8 @@
|
|
|
5472
5657
|
{
|
|
5473
5658
|
"type": "null"
|
|
5474
5659
|
}
|
|
5475
|
-
]
|
|
5660
|
+
],
|
|
5661
|
+
"description": "The shard this attempt ran in; null for an unsharded run."
|
|
5476
5662
|
}
|
|
5477
5663
|
},
|
|
5478
5664
|
"required": [
|
|
@@ -5552,7 +5738,7 @@
|
|
|
5552
5738
|
"local_groups_finished_not_mass_failure",
|
|
5553
5739
|
"all_groups_finished_not_mass_failure"
|
|
5554
5740
|
],
|
|
5555
|
-
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one.
|
|
5741
|
+
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider), the default; local_ - runs no shard reported a CI provider for; all_ - both (since 1.0.0-beta.8; before it, ci_groups_finished_not_mass_failure was the only value, and since 1.0.0-beta.6 - it was ci_completed_not_excluded, which counted every completed shard on its own)."
|
|
5556
5742
|
},
|
|
5557
5743
|
"days": {
|
|
5558
5744
|
"type": "integer",
|
|
@@ -5857,7 +6043,7 @@
|
|
|
5857
6043
|
"local_groups_finished_not_mass_failure",
|
|
5858
6044
|
"all_groups_finished_not_mass_failure"
|
|
5859
6045
|
],
|
|
5860
|
-
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one.
|
|
6046
|
+
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider), the default; local_ - runs no shard reported a CI provider for; all_ - both (since 1.0.0-beta.8; before it, ci_groups_finished_not_mass_failure was the only value, and since 1.0.0-beta.6 - it was ci_completed_not_excluded, which counted every completed shard on its own)."
|
|
5861
6047
|
},
|
|
5862
6048
|
"days": {
|
|
5863
6049
|
"type": "integer",
|
|
@@ -6165,7 +6351,7 @@
|
|
|
6165
6351
|
},
|
|
6166
6352
|
"counted": {
|
|
6167
6353
|
"type": "boolean",
|
|
6168
|
-
"description": "Whether the run is in the metrics' population (see population on the metrics endpoints): a run of the source parameter's kind - CI by default -
|
|
6354
|
+
"description": "Whether the run is in the metrics' population (see population on the metrics endpoints): a run of the source parameter's kind - CI by default - that finished (all its shards, or overdue with every shard that started completed, timed out or stopped by the monthly result limit - since 1.0.0-beta.10, the limit since 1.0.0-beta.11), not a mass failure, and the test not just skipped by a cancellation (since 1.0.0-beta.6; the source since 1.0.0-beta.8)."
|
|
6169
6355
|
},
|
|
6170
6356
|
"branch": {
|
|
6171
6357
|
"anyOf": [
|
|
@@ -6285,7 +6471,7 @@
|
|
|
6285
6471
|
"local_groups_finished",
|
|
6286
6472
|
"all_groups_finished"
|
|
6287
6473
|
],
|
|
6288
|
-
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount).
|
|
6474
|
+
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount). A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider), the default; local_ - runs no shard reported a CI provider for; all_ - both (since 1.0.0-beta.9)."
|
|
6289
6475
|
},
|
|
6290
6476
|
"window": {
|
|
6291
6477
|
"type": "object",
|
|
@@ -6424,7 +6610,7 @@
|
|
|
6424
6610
|
},
|
|
6425
6611
|
"unfinishedRuns": {
|
|
6426
6612
|
"type": "integer",
|
|
6427
|
-
"description": "Runs of the source in the window that have not finished - a shard still running, timed out or never reported. In no figure (since 1.0.0-beta.9)."
|
|
6613
|
+
"description": "Runs of the source in the window that have not finished - a shard still running, or one timed out, stopped by the monthly result limit or never reported while the run is not yet overdue (an overdue one is finished and counted since 1.0.0-beta.10; a limit-stopped one since 1.0.0-beta.11). In no figure (since 1.0.0-beta.9)."
|
|
6428
6614
|
}
|
|
6429
6615
|
},
|
|
6430
6616
|
"required": [
|
|
@@ -6462,7 +6648,7 @@
|
|
|
6462
6648
|
"local_groups_finished",
|
|
6463
6649
|
"all_groups_finished"
|
|
6464
6650
|
],
|
|
6465
|
-
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount).
|
|
6651
|
+
"description": "Which runs every error figure counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - runs that were a mass failure as a whole included (see massFailureRunCount). A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider), the default; local_ - runs no shard reported a CI provider for; all_ - both (since 1.0.0-beta.9)."
|
|
6466
6652
|
},
|
|
6467
6653
|
"window": {
|
|
6468
6654
|
"type": "object",
|
|
@@ -6770,15 +6956,14 @@
|
|
|
6770
6956
|
]
|
|
6771
6957
|
},
|
|
6772
6958
|
"status": {
|
|
6773
|
-
"
|
|
6774
|
-
|
|
6775
|
-
|
|
6776
|
-
|
|
6777
|
-
|
|
6778
|
-
|
|
6779
|
-
}
|
|
6959
|
+
"type": "string",
|
|
6960
|
+
"enum": [
|
|
6961
|
+
"passed",
|
|
6962
|
+
"failed",
|
|
6963
|
+
"cancelled",
|
|
6964
|
+
"incomplete"
|
|
6780
6965
|
],
|
|
6781
|
-
"description": "The run's status over all its shards: passed or
|
|
6966
|
+
"description": "The run's status over all its shards, as GET /runs reports it - a finished run, so never running: passed, failed, cancelled, or incomplete for a run finished by the run timeout - a shard timed out, was stopped by the monthly result limit or never reported (incomplete since 1.0.0-beta.10, the limit since 1.0.0-beta.11; cancelled since 1.0.0-beta.14 - before it, a cancelled run reported the outcome beneath the cancel)."
|
|
6782
6967
|
},
|
|
6783
6968
|
"startedAt": {
|
|
6784
6969
|
"type": "string",
|
|
@@ -6843,11 +7028,11 @@
|
|
|
6843
7028
|
},
|
|
6844
7029
|
"runsInWindow": {
|
|
6845
7030
|
"type": "integer",
|
|
6846
|
-
"description": "Finished runs of the source in the window on branchScope (see the source parameter and testMetrics.population), each counted once however many shards it had, by the start of its first shard; mass-failure and
|
|
7031
|
+
"description": "Finished runs of the source in the window on branchScope (see the source parameter and testMetrics.population), each counted once however many shards it had, by the start of its first shard; mass-failure, cancelled and incomplete runs included (incomplete - finished by the run timeout - since 1.0.0-beta.10). The runs windowStability is counted over."
|
|
6847
7032
|
},
|
|
6848
7033
|
"unfinishedRuns": {
|
|
6849
7034
|
"type": "integer",
|
|
6850
|
-
"description": "Runs of the source in the window on branchScope that have not finished - a shard still running, timed out or never reported. In no other figure (since 1.0.0-beta.7; of the source since 1.0.0-beta.8)."
|
|
7035
|
+
"description": "Runs of the source in the window on branchScope that have not finished - a shard still running, or one timed out, stopped by the monthly result limit or never reported while the run is not yet overdue (an overdue one is finished and counted since 1.0.0-beta.10; a limit-stopped one since 1.0.0-beta.11). In no other figure (since 1.0.0-beta.7; of the source since 1.0.0-beta.8)."
|
|
6851
7036
|
},
|
|
6852
7037
|
"suiteSize": {
|
|
6853
7038
|
"type": "integer",
|
|
@@ -6921,33 +7106,33 @@
|
|
|
6921
7106
|
"local_groups_finished_not_mass_failure",
|
|
6922
7107
|
"all_groups_finished_not_mass_failure"
|
|
6923
7108
|
],
|
|
6924
|
-
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one.
|
|
7109
|
+
"description": "Which runs every number counts: runs whose shards all finished, each counted once however many shards it ran in, placed in the period by the start of their first shard - without runs that were a mass failure as a whole (at least 20 tests, over 80% of them failed). Every branch unless the branch parameter names one. A run finishes when all its shards have, or - once it is overdue by the project's run timeout, counted from its latest shard's start - when every shard that started has completed, timed out or been stopped by the monthly result limit; such a run counts like a cancelled one, with the results its shards reported (since 1.0.0-beta.10; the monthly result limit since 1.0.0-beta.11). Runs still running are not counted. The prefix names the source parameter: ci_ - CI runs (a shard reported a CI provider), the default; local_ - runs no shard reported a CI provider for; all_ - both (since 1.0.0-beta.8; before it, ci_groups_finished_not_mass_failure was the only value, and since 1.0.0-beta.6 - it was ci_completed_not_excluded, which counted every completed shard on its own)."
|
|
6925
7110
|
},
|
|
6926
7111
|
"days": {
|
|
6927
7112
|
"type": "integer",
|
|
6928
7113
|
"description": "The period in days: the same as window.days."
|
|
6929
7114
|
},
|
|
6930
|
-
"
|
|
7115
|
+
"flakyTests": {
|
|
6931
7116
|
"type": "integer",
|
|
6932
|
-
"description": "Of
|
|
7117
|
+
"description": "Of tests, the tests with at least one flaky run in the period (a final attempt that passed after a retry) - the dashboard's Flaky tests tile. Called testsWithRetries until 1.0.0-beta.15."
|
|
6933
7118
|
},
|
|
6934
7119
|
"failingTests": {
|
|
6935
7120
|
"type": "integer",
|
|
6936
|
-
"description": "Of
|
|
7121
|
+
"description": "Of tests, the tests with at least one failed run in the period."
|
|
6937
7122
|
},
|
|
6938
|
-
"
|
|
7123
|
+
"tests": {
|
|
6939
7124
|
"type": "integer",
|
|
6940
|
-
"description": "Tests with at least one run of the population in the period - CI runs by default
|
|
7125
|
+
"description": "Tests with at least one run of the population in the period - CI runs by default. Called ciTests until 1.0.0-beta.15."
|
|
6941
7126
|
}
|
|
6942
7127
|
},
|
|
6943
7128
|
"required": [
|
|
6944
7129
|
"population",
|
|
6945
7130
|
"days",
|
|
6946
|
-
"
|
|
7131
|
+
"flakyTests",
|
|
6947
7132
|
"failingTests",
|
|
6948
|
-
"
|
|
7133
|
+
"tests"
|
|
6949
7134
|
],
|
|
6950
|
-
"description": "The dashboard's
|
|
7135
|
+
"description": "The dashboard's Flaky tests and Failing tests tiles and their denominator, counted like /tests on branchScope (since 1.0.0-beta.7; every branch before). Mass-failure runs are left out here, as in every per-test metric."
|
|
6951
7136
|
},
|
|
6952
7137
|
"slowestTests": {
|
|
6953
7138
|
"type": "array",
|
|
@@ -7045,7 +7230,7 @@
|
|
|
7045
7230
|
"durationMs"
|
|
7046
7231
|
]
|
|
7047
7232
|
},
|
|
7048
|
-
"description": "The last 30 finished runs of the source on branchScope, whatever the window, oldest first - one point per run, passed counting first-try passes, durationMs from its first shard's start to its last shard's finish. Cancelled runs are left out."
|
|
7233
|
+
"description": "The last 30 finished runs of the source on branchScope, whatever the window, oldest first - one point per run, passed counting first-try passes, durationMs from its first shard's start to its last shard's finish. Cancelled and incomplete runs are left out (incomplete since 1.0.0-beta.10)."
|
|
7049
7234
|
}
|
|
7050
7235
|
},
|
|
7051
7236
|
"required": [
|
package/dist/result.js
CHANGED
|
@@ -17,7 +17,7 @@ export function expiryLine(isoDate) {
|
|
|
17
17
|
const shown = Number.isNaN(parsed.getTime())
|
|
18
18
|
? isoDate
|
|
19
19
|
: parsed.toISOString().slice(0, 10);
|
|
20
|
-
return `Note: this Squally read key expires on ${shown}. Create a new one in
|
|
20
|
+
return `Note: this Squally read key expires on ${shown}. Create a new one in Organization -> Read keys before then; expired keys are not renewed.`;
|
|
21
21
|
}
|
|
22
22
|
export function successResult(data, keyExpiresAt) {
|
|
23
23
|
const content = [text(JSON.stringify(data, null, 2))];
|
package/dist/tools.js
CHANGED
|
@@ -23,10 +23,15 @@ export const TOOLS = [
|
|
|
23
23
|
name: "squally-find-run",
|
|
24
24
|
title: "Find runs",
|
|
25
25
|
operationId: "listRuns",
|
|
26
|
-
description: "Finds
|
|
27
|
-
"
|
|
28
|
-
"
|
|
29
|
-
|
|
26
|
+
description: "Finds runs of a project, CI and local alike - the latest, or filtered by branch, commit SHA, " +
|
|
27
|
+
"status or source (all by default; ci or local narrows it, and each run says local). Status " +
|
|
28
|
+
"is passed, failed, cancelled, incomplete or running; settled is false while the run can " +
|
|
29
|
+
"still change. One row per logical run: a sharded run is collapsed into a single entry, its " +
|
|
30
|
+
"counters over all its shards; each job of a CI run that sets a job key is its own run " +
|
|
31
|
+
"(jobKey). attempt and rerunOf say which attempt of its CI run it is and which run it " +
|
|
32
|
+
"re-ran; ciRunUrl links the CI run. Returns summary counters only; use squally-get-run for " +
|
|
33
|
+
"the per-test rows.",
|
|
34
|
+
answers: "Which runs happened — latest, or by branch, commit SHA, status or source. Counters only.",
|
|
30
35
|
cost: "cheap",
|
|
31
36
|
},
|
|
32
37
|
{
|
|
@@ -34,7 +39,10 @@ export const TOOLS = [
|
|
|
34
39
|
title: "Get one run",
|
|
35
40
|
operationId: "getRun",
|
|
36
41
|
description: "One run with its per-test result rows, across every shard. Use this to find out which " +
|
|
37
|
-
"test is red in a run; the run list carries only the counters."
|
|
42
|
+
"test is red in a run; the run list carries only the counters. Its status is cancelled " +
|
|
43
|
+
"for a cancelled run (cancellation says how and when), and local says it ran without CI. " +
|
|
44
|
+
"Like squally-find-run it carries settled, jobKey, attempt, rerunOf and ciRunUrl - for a " +
|
|
45
|
+
"re-run, rerunOf.id is the run it re-ran, to compare with.",
|
|
38
46
|
answers: "One run with its per-test rows, across all shards. Which test is red.",
|
|
39
47
|
cost: "cheap",
|
|
40
48
|
},
|
|
@@ -56,13 +64,15 @@ export const TOOLS = [
|
|
|
56
64
|
"/ runs), flakyRate (runs that passed only after a retry / runs), failureRate (failed " +
|
|
57
65
|
"runs / runs), time lost to retries and failures, its last 20 runs (each with the run " +
|
|
58
66
|
"id squally-find-run gives; runs it skipped included, as result \"skipped\" - shown, " +
|
|
59
|
-
"never counted in runs or any rate) and the branches it ran on. Counts the
|
|
60
|
-
"
|
|
61
|
-
"
|
|
62
|
-
"
|
|
63
|
-
"
|
|
67
|
+
"never counted in runs or any rate) and the branches it ran on. Counts the finished " +
|
|
68
|
+
"CI runs of the period - a sharded run counts once, by its first shard's start - " +
|
|
69
|
+
"without runs that were a mass failure as a whole; source=local or source=all counts " +
|
|
70
|
+
"local runs or both instead (population names which). A run still in progress counts " +
|
|
71
|
+
"once all its shards have finished, or once the run timeout has passed, with the " +
|
|
72
|
+
"results its shards reported - also when a shard was stopped by the monthly result " +
|
|
73
|
+
"limit. There " +
|
|
64
74
|
"is no verdict - the tool returns the numbers and you judge them. runs = 0 " +
|
|
65
|
-
"with null rates is a valid answer: no counted
|
|
75
|
+
"with null rates is a valid answer: no counted run of this test in the period. " +
|
|
66
76
|
"Cheap; for one test use this, not squally-list-tests.",
|
|
67
77
|
answers: "One test's stability, flaky rate and failure rate over a period.",
|
|
68
78
|
cost: "cheap",
|