squally-mcp 0.5.0 → 0.6.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +18 -10
- package/dist/config.js +2 -2
- package/dist/instructions.js +10 -1
- package/dist/openapi/v1.json +175 -20
- package/dist/result.js +1 -1
- package/dist/tools.js +14 -8
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -24,12 +24,12 @@ Worth knowing before you hand a key to an assistant. Through this server an
|
|
|
24
24
|
agent can read:
|
|
25
25
|
|
|
26
26
|
- your **projects**, their names and stable branches;
|
|
27
|
-
- your **CI
|
|
28
|
-
timing, pass/fail counts;
|
|
27
|
+
- your **runs**, CI and local — commit SHA and message, branch, author, pull
|
|
28
|
+
request, job key, a link to the CI run, timing, pass/fail counts;
|
|
29
29
|
- **per-test results** for a run, including which shard ran what;
|
|
30
30
|
- for a failing attempt: the **error message and stack**, the **code snippet**
|
|
31
|
-
— **verbatim source code from your test file** — and the **
|
|
32
|
-
prompt
|
|
31
|
+
— **verbatim source code from your test file** — and the **full-context
|
|
32
|
+
prompt** (the text behind Copy the full context in Squally), which carries the same snippet plus an **ARIA snapshot of your
|
|
33
33
|
application at the moment of failure** (whatever was on screen, truncated to
|
|
34
34
|
3000 characters);
|
|
35
35
|
- **per-test metrics**: how often each test passed on the first try, passed
|
|
@@ -40,13 +40,13 @@ It cannot see screenshots, videos or traces — artifacts are deliberately out o
|
|
|
40
40
|
scope — and it cannot see any key, secret or webhook URL.
|
|
41
41
|
|
|
42
42
|
The key reaches **every project in the organization**. Create one per machine so
|
|
43
|
-
a single revocation does not lock out everything, and revoke it
|
|
44
|
-
|
|
43
|
+
a single revocation does not lock out everything, and revoke it under
|
|
44
|
+
Organization › Read keys when the machine is retired.
|
|
45
45
|
|
|
46
46
|
## Setup
|
|
47
47
|
|
|
48
|
-
You need an **organization read key** (`sqly_ro_…`): Squally →
|
|
49
|
-
keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
48
|
+
You need an **organization read key** (`sqly_ro_…`): Squally → Organization ›
|
|
49
|
+
Read keys → *Create read key*. It is shown once. It is not the same thing as the
|
|
50
50
|
project ingest key your Playwright reporter uses — this server refuses that one
|
|
51
51
|
by name.
|
|
52
52
|
|
|
@@ -140,7 +140,7 @@ squally-mcp trusts your operating system's certificate store, like your browser
|
|
|
140
140
|
| `squally-list-projects` | Which projects exist. **Start here** — every other tool needs a `projectId` from it. | cheap |
|
|
141
141
|
| `squally-find-run` | Which runs happened — latest, or by branch, commit SHA, status or source. Counters only. | cheap |
|
|
142
142
|
| `squally-get-run` | One run with its per-test rows, across all shards. Which test is red. | cheap |
|
|
143
|
-
| `squally-debug-failure` | Every attempt of one test in one run: error, stack,
|
|
143
|
+
| `squally-debug-failure` | Every attempt of one test in one run: error, stack, full-context prompt. | cheap |
|
|
144
144
|
| `squally-get-test-metrics` | One test's stability, flaky rate and failure rate over a period. | cheap — one test |
|
|
145
145
|
| `squally-list-tests` | Every test with a CI run in the period, ranked by stability, flaky rate or failures. | cheap — two queries, paged |
|
|
146
146
|
| `squally-list-errors` | Error signatures in a period: what keeps failing, grouped, and how its runs ended. | cheap |
|
|
@@ -159,6 +159,14 @@ any of them. A cancelled run is `cancelled`; `cancellation` says how, when
|
|
|
159
159
|
and by whom. A sharded run is one row, its counters over all its shards with
|
|
160
160
|
every test once, and the tests a cancel cut off count nowhere.
|
|
161
161
|
|
|
162
|
+
Each run also says whether it is `settled` - final and counted; an
|
|
163
|
+
`incomplete` run that is not settled still waits for a shard within the run
|
|
164
|
+
timeout - and, from squally-reporter 0.11.0, which job of its CI run it is
|
|
165
|
+
(`jobKey`, from `SQUALLY_CI_JOB_KEY`; each job is its own run), which
|
|
166
|
+
`attempt`, which run it re-ran (`rerunOf`) and where the CI run is
|
|
167
|
+
(`ciRunUrl`; for GitHub Actions also for older reporters when the project
|
|
168
|
+
has one linked repository).
|
|
169
|
+
|
|
162
170
|
### What the numbers mean
|
|
163
171
|
|
|
164
172
|
The two test tools count **finished CI runs** in the period (7, 14, 30 or 90
|
|
@@ -236,7 +244,7 @@ The same numbers as that test's row, for one test (abridged):
|
|
|
236
244
|
|
|
237
245
|
Pass the name exactly as Squally shows it. When it exists in several spec
|
|
238
246
|
files, the tool answers with the `filePath` values to retry with; a test with
|
|
239
|
-
results but no counted
|
|
247
|
+
results but no counted run in the period answers `runs: 0` and `null`
|
|
240
248
|
rates — an answer, not an error.
|
|
241
249
|
|
|
242
250
|
## Development
|
package/dist/config.js
CHANGED
|
@@ -55,7 +55,7 @@ export function resolveConfig(env, version) {
|
|
|
55
55
|
return {
|
|
56
56
|
ok: false,
|
|
57
57
|
message: "SQUALLY_API_KEY is not set. Create an organization read key in Squally under " +
|
|
58
|
-
"
|
|
58
|
+
"Organization -> Read keys and set it in this server's environment.",
|
|
59
59
|
};
|
|
60
60
|
}
|
|
61
61
|
// The order matters: an ingest key DOES start with sqly_, so the read-key
|
|
@@ -65,7 +65,7 @@ export function resolveConfig(env, version) {
|
|
|
65
65
|
return {
|
|
66
66
|
ok: false,
|
|
67
67
|
message: "SQUALLY_API_KEY looks like a project ingest key; the MCP server needs an " +
|
|
68
|
-
"organization read key (
|
|
68
|
+
"organization read key (Organization -> Read keys). A read key starts with " +
|
|
69
69
|
`${READ_KEY_PREFIX} and is created separately from the ingest key your ` +
|
|
70
70
|
"Playwright reporter uses.",
|
|
71
71
|
};
|
package/dist/instructions.js
CHANGED
|
@@ -38,6 +38,13 @@
|
|
|
38
38
|
// here because a model that knew the old shape looks for a flag that no longer
|
|
39
39
|
// exists, or reads "cancelled" as an unknown value.
|
|
40
40
|
//
|
|
41
|
+
// SETTLED AND THE JOB KEY SINCE 0.6.0 (API 1.0.0-beta.15): a run that is not
|
|
42
|
+
// settled can still change - an incomplete one may still be waiting for a
|
|
43
|
+
// shard - and one CI run can be several Squally runs, one per job key. Said
|
|
44
|
+
// here because the newest run is the one an agent asks about, and an
|
|
45
|
+
// unsettled "incomplete" read as final, or two runs of one commit read as a
|
|
46
|
+
// duplicate, is the mistake.
|
|
47
|
+
//
|
|
41
48
|
// Exported as `squally-mcp/instructions` (package.json), so it stays a plain
|
|
42
49
|
// constant - test/exports.test.js imports it through the package name.
|
|
43
50
|
export const INSTRUCTIONS = [
|
|
@@ -52,6 +59,8 @@ export const INSTRUCTIONS = [
|
|
|
52
59
|
"first try / runs; flakyRate = runs that passed only after a retry / runs;",
|
|
53
60
|
"failureRate = failed runs / runs. There is no verdict: the tools return numbers,",
|
|
54
61
|
"and you judge them. If a test name is ambiguous, repeat with filePath from the",
|
|
55
|
-
"error. A run's status is passed, failed, cancelled, incomplete or running
|
|
62
|
+
"error. A run's status is passed, failed, cancelled, incomplete or running;",
|
|
63
|
+
"a run that is not settled can still change. Each job of a CI run that sets a",
|
|
64
|
+
"job key is its own run (jobKey).",
|
|
56
65
|
"Errors carry a code and an action; follow the action.",
|
|
57
66
|
].join(" ");
|
package/dist/openapi/v1.json
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema",
|
|
4
4
|
"info": {
|
|
5
5
|
"title": "Squally Read API",
|
|
6
|
-
"version": "1.0.0-beta.
|
|
6
|
+
"version": "1.0.0-beta.16",
|
|
7
7
|
"description": "Read access to a Squally organization's test results, for scripts, CI jobs and AI agents.\n\n**Authentication.** Every request carries an organization read key as `Authorization: Bearer sqly_ro_…`. A read key is organization-wide (every project), created by an organization admin in Organization › Read keys, and expires after 30, 60, 90, 180 days as chosen at creation. There is no renewal: create a new key and revoke the old one. Project ingest keys (`sqly_…`) are refused with `wrong_key_type`.\n\n**Expiry warning.** From 7 days before a key expires, every authenticated response carries `Squally-Key-Expires-At` with the expiry instant (ISO-8601).\n\n**Rate limit.** Currently 60 requests per 60 seconds per key, counted per `Authorization` header - a read key has its own budget, shared with nothing else. Over the limit the edge answers 429 without a JSON body.\n\n**Retries.** 429: wait 60 seconds, then retry. The edge counts requests per key in 60-second windows and sends no Retry-After header. 5xx: retry with exponential backoff, starting at 1 second and capped at 30 seconds. 400, 401, 403, 404: never retry - the same request gets the same answer.\n\n**Plans.** The read API is part of the Standard and Pro plans. A key of an organization on Free is answered 403 `tier_too_low`; keys are not revoked on a downgrade and work again after an upgrade.\n\n**Errors.** Every error is `{\"error\": \"<sentence>\", \"code\": \"<slug>\"}`. The sentence may be reworded; the slug may not. An id that does not exist and an id that belongs to another organization get the same 404, byte for byte.\n\n**Test names in paths.** Test names are Playwright title paths and often contain `/`: URL-encode them, including `/` as `%2F`.\n\n**Sharded runs** are returned as one logical run: /runs lists one row per logical run, and the run routes answer for all shards whichever shard's id you pass.\n\n**Stability.** v1, beta. The API may change until the beta ends; every change is listed in the changelog (docs/openapi/CHANGELOG.md)."
|
|
8
8
|
},
|
|
9
9
|
"servers": [
|
|
@@ -337,7 +337,7 @@
|
|
|
337
337
|
"get": {
|
|
338
338
|
"operationId": "listRuns",
|
|
339
339
|
"summary": "List runs",
|
|
340
|
-
"description": "Runs in a period, newest first - one row per logical run (a sharded CI run is one row), placed in the period and ordered by the start of its first shard (since 1.0.0-beta.13). CI and local runs both, unless source narrows it. Filter by branch, commit SHA prefix, status or source; page with the cursors. Cheap to moderate: bounded by the period's runs.",
|
|
340
|
+
"description": "Runs in a period, newest first - one row per logical run (a sharded CI run is one row; each job of a CI run that sets a job key is its own run, see jobKey), placed in the period and ordered by the start of its first shard (since 1.0.0-beta.13). CI and local runs both, unless source narrows it. Filter by branch, commit SHA prefix, status or source; page with the cursors. Cheap to moderate: bounded by the period's runs.",
|
|
341
341
|
"tags": [
|
|
342
342
|
"Runs"
|
|
343
343
|
],
|
|
@@ -494,6 +494,11 @@
|
|
|
494
494
|
"status": "passed",
|
|
495
495
|
"cancellation": null,
|
|
496
496
|
"local": false,
|
|
497
|
+
"settled": true,
|
|
498
|
+
"jobKey": null,
|
|
499
|
+
"attempt": 1,
|
|
500
|
+
"rerunOf": null,
|
|
501
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18144270113/attempts/1",
|
|
497
502
|
"startedAt": "2026-09-24T07:31:12.000Z",
|
|
498
503
|
"finishedAt": "2026-09-24T07:39:47.000Z",
|
|
499
504
|
"durationMs": 515000,
|
|
@@ -519,6 +524,11 @@
|
|
|
519
524
|
"status": "failed",
|
|
520
525
|
"cancellation": null,
|
|
521
526
|
"local": false,
|
|
527
|
+
"settled": true,
|
|
528
|
+
"jobKey": null,
|
|
529
|
+
"attempt": 1,
|
|
530
|
+
"rerunOf": null,
|
|
531
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18127766204/attempts/1",
|
|
522
532
|
"startedAt": "2026-09-23T16:05:40.000Z",
|
|
523
533
|
"finishedAt": "2026-09-23T16:12:02.000Z",
|
|
524
534
|
"durationMs": 382000,
|
|
@@ -545,6 +555,11 @@
|
|
|
545
555
|
"byUserId": null
|
|
546
556
|
},
|
|
547
557
|
"local": false,
|
|
558
|
+
"settled": true,
|
|
559
|
+
"jobKey": null,
|
|
560
|
+
"attempt": 1,
|
|
561
|
+
"rerunOf": null,
|
|
562
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18127598830/attempts/1",
|
|
548
563
|
"startedAt": "2026-09-23T15:55:02.000Z",
|
|
549
564
|
"finishedAt": "2026-09-23T15:58:31.000Z",
|
|
550
565
|
"durationMs": 209000,
|
|
@@ -851,6 +866,11 @@
|
|
|
851
866
|
"status": "passed",
|
|
852
867
|
"cancellation": null,
|
|
853
868
|
"local": false,
|
|
869
|
+
"settled": true,
|
|
870
|
+
"jobKey": null,
|
|
871
|
+
"attempt": 1,
|
|
872
|
+
"rerunOf": null,
|
|
873
|
+
"ciRunUrl": "https://github.com/example-shop/storefront/actions/runs/18144270113/attempts/1",
|
|
854
874
|
"startedAt": "2026-09-24T07:31:12.000Z",
|
|
855
875
|
"finishedAt": "2026-09-24T07:39:47.000Z",
|
|
856
876
|
"durationMs": 515000,
|
|
@@ -1693,7 +1713,7 @@
|
|
|
1693
1713
|
"get": {
|
|
1694
1714
|
"operationId": "listTests",
|
|
1695
1715
|
"summary": "List tests",
|
|
1696
|
-
"description": "Every test with a run of the source (CI by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Counts
|
|
1716
|
+
"description": "Every test with a run of the source (CI by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Counts finished runs of the source - CI by default - without mass-failure runs, on every branch unless branch names one (see population). Moderate: aggregate queries over the period's results of the source, paged in the database; use the metrics route for one test.",
|
|
1697
1717
|
"tags": [
|
|
1698
1718
|
"Tests"
|
|
1699
1719
|
],
|
|
@@ -3116,7 +3136,7 @@
|
|
|
3116
3136
|
"get": {
|
|
3117
3137
|
"operationId": "listErrors",
|
|
3118
3138
|
"summary": "List errors",
|
|
3119
|
-
"description": "List errors. The project's errors in a period - failures grouped by fingerprint - most frequent first, over the finished runs of the source (see population), all shards of a run one run. Every count covers the period; firstSeen is the first occurrence still stored, not bounded by it. Moderate: one aggregate query.",
|
|
3139
|
+
"description": "List errors. The project's errors in a period - failures grouped by fingerprint - most frequent first: by occurrences, then runs, then tests, ties by fingerprint, so a page never changes order between two requests over the same data. Over the finished runs of the source (see population), all shards of a run one run. Every count covers the period; firstSeen is the first occurrence still stored, not bounded by it. Moderate: one aggregate query, paged in the database.",
|
|
3120
3140
|
"tags": [
|
|
3121
3141
|
"Errors"
|
|
3122
3142
|
],
|
|
@@ -3841,7 +3861,7 @@
|
|
|
3841
3861
|
"get": {
|
|
3842
3862
|
"operationId": "getOverview",
|
|
3843
3863
|
"summary": "Get dashboard figures",
|
|
3844
|
-
"description": "The dashboard for a period: last run, suite stability, time lost, the two test-metric tiles (testMetrics), slowest tests, time sinks, recent runs. Every figure counts finished runs of the source - CI by default - and testMetrics.population names it. Moderate:
|
|
3864
|
+
"description": "The dashboard for a period: last run, suite stability, time lost, the two test-metric tiles (testMetrics), slowest tests, time sinks, recent runs. Every figure counts finished runs of the source - CI by default - and testMetrics.population names it. Moderate: aggregate queries over the period's finished runs, run in the database - no result rows are loaded.",
|
|
3845
3865
|
"tags": [
|
|
3846
3866
|
"Project"
|
|
3847
3867
|
],
|
|
@@ -3957,9 +3977,9 @@
|
|
|
3957
3977
|
"testMetrics": {
|
|
3958
3978
|
"population": "ci_groups_finished_not_mass_failure",
|
|
3959
3979
|
"days": 30,
|
|
3960
|
-
"
|
|
3980
|
+
"flakyTests": 7,
|
|
3961
3981
|
"failingTests": 2,
|
|
3962
|
-
"
|
|
3982
|
+
"tests": 409
|
|
3963
3983
|
},
|
|
3964
3984
|
"slowestTests": [
|
|
3965
3985
|
{
|
|
@@ -4666,6 +4686,67 @@
|
|
|
4666
4686
|
"type": "boolean",
|
|
4667
4687
|
"description": "No shard of the run reported a CI provider: a local run (since 1.0.0-beta.14). GET /runs lists both kinds unless source says otherwise."
|
|
4668
4688
|
},
|
|
4689
|
+
"settled": {
|
|
4690
|
+
"type": "boolean",
|
|
4691
|
+
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while an incomplete run still waits for a shard within the run time limit - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
4692
|
+
},
|
|
4693
|
+
"jobKey": {
|
|
4694
|
+
"anyOf": [
|
|
4695
|
+
{
|
|
4696
|
+
"type": "string"
|
|
4697
|
+
},
|
|
4698
|
+
{
|
|
4699
|
+
"type": "null"
|
|
4700
|
+
}
|
|
4701
|
+
],
|
|
4702
|
+
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check, Squally · <key>. null when the job set none. Since 1.0.0-beta.15."
|
|
4703
|
+
},
|
|
4704
|
+
"attempt": {
|
|
4705
|
+
"anyOf": [
|
|
4706
|
+
{
|
|
4707
|
+
"type": "integer"
|
|
4708
|
+
},
|
|
4709
|
+
{
|
|
4710
|
+
"type": "null"
|
|
4711
|
+
}
|
|
4712
|
+
],
|
|
4713
|
+
"description": "Which attempt of its CI run this is - GitHub Actions' run attempt, 2 after Re-run failed jobs. null when the reporter did not say and the run id did not show it. Since 1.0.0-beta.15."
|
|
4714
|
+
},
|
|
4715
|
+
"rerunOf": {
|
|
4716
|
+
"anyOf": [
|
|
4717
|
+
{
|
|
4718
|
+
"type": "object",
|
|
4719
|
+
"properties": {
|
|
4720
|
+
"id": {
|
|
4721
|
+
"type": "string",
|
|
4722
|
+
"description": "The earlier attempt's id - the one GET /runs/{runId} takes."
|
|
4723
|
+
},
|
|
4724
|
+
"runNumber": {
|
|
4725
|
+
"type": "integer"
|
|
4726
|
+
}
|
|
4727
|
+
},
|
|
4728
|
+
"required": [
|
|
4729
|
+
"id",
|
|
4730
|
+
"runNumber"
|
|
4731
|
+
]
|
|
4732
|
+
},
|
|
4733
|
+
{
|
|
4734
|
+
"type": "null"
|
|
4735
|
+
}
|
|
4736
|
+
],
|
|
4737
|
+
"description": "The run this one re-ran: the latest earlier attempt of the same CI run (and job key) that reported to Squally. null for a first attempt, when the earlier attempt never reported, or once retention deleted it. Since 1.0.0-beta.15."
|
|
4738
|
+
},
|
|
4739
|
+
"ciRunUrl": {
|
|
4740
|
+
"anyOf": [
|
|
4741
|
+
{
|
|
4742
|
+
"type": "string"
|
|
4743
|
+
},
|
|
4744
|
+
{
|
|
4745
|
+
"type": "null"
|
|
4746
|
+
}
|
|
4747
|
+
],
|
|
4748
|
+
"description": "The CI run's page - the run page's View in CI link: the URL the reporter sent (squally-reporter 0.11.0; GitHub Actions links the attempt), else for GitHub Actions one built from the project's linked repository when it has exactly one. https only; null when neither applies. Since 1.0.0-beta.15."
|
|
4749
|
+
},
|
|
4669
4750
|
"startedAt": {
|
|
4670
4751
|
"type": "string",
|
|
4671
4752
|
"format": "date-time"
|
|
@@ -4727,7 +4808,8 @@
|
|
|
4727
4808
|
"type": "object",
|
|
4728
4809
|
"properties": {
|
|
4729
4810
|
"received": {
|
|
4730
|
-
"type": "integer"
|
|
4811
|
+
"type": "integer",
|
|
4812
|
+
"description": "How many shards reported."
|
|
4731
4813
|
},
|
|
4732
4814
|
"expected": {
|
|
4733
4815
|
"anyOf": [
|
|
@@ -4737,7 +4819,8 @@
|
|
|
4737
4819
|
{
|
|
4738
4820
|
"type": "null"
|
|
4739
4821
|
}
|
|
4740
|
-
]
|
|
4822
|
+
],
|
|
4823
|
+
"description": "How many shards the run waits for: the shard total the CI run declared - for a re-run, the shards it re-ran (those that had not passed before it; the others are not run again and do not report). null when no shard declared a total. The run is complete when every expected shard has completed."
|
|
4741
4824
|
}
|
|
4742
4825
|
},
|
|
4743
4826
|
"required": [
|
|
@@ -4761,6 +4844,11 @@
|
|
|
4761
4844
|
"status",
|
|
4762
4845
|
"cancellation",
|
|
4763
4846
|
"local",
|
|
4847
|
+
"settled",
|
|
4848
|
+
"jobKey",
|
|
4849
|
+
"attempt",
|
|
4850
|
+
"rerunOf",
|
|
4851
|
+
"ciRunUrl",
|
|
4764
4852
|
"startedAt",
|
|
4765
4853
|
"finishedAt",
|
|
4766
4854
|
"durationMs",
|
|
@@ -4974,6 +5062,67 @@
|
|
|
4974
5062
|
"type": "boolean",
|
|
4975
5063
|
"description": "No shard of the run reported a CI provider: a local run (since 1.0.0-beta.14). GET /runs lists both kinds unless source says otherwise."
|
|
4976
5064
|
},
|
|
5065
|
+
"settled": {
|
|
5066
|
+
"type": "boolean",
|
|
5067
|
+
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while an incomplete run still waits for a shard within the run time limit - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
5068
|
+
},
|
|
5069
|
+
"jobKey": {
|
|
5070
|
+
"anyOf": [
|
|
5071
|
+
{
|
|
5072
|
+
"type": "string"
|
|
5073
|
+
},
|
|
5074
|
+
{
|
|
5075
|
+
"type": "null"
|
|
5076
|
+
}
|
|
5077
|
+
],
|
|
5078
|
+
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check, Squally · <key>. null when the job set none. Since 1.0.0-beta.15."
|
|
5079
|
+
},
|
|
5080
|
+
"attempt": {
|
|
5081
|
+
"anyOf": [
|
|
5082
|
+
{
|
|
5083
|
+
"type": "integer"
|
|
5084
|
+
},
|
|
5085
|
+
{
|
|
5086
|
+
"type": "null"
|
|
5087
|
+
}
|
|
5088
|
+
],
|
|
5089
|
+
"description": "Which attempt of its CI run this is - GitHub Actions' run attempt, 2 after Re-run failed jobs. null when the reporter did not say and the run id did not show it. Since 1.0.0-beta.15."
|
|
5090
|
+
},
|
|
5091
|
+
"rerunOf": {
|
|
5092
|
+
"anyOf": [
|
|
5093
|
+
{
|
|
5094
|
+
"type": "object",
|
|
5095
|
+
"properties": {
|
|
5096
|
+
"id": {
|
|
5097
|
+
"type": "string",
|
|
5098
|
+
"description": "The earlier attempt's id - the one GET /runs/{runId} takes."
|
|
5099
|
+
},
|
|
5100
|
+
"runNumber": {
|
|
5101
|
+
"type": "integer"
|
|
5102
|
+
}
|
|
5103
|
+
},
|
|
5104
|
+
"required": [
|
|
5105
|
+
"id",
|
|
5106
|
+
"runNumber"
|
|
5107
|
+
]
|
|
5108
|
+
},
|
|
5109
|
+
{
|
|
5110
|
+
"type": "null"
|
|
5111
|
+
}
|
|
5112
|
+
],
|
|
5113
|
+
"description": "The run this one re-ran: the latest earlier attempt of the same CI run (and job key) that reported to Squally. null for a first attempt, when the earlier attempt never reported, or once retention deleted it. Since 1.0.0-beta.15."
|
|
5114
|
+
},
|
|
5115
|
+
"ciRunUrl": {
|
|
5116
|
+
"anyOf": [
|
|
5117
|
+
{
|
|
5118
|
+
"type": "string"
|
|
5119
|
+
},
|
|
5120
|
+
{
|
|
5121
|
+
"type": "null"
|
|
5122
|
+
}
|
|
5123
|
+
],
|
|
5124
|
+
"description": "The CI run's page - the run page's View in CI link: the URL the reporter sent (squally-reporter 0.11.0; GitHub Actions links the attempt), else for GitHub Actions one built from the project's linked repository when it has exactly one. https only; null when neither applies. Since 1.0.0-beta.15."
|
|
5125
|
+
},
|
|
4977
5126
|
"startedAt": {
|
|
4978
5127
|
"type": "string",
|
|
4979
5128
|
"format": "date-time"
|
|
@@ -5013,7 +5162,8 @@
|
|
|
5013
5162
|
{
|
|
5014
5163
|
"type": "null"
|
|
5015
5164
|
}
|
|
5016
|
-
]
|
|
5165
|
+
],
|
|
5166
|
+
"description": "How many shards the run waits for: the shard total the CI run declared - for a re-run, the shards it re-ran (those that had not passed before it; the others are not run again and do not report). null when no shard declared a total. The run is complete when every expected shard has completed."
|
|
5017
5167
|
},
|
|
5018
5168
|
"received": {
|
|
5019
5169
|
"type": "array",
|
|
@@ -5082,6 +5232,11 @@
|
|
|
5082
5232
|
"status",
|
|
5083
5233
|
"cancellation",
|
|
5084
5234
|
"local",
|
|
5235
|
+
"settled",
|
|
5236
|
+
"jobKey",
|
|
5237
|
+
"attempt",
|
|
5238
|
+
"rerunOf",
|
|
5239
|
+
"ciRunUrl",
|
|
5085
5240
|
"startedAt",
|
|
5086
5241
|
"finishedAt",
|
|
5087
5242
|
"durationMs",
|
|
@@ -5473,7 +5628,7 @@
|
|
|
5473
5628
|
"type": "null"
|
|
5474
5629
|
}
|
|
5475
5630
|
],
|
|
5476
|
-
"description": "A prompt for an assistant that has the repository open, byte-identical to the Copy
|
|
5631
|
+
"description": "A prompt for an assistant that has the repository open, byte-identical to the text that Copy the full context puts on the clipboard (Debug with AI on the attempt page). null for an attempt without a real failure (a pass, a test.fail() marker, no error text)."
|
|
5477
5632
|
},
|
|
5478
5633
|
"shard": {
|
|
5479
5634
|
"anyOf": [
|
|
@@ -5503,7 +5658,7 @@
|
|
|
5503
5658
|
"type": "null"
|
|
5504
5659
|
}
|
|
5505
5660
|
],
|
|
5506
|
-
"description": "The shard
|
|
5661
|
+
"description": "The shard this attempt ran in; null for an unsharded run."
|
|
5507
5662
|
}
|
|
5508
5663
|
},
|
|
5509
5664
|
"required": [
|
|
@@ -6957,27 +7112,27 @@
|
|
|
6957
7112
|
"type": "integer",
|
|
6958
7113
|
"description": "The period in days: the same as window.days."
|
|
6959
7114
|
},
|
|
6960
|
-
"
|
|
7115
|
+
"flakyTests": {
|
|
6961
7116
|
"type": "integer",
|
|
6962
|
-
"description": "Of
|
|
7117
|
+
"description": "Of tests, the tests with at least one flaky run in the period (a final attempt that passed after a retry) - the dashboard's Flaky tests tile. Called testsWithRetries until 1.0.0-beta.15."
|
|
6963
7118
|
},
|
|
6964
7119
|
"failingTests": {
|
|
6965
7120
|
"type": "integer",
|
|
6966
|
-
"description": "Of
|
|
7121
|
+
"description": "Of tests, the tests with at least one failed run in the period."
|
|
6967
7122
|
},
|
|
6968
|
-
"
|
|
7123
|
+
"tests": {
|
|
6969
7124
|
"type": "integer",
|
|
6970
|
-
"description": "Tests with at least one run of the population in the period - CI runs by default
|
|
7125
|
+
"description": "Tests with at least one run of the population in the period - CI runs by default. Called ciTests until 1.0.0-beta.15."
|
|
6971
7126
|
}
|
|
6972
7127
|
},
|
|
6973
7128
|
"required": [
|
|
6974
7129
|
"population",
|
|
6975
7130
|
"days",
|
|
6976
|
-
"
|
|
7131
|
+
"flakyTests",
|
|
6977
7132
|
"failingTests",
|
|
6978
|
-
"
|
|
7133
|
+
"tests"
|
|
6979
7134
|
],
|
|
6980
|
-
"description": "The dashboard's
|
|
7135
|
+
"description": "The dashboard's Flaky tests and Failing tests tiles and their denominator, counted like /tests on branchScope (since 1.0.0-beta.7; every branch before). Mass-failure runs are left out here, as in every per-test metric."
|
|
6981
7136
|
},
|
|
6982
7137
|
"slowestTests": {
|
|
6983
7138
|
"type": "array",
|
package/dist/result.js
CHANGED
|
@@ -17,7 +17,7 @@ export function expiryLine(isoDate) {
|
|
|
17
17
|
const shown = Number.isNaN(parsed.getTime())
|
|
18
18
|
? isoDate
|
|
19
19
|
: parsed.toISOString().slice(0, 10);
|
|
20
|
-
return `Note: this Squally read key expires on ${shown}. Create a new one in
|
|
20
|
+
return `Note: this Squally read key expires on ${shown}. Create a new one in Organization -> Read keys before then; expired keys are not renewed.`;
|
|
21
21
|
}
|
|
22
22
|
export function successResult(data, keyExpiresAt) {
|
|
23
23
|
const content = [text(JSON.stringify(data, null, 2))];
|
package/dist/tools.js
CHANGED
|
@@ -25,9 +25,12 @@ export const TOOLS = [
|
|
|
25
25
|
operationId: "listRuns",
|
|
26
26
|
description: "Finds runs of a project, CI and local alike - the latest, or filtered by branch, commit SHA, " +
|
|
27
27
|
"status or source (all by default; ci or local narrows it, and each run says local). Status " +
|
|
28
|
-
"is passed, failed, cancelled, incomplete or running
|
|
29
|
-
"is collapsed into a single entry, its
|
|
30
|
-
"counters
|
|
28
|
+
"is passed, failed, cancelled, incomplete or running; settled is false while the run can " +
|
|
29
|
+
"still change. One row per logical run: a sharded run is collapsed into a single entry, its " +
|
|
30
|
+
"counters over all its shards; each job of a CI run that sets a job key is its own run " +
|
|
31
|
+
"(jobKey). attempt and rerunOf say which attempt of its CI run it is and which run it " +
|
|
32
|
+
"re-ran; ciRunUrl links the CI run. Returns summary counters only; use squally-get-run for " +
|
|
33
|
+
"the per-test rows.",
|
|
31
34
|
answers: "Which runs happened — latest, or by branch, commit SHA, status or source. Counters only.",
|
|
32
35
|
cost: "cheap",
|
|
33
36
|
},
|
|
@@ -37,7 +40,9 @@ export const TOOLS = [
|
|
|
37
40
|
operationId: "getRun",
|
|
38
41
|
description: "One run with its per-test result rows, across every shard. Use this to find out which " +
|
|
39
42
|
"test is red in a run; the run list carries only the counters. Its status is cancelled " +
|
|
40
|
-
"for a cancelled run (cancellation says how and when), and local says it ran without CI."
|
|
43
|
+
"for a cancelled run (cancellation says how and when), and local says it ran without CI. " +
|
|
44
|
+
"Like squally-find-run it carries settled, jobKey, attempt, rerunOf and ciRunUrl - for a " +
|
|
45
|
+
"re-run, rerunOf.id is the run it re-ran, to compare with.",
|
|
41
46
|
answers: "One run with its per-test rows, across all shards. Which test is red.",
|
|
42
47
|
cost: "cheap",
|
|
43
48
|
},
|
|
@@ -46,9 +51,10 @@ export const TOOLS = [
|
|
|
46
51
|
title: "Debug a failing test",
|
|
47
52
|
operationId: "getRunTestAttempts",
|
|
48
53
|
description: "Every attempt of one test in one run, oldest first, each with the error, the stack and " +
|
|
49
|
-
"the
|
|
54
|
+
"the full-context prompt (the same text Copy the full context puts on the clipboard in " +
|
|
55
|
+
"Squally's Debug with AI panel). This is the " +
|
|
50
56
|
"tool for 'why did this test fail'. If the test name is ambiguous, pass filePath.",
|
|
51
|
-
answers: "Every attempt of one test in one run: error, stack,
|
|
57
|
+
answers: "Every attempt of one test in one run: error, stack, full-context prompt.",
|
|
52
58
|
cost: "cheap",
|
|
53
59
|
},
|
|
54
60
|
{
|
|
@@ -67,7 +73,7 @@ export const TOOLS = [
|
|
|
67
73
|
"results its shards reported - also when a shard was stopped by the monthly result " +
|
|
68
74
|
"limit. There " +
|
|
69
75
|
"is no verdict - the tool returns the numbers and you judge them. runs = 0 " +
|
|
70
|
-
"with null rates is a valid answer: no counted
|
|
76
|
+
"with null rates is a valid answer: no counted run of this test in the period. " +
|
|
71
77
|
"Cheap; for one test use this, not squally-list-tests.",
|
|
72
78
|
answers: "One test's stability, flaky rate and failure rate over a period.",
|
|
73
79
|
cost: "cheap",
|
|
@@ -86,7 +92,7 @@ export const TOOLS = [
|
|
|
86
92
|
},
|
|
87
93
|
{
|
|
88
94
|
name: "squally-list-errors",
|
|
89
|
-
title: "List
|
|
95
|
+
title: "List errors",
|
|
90
96
|
operationId: "listErrors",
|
|
91
97
|
description: "The project's error signatures in a period: failures grouped by fingerprint, with how " +
|
|
92
98
|
"often and how recently each occurred. Answers 'is this failure one instance of something " +
|