squally-mcp 0.8.4 → 0.9.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +42 -9
- package/dist/api.js +7 -0
- package/dist/instructions.js +17 -0
- package/dist/openapi/v1.json +1072 -210
- package/dist/openapi.d.ts +2 -1
- package/dist/openapi.js +39 -2
- package/dist/tools.js +30 -13
- package/dist/validate.js +5 -0
- package/package.json +1 -1
package/dist/openapi/v1.json
CHANGED
|
@@ -3,8 +3,8 @@
|
|
|
3
3
|
"jsonSchemaDialect": "https://json-schema.org/draft/2020-12/schema",
|
|
4
4
|
"info": {
|
|
5
5
|
"title": "Squally Read API",
|
|
6
|
-
"version": "1.0.0-beta.
|
|
7
|
-
"description": "Read access to a Squally organization's test results, for scripts, CI jobs and AI agents.\n\n**Authentication.** Every request carries an organization read key as `Authorization: Bearer sqly_ro_…`.
|
|
6
|
+
"version": "1.0.0-beta.19",
|
|
7
|
+
"description": "Read access to a Squally organization's test results, for scripts, CI jobs and AI agents.\n\n**Authentication.** Every request goes to `https://app.squally.dev/api/v1` plus the path and carries an organization read key as `Authorization: Bearer sqly_ro_…`. With the key in `SQUALLY_API_KEY` - the variable the MCP server reads too - the first request lists the organization's projects:\n```sh\ncurl -H \"Authorization: Bearer $SQUALLY_API_KEY\" https://app.squally.dev/api/v1/projects\n```\nA read key is organization-wide (every project), created by an organization admin in Organization › Read keys, and expires after 30, 60, 90, 180 days as chosen at creation. There is no renewal: create a new key and revoke the old one. Project ingest keys (`sqly_…`) are refused with `wrong_key_type`.\n\n**Expiry warning.** From 7 days before a key expires, every authenticated response carries `Squally-Key-Expires-At` with the expiry instant (ISO-8601). Once it has passed, every request with the key is answered 401 `key_expired` (\"This read key expired on <date>. Create a new one in Organization › Read keys.\"); an expired key is never extended. To rotate without a gap: create a new key, switch your callers to it, then revoke the old one - an organization may hold several active keys, and each works until it expires or is revoked.\n\n**Rate limit.** Currently 60 requests per 60 seconds per key - a read key has its own budget, shared with nothing else. Over the limit the API answers 429 with `Retry-After: 60` and the usual error body, `{\"error\": \"Too many requests with this read key. Wait 60 seconds, then retry.\", \"code\": \"rate_limited\"}`. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no `Retry-After`: wait 60 seconds just the same.\n\n**Counts.** A run's `total`, `passed`, `failed`, `flaky` and `skipped` count tests, each test once over all the run's shards by its final attempt: `passed` on the first try, `flaky` after a retry, `failed` when the final attempt failed; `total` = `passed` + `failed` + `skipped` + `flaky`. Figures over a period - a test's runs, an error's occurrences - count finished runs of the `source` parameter's kind, as each field's description says.\n\n**Playwright projects.** A run counts like Playwright: one row and one count per project's variant of a test - the run routes. Anything collected across runs counts a test ONCE across its Playwright projects and names them - `/tests`, a test's metrics and history, `/errors`, `/overview`: a run failed for the test when it failed in any project, flaky when it passed only after a retry in some project and failed in none. Since 1.0.0-beta.19; each project's variant was its own test before.\n\n**Run numbers.** `runNumber` is the run's number in its project, the #12 the app shows: counted up from 1 per project, one number for all shards of a run, never given twice. Runs removed by the plan's retention leave gaps. The run routes take the run's `id`, not its number.\n\n**Paging.** `/projects` and `/projects/{projectId}/branches` answer with every item at once, no paging. `/runs` pages with a cursor (`cursor`, `direction`, `perPage`); `/tests` and `/errors` page by number (`page`, 1-based - a page past the end answers the last page - and `perPage`).\n\n**Retries.** 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. 5xx: retry with exponential backoff, starting at 1 second and capped at 30 seconds. 400, 401, 403, 404: never retry - the same request gets the same answer.\n\n**Plans.** The read API is part of the Standard and Pro plans. A key of an organization on Free is answered 403 `tier_too_low`; keys are not revoked on a downgrade and work again after an upgrade.\n\n**Errors.** Every error is `{\"error\": \"<sentence>\", \"code\": \"<slug>\"}`. The sentence may be reworded; the slug may not. An id that does not exist and an id that belongs to another organization get the same 404, byte for byte.\n\n**Test names in paths.** Test names are Playwright title paths and often contain `/`: URL-encode them, including `/` as `%2F`. The /tests/{testName} routes take a test's name without its project, as /tests lists it; a run's attempts route takes the name the run lists, with its project.\n\n**Sharded runs** are returned as one logical run: /runs lists one row per logical run, and the run routes answer for all shards whichever shard's id you pass.\n\n**Stability.** v1, beta. The API may change until the beta ends; every change is listed in the changelog (docs/openapi/CHANGELOG.md)."
|
|
8
8
|
},
|
|
9
9
|
"servers": [
|
|
10
10
|
{
|
|
@@ -141,8 +141,8 @@
|
|
|
141
141
|
"code": "rate_limited",
|
|
142
142
|
"status": 429,
|
|
143
143
|
"when": "More than 60 requests within 60 seconds with one read key.",
|
|
144
|
-
"action": "Wait
|
|
145
|
-
"hasBody":
|
|
144
|
+
"action": "Wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows.",
|
|
145
|
+
"hasBody": true
|
|
146
146
|
},
|
|
147
147
|
{
|
|
148
148
|
"code": "internal_error",
|
|
@@ -305,7 +305,44 @@
|
|
|
305
305
|
}
|
|
306
306
|
},
|
|
307
307
|
"429": {
|
|
308
|
-
"description": "Rate limited
|
|
308
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
309
|
+
"headers": {
|
|
310
|
+
"Retry-After": {
|
|
311
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
312
|
+
"schema": {
|
|
313
|
+
"type": "integer",
|
|
314
|
+
"example": 60
|
|
315
|
+
}
|
|
316
|
+
}
|
|
317
|
+
},
|
|
318
|
+
"content": {
|
|
319
|
+
"application/json": {
|
|
320
|
+
"schema": {
|
|
321
|
+
"type": "object",
|
|
322
|
+
"properties": {
|
|
323
|
+
"error": {
|
|
324
|
+
"type": "string",
|
|
325
|
+
"description": "A sentence for a human. May be reworded."
|
|
326
|
+
},
|
|
327
|
+
"code": {
|
|
328
|
+
"type": "string",
|
|
329
|
+
"enum": [
|
|
330
|
+
"rate_limited"
|
|
331
|
+
],
|
|
332
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
333
|
+
}
|
|
334
|
+
},
|
|
335
|
+
"required": [
|
|
336
|
+
"error",
|
|
337
|
+
"code"
|
|
338
|
+
]
|
|
339
|
+
},
|
|
340
|
+
"example": {
|
|
341
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
342
|
+
"code": "rate_limited"
|
|
343
|
+
}
|
|
344
|
+
}
|
|
345
|
+
}
|
|
309
346
|
},
|
|
310
347
|
"500": {
|
|
311
348
|
"description": "Error: internal_error",
|
|
@@ -785,7 +822,44 @@
|
|
|
785
822
|
}
|
|
786
823
|
},
|
|
787
824
|
"429": {
|
|
788
|
-
"description": "Rate limited
|
|
825
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
826
|
+
"headers": {
|
|
827
|
+
"Retry-After": {
|
|
828
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
829
|
+
"schema": {
|
|
830
|
+
"type": "integer",
|
|
831
|
+
"example": 60
|
|
832
|
+
}
|
|
833
|
+
}
|
|
834
|
+
},
|
|
835
|
+
"content": {
|
|
836
|
+
"application/json": {
|
|
837
|
+
"schema": {
|
|
838
|
+
"type": "object",
|
|
839
|
+
"properties": {
|
|
840
|
+
"error": {
|
|
841
|
+
"type": "string",
|
|
842
|
+
"description": "A sentence for a human. May be reworded."
|
|
843
|
+
},
|
|
844
|
+
"code": {
|
|
845
|
+
"type": "string",
|
|
846
|
+
"enum": [
|
|
847
|
+
"rate_limited"
|
|
848
|
+
],
|
|
849
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
850
|
+
}
|
|
851
|
+
},
|
|
852
|
+
"required": [
|
|
853
|
+
"error",
|
|
854
|
+
"code"
|
|
855
|
+
]
|
|
856
|
+
},
|
|
857
|
+
"example": {
|
|
858
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
859
|
+
"code": "rate_limited"
|
|
860
|
+
}
|
|
861
|
+
}
|
|
862
|
+
}
|
|
789
863
|
},
|
|
790
864
|
"500": {
|
|
791
865
|
"description": "Error: internal_error",
|
|
@@ -1126,7 +1200,44 @@
|
|
|
1126
1200
|
}
|
|
1127
1201
|
},
|
|
1128
1202
|
"429": {
|
|
1129
|
-
"description": "Rate limited
|
|
1203
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
1204
|
+
"headers": {
|
|
1205
|
+
"Retry-After": {
|
|
1206
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
1207
|
+
"schema": {
|
|
1208
|
+
"type": "integer",
|
|
1209
|
+
"example": 60
|
|
1210
|
+
}
|
|
1211
|
+
}
|
|
1212
|
+
},
|
|
1213
|
+
"content": {
|
|
1214
|
+
"application/json": {
|
|
1215
|
+
"schema": {
|
|
1216
|
+
"type": "object",
|
|
1217
|
+
"properties": {
|
|
1218
|
+
"error": {
|
|
1219
|
+
"type": "string",
|
|
1220
|
+
"description": "A sentence for a human. May be reworded."
|
|
1221
|
+
},
|
|
1222
|
+
"code": {
|
|
1223
|
+
"type": "string",
|
|
1224
|
+
"enum": [
|
|
1225
|
+
"rate_limited"
|
|
1226
|
+
],
|
|
1227
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
1228
|
+
}
|
|
1229
|
+
},
|
|
1230
|
+
"required": [
|
|
1231
|
+
"error",
|
|
1232
|
+
"code"
|
|
1233
|
+
]
|
|
1234
|
+
},
|
|
1235
|
+
"example": {
|
|
1236
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
1237
|
+
"code": "rate_limited"
|
|
1238
|
+
}
|
|
1239
|
+
}
|
|
1240
|
+
}
|
|
1130
1241
|
},
|
|
1131
1242
|
"500": {
|
|
1132
1243
|
"description": "Error: internal_error",
|
|
@@ -1261,7 +1372,7 @@
|
|
|
1261
1372
|
}
|
|
1262
1373
|
}
|
|
1263
1374
|
],
|
|
1264
|
-
"copyPrompt": "You have the repository open. Find the test below, explain the failure and propose a fix.\n\nFailing test: chromium > checkout.spec.ts > Checkout > applies a coupon code\nFile: tests/checkout.spec.ts\
|
|
1375
|
+
"copyPrompt": "You have the repository open. Find the test below, explain the failure and propose a fix.\n\nFailing test: chromium > checkout.spec.ts > Checkout > applies a coupon code\nFile: tests/checkout.spec.ts\nProject: chromium\nRun branch: main\nAttempt: first try\nOutcome: a later retry of this test passed - a flaky run.\n\nError message:\nError: expect(locator).toHaveText(expected) failed\n\nLocator: getByTestId('order-total')\nExpected: \"€ 45.00\"\nReceived: \"€ 50.00\"\nTimeout: 5000ms\nStack (top frames):\nat /home/runner/work/storefront-web/tests/checkout.spec.ts:48:52\nAttempt duration: 6210 ms\nFailure category: Assertion\nLast 30 days on CI: flaky rate 50% (4/8), failure rate 0% (0/8)",
|
|
1265
1376
|
"shard": {
|
|
1266
1377
|
"index": 1,
|
|
1267
1378
|
"runId": "b1e0c6d2-7a43-4f19-8d5e-0c3a9e6f2b71"
|
|
@@ -1518,7 +1629,44 @@
|
|
|
1518
1629
|
}
|
|
1519
1630
|
},
|
|
1520
1631
|
"429": {
|
|
1521
|
-
"description": "Rate limited
|
|
1632
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
1633
|
+
"headers": {
|
|
1634
|
+
"Retry-After": {
|
|
1635
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
1636
|
+
"schema": {
|
|
1637
|
+
"type": "integer",
|
|
1638
|
+
"example": 60
|
|
1639
|
+
}
|
|
1640
|
+
}
|
|
1641
|
+
},
|
|
1642
|
+
"content": {
|
|
1643
|
+
"application/json": {
|
|
1644
|
+
"schema": {
|
|
1645
|
+
"type": "object",
|
|
1646
|
+
"properties": {
|
|
1647
|
+
"error": {
|
|
1648
|
+
"type": "string",
|
|
1649
|
+
"description": "A sentence for a human. May be reworded."
|
|
1650
|
+
},
|
|
1651
|
+
"code": {
|
|
1652
|
+
"type": "string",
|
|
1653
|
+
"enum": [
|
|
1654
|
+
"rate_limited"
|
|
1655
|
+
],
|
|
1656
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
1657
|
+
}
|
|
1658
|
+
},
|
|
1659
|
+
"required": [
|
|
1660
|
+
"error",
|
|
1661
|
+
"code"
|
|
1662
|
+
]
|
|
1663
|
+
},
|
|
1664
|
+
"example": {
|
|
1665
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
1666
|
+
"code": "rate_limited"
|
|
1667
|
+
}
|
|
1668
|
+
}
|
|
1669
|
+
}
|
|
1522
1670
|
},
|
|
1523
1671
|
"500": {
|
|
1524
1672
|
"description": "Error: internal_error",
|
|
@@ -1754,7 +1902,44 @@
|
|
|
1754
1902
|
}
|
|
1755
1903
|
},
|
|
1756
1904
|
"429": {
|
|
1757
|
-
"description": "Rate limited
|
|
1905
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
1906
|
+
"headers": {
|
|
1907
|
+
"Retry-After": {
|
|
1908
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
1909
|
+
"schema": {
|
|
1910
|
+
"type": "integer",
|
|
1911
|
+
"example": 60
|
|
1912
|
+
}
|
|
1913
|
+
}
|
|
1914
|
+
},
|
|
1915
|
+
"content": {
|
|
1916
|
+
"application/json": {
|
|
1917
|
+
"schema": {
|
|
1918
|
+
"type": "object",
|
|
1919
|
+
"properties": {
|
|
1920
|
+
"error": {
|
|
1921
|
+
"type": "string",
|
|
1922
|
+
"description": "A sentence for a human. May be reworded."
|
|
1923
|
+
},
|
|
1924
|
+
"code": {
|
|
1925
|
+
"type": "string",
|
|
1926
|
+
"enum": [
|
|
1927
|
+
"rate_limited"
|
|
1928
|
+
],
|
|
1929
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
1930
|
+
}
|
|
1931
|
+
},
|
|
1932
|
+
"required": [
|
|
1933
|
+
"error",
|
|
1934
|
+
"code"
|
|
1935
|
+
]
|
|
1936
|
+
},
|
|
1937
|
+
"example": {
|
|
1938
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
1939
|
+
"code": "rate_limited"
|
|
1940
|
+
}
|
|
1941
|
+
}
|
|
1942
|
+
}
|
|
1758
1943
|
},
|
|
1759
1944
|
"500": {
|
|
1760
1945
|
"description": "Error: internal_error",
|
|
@@ -1799,7 +1984,7 @@
|
|
|
1799
1984
|
"get": {
|
|
1800
1985
|
"operationId": "listTests",
|
|
1801
1986
|
"summary": "List tests",
|
|
1802
|
-
"description": "Every test with a run of the source (all by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Counts finished runs of the source - all by default - without mass-failure runs, on every branch unless branch names one (see population). Moderate: aggregate queries over the period's results of the source, paged in the database; use the metrics route for one test.",
|
|
1987
|
+
"description": "Every test with a run of the source (all by default) in the period and its metrics - stability, flaky rate, failure rate, runs, time lost, its median duration, its last 20 runs, the branches it ran on and where it fails most - ranked by the chosen sort. Each test once across its Playwright projects, its projects named and each project's own numbers in variants (since 1.0.0-beta.19; one item per project before). Counts finished runs of the source - all by default - without mass-failure runs, on every branch unless branch names one (see population). Moderate: aggregate queries over the period's results of the source, paged in the database; use the metrics route for one test.",
|
|
1803
1988
|
"tags": [
|
|
1804
1989
|
"Tests"
|
|
1805
1990
|
],
|
|
@@ -1867,14 +2052,17 @@
|
|
|
1867
2052
|
}
|
|
1868
2053
|
},
|
|
1869
2054
|
{
|
|
1870
|
-
"name": "
|
|
2055
|
+
"name": "project",
|
|
1871
2056
|
"in": "query",
|
|
1872
2057
|
"required": false,
|
|
1873
|
-
"description": "Only
|
|
2058
|
+
"description": "Only these Playwright projects, exactly (e.g. chromium) - repeat the parameter for several: ?project=chromium&project=firefox. Each test then counts only those projects' variants. Every project when absent. Called browser, one value, until 1.0.0-beta.19 - browser is now a 400.",
|
|
1874
2059
|
"schema": {
|
|
1875
|
-
"type": "
|
|
1876
|
-
"
|
|
1877
|
-
|
|
2060
|
+
"type": "array",
|
|
2061
|
+
"items": {
|
|
2062
|
+
"type": "string",
|
|
2063
|
+
"minLength": 1,
|
|
2064
|
+
"maxLength": 200
|
|
2065
|
+
}
|
|
1878
2066
|
}
|
|
1879
2067
|
},
|
|
1880
2068
|
{
|
|
@@ -1941,17 +2129,20 @@
|
|
|
1941
2129
|
"since": "2026-08-25T08:00:00.000Z",
|
|
1942
2130
|
"until": "2026-09-24T08:00:00.000Z",
|
|
1943
2131
|
"branch": null,
|
|
1944
|
-
"
|
|
2132
|
+
"projectFilter": [],
|
|
1945
2133
|
"sort": "stability",
|
|
1946
2134
|
"page": 1,
|
|
1947
2135
|
"perPage": 3,
|
|
1948
|
-
"total":
|
|
1949
|
-
"pageCount":
|
|
2136
|
+
"total": 205,
|
|
2137
|
+
"pageCount": 69,
|
|
1950
2138
|
"items": [
|
|
1951
2139
|
{
|
|
1952
|
-
"testName": "
|
|
2140
|
+
"testName": "checkout.spec.ts > Checkout > applies a coupon code",
|
|
1953
2141
|
"filePath": "tests/checkout.spec.ts",
|
|
1954
|
-
"
|
|
2142
|
+
"projects": [
|
|
2143
|
+
"chromium",
|
|
2144
|
+
"firefox"
|
|
2145
|
+
],
|
|
1955
2146
|
"runs": 8,
|
|
1956
2147
|
"stableRuns": 4,
|
|
1957
2148
|
"flakyRuns": 4,
|
|
@@ -1960,6 +2151,7 @@
|
|
|
1960
2151
|
"flakyRate": 0.5,
|
|
1961
2152
|
"failureRate": 0,
|
|
1962
2153
|
"timeLostMs": 7361,
|
|
2154
|
+
"medianDurationMs": 1840,
|
|
1963
2155
|
"recentResults": [
|
|
1964
2156
|
{
|
|
1965
2157
|
"runId": "5c8e2a17-9d4b-4f60-b3a1-7e2d9c0f4b58",
|
|
@@ -2026,12 +2218,37 @@
|
|
|
2026
2218
|
"branch": "main",
|
|
2027
2219
|
"kind": "retries",
|
|
2028
2220
|
"count": 3
|
|
2029
|
-
}
|
|
2221
|
+
},
|
|
2222
|
+
"variants": [
|
|
2223
|
+
{
|
|
2224
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
2225
|
+
"project": "chromium",
|
|
2226
|
+
"runs": 8,
|
|
2227
|
+
"stableRuns": 4,
|
|
2228
|
+
"flakyRuns": 4,
|
|
2229
|
+
"failedRuns": 0,
|
|
2230
|
+
"timeLostMs": 7361,
|
|
2231
|
+
"medianDurationMs": 1840
|
|
2232
|
+
},
|
|
2233
|
+
{
|
|
2234
|
+
"testName": "firefox > checkout.spec.ts > Checkout > applies a coupon code",
|
|
2235
|
+
"project": "firefox",
|
|
2236
|
+
"runs": 8,
|
|
2237
|
+
"stableRuns": 8,
|
|
2238
|
+
"flakyRuns": 0,
|
|
2239
|
+
"failedRuns": 0,
|
|
2240
|
+
"timeLostMs": 0,
|
|
2241
|
+
"medianDurationMs": 1795
|
|
2242
|
+
}
|
|
2243
|
+
]
|
|
2030
2244
|
},
|
|
2031
2245
|
{
|
|
2032
|
-
"testName": "
|
|
2246
|
+
"testName": "search.spec.ts > Search > filters by date",
|
|
2033
2247
|
"filePath": "tests/search.spec.ts",
|
|
2034
|
-
"
|
|
2248
|
+
"projects": [
|
|
2249
|
+
"chromium",
|
|
2250
|
+
"firefox"
|
|
2251
|
+
],
|
|
2035
2252
|
"runs": 8,
|
|
2036
2253
|
"stableRuns": 5,
|
|
2037
2254
|
"flakyRuns": 0,
|
|
@@ -2040,6 +2257,7 @@
|
|
|
2040
2257
|
"flakyRate": 0,
|
|
2041
2258
|
"failureRate": 0.375,
|
|
2042
2259
|
"timeLostMs": 14308,
|
|
2260
|
+
"medianDurationMs": 2385,
|
|
2043
2261
|
"recentResults": [
|
|
2044
2262
|
{
|
|
2045
2263
|
"runId": "5c8e2a17-9d4b-4f60-b3a1-7e2d9c0f4b58",
|
|
@@ -2106,12 +2324,37 @@
|
|
|
2106
2324
|
"branch": "main",
|
|
2107
2325
|
"kind": "failures",
|
|
2108
2326
|
"count": 2
|
|
2109
|
-
}
|
|
2327
|
+
},
|
|
2328
|
+
"variants": [
|
|
2329
|
+
{
|
|
2330
|
+
"testName": "chromium > search.spec.ts > Search > filters by date",
|
|
2331
|
+
"project": "chromium",
|
|
2332
|
+
"runs": 8,
|
|
2333
|
+
"stableRuns": 5,
|
|
2334
|
+
"flakyRuns": 0,
|
|
2335
|
+
"failedRuns": 3,
|
|
2336
|
+
"timeLostMs": 14308,
|
|
2337
|
+
"medianDurationMs": 2385
|
|
2338
|
+
},
|
|
2339
|
+
{
|
|
2340
|
+
"testName": "firefox > search.spec.ts > Search > filters by date",
|
|
2341
|
+
"project": "firefox",
|
|
2342
|
+
"runs": 8,
|
|
2343
|
+
"stableRuns": 8,
|
|
2344
|
+
"flakyRuns": 0,
|
|
2345
|
+
"failedRuns": 0,
|
|
2346
|
+
"timeLostMs": 0,
|
|
2347
|
+
"medianDurationMs": 2290
|
|
2348
|
+
}
|
|
2349
|
+
]
|
|
2110
2350
|
},
|
|
2111
2351
|
{
|
|
2112
|
-
"testName": "
|
|
2352
|
+
"testName": "auth.spec.ts > Login > remembers the device",
|
|
2113
2353
|
"filePath": "tests/auth.spec.ts",
|
|
2114
|
-
"
|
|
2354
|
+
"projects": [
|
|
2355
|
+
"chromium",
|
|
2356
|
+
"firefox"
|
|
2357
|
+
],
|
|
2115
2358
|
"runs": 7,
|
|
2116
2359
|
"stableRuns": 6,
|
|
2117
2360
|
"flakyRuns": 1,
|
|
@@ -2120,6 +2363,7 @@
|
|
|
2120
2363
|
"flakyRate": 0.14285714285714285,
|
|
2121
2364
|
"failureRate": 0,
|
|
2122
2365
|
"timeLostMs": 3071,
|
|
2366
|
+
"medianDurationMs": 3012,
|
|
2123
2367
|
"recentResults": [
|
|
2124
2368
|
{
|
|
2125
2369
|
"runId": "5c8e2a17-9d4b-4f60-b3a1-7e2d9c0f4b58",
|
|
@@ -2186,7 +2430,29 @@
|
|
|
2186
2430
|
"branch": "feature/saved-carts",
|
|
2187
2431
|
"kind": "retries",
|
|
2188
2432
|
"count": 1
|
|
2189
|
-
}
|
|
2433
|
+
},
|
|
2434
|
+
"variants": [
|
|
2435
|
+
{
|
|
2436
|
+
"testName": "chromium > auth.spec.ts > Login > remembers the device",
|
|
2437
|
+
"project": "chromium",
|
|
2438
|
+
"runs": 7,
|
|
2439
|
+
"stableRuns": 6,
|
|
2440
|
+
"flakyRuns": 1,
|
|
2441
|
+
"failedRuns": 0,
|
|
2442
|
+
"timeLostMs": 3071,
|
|
2443
|
+
"medianDurationMs": 3012
|
|
2444
|
+
},
|
|
2445
|
+
{
|
|
2446
|
+
"testName": "firefox > auth.spec.ts > Login > remembers the device",
|
|
2447
|
+
"project": "firefox",
|
|
2448
|
+
"runs": 7,
|
|
2449
|
+
"stableRuns": 7,
|
|
2450
|
+
"flakyRuns": 0,
|
|
2451
|
+
"failedRuns": 0,
|
|
2452
|
+
"timeLostMs": 0,
|
|
2453
|
+
"medianDurationMs": 2954
|
|
2454
|
+
}
|
|
2455
|
+
]
|
|
2190
2456
|
}
|
|
2191
2457
|
]
|
|
2192
2458
|
}
|
|
@@ -2386,7 +2652,44 @@
|
|
|
2386
2652
|
}
|
|
2387
2653
|
},
|
|
2388
2654
|
"429": {
|
|
2389
|
-
"description": "Rate limited
|
|
2655
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
2656
|
+
"headers": {
|
|
2657
|
+
"Retry-After": {
|
|
2658
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
2659
|
+
"schema": {
|
|
2660
|
+
"type": "integer",
|
|
2661
|
+
"example": 60
|
|
2662
|
+
}
|
|
2663
|
+
}
|
|
2664
|
+
},
|
|
2665
|
+
"content": {
|
|
2666
|
+
"application/json": {
|
|
2667
|
+
"schema": {
|
|
2668
|
+
"type": "object",
|
|
2669
|
+
"properties": {
|
|
2670
|
+
"error": {
|
|
2671
|
+
"type": "string",
|
|
2672
|
+
"description": "A sentence for a human. May be reworded."
|
|
2673
|
+
},
|
|
2674
|
+
"code": {
|
|
2675
|
+
"type": "string",
|
|
2676
|
+
"enum": [
|
|
2677
|
+
"rate_limited"
|
|
2678
|
+
],
|
|
2679
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
2680
|
+
}
|
|
2681
|
+
},
|
|
2682
|
+
"required": [
|
|
2683
|
+
"error",
|
|
2684
|
+
"code"
|
|
2685
|
+
]
|
|
2686
|
+
},
|
|
2687
|
+
"example": {
|
|
2688
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
2689
|
+
"code": "rate_limited"
|
|
2690
|
+
}
|
|
2691
|
+
}
|
|
2692
|
+
}
|
|
2390
2693
|
},
|
|
2391
2694
|
"500": {
|
|
2392
2695
|
"description": "Error: internal_error",
|
|
@@ -2431,7 +2734,7 @@
|
|
|
2431
2734
|
"get": {
|
|
2432
2735
|
"operationId": "getTestMetrics",
|
|
2433
2736
|
"summary": "Get test metrics",
|
|
2434
|
-
"description": "One test's metrics over the period: the fields of its /tests row, flat, beside the same period. A test with results but no run of the source in the period answers zeros and null rates rather than 404. Cheap: the list's queries, narrowed to one test; prefer it to /tests when the question is about one test.",
|
|
2737
|
+
"description": "One test's metrics over the period: the fields of its /tests row, flat, beside the same period - across its Playwright projects, or one project's with project. A test with results but no run of the source in the period answers zeros and null rates rather than 404. Cheap: the list's queries, narrowed to one test; prefer it to /tests when the question is about one test.",
|
|
2435
2738
|
"tags": [
|
|
2436
2739
|
"Tests"
|
|
2437
2740
|
],
|
|
@@ -2449,7 +2752,7 @@
|
|
|
2449
2752
|
"name": "testName",
|
|
2450
2753
|
"in": "path",
|
|
2451
2754
|
"required": true,
|
|
2452
|
-
"description": "The
|
|
2755
|
+
"description": "The test's name WITHOUT its Playwright project, as /tests lists it (e.g. cart.spec.ts > adds an item): every project of the test. A stored name WITH its project (chromium > cart.spec.ts > adds an item, the identity before 1.0.0-beta.19) is that test in that project only. URL-encode it, including \"/\" as %2F. A name used in several spec files needs ?filePath=.",
|
|
2453
2756
|
"schema": {
|
|
2454
2757
|
"type": "string"
|
|
2455
2758
|
}
|
|
@@ -2463,6 +2766,17 @@
|
|
|
2463
2766
|
"type": "string"
|
|
2464
2767
|
}
|
|
2465
2768
|
},
|
|
2769
|
+
{
|
|
2770
|
+
"name": "project",
|
|
2771
|
+
"in": "query",
|
|
2772
|
+
"required": false,
|
|
2773
|
+
"description": "Only this Playwright project's variant of the test, exactly (e.g. chromium) - the numbers as they were per variant before 1.0.0-beta.19. Every project of the test when absent. A project the test did not run in is 404 test_not_found.",
|
|
2774
|
+
"schema": {
|
|
2775
|
+
"type": "string",
|
|
2776
|
+
"minLength": 1,
|
|
2777
|
+
"maxLength": 200
|
|
2778
|
+
}
|
|
2779
|
+
},
|
|
2466
2780
|
{
|
|
2467
2781
|
"name": "days",
|
|
2468
2782
|
"in": "query",
|
|
@@ -2529,9 +2843,13 @@
|
|
|
2529
2843
|
"since": "2026-08-25T08:00:00.000Z",
|
|
2530
2844
|
"until": "2026-09-24T08:00:00.000Z",
|
|
2531
2845
|
"branch": null,
|
|
2532
|
-
"
|
|
2846
|
+
"projectFilter": null,
|
|
2847
|
+
"testName": "checkout.spec.ts > Checkout > applies a coupon code",
|
|
2533
2848
|
"filePath": "tests/checkout.spec.ts",
|
|
2534
|
-
"
|
|
2849
|
+
"projects": [
|
|
2850
|
+
"chromium",
|
|
2851
|
+
"firefox"
|
|
2852
|
+
],
|
|
2535
2853
|
"runs": 8,
|
|
2536
2854
|
"stableRuns": 4,
|
|
2537
2855
|
"flakyRuns": 4,
|
|
@@ -2540,6 +2858,7 @@
|
|
|
2540
2858
|
"flakyRate": 0.5,
|
|
2541
2859
|
"failureRate": 0,
|
|
2542
2860
|
"timeLostMs": 7361,
|
|
2861
|
+
"medianDurationMs": 1840,
|
|
2543
2862
|
"recentResults": [
|
|
2544
2863
|
{
|
|
2545
2864
|
"runId": "5c8e2a17-9d4b-4f60-b3a1-7e2d9c0f4b58",
|
|
@@ -2606,7 +2925,29 @@
|
|
|
2606
2925
|
"branch": "main",
|
|
2607
2926
|
"kind": "retries",
|
|
2608
2927
|
"count": 3
|
|
2609
|
-
}
|
|
2928
|
+
},
|
|
2929
|
+
"variants": [
|
|
2930
|
+
{
|
|
2931
|
+
"testName": "chromium > checkout.spec.ts > Checkout > applies a coupon code",
|
|
2932
|
+
"project": "chromium",
|
|
2933
|
+
"runs": 8,
|
|
2934
|
+
"stableRuns": 4,
|
|
2935
|
+
"flakyRuns": 4,
|
|
2936
|
+
"failedRuns": 0,
|
|
2937
|
+
"timeLostMs": 7361,
|
|
2938
|
+
"medianDurationMs": 1840
|
|
2939
|
+
},
|
|
2940
|
+
{
|
|
2941
|
+
"testName": "firefox > checkout.spec.ts > Checkout > applies a coupon code",
|
|
2942
|
+
"project": "firefox",
|
|
2943
|
+
"runs": 8,
|
|
2944
|
+
"stableRuns": 8,
|
|
2945
|
+
"flakyRuns": 0,
|
|
2946
|
+
"failedRuns": 0,
|
|
2947
|
+
"timeLostMs": 0,
|
|
2948
|
+
"medianDurationMs": 1795
|
|
2949
|
+
}
|
|
2950
|
+
]
|
|
2610
2951
|
}
|
|
2611
2952
|
}
|
|
2612
2953
|
}
|
|
@@ -2820,10 +3161,16 @@
|
|
|
2820
3161
|
}
|
|
2821
3162
|
},
|
|
2822
3163
|
"429": {
|
|
2823
|
-
"description": "Rate limited
|
|
2824
|
-
|
|
2825
|
-
|
|
2826
|
-
|
|
3164
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
3165
|
+
"headers": {
|
|
3166
|
+
"Retry-After": {
|
|
3167
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
3168
|
+
"schema": {
|
|
3169
|
+
"type": "integer",
|
|
3170
|
+
"example": 60
|
|
3171
|
+
}
|
|
3172
|
+
}
|
|
3173
|
+
},
|
|
2827
3174
|
"content": {
|
|
2828
3175
|
"application/json": {
|
|
2829
3176
|
"schema": {
|
|
@@ -2836,7 +3183,38 @@
|
|
|
2836
3183
|
"code": {
|
|
2837
3184
|
"type": "string",
|
|
2838
3185
|
"enum": [
|
|
2839
|
-
"
|
|
3186
|
+
"rate_limited"
|
|
3187
|
+
],
|
|
3188
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
3189
|
+
}
|
|
3190
|
+
},
|
|
3191
|
+
"required": [
|
|
3192
|
+
"error",
|
|
3193
|
+
"code"
|
|
3194
|
+
]
|
|
3195
|
+
},
|
|
3196
|
+
"example": {
|
|
3197
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
3198
|
+
"code": "rate_limited"
|
|
3199
|
+
}
|
|
3200
|
+
}
|
|
3201
|
+
}
|
|
3202
|
+
},
|
|
3203
|
+
"500": {
|
|
3204
|
+
"description": "Error: internal_error",
|
|
3205
|
+
"content": {
|
|
3206
|
+
"application/json": {
|
|
3207
|
+
"schema": {
|
|
3208
|
+
"type": "object",
|
|
3209
|
+
"properties": {
|
|
3210
|
+
"error": {
|
|
3211
|
+
"type": "string",
|
|
3212
|
+
"description": "A sentence for a human. May be reworded."
|
|
3213
|
+
},
|
|
3214
|
+
"code": {
|
|
3215
|
+
"type": "string",
|
|
3216
|
+
"enum": [
|
|
3217
|
+
"internal_error"
|
|
2840
3218
|
],
|
|
2841
3219
|
"description": "Stable slug - match on this, not on the sentence."
|
|
2842
3220
|
}
|
|
@@ -2865,7 +3243,7 @@
|
|
|
2865
3243
|
"get": {
|
|
2866
3244
|
"operationId": "getTestHistory",
|
|
2867
3245
|
"summary": "Get test history",
|
|
2868
|
-
"description": "Get a test's history across runs. One test in every run of a period, newest first: final status, duration, attempts and time spent on retries. Every run is listed; source decides which are counted. Moderate.",
|
|
3246
|
+
"description": "Get a test's history across runs. One test in every run of a period, newest first: final status, duration, attempts and time spent on retries - across its Playwright projects, each project's own result beside it, or one project's with project. Every run is listed; source decides which are counted. Moderate.",
|
|
2869
3247
|
"tags": [
|
|
2870
3248
|
"Tests"
|
|
2871
3249
|
],
|
|
@@ -2883,7 +3261,7 @@
|
|
|
2883
3261
|
"name": "testName",
|
|
2884
3262
|
"in": "path",
|
|
2885
3263
|
"required": true,
|
|
2886
|
-
"description": "The
|
|
3264
|
+
"description": "The test's name WITHOUT its Playwright project, as /tests lists it (e.g. cart.spec.ts > adds an item): every project of the test. A stored name WITH its project (chromium > cart.spec.ts > adds an item, the identity before 1.0.0-beta.19) is that test in that project only. URL-encode it, including \"/\" as %2F. A name used in several spec files needs ?filePath=.",
|
|
2887
3265
|
"schema": {
|
|
2888
3266
|
"type": "string"
|
|
2889
3267
|
}
|
|
@@ -2897,6 +3275,17 @@
|
|
|
2897
3275
|
"type": "string"
|
|
2898
3276
|
}
|
|
2899
3277
|
},
|
|
3278
|
+
{
|
|
3279
|
+
"name": "project",
|
|
3280
|
+
"in": "query",
|
|
3281
|
+
"required": false,
|
|
3282
|
+
"description": "Only this Playwright project's variant of the test, exactly (e.g. chromium) - the numbers as they were per variant before 1.0.0-beta.19. Every project of the test when absent. A project the test did not run in is 404 test_not_found.",
|
|
3283
|
+
"schema": {
|
|
3284
|
+
"type": "string",
|
|
3285
|
+
"minLength": 1,
|
|
3286
|
+
"maxLength": 200
|
|
3287
|
+
}
|
|
3288
|
+
},
|
|
2900
3289
|
{
|
|
2901
3290
|
"name": "days",
|
|
2902
3291
|
"in": "query",
|
|
@@ -2948,8 +3337,9 @@
|
|
|
2948
3337
|
"name": "storefront-web"
|
|
2949
3338
|
},
|
|
2950
3339
|
"test": {
|
|
2951
|
-
"name": "
|
|
2952
|
-
"filePath": "tests/checkout.spec.ts"
|
|
3340
|
+
"name": "checkout.spec.ts > Checkout > applies a coupon code",
|
|
3341
|
+
"filePath": "tests/checkout.spec.ts",
|
|
3342
|
+
"project": null
|
|
2953
3343
|
},
|
|
2954
3344
|
"window": {
|
|
2955
3345
|
"days": 30,
|
|
@@ -2969,14 +3359,26 @@
|
|
|
2969
3359
|
"durationMs": 1840,
|
|
2970
3360
|
"timedOut": false,
|
|
2971
3361
|
"timeoutMs": 30000,
|
|
2972
|
-
"attempts":
|
|
2973
|
-
"wastedMs": 6210
|
|
3362
|
+
"attempts": 3,
|
|
3363
|
+
"wastedMs": 6210,
|
|
3364
|
+
"projects": [
|
|
3365
|
+
{
|
|
3366
|
+
"project": "chromium",
|
|
3367
|
+
"status": "passed",
|
|
3368
|
+
"attempts": 2
|
|
3369
|
+
},
|
|
3370
|
+
{
|
|
3371
|
+
"project": "firefox",
|
|
3372
|
+
"status": "passed",
|
|
3373
|
+
"attempts": 1
|
|
3374
|
+
}
|
|
3375
|
+
]
|
|
2974
3376
|
},
|
|
2975
3377
|
{
|
|
2976
3378
|
"runId": "7d2f4b19-3c8e-4a61-9e07-5b1d8c2a6f34",
|
|
2977
3379
|
"runNumber": 1283,
|
|
2978
|
-
"local":
|
|
2979
|
-
"counted":
|
|
3380
|
+
"local": false,
|
|
3381
|
+
"counted": true,
|
|
2980
3382
|
"branch": "feature/saved-carts",
|
|
2981
3383
|
"testedRevision": "c4f7a1e9b3d5c8e2f0a6b4d9e1c7f3a5b8d2e0f6",
|
|
2982
3384
|
"startedAt": "2026-09-23T16:05:40.000Z",
|
|
@@ -2984,8 +3386,20 @@
|
|
|
2984
3386
|
"durationMs": 1795,
|
|
2985
3387
|
"timedOut": null,
|
|
2986
3388
|
"timeoutMs": null,
|
|
2987
|
-
"attempts":
|
|
2988
|
-
"wastedMs": 0
|
|
3389
|
+
"attempts": 2,
|
|
3390
|
+
"wastedMs": 0,
|
|
3391
|
+
"projects": [
|
|
3392
|
+
{
|
|
3393
|
+
"project": "chromium",
|
|
3394
|
+
"status": "passed",
|
|
3395
|
+
"attempts": 1
|
|
3396
|
+
},
|
|
3397
|
+
{
|
|
3398
|
+
"project": "firefox",
|
|
3399
|
+
"status": "passed",
|
|
3400
|
+
"attempts": 1
|
|
3401
|
+
}
|
|
3402
|
+
]
|
|
2989
3403
|
}
|
|
2990
3404
|
]
|
|
2991
3405
|
}
|
|
@@ -3201,7 +3615,44 @@
|
|
|
3201
3615
|
}
|
|
3202
3616
|
},
|
|
3203
3617
|
"429": {
|
|
3204
|
-
"description": "Rate limited
|
|
3618
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
3619
|
+
"headers": {
|
|
3620
|
+
"Retry-After": {
|
|
3621
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
3622
|
+
"schema": {
|
|
3623
|
+
"type": "integer",
|
|
3624
|
+
"example": 60
|
|
3625
|
+
}
|
|
3626
|
+
}
|
|
3627
|
+
},
|
|
3628
|
+
"content": {
|
|
3629
|
+
"application/json": {
|
|
3630
|
+
"schema": {
|
|
3631
|
+
"type": "object",
|
|
3632
|
+
"properties": {
|
|
3633
|
+
"error": {
|
|
3634
|
+
"type": "string",
|
|
3635
|
+
"description": "A sentence for a human. May be reworded."
|
|
3636
|
+
},
|
|
3637
|
+
"code": {
|
|
3638
|
+
"type": "string",
|
|
3639
|
+
"enum": [
|
|
3640
|
+
"rate_limited"
|
|
3641
|
+
],
|
|
3642
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
3643
|
+
}
|
|
3644
|
+
},
|
|
3645
|
+
"required": [
|
|
3646
|
+
"error",
|
|
3647
|
+
"code"
|
|
3648
|
+
]
|
|
3649
|
+
},
|
|
3650
|
+
"example": {
|
|
3651
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
3652
|
+
"code": "rate_limited"
|
|
3653
|
+
}
|
|
3654
|
+
}
|
|
3655
|
+
}
|
|
3205
3656
|
},
|
|
3206
3657
|
"500": {
|
|
3207
3658
|
"description": "Error: internal_error",
|
|
@@ -3358,16 +3809,19 @@
|
|
|
3358
3809
|
"fileCount": 1,
|
|
3359
3810
|
"testCount": 1,
|
|
3360
3811
|
"testNames": [
|
|
3361
|
-
"
|
|
3812
|
+
"checkout.spec.ts > Checkout > applies a coupon code"
|
|
3362
3813
|
],
|
|
3363
|
-
"
|
|
3364
|
-
|
|
3365
|
-
|
|
3366
|
-
"
|
|
3367
|
-
"
|
|
3368
|
-
"
|
|
3814
|
+
"projects": [
|
|
3815
|
+
"chromium"
|
|
3816
|
+
],
|
|
3817
|
+
"occurrenceCount": 4,
|
|
3818
|
+
"runCount": 4,
|
|
3819
|
+
"failedCount": 0,
|
|
3820
|
+
"flakyCount": 4,
|
|
3821
|
+
"failedRunCount": 0,
|
|
3822
|
+
"flakyRunCount": 4,
|
|
3369
3823
|
"massFailureRunCount": 0,
|
|
3370
|
-
"firstSeen": "2026-09-
|
|
3824
|
+
"firstSeen": "2026-09-14T15:14:27.000Z",
|
|
3371
3825
|
"lastSeen": "2026-09-24T07:34:05.000Z"
|
|
3372
3826
|
}
|
|
3373
3827
|
],
|
|
@@ -3570,7 +4024,44 @@
|
|
|
3570
4024
|
}
|
|
3571
4025
|
},
|
|
3572
4026
|
"429": {
|
|
3573
|
-
"description": "Rate limited
|
|
4027
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
4028
|
+
"headers": {
|
|
4029
|
+
"Retry-After": {
|
|
4030
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
4031
|
+
"schema": {
|
|
4032
|
+
"type": "integer",
|
|
4033
|
+
"example": 60
|
|
4034
|
+
}
|
|
4035
|
+
}
|
|
4036
|
+
},
|
|
4037
|
+
"content": {
|
|
4038
|
+
"application/json": {
|
|
4039
|
+
"schema": {
|
|
4040
|
+
"type": "object",
|
|
4041
|
+
"properties": {
|
|
4042
|
+
"error": {
|
|
4043
|
+
"type": "string",
|
|
4044
|
+
"description": "A sentence for a human. May be reworded."
|
|
4045
|
+
},
|
|
4046
|
+
"code": {
|
|
4047
|
+
"type": "string",
|
|
4048
|
+
"enum": [
|
|
4049
|
+
"rate_limited"
|
|
4050
|
+
],
|
|
4051
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
4052
|
+
}
|
|
4053
|
+
},
|
|
4054
|
+
"required": [
|
|
4055
|
+
"error",
|
|
4056
|
+
"code"
|
|
4057
|
+
]
|
|
4058
|
+
},
|
|
4059
|
+
"example": {
|
|
4060
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
4061
|
+
"code": "rate_limited"
|
|
4062
|
+
}
|
|
4063
|
+
}
|
|
4064
|
+
}
|
|
3574
4065
|
},
|
|
3575
4066
|
"500": {
|
|
3576
4067
|
"description": "Error: internal_error",
|
|
@@ -3704,27 +4195,33 @@
|
|
|
3704
4195
|
"fileCount": 1,
|
|
3705
4196
|
"testCount": 1,
|
|
3706
4197
|
"testNames": [
|
|
3707
|
-
"
|
|
4198
|
+
"checkout.spec.ts > Checkout > applies a coupon code"
|
|
3708
4199
|
],
|
|
3709
|
-
"
|
|
3710
|
-
|
|
3711
|
-
|
|
3712
|
-
"
|
|
3713
|
-
"
|
|
3714
|
-
"
|
|
4200
|
+
"projects": [
|
|
4201
|
+
"chromium"
|
|
4202
|
+
],
|
|
4203
|
+
"occurrenceCount": 4,
|
|
4204
|
+
"runCount": 4,
|
|
4205
|
+
"failedCount": 0,
|
|
4206
|
+
"flakyCount": 4,
|
|
4207
|
+
"failedRunCount": 0,
|
|
4208
|
+
"flakyRunCount": 4,
|
|
3715
4209
|
"massFailureRunCount": 0,
|
|
3716
|
-
"firstSeen": "2026-09-
|
|
4210
|
+
"firstSeen": "2026-09-14T15:14:27.000Z",
|
|
3717
4211
|
"lastSeen": "2026-09-24T07:34:05.000Z"
|
|
3718
4212
|
},
|
|
3719
4213
|
"windowEmpty": false,
|
|
3720
4214
|
"tests": [
|
|
3721
4215
|
{
|
|
3722
|
-
"testName": "
|
|
4216
|
+
"testName": "checkout.spec.ts > Checkout > applies a coupon code",
|
|
3723
4217
|
"filePath": "tests/checkout.spec.ts",
|
|
3724
|
-
"
|
|
3725
|
-
|
|
3726
|
-
|
|
3727
|
-
"
|
|
4218
|
+
"projects": [
|
|
4219
|
+
"chromium"
|
|
4220
|
+
],
|
|
4221
|
+
"occurrenceCount": 4,
|
|
4222
|
+
"runCount": 4,
|
|
4223
|
+
"failedCount": 0,
|
|
4224
|
+
"flakyCount": 4
|
|
3728
4225
|
}
|
|
3729
4226
|
],
|
|
3730
4227
|
"runs": [
|
|
@@ -3733,12 +4230,18 @@
|
|
|
3733
4230
|
"runNumber": 1284,
|
|
3734
4231
|
"branch": "main",
|
|
3735
4232
|
"startedAt": "2026-09-24T07:31:12.000Z",
|
|
3736
|
-
"
|
|
3737
|
-
"
|
|
3738
|
-
"
|
|
3739
|
-
|
|
3740
|
-
|
|
3741
|
-
|
|
4233
|
+
"status": "flaky",
|
|
4234
|
+
"massFailure": false,
|
|
4235
|
+
"tests": [
|
|
4236
|
+
{
|
|
4237
|
+
"testName": "checkout.spec.ts > Checkout > applies a coupon code",
|
|
4238
|
+
"filePath": "tests/checkout.spec.ts",
|
|
4239
|
+
"projects": [
|
|
4240
|
+
"chromium"
|
|
4241
|
+
],
|
|
4242
|
+
"finalStatus": "passed"
|
|
4243
|
+
}
|
|
4244
|
+
]
|
|
3742
4245
|
}
|
|
3743
4246
|
]
|
|
3744
4247
|
}
|
|
@@ -3946,7 +4449,44 @@
|
|
|
3946
4449
|
}
|
|
3947
4450
|
},
|
|
3948
4451
|
"429": {
|
|
3949
|
-
"description": "Rate limited
|
|
4452
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
4453
|
+
"headers": {
|
|
4454
|
+
"Retry-After": {
|
|
4455
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
4456
|
+
"schema": {
|
|
4457
|
+
"type": "integer",
|
|
4458
|
+
"example": 60
|
|
4459
|
+
}
|
|
4460
|
+
}
|
|
4461
|
+
},
|
|
4462
|
+
"content": {
|
|
4463
|
+
"application/json": {
|
|
4464
|
+
"schema": {
|
|
4465
|
+
"type": "object",
|
|
4466
|
+
"properties": {
|
|
4467
|
+
"error": {
|
|
4468
|
+
"type": "string",
|
|
4469
|
+
"description": "A sentence for a human. May be reworded."
|
|
4470
|
+
},
|
|
4471
|
+
"code": {
|
|
4472
|
+
"type": "string",
|
|
4473
|
+
"enum": [
|
|
4474
|
+
"rate_limited"
|
|
4475
|
+
],
|
|
4476
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
4477
|
+
}
|
|
4478
|
+
},
|
|
4479
|
+
"required": [
|
|
4480
|
+
"error",
|
|
4481
|
+
"code"
|
|
4482
|
+
]
|
|
4483
|
+
},
|
|
4484
|
+
"example": {
|
|
4485
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
4486
|
+
"code": "rate_limited"
|
|
4487
|
+
}
|
|
4488
|
+
}
|
|
4489
|
+
}
|
|
3950
4490
|
},
|
|
3951
4491
|
"500": {
|
|
3952
4492
|
"description": "Error: internal_error",
|
|
@@ -4080,14 +4620,14 @@
|
|
|
4080
4620
|
"startedAt": "2026-09-24T07:31:12.000Z",
|
|
4081
4621
|
"finishedAt": "2026-09-24T07:39:47.000Z",
|
|
4082
4622
|
"total": 412,
|
|
4083
|
-
"passed":
|
|
4084
|
-
"flaky":
|
|
4623
|
+
"passed": 408,
|
|
4624
|
+
"flaky": 1,
|
|
4085
4625
|
"failed": 0,
|
|
4086
|
-
"skipped":
|
|
4626
|
+
"skipped": 3
|
|
4087
4627
|
},
|
|
4088
4628
|
"runsInWindow": 11,
|
|
4089
4629
|
"unfinishedRuns": 1,
|
|
4090
|
-
"suiteSize":
|
|
4630
|
+
"suiteSize": 206,
|
|
4091
4631
|
"windowStability": 99,
|
|
4092
4632
|
"timeLost": {
|
|
4093
4633
|
"lostMs": 5112000,
|
|
@@ -4095,30 +4635,33 @@
|
|
|
4095
4635
|
"wastedAttempts": 71,
|
|
4096
4636
|
"wallClockMs": 213000000
|
|
4097
4637
|
},
|
|
4098
|
-
"failingTests": {
|
|
4099
|
-
"failing": 1,
|
|
4100
|
-
"testsRun": 412
|
|
4101
|
-
},
|
|
4102
4638
|
"testMetrics": {
|
|
4103
4639
|
"population": "ci_groups_finished_not_mass_failure",
|
|
4104
4640
|
"days": 30,
|
|
4105
4641
|
"flakyTests": 7,
|
|
4106
4642
|
"failingTests": 2,
|
|
4107
|
-
"tests":
|
|
4643
|
+
"tests": 205
|
|
4108
4644
|
},
|
|
4109
4645
|
"slowestTests": [
|
|
4110
4646
|
{
|
|
4111
|
-
"testName": "
|
|
4647
|
+
"testName": "checkout.spec.ts > Checkout > pays with a saved card",
|
|
4112
4648
|
"filePath": "tests/checkout.spec.ts",
|
|
4649
|
+
"projects": [
|
|
4650
|
+
"chromium",
|
|
4651
|
+
"firefox"
|
|
4652
|
+
],
|
|
4113
4653
|
"medianMs": 14820,
|
|
4114
|
-
"runs":
|
|
4654
|
+
"runs": 11
|
|
4115
4655
|
}
|
|
4116
4656
|
],
|
|
4117
4657
|
"timeSinks": [
|
|
4118
4658
|
{
|
|
4119
|
-
"testName": "
|
|
4659
|
+
"testName": "checkout.spec.ts > Checkout > applies a coupon code",
|
|
4120
4660
|
"filePath": "tests/checkout.spec.ts",
|
|
4121
|
-
"
|
|
4661
|
+
"projects": [
|
|
4662
|
+
"chromium"
|
|
4663
|
+
],
|
|
4664
|
+
"lostMs": 18630
|
|
4122
4665
|
}
|
|
4123
4666
|
],
|
|
4124
4667
|
"runsSeries": [
|
|
@@ -4332,7 +4875,44 @@
|
|
|
4332
4875
|
}
|
|
4333
4876
|
},
|
|
4334
4877
|
"429": {
|
|
4335
|
-
"description": "Rate limited
|
|
4878
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
4879
|
+
"headers": {
|
|
4880
|
+
"Retry-After": {
|
|
4881
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
4882
|
+
"schema": {
|
|
4883
|
+
"type": "integer",
|
|
4884
|
+
"example": 60
|
|
4885
|
+
}
|
|
4886
|
+
}
|
|
4887
|
+
},
|
|
4888
|
+
"content": {
|
|
4889
|
+
"application/json": {
|
|
4890
|
+
"schema": {
|
|
4891
|
+
"type": "object",
|
|
4892
|
+
"properties": {
|
|
4893
|
+
"error": {
|
|
4894
|
+
"type": "string",
|
|
4895
|
+
"description": "A sentence for a human. May be reworded."
|
|
4896
|
+
},
|
|
4897
|
+
"code": {
|
|
4898
|
+
"type": "string",
|
|
4899
|
+
"enum": [
|
|
4900
|
+
"rate_limited"
|
|
4901
|
+
],
|
|
4902
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
4903
|
+
}
|
|
4904
|
+
},
|
|
4905
|
+
"required": [
|
|
4906
|
+
"error",
|
|
4907
|
+
"code"
|
|
4908
|
+
]
|
|
4909
|
+
},
|
|
4910
|
+
"example": {
|
|
4911
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
4912
|
+
"code": "rate_limited"
|
|
4913
|
+
}
|
|
4914
|
+
}
|
|
4915
|
+
}
|
|
4336
4916
|
},
|
|
4337
4917
|
"500": {
|
|
4338
4918
|
"description": "Error: internal_error",
|
|
@@ -4575,7 +5155,44 @@
|
|
|
4575
5155
|
}
|
|
4576
5156
|
},
|
|
4577
5157
|
"429": {
|
|
4578
|
-
"description": "Rate limited
|
|
5158
|
+
"description": "Rate limited: more than 60 requests within 60 seconds with one read key. 429: wait the seconds Retry-After names (60), then retry. Requests are counted per key in 60-second windows. A caller far beyond the limit can be stopped earlier by the edge firewall with a bare 429 - no body, no Retry-After; wait 60 seconds just the same.",
|
|
5159
|
+
"headers": {
|
|
5160
|
+
"Retry-After": {
|
|
5161
|
+
"description": "Seconds to wait before retrying: 60.",
|
|
5162
|
+
"schema": {
|
|
5163
|
+
"type": "integer",
|
|
5164
|
+
"example": 60
|
|
5165
|
+
}
|
|
5166
|
+
}
|
|
5167
|
+
},
|
|
5168
|
+
"content": {
|
|
5169
|
+
"application/json": {
|
|
5170
|
+
"schema": {
|
|
5171
|
+
"type": "object",
|
|
5172
|
+
"properties": {
|
|
5173
|
+
"error": {
|
|
5174
|
+
"type": "string",
|
|
5175
|
+
"description": "A sentence for a human. May be reworded."
|
|
5176
|
+
},
|
|
5177
|
+
"code": {
|
|
5178
|
+
"type": "string",
|
|
5179
|
+
"enum": [
|
|
5180
|
+
"rate_limited"
|
|
5181
|
+
],
|
|
5182
|
+
"description": "Stable slug - match on this, not on the sentence."
|
|
5183
|
+
}
|
|
5184
|
+
},
|
|
5185
|
+
"required": [
|
|
5186
|
+
"error",
|
|
5187
|
+
"code"
|
|
5188
|
+
]
|
|
5189
|
+
},
|
|
5190
|
+
"example": {
|
|
5191
|
+
"error": "Too many requests with this read key. Wait 60 seconds, then retry.",
|
|
5192
|
+
"code": "rate_limited"
|
|
5193
|
+
}
|
|
5194
|
+
}
|
|
5195
|
+
}
|
|
4579
5196
|
},
|
|
4580
5197
|
"500": {
|
|
4581
5198
|
"description": "Error: internal_error",
|
|
@@ -4804,7 +5421,7 @@
|
|
|
4804
5421
|
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while the run still waits for a shard within the run time limit, counted from its shards' last activity - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
4805
5422
|
},
|
|
4806
5423
|
"jobKey": {
|
|
4807
|
-
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check,
|
|
5424
|
+
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check, <check name> · <key>: the check name of the linked repository, \"Squally\" for the first project that links a repository. null when the job set none. Since 1.0.0-beta.15.",
|
|
4808
5425
|
"type": [
|
|
4809
5426
|
"string",
|
|
4810
5427
|
"null"
|
|
@@ -5127,7 +5744,7 @@
|
|
|
5127
5744
|
"description": "Whether the run is settled: its status and counts are final and it counts in the dashboard's figures. false while a shard is running, and while the run still waits for a shard within the run time limit, counted from its shards' last activity - once that passes, the hourly check settles it as incomplete. The dashboard's unfinished runs are the ones not settled. Since 1.0.0-beta.15."
|
|
5128
5745
|
},
|
|
5129
5746
|
"jobKey": {
|
|
5130
|
-
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check,
|
|
5747
|
+
"description": "The job key the reporter sent (SQUALLY_CI_JOB_KEY, squally-reporter 0.11.0): which job of a CI run this is, for example the browser of a matrix. Each job is its own run with its own GitHub check, <check name> · <key>: the check name of the linked repository, \"Squally\" for the first project that links a repository. null when the job set none. Since 1.0.0-beta.15.",
|
|
5131
5748
|
"type": [
|
|
5132
5749
|
"string",
|
|
5133
5750
|
"null"
|
|
@@ -5905,12 +6522,12 @@
|
|
|
5905
6522
|
"null"
|
|
5906
6523
|
]
|
|
5907
6524
|
},
|
|
5908
|
-
"
|
|
5909
|
-
"
|
|
5910
|
-
"
|
|
5911
|
-
"string"
|
|
5912
|
-
|
|
5913
|
-
|
|
6525
|
+
"projectFilter": {
|
|
6526
|
+
"type": "array",
|
|
6527
|
+
"items": {
|
|
6528
|
+
"type": "string"
|
|
6529
|
+
},
|
|
6530
|
+
"description": "The project filter applied - the project parameter's values, code-point order; empty for every project. Each item then holds only those projects' variants (since 1.0.0-beta.19; browser, one value, before)."
|
|
5914
6531
|
},
|
|
5915
6532
|
"sort": {
|
|
5916
6533
|
"type": "string",
|
|
@@ -5930,7 +6547,7 @@
|
|
|
5930
6547
|
},
|
|
5931
6548
|
"total": {
|
|
5932
6549
|
"type": "integer",
|
|
5933
|
-
"description": "Tests matching the query across all pages: every test with at least one run of the population in the period, narrowed by search, branch and
|
|
6550
|
+
"description": "Tests matching the query across all pages - each test once across projects: every test with at least one run of the population in the period, narrowed by search, branch and project."
|
|
5934
6551
|
},
|
|
5935
6552
|
"pageCount": {
|
|
5936
6553
|
"type": "integer"
|
|
@@ -5942,7 +6559,7 @@
|
|
|
5942
6559
|
"properties": {
|
|
5943
6560
|
"testName": {
|
|
5944
6561
|
"type": "string",
|
|
5945
|
-
"description": "The
|
|
6562
|
+
"description": "The test's name WITHOUT its Playwright project (\"cart.spec.ts > adds an item\" for chromium's and firefox's \"<project> > cart.spec.ts > adds an item\") - the identity GET /tests/{testName} takes. Since 1.0.0-beta.19; the stored name with its project, one item per project, before."
|
|
5946
6563
|
},
|
|
5947
6564
|
"filePath": {
|
|
5948
6565
|
"type": [
|
|
@@ -5950,28 +6567,28 @@
|
|
|
5950
6567
|
"null"
|
|
5951
6568
|
]
|
|
5952
6569
|
},
|
|
5953
|
-
"
|
|
5954
|
-
"
|
|
5955
|
-
"
|
|
5956
|
-
"string"
|
|
5957
|
-
|
|
5958
|
-
|
|
6570
|
+
"projects": {
|
|
6571
|
+
"type": "array",
|
|
6572
|
+
"items": {
|
|
6573
|
+
"type": "string"
|
|
6574
|
+
},
|
|
6575
|
+
"description": "Every Playwright project the test ran in, code-point order; empty when it ran under none. Replaces browser (1.0.0-beta.19)."
|
|
5959
6576
|
},
|
|
5960
6577
|
"runs": {
|
|
5961
6578
|
"type": "integer",
|
|
5962
|
-
"description": "Runs of the population in which the test has a final attempt that did not skip: stableRuns + flakyRuns + failedRuns. A sharded run counts once; the final attempt is the one with the most retries, then from the lowest shard."
|
|
6579
|
+
"description": "Runs of the population in which the test has a final attempt that did not skip in any of its projects: stableRuns + flakyRuns + failedRuns. A sharded run counts once; per project, the final attempt is the one with the most retries, then from the lowest shard."
|
|
5963
6580
|
},
|
|
5964
6581
|
"stableRuns": {
|
|
5965
6582
|
"type": "integer",
|
|
5966
|
-
"description": "Of runs, those
|
|
6583
|
+
"description": "Of runs, those in which it passed on the first try wherever it ran."
|
|
5967
6584
|
},
|
|
5968
6585
|
"flakyRuns": {
|
|
5969
6586
|
"type": "integer",
|
|
5970
|
-
"description": "Of runs, those
|
|
6587
|
+
"description": "Of runs, those in which it passed only after a retry in some project and failed in none."
|
|
5971
6588
|
},
|
|
5972
6589
|
"failedRuns": {
|
|
5973
6590
|
"type": "integer",
|
|
5974
|
-
"description": "Of runs, those
|
|
6591
|
+
"description": "Of runs, those in which its final attempt failed in any project - failed wins over flaky."
|
|
5975
6592
|
},
|
|
5976
6593
|
"stability": {
|
|
5977
6594
|
"description": "stableRuns / runs, a fraction from 0 to 1. null only when runs is 0.",
|
|
@@ -5996,7 +6613,18 @@
|
|
|
5996
6613
|
},
|
|
5997
6614
|
"timeLostMs": {
|
|
5998
6615
|
"type": "integer",
|
|
5999
|
-
"description": "Time spent on attempts that did not count, in ms, summed over runs: a failed
|
|
6616
|
+
"description": "Time spent on attempts that did not count, in ms, summed over runs and projects: a failed final attempt loses every attempt it made, one rescued by a retry the attempts before the pass, each at the final attempt's duration. 0 when every run passed on the first try."
|
|
6617
|
+
},
|
|
6618
|
+
"medianDurationMs": {
|
|
6619
|
+
"anyOf": [
|
|
6620
|
+
{
|
|
6621
|
+
"type": "integer"
|
|
6622
|
+
},
|
|
6623
|
+
{
|
|
6624
|
+
"type": "null"
|
|
6625
|
+
}
|
|
6626
|
+
],
|
|
6627
|
+
"description": "Per run the slowest project's final attempt, the median over the runs (the lower middle); null when no run names a duration (since 1.0.0-beta.19)."
|
|
6000
6628
|
},
|
|
6001
6629
|
"recentResults": {
|
|
6002
6630
|
"type": "array",
|
|
@@ -6040,7 +6668,7 @@
|
|
|
6040
6668
|
"result"
|
|
6041
6669
|
]
|
|
6042
6670
|
},
|
|
6043
|
-
"description": "The test's last 20 runs in the period, oldest first,
|
|
6671
|
+
"description": "The test's last 20 runs in the period, oldest first, each over its projects: failed when it failed in any, flaky when it passed after a retry in one and failed in none, stable, or skipped when every project skipped. Empty when it has none."
|
|
6044
6672
|
},
|
|
6045
6673
|
"branches": {
|
|
6046
6674
|
"type": "array",
|
|
@@ -6085,12 +6713,70 @@
|
|
|
6085
6713
|
}
|
|
6086
6714
|
],
|
|
6087
6715
|
"description": "The branch with the most failed runs (kind failures, count = those runs); when no run failed, the branch with the most flaky runs (kind retries, count = those runs); null when neither. Ties go to the branch name in code-point order, a null branch last."
|
|
6716
|
+
},
|
|
6717
|
+
"variants": {
|
|
6718
|
+
"type": "array",
|
|
6719
|
+
"items": {
|
|
6720
|
+
"type": "object",
|
|
6721
|
+
"properties": {
|
|
6722
|
+
"testName": {
|
|
6723
|
+
"type": "string",
|
|
6724
|
+
"description": "The variant's stored name: Playwright's title path, its project first - the name GET /runs/{runId} and the attempts route carry."
|
|
6725
|
+
},
|
|
6726
|
+
"project": {
|
|
6727
|
+
"description": "Its Playwright project; null for a result that ran under no named project.",
|
|
6728
|
+
"type": [
|
|
6729
|
+
"string",
|
|
6730
|
+
"null"
|
|
6731
|
+
]
|
|
6732
|
+
},
|
|
6733
|
+
"runs": {
|
|
6734
|
+
"type": "integer",
|
|
6735
|
+
"description": "Runs of the population in which this project's final attempt did not skip."
|
|
6736
|
+
},
|
|
6737
|
+
"stableRuns": {
|
|
6738
|
+
"type": "integer"
|
|
6739
|
+
},
|
|
6740
|
+
"flakyRuns": {
|
|
6741
|
+
"type": "integer"
|
|
6742
|
+
},
|
|
6743
|
+
"failedRuns": {
|
|
6744
|
+
"type": "integer"
|
|
6745
|
+
},
|
|
6746
|
+
"timeLostMs": {
|
|
6747
|
+
"type": "integer",
|
|
6748
|
+
"description": "This project's time lost - see the test's timeLostMs."
|
|
6749
|
+
},
|
|
6750
|
+
"medianDurationMs": {
|
|
6751
|
+
"anyOf": [
|
|
6752
|
+
{
|
|
6753
|
+
"type": "integer"
|
|
6754
|
+
},
|
|
6755
|
+
{
|
|
6756
|
+
"type": "null"
|
|
6757
|
+
}
|
|
6758
|
+
],
|
|
6759
|
+
"description": "The median of this project's final-attempt durations over its runs, the lower middle; null when none names one."
|
|
6760
|
+
}
|
|
6761
|
+
},
|
|
6762
|
+
"required": [
|
|
6763
|
+
"testName",
|
|
6764
|
+
"project",
|
|
6765
|
+
"runs",
|
|
6766
|
+
"stableRuns",
|
|
6767
|
+
"flakyRuns",
|
|
6768
|
+
"failedRuns",
|
|
6769
|
+
"timeLostMs",
|
|
6770
|
+
"medianDurationMs"
|
|
6771
|
+
]
|
|
6772
|
+
},
|
|
6773
|
+
"description": "One item per project's variant, project in code-point order (none last), each with its own numbers - the Test explorer's breakdown (since 1.0.0-beta.19)."
|
|
6088
6774
|
}
|
|
6089
6775
|
},
|
|
6090
6776
|
"required": [
|
|
6091
6777
|
"testName",
|
|
6092
6778
|
"filePath",
|
|
6093
|
-
"
|
|
6779
|
+
"projects",
|
|
6094
6780
|
"runs",
|
|
6095
6781
|
"stableRuns",
|
|
6096
6782
|
"flakyRuns",
|
|
@@ -6099,9 +6785,11 @@
|
|
|
6099
6785
|
"flakyRate",
|
|
6100
6786
|
"failureRate",
|
|
6101
6787
|
"timeLostMs",
|
|
6788
|
+
"medianDurationMs",
|
|
6102
6789
|
"recentResults",
|
|
6103
6790
|
"branches",
|
|
6104
|
-
"topBranch"
|
|
6791
|
+
"topBranch",
|
|
6792
|
+
"variants"
|
|
6105
6793
|
]
|
|
6106
6794
|
}
|
|
6107
6795
|
}
|
|
@@ -6113,7 +6801,7 @@
|
|
|
6113
6801
|
"since",
|
|
6114
6802
|
"until",
|
|
6115
6803
|
"branch",
|
|
6116
|
-
"
|
|
6804
|
+
"projectFilter",
|
|
6117
6805
|
"sort",
|
|
6118
6806
|
"page",
|
|
6119
6807
|
"perPage",
|
|
@@ -6170,38 +6858,45 @@
|
|
|
6170
6858
|
"null"
|
|
6171
6859
|
]
|
|
6172
6860
|
},
|
|
6173
|
-
"
|
|
6174
|
-
"
|
|
6175
|
-
"description": "The stored name: Playwright's title path, the Playwright project first when there is one. The identity other test routes take."
|
|
6176
|
-
},
|
|
6177
|
-
"filePath": {
|
|
6861
|
+
"projectFilter": {
|
|
6862
|
+
"description": "The one project the numbers count - the project parameter, or the project of the variant name the path gave; null for every project of the test (since 1.0.0-beta.19).",
|
|
6178
6863
|
"type": [
|
|
6179
6864
|
"string",
|
|
6180
6865
|
"null"
|
|
6181
6866
|
]
|
|
6182
6867
|
},
|
|
6183
|
-
"
|
|
6184
|
-
"
|
|
6868
|
+
"testName": {
|
|
6869
|
+
"type": "string",
|
|
6870
|
+
"description": "The test's name WITHOUT its Playwright project (\"cart.spec.ts > adds an item\" for chromium's and firefox's \"<project> > cart.spec.ts > adds an item\") - the identity GET /tests/{testName} takes. Since 1.0.0-beta.19; the stored name with its project, one item per project, before."
|
|
6871
|
+
},
|
|
6872
|
+
"filePath": {
|
|
6185
6873
|
"type": [
|
|
6186
6874
|
"string",
|
|
6187
6875
|
"null"
|
|
6188
6876
|
]
|
|
6189
6877
|
},
|
|
6878
|
+
"projects": {
|
|
6879
|
+
"type": "array",
|
|
6880
|
+
"items": {
|
|
6881
|
+
"type": "string"
|
|
6882
|
+
},
|
|
6883
|
+
"description": "Every Playwright project the test ran in, code-point order; empty when it ran under none. Replaces browser (1.0.0-beta.19)."
|
|
6884
|
+
},
|
|
6190
6885
|
"runs": {
|
|
6191
6886
|
"type": "integer",
|
|
6192
|
-
"description": "Runs of the population in which the test has a final attempt that did not skip: stableRuns + flakyRuns + failedRuns. A sharded run counts once; the final attempt is the one with the most retries, then from the lowest shard."
|
|
6887
|
+
"description": "Runs of the population in which the test has a final attempt that did not skip in any of its projects: stableRuns + flakyRuns + failedRuns. A sharded run counts once; per project, the final attempt is the one with the most retries, then from the lowest shard."
|
|
6193
6888
|
},
|
|
6194
6889
|
"stableRuns": {
|
|
6195
6890
|
"type": "integer",
|
|
6196
|
-
"description": "Of runs, those
|
|
6891
|
+
"description": "Of runs, those in which it passed on the first try wherever it ran."
|
|
6197
6892
|
},
|
|
6198
6893
|
"flakyRuns": {
|
|
6199
6894
|
"type": "integer",
|
|
6200
|
-
"description": "Of runs, those
|
|
6895
|
+
"description": "Of runs, those in which it passed only after a retry in some project and failed in none."
|
|
6201
6896
|
},
|
|
6202
6897
|
"failedRuns": {
|
|
6203
6898
|
"type": "integer",
|
|
6204
|
-
"description": "Of runs, those
|
|
6899
|
+
"description": "Of runs, those in which its final attempt failed in any project - failed wins over flaky."
|
|
6205
6900
|
},
|
|
6206
6901
|
"stability": {
|
|
6207
6902
|
"description": "stableRuns / runs, a fraction from 0 to 1. null only when runs is 0.",
|
|
@@ -6226,7 +6921,18 @@
|
|
|
6226
6921
|
},
|
|
6227
6922
|
"timeLostMs": {
|
|
6228
6923
|
"type": "integer",
|
|
6229
|
-
"description": "Time spent on attempts that did not count, in ms, summed over runs: a failed
|
|
6924
|
+
"description": "Time spent on attempts that did not count, in ms, summed over runs and projects: a failed final attempt loses every attempt it made, one rescued by a retry the attempts before the pass, each at the final attempt's duration. 0 when every run passed on the first try."
|
|
6925
|
+
},
|
|
6926
|
+
"medianDurationMs": {
|
|
6927
|
+
"anyOf": [
|
|
6928
|
+
{
|
|
6929
|
+
"type": "integer"
|
|
6930
|
+
},
|
|
6931
|
+
{
|
|
6932
|
+
"type": "null"
|
|
6933
|
+
}
|
|
6934
|
+
],
|
|
6935
|
+
"description": "Per run the slowest project's final attempt, the median over the runs (the lower middle); null when no run names a duration (since 1.0.0-beta.19)."
|
|
6230
6936
|
},
|
|
6231
6937
|
"recentResults": {
|
|
6232
6938
|
"type": "array",
|
|
@@ -6270,7 +6976,7 @@
|
|
|
6270
6976
|
"result"
|
|
6271
6977
|
]
|
|
6272
6978
|
},
|
|
6273
|
-
"description": "The test's last 20 runs in the period, oldest first,
|
|
6979
|
+
"description": "The test's last 20 runs in the period, oldest first, each over its projects: failed when it failed in any, flaky when it passed after a retry in one and failed in none, stable, or skipped when every project skipped. Empty when it has none."
|
|
6274
6980
|
},
|
|
6275
6981
|
"branches": {
|
|
6276
6982
|
"type": "array",
|
|
@@ -6315,6 +7021,64 @@
|
|
|
6315
7021
|
}
|
|
6316
7022
|
],
|
|
6317
7023
|
"description": "The branch with the most failed runs (kind failures, count = those runs); when no run failed, the branch with the most flaky runs (kind retries, count = those runs); null when neither. Ties go to the branch name in code-point order, a null branch last."
|
|
7024
|
+
},
|
|
7025
|
+
"variants": {
|
|
7026
|
+
"type": "array",
|
|
7027
|
+
"items": {
|
|
7028
|
+
"type": "object",
|
|
7029
|
+
"properties": {
|
|
7030
|
+
"testName": {
|
|
7031
|
+
"type": "string",
|
|
7032
|
+
"description": "The variant's stored name: Playwright's title path, its project first - the name GET /runs/{runId} and the attempts route carry."
|
|
7033
|
+
},
|
|
7034
|
+
"project": {
|
|
7035
|
+
"description": "Its Playwright project; null for a result that ran under no named project.",
|
|
7036
|
+
"type": [
|
|
7037
|
+
"string",
|
|
7038
|
+
"null"
|
|
7039
|
+
]
|
|
7040
|
+
},
|
|
7041
|
+
"runs": {
|
|
7042
|
+
"type": "integer",
|
|
7043
|
+
"description": "Runs of the population in which this project's final attempt did not skip."
|
|
7044
|
+
},
|
|
7045
|
+
"stableRuns": {
|
|
7046
|
+
"type": "integer"
|
|
7047
|
+
},
|
|
7048
|
+
"flakyRuns": {
|
|
7049
|
+
"type": "integer"
|
|
7050
|
+
},
|
|
7051
|
+
"failedRuns": {
|
|
7052
|
+
"type": "integer"
|
|
7053
|
+
},
|
|
7054
|
+
"timeLostMs": {
|
|
7055
|
+
"type": "integer",
|
|
7056
|
+
"description": "This project's time lost - see the test's timeLostMs."
|
|
7057
|
+
},
|
|
7058
|
+
"medianDurationMs": {
|
|
7059
|
+
"anyOf": [
|
|
7060
|
+
{
|
|
7061
|
+
"type": "integer"
|
|
7062
|
+
},
|
|
7063
|
+
{
|
|
7064
|
+
"type": "null"
|
|
7065
|
+
}
|
|
7066
|
+
],
|
|
7067
|
+
"description": "The median of this project's final-attempt durations over its runs, the lower middle; null when none names one."
|
|
7068
|
+
}
|
|
7069
|
+
},
|
|
7070
|
+
"required": [
|
|
7071
|
+
"testName",
|
|
7072
|
+
"project",
|
|
7073
|
+
"runs",
|
|
7074
|
+
"stableRuns",
|
|
7075
|
+
"flakyRuns",
|
|
7076
|
+
"failedRuns",
|
|
7077
|
+
"timeLostMs",
|
|
7078
|
+
"medianDurationMs"
|
|
7079
|
+
]
|
|
7080
|
+
},
|
|
7081
|
+
"description": "One item per project's variant, project in code-point order (none last), each with its own numbers - the Test explorer's breakdown (since 1.0.0-beta.19)."
|
|
6318
7082
|
}
|
|
6319
7083
|
},
|
|
6320
7084
|
"required": [
|
|
@@ -6324,9 +7088,10 @@
|
|
|
6324
7088
|
"since",
|
|
6325
7089
|
"until",
|
|
6326
7090
|
"branch",
|
|
7091
|
+
"projectFilter",
|
|
6327
7092
|
"testName",
|
|
6328
7093
|
"filePath",
|
|
6329
|
-
"
|
|
7094
|
+
"projects",
|
|
6330
7095
|
"runs",
|
|
6331
7096
|
"stableRuns",
|
|
6332
7097
|
"flakyRuns",
|
|
@@ -6335,9 +7100,11 @@
|
|
|
6335
7100
|
"flakyRate",
|
|
6336
7101
|
"failureRate",
|
|
6337
7102
|
"timeLostMs",
|
|
7103
|
+
"medianDurationMs",
|
|
6338
7104
|
"recentResults",
|
|
6339
7105
|
"branches",
|
|
6340
|
-
"topBranch"
|
|
7106
|
+
"topBranch",
|
|
7107
|
+
"variants"
|
|
6341
7108
|
]
|
|
6342
7109
|
},
|
|
6343
7110
|
"TestHistoryResponse": {
|
|
@@ -6362,18 +7129,27 @@
|
|
|
6362
7129
|
"type": "object",
|
|
6363
7130
|
"properties": {
|
|
6364
7131
|
"name": {
|
|
6365
|
-
"type": "string"
|
|
7132
|
+
"type": "string",
|
|
7133
|
+
"description": "The test's name without its Playwright project (since 1.0.0-beta.19; the name as given before)."
|
|
6366
7134
|
},
|
|
6367
7135
|
"filePath": {
|
|
6368
7136
|
"type": [
|
|
6369
7137
|
"string",
|
|
6370
7138
|
"null"
|
|
6371
7139
|
]
|
|
7140
|
+
},
|
|
7141
|
+
"project": {
|
|
7142
|
+
"description": "The one project the history is of - the project parameter, or the project of the variant name the path gave; null for every project (since 1.0.0-beta.19).",
|
|
7143
|
+
"type": [
|
|
7144
|
+
"string",
|
|
7145
|
+
"null"
|
|
7146
|
+
]
|
|
6372
7147
|
}
|
|
6373
7148
|
},
|
|
6374
7149
|
"required": [
|
|
6375
7150
|
"name",
|
|
6376
|
-
"filePath"
|
|
7151
|
+
"filePath",
|
|
7152
|
+
"project"
|
|
6377
7153
|
]
|
|
6378
7154
|
},
|
|
6379
7155
|
"window": {
|
|
@@ -6436,7 +7212,7 @@
|
|
|
6436
7212
|
},
|
|
6437
7213
|
"status": {
|
|
6438
7214
|
"type": "string",
|
|
6439
|
-
"description": "The final attempt's status across the run's shards, read through test.fail()."
|
|
7215
|
+
"description": "The final attempt's status across the run's shards, read through test.fail() - over the test's projects: failed when it failed in any, passed when it passed in any, skipped otherwise (since 1.0.0-beta.19)."
|
|
6440
7216
|
},
|
|
6441
7217
|
"durationMs": {
|
|
6442
7218
|
"anyOf": [
|
|
@@ -6446,10 +7222,11 @@
|
|
|
6446
7222
|
{
|
|
6447
7223
|
"type": "null"
|
|
6448
7224
|
}
|
|
6449
|
-
]
|
|
7225
|
+
],
|
|
7226
|
+
"description": "The final attempt's duration - across projects, the slowest project's (since 1.0.0-beta.19)."
|
|
6450
7227
|
},
|
|
6451
7228
|
"timedOut": {
|
|
6452
|
-
"description": "The final attempt's. Whether the attempt ended by running into its time limit (Playwright's timedOut). null when the reporter did not say - every result from a reporter before 0.10.0.",
|
|
7229
|
+
"description": "The final attempt's - across projects, true when any project's was. Whether the attempt ended by running into its time limit (Playwright's timedOut). null when the reporter did not say - every result from a reporter before 0.10.0.",
|
|
6453
7230
|
"type": [
|
|
6454
7231
|
"boolean",
|
|
6455
7232
|
"null"
|
|
@@ -6464,13 +7241,43 @@
|
|
|
6464
7241
|
"type": "null"
|
|
6465
7242
|
}
|
|
6466
7243
|
],
|
|
6467
|
-
"description": "The final attempt's. The time limit Playwright applied to the attempt, in milliseconds; 0 means no limit. null when the reporter did not say."
|
|
7244
|
+
"description": "The final attempt's - across projects, the largest. The time limit Playwright applied to the attempt, in milliseconds; 0 means no limit. null when the reporter did not say."
|
|
6468
7245
|
},
|
|
6469
7246
|
"attempts": {
|
|
6470
|
-
"type": "integer"
|
|
7247
|
+
"type": "integer",
|
|
7248
|
+
"description": "Every attempt of the test in the run, every project's."
|
|
6471
7249
|
},
|
|
6472
7250
|
"wastedMs": {
|
|
6473
|
-
"type": "integer"
|
|
7251
|
+
"type": "integer",
|
|
7252
|
+
"description": "Duration of the attempts that did not count - every attempt but each project's final one."
|
|
7253
|
+
},
|
|
7254
|
+
"projects": {
|
|
7255
|
+
"type": "array",
|
|
7256
|
+
"items": {
|
|
7257
|
+
"type": "object",
|
|
7258
|
+
"properties": {
|
|
7259
|
+
"project": {
|
|
7260
|
+
"type": [
|
|
7261
|
+
"string",
|
|
7262
|
+
"null"
|
|
7263
|
+
]
|
|
7264
|
+
},
|
|
7265
|
+
"status": {
|
|
7266
|
+
"type": "string",
|
|
7267
|
+
"description": "This project's final attempt, through test.fail(): passed, failed or skipped."
|
|
7268
|
+
},
|
|
7269
|
+
"attempts": {
|
|
7270
|
+
"type": "integer",
|
|
7271
|
+
"description": "This project's attempts in the run."
|
|
7272
|
+
}
|
|
7273
|
+
},
|
|
7274
|
+
"required": [
|
|
7275
|
+
"project",
|
|
7276
|
+
"status",
|
|
7277
|
+
"attempts"
|
|
7278
|
+
]
|
|
7279
|
+
},
|
|
7280
|
+
"description": "Each project's own result in the run, project in code-point order (since 1.0.0-beta.19). One item under the project parameter."
|
|
6474
7281
|
}
|
|
6475
7282
|
},
|
|
6476
7283
|
"required": [
|
|
@@ -6486,7 +7293,8 @@
|
|
|
6486
7293
|
"timedOut",
|
|
6487
7294
|
"timeoutMs",
|
|
6488
7295
|
"attempts",
|
|
6489
|
-
"wastedMs"
|
|
7296
|
+
"wastedMs",
|
|
7297
|
+
"projects"
|
|
6490
7298
|
]
|
|
6491
7299
|
}
|
|
6492
7300
|
}
|
|
@@ -6596,13 +7404,22 @@
|
|
|
6596
7404
|
"type": "integer"
|
|
6597
7405
|
},
|
|
6598
7406
|
"testCount": {
|
|
6599
|
-
"type": "integer"
|
|
7407
|
+
"type": "integer",
|
|
7408
|
+
"description": "The tests the error hit, each once across Playwright projects - a name without its project and a file (since 1.0.0-beta.19; each project's variant its own before)."
|
|
6600
7409
|
},
|
|
6601
7410
|
"testNames": {
|
|
6602
7411
|
"type": "array",
|
|
6603
7412
|
"items": {
|
|
6604
7413
|
"type": "string"
|
|
6605
|
-
}
|
|
7414
|
+
},
|
|
7415
|
+
"description": "Their names without the project, code-point order (since 1.0.0-beta.19; the stored names with it before)."
|
|
7416
|
+
},
|
|
7417
|
+
"projects": {
|
|
7418
|
+
"type": "array",
|
|
7419
|
+
"items": {
|
|
7420
|
+
"type": "string"
|
|
7421
|
+
},
|
|
7422
|
+
"description": "The Playwright projects those tests ran in, code-point order; empty when none ran under a named project (since 1.0.0-beta.19)."
|
|
6606
7423
|
},
|
|
6607
7424
|
"occurrenceCount": {
|
|
6608
7425
|
"type": "integer"
|
|
@@ -6613,11 +7430,11 @@
|
|
|
6613
7430
|
},
|
|
6614
7431
|
"failedCount": {
|
|
6615
7432
|
"type": "integer",
|
|
6616
|
-
"description": "Of the (run, test) pairs the error hit
|
|
7433
|
+
"description": "Of the (run, test) pairs the error hit - a test once across its projects - how many ended failed: the test's final attempt across all the run's shards failed in any project the error hit (since 1.0.0-beta.9; across projects since 1.0.0-beta.19, each project's variant its own pair before)."
|
|
6617
7434
|
},
|
|
6618
7435
|
"flakyCount": {
|
|
6619
7436
|
"type": "integer",
|
|
6620
|
-
"description": "Of the same pairs, how many ended flaky: the final attempt passed after a retry. A pair whose final
|
|
7437
|
+
"description": "Of the same pairs, how many ended flaky: the final attempt passed after a retry in a project the error hit and failed in none. A pair whose final attempts skipped is neither (since 1.0.0-beta.9; across projects since 1.0.0-beta.19)."
|
|
6621
7438
|
},
|
|
6622
7439
|
"failedRunCount": {
|
|
6623
7440
|
"type": "integer",
|
|
@@ -6650,6 +7467,7 @@
|
|
|
6650
7467
|
"fileCount",
|
|
6651
7468
|
"testCount",
|
|
6652
7469
|
"testNames",
|
|
7470
|
+
"projects",
|
|
6653
7471
|
"occurrenceCount",
|
|
6654
7472
|
"runCount",
|
|
6655
7473
|
"failedCount",
|
|
@@ -6754,13 +7572,22 @@
|
|
|
6754
7572
|
"type": "integer"
|
|
6755
7573
|
},
|
|
6756
7574
|
"testCount": {
|
|
6757
|
-
"type": "integer"
|
|
7575
|
+
"type": "integer",
|
|
7576
|
+
"description": "The tests the error hit, each once across Playwright projects - a name without its project and a file (since 1.0.0-beta.19; each project's variant its own before)."
|
|
6758
7577
|
},
|
|
6759
7578
|
"testNames": {
|
|
6760
7579
|
"type": "array",
|
|
6761
7580
|
"items": {
|
|
6762
7581
|
"type": "string"
|
|
6763
|
-
}
|
|
7582
|
+
},
|
|
7583
|
+
"description": "Their names without the project, code-point order (since 1.0.0-beta.19; the stored names with it before)."
|
|
7584
|
+
},
|
|
7585
|
+
"projects": {
|
|
7586
|
+
"type": "array",
|
|
7587
|
+
"items": {
|
|
7588
|
+
"type": "string"
|
|
7589
|
+
},
|
|
7590
|
+
"description": "The Playwright projects those tests ran in, code-point order; empty when none ran under a named project (since 1.0.0-beta.19)."
|
|
6764
7591
|
},
|
|
6765
7592
|
"occurrenceCount": {
|
|
6766
7593
|
"type": "integer"
|
|
@@ -6771,11 +7598,11 @@
|
|
|
6771
7598
|
},
|
|
6772
7599
|
"failedCount": {
|
|
6773
7600
|
"type": "integer",
|
|
6774
|
-
"description": "Of the (run, test) pairs the error hit
|
|
7601
|
+
"description": "Of the (run, test) pairs the error hit - a test once across its projects - how many ended failed: the test's final attempt across all the run's shards failed in any project the error hit (since 1.0.0-beta.9; across projects since 1.0.0-beta.19, each project's variant its own pair before)."
|
|
6775
7602
|
},
|
|
6776
7603
|
"flakyCount": {
|
|
6777
7604
|
"type": "integer",
|
|
6778
|
-
"description": "Of the same pairs, how many ended flaky: the final attempt passed after a retry. A pair whose final
|
|
7605
|
+
"description": "Of the same pairs, how many ended flaky: the final attempt passed after a retry in a project the error hit and failed in none. A pair whose final attempts skipped is neither (since 1.0.0-beta.9; across projects since 1.0.0-beta.19)."
|
|
6779
7606
|
},
|
|
6780
7607
|
"failedRunCount": {
|
|
6781
7608
|
"type": "integer",
|
|
@@ -6808,6 +7635,7 @@
|
|
|
6808
7635
|
"fileCount",
|
|
6809
7636
|
"testCount",
|
|
6810
7637
|
"testNames",
|
|
7638
|
+
"projects",
|
|
6811
7639
|
"occurrenceCount",
|
|
6812
7640
|
"runCount",
|
|
6813
7641
|
"failedCount",
|
|
@@ -6829,7 +7657,8 @@
|
|
|
6829
7657
|
"type": "object",
|
|
6830
7658
|
"properties": {
|
|
6831
7659
|
"testName": {
|
|
6832
|
-
"type": "string"
|
|
7660
|
+
"type": "string",
|
|
7661
|
+
"description": "The test's name without its Playwright project (since 1.0.0-beta.19)."
|
|
6833
7662
|
},
|
|
6834
7663
|
"filePath": {
|
|
6835
7664
|
"type": [
|
|
@@ -6837,31 +7666,41 @@
|
|
|
6837
7666
|
"null"
|
|
6838
7667
|
]
|
|
6839
7668
|
},
|
|
7669
|
+
"projects": {
|
|
7670
|
+
"type": "array",
|
|
7671
|
+
"items": {
|
|
7672
|
+
"type": "string"
|
|
7673
|
+
},
|
|
7674
|
+
"description": "The projects it hit the error in, code-point order; empty when none is named (since 1.0.0-beta.19)."
|
|
7675
|
+
},
|
|
6840
7676
|
"occurrenceCount": {
|
|
6841
|
-
"type": "integer"
|
|
7677
|
+
"type": "integer",
|
|
7678
|
+
"description": "Attempts with this error, every project's."
|
|
6842
7679
|
},
|
|
6843
7680
|
"runCount": {
|
|
6844
7681
|
"type": "integer",
|
|
6845
|
-
"description": "Runs in which this test hit the error (since 1.0.0-beta.9)."
|
|
7682
|
+
"description": "Runs in which this test hit the error - a run once, however many of its projects (since 1.0.0-beta.9; across projects since 1.0.0-beta.19)."
|
|
6846
7683
|
},
|
|
6847
7684
|
"failedCount": {
|
|
6848
7685
|
"type": "integer",
|
|
6849
|
-
"description": "Of those runs, the ones the test ended failed (since 1.0.0-beta.9)."
|
|
7686
|
+
"description": "Of those runs, the ones the test ended failed in any project the error hit (since 1.0.0-beta.9)."
|
|
6850
7687
|
},
|
|
6851
7688
|
"flakyCount": {
|
|
6852
7689
|
"type": "integer",
|
|
6853
|
-
"description": "Of those runs, the ones
|
|
7690
|
+
"description": "Of those runs, the ones it ended flaky - passed after a retry in a project the error hit and failed in none (since 1.0.0-beta.9)."
|
|
6854
7691
|
}
|
|
6855
7692
|
},
|
|
6856
7693
|
"required": [
|
|
6857
7694
|
"testName",
|
|
6858
7695
|
"filePath",
|
|
7696
|
+
"projects",
|
|
6859
7697
|
"occurrenceCount",
|
|
6860
7698
|
"runCount",
|
|
6861
7699
|
"failedCount",
|
|
6862
7700
|
"flakyCount"
|
|
6863
7701
|
]
|
|
6864
|
-
}
|
|
7702
|
+
},
|
|
7703
|
+
"description": "One item per test the error hit, each once across Playwright projects - the error page's Affected tests (since 1.0.0-beta.19; one per project's variant before). Their failedCount and flakyCount sum to the signature's."
|
|
6865
7704
|
},
|
|
6866
7705
|
"runs": {
|
|
6867
7706
|
"type": "array",
|
|
@@ -6870,7 +7709,7 @@
|
|
|
6870
7709
|
"properties": {
|
|
6871
7710
|
"runId": {
|
|
6872
7711
|
"type": "string",
|
|
6873
|
-
"description": "The run, by the id GET /projects/{projectId}/runs gives it - one row per run
|
|
7712
|
+
"description": "The run, by the id GET /projects/{projectId}/runs gives it - one row per run, however many shards and tests (since 1.0.0-beta.19; one row per run and test before)."
|
|
6874
7713
|
},
|
|
6875
7714
|
"runNumber": {
|
|
6876
7715
|
"type": "integer"
|
|
@@ -6885,30 +7724,54 @@
|
|
|
6885
7724
|
"type": "string",
|
|
6886
7725
|
"format": "date-time"
|
|
6887
7726
|
},
|
|
6888
|
-
"
|
|
6889
|
-
"type": "string"
|
|
6890
|
-
},
|
|
6891
|
-
"filePath": {
|
|
6892
|
-
"type": [
|
|
6893
|
-
"string",
|
|
6894
|
-
"null"
|
|
6895
|
-
]
|
|
6896
|
-
},
|
|
6897
|
-
"attempts": {
|
|
6898
|
-
"type": "integer",
|
|
6899
|
-
"description": "Every attempt of the test in the run, retries and every shard included."
|
|
6900
|
-
},
|
|
6901
|
-
"occurrences": {
|
|
6902
|
-
"type": "integer",
|
|
6903
|
-
"description": "How many of those carry this error."
|
|
6904
|
-
},
|
|
6905
|
-
"finalStatus": {
|
|
7727
|
+
"status": {
|
|
6906
7728
|
"type": "string",
|
|
6907
|
-
"
|
|
7729
|
+
"enum": [
|
|
7730
|
+
"failed",
|
|
7731
|
+
"flaky",
|
|
7732
|
+
"skipped"
|
|
7733
|
+
],
|
|
7734
|
+
"description": "How the run ended for the tests the error hit - the error page's mark: failed when one ended failed, flaky when a retry rescued every one, skipped otherwise (since 1.0.0-beta.19)."
|
|
6908
7735
|
},
|
|
6909
7736
|
"massFailure": {
|
|
6910
7737
|
"type": "boolean",
|
|
6911
7738
|
"description": "The run was a mass failure as a whole (since 1.0.0-beta.9)."
|
|
7739
|
+
},
|
|
7740
|
+
"tests": {
|
|
7741
|
+
"type": "array",
|
|
7742
|
+
"items": {
|
|
7743
|
+
"type": "object",
|
|
7744
|
+
"properties": {
|
|
7745
|
+
"testName": {
|
|
7746
|
+
"type": "string",
|
|
7747
|
+
"description": "The test's name without its Playwright project."
|
|
7748
|
+
},
|
|
7749
|
+
"filePath": {
|
|
7750
|
+
"type": [
|
|
7751
|
+
"string",
|
|
7752
|
+
"null"
|
|
7753
|
+
]
|
|
7754
|
+
},
|
|
7755
|
+
"projects": {
|
|
7756
|
+
"type": "array",
|
|
7757
|
+
"items": {
|
|
7758
|
+
"type": "string"
|
|
7759
|
+
},
|
|
7760
|
+
"description": "The projects it hit the error in, in this run, code-point order."
|
|
7761
|
+
},
|
|
7762
|
+
"finalStatus": {
|
|
7763
|
+
"type": "string",
|
|
7764
|
+
"description": "The test's final attempt in the run, through test.fail(), over those projects: failed when any failed, passed when any passed, skipped otherwise."
|
|
7765
|
+
}
|
|
7766
|
+
},
|
|
7767
|
+
"required": [
|
|
7768
|
+
"testName",
|
|
7769
|
+
"filePath",
|
|
7770
|
+
"projects",
|
|
7771
|
+
"finalStatus"
|
|
7772
|
+
]
|
|
7773
|
+
},
|
|
7774
|
+
"description": "The tests the error hit in this run, each once across projects, by name then file (since 1.0.0-beta.19)."
|
|
6912
7775
|
}
|
|
6913
7776
|
},
|
|
6914
7777
|
"required": [
|
|
@@ -6916,14 +7779,12 @@
|
|
|
6916
7779
|
"runNumber",
|
|
6917
7780
|
"branch",
|
|
6918
7781
|
"startedAt",
|
|
6919
|
-
"
|
|
6920
|
-
"
|
|
6921
|
-
"
|
|
6922
|
-
"occurrences",
|
|
6923
|
-
"finalStatus",
|
|
6924
|
-
"massFailure"
|
|
7782
|
+
"status",
|
|
7783
|
+
"massFailure",
|
|
7784
|
+
"tests"
|
|
6925
7785
|
]
|
|
6926
|
-
}
|
|
7786
|
+
},
|
|
7787
|
+
"description": "The runs with this error, newest first, one row per run - the error page's runs list (since 1.0.0-beta.19)."
|
|
6927
7788
|
}
|
|
6928
7789
|
},
|
|
6929
7790
|
"required": [
|
|
@@ -7077,7 +7938,7 @@
|
|
|
7077
7938
|
},
|
|
7078
7939
|
"suiteSize": {
|
|
7079
7940
|
"type": "integer",
|
|
7080
|
-
"description": "The largest run in the window, every test once over all its shards."
|
|
7941
|
+
"description": "The largest run in the window, every test once over all its shards and across its Playwright projects - Analytics' Suite size (across projects since 1.0.0-beta.19; each project's variant its own before)."
|
|
7081
7942
|
},
|
|
7082
7943
|
"windowStability": {
|
|
7083
7944
|
"description": "Suite stability, 0-100: of the final test results in the window's finished runs of the source on the branch (skipped results left out), the share that passed on the first try, rounded. Mass-failure runs count (since 1.0.0-beta.7). A result that passed only after a retry counts against it. null when no such result exists.",
|
|
@@ -7113,22 +7974,6 @@
|
|
|
7113
7974
|
],
|
|
7114
7975
|
"description": "Time spent on attempts a later retry superseded, in the window's finished runs of the source on the branch; wallClockMs is the time those runs used - each shard's own duration, summed - and lostPercent is lostMs of it."
|
|
7115
7976
|
},
|
|
7116
|
-
"failingTests": {
|
|
7117
|
-
"type": "object",
|
|
7118
|
-
"properties": {
|
|
7119
|
-
"failing": {
|
|
7120
|
-
"type": "integer"
|
|
7121
|
-
},
|
|
7122
|
-
"testsRun": {
|
|
7123
|
-
"type": "integer"
|
|
7124
|
-
}
|
|
7125
|
-
},
|
|
7126
|
-
"required": [
|
|
7127
|
-
"failing",
|
|
7128
|
-
"testsRun"
|
|
7129
|
-
],
|
|
7130
|
-
"description": "Tests whose final attempt failed in at least one of the window's finished runs of the source on the branch, out of the tests those runs reported (a skipped test counts as reported)."
|
|
7131
|
-
},
|
|
7132
7977
|
"testMetrics": {
|
|
7133
7978
|
"type": "object",
|
|
7134
7979
|
"properties": {
|
|
@@ -7147,15 +7992,15 @@
|
|
|
7147
7992
|
},
|
|
7148
7993
|
"flakyTests": {
|
|
7149
7994
|
"type": "integer",
|
|
7150
|
-
"description": "Of tests, the tests with at least one flaky run in the period
|
|
7995
|
+
"description": "Of tests, the tests with at least one flaky run in the period - passed only after a retry in some project and failed in none - the dashboard's Flaky tests tile. Called testsWithRetries until 1.0.0-beta.15; across projects since 1.0.0-beta.19."
|
|
7151
7996
|
},
|
|
7152
7997
|
"failingTests": {
|
|
7153
7998
|
"type": "integer",
|
|
7154
|
-
"description": "Of tests, the tests with at least one failed run in the period."
|
|
7999
|
+
"description": "Of tests, the tests with at least one failed run in the period - failed in any project. The dashboard's Failing tests tile; the top-level failingTests went in 1.0.0-beta.19."
|
|
7155
8000
|
},
|
|
7156
8001
|
"tests": {
|
|
7157
8002
|
"type": "integer",
|
|
7158
|
-
"description": "Tests with at least one run of the population in the period -
|
|
8003
|
+
"description": "Tests with at least one run of the population in the period, each once across Playwright projects - runs of the source parameter's kind, all by default since 1.0.0-beta.18 (CI before). Called ciTests until 1.0.0-beta.15; across projects since 1.0.0-beta.19."
|
|
7159
8004
|
}
|
|
7160
8005
|
},
|
|
7161
8006
|
"required": [
|
|
@@ -7165,7 +8010,7 @@
|
|
|
7165
8010
|
"failingTests",
|
|
7166
8011
|
"tests"
|
|
7167
8012
|
],
|
|
7168
|
-
"description": "The dashboard's Flaky tests and Failing tests tiles and their denominator,
|
|
8013
|
+
"description": "The dashboard's Flaky tests and Failing tests tiles and their denominator - each test once across Playwright projects, a run once (since 1.0.0-beta.19; each project's variant its own before) - on the branch (since 1.0.0-beta.7; every branch before). Mass-failure runs are left out here, as in every per-test metric."
|
|
7169
8014
|
},
|
|
7170
8015
|
"slowestTests": {
|
|
7171
8016
|
"type": "array",
|
|
@@ -7173,7 +8018,8 @@
|
|
|
7173
8018
|
"type": "object",
|
|
7174
8019
|
"properties": {
|
|
7175
8020
|
"testName": {
|
|
7176
|
-
"type": "string"
|
|
8021
|
+
"type": "string",
|
|
8022
|
+
"description": "The test's name without its Playwright project (since 1.0.0-beta.19)."
|
|
7177
8023
|
},
|
|
7178
8024
|
"filePath": {
|
|
7179
8025
|
"type": [
|
|
@@ -7181,6 +8027,13 @@
|
|
|
7181
8027
|
"null"
|
|
7182
8028
|
]
|
|
7183
8029
|
},
|
|
8030
|
+
"projects": {
|
|
8031
|
+
"type": "array",
|
|
8032
|
+
"items": {
|
|
8033
|
+
"type": "string"
|
|
8034
|
+
},
|
|
8035
|
+
"description": "Every project it ran in, code-point order (since 1.0.0-beta.19)."
|
|
8036
|
+
},
|
|
7184
8037
|
"medianMs": {
|
|
7185
8038
|
"type": "number"
|
|
7186
8039
|
},
|
|
@@ -7191,11 +8044,12 @@
|
|
|
7191
8044
|
"required": [
|
|
7192
8045
|
"testName",
|
|
7193
8046
|
"filePath",
|
|
8047
|
+
"projects",
|
|
7194
8048
|
"medianMs",
|
|
7195
8049
|
"runs"
|
|
7196
8050
|
]
|
|
7197
8051
|
},
|
|
7198
|
-
"description": "Up to 5 tests by
|
|
8052
|
+
"description": "Up to 5 tests by their median duration - per run the slowest project's final attempt, the median over the runs - at least 3 runs; skipped results left out. Analytics' Slowest tests (across projects since 1.0.0-beta.19; per project's variant before)."
|
|
7199
8053
|
},
|
|
7200
8054
|
"timeSinks": {
|
|
7201
8055
|
"type": "array",
|
|
@@ -7203,7 +8057,8 @@
|
|
|
7203
8057
|
"type": "object",
|
|
7204
8058
|
"properties": {
|
|
7205
8059
|
"testName": {
|
|
7206
|
-
"type": "string"
|
|
8060
|
+
"type": "string",
|
|
8061
|
+
"description": "The test's name without its Playwright project (since 1.0.0-beta.19)."
|
|
7207
8062
|
},
|
|
7208
8063
|
"filePath": {
|
|
7209
8064
|
"type": [
|
|
@@ -7211,6 +8066,13 @@
|
|
|
7211
8066
|
"null"
|
|
7212
8067
|
]
|
|
7213
8068
|
},
|
|
8069
|
+
"projects": {
|
|
8070
|
+
"type": "array",
|
|
8071
|
+
"items": {
|
|
8072
|
+
"type": "string"
|
|
8073
|
+
},
|
|
8074
|
+
"description": "The projects whose attempts lost the time, code-point order (since 1.0.0-beta.19)."
|
|
8075
|
+
},
|
|
7214
8076
|
"lostMs": {
|
|
7215
8077
|
"type": "number"
|
|
7216
8078
|
}
|
|
@@ -7218,10 +8080,11 @@
|
|
|
7218
8080
|
"required": [
|
|
7219
8081
|
"testName",
|
|
7220
8082
|
"filePath",
|
|
8083
|
+
"projects",
|
|
7221
8084
|
"lostMs"
|
|
7222
8085
|
]
|
|
7223
8086
|
},
|
|
7224
|
-
"description": "Up to 5 tests by the time their superseded attempts cost in the window (at least a second each), worst first."
|
|
8087
|
+
"description": "Up to 5 tests by the time their superseded attempts cost in the window (at least a second each), worst first - each test once across projects, the Overview's Where the time goes (since 1.0.0-beta.19)."
|
|
7225
8088
|
},
|
|
7226
8089
|
"runsSeries": {
|
|
7227
8090
|
"type": "array",
|
|
@@ -7268,7 +8131,6 @@
|
|
|
7268
8131
|
"suiteSize",
|
|
7269
8132
|
"windowStability",
|
|
7270
8133
|
"timeLost",
|
|
7271
|
-
"failingTests",
|
|
7272
8134
|
"testMetrics",
|
|
7273
8135
|
"slowestTests",
|
|
7274
8136
|
"timeSinks",
|