@ai-ecoverse/slicc-bios 1.32.0 → 1.32.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -574,7 +574,7 @@ No test calls a real model. Every test serves an empty `packages/agent/` project
|
|
|
574
574
|
|
|
575
575
|
### CI
|
|
576
576
|
|
|
577
|
-
`ci.yml` runs on pull requests, the merge queue and pushes to `main`. A `ci / plan` job ([`scripts/ci/plan.mjs`](scripts/ci/plan.mjs)) splits the integration tests into 14 shards: each `guard-set-*.test.mjs` becomes one item per detach variant of the pinned guard vectors (run with `GUARD_VARIANTS`), every other test file is one item, and the items go in a random order, seeded with the run id, to the least-loaded shard by the times in [`scripts/ci/durations.json`](scripts/ci/durations.json), the longest items first. Each shard (`ci / integration (s01)` …) runs its items with [`scripts/ci/shard.mjs`](scripts/ci/shard.mjs), one `node --test` per guard-set file with its variants (the other files join the first), counting a test the guard-set helper reports for that run's guard-set file, keeps the harness's artifacts and coverage from every run, and writes its verdict line. `ci / test`, the required check, fails without a passing verdict from every shard the plan named.
|
|
577
|
+
`ci.yml` runs on pull requests, the merge queue and pushes to `main`. Its jobs run on the Cloud Run pools (`CERT_RUNNER` for the shards, `CERT_LIGHT_RUNNER` for the rest) for same-repo pull requests, the merge queue, dispatches and `main`, and on `ubuntu-latest` for forks; until 2026-10-20 the merge queue ran on `ubuntu-latest`, where its shards waited for GitHub's runners. A `ci / plan` job ([`scripts/ci/plan.mjs`](scripts/ci/plan.mjs)) splits the integration tests into 14 shards: each `guard-set-*.test.mjs` becomes one item per detach variant of the pinned guard vectors (run with `GUARD_VARIANTS`), every other test file is one item, and the items go in a random order, seeded with the run id, to the least-loaded shard by the times in [`scripts/ci/durations.json`](scripts/ci/durations.json), the longest items first. Each shard (`ci / integration (s01)` …) runs its items with [`scripts/ci/shard.mjs`](scripts/ci/shard.mjs), one `node --test` per guard-set file with its variants (the other files join the first), counting a test the guard-set helper reports for that run's guard-set file, keeps the harness's artifacts and coverage from every run, and writes its verdict line. `ci / test`, the required check, fails without a passing verdict from every shard the plan named.
|
|
578
578
|
|
|
579
579
|
On a pull request, the plan first reads the files the pull request changes (`git diff HEAD^1 HEAD` on its merge commit) and picks a mode ([`scripts/ci/scope.mjs`](scripts/ci/scope.mjs)):
|
|
580
580
|
|
|
@@ -586,7 +586,7 @@ On a pull request, the plan first reads the files the pull request changes (`git
|
|
|
586
586
|
|
|
587
587
|
[`scripts/ci/scopes.json`](scripts/ci/scopes.json) maps each package name or pattern to its rows (`full` for the harness, Playwright and the installer's tar, `vendors` for the slicc CLI). The plan prints its mode and why in the job summary, and `ci / test` passes only when the jobs that mode skips were skipped and every other job passed: `ci: PASS docs plan, 0 shards, skipped integration and vendors`. The merge queue always runs everything, so `main` is never less tested.
|
|
588
588
|
|
|
589
|
-
Every shard runs the files with a failing test once more and passes when they pass (`retry`, skipped when it wouldn't fit the shard's 62 minutes). From 2026-10-17 to 2026-10-18 a two-by-two experiment
|
|
589
|
+
Every shard runs the files with a failing test once more and passes when they pass (`retry`, skipped when it wouldn't fit the shard's 62 minutes), and a shard that still fails cancels the others (`fail-fast`). Both were tested first. From 2026-10-17 to 2026-10-18 a two-by-two experiment by pull request number found that 0 of 28 merge queue runs with retry failed, against 15 of 42 without it. From 2026-10-18 to 2026-10-20 the number's parity picked fail-fast or not, with retry on both sides: over 525 runs, failing runs finished after a median of 29 minutes with fail-fast against 47 without (the first shard went red after about 26 minutes in both), passing runs took the same time (46 and 45 minutes), and the 65 pull requests merged since leaned the same way (mean CI wait 1.8 h against 2.4 h, time to merge 4.0 h against 4.9 h, not yet significant). A run's arm is now always `fail-fast+retry`. The plan, with the arm, the seed and each shard's items, is in the job summary and the `experiment-plan` artifact; each shard's record (`artifacts/experiment/<shard>.json`: its groups, the first run and any retry with their exit codes, counts and failed files) is in its `integration-<shard>` artifact, with each item's time and every test that failed and then passed on the retry. `ci / test` joins them into `summary.json` (the `experiment-summary` artifact), and for each test that flipped it files an issue labeled `flaky-test`, or comments on the open one for that test, with the run, the arm, the shard and the first run's error ([`scripts/ci/summary.mjs`](scripts/ci/summary.mjs)); this step never fails the check, and a fork's read-only token files nothing. `node scripts/ci/durations.mjs` refreshes `durations.json` with the median times of the last 40 runs' summaries.
|
|
590
590
|
|
|
591
591
|
## Hosting
|
|
592
592
|
|