@cratis/pi 7.101.0 → 7.102.0

package/package/corpus/harnesses/copilot/instructions/github-actions.instructions.md

@@ -45,6 +45,23 @@ paths:
   `cancel-in-progress: true`, keyed `${{ github.workflow }}-${{ github.ref }}`.
   **Never** cancel-in-progress on publish, release, or deploy workflows — a cancelled
   half-publish is worse than a queued one.
+- **Publish and release workflows never cancel** (`cancel-in-progress: false`) and set
+  `queue: max`. A group holds one running and one pending run; a newer arrival replaces
+  the pending one. Merges into the default branch share a group (a push, a repository-wide
+  key, or `github.ref` on a merged `pull_request` `closed` event, which is the base
+  branch), so without `queue: max` a newer merge replaces the older pending release.
+  `queue: max` keeps up to 100 pending runs, processed in order, and applies only while
+  `cancel-in-progress` is false or unset. Keep a single pending slot only for a documented
+  design where each run releases everything since the last release, so a replaced run
+  loses nothing; say so in a comment beside the `concurrency` block.
+- **A required check must not be skippable by `paths:` alone.** A workflow that never
+  starts never reports, and the pull request hangs. Gate jobs on a change-detection job
+  and require one always-run aggregator job (`if: always()`, checks every `needs.*.result`).
+- **Pin third-party actions by full commit SHA** (version in a comment) and set minimal
+  `permissions:` on each job. Run `zizmor` locally on edited workflows: its `unpinned-uses`
+  and `excessive-permissions` audits check exactly this. `actionlint` checks syntax,
+  expressions and embedded shell, not pinning. Do not assume CI runs either tool.
+- To make a workflow faster, use the `cratis-ci-pipeline-performance` skill.
 
 ## Matrices and operating systems
 

package/package/corpus/rules/github-actions.md

@@ -44,6 +44,23 @@ paths:
   `cancel-in-progress: true`, keyed `${{ github.workflow }}-${{ github.ref }}`.
   **Never** cancel-in-progress on publish, release, or deploy workflows — a cancelled
   half-publish is worse than a queued one.
+- **Publish and release workflows never cancel** (`cancel-in-progress: false`) and set
+  `queue: max`. A group holds one running and one pending run; a newer arrival replaces
+  the pending one. Merges into the default branch share a group (a push, a repository-wide
+  key, or `github.ref` on a merged `pull_request` `closed` event, which is the base
+  branch), so without `queue: max` a newer merge replaces the older pending release.
+  `queue: max` keeps up to 100 pending runs, processed in order, and applies only while
+  `cancel-in-progress` is false or unset. Keep a single pending slot only for a documented
+  design where each run releases everything since the last release, so a replaced run
+  loses nothing; say so in a comment beside the `concurrency` block.
+- **A required check must not be skippable by `paths:` alone.** A workflow that never
+  starts never reports, and the pull request hangs. Gate jobs on a change-detection job
+  and require one always-run aggregator job (`if: always()`, checks every `needs.*.result`).
+- **Pin third-party actions by full commit SHA** (version in a comment) and set minimal
+  `permissions:` on each job. Run `zizmor` locally on edited workflows: its `unpinned-uses`
+  and `excessive-permissions` audits check exactly this. `actionlint` checks syntax,
+  expressions and embedded shell, not pinning. Do not assume CI runs either tool.
+- To make a workflow faster, use the `cratis-ci-pipeline-performance` skill.
 
 ## Matrices and operating systems
 

package/package/corpus/skills/cratis-ci-pipeline-performance/LICENSE

@@ -0,0 +1,2 @@
+Copyright (c) Cratis. All rights reserved.
+Licensed under the MIT license. See LICENSE file in the canonical Cratis/AI repository for the full license text.

package/package/corpus/skills/cratis-ci-pipeline-performance/SKILL.md

@@ -0,0 +1,201 @@
+---
+name: cratis-ci-pipeline-performance
+description: Make a Cratis repository's GitHub Actions CI/CD faster and cheaper by measuring workflow timings first, picking the lever that fits the measured bottleneck (build once and shard, caching, path filters with an aggregator, concurrency, runner routing, publish from artifacts) and verifying before and after. Use when a workflow is slow, queues, burns runner minutes, reruns or cancels a lot, or when editing .github/workflows to speed up CI. Not for authoring a release (use cratis-governed-release-methodology), recurring regressions on one surface (use cratis-engineering-regression-cluster), runtime performance of application code (use cratis-performance-review), or fixing a red build.
+license: LICENSE
+---
+
+# Measure the workflow, then pick the lever
+
+A workflow is slow for one of four different reasons, and each has a different
+fix. Adding runners cures a queue and does nothing for a serial build; a faster
+build does nothing for a workflow that fails a quarter of the time. Measure,
+classify, change one lever, measure again. Never optimize from the YAML alone.
+
+Capacity, timeout, concurrency and runner invariants live in
+[github-actions.md](../../rules/github-actions.md); obey them and do not restate them in a change.
+Workflow-only pull requests carry the `no-release` label per
+[pull-requests.md](../../rules/pull-requests.md).
+
+## 1. Measure first
+
+Execute the bundled script (needs `gh` authenticated for the repository, and `jq`):
+
+```bash
+bash scripts/ci-timings.sh --repo OWNER/REPO --workflow ci.yml --limit 50 --jobs 10
+```
+
+Add `--branch main` or `--event pull_request` to compare populations; PR and
+default-branch runs behave differently. `bash scripts/ci-timings.sh --self-test`
+proves the script itself works without network access. Exit codes: `0` report
+printed, `2` usage error, `3` no runs or no data, `4` missing `gh`/`jq` or `gh`
+not authenticated. Exit `3` and `4` are an unknown, not a clean workflow: say the
+measurement was not made.
+
+The report separates **wall clock** (what a developer waits for) from
+**runner-minutes** (what the organization pays for or occupies). A change can
+cut one and raise the other; report both. It prints a one-line classification:
+
+| Class | Hint threshold | Meaning | First lever |
+|---|---|---|---|
+| failure-bound | failed or rerun >= 15% of completed runs | Time and minutes go into runs that are redone | Flaky and hung jobs; fail fast |
+| cancel-bound | cancelled >= 20% | Superseded work still ran heavy legs | Cheap gate first, path filters, smaller PR matrix |
+| queue-bound | median longest job wait >= 25% of wall | Jobs wait for runners, not for work | Fewer, bigger jobs; runner routing |
+| execution-bound | none of the above | The critical path is serial work | Build once and shard; caching |
+
+The thresholds are a hint, not a verdict. Several classes can hold at once; the
+script lists extras after the primary class. Then find the **critical path over
+the required checks**: the longest chain of `needs` from the start to the last
+required job. Only work on that chain shortens wall clock; shaving a parallel
+sibling does not.
+
+Without `gh` access, read the workflow YAML, state that the result is
+**static-only analysis**, and rank findings as hypotheses to measure later.
+
+## 2. Guardrails before proposing a fix
+
+- **Never hide required validation.** Release, schema, migration, analyzer and
+  shared-library checks stay on the path that gates a merge or a release. Moving
+  them to a schedule needs an explicit owner decision, not an optimization.
+- **Say what less parallelism costs.** Merging jobs lowers queue and setup cost
+  but lengthens the critical path when the merged work was parallel. State the
+  expected wall change and runner-minute change together.
+- **Separate repository changes from organization settings.** Workflow YAML is a
+  repository change. Runner capacity, scale sets and plan limits are not; report
+  them as a request to the owner and do not work around them in YAML.
+- **One lever per change.** Two levers at once make the measured difference
+  unattributable.
+- **Rank by what it saves.** Estimate minutes per day as (saving per run) x
+  (runs per day) from the report, and take the top one to three items.
+
+## 3. Lever catalogue
+
+Each lever lists when it fits, the Cratis default and the main pitfall. Read the
+named reference before applying one.
+
+**Build once, shard the tests.** Fits execution-bound workflows where build and
+tests run serially or several jobs each rebuild. Default: one job builds in
+`Release`, uploads the output with `actions/upload-artifact`, and a matrix of shard jobs downloads it and runs
+`dotnet test --no-build`. Pitfall: a shard whose filter matches nothing still
+passes, so assert that every shard ran tests. Read
+`references/build-once-and-shard.md` when applying or reviewing this.
+
+**Fewer, bigger jobs.** Fits queue-bound workflows: every job pays a scheduler
+start, checkout and toolchain setup. Default: merge small sibling jobs that share
+a setup; keep shards only where the work is long enough to repay a start. Pitfall:
+merging the long pole into a serial chain. Read `references/build-once-and-shard.md`.
+
+**Dependency caching.** Fits any job with a measurable restore or install step.
+Default: the shared `Cratis/Workflows` composite actions `dotnet-cache` and
+`yarn-cache` (restore on pull requests, save on the default branch), NuGet lock files and
+`yarn install --immutable`. Pitfall: a cache key that changes every run, or a
+different path list per workflow, which creates separate caches. Read
+`references/caching.md`.
+
+**Docker layer cache.** Fits image builds that repeat unchanged layers. Default:
+`docker/build-push-action` with `cache-from`/`cache-to` of type `gha` and a custom
+`scope` per image and platform. Pitfall: the default scope is shared, so builds
+overwrite each other's cache. Read `references/caching.md`.
+
+**Path filters with an always-run aggregator.** Fits cancel-bound workflows and
+documentation-only changes. Default: filter heavy jobs by changed paths, and make
+a single aggregator job with `if: always()` check every `needs.*.result`; require
+only the aggregator. Pitfall: a required check skipped by `paths:` never reports
+and the pull request hangs. Read `references/required-checks-and-path-filters.md`.
+
+**Concurrency.** Pull-request verification uses `cancel-in-progress: true` keyed
+by workflow and ref. Publish never cancels; where releases share one group, `queue:
+max` keeps up to 100 pending runs in order instead of replacing the single pending
+one. Pitfall: leaving the single pending slot where a replaced merge's run alone
+would have released something (merges into the default branch share a group, also
+when keyed on `github.ref` for merged `pull_request` `closed` events). A workflow
+whose release tooling aggregates since the last release may keep one pending slot,
+documented. Read `references/publish-pipelines.md`.
+
+**Runner routing.** Fits queue-bound workflows. Default: private-repository jobs
+on the organization scale set `cratis-arc` or a repository variable that falls
+back to a hosted label, hosted runners for public repositories. Pitfall: never
+route `pull_request` jobs of a public repository to self-hosted runners; that is
+fork code on our infrastructure. The rule owns the details.
+
+**Publish from tested artifacts.** Fits release workflows that rebuild what the
+pull-request run already built and tested. Default: build and pack once, then
+publish exactly those packages. Pitfall: a second build can differ from the
+tested one. Read `references/publish-pipelines.md`.
+
+**Fail fast on cheap gates.** Fits failure-bound and cancel-bound workflows.
+Default: formatting, lint and unit-level checks run first, and expensive legs
+depend on them with `needs`. Pitfall: serializing checks that were parallel; keep
+the cheap gate short enough to repay its place on the critical path.
+
+**Expensive matrices off the PR path.** Fits cancel-bound workflows with a wide
+OS or framework matrix. Default: full matrix on `schedule` and
+`workflow_dispatch`, a Linux-only representative smoke on pull requests (the
+rule states this). Pitfall: the full matrix needs an owner who reads its failures.
+
+**Flaky and hung jobs.** Fits failure-bound workflows. Default: a
+`timeout-minutes` on every job, `--blame-hang`, retry only a diagnosed flake, and
+quarantine a test with an issue. Pitfall: a blanket rerun hides the defect and
+doubles minutes. Read `references/flaky-and-hung-jobs.md`.
+
+**Merge queue.** Fits repositories where passing pull requests break each other
+after merge. Default: add the `merge_group` trigger, and make the aggregator
+handle it. Pitfall: without `merge_group` the required check never reports in the
+queue. Read `references/required-checks-and-path-filters.md`.
+
+**Compile once, analyze once.** Fits any workflow where one push compiles the same
+code several times. Default: build once and reuse the output; run analyzers in one
+parallel lint job instead of in every build, and let that job fail early on compile
+and analyzer errors before long test legs. Pitfall: dropping analyzers from the only
+build that consumers enforce. Read `references/test-setup-cost.md`.
+
+**Cores per job before job count.** Fits uniformly slow jobs. Default: check `nproc`
+and the runner's resources before sharding; undersized runners make every job slow.
+Pitfall: adding jobs to an undersized pool. Read `references/runner-sizing-and-images.md`.
+
+**Hidden test setup.** Fits test lanes whose step time far exceeds the runner's
+reported test time. Default: profile the test host and fix per-class setup upstream
+rather than sharding it. Pitfall: sharding that divides the waste. Read
+`references/test-setup-cost.md`.
+
+**Test a guarantee where it can break.** Fits suites that re-test a dependency.
+Default: run them upstream, or on a dependency pin change or nightly; use snapshots
+for shape and post-deploy synthetic checks where possible. Pitfall: relocating the
+only check of a required guarantee. Read `references/test-setup-cost.md`.
+
+**Images and runner contents.** Fits jobs that pull unused images or install tools per
+run. Default: pull only what the job uses, in parallel, and bake tools into the runner
+image; content-addressed caches before warm volumes, publish stays cold. Read
+`references/runner-sizing-and-images.md`.
+
+Speed work must not weaken supply-chain hygiene. Read
+`references/security-and-pinning.md` when a change adds or updates an action,
+touches `permissions:`, uses `pull_request_target`, or adds a cache.
+
+## 4. Verify
+
+1. Re-run the same `ci-timings.sh` command on the same workflow after enough new
+   runs exist (at least the sample size of the baseline; say so when fewer).
+2. Report expected against measured savings, wall and runner-minutes separately,
+   and whether the classification moved. A saving that is not measured is labeled
+   as expected.
+3. Run the linters locally on every edited workflow when installed: `actionlint`
+   (syntax, expressions, embedded shell) and `zizmor` (security audits such as
+   `unpinned-uses` and `excessive-permissions`). Pass means exit `0` with no
+   findings; a tool that is not installed is reported as not run, not as clean.
+   Do not assume CI runs either.
+4. Compare required check names before and after; a renamed job silently removes
+   a required check. See `references/required-checks-and-path-filters.md`.
+5. Confirm the workflow change ships as `no-release`.
+
+## References
+
+- Read `references/build-once-and-shard.md` when changing how a build is shared across jobs or how tests are split.
+- Read `references/caching.md` when adding or changing a dependency, Docker or build cache.
+- Read `references/required-checks-and-path-filters.md` when adding path filters, an aggregator, or `merge_group`.
+- Read `references/publish-pipelines.md` when a release or deploy workflow queues, drops runs or rebuilds.
+- Read `references/flaky-and-hung-jobs.md` when runs fail, hang or are rerun.
+- Read `references/test-setup-cost.md` when test steps outlast reported test time, tests duplicate a dependency's suite, or compiles repeat per push.
+- Read `references/runner-sizing-and-images.md` when jobs are uniformly slow, pull images, install tools or reuse workspaces.
+- Read `references/security-and-pinning.md` when a change touches actions, permissions, triggers or caches.
+- Read `references/sources.md` when attributing or extending the method.
+- Execute `scripts/ci-timings.sh` for every measurement; read it only to change it. Its self-test reads the synthetic inputs `scripts/fixtures/runs.json` and `scripts/fixtures/jobs/1003.json`, `scripts/fixtures/jobs/1007.json`, `scripts/fixtures/jobs/1010.json`; edit them only together with the expected lines in the script.

package/package/corpus/skills/cratis-ci-pipeline-performance/references/build-once-and-shard.md

@@ -0,0 +1,158 @@
+# Build once, shard the tests
+
+## Contents
+
+- When it fits
+- Shape
+- Decisions
+- Pitfalls
+- Fewer, bigger jobs
+
+Use this when one workflow builds the same code more than once, or runs build and
+tests serially in one long job. The pattern: a single build job compiles in
+`Release` and uploads the output; a shard matrix downloads it and runs the tests
+with `--no-build`. The wall clock becomes build + the slowest shard instead of
+build + all tests.
+
+## When it fits
+
+- The report shows `execution-bound` and one job holds most of the span.
+- Several jobs each restore and compile the same solution.
+- Tests are independent by project or namespace.
+
+It does not fit when the build is short compared with job start cost, or when
+the report is `queue-bound`: more shards are more independently scheduled jobs.
+Merge shards first, then re-measure.
+
+## Shape
+
+```yaml
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    timeout-minutes: 45
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@v7 # pin by full commit SHA
+      - uses: actions/setup-dotnet@v6 # pin by full commit SHA
+      - uses: Cratis/Workflows/.github/actions/dotnet-cache@main
+      - run: dotnet build Repo.slnx --configuration Release
+      - name: Archive build output
+        run: |
+          set -euo pipefail
+          find . -type d -name bin -not -path '*/node_modules/*' -not -path './.git/*' -print0 \
+            | tar --null -T - --zstd -cf build-output.tar.zst
+      - uses: actions/upload-artifact@v7 # pin by full commit SHA
+        with:
+          name: build-output
+          path: build-output.tar.zst
+          compression-level: 0
+          retention-days: 1
+          if-no-files-found: error
+
+  specs:
+    name: ${{ matrix.shard }}
+    needs: [build]
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - shard: specs-1
+            projects: Source/Core.Specs Source/Storage.Specs
+          - shard: specs-2
+            projects: Source/Api.Specs
+          - shard: specs-3
+            projects: Source/Client.Specs Source/Tools.Specs
+    steps:
+      - uses: actions/checkout@v7 # pin by full commit SHA
+      - uses: actions/setup-dotnet@v6 # pin by full commit SHA
+      - uses: actions/download-artifact@v8 # pin by full commit SHA
+        with:
+          name: build-output
+      - run: tar --zstd -xf build-output.tar.zst
+      - name: Test the shard's projects
+        env:
+          PROJECTS: ${{ matrix.projects }}
+        run: |
+          failed=0
+          for project in $PROJECTS; do
+            dotnet test "$project" --no-build --configuration Release \
+              --logger "trx;LogFileName=$(basename "$project").trx" \
+              --results-directory "$RUNNER_TEMP/test-results" || failed=1
+          done
+          exit "$failed"
+      - name: Verify every project ran tests
+        if: ${{ !cancelled() }}
+        env:
+          PROJECTS: ${{ matrix.projects }}
+        run: |
+          failed=0
+          for project in $PROJECTS; do
+            file="$RUNNER_TEMP/test-results/$(basename "$project").trx"
+            executed=$(grep -o -m1 'executed="[0-9]*"' "$file" 2>/dev/null | grep -o '[0-9]*' || echo 0)
+            echo "$project executed $executed tests"
+            [ "$executed" -gt 0 ] || failed=1
+          done
+          exit "$failed"
+      - uses: actions/upload-artifact@v7 # pin by full commit SHA
+        if: ${{ !cancelled() }}
+        with:
+          name: test-results-${{ matrix.shard }}
+          path: ${{ runner.temp }}/test-results/*.trx
+          if-no-files-found: warn
+```
+
+Each matrix entry carries its own `projects` list, which the test step reads, so
+shards run different tests. For a test framework filter instead of a project list,
+carry the filter in the entry (`filter: FullyQualifiedName~Core`) and pass it as
+`--filter`. Keep the assignment in the workflow or one checked-in script so that
+adding a project cannot leave it unassigned; review the list when a project is added.
+
+To balance, read each shard job's median duration from the slowest-jobs table of
+`ci-timings.sh`, move a project from the longest shard to the shortest, and
+re-measure; stop when the spread is within the job start cost.
+
+## Decisions
+
+- **Artifact, not cache, between jobs of one run.** A cache keyed on the commit
+  SHA is immutable: one failed save poisons the key for every later attempt and a
+  rerun cannot recover. An artifact belongs to the run and is the intended
+  primitive for passing build output between its jobs.
+- **Archive with `tar`.** Artifact upload does not preserve file permissions or
+  empty directories; a tar keeps both. Output that was already compressed should
+  not be compressed again: `compression-level: 0`.
+- **Include every consumed output.** Frontend-dependent specs need the built
+  `dist` folders as well as `bin`; list them in the archive.
+- **Short retention.** `retention-days: 1` for build output nobody downloads by
+  hand. Test results can live longer.
+- **Same configuration end to end.** Test with the configuration that built
+  (`--configuration Release --no-build`); a different one silently rebuilds.
+- **`fail-fast: false`** on shard matrices, so one failing shard does not hide the
+  results of the others, and a rerun of failed jobs reruns only the failed shard.
+- **Upload results with `if: ${{ !cancelled() }}`** so failures keep their TRX.
+
+## Pitfalls
+
+- A filter or project list that matches nothing passes `dotnet test`. End every
+  shard with a check that each assigned project or namespace ran at least one test,
+  and print the counts. Zero tests is a failed shard.
+- Shards balance by test time, not by project count (see the balancing steps
+  after the example).
+- A shard that downloads a large artifact pays that transfer on every job; measure
+  the download step in the slowest-steps table before adding shards.
+- Static assets outside `bin`/`dist` (generated sources, local tool caches) must
+  be in the archive or regenerated by the shard.
+
+## Fewer, bigger jobs
+
+When the report is `queue-bound`, every extra job costs a scheduler start, a
+checkout and a toolchain setup. Merge sibling jobs that share setup (several
+lint or format checks, several small build legs) into one job with several steps.
+Keep a job separate only when it is on the critical path and long enough that
+parallelism repays its start. State the expected effect on both wall clock and
+runner-minutes, since merging parallel work lengthens the chain.

package/package/corpus/skills/cratis-ci-pipeline-performance/references/caching.md

@@ -0,0 +1,86 @@
+# Caching
+
+Use this when a job spends measurable time restoring packages, installing a
+frontend toolchain or rebuilding Docker layers. Check the slowest-steps table
+first: a step under a minute is not worth a cache.
+
+## Contents
+
+- Shared Cratis cache actions
+- NuGet and Yarn determinism
+- Docker layer cache
+- Scope, eviction and safety
+
+## Shared Cratis cache actions
+
+The `Cratis/Workflows` repository ships two composite actions. Use them instead
+of hand-written `actions/cache` steps so every workflow of a repository uses one
+key and one path list.
+
+```yaml
+- uses: Cratis/Workflows/.github/actions/dotnet-cache@main
+- uses: Cratis/Workflows/.github/actions/yarn-cache@main
+  with:
+    working-directory: Source/JavaScript
+```
+
+- `mode` is `auto` by default: pull requests only restore; a push to the default
+  branch, a schedule or a manual run on the default branch saves. Use `restore`
+  to never save and `save` to always save.
+- `dotnet-cache` inputs: `path`, `key-files`, `key-prefix`, `mode`. Keep `path`
+  identical everywhere; a different path list is a different cache. Bump the
+  version suffix of `key-prefix` to start a fresh cache.
+- Outputs: `cache-hit`, `matched-key`, `key`, `mode`.
+- A cache is only saved from the default branch, so a repository with no push
+  workflow on that branch never warms one. Add a scheduled warm-up workflow that
+  runs the same restore and build.
+
+## NuGet and Yarn determinism
+
+A cache is only as stable as its key. Make the key inputs deterministic.
+
+- NuGet: enable lock files (`<RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>`)
+  and restore with `dotnet restore --locked-mode` in CI, so the lock file hash
+  keys the cache and a drifting dependency fails loudly.
+- Yarn: commit `yarn.lock` and install with `yarn install --immutable`; the
+  install fails instead of rewriting the lock file.
+- Cache the package download directories, not `bin`/`obj` output. Build output
+  caches across commits are rarely correct; share build output within a run as an
+  artifact instead (see `build-once-and-shard.md`).
+
+## Docker layer cache
+
+```yaml
+- uses: docker/setup-buildx-action@v4 # pin by full commit SHA
+- uses: docker/build-push-action@v7 # pin by full commit SHA
+  with:
+    context: .
+    push: false
+    cache-from: type=gha,scope=api-linux-amd64
+    cache-to: type=gha,scope=api-linux-amd64,mode=max
+```
+
+- Set a custom `scope` per image and platform. The default scope is shared, so
+  two builds overwrite each other's cache and both see misses.
+- `mode=max` caches intermediate layers of multi-stage builds; the default caches
+  only the final image layers.
+- Order the Dockerfile so rarely changing layers (base image, dependency
+  restore) come before source copies.
+- When an image is built once and pulled by several jobs, pushing it to a
+  registry and pulling it costs a transfer per job; measure that pull in the
+  slowest-steps table against rebuilding from cache.
+
+## Scope, eviction and safety
+
+- A pull request run can read caches of its own ref and of the base branch; a
+  cache saved on the default branch is readable by every branch. A cache saved by
+  one pull request is not visible to others.
+- Repository cache storage is bounded (10 GB by default). When full, or after a
+  week without access, entries are evicted, least recently used first. A large
+  rarely used cache can evict a hot one.
+- Caches must not contain secrets, tokens or credentials: restored content is
+  executed and readable by later runs of the same scope.
+- Never restore a cache written by an untrusted pull request into a privileged
+  workflow. See `security-and-pinning.md`.
+- Verify a cache works by reading the `cache-hit` output (or the restore step
+  log) on a second run, then re-measure with `ci-timings.sh`.

package/package/corpus/skills/cratis-ci-pipeline-performance/references/flaky-and-hung-jobs.md

@@ -0,0 +1,70 @@
+# Flaky and hung jobs
+
+Use this when the report is `failure-bound` (failed or rerun at or above the
+hint threshold) or a job occasionally runs to its timeout. Reruns and failures
+cost runner-minutes and developer trust; diagnose before retrying.
+
+## Diagnose
+
+1. Run `ci-timings.sh` with `--branch main` and again with `--event pull_request`.
+   Failures on the default branch with no code change are flakes or environment
+   problems; failures only on pull requests are mostly real.
+2. Group the failing runs by job and step name. One step that fails across many
+   runs is a defect in that step, not noise.
+3. Separate the cause: a hung test, a timeout from an external service, an
+   out-of-memory kill, or an infrastructure cancellation.
+
+## Bound every job
+
+- `timeout-minutes` on every job, per the rule. A hung job otherwise holds a
+  runner slot for up to six hours.
+- Set the job timeout above the test timeout, so dump collection and cleanup have
+  time to finish.
+
+## Find hangs and crashes in .NET
+
+```bash
+dotnet test Repo.slnx --configuration Release --no-build \
+  --blame-hang --blame-hang-timeout 5m --blame-crash \
+  --results-directory TestResults
+```
+
+- This is the VSTest form. Microsoft.Testing.Platform projects use their own
+  crash and hang dump extensions and options; do not mix the two sets of switches.
+- Upload `TestResults` as an artifact with `if: ${{ !cancelled() }}`, so the dump
+  survives the failure.
+
+## Retry only a diagnosed flake
+
+- Retry the smallest unit: a single test or a single failed job, never the whole
+  suite. With Microsoft.Testing.Platform, the retry extension provides
+  `--retry-failed-tests` with a failure-percentage ceiling; plain VSTest has no
+  equivalent flag.
+- Keep the first-attempt failure visible. A retry that turns red green without a
+  record hides the flake and doubles minutes.
+- Retrying a deterministic failure only consumes capacity.
+
+## Quarantine
+
+A quarantined test is a decision with an owner, not a deletion:
+
+- Mark it with the framework's skip or category mechanism and a reason that names
+  an issue.
+- Run quarantined tests in a separate, visible, non-required lane so they keep
+  producing signal.
+- Give the issue an owner and an expiry; an entry nobody revisits is a check
+  quietly deleted.
+
+## Fail fast
+
+Put the cheap, high-signal gates (format, lint, unit-level tests) before the
+expensive legs and make the expensive legs `needs` them. A failure then costs
+minutes instead of the full pipeline. Keep the gate short: a slow gate on the
+critical path moves the cost instead of removing it.
+
+## Cancelled work
+
+A high cancelled share is expected on pull requests with `cancel-in-progress`.
+It is waste only when the cancelled runs had already spent their expensive legs.
+Reduce it by ordering cheap legs first and by keeping the full matrix off the
+pull-request path, not by removing the cancellation.

package/package/corpus/skills/cratis-ci-pipeline-performance/references/publish-pipelines.md

@@ -0,0 +1,67 @@
+# Publish and release pipelines
+
+Use this when a release, package-publish or deploy workflow queues, loses runs,
+or rebuilds what a pull-request run already built and tested. Release method and
+assurance tiers belong to `cratis-governed-release-methodology`.
+
+## Concurrency for publishing
+
+```yaml
+concurrency:
+  group: publish-${{ github.repository }}
+  cancel-in-progress: false
+  queue: max
+```
+
+- Pull-request verification cancels superseded runs. Publish never does: a
+  cancelled half-publish is worse than a queued one.
+- A concurrency group holds one running and, by default, one pending run; a newer
+  arrival replaces the pending one. `queue: max` keeps up to 100 pending runs in
+  the group and processes them in order. It is valid only while
+  `cancel-in-progress` is `false` or unset; never combine it with `true`.
+- Replacement only happens between runs of the same group, so read the group key.
+  Merges into the default branch share one group: a push, a repository-wide key, or
+  `github.ref` on a merged `pull_request` `closed` event, where `github.ref` is the
+  base branch (for example `refs/heads/main`). A newer merge's pending run then
+  replaces the older pending one unless `queue: max` is set. Only a `closed` event
+  of an unmerged pull request keeps `refs/pull/N/merge`, a group of its own.
+- A replaced pending run is lost work only when that run alone would have released
+  something. Release tooling that aggregates everything since the last release
+  loses nothing when a pending run is replaced; such a workflow may keep one
+  pending slot, with a comment beside the `concurrency` block saying so.
+- Groups are repository-scoped; this is not an organization-wide release lock.
+- Check the symptom in the report: many cancelled publish runs under one group
+  point at a replaced pending slot.
+
+## Publish from the tested artifacts
+
+A publish job that rebuilds can ship something different from what was tested,
+and pays the build twice.
+
+- Build once per publish run (per architecture where a native build differs),
+  pack with `--no-build`, and publish exactly those packages and images.
+- Share build output between jobs of the run as an artifact (see
+  `build-once-and-shard.md`). Do not pull a pull-request run's artifact into a
+  release: artifacts expire and belong to another commit.
+- Build the way consumers' strictest CI builds (`Release`, analyzers on); a
+  `--no-build` pack of a `Debug` build ships the wrong thing.
+- When several services publish images from one build, build the image once,
+  push it, and have the per-service jobs reuse it; compare the per-job pull time
+  against rebuilding from the Docker cache.
+- Keep a version resolved once in an early job and passed down as an output, so
+  every job in the run packs the same version.
+
+## Timeouts and permissions
+
+- Publish and release jobs use a 60-minute `timeout-minutes` tier, per the rule.
+- Grant `contents: write` or `packages: write` on the job that publishes, not on
+  the workflow, and keep build jobs on `contents: read`.
+- Prefer OIDC trusted publishing to a long-lived registry API key where the
+  registry supports it. See `security-and-pinning.md`.
+
+## Speed without weakening the release
+
+- Do not skip validation on the release path to save minutes; move other work off
+  it instead (docs jobs, matrices already covered by schedule).
+- Canary before promotion and recovery disposition are release-method questions;
+  hand them to `cratis-governed-release-methodology`.