@htekdev/actions-debugger 1.0.5 → 1.0.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/errors/caching-artifacts/cache-10gb-limit-eviction.yml +135 -0
- package/errors/runner-environment/dockerhub-pull-rate-limit.yml +125 -0
- package/errors/runner-environment/service-container-localhost-vs-hostname.yml +130 -0
- package/errors/runner-environment/setup-python-cache-no-dependency-file.yml +126 -0
- package/errors/runner-environment/ubuntu-latest-to-24-04-migration.yml +138 -0
- package/errors/silent-failures/github-output-multiline-truncated.yml +112 -0
- package/errors/triggers/merge-group-trigger-missing.yml +122 -0
- package/errors/yaml-syntax/set-output-deprecated-commands.yml +130 -0
- package/package.json +1 -1
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
id: caching-artifacts-011
|
|
2
|
+
title: "Repository Cache Storage Limit (10 GB) — Silent Eviction Causes Cache Miss Spike"
|
|
3
|
+
category: caching-artifacts
|
|
4
|
+
severity: silent-failure
|
|
5
|
+
tags:
|
|
6
|
+
- cache
|
|
7
|
+
- storage-limit
|
|
8
|
+
- eviction
|
|
9
|
+
- cache-miss
|
|
10
|
+
- actions/cache
|
|
11
|
+
- 10gb
|
|
12
|
+
- lru
|
|
13
|
+
patterns:
|
|
14
|
+
- regex: "cache.*evict|evict.*cache"
|
|
15
|
+
flags: "i"
|
|
16
|
+
- regex: "cache size.*limit|limit.*cache size"
|
|
17
|
+
flags: "i"
|
|
18
|
+
- regex: "exceeds.*10.*GB|10.*GB.*limit"
|
|
19
|
+
flags: "i"
|
|
20
|
+
- regex: "cache.*storage.*limit.*exceeded"
|
|
21
|
+
flags: "i"
|
|
22
|
+
error_messages:
|
|
23
|
+
- "Cache entry is too big. Maximum cache size is 10GB"
|
|
24
|
+
- "Warning: Cache size of XY GB (ZZZZ MB) is over the 10 GB limit, so some older caches will be removed"
|
|
25
|
+
- "Failed to save cache entry. Exiting with error: Cache storage quota has been reached for the current repository"
|
|
26
|
+
root_cause: |
|
|
27
|
+
GitHub limits the total Actions cache storage per repository to **10 GB**. When the
|
|
28
|
+
10 GB limit is reached, GitHub automatically evicts the **least-recently-used (LRU)**
|
|
29
|
+
cache entries to make room for new ones. Additionally, cache entries that have not been
|
|
30
|
+
accessed in **7 days** are automatically deleted.
|
|
31
|
+
|
|
32
|
+
**This is a silent failure** because:
|
|
33
|
+
- Eviction happens asynchronously — the job that triggers eviction does not fail.
|
|
34
|
+
- The workflow that was relying on an evicted cache simply gets a cache miss on the
|
|
35
|
+
next run and reinstalls from scratch, adding minutes to build time.
|
|
36
|
+
- There is no notification, no warning in the workflow log, and no failed check.
|
|
37
|
+
- Engineers often diagnose this as a "flaky cache" without realizing the 10 GB limit
|
|
38
|
+
is being hit regularly.
|
|
39
|
+
|
|
40
|
+
**Common causes of limit exhaustion:**
|
|
41
|
+
1. **Matrix builds with many OS/version combinations** each storing large dependency
|
|
42
|
+
caches (node_modules, .cargo, ~/.gradle, etc.).
|
|
43
|
+
2. **Large monorepo caches** where the entire dependency tree exceeds 10 GB.
|
|
44
|
+
3. **Docker layer caches** stored via `actions/cache` for repeated builds.
|
|
45
|
+
4. **Accumulation without cleanup** — cache keys rotate on every dependency update
|
|
46
|
+
but old entries aren't explicitly purged.
|
|
47
|
+
|
|
48
|
+
**How to check current cache usage:**
|
|
49
|
+
Repository → Actions → Management → Caches (or via `gh cache list --repo owner/repo`).
|
|
50
|
+
fix: |
|
|
51
|
+
Reduce total cache storage by improving cache key strategy and explicitly pruning old
|
|
52
|
+
cache entries.
|
|
53
|
+
|
|
54
|
+
**Immediate steps:**
|
|
55
|
+
1. Audit current cache usage with `gh cache list --repo owner/repo --sort size --order desc`
|
|
56
|
+
2. Delete oversized or stale entries with `gh cache delete <id> --repo owner/repo`
|
|
57
|
+
3. Review matrix build cache keys — large matrices with OS+version combinations
|
|
58
|
+
multiply cache storage proportionally.
|
|
59
|
+
|
|
60
|
+
**Structural fixes:**
|
|
61
|
+
- Use more specific cache keys to avoid storing redundant versions.
|
|
62
|
+
- Split large caches into smaller focused caches (dependencies vs. build outputs).
|
|
63
|
+
- Add a periodic workflow to prune old caches before the limit is reached.
|
|
64
|
+
- For Docker layer caches, consider using GitHub Container Registry or a dedicated
|
|
65
|
+
cache registry instead of Actions cache.
|
|
66
|
+
fix_code:
|
|
67
|
+
- language: yaml
|
|
68
|
+
label: "Check cache usage and delete stale entries in a workflow"
|
|
69
|
+
code: |
|
|
70
|
+
jobs:
|
|
71
|
+
cleanup-caches:
|
|
72
|
+
runs-on: ubuntu-latest
|
|
73
|
+
steps:
|
|
74
|
+
- name: List and clean old caches
|
|
75
|
+
env:
|
|
76
|
+
GH_TOKEN: ${{ github.token }}
|
|
77
|
+
run: |
|
|
78
|
+
# List all caches sorted by last accessed time (oldest first)
|
|
79
|
+
gh cache list \
|
|
80
|
+
--repo ${{ github.repository }} \
|
|
81
|
+
--sort last-accessed \
|
|
82
|
+
--order asc \
|
|
83
|
+
--limit 100 \
|
|
84
|
+
--json id,key,sizeInBytes,lastAccessedAt \
|
|
85
|
+
| jq -r '.[] | "\(.id)\t\(.sizeInBytes)\t\(.lastAccessedAt)\t\(.key)"'
|
|
86
|
+
- language: yaml
|
|
87
|
+
label: "Prune caches older than N days using the Actions API"
|
|
88
|
+
code: |
|
|
89
|
+
jobs:
|
|
90
|
+
prune-caches:
|
|
91
|
+
runs-on: ubuntu-latest
|
|
92
|
+
permissions:
|
|
93
|
+
actions: write
|
|
94
|
+
steps:
|
|
95
|
+
- name: Delete caches not accessed in 5 days
|
|
96
|
+
env:
|
|
97
|
+
GH_TOKEN: ${{ github.token }}
|
|
98
|
+
REPO: ${{ github.repository }}
|
|
99
|
+
run: |
|
|
100
|
+
CUTOFF=$(date -d '5 days ago' --iso-8601=seconds)
|
|
101
|
+
gh api "repos/$REPO/actions/caches" \
|
|
102
|
+
--paginate \
|
|
103
|
+
--jq ".actions_caches[] | select(.last_accessed_at < \"$CUTOFF\") | .id" \
|
|
104
|
+
| xargs -I{} gh api --method DELETE "repos/$REPO/actions/caches/{}"
|
|
105
|
+
- language: yaml
|
|
106
|
+
label: "Optimize matrix builds to share a single cache entry"
|
|
107
|
+
code: |
|
|
108
|
+
jobs:
|
|
109
|
+
build:
|
|
110
|
+
strategy:
|
|
111
|
+
matrix:
|
|
112
|
+
os: [ubuntu-latest, windows-latest, macos-latest]
|
|
113
|
+
node: ['18', '20', '22']
|
|
114
|
+
runs-on: ${{ matrix.os }}
|
|
115
|
+
steps:
|
|
116
|
+
- uses: actions/cache@v4
|
|
117
|
+
with:
|
|
118
|
+
path: ~/.npm
|
|
119
|
+
# Share cache across Node versions — hash only package-lock.json
|
|
120
|
+
key: npm-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}
|
|
121
|
+
restore-keys: npm-${{ runner.os }}-
|
|
122
|
+
prevention:
|
|
123
|
+
- "Monitor total cache usage regularly with `gh cache list --repo owner/repo` or the GitHub UI (Actions → Caches) — set up an alert when approaching 8 GB."
|
|
124
|
+
- "Add a weekly scheduled workflow that prunes caches older than 5-7 days to stay well under the 10 GB limit."
|
|
125
|
+
- "Use `actions: write` permission and the REST API to manage caches programmatically as part of your CI housekeeping."
|
|
126
|
+
- "Design cache keys to be OS-specific but NOT dependency-version-specific where possible — this reduces the number of unique cache entries stored simultaneously."
|
|
127
|
+
- "For Docker layer caches, prefer GitHub Container Registry or an external caching service to avoid eating into the 10 GB Actions cache budget."
|
|
128
|
+
- "Check for cache key patterns that change too frequently (e.g., including `github.run_id`) — these create orphaned entries that accumulate until evicted."
|
|
129
|
+
docs:
|
|
130
|
+
- url: "https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy"
|
|
131
|
+
label: "GitHub Docs: Cache usage limits and eviction policy"
|
|
132
|
+
- url: "https://docs.github.com/en/rest/actions/cache?apiVersion=2022-11-28"
|
|
133
|
+
label: "GitHub REST API: Actions cache endpoints"
|
|
134
|
+
- url: "https://github.com/actions/cache/blob/main/tips-and-workarounds.md"
|
|
135
|
+
label: "actions/cache: Tips and workarounds"
|
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
id: runner-environment-024
|
|
2
|
+
title: "Docker Hub Pull Rate Limit — toomanyrequests in GitHub Actions"
|
|
3
|
+
category: runner-environment
|
|
4
|
+
severity: error
|
|
5
|
+
tags:
|
|
6
|
+
- docker
|
|
7
|
+
- docker-hub
|
|
8
|
+
- rate-limit
|
|
9
|
+
- container
|
|
10
|
+
- pull
|
|
11
|
+
- authentication
|
|
12
|
+
patterns:
|
|
13
|
+
- regex: "toomanyrequests: You have reached your pull rate limit"
|
|
14
|
+
flags: "i"
|
|
15
|
+
- regex: "You have reached your unauthenticated pull rate limit"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "pull rate limit reached"
|
|
18
|
+
flags: "i"
|
|
19
|
+
- regex: "ERROR: toomanyrequests"
|
|
20
|
+
flags: "i"
|
|
21
|
+
- regex: "429 Too Many Requests.*docker"
|
|
22
|
+
flags: "i"
|
|
23
|
+
error_messages:
|
|
24
|
+
- "toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
|
|
25
|
+
- "Error response from daemon: toomanyrequests: You have reached your unauthenticated pull rate limit."
|
|
26
|
+
- "pull access denied, repository does not exist or may require 'docker login'"
|
|
27
|
+
root_cause: |
|
|
28
|
+
Docker Hub enforces anonymous pull rate limits on all IP addresses. GitHub-hosted
|
|
29
|
+
runners share a pool of egress IPs — multiple concurrent workflow runs from different
|
|
30
|
+
repositories hit Docker Hub from the same IP address, exhausting the shared quota
|
|
31
|
+
rapidly.
|
|
32
|
+
|
|
33
|
+
**Rate limit tiers (as of 2024):**
|
|
34
|
+
- **Unauthenticated (anonymous):** 100 pulls per 6 hours per IP
|
|
35
|
+
- **Authenticated free account:** 200 pulls per 6 hours per account
|
|
36
|
+
- **Docker Hub Pro/Team/Business:** unlimited
|
|
37
|
+
|
|
38
|
+
Because GitHub-hosted runners use a small set of shared egress IPs, popular runners
|
|
39
|
+
can hit the anonymous 100-pull limit within minutes during peak hours. The error
|
|
40
|
+
surfaces as a hard failure when the runner attempts `docker pull` for job containers,
|
|
41
|
+
service containers, container actions, or explicit `docker pull` commands in steps.
|
|
42
|
+
|
|
43
|
+
Even after adding `docker/login-action`, workflows can still hit the limit if they use
|
|
44
|
+
actions that internally pull images (e.g., `docker/build-push-action`, `docker/bake-action`)
|
|
45
|
+
before the login step has executed.
|
|
46
|
+
fix: |
|
|
47
|
+
Add `docker/login-action` as the **first step** in any job that pulls from Docker Hub.
|
|
48
|
+
The action authenticates with Docker Hub, switching from anonymous to authenticated rate
|
|
49
|
+
limits (200 pulls/6 hours per account) or unlimited for Pro/Team accounts.
|
|
50
|
+
|
|
51
|
+
**Recommended approach:**
|
|
52
|
+
1. Create a Docker Hub account (free tier) or use an existing one.
|
|
53
|
+
2. Store credentials as repository secrets: `DOCKERHUB_USERNAME` and `DOCKERHUB_TOKEN`
|
|
54
|
+
(use an Access Token, not your password — generate at hub.docker.com > Account Settings > Security).
|
|
55
|
+
3. Add the login step before any image-pulling step.
|
|
56
|
+
|
|
57
|
+
**For self-hosted runners:** Configure a Docker Hub mirror or use GitHub Packages /
|
|
58
|
+
Amazon ECR / Google Artifact Registry to cache images and avoid Docker Hub entirely.
|
|
59
|
+
fix_code:
|
|
60
|
+
- language: yaml
|
|
61
|
+
label: "Authenticate with Docker Hub before pulling images"
|
|
62
|
+
code: |
|
|
63
|
+
jobs:
|
|
64
|
+
build:
|
|
65
|
+
runs-on: ubuntu-latest
|
|
66
|
+
steps:
|
|
67
|
+
# Login FIRST — before any step that pulls from Docker Hub
|
|
68
|
+
- name: Log in to Docker Hub
|
|
69
|
+
uses: docker/login-action@v3
|
|
70
|
+
with:
|
|
71
|
+
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
72
|
+
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
73
|
+
|
|
74
|
+
- uses: actions/checkout@v4
|
|
75
|
+
|
|
76
|
+
- name: Build and push
|
|
77
|
+
uses: docker/build-push-action@v5
|
|
78
|
+
with:
|
|
79
|
+
push: true
|
|
80
|
+
tags: myorg/myimage:latest
|
|
81
|
+
- language: yaml
|
|
82
|
+
label: "Authenticate for jobs using a container image"
|
|
83
|
+
code: |
|
|
84
|
+
jobs:
|
|
85
|
+
test:
|
|
86
|
+
runs-on: ubuntu-latest
|
|
87
|
+
container:
|
|
88
|
+
image: python:3.12
|
|
89
|
+
credentials:
|
|
90
|
+
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
91
|
+
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
92
|
+
steps:
|
|
93
|
+
- uses: actions/checkout@v4
|
|
94
|
+
- run: python -m pytest
|
|
95
|
+
- language: yaml
|
|
96
|
+
label: "Authenticate for service containers pulling from Docker Hub"
|
|
97
|
+
code: |
|
|
98
|
+
jobs:
|
|
99
|
+
test:
|
|
100
|
+
runs-on: ubuntu-latest
|
|
101
|
+
services:
|
|
102
|
+
postgres:
|
|
103
|
+
image: postgres:16
|
|
104
|
+
credentials:
|
|
105
|
+
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
106
|
+
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
107
|
+
env:
|
|
108
|
+
POSTGRES_PASSWORD: postgres
|
|
109
|
+
steps:
|
|
110
|
+
- uses: actions/checkout@v4
|
|
111
|
+
prevention:
|
|
112
|
+
- "Store a Docker Hub access token (not password) in secrets and use docker/login-action@v3 in all jobs that pull images."
|
|
113
|
+
- "Place the docker/login-action step first in the job — before any action that internally pulls Docker Hub images."
|
|
114
|
+
- "Consider mirroring frequently-used base images to GitHub Container Registry (ghcr.io) or your cloud provider's registry to eliminate Docker Hub dependency."
|
|
115
|
+
- "Use Docker Hub's Automated Builds or a scheduled workflow to keep mirror images up to date."
|
|
116
|
+
- "For service containers and job containers, always provide `credentials:` block alongside `image:` when pulling from Docker Hub."
|
|
117
|
+
docs:
|
|
118
|
+
- url: "https://docs.docker.com/docker-hub/download-rate-limit/"
|
|
119
|
+
label: "Docker Hub: Download rate limit documentation"
|
|
120
|
+
- url: "https://github.com/docker/login-action"
|
|
121
|
+
label: "docker/login-action — GitHub Marketplace"
|
|
122
|
+
- url: "https://docs.github.com/en/actions/use-cases-and-examples/publishing-packages/publishing-docker-images"
|
|
123
|
+
label: "GitHub Docs: Publishing Docker images"
|
|
124
|
+
- url: "https://github.com/actions/runner-images/issues/1445"
|
|
125
|
+
label: "actions/runner-images #1445 — Did Docker Hub rate limit affect GitHub Action?"
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
id: runner-environment-025
|
|
2
|
+
title: "Service Container Network: localhost Works on Runner, Service Name Required in Container Jobs"
|
|
3
|
+
category: runner-environment
|
|
4
|
+
severity: error
|
|
5
|
+
tags:
|
|
6
|
+
- service-containers
|
|
7
|
+
- networking
|
|
8
|
+
- localhost
|
|
9
|
+
- postgres
|
|
10
|
+
- mysql
|
|
11
|
+
- redis
|
|
12
|
+
- container-jobs
|
|
13
|
+
- docker-networking
|
|
14
|
+
patterns:
|
|
15
|
+
- regex: "Connection refused.*(?:localhost|127\\.0\\.0\\.1).*(?:5432|3306|6379|27017)"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "could not connect to server.*Connection refused.*localhost"
|
|
18
|
+
flags: "i"
|
|
19
|
+
- regex: "ECONNREFUSED.*(?:5432|3306|6379|27017)"
|
|
20
|
+
flags: "i"
|
|
21
|
+
- regex: "dial tcp 127\\.0\\.0\\.1:\\d+: connect: connection refused"
|
|
22
|
+
flags: "i"
|
|
23
|
+
error_messages:
|
|
24
|
+
- "Error: connect ECONNREFUSED 127.0.0.1:5432"
|
|
25
|
+
- "could not connect to server: Connection refused (Is the server running on host \"localhost\" (127.0.0.1) and accepting TCP/IP connections on port 5432?)"
|
|
26
|
+
- "OperationalError: connection to server at \"localhost\" (127.0.0.1), port 5432 failed: Connection refused"
|
|
27
|
+
- "Error: getaddrinfo ENOTFOUND postgres"
|
|
28
|
+
root_cause: |
|
|
29
|
+
GitHub Actions service containers run as Docker containers on the runner machine.
|
|
30
|
+
The correct hostname to reach them depends on **how the job itself runs**:
|
|
31
|
+
|
|
32
|
+
**Scenario A — Job runs directly on the runner (no `container:` key):**
|
|
33
|
+
The runner process runs natively on the VM. Docker maps service container ports to the
|
|
34
|
+
runner's loopback interface. Services are reachable at `localhost:<mapped_port>`.
|
|
35
|
+
Using the service name (e.g. `postgres`) as a hostname will **fail** with ENOTFOUND.
|
|
36
|
+
|
|
37
|
+
**Scenario B — Job runs inside a container (`container:` key present):**
|
|
38
|
+
Both the job container and service containers are on the same Docker user-defined network.
|
|
39
|
+
Docker's built-in DNS resolves container names. Services are reachable by their
|
|
40
|
+
**service name** (e.g. `postgres`, `redis`) as the hostname, NOT `localhost`.
|
|
41
|
+
Using `localhost` here will produce "Connection refused" because the service port is
|
|
42
|
+
not bound to the job container's loopback interface.
|
|
43
|
+
|
|
44
|
+
This asymmetry is the root of most service container connectivity bugs:
|
|
45
|
+
developers copy a working non-containerized workflow config into a containerized job
|
|
46
|
+
(or vice versa) without updating the hostname.
|
|
47
|
+
|
|
48
|
+
Additionally, when using `ports:` mapping in service definitions, the runner maps the
|
|
49
|
+
container port to a randomly assigned host port (unless pinned with `host:container`
|
|
50
|
+
syntax). The `${{ job.services.postgres.ports['5432'] }}` expression returns the
|
|
51
|
+
actual host-side port, which may differ from 5432.
|
|
52
|
+
fix: |
|
|
53
|
+
**For jobs running directly on the runner (no `container:` key):**
|
|
54
|
+
Connect to services via `localhost` (or `127.0.0.1`) and the mapped port.
|
|
55
|
+
Use `${{ job.services.<service>.ports['<container_port>'] }}` to get the dynamic host port
|
|
56
|
+
when not using explicit port pinning.
|
|
57
|
+
|
|
58
|
+
**For jobs running in a container (`container:` key):**
|
|
59
|
+
Connect to services via the **service name** as the hostname and the container's
|
|
60
|
+
internal port directly. No port mapping is required — both containers share the same
|
|
61
|
+
Docker network and communicate directly.
|
|
62
|
+
|
|
63
|
+
Use the `options: --add-host` trick only if you need the same workflow YAML to work
|
|
64
|
+
in both contexts (rare — avoid this complexity).
|
|
65
|
+
fix_code:
|
|
66
|
+
- language: yaml
|
|
67
|
+
label: "Correct connectivity for a runner-native job (no container:)"
|
|
68
|
+
code: |
|
|
69
|
+
jobs:
|
|
70
|
+
test:
|
|
71
|
+
runs-on: ubuntu-latest
|
|
72
|
+
# No 'container:' key — job runs directly on the runner VM
|
|
73
|
+
services:
|
|
74
|
+
postgres:
|
|
75
|
+
image: postgres:16
|
|
76
|
+
env:
|
|
77
|
+
POSTGRES_PASSWORD: postgres
|
|
78
|
+
ports:
|
|
79
|
+
- 5432:5432 # pin host:container port for simplicity
|
|
80
|
+
options: >-
|
|
81
|
+
--health-cmd pg_isready
|
|
82
|
+
--health-interval 10s
|
|
83
|
+
--health-timeout 5s
|
|
84
|
+
--health-retries 5
|
|
85
|
+
steps:
|
|
86
|
+
- uses: actions/checkout@v4
|
|
87
|
+
- name: Run tests
|
|
88
|
+
env:
|
|
89
|
+
# Use localhost — runner-native job maps service ports to loopback
|
|
90
|
+
DATABASE_URL: postgres://postgres:postgres@localhost:5432/testdb
|
|
91
|
+
run: npm test
|
|
92
|
+
- language: yaml
|
|
93
|
+
label: "Correct connectivity for a containerized job (container: key present)"
|
|
94
|
+
code: |
|
|
95
|
+
jobs:
|
|
96
|
+
test:
|
|
97
|
+
runs-on: ubuntu-latest
|
|
98
|
+
container:
|
|
99
|
+
image: node:20 # Job itself runs inside a container
|
|
100
|
+
services:
|
|
101
|
+
postgres:
|
|
102
|
+
image: postgres:16
|
|
103
|
+
env:
|
|
104
|
+
POSTGRES_PASSWORD: postgres
|
|
105
|
+
# NO ports: mapping needed — containers share the Docker network
|
|
106
|
+
options: >-
|
|
107
|
+
--health-cmd pg_isready
|
|
108
|
+
--health-interval 10s
|
|
109
|
+
--health-timeout 5s
|
|
110
|
+
--health-retries 5
|
|
111
|
+
steps:
|
|
112
|
+
- uses: actions/checkout@v4
|
|
113
|
+
- name: Run tests
|
|
114
|
+
env:
|
|
115
|
+
# Use service NAME "postgres" — Docker DNS resolves it within the network
|
|
116
|
+
DATABASE_URL: postgres://postgres:postgres@postgres:5432/testdb
|
|
117
|
+
run: npm test
|
|
118
|
+
prevention:
|
|
119
|
+
- "When your job uses `container:`, always connect to services via the service name (e.g. `postgres`, `redis`), NOT `localhost`."
|
|
120
|
+
- "When your job runs natively (no `container:`), always connect to services via `localhost` and the mapped host port."
|
|
121
|
+
- "Use `options: --health-cmd / --health-interval / --health-retries` on service containers to ensure they are ready before steps execute."
|
|
122
|
+
- "Avoid using `localhost` as the database hostname in connection strings — parameterize it so the same code works in both runner-native and containerized jobs."
|
|
123
|
+
- "If you move a job from runner-native to containerized (or vice versa), always audit all service hostnames in env variables and connection strings."
|
|
124
|
+
docs:
|
|
125
|
+
- url: "https://docs.github.com/en/actions/use-cases-and-examples/using-containerized-services/about-service-containers"
|
|
126
|
+
label: "GitHub Docs: About service containers"
|
|
127
|
+
- url: "https://docs.github.com/en/actions/use-cases-and-examples/using-containerized-services/creating-postgresql-service-containers"
|
|
128
|
+
label: "GitHub Docs: Creating PostgreSQL service containers"
|
|
129
|
+
- url: "https://docs.github.com/en/actions/use-cases-and-examples/using-containerized-services/creating-redis-service-containers"
|
|
130
|
+
label: "GitHub Docs: Creating Redis service containers"
|
|
@@ -0,0 +1,126 @@
|
|
|
1
|
+
id: runner-environment-027
|
|
2
|
+
title: "actions/setup-python cache Fails — No Dependency File Found for pip/poetry/pipenv"
|
|
3
|
+
category: runner-environment
|
|
4
|
+
severity: error
|
|
5
|
+
tags:
|
|
6
|
+
- setup-python
|
|
7
|
+
- cache
|
|
8
|
+
- pip
|
|
9
|
+
- poetry
|
|
10
|
+
- pipenv
|
|
11
|
+
- cache-dependency-path
|
|
12
|
+
- requirements.txt
|
|
13
|
+
- pyproject.toml
|
|
14
|
+
patterns:
|
|
15
|
+
- regex: "No file.*found for the supported package managers"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "Error: No file in .* found for the supported package managers"
|
|
18
|
+
flags: "i"
|
|
19
|
+
- regex: "setup-python.*cache.*dependency.*not found"
|
|
20
|
+
flags: "i"
|
|
21
|
+
- regex: "dependencies were not found for the cache-dependency-path"
|
|
22
|
+
flags: "i"
|
|
23
|
+
- regex: "Could not find .* in .*requirements"
|
|
24
|
+
flags: "i"
|
|
25
|
+
error_messages:
|
|
26
|
+
- "Error: No file in /home/runner/work/my-repo/my-repo found for the supported package managers (pip, pipenv, poetry), file patterns (requirements*.txt, Pipfile.lock, poetry.lock)"
|
|
27
|
+
- "Error: No file in /home/runner/work found for the supported package managers (pip), file patterns (requirements*.txt)"
|
|
28
|
+
- "dependencies were not found for the cache-dependency-path input"
|
|
29
|
+
root_cause: |
|
|
30
|
+
When `actions/setup-python` is configured with `cache: 'pip'` (or `'poetry'` /
|
|
31
|
+
`'pipenv'`), it looks for a dependency lockfile to use as the cache key. The action
|
|
32
|
+
searches the repository for these file patterns:
|
|
33
|
+
|
|
34
|
+
| Cache type | File patterns searched |
|
|
35
|
+
|------------|---------------------------------------------------|
|
|
36
|
+
| `pip` | `requirements*.txt`, `requirements/*.txt` |
|
|
37
|
+
| `poetry` | `poetry.lock` |
|
|
38
|
+
| `pipenv` | `Pipfile.lock` |
|
|
39
|
+
|
|
40
|
+
If no matching file is found (either because the project uses a non-standard layout,
|
|
41
|
+
the file has a custom name, or the lockfile is gitignored), the action fails with this
|
|
42
|
+
error.
|
|
43
|
+
|
|
44
|
+
**Common causes:**
|
|
45
|
+
1. **Non-standard requirements file name**: `deps.txt`, `dev-requirements.txt` (outside
|
|
46
|
+
`requirements*.txt` glob), or stored in a subdirectory not matching the search path.
|
|
47
|
+
2. **pyproject.toml without poetry.lock**: Modern Python projects using `pyproject.toml`
|
|
48
|
+
with pip or flit don't generate a lockfile by default.
|
|
49
|
+
3. **Monorepo layout**: The requirements file is in a subdirectory (e.g.,
|
|
50
|
+
`backend/requirements.txt`) but the default search is from the repo root.
|
|
51
|
+
4. **Gitignored lockfiles**: `poetry.lock` or `Pipfile.lock` is in `.gitignore`, so
|
|
52
|
+
setup-python can't find it on the runner.
|
|
53
|
+
|
|
54
|
+
The `cache-dependency-path` input was added (actions/setup-python #361) to address
|
|
55
|
+
non-standard layouts, but is often overlooked in workflow templates.
|
|
56
|
+
fix: |
|
|
57
|
+
Set the `cache-dependency-path` input to point to your actual dependency file, or
|
|
58
|
+
ensure the file follows the default naming convention.
|
|
59
|
+
|
|
60
|
+
**Option 1 (preferred):** Use `cache-dependency-path` to specify the exact path or glob.
|
|
61
|
+
**Option 2:** Rename your requirements file to match the default patterns (`requirements.txt`
|
|
62
|
+
or `requirements-*.txt`).
|
|
63
|
+
**Option 3:** If you don't need caching, remove `cache:` from the setup-python step entirely.
|
|
64
|
+
**Option 4 (pyproject.toml):** If using pip with pyproject.toml, create a
|
|
65
|
+
`requirements.txt` (or use `pip-compile`) to generate a lockfile for cache keying.
|
|
66
|
+
fix_code:
|
|
67
|
+
- language: yaml
|
|
68
|
+
label: "Specify custom requirements file path with cache-dependency-path"
|
|
69
|
+
code: |
|
|
70
|
+
- uses: actions/setup-python@v5
|
|
71
|
+
with:
|
|
72
|
+
python-version: '3.12'
|
|
73
|
+
cache: 'pip'
|
|
74
|
+
cache-dependency-path: |
|
|
75
|
+
backend/requirements.txt
|
|
76
|
+
backend/requirements-dev.txt
|
|
77
|
+
- language: yaml
|
|
78
|
+
label: "Use cache-dependency-path glob for monorepo with multiple requirements files"
|
|
79
|
+
code: |
|
|
80
|
+
- uses: actions/setup-python@v5
|
|
81
|
+
with:
|
|
82
|
+
python-version: '3.12'
|
|
83
|
+
cache: 'pip'
|
|
84
|
+
cache-dependency-path: '**/requirements*.txt'
|
|
85
|
+
- language: yaml
|
|
86
|
+
label: "Poetry project with explicit cache-dependency-path"
|
|
87
|
+
code: |
|
|
88
|
+
- uses: actions/setup-python@v5
|
|
89
|
+
with:
|
|
90
|
+
python-version: '3.12'
|
|
91
|
+
cache: 'poetry'
|
|
92
|
+
cache-dependency-path: 'pyproject/poetry.lock'
|
|
93
|
+
- language: yaml
|
|
94
|
+
label: "pyproject.toml project using pip — generate a lockfile for cache keying"
|
|
95
|
+
code: |
|
|
96
|
+
- uses: actions/setup-python@v5
|
|
97
|
+
with:
|
|
98
|
+
python-version: '3.12'
|
|
99
|
+
# No cache here — use actions/cache manually with hash of pyproject.toml
|
|
100
|
+
- uses: actions/cache@v4
|
|
101
|
+
with:
|
|
102
|
+
path: ~/.cache/pip
|
|
103
|
+
key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml') }}
|
|
104
|
+
restore-keys: |
|
|
105
|
+
${{ runner.os }}-pip-
|
|
106
|
+
- language: yaml
|
|
107
|
+
label: "Skip caching entirely when no lockfile exists"
|
|
108
|
+
code: |
|
|
109
|
+
- uses: actions/setup-python@v5
|
|
110
|
+
with:
|
|
111
|
+
python-version: '3.12'
|
|
112
|
+
# Omit 'cache:' entirely — no caching, no failure
|
|
113
|
+
- run: pip install -r deps.txt
|
|
114
|
+
prevention:
|
|
115
|
+
- "Always set `cache-dependency-path` explicitly rather than relying on the default file search — it makes the workflow self-documenting and prevents surprises on rename."
|
|
116
|
+
- "Commit `poetry.lock` and `Pipfile.lock` to version control — these files serve as both the reproducible install spec and the cache key."
|
|
117
|
+
- "In monorepos, use a glob pattern like `**/requirements*.txt` to match files across subdirectories."
|
|
118
|
+
- "If pyproject.toml is your only dependency file, use `actions/cache@v4` directly with `hashFiles('pyproject.toml')` instead of setup-python's built-in cache."
|
|
119
|
+
- "Pin to `actions/setup-python@v5` or later — older versions have different cache file discovery behavior."
|
|
120
|
+
docs:
|
|
121
|
+
- url: "https://github.com/actions/setup-python?tab=readme-ov-file#caching-packages-dependencies"
|
|
122
|
+
label: "actions/setup-python: Caching packages dependencies"
|
|
123
|
+
- url: "https://github.com/actions/setup-python/issues/361"
|
|
124
|
+
label: "actions/setup-python #361: Support cache-dependency-paths outside the current directory"
|
|
125
|
+
- url: "https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows"
|
|
126
|
+
label: "GitHub Docs: Caching dependencies to speed up workflows"
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
id: runner-environment-026
|
|
2
|
+
title: "ubuntu-latest Switched to Ubuntu 24.04 — Breaks Python 3.8/3.9, System Packages, OpenSSL"
|
|
3
|
+
category: runner-environment
|
|
4
|
+
severity: error
|
|
5
|
+
tags:
|
|
6
|
+
- ubuntu
|
|
7
|
+
- ubuntu-24.04
|
|
8
|
+
- runner-image
|
|
9
|
+
- python
|
|
10
|
+
- openssl
|
|
11
|
+
- apt
|
|
12
|
+
- migration
|
|
13
|
+
- breaking-change
|
|
14
|
+
patterns:
|
|
15
|
+
- regex: "Unable to locate package python3\\.[89]"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "python3\\.[89].*not found.*ubuntu.*24"
|
|
18
|
+
flags: "i"
|
|
19
|
+
- regex: "E: Package 'python3\\.[89]' has no installation candidate"
|
|
20
|
+
flags: "i"
|
|
21
|
+
- regex: "error.*openssl.*libssl1\\.1.*not available"
|
|
22
|
+
flags: "i"
|
|
23
|
+
- regex: "deadsnakes.*ppa.*focal.*not available.*noble"
|
|
24
|
+
flags: "i"
|
|
25
|
+
- regex: "ubuntu-latest.*ubuntu 24"
|
|
26
|
+
flags: "i"
|
|
27
|
+
error_messages:
|
|
28
|
+
- "E: Package 'python3.8' has no installation candidate"
|
|
29
|
+
- "E: Package 'python3.9' has no installation candidate"
|
|
30
|
+
- "E: Package 'libssl1.1' has no installation candidate"
|
|
31
|
+
- "Error: The version '3.8' with architecture 'x64' was not found for Ubuntu 24.04"
|
|
32
|
+
- "Error: deadsnakes/ppa does not provide Python 3.8 packages for Ubuntu Noble (24.04)"
|
|
33
|
+
root_cause: |
|
|
34
|
+
GitHub began rolling out Ubuntu 24.04 as the new target for `ubuntu-latest` on
|
|
35
|
+
**December 5, 2024**, completing the transition on **January 17, 2025** (tracked in
|
|
36
|
+
actions/runner-images #10636, 266 👍 reactions). Any workflow using `runs-on: ubuntu-latest`
|
|
37
|
+
without explicit version pinning now runs on Ubuntu 24.04 ("Noble Numbat") instead of 22.04.
|
|
38
|
+
|
|
39
|
+
**Key breaking changes in Ubuntu 24.04:**
|
|
40
|
+
|
|
41
|
+
1. **Python 3.8 and 3.9 are end-of-life and not available** — not via `apt` and not via
|
|
42
|
+
the `deadsnakes/ppa` PPA (which only backports for supported Ubuntu releases). Workflows
|
|
43
|
+
that `apt-get install python3.8` or use `setup-python` with version `3.8` or `3.9` fail.
|
|
44
|
+
|
|
45
|
+
2. **OpenSSL 1.1 removed** — Ubuntu 24.04 ships only OpenSSL 3.x. Packages and Python
|
|
46
|
+
versions that were compiled against OpenSSL 1.1 (`libssl1.1`) will fail to install or
|
|
47
|
+
run. This affects older Ruby gems, some Go binaries, and legacy pip packages.
|
|
48
|
+
|
|
49
|
+
3. **System Python is 3.12** — the `python3` system binary points to Python 3.12, which
|
|
50
|
+
may break scripts that relied on the 3.10 default in 22.04.
|
|
51
|
+
|
|
52
|
+
4. **`apt` package name changes** — some packages were renamed or split between 22.04 and
|
|
53
|
+
24.04 (e.g., `python3-distutils` is gone; functionality merged into `python3`).
|
|
54
|
+
|
|
55
|
+
5. **`/usr/bin/python` symlink absent by default** — workflows calling bare `python`
|
|
56
|
+
(not `python3`) fail with "python: command not found".
|
|
57
|
+
fix: |
|
|
58
|
+
**Short-term — pin to ubuntu-22.04:**
|
|
59
|
+
Replace `runs-on: ubuntu-latest` with `runs-on: ubuntu-22.04` to freeze the runner image
|
|
60
|
+
while you migrate. Note: ubuntu-22.04 will eventually reach end-of-support (~April 2027).
|
|
61
|
+
|
|
62
|
+
**Long-term — migrate to supported Python/package versions:**
|
|
63
|
+
- Replace Python 3.8/3.9 with Python 3.10, 3.11, or 3.12 using `actions/setup-python`.
|
|
64
|
+
`actions/setup-python` downloads a pre-built binary from the tool cache — it works
|
|
65
|
+
regardless of the runner's system Python.
|
|
66
|
+
- Replace `libssl1.1` dependencies by upgrading the affected package to an OpenSSL-3
|
|
67
|
+
compatible version, or adding a compatibility shim if available.
|
|
68
|
+
- Replace bare `python` calls with `python3`.
|
|
69
|
+
|
|
70
|
+
**For `deadsnakes` PPA users:** `deadsnakes/ppa` only provides Python 3.8/3.9 for
|
|
71
|
+
Ubuntu 20.04 (Focal) and 22.04 (Jammy) — not 24.04 (Noble). There is no supported
|
|
72
|
+
workaround; upgrade to a supported Python version.
|
|
73
|
+
fix_code:
|
|
74
|
+
- language: yaml
|
|
75
|
+
label: "Pin to ubuntu-22.04 as a short-term fix while migrating"
|
|
76
|
+
code: |
|
|
77
|
+
jobs:
|
|
78
|
+
test:
|
|
79
|
+
# Pin explicitly until Python 3.8/3.9 dependencies are upgraded
|
|
80
|
+
runs-on: ubuntu-22.04
|
|
81
|
+
steps:
|
|
82
|
+
- uses: actions/checkout@v4
|
|
83
|
+
- run: python3.9 -m pytest
|
|
84
|
+
- language: yaml
|
|
85
|
+
label: "Use actions/setup-python to install any Python version on ubuntu-latest"
|
|
86
|
+
code: |
|
|
87
|
+
jobs:
|
|
88
|
+
test:
|
|
89
|
+
runs-on: ubuntu-latest # Now 24.04
|
|
90
|
+
steps:
|
|
91
|
+
- uses: actions/checkout@v4
|
|
92
|
+
- name: Set up Python
|
|
93
|
+
uses: actions/setup-python@v5
|
|
94
|
+
with:
|
|
95
|
+
# setup-python downloads pre-built binaries — works on any runner OS
|
|
96
|
+
python-version: '3.9'
|
|
97
|
+
- run: python --version # 3.9.x from setup-python's tool cache
|
|
98
|
+
- run: pip install -r requirements.txt
|
|
99
|
+
- language: yaml
|
|
100
|
+
label: "Test matrix against multiple Python versions to find breakages early"
|
|
101
|
+
code: |
|
|
102
|
+
jobs:
|
|
103
|
+
test:
|
|
104
|
+
runs-on: ubuntu-latest
|
|
105
|
+
strategy:
|
|
106
|
+
matrix:
|
|
107
|
+
python-version: ['3.9', '3.10', '3.11', '3.12']
|
|
108
|
+
steps:
|
|
109
|
+
- uses: actions/checkout@v4
|
|
110
|
+
- uses: actions/setup-python@v5
|
|
111
|
+
with:
|
|
112
|
+
python-version: ${{ matrix.python-version }}
|
|
113
|
+
- run: pip install -r requirements.txt && pytest
|
|
114
|
+
- language: yaml
|
|
115
|
+
label: "Fix bare 'python' command not found on Ubuntu 24.04"
|
|
116
|
+
code: |
|
|
117
|
+
steps:
|
|
118
|
+
- name: Ensure python symlink exists
|
|
119
|
+
run: |
|
|
120
|
+
# Ubuntu 24.04 no longer creates /usr/bin/python by default
|
|
121
|
+
sudo ln -sf /usr/bin/python3 /usr/bin/python
|
|
122
|
+
# Or: use actions/setup-python which sets up the PATH correctly
|
|
123
|
+
prevention:
|
|
124
|
+
- "Pin `runs-on: ubuntu-22.04` explicitly if your workflow depends on Python 3.8/3.9, libssl1.1, or other 22.04-specific packages."
|
|
125
|
+
- "Use `actions/setup-python@v5` to install a specific Python version instead of relying on `apt-get install python3.x` — it works across all Ubuntu versions."
|
|
126
|
+
- "When you upgrade Ubuntu runner versions, scan `apt-get install` lines in run steps for packages that may not exist in the new Ubuntu release."
|
|
127
|
+
- "Replace bare `python` invocations with `python3` or use `actions/setup-python` which adds an alias."
|
|
128
|
+
- "Test your workflows against `ubuntu-24.04` explicitly before `ubuntu-latest` changes, by running on both runner images in a matrix."
|
|
129
|
+
- "Subscribe to https://github.com/actions/runner-images/issues for announcements about upcoming `ubuntu-latest` image transitions."
|
|
130
|
+
docs:
|
|
131
|
+
- url: "https://github.com/actions/runner-images/issues/10636"
|
|
132
|
+
label: "actions/runner-images #10636 — Ubuntu-latest workflows will use Ubuntu-24.04 image"
|
|
133
|
+
- url: "https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md"
|
|
134
|
+
label: "Ubuntu 24.04 runner image — installed software reference"
|
|
135
|
+
- url: "https://github.com/actions/setup-python"
|
|
136
|
+
label: "actions/setup-python — install any Python version on any runner"
|
|
137
|
+
- url: "https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources"
|
|
138
|
+
label: "GitHub Docs: Supported runners and hardware resources"
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
id: silent-failures-011
|
|
2
|
+
title: "GITHUB_OUTPUT Multiline Values Silently Truncated — Heredoc EOF Required"
|
|
3
|
+
category: silent-failures
|
|
4
|
+
severity: silent-failure
|
|
5
|
+
tags:
|
|
6
|
+
- github-output
|
|
7
|
+
- multiline
|
|
8
|
+
- output
|
|
9
|
+
- heredoc
|
|
10
|
+
- truncation
|
|
11
|
+
- environment-files
|
|
12
|
+
patterns:
|
|
13
|
+
- regex: "echo\\s+['\"]?\\w+=.*\\$\\([^)]+\\).*>>\\s+\\$GITHUB_OUTPUT"
|
|
14
|
+
flags: "i"
|
|
15
|
+
- regex: "invalid value.*GITHUB_OUTPUT.*contains newline"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "Error: Unable to process file command 'output'"
|
|
18
|
+
flags: "i"
|
|
19
|
+
- regex: "GITHUB_OUTPUT.*multiline"
|
|
20
|
+
flags: "i"
|
|
21
|
+
error_messages:
|
|
22
|
+
- "Error: Unable to process file command 'output' successfully."
|
|
23
|
+
- "Invalid format 'key=line1\\nline2' - value contains newline, use heredoc syntax"
|
|
24
|
+
- "Warning: Unexpected input(s) 'output', valid inputs are"
|
|
25
|
+
root_cause: |
|
|
26
|
+
The `GITHUB_OUTPUT` environment file protocol (introduced November 2022 as a replacement
|
|
27
|
+
for the deprecated `::set-output` command) uses a line-based key=value format that does
|
|
28
|
+
NOT support literal newline characters in values when using simple `echo "key=value"` syntax.
|
|
29
|
+
|
|
30
|
+
When a step sets a multiline output using the naive approach:
|
|
31
|
+
```bash
|
|
32
|
+
echo "REPORT=$(cat report.txt)" >> $GITHUB_OUTPUT
|
|
33
|
+
```
|
|
34
|
+
the shell expands `$(cat report.txt)` into a multiline string, which is then written as:
|
|
35
|
+
```
|
|
36
|
+
REPORT=line1
|
|
37
|
+
line2
|
|
38
|
+
line3
|
|
39
|
+
```
|
|
40
|
+
The runner reads the first newline as end-of-value for `REPORT`, then tries to parse `line2`
|
|
41
|
+
as a new key=value entry (which fails silently or errors). The downstream step reading
|
|
42
|
+
`${{ steps.id.outputs.REPORT }}` receives only `line1` — or an empty string if the parser
|
|
43
|
+
rejects the entry entirely.
|
|
44
|
+
|
|
45
|
+
This is a **silent failure** because the workflow step itself does not fail — the `echo`
|
|
46
|
+
command exits successfully. The broken output only surfaces downstream, often as an empty
|
|
47
|
+
variable or a truncated value that's hard to trace back to the output-setting step.
|
|
48
|
+
|
|
49
|
+
The same issue affects `$GITHUB_ENV` for multiline environment variable values.
|
|
50
|
+
fix: |
|
|
51
|
+
Use the **heredoc (multi-line delimiter) syntax** documented by GitHub for multiline values.
|
|
52
|
+
The format is:
|
|
53
|
+
```
|
|
54
|
+
{name}<<{delimiter}
|
|
55
|
+
{value}
|
|
56
|
+
{delimiter}
|
|
57
|
+
```
|
|
58
|
+
where `{delimiter}` is any string that does NOT appear in the value itself. `EOF` is the
|
|
59
|
+
conventional choice, but using a random string (e.g. `$(uuidgen)`) prevents injection
|
|
60
|
+
attacks if the value is untrusted.
|
|
61
|
+
|
|
62
|
+
**For `$GITHUB_ENV` multiline values:** use the identical heredoc pattern with `$GITHUB_ENV`.
|
|
63
|
+
|
|
64
|
+
**For JavaScript/TypeScript actions:** use `core.setOutput('name', value)` from
|
|
65
|
+
`@actions/core` — it handles serialization automatically and is not subject to this limitation.
|
|
66
|
+
fix_code:
|
|
67
|
+
- language: yaml
|
|
68
|
+
label: "Correct multiline GITHUB_OUTPUT using heredoc EOF syntax"
|
|
69
|
+
code: |
|
|
70
|
+
- name: Set multiline output
|
|
71
|
+
id: report
|
|
72
|
+
run: |
|
|
73
|
+
# WRONG: echo "REPORT=$(cat report.txt)" >> $GITHUB_OUTPUT
|
|
74
|
+
# Correct: heredoc syntax
|
|
75
|
+
echo "REPORT<<EOF" >> $GITHUB_OUTPUT
|
|
76
|
+
cat report.txt >> $GITHUB_OUTPUT
|
|
77
|
+
echo "EOF" >> $GITHUB_OUTPUT
|
|
78
|
+
|
|
79
|
+
- name: Use multiline output
|
|
80
|
+
run: |
|
|
81
|
+
echo "${{ steps.report.outputs.REPORT }}"
|
|
82
|
+
- language: yaml
|
|
83
|
+
label: "Use random delimiter to prevent injection from untrusted content"
|
|
84
|
+
code: |
|
|
85
|
+
- name: Set multiline output safely
|
|
86
|
+
id: data
|
|
87
|
+
run: |
|
|
88
|
+
DELIMITER=$(openssl rand -hex 16)
|
|
89
|
+
echo "JSON_OUTPUT<<${DELIMITER}" >> $GITHUB_OUTPUT
|
|
90
|
+
curl -s https://api.example.com/data >> $GITHUB_OUTPUT
|
|
91
|
+
echo "${DELIMITER}" >> $GITHUB_OUTPUT
|
|
92
|
+
- language: yaml
|
|
93
|
+
label: "Multiline GITHUB_ENV using heredoc (same pattern)"
|
|
94
|
+
code: |
|
|
95
|
+
- name: Set multiline environment variable
|
|
96
|
+
run: |
|
|
97
|
+
echo "CHANGELOG<<EOF" >> $GITHUB_ENV
|
|
98
|
+
git log --oneline -10 >> $GITHUB_ENV
|
|
99
|
+
echo "EOF" >> $GITHUB_ENV
|
|
100
|
+
prevention:
|
|
101
|
+
- "Never use `echo \"KEY=$(command)\" >> $GITHUB_OUTPUT` when the command output may span multiple lines."
|
|
102
|
+
- "Always use the heredoc EOF syntax for any output value that could contain newlines: `echo \"KEY<<EOF\" >> $GITHUB_OUTPUT && <value> >> $GITHUB_OUTPUT && echo \"EOF\" >> $GITHUB_OUTPUT`."
|
|
103
|
+
- "Use a random delimiter (`$(openssl rand -hex 16)`) instead of `EOF` when the value originates from untrusted input to prevent delimiter injection."
|
|
104
|
+
- "In JavaScript/TypeScript actions, use `@actions/core`'s `core.setOutput()` which handles serialization automatically."
|
|
105
|
+
- "Test multiline outputs locally with `act` or by printing `cat $GITHUB_OUTPUT` in a subsequent step to verify the full value was captured."
|
|
106
|
+
docs:
|
|
107
|
+
- url: "https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#multiline-strings"
|
|
108
|
+
label: "GitHub Docs: Workflow commands — Multiline strings"
|
|
109
|
+
- url: "https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/passing-information-between-jobs"
|
|
110
|
+
label: "GitHub Docs: Passing information between jobs"
|
|
111
|
+
- url: "https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/"
|
|
112
|
+
label: "GitHub Changelog: Deprecating save-state and set-output commands"
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
id: triggers-007
|
|
2
|
+
title: "merge_group Trigger Missing — Required Checks Never Run in Merge Queue"
|
|
3
|
+
category: triggers
|
|
4
|
+
severity: silent-failure
|
|
5
|
+
tags:
|
|
6
|
+
- merge_group
|
|
7
|
+
- merge-queue
|
|
8
|
+
- required-checks
|
|
9
|
+
- branch-protection
|
|
10
|
+
- triggers
|
|
11
|
+
- pull_request
|
|
12
|
+
patterns:
|
|
13
|
+
- regex: "merge_group"
|
|
14
|
+
flags: "i"
|
|
15
|
+
- regex: "Expected.*Waiting.*merge queue"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "merge queue.*required.*status check.*waiting"
|
|
18
|
+
flags: "i"
|
|
19
|
+
error_messages:
|
|
20
|
+
- "Required check 'CI' is expected — Waiting"
|
|
21
|
+
- "All checks have passed except those that are waiting for merge queue"
|
|
22
|
+
- "The merge queue is waiting for the required check to pass"
|
|
23
|
+
root_cause: |
|
|
24
|
+
GitHub's merge queue (enabled via branch protection rules → "Require merge queue")
|
|
25
|
+
creates a `merge_group` event when a PR is added to the queue. This event is distinct
|
|
26
|
+
from `pull_request` and `push` — a workflow must explicitly declare `on: merge_group:`
|
|
27
|
+
to receive it.
|
|
28
|
+
|
|
29
|
+
When a workflow is required by branch protection but does NOT include `merge_group`
|
|
30
|
+
as a trigger, the merge queue adds the PR to a temporary merge group branch (format:
|
|
31
|
+
`gh-readonly-queue/{base}/{pr-number}`) but the required workflow **never starts**.
|
|
32
|
+
GitHub shows the required check as "Expected — Waiting" indefinitely, and the PR
|
|
33
|
+
cannot merge.
|
|
34
|
+
|
|
35
|
+
**Why this is subtle:**
|
|
36
|
+
- The workflow runs fine for normal `pull_request` events (dev branch → PR, all checks
|
|
37
|
+
pass).
|
|
38
|
+
- The failure only manifests once the PR is actually added to the merge queue.
|
|
39
|
+
- The "Waiting" status in the merge queue looks different from a failed check, making
|
|
40
|
+
it unclear that the workflow trigger is misconfigured.
|
|
41
|
+
|
|
42
|
+
**The merge_group event payload** is slightly different from `pull_request`:
|
|
43
|
+
- `github.event.merge_group.base_ref` — the base branch
|
|
44
|
+
- `github.event.merge_group.head_sha` — the merged commit SHA to test
|
|
45
|
+
- `github.event.merge_group.head_ref` — the temporary merge branch name
|
|
46
|
+
fix: |
|
|
47
|
+
Add `merge_group:` to the `on:` block of every workflow that is listed as a required
|
|
48
|
+
status check for merge queue-protected branches.
|
|
49
|
+
|
|
50
|
+
**Important:** `merge_group` does NOT automatically run `pull_request` workflows — it
|
|
51
|
+
is a completely separate event type. You must add it explicitly.
|
|
52
|
+
|
|
53
|
+
If your workflow has conditions that reference `github.event_name == 'pull_request'`,
|
|
54
|
+
update those conditions to also allow `merge_group` events, or restructure them to
|
|
55
|
+
check `github.event.pull_request || github.event.merge_group`.
|
|
56
|
+
fix_code:
|
|
57
|
+
- language: yaml
|
|
58
|
+
label: "Add merge_group trigger to an existing CI workflow"
|
|
59
|
+
code: |
|
|
60
|
+
on:
|
|
61
|
+
pull_request:
|
|
62
|
+
branches: [main, develop]
|
|
63
|
+
merge_group: # <-- Add this to receive merge queue events
|
|
64
|
+
types: [checks_requested]
|
|
65
|
+
|
|
66
|
+
jobs:
|
|
67
|
+
ci:
|
|
68
|
+
runs-on: ubuntu-latest
|
|
69
|
+
steps:
|
|
70
|
+
- uses: actions/checkout@v4
|
|
71
|
+
- name: Run tests
|
|
72
|
+
run: npm test
|
|
73
|
+
- language: yaml
|
|
74
|
+
label: "Conditional logic that handles both pull_request and merge_group"
|
|
75
|
+
code: |
|
|
76
|
+
on:
|
|
77
|
+
pull_request:
|
|
78
|
+
merge_group:
|
|
79
|
+
types: [checks_requested]
|
|
80
|
+
|
|
81
|
+
jobs:
|
|
82
|
+
ci:
|
|
83
|
+
runs-on: ubuntu-latest
|
|
84
|
+
steps:
|
|
85
|
+
- uses: actions/checkout@v4
|
|
86
|
+
|
|
87
|
+
# Access the base SHA correctly for both event types
|
|
88
|
+
- name: Get base SHA
|
|
89
|
+
id: base
|
|
90
|
+
run: |
|
|
91
|
+
if [ "${{ github.event_name }}" = "merge_group" ]; then
|
|
92
|
+
echo "sha=${{ github.event.merge_group.base_sha }}" >> $GITHUB_OUTPUT
|
|
93
|
+
else
|
|
94
|
+
echo "sha=${{ github.event.pull_request.base.sha }}" >> $GITHUB_OUTPUT
|
|
95
|
+
fi
|
|
96
|
+
- language: yaml
|
|
97
|
+
label: "Reusable workflow that supports merge_group via workflow_call"
|
|
98
|
+
code: |
|
|
99
|
+
# In the caller workflow:
|
|
100
|
+
on:
|
|
101
|
+
pull_request:
|
|
102
|
+
merge_group:
|
|
103
|
+
types: [checks_requested]
|
|
104
|
+
|
|
105
|
+
jobs:
|
|
106
|
+
ci:
|
|
107
|
+
uses: ./.github/workflows/ci-reusable.yml
|
|
108
|
+
with:
|
|
109
|
+
ref: ${{ github.event.pull_request.head.sha || github.event.merge_group.head_sha }}
|
|
110
|
+
prevention:
|
|
111
|
+
- "When enabling merge queue on a branch protection rule, immediately check that every required workflow has `merge_group` in its `on:` triggers."
|
|
112
|
+
- "Add a repository-level workflow audit that lists all required status checks and verifies each has a matching `merge_group` trigger."
|
|
113
|
+
- "Test the merge queue setup by creating a test PR and attempting to add it to the queue — if checks show 'Waiting', add the trigger."
|
|
114
|
+
- "Third-party Actions and path-filter actions (dorny/paths-filter, tj-actions/changed-files) may need explicit `merge_group` support — check their changelogs."
|
|
115
|
+
- "GitHub's documentation now includes a merge queue section — review it when enabling the feature in your organization."
|
|
116
|
+
docs:
|
|
117
|
+
- url: "https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue"
|
|
118
|
+
label: "GitHub Docs: Managing a merge queue"
|
|
119
|
+
- url: "https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#merge_group"
|
|
120
|
+
label: "GitHub Actions: merge_group event trigger"
|
|
121
|
+
- url: "https://github.blog/changelog/2023-02-08-pull-request-merge-queue-public-beta/"
|
|
122
|
+
label: "GitHub Changelog: Pull request merge queue public beta (Feb 2023)"
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
id: yaml-syntax-015
|
|
2
|
+
title: "Deprecated ::set-output:: / ::save-state:: / ::add-path:: Workflow Commands"
|
|
3
|
+
category: yaml-syntax
|
|
4
|
+
severity: warning
|
|
5
|
+
tags:
|
|
6
|
+
- set-output
|
|
7
|
+
- save-state
|
|
8
|
+
- add-path
|
|
9
|
+
- deprecated
|
|
10
|
+
- GITHUB_OUTPUT
|
|
11
|
+
- GITHUB_ENV
|
|
12
|
+
- GITHUB_PATH
|
|
13
|
+
- workflow-commands
|
|
14
|
+
patterns:
|
|
15
|
+
- regex: "The `set-output` command is deprecated"
|
|
16
|
+
flags: "i"
|
|
17
|
+
- regex: "The `save-state` command is deprecated"
|
|
18
|
+
flags: "i"
|
|
19
|
+
- regex: "The `set-env` command is disabled"
|
|
20
|
+
flags: "i"
|
|
21
|
+
- regex: "::set-output name="
|
|
22
|
+
flags: ""
|
|
23
|
+
- regex: "::save-state name="
|
|
24
|
+
flags: ""
|
|
25
|
+
- regex: "::add-path::"
|
|
26
|
+
flags: ""
|
|
27
|
+
- regex: "workflow commands.*deprecated.*will be disabled"
|
|
28
|
+
flags: "i"
|
|
29
|
+
error_messages:
|
|
30
|
+
- "Warning: The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter"
|
|
31
|
+
- "Warning: The `save-state` command is deprecated and will be disabled soon. Please upgrade to using Environment Files."
|
|
32
|
+
- "Error: The `set-env` command is disabled. Please upgrade to using Environment Files."
|
|
33
|
+
- "Error: The `add-path` command is disabled. Please upgrade to using Environment Files."
|
|
34
|
+
root_cause: |
|
|
35
|
+
GitHub Actions introduced a new Environment Files mechanism in 2020 to replace the
|
|
36
|
+
older `echo "::command::"` workflow commands. The old commands were deprecated in
|
|
37
|
+
October 2022 (security advisory: injection attacks could hijack `::set-output::` via
|
|
38
|
+
untrusted log output) and disabled shortly after.
|
|
39
|
+
|
|
40
|
+
**Deprecated commands and their replacements:**
|
|
41
|
+
| Old command | Replacement |
|
|
42
|
+
|-------------------------------------|------------------------------------------------|
|
|
43
|
+
| `echo "::set-output name=K::V"` | `echo "K=V" >> $GITHUB_OUTPUT` |
|
|
44
|
+
| `echo "::save-state name=K::V"` | `echo "K=V" >> $GITHUB_STATE` |
|
|
45
|
+
| `echo "::set-env name=K::V"` | `echo "K=V" >> $GITHUB_ENV` |
|
|
46
|
+
| `echo "::add-path::PATH"` | `echo "PATH" >> $GITHUB_PATH` |
|
|
47
|
+
|
|
48
|
+
These commands still appear in:
|
|
49
|
+
- Third-party GitHub Actions that haven't been updated to a newer major version
|
|
50
|
+
- Scripts copied from old Stack Overflow answers or blog posts
|
|
51
|
+
- Custom scripts that use the `::set-output::` form directly
|
|
52
|
+
- Older composite actions where steps use `echo "::set-output name=result::$value"`
|
|
53
|
+
|
|
54
|
+
When the runner encounters these commands, it emits a deprecation warning. If the
|
|
55
|
+
commands are later hard-disabled for a repository (or for a specific runner version),
|
|
56
|
+
the output variable is simply never set — creating a silent downstream failure.
|
|
57
|
+
fix: |
|
|
58
|
+
Replace all deprecated `::command::` syntax with the Environment Files equivalents
|
|
59
|
+
in your workflow YAML, composite action scripts, and any shell scripts that produce
|
|
60
|
+
outputs or modify the environment.
|
|
61
|
+
|
|
62
|
+
**Key rules:**
|
|
63
|
+
- `GITHUB_OUTPUT`, `GITHUB_ENV`, `GITHUB_STATE`, `GITHUB_PATH` are all file paths
|
|
64
|
+
that are set as environment variables by the runner.
|
|
65
|
+
- Append to these files — never overwrite them (`>>` not `>`).
|
|
66
|
+
- For multiline values, use the heredoc delimiter syntax (see fix_code below).
|
|
67
|
+
- On Windows (PowerShell), use `"K=V" | Out-File -FilePath $env:GITHUB_OUTPUT -Append`
|
|
68
|
+
or simply `echo "K=V" >> $env:GITHUB_OUTPUT` (cmd-style append works in pwsh too).
|
|
69
|
+
fix_code:
|
|
70
|
+
- language: yaml
|
|
71
|
+
label: "Replace ::set-output:: with GITHUB_OUTPUT (bash)"
|
|
72
|
+
code: |
|
|
73
|
+
- name: Set output (modern)
|
|
74
|
+
id: my-step
|
|
75
|
+
run: |
|
|
76
|
+
# Old (deprecated):
|
|
77
|
+
# echo "::set-output name=version::1.2.3"
|
|
78
|
+
|
|
79
|
+
# New (use environment file):
|
|
80
|
+
echo "version=1.2.3" >> $GITHUB_OUTPUT
|
|
81
|
+
|
|
82
|
+
- name: Use output downstream
|
|
83
|
+
run: echo "Version is ${{ steps.my-step.outputs.version }}"
|
|
84
|
+
- language: yaml
|
|
85
|
+
label: "Replace ::set-output:: with GITHUB_OUTPUT (PowerShell)"
|
|
86
|
+
code: |
|
|
87
|
+
- name: Set output (PowerShell modern)
|
|
88
|
+
id: my-step
|
|
89
|
+
shell: pwsh
|
|
90
|
+
run: |
|
|
91
|
+
# Old (deprecated):
|
|
92
|
+
# Write-Output "::set-output name=version::1.2.3"
|
|
93
|
+
|
|
94
|
+
# New:
|
|
95
|
+
"version=1.2.3" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append
|
|
96
|
+
- language: yaml
|
|
97
|
+
label: "Replace ::save-state:: with GITHUB_STATE"
|
|
98
|
+
code: |
|
|
99
|
+
- name: Save state in pre step
|
|
100
|
+
run: echo "cleanup_token=${{ secrets.TOKEN }}" >> $GITHUB_STATE
|
|
101
|
+
|
|
102
|
+
- name: Read state in post step
|
|
103
|
+
run: echo "Token was $STATE_CLEANUP_TOKEN"
|
|
104
|
+
# State values are exposed as STATE_<NAME> environment variables
|
|
105
|
+
- language: yaml
|
|
106
|
+
label: "Multiline output value with heredoc delimiter"
|
|
107
|
+
code: |
|
|
108
|
+
- name: Set multiline output
|
|
109
|
+
id: changelog
|
|
110
|
+
run: |
|
|
111
|
+
EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
|
|
112
|
+
echo "notes<<$EOF" >> $GITHUB_OUTPUT
|
|
113
|
+
echo "Line 1 of release notes" >> $GITHUB_OUTPUT
|
|
114
|
+
echo "Line 2 of release notes" >> $GITHUB_OUTPUT
|
|
115
|
+
echo "$EOF" >> $GITHUB_OUTPUT
|
|
116
|
+
prevention:
|
|
117
|
+
- "Audit all workflows and composite actions for `::set-output::`, `::save-state::`, `::set-env::`, and `::add-path::` patterns before they become hard failures."
|
|
118
|
+
- "Add a CI lint step using `grep -r '::set-output' .github/` to catch regressions."
|
|
119
|
+
- "When using third-party Actions, pin to a version tag that uses the modern file-based outputs — check the action's CHANGELOG for 'GITHUB_OUTPUT migration'."
|
|
120
|
+
- "For cross-platform workflows, test the `$GITHUB_OUTPUT` / `$env:GITHUB_OUTPUT` equivalence — both work on ubuntu/macos/windows runners."
|
|
121
|
+
- "Enable the 'Deprecation warnings as errors' setting in your organization's Actions policy to catch deprecated commands in CI before they silently fail."
|
|
122
|
+
docs:
|
|
123
|
+
- url: "https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-output-parameter"
|
|
124
|
+
label: "GitHub Actions: Setting an output parameter (Environment Files)"
|
|
125
|
+
- url: "https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/"
|
|
126
|
+
label: "GitHub Changelog: Deprecating save-state and set-output commands (Oct 2022)"
|
|
127
|
+
- url: "https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/"
|
|
128
|
+
label: "GitHub Changelog: Deprecating set-env and add-path commands (Oct 2020)"
|
|
129
|
+
- url: "https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#understanding-the-risk-of-script-injections"
|
|
130
|
+
label: "Security hardening: Understanding the risk of script injections"
|
package/package.json
CHANGED