lintsmith 0.0.16__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. lintsmith-0.0.16/.gitignore +29 -0
  2. lintsmith-0.0.16/LICENSE +21 -0
  3. lintsmith-0.0.16/PKG-INFO +210 -0
  4. lintsmith-0.0.16/README.md +186 -0
  5. lintsmith-0.0.16/docs/benchmarks/README.md +185 -0
  6. lintsmith-0.0.16/pyproject.toml +92 -0
  7. lintsmith-0.0.16/src/lintsmith/__init__.py +8 -0
  8. lintsmith-0.0.16/src/lintsmith/__main__.py +6 -0
  9. lintsmith-0.0.16/src/lintsmith/bench.py +172 -0
  10. lintsmith-0.0.16/src/lintsmith/cli.py +59 -0
  11. lintsmith-0.0.16/src/lintsmith/config.py +81 -0
  12. lintsmith-0.0.16/src/lintsmith/data/filetypes.json +475 -0
  13. lintsmith-0.0.16/src/lintsmith/data/linters.json +1184 -0
  14. lintsmith-0.0.16/src/lintsmith/data/prune-node-modules.js +91 -0
  15. lintsmith-0.0.16/src/lintsmith/drift.py +186 -0
  16. lintsmith-0.0.16/src/lintsmith/generate.py +348 -0
  17. lintsmith-0.0.16/src/lintsmith/mapping_sync.py +163 -0
  18. lintsmith-0.0.16/src/lintsmith/release_notes.py +164 -0
  19. lintsmith-0.0.16/src/lintsmith/report.py +241 -0
  20. lintsmith-0.0.16/src/lintsmith/repository_settings.py +146 -0
  21. lintsmith-0.0.16/src/lintsmith/smoke.py +318 -0
  22. lintsmith-0.0.16/src/lintsmith/upstream.py +109 -0
  23. lintsmith-0.0.16/tests/__init__.py +0 -0
  24. lintsmith-0.0.16/tests/conftest.py +27 -0
  25. lintsmith-0.0.16/tests/fixtures/ghwm-workflow.yml +29 -0
  26. lintsmith-0.0.16/tests/shared.py +63 -0
  27. lintsmith-0.0.16/tests/test_bench.py +296 -0
  28. lintsmith-0.0.16/tests/test_cli.py +73 -0
  29. lintsmith-0.0.16/tests/test_config.py +119 -0
  30. lintsmith-0.0.16/tests/test_drift.py +218 -0
  31. lintsmith-0.0.16/tests/test_generate.py +336 -0
  32. lintsmith-0.0.16/tests/test_mapping_sync.py +162 -0
  33. lintsmith-0.0.16/tests/test_release_notes.py +130 -0
  34. lintsmith-0.0.16/tests/test_report.py +338 -0
  35. lintsmith-0.0.16/tests/test_repository_settings.py +162 -0
  36. lintsmith-0.0.16/tests/test_smoke.py +618 -0
  37. lintsmith-0.0.16/tests/test_upstream.py +383 -0
@@ -0,0 +1,29 @@
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *.egg-info/
5
+ dist/
6
+ .venv/
7
+ .mypy_cache/
8
+ .pytest_cache/
9
+ .ruff_cache/
10
+
11
+ # Build / test artifacts
12
+ .coverage
13
+ pytest-coverage.txt
14
+ coverage.xml
15
+ junit.xml
16
+ htmlcov/
17
+ node_modules/
18
+
19
+ # Benchmark output
20
+ bench-results.jsonl
21
+ bench-*.jsonl
22
+ results.md
23
+ lint-*.log
24
+ exit-*.txt
25
+
26
+ .DS_Store
27
+
28
+ # APM dependencies
29
+ apm_modules/
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Peter Meyer Elphick Frandsen
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,210 @@
1
+ Metadata-Version: 2.5
2
+ Name: lintsmith
3
+ Version: 0.0.16
4
+ Summary: Lean, tailored Super-Linter images: only the linters you enable, plus any extra tools you need.
5
+ Project-URL: Homepage, https://github.com/ghwfxlab/lintsmith
6
+ Project-URL: Repository, https://github.com/ghwfxlab/lintsmith
7
+ Project-URL: Issues, https://github.com/ghwfxlab/lintsmith/issues
8
+ Author: Peter Meyer Elphick Frandsen
9
+ License: MIT
10
+ License-File: LICENSE
11
+ Keywords: ci-cd,docker,github-actions,linting,super-linter
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: License :: OSI Approved :: MIT License
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Programming Language :: Python :: 3.14
20
+ Classifier: Topic :: Software Development :: Build Tools
21
+ Requires-Python: >=3.12
22
+ Requires-Dist: pyyaml>=6.0
23
+ Description-Content-Type: text/markdown
24
+
25
+ # lintsmith
26
+
27
+ Generate lean, pinned [Super-Linter](https://github.com/super-linter/super-linter)
28
+ images that contain only the linters a repository uses, to cut the cold
29
+ pull-and-unpack time in CI.
30
+
31
+ Formerly `superlint-lean`. The old PyPI package and repository URL keep working, but
32
+ new releases are published as `lintsmith`.
33
+
34
+ Status: working, early. On GitHub-hosted runners a lean image pulls 5x to 8x
35
+ faster than upstream `slim`:
36
+
37
+ | Stack | Upstream slim | Lean | Faster |
38
+ | --- | --- | --- | --- |
39
+ | ghwm | 70.7 s | 10.7 s | 6.6x |
40
+ | python | 67.9 s | 11.6 s | 5.9x |
41
+ | frontend-react | 68.7 s | 10.4 s | 6.6x |
42
+ | dotnet (against upstream standard) | 91.0 s | 16.9 s | 5.4x |
43
+
44
+ Cold pull, median of 3, upstream v9.0.0. All profiles, the method and the
45
+ caveats are in [docs/benchmarks](docs/benchmarks/README.md).
46
+
47
+ ## Idea
48
+
49
+ 1. A repository declares the linters it needs, or they are discovered from the
50
+ `VALIDATE_*` flags in its Super-Linter workflow.
51
+ 2. A compiler builds an overlay image: same Python/Alpine base as upstream, plus
52
+ only the needed tool paths copied from the upstream release image.
53
+ 3. A PR check fails when a change adds file types the current image does not
54
+ cover.
55
+ 4. A scheduled job rebuilds when upstream publishes a new release.
56
+ 5. The workflows are distributed through the ghwm registry; see
57
+ [Run the lean image with the ghwm registry workflows](docs/using-with-ghwm.md).
58
+
59
+ ## Install
60
+
61
+ The CLI is a Python package, managed with [uv](https://docs.astral.sh/uv/):
62
+
63
+ ```sh
64
+ uv tool install lintsmith # or: pipx install lintsmith
65
+ uvx lintsmith --help # run without installing
66
+ ```
67
+
68
+ The latest unreleased code is available with
69
+ `uvx --from git+https://github.com/ghwfxlab/lintsmith lintsmith --help`.
70
+
71
+ Commands:
72
+
73
+ | Command | Purpose |
74
+ | --- | --- |
75
+ | `lintsmith generate` | `VALIDATE_*` flags (env files or a workflow) -> Dockerfile, tool self-checks and plan |
76
+ | `lintsmith drift` | PR drift check: fails when changed files are not covered by the enabled linters |
77
+ | `lintsmith sync-mapping` | Compare the linter mapping with an upstream checkout and add draft entries |
78
+ | `lintsmith release-notes` | Release notes with before/after benchmarks of two upstream versions |
79
+ | `lintsmith compare`, `summarize` | Markdown tables from benchmark results: lean against baseline, and every variant on its own |
80
+ | `lintsmith repository-settings` | Print or apply the repository settings in `docs/repository-settings.md` (maintainers) |
81
+ | `lintsmith bench` | Time cold or warm pulls and container start of an image (the benchmark workflow) |
82
+ | `lintsmith select-upstream` | Pick the upstream release to sync to, honouring a cooldown |
83
+ | `lintsmith smoke` | Lint upstream's good and bad fixtures with a built image, one linter at a time |
84
+
85
+ ## Layout
86
+
87
+ | Path | Purpose |
88
+ | --- | --- |
89
+ | `src/lintsmith/` | The package: `cli.py`, `generate.py`, `drift.py`, `config.py` (reads workflows and env files with upstream's `VALIDATE_*` semantics), `mapping_sync.py`, `release_notes.py` |
90
+ | `src/lintsmith/data/` | `linters.json` (flag -> upstream paths, venvs, npm and apk packages), `filetypes.json` (file type rules transcribed from upstream), `prune-node-modules.js` |
91
+ | `tests/` | pytest suite (Arrange/Act/Assert), using the ghwm workflow as a fixture |
92
+ | `base/opinionated.env` | The opinionated base: the checks almost every repository wants (EditorConfig, env, actionlint, zizmor, gitleaks, JSON, Markdown, natural language, YAML, shell, hadolint) |
93
+ | `toolchains/` | Example toolchains on top of the base: python, Java, dotnet, rust, c-cpp, frontend-{vanilla,react,vue,angular,svelte}, base-only |
94
+ | `examples/` | `ghwm.env` (a real repository's exact flags, without the base) and `with-extras.env` |
95
+ | `.github/workflows/` | `tests`, `profiles` (benchmarks), `upstream-sync`, `release`, `self-image` (this repository's own lean image), and the reusable `drift-blocker` |
96
+
97
+ ## Development
98
+
99
+ ```sh
100
+ make install # uv sync
101
+ make setup-precommit # pre-commit hooks (super-linter and the tests)
102
+ make test # pytest with coverage
103
+ make lint # ruff
104
+ make type-check # mypy
105
+ make lang # textlint for the docs
106
+ make super-linter # super-linter via Docker
107
+ make build # sdist and wheel
108
+ ```
109
+
110
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for the workflow, [AGENTS.md](AGENTS.md)
111
+ for the guide used by coding agents, [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
112
+ and [docs/reference/cli.md](docs/reference/cli.md) for the internals and the
113
+ command reference, [docs/verifying-images.md](docs/verifying-images.md) to
114
+ check a pulled image, and [docs/repository-settings.md](docs/repository-settings.md)
115
+ for the repository settings. Report vulnerabilities as described in
116
+ [SECURITY.md](SECURITY.md).
117
+
118
+ ## Configuration
119
+
120
+ Nothing is fixed. The base and the toolchains are plain env files with
121
+ `VALIDATE_*=true` lines, and the generator takes any combination of them:
122
+
123
+ - leave the base out, or write your own
124
+ - stack several toolchains (`--env toolchains/python.env --env toolchains/rust.env`)
125
+ - drop single linters from a combination with `--exclude NATURAL_LANGUAGE`
126
+ - add tools Super-Linter does not ship: Alpine packages (`--extra-apk`), npm
127
+ packages (`--extra-npm`) and pip packages (`--extra-pip`), plus `--extra-check`
128
+ commands that must succeed in the built image. They can also be written in an
129
+ env file (`EXTRA_APK=`, `EXTRA_NPM=`, `EXTRA_PIP=`, `EXTRA_CHECK=`), see
130
+ `examples/with-extras.env`. Extras are installed at build time from the
131
+ package registries, in their own layers (`/opt/extras`), and npm installs run
132
+ with `--ignore-scripts`
133
+
134
+ The toolchains in this repository, and the numbers in
135
+ [docs/benchmarks](docs/benchmarks/README.md), are examples of combinations, not
136
+ the supported set.
137
+
138
+ The reason for an opinionated base is that a repository using one linter has no
139
+ use for Super-Linter. The value is in running a broad set of checks, so the base
140
+ is broad and each toolchain adds a stack on top.
141
+
142
+ ## Block pull requests that outgrow the image
143
+
144
+ The image only contains the linters you enabled. If a pull request adds a file
145
+ type whose linters are not enabled (a `.rs` file when no Rust linter is on),
146
+ Super-Linter would silently skip it. The drift blocker fails that pull request
147
+ and says which linter to enable:
148
+
149
+ ```yaml
150
+ jobs:
151
+ drift:
152
+ permissions:
153
+ contents: read
154
+ uses: ghwfxlab/lintsmith/.github/workflows/drift-blocker.yml@main
155
+ with:
156
+ workflow: .github/workflows/super-linter.yaml # the workflow with the super-linter step
157
+ allow_ext: "txt lock" # file types you deliberately do not lint
158
+ ```
159
+
160
+ The configuration is read from the `env:` of the Super-Linter step, with the
161
+ upstream rules: if any `VALIDATE_X` is `true`, only those linters run; if only
162
+ `false` values are set, everything else runs. `FILTER_REGEX_INCLUDE` and
163
+ `FILTER_REGEX_EXCLUDE` are respected. The same discovery works in the compiler:
164
+ `lintsmith generate --workflow .github/workflows/super-linter.yaml ...`. Content-based
165
+ detections (OpenAPI, CloudFormation, Kubernetes, ARM, AWS States, Ansible) are
166
+ not modelled, so a YAML file of those kinds is only checked as plain YAML.
167
+
168
+ ## Try it locally
169
+
170
+ ```sh
171
+ lintsmith generate --env base/opinionated.env --env toolchains/python.env \
172
+ --upstream-tag v9.0.0 --upstream-dockerfile <upstream Dockerfile at that tag> --out build/python
173
+ docker buildx build build/python -t lean-python
174
+ docker run --rm -v "$PWD/build/python/checks.sh:/checks.sh:ro" --entrypoint bash lean-python /checks.sh
175
+ ```
176
+
177
+ Profiles marked as needing the upstream `standard` image (dotnet, rust) are
178
+ compared against `standard`, not `slim`. Upstream has no Svelte-specific linter,
179
+ so the Svelte profile covers the JS, TS, CSS and HTML around it.
180
+
181
+ ## Limitations
182
+
183
+ - Only linters in the mapping (`src/lintsmith/data/linters.json`, 95 flags
184
+ for upstream v9.0.0) can be selected. An unknown flag stops `generate` with a
185
+ message, and new upstream linters start as drafts that need a review (see
186
+ [CONTRIBUTING.md](CONTRIBUTING.md#extending-the-mapping)).
187
+ - The generator reads upstream's published image and `Dockerfile`. A release
188
+ that moves tools around can break the mapping; `upstream-sync` reports it
189
+ before the pin moves.
190
+ - The image is rebuilt after a change reaches `main`. A pull request that
191
+ changes the enabled linters is still linted with the previous image, and the
192
+ drift check only catches file types that no enabled linter covers.
193
+ - The saving comes from leaving linters out. With every mapped linter enabled the
194
+ image is only about 11% smaller than upstream standard.
195
+ - A runner that already holds the upstream image gains nothing: the pull is a
196
+ no-op either way.
197
+ - Pulling a private image needs credentials: `packages: read` for GHCR in the
198
+ same organization, otherwise a registry login.
199
+ - Only `linux/amd64` is tested. Images are not signed yet.
200
+ - zstd layers are published and pull 0% to 15% faster than gzip; that is small next to leaving linters out.
201
+
202
+ ## Licence
203
+
204
+ This repository is MIT licensed (see [LICENSE](LICENSE)). It contains no
205
+ third-party linters. A generated image is built by copying files out of the
206
+ published Super-Linter image, so it contains Super-Linter (MIT) and the linters
207
+ you select, each under its own licence. If you publish such an image you
208
+ redistribute those tools and must follow their licences; check them for the
209
+ linters you pick. The smoke test reads Super-Linter's test fixtures at build
210
+ time and does not copy them into this repository or into an image.
@@ -0,0 +1,186 @@
1
+ # lintsmith
2
+
3
+ Generate lean, pinned [Super-Linter](https://github.com/super-linter/super-linter)
4
+ images that contain only the linters a repository uses, to cut the cold
5
+ pull-and-unpack time in CI.
6
+
7
+ Formerly `superlint-lean`. The old PyPI package and repository URL keep working, but
8
+ new releases are published as `lintsmith`.
9
+
10
+ Status: working, early. On GitHub-hosted runners a lean image pulls 5x to 8x
11
+ faster than upstream `slim`:
12
+
13
+ | Stack | Upstream slim | Lean | Faster |
14
+ | --- | --- | --- | --- |
15
+ | ghwm | 70.7 s | 10.7 s | 6.6x |
16
+ | python | 67.9 s | 11.6 s | 5.9x |
17
+ | frontend-react | 68.7 s | 10.4 s | 6.6x |
18
+ | dotnet (against upstream standard) | 91.0 s | 16.9 s | 5.4x |
19
+
20
+ Cold pull, median of 3, upstream v9.0.0. All profiles, the method and the
21
+ caveats are in [docs/benchmarks](docs/benchmarks/README.md).
22
+
23
+ ## Idea
24
+
25
+ 1. A repository declares the linters it needs, or they are discovered from the
26
+ `VALIDATE_*` flags in its Super-Linter workflow.
27
+ 2. A compiler builds an overlay image: same Python/Alpine base as upstream, plus
28
+ only the needed tool paths copied from the upstream release image.
29
+ 3. A PR check fails when a change adds file types the current image does not
30
+ cover.
31
+ 4. A scheduled job rebuilds when upstream publishes a new release.
32
+ 5. The workflows are distributed through the ghwm registry; see
33
+ [Run the lean image with the ghwm registry workflows](docs/using-with-ghwm.md).
34
+
35
+ ## Install
36
+
37
+ The CLI is a Python package, managed with [uv](https://docs.astral.sh/uv/):
38
+
39
+ ```sh
40
+ uv tool install lintsmith # or: pipx install lintsmith
41
+ uvx lintsmith --help # run without installing
42
+ ```
43
+
44
+ The latest unreleased code is available with
45
+ `uvx --from git+https://github.com/ghwfxlab/lintsmith lintsmith --help`.
46
+
47
+ Commands:
48
+
49
+ | Command | Purpose |
50
+ | --- | --- |
51
+ | `lintsmith generate` | `VALIDATE_*` flags (env files or a workflow) -> Dockerfile, tool self-checks and plan |
52
+ | `lintsmith drift` | PR drift check: fails when changed files are not covered by the enabled linters |
53
+ | `lintsmith sync-mapping` | Compare the linter mapping with an upstream checkout and add draft entries |
54
+ | `lintsmith release-notes` | Release notes with before/after benchmarks of two upstream versions |
55
+ | `lintsmith compare`, `summarize` | Markdown tables from benchmark results: lean against baseline, and every variant on its own |
56
+ | `lintsmith repository-settings` | Print or apply the repository settings in `docs/repository-settings.md` (maintainers) |
57
+ | `lintsmith bench` | Time cold or warm pulls and container start of an image (the benchmark workflow) |
58
+ | `lintsmith select-upstream` | Pick the upstream release to sync to, honouring a cooldown |
59
+ | `lintsmith smoke` | Lint upstream's good and bad fixtures with a built image, one linter at a time |
60
+
61
+ ## Layout
62
+
63
+ | Path | Purpose |
64
+ | --- | --- |
65
+ | `src/lintsmith/` | The package: `cli.py`, `generate.py`, `drift.py`, `config.py` (reads workflows and env files with upstream's `VALIDATE_*` semantics), `mapping_sync.py`, `release_notes.py` |
66
+ | `src/lintsmith/data/` | `linters.json` (flag -> upstream paths, venvs, npm and apk packages), `filetypes.json` (file type rules transcribed from upstream), `prune-node-modules.js` |
67
+ | `tests/` | pytest suite (Arrange/Act/Assert), using the ghwm workflow as a fixture |
68
+ | `base/opinionated.env` | The opinionated base: the checks almost every repository wants (EditorConfig, env, actionlint, zizmor, gitleaks, JSON, Markdown, natural language, YAML, shell, hadolint) |
69
+ | `toolchains/` | Example toolchains on top of the base: python, Java, dotnet, rust, c-cpp, frontend-{vanilla,react,vue,angular,svelte}, base-only |
70
+ | `examples/` | `ghwm.env` (a real repository's exact flags, without the base) and `with-extras.env` |
71
+ | `.github/workflows/` | `tests`, `profiles` (benchmarks), `upstream-sync`, `release`, `self-image` (this repository's own lean image), and the reusable `drift-blocker` |
72
+
73
+ ## Development
74
+
75
+ ```sh
76
+ make install # uv sync
77
+ make setup-precommit # pre-commit hooks (super-linter and the tests)
78
+ make test # pytest with coverage
79
+ make lint # ruff
80
+ make type-check # mypy
81
+ make lang # textlint for the docs
82
+ make super-linter # super-linter via Docker
83
+ make build # sdist and wheel
84
+ ```
85
+
86
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for the workflow, [AGENTS.md](AGENTS.md)
87
+ for the guide used by coding agents, [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
88
+ and [docs/reference/cli.md](docs/reference/cli.md) for the internals and the
89
+ command reference, [docs/verifying-images.md](docs/verifying-images.md) to
90
+ check a pulled image, and [docs/repository-settings.md](docs/repository-settings.md)
91
+ for the repository settings. Report vulnerabilities as described in
92
+ [SECURITY.md](SECURITY.md).
93
+
94
+ ## Configuration
95
+
96
+ Nothing is fixed. The base and the toolchains are plain env files with
97
+ `VALIDATE_*=true` lines, and the generator takes any combination of them:
98
+
99
+ - leave the base out, or write your own
100
+ - stack several toolchains (`--env toolchains/python.env --env toolchains/rust.env`)
101
+ - drop single linters from a combination with `--exclude NATURAL_LANGUAGE`
102
+ - add tools Super-Linter does not ship: Alpine packages (`--extra-apk`), npm
103
+ packages (`--extra-npm`) and pip packages (`--extra-pip`), plus `--extra-check`
104
+ commands that must succeed in the built image. They can also be written in an
105
+ env file (`EXTRA_APK=`, `EXTRA_NPM=`, `EXTRA_PIP=`, `EXTRA_CHECK=`), see
106
+ `examples/with-extras.env`. Extras are installed at build time from the
107
+ package registries, in their own layers (`/opt/extras`), and npm installs run
108
+ with `--ignore-scripts`
109
+
110
+ The toolchains in this repository, and the numbers in
111
+ [docs/benchmarks](docs/benchmarks/README.md), are examples of combinations, not
112
+ the supported set.
113
+
114
+ The reason for an opinionated base is that a repository using one linter has no
115
+ use for Super-Linter. The value is in running a broad set of checks, so the base
116
+ is broad and each toolchain adds a stack on top.
117
+
118
+ ## Block pull requests that outgrow the image
119
+
120
+ The image only contains the linters you enabled. If a pull request adds a file
121
+ type whose linters are not enabled (a `.rs` file when no Rust linter is on),
122
+ Super-Linter would silently skip it. The drift blocker fails that pull request
123
+ and says which linter to enable:
124
+
125
+ ```yaml
126
+ jobs:
127
+ drift:
128
+ permissions:
129
+ contents: read
130
+ uses: ghwfxlab/lintsmith/.github/workflows/drift-blocker.yml@main
131
+ with:
132
+ workflow: .github/workflows/super-linter.yaml # the workflow with the super-linter step
133
+ allow_ext: "txt lock" # file types you deliberately do not lint
134
+ ```
135
+
136
+ The configuration is read from the `env:` of the Super-Linter step, with the
137
+ upstream rules: if any `VALIDATE_X` is `true`, only those linters run; if only
138
+ `false` values are set, everything else runs. `FILTER_REGEX_INCLUDE` and
139
+ `FILTER_REGEX_EXCLUDE` are respected. The same discovery works in the compiler:
140
+ `lintsmith generate --workflow .github/workflows/super-linter.yaml ...`. Content-based
141
+ detections (OpenAPI, CloudFormation, Kubernetes, ARM, AWS States, Ansible) are
142
+ not modelled, so a YAML file of those kinds is only checked as plain YAML.
143
+
144
+ ## Try it locally
145
+
146
+ ```sh
147
+ lintsmith generate --env base/opinionated.env --env toolchains/python.env \
148
+ --upstream-tag v9.0.0 --upstream-dockerfile <upstream Dockerfile at that tag> --out build/python
149
+ docker buildx build build/python -t lean-python
150
+ docker run --rm -v "$PWD/build/python/checks.sh:/checks.sh:ro" --entrypoint bash lean-python /checks.sh
151
+ ```
152
+
153
+ Profiles marked as needing the upstream `standard` image (dotnet, rust) are
154
+ compared against `standard`, not `slim`. Upstream has no Svelte-specific linter,
155
+ so the Svelte profile covers the JS, TS, CSS and HTML around it.
156
+
157
+ ## Limitations
158
+
159
+ - Only linters in the mapping (`src/lintsmith/data/linters.json`, 95 flags
160
+ for upstream v9.0.0) can be selected. An unknown flag stops `generate` with a
161
+ message, and new upstream linters start as drafts that need a review (see
162
+ [CONTRIBUTING.md](CONTRIBUTING.md#extending-the-mapping)).
163
+ - The generator reads upstream's published image and `Dockerfile`. A release
164
+ that moves tools around can break the mapping; `upstream-sync` reports it
165
+ before the pin moves.
166
+ - The image is rebuilt after a change reaches `main`. A pull request that
167
+ changes the enabled linters is still linted with the previous image, and the
168
+ drift check only catches file types that no enabled linter covers.
169
+ - The saving comes from leaving linters out. With every mapped linter enabled the
170
+ image is only about 11% smaller than upstream standard.
171
+ - A runner that already holds the upstream image gains nothing: the pull is a
172
+ no-op either way.
173
+ - Pulling a private image needs credentials: `packages: read` for GHCR in the
174
+ same organization, otherwise a registry login.
175
+ - Only `linux/amd64` is tested. Images are not signed yet.
176
+ - zstd layers are published and pull 0% to 15% faster than gzip; that is small next to leaving linters out.
177
+
178
+ ## Licence
179
+
180
+ This repository is MIT licensed (see [LICENSE](LICENSE)). It contains no
181
+ third-party linters. A generated image is built by copying files out of the
182
+ published Super-Linter image, so it contains Super-Linter (MIT) and the linters
183
+ you select, each under its own licence. If you publish such an image you
184
+ redistribute those tools and must follow their licences; check them for the
185
+ linters you pick. The smoke test reads Super-Linter's test fixtures at build
186
+ time and does not copy them into this repository or into an image.
@@ -0,0 +1,185 @@
1
+ # Benchmarks
2
+
3
+ Question: how much faster is a lean image than upstream `slim` on a cold runner,
4
+ and is that enough to justify this tool?
5
+
6
+ ## Go / no-go
7
+
8
+ Go if the lean image for the first target (ghwm) pulls in about 20 s or less, or
9
+ is at least 3x faster than slim on a **GitHub-hosted** runner. Otherwise the
10
+ honest outcome is a "slim plus caching" write-up.
11
+
12
+ **Verdict: go.** On a GitHub-hosted runner the ghwm image pulls in 10.7 s against
13
+ 70.7 s for slim (6.6x), and every example profile is at least 5x faster.
14
+
15
+ ## When benchmarks run
16
+
17
+ Nothing runs on upstream's behalf unless upstream has a newer release than the
18
+ one pinned in `upstream-version.txt`:
19
+
20
+ 1. The weekly `upstream-sync` run makes one API call. If the pin is current it
21
+ stops there, with no checkout of upstream, no build and no benchmark.
22
+ 2. If upstream is ahead it adds missing linters to the mapping, checks them with
23
+ an all-linters image and opens a pull request that bumps the pin.
24
+ 3. Merging that pull request runs the `release` workflow: it benchmarks the old
25
+ version (only if no results are stored for it) and the new one with the same
26
+ profiles, and publishes a GitHub release `upstream-<tag>` whose notes show
27
+ the numbers before and after (`scripts/release_notes.py`).
28
+
29
+ Results are stored on the `bench-results` branch under `results/by-tag/<tag>/`.
30
+ The profiles used for releases come from the repository variable
31
+ `RELEASE_PROFILES` (default `ghwm,python,frontend-react,dotnet`). Starting the
32
+ `Toolchain benchmark` workflow by hand is still possible, as an explicit
33
+ decision, for example to add a profile to the stored results.
34
+
35
+ ## Pipeline under test
36
+
37
+ config -> create image -> push to GHCR -> pull on a runner. The "create
38
+ image" step is `lintsmith generate` followed by `docker build`. Each
39
+ scenario compares upstream slim ("without") with the generated lean image
40
+ ("with").
41
+
42
+ ## Scenarios
43
+
44
+ 1. **Cold, GitHub-hosted runner.** Headline number: what most users pay.
45
+ 2. **Cold, self-hosted runner.** Image removed before each pull.
46
+ 3. **Warm, self-hosted runner with the full image preloaded.** Tests whether a
47
+ smaller image helps when nothing has to be downloaded. Expect little change
48
+ in pull time; start time and disk use are the other things to look at.
49
+ 4. **Upgrade pull** (not yet automated): previous upstream tag present, pull the
50
+ next. Needs a second build.
51
+
52
+ ## Method
53
+
54
+ - Workflow: `.github/workflows/profiles.yaml` (manual dispatch or called by the release workflow, one job per stack profile; runs on `ubuntu-latest` unless the `RUNNER_LABELS` repository variable or the `runner_labels` input says otherwise).
55
+ - Variants:
56
+ - `slim`: `ghcr.io/super-linter/super-linter:slim-<tag>`
57
+ - `lean-gzip`: generated Dockerfile, default gzip layers
58
+ - `lean-zstd`: same image, zstd-compressed layers
59
+ - Runners: `ubuntu-latest` (headline numbers) and optionally a self-hosted runner (repository variable `RUNNER_LABELS`), cold and warm (`BENCH_MODE`).
60
+ - Each run removes the image locally, then times `docker pull`. With `crane`
61
+ available it also records a download-only pull and a `docker load`, giving a
62
+ rough download vs unpack split.
63
+ - Median of N runs (default 5). Sizes are compressed layer size and
64
+ uncompressed image size.
65
+
66
+ Self-hosted numbers are secondary. Runners that keep layers between jobs, or
67
+ that sit close to the registry, will make every variant look faster than most
68
+ users will see.
69
+
70
+ ## Caveats
71
+
72
+ - `/action/linterVersions.txt` lists all upstream tools, not only the kept ones.
73
+ - The self-hosted zstd numbers below come from a run before the cold-pull
74
+ cleanup of `lintsmith bench`. Within that run the zstd pull followed the
75
+ gzip pull of the same image, so they may be too low.
76
+
77
+ ## What the benchmark covers
78
+
79
+ The tool is fully configurable, so the benchmark is only a set of example
80
+ combinations: the opinionated base plus one example toolchain each
81
+ (`toolchains/`), and the ghwm repository's exact flags (`examples/ghwm.env`).
82
+ There are no single-linter profiles on purpose: a repository that needs one
83
+ linter has no reason to use Super-Linter. `base-only` measures the base on its
84
+ own, as the floor for the toolchains.
85
+
86
+ ## Results
87
+
88
+ ### GitHub-hosted runner (headline)
89
+
90
+ Run 38303678148, upstream v9.0.0, `ubuntu-latest`, median of 3 cold pulls. Before
91
+ every cold pull the script removes the image, every other unused image and the
92
+ build cache, so no variant can reuse another's layers. The upstream baseline is
93
+ `slim-v9.0.0`, except rust and dotnet, which need `standard-v9.0.0`. Every
94
+ profile except `ghwm` is the shared base (`base/opinionated.env`) plus its stack
95
+ flags. Raw data is on the `bench-results` branch under `results/by-tag/v9.0.0/`.
96
+
97
+ | Profile | Upstream pull | Lean gzip | Lean zstd | Faster (gzip) | Faster (zstd) | Compressed: upstream to lean |
98
+ | --- | --- | --- | --- | --- | --- | --- |
99
+ | ghwm | 70.7 s | 10.7 s | 9.8 s | 6.6x | 7.2x | 1310 to 170 MB (-87%) |
100
+ | python | 67.9 s | 11.6 s | 11.5 s | 5.9x | 5.9x | 1310 to 210 MB (-84%) |
101
+ | Java | 68.3 s | 10.7 s | 9.7 s | 6.4x | 7.0x | 1310 to 233 MB (-82%) |
102
+ | c-cpp | 77.6 s | 11.0 s | 10.5 s | 7.1x | 7.4x | 1310 to 156 MB (-88%) |
103
+ | frontend-vanilla | 68.3 s | 9.8 s | 8.6 s | 7.0x | 8.0x | 1310 to 152 MB (-88%) |
104
+ | frontend-react | 68.7 s | 10.4 s | 9.4 s | 6.6x | 7.3x | 1310 to 156 MB (-88%) |
105
+ | frontend-vue | 68.3 s | 9.5 s | 8.7 s | 7.2x | 7.8x | 1310 to 152 MB (-88%) |
106
+ | frontend-angular | 68.7 s | 9.5 s | 8.8 s | 7.2x | 7.8x | 1310 to 152 MB (-88%) |
107
+ | frontend-svelte | 68.5 s | 9.6 s | 8.7 s | 7.2x | 7.9x | 1310 to 152 MB (-88%) |
108
+ | base-only | 59.7 s | 8.4 s | 8.2 s | 7.1x | 7.3x | 1310 to 150 MB (-89%) |
109
+ | rust | 81.2 s | 14.1 s | 14.3 s | 5.8x | 5.7x | 1798 to 376 MB (-79%) |
110
+ | dotnet | 91.0 s | 16.9 s | 15.4 s | 5.4x | 5.9x | 1798 to 378 MB (-79%) |
111
+
112
+ What to take from it:
113
+
114
+ - A lean image pulls 5x to 8x faster, because 79% to 89% fewer compressed bytes
115
+ have to be downloaded and unpacked. The saving is about the same for every
116
+ stack that only drops other stacks' tools.
117
+ - zstd layers pull 0% to 15% faster than gzip layers (up to 1.5 s), a small
118
+ extra gain on top of the 5x to 8x. Docker on the runner decompresses both
119
+ without trouble.
120
+ - Baseline drift (first cold readings against the end-of-run bracket) stayed at
121
+ 9% or below on every profile.
122
+ - Warm runner (image already present): pull is a no-op (0.2 s to 0.7 s) for both
123
+ images. A smaller image does not help a runner that already holds the full one.
124
+ - The "unpacked" sizes on this runner (3995 MB for slim) are lower than on the
125
+ self-hosted runner (5789 MB). The reason is not established, so only
126
+ compressed sizes are compared above (tracked in #30).
127
+
128
+ An earlier hosted run (38284154555) reported zstd cold pulls of 0.2 s to 0.7 s.
129
+ That was a measurement error: the zstd pull reused the layers of the gzip image
130
+ still on the runner. It is fixed in `lintsmith bench` and those numbers
131
+ were replaced by this run.
132
+
133
+ ### Self-hosted runner (secondary)
134
+
135
+ Self-hosted numbers are secondary: a runner that keeps layers between jobs, or
136
+ sits close to the registry, makes every variant look faster than most users will
137
+ see.
138
+
139
+ Run 38144305432, upstream v9.0.0, a self-hosted runner (Proxmox VM,
140
+ 2 CPU, 4 GB RAM, 40 GB disk), median of 3 cold pulls. Baseline is upstream
141
+ `slim-v9.0.0`. Every profile except `ghwm` is the shared base
142
+ (`base/opinionated.env`) plus its stack flags. Raw data is on the `bench-results`
143
+ branch under `results/2026-10-11-run-38144305432/`.
144
+
145
+ | Profile | Baseline pull | Lean gzip | Lean zstd | Unpacked: base to lean | Compressed: base to lean | Build gzip+zstd (incl. push) |
146
+ | --- | --- | --- | --- | --- | --- | --- |
147
+ | ghwm | 75.6 s | 10.2 s | 4.6 s | 5789 to 771 MB (-87%) | 1310 to 170 MB (-87%) | 129 s |
148
+ | python | 73.4 s | 12.4 s | 5.7 s | 5789 to 965 MB (-83%) | 1310 to 210 MB (-84%) | 153 s |
149
+ | Java | 71.7 s | 13.2 s | 5.3 s | 5789 to 1004 MB (-83%) | 1310 to 233 MB (-82%) | 182 s |
150
+ | c-cpp | 69.2 s | 9.8 s | 3.6 s | 5789 to 726 MB (-87%) | 1310 to 156 MB (-88%) | 124 s |
151
+ | frontend-vanilla | 71.1 s | 9.4 s | 3.4 s | 5789 to 722 MB (-88%) | 1310 to 152 MB (-88%) | 128 s |
152
+ | frontend-react | 71.1 s | 9.7 s | 3.6 s | 5789 to 755 MB (-87%) | 1310 to 156 MB (-88%) | 133 s |
153
+ | frontend-vue | 71.3 s | 9.3 s | 3.6 s | 5789 to 722 MB (-88%) | 1310 to 152 MB (-88%) | 130 s |
154
+ | frontend-angular | 72.7 s | 8.9 s | 3.7 s | 5789 to 722 MB (-88%) | 1310 to 152 MB (-88%) | 135 s |
155
+ | frontend-svelte | 71.9 s | 9.2 s | 4.1 s | 5789 to 722 MB (-88%) | 1310 to 152 MB (-88%) | 139 s |
156
+ | rust | 96.0 s | 16.9 s | 6.6 s | 7885 to 1540 MB (-80%) | 1798 to 376 MB (-79%) | 177 s |
157
+ | dotnet | 98.6 s | 15.1 s | 7.8 s | 7885 to 1555 MB (-80%) | 1798 to 378 MB (-79%) | 200 s |
158
+
159
+ Warm runner (image already present): pull is a no-op, about 1.5 s for the
160
+ baseline and 0.9 s for lean; container start 0.4 s vs 0.3 s. A smaller image
161
+ does not make a runner that already holds the full image meaningfully faster.
162
+
163
+ Baseline drift (first vs last cold pull) stayed within 5% on every profile.
164
+ Build time is gzip and zstd together; later runs record them separately.
165
+
166
+ dotnet and rust (run 38152523859, same runner) are compared against the upstream
167
+ `standard` image (`v9.0.0`), which is larger than slim: 7885 MB unpacked, 1798 MB
168
+ compressed. Their baseline drift was 2% and 13%. In that run the build time was
169
+ recorded per variant (gzip 167 s / zstd 32 s for dotnet, 141 s / 36 s for rust).
170
+ The zstd build runs second and reuses the build cache, so its time is not a
171
+ fair cold-build number.
172
+
173
+ ### Every mapped linter at once
174
+
175
+ Run 38165304659 built an image with all 95 mapped flags (`all-linters`, based on
176
+ upstream `standard`) to verify the mapping: 80 tool checks ran and all tools
177
+ worked (the one reported failure was a false positive in the checker, fixed).
178
+ The lean image is 7037 MB unpacked and 1679 MB compressed against 7885 MB and
179
+ 1798 MB for upstream standard, with a cold pull of 72.4 s (gzip) and 26.5 s
180
+ (zstd). With every linter enabled the size saving is only about 11%, as expected:
181
+ the saving comes from leaving linters out, so the tool pays off for repositories
182
+ that use a subset. The baseline pull of this run did not complete.
183
+
184
+ Open: the upgrade-pull scenario (previous tag present, new tag pulled) and why
185
+ unpacked sizes differ between the two runners.