c2cciutils 1.8.0.dev64__tar.gz → 1.8.0.dev68__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Potentially problematic release.
This version of c2cciutils might be problematic. Click here for more details.
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/PKG-INFO +1 -219
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/README.md +0 -203
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/__init__.py +1 -194
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/applications-versions.yaml +0 -1
- c2cciutils-1.8.0.dev68/c2cciutils/configuration.py +206 -0
- c2cciutils-1.8.0.dev68/c2cciutils/schema.json +102 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/pyproject.toml +1 -28
- c2cciutils-1.8.0.dev64/c2cciutils/configuration.py +0 -803
- c2cciutils-1.8.0.dev64/c2cciutils/lib/docker.py +0 -141
- c2cciutils-1.8.0.dev64/c2cciutils/lib/oidc.py +0 -188
- c2cciutils-1.8.0.dev64/c2cciutils/package-lock.json +0 -370
- c2cciutils-1.8.0.dev64/c2cciutils/package.json +0 -9
- c2cciutils-1.8.0.dev64/c2cciutils/publish.py +0 -451
- c2cciutils-1.8.0.dev64/c2cciutils/schema-applications.json +0 -50
- c2cciutils-1.8.0.dev64/c2cciutils/schema.json +0 -417
- c2cciutils-1.8.0.dev64/c2cciutils/scripts/clean.py +0 -103
- c2cciutils-1.8.0.dev64/c2cciutils/scripts/docker_versions_gen.py +0 -33
- c2cciutils-1.8.0.dev64/c2cciutils/scripts/pin_pipenv.py +0 -54
- c2cciutils-1.8.0.dev64/c2cciutils/scripts/publish.py +0 -477
- c2cciutils-1.8.0.dev64/c2cciutils/scripts/trigger_image_update.py +0 -84
- c2cciutils-1.8.0.dev64/c2cciutils/scripts/version.py +0 -245
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/LICENSE +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/applications.yaml +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/applications_definition.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/branches.graphql +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/commits.graphql +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/default_branch.graphql +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/env.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/__init__.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/docker_logs.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/download_applications.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/env.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/k8s/__init__.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/k8s/db.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/k8s/install.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/k8s/logs.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/k8s/wait.py +0 -0
- {c2cciutils-1.8.0.dev64 → c2cciutils-1.8.0.dev68}/c2cciutils/scripts/main.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.3
|
|
2
2
|
Name: c2cciutils
|
|
3
|
-
Version: 1.8.0.
|
|
3
|
+
Version: 1.8.0.dev68
|
|
4
4
|
Summary: Common utilities for Camptocamp CI
|
|
5
5
|
License: FreeBSD
|
|
6
6
|
Keywords: ci
|
|
@@ -21,24 +21,9 @@ Classifier: Programming Language :: Python :: 3.11
|
|
|
21
21
|
Classifier: Programming Language :: Python :: 3.12
|
|
22
22
|
Classifier: Programming Language :: Python :: 3.13
|
|
23
23
|
Classifier: Typing :: Typed
|
|
24
|
-
Provides-Extra: audit
|
|
25
|
-
Provides-Extra: checks
|
|
26
|
-
Provides-Extra: pr-checks
|
|
27
|
-
Provides-Extra: publish
|
|
28
|
-
Provides-Extra: publish-plugins
|
|
29
|
-
Provides-Extra: version
|
|
30
24
|
Requires-Dist: PyYAML (==6.0.2)
|
|
31
|
-
Requires-Dist: debian-inspector (==31.1.0)
|
|
32
|
-
Requires-Dist: defusedxml (==0.7.1)
|
|
33
|
-
Requires-Dist: google-api-python-client (==2.160.0) ; extra == "publish"
|
|
34
|
-
Requires-Dist: google-auth-httplib2 (==0.2.0) ; extra == "publish"
|
|
35
|
-
Requires-Dist: google-auth-oauthlib (==1.2.1) ; extra == "publish"
|
|
36
|
-
Requires-Dist: id (==1.5.0) ; extra == "publish"
|
|
37
|
-
Requires-Dist: multi-repo-automation (==1.5.0) ; extra == "version"
|
|
38
25
|
Requires-Dist: requests (==2.32.3)
|
|
39
26
|
Requires-Dist: ruamel.yaml (==0.18.10)
|
|
40
|
-
Requires-Dist: security-md (==1.0.0)
|
|
41
|
-
Requires-Dist: twine (==6.1.0) ; extra == "publish"
|
|
42
27
|
Project-URL: Repository, https://github.com/camptocamp/c2cciutils
|
|
43
28
|
Description-Content-Type: text/markdown
|
|
44
29
|
|
|
@@ -90,7 +75,6 @@ C2cciutils make easier to have those workflows in a project:
|
|
|
90
75
|
|
|
91
76
|
- `auto-review.yaml`: Auto review the Renovate pull requests
|
|
92
77
|
- `backport.yaml`: Trigger the backports (work with labels)
|
|
93
|
-
- `clean.yaml`: Clean the Docker images related on a deleted feature branch
|
|
94
78
|
- `main.yaml`: Main workflow especially with the c2cciutils-checks command
|
|
95
79
|
|
|
96
80
|
All the provided commands used in the workflow:
|
|
@@ -98,8 +82,6 @@ All the provided commands used in the workflow:
|
|
|
98
82
|
- `c2cciutils`: some generic tools.
|
|
99
83
|
- `c2cciutils-version`: Create a new version of the project.
|
|
100
84
|
- `c2cciutils-env`: Print some environment information.
|
|
101
|
-
- `c2cciutils-publish`: Publish the project.
|
|
102
|
-
- `c2cciutils-clean`: Delete Docker images on Docker Hub after corresponding branch have been deleted.
|
|
103
85
|
|
|
104
86
|
## Utilities
|
|
105
87
|
|
|
@@ -146,7 +128,6 @@ In the CI we need to have the following secrets::
|
|
|
146
128
|
## Use locally, in the projects that use c2cciutils
|
|
147
129
|
|
|
148
130
|
Install it: `python3 -m pip install --user --requirement ci/requirements.txt`
|
|
149
|
-
Dry run publish: `GITHUB_REF=... c2cciutils-publish --dry-run ...`
|
|
150
131
|
|
|
151
132
|
## Configuration
|
|
152
133
|
|
|
@@ -203,205 +184,6 @@ Select a formatter:
|
|
|
203
184
|
- Configure Default Formatter...
|
|
204
185
|
- Select the formatter
|
|
205
186
|
|
|
206
|
-
## Publishing
|
|
207
|
-
|
|
208
|
-
### To pypi
|
|
209
|
-
|
|
210
|
-
The config is like this:
|
|
211
|
-
|
|
212
|
-
```yaml
|
|
213
|
-
versions:
|
|
214
|
-
# List of kinds of versions you want to publish, that can be:
|
|
215
|
-
# rebuild (specified with --type),
|
|
216
|
-
# version_tag, version_branch, feature_branch, feature_tag (for pull request)
|
|
217
|
-
```
|
|
218
|
-
|
|
219
|
-
It we have a `setup.py` file, we will be in legacy mode:
|
|
220
|
-
When publishing, the version computed from arguments or `GITHUB_REF` is put in environment variable `VERSION`, thus you should use it in `setup.py`, example:
|
|
221
|
-
|
|
222
|
-
```python
|
|
223
|
-
VERSION = os.environ.get("VERSION", "1.0.0")
|
|
224
|
-
```
|
|
225
|
-
|
|
226
|
-
Also we consider that we use `poetry` with [poetry-dynamic-versioning](https://pypi.org/project/poetry-dynamic-versioning/) to manage the version, and [poetry-plugin-tweak-dependencies-version](https://pypi.org/project/poetry-plugin-tweak-dependencies-version/) to manage the dependencies versions.
|
|
227
|
-
|
|
228
|
-
Example of configuration:
|
|
229
|
-
|
|
230
|
-
```toml
|
|
231
|
-
[tool.poetry-dynamic-versioning]
|
|
232
|
-
enable = true
|
|
233
|
-
vcs = "git"
|
|
234
|
-
pattern = "^(?P<base>\\d+(\\.\\d+)*)"
|
|
235
|
-
format-jinja = """
|
|
236
|
-
{%- if env.get("VERSION_TYPE") == "version_branch" -%}
|
|
237
|
-
{{serialize_pep440(bump_version(base, 1 if env.get("IS_MASTER") == "TRUE" else 2), dev=distance)}}
|
|
238
|
-
{%- elif distance == 0 -%}
|
|
239
|
-
{{serialize_pep440(base)}}
|
|
240
|
-
{%- else -%}
|
|
241
|
-
{{serialize_pep440(bump_version(base), dev=distance)}}
|
|
242
|
-
{%- endif -%}
|
|
243
|
-
"""
|
|
244
|
-
|
|
245
|
-
```
|
|
246
|
-
|
|
247
|
-
Note that we can access to the environment variables `VERSION`,`VERSION_TYPE` and `IS_MASTER`.
|
|
248
|
-
|
|
249
|
-
Then by default:
|
|
250
|
-
|
|
251
|
-
- Tag with `1.2.3` => release `1.2.3`
|
|
252
|
-
- Commit on feature branch just do a validation
|
|
253
|
-
- Commit on `master` branch after the tag 1.3.0 => release `1.4.0.dev1`
|
|
254
|
-
- Commit on `1.3` branch after the tag 1.3.0 => release `1.3.1.dev1`
|
|
255
|
-
|
|
256
|
-
#### Authentication
|
|
257
|
-
|
|
258
|
-
If the file `~/.pypirc` exists we consider that we ar already logged in also
|
|
259
|
-
we will do the login with the `pypi` server with OpenID Connect (OIDC).
|
|
260
|
-
|
|
261
|
-
The OIDC login is recommended because it didn't needs any additional secrets,
|
|
262
|
-
but it need some configuration on pypi in the package,
|
|
263
|
-
see the [GitHub Documentation](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi#adding-the-identity-provider-to-pypi).
|
|
264
|
-
|
|
265
|
-
#### Integration if the package directly in a Docker image
|
|
266
|
-
|
|
267
|
-
To make it working in the `Dockerfile` you should have in the `poetry` stage:
|
|
268
|
-
|
|
269
|
-
```Dockerfile
|
|
270
|
-
ENV POETRY_DYNAMIC_VERSIONING_BYPASS=dev
|
|
271
|
-
RUN poetry export --extras=checks --extras=publish --output=requirements.txt \
|
|
272
|
-
&& poetry export --with=dev --output=requirements-dev.txt
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
And in the `run` stage
|
|
276
|
-
|
|
277
|
-
```Dockerfile
|
|
278
|
-
ARG VERSION=dev
|
|
279
|
-
RUN --mount=type=cache,target=/root/.cache \
|
|
280
|
-
POETRY_DYNAMIC_VERSIONING_BYPASS=${VERSION} python3 -m pip install --disable-pip-version-check --no-deps --editable=.
|
|
281
|
-
```
|
|
282
|
-
|
|
283
|
-
And in the `Makefile`:
|
|
284
|
-
|
|
285
|
-
```Makefile
|
|
286
|
-
VERSION = $(strip $(shell poetry version --short))
|
|
287
|
-
|
|
288
|
-
.PHONY: build
|
|
289
|
-
build: ## Build the Docker images
|
|
290
|
-
docker build --build-arg=VERSION=$(VERSION) --tag=$(GITHUB_REPOSITORY) .
|
|
291
|
-
```
|
|
292
|
-
|
|
293
|
-
### To Docker registry
|
|
294
|
-
|
|
295
|
-
The config is like this:
|
|
296
|
-
|
|
297
|
-
```yaml
|
|
298
|
-
latest: True
|
|
299
|
-
images:
|
|
300
|
-
- # The base name of the image we want to publish
|
|
301
|
-
name:
|
|
302
|
-
repository:
|
|
303
|
-
<internal_name>:
|
|
304
|
-
# The fqdn name of the server if not Docker hub
|
|
305
|
-
server:
|
|
306
|
-
# List of kinds of versions you want to publish, that can be: rebuild (specified using --type),
|
|
307
|
-
# version_tag, version_branch, feature_branch, feature_tag (for pull request)
|
|
308
|
-
version:
|
|
309
|
-
# List of tags we want to publish interpreted with `format(version=version)`
|
|
310
|
-
# e.g. if you use `{version}-lite` when you publish the version `1.2.3` the source tag
|
|
311
|
-
# (that should be built by the application build) is `latest-lite`, and it will be published
|
|
312
|
-
# with the tag `1.2.3-lite`.
|
|
313
|
-
tags:
|
|
314
|
-
# If your images are published by different jobs you can separate them in different groups
|
|
315
|
-
# and publish them with `c2cciutils-publish --group=<group>`
|
|
316
|
-
group:
|
|
317
|
-
```
|
|
318
|
-
|
|
319
|
-
By default, the last line of the `SECURITY.md` file will be published (`docker`) with the tag
|
|
320
|
-
`latest`. Set `latest` to `False` to disable it.
|
|
321
|
-
|
|
322
|
-
With the `c2cciutils-clean` the images on Docker hub for `feature_branch` will be removed on branch removing.
|
|
323
|
-
|
|
324
|
-
## Download applications
|
|
325
|
-
|
|
326
|
-
In case some executables or applications from GitHub releases or any other URLs are required on the CI host
|
|
327
|
-
and are not handled by any dependency manager, we provide a set of tools to install them and manage upgrades
|
|
328
|
-
through Renovate.
|
|
329
|
-
|
|
330
|
-
Create an application file (e.-g. `applications.yaml`) with:
|
|
331
|
-
|
|
332
|
-
```yaml
|
|
333
|
-
# yaml-language-server: $schema=https://raw.githubusercontent.com/camptocamp/c2cciutils/master/c2cciutils/schema-applications.json
|
|
334
|
-
|
|
335
|
-
# Application from GitHub release
|
|
336
|
-
<organization>/<project>:
|
|
337
|
-
get-file-name: <file name present in the release>
|
|
338
|
-
to-file-name: <The file name you want to create in ~/.local/bin>
|
|
339
|
-
finish-command: # The command you want to run after the file is downloaded
|
|
340
|
-
- - chmod # To be executable (usually required)
|
|
341
|
-
- +x
|
|
342
|
-
- <to-file-name>
|
|
343
|
-
- - <to-file-name> # Print the version of the application
|
|
344
|
-
- --version
|
|
345
|
-
# Application from GitHub release in a tar file (or tar.gz)
|
|
346
|
-
<organization>/<project>:
|
|
347
|
-
get-file-name: <file name present in the release>
|
|
348
|
-
type: tar
|
|
349
|
-
tar-file-name: <The file name available in the tar file>
|
|
350
|
-
to-file-name: <The file name you want to create in ~/.local/bin>
|
|
351
|
-
finish-command: [...] # The command you want to run after the file is downloaded
|
|
352
|
-
# Application from an URL
|
|
353
|
-
<application reference name>:
|
|
354
|
-
url-pattern: <The URL used to download the application>
|
|
355
|
-
to-file-name: <The file name you want to create in ~/.local/bin>
|
|
356
|
-
finish-command: [...] # The command you want to run after the file is downloaded
|
|
357
|
-
```
|
|
358
|
-
|
|
359
|
-
In the attributes `url-pattern`, `get-file-name` you can use the following variables:
|
|
360
|
-
|
|
361
|
-
- `{version}`: The version of the application present in the version file.
|
|
362
|
-
- `{version_quote}`: The URL encoded version.
|
|
363
|
-
- `{short_version}`: The version without the `v` prefix.
|
|
364
|
-
|
|
365
|
-
The `applications-versions.yaml` file is a map of applications and their versions.
|
|
366
|
-
|
|
367
|
-
Add in your Renovate configuration:
|
|
368
|
-
|
|
369
|
-
```json5
|
|
370
|
-
regexManagers: [
|
|
371
|
-
{
|
|
372
|
-
fileMatch: ['^applications-versions.yaml$'],
|
|
373
|
-
matchStrings: [
|
|
374
|
-
'(?<depName>[^\\s]+): (?<currentValue>[^\\s]+) # (?<datasource>[^\\s]+)',
|
|
375
|
-
],
|
|
376
|
-
},
|
|
377
|
-
],
|
|
378
|
-
```
|
|
379
|
-
|
|
380
|
-
Now you need to call `c2cciutils-download-applications --applications-file=applications.yaml --versions-file=applications-version.yaml`
|
|
381
|
-
to install required applications on CI host before using them (an already installed application is installed only if needed).
|
|
382
|
-
|
|
383
|
-
## Use Renovate to trigger a new build instead of the legacy rebuild
|
|
384
|
-
|
|
385
|
-
Run the command `c2cciutils-docker-versions-gen camptocamp/image[:tag]` to generate a file that is a kind of package lock of the Debian packages in the file `ci/dpkg-versions.yaml`.
|
|
386
|
-
|
|
387
|
-
Add in your renovate configuration:
|
|
388
|
-
|
|
389
|
-
```javascript
|
|
390
|
-
regexManagers: [
|
|
391
|
-
{
|
|
392
|
-
fileMatch: ['^ci/dpkg-versions.yaml$'],
|
|
393
|
-
matchStrings: [" *(?<depName>[^'\\s]+): '?(?<currentValue>[^'\\s/]*[0-9][^'\\s/]*)'?"],
|
|
394
|
-
datasourceTemplate: 'repology',
|
|
395
|
-
versioningTemplate: 'loose',
|
|
396
|
-
},
|
|
397
|
-
],
|
|
398
|
-
```
|
|
399
|
-
|
|
400
|
-
When a new version of a Debian package will be available:
|
|
401
|
-
|
|
402
|
-
- Renovate will automatically open a pull request to update the file `ci/dpkg-versions.yaml`.
|
|
403
|
-
- And the continuous integration will build a new fresh Docker image with latest versions of all Debian packages.
|
|
404
|
-
|
|
405
187
|
## Kubernetes
|
|
406
188
|
|
|
407
189
|
C2cciutils provide some commands for Kubernetes.
|
|
@@ -46,7 +46,6 @@ C2cciutils make easier to have those workflows in a project:
|
|
|
46
46
|
|
|
47
47
|
- `auto-review.yaml`: Auto review the Renovate pull requests
|
|
48
48
|
- `backport.yaml`: Trigger the backports (work with labels)
|
|
49
|
-
- `clean.yaml`: Clean the Docker images related on a deleted feature branch
|
|
50
49
|
- `main.yaml`: Main workflow especially with the c2cciutils-checks command
|
|
51
50
|
|
|
52
51
|
All the provided commands used in the workflow:
|
|
@@ -54,8 +53,6 @@ All the provided commands used in the workflow:
|
|
|
54
53
|
- `c2cciutils`: some generic tools.
|
|
55
54
|
- `c2cciutils-version`: Create a new version of the project.
|
|
56
55
|
- `c2cciutils-env`: Print some environment information.
|
|
57
|
-
- `c2cciutils-publish`: Publish the project.
|
|
58
|
-
- `c2cciutils-clean`: Delete Docker images on Docker Hub after corresponding branch have been deleted.
|
|
59
56
|
|
|
60
57
|
## Utilities
|
|
61
58
|
|
|
@@ -102,7 +99,6 @@ In the CI we need to have the following secrets::
|
|
|
102
99
|
## Use locally, in the projects that use c2cciutils
|
|
103
100
|
|
|
104
101
|
Install it: `python3 -m pip install --user --requirement ci/requirements.txt`
|
|
105
|
-
Dry run publish: `GITHUB_REF=... c2cciutils-publish --dry-run ...`
|
|
106
102
|
|
|
107
103
|
## Configuration
|
|
108
104
|
|
|
@@ -159,205 +155,6 @@ Select a formatter:
|
|
|
159
155
|
- Configure Default Formatter...
|
|
160
156
|
- Select the formatter
|
|
161
157
|
|
|
162
|
-
## Publishing
|
|
163
|
-
|
|
164
|
-
### To pypi
|
|
165
|
-
|
|
166
|
-
The config is like this:
|
|
167
|
-
|
|
168
|
-
```yaml
|
|
169
|
-
versions:
|
|
170
|
-
# List of kinds of versions you want to publish, that can be:
|
|
171
|
-
# rebuild (specified with --type),
|
|
172
|
-
# version_tag, version_branch, feature_branch, feature_tag (for pull request)
|
|
173
|
-
```
|
|
174
|
-
|
|
175
|
-
It we have a `setup.py` file, we will be in legacy mode:
|
|
176
|
-
When publishing, the version computed from arguments or `GITHUB_REF` is put in environment variable `VERSION`, thus you should use it in `setup.py`, example:
|
|
177
|
-
|
|
178
|
-
```python
|
|
179
|
-
VERSION = os.environ.get("VERSION", "1.0.0")
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
Also we consider that we use `poetry` with [poetry-dynamic-versioning](https://pypi.org/project/poetry-dynamic-versioning/) to manage the version, and [poetry-plugin-tweak-dependencies-version](https://pypi.org/project/poetry-plugin-tweak-dependencies-version/) to manage the dependencies versions.
|
|
183
|
-
|
|
184
|
-
Example of configuration:
|
|
185
|
-
|
|
186
|
-
```toml
|
|
187
|
-
[tool.poetry-dynamic-versioning]
|
|
188
|
-
enable = true
|
|
189
|
-
vcs = "git"
|
|
190
|
-
pattern = "^(?P<base>\\d+(\\.\\d+)*)"
|
|
191
|
-
format-jinja = """
|
|
192
|
-
{%- if env.get("VERSION_TYPE") == "version_branch" -%}
|
|
193
|
-
{{serialize_pep440(bump_version(base, 1 if env.get("IS_MASTER") == "TRUE" else 2), dev=distance)}}
|
|
194
|
-
{%- elif distance == 0 -%}
|
|
195
|
-
{{serialize_pep440(base)}}
|
|
196
|
-
{%- else -%}
|
|
197
|
-
{{serialize_pep440(bump_version(base), dev=distance)}}
|
|
198
|
-
{%- endif -%}
|
|
199
|
-
"""
|
|
200
|
-
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
Note that we can access to the environment variables `VERSION`,`VERSION_TYPE` and `IS_MASTER`.
|
|
204
|
-
|
|
205
|
-
Then by default:
|
|
206
|
-
|
|
207
|
-
- Tag with `1.2.3` => release `1.2.3`
|
|
208
|
-
- Commit on feature branch just do a validation
|
|
209
|
-
- Commit on `master` branch after the tag 1.3.0 => release `1.4.0.dev1`
|
|
210
|
-
- Commit on `1.3` branch after the tag 1.3.0 => release `1.3.1.dev1`
|
|
211
|
-
|
|
212
|
-
#### Authentication
|
|
213
|
-
|
|
214
|
-
If the file `~/.pypirc` exists we consider that we ar already logged in also
|
|
215
|
-
we will do the login with the `pypi` server with OpenID Connect (OIDC).
|
|
216
|
-
|
|
217
|
-
The OIDC login is recommended because it didn't needs any additional secrets,
|
|
218
|
-
but it need some configuration on pypi in the package,
|
|
219
|
-
see the [GitHub Documentation](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi#adding-the-identity-provider-to-pypi).
|
|
220
|
-
|
|
221
|
-
#### Integration if the package directly in a Docker image
|
|
222
|
-
|
|
223
|
-
To make it working in the `Dockerfile` you should have in the `poetry` stage:
|
|
224
|
-
|
|
225
|
-
```Dockerfile
|
|
226
|
-
ENV POETRY_DYNAMIC_VERSIONING_BYPASS=dev
|
|
227
|
-
RUN poetry export --extras=checks --extras=publish --output=requirements.txt \
|
|
228
|
-
&& poetry export --with=dev --output=requirements-dev.txt
|
|
229
|
-
```
|
|
230
|
-
|
|
231
|
-
And in the `run` stage
|
|
232
|
-
|
|
233
|
-
```Dockerfile
|
|
234
|
-
ARG VERSION=dev
|
|
235
|
-
RUN --mount=type=cache,target=/root/.cache \
|
|
236
|
-
POETRY_DYNAMIC_VERSIONING_BYPASS=${VERSION} python3 -m pip install --disable-pip-version-check --no-deps --editable=.
|
|
237
|
-
```
|
|
238
|
-
|
|
239
|
-
And in the `Makefile`:
|
|
240
|
-
|
|
241
|
-
```Makefile
|
|
242
|
-
VERSION = $(strip $(shell poetry version --short))
|
|
243
|
-
|
|
244
|
-
.PHONY: build
|
|
245
|
-
build: ## Build the Docker images
|
|
246
|
-
docker build --build-arg=VERSION=$(VERSION) --tag=$(GITHUB_REPOSITORY) .
|
|
247
|
-
```
|
|
248
|
-
|
|
249
|
-
### To Docker registry
|
|
250
|
-
|
|
251
|
-
The config is like this:
|
|
252
|
-
|
|
253
|
-
```yaml
|
|
254
|
-
latest: True
|
|
255
|
-
images:
|
|
256
|
-
- # The base name of the image we want to publish
|
|
257
|
-
name:
|
|
258
|
-
repository:
|
|
259
|
-
<internal_name>:
|
|
260
|
-
# The fqdn name of the server if not Docker hub
|
|
261
|
-
server:
|
|
262
|
-
# List of kinds of versions you want to publish, that can be: rebuild (specified using --type),
|
|
263
|
-
# version_tag, version_branch, feature_branch, feature_tag (for pull request)
|
|
264
|
-
version:
|
|
265
|
-
# List of tags we want to publish interpreted with `format(version=version)`
|
|
266
|
-
# e.g. if you use `{version}-lite` when you publish the version `1.2.3` the source tag
|
|
267
|
-
# (that should be built by the application build) is `latest-lite`, and it will be published
|
|
268
|
-
# with the tag `1.2.3-lite`.
|
|
269
|
-
tags:
|
|
270
|
-
# If your images are published by different jobs you can separate them in different groups
|
|
271
|
-
# and publish them with `c2cciutils-publish --group=<group>`
|
|
272
|
-
group:
|
|
273
|
-
```
|
|
274
|
-
|
|
275
|
-
By default, the last line of the `SECURITY.md` file will be published (`docker`) with the tag
|
|
276
|
-
`latest`. Set `latest` to `False` to disable it.
|
|
277
|
-
|
|
278
|
-
With the `c2cciutils-clean` the images on Docker hub for `feature_branch` will be removed on branch removing.
|
|
279
|
-
|
|
280
|
-
## Download applications
|
|
281
|
-
|
|
282
|
-
In case some executables or applications from GitHub releases or any other URLs are required on the CI host
|
|
283
|
-
and are not handled by any dependency manager, we provide a set of tools to install them and manage upgrades
|
|
284
|
-
through Renovate.
|
|
285
|
-
|
|
286
|
-
Create an application file (e.-g. `applications.yaml`) with:
|
|
287
|
-
|
|
288
|
-
```yaml
|
|
289
|
-
# yaml-language-server: $schema=https://raw.githubusercontent.com/camptocamp/c2cciutils/master/c2cciutils/schema-applications.json
|
|
290
|
-
|
|
291
|
-
# Application from GitHub release
|
|
292
|
-
<organization>/<project>:
|
|
293
|
-
get-file-name: <file name present in the release>
|
|
294
|
-
to-file-name: <The file name you want to create in ~/.local/bin>
|
|
295
|
-
finish-command: # The command you want to run after the file is downloaded
|
|
296
|
-
- - chmod # To be executable (usually required)
|
|
297
|
-
- +x
|
|
298
|
-
- <to-file-name>
|
|
299
|
-
- - <to-file-name> # Print the version of the application
|
|
300
|
-
- --version
|
|
301
|
-
# Application from GitHub release in a tar file (or tar.gz)
|
|
302
|
-
<organization>/<project>:
|
|
303
|
-
get-file-name: <file name present in the release>
|
|
304
|
-
type: tar
|
|
305
|
-
tar-file-name: <The file name available in the tar file>
|
|
306
|
-
to-file-name: <The file name you want to create in ~/.local/bin>
|
|
307
|
-
finish-command: [...] # The command you want to run after the file is downloaded
|
|
308
|
-
# Application from an URL
|
|
309
|
-
<application reference name>:
|
|
310
|
-
url-pattern: <The URL used to download the application>
|
|
311
|
-
to-file-name: <The file name you want to create in ~/.local/bin>
|
|
312
|
-
finish-command: [...] # The command you want to run after the file is downloaded
|
|
313
|
-
```
|
|
314
|
-
|
|
315
|
-
In the attributes `url-pattern`, `get-file-name` you can use the following variables:
|
|
316
|
-
|
|
317
|
-
- `{version}`: The version of the application present in the version file.
|
|
318
|
-
- `{version_quote}`: The URL encoded version.
|
|
319
|
-
- `{short_version}`: The version without the `v` prefix.
|
|
320
|
-
|
|
321
|
-
The `applications-versions.yaml` file is a map of applications and their versions.
|
|
322
|
-
|
|
323
|
-
Add in your Renovate configuration:
|
|
324
|
-
|
|
325
|
-
```json5
|
|
326
|
-
regexManagers: [
|
|
327
|
-
{
|
|
328
|
-
fileMatch: ['^applications-versions.yaml$'],
|
|
329
|
-
matchStrings: [
|
|
330
|
-
'(?<depName>[^\\s]+): (?<currentValue>[^\\s]+) # (?<datasource>[^\\s]+)',
|
|
331
|
-
],
|
|
332
|
-
},
|
|
333
|
-
],
|
|
334
|
-
```
|
|
335
|
-
|
|
336
|
-
Now you need to call `c2cciutils-download-applications --applications-file=applications.yaml --versions-file=applications-version.yaml`
|
|
337
|
-
to install required applications on CI host before using them (an already installed application is installed only if needed).
|
|
338
|
-
|
|
339
|
-
## Use Renovate to trigger a new build instead of the legacy rebuild
|
|
340
|
-
|
|
341
|
-
Run the command `c2cciutils-docker-versions-gen camptocamp/image[:tag]` to generate a file that is a kind of package lock of the Debian packages in the file `ci/dpkg-versions.yaml`.
|
|
342
|
-
|
|
343
|
-
Add in your renovate configuration:
|
|
344
|
-
|
|
345
|
-
```javascript
|
|
346
|
-
regexManagers: [
|
|
347
|
-
{
|
|
348
|
-
fileMatch: ['^ci/dpkg-versions.yaml$'],
|
|
349
|
-
matchStrings: [" *(?<depName>[^'\\s]+): '?(?<currentValue>[^'\\s/]*[0-9][^'\\s/]*)'?"],
|
|
350
|
-
datasourceTemplate: 'repology',
|
|
351
|
-
versioningTemplate: 'loose',
|
|
352
|
-
},
|
|
353
|
-
],
|
|
354
|
-
```
|
|
355
|
-
|
|
356
|
-
When a new version of a Debian package will be available:
|
|
357
|
-
|
|
358
|
-
- Renovate will automatically open a pull request to update the file `ci/dpkg-versions.yaml`.
|
|
359
|
-
- And the continuous integration will build a new fresh Docker image with latest versions of all Debian packages.
|
|
360
|
-
|
|
361
158
|
## Kubernetes
|
|
362
159
|
|
|
363
160
|
C2cciutils provide some commands for Kubernetes.
|