c2cciutils 1.7.0.dev174__py3-none-any.whl → 1.8.0.dev45__py3-none-any.whl

{c2cciutils-1.7.0.dev174.dist-info → c2cciutils-1.8.0.dev45.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: c2cciutils
-Version: 1.7.0.dev174
+Version: 1.8.0.dev45
 Summary: Common utilities for Camptocamp CI
 Home-page: https://github.com/camptocamp/c2cciutils
 License: FreeBSD
@@ -18,29 +18,28 @@ Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Typing :: Typed
 Provides-Extra: audit
 Provides-Extra: checks
-Provides-Extra: pr_checks
+Provides-Extra: pr-checks
 Provides-Extra: publish
-Provides-Extra: publish_plugins
+Provides-Extra: publish-plugins
 Provides-Extra: version
-Requires-Dist: Markdown (>=3.0.0,<4.0.0)
 Requires-Dist: PyYAML (>=6.0.0,<7.0.0)
-Requires-Dist: codespell (==2.2.6); extra == "pr_checks"
 Requires-Dist: debian-inspector (>=31.0.0,<32.0.0)
 Requires-Dist: defusedxml (>=0.0.0,<1.0.0)
-Requires-Dist: google-api-python-client (>=2.0.0,<3.0.0); extra == "publish"
-Requires-Dist: google-auth-httplib2 (>=0.0.0,<1.0.0); extra == "publish"
-Requires-Dist: google-auth-oauthlib (>=1.0.0,<2.0.0); extra == "publish"
-Requires-Dist: markdown-table (>=2020.0.0,<2021.0.0)
-Requires-Dist: multi-repo-automation (>=1.0.0,<2.0.0); extra == "version"
-Requires-Dist: python-magic (>=0.0.0,<1.0.0)
+Requires-Dist: google-api-python-client (>=2.0.0,<3.0.0) ; extra == "publish"
+Requires-Dist: google-auth-httplib2 (>=0.0.0,<1.0.0) ; extra == "publish"
+Requires-Dist: google-auth-oauthlib (>=1.0.0,<2.0.0) ; extra == "publish"
+Requires-Dist: id (>=1.0.0,<2.0.0) ; extra == "publish"
+Requires-Dist: multi-repo-automation (>=1.0.0,<2.0.0) ; extra == "version"
 Requires-Dist: requests (>=2.0.0,<3.0.0)
 Requires-Dist: ruamel.yaml (>=0.0.0,<1.0.0)
-Requires-Dist: toml (>=0.0.0,<1.0.0)
-Requires-Dist: tomlkit (>=0.0.0,<1.0.0); extra == "publish"
-Requires-Dist: twine (>=5.0.0,<6.0.0); extra == "publish"
+Requires-Dist: security-md (>=1.0.0,<2.0.0)
+Requires-Dist: twine (>=6.0.0,<7.0.0) ; extra == "publish"
 Project-URL: Repository, https://github.com/camptocamp/c2cciutils
 Description-Content-Type: text/markdown
 
@@ -48,47 +47,14 @@ Description-Content-Type: text/markdown
 
 ## Publishing
 
-The main goals of C2C CI utils is to offer the commands and the workflows to have the following project structure:
-
-Have stabilization branches named by default `<major>.<minor>`.
-Have the release named by default `<major>.<minor>.<patch>`.
-
-With C2C CI utils you can publish a python package and a Docker image from the same repository.
-
-The default publishing are:
-
-- Push on the `<major>.<minor>` branch will publish Docker images.
-- Create the tag `<major>.<minor>.<patch>` will publish the Docker images, and the Python package.
-- Push on a feature branch (whatever other name) will publish the Docker images.
-- Delete a feature branch will delete the Docker images.
-- Push on the `master` branch will publish the Docker images with the master tag (Publishing a python package is also possible).
-- The version at the last line of the `SECURITY.md` of the `master` branch will be also published using the `latest` tag,
-  this will respect the `tags` present in the configuration
-- In the `SECURITY.md` file of the `master` branch we can also add a column `Alternate Tag` to publish the Docker images with another tag,
-  this will respect the `tags` present in the configuration (only for Docker).
-
-The Docker images are published on Docker Hub and GitHub Container Registry.
-
-You can run the publishing locally in dry-run mode:
-
-```bash
-GITHUB_REF=... c2cciutils-publish --dry-run ...
-```
+The main goals of C2C CI utils is to offer the commands to publish the project,
+see the [documentation](https://github.com/camptocamp/c2cciutils/wiki/Publishing).
 
 ## Changelog
 
 When we create a tag by default with the `changelog` workflow a release is created on GitHub, a changelog is
 generated and added to the release.
 
-## Security
-
-The security is managed by the `c2cciutils-audit` command with Snyk, it will audit the dependencies of the project on every
-stabilization branches, if possible a pull request is created automatically to update the dependencies.
-
-When we publish a Docker image the generated image is monitored by Snyk, this means that Snyk will search
-for all the dependencies and send the list to the Snyk web site to be monitored.
-We also do a test of the image and log the result (This will never cause the build to fail).
-
 ## Checks
 
 C2C CI utils will no more provide a tool to do a check of the project, this is replaced by `pre-commit`,
@@ -123,21 +89,16 @@ workflow will delete the workflows older than 500 days.
 
 C2cciutils make easier to have those workflows in a project:
 
-- `audit.yaml`: Audit the stabilization branches of the application against vulnerabilities in the python and node dependency
 - `auto-review.yaml`: Auto review the Renovate pull requests
 - `backport.yaml`: Trigger the backports (work with labels)
 - `clean.yaml`: Clean the Docker images related on a deleted feature branch
 - `main.yaml`: Main workflow especially with the c2cciutils-checks command
-- `changelog.yaml`: Generate the changelog and create the release on GitHub
-- `delete-old-workflows-run.yaml`: Delete the old workflows
-- `pr-checks.yaml`: Run the checks on the pull requests
 
 All the provided commands used in the workflow:
 
 - `c2cciutils`: some generic tools.
 - `c2cciutils-version`: Create a new version of the project.
-- `c2cciutils-checks`: Run the checks on the code (those checks don't need any project dependencies).
-- `c2cciutils-audit`: Do the audit, the main difference with checks is that it can change between runs on the same code.
+- `c2cciutils-env`: Print some environment information.
 - `c2cciutils-publish`: Publish the project.
 - `c2cciutils-clean`: Delete Docker images on Docker Hub after corresponding branch have been deleted.
 
@@ -198,7 +159,6 @@ You can override the configuration with the file `ci/config.yaml`.
 At the base of the configuration you have:
 
 - `version`: Contains some regular expressions to find the versions branches and tags, and to convert them into application versions.
-- `audit`: The audit configuration, see `c2cciutils/audit.py` for more information.
 - `publish`: The publishing configuration, see `c2cciutils/publish.py` for more information.
 
 Many actions can be disabled by setting the corresponding configuration part to `False`.
@@ -294,11 +254,22 @@ Then by default:
 - Commit on `master` branch after the tag 1.3.0 => release `1.4.0.dev1`
 - Commit on `1.3` branch after the tag 1.3.0 => release `1.3.1.dev1`
 
+#### Authentication
+
+If the file `~/.pypirc` exists we consider that we ar already logged in also
+we will do the login with the `pypi` server with OpenID Connect (OIDC).
+
+The OIDC login is recommended because it didn't needs any additional secrets,
+but it need some configuration on pypi in the package,
+see the [GitHub Documentation](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi#adding-the-identity-provider-to-pypi).
+
+#### Integration if the package directly in a Docker image
+
 To make it working in the `Dockerfile` you should have in the `poetry` stage:
 
 ```Dockerfile
 ENV POETRY_DYNAMIC_VERSIONING_BYPASS=dev
-RUN poetry export --extras=checks --extras=publish --extras=audit --output=requirements.txt \
+RUN poetry export --extras=checks --extras=publish --output=requirements.txt \
     && poetry export --with=dev --output=requirements-dev.txt
 ```
 
@@ -336,7 +307,7 @@ repository:
     # List of kinds of versions you want to publish, that can be: rebuild (specified using --type),
     # version_tag, version_branch, feature_branch, feature_tag (for pull request)
     version:
-    # List of tags we want to publish interpreted with `template(version=version)`
+    # List of tags we want to publish interpreted with `format(version=version)`
     # e.g. if you use `{version}-lite` when you publish the version `1.2.3` the source tag
     # (that should be built by the application build) is `latest-lite`, and it will be published
     # with the tag `1.2.3-lite`.

c2cciutils-1.8.0.dev45.dist-info/RECORD

@@ -0,0 +1,37 @@
+c2cciutils/__init__.py,sha256=-J6_XPwyZ3Xbix6AhmAONbltSf9okJsXwQvBj6TuP_U,13353
+c2cciutils/applications-versions.yaml,sha256=uTCcyFKt2GJIhUNxrOl3hEZo-6ee0x5ftb9Zncci6dk,224
+c2cciutils/applications.yaml,sha256=yn0XRi08cS29A_jXPofcBPxsGBv7PEBliztjRC3WtfM,504
+c2cciutils/applications_definition.py,sha256=inAh3vJitu3S9n62Ntv6N-avNUQlUd5FatNvDq4VpxM,1305
+c2cciutils/branches.graphql,sha256=UZrj1RO-H527M1SKqWm1VnkWtNsuKTnPTf4BCU2YcOU,358
+c2cciutils/commits.graphql,sha256=3HAuIEig5V7j1L-6mqBaOkiTD3Fb8_gl1ilpZjPJf74,308
+c2cciutils/configuration.py,sha256=LwJw-l9zB3zy7LtjjrWHFATIuxdRVMJZ-mLbTbFiug8,17062
+c2cciutils/default_branch.graphql,sha256=CaP3rRsNiyg_7RvqbMk0tOJr0aqWd8cOeSV-ZKgvKY4,131
+c2cciutils/env.py,sha256=daNqFY-6-X7PGuPz5OaN634KqYtsawj9Dj-Zt-bxctw,3338
+c2cciutils/lib/docker.py,sha256=ULklJgc-8sfy1vI8p_cbYYEq_u73BKNIGorMxzEKA2I,5663
+c2cciutils/lib/oidc.py,sha256=inulioX8__IJEF-yJO8pJNJb-CnU1zdSLdXrD7uP2Qc,6297
+c2cciutils/package-lock.json,sha256=3m-e-2KYk1OBfeUJ1ICuYXx1Z9sEgX7YUW4xSI7QiIM,13406
+c2cciutils/package.json,sha256=M-ah0GP5yl-tipGXquiCD9m6wBPqUwonCrmHLG9JwT4,134
+c2cciutils/publish.py,sha256=Pc8BdEMZiqDiNIP3tlysNf1nxc-0ZM0lnymfKcqapg8,17158
+c2cciutils/schema-applications.json,sha256=Tus-s9NB7uwKhTrQwhWQM4_oJygF_yHUqShtZhN1IxE,1551
+c2cciutils/schema.json,sha256=G0WbvTVn9zdDjilFr28uZqF7zDVADl7uQus12_TVnSs,14559
+c2cciutils/scripts/__init__.py,sha256=YvySYhMJ9eUqBc9FgH6hNbZHbTU25D7z6FIbYJiRW_U,34
+c2cciutils/scripts/clean.py,sha256=oB4A4TvSNoo7vGWzy5m0p2QDyGM-ix0FepxPIqqNlyg,2987
+c2cciutils/scripts/docker_logs.py,sha256=-_LeoavnKqNNeDJMwkAMKrH91JBUiJuAScMpCA6VfTA,1766
+c2cciutils/scripts/docker_versions_gen.py,sha256=yp1I_UqxyIABKxHek3q3UYWh0YR8tqKWzliC2kqZOa0,1315
+c2cciutils/scripts/download_applications.py,sha256=x0SPm4MCWCIYMp1AWekIpbOicwUixitEAJ69quMq698,4488
+c2cciutils/scripts/env.py,sha256=YZwV39D_vQ454ytHByI84gXP6VrtTW3eM8MdFLUBCw0,363
+c2cciutils/scripts/k8s/__init__.py,sha256=m8_lELJYRmzbAW7aGZL_T3tNwuFi1t8ot8F271HZp9E,67
+c2cciutils/scripts/k8s/db.py,sha256=GK1tzzyCqrCyIJzcBdIXjyNJUXrBPFNa-9fdtwxyrlU,3268
+c2cciutils/scripts/k8s/install.py,sha256=OIQ8KHA3Pst2pjO2E-J5FYNaBHW-i-fqCXlAUcG1tw0,933
+c2cciutils/scripts/k8s/logs.py,sha256=-xJYu8BBUmSmMrPEwiTBKZjJBRyIlMp1depCB04_NWs,2655
+c2cciutils/scripts/k8s/wait.py,sha256=mZyQbmSwuC2BZuJlfJDjkLSKWXz4WkIkihOjRzdBa6Y,5689
+c2cciutils/scripts/main.py,sha256=DfljDzEGCWwP7FZG0RRZaXD6PmOVaXMJjnNBQ0-rAWM,785
+c2cciutils/scripts/pin_pipenv.py,sha256=jBTwlolcEL0MUyq6VYzO-adkcL1gqN7B3kBb3UjTo2k,2150
+c2cciutils/scripts/publish.py,sha256=wzXSBn_60DzMJVdO8i2NjjO6Vwg6L6efH_T6ZR6LRsU,20513
+c2cciutils/scripts/trigger_image_update.py,sha256=kEQHs71ymRNFh5VpKt0UMJILxcRPFZupl24iJ2EA3Bs,2772
+c2cciutils/scripts/version.py,sha256=Srk22CQudxT8JixS-ZBsxeJvWryYVzN_NiWpcJvgrWI,8931
+c2cciutils-1.8.0.dev45.dist-info/LICENSE,sha256=K_e76Y2cY12AHvtanvNmIEJ0nWsNh_WFMDK020TxMcY,1307
+c2cciutils-1.8.0.dev45.dist-info/METADATA,sha256=v7iKCTMwlIgCjTOxcdxPPonwrlQgTIXfl9yunTXi3FU,17852
+c2cciutils-1.8.0.dev45.dist-info/WHEEL,sha256=Nq82e9rUAnEjt98J6MlVmMCZb-t9cYE2Ir1kpBmnWfs,88
+c2cciutils-1.8.0.dev45.dist-info/entry_points.txt,sha256=54llpn1q67tS78NbLwA8rwSDWnEQ6aE3FaqMN0ctNN8,918
+c2cciutils-1.8.0.dev45.dist-info/RECORD,,

{c2cciutils-1.7.0.dev174.dist-info → c2cciutils-1.8.0.dev45.dist-info}/WHEEL

@@ -1,4 +1,4 @@
 Wheel-Version: 1.0
-Generator: poetry-core 1.2.0
+Generator: poetry-core 1.9.1
 Root-Is-Purelib: true
 Tag: py3-none-any

{c2cciutils-1.7.0.dev174.dist-info → c2cciutils-1.8.0.dev45.dist-info}/entry_points.txt

@@ -1,11 +1,9 @@
 [console_scripts]
 c2cciutils=c2cciutils.scripts.main:main
-c2cciutils-audit=c2cciutils.scripts.audit:main
 c2cciutils-checks=c2cciutils.scripts.env:main
 c2cciutils-clean=c2cciutils.scripts.clean:main
 c2cciutils-docker-logs=c2cciutils.scripts.docker_logs:main
 c2cciutils-docker-versions-gen=c2cciutils.scripts.docker_versions_gen:main
-c2cciutils-docker-versions-update=c2cciutils.scripts.docker_versions_update:main
 c2cciutils-download-applications=c2cciutils.scripts.download_applications:main
 c2cciutils-env=c2cciutils.scripts.env:main
 c2cciutils-google-calendar=c2cciutils.publish:main_calendar
@@ -15,7 +13,6 @@ c2cciutils-k8s-logs=c2cciutils.scripts.k8s.logs:main
 c2cciutils-k8s-wait=c2cciutils.scripts.k8s.wait:main
 c2cciutils-pin-pipenv=c2cciutils.scripts.pin_pipenv:main
 c2cciutils-publish=c2cciutils.scripts.publish:main
-c2cciutils-pull-request-checks=c2cciutils.scripts.pr_checks:main
 c2cciutils-trigger-image-update=c2cciutils.scripts.trigger_image_update:main
 c2cciutils-version=c2cciutils.scripts.version:main
 

c2cciutils/audit.py

@@ -1,229 +0,0 @@
-"""
-The auditing functions.
-"""
-
-import datetime
-import json
-import os.path
-import subprocess  # nosec
-import sys
-from argparse import Namespace
-
-import c2cciutils
-import c2cciutils.configuration
-import c2cciutils.security
-
-
-def print_versions(
-    config: c2cciutils.configuration.PrintVersions,
-    full_config: c2cciutils.configuration.Configuration,
-    args: Namespace,
-) -> bool:
-    """
-    Print the versions.
-
-    Arguments:
-        config: The audit section config
-        full_config: All the CI config
-        args: The parsed command arguments
-    """
-    del full_config, args
-
-    print("::group::Versions")
-    c2cciutils.print_versions(config)
-    print("::endgroup::")
-
-    return True
-
-
-def snyk(
-    config: c2cciutils.configuration.AuditSnykConfig,
-    full_config: c2cciutils.configuration.Configuration,
-    args: Namespace,
-) -> bool:
-    """
-    Audit the code with Snyk.
-    """
-    del full_config
-
-    one_done = False
-    install_success = True
-    test_success = True
-
-    for file in (
-        subprocess.run(
-            ["git", "ls-files", "requirements.txt", "*/requirements.txt"], stdout=subprocess.PIPE, check=True
-        )
-        .stdout.decode()
-        .strip()
-        .split("\n")
-    ):
-        if not file:
-            continue
-        if file in config.get(
-            "files_no_install", c2cciutils.configuration.AUDIT_SNYK_FILES_NO_INSTALL_DEFAULT
-        ):
-            continue
-        print(f"::notice::Install from: {file}")
-        if not one_done:
-            print("::group::Install dependencies")
-            one_done = True
-        sys.stdout.flush()
-        sys.stderr.flush()
-        proc = subprocess.run(  # pylint: disable=subprocess-run-check
-            [
-                "pip",
-                "install",
-                *config.get(
-                    "pip_install_arguments", c2cciutils.configuration.AUDIT_SNYK_PIP_INSTALL_ARGUMENTS_DEFAULT
-                ),
-                f"--requirement={file}",
-            ]
-        )
-        if proc.returncode != 0:
-            print(f"::error::With error from: {file}")
-        install_success &= proc.returncode == 0
-
-    for file in (
-        subprocess.run(["git", "ls-files", "Pipfile", "*/Pipfile"], stdout=subprocess.PIPE, check=True)
-        .stdout.decode()
-        .strip()
-        .split("\n")
-    ):
-        if not file:
-            continue
-        if file in config.get(
-            "files_no_install", c2cciutils.configuration.AUDIT_SNYK_FILES_NO_INSTALL_DEFAULT
-        ):
-            continue
-        if not one_done:
-            print("::group::Install dependencies")
-            one_done = True
-        print(f"::notice::Install from: {file}")
-        directory = os.path.dirname(os.path.abspath(file))
-
-        sys.stdout.flush()
-        sys.stderr.flush()
-        proc = subprocess.run(  # pylint: disable=subprocess-run-check
-            [
-                "pipenv",
-                "sync",
-                *config.get(
-                    "pipenv_sync_arguments", c2cciutils.configuration.AUDIT_SNYK_PIPENV_SYNC_ARGUMENTS_DEFAULT
-                ),
-            ],
-            cwd=directory,
-        )
-        if proc.returncode != 0:
-            print(f"::error::With error from: {file}")
-        install_success &= proc.returncode == 0
-
-    if one_done:
-        print("::endgroup::")
-    if not install_success:
-        print("::error::Error while installing the dependencies")
-
-    snyk_exec, env = c2cciutils.snyk_exec()
-    if not args.fix:
-        command = [snyk_exec, "monitor", f"--target-reference={args.branch}"] + config.get(
-            "monitor_arguments", c2cciutils.configuration.AUDIT_SNYK_MONITOR_ARGUMENTS_DEFAULT
-        )
-        print(f"::group::Run: {' '.join(command)}")
-        sys.stdout.flush()
-        sys.stderr.flush()
-        subprocess.run(command, env=env)  # pylint: disable=subprocess-run-check
-        print("::endgroup::")
-
-        command = [snyk_exec, "test"] + config.get(
-            "test_arguments", c2cciutils.configuration.AUDIT_SNYK_TEST_ARGUMENTS_DEFAULT
-        )
-        print(f"::group::Run: {' '.join(command)}")
-        sys.stdout.flush()
-        sys.stderr.flush()
-        test_proc = subprocess.run(command, env=env)  # pylint: disable=subprocess-run-check
-        print("::endgroup::")
-        if test_proc.returncode != 0:
-            test_success = False
-            print("::error::With error")
-
-        # Clean all the changes to isolate the fix diff
-        subprocess.run(["git", "reset", "--hard"], check=True)
-
-    command = [snyk_exec, "fix"] + config.get(
-        "fix_arguments", c2cciutils.configuration.AUDIT_SNYK_FIX_ARGUMENTS_DEFAULT
-    )
-    print(f"::group::Run: {' '.join(command)}")
-    sys.stdout.flush()
-    sys.stderr.flush()
-    snyk_fix_proc = subprocess.run(  # pylint: disable=subprocess-run-check
-        command, env={**env, "FORCE_COLOR": "false"}, stdout=subprocess.PIPE, encoding="utf-8"
-    )
-    snyk_fix_message = snyk_fix_proc.stdout.strip()
-    print("::endgroup::")
-
-    if not args.fix:
-        current_branch = c2cciutils.get_branch(args.branch)
-        fix_github_create_pull_request_arguments = config.get(
-            "fix_github_create_pull_request_arguments",
-            c2cciutils.configuration.AUDIT_SNYK_FIX_PULL_REQUEST_ARGUMENTS_DEFAULT,
-        )
-        has_diff = c2cciutils.create_pull_request_if_needed(
-            current_branch,
-            f"snyk-fix/{current_branch}",
-            "Snyk auto fix",
-            [f"--body={snyk_fix_message}", *fix_github_create_pull_request_arguments],
-        )
-
-    return install_success and test_success and not has_diff
-
-
-def outdated_versions(
-    config: None,
-    full_config: c2cciutils.configuration.Configuration,
-    args: Namespace,
-) -> bool:
-    """
-    Check that the versions from the SECURITY.md are not outdated.
-
-    Arguments:
-        config: The audit section config
-        full_config: All the CI config
-        args: The parsed command arguments
-    """
-    del config, full_config
-
-    repo = c2cciutils.get_repository().split("/")
-    json_response = c2cciutils.graphql(
-        "default_branch.graphql",
-        {"name": repo[1], "owner": repo[0]},
-    )
-
-    if "errors" in json_response:
-        raise RuntimeError(json.dumps(json_response["errors"], indent=2))
-    if json_response["repository"]["defaultBranchRef"]["name"] != c2cciutils.get_branch(args.branch):
-        return True
-
-    success = True
-
-    if not os.path.exists("SECURITY.md"):
-        return True
-
-    with open("SECURITY.md", encoding="utf-8") as security_file:
-        security = c2cciutils.security.Security(security_file.read())
-
-    version_index = security.headers.index("Version")
-    date_index = security.headers.index("Supported Until")
-
-    for row in security.data:
-        str_date = row[date_index]
-        if str_date not in ("Unsupported", "Best effort", "To be defined"):
-            date = datetime.datetime.strptime(row[date_index], "%d/%m/%Y")
-            if date < datetime.datetime.now():
-                c2cciutils.error(
-                    "versions",
-                    f"The version '{row[version_index]}' is outdated, it can be set to "
-                    "'Unsupported', 'Best effort' or 'To be defined'",
-                    "SECURITY.md",
-                )
-                success = False
-    return success