continuous-delivery-scripts 4.1.0__tar.gz → 4.3.0__tar.gz

{continuous_delivery_scripts-4.1.0/continuous_delivery_scripts.egg-info → continuous_delivery_scripts-4.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: continuous-delivery-scripts
-Version: 4.1.0
+Version: 4.3.0
 Summary: Git-based CI/CD tools for releases, changelogs, SPDX SBOMs and licence reporting
 Home-page: https://github.com/ARMmbed/continuous-delivery-scripts
 Author: Arm
@@ -239,17 +239,18 @@ same commands whichever CI system the team chooses.
 
 | Command | What it does | Key arguments | Typical environment |
 | --- | --- | --- | --- |
-| `cd-assert-news` | Validates a branch's news files; can add a missing file for a configured dependency-update branch. | `--current-branch` (`-b`), `--local` (`-l`) | CI; `--local` for developer checks |
-| `cd-create-news-file` | Creates a one-line news fragment in the configured news directory. | Required news text; `--type` (`-t`), `--ref-number` (`-n`) | Developer |
-| `cd-determine-version` | Calculates and prints the **project's prospective release version** without generating a changelog. | Required `--release-type` (`-t`) | Developer preview or CI |
-| `cd-generate-news` | Updates the project version and, for beta or production releases, builds the changelog from news fragments; prints the resulting version. | Required `--release-type` (`-t`) | CI release flow |
-| `cd-get-config` | Prints a project configuration value. | Either `--key` (`-k`) or `--config-variable` (`-c`) | Developer or CI |
-| `cd-tag-and-release` | Runs the release flow, including documentation, licensing summaries, Git tagging, packaging and publication through the selected [language plugin](./continuous_delivery_scripts/plugins). | Required `--release-type` (`-t`); optional `--current-branch` (`-b`) | CI release flow |
-| `cd-generate-docs` | Generates code documentation in the configured output directory (or one specified on the command line). | `--output_directory` | Developer preview or CI |
-| `cd-generate-spdx` | Generates SPDX documents and third-party licence summaries, then checks licence compliance when project metadata is available. | Required `--output-dir` (`-o`); create the directory first | CI audit; developer review also possible |
-| `cd-license-files` | Adds or updates source-file licence and copyright headers when the language plugin supports them. | `--verbose` (`-v`) | Developer or CI |
-| `cd-record-secrets` | Records accepted findings in the project's [detect-secrets](https://github.com/Yelp/detect-secrets) registry. | `--registry-file` (`-r`); defaults to the configured registry | Developer (registry maintenance) |
-| `cd-detect-secrets` | Checks Git-tracked files against that registry and fails if new secrets are found. | `--registry-file` (`-r`); defaults to the configured registry | CI; developer checks also possible |
+| [`cd-assert-news`](./guides/checking-news-fragments.md) | Validates a branch's news files; can add a missing file for a configured dependency-update branch. | `--current-branch` (`-b`), `--local` (`-l`) | CI; `--local` for developer checks |
+| [`cd-create-news-file`](./guides/creating-news-fragments.md) | Creates a one-line news fragment in the configured news directory. | Required news text; `--type` (`-t`), `--ref-number` (`-n`) | Developer |
+| [`cd-determine-version`](./guides/previewing-versions.md) | Calculates and prints the **project's prospective release version** without generating a changelog. | Required `--release-type` (`-t`) | Developer preview or CI |
+| [`cd-generate-news`](./guides/managing-changelogs.md) | Updates the project version and, for beta or production releases, builds the changelog from news fragments; prints the resulting version. | Required `--release-type` (`-t`) | CI release flow |
+| [`cd-get-config`](./guides/reading-project-configuration.md) | Prints a project configuration value. | Either `--key` (`-k`) or `--config-variable` (`-c`) | Developer or CI |
+| [`cd-tag-and-release`](./guides/automating-releases.md) | Runs the release flow, including documentation, licensing summaries, Git tagging, packaging and publication through the selected [language plugin](./continuous_delivery_scripts/plugins). | Required `--release-type` (`-t`); optional `--current-branch` (`-b`) | CI release flow |
+| [`cd-generate-docs`](./guides/generating-code-documentation.md) | Generates code documentation in the configured output directory (or one specified on the command line). | `--output_directory` | Developer preview or CI |
+| [`cd-generate-spdx`](./guides/generating-an-spdx-sbom.md) | Generates SPDX documents and third-party licence summaries, then checks licence compliance when project metadata is available. | Required `--output-dir` (`-o`); create the directory first | CI audit; developer review also possible |
+| [`cd-check-licence-compliance`](./guides/checking-licence-compliance.md) | Checks project and dependency licences without creating SPDX documents; optionally writes third-party IP summaries. | Optional `--output-dir` (`-o`) for reports | CI gate or developer review |
+| [`cd-license-files`](./guides/licence-header-management.md) | Adds or updates source-file licence and copyright headers when the language plugin supports them. | `--verbose` (`-v`) | Developer or CI |
+| [`cd-record-secrets`](./guides/recording-secrets.md) | Records accepted findings in the project's [detect-secrets](https://github.com/Yelp/detect-secrets) registry. | `--registry-file` (`-r`); defaults to the configured registry | Developer (registry maintenance) |
+| [`cd-detect-secrets`](./guides/checking-for-secrets.md) | Checks Git-tracked files against that registry and fails if new secrets are found. | `--registry-file` (`-r`); defaults to the configured registry | CI; developer checks also possible |
 
 For example, `cd-create-news-file "Fix dependency resolution" --type bugfix`
 adds a news fragment. For release workflow details, see the

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/README.md

@@ -178,17 +178,18 @@ same commands whichever CI system the team chooses.
 
 | Command | What it does | Key arguments | Typical environment |
 | --- | --- | --- | --- |
-| `cd-assert-news` | Validates a branch's news files; can add a missing file for a configured dependency-update branch. | `--current-branch` (`-b`), `--local` (`-l`) | CI; `--local` for developer checks |
-| `cd-create-news-file` | Creates a one-line news fragment in the configured news directory. | Required news text; `--type` (`-t`), `--ref-number` (`-n`) | Developer |
-| `cd-determine-version` | Calculates and prints the **project's prospective release version** without generating a changelog. | Required `--release-type` (`-t`) | Developer preview or CI |
-| `cd-generate-news` | Updates the project version and, for beta or production releases, builds the changelog from news fragments; prints the resulting version. | Required `--release-type` (`-t`) | CI release flow |
-| `cd-get-config` | Prints a project configuration value. | Either `--key` (`-k`) or `--config-variable` (`-c`) | Developer or CI |
-| `cd-tag-and-release` | Runs the release flow, including documentation, licensing summaries, Git tagging, packaging and publication through the selected [language plugin](./continuous_delivery_scripts/plugins). | Required `--release-type` (`-t`); optional `--current-branch` (`-b`) | CI release flow |
-| `cd-generate-docs` | Generates code documentation in the configured output directory (or one specified on the command line). | `--output_directory` | Developer preview or CI |
-| `cd-generate-spdx` | Generates SPDX documents and third-party licence summaries, then checks licence compliance when project metadata is available. | Required `--output-dir` (`-o`); create the directory first | CI audit; developer review also possible |
-| `cd-license-files` | Adds or updates source-file licence and copyright headers when the language plugin supports them. | `--verbose` (`-v`) | Developer or CI |
-| `cd-record-secrets` | Records accepted findings in the project's [detect-secrets](https://github.com/Yelp/detect-secrets) registry. | `--registry-file` (`-r`); defaults to the configured registry | Developer (registry maintenance) |
-| `cd-detect-secrets` | Checks Git-tracked files against that registry and fails if new secrets are found. | `--registry-file` (`-r`); defaults to the configured registry | CI; developer checks also possible |
+| [`cd-assert-news`](./guides/checking-news-fragments.md) | Validates a branch's news files; can add a missing file for a configured dependency-update branch. | `--current-branch` (`-b`), `--local` (`-l`) | CI; `--local` for developer checks |
+| [`cd-create-news-file`](./guides/creating-news-fragments.md) | Creates a one-line news fragment in the configured news directory. | Required news text; `--type` (`-t`), `--ref-number` (`-n`) | Developer |
+| [`cd-determine-version`](./guides/previewing-versions.md) | Calculates and prints the **project's prospective release version** without generating a changelog. | Required `--release-type` (`-t`) | Developer preview or CI |
+| [`cd-generate-news`](./guides/managing-changelogs.md) | Updates the project version and, for beta or production releases, builds the changelog from news fragments; prints the resulting version. | Required `--release-type` (`-t`) | CI release flow |
+| [`cd-get-config`](./guides/reading-project-configuration.md) | Prints a project configuration value. | Either `--key` (`-k`) or `--config-variable` (`-c`) | Developer or CI |
+| [`cd-tag-and-release`](./guides/automating-releases.md) | Runs the release flow, including documentation, licensing summaries, Git tagging, packaging and publication through the selected [language plugin](./continuous_delivery_scripts/plugins). | Required `--release-type` (`-t`); optional `--current-branch` (`-b`) | CI release flow |
+| [`cd-generate-docs`](./guides/generating-code-documentation.md) | Generates code documentation in the configured output directory (or one specified on the command line). | `--output_directory` | Developer preview or CI |
+| [`cd-generate-spdx`](./guides/generating-an-spdx-sbom.md) | Generates SPDX documents and third-party licence summaries, then checks licence compliance when project metadata is available. | Required `--output-dir` (`-o`); create the directory first | CI audit; developer review also possible |
+| [`cd-check-licence-compliance`](./guides/checking-licence-compliance.md) | Checks project and dependency licences without creating SPDX documents; optionally writes third-party IP summaries. | Optional `--output-dir` (`-o`) for reports | CI gate or developer review |
+| [`cd-license-files`](./guides/licence-header-management.md) | Adds or updates source-file licence and copyright headers when the language plugin supports them. | `--verbose` (`-v`) | Developer or CI |
+| [`cd-record-secrets`](./guides/recording-secrets.md) | Records accepted findings in the project's [detect-secrets](https://github.com/Yelp/detect-secrets) registry. | `--registry-file` (`-r`); defaults to the configured registry | Developer (registry maintenance) |
+| [`cd-detect-secrets`](./guides/checking-for-secrets.md) | Checks Git-tracked files against that registry and fails if new secrets are found. | `--registry-file` (`-r`); defaults to the configured registry | CI; developer checks also possible |
 
 For example, `cd-create-news-file "Fix dependency resolution" --type bugfix`
 adds a news fragment. For release workflow details, see the

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/continuous_delivery_scripts/_version.py

@@ -11,8 +11,8 @@ References
 This file is autogenerated, do not modify by hand.
 """
 
-__version__ = "4.1.0"
-COMMIT = "298fb1cfdf2636976e5ea78d65207793279edcca"
+__version__ = "4.3.0"
+COMMIT = "3e8ffc7bd8472928ca195f17fb278d40476ef8a2"
 MAJOR = 4
-MINOR = 1
+MINOR = 3
 PATCH = 0

continuous_delivery_scripts-4.3.0/continuous_delivery_scripts/check_licence_compliance.py

@@ -0,0 +1,48 @@
+#
+# Copyright (C) 2020-2026 Arm Limited or its affiliates and Contributors. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+"""Check project and dependency licences without generating SPDX documents."""
+
+import argparse
+import logging
+import sys
+from pathlib import Path
+
+from continuous_delivery_scripts.language_specifics import get_language_specifics
+from continuous_delivery_scripts.utils.logging import log_exception, set_log_level
+
+logger = logging.getLogger(__name__)
+
+
+def main() -> int:
+    """Check licence compliance and optionally write third-party IP summaries."""
+    parser = argparse.ArgumentParser(description="Check project and dependency licence compliance.")
+    parser.add_argument(
+        "-o",
+        "--output-dir",
+        type=Path,
+        help="Write third-party IP summaries to this existing directory (no SPDX documents).",
+    )
+    parser.add_argument("-v", "--verbose", action="count", default=0, help="Increase logging verbosity.")
+    args = parser.parse_args()
+    set_log_level(args.verbose)
+
+    try:
+        plugin = get_language_specifics()
+        if not plugin.can_get_project_metadata():
+            raise ValueError("The selected language plugin cannot provide metadata for licence compliance checks.")
+        project = plugin.get_current_spdx_project()
+        if project is None:
+            raise ValueError("The selected language plugin did not return a project for licence compliance checks.")
+        if args.output_dir is not None:
+            project.generate_licensing_summary(args.output_dir)
+        project.check_licence_compliance()
+        return 0
+    except Exception as error:
+        log_exception(logger, error)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/continuous_delivery_scripts/generate_docs.py

@@ -93,7 +93,7 @@ def _guide_settings() -> tuple[Path, Path, Path, str] | None:
 def _render_guide(source: Path, destination: Path, project_name: str, nav_prefix: str, has_api_index: bool) -> str:
     """Render one Markdown guide to an HTML page and return its title."""
     text = source.read_text(encoding="utf8")
-    title = text.splitlines()[0].removeprefix("# ").strip()
+    title = next((line[2:].strip() for line in text.splitlines() if line.startswith("# ")), source.stem)
     body: str = markdown.markdown(text, extensions=["fenced_code", "tables"])
     body = re.sub(r'href="([a-z0-9-]+)\.md(#[^"]*)?"', _html_guide_link, body)
     description = f"{title} — {project_name}."
@@ -124,10 +124,13 @@ def _write_guide_index(
 ) -> None:
     """Write the project-specific landing page for the published guides."""
     guide_prefix = guide_folder.as_posix()
+    licence = configuration.get_value_or_default(ConfigurationVariable.FILE_LICENCE_IDENTIFIER, None)
+    licence_details = f"<p>Project licence: {escape(str(licence))}</p>" if licence else ""
     overview = (
         f"<h1>{escape(project_name)}</h1>"
         f"<p>Task guides for {escape(project_name)}, with setup, examples and project workflows.</p>"
-        f'<p><a href="{escape(guide_prefix)}/index.html">All task guides</a>'
+        + licence_details
+        + f'<p><a href="{escape(guide_prefix)}/index.html">All task guides</a>'
         + (' · <a href="api.html">API reference</a>' if has_api_index else "")
         + "</p><ul>"
         + "".join(links)

continuous_delivery_scripts-4.3.0/continuous_delivery_scripts/plugins/GOLAND.MD

@@ -0,0 +1,52 @@
+# Golang plugin
+
+This plugin is used for golang projects when `PROGRAMMING_LANGUAGE` is set to `golang`.
+
+In order to work, `go > 1.18` is required.
+
+## Release
+The plugin leverages [goreleaser](https://goreleaser.com/) to package and release projects.
+Therefore, a `.goreleaser.yml` file must be present in the go source directory i.e. where the mod file lies.
+
+## Documentation
+The plugin uses [doc2go](https://abhinav.github.io/doc2go) to generate the code reference documentation.
+
+## SPDX and third-party IP reporting
+
+The Go plugin uses [go-licenses v2](https://github.com/google/go-licenses) to
+discover the licences of packages imported by Go modules. It checks for a
+usable `go-licenses` binary on `PATH` before installing
+`github.com/google/go-licenses/v2@latest` with `go install`. The Go toolchain
+must be available; an installed binary can also be located under `GOBIN` or
+the first `GOPATH` entry's `bin` directory.
+
+Set `PROGRAMMING_LANGUAGE = "golang"`, `PROJECT_NAME`, `PROJECT_ROOT` and
+`SOURCE_DIR` in `[ProjectConfig]`. Put a `go.mod` under the source or project
+root; `go.work` modules and nested Go modules are also considered. Supply
+`FILE_LICENCE_IDENTIFIER` for the project licence and set `PROJECT_UUID`
+and `[spdx]` namespace fields as explained in the
+[SPDX configuration guide](../../DEVELOPMENT.md#spdx-document-identity).
+
+With the project and its Go dependencies available, run:
+
+```bash
+mkdir -p licensing
+cd-generate-spdx --output-dir licensing
+```
+
+For each module, the plugin runs `go-licenses report ./...` with a template
+to collect dependency package names, versions, licence URLs and SPDX licence
+names. The [go-licenses custom-template documentation](https://github.com/google/go-licenses/blob/master/README.md#reports-with-custom-templates)
+defines the `Name`, `Version`, `LicenseURL` and `LicenseName` fields used by the
+plugin; the default CSV report has no `Version` field. The metadata then enters
+the **shared** SPDX writer, HTML/CSV/text/JSON
+third-party IP report and accepted-licence policy used by other plugins.
+To include the shared licence summary in release documentation, set
+`GENERATE_LICENSING_SUMMARY_ON_RELEASE = true` in `[ProjectConfig]` (the default
+is `false`).
+The Go tool's own `check` policy is not used; configure
+`ACCEPTED_THIRD_PARTY_LICENCES` and documented exceptions in
+`PACKAGES_WITH_CHECKED_LICENCE` in `pyproject.toml` instead. Unknown licences
+remain unknown so that the shared audit can flag them. The report reflects
+dependencies built for the current Go environment and excludes test-only
+imports unless the Go tool is invoked separately with `--include_tests`.

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/continuous_delivery_scripts/plugins/golang.py

@@ -4,19 +4,23 @@
 #
 """Plugin for Golang projects."""
 
+import csv
+import io
 import json
 import logging
 import os
 import shutil
 from pathlib import Path
 from subprocess import check_call, check_output
-from typing import TYPE_CHECKING, Optional, List, Dict, MutableMapping
+from tempfile import TemporaryDirectory
+from typing import Optional, List, Dict, MutableMapping
 
+from continuous_delivery_scripts.spdx_report.spdx_project import SpdxProject
 from continuous_delivery_scripts.utils.configuration import (
     configuration,
     ConfigurationVariable,
 )
-from continuous_delivery_scripts.utils.definitions import CommitType
+from continuous_delivery_scripts.utils.definitions import CommitType, UNKNOWN
 from continuous_delivery_scripts.utils.git_helpers import (
     LocalProjectRepository,
     GitWrapper,
@@ -25,9 +29,12 @@ from continuous_delivery_scripts.utils.language_specifics_base import (
     BaseLanguage,
     get_language_from_file_name,
 )
-
-if TYPE_CHECKING:
-    from continuous_delivery_scripts.spdx_report.spdx_project import SpdxProject
+from continuous_delivery_scripts.utils.package_helpers import (
+    LicenceSource,
+    PackageMetadata,
+    ProjectMetadata,
+    ProjectMetadataFetcher,
+)
 
 logger = logging.getLogger(__name__)
 
@@ -37,6 +44,9 @@ ENVVAR_GORELEASER_GIT_TOKEN = "GITHUB_TOKEN"
 ENVVAR_GORELEASER_CUSTOMISED_TAG = "GORELEASER_CURRENT_TAG"
 ENVVAR_GO_MOD = "GO111MODULE"
 GO_MOD_ON_VALUE = "on"
+# go-licenses report --template receives Name, Version, LicenseURL and LicenseName for each library.
+# https://github.com/google/go-licenses/blob/master/README.md#reports-with-custom-templates
+GO_LICENSES_TEMPLATE = "{{range .}}{{.Name}}\t{{.Version}}\t{{.LicenseURL}}\t{{.LicenseName}}\n{{end}}"
 
 
 def _generate_doc2go_command_list(output_directory: Path, module: str) -> List[str]:
@@ -81,24 +91,147 @@ def _install_goreleaser_command_list() -> List[str]:
     return ["go", "install", "github.com/goreleaser/goreleaser/v2@latest"]
 
 
+def _install_go_licenses_command_list() -> List[str]:
+    return ["go", "install", "github.com/google/go-licenses/v2@latest"]
+
+
+def _installed_go_tool_path(tool_name: str, env: MutableMapping[str, str]) -> str:
+    """Find a binary installed using go install, even when Go's bin directory is not on PATH."""
+    go_bin = check_output(["go", "env", "GOBIN"], env=env, encoding="utf8").strip()
+    if not go_bin:
+        go_path = check_output(["go", "env", "GOPATH"], env=env, encoding="utf8").strip()
+        go_bin = str(Path(go_path.split(os.pathsep)[0]) / "bin")
+    executable = Path(go_bin) / (f"{tool_name}.exe" if os.name == "nt" else tool_name)
+    if not executable.is_file():
+        raise FileNotFoundError(f"Could not find installed {tool_name} binary: {executable}")
+    return str(executable)
+
+
+def _candidate_go_module_directories(include_project_root: bool = False) -> List[Path]:
+    """Share module selection between release tags and licence reporting."""
+    candidates = [SRC_DIR]
+    if include_project_root:
+        candidates.append(ROOT_DIR)
+    workspace_modules = _determine_go_work_module_directories()
+    candidates.extend(workspace_modules if workspace_modules else _determine_go_subproject_directories())
+    return list(dict.fromkeys(candidates))
+
+
+def _go_licence_module_directories() -> List[Path]:
+    """Find actual Go modules for dependency reporting."""
+    modules = [
+        path for path in _candidate_go_module_directories(include_project_root=True) if (path / "go.mod").is_file()
+    ]
+    if not modules:
+        raise FileNotFoundError(f"No go.mod found in {SRC_DIR}, {ROOT_DIR} or their Go workspace modules.")
+    return modules
+
+
+def _go_module_name(module_dir: Path, env: MutableMapping[str, str]) -> str:
+    """Read the import path declared by a Go module."""
+    module = json.loads(check_output(["go", "list", "-m", "-json"], cwd=module_dir, env=env, encoding="utf8"))
+    return str(module["Path"])
+
+
+def _parse_go_licences(output: str) -> List[PackageMetadata]:
+    """Translate go-licenses template output into shared package metadata."""
+    packages = []
+    for row in csv.reader(io.StringIO(output), delimiter="\t"):
+        if len(row) != 4 or not row[0].strip():
+            raise ValueError(f"Unexpected go-licenses report row: {row}")
+        name, version, licence_url, licence = (item.strip() for item in row)
+        known_licence = bool(licence and licence.lower() != UNKNOWN)
+        evidence = [{"kind": "licence", "path": licence_url, "text": ""}] if licence_url.startswith("https://") else []
+        packages.append(
+            PackageMetadata.from_fields(
+                name=name,
+                version=version or UNKNOWN,
+                licence=licence if known_licence else UNKNOWN,
+                licence_source=LicenceSource.TOOL if known_licence else LicenceSource.UNKNOWN,
+                declared_licence=licence or UNKNOWN,
+                url=licence_url or UNKNOWN,
+                licence_evidence=evidence,
+            )
+        )
+    return packages
+
+
+class GoProjectMetadataFetcher(ProjectMetadataFetcher):
+    """Retrieve Go dependency licences for the shared SPDX and compliance report."""
+
+    def __init__(self) -> None:
+        """Initialise with the configured project name."""
+        super().__init__(str(configuration.get_value(ConfigurationVariable.PROJECT_NAME)))
+
+    def fetch_project_metadata(self) -> ProjectMetadata:
+        """Collect module dependencies using go-licenses without replacing shared policy checks."""
+        module_directories = _go_licence_module_directories()
+        env = os.environ.copy()
+        env[ENVVAR_GO_MOD] = GO_MOD_ON_VALUE
+        executable = _ensure_go_tool_installed(
+            tool_name="go-licenses",
+            version_command=["go-licenses", "report", "--help"],
+            install_command=_install_go_licenses_command_list(),
+            env=env,
+        )
+        project = ProjectMetadata(self._package_name)
+        project.project_metadata = PackageMetadata.from_fields(
+            name=self._package_name,
+            licence=str(configuration.get_value(ConfigurationVariable.FILE_LICENCE_IDENTIFIER)),
+            licence_source=LicenceSource.CONFIGURATION,
+        )
+        dependencies: Dict[str, PackageMetadata] = {}
+        modules = [(module_dir, _go_module_name(module_dir, env)) for module_dir in module_directories]
+        with TemporaryDirectory() as temporary_dir:
+            template_path = Path(temporary_dir) / "go-licenses.tpl"
+            template_path.write_text(GO_LICENSES_TEMPLATE, encoding="utf8")
+            for module_dir, _ in modules:
+                output = check_output(
+                    [executable, "report", "./...", "--template", str(template_path)],
+                    cwd=module_dir,
+                    env=env,
+                    encoding="utf8",
+                )
+                for package in _parse_go_licences(output):
+                    if any(package.name == name or package.name.startswith(f"{name}/") for _, name in modules):
+                        continue
+                    existing = dependencies.get(package.name)
+                    if existing and (existing.version, existing.licence) != (package.version, package.licence):
+                        candidates = existing.licence_candidates or [existing.licence]
+                        dependencies[package.name] = PackageMetadata.from_fields(
+                            name=package.name,
+                            version=UNKNOWN if existing.version != package.version else package.version,
+                            licence=UNKNOWN,
+                            licence_source=LicenceSource.UNKNOWN,
+                            licence_candidates=sorted(set(candidates + [package.licence])),
+                            licence_evidence=existing.licence_evidence + package.licence_evidence,
+                        )
+                    elif existing is None:
+                        dependencies[package.name] = package
+        for dependency in dependencies.values():
+            project.add_dependency_metadata(dependency)
+        return project
+
+
 def _ensure_go_tool_installed(
     tool_name: str,
     version_command: List[str],
     install_command: List[str],
     env: MutableMapping[str, str],
-) -> None:
-    """Ensure a Go-based tool is available on PATH before use."""
+) -> str:
+    """Return a usable Go tool on PATH or install it into the Go binary directory."""
     tool_path = shutil.which(tool_name)
     if tool_path:
         try:
             check_output(version_command, env=env)
             logger.info("Using %s from PATH: %s", tool_name, tool_path)
-            return
+            return tool_path
         except Exception as exception:
             logger.warning("Could not use %s from PATH: %s", tool_name, exception)
 
     logger.info("Installing %s with go install.", tool_name)
     check_call(install_command, env=env)
+    return _installed_go_tool_path(tool_name, env)
 
 
 def _call_doc2go(output_directory: Path, module: str) -> None:
@@ -212,15 +345,9 @@ def _determine_go_module_tag(version: str) -> List[str]:
     https://go.dev/ref/mod#workspaces, and
     https://github.com/golang/go/wiki/Modules/a549b3e4b7ad6be6e7d11c37ef247bb2279c8146#faqs--multi-module-repositories.
     """
-    module_directories = [SRC_DIR]
-    go_work_module_directories = _determine_go_work_module_directories()
-    if go_work_module_directories:
-        module_directories.extend(go_work_module_directories)
-    else:
-        module_directories.extend(_determine_go_subproject_directories())
-
     tags = [
-        _determine_go_module_tag_for_directory(module_directory, version) for module_directory in module_directories
+        _determine_go_module_tag_for_directory(module_directory, version)
+        for module_directory in _candidate_go_module_directories()
     ]
     return list(dict.fromkeys([tag for tag in tags if tag]))
 
@@ -263,7 +390,7 @@ class Go(BaseLanguage):
 
     def can_get_project_metadata(self) -> bool:
         """States whether project metadata can be retrieved."""
-        return False
+        return True
 
     def get_secret_registry_exclude_files(self) -> List[str]:
         """Gets additional detect-secrets exclude patterns for Go projects."""
@@ -274,10 +401,9 @@ class Go(BaseLanguage):
             (r"^\.github[\\/]workflows[\\/].*"),
         ]
 
-    def get_current_spdx_project(self) -> Optional["SpdxProject"]:
+    def get_current_spdx_project(self) -> Optional[SpdxProject]:
         """Gets current SPDX description."""
-        # TODO
-        return None
+        return SpdxProject(GoProjectMetadataFetcher())
 
     def should_clean_before_packaging(self) -> bool:
         """States whether the repository must be cleaned before packaging happens."""

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/continuous_delivery_scripts/spdx_report/spdx_document.py

@@ -6,6 +6,7 @@
 
 from pathlib import Path
 from typing import TYPE_CHECKING, List, Optional
+from urllib.parse import urlsplit
 
 from continuous_delivery_scripts.spdx_report.spdx_dependency import (
     DependencySpdxDocumentRef,
@@ -23,7 +24,8 @@ from continuous_delivery_scripts.utils.configuration import (
     ConfigurationVariable,
 )
 from continuous_delivery_scripts.utils.hash_helpers import generate_uuid_based_on_str
-from continuous_delivery_scripts.utils.python.package_helpers import PackageMetadata
+from continuous_delivery_scripts.utils.package_helpers import PackageMetadata
+from continuous_delivery_scripts.utils.third_party_licences import LICENSE_REF_PROPRIETARY
 
 if TYPE_CHECKING:
     from spdx.document import Document
@@ -230,7 +232,7 @@ class SpdxDocument:
             the corresponding document
         """
         from spdx.creationinfo import Person, Organization, Tool
-        from spdx.document import Document, License
+        from spdx.document import Document, ExtractedLicense, License
         from spdx.review import Review
         from spdx.version import Version
 
@@ -240,7 +242,7 @@ class SpdxDocument:
         doc.namespace = determine_spdx_value(self.document_namespace)
         doc.spdx_id = "SPDXRef-DOCUMENT"
         doc.comment = determine_spdx_value(
-            "This document was created automatically using available information from python packages."
+            "This document was created automatically using available project and dependency information."
         )
         doc.data_license = License.from_identifier("CC0-1.0")
         doc.creation_info.add_creator(Person(self.author, self.author_email))
@@ -262,7 +264,35 @@ class SpdxDocument:
         #  be described in a file and hence, all dependencies are described
         #  in separate files. Find out what to do with dependencies when new
         #  tools are released as it is not entirely clear in the specification
-        doc.package = self.generate_spdx_package().generate_spdx_package()
+        spdx_package = self.generate_spdx_package()
+        doc.package = spdx_package.generate_spdx_package()
+        if LICENSE_REF_PROPRIETARY in spdx_package.main_licence or LICENSE_REF_PROPRIETARY in spdx_package.licence:
+            proprietary_licence = ExtractedLicense(LICENSE_REF_PROPRIETARY)
+            proprietary_licence.full_name = "Proprietary licence"
+            proprietary_licence.text = "Proprietary licence terms are not included in this SPDX document."
+            proprietary_licence.comment = (
+                "This is a locally defined licence reference, not an SPDX License List identifier. "
+                "See https://spdx.github.io/spdx-spec/v2.3/other-licensing-information-detected/ "
+                "for the LicenseRef format; consult the rights holder for the licence terms."
+            )
+            if LICENSE_REF_PROPRIETARY in spdx_package.main_licence:
+                licence_urls = {
+                    evidence.get("path", "")
+                    for evidence in self._package_metadata.licence_evidence
+                    if evidence.get("kind") == "licence"
+                }
+                for url in sorted(licence_urls):
+                    try:
+                        parsed_url = urlsplit(url)
+                    except ValueError:
+                        continue
+                    if (
+                        parsed_url.scheme in ("http", "https")
+                        and parsed_url.netloc
+                        and not any(c.isspace() for c in url)
+                    ):
+                        proprietary_licence.add_xref(url)
+            doc.add_extr_lic(proprietary_licence)
 
         for external_reference in self.external_refs:
             doc.add_ext_document_reference(external_reference.generate_external_reference())

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/continuous_delivery_scripts/spdx_report/spdx_helpers.py

@@ -67,7 +67,7 @@ COPYRIGHT_PATTERN = r"Copyright.*$"
 COPYRIGHT_REGEX_PATTERN = re.compile(COPYRIGHT_PATTERN, flags=re.MULTILINE | re.IGNORECASE)
 # Specification of the identifier based on https://spdx.org/spdx-specification-21-web-version#h.twlc0ztnng3b
 # and https://spdx.org/ids-how
-SPDX_LICENCE_IDENTIFIER_PATTERN = r"SPDX-License-Identifier: ([\.\w+\-\(\)\s]+)[\*]?$"
+SPDX_LICENCE_IDENTIFIER_PATTERN = r"SPDX-License-Identifier: ([\.\w+\-\(\)\t ]+)[\*]?$"
 SPDX_IDENTIFIER_REGEX_PATTERN = re.compile(SPDX_LICENCE_IDENTIFIER_PATTERN, re.MULTILINE)
 THIRD_PARTY_CONFIG_NAMESPACE = "spdx"
 PATHS_TO_EXCLUDE = [
@@ -77,6 +77,7 @@ PATHS_TO_EXCLUDE = [
     "*.rdf",
     "*.pyc",
     "*.bin",
+    "*.exe",
     "*.tar",
     "*.zip",
     "**/__pycache__/**",

{continuous_delivery_scripts-4.1.0 → continuous_delivery_scripts-4.3.0}/continuous_delivery_scripts/spdx_report/spdx_package.py

@@ -6,18 +6,21 @@
 
 from dataclasses import dataclass
 from pathlib import Path
+import re
 from typing import TYPE_CHECKING, List, Optional
 
 from license_expression import ExpressionError
 
 from continuous_delivery_scripts.spdx_report.spdx_file import SpdxFile
+from continuous_delivery_scripts.utils.hash_helpers import generate_uuid_based_on_str
 from continuous_delivery_scripts.spdx_report.spdx_helpers import (
     determine_spdx_value,
     list_project_files_for_licensing,
 )
 from continuous_delivery_scripts.utils.definitions import UNKNOWN
-from continuous_delivery_scripts.utils.python.package_helpers import PackageMetadata
+from continuous_delivery_scripts.utils.package_helpers import PackageMetadata
 from continuous_delivery_scripts.utils.third_party_licences import (
+    LICENSE_REF_PROPRIETARY,
     UNKNOWN_LICENCE,
     cleanse_licence_expression,
     is_licence_accepted,
@@ -30,7 +33,7 @@ if TYPE_CHECKING:
 
 @dataclass(frozen=True, order=True)
 class PackageInfo:
-    """Definition of a Python package.
+    """Definition of a project or dependency package.
 
     Attributes:
         metadata: metadata about an installed package.
@@ -90,7 +93,9 @@ class SpdxPackage:
         Returns:
             An ID
         """
-        return self.name if self._is_dependency else self._package_info.uuid
+        if not self._is_dependency:
+            return self._package_info.uuid
+        return self.name if re.fullmatch(r"[A-Za-z0-9.-]+", self.name) else str(generate_uuid_based_on_str(self.name))
 
     @property
     def is_dependency(self) -> bool:
@@ -137,7 +142,14 @@ class SpdxPackage:
     @property
     def is_main_licence_accepted(self) -> bool:
         """States whether the main licence of the package is part of the accepted licence list."""
-        return bool(is_licence_accepted(self.main_licence))
+        return bool(is_licence_accepted(self.main_licence, self._project_licence_refs))
+
+    @property
+    def _project_licence_refs(self) -> List[str]:
+        """Allow the project's own proprietary licence without permitting it for dependencies."""
+        if not self._is_dependency and LICENSE_REF_PROPRIETARY in self.main_licence:
+            return [LICENSE_REF_PROPRIETARY]
+        return []
 
     @property
     def licence(self) -> str:
@@ -158,7 +170,7 @@ class SpdxPackage:
     @property
     def is_licence_accepted(self) -> bool:
         """States whether the actual package's licence of the package is part of the accepted licence list."""
-        return bool(is_licence_accepted(self.licence))
+        return bool(is_licence_accepted(self.licence, self._project_licence_refs))
 
     @property
     def author(self) -> str: