grz-cli 2.0.0__tar.gz → 3.0.0__tar.gz

{grz_cli-2.0.0 → grz_cli-3.0.0}/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Changelog
 
+## [3.0.0](https://github.com/BfArM-MVH/grz-tools/compare/grz-cli-v2.0.0...grz-cli-v3.0.0) (2026-09-26)
+
+
+### ⚠ BREAKING CHANGES
+
+* **grzctl:** the grzctl config has no `keys` section. Each inbox names its private key. An inbox's `private_key_passphrase` comes before `C4GH_PASSPHRASE`.
+* **grz-cli:** `Crypt4GH.prepare_c4gh_keys`, `Crypt4GH.decrypt_file`, `Submission.encrypt`, `EncryptedSubmission.decrypt`, `Worker.encrypt` and `Worker.decrypt` take `X25519PrivateKey` and `X25519PublicKey` objects instead of paths. The `Crypt4GH` key loaders return these objects.
+
+### Features
+
+* **grz-cli,grz-common,grz-db,grzctl:** record why a submission failed and classify S3 errors ([#690](https://github.com/BfArM-MVH/grz-tools/issues/690)) ([f1d3fc8](https://github.com/BfArM-MVH/grz-tools/commit/f1d3fc804387c4fb6de861cccb6a99b521ac0056))
+* **grz-cli:** accept the submitter private key inline ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+* **grz-cli:** add keys.submitter_private_key_passphrase ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+* **grz-common:** load crypt4gh keys in memory ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+* **grz-common:** recommend grz-cli 3.0.0 in the bundled version policy ([bf3e6bf](https://github.com/BfArM-MVH/grz-tools/commit/bf3e6bf4f2ed21cd1c7bdf4d6aa8710ece4f60d3))
+* **grzctl:** accept every crypt4gh key in the config inline ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+* **grzctl:** Centralized version.json and its distribution using grzctl ([#667](https://github.com/BfArM-MVH/grz-tools/issues/667)) ([6ed9cb0](https://github.com/BfArM-MVH/grz-tools/commit/6ed9cb04bede9fc9b34b7ce6d3aa9a0df9d42ae4))
+* **grzctl:** configure each crypt4gh key where grzctl uses it ([#694](https://github.com/BfArM-MVH/grz-tools/issues/694)) ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+
+
+### Bug Fixes
+
+* **grz-cli,grz-common,grz-db,grzctl:** require grz-common 4, grz-db 4 and grz-pydantic-models 4 ([#688](https://github.com/BfArM-MVH/grz-tools/issues/688)) ([bf3e6bf](https://github.com/BfArM-MVH/grz-tools/commit/bf3e6bf4f2ed21cd1c7bdf4d6aa8710ece4f60d3))
+* **grz-common:** keep crypt4gh private keys out of DEBUG logs ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+* **grz-common:** leave the raw input out of config validation errors ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+* **grzctl,grz-common,grz-cli:** drop grzctl submit and grzctl's grz-cli dependency ([#675](https://github.com/BfArM-MVH/grz-tools/issues/675)) ([e6f27c1](https://github.com/BfArM-MVH/grz-tools/commit/e6f27c1714fc73d5159b29356914bc345a797c0a))
+* **grzctl:** keep the stored upload date in db submission populate ([f1d3fc8](https://github.com/BfArM-MVH/grz-tools/commit/f1d3fc804387c4fb6de861cccb6a99b521ac0056))
+* **grzctl:** mask db.author.private_key in dump-config ([06f12e6](https://github.com/BfArM-MVH/grz-tools/commit/06f12e625878c87413fbc1afa77ce8708c03a3df))
+
 ## [2.0.0](https://github.com/BfArM-MVH/grz-tools/compare/grz-cli-v1.8.0...grz-cli-v2.0.0) (2026-08-13)
 
 

{grz_cli-2.0.0 → grz_cli-3.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: grz-cli
-Version: 2.0.0
+Version: 3.0.0
 Summary: Tool for validation, encryption and upload of MV submissions to GDCs.
 Project-URL: Homepage, https://github.com/BfArM-MVH/grz-tools
 Project-URL: Repository, https://github.com/BfArM-MVH/grz-tools
@@ -13,7 +13,7 @@ Keywords: GDC,GRZ,S3
 Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Requires-Python: >=3.12
-Requires-Dist: grz-common<4,>=3
+Requires-Dist: grz-common<5,>=4
 Requires-Dist: packaging<27,>=23
 Description-Content-Type: text/markdown
 

{grz_cli-2.0.0 → grz_cli-3.0.0}/pyproject.toml

@@ -14,7 +14,7 @@ authors = [
 dynamic = ["version"]
 dependencies = [
     "packaging >=23,<27",
-    "grz-common >=3,<4",
+    "grz-common >=4,<5",
 ]
 classifiers = [
     "Programming Language :: Python :: 3",
@@ -30,6 +30,7 @@ test = [
     "pytest-cov",
     "pytest-mock",
     "pytest-sugar",
+    "pytest-xdist",
 ]
 
 

grz_cli-3.0.0/src/grz_cli/__init__.py

@@ -0,0 +1 @@
+__version__ = "3.0.0"

{grz_cli-2.0.0 → grz_cli-3.0.0}/src/grz_cli/cli.py

@@ -11,6 +11,7 @@ from pathlib import Path
 from textwrap import dedent
 
 import click
+import grz_common.exceptions as grzexc
 import grz_pydantic_models.submission.metadata
 from grz_common.cli import config_file
 from grz_common.cli.dump_config import dump_config
@@ -95,7 +96,12 @@ def main():
     Main entry point for the CLI application.
     """
     cli = build_cli()
-    cli()
+    try:
+        cli()
+    except grzexc.GrzError as e:
+        # an expected failure, such as an invalid metadata.json: log its message, not a traceback
+        log.error(e)
+        sys.exit(1)
 
 
 if __name__ == "__main__":

{grz_cli-2.0.0 → grz_cli-3.0.0}/src/grz_cli/commands/encrypt.py

@@ -3,11 +3,12 @@
 import logging
 import sys
 from pathlib import Path
-from tempfile import NamedTemporaryFile
 from typing import Any
 
 import click
 import grz_common.cli as grzcli
+from grz_common.models.base import get_secret_value
+from grz_common.utils.crypt import Crypt4GH
 from grz_common.workers.worker import Worker
 
 from ..models.config import EncryptConfig
@@ -34,9 +35,26 @@ def encrypt(configuration: dict[str, Any], submission_dir, force, check_validati
     """
     config = EncryptConfig.model_validate(configuration)
 
-    submitter_privkey_path = config.keys.submitter_private_key_path
-    if submitter_privkey_path == "":
-        submitter_privkey_path = None
+    if config.keys.grz_public_key is not None:
+        grz_public_key = Crypt4GH.load_public_key(config.keys.grz_public_key, key_name="keys.grz_public_key")
+    elif config.keys.grz_public_key_path is not None:
+        grz_public_key = Crypt4GH.retrieve_public_key(config.keys.grz_public_key_path)
+    else:
+        # This case cannot occur here, but an explicit check is needed for type-checking.
+        sys.exit("Either keys.grz_public_key or keys.grz_public_key_path must be set for encryption.")
+
+    submitter_private_key = None
+    passphrase = get_secret_value(config.keys.submitter_private_key_passphrase)
+    if config.keys.submitter_private_key is not None:
+        submitter_private_key = Crypt4GH.load_private_key(
+            config.keys.submitter_private_key.get_secret_value(),
+            passphrase=passphrase,
+            key_name="keys.submitter_private_key",
+        )
+    elif config.keys.submitter_private_key_path is not None:
+        submitter_private_key = Crypt4GH.retrieve_private_key(
+            config.keys.submitter_private_key_path, passphrase=passphrase
+        )
 
     log.info("Starting encryption...")
 
@@ -48,25 +66,11 @@ def encrypt(configuration: dict[str, Any], submission_dir, force, check_validati
         log_dir=submission_dir / "logs",
         encrypted_files_dir=submission_dir / "encrypted_files",
     )
-    if pubkey := config.keys.grz_public_key:
-        with NamedTemporaryFile("w") as f:
-            f.write(pubkey)
-            f.flush()
-            worker_inst.encrypt(
-                f.name,
-                submitter_private_key_path=submitter_privkey_path,
-                force=force,
-                check_validation_logs=check_validation_logs,
-            )
-    else:
-        # This case cannot occur here, but an explicit check is needed for type-checking.
-        if config.keys.grz_public_key_path is None:
-            sys.exit("GRZ public key path is required for encryption.")
-        worker_inst.encrypt(
-            config.keys.grz_public_key_path,
-            submitter_private_key_path=submitter_privkey_path,
-            force=force,
-            check_validation_logs=check_validation_logs,
-        )
+    worker_inst.encrypt(
+        grz_public_key,
+        submitter_private_key=submitter_private_key,
+        force=force,
+        check_validation_logs=check_validation_logs,
+    )
 
     log.info("Encryption successful!")

grz_cli-3.0.0/src/grz_cli/models/config.py

@@ -0,0 +1,74 @@
+import logging
+from typing import Self
+
+from grz_common.models.base import Crypt4GHPublicKey, FilePath, IgnoringBaseModel
+from grz_common.models.identifiers import IdentifiersConfigModel
+from grz_common.models.s3 import S3ConfigModel
+from pydantic import ConfigDict, SecretStr, model_validator
+
+log = logging.getLogger(__name__)
+
+
+class KeyModel(IgnoringBaseModel):
+    grz_public_key: Crypt4GHPublicKey | None = None
+    """
+    The public key of the recipient (the associated GRZ).
+    """
+
+    grz_public_key_path: FilePath | None = None
+    """
+    Path to the crypt4gh public key of the recipient (the associated GRZ).
+    """
+
+    submitter_private_key: SecretStr | None = None
+    """
+    The submitter's crypt4gh private key (optional).
+    grz-cli signs the encrypted files with this key if it is set.
+    If neither this field nor ``submitter_private_key_path`` is set, grz-cli signs them with a random key.
+    """
+
+    submitter_private_key_path: FilePath | None = None
+    """
+    Path to the submitter's crypt4gh private key (optional).
+    grz-cli signs the encrypted files with this key if it is set.
+    If neither this field nor ``submitter_private_key`` is set, grz-cli signs them with a random key.
+    """
+
+    submitter_private_key_passphrase: SecretStr | None = None
+    """
+    Passphrase to the submitter's crypt4gh private key (optional).
+    grz-cli takes the passphrase from this field, else from ``C4GH_PASSPHRASE``, else from a prompt.
+    """
+
+    @model_validator(mode="after")
+    def validate_grz_public_key(self) -> Self:
+        if self.grz_public_key is None and self.grz_public_key_path is None:
+            raise ValueError("Either grz_public_key or grz_public_key_path must be set.")
+        if self.grz_public_key is not None and self.grz_public_key_path is not None:
+            raise ValueError("Only one of grz_public_key or grz_public_key_path must be set.")
+        return self
+
+    @model_validator(mode="after")
+    def validate_submitter_private_key(self) -> Self:
+        if self.submitter_private_key is not None and self.submitter_private_key_path is not None:
+            raise ValueError("Only one of submitter_private_key or submitter_private_key_path must be set.")
+        return self
+
+
+class KeyConfigModel(IgnoringBaseModel):
+    # errors would show the raw input, and SecretStr does not mask the submitter private key and its passphrase there
+    model_config = ConfigDict(hide_input_in_errors=True)
+
+    keys: KeyModel
+
+
+class UploadConfig(S3ConfigModel):
+    pass
+
+
+class EncryptConfig(KeyConfigModel):
+    pass
+
+
+class ValidateConfig(IdentifiersConfigModel):
+    pass

{grz_cli-2.0.0 → grz_cli-3.0.0}/src/grz_cli/utils/version_check.py

@@ -4,7 +4,7 @@ from datetime import UTC, datetime
 from importlib.metadata import version
 
 from grz_common.models.s3 import S3Options
-from grz_common.models.version import VersionFile, VersionInfo
+from grz_common.models.version import VERSION_FILE_KEY, VersionFile, VersionInfo
 from packaging import version as pkg_version
 
 logger = logging.getLogger(__name__)
@@ -37,7 +37,7 @@ def _select_active_policy(
 
 def check_version_and_exit_if_needed(
     s3_options: S3Options,
-    version_file_key: str = "version.json",
+    version_file_key: str = VERSION_FILE_KEY,
 ) -> None:
     """Validate the installed grz-cli version against the policy defined in version.json."""
     version_file = VersionFile.from_s3(s3_options, version_file_key)
@@ -46,18 +46,6 @@ def check_version_and_exit_if_needed(
     _check_policy_and_exit_if_needed("grz-cli", current_version, version_file.grzcli_version)
 
 
-def check_metadata_version_and_exit_if_needed(
-    s3_options: S3Options,
-    metadata_schema_version: str,
-    version_file_key: str = "version.json",
-) -> None:
-    """Validate the metadata schema version against the policy defined in version.json."""
-    version_file = VersionFile.from_s3(s3_options, version_file_key)
-
-    current_version = pkg_version.Version(metadata_schema_version)
-    _check_policy_and_exit_if_needed("metadata", current_version, version_file.metadata_version)
-
-
 def _check_policy_and_exit_if_needed(
     subject: str,
     current_version: pkg_version.Version,

grz_cli-3.0.0/tests/test_config.py

@@ -0,0 +1,109 @@
+"""Tests for ``KeyModel``, the keys section of the grz-cli config, which moved here from grz-common."""
+
+import importlib.util
+from pathlib import Path
+
+import pytest
+from grz_cli.models.config import EncryptConfig, KeyModel
+from pydantic import ValidationError
+
+PUBLIC_KEY = "-----BEGIN CRYPT4GH PUBLIC KEY-----\n7JZ9eRjhOo1zB8HfoQK1ULCR3Wpnl91hF2K8FtpmeQ8=\n-----END CRYPT4GH PUBLIC KEY-----\n"
+"""A crypt4gh public key as text. ``Crypt4GHPublicKey`` only checks for the markers, so it need not decode."""
+
+SUBMITTER_PRIVATE_KEY = "submitter-private-key"
+"""Stands for a private key given inline. The model does not parse it, so it need not be a key."""
+
+
+def test_grz_common_keeps_no_copy_of_the_key_models():
+    assert importlib.util.find_spec("grz_common.models.keys") is None
+
+
+def test_encrypt_config_reads_the_keys_section(tmp_path: Path):
+    public_key_path = tmp_path / "grz.pub"
+    public_key_path.write_text(PUBLIC_KEY)
+
+    config = EncryptConfig.model_validate({"keys": {"grz_public_key_path": str(public_key_path)}})
+
+    assert config.keys.grz_public_key_path == public_key_path
+
+
+def test_a_config_with_the_removed_grz_private_key_path_still_loads():
+    """grz-cli never read ``keys.grz_private_key_path``. Old configs still set it, so the model ignores it."""
+    keys = KeyModel.model_validate({"grz_public_key": PUBLIC_KEY, "grz_private_key_path": "/no/such/key.sec"})
+
+    assert "grz_private_key_path" not in keys.model_dump()
+
+
+def test_neither_grz_public_key_nor_grz_public_key_path_fails():
+    with pytest.raises(ValidationError, match="Either grz_public_key or grz_public_key_path must be set"):
+        KeyModel()
+
+
+def test_both_grz_public_key_and_grz_public_key_path_fails(tmp_path: Path):
+    public_key_path = tmp_path / "grz.pub"
+    public_key_path.write_text(PUBLIC_KEY)
+
+    with pytest.raises(ValidationError, match="Only one of grz_public_key or grz_public_key_path must be set"):
+        KeyModel(grz_public_key=PUBLIC_KEY, grz_public_key_path=public_key_path)
+
+
+@pytest.mark.parametrize(
+    "public_key",
+    [
+        PUBLIC_KEY,
+        "-----BEGIN CRYPT4GH PUBLIC KEY-----\n7JZ9eRjhOo1zB8HfoQK1ULCR3Wpnl91hF2K8FtpmeQ8=\n",
+        "7JZ9eRjhOo1zB8HfoQK1ULCR3Wpnl91hF2K8FtpmeQ8=\n-----END CRYPT4GH PUBLIC KEY-----\n",
+    ],
+    ids=["both markers", "BEGIN marker only", "END marker only"],
+)
+def test_grz_public_key_with_a_marker_passes(public_key: str):
+    assert KeyModel(grz_public_key=public_key).grz_public_key == public_key
+
+
+def test_grz_public_key_without_markers_fails():
+    with pytest.raises(ValidationError, match="Invalid public key format"):
+        KeyModel(grz_public_key="7JZ9eRjhOo1zB8HfoQK1ULCR3Wpnl91hF2K8FtpmeQ8=")
+
+
+def test_both_submitter_private_key_and_submitter_private_key_path_fails(tmp_path: Path):
+    private_key_path = tmp_path / "submitter.sec"
+    private_key_path.write_text(SUBMITTER_PRIVATE_KEY)
+
+    with pytest.raises(
+        ValidationError, match="Only one of submitter_private_key or submitter_private_key_path must be set"
+    ):
+        KeyModel(
+            grz_public_key=PUBLIC_KEY,
+            submitter_private_key=SUBMITTER_PRIVATE_KEY,
+            submitter_private_key_path=private_key_path,
+        )
+
+
+def test_the_submitter_private_key_is_masked_in_dumps():
+    keys = KeyModel(grz_public_key=PUBLIC_KEY, submitter_private_key=SUBMITTER_PRIVATE_KEY)
+
+    assert SUBMITTER_PRIVATE_KEY not in repr(keys)
+    assert SUBMITTER_PRIVATE_KEY not in keys.model_dump_json()
+
+
+def test_the_submitter_private_key_passphrase_is_masked_in_dumps():
+    passphrase = "submitter-passphrase"
+    keys = KeyModel(grz_public_key=PUBLIC_KEY, submitter_private_key_passphrase=passphrase)
+
+    assert passphrase not in repr(keys)
+    assert passphrase not in keys.model_dump_json()
+
+
+def test_a_config_error_does_not_show_the_submitter_private_key(tmp_path: Path):
+    private_key_path = tmp_path / "submitter.sec"
+    private_key_path.write_text(SUBMITTER_PRIVATE_KEY)
+    keys = {
+        "grz_public_key": PUBLIC_KEY,
+        "submitter_private_key": SUBMITTER_PRIVATE_KEY,
+        "submitter_private_key_path": str(private_key_path),
+    }
+
+    with pytest.raises(ValidationError) as exc_info:
+        EncryptConfig.model_validate({"keys": keys})
+
+    assert SUBMITTER_PRIVATE_KEY not in str(exc_info.value)

grz_cli-3.0.0/tests/test_encrypt.py

@@ -0,0 +1,54 @@
+"""Tests for ``grz-cli encrypt``, which loads the submitter private key with its configured passphrase."""
+
+from pathlib import Path
+
+import click.testing
+import crypt4gh.keys
+import crypt4gh.keys.c4gh
+import pytest
+import yaml
+from grz_cli.cli import build_cli
+from pytest_mock import MockerFixture
+
+PASSPHRASE = "right-passphrase"
+
+
+@pytest.fixture
+def no_prompt(monkeypatch):
+    """Fail the test if the passphrase prompt opens, and set a wrong ``C4GH_PASSPHRASE``.
+
+    The configured passphrase comes first, so the wrong one in the environment must not matter.
+    """
+    monkeypatch.setenv("C4GH_PASSPHRASE", "wrong-passphrase")
+
+    def _fail(*args, **kwargs):
+        raise AssertionError("the passphrase prompt must not open")
+
+    monkeypatch.setattr("grz_common.utils.crypt.getpass", _fail)
+
+
+@pytest.mark.parametrize("inline", [False, True], ids=["path", "inline"])
+def test_encrypt_signs_with_the_submitter_key_and_its_configured_passphrase(
+    tmp_path: Path, submission_dir: Path, mocker: MockerFixture, no_prompt, inline: bool
+):
+    crypt4gh.keys.c4gh.generate(tmp_path / "grz.sec", tmp_path / "grz.pub", None, comment=None)
+    private_key_path = tmp_path / "le.sec"
+    crypt4gh.keys.c4gh.generate(private_key_path, tmp_path / "le.pub", PASSPHRASE.encode(), comment=None)
+    keys = {"grz_public_key_path": str(tmp_path / "grz.pub"), "submitter_private_key_passphrase": PASSPHRASE}
+    if inline:
+        keys["submitter_private_key"] = private_key_path.read_text()
+    else:
+        keys["submitter_private_key_path"] = str(private_key_path)
+    config_path = tmp_path / "config.yaml"
+    config_path.write_text(yaml.dump({"keys": keys}))
+    worker_encrypt = mocker.patch("grz_cli.commands.encrypt.Worker.encrypt")
+
+    result = click.testing.CliRunner().invoke(
+        build_cli(),
+        ["encrypt", "--submission-dir", str(submission_dir), "--config-file", str(config_path)],
+        catch_exceptions=False,
+    )
+
+    assert result.exit_code == 0, result.output
+    signing_key = worker_encrypt.call_args.kwargs["submitter_private_key"]
+    assert signing_key.private_bytes_raw() == crypt4gh.keys.get_private_key(private_key_path, lambda: PASSPHRASE)

grz_cli-2.0.0/src/grz_cli/__init__.py

@@ -1 +0,0 @@
-__version__ = "2.0.0"

grz_cli-2.0.0/src/grz_cli/models/config.py

@@ -1,19 +0,0 @@
-import logging
-
-from grz_common.models.identifiers import IdentifiersConfigModel
-from grz_common.models.keys import KeyConfigModel
-from grz_common.models.s3 import S3ConfigModel
-
-log = logging.getLogger(__name__)
-
-
-class UploadConfig(S3ConfigModel):
-    pass
-
-
-class EncryptConfig(KeyConfigModel):
-    pass
-
-
-class ValidateConfig(IdentifiersConfigModel):
-    pass