genesis-devtools 0.2.4__tar.gz → 0.2.6__tar.gz

genesis_devtools-0.2.6/ChangeLog

@@ -0,0 +1,7 @@
+CHANGES
+=======
+
+0.2.6
+-----
+
+* Encryption of backups

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: genesis-devtools
-Version: 0.2.4
+Version: 0.2.6
 Summary: Tools to manager life cycle of Genesis projects.
 Home-page: https://github.com/infraguys/genesis_devtools
 Author: Genesis Corporation
@@ -23,6 +23,7 @@ Requires-Dist: pyyaml<7.0.0,>=6.0.0
 Requires-Dist: prettytable<4.0.0,>=3.7.0
 Requires-Dist: GitPython<4.0.0,>=3.1.30
 Requires-Dist: bazooka<2.0.0,>=1.0.0
+Requires-Dist: cryptography<47.0.0,>=45.0.5
 Dynamic: author
 Dynamic: author-email
 Dynamic: classifier
@@ -54,7 +55,7 @@ Before you can install and use genesis tools you need to install several require
 Install packages
 ```sh
 sudo apt update
-sudo apt install qemu-kvm libvirt-daemon-system libvirt-dev mkisofs
+sudo apt install qemu-kvm qemu-utils libvirt-daemon-system libvirt-dev mkisofs
 ```
 
 Add user to group
@@ -342,7 +343,7 @@ genesis backup -oneshot
 
 This command will backup the current installation to the specified directory once and exit.
 
-### Compuressed Backup
+### Compressed Backup
 
 Run backup of all libvirt domains and store a compressed archive of the backup in the current directory:
 
@@ -350,6 +351,25 @@ Run backup of all libvirt domains and store a compressed archive of the backup i
 genesis backup --compress
 ```
 
+### Encrypted Backup
+
+Run backup of all libvirt domains and store an encrypted archive of the backup in the current directory:
+
+NOTE: It works only with `--compress` flag
+
+You need to set the environment variables `GEN_DEV_BACKUP_KEY` and `GEN_DEV_BACKUP_IV` to encrypt the backup. The key and IV must be greater or equal to 6 bytes and less or equal to 16 bytes.
+```bash
+export GEN_DEV_BACKUP_KEY=secret_key
+export GEN_DEV_BACKUP_IV=secret_iv
+
+genesis backup --compress --encrypt
+```
+
+For decryption, use the `genesis backup-decrypt` command.
+```bash
+genesis backup-decrypt backup.tar.gz.encrypted
+```
+
 ### Rotation
 
 For the periodic backup, you can set the number of backups to keep(rotation number). The default value is 5. Use the `--rotate` option to set the number of backups to keep:

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/README.md

@@ -20,7 +20,7 @@ Before you can install and use genesis tools you need to install several require
 Install packages
 ```sh
 sudo apt update
-sudo apt install qemu-kvm libvirt-daemon-system libvirt-dev mkisofs
+sudo apt install qemu-kvm qemu-utils libvirt-daemon-system libvirt-dev mkisofs
 ```
 
 Add user to group
@@ -308,7 +308,7 @@ genesis backup -oneshot
 
 This command will backup the current installation to the specified directory once and exit.
 
-### Compuressed Backup
+### Compressed Backup
 
 Run backup of all libvirt domains and store a compressed archive of the backup in the current directory:
 
@@ -316,6 +316,25 @@ Run backup of all libvirt domains and store a compressed archive of the backup i
 genesis backup --compress
 ```
 
+### Encrypted Backup
+
+Run backup of all libvirt domains and store an encrypted archive of the backup in the current directory:
+
+NOTE: It works only with `--compress` flag
+
+You need to set the environment variables `GEN_DEV_BACKUP_KEY` and `GEN_DEV_BACKUP_IV` to encrypt the backup. The key and IV must be greater or equal to 6 bytes and less or equal to 16 bytes.
+```bash
+export GEN_DEV_BACKUP_KEY=secret_key
+export GEN_DEV_BACKUP_IV=secret_iv
+
+genesis backup --compress --encrypt
+```
+
+For decryption, use the `genesis backup-decrypt` command.
+```bash
+genesis backup-decrypt backup.tar.gz.encrypted
+```
+
 ### Rotation
 
 For the periodic backup, you can set the number of backups to keep(rotation number). The default value is 5. Use the `--rotate` option to set the number of backups to keep:
@@ -330,4 +349,4 @@ Backups can take a lot of disk space and it can be a reason to crash the whole s
 
 ```bash
 genesis backup --min-free-space 50
-``` 
+``` 

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools/backup.py

@@ -29,8 +29,57 @@ from genesis_devtools import utils
 from genesis_devtools.infra.libvirt import libvirt
 
 
+class EnctryptionCreds(tp.NamedTuple):
+    LEN = 16
+    MIN_LEN = 6
+
+    key: bytes
+    iv: bytes
+
+    @classmethod
+    def validate_env(cls):
+        if not os.environ.get("GEN_DEV_BACKUP_KEY") or not os.environ.get(
+            "GEN_DEV_BACKUP_IV"
+        ):
+            raise ValueError(
+                (
+                    "Define environment variables GEN_DEV_BACKUP_KEY "
+                    "and GEN_DEV_BACKUP_IV."
+                )
+            )
+
+        key = os.environ["GEN_DEV_BACKUP_KEY"]
+        iv = os.environ["GEN_DEV_BACKUP_IV"]
+
+        if (
+            cls.MIN_LEN <= len(key) < cls.LEN
+            and cls.MIN_LEN <= len(iv) < cls.LEN
+        ):
+            return
+
+        raise ValueError(
+            f"Key and IV must be greater or equal than {cls.MIN_LEN} "
+            f"bytes and less or equal to {cls.LEN} bytes."
+        )
+
+    @classmethod
+    def from_env(cls):
+        key = os.environ["GEN_DEV_BACKUP_KEY"]
+        iv = os.environ["GEN_DEV_BACKUP_IV"]
+        key = key + "0" * (cls.LEN - len(key))
+        iv = iv + "0" * (cls.LEN - len(iv))
+
+        return cls(
+            key=key.encode(),
+            iv=iv.encode(),
+        )
+
+
 def _do_backup(
-    backup_path: str, domains: tp.List[str], compress: bool = False
+    backup_path: str,
+    domains: tp.List[str],
+    compress: bool = False,
+    encryption: EnctryptionCreds | None = None,
 ) -> None:
     os.makedirs(backup_path, exist_ok=True)
 
@@ -75,19 +124,35 @@ def _do_backup(
         return
 
     click.echo(f"Compressing {backup_path}")
+    compressed_backup_path = f"{backup_path}.tar.gz"
     compress_directory = os.path.dirname(backup_path)
     try:
         utils.compress_dir(backup_path, compress_directory)
     except Exception:
         click.secho(f"Compression of {backup_path} failed", fg="red")
-        compressed_backup_path = f"{backup_path}.tar.gz"
-
         if os.path.exists(compressed_backup_path):
             os.remove(compressed_backup_path)
         return
 
     click.secho(f"Compression of {backup_path} done", fg="green")
-    shutil.rmtree(backup_path)
+
+    if not encryption:
+        shutil.rmtree(backup_path)
+        return
+
+    click.echo(f"Encrypting {compressed_backup_path}")
+    try:
+        utils.encrypt_file(
+            compressed_backup_path, encryption.key, encryption.iv
+        )
+    except Exception:
+        click.secho(f"Encryption of {compressed_backup_path} failed", fg="red")
+        return
+    finally:
+        shutil.rmtree(backup_path)
+
+    os.remove(compressed_backup_path)
+    click.secho(f"Encryption of {compressed_backup_path} done", fg="green")
 
 
 def _terminate_backup_process(backup_process: mp.Process) -> None:
@@ -115,6 +180,7 @@ def backup(
     backup_path: str,
     domains: tp.List[str],
     compress: bool = False,
+    encryption: EnctryptionCreds | None = None,
     min_free_disk_space_gb: int = 50,
 ) -> None:
     if not os.path.exists(backup_path):
@@ -134,7 +200,9 @@ def backup(
     # Run the actual backup process in another process.
     # The current process will track the free disk space.
     backup_process = mp.Process(
-        target=_do_backup, args=(backup_path, domains, compress), daemon=True
+        target=_do_backup,
+        args=(backup_path, domains, compress, encryption),
+        daemon=True,
     )
     backup_process.start()
 

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools/builder/packer.py

@@ -127,8 +127,13 @@ class PackerBuilder(base.DummyImageBuilder):
         result = []
 
         for env in envs:
+            # Check if the env is a wildcard
+            if env.endswith("*"):
+                for _env in (e for e in os.environ if e.startswith(env[:-1])):
+                    result.append(f'{_env}="{os.environ[_env]}"')
+                continue
             # Check if there a default value for the env
-            if "=" in env:
+            elif "=" in env:
                 name, value = tuple(e.strip() for e in env.split("="))
             else:
                 name, value = env.strip(), ""

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools/cmd/cli.py

@@ -385,7 +385,7 @@ def get_project_version_cmd(element_dir: str) -> None:
     "-p",
     "--period",
     default=c.BackupPeriod.D1.value,
-    type=click.Choice([p for p in c.BackupPeriod]),
+    type=click.Choice([p.value for p in c.BackupPeriod]),
     show_default=True,
     help="the regularity of backups",
 )
@@ -393,7 +393,7 @@ def get_project_version_cmd(element_dir: str) -> None:
     "-o",
     "--offset",
     default=None,
-    type=click.Choice([p for p in c.BackupPeriod]),
+    type=click.Choice([p.value for p in c.BackupPeriod]),
     show_default=True,
     help=(
         "The time offset of the first backup. If not provided, "
@@ -411,12 +411,22 @@ def get_project_version_cmd(element_dir: str) -> None:
 @click.option(
     "-c",
     "--compress",
-    default=False,
-    type=bool,
     show_default=True,
     is_flag=True,
     help="Compress the backup.",
 )
+@click.option(
+    "-e",
+    "--encrypt",
+    show_default=True,
+    is_flag=True,
+    help=(
+        "Encrypt the backup. Works only with the compress flag. "
+        "Use environment variable to specify the encryption key "
+        "and the initialization vector: "
+        "GEN_DEV_BACKUP_KEY and GEN_DEV_BACKUP_IV"
+    ),
+)
 @click.option(
     "-s",
     "--min-free-space",
@@ -447,14 +457,42 @@ def bakcup_cmd(
     offset: c.BackupPeriod | None,
     oneshot: bool,
     compress: bool,
+    encrypt: bool,
     min_free_space: int,
     rotate: int,
 ) -> None:
+    if not compress and encrypt:
+        raise click.UsageError(
+            "The encrypt flag can be used only with the compress flag."
+        )
+
+    # Need to specify encryption key and initialization vector via
+    # environment variables.
+    if encrypt:
+        try:
+            backup.EnctryptionCreds.validate_env()
+        except ValueError:
+            raise click.UsageError(
+                (
+                    "Define environment variables GEN_DEV_BACKUP_KEY "
+                    "and GEN_DEV_BACKUP_IV. "
+                    "Key and IV must be greater or equal than "
+                    f"{backup.EnctryptionCreds.MIN_LEN} bytes and less or "
+                    f"equal to {backup.EnctryptionCreds.LEN} bytes."
+                )
+            )
+
+        encryption = backup.EnctryptionCreds.from_env()
+    else:
+        encryption = None
+
     # Do a single backup and exit
     if oneshot:
         domains = _domains_for_backup(name, raise_on_domain_absence=True)
         backup_path = utils.backup_path(backup_dir)
-        backup.backup(backup_path, domains, compress, min_free_space)
+        backup.backup(
+            backup_path, domains, compress, encryption, min_free_space
+        )
         return
 
     offset = offset or period
@@ -475,7 +513,9 @@ def bakcup_cmd(
         click.secho(f"Backup started at {time.strftime('%Y-%m-%d %H:%M:%S')}")
         backup_path = utils.backup_path(backup_dir)
 
-        backup.backup(backup_path, domains, compress, min_free_space)
+        backup.backup(
+            backup_path, domains, compress, encryption, min_free_space
+        )
         click.secho(
             f"Next backup at: {time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(next_ts))}"
         )
@@ -490,6 +530,35 @@ def bakcup_cmd(
         time.sleep(timeout)
 
 
+@main.command("backup-decrypt", help="Decrypt a backup file")
+@click.argument("path", type=click.Path(exists=True))
+def bakcup_decrypt_cmd(path: str) -> None:
+    # Need to specify encryption key and initialization vector via
+    # environment variables.
+
+    try:
+        backup.EnctryptionCreds.validate_env()
+    except ValueError:
+        raise click.UsageError(
+            (
+                "Define environment variables GEN_DEV_BACKUP_KEY "
+                "and GEN_DEV_BACKUP_IV. "
+                "Key and IV must be greater or equal than "
+                f"{backup.EnctryptionCreds.MIN_LEN} bytes and less or "
+                f"equal to {backup.EnctryptionCreds.LEN} bytes."
+            )
+        )
+
+    encryption = backup.EnctryptionCreds.from_env()
+
+    utils.decrypt_file(
+        path,
+        encryption.key,
+        encryption.iv,
+    )
+    click.secho(f"The {path} file has been decrypted.", fg="green")
+
+
 def _domains_for_backup(
     names: tp.List[str] | None = None, raise_on_domain_absence: bool = False
 ) -> tp.List[str]:

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools/constants.py

@@ -22,6 +22,7 @@ DEF_GEN_CFG_FILE_NAME = "genesis.yaml"
 DEF_GEN_WORK_DIR_NAME = "genesis"
 DEF_GEN_OUTPUT_DIR_NAME = "output"
 RC_BRANCHES = ("master", "main")
+ENCRYPTED_EXTENSION = ".encrypted"
 
 # ENV vars
 ENV_GEN_DEV_KEYS = "GEN_DEV_KEYS"

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools/utils.py

@@ -24,6 +24,10 @@ from importlib.metadata import entry_points
 import yaml
 
 import git
+from cryptography.hazmat.primitives import ciphers
+from cryptography.hazmat.primitives.ciphers import modes as cipher_models
+from cryptography.hazmat.primitives.ciphers import algorithms
+from cryptography.hazmat import backends as crypto_back
 
 import genesis_devtools.constants as c
 
@@ -234,3 +238,89 @@ def compress_dir(
 
     # Create a zip archive of the directory
     shutil.make_archive(archive_base_name, compression_format, directory)
+
+
+def encrypt_file(
+    path: str,
+    key: bytes,
+    iv: bytes,
+    chunk_size_kb: int = 128,
+    extension: str = c.ENCRYPTED_EXTENSION,
+) -> None:
+    # Ensure that the key and iv are the correct lengths
+    # for AES (16 bytes for AES-128)
+    if len(key) != 16 or len(iv) != 16:
+        raise ValueError("Key and IV must be 16 bytes long")
+
+    # Create a cipher object
+    cipher = ciphers.Cipher(
+        algorithms.AES(key),
+        cipher_models.CFB(iv),
+        backend=crypto_back.default_backend(),
+    )
+    chunk_size = chunk_size_kb << 10
+    encrypted_path = path + extension
+
+    # Open the input file and the temporary output file
+    try:
+        with open(path, "rb") as infile, open(encrypted_path, "wb") as outfile:
+            # Create an encryptor object
+            encryptor = cipher.encryptor()
+
+            # Read the file in chunks and write the encrypted
+            # data to the temp file
+            while True:
+                chunk = infile.read(chunk_size)
+                if len(chunk) == 0:
+                    break
+                outfile.write(encryptor.update(chunk))
+            outfile.write(encryptor.finalize())
+    except Exception as e:
+        os.remove(encrypted_path)
+        raise e
+
+
+def decrypt_file(
+    path: str,
+    key: bytes,
+    iv: bytes,
+    chunk_size_kb: int = 128,
+    extension: str = c.ENCRYPTED_EXTENSION,
+) -> None:
+    # Ensure that the key and iv are the correct lengths
+    # for AES (16 bytes for AES-128)
+    if len(key) != 16 or len(iv) != 16:
+        raise ValueError("Key and IV must be 16 bytes long")
+
+    # Create a cipher object
+    cipher = ciphers.Cipher(
+        algorithms.AES(key),
+        cipher_models.CFB(iv),
+        backend=crypto_back.default_backend(),
+    )
+    chunk_size = chunk_size_kb << 10
+    plain_path = path[: -len(extension)] if path.endswith(extension) else path
+    tmp_path = plain_path + ".tmp"
+
+    # Open the encrypted file and the temporary output file
+    try:
+        with open(path, "rb") as infile, open(tmp_path, "wb") as outfile:
+            # Create a decryptor object
+            decryptor = cipher.decryptor()
+
+            # Read the file in chunks and write the decrypted data to the temp file
+            while True:
+                chunk = infile.read(chunk_size)
+                if len(chunk) == 0:
+                    break
+                outfile.write(decryptor.update(chunk))
+            outfile.write(decryptor.finalize())
+    except Exception as e:
+        os.remove(tmp_path)
+        raise e
+
+    # Replace the encrypted file with the decrypted temp file
+    if plain_path == path:
+        os.replace(tmp_path, plain_path)
+    else:
+        shutil.move(tmp_path, plain_path)

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: genesis-devtools
-Version: 0.2.4
+Version: 0.2.6
 Summary: Tools to manager life cycle of Genesis projects.
 Home-page: https://github.com/infraguys/genesis_devtools
 Author: Genesis Corporation
@@ -23,6 +23,7 @@ Requires-Dist: pyyaml<7.0.0,>=6.0.0
 Requires-Dist: prettytable<4.0.0,>=3.7.0
 Requires-Dist: GitPython<4.0.0,>=3.1.30
 Requires-Dist: bazooka<2.0.0,>=1.0.0
+Requires-Dist: cryptography<47.0.0,>=45.0.5
 Dynamic: author
 Dynamic: author-email
 Dynamic: classifier
@@ -54,7 +55,7 @@ Before you can install and use genesis tools you need to install several require
 Install packages
 ```sh
 sudo apt update
-sudo apt install qemu-kvm libvirt-daemon-system libvirt-dev mkisofs
+sudo apt install qemu-kvm qemu-utils libvirt-daemon-system libvirt-dev mkisofs
 ```
 
 Add user to group
@@ -342,7 +343,7 @@ genesis backup -oneshot
 
 This command will backup the current installation to the specified directory once and exit.
 
-### Compuressed Backup
+### Compressed Backup
 
 Run backup of all libvirt domains and store a compressed archive of the backup in the current directory:
 
@@ -350,6 +351,25 @@ Run backup of all libvirt domains and store a compressed archive of the backup i
 genesis backup --compress
 ```
 
+### Encrypted Backup
+
+Run backup of all libvirt domains and store an encrypted archive of the backup in the current directory:
+
+NOTE: It works only with `--compress` flag
+
+You need to set the environment variables `GEN_DEV_BACKUP_KEY` and `GEN_DEV_BACKUP_IV` to encrypt the backup. The key and IV must be greater or equal to 6 bytes and less or equal to 16 bytes.
+```bash
+export GEN_DEV_BACKUP_KEY=secret_key
+export GEN_DEV_BACKUP_IV=secret_iv
+
+genesis backup --compress --encrypt
+```
+
+For decryption, use the `genesis backup-decrypt` command.
+```bash
+genesis backup-decrypt backup.tar.gz.encrypted
+```
+
 ### Rotation
 
 For the periodic backup, you can set the number of backups to keep(rotation number). The default value is 5. Use the `--rotate` option to set the number of backups to keep:

genesis_devtools-0.2.6/genesis_devtools.egg-info/pbr.json

@@ -0,0 +1 @@
+{"git_version": "6371a1f", "is_release": false}

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/genesis_devtools.egg-info/requires.txt

@@ -4,3 +4,4 @@ pyyaml<7.0.0,>=6.0.0
 prettytable<4.0.0,>=3.7.0
 GitPython<4.0.0,>=3.1.30
 bazooka<2.0.0,>=1.0.0
+cryptography<47.0.0,>=45.0.5

{genesis_devtools-0.2.4 → genesis_devtools-0.2.6}/requirements.txt

@@ -4,3 +4,4 @@ pyyaml>=6.0.0,<7.0.0  # MIT
 prettytable>=3.7.0,<4.0.0  # MIT
 GitPython>=3.1.30,<4.0.0
 bazooka>=1.0.0,<2.0.0  # Apache-2.0
+cryptography>=45.0.5,<47.0.0 # BSD-3

genesis_devtools-0.2.4/ChangeLog

@@ -1,7 +0,0 @@
-CHANGES
-=======
-
-0.2.4
------
-
-* Bugfix: Building several images for one element (#47)

genesis_devtools-0.2.4/genesis_devtools.egg-info/pbr.json

@@ -1 +0,0 @@
-{"git_version": "5308a95", "is_release": false}