glpkg 1.2.0__tar.gz → 1.4.0__tar.gz

{glpkg-1.2.0/src/glpkg.egg-info → glpkg-1.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: glpkg
-Version: 1.2.0
+Version: 1.4.0
 Summary: Tool to make GitLab generic package registry operations easy.
 Author-email: bugproduction <bugproduction@outlook.com>
 License-Expression: MIT
@@ -15,6 +15,7 @@ Dynamic: license-file
 
 glpkg is a tool that makes it easy to work with [GitLab generic packages](https://docs.gitlab.com/user/packages/generic_packages/).
 
+Generic package registries of GitLab projects are supported. Group registries are not, as the GitLab REST API for groups is very limited.
 
 ## Installation
 
@@ -26,7 +27,6 @@ pip install glpkg
 
 To check the installation and version,  run:
 
-
 ```bash
 glpkg --version
 ```
@@ -105,10 +105,49 @@ Where:
 - `12345` is your projects ID ([Find the Project ID](https://docs.gitlab.com/user/project/working_with_projects/#find-the-project-id)) or the path of the project (like `namespace/project`)
 - `mypackagename` is the name of the generic package
 - `1.0` is the version of the generic package to which the file is uploaded
-- `my-file.txt` is the file that is uploaded to the generic package. Currently, only relative paths are supported, and the relative path (e.g. `folder/file.txt`) is preserved when uploading the file to the registry.
+- `my-file.txt` is the file that is uploaded to the generic package. Only relative paths are supported, and the relative path (e.g. `folder/file.txt`) is preserved when uploading the file to the package.
 
 > A GitLab generic package may have multiple files with the same file name. However, it likely is not a great idea, as they cannot be downloaded separately from the GitLab API.
 
+To upload multiple files, or to upload a single file from a different directory, use `--source` argument. If no `--file` argument is set, all of the files in the source directory are uploaded, recursively. As an example, to upload all files from a `upload` folder to the package:
+
+```bash
+glpkg upload --project 12345 --name mypackagename --version 1.0 --source upload
+```
+
+Note: a file in `upload` folder will be uploaded to the root of the package. If you want to upload the file to a `dir` folder in the package, make a structure in the upload folder, like `upload/dir/`.
+
+### Delete a file from a generic package
+
+To delete a file from a specific generic package, run
+
+```bash
+glpkg delete --project 12345 --name mypackagename --version 1.0 --file my-file.txt
+```
+
+Where
+- `12345` is your projects ID ([Find the Project ID](https://docs.gitlab.com/user/project/working_with_projects/#find-the-project-id)) or the path of the project (like `namespace/project`)
+- `mypackagename` is the name of the generic package
+- `1.0` is the version of the generic package from the file is deleted
+- `my-file.txt` is the file that is deleted. Only relative paths are supported. Note that the package may contain multiple files with same name. In this case, one file of them is deleted.
+
+The token that is used to delete files must have at least Maintainer role in the project.
+
+### Delete a package version
+
+To delete a specific generic package version, run
+
+```bash
+glpkg delete --project 12345 --name mypackagename --version 1.0
+```
+
+Where
+- `12345` is your projects ID ([Find the Project ID](https://docs.gitlab.com/user/project/working_with_projects/#find-the-project-id)) or the path of the project (like `namespace/project`)
+- `mypackagename` is the name of the generic package
+- `1.0` is the version of the generic package that is deleted
+
+The token that is used to delete packages must have at least Maintainer role in the project.
+
 ### Use in GitLab pipelines
 
 If you use the tool in a GitLab pipeline, setting argument `--ci` uses [GitLab predefined variables](https://docs.gitlab.com/ci/variables/predefined_variables/) to configure the tool. In this case `CI_SERVER_HOST`, `CI_PROJECT_ID`, and `CI_JOB_TOKEN` environment variables are used. The `--project`, and `--token` arguments can still be used to override the project ID or to use a personal or project access token instead of `CI_JOB_TOKEN`.
@@ -120,11 +159,3 @@ glpkg upload --ci --name mypackagename --version 1.0 --file my-file.txt
 ```
 
 To use the `CI_JOB_TOKEN` with package registry of another projects, add `--project <otherproject ID>` argument. Remember that you may need to add [permissions for the CI_JOB_TOKEN](https://docs.gitlab.com/ci/jobs/ci_job_token/#control-job-token-access-to-your-project) in the other project.
-
-
-## Limitations
-
-The tool is not perfect (yet) and has limitations. The following limitations are known, but more can exist:
-
-- Uploading files must be done one-by-one.
-- Only project registries are supported for now.

{glpkg-1.2.0 → glpkg-1.4.0}/README.md

@@ -2,6 +2,7 @@
 
 glpkg is a tool that makes it easy to work with [GitLab generic packages](https://docs.gitlab.com/user/packages/generic_packages/).
 
+Generic package registries of GitLab projects are supported. Group registries are not, as the GitLab REST API for groups is very limited.
 
 ## Installation
 
@@ -13,7 +14,6 @@ pip install glpkg
 
 To check the installation and version,  run:
 
-
 ```bash
 glpkg --version
 ```
@@ -92,10 +92,49 @@ Where:
 - `12345` is your projects ID ([Find the Project ID](https://docs.gitlab.com/user/project/working_with_projects/#find-the-project-id)) or the path of the project (like `namespace/project`)
 - `mypackagename` is the name of the generic package
 - `1.0` is the version of the generic package to which the file is uploaded
-- `my-file.txt` is the file that is uploaded to the generic package. Currently, only relative paths are supported, and the relative path (e.g. `folder/file.txt`) is preserved when uploading the file to the registry.
+- `my-file.txt` is the file that is uploaded to the generic package. Only relative paths are supported, and the relative path (e.g. `folder/file.txt`) is preserved when uploading the file to the package.
 
 > A GitLab generic package may have multiple files with the same file name. However, it likely is not a great idea, as they cannot be downloaded separately from the GitLab API.
 
+To upload multiple files, or to upload a single file from a different directory, use `--source` argument. If no `--file` argument is set, all of the files in the source directory are uploaded, recursively. As an example, to upload all files from a `upload` folder to the package:
+
+```bash
+glpkg upload --project 12345 --name mypackagename --version 1.0 --source upload
+```
+
+Note: a file in `upload` folder will be uploaded to the root of the package. If you want to upload the file to a `dir` folder in the package, make a structure in the upload folder, like `upload/dir/`.
+
+### Delete a file from a generic package
+
+To delete a file from a specific generic package, run
+
+```bash
+glpkg delete --project 12345 --name mypackagename --version 1.0 --file my-file.txt
+```
+
+Where
+- `12345` is your projects ID ([Find the Project ID](https://docs.gitlab.com/user/project/working_with_projects/#find-the-project-id)) or the path of the project (like `namespace/project`)
+- `mypackagename` is the name of the generic package
+- `1.0` is the version of the generic package from the file is deleted
+- `my-file.txt` is the file that is deleted. Only relative paths are supported. Note that the package may contain multiple files with same name. In this case, one file of them is deleted.
+
+The token that is used to delete files must have at least Maintainer role in the project.
+
+### Delete a package version
+
+To delete a specific generic package version, run
+
+```bash
+glpkg delete --project 12345 --name mypackagename --version 1.0
+```
+
+Where
+- `12345` is your projects ID ([Find the Project ID](https://docs.gitlab.com/user/project/working_with_projects/#find-the-project-id)) or the path of the project (like `namespace/project`)
+- `mypackagename` is the name of the generic package
+- `1.0` is the version of the generic package that is deleted
+
+The token that is used to delete packages must have at least Maintainer role in the project.
+
 ### Use in GitLab pipelines
 
 If you use the tool in a GitLab pipeline, setting argument `--ci` uses [GitLab predefined variables](https://docs.gitlab.com/ci/variables/predefined_variables/) to configure the tool. In this case `CI_SERVER_HOST`, `CI_PROJECT_ID`, and `CI_JOB_TOKEN` environment variables are used. The `--project`, and `--token` arguments can still be used to override the project ID or to use a personal or project access token instead of `CI_JOB_TOKEN`.
@@ -107,11 +146,3 @@ glpkg upload --ci --name mypackagename --version 1.0 --file my-file.txt
 ```
 
 To use the `CI_JOB_TOKEN` with package registry of another projects, add `--project <otherproject ID>` argument. Remember that you may need to add [permissions for the CI_JOB_TOKEN](https://docs.gitlab.com/ci/jobs/ci_job_token/#control-job-token-access-to-your-project) in the other project.
-
-
-## Limitations
-
-The tool is not perfect (yet) and has limitations. The following limitations are known, but more can exist:
-
-- Uploading files must be done one-by-one.
-- Only project registries are supported for now.

{glpkg-1.2.0 → glpkg-1.4.0}/pyproject.toml

@@ -31,3 +31,9 @@ where = ["src"]
 [tool.pytest.ini_options]
 pythonpath = ["src"]
 addopts = "--cov=src --cov-report term"
+
+[tool.pylint.'MESSAGES CONTROL']
+min-public-methods = 1
+max-args = 6
+max-positional-arguments = 6
+ignore = ["build", "test"]

glpkg-1.4.0/src/gitlab/__init__.py

@@ -0,0 +1,5 @@
+"""GitLab packages initialization module"""
+
+from gitlab.packages import Packages
+
+__version__ = "1.4.0"

glpkg-1.4.0/src/gitlab/__main__.py

@@ -0,0 +1,29 @@
+"""GitLab packages main module"""
+
+import logging
+import sys
+
+from gitlab.cli_handler import CLIHandler
+
+
+def cli() -> int:
+    """
+    Runs the main program of the glpkg.
+
+    Uses arguments from command line and executes the given command.
+
+    Return
+    ------
+    int
+        Zero when everything is fine, non-zero otherwise.
+    """
+    logging.basicConfig(
+        level=logging.DEBUG,
+        handlers=[logging.FileHandler("glpkg.log")],  # , logging.StreamHandler()],
+    )
+    handler = CLIHandler()
+    return handler.do_it()
+
+
+if __name__ == "__main__":
+    sys.exit(cli())

glpkg-1.4.0/src/gitlab/cli_handler.py

@@ -0,0 +1,419 @@
+"""CLI handler for glpkg"""
+
+import argparse
+from netrc import netrc
+import os
+import sys
+import urllib
+from gitlab import Packages, __version__
+
+
+class CLIHandler:
+    """Class to parse CLI arguments and run the requested command"""
+
+    def __init__(self):
+        """
+        Creates a new instance of CLIHandler.
+
+        Parses the arguments from command line and prepares everything
+        to be ready to executed. Just use do_it to run it!
+        """
+        parser = argparse.ArgumentParser(
+            description="Toolbox for GitLab generic packages"
+        )
+        parser.add_argument("-v", "--version", action="store_true")
+        parser.set_defaults(action=self._print_version)
+        subparsers = parser.add_subparsers()
+        list_parser = subparsers.add_parser(
+            name="list",
+            description="Lists the available version of a package from the "
+            "package registry.",
+        )
+        self._register_list_parser(list_parser)
+        download_parser = subparsers.add_parser(
+            name="download",
+            description="Downloads all files from a specific package version "
+            "to the current directory.",
+        )
+        self._register_download_parser(download_parser)
+        upload_parser = subparsers.add_parser(
+            name="upload", description="Uploads file to a specific package version."
+        )
+        self._register_upload_parser(upload_parser)
+        delete_parser = subparsers.add_parser(
+            name="delete",
+            description="Deletes a specific package version or a specific file from "
+            "a specific package version, depending whether the --file argument is "
+            "set or not.",
+        )
+        self._register_delete_parser(delete_parser)
+        self.args = parser.parse_args()
+
+    def _print_version(self, _args: argparse.Namespace) -> int:
+        """
+        A handler for printing the version of the tool to the console.
+
+        Parameters
+        ----------
+        _args : argparse.Namespace
+            Unused.
+
+        Return
+        ------
+        int
+            Zero when printing to console succeeded.
+        """
+        print(__version__)
+        return 0
+
+    def do_it(self) -> int:
+        """
+        Executes the requested command.
+
+        In case of error, prints to stderr.
+
+        Return
+        ------
+        int
+            Zero when everything went fine, non-zero otherwise.
+        """
+        ret = 1
+        try:
+            ret = self.args.action(self.args)
+        except urllib.error.HTTPError as e:
+            # GitLab API returns 404 when a resource is not found
+            # but also when the user has no access to the resource
+            print("Oops! Something did go wrong.", file=sys.stderr)
+            print(e, file=sys.stderr)
+            print(
+                "Note that Error 404 may also indicate authentication issues with GitLab API.",
+                file=sys.stderr,
+            )
+            print("Check your arguments and credentials.", file=sys.stderr)
+        return ret
+
+    def _register_common_arguments(self, parser: argparse.ArgumentParser) -> None:
+        """
+        Registers common arguments to the parser:
+            - host
+            - ci
+            - project
+            - name
+            - token
+            - netrc
+
+        Parameters
+        ----------
+        parser:
+            The argparser where to register the common arguments.
+        """
+        group = parser.add_mutually_exclusive_group()
+        group.add_argument(
+            "-H",
+            "--host",
+            default="gitlab.com",
+            type=str,
+            help="The host address of GitLab instance without scheme, "
+            "for example gitlab.com. Note that only https scheme is supported.",
+        )
+        group.add_argument(
+            "-c",
+            "--ci",
+            action="store_true",
+            help="Use this in GitLab jobs. In this case CI_SERVER_HOST, CI_PROJECT_ID, "
+            "and CI_JOB_TOKEN variables from the environment are used. --project and --token "
+            "can be used to override project ID and the CI_JOB_TOKEN to a personal or project "
+            "access token.",
+        )
+        parser.add_argument(
+            "-p",
+            "--project",
+            type=str,
+            help="The project ID or path. For example 123456 or namespace/project.",
+        )
+        parser.add_argument("-n", "--name", type=str, help="The package name.")
+        group2 = parser.add_mutually_exclusive_group()
+        group2.add_argument(
+            "-t",
+            "--token",
+            type=str,
+            help="Private or project access token that is used to authenticate with "
+            "the package registry. Leave empty if the registry is public. The token "
+            "must have 'read API' or 'API' scope.",
+        )
+        group2.add_argument(
+            "--netrc",
+            action="store_true",
+            help="Set to use a token from .netrc file (~/.netrc) for the host. The "
+            ".netrc username is ignored due to API restrictions. PRIVATE-TOKEN is used "
+            "instead. Note that .netrc file access rights must be correct.",
+        )
+
+    def _register_download_parser(self, parser: argparse.ArgumentParser):
+        """
+        Registers the download command related arguments to the parser:
+            - version
+            - file
+            - destination
+            - action
+
+        Additionally, registers the common args.
+
+        Parameters
+        ----------
+        parser:
+            The argparser where to register the download arguments.
+        """
+        self._register_common_arguments(parser)
+        parser.add_argument("-v", "--version", type=str, help="The package version.")
+        parser.add_argument(
+            "-f",
+            "--file",
+            type=str,
+            help="The file to download from the package. If not defined, all "
+            "files are downloaded.",
+        )
+        parser.add_argument(
+            "-d",
+            "--destination",
+            default="",
+            type=str,
+            help="The path where the file(s) are downloaded. If not defined, "
+            "the current working directory is used.",
+        )
+        parser.set_defaults(action=self._download_handler)
+
+    def _args(self, args) -> tuple[str, str, str, str, str]:
+        """
+        Returns the connection parameters according to the args
+
+        Parameters
+        ----------
+        args:
+            The args that are used to determined the connection parameters
+
+        Returns
+        -------
+        host : str
+            The GitLab host name
+        project : str
+            The Project ID or name to use
+        name : str
+            The package name
+        token_user : str
+            The token user according to the args. If ci is used, returns `JOB-TOKEN`, else
+            `PRIVATE-TOKEN`.
+        token : str
+            The token according to the args. If token is set, returns it. If netrc is set,
+            reads the token from the .netrc file. If ci is set, reads the environment
+            variable CI_JOB_TOKEN. Otherwise returns None.
+        """
+        if args.ci:
+            host = os.environ["CI_SERVER_HOST"]
+            project = os.environ["CI_PROJECT_ID"]
+            token = os.environ["CI_JOB_TOKEN"]
+            token_user = "JOB-TOKEN"
+            if args.project:
+                project = args.project
+            if args.token:
+                token = args.token
+                token_user = "PRIVATE-TOKEN"
+        else:
+            host = args.host
+            project = args.project
+            token = args.token
+            token_user = "PRIVATE-TOKEN"
+        if args.netrc:
+            _, _, token = netrc().authenticators(host)
+            token_user = "PRIVATE-TOKEN"
+        name = args.name
+        return host, project, name, token_user, token
+
+    def _download_handler(self, args: argparse.Namespace) -> int:
+        """
+        Downloads package file(s) from GitLab package registry.
+
+        Parameters
+        ----------
+        args : argparse.Namespace
+            The parsed arguments
+
+        Returns
+        -------
+        int
+            Zero if everything goes well, non-zero otherwise
+        """
+        ret = 1
+        host, project, name, token_user, token = self._args(args)
+        version = args.version
+        destination = args.destination
+        packages = Packages(host, token_user, token)
+        package_id = packages.get_id(project, name, version)
+        if package_id:
+            files = []
+            if args.file:
+                files.append(args.file)
+            else:
+                files = packages.get_files(project, package_id).keys()
+            for file in files:
+                ret = packages.download_file(project, name, version, file, destination)
+                if ret:
+                    print("Failed to download file " + file)
+                    break
+        else:
+            print("No package " + name + " version " + version + " found!")
+        return ret
+
+    def _register_list_parser(self, parser: argparse.ArgumentParser):
+        """
+        Registers the list command related arguments to the parser:
+            - action
+
+        Additionally, registers the common args.
+
+        Parameters
+        ----------
+        parser:
+            The argparser where to register the list arguments.
+        """
+        self._register_common_arguments(parser)
+        parser.set_defaults(action=self._list_packages)
+
+    def _list_packages(self, args: argparse.Namespace) -> int:
+        """
+        List package versions from GitLab package registry.
+
+        Parameters
+        ----------
+        args : argparse.Namespace
+            The parsed arguments
+
+        Returns
+        -------
+        int
+            Zero if everything goes well, non-zero otherwise
+        """
+        host, project, name, token_user, token = self._args(args)
+        packages = Packages(host, token_user, token)
+        package_list = packages.get_versions(project, name)
+        print("Name" + "\t\t" + "Version")
+        for package in package_list:
+            print(package["name"] + "\t" + package["version"])
+
+    def _register_upload_parser(self, parser: argparse.ArgumentParser):
+        """
+        Registers the upload command related arguments to the parser:
+            - version
+            - file
+            - action
+
+        Additionally, registers the common args.
+
+        Parameters
+        ----------
+        parser:
+            The argparser where to register the upload arguments.
+        """
+        self._register_common_arguments(parser)
+        parser.add_argument("-v", "--version", type=str, help="The package version.")
+        parser.add_argument(
+            "-f",
+            "--file",
+            type=str,
+            help="The file to be uploaded, for example my_file.txt. Note that "
+            "only relative paths (to the source) are supported and the relative "
+            "path is preserved when uploading the file. If left undefined, all files "
+            "of the source directory are uploaded. For example --source=temp --file=myfile "
+            "will upload myfile to the GitLab generic package root. However using --source=. "
+            "(or omittinge source) --file=temp/myfile will upload the file to temp folder "
+            "in the GitLab package.",
+        )
+        parser.add_argument(
+            "-s",
+            "--source",
+            type=str,
+            default="",
+            help="The source directory of the uploaded file(s). Defaults to current"
+            "working directory.",
+        )
+        parser.set_defaults(action=self._upload)
+
+    def _upload(self, args: argparse.Namespace) -> int:
+        """
+        Uploads a file to a GitLab package registry
+
+        Parameters
+        ----------
+        args : argparse.Namespace
+            The arguments from command line
+
+        Returns
+        -------
+        int
+            Zero if everything went fine, non-zero otherwise.
+        """
+        ret = 0
+        host, project, name, token_user, token = self._args(args)
+        version = args.version
+        file = args.file
+        source = args.source
+        if file:
+            # Check if the uploaded file exists.
+            if not os.path.isfile(os.path.join(source, file)):
+                print("File " + file + " does not exist!")
+                ret = 1
+        if not ret:
+            packages = Packages(host, token_user, token)
+            ret = packages.upload_file(project, name, version, file, source)
+        return ret
+
+    def _register_delete_parser(self, parser: argparse.ArgumentParser):
+        """
+        Registers the delete command related arguments to the parser:
+            - version
+            - file
+            - action
+
+        Additionally, registers the common args.
+
+        Parameters
+        ----------
+        parser:
+            The argparser where to register the upload arguments.
+        """
+        self._register_common_arguments(parser)
+        parser.add_argument("-v", "--version", type=str, help="The package version.")
+        parser.add_argument(
+            "-f",
+            "--file",
+            type=str,
+            help="The file to be deleted, for example my_file.txt. Note that "
+            "only relative paths (to the package root) are supported. If undefined, "
+            "the package version is deleted.",
+        )
+        parser.set_defaults(action=self._delete)
+
+    def _delete(self, args: argparse.Namespace) -> int:
+        """
+        Deletes a file from a GitLab generic package
+
+        Parameters
+        ----------
+        args : argparse.Namespace
+            The arguments from command line
+
+        Returns
+        -------
+        int
+            Zero if everything went fine, non-zero otherwise.
+        """
+        ret = 0
+        host, project, name, token_user, token = self._args(args)
+        version = args.version
+        file = args.file
+        packages = Packages(host, token_user, token)
+        if file:
+            ret = packages.delete_file(project, name, version, file)
+        else:
+            ret = packages.delete_package(project, name, version)
+        return ret