gitlabform 5.4.0__tar.gz → 5.5.0__tar.gz

{gitlabform-5.4.0 → gitlabform-5.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gitlabform
-Version: 5.4.0
+Version: 5.5.0
 Summary: 🏗 Specialized configuration as a code tool for GitLab projects, groups and more using hierarchical configuration written in YAML
 Project-URL: Homepage, https://gitlabform.github.io/gitlabform/
 Project-URL: Repository, https://github.com/gitlabform/gitlabform.git

{gitlabform-5.4.0 → gitlabform-5.5.0}/docs/changelog.md

@@ -5,6 +5,18 @@ This changelog follows industry-standard best practices. Each release includes t
 For details on how to migrate between major versions, please refer to the [upgrade guide](upgrade.md), which contains important instructions for breaking changes.
 
 ---
+## 5.5.0
+
+### Features
+
+* added support for group level branch protection settings. Resolves [#1049](https://github.com/gitlabform/gitlabform/issues/1049). [#1251](https://github.com/gitlabform/gitlabform/pull/1251), ([rickbrouwer](https://github.com/rickbrouwer)).
+
+### Dependencies
+
+* Update various dependencies to newer versions.
+
+Thanks to all the contributors of this release!
+
 ## 5.4.0
 
 ### Build

{gitlabform-5.4.0 → gitlabform-5.5.0}/docs/reference/protected_branches.md

@@ -2,7 +2,49 @@
 
 This section purpose is to manage the [protected branches](https://docs.gitlab.com/user/project/repository/branches/protected/).
 
-## Community Edition vs Enterprise Edition
+GitLabForm supports two different ways to manage branch protection:
+
+* **Group-level protected branches** (`group_protected_branches`) — Uses GitLab's [Group Protected Branches API](https://docs.gitlab.com/ee/api/group_protected_branches.html) to set branch protection rules that are inherited by all projects in a top-level group. Requires GitLab Premium.
+* **Project-level protected branches** (`branches`) — Sets branch protection rules on individual projects. When configured under a group wildcard (e.g. `group/*`), GitLabForm applies the rules to each project separately. Works with all GitLab tiers.
+
+## Group-level protected branches
+
+This section purpose is to manage the [group-level protected branches](https://docs.gitlab.com/ee/api/group_protected_branches.html).
+
+!!! info
+
+    This section requires GitLab Premium (paid). (This is a GitLab limitation, not GitLabForm's.)
+
+!!! warning
+
+    Protected branch settings for groups are restricted to top-level groups only. (This is a GitLab limitation, not GitLabForm's.) GitLabForm applies this section only to the top-level group; when the configuration is inherited by subgroups via the `group_1/*` wildcard, GitLabForm skips the subgroups.
+
+    Group protected branches only support access levels. Individual users and groups cannot be specified. (This is a GitLab API limitation).
+
+Example:
+
+```yaml
+projects_and_groups:
+  group_1/*:
+    group_protected_branches:
+      main:
+        protected: true
+        push_access_level: no access
+        merge_access_level: maintainer
+      release/*:
+        protected: true
+        push_access_level: no access
+        merge_access_level: maintainer
+        unprotect_access_level: maintainer
+```
+
+!!! note
+
+    A group-level protected branch also appears under the protected branches list of every project in the group, marked with a lock icon. Inherited rules cannot be edited or removed from the project. Projects can still define their own additional protected branch rules on top of the inherited ones via the project-level [`branches`](#project-level-protected-branches) section.
+
+## Project-level protected branches
+
+### Community Edition vs Enterprise Edition
 Note: that Gitlab Community Edition does not support setting `unprotect_access_level` and will always return `None` from the API, and there is also no way to manually set this through the UI.
 
 ### Functionality Differences
@@ -13,7 +55,7 @@ In Gitlab EE versions <=15.6.0 and Gitlab Community Edition, GitLabForm uses old
 
 In later versions of Gitlab EE, GitLabForm will modify the Branch Protection rules in-place, see the [V4->V5 upgrade notes](../upgrade.md)
 
-## Common features
+### Common features
 
 The key names here may be:
 
@@ -63,7 +105,7 @@ projects_and_groups:
         allow_force_push: true
 ```
 
-## Premium-only features
+### Premium-only features
 
 !!! info
 
@@ -106,4 +148,4 @@ projects_and_groups:
           - group_id: 456 # ...or group ids, if you know them...
         allowed_to_unprotect:
           - access_level: maintainer # ...or the whole access levels
-```
+```

{gitlabform-5.4.0 → gitlabform-5.5.0}/gitlabform/configuration/groups.py

@@ -24,8 +24,7 @@ class ConfigurationGroups(ConfigurationCommon, ABC):
         for element in projects_and_groups.keys():
             if element.endswith("/*"):
                 # cut off that "/*"
-                group_name = element[:-2]
-                groups.append(group_name)
+                groups.append(element[:-2])
         return sorted(groups)
 
     def is_group_skipped(self, group):

{gitlabform-5.4.0 → gitlabform-5.5.0}/gitlabform/processors/group/__init__.py

@@ -30,6 +30,9 @@ from gitlabform.processors.group.group_push_rules_processor import (
 from gitlabform.processors.group.group_hooks_processor import (
     GroupHooksProcessor,
 )
+from gitlabform.processors.group.group_protected_branches_processor import (
+    GroupProtectedBranchesProcessor,
+)
 
 
 class GroupProcessors(AbstractProcessors):
@@ -45,4 +48,5 @@ class GroupProcessors(AbstractProcessors):
             GroupLabelsProcessor(gitlab),
             GroupHooksProcessor(gitlab),
             GroupPushRulesProcessor(gitlab),
+            GroupProtectedBranchesProcessor(gitlab, strict),
         ]

gitlabform-5.5.0/gitlabform/processors/group/group_protected_branches_processor.py

@@ -0,0 +1,146 @@
+import sys
+from typing import Optional
+
+from logging import info, warning, error, critical, debug
+from gitlab import GitlabGetError, GitlabDeleteError, GitlabOperationError
+from gitlab.v4.objects.branches import GroupProtectedBranch
+from gitlab.v4.objects.groups import Group
+
+from gitlabform.constants import EXIT_INVALID_INPUT, EXIT_PROCESSING_ERROR
+from gitlabform.gitlab import GitLab
+from gitlabform.processors.abstract_processor import AbstractProcessor
+from gitlabform.processors.util.branch_protection import BranchProtection
+
+
+class GroupProtectedBranchesProcessor(AbstractProcessor):
+
+    def __init__(self, gitlab: GitLab, strict: bool):
+        super().__init__("group_protected_branches", gitlab)
+        self.strict = strict
+
+        self.custom_diff_analyzers["merge_access_levels"] = BranchProtection.naive_access_level_diff_analyzer
+        self.custom_diff_analyzers["push_access_levels"] = BranchProtection.naive_access_level_diff_analyzer
+        self.custom_diff_analyzers["unprotect_access_levels"] = BranchProtection.naive_access_level_diff_analyzer
+
+    def _can_proceed(self, group: str, configuration: dict):
+        if "/" in group:
+            debug(f"Skipping group_protected_branches for '{group}' - only supported for top-level groups")
+            return False
+
+        for branch in sorted(configuration["group_protected_branches"]):
+            branch_config = configuration["group_protected_branches"][branch]
+            if branch_config.get("protected") is None:
+                critical(
+                    f"The Protected key is mandatory in group_protected_branches configuration, fix {branch} YAML config"
+                )
+                sys.exit(EXIT_INVALID_INPUT)
+
+            for key in ("allowed_to_push", "allowed_to_merge", "allowed_to_unprotect"):
+                if BranchProtection.branch_protection_config_contains_user_or_group(branch_config, key):
+                    warning(
+                        f"Group protected branches do not support individual users or groups in '{key}' for branch '{branch}'. "
+                        f"Only access_level is supported. The user/group entry will be ignored by GitLab."
+                    )
+
+        return True
+
+    def _process_configuration(self, group: str, configuration: dict):
+
+        gitlab_group: Group = self.gl.get_group_by_path_cached(group)
+
+        for branch in sorted(configuration["group_protected_branches"]):
+            branch_configuration: dict = configuration["group_protected_branches"][branch]
+
+            self.process_branch_protection(gitlab_group, branch, branch_configuration)
+
+    def process_branch_protection(self, group: Group, branch_name: str, branch_config: dict):
+        protected_branch: Optional[GroupProtectedBranch] = None
+
+        try:
+            protected_branch = group.protectedbranches.get(branch_name)
+        except GitlabGetError:
+            info(f"The branch '{branch_name}' is not protected at group level!")
+
+        if branch_config.get("protected"):
+            if not protected_branch:
+                info(f"Creating group-level branch protection for {branch_name}")
+                self.protect_branch(group, branch_name, branch_config, False)
+                return
+
+            transformed_branch_config = BranchProtection.map_config_to_protected_branch_get_data(branch_config)
+
+            protected_branch_api_patch_data: dict = {}
+
+            special_list_keys = [
+                "merge_access_levels",
+                "push_access_levels",
+                "unprotect_access_levels",
+            ]
+            for key, value in transformed_branch_config.items():
+                if key not in special_list_keys:
+                    existing_value = getattr(protected_branch, key, None)
+                    if existing_value != value:
+                        info(f"Creating data to update {key} as necessary")
+                        protected_branch_api_patch_data[key] = value
+
+            info("Creating data to update merge_access_levels as necessary")
+            merge_access_items_patch_data = BranchProtection.build_patch_request_data(
+                transformed_access_levels=transformed_branch_config.get("merge_access_levels"),
+                existing_records=tuple(BranchProtection.get_list_attribute(protected_branch, "merge_access_levels")),
+            )
+            if len(merge_access_items_patch_data) > 0:
+                protected_branch_api_patch_data["allowed_to_merge"] = merge_access_items_patch_data
+
+            info("Creating data to update push_access_levels as necessary")
+            push_access_items_patch_data = BranchProtection.build_patch_request_data(
+                transformed_access_levels=transformed_branch_config.get("push_access_levels"),
+                existing_records=tuple(BranchProtection.get_list_attribute(protected_branch, "push_access_levels")),
+            )
+            if len(push_access_items_patch_data) > 0:
+                protected_branch_api_patch_data["allowed_to_push"] = push_access_items_patch_data
+
+            info("Creating data to update unprotect_access_levels as necessary")
+            unprotect_access_items_patch_data = BranchProtection.build_patch_request_data(
+                transformed_access_levels=transformed_branch_config.get("unprotect_access_levels"),
+                existing_records=tuple(
+                    BranchProtection.get_list_attribute(protected_branch, "unprotect_access_levels")
+                ),
+            )
+            if len(unprotect_access_items_patch_data) > 0:
+                protected_branch_api_patch_data["allowed_to_unprotect"] = unprotect_access_items_patch_data
+
+            if protected_branch_api_patch_data != {}:
+                info(f"Updating group-level protected branch {branch_name} with {protected_branch_api_patch_data}")
+                self.protect_branch(group, branch_name, protected_branch_api_patch_data, True)
+
+        elif protected_branch and not branch_config.get("protected"):
+            info(f"Removing group-level branch protection for {branch_name}")
+            self.unprotect_branch(protected_branch)
+
+    def protect_branch(self, group: Group, branch_name: str, branch_config: dict, update_only: bool = False):
+        try:
+            if update_only:
+                group.protectedbranches.update(branch_name, branch_config)
+            else:
+                group.protectedbranches.create({"name": branch_name, **branch_config})
+        except GitlabOperationError as e:
+            message = f"Protecting branch '{branch_name}' at group level failed! Error '{e.error_message}"
+
+            if self.strict:
+                critical(message)
+                sys.exit(EXIT_PROCESSING_ERROR)
+            else:
+                error(message)
+
+    def unprotect_branch(self, protected_branch: GroupProtectedBranch):
+        try:
+            protected_branch.delete()
+        except GitlabDeleteError as e:
+            message = (
+                f"Branch '{protected_branch.name}' could not be unprotected at group level! Error '{e.error_message}'"
+            )
+            if self.strict:
+                critical(message)
+                sys.exit(EXIT_PROCESSING_ERROR)
+            else:
+                warning(message)

gitlabform-5.5.0/gitlabform/processors/project/branches_processor.py

@@ -0,0 +1,277 @@
+import sys
+from typing import Optional, Any
+
+from logging import info, warning, error, critical
+from gitlab import (
+    GitlabGetError,
+    GitlabDeleteError,
+    GitlabOperationError,
+)
+from gitlab.v4.objects import Project, ProjectProtectedBranch
+
+from gitlabform.constants import EXIT_INVALID_INPUT, EXIT_PROCESSING_ERROR
+from gitlabform.gitlab import GitLab
+from gitlabform.processors.abstract_processor import AbstractProcessor
+from gitlabform.processors.util.branch_protection import BranchProtection
+
+
+class BranchesProcessor(AbstractProcessor):
+    """
+    Processor for branch protection settings.
+
+    This processor is complex because GitLab's Protected Branches API uses different
+    data structures for Create (POST), Get (GET), and Update (PATCH) operations.
+
+    It implements 'Additive Design' (existing rules are preserved) and
+    'Raw Parameter Passing' (arbitrary keys in config are sent to the API).
+    """
+
+    def __init__(self, gitlab: GitLab, strict: bool):
+        super().__init__("branches", gitlab)
+        self.strict = strict
+
+        # Protected Branch API: https://docs.gitlab.com/api/protected_branches/#update-a-protected-branch
+        # Behind the scenes gitlab will map "allowed_to_merge" and "merge_access_level" to "merge_access_levels"
+        # and the same for unprotect and push access, so we need some custom diff analyzers to validate if the config
+        # has actually changed
+        self.custom_diff_analyzers["merge_access_levels"] = BranchProtection.naive_access_level_diff_analyzer
+        self.custom_diff_analyzers["push_access_levels"] = BranchProtection.naive_access_level_diff_analyzer
+        self.custom_diff_analyzers["unprotect_access_levels"] = BranchProtection.naive_access_level_diff_analyzer
+
+    def _can_proceed(self, project_or_group: str, configuration: dict):
+        for branch in sorted(configuration["branches"]):
+            branch_config = configuration["branches"][branch]
+            if branch_config.get("protected") is None:
+                critical(f"The Protected key is mandatory in branches configuration, fix {branch} YAML config")
+                sys.exit(EXIT_INVALID_INPUT)
+
+        return True
+
+    def _process_configuration(self, project_and_group: str, configuration: dict):
+        """
+        Called from process defined in abstract_processor.py after checking self._can_proceed
+        Iterates through all branches defined in the configuration and applies protection rules.
+        """
+
+        project: Project = self.gl.get_project_by_path_cached(project_and_group)
+
+        for branch in sorted(configuration["branches"]):
+            branch_configuration: dict = self.convert_user_and_group_names_to_ids(configuration["branches"][branch])
+
+            self.process_branch_protection(project, branch, branch_configuration)
+
+    def process_branch_protection(self, project: Project, branch_name: str, branch_config: dict):
+        """
+        High-level logic for processing branch protection.
+
+        1. Validates branch existence (unless wildcard).
+        2. Handles 'protected: false' (unprotecting).
+        3. Handles 'protected: true':
+           - If not currently protected: Create protection.
+           - If protected: Update using PATCH (EE > 15.6) or Unprotect/Reprotect (CE/Old EE).
+        """
+        protected_branch: Optional[ProjectProtectedBranch] = None
+
+        # If protected branch name contains a supported wildcard do not try looking it up
+        if not self.branch_name_contains_supported_wildcard(branch_name):
+            # Check branch we are trying to protect actually exists first
+            try:
+                project.branches.get(branch_name)
+            except GitlabGetError:
+                message = f"Branch '{branch_name}' not found when processing it to be protected/unprotected!"
+                if self.strict:
+                    critical(message)
+                    sys.exit(EXIT_INVALID_INPUT)
+                else:
+                    warning(message)
+
+        try:
+            protected_branch = project.protectedbranches.get(branch_name)
+        except GitlabGetError:
+            message = f"The branch '{branch_name}' is not protected!"
+            info(message)
+
+        if branch_config.get("protected"):
+            if not protected_branch:
+                info(f"Creating branch protection for {branch_name}")
+                self.protect_branch(project, branch_name, branch_config, False)
+                return
+
+            # https://docs.gitlab.com/api/protected_branches/#update-a-protected-branch was only introduced after 15.6
+            # for user's on older versions of Gitlab or Community Edition (https://gitlab.com/rluna-gitlab/gitlab-ce/-/work_items/37)
+            # We need to unprotect and then reprotect the branch to apply the
+            # defined configuration
+            if self.gitlab.is_version_less_than("15.6.0") or (self.gitlab.enterprise == False):
+                self.process_branch_config_gitlab_under_15_6_0_or_ce(
+                    branch_config, branch_name, project, protected_branch
+                )
+                return
+
+            # For later Gitlab versions we dynamically generate the data to send to the update endpoint based on the
+            # configured state and the current Gitlab state so do not need to check _needs_update first.
+
+            # Gitlab returns the allowed_to_merge etc. data in a different format from GET endpoint than it takes in
+            # the POST (create) or PATCH (update) endpoints
+            # GET: https://docs.gitlab.com/api/protected_branches/#get-a-single-protected-branch-or-wildcard-protected-branch
+            # POST: https://docs.gitlab.com/api/protected_branches/#protect-repository-branches
+            # PATCH: https://docs.gitlab.com/api/protected_branches/#update-a-protected-branch
+            # Therefore we first transform the configured YAML into a state matching the gitlab GET endpoint so we can
+            # compare states easier to determine what PATCH request we need to send (if any)
+            transformed_branch_config = BranchProtection.map_config_to_protected_branch_get_data(branch_config)
+
+            protected_branch_api_patch_data: dict = {}
+
+            # RAW PARAMETER PASSING (Top-Level):
+            # Iterate through any top-level flags (e.g., allow_force_push, code_owner_approval_required)
+            # and include them in the PATCH request if they differ from the current GitLab state.
+            special_list_keys = [
+                "merge_access_levels",
+                "push_access_levels",
+                "unprotect_access_levels",
+            ]
+            for key, value in transformed_branch_config.items():
+                if key not in special_list_keys:
+                    # Check if this attribute exists on the GitLab object and needs an update
+                    existing_value = getattr(protected_branch, key, None)
+                    if existing_value != value:
+                        info(f"Creating data to update {key} as necessary")
+                        protected_branch_api_patch_data[key] = value
+
+            info("Creating data to update merge_access_levels as necessary")
+            merge_access_items_patch_data = BranchProtection.build_patch_request_data(
+                transformed_access_levels=transformed_branch_config.get("merge_access_levels"),
+                existing_records=tuple(BranchProtection.get_list_attribute(protected_branch, "merge_access_levels")),
+            )
+            if len(merge_access_items_patch_data) > 0:
+                protected_branch_api_patch_data["allowed_to_merge"] = merge_access_items_patch_data
+
+            info("Creating data to update push_access_levels as necessary")
+            push_access_items_patch_data = BranchProtection.build_patch_request_data(
+                transformed_access_levels=transformed_branch_config.get("push_access_levels"),
+                existing_records=tuple(BranchProtection.get_list_attribute(protected_branch, "push_access_levels")),
+            )
+            if len(push_access_items_patch_data) > 0:
+                protected_branch_api_patch_data["allowed_to_push"] = push_access_items_patch_data
+
+            info("Creating data to update unprotect_access_levels as necessary")
+
+            unprotect_access_items_patch_data = BranchProtection.build_patch_request_data(
+                transformed_access_levels=transformed_branch_config.get("unprotect_access_levels"),
+                existing_records=tuple(
+                    BranchProtection.get_list_attribute(protected_branch, "unprotect_access_levels")
+                ),
+            )
+
+            if len(unprotect_access_items_patch_data) > 0:
+                protected_branch_api_patch_data["allowed_to_unprotect"] = unprotect_access_items_patch_data
+
+            if protected_branch_api_patch_data != {}:
+                # We have some updates to make
+                info(f"Updating protected branch {branch_name} with {protected_branch_api_patch_data}")
+                self.protect_branch(project, branch_name, protected_branch_api_patch_data, True)
+
+        elif protected_branch and not branch_config.get("protected"):
+            info(f"Removing branch protection for {branch_name}")
+            self.unprotect_branch(protected_branch)
+
+    def process_branch_config_gitlab_under_15_6_0_or_ce(self, branch_config, branch_name, project, protected_branch):
+        """
+        Processes the branches configuration for gitlab version <=15.6.0 or Community Edition,
+        where in-place updates (PATCH) are not supported or effective.
+        If a change is detected, the branch is unprotected and then reprotected from scratch.
+        """
+
+        # Gitlab returns the allowed_to_merge etc data in a different format from GET endpoint than it takes in
+        # the POST (create) endpoint
+        # GET: https://docs.gitlab.com/api/protected_branches/#get-a-single-protected-branch-or-wildcard-protected-branch
+        # POST: https://docs.gitlab.com/api/protected_branches/#protect-repository-branches
+        # Therefore we first transform the configured YAML into a state matching the gitlab GET endpoint,
+        # before checking if it needs_update
+        if self._needs_update(
+            protected_branch.attributes, BranchProtection.map_config_to_protected_branch_get_data(branch_config)
+        ):
+            info(
+                f"Gitlab version is less than 15.6.0, so un-protecting and reprotecting branch {branch_name} to apply new config..."
+            )
+            self.unprotect_branch(protected_branch)
+
+            # Send the untransformed config to the POST endpoint, as GitlabForm YAML structure conforms to the POST inputs
+            self.protect_branch(project, branch_name, branch_config, False)
+
+    def protect_branch(self, project: Project, branch_name: str, branch_config: dict, update_only: bool = False):
+        """
+        Create or update branch protection using given config.
+        Raise exception if running in strict mode.
+
+        args:
+            project (Project): Gitlab project
+            branch_name (str): Name of branch on the project to protect or update protection on
+            branch_config (dict): Branch protection configuration to apply
+            update_only (bool):
+                If True, update branch protection of branch with branch_name,
+                If False create a new protected branch with branch_name
+        """
+        try:
+            if update_only:
+                project.protectedbranches.update(branch_name, branch_config)
+            else:
+                project.protectedbranches.create({"name": branch_name, **branch_config})
+        except GitlabOperationError as e:
+            message = f"Protecting branch '{branch_name}' failed! Error '{e.error_message}"
+
+            if self.strict:
+                critical(message)
+                sys.exit(EXIT_PROCESSING_ERROR)
+            else:
+                error(message)
+
+    def unprotect_branch(self, protected_branch: ProjectProtectedBranch):
+        """
+        Unprotect a branch.
+        Raise exception if running in strict mode.
+        """
+        try:
+            # The delete method doesn't delete the actual branch.
+            # It only unprotects the branch.
+            protected_branch.delete()
+        except GitlabDeleteError as e:
+            message = f"Branch '{protected_branch.name}' could not be unprotected! Error '{e.error_message}'"
+            if self.strict:
+                critical(message)
+                sys.exit(EXIT_PROCESSING_ERROR)
+            else:
+                warning(message)
+
+    def convert_user_and_group_names_to_ids(self, branch_config: dict):
+        """
+        Pre-processor to resolve names to IDs.
+        Translates 'user: username' or 'group: name' into 'user_id' or 'group_id' as
+        config by replacing them with ids.
+        """
+        info("Transforming User and Group names in Branch configuration to Ids")
+
+        for key in branch_config:
+            if isinstance(branch_config[key], list):
+                for item in branch_config[key]:
+                    if isinstance(item, dict):
+                        if "user" in item:
+                            username = item.pop("user")
+                            user_id = self.gl.get_user_id_cached(username)
+                            if user_id is None:
+                                raise GitlabGetError(
+                                    f"transform_branch_config - No users found when searching for username {username}",
+                                    404,
+                                )
+                            item["user_id"] = user_id
+                        elif "group" in item:
+                            item["group_id"] = self.gl.get_group_id(item.pop("group"))
+
+        return branch_config
+
+    @staticmethod
+    def branch_name_contains_supported_wildcard(branch):
+        """
+        Gitlab supports "*" wildcards when protecting branches:
+        https://docs.gitlab.com/user/project/repository/branches/protected/#use-wildcard-rules
+        """
+        return "*" in branch