apache-airflow-providers-git 0.0.8__tar.gz → 0.0.9rc1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of apache-airflow-providers-git might be problematic. Click here for more details.

Files changed (31) hide show
  1. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/PKG-INFO +34 -13
  2. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/README.rst +29 -9
  3. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/changelog.rst +21 -4
  4. apache_airflow_providers_git-0.0.9rc1/docs/index.rst +129 -0
  5. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/provider.yaml +2 -1
  6. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/pyproject.toml +6 -4
  7. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/__init__.py +1 -1
  8. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/bundles/git.py +60 -19
  9. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/hooks/git.py +1 -1
  10. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/version_compat.py +1 -6
  11. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/unit/git/bundles/test_git.py +64 -2
  12. apache_airflow_providers_git-0.0.8/docs/index.rst +0 -100
  13. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/.latest-doc-only-change.txt +0 -0
  14. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/bundles/index.rst +0 -0
  15. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/commits.rst +0 -0
  16. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/conf.py +0 -0
  17. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/connections/git.rst +0 -0
  18. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/installing-providers-from-sources.rst +0 -0
  19. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/docs/security.rst +0 -0
  20. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/__init__.py +0 -0
  21. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/__init__.py +0 -0
  22. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/LICENSE +0 -0
  23. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/bundles/__init__.py +0 -0
  24. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/get_provider_info.py +0 -0
  25. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/src/airflow/providers/git/hooks/__init__.py +0 -0
  26. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/conftest.py +0 -0
  27. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/unit/__init__.py +0 -0
  28. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/unit/git/__init__.py +0 -0
  29. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/unit/git/bundles/__init__.py +0 -0
  30. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/unit/git/hooks/__init__.py +0 -0
  31. {apache_airflow_providers_git-0.0.8 → apache_airflow_providers_git-0.0.9rc1}/tests/unit/git/hooks/test_git.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: apache-airflow-providers-git
3
- Version: 0.0.8
3
+ Version: 0.0.9rc1
4
4
  Summary: Provider package apache-airflow-providers-git for Apache Airflow
5
5
  Keywords: airflow-provider,git,airflow,integration
6
6
  Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -20,11 +20,12 @@ Classifier: Programming Language :: Python :: 3.11
20
20
  Classifier: Programming Language :: Python :: 3.12
21
21
  Classifier: Programming Language :: Python :: 3.13
22
22
  Classifier: Topic :: System :: Monitoring
23
- Requires-Dist: apache-airflow>=3.0.0
23
+ Requires-Dist: apache-airflow>=3.0.0rc1
24
+ Requires-Dist: apache-airflow-providers-common-compat>=1.8.0rc1
24
25
  Requires-Dist: GitPython>=3.1.44
25
26
  Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
26
- Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8/changelog.html
27
- Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8
27
+ Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-git/0.0.9/changelog.html
28
+ Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-git/0.0.9
28
29
  Project-URL: Mastodon, https://fosstodon.org/@airflow
29
30
  Project-URL: Slack Chat, https://s.apache.org/airflow-slack
30
31
  Project-URL: Source Code, https://github.com/apache/airflow
@@ -55,7 +56,7 @@ Project-URL: YouTube, https://www.youtube.com/channel/UCSXwxpWZQ7XZ1WL3wqevChA/
55
56
 
56
57
  Package ``apache-airflow-providers-git``
57
58
 
58
- Release: ``0.0.8``
59
+ Release: ``0.0.9``
59
60
 
60
61
 
61
62
  `Distributed version control system (GIT) <https://git-scm.com/>`__
@@ -68,7 +69,7 @@ This is a provider package for ``git`` provider. All classes for this provider p
68
69
  are in ``airflow.providers.git`` python package.
69
70
 
70
71
  You can find package information and changelog for the provider
71
- in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8/>`_.
72
+ in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.9/>`_.
72
73
 
73
74
  Installation
74
75
  ------------
@@ -82,13 +83,33 @@ The package supports the following python versions: 3.10,3.11,3.12,3.13
82
83
  Requirements
83
84
  ------------
84
85
 
85
- ================== ==================
86
- PIP package Version required
87
- ================== ==================
88
- ``apache-airflow`` ``>=3.0.0``
89
- ``GitPython`` ``>=3.1.44``
90
- ================== ==================
86
+ ========================================== ==================
87
+ PIP package Version required
88
+ ========================================== ==================
89
+ ``apache-airflow`` ``>=3.0.0``
90
+ ``apache-airflow-providers-common-compat`` ``>=1.8.0``
91
+ ``GitPython`` ``>=3.1.44``
92
+ ========================================== ==================
93
+
94
+ Cross provider package dependencies
95
+ -----------------------------------
96
+
97
+ Those are dependencies that might be needed in order to use all the features of the package.
98
+ You need to install the specified providers in order to use them.
99
+
100
+ You can install such cross-provider dependencies when installing from PyPI. For example:
101
+
102
+ .. code-block:: bash
103
+
104
+ pip install apache-airflow-providers-git[common.compat]
105
+
106
+
107
+ ================================================================================================================== =================
108
+ Dependent package Extra
109
+ ================================================================================================================== =================
110
+ `apache-airflow-providers-common-compat <https://airflow.apache.org/docs/apache-airflow-providers-common-compat>`_ ``common.compat``
111
+ ================================================================================================================== =================
91
112
 
92
113
  The changelog for the provider package can be found in the
93
- `changelog <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8/changelog.html>`_.
114
+ `changelog <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.9/changelog.html>`_.
94
115
 
@@ -23,7 +23,7 @@
23
23
 
24
24
  Package ``apache-airflow-providers-git``
25
25
 
26
- Release: ``0.0.8``
26
+ Release: ``0.0.9``
27
27
 
28
28
 
29
29
  `Distributed version control system (GIT) <https://git-scm.com/>`__
@@ -36,7 +36,7 @@ This is a provider package for ``git`` provider. All classes for this provider p
36
36
  are in ``airflow.providers.git`` python package.
37
37
 
38
38
  You can find package information and changelog for the provider
39
- in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8/>`_.
39
+ in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.9/>`_.
40
40
 
41
41
  Installation
42
42
  ------------
@@ -50,12 +50,32 @@ The package supports the following python versions: 3.10,3.11,3.12,3.13
50
50
  Requirements
51
51
  ------------
52
52
 
53
- ================== ==================
54
- PIP package Version required
55
- ================== ==================
56
- ``apache-airflow`` ``>=3.0.0``
57
- ``GitPython`` ``>=3.1.44``
58
- ================== ==================
53
+ ========================================== ==================
54
+ PIP package Version required
55
+ ========================================== ==================
56
+ ``apache-airflow`` ``>=3.0.0``
57
+ ``apache-airflow-providers-common-compat`` ``>=1.8.0``
58
+ ``GitPython`` ``>=3.1.44``
59
+ ========================================== ==================
60
+
61
+ Cross provider package dependencies
62
+ -----------------------------------
63
+
64
+ Those are dependencies that might be needed in order to use all the features of the package.
65
+ You need to install the specified providers in order to use them.
66
+
67
+ You can install such cross-provider dependencies when installing from PyPI. For example:
68
+
69
+ .. code-block:: bash
70
+
71
+ pip install apache-airflow-providers-git[common.compat]
72
+
73
+
74
+ ================================================================================================================== =================
75
+ Dependent package Extra
76
+ ================================================================================================================== =================
77
+ `apache-airflow-providers-common-compat <https://airflow.apache.org/docs/apache-airflow-providers-common-compat>`_ ``common.compat``
78
+ ================================================================================================================== =================
59
79
 
60
80
  The changelog for the provider package can be found in the
61
- `changelog <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8/changelog.html>`_.
81
+ `changelog <https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.9/changelog.html>`_.
@@ -19,11 +19,30 @@
19
19
  Changelog
20
20
  ---------
21
21
 
22
- 0.0.8
22
+ 0.0.9
23
23
  .....
24
24
 
25
+ Bug Fixes
26
+ ~~~~~~~~~
27
+
28
+ * ``Fix corrupted bare Git repository recovery in DAG bundles (#56206)``
29
+
30
+ Misc
31
+ ~~~~
32
+
33
+ * ``Migrate git provider to ''common.compat'' (#57008)``
34
+
35
+ Doc-only
36
+ ~~~~~~~~
37
+
38
+ * ``Remove placeholder Release Date in changelog and index files (#56056)``
39
+
40
+ .. Below changes are excluded from the changelog. Move them to
41
+ appropriate section above if needed. Do not delete the lines(!):
42
+
43
+ 0.0.8
44
+ .....
25
45
 
26
- Release Date: ``|PypiReleaseDate|``
27
46
 
28
47
  Bug Fixes
29
48
  ~~~~~~~~~
@@ -37,8 +56,6 @@ Bug Fixes
37
56
  .....
38
57
 
39
58
 
40
- Release Date: ``|PypiReleaseDate|``
41
-
42
59
  Bug Fixes
43
60
  ~~~~~~~~~
44
61
 
@@ -0,0 +1,129 @@
1
+
2
+ .. Licensed to the Apache Software Foundation (ASF) under one
3
+ or more contributor license agreements. See the NOTICE file
4
+ distributed with this work for additional information
5
+ regarding copyright ownership. The ASF licenses this file
6
+ to you under the Apache License, Version 2.0 (the
7
+ "License"); you may not use this file except in compliance
8
+ with the License. You may obtain a copy of the License at
9
+
10
+ .. http://www.apache.org/licenses/LICENSE-2.0
11
+
12
+ .. Unless required by applicable law or agreed to in writing,
13
+ software distributed under the License is distributed on an
14
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15
+ KIND, either express or implied. See the License for the
16
+ specific language governing permissions and limitations
17
+ under the License.
18
+
19
+ ``apache-airflow-providers-git``
20
+ ================================
21
+
22
+
23
+ .. toctree::
24
+ :hidden:
25
+ :maxdepth: 1
26
+ :caption: Basics
27
+
28
+ Home <self>
29
+ Changelog <changelog>
30
+ Security <security>
31
+
32
+ .. toctree::
33
+ :hidden:
34
+ :maxdepth: 1
35
+ :caption: References
36
+
37
+ Connection Types <connections/git>
38
+ Python API <_api/airflow/providers/git/index>
39
+
40
+ .. toctree::
41
+ :hidden:
42
+ :maxdepth: 1
43
+ :caption: Bundles
44
+
45
+ Bundles <bundles/index>
46
+
47
+ .. toctree::
48
+ :hidden:
49
+ :maxdepth: 1
50
+ :caption: Resources
51
+
52
+ PyPI Repository <https://pypi.org/project/apache-airflow-providers-git/>
53
+ Installing from sources <installing-providers-from-sources>
54
+ Python API <_api/airflow/providers/git/index>
55
+
56
+
57
+
58
+ .. THE REMAINDER OF THE FILE IS AUTOMATICALLY GENERATED. IT WILL BE OVERWRITTEN AT RELEASE TIME!
59
+
60
+
61
+ .. toctree::
62
+ :hidden:
63
+ :maxdepth: 1
64
+ :caption: Commits
65
+
66
+ Detailed list of commits <commits>
67
+
68
+
69
+ apache-airflow-providers-git package
70
+ ------------------------------------------------------
71
+
72
+ `Distributed version control system (GIT) <https://git-scm.com/>`__
73
+
74
+
75
+ Release: 0.0.9
76
+
77
+ Provider package
78
+ ----------------
79
+
80
+ This package is for the ``git`` provider.
81
+ All classes for this package are included in the ``airflow.providers.git`` python package.
82
+
83
+ Installation
84
+ ------------
85
+
86
+ You can install this package on top of an existing Airflow installation via
87
+ ``pip install apache-airflow-providers-git``.
88
+ For the minimum Airflow version supported, see ``Requirements`` below.
89
+
90
+ Requirements
91
+ ------------
92
+
93
+ The minimum Apache Airflow version supported by this provider distribution is ``3.0.0``.
94
+
95
+ ========================================== ==================
96
+ PIP package Version required
97
+ ========================================== ==================
98
+ ``apache-airflow`` ``>=3.0.0``
99
+ ``apache-airflow-providers-common-compat`` ``>=1.8.0``
100
+ ``GitPython`` ``>=3.1.44``
101
+ ========================================== ==================
102
+
103
+ Cross provider package dependencies
104
+ -----------------------------------
105
+
106
+ Those are dependencies that might be needed in order to use all the features of the package.
107
+ You need to install the specified provider distributions in order to use them.
108
+
109
+ You can install such cross-provider dependencies when installing from PyPI. For example:
110
+
111
+ .. code-block:: bash
112
+
113
+ pip install apache-airflow-providers-git[common.compat]
114
+
115
+
116
+ ================================================================================================================== =================
117
+ Dependent package Extra
118
+ ================================================================================================================== =================
119
+ `apache-airflow-providers-common-compat <https://airflow.apache.org/docs/apache-airflow-providers-common-compat>`_ ``common.compat``
120
+ ================================================================================================================== =================
121
+
122
+ Downloading official packages
123
+ -----------------------------
124
+
125
+ You can download officially released packages and verify their checksums and signatures from the
126
+ `Official Apache Download site <https://downloads.apache.org/airflow/providers/>`_
127
+
128
+ * `The apache-airflow-providers-git 0.0.9 sdist package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_git-0.0.9.tar.gz>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_git-0.0.9.tar.gz.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_git-0.0.9.tar.gz.sha512>`__)
129
+ * `The apache-airflow-providers-git 0.0.9 wheel package <https://downloads.apache.org/airflow/providers/apache_airflow_providers_git-0.0.9-py3-none-any.whl>`_ (`asc <https://downloads.apache.org/airflow/providers/apache_airflow_providers_git-0.0.9-py3-none-any.whl.asc>`__, `sha512 <https://downloads.apache.org/airflow/providers/apache_airflow_providers_git-0.0.9-py3-none-any.whl.sha512>`__)
@@ -22,12 +22,13 @@ description: |
22
22
  `Distributed version control system (GIT) <https://git-scm.com/>`__
23
23
 
24
24
  state: ready
25
- source-date-epoch: 1757950328
25
+ source-date-epoch: 1761116758
26
26
  # Note that those versions are maintained by release manager - do not update them manually
27
27
  # with the exception of case where other provider in sources has >= new provider version.
28
28
  # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
29
29
  # to be done in the same PR
30
30
  versions:
31
+ - 0.0.9
31
32
  - 0.0.8
32
33
  - 0.0.7
33
34
  - 0.0.6
@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
25
25
 
26
26
  [project]
27
27
  name = "apache-airflow-providers-git"
28
- version = "0.0.8"
28
+ version = "0.0.9rc1"
29
29
  description = "Provider package apache-airflow-providers-git for Apache Airflow"
30
30
  readme = "README.rst"
31
31
  authors = [
@@ -57,7 +57,8 @@ requires-python = ">=3.10"
57
57
  # Make sure to run ``prek update-providers-dependencies --all-files``
58
58
  # After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
59
59
  dependencies = [
60
- "apache-airflow>=3.0.0",
60
+ "apache-airflow>=3.0.0rc1",
61
+ "apache-airflow-providers-common-compat>=1.8.0rc1",
61
62
  "GitPython>=3.1.44",
62
63
  ]
63
64
 
@@ -66,6 +67,7 @@ dev = [
66
67
  "apache-airflow",
67
68
  "apache-airflow-task-sdk",
68
69
  "apache-airflow-devel-common",
70
+ "apache-airflow-providers-common-compat",
69
71
  # Additional devel dependencies (do not remove this line and add extra development dependencies)
70
72
  ]
71
73
 
@@ -95,8 +97,8 @@ apache-airflow-providers-common-sql = {workspace = true}
95
97
  apache-airflow-providers-standard = {workspace = true}
96
98
 
97
99
  [project.urls]
98
- "Documentation" = "https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8"
99
- "Changelog" = "https://airflow.apache.org/docs/apache-airflow-providers-git/0.0.8/changelog.html"
100
+ "Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-git/0.0.9"
101
+ "Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-git/0.0.9/changelog.html"
100
102
  "Bug Tracker" = "https://github.com/apache/airflow/issues"
101
103
  "Source Code" = "https://github.com/apache/airflow"
102
104
  "Slack Chat" = "https://s.apache.org/airflow-slack"
@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
29
29
 
30
30
  __all__ = ["__version__"]
31
31
 
32
- __version__ = "0.0.8"
32
+ __version__ = "0.0.9"
33
33
 
34
34
  if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
35
35
  "3.0.0"
@@ -17,13 +17,15 @@
17
17
  from __future__ import annotations
18
18
 
19
19
  import os
20
+ import shutil
20
21
  from contextlib import nullcontext
21
22
  from pathlib import Path
22
23
  from urllib.parse import urlparse
23
24
 
24
25
  import structlog
25
26
  from git import Repo
26
- from git.exc import BadName, GitCommandError, NoSuchPathError
27
+ from git.exc import BadName, GitCommandError, InvalidGitRepositoryError, NoSuchPathError
28
+ from tenacity import retry, retry_if_exception_type, stop_after_attempt
27
29
 
28
30
  from airflow.dag_processing.bundles.base import BaseDagBundle
29
31
  from airflow.exceptions import AirflowException
@@ -91,11 +93,21 @@ class GitDagBundle(BaseDagBundle):
91
93
  with self.lock():
92
94
  cm = self.hook.configure_hook_env() if self.hook else nullcontext()
93
95
  with cm:
94
- self._clone_bare_repo_if_required()
96
+ try:
97
+ self._clone_bare_repo_if_required()
98
+ except GitCommandError as e:
99
+ raise RuntimeError("Error cloning repository") from e
100
+ except InvalidGitRepositoryError as e:
101
+ raise RuntimeError(f"Invalid git repository at {self.bare_repo_path}") from e
95
102
  self._ensure_version_in_bare_repo()
96
103
  self.bare_repo.close()
97
104
 
98
- self._clone_repo_if_required()
105
+ try:
106
+ self._clone_repo_if_required()
107
+ except GitCommandError as e:
108
+ raise RuntimeError("Error cloning repository") from e
109
+ except InvalidGitRepositoryError as e:
110
+ raise RuntimeError(f"Invalid git repository at {self.repo_path}") from e
99
111
  self.repo.git.checkout(self.tracking_ref)
100
112
  self._log.debug("bundle initialize", version=self.version)
101
113
  if self.version:
@@ -113,36 +125,65 @@ class GitDagBundle(BaseDagBundle):
113
125
  self._initialize()
114
126
  super().initialize()
115
127
 
128
+ @retry(
129
+ retry=retry_if_exception_type((InvalidGitRepositoryError, GitCommandError)),
130
+ stop=stop_after_attempt(2),
131
+ reraise=True,
132
+ )
116
133
  def _clone_repo_if_required(self) -> None:
117
- if not os.path.exists(self.repo_path):
118
- self._log.info("Cloning repository", repo_path=self.repo_path, bare_repo_path=self.bare_repo_path)
119
- try:
134
+ try:
135
+ if not os.path.exists(self.repo_path):
136
+ self._log.info(
137
+ "Cloning repository", repo_path=self.repo_path, bare_repo_path=self.bare_repo_path
138
+ )
120
139
  Repo.clone_from(
121
140
  url=self.bare_repo_path,
122
141
  to_path=self.repo_path,
123
142
  )
124
- except NoSuchPathError as e:
125
- # Protection should the bare repo be removed manually
126
- raise AirflowException("Repository path: %s not found", self.bare_repo_path) from e
127
- else:
128
- self._log.debug("repo exists", repo_path=self.repo_path)
129
- self.repo = Repo(self.repo_path)
130
-
143
+ else:
144
+ self._log.debug("repo exists", repo_path=self.repo_path)
145
+ self.repo = Repo(self.repo_path)
146
+ except NoSuchPathError as e:
147
+ # Protection should the bare repo be removed manually
148
+ raise AirflowException("Repository path: %s not found", self.bare_repo_path) from e
149
+ except (InvalidGitRepositoryError, GitCommandError) as e:
150
+ self._log.warning(
151
+ "Repository clone/open failed, cleaning up and retrying",
152
+ repo_path=self.repo_path,
153
+ exc=e,
154
+ )
155
+ if os.path.exists(self.repo_path):
156
+ shutil.rmtree(self.repo_path)
157
+ raise
158
+
159
+ @retry(
160
+ retry=retry_if_exception_type((InvalidGitRepositoryError, GitCommandError)),
161
+ stop=stop_after_attempt(2),
162
+ reraise=True,
163
+ )
131
164
  def _clone_bare_repo_if_required(self) -> None:
132
165
  if not self.repo_url:
133
166
  raise AirflowException(f"Connection {self.git_conn_id} doesn't have a host url")
134
- if not os.path.exists(self.bare_repo_path):
135
- self._log.info("Cloning bare repository", bare_repo_path=self.bare_repo_path)
136
- try:
167
+
168
+ try:
169
+ if not os.path.exists(self.bare_repo_path):
170
+ self._log.info("Cloning bare repository", bare_repo_path=self.bare_repo_path)
137
171
  Repo.clone_from(
138
172
  url=self.repo_url,
139
173
  to_path=self.bare_repo_path,
140
174
  bare=True,
141
175
  env=self.hook.env if self.hook else None,
142
176
  )
143
- except GitCommandError as e:
144
- raise AirflowException("Error cloning repository") from e
145
- self.bare_repo = Repo(self.bare_repo_path)
177
+ self.bare_repo = Repo(self.bare_repo_path)
178
+ except (InvalidGitRepositoryError, GitCommandError) as e:
179
+ self._log.warning(
180
+ "Bare repository clone/open failed, cleaning up and retrying",
181
+ bare_repo_path=self.bare_repo_path,
182
+ exc=e,
183
+ )
184
+ if os.path.exists(self.bare_repo_path):
185
+ shutil.rmtree(self.bare_repo_path)
186
+ raise
146
187
 
147
188
  def _ensure_version_in_bare_repo(self) -> None:
148
189
  if not self.version:
@@ -25,7 +25,7 @@ import tempfile
25
25
  from typing import Any
26
26
 
27
27
  from airflow.exceptions import AirflowException
28
- from airflow.providers.git.version_compat import BaseHook
28
+ from airflow.providers.common.compat.sdk import BaseHook
29
29
 
30
30
  log = logging.getLogger(__name__)
31
31
 
@@ -35,9 +35,4 @@ def get_base_airflow_version_tuple() -> tuple[int, int, int]:
35
35
  AIRFLOW_V_3_0_PLUS = get_base_airflow_version_tuple() >= (3, 0, 0)
36
36
  AIRFLOW_V_3_1_PLUS: bool = get_base_airflow_version_tuple() >= (3, 1, 0)
37
37
 
38
- if AIRFLOW_V_3_1_PLUS:
39
- from airflow.sdk import BaseHook
40
- else:
41
- from airflow.hooks.base import BaseHook # type: ignore[attr-defined,no-redef]
42
-
43
- __all__ = ["AIRFLOW_V_3_0_PLUS", "AIRFLOW_V_3_1_PLUS", "BaseHook"]
38
+ __all__ = ["AIRFLOW_V_3_0_PLUS", "AIRFLOW_V_3_1_PLUS"]
@@ -26,7 +26,7 @@ from unittest.mock import patch
26
26
 
27
27
  import pytest
28
28
  from git import Repo
29
- from git.exc import GitCommandError, NoSuchPathError
29
+ from git.exc import GitCommandError, InvalidGitRepositoryError, NoSuchPathError
30
30
 
31
31
  from airflow.dag_processing.bundles.base import get_bundle_storage_root_path
32
32
  from airflow.exceptions import AirflowException
@@ -653,7 +653,7 @@ class TestGitDagBundle:
653
653
  mock_clone.side_effect = GitCommandError("clone", "Simulated error")
654
654
  bundle = GitDagBundle(name="test", git_conn_id=CONN_HTTPS, tracking_ref="main")
655
655
  with pytest.raises(
656
- AirflowException,
656
+ RuntimeError,
657
657
  match=re.escape("Error cloning repository"),
658
658
  ):
659
659
  bundle.initialize()
@@ -745,3 +745,65 @@ class TestGitDagBundle:
745
745
  bundle._clone_bare_repo_if_required()
746
746
  _, kwargs = mock_gitRepo.clone_from.call_args
747
747
  assert kwargs["env"] == EXPECTED_ENV
748
+
749
+ @mock.patch("airflow.providers.git.bundles.git.GitHook")
750
+ @mock.patch("airflow.providers.git.bundles.git.shutil.rmtree")
751
+ @mock.patch("airflow.providers.git.bundles.git.os.path.exists")
752
+ def test_clone_bare_repo_invalid_repository_error_retry(self, mock_exists, mock_rmtree, mock_githook):
753
+ """Test that InvalidGitRepositoryError triggers cleanup and retry."""
754
+ mock_githook.return_value.repo_url = "git@github.com:apache/airflow.git"
755
+ mock_githook.return_value.env = {}
756
+
757
+ # Set up exists to return True for the bare repo path (simulating corrupted repo exists)
758
+ mock_exists.return_value = True
759
+
760
+ with mock.patch("airflow.providers.git.bundles.git.Repo") as mock_repo_class:
761
+ # First call to Repo() raises InvalidGitRepositoryError, second call succeeds
762
+ mock_repo_class.side_effect = [
763
+ InvalidGitRepositoryError("Invalid git repository"),
764
+ mock.MagicMock(), # Second attempt succeeds
765
+ ]
766
+
767
+ # Mock successful clone_from for the retry attempt
768
+ mock_repo_class.clone_from = mock.MagicMock()
769
+
770
+ bundle = GitDagBundle(name="test", git_conn_id=CONN_HTTPS, tracking_ref="main")
771
+
772
+ # This should not raise an exception due to retry logic
773
+ bundle._clone_bare_repo_if_required()
774
+
775
+ # Verify cleanup was called
776
+ mock_rmtree.assert_called_once_with(bundle.bare_repo_path)
777
+
778
+ # Verify Repo was called twice (failed attempt + retry)
779
+ assert mock_repo_class.call_count == 2
780
+
781
+ @mock.patch("airflow.providers.git.bundles.git.GitHook")
782
+ @mock.patch("airflow.providers.git.bundles.git.shutil.rmtree")
783
+ @mock.patch("airflow.providers.git.bundles.git.os.path.exists")
784
+ def test_clone_bare_repo_invalid_repository_error_retry_fails(
785
+ self, mock_exists, mock_rmtree, mock_githook
786
+ ):
787
+ """Test that InvalidGitRepositoryError after retry is re-raised (wrapped in AirflowException by caller)."""
788
+ mock_githook.return_value.repo_url = "git@github.com:apache/airflow.git"
789
+ mock_githook.return_value.env = {}
790
+
791
+ # Set up exists to return True for the bare repo path
792
+ mock_exists.return_value = True
793
+
794
+ with mock.patch("airflow.providers.git.bundles.git.Repo") as mock_repo_class:
795
+ # Both calls to Repo() raise InvalidGitRepositoryError
796
+ mock_repo_class.side_effect = InvalidGitRepositoryError("Invalid git repository")
797
+
798
+ bundle = GitDagBundle(name="test", git_conn_id=CONN_HTTPS, tracking_ref="main")
799
+
800
+ # The raw exception is raised by the method itself, but wrapped by _initialize
801
+ with pytest.raises(InvalidGitRepositoryError, match="Invalid git repository"):
802
+ bundle._clone_bare_repo_if_required()
803
+
804
+ # Verify cleanup was called twice (once for each failed attempt)
805
+ assert mock_rmtree.call_count == 2
806
+ mock_rmtree.assert_called_with(bundle.bare_repo_path)
807
+
808
+ # Verify Repo was called twice (failed attempt + failed retry)
809
+ assert mock_repo_class.call_count == 2
@@ -1,100 +0,0 @@
1
-
2
- .. Licensed to the Apache Software Foundation (ASF) under one
3
- or more contributor license agreements. See the NOTICE file
4
- distributed with this work for additional information
5
- regarding copyright ownership. The ASF licenses this file
6
- to you under the Apache License, Version 2.0 (the
7
- "License"); you may not use this file except in compliance
8
- with the License. You may obtain a copy of the License at
9
-
10
- .. http://www.apache.org/licenses/LICENSE-2.0
11
-
12
- .. Unless required by applicable law or agreed to in writing,
13
- software distributed under the License is distributed on an
14
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15
- KIND, either express or implied. See the License for the
16
- specific language governing permissions and limitations
17
- under the License.
18
-
19
- ``apache-airflow-providers-git``
20
- ================================
21
-
22
-
23
- .. toctree::
24
- :hidden:
25
- :maxdepth: 1
26
- :caption: Basics
27
-
28
- Home <self>
29
- Changelog <changelog>
30
- Security <security>
31
-
32
- .. toctree::
33
- :hidden:
34
- :maxdepth: 1
35
- :caption: References
36
-
37
- Connection Types <connections/git>
38
- Python API <_api/airflow/providers/git/index>
39
-
40
- .. toctree::
41
- :hidden:
42
- :maxdepth: 1
43
- :caption: Bundles
44
-
45
- Bundles <bundles/index>
46
-
47
- .. toctree::
48
- :hidden:
49
- :maxdepth: 1
50
- :caption: Resources
51
-
52
- PyPI Repository <https://pypi.org/project/apache-airflow-providers-git/>
53
- Installing from sources <installing-providers-from-sources>
54
- Python API <_api/airflow/providers/git/index>
55
-
56
-
57
-
58
- .. THE REMAINDER OF THE FILE IS AUTOMATICALLY GENERATED. IT WILL BE OVERWRITTEN AT RELEASE TIME!
59
-
60
-
61
- .. toctree::
62
- :hidden:
63
- :maxdepth: 1
64
- :caption: Commits
65
-
66
- Detailed list of commits <commits>
67
-
68
-
69
- apache-airflow-providers-git package
70
- ------------------------------------------------------
71
-
72
- `Distributed version control system (GIT) <https://git-scm.com/>`__
73
-
74
-
75
- Release: 0.0.8
76
-
77
- Provider package
78
- ----------------
79
-
80
- This package is for the ``git`` provider.
81
- All classes for this package are included in the ``airflow.providers.git`` python package.
82
-
83
- Installation
84
- ------------
85
-
86
- You can install this package on top of an existing Airflow installation via
87
- ``pip install apache-airflow-providers-git``.
88
- For the minimum Airflow version supported, see ``Requirements`` below.
89
-
90
- Requirements
91
- ------------
92
-
93
- The minimum Apache Airflow version supported by this provider distribution is ``3.0.0``.
94
-
95
- ================== ==================
96
- PIP package Version required
97
- ================== ==================
98
- ``apache-airflow`` ``>=3.0.0``
99
- ``GitPython`` ``>=3.1.44``
100
- ================== ==================