airbyte-source-gitlab 2.1.1__tar.gz → 3.0.0__tar.gz

Sign up to get free protection for your applications and to get access to all the features.
Files changed (52) hide show
  1. airbyte_source_gitlab-3.0.0/PKG-INFO +111 -0
  2. airbyte_source_gitlab-3.0.0/README.md +91 -0
  3. airbyte_source_gitlab-3.0.0/pyproject.toml +34 -0
  4. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/config_migrations.py +0 -2
  5. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/epic_issues.json +6 -0
  6. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/groups.json +20 -0
  7. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/issues.json +30 -0
  8. airbyte_source_gitlab-3.0.0/source_gitlab/schemas/merge_request_commits.json +85 -0
  9. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/merge_requests.json +3 -0
  10. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/pipelines_extended.json +3 -0
  11. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/projects.json +18 -0
  12. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/streams.py +3 -1
  13. airbyte-source-gitlab-2.1.1/PKG-INFO +0 -141
  14. airbyte-source-gitlab-2.1.1/README.md +0 -147
  15. airbyte-source-gitlab-2.1.1/airbyte_source_gitlab.egg-info/PKG-INFO +0 -141
  16. airbyte-source-gitlab-2.1.1/airbyte_source_gitlab.egg-info/SOURCES.txt +0 -47
  17. airbyte-source-gitlab-2.1.1/airbyte_source_gitlab.egg-info/dependency_links.txt +0 -1
  18. airbyte-source-gitlab-2.1.1/airbyte_source_gitlab.egg-info/entry_points.txt +0 -2
  19. airbyte-source-gitlab-2.1.1/airbyte_source_gitlab.egg-info/requires.txt +0 -8
  20. airbyte-source-gitlab-2.1.1/airbyte_source_gitlab.egg-info/top_level.txt +0 -2
  21. airbyte-source-gitlab-2.1.1/integration_tests/__init__.py +0 -0
  22. airbyte-source-gitlab-2.1.1/integration_tests/abnormal_state.json +0 -54
  23. airbyte-source-gitlab-2.1.1/integration_tests/acceptance.py +0 -16
  24. airbyte-source-gitlab-2.1.1/integration_tests/configured_catalog.json +0 -271
  25. airbyte-source-gitlab-2.1.1/integration_tests/incremental_catalog.json +0 -60
  26. airbyte-source-gitlab-2.1.1/integration_tests/invalid_config.json +0 -11
  27. airbyte-source-gitlab-2.1.1/integration_tests/sample_config.json +0 -7
  28. airbyte-source-gitlab-2.1.1/integration_tests/sample_state.json +0 -22
  29. airbyte-source-gitlab-2.1.1/setup.cfg +0 -137
  30. airbyte-source-gitlab-2.1.1/setup.py +0 -37
  31. airbyte-source-gitlab-2.1.1/source_gitlab/schemas/merge_request_commits.json +0 -234
  32. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/__init__.py +0 -0
  33. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/run.py +0 -0
  34. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/branches.json +0 -0
  35. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/commits.json +0 -0
  36. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/deployments.json +0 -0
  37. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/epics.json +0 -0
  38. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/group_issue_boards.json +0 -0
  39. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/group_labels.json +0 -0
  40. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/group_members.json +0 -0
  41. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/group_milestones.json +0 -0
  42. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/jobs.json +0 -0
  43. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/pipelines.json +0 -0
  44. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/project_labels.json +0 -0
  45. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/project_members.json +0 -0
  46. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/project_milestones.json +0 -0
  47. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/releases.json +0 -0
  48. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/tags.json +0 -0
  49. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/schemas/users.json +0 -0
  50. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/source.py +0 -0
  51. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/spec.json +0 -0
  52. {airbyte-source-gitlab-2.1.1 → airbyte_source_gitlab-3.0.0}/source_gitlab/utils.py +0 -0
@@ -0,0 +1,111 @@
1
+ Metadata-Version: 2.1
2
+ Name: airbyte-source-gitlab
3
+ Version: 3.0.0
4
+ Summary: Source implementation for Gitlab.
5
+ Home-page: https://airbyte.com
6
+ License: MIT
7
+ Author: Airbyte
8
+ Author-email: contact@airbyte.io
9
+ Requires-Python: >=3.9,<3.12
10
+ Classifier: License :: OSI Approved :: MIT License
11
+ Classifier: Programming Language :: Python :: 3
12
+ Classifier: Programming Language :: Python :: 3.9
13
+ Classifier: Programming Language :: Python :: 3.10
14
+ Classifier: Programming Language :: Python :: 3.11
15
+ Requires-Dist: airbyte-cdk (==0.58.8)
16
+ Requires-Dist: vcrpy (==4.1.1)
17
+ Project-URL: Documentation, https://docs.airbyte.com/integrations/sources/gitlab
18
+ Project-URL: Repository, https://github.com/airbytehq/airbyte
19
+ Description-Content-Type: text/markdown
20
+
21
+ # Gitlab source connector
22
+
23
+
24
+ This is the repository for the Gitlab source connector, written in Python.
25
+ For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.com/integrations/sources/gitlab).
26
+
27
+ ## Local development
28
+
29
+ ### Prerequisites
30
+ * Python (~=3.9)
31
+ * Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
32
+
33
+
34
+ ### Installing the connector
35
+ From this connector directory, run:
36
+ ```bash
37
+ poetry install --with dev
38
+ ```
39
+
40
+
41
+ ### Create credentials
42
+ **If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/gitlab)
43
+ to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_gitlab/spec.yaml` file.
44
+ Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
45
+ See `sample_files/sample_config.json` for a sample config file.
46
+
47
+
48
+ ### Locally running the connector
49
+ ```
50
+ poetry run source-gitlab spec
51
+ poetry run source-gitlab check --config secrets/config.json
52
+ poetry run source-gitlab discover --config secrets/config.json
53
+ poetry run source-gitlab read --config secrets/config.json --catalog sample_files/configured_catalog.json
54
+ ```
55
+
56
+ ### Running unit tests
57
+ To run unit tests locally, from the connector directory run:
58
+ ```
59
+ poetry run pytest unit_tests
60
+ ```
61
+
62
+ ### Building the docker image
63
+ 1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
64
+ 2. Run the following command to build the docker image:
65
+ ```bash
66
+ airbyte-ci connectors --name=source-gitlab build
67
+ ```
68
+
69
+ An image will be available on your host with the tag `airbyte/source-gitlab:dev`.
70
+
71
+
72
+ ### Running as a docker container
73
+ Then run any of the connector commands as follows:
74
+ ```
75
+ docker run --rm airbyte/source-gitlab:dev spec
76
+ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-gitlab:dev check --config /secrets/config.json
77
+ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-gitlab:dev discover --config /secrets/config.json
78
+ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-gitlab:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
79
+ ```
80
+
81
+ ### Running our CI test suite
82
+ You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
83
+ ```bash
84
+ airbyte-ci connectors --name=source-gitlab test
85
+ ```
86
+
87
+ ### Customizing acceptance Tests
88
+ Customize `acceptance-test-config.yml` file to configure acceptance tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
89
+ If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
90
+
91
+ ### Dependency Management
92
+ All of your dependencies should be managed via Poetry.
93
+ To add a new dependency, run:
94
+ ```bash
95
+ poetry add <package-name>
96
+ ```
97
+
98
+ Please commit the changes to `pyproject.toml` and `poetry.lock` files.
99
+
100
+ ## Publishing a new version of the connector
101
+ You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
102
+ 1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-gitlab test`
103
+ 2. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)):
104
+ - bump the `dockerImageTag` value in in `metadata.yaml`
105
+ - bump the `version` value in `pyproject.toml`
106
+ 3. Make sure the `metadata.yaml` content is up to date.
107
+ 4. Make sure the connector documentation and its changelog is up to date (`docs/integrations/sources/gitlab.md`).
108
+ 5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
109
+ 6. Pat yourself on the back for being an awesome contributor.
110
+ 7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
111
+ 8. Once your PR is merged, the new version of the connector will be automatically published to Docker Hub and our connector registry.
@@ -0,0 +1,91 @@
1
+ # Gitlab source connector
2
+
3
+
4
+ This is the repository for the Gitlab source connector, written in Python.
5
+ For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.com/integrations/sources/gitlab).
6
+
7
+ ## Local development
8
+
9
+ ### Prerequisites
10
+ * Python (~=3.9)
11
+ * Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
12
+
13
+
14
+ ### Installing the connector
15
+ From this connector directory, run:
16
+ ```bash
17
+ poetry install --with dev
18
+ ```
19
+
20
+
21
+ ### Create credentials
22
+ **If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/gitlab)
23
+ to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_gitlab/spec.yaml` file.
24
+ Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
25
+ See `sample_files/sample_config.json` for a sample config file.
26
+
27
+
28
+ ### Locally running the connector
29
+ ```
30
+ poetry run source-gitlab spec
31
+ poetry run source-gitlab check --config secrets/config.json
32
+ poetry run source-gitlab discover --config secrets/config.json
33
+ poetry run source-gitlab read --config secrets/config.json --catalog sample_files/configured_catalog.json
34
+ ```
35
+
36
+ ### Running unit tests
37
+ To run unit tests locally, from the connector directory run:
38
+ ```
39
+ poetry run pytest unit_tests
40
+ ```
41
+
42
+ ### Building the docker image
43
+ 1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
44
+ 2. Run the following command to build the docker image:
45
+ ```bash
46
+ airbyte-ci connectors --name=source-gitlab build
47
+ ```
48
+
49
+ An image will be available on your host with the tag `airbyte/source-gitlab:dev`.
50
+
51
+
52
+ ### Running as a docker container
53
+ Then run any of the connector commands as follows:
54
+ ```
55
+ docker run --rm airbyte/source-gitlab:dev spec
56
+ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-gitlab:dev check --config /secrets/config.json
57
+ docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-gitlab:dev discover --config /secrets/config.json
58
+ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-gitlab:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
59
+ ```
60
+
61
+ ### Running our CI test suite
62
+ You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
63
+ ```bash
64
+ airbyte-ci connectors --name=source-gitlab test
65
+ ```
66
+
67
+ ### Customizing acceptance Tests
68
+ Customize `acceptance-test-config.yml` file to configure acceptance tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
69
+ If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
70
+
71
+ ### Dependency Management
72
+ All of your dependencies should be managed via Poetry.
73
+ To add a new dependency, run:
74
+ ```bash
75
+ poetry add <package-name>
76
+ ```
77
+
78
+ Please commit the changes to `pyproject.toml` and `poetry.lock` files.
79
+
80
+ ## Publishing a new version of the connector
81
+ You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
82
+ 1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-gitlab test`
83
+ 2. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)):
84
+ - bump the `dockerImageTag` value in in `metadata.yaml`
85
+ - bump the `version` value in `pyproject.toml`
86
+ 3. Make sure the `metadata.yaml` content is up to date.
87
+ 4. Make sure the connector documentation and its changelog is up to date (`docs/integrations/sources/gitlab.md`).
88
+ 5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
89
+ 6. Pat yourself on the back for being an awesome contributor.
90
+ 7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
91
+ 8. Once your PR is merged, the new version of the connector will be automatically published to Docker Hub and our connector registry.
@@ -0,0 +1,34 @@
1
+ [build-system]
2
+ requires = [
3
+ "poetry-core>=1.0.0",
4
+ ]
5
+ build-backend = "poetry.core.masonry.api"
6
+
7
+ [tool.poetry]
8
+ version = "3.0.0"
9
+ name = "airbyte-source-gitlab"
10
+ description = "Source implementation for Gitlab."
11
+ authors = [
12
+ "Airbyte <contact@airbyte.io>",
13
+ ]
14
+ license = "MIT"
15
+ readme = "README.md"
16
+ documentation = "https://docs.airbyte.com/integrations/sources/gitlab"
17
+ homepage = "https://airbyte.com"
18
+ repository = "https://github.com/airbytehq/airbyte"
19
+ packages = [
20
+ { include = "source_gitlab" },
21
+ ]
22
+
23
+ [tool.poetry.dependencies]
24
+ python = "^3.9,<3.12"
25
+ airbyte-cdk = "==0.58.8"
26
+ vcrpy = "==4.1.1"
27
+
28
+ [tool.poetry.scripts]
29
+ source-gitlab = "source_gitlab.run:run"
30
+
31
+ [tool.poetry.group.dev.dependencies]
32
+ pytest-mock = "^3.12.0"
33
+ requests-mock = "^1.9.3"
34
+ pytest = "^6.1"
@@ -95,12 +95,10 @@ class MigrateStringToArray(ABC):
95
95
 
96
96
 
97
97
  class MigrateGroups(MigrateStringToArray):
98
-
99
98
  migrate_from_key: str = "groups"
100
99
  migrate_to_key: str = "groups_list"
101
100
 
102
101
 
103
102
  class MigrateProjects(MigrateStringToArray):
104
-
105
103
  migrate_from_key: str = "projects"
106
104
  migrate_to_key: str = "projects_list"
@@ -216,6 +216,12 @@
216
216
  },
217
217
  "group_id": {
218
218
  "type": ["null", "integer"]
219
+ },
220
+ "human_readable_end_date": {
221
+ "type": ["null", "string"]
222
+ },
223
+ "human_readable_timestamp": {
224
+ "type": ["null", "string"]
219
225
  }
220
226
  }
221
227
  },
@@ -38,6 +38,17 @@
38
38
  }
39
39
  }
40
40
  }
41
+ },
42
+ "allowed_to_push": {
43
+ "type": ["null", "array"],
44
+ "items": {
45
+ "type": ["null", "object"],
46
+ "properties": {
47
+ "access_level": {
48
+ "type": ["null", "integer"]
49
+ }
50
+ }
51
+ }
41
52
  }
42
53
  }
43
54
  },
@@ -74,6 +85,9 @@
74
85
  "subgroup_creation_level": {
75
86
  "type": ["null", "string"]
76
87
  },
88
+ "enabled_git_access_protocol": {
89
+ "type": ["null", "string"]
90
+ },
77
91
  "emails_disabled": {
78
92
  "type": ["null", "boolean"]
79
93
  },
@@ -153,6 +167,12 @@
153
167
  },
154
168
  "service_access_tokens_expiration_enforced": {
155
169
  "type": ["null", "boolean"]
170
+ },
171
+ "lock_math_rendering_limits_enabled": {
172
+ "type": ["null", "boolean"]
173
+ },
174
+ "math_rendering_limits_enabled": {
175
+ "type": ["null", "boolean"]
156
176
  }
157
177
  }
158
178
  }
@@ -118,6 +118,9 @@
118
118
  },
119
119
  "id": {
120
120
  "type": ["null", "integer"]
121
+ },
122
+ "locked": {
123
+ "type": ["null", "boolean"]
121
124
  }
122
125
  }
123
126
  },
@@ -141,6 +144,12 @@
141
144
  },
142
145
  "username": {
143
146
  "type": ["null", "string"]
147
+ },
148
+ "id": {
149
+ "type": ["null", "integer"]
150
+ },
151
+ "locked": {
152
+ "type": ["null", "boolean"]
144
153
  }
145
154
  }
146
155
  },
@@ -164,6 +173,18 @@
164
173
  },
165
174
  "username": {
166
175
  "type": ["null", "string"]
176
+ },
177
+ "human_readable_end_date": {
178
+ "type": ["null", "string"]
179
+ },
180
+ "human_readable_timestamp": {
181
+ "type": ["null", "string"]
182
+ },
183
+ "id": {
184
+ "type": ["null", "integer"]
185
+ },
186
+ "locked": {
187
+ "type": ["null", "boolean"]
167
188
  }
168
189
  }
169
190
  },
@@ -251,6 +272,15 @@
251
272
  },
252
273
  "group_id": {
253
274
  "type": ["null", "integer"]
275
+ },
276
+ "locked": {
277
+ "type": ["null", "boolean"]
278
+ },
279
+ "human_readable_end_date": {
280
+ "type": ["null", "string"]
281
+ },
282
+ "human_readable_timestamp": {
283
+ "type": ["null", "string"]
254
284
  }
255
285
  }
256
286
  },
@@ -0,0 +1,85 @@
1
+ {
2
+ "$schema": "https://json-schema.org/draft-07/schema#",
3
+ "type": "object",
4
+ "properties": {
5
+ "project_id": {
6
+ "type": ["null", "integer"]
7
+ },
8
+ "id": {
9
+ "type": ["null", "string"]
10
+ },
11
+ "short_id": {
12
+ "type": ["null", "string"]
13
+ },
14
+ "created_at": {
15
+ "type": ["null", "string"],
16
+ "format": "date-time"
17
+ },
18
+ "parent_ids": {
19
+ "type": ["null", "array"],
20
+ "items": {
21
+ "type": ["null", "string"]
22
+ }
23
+ },
24
+ "title": {
25
+ "type": ["null", "string"]
26
+ },
27
+ "message": {
28
+ "type": ["null", "string"]
29
+ },
30
+ "author_name": {
31
+ "type": ["null", "string"]
32
+ },
33
+ "author_email": {
34
+ "type": ["null", "string"]
35
+ },
36
+ "authored_date": {
37
+ "type": ["null", "string"],
38
+ "format": "date-time"
39
+ },
40
+ "extended_trailers": {
41
+ "type": ["null", "object"],
42
+ "properties": {
43
+ "Cc": {
44
+ "type": ["null", "array"],
45
+ "items": {
46
+ "type": ["null", "string"]
47
+ }
48
+ }
49
+ }
50
+ },
51
+ "committer_name": {
52
+ "type": ["null", "string"]
53
+ },
54
+ "committer_email": {
55
+ "type": ["null", "string"]
56
+ },
57
+ "committed_date": {
58
+ "type": ["null", "string"],
59
+ "format": "date-time"
60
+ },
61
+ "trailers": {
62
+ "type": ["null", "object"]
63
+ },
64
+ "web_url": {
65
+ "type": ["null", "string"]
66
+ },
67
+ "stats": {
68
+ "type": ["null", "object"],
69
+ "properties": {
70
+ "additions": {
71
+ "type": ["null", "integer"]
72
+ },
73
+ "deletions": {
74
+ "type": ["null", "integer"]
75
+ },
76
+ "total": {
77
+ "type": ["null", "integer"]
78
+ }
79
+ }
80
+ },
81
+ "merge_request_iid": {
82
+ "type": ["null", "integer"]
83
+ }
84
+ }
85
+ }
@@ -194,6 +194,9 @@
194
194
  },
195
195
  "web_url": {
196
196
  "type": ["null", "string"]
197
+ },
198
+ "locked": {
199
+ "type": ["null", "boolean"]
197
200
  }
198
201
  }
199
202
  }
@@ -63,6 +63,9 @@
63
63
  },
64
64
  "web_url": {
65
65
  "type": ["null", "string"]
66
+ },
67
+ "locked": {
68
+ "type": ["null", "boolean"]
66
69
  }
67
70
  }
68
71
  },
@@ -123,6 +123,9 @@
123
123
  },
124
124
  "members": {
125
125
  "type": ["null", "string"]
126
+ },
127
+ "cluster_agents": {
128
+ "type": ["null", "string"]
126
129
  }
127
130
  }
128
131
  },
@@ -329,6 +332,15 @@
329
332
  },
330
333
  "packages_size": {
331
334
  "type": ["null", "integer"]
335
+ },
336
+ "container_registry_size": {
337
+ "type": ["null", "integer"]
338
+ },
339
+ "pipeline_artifacts_size": {
340
+ "type": ["null", "integer"]
341
+ },
342
+ "uploads_size": {
343
+ "type": ["null", "integer"]
332
344
  }
333
345
  }
334
346
  },
@@ -492,6 +504,12 @@
492
504
  },
493
505
  "ci_restrict_pipeline_cancellation_role": {
494
506
  "type": ["null", "string"]
507
+ },
508
+ "repository_object_format": {
509
+ "type": ["null", "string"]
510
+ },
511
+ "warn_about_potentially_unwanted_characters": {
512
+ "type": ["null", "boolean"]
495
513
  }
496
514
  }
497
515
  }
@@ -339,8 +339,10 @@ class MergeRequests(IncrementalGitlabChildStream):
339
339
 
340
340
 
341
341
  class MergeRequestCommits(GitlabChildStream):
342
+ """Docs: https://docs.gitlab.com/ee/api/merge_requests.html#get-single-merge-request-commits"""
343
+
342
344
  path_list = ["project_id", "iid"]
343
- path_template = "projects/{project_id}/merge_requests/{iid}"
345
+ path_template = "projects/{project_id}/merge_requests/{iid}/commits"
344
346
 
345
347
  def transform(self, record, stream_slice: Mapping[str, Any] = None, **kwargs):
346
348
  super().transform(record, stream_slice, **kwargs)
@@ -1,141 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: airbyte-source-gitlab
3
- Version: 2.1.1
4
- Summary: Source implementation for Gitlab.
5
- Author: Airbyte
6
- Author-email: contact@airbyte.io
7
- Description-Content-Type: text/markdown
8
- Requires-Dist: airbyte-cdk
9
- Requires-Dist: vcrpy==4.1.1
10
- Provides-Extra: tests
11
- Requires-Dist: requests-mock~=1.9.3; extra == "tests"
12
- Requires-Dist: pytest~=6.1; extra == "tests"
13
- Requires-Dist: requests_mock; extra == "tests"
14
- Requires-Dist: pytest-mock; extra == "tests"
15
-
16
- # Gitlab Source
17
-
18
- This is the repository for the Gitlab source connector, written in Python.
19
- For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/gitlab).
20
-
21
-
22
- **To iterate on this connector, make sure to complete this prerequisites section.**
23
-
24
- From this connector directory, create a virtual environment:
25
- ```
26
- python -m venv .venv
27
- ```
28
-
29
- This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
30
- development environment of choice. To activate it from the terminal, run:
31
- ```
32
- source .venv/bin/activate
33
- pip install -r requirements.txt
34
- ```
35
- If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
36
-
37
- Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
38
- used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
39
- If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
40
- should work as you expect.
41
-
42
- **If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/gitlab)
43
- to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_gitlab/spec.json` file.
44
- Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
45
- See `integration_tests/sample_config.json` for a sample config file.
46
-
47
- **If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source gitlab test creds`
48
- and place them into `secrets/config.json`.
49
-
50
- ```
51
- python main.py spec
52
- python main.py check --config secrets/config.json
53
- python main.py discover --config secrets/config.json
54
- python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
55
- ```
56
-
57
-
58
-
59
-
60
- The Airbyte way of building this connector is to use our `airbyte-ci` tool.
61
- You can follow install instructions [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md#L1).
62
- Then running the following command will build your connector:
63
-
64
- ```bash
65
- airbyte-ci connectors --name=source-gitlab build
66
- ```
67
- Once the command is done, you will find your connector image in your local docker registry: `airbyte/source-gitlab:dev`.
68
-
69
- When contributing on our connector you might need to customize the build process to add a system dependency or set an env var.
70
- You can customize our build process by adding a `build_customization.py` module to your connector.
71
- This module should contain a `pre_connector_install` and `post_connector_install` async function that will mutate the base image and the connector container respectively.
72
- It will be imported at runtime by our build process and the functions will be called if they exist.
73
-
74
- Here is an example of a `build_customization.py` module:
75
- ```python
76
- from __future__ import annotations
77
-
78
- from typing import TYPE_CHECKING
79
-
80
- if TYPE_CHECKING:
81
- from dagger import Container
82
-
83
-
84
- async def pre_connector_install(base_image_container: Container) -> Container:
85
- return await base_image_container.with_env_variable("MY_PRE_BUILD_ENV_VAR", "my_pre_build_env_var_value")
86
-
87
- async def post_connector_install(connector_container: Container) -> Container:
88
- return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value")
89
- ```
90
-
91
- This connector is built using our dynamic built process in `airbyte-ci`.
92
- The base image used to build it is defined within the metadata.yaml file under the `connectorBuildOptions`.
93
- The build logic is defined using [Dagger](https://dagger.io/) [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/pipelines/builds/python_connectors.py).
94
- It does not rely on a Dockerfile.
95
-
96
- If you would like to patch our connector and build your own a simple approach would be to:
97
-
98
- 1. Create your own Dockerfile based on the latest version of the connector image.
99
- ```Dockerfile
100
- FROM airbyte/source-gitlab:latest
101
-
102
- COPY . ./airbyte/integration_code
103
- RUN pip install ./airbyte/integration_code
104
-
105
- ```
106
- Please use this as an example. This is not optimized.
107
-
108
- 2. Build your image:
109
- ```bash
110
- docker build -t airbyte/source-gitlab:dev .
111
- docker run airbyte/source-gitlab:dev spec
112
- ```
113
- Then run any of the connector commands as follows:
114
- ```
115
- docker run --rm airbyte/source-gitlab:dev spec
116
- docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-gitlab:dev check --config /secrets/config.json
117
- docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-gitlab:dev discover --config /secrets/config.json
118
- docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-gitlab:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
119
- ```
120
-
121
- You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
122
- ```bash
123
- airbyte-ci connectors --name=source-gitlab test
124
- ```
125
-
126
- Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
127
- If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
128
-
129
- All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
130
- We split dependencies between two groups, dependencies that are:
131
- * required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
132
- * required for the testing need to go to `TEST_REQUIREMENTS` list
133
-
134
- You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
135
- 1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-gitlab test`
136
- 2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
137
- 3. Make sure the `metadata.yaml` content is up to date.
138
- 4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/gitlab.md`).
139
- 5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
140
- 6. Pat yourself on the back for being an awesome contributor.
141
- 7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.