airbyte-source-github 2.1.24__tar.gz → 2.1.25__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/PKG-INFO +1 -1
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/pyproject.toml +1 -1
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/errors_handlers.py +15 -3
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/source.py +2 -2
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/streams.py +31 -13
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/README.md +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/__init__.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/backoff_strategies.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/config_migrations.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/constants.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/github_schema.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/graphql.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/run.py +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/assignees.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/branches.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/collaborators.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/comments.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/commit_comment_reactions.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/commit_comments.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/commits.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/contributor_activity.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/deployments.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/events.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issue_comment_reactions.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issue_events.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issue_labels.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issue_milestones.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issue_reactions.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issue_timeline_events.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issues.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/organizations.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/project_cards.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/project_columns.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/projects.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/projects_v2.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/pull_request_comment_reactions.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/pull_request_commits.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/pull_request_stats.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/pull_requests.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/releases.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/repositories.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/review_comments.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/reviews.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/events/comment.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/events/commented.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/events/committed.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/events/cross_referenced.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/events/reviewed.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/reaction.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/reactions.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/user.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/user_graphql.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/stargazers.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/tags.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/team_members.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/team_memberships.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/teams.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/users.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/workflow_jobs.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/workflow_runs.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/workflows.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/spec.json +0 -0
- {airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/utils.py +0 -0
|
@@ -3,7 +3,7 @@ requires = [ "poetry-core>=1.0.0",]
|
|
|
3
3
|
build-backend = "poetry.core.masonry.api"
|
|
4
4
|
|
|
5
5
|
[tool.poetry]
|
|
6
|
-
version = "2.1.
|
|
6
|
+
version = "2.1.25"
|
|
7
7
|
name = "airbyte-source-github"
|
|
8
8
|
description = "Source implementation for GitHub."
|
|
9
9
|
authors = [ "Airbyte <contact@airbyte.io>",]
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/errors_handlers.py
RENAMED
|
@@ -23,7 +23,12 @@ GITHUB_DEFAULT_ERROR_MAPPING = DEFAULT_ERROR_MAPPING | {
|
|
|
23
23
|
403: ErrorResolution(
|
|
24
24
|
response_action=ResponseAction.FAIL,
|
|
25
25
|
failure_type=FailureType.config_error,
|
|
26
|
-
error_message=
|
|
26
|
+
error_message=(
|
|
27
|
+
"GitHub denied access (HTTP 403). Your token may be missing required scopes "
|
|
28
|
+
"(this connector typically needs: repo, read:org, read:user, read:project, workflow), "
|
|
29
|
+
"or this organization may require SAML SSO authorization. "
|
|
30
|
+
"See https://docs.github.com/en/rest/using-the-rest-api/troubleshooting-the-rest-api"
|
|
31
|
+
),
|
|
27
32
|
),
|
|
28
33
|
404: ErrorResolution(
|
|
29
34
|
response_action=ResponseAction.RETRY,
|
|
@@ -92,11 +97,18 @@ class GithubStreamABCErrorHandler(HttpStatusErrorHandler):
|
|
|
92
97
|
return ErrorResolution(
|
|
93
98
|
response_action=ResponseAction.RATE_LIMITED,
|
|
94
99
|
failure_type=FailureType.transient_error,
|
|
95
|
-
error_message=
|
|
100
|
+
error_message=(
|
|
101
|
+
f"GitHub rate limit hit for stream `{self.stream.name}` "
|
|
102
|
+
f"(HTTP {response_or_exception.status_code}). "
|
|
103
|
+
f"Waiting for the rate limit window to reset before retrying."
|
|
104
|
+
),
|
|
96
105
|
)
|
|
97
106
|
|
|
98
107
|
if is_conflict_with_empty_repository(response_or_exception=response_or_exception):
|
|
99
|
-
log_message =
|
|
108
|
+
log_message = (
|
|
109
|
+
f"Skipping `{self.stream.name}` for this repository: GitHub returned 409 Conflict "
|
|
110
|
+
f"with message 'Git Repository is empty.' This means the repository has no commits."
|
|
111
|
+
)
|
|
100
112
|
return ErrorResolution(
|
|
101
113
|
response_action=ResponseAction.IGNORE,
|
|
102
114
|
failure_type=FailureType.config_error,
|
|
@@ -185,9 +185,9 @@ class SourceGithub(AbstractSource):
|
|
|
185
185
|
org_name = message.split("https://api.github.com/orgs/")[1].split("/")[0]
|
|
186
186
|
user_message = f'Organization name: "{org_name}" is unknown, "repository" config option should be updated. Please validate your repository config.'
|
|
187
187
|
elif "401 Client Error: Unauthorized for url" in message or ("Error: Unauthorized" in message and "401" in message):
|
|
188
|
-
# 401 Client Error: Unauthorized for url: https://api.github.com/orgs/datarootsio/repos?per_page=100&sort=updated&direction=desc
|
|
189
188
|
user_message = (
|
|
190
|
-
"
|
|
189
|
+
"GitHub authentication failed (HTTP 401). Please verify your Personal Access Token or OAuth credentials "
|
|
190
|
+
"are valid and not expired."
|
|
191
191
|
)
|
|
192
192
|
return user_message
|
|
193
193
|
|
|
@@ -151,44 +151,62 @@ class GithubStreamABC(HttpStream, ABC):
|
|
|
151
151
|
elif isinstance(self, TeamMemberships):
|
|
152
152
|
error_msg = f"Syncing `{self.__class__.__name__}` stream for organization `{organisation}`, team `{stream_slice.get('team_slug')}` and user `{stream_slice.get('username')}` isn't available: User has no team membership. Skipping..."
|
|
153
153
|
else:
|
|
154
|
-
error_msg =
|
|
154
|
+
error_msg = (
|
|
155
|
+
f"Skipping `{self.__class__.__name__}` for repository `{repository}`: "
|
|
156
|
+
f"GitHub returned 404 Not Found. The repository may not exist, may have been deleted, "
|
|
157
|
+
f"or the configured token may lack access to it."
|
|
158
|
+
)
|
|
155
159
|
elif e._exception.response.status_code == requests.codes.FORBIDDEN:
|
|
156
|
-
|
|
160
|
+
api_message = (e._exception.response.json() or {}).get("message", "")
|
|
157
161
|
# When using the `check_connection` method, we should raise an error if we do not have access to the repository.
|
|
158
162
|
if isinstance(self, Repositories):
|
|
159
163
|
raise e
|
|
160
164
|
# When `403` for the stream, that has no access to the organization's teams, based on OAuth Apps Restrictions:
|
|
161
165
|
# https://docs.github.com/en/organizations/restricting-access-to-your-organizations-data/enabling-oauth-app-access-restrictions-for-your-organization
|
|
162
166
|
# For all `Organisation` based streams
|
|
163
|
-
elif isinstance(self, Organizations
|
|
167
|
+
elif isinstance(self, (Organizations, Teams, Users)):
|
|
164
168
|
error_msg = (
|
|
165
|
-
f"
|
|
169
|
+
f"Skipping `{self.name}` for organization `{organisation}`: "
|
|
170
|
+
f"GitHub denied access (HTTP 403). Your token may be missing the `read:org` scope, "
|
|
171
|
+
f"or this organization may require SAML SSO authorization. "
|
|
172
|
+
f"GitHub message: {api_message!r}"
|
|
166
173
|
)
|
|
167
174
|
# For all other `Repository` base streams
|
|
168
175
|
else:
|
|
169
176
|
error_msg = (
|
|
170
|
-
f"
|
|
177
|
+
f"Skipping `{self.name}` for repository `{repository}`: "
|
|
178
|
+
f"GitHub denied access (HTTP 403). Your token may be missing required scopes, "
|
|
179
|
+
f"or this organization may require SAML SSO authorization. "
|
|
180
|
+
f"GitHub message: {api_message!r}"
|
|
171
181
|
)
|
|
172
182
|
elif e._exception.response.status_code == requests.codes.UNAUTHORIZED:
|
|
183
|
+
api_message = (e._exception.response.json() or {}).get("message", "")
|
|
173
184
|
if self.access_token_type == constants.PERSONAL_ACCESS_TOKEN_TITLE:
|
|
174
|
-
|
|
175
|
-
|
|
185
|
+
self.logger.error(
|
|
186
|
+
f"GitHub authentication failed (HTTP 401) for stream `{self.name}`. "
|
|
187
|
+
f"Your Personal Access Token may need to be renewed. GitHub message: {api_message!r}"
|
|
188
|
+
)
|
|
176
189
|
raise e
|
|
177
190
|
elif e._exception.response.status_code == requests.codes.GONE and isinstance(self, Projects):
|
|
178
|
-
# Some repos don't have projects enabled and we
|
|
191
|
+
# Some repos don't have projects enabled and we get "410 Client Error: Gone for
|
|
179
192
|
# url: https://api.github.com/repos/xyz/projects?per_page=100" error.
|
|
180
|
-
error_msg =
|
|
193
|
+
error_msg = (
|
|
194
|
+
f"GitHub Projects (classic) is disabled for repository `{stream_slice['repository']}`. "
|
|
195
|
+
f"Skipping the `Projects` stream for this repository."
|
|
196
|
+
)
|
|
181
197
|
elif e._exception.response.status_code == requests.codes.CONFLICT:
|
|
182
198
|
error_msg = (
|
|
183
|
-
f"
|
|
184
|
-
f"
|
|
199
|
+
f"Skipping `{self.name}` for repository `{stream_slice['repository']}`: "
|
|
200
|
+
f"GitHub returned 409 Conflict. The repository is likely empty (no commits)."
|
|
185
201
|
)
|
|
186
202
|
elif e._exception.response.status_code == requests.codes.SERVER_ERROR and isinstance(self, WorkflowRuns):
|
|
187
203
|
error_msg = f"Syncing `{self.name}` stream isn't available for repository `{stream_slice['repository']}`."
|
|
188
204
|
elif e._exception.response.status_code == requests.codes.BAD_GATEWAY:
|
|
189
|
-
error_msg =
|
|
205
|
+
error_msg = (
|
|
206
|
+
f"GitHub returned HTTP 502 Bad Gateway for stream `{self.name}` after exhausting retries. "
|
|
207
|
+
f"This is usually transient — the next sync attempt should succeed."
|
|
208
|
+
)
|
|
190
209
|
else:
|
|
191
|
-
# most probably here we're facing a 500 server error and a risk to get a non-json response, so lets output response.text
|
|
192
210
|
self.logger.error(f"Undefined error while reading records: {e._exception.response.text}")
|
|
193
211
|
raise e
|
|
194
212
|
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/backoff_strategies.py
RENAMED
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/config_migrations.py
RENAMED
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/github_schema.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/assignees.json
RENAMED
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/branches.json
RENAMED
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/comments.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/commits.json
RENAMED
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/deployments.json
RENAMED
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/events.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/issues.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/projects.json
RENAMED
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/projects_v2.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/releases.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/reviews.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/shared/user.json
RENAMED
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/stargazers.json
RENAMED
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/tags.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/teams.json
RENAMED
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/users.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
{airbyte_source_github-2.1.24 → airbyte_source_github-2.1.25}/source_github/schemas/workflows.json
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|