airbyte-source-google-search-console 1.4.2__py3-none-any.whl → 1.4.3__py3-none-any.whl

Sign up to get free protection for your applications and to get access to all the features.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: airbyte-source-google-search-console
3
- Version: 1.4.2
3
+ Version: 1.4.3
4
4
  Summary: Source implementation for Google Search Console.
5
5
  Home-page: https://airbyte.com
6
6
  License: Elv2
@@ -12,7 +12,7 @@ Classifier: Programming Language :: Python :: 3
12
12
  Classifier: Programming Language :: Python :: 3.9
13
13
  Classifier: Programming Language :: Python :: 3.10
14
14
  Classifier: Programming Language :: Python :: 3.11
15
- Requires-Dist: airbyte-cdk (==0.80.0)
15
+ Requires-Dist: airbyte-cdk (==0.90.0)
16
16
  Requires-Dist: google-api-python-client (==2.105.0)
17
17
  Requires-Dist: google-auth (==2.23.3)
18
18
  Project-URL: Documentation, https://docs.airbyte.com/integrations/sources/google-search-console
@@ -21,32 +21,33 @@ Description-Content-Type: text/markdown
21
21
 
22
22
  # Google-Search-Console source connector
23
23
 
24
-
25
24
  This is the repository for the Google-Search-Console source connector, written in Python.
26
25
  For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.com/integrations/sources/google-search-console).
27
26
 
28
27
  ## Local development
29
28
 
30
29
  ### Prerequisites
31
- * Python (~=3.9)
32
- * Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
33
30
 
31
+ - Python (~=3.9)
32
+ - Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
34
33
 
35
34
  ### Installing the connector
35
+
36
36
  From this connector directory, run:
37
+
37
38
  ```bash
38
39
  poetry install --with dev
39
40
  ```
40
41
 
41
-
42
42
  ### Create credentials
43
+
43
44
  **If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/google-search-console)
44
45
  to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_google_search_console/spec.yaml` file.
45
46
  Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
46
47
  See `sample_files/sample_config.json` for a sample config file.
47
48
 
48
-
49
49
  ### Locally running the connector
50
+
50
51
  ```
51
52
  poetry run source-google-search-console spec
52
53
  poetry run source-google-search-console check --config secrets/config.json
@@ -55,23 +56,28 @@ poetry run source-google-search-console read --config secrets/config.json --cata
55
56
  ```
56
57
 
57
58
  ### Running unit tests
59
+
58
60
  To run unit tests locally, from the connector directory run:
61
+
59
62
  ```
60
63
  poetry run pytest unit_tests
61
64
  ```
62
65
 
63
66
  ### Building the docker image
67
+
64
68
  1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
65
69
  2. Run the following command to build the docker image:
70
+
66
71
  ```bash
67
72
  airbyte-ci connectors --name=source-google-search-console build
68
73
  ```
69
74
 
70
75
  An image will be available on your host with the tag `airbyte/source-google-search-console:dev`.
71
76
 
72
-
73
77
  ### Running as a docker container
78
+
74
79
  Then run any of the connector commands as follows:
80
+
75
81
  ```
76
82
  docker run --rm airbyte/source-google-search-console:dev spec
77
83
  docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-google-search-console:dev check --config /secrets/config.json
@@ -80,18 +86,23 @@ docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integrat
80
86
  ```
81
87
 
82
88
  ### Running our CI test suite
89
+
83
90
  You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
91
+
84
92
  ```bash
85
93
  airbyte-ci connectors --name=source-google-search-console test
86
94
  ```
87
95
 
88
96
  ### Customizing acceptance Tests
97
+
89
98
  Customize `acceptance-test-config.yml` file to configure acceptance tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
90
99
  If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
91
100
 
92
101
  ### Dependency Management
93
- All of your dependencies should be managed via Poetry.
102
+
103
+ All of your dependencies should be managed via Poetry.
94
104
  To add a new dependency, run:
105
+
95
106
  ```bash
96
107
  poetry add <package-name>
97
108
  ```
@@ -99,14 +110,17 @@ poetry add <package-name>
99
110
  Please commit the changes to `pyproject.toml` and `poetry.lock` files.
100
111
 
101
112
  ## Publishing a new version of the connector
113
+
102
114
  You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
115
+
103
116
  1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-google-search-console test`
104
- 2. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)):
105
- - bump the `dockerImageTag` value in in `metadata.yaml`
106
- - bump the `version` value in `pyproject.toml`
117
+ 2. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)):
118
+ - bump the `dockerImageTag` value in in `metadata.yaml`
119
+ - bump the `version` value in `pyproject.toml`
107
120
  3. Make sure the `metadata.yaml` content is up to date.
108
121
  4. Make sure the connector documentation and its changelog is up to date (`docs/integrations/sources/google-search-console.md`).
109
122
  5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
110
123
  6. Pat yourself on the back for being an awesome contributor.
111
124
  7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
112
125
  8. Once your PR is merged, the new version of the connector will be automatically published to Docker Hub and our connector registry.
126
+
@@ -17,10 +17,10 @@ source_google_search_console/schemas/search_analytics_site_report_by_site.json,s
17
17
  source_google_search_console/schemas/sitemaps.json,sha256=coyPSZCAfzMheybfRp4WPAZCp5JF2KGRF2rWK8oC080,1775
18
18
  source_google_search_console/schemas/sites.json,sha256=WNiCRuStPL1YkJiFa8FEbNJmqaERAOf9Yow6ygIumvo,383
19
19
  source_google_search_console/service_account_authenticator.py,sha256=ocSisT5IYegoZf4atsGm9u2lJCdmeSf_9o4q7YN3vD4,1225
20
- source_google_search_console/source.py,sha256=TEqjXQr_X_8J0fPrLhQVIzIIgfspF7sgLUjAjKNtY-Q,10013
20
+ source_google_search_console/source.py,sha256=s-ZFiZ0iDL_OGOndO9tyBAoiiLSPtCf5kVmHWsxE25k,10029
21
21
  source_google_search_console/spec.json,sha256=WYtFvaSqWYGm1Dt2yV9G92U78Q94rh9oarbxJe3H7xo,8470
22
- source_google_search_console/streams.py,sha256=IyJ2lvLkC9axG-OS5cPzYantmIAwci8ltwDzy10kTdI,18404
23
- airbyte_source_google_search_console-1.4.2.dist-info/METADATA,sha256=L4Sd4OZ-21Rs0DN-21M4D9UYmIEZmHKB06gGGus4_PI,5609
24
- airbyte_source_google_search_console-1.4.2.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
25
- airbyte_source_google_search_console-1.4.2.dist-info/entry_points.txt,sha256=DMcgc9bCX-Vt6hm_68pa77qS3eGdeMhg-UdlFc-XKUM,85
26
- airbyte_source_google_search_console-1.4.2.dist-info/RECORD,,
22
+ source_google_search_console/streams.py,sha256=8uj_SjdDykkb6vf-gcDqP6j6Ve7bu8rw3NT_LFtUQxE,18331
23
+ airbyte_source_google_search_console-1.4.3.dist-info/METADATA,sha256=WtR81PaRJEnP739ng6YWj1Og_fF4uzY9HTPddEqIoQM,5619
24
+ airbyte_source_google_search_console-1.4.3.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
25
+ airbyte_source_google_search_console-1.4.3.dist-info/entry_points.txt,sha256=DMcgc9bCX-Vt6hm_68pa77qS3eGdeMhg-UdlFc-XKUM,85
26
+ airbyte_source_google_search_console-1.4.3.dist-info/RECORD,,
@@ -13,7 +13,7 @@ from airbyte_cdk.logger import AirbyteLogger
13
13
  from airbyte_cdk.models import FailureType, SyncMode
14
14
  from airbyte_cdk.sources import AbstractSource
15
15
  from airbyte_cdk.sources.streams import Stream
16
- from airbyte_cdk.sources.streams.http.auth import Oauth2Authenticator
16
+ from airbyte_cdk.sources.streams.http.requests_native_auth import Oauth2Authenticator
17
17
  from airbyte_cdk.utils import AirbyteTracedException
18
18
  from source_google_search_console.exceptions import (
19
19
  InvalidSiteURLValidationError,
@@ -11,7 +11,7 @@ import pendulum
11
11
  import requests
12
12
  from airbyte_cdk.models import SyncMode
13
13
  from airbyte_cdk.sources.streams.http import HttpStream
14
- from airbyte_cdk.sources.streams.http.auth import HttpAuthenticator
14
+ from requests.auth import AuthBase
15
15
 
16
16
  BASE_URL = "https://www.googleapis.com/webmasters/v3/"
17
17
  ROW_LIMIT = 25000
@@ -30,7 +30,7 @@ class GoogleSearchConsole(HttpStream, ABC):
30
30
 
31
31
  def __init__(
32
32
  self,
33
- authenticator: Union[HttpAuthenticator, requests.auth.AuthBase],
33
+ authenticator: AuthBase,
34
34
  site_urls: list,
35
35
  start_date: str,
36
36
  end_date: str,
@@ -366,7 +366,7 @@ class SearchByKeyword(SearchAnalytics):
366
366
  ) -> Optional[Union[Dict[str, Any], str]]:
367
367
  data = super().request_body_json(stream_state, stream_slice, next_page_token)
368
368
 
369
- stream = SearchAppearance(self.authenticator, self._site_urls, self._start_date, self._end_date)
369
+ stream = SearchAppearance(self._session.auth, self._site_urls, self._start_date, self._end_date)
370
370
  keywords_records = stream.read_records(sync_mode=SyncMode.full_refresh, stream_state=stream_state, stream_slice=stream_slice)
371
371
  keywords = {record["searchAppearance"] for record in keywords_records}
372
372
  filters = []