airbyte-source-convex 0.4.0__tar.gz

airbyte-source-convex-0.4.0/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.1
+Name: airbyte-source-convex
+Version: 0.4.0
+Summary: Source implementation for Convex.
+Author: Airbyte
+Author-email: contact@airbyte.io
+Description-Content-Type: text/markdown
+Requires-Dist: airbyte-cdk~=0.2
+Provides-Extra: tests
+Requires-Dist: requests-mock~=1.9.3; extra == "tests"
+Requires-Dist: pytest~=6.1; extra == "tests"
+Requires-Dist: pytest-mock~=3.6.1; extra == "tests"
+Requires-Dist: responses~=0.13.3; extra == "tests"
+
+# Convex Source
+
+This is the repository for the Convex source connector, written in Python.
+For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/convex).
+
+
+**To iterate on this connector, make sure to complete this prerequisites section.**
+
+
+From this connector directory, create a virtual environment:
+```
+python -m venv .venv
+```
+
+This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
+development environment of choice. To activate it from the terminal, run:
+```
+source .venv/bin/activate
+pip install -r requirements.txt
+pip install '.[tests]'
+```
+If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
+
+Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
+used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
+If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
+should work as you expect.
+
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/convex)
+to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_convex/spec.yaml` file.
+Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
+See `integration_tests/sample_config.json` for a sample config file.
+
+**If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source convex test creds`
+and place them into `secrets/config.json`.
+
+```
+python main.py spec
+python main.py check --config secrets/config.json
+python main.py discover --config secrets/config.json
+python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
+```
+
+
+
+**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
+```bash
+airbyte-ci connectors --name=source-convex build
+```
+
+An image will be built with the tag `airbyte/source-convex:dev`.
+
+**Via `docker build`:**
+```bash
+docker build -t airbyte/source-convex:dev .
+```
+
+Then run any of the connector commands as follows:
+```
+docker run --rm airbyte/source-convex:dev spec
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev check --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev discover --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-convex:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
+```
+
+You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
+```bash
+airbyte-ci connectors --name=source-convex test
+```
+
+Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+
+All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
+We split dependencies between two groups, dependencies that are:
+* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
+* required for the testing need to go to `TEST_REQUIREMENTS` list
+
+You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
+1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-convex test`
+2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
+3. Make sure the `metadata.yaml` content is up to date.
+4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/convex.md`).
+5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
+6. Pat yourself on the back for being an awesome contributor.
+7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.

airbyte-source-convex-0.4.0/README.md

@@ -0,0 +1,100 @@
+# Convex Source
+
+This is the repository for the Convex source connector, written in Python.
+For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/convex).
+
+## Local development
+
+### Prerequisites
+**To iterate on this connector, make sure to complete this prerequisites section.**
+
+#### Minimum Python version required `= 3.9.0`
+
+#### Build & Activate Virtual Environment and install dependencies
+From this connector directory, create a virtual environment:
+```
+python -m venv .venv
+```
+
+This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
+development environment of choice. To activate it from the terminal, run:
+```
+source .venv/bin/activate
+pip install -r requirements.txt
+pip install '.[tests]'
+```
+If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
+
+Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
+used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
+If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
+should work as you expect.
+
+#### Create credentials
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/convex)
+to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_convex/spec.yaml` file.
+Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
+See `integration_tests/sample_config.json` for a sample config file.
+
+**If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source convex test creds`
+and place them into `secrets/config.json`.
+
+### Locally running the connector
+```
+python main.py spec
+python main.py check --config secrets/config.json
+python main.py discover --config secrets/config.json
+python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
+```
+
+### Locally running the connector docker image
+
+
+#### Build
+**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
+```bash
+airbyte-ci connectors --name=source-convex build
+```
+
+An image will be built with the tag `airbyte/source-convex:dev`.
+
+**Via `docker build`:**
+```bash
+docker build -t airbyte/source-convex:dev .
+```
+
+#### Run
+Then run any of the connector commands as follows:
+```
+docker run --rm airbyte/source-convex:dev spec
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev check --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev discover --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-convex:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
+```
+
+## Testing
+You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
+```bash
+airbyte-ci connectors --name=source-convex test
+```
+
+### Customizing acceptance Tests
+Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+
+## Dependency Management
+All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
+We split dependencies between two groups, dependencies that are:
+* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
+* required for the testing need to go to `TEST_REQUIREMENTS` list
+
+### Publishing a new version of the connector
+You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
+1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-convex test`
+2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
+3. Make sure the `metadata.yaml` content is up to date.
+4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/convex.md`).
+5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
+6. Pat yourself on the back for being an awesome contributor.
+7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
+

airbyte-source-convex-0.4.0/airbyte_source_convex.egg-info/PKG-INFO

@@ -0,0 +1,100 @@
+Metadata-Version: 2.1
+Name: airbyte-source-convex
+Version: 0.4.0
+Summary: Source implementation for Convex.
+Author: Airbyte
+Author-email: contact@airbyte.io
+Description-Content-Type: text/markdown
+Requires-Dist: airbyte-cdk~=0.2
+Provides-Extra: tests
+Requires-Dist: requests-mock~=1.9.3; extra == "tests"
+Requires-Dist: pytest~=6.1; extra == "tests"
+Requires-Dist: pytest-mock~=3.6.1; extra == "tests"
+Requires-Dist: responses~=0.13.3; extra == "tests"
+
+# Convex Source
+
+This is the repository for the Convex source connector, written in Python.
+For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/convex).
+
+
+**To iterate on this connector, make sure to complete this prerequisites section.**
+
+
+From this connector directory, create a virtual environment:
+```
+python -m venv .venv
+```
+
+This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
+development environment of choice. To activate it from the terminal, run:
+```
+source .venv/bin/activate
+pip install -r requirements.txt
+pip install '.[tests]'
+```
+If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
+
+Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
+used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
+If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
+should work as you expect.
+
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/convex)
+to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_convex/spec.yaml` file.
+Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
+See `integration_tests/sample_config.json` for a sample config file.
+
+**If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source convex test creds`
+and place them into `secrets/config.json`.
+
+```
+python main.py spec
+python main.py check --config secrets/config.json
+python main.py discover --config secrets/config.json
+python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
+```
+
+
+
+**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
+```bash
+airbyte-ci connectors --name=source-convex build
+```
+
+An image will be built with the tag `airbyte/source-convex:dev`.
+
+**Via `docker build`:**
+```bash
+docker build -t airbyte/source-convex:dev .
+```
+
+Then run any of the connector commands as follows:
+```
+docker run --rm airbyte/source-convex:dev spec
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev check --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev discover --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-convex:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
+```
+
+You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
+```bash
+airbyte-ci connectors --name=source-convex test
+```
+
+Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+
+All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
+We split dependencies between two groups, dependencies that are:
+* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
+* required for the testing need to go to `TEST_REQUIREMENTS` list
+
+You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
+1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-convex test`
+2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
+3. Make sure the `metadata.yaml` content is up to date.
+4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/convex.md`).
+5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
+6. Pat yourself on the back for being an awesome contributor.
+7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.

airbyte-source-convex-0.4.0/airbyte_source_convex.egg-info/SOURCES.txt

@@ -0,0 +1,24 @@
+README.md
+setup.cfg
+setup.py
+airbyte_source_convex.egg-info/PKG-INFO
+airbyte_source_convex.egg-info/SOURCES.txt
+airbyte_source_convex.egg-info/dependency_links.txt
+airbyte_source_convex.egg-info/entry_points.txt
+airbyte_source_convex.egg-info/requires.txt
+airbyte_source_convex.egg-info/top_level.txt
+integration_tests/__init__.py
+integration_tests/abnormal_state.json
+integration_tests/acceptance.py
+integration_tests/configured_catalog.json
+integration_tests/invalid_config.json
+integration_tests/sample_config.json
+integration_tests/sample_state.json
+source_convex/__init__.py
+source_convex/run.py
+source_convex/source.py
+source_convex/spec.yaml
+unit_tests/__init__.py
+unit_tests/test_incremental_streams.py
+unit_tests/test_source.py
+unit_tests/test_streams.py

airbyte-source-convex-0.4.0/airbyte_source_convex.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

airbyte-source-convex-0.4.0/airbyte_source_convex.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+source-convex = source_convex.run:run

airbyte-source-convex-0.4.0/airbyte_source_convex.egg-info/requires.txt

@@ -0,0 +1,7 @@
+airbyte-cdk~=0.2
+
+[tests]
+requests-mock~=1.9.3
+pytest~=6.1
+pytest-mock~=3.6.1
+responses~=0.13.3

airbyte-source-convex-0.4.0/airbyte_source_convex.egg-info/top_level.txt

@@ -0,0 +1,3 @@
+integration_tests
+source_convex
+unit_tests

airbyte-source-convex-0.4.0/integration_tests/__init__.py

@@ -0,0 +1,3 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#

airbyte-source-convex-0.4.0/integration_tests/abnormal_state.json

@@ -0,0 +1,12 @@
+{
+  "posts": {
+    "snapshot_cursor": "hi",
+    "snapshot_has_more": false,
+    "delta_cursor": 2652635567679741986
+  },
+  "users": {
+    "snapshot_cursor": "hi",
+    "snapshot_has_more": false,
+    "delta_cursor": 2660025892355943945
+  }
+}

airbyte-source-convex-0.4.0/integration_tests/acceptance.py

@@ -0,0 +1,14 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+import pytest
+
+pytest_plugins = ("connector_acceptance_test.plugin",)
+
+
+@pytest.fixture(scope="session", autouse=True)
+def connector_setup():
+    """This fixture is a placeholder for external resources that acceptance test might require."""
+    yield

airbyte-source-convex-0.4.0/integration_tests/configured_catalog.json

@@ -0,0 +1,56 @@
+{
+  "streams": [
+    {
+      "sync_mode": "incremental",
+      "destination_sync_mode": "append",
+      "stream": {
+        "name": "posts",
+        "json_schema": {
+          "type": "object",
+          "properties": {
+            "_creationTime": { "type": "number" },
+            "_id": {
+              "type": "object",
+              "properties": { "$id": { "type": "string" } }
+            },
+            "author": {
+              "type": "object",
+              "properties": { "$id": { "type": "string" } }
+            },
+            "body": { "type": "string" },
+            "time": { "type": "number" },
+            "_ts": { "type": "number" }
+          }
+        },
+        "supported_sync_modes": ["full_refresh", "incremental"],
+        "source_defined_cursor": true,
+        "default_cursor_field": ["_ts"],
+        "source_defined_primary_key": [["_id"]]
+      }
+    },
+    {
+      "sync_mode": "incremental",
+      "destination_sync_mode": "append",
+      "stream": {
+        "name": "users",
+        "json_schema": {
+          "type": "object",
+          "properties": {
+            "_creationTime": { "type": "number" },
+            "_id": {
+              "type": "object",
+              "properties": { "$id": { "type": "string" } }
+            },
+            "name": { "type": "string" },
+            "tokenIdentifier": { "type": "string" },
+            "_ts": { "type": "number" }
+          }
+        },
+        "supported_sync_modes": ["full_refresh", "incremental"],
+        "source_defined_cursor": true,
+        "default_cursor_field": ["_ts"],
+        "source_defined_primary_key": [["_id"]]
+      }
+    }
+  ]
+}

airbyte-source-convex-0.4.0/integration_tests/invalid_config.json

@@ -0,0 +1,4 @@
+{
+  "deployment_url": "https://murky-swan-635.convex.cloud",
+  "access_key": "bad"
+}

airbyte-source-convex-0.4.0/integration_tests/sample_config.json

@@ -0,0 +1,4 @@
+{
+  "deployment_url": "https://descriptive-vulture-260.convex.cloud",
+  "access_key": "Your access token"
+}

airbyte-source-convex-0.4.0/integration_tests/sample_state.json

@@ -0,0 +1,12 @@
+{
+  "posts": {
+    "snapshot_cursor": "hi",
+    "snapshot_has_more": false,
+    "delta_cursor": 1
+  },
+  "users": {
+    "snapshot_cursor": null,
+    "snapshot_has_more": true,
+    "delta_cursor": null
+  }
+}

airbyte-source-convex-0.4.0/setup.cfg

@@ -0,0 +1,97 @@
+[metadata]
+name = airbyte-source-convex
+version = 0.4.0
+author = Airbyte
+author_email = contact@airbyte.io
+long_description = # Convex Source
+	
+	This is the repository for the Convex source connector, written in Python.
+	For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/convex).
+	
+	
+	**To iterate on this connector, make sure to complete this prerequisites section.**
+	
+	
+	From this connector directory, create a virtual environment:
+	```
+	python -m venv .venv
+	```
+	
+	This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
+	development environment of choice. To activate it from the terminal, run:
+	```
+	source .venv/bin/activate
+	pip install -r requirements.txt
+	pip install '.[tests]'
+	```
+	If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
+	
+	Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
+	used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
+	If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
+	should work as you expect.
+	
+	**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/convex)
+	to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_convex/spec.yaml` file.
+	Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
+	See `integration_tests/sample_config.json` for a sample config file.
+	
+	**If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source convex test creds`
+	and place them into `secrets/config.json`.
+	
+	```
+	python main.py spec
+	python main.py check --config secrets/config.json
+	python main.py discover --config secrets/config.json
+	python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
+	```
+	
+	
+	
+	**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
+	```bash
+	airbyte-ci connectors --name=source-convex build
+	```
+	
+	An image will be built with the tag `airbyte/source-convex:dev`.
+	
+	**Via `docker build`:**
+	```bash
+	docker build -t airbyte/source-convex:dev .
+	```
+	
+	Then run any of the connector commands as follows:
+	```
+	docker run --rm airbyte/source-convex:dev spec
+	docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev check --config /secrets/config.json
+	docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-convex:dev discover --config /secrets/config.json
+	docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-convex:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
+	```
+	
+	You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
+	```bash
+	airbyte-ci connectors --name=source-convex test
+	```
+	
+	Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+	If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+	
+	All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
+	We split dependencies between two groups, dependencies that are:
+	* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
+	* required for the testing need to go to `TEST_REQUIREMENTS` list
+	
+	You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
+	1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-convex test`
+	2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
+	3. Make sure the `metadata.yaml` content is up to date.
+	4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/convex.md`).
+	5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
+	6. Pat yourself on the back for being an awesome contributor.
+	7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
+long_description_content_type = text/markdown
+
+[egg_info]
+tag_build = 
+tag_date = 0
+

airbyte-source-convex-0.4.0/setup.py

@@ -0,0 +1,44 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+from setuptools import find_packages, setup
+
+MAIN_REQUIREMENTS = [
+    "airbyte-cdk~=0.2",
+]
+
+TEST_REQUIREMENTS = [
+    "requests-mock~=1.9.3",
+    "pytest~=6.1",
+    "pytest-mock~=3.6.1",
+    "responses~=0.13.3",
+]
+
+setup(
+    entry_points={
+        "console_scripts": [
+            "source-convex=source_convex.run:run",
+        ],
+    },
+    description="Source implementation for Convex.",
+    packages=find_packages(),
+    install_requires=MAIN_REQUIREMENTS,
+    package_data={
+        "": [
+            # Include yaml files in the package (if any)
+            "*.yml",
+            "*.yaml",
+            # Include all json files in the package, up to 4 levels deep
+            "*.json",
+            "*/*.json",
+            "*/*/*.json",
+            "*/*/*/*.json",
+            "*/*/*/*/*.json",
+        ]
+    },
+    extras_require={
+        "tests": TEST_REQUIREMENTS,
+    },
+)

airbyte-source-convex-0.4.0/source_convex/__init__.py

@@ -0,0 +1,8 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+from .source import SourceConvex
+
+__all__ = ["SourceConvex"]

airbyte-source-convex-0.4.0/source_convex/run.py

@@ -0,0 +1,14 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+import sys
+
+from airbyte_cdk.entrypoint import launch
+from source_convex import SourceConvex
+
+
+def run():
+    source = SourceConvex()
+    launch(source, sys.argv[1:])

airbyte-source-convex-0.4.0/source_convex/source.py

@@ -0,0 +1,243 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+from datetime import datetime
+from json import JSONDecodeError
+from typing import Any, Dict, Iterable, Iterator, List, Mapping, MutableMapping, Optional, Tuple, TypedDict, cast
+
+import requests
+from airbyte_cdk.models import SyncMode
+from airbyte_cdk.sources import AbstractSource
+from airbyte_cdk.sources.streams import IncrementalMixin, Stream
+from airbyte_cdk.sources.streams.http import HttpStream
+from airbyte_cdk.sources.streams.http.requests_native_auth.token import TokenAuthenticator
+
+ConvexConfig = TypedDict(
+    "ConvexConfig",
+    {
+        "deployment_url": str,
+        "access_key": str,
+    },
+)
+
+ConvexState = TypedDict(
+    "ConvexState",
+    {
+        "snapshot_cursor": Optional[str],
+        "snapshot_has_more": bool,
+        "delta_cursor": Optional[int],
+    },
+)
+
+CONVEX_CLIENT_VERSION = "0.4.0"
+
+
+# Source
+class SourceConvex(AbstractSource):
+    def _json_schemas(self, config: ConvexConfig) -> requests.Response:
+        deployment_url = config["deployment_url"]
+        access_key = config["access_key"]
+        url = f"{deployment_url}/api/json_schemas?deltaSchema=true&format=json"
+        headers = {
+            "Authorization": f"Convex {access_key}",
+            "Convex-Client": f"airbyte-export-{CONVEX_CLIENT_VERSION}",
+        }
+        return requests.get(url, headers=headers)
+
+    def check_connection(self, logger: Any, config: Mapping[str, Any]) -> Tuple[bool, Any]:
+        """
+        Connection check to validate that the user-provided config can be used to connect to the underlying API
+
+        :param config:  the user-input config object conforming to the connector's spec.yaml
+        :param logger:  logger object
+        :return Tuple[bool, any]: (True, None) if the input config can be used to connect to the API successfully, (False, error) otherwise.
+        """
+        config = cast(ConvexConfig, config)
+        resp = self._json_schemas(config)
+        if resp.status_code == 200:
+            return True, None
+        else:
+            return False, format_http_error("Connection to Convex via json_schemas endpoint failed", resp)
+
+    def streams(self, config: Mapping[str, Any]) -> List[Stream]:
+        """
+        :param config: A Mapping of the user input configuration as defined in the connector spec.
+        """
+        config = cast(ConvexConfig, config)
+        resp = self._json_schemas(config)
+        if resp.status_code != 200:
+            raise Exception(format_http_error("Failed request to json_schemas", resp))
+        json_schemas = resp.json()
+        table_names = list(json_schemas.keys())
+        return [
+            ConvexStream(
+                config["deployment_url"],
+                config["access_key"],
+                "json",  # Use `json` export format
+                table_name,
+                json_schemas[table_name],
+            )
+            for table_name in table_names
+        ]
+
+
+class ConvexStream(HttpStream, IncrementalMixin):
+    def __init__(
+        self,
+        deployment_url: str,
+        access_key: str,
+        fmt: str,
+        table_name: str,
+        json_schema: Dict[str, Any],
+    ):
+        self.deployment_url = deployment_url
+        self.fmt = fmt
+        self.table_name = table_name
+        if json_schema:
+            json_schema["additionalProperties"] = True
+            json_schema["properties"]["_ab_cdc_lsn"] = {"type": "number"}
+            json_schema["properties"]["_ab_cdc_updated_at"] = {"type": "string"}
+            json_schema["properties"]["_ab_cdc_deleted_at"] = {"anyOf": [{"type": "string"}, {"type": "null"}]}
+        else:
+            json_schema = {}
+        self.json_schema = json_schema
+        self._snapshot_cursor_value: Optional[str] = None
+        self._snapshot_has_more = True
+        self._delta_cursor_value: Optional[int] = None
+        self._delta_has_more = True
+        super().__init__(TokenAuthenticator(access_key, "Convex"))
+
+    @property
+    def name(self) -> str:
+        return self.table_name
+
+    @property
+    def url_base(self) -> str:
+        return self.deployment_url
+
+    def get_json_schema(self) -> Mapping[str, Any]:  # type: ignore[override]
+        return self.json_schema
+
+    primary_key = "_id"
+    cursor_field = "_ts"
+
+    # Checkpoint stream reads after this many records. This prevents re-reading of data if the stream fails for any reason.
+    state_checkpoint_interval = 128
+
+    @property
+    def state(self) -> MutableMapping[str, Any]:
+        value: ConvexState = {
+            "snapshot_cursor": self._snapshot_cursor_value,
+            "snapshot_has_more": self._snapshot_has_more,
+            "delta_cursor": self._delta_cursor_value,
+        }
+        return cast(MutableMapping[str, Any], value)
+
+    @state.setter
+    def state(self, value: MutableMapping[str, Any]) -> None:
+        state = cast(ConvexState, value)
+        self._snapshot_cursor_value = state["snapshot_cursor"]
+        self._snapshot_has_more = state["snapshot_has_more"]
+        self._delta_cursor_value = state["delta_cursor"]
+
+    def next_page_token(self, response: requests.Response) -> Optional[ConvexState]:
+        if response.status_code != 200:
+            raise Exception(format_http_error("Failed request", response))
+        resp_json = response.json()
+        if self._snapshot_has_more:
+            self._snapshot_cursor_value = resp_json["cursor"]
+            self._snapshot_has_more = resp_json["hasMore"]
+            self._delta_cursor_value = resp_json["snapshot"]
+        else:
+            self._delta_cursor_value = resp_json["cursor"]
+            self._delta_has_more = resp_json["hasMore"]
+        has_more = self._snapshot_has_more or self._delta_has_more
+        return cast(ConvexState, self.state) if has_more else None
+
+    def path(
+        self,
+        stream_state: Optional[Mapping[str, Any]] = None,
+        stream_slice: Optional[Mapping[str, Any]] = None,
+        next_page_token: Optional[Mapping[str, Any]] = None,
+    ) -> str:
+        # https://docs.convex.dev/http-api/#sync
+        if self._snapshot_has_more:
+            return "/api/list_snapshot"
+        else:
+            return "/api/document_deltas"
+
+    def parse_response(
+        self,
+        response: requests.Response,
+        stream_state: Mapping[str, Any],
+        stream_slice: Optional[Mapping[str, Any]] = None,
+        next_page_token: Optional[Mapping[str, Any]] = None,
+    ) -> Iterable[Mapping[str, Any]]:
+        if response.status_code != 200:
+            raise Exception(format_http_error("Failed request", response))
+        resp_json = response.json()
+        return list(resp_json["values"])
+
+    def request_params(
+        self,
+        stream_state: Optional[Mapping[str, Any]],
+        stream_slice: Optional[Mapping[str, Any]] = None,
+        next_page_token: Optional[Mapping[str, Any]] = None,
+    ) -> MutableMapping[str, Any]:
+        params: Dict[str, Any] = {"tableName": self.table_name, "format": self.fmt}
+        if self._snapshot_has_more:
+            if self._snapshot_cursor_value:
+                params["cursor"] = self._snapshot_cursor_value
+            if self._delta_cursor_value:
+                params["snapshot"] = self._delta_cursor_value
+        else:
+            if self._delta_cursor_value:
+                params["cursor"] = self._delta_cursor_value
+        return params
+
+    def request_headers(
+        self,
+        stream_state: Optional[Mapping[str, Any]],
+        stream_slice: Optional[Mapping[str, Any]] = None,
+        next_page_token: Optional[Mapping[str, Any]] = None,
+    ) -> Dict[str, str]:
+        """
+        Custom headers for each HTTP request, not including Authorization.
+        """
+        return {
+            "Convex-Client": f"airbyte-export-{CONVEX_CLIENT_VERSION}",
+        }
+
+    def get_updated_state(self, current_stream_state: ConvexState, latest_record: Mapping[str, Any]) -> ConvexState:
+        """
+        This (deprecated) method is still used by AbstractSource to update state between calls to `read_records`.
+        """
+        return cast(ConvexState, self.state)
+
+    def read_records(self, sync_mode: SyncMode, *args: Any, **kwargs: Any) -> Iterator[Any]:
+        self._delta_has_more = sync_mode == SyncMode.incremental
+        for read_record in super().read_records(sync_mode, *args, **kwargs):
+            record = dict(read_record)
+            ts_ns = record["_ts"]
+            ts_seconds = ts_ns / 1e9  # convert from nanoseconds.
+            # equivalent of java's `new Timestamp(transactionMillis).toInstant().toString()`
+            ts_datetime = datetime.utcfromtimestamp(ts_seconds)
+            ts = ts_datetime.isoformat()
+            # DebeziumEventUtils.CDC_LSN
+            record["_ab_cdc_lsn"] = ts_ns
+            # DebeziumEventUtils.CDC_DELETED_AT
+            record["_ab_cdc_updated_at"] = ts
+            record["_deleted"] = "_deleted" in record and record["_deleted"]
+            # DebeziumEventUtils.CDC_DELETED_AT
+            record["_ab_cdc_deleted_at"] = ts if record["_deleted"] else None
+            yield record
+
+
+def format_http_error(context: str, resp: requests.Response) -> str:
+    try:
+        err = resp.json()
+        return f"{context}: {resp.status_code}: {err['code']}: {err['message']}"
+    except (JSONDecodeError, KeyError):
+        return f"{context}: {resp.text}"

airbyte-source-convex-0.4.0/source_convex/spec.yaml

@@ -0,0 +1,21 @@
+documentationUrl: https://docs.airbyte.com/integrations/sources/convex
+connectionSpecification:
+  additionalProperties: true
+  $schema: http://json-schema.org/draft-07/schema#
+  title: Convex Source Spec
+  type: object
+  required:
+    - deployment_url
+    - access_key
+  properties:
+    deployment_url:
+      type: string
+      title: Deployment Url
+      examples:
+        - https://murky-swan-635.convex.cloud
+        - https://cluttered-owl-337.convex.cloud
+    access_key:
+      type: string
+      title: Access Key
+      description: API access key used to retrieve data from Convex.
+      airbyte_secret: true

airbyte-source-convex-0.4.0/unit_tests/__init__.py

@@ -0,0 +1,3 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#

airbyte-source-convex-0.4.0/unit_tests/test_incremental_streams.py

@@ -0,0 +1,88 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+from unittest.mock import MagicMock
+
+from airbyte_cdk.models import SyncMode
+from pytest import fixture
+from source_convex.source import ConvexStream
+
+
+@fixture
+def patch_incremental_base_class(mocker):
+    # Mock abstract methods to enable instantiating abstract class
+    mocker.patch.object(ConvexStream, "path", "v0/example_endpoint")
+    mocker.patch.object(ConvexStream, "primary_key", "test_primary_key")
+    mocker.patch.object(ConvexStream, "__abstractmethods__", set())
+
+
+def test_cursor_field(patch_incremental_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    expected_cursor_field = "_ts"
+    assert stream.cursor_field == expected_cursor_field
+
+
+def test_get_updated_state(patch_incremental_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    resp = MagicMock()
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 123}], "cursor": 1234, "snapshot": 3000, "hasMore": True}
+    resp.status_code = 200
+    stream.parse_response(resp, {})
+    stream.next_page_token(resp)
+    assert stream.get_updated_state(None, None) == {
+        "snapshot_cursor": 1234,
+        "snapshot_has_more": True,
+        "delta_cursor": 3000,
+    }
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1235}], "cursor": 1235, "snapshot": 3000, "hasMore": False}
+    stream.parse_response(resp, {})
+    stream.next_page_token(resp)
+    assert stream.get_updated_state(None, None) == {
+        "snapshot_cursor": 1235,
+        "snapshot_has_more": False,
+        "delta_cursor": 3000,
+    }
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1235}], "cursor": 8000, "hasMore": True}
+    stream.parse_response(resp, {})
+    stream.next_page_token(resp)
+    assert stream.get_updated_state(None, None) == {
+        "snapshot_cursor": 1235,
+        "snapshot_has_more": False,
+        "delta_cursor": 8000,
+    }
+    assert stream._delta_has_more is True
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1235}], "cursor": 9000, "hasMore": False}
+    stream.parse_response(resp, {})
+    stream.next_page_token(resp)
+    assert stream.get_updated_state(None, None) == {
+        "snapshot_cursor": 1235,
+        "snapshot_has_more": False,
+        "delta_cursor": 9000,
+    }
+    assert stream._delta_has_more is False
+
+
+def test_stream_slices(patch_incremental_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    inputs = {"sync_mode": SyncMode.incremental, "cursor_field": [], "stream_state": {}}
+    expected_stream_slice = [None]
+    assert stream.stream_slices(**inputs) == expected_stream_slice
+
+
+def test_supports_incremental(patch_incremental_base_class, mocker):
+    mocker.patch.object(ConvexStream, "cursor_field", "dummy_field")
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    assert stream.supports_incremental
+
+
+def test_source_defined_cursor(patch_incremental_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    assert stream.source_defined_cursor
+
+
+def test_stream_checkpoint_interval(patch_incremental_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    expected_checkpoint_interval = 128
+    assert stream.state_checkpoint_interval == expected_checkpoint_interval

airbyte-source-convex-0.4.0/unit_tests/test_source.py

@@ -0,0 +1,115 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+from unittest.mock import MagicMock
+
+import responses
+from source_convex.source import SourceConvex
+
+
+def setup_responses():
+    sample_shapes_resp = {
+        "posts": {
+            "type": "object",
+            "properties": {
+                "_creationTime": {"type": "number"},
+                "_id": {"$description": "Id(posts)", "type": "object", "properties": {"$id": {"type": "string"}}},
+                "author": {"$description": "Id(users)", "type": "object", "properties": {"$id": {"type": "string"}}},
+                "body": {"type": "string"},
+                "_ts": {"type": "integer"},
+                "_deleted": {"type": "boolean"},
+            },
+            "$schema": "http://json-schema.org/draft-07/schema#",
+        },
+        "users": {
+            "type": "object",
+            "properties": {
+                "_creationTime": {"type": "number"},
+                "_id": {"$description": "Id(users)", "type": "object", "properties": {"$id": {"type": "string"}}},
+                "name": {"type": "string"},
+                "tokenIdentifier": {"type": "string"},
+                "_ts": {"type": "integer"},
+                "_deleted": {"type": "boolean"},
+            },
+            "$schema": "http://json-schema.org/draft-07/schema#",
+        },
+    }
+    responses.add(
+        responses.GET,
+        "https://murky-swan-635.convex.cloud/api/json_schemas?deltaSchema=true&format=json",
+        json=sample_shapes_resp,
+    )
+    responses.add(
+        responses.GET,
+        "https://curious-giraffe-964.convex.cloud/api/json_schemas?deltaSchema=true&format=json",
+        json={"code": "Error code", "message": "Error message"},
+        status=400,
+    )
+
+
+@responses.activate
+def test_check_connection(mocker):
+    setup_responses()
+    source = SourceConvex()
+    logger_mock = MagicMock()
+    assert source.check_connection(
+        logger_mock,
+        {
+            "deployment_url": "https://murky-swan-635.convex.cloud",
+            "access_key": "test_api_key",
+        },
+    ) == (True, None)
+
+
+@responses.activate
+def test_check_bad_connection(mocker):
+    setup_responses()
+    source = SourceConvex()
+    logger_mock = MagicMock()
+    assert source.check_connection(
+        logger_mock,
+        {
+            "deployment_url": "https://curious-giraffe-964.convex.cloud",
+            "access_key": "test_api_key",
+        },
+    ) == (False, "Connection to Convex via json_schemas endpoint failed: 400: Error code: Error message")
+
+
+@responses.activate
+def test_streams(mocker):
+    setup_responses()
+    source = SourceConvex()
+    streams = source.streams(
+        {
+            "deployment_url": "https://murky-swan-635.convex.cloud",
+            "access_key": "test_api_key",
+        }
+    )
+    assert len(streams) == 2
+    streams.sort(key=lambda stream: stream.table_name)
+    assert streams[0].table_name == "posts"
+    assert streams[1].table_name == "users"
+    assert all(stream.deployment_url == "https://murky-swan-635.convex.cloud" for stream in streams)
+    assert all(stream._session.auth.get_auth_header() == {"Authorization": "Convex test_api_key"} for stream in streams)
+    shapes = [stream.get_json_schema() for stream in streams]
+    assert all(shape["type"] == "object" for shape in shapes)
+    properties = [shape["properties"] for shape in shapes]
+    assert [
+        props["_id"]
+        == {
+            "type": "object",
+            "properties": {
+                "$id": {"type": "string"},
+            },
+        }
+        for props in properties
+    ]
+    assert [props["_ts"] == {"type": "number"} for props in properties]
+    assert [props["_creationTime"] == {"type": "number"} for props in properties]
+    assert set(properties[0].keys()) == set(
+        ["_id", "_ts", "_deleted", "_creationTime", "author", "body", "_ab_cdc_lsn", "_ab_cdc_updated_at", "_ab_cdc_deleted_at"]
+    )
+    assert set(properties[1].keys()) == set(
+        ["_id", "_ts", "_deleted", "_creationTime", "name", "tokenIdentifier", "_ab_cdc_lsn", "_ab_cdc_updated_at", "_ab_cdc_deleted_at"]
+    )

airbyte-source-convex-0.4.0/unit_tests/test_streams.py

@@ -0,0 +1,169 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+from http import HTTPStatus
+from unittest.mock import MagicMock
+
+import pytest
+import requests
+import responses
+from airbyte_cdk.models import SyncMode
+from source_convex.source import ConvexStream
+
+
+@pytest.fixture
+def patch_base_class(mocker):
+    # Mock abstract methods to enable instantiating abstract class
+    mocker.patch.object(ConvexStream, "primary_key", "test_primary_key")
+    mocker.patch.object(ConvexStream, "__abstractmethods__", set())
+
+
+def test_request_params(patch_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    inputs = {"stream_slice": None, "stream_state": None, "next_page_token": None}
+    expected_params = {"tableName": "messages", "format": "json"}
+    assert stream.request_params(**inputs) == expected_params
+    stream._snapshot_cursor_value = 1234
+    expected_params = {"tableName": "messages", "format": "json", "cursor": 1234}
+    assert stream.request_params(**inputs) == expected_params
+    stream._snapshot_has_more = False
+    stream._delta_cursor_value = 2345
+    expected_params = {"tableName": "messages", "format": "json", "cursor": 2345}
+    assert stream.request_params(**inputs) == expected_params
+
+
+def test_next_page_token(patch_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    resp = MagicMock()
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 123}], "cursor": 1234, "snapshot": 5000, "hasMore": True}
+    resp.status_code = 200
+    stream.parse_response(resp, {})
+    assert stream.next_page_token(resp) == {
+        "snapshot_cursor": 1234,
+        "snapshot_has_more": True,
+        "delta_cursor": 5000,
+    }
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1235}], "cursor": 1235, "snapshot": 5000, "hasMore": False}
+    stream.parse_response(resp, {})
+    assert stream.next_page_token(resp) == {
+        "snapshot_cursor": 1235,
+        "snapshot_has_more": False,
+        "delta_cursor": 5000,
+    }
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1235}], "cursor": 6000, "hasMore": True}
+    stream.parse_response(resp, {})
+    assert stream.next_page_token(resp) == {
+        "snapshot_cursor": 1235,
+        "snapshot_has_more": False,
+        "delta_cursor": 6000,
+    }
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1235}], "cursor": 7000, "hasMore": False}
+    stream.parse_response(resp, {})
+    assert stream.next_page_token(resp) is None
+    assert stream.state == {"snapshot_cursor": 1235, "snapshot_has_more": False, "delta_cursor": 7000}
+
+
+@responses.activate
+def test_read_records_full_refresh(patch_base_class):
+    stream = ConvexStream("http://mocked_base_url:8080", "accesskey", "json", "messages", None)
+    snapshot0_resp = {"values": [{"_id": "my_id", "field": "f", "_ts": 123}], "cursor": 1234, "snapshot": 5000, "hasMore": True}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/list_snapshot?tableName=messages&format=json",
+        json=snapshot0_resp,
+    )
+    snapshot1_resp = {"values": [{"_id": "an_id", "field": "b", "_ts": 100}], "cursor": 2345, "snapshot": 5000, "hasMore": True}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/list_snapshot?tableName=messages&format=json&cursor=1234&snapshot=5000",
+        json=snapshot1_resp,
+    )
+    snapshot2_resp = {"values": [{"_id": "a_id", "field": "x", "_ts": 300}], "cursor": 3456, "snapshot": 5000, "hasMore": False}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/list_snapshot?tableName=messages&format=json&cursor=2345&snapshot=5000",
+        json=snapshot2_resp,
+    )
+    records = list(stream.read_records(SyncMode.full_refresh))
+    assert len(records) == 3
+    assert [record["field"] for record in records] == ["f", "b", "x"]
+    assert stream.state == {"delta_cursor": 5000, "snapshot_cursor": 3456, "snapshot_has_more": False}
+
+
+@responses.activate
+def test_read_records_incremental(patch_base_class):
+    stream = ConvexStream("http://mocked_base_url:8080", "accesskey", "json", "messages", None)
+    snapshot0_resp = {"values": [{"_id": "my_id", "field": "f", "_ts": 123}], "cursor": 1234, "snapshot": 5000, "hasMore": True}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/list_snapshot?tableName=messages&format=json",
+        json=snapshot0_resp,
+    )
+    snapshot1_resp = {"values": [{"_id": "an_id", "field": "b", "_ts": 100}], "cursor": 2345, "snapshot": 5000, "hasMore": False}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/list_snapshot?tableName=messages&format=json&cursor=1234&snapshot=5000",
+        json=snapshot1_resp,
+    )
+    delta0_resp = {"values": [{"_id": "a_id", "field": "x", "_ts": 300}], "cursor": 6000, "hasMore": True}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/document_deltas?tableName=messages&format=json&cursor=5000",
+        json=delta0_resp,
+    )
+    delta1_resp = {"values": [{"_id": "a_id", "field": "x", "_ts": 400}], "cursor": 7000, "hasMore": False}
+    responses.add(
+        responses.GET,
+        "http://mocked_base_url:8080/api/document_deltas?tableName=messages&format=json&cursor=6000",
+        json=delta1_resp,
+    )
+    records = list(stream.read_records(SyncMode.incremental))
+    assert len(records) == 4
+    assert [record["field"] for record in records] == ["f", "b", "x", "x"]
+    assert stream.state == {"delta_cursor": 7000, "snapshot_cursor": 2345, "snapshot_has_more": False}
+
+
+def test_parse_response(patch_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    resp = MagicMock()
+    resp.json = lambda: {"values": [{"_id": "my_id", "field": "f", "_ts": 1234}], "cursor": 1234, "snapshot": 2000, "hasMore": True}
+    resp.status_code = 200
+    inputs = {"response": resp, "stream_state": {}}
+    expected_parsed_objects = [{"_id": "my_id", "field": "f", "_ts": 1234}]
+    assert stream.parse_response(**inputs) == expected_parsed_objects
+
+
+def test_request_headers(patch_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    inputs = {"stream_slice": None, "stream_state": None, "next_page_token": None}
+    assert stream.request_headers(**inputs) == {"Convex-Client": "airbyte-export-0.4.0"}
+
+
+def test_http_method(patch_base_class):
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    expected_method = "GET"
+    assert stream.http_method == expected_method
+
+
+@pytest.mark.parametrize(
+    ("http_status", "should_retry"),
+    [
+        (HTTPStatus.OK, False),
+        (HTTPStatus.BAD_REQUEST, False),
+        (HTTPStatus.TOO_MANY_REQUESTS, True),
+        (HTTPStatus.INTERNAL_SERVER_ERROR, True),
+    ],
+)
+def test_should_retry(patch_base_class, http_status, should_retry):
+    response_mock = MagicMock()
+    response_mock.status_code = http_status
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    assert stream.should_retry(response_mock) == should_retry
+
+
+def test_backoff_time(patch_base_class):
+    response_mock = MagicMock()
+    stream = ConvexStream("murky-swan-635", "accesskey", "json", "messages", None)
+    expected_backoff_time = None
+    assert stream.backoff_time(response_mock) == expected_backoff_time