airbyte-source-commcare 0.1.0__py3-none-any.whl

airbyte_source_commcare-0.1.0.dist-info/METADATA

@@ -0,0 +1,102 @@
+Metadata-Version: 2.1
+Name: airbyte-source-commcare
+Version: 0.1.0
+Summary: Source implementation for Commcare.
+Author: Airbyte
+Author-email: contact@airbyte.io
+Description-Content-Type: text/markdown
+Requires-Dist: airbyte-cdk
+Requires-Dist: bigquery-schema-generator ~=1.5
+Requires-Dist: gbqschema-converter ~=1.2.0
+Requires-Dist: flatten-json ~=0.1.13
+Provides-Extra: tests
+Requires-Dist: requests-mock ~=1.9.3 ; extra == 'tests'
+Requires-Dist: pytest ~=6.1 ; extra == 'tests'
+Requires-Dist: pytest-mock ~=3.6.1 ; extra == 'tests'
+
+# Commcare Source
+
+This is the repository for the Commcare source connector, written in Python.
+For information about how to use this connector within Airbyte, see [the documentation](https://docs.airbyte.io/integrations/sources/commcare).
+
+
+**To iterate on this connector, make sure to complete this prerequisites section.**
+
+
+From this connector directory, create a virtual environment:
+```
+python -m venv .venv
+```
+
+This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
+development environment of choice. To activate it from the terminal, run:
+```
+source .venv/bin/activate
+pip install -r requirements.txt
+pip install '.[tests]'
+```
+If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
+
+Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
+used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
+If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
+should work as you expect.
+
+**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/commcare)
+to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_commcare/spec.yaml` file.
+Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
+See `integration_tests/sample_config.json` for a sample config file.
+
+**If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source commcare test creds`
+and place them into `secrets/config.json`.
+
+```
+python main.py spec
+python main.py check --config secrets/config.json
+python main.py discover --config secrets/config.json
+python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
+```
+
+
+
+**Via [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) (recommended):**
+```bash
+airbyte-ci connectors --name=source-commcare build
+```
+
+An image will be built with the tag `airbyte/source-commcare:dev`.
+
+**Via `docker build`:**
+```bash
+docker build -t airbyte/source-commcare:dev .
+```
+
+Then run any of the connector commands as follows:
+```
+docker run --rm airbyte/source-commcare:dev spec
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-commcare:dev check --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-commcare:dev discover --config /secrets/config.json
+docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-commcare:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
+```
+
+You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
+```bash
+airbyte-ci connectors --name=source-commcare test
+```
+
+Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
+If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
+
+All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
+We split dependencies between two groups, dependencies that are:
+* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
+* required for the testing need to go to `TEST_REQUIREMENTS` list
+
+You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
+1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-commcare test`
+2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
+3. Make sure the `metadata.yaml` content is up to date.
+4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/commcare.md`).
+5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
+6. Pat yourself on the back for being an awesome contributor.
+7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.

airbyte_source_commcare-0.1.0.dist-info/RECORD

@@ -0,0 +1,19 @@
+integration_tests/__init__.py,sha256=4Hw-PX1-VgESLF16cDdvuYCzGJtHntThLF4qIiULWeo,61
+integration_tests/abnormal_state.json,sha256=-DJMcBhq1do1ntOZXzsaGmxvbDTmYmFS0cSSaUoRLZ4,86
+integration_tests/acceptance.py,sha256=8eU9iSDbmHyufPvAouJGhPMgPAFTCP8IKIKHLm7u5TE,435
+integration_tests/catalog.json,sha256=yj0WO6sFU4GCciYUBWjzvvfqrBh869doeOC2Pp5EI1Y,3
+integration_tests/configured_catalog.json,sha256=1tbuuekIkjKW6NKdwW9ulYAYNjErYgAJqDsq08etQLE,397
+integration_tests/invalid_config.json,sha256=kXy1C-2u6aW122jeDBlv1taNZDYjzdjRyer_2B1xgk8,142
+integration_tests/sample_config.json,sha256=Lh9_zss5lxbqGFYpZnvi8qo5eIMNcuQ6GBDKbw1dVhM,115
+integration_tests/sample_state.json,sha256=HBR6G9QPoPDFAlfAKwRmeNaQ5_56xehMLtwWr4mxd5k,63
+source_commcare/__init__.py,sha256=e2gcZC5sfkJ_uidHqoPCVmqUdL9mhpMrw4Ix4uIzSng,128
+source_commcare/run.py,sha256=aZZ10q2rM_fY8lmNk4Gmb_pOhDJvv-biHN6V-ut5Y6I,236
+source_commcare/source.py,sha256=X_4bNim1LBDCVjqnhWViaFgsjPvMTXkZZ1yNRGRf1Aw,12956
+source_commcare/spec.yaml,sha256=CMpevuCDKMFZmq16CdeilpvXKbCiQPjmI1fcgzvJQuQ,1019
+unit_tests/__init__.py,sha256=4Hw-PX1-VgESLF16cDdvuYCzGJtHntThLF4qIiULWeo,61
+unit_tests/test_source.py,sha256=qneLJGlvcpoyFlnvwzwP2voDtzi8z6c_N-lu0won5wM,682
+airbyte_source_commcare-0.1.0.dist-info/METADATA,sha256=-GDqr4YKVK8LGyoTB9nmPitvdARyMlTt9LOzmE5dJUc,5536
+airbyte_source_commcare-0.1.0.dist-info/WHEEL,sha256=oiQVh_5PnQM0E3gPdiz09WCNmwiHDMaGer_elqB3coM,92
+airbyte_source_commcare-0.1.0.dist-info/entry_points.txt,sha256=zVJihl1jgNaayMgxgA1G9KN85LZ9T8bbss4VfqE78Bg,60
+airbyte_source_commcare-0.1.0.dist-info/top_level.txt,sha256=WCua9CoU5uJ26Af2zVYpamondw5woLFReykWLPCjX7s,45
+airbyte_source_commcare-0.1.0.dist-info/RECORD,,

airbyte_source_commcare-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: bdist_wheel (0.42.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

airbyte_source_commcare-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+source-commcare = source_commcare.run:run

airbyte_source_commcare-0.1.0.dist-info/top_level.txt

@@ -0,0 +1,3 @@
+integration_tests
+source_commcare
+unit_tests

integration_tests/__init__.py

@@ -0,0 +1,3 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#

integration_tests/abnormal_state.json

@@ -0,0 +1,5 @@
+{
+  "Assess a referred patient": {
+    "indexed_on": "2023-11-25T20:30:30.2423"
+  }
+}

integration_tests/acceptance.py

@@ -0,0 +1,16 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+import pytest
+
+pytest_plugins = ("connector_acceptance_test.plugin",)
+
+
+@pytest.fixture(scope="session", autouse=True)
+def connector_setup():
+    """This fixture is a placeholder for external resources that acceptance test might require."""
+    # TODO: setup test dependencies if needed. otherwise remove the TODO comments
+    yield
+    # TODO: clean up test dependencies

integration_tests/catalog.json

@@ -0,0 +1 @@
+{}

integration_tests/configured_catalog.json

@@ -0,0 +1,16 @@
+{
+  "streams": [
+    {
+      "stream": {
+        "name": "Assess a referred patient",
+        "json_schema": {},
+        "supported_sync_modes": ["full_refresh", "incremental"],
+        "source_defined_cursor": true,
+        "default_cursor_field": ["indexed_on"]
+      },
+      "sync_mode": "incremental",
+      "cursor_field": ["indexed_on"],
+      "destination_sync_mode": "append"
+    }
+  ]
+}

integration_tests/invalid_config.json

@@ -0,0 +1,6 @@
+{
+  "app_id": "wrong app_id",
+  "api_key": "wrong api key",
+  "start_date": "This has the wrong format",
+  "project_space": "project_space"
+}

integration_tests/sample_config.json

@@ -0,0 +1,6 @@
+{
+  "app_id": "App ID",
+  "api_key": "API KEY",
+  "project_space": "project_space",
+  "start_date": "Start Date"
+}

integration_tests/sample_state.json

@@ -0,0 +1,5 @@
+{
+  "todo-stream-name": {
+    "todo-field-name": "value"
+  }
+}

source_commcare/__init__.py

@@ -0,0 +1,8 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+from .source import SourceCommcare
+
+__all__ = ["SourceCommcare"]

source_commcare/run.py

@@ -0,0 +1,14 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+
+import sys
+
+from airbyte_cdk.entrypoint import launch
+from source_commcare import SourceCommcare
+
+
+def run():
+    source = SourceCommcare()
+    launch(source, sys.argv[1:])

source_commcare/source.py

@@ -0,0 +1,337 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+import re
+from abc import ABC
+from datetime import datetime
+from typing import Any, Iterable, List, Mapping, MutableMapping, Optional, Tuple
+from urllib.parse import parse_qs
+
+import requests
+from airbyte_cdk.models import SyncMode
+from airbyte_cdk.sources import AbstractSource
+from airbyte_cdk.sources.streams import IncrementalMixin, Stream
+from airbyte_cdk.sources.streams.http import HttpStream
+from airbyte_cdk.sources.streams.http.requests_native_auth import TokenAuthenticator
+from flatten_json import flatten
+
+
+# Basic full refresh stream
+class CommcareStream(HttpStream, ABC):
+    def __init__(self, project_space, **kwargs):
+        super().__init__(**kwargs)
+        self.project_space = project_space
+
+    @property
+    def url_base(self) -> str:
+        return f"https://www.commcarehq.org/a/{self.project_space}/api/v0.5/"
+
+    # These class variables save state
+    # forms holds form ids and we filter cases which contain one of these form ids
+    # last_form_date stores the date of the last form read so the next cycle for forms and cases starts at the same timestamp
+    forms = set()
+    last_form_date = None
+    schemas = {}
+    unwantedfields = re.compile(r"^(case_|update_|meta|create_|commcare_).*$")
+
+    @property
+    def dateformat(self):
+        return "%Y-%m-%dT%H:%M:%S.%f"
+
+    def scrubUnwantedFields(self, form):
+        newform = {k: v for k, v in form.items() if not self.unwantedfields.match(k)}
+        return newform
+
+    def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:
+        try:
+            # Server returns status 500 when there are no more rows.
+            # raise an error if server returns an error
+            response.raise_for_status()
+            meta = response.json()["meta"]
+            return parse_qs(meta["next"][1:])
+        except Exception as ex:
+            return ex
+
+    def request_params(
+        self, stream_state: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> MutableMapping[str, Any]:
+
+        params = {"format": "json"}
+        return params
+
+
+class Application(CommcareStream):
+    primary_key = "id"
+
+    def __init__(self, app_id, **kwargs):
+        super().__init__(**kwargs)
+        self.app_id = app_id
+
+    def path(
+        self, stream_state: Mapping[str, Any] = None, stream_slice: Mapping[str, Any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> str:
+        return f"application/{self.app_id}/"
+
+    def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:
+        return None
+
+    def request_params(
+        self, stream_state: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> MutableMapping[str, Any]:
+
+        params = {"format": "json", "extras": "true"}
+        return params
+
+    def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapping]:
+        yield response.json()
+
+
+class IncrementalStream(CommcareStream, IncrementalMixin):
+    cursor_field = "indexed_on"
+    _cursor_value = None
+
+    @property
+    def state(self) -> Mapping[str, Any]:
+        if self._cursor_value:
+            return {self.cursor_field: self._cursor_value}
+
+    @state.setter
+    def state(self, value: Mapping[str, Any]):
+        self._cursor_value = datetime.strptime(value[self.cursor_field], self.dateformat)
+
+    @property
+    def sync_mode(self):
+        return SyncMode.incremental
+
+    @property
+    def supported_sync_modes(self):
+        return [SyncMode.incremental]
+
+    def next_page_token(self, response: requests.Response) -> Optional[Mapping[str, Any]]:
+        try:
+            # Server returns status 500 when there are no more rows.
+            # raise an error if server returns an error
+            response.raise_for_status()
+            meta = response.json()["meta"]
+            if meta["next"]:
+                return parse_qs(meta["next"][1:])
+            return None
+        except Exception:
+            return None
+
+    def request_params(
+        self, stream_state: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> MutableMapping[str, Any]:
+
+        params = {"format": "json"}
+        if next_page_token:
+            params.update(next_page_token)
+        return params
+
+    def parse_response(self, response: requests.Response, **kwargs) -> Iterable[Mapping]:
+        for o in iter(response.json()["objects"]):
+            yield o
+        return None
+
+
+class Case(IncrementalStream):
+
+    """
+    docs: https://www.commcarehq.org/a/[domain]/api/[version]/case/
+    """
+
+    cursor_field = "indexed_on"
+    primary_key = "id"
+
+    def __init__(self, start_date, app_id, schema, **kwargs):
+        super().__init__(**kwargs)
+        self._cursor_value = datetime.strptime(start_date, "%Y-%m-%dT%H:%M:%SZ")
+        self.schema = schema
+
+    def get_json_schema(self):
+        return self.schema
+
+    @property
+    def name(self):
+        # Airbyte orders streams in alpha order but since we have dependent peers and we need to
+        # pull all forms before cases, we name this stream to
+        # ensure this stream gets pulled last (assuming ascii stream names only)
+        return "zzz_case"
+
+    def path(
+        self, stream_state: Mapping[str, Any] = None, stream_slice: Mapping[str, Any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> str:
+        return "case"
+
+    def request_params(
+        self, stream_state: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> MutableMapping[str, Any]:
+
+        # start date is what we saved for forms
+        # if self.cursor_field in self.state else (CommcareStream.last_form_date or self.initial_date)
+        ix = self.state[self.cursor_field]
+        params = {"format": "json", "indexed_on_start": ix.strftime(self.dateformat), "order_by": "indexed_on", "limit": "5000"}
+        if next_page_token:
+            params.update(next_page_token)
+        return params
+
+    def read_records(self, *args, **kwargs) -> Iterable[Mapping[str, Any]]:
+        for record in super().read_records(*args, **kwargs):
+            found = False
+            for f in record["xform_ids"]:
+                if f in CommcareStream.forms:
+                    found = True
+                    break
+            if found:
+                self._cursor_value = datetime.strptime(record[self.cursor_field], self.dateformat)
+                # Make indexed_on tz aware
+                record.update({"streamname": "case", "indexed_on": record["indexed_on"] + "Z"})
+                # convert xform_ids field from array to comma separated list so flattening won't create
+                # one field per item. This is because some cases have up to 2000 xform_ids and we don't want 2000 extra
+                # fields in the schema
+                record["xform_ids"] = ",".join(record["xform_ids"])
+                frec = flatten(record)
+                yield frec
+        if self._cursor_value.microsecond == 0:
+            # Airbyte converts the cursor_field value (datetime) to string when it saves the state and
+            # our state setter parses the saved state with a format that contains microseconds
+            # self._cursor_value must have non-zero microseconds for the formatting and parsing to work correctly.
+            # This issue would also occur if an incoming record had a timestamp with zero microseconds
+            self._cursor_value = self._cursor_value.replace(microsecond=10)
+        # This cycle of pull is complete so clear out the form ids we saved for this cycle
+        CommcareStream.forms.clear()
+
+
+class Form(IncrementalStream):
+    """
+    docs: https://www.commcarehq.org/a/[domain]/api/[version]/form/
+    """
+
+    cursor_field = "indexed_on"
+    primary_key = "id"
+
+    def __init__(self, start_date, app_id, name, xmlns, schema, **kwargs):
+        super().__init__(**kwargs)
+        self.app_id = app_id
+        self._cursor_value = datetime.strptime(start_date, "%Y-%m-%dT%H:%M:%SZ")
+        self.streamname = name
+        self.xmlns = xmlns
+        self.schema = schema
+
+    @property
+    def name(self):
+        return self.streamname
+
+    def get_json_schema(self):
+        return self.schema
+
+    def path(
+        self, stream_state: Mapping[str, Any] = None, stream_slice: Mapping[str, Any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> str:
+        return "form"
+
+    def request_params(
+        self, stream_state: Mapping[str, Any], stream_slice: Mapping[str, any] = None, next_page_token: Mapping[str, Any] = None
+    ) -> MutableMapping[str, Any]:
+
+        # if self.cursor_field in self.state else self.initial_date
+        ix = self.state[self.cursor_field]
+        params = {
+            "format": "json",
+            "app_id": self.app_id,
+            "indexed_on_start": ix.strftime(self.dateformat),
+            "order_by": "indexed_on",
+            "limit": "1000",
+            "xmlns": self.xmlns,
+        }
+        if next_page_token:
+            params.update(next_page_token)
+        return params
+
+    def read_records(self, *args, **kwargs) -> Iterable[Mapping[str, Any]]:
+        upd = {"streamname": self.streamname, "xmlns": self.xmlns}
+        for record in super().read_records(*args, **kwargs):
+            self._cursor_value = datetime.strptime(record[self.cursor_field], self.dateformat)
+            CommcareStream.forms.add(record["id"])
+            form = record["form"]
+            form.update(upd)
+            # Append Z to make it timezone aware
+            form.update({"id": record["id"], "indexed_on": record["indexed_on"] + "Z"})
+            newform = self.scrubUnwantedFields(form)
+            yield flatten(newform)
+        if self._cursor_value.microsecond == 0:
+            # Airbyte converts the cursor_field value (datetime) to string when it saves the state and
+            # our state setter parses the saved state with a format that contains microseconds
+            # self._cursor_value must have non-zero microseconds for the formatting and parsing to work correctly.
+            # This issue would also occur if an incoming record had a timestamp with zero microseconds
+            self._cursor_value = self._cursor_value.replace(microsecond=10)
+
+
+# Source
+class SourceCommcare(AbstractSource):
+    def check_connection(self, logger, config) -> Tuple[bool, any]:
+        if "api_key" not in config:
+            return False, None
+        return True, None
+
+    def base_schema(self):
+        return {
+            "$schema": "http://json-schema.org/draft-07/schema#",
+            "type": "object",
+            "properties": {"id": {"type": "string"}, "indexed_on": {"type": "string", "format": "date-time"}},
+        }
+
+    def streams(self, config: Mapping[str, Any]) -> List[Stream]:
+        auth = TokenAuthenticator(config["api_key"], auth_method="ApiKey")
+        args = {
+            "authenticator": auth,
+        }
+        appdata = Application(**{**args, "app_id": config["app_id"], "project_space": config["project_space"]}).read_records(
+            sync_mode=SyncMode.full_refresh
+        )
+
+        # Generate streams for forms, one per xmlns and one stream for cases.
+        streams = self.generate_streams(args, config, appdata)
+        return streams
+
+    def generate_streams(self, args, config, appdata):
+        form_args = {"app_id": config["app_id"], "start_date": config["start_date"], "project_space": config["project_space"], **args}
+        streams = []
+        name2xmlns = {}
+
+        # Collect the form names and xmlns from the application
+        for record in appdata:
+            mods = record["modules"]
+            for m in mods:
+                forms = m["forms"]
+                for f in forms:
+                    xmlns = f["xmlns"]
+                    formname = ""
+                    if "en" in f["name"]:
+                        formname = f["name"]["en"].strip()
+                    else:
+                        # Unknown forms are named UNNAMED_xxxxx where xxxxx are the last 5 difits of the XMLNS
+                        # This convention gives us repeatable names
+                        formname = f"Unnamed_{xmlns[-5:]}"
+
+                    name = formname
+                    name2xmlns[name] = xmlns
+
+        # Create the streams from the collected names
+        # Sorted by name
+        for k in sorted(name2xmlns):
+            key = name2xmlns[k]
+            stream = Form(name=k, xmlns=key, schema=self.base_schema(), **form_args)
+            streams.append(stream)
+
+        stream = Case(
+            app_id=config["app_id"],
+            start_date=config["start_date"],
+            schema=self.base_schema(),
+            project_space=config["project_space"],
+            **args,
+        )
+        streams.append(stream)
+
+        return streams

source_commcare/spec.yaml

@@ -0,0 +1,38 @@
+documentationUrl: https://docsurl.com
+connectionSpecification:
+  $schema: http://json-schema.org/draft-07/schema#
+  title: Commcare Source Spec
+  type: object
+  required:
+    - api_key
+    - app_id
+    - start_date
+  properties:
+    api_key:
+      type: string
+      title: API Key
+      description: >-
+        Commcare API Key
+      airbyte_secret: true
+      order: 0
+    project_space:
+      type: string
+      title: Project Space
+      description: >-
+        Project Space for commcare
+      order: 1
+    app_id:
+      type: string
+      title: Application ID
+      description: >-
+        The Application ID we are interested in
+      airbyte_secret: true
+      order: 2
+    start_date:
+      type: string
+      title: Start date for extracting records
+      pattern: ^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}Z$
+      default: "2022-10-01T00:00:00Z"
+      description: >-
+        UTC date and time in the format 2017-01-25T00:00:00Z. Only records after this date will be replicated.
+      order: 3

unit_tests/__init__.py

@@ -0,0 +1,3 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#

unit_tests/test_source.py

@@ -0,0 +1,25 @@
+#
+# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
+#
+
+from unittest.mock import MagicMock, Mock
+
+import pytest
+from source_commcare.source import SourceCommcare
+
+
+@pytest.fixture(name="config")
+def config_fixture():
+    return {"api_key": "apikey", "app_id": "appid", "start_date": "2022-01-01T00:00:00Z"}
+
+
+def test_check_connection_ok(mocker, config):
+    source = SourceCommcare()
+    logger_mock = Mock()
+    assert source.check_connection(logger_mock, config=config) == (True, None)
+
+
+def test_check_connection_fail(mocker, config):
+    source = SourceCommcare()
+    logger_mock = MagicMock()
+    assert source.check_connection(logger_mock, config={}) == (False, None)