ingestr 0.13.46__tar.gz → 0.13.48__tar.gz

{ingestr-0.13.46 → ingestr-0.13.48}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ingestr
-Version: 0.13.46
+Version: 0.13.48
 Summary: ingestr is a command-line application that ingests data from various sources and stores them in any database.
 Project-URL: Homepage, https://github.com/bruin-data/ingestr
 Project-URL: Issues, https://github.com/bruin-data/ingestr/issues

{ingestr-0.13.46 → ingestr-0.13.48}/docs/.vitepress/config.mjs

@@ -140,6 +140,7 @@ export default defineConfig({
               { text: "Shopify", link: "/supported-sources/shopify.md" },
               { text: "Slack", link: "/supported-sources/slack.md" },
               { text: "Spanner", link: "/supported-sources/spanner.md" },
+              { text: "Solidgate", link: "/supported-sources/solidgate.md" },
               { text: "Stripe", link: "/supported-sources/stripe.md" },
               { text: "TikTok Ads", link: "/supported-sources/tiktok-ads.md" },
               { text: "Zendesk", link: "/supported-sources/zendesk.md" },

ingestr-0.13.48/docs/supported-sources/frankfurter.md

@@ -0,0 +1,48 @@
+# Frankfurter
+
+[Frankfurter API](https://www.frankfurter.dev/) is an online platform which fetches current and historical exchnge rate data.
+
+ingestr supports Frankfurter as a source primarily to demonstrate ingestr's features since the API doesn't require any authentication. 
+
+## URI format
+
+The URI format for Frankfurter is as follows:
+
+```plaintext
+frankfurter://?base=<currency-code-here>
+```
+
+URI parameters:
+- `base`: defines the base currency code (e.g. EUR, USD, IDR) used to calculate the exchange rates. 
+
+
+## Set-Up Frankfurter Integration
+
+Let's say you want to fetch the exchange rates for a certain period with the base currency as Indian Rupees. Here's a sample command that will copy this data into your DuckDB database:
+
+```bash
+ingestr ingest \
+--source-uri 'frankfurter://?base=INR' \
+--interval-start '2025-03-20' \ 
+--interval-end '2025-03-28' \       
+--source-table 'exchange_rates' \    
+--dest-uri 'duckdb///frankfurter.duckdb' \
+--dest-table 'my_schema.exchange_rates'
+```
+
+The result of this command will be a list of currency exchange rates from 20.03.2025-28.03.2025 with INR as the base currency in your DuckDB database. 
+
+## Tables
+
+- `latest`: Fetches the latest exchange rates.
+- `exchange_rates`: Fetches historical exchange rates for a specified date range.
+- `currencies`: Fetches a list of the available currencies and their ISO 4217 currency code (e.g. `USD`, `EUR`, `GBP`).
+
+Use these as `--source-table` parameter in the `ingestr ingest` command.
+
+
+**Notes**:
+- The arguments `--interval-start` and `--interval-end` are only relevant for the table exchange_rates.
+- If a start date but no end date is specified, then the end date will default to today's date and ingestr will retrieve data up until the latest published data.
+- Note that the [Frankfurter API](https://www.frankfurter.dev/) only publishes updates Monday-Friday. If the given date is on the weekend, the date will default to the previous Friday.
+

ingestr-0.13.48/docs/supported-sources/s3.md

@@ -0,0 +1,136 @@
+# Amazon S3
+
+Amazon Simple Storage Service (S3) is a scalable cloud storage service offered by Amazon Web Services (AWS). It allows users to store and retrieve extensive amounts of data from anywhere on the web.
+
+`ingestr` supports Amazon S3 as both a data source and destination.
+
+## URI Format
+
+The URI for connecting to Amazon S3 is structured as follows:
+
+```plaintext
+s3://?access_key_id=<your_access_key_id>&secret_access_key=<your_secret_access_key>
+```
+
+**URI Parameters:**
+
+*   `access_key_id`: Your AWS access key ID.
+*   `secret_access_key`: Your AWS secret access key.
+*   `endpoint_url`: URL of an S3-Compatiable API Server (optional, destination only)
+*   `layout`: Layout template (optional, destination only)
+
+These credentials are required to authenticate and authorize access to your S3 buckets.
+
+The `--source-table` parameter specifies the S3 bucket and file pattern using the following format:
+
+```
+<bucket-name>/<file-glob-pattern>
+```
+
+## Setting up an S3 Integration
+
+To integrate `ingestr` with Amazon S3, you need an `access_key_id` and a `secret_access_key`. For guidance on obtaining these credentials, refer to the dltHub documentation on [AWS credentials](https://dlthub.com/docs/dlt-ecosystem/verified-sources/filesystem/basic#get-credentials).
+
+Once you have your credentials, you can configure the S3 URI. The `bucket_name` and `path_to_files` (file glob pattern) are specified in the `--source-table` argument.
+
+### Example: Loading data from S3
+
+Let's assume the following details:
+*   `access_key_id`: `AKC3YOW7E`
+*   `secret_access_key`: `XCtkpL5B`
+*   S3 bucket name: `my_bucket`
+*   Path to files within the bucket: `students/students_details.csv`
+
+The following command demonstrates how to copy data from the specified S3 location to a DuckDB database:
+
+```sh
+ingestr ingest \
+    --source-uri 's3://?access_key_id=AKC3YOW7E&secret_access_key=XCtkpL5B' \
+    --source-table 'my_bucket/students/students_details.csv' \
+    --dest-uri duckdb:///s3_data.duckdb \
+    --dest-table 'processed_students.student_details'
+```
+
+This command will create a table named `student_details` within the `processed_students` schema (or equivalent grouping) in the DuckDB database file located at `s3_data.duckdb`.
+
+### Example: Uploading data to S3
+For this, example we'll assume that:
+* `records.db` is a duckdb database.
+* has a table called `public.users`.
+* the S3 credentials are the same as the example above.
+
+The following command demonstrates how to copy data from a local duckdb database to S3:
+```sh
+ingestr ingest \
+    --source-uri 'duckdb:///records.db' \
+    --source-table 'public.users' \
+    --dest-uri 's3://?access_key_id=AKC3YOW7E&secret_access_key=XCtkpL5B' \
+    --dest-table 'my_bucket/records'
+```
+
+This will result in a file structure like the following:
+```
+my_bucket/
+└── records
+    ├── _dlt_loads
+    ├── _dlt_pipeline_state
+    ├── _dlt_version
+    └── users
+        └── <load_id>.<file_id>.parquet
+```
+
+The value of `load_id` and `file_id` is determined at runtime. The default layout creates a folder with the same table name as the source and places the data inside a parquet file. This layout is configurable using the `layout` parameter.
+
+For example, if you would like to create a parquet file with the same name as the source table (as opposed to a folder) you can set `layout` to `{table_name}.{ext}` in the commandline above:
+
+```sh
+ingestr ingest \
+    --source-uri 'duckdb:///records.db' \
+    --source-table 'public.users' \
+    --dest-uri 's3://?layout={table_name}.{ext}&access_key_id=AKC3YOW7E&secret_access_key=XCtkpL5B' \ 
+    --dest-table 'my_bucket/records'
+```
+
+Result:
+```
+my_bucket/
+└── records
+    ├── _dlt_loads
+    ├── _dlt_pipeline_state
+    ├── _dlt_version
+    └── users.parquet
+```
+
+List of available Layout variables is available [here](https://dlthub.com/docs/dlt-ecosystem/destinations/filesystem#available-layout-placeholders)
+
+### Working with S3-Compatiable object stores
+`ingestr` support S3 compatiable storage services like [Minio](https://min.io/), Digital Ocean [spaces](https://www.digitalocean.com/products/spaces) and Cloudflare [R2](https://developers.cloudflare.com/r2/). You can set the `endpoint_url` in your destination URI to write data to these object stores.
+
+For example, if you're running minio on `localhost:9000`, you can write the same data as the [example above](#example-uploading-data-to-s3) by running:
+```sh
+ingestr ingest \
+    --source-uri 'duckdb:///records.db' \
+    --source-table 'public.users' \
+    --dest-uri 's3://?endpoint_url=http://localhost:9000&access_key_id=AKC3YOW7E&secret_access_key=XCtkpL5B' \
+    --dest-table 'my_bucket/records'
+```
+
+::: info NOTE
+S3-Compatiable object stores are currently only supported as destinations. 
+:::
+
+### File Glob Pattern Examples:
+
+::: warning
+Glob patterns only apply when loading data from S3 as source.
+:::
+
+The `<file-glob-pattern>` in the `--source-table` argument allows for flexible file selection. Here are some common patterns and their descriptions:
+
+| Pattern                                     | Description                                                                                                |
+| :------------------------------------------ | :--------------------------------------------------------------------------------------------------------- |
+| `bucket/**/*.csv`                           | Retrieves all CSV files recursively from `s3://bucket`.                                                    |
+| `bucket/*.csv`                              | Retrieves all CSV files located at the root level of `s3://bucket`.                                        |
+| `bucket/myFolder/**/*.jsonl`                | Retrieves all JSONL files recursively from the `myFolder` directory and its subdirectories in `s3://bucket`. |
+| `bucket/myFolder/mySubFolder/users.parquet` | Retrieves the specific `users.parquet` file from the `myFolder/mySubFolder/` path in `s3://bucket`.        |
+| `bucket/employees.jsonl`                    | Retrieves the `employees.jsonl` file located at the root level of the `s3://bucket`.                       |

ingestr-0.13.48/docs/supported-sources/solidgate.md

@@ -0,0 +1,46 @@
+# Solidgate
+
+[Solidgate](https://solidgate.com//) is a one-stop payment processing platform that ensures seamless online purchases and streamlined payment infrastructure.
+
+ingestr supports Solidgate as a source.
+
+## URI format
+
+The URI format for Solidgate is as follows:
+
+```plaintext
+solidgate://?public_key=<your-public-key>&secret_key=<your-secret-key>
+```
+
+URI parameters:
+
+- `public_key`: The public API key used to identify the account.
+
+- `secret_key`: The secret API key used to authenticate requests to the Solidgate API.
+
+
+## Setting up a Solidgate Integration
+
+Solidgate requires a few steps to set up an integration. Please follow the [guide](https://docs.solidgate.com/payments/integrate/access-to-api/#retrieve-your-credentials).
+
+Once you complete the guide, you should have `public_key` and `secret_key`, here’s a sample command that ingests data from Solidgate into a DuckDB database:
+
+```sh
+ingestr ingestr ingest \
+--source-uri "solidgate://?public_key=api_pk_test&secret_key=api_sk_test" \
+--source-table "apm-orders" \
+--dest-uri "duckdb:///solidgate.db" \
+--dest-table "dest.apmorders"
+```
+
+The result of this command will be a table in the `Solidgate.duckdb` database with JSON columns.
+
+## Tables
+
+Solidgate source allows ingesting the following sources into separate tables:
+- `subscriptions`: Provides a comprehensive view of customer subscriptions, including subscription IDs, statuses, and key timestamps such as creation, update, and expiration dates.
+- `apm-orders`: Provides essential information for anti-fraud purposes, including order IDs, transaction statuses, amounts, currencies, and payment methods, along with crucial customer details such as email addresses.
+- `card-orders`: Provides detailed information on orders processed via card payments, including transaction data, payment status, and customer details.
+
+Use these as `--source-table` parameter in the `ingestr ingest` command.
+

ingestr-0.13.48/ingestr/src/buildinfo.py

@@ -0,0 +1 @@
+version = "v0.13.48"

{ingestr-0.13.46 → ingestr-0.13.48}/ingestr/src/factory.py

@@ -54,6 +54,7 @@ from ingestr.src.sources import (
     SalesforceSource,
     ShopifySource,
     SlackSource,
+    SolidgateSource,
     SqlSource,
     StripeAnalyticsSource,
     TikTokSource,
@@ -159,6 +160,7 @@ class SourceDestinationFactory:
         "phantombuster": PhantombusterSource,
         "elasticsearch": ElasticsearchSource,
         "attio": AttioSource,
+        "solidgate": SolidgateSource,
     }
     destinations: Dict[str, Type[DestinationProtocol]] = {
         "bigquery": BigQueryDestination,

ingestr-0.13.48/ingestr/src/solidgate/__init__.py

@@ -0,0 +1,97 @@
+from typing import Iterable, Iterator
+
+import dlt
+import pendulum
+from dlt.sources import DltResource
+
+from .helpers import SolidgateClient
+
+
+@dlt.source(max_table_nesting=0)
+def solidgate_source(
+    start_date: pendulum.DateTime,
+    end_date: pendulum.DateTime | None,
+    public_key: str,
+    secret_key: str,
+) -> Iterable[DltResource]:
+    solidgate_client = SolidgateClient(public_key, secret_key)
+
+    @dlt.resource(
+        name="subscriptions",
+        write_disposition="merge",
+        primary_key="id",
+        columns={
+            "created_at": {"data_type": "timestamp", "partition": True},
+        },
+    )
+    def fetch_all_subscriptions(
+        dateTime=dlt.sources.incremental(
+            "updated_at",
+            initial_value=start_date,
+            end_value=end_date,
+            range_start="closed",
+            range_end="closed",
+        ),
+    ) -> Iterator[dict]:
+        path = "subscriptions"
+        if dateTime.end_value is None:
+            end_dt = pendulum.now(tz="UTC")
+        else:
+            end_dt = dateTime.end_value
+
+        start_dt = dateTime.last_value
+        yield solidgate_client.fetch_data(path, date_from=start_dt, date_to=end_dt)
+
+    @dlt.resource(
+        name="apm-orders",
+        write_disposition="merge",
+        primary_key="order_id",
+        columns={
+            "created_at": {"data_type": "timestamp", "partition": True},
+        },
+    )
+    def fetch_apm_orders(
+        dateTime=dlt.sources.incremental(
+            "updated_at",
+            initial_value=start_date,
+            end_value=end_date,
+            range_start="closed",
+            range_end="closed",
+        ),
+    ) -> Iterator[dict]:
+        path = "apm-orders"
+        if dateTime.end_value is None:
+            end_dt = pendulum.now(tz="UTC")
+        else:
+            end_dt = dateTime.end_value
+
+        start_dt = dateTime.last_value
+        yield solidgate_client.fetch_data(path, date_from=start_dt, date_to=end_dt)
+
+    @dlt.resource(
+        name="card-orders",
+        write_disposition="merge",
+        primary_key="order_id",
+        columns={
+            "created_at": {"data_type": "timestamp", "partition": True},
+        },
+    )
+    def fetch_card_orders(
+        dateTime=dlt.sources.incremental(
+            "updated_at",
+            initial_value=start_date,
+            end_value=end_date,
+            range_start="closed",
+            range_end="closed",
+        ),
+    ) -> Iterator[dict]:
+        path = "card-orders"
+        if dateTime.end_value is None:
+            end_dt = pendulum.now(tz="UTC")
+        else:
+            end_dt = dateTime.end_value
+
+        start_dt = dateTime.last_value
+        yield solidgate_client.fetch_data(path, date_from=start_dt, date_to=end_dt)
+
+    return fetch_all_subscriptions, fetch_apm_orders, fetch_card_orders

ingestr-0.13.48/ingestr/src/solidgate/helpers.py

@@ -0,0 +1,78 @@
+import base64
+import hashlib
+import hmac
+import json
+
+import pendulum
+
+from ingestr.src.http_client import create_client
+
+
+class SolidgateClient:
+    def __init__(self, public_key, secret_key):
+        self.base_url = "https://reports.solidgate.com/api/v1"
+        self.public_key = public_key
+        self.secret_key = secret_key
+        self.client = create_client()
+
+    def fetch_data(
+        self,
+        path: str,
+        date_from: pendulum.DateTime,
+        date_to: pendulum.DateTime,
+    ):
+        request_payload = {
+            "date_from": date_from.format("YYYY-MM-DD HH:mm:ss"),
+            "date_to": date_to.format("YYYY-MM-DD HH:mm:ss"),
+        }
+
+        json_string = json.dumps(request_payload)
+        signature = self.generateSignature(json_string)
+        headers = {
+            "merchant": self.public_key,
+            "Signature": signature,
+            "Content-Type": "application/json",
+        }
+
+        next_page_iterator = None
+        url = f"{self.base_url}/{path}"
+
+        while True:
+            payload = request_payload.copy()
+            if next_page_iterator:
+                payload["page_iterator"] = next_page_iterator
+
+            response = self.client.post(url, headers=headers, json=payload)
+            response.raise_for_status()
+            response_json = response.json()
+
+            if path == "subscriptions":
+                data = response_json["subscriptions"]
+                for _, value in data.items():
+                    if "updated_at" in value:
+                        value["updated_at"] = pendulum.parse(
+                            value["updated_at"]
+                        )
+                    yield value
+
+            else:
+                data = response_json["orders"]
+                for value in data:
+                    if "updated_at" in value:
+                        value["updated_at"] = pendulum.parse(
+                        value["updated_at"]
+                    )
+                    yield value
+
+            next_page_iterator = response_json.get("metadata", {}).get(
+                "next_page_iterator"
+            )
+            if not next_page_iterator or next_page_iterator == "None":
+                break
+
+    def generateSignature(self, json_string):
+        data = self.public_key + json_string + self.public_key
+        hmac_hash = hmac.new(
+            self.secret_key.encode("utf-8"), data.encode("utf-8"), hashlib.sha512
+        ).digest()
+        return base64.b64encode(hmac_hash.hex().encode("utf-8")).decode("utf-8")

{ingestr-0.13.46 → ingestr-0.13.48}/ingestr/src/sources.py

@@ -698,7 +698,10 @@ class StripeAnalyticsSource:
             raise ValueError("api_key in the URI is required to connect to Stripe")
 
         endpoint = None
-        table = str.capitalize(table)
+        if table == "balancetransaction":
+            table = "BalanceTransaction"
+        else:
+            table = table.capitalize()
 
         if table in [
             "Subscription",
@@ -2428,3 +2431,45 @@ class AttioSource:
             )
         except ResourcesNotFoundError:
             raise UnsupportedResourceError(table_name, "Attio")
+
+
+class SolidgateSource:
+    def handles_incrementality(self) -> bool:
+        return True
+
+    def dlt_source(self, uri: str, table: str, **kwargs):
+        parsed_uri = urlparse(uri)
+        query_params = parse_qs(parsed_uri.query)
+        public_key = query_params.get("public_key")
+        secret_key = query_params.get("secret_key")
+
+        if public_key is None:
+            raise MissingValueError("public_key", "Solidgate")
+
+        if secret_key is None:
+            raise MissingValueError("secret_key", "Solidgate")
+
+        table_name = table.replace(" ", "")
+
+        start_date = kwargs.get("interval_start")
+        if start_date is None:
+            start_date = pendulum.yesterday().in_tz("UTC")
+        else:
+            start_date = ensure_pendulum_datetime(start_date).in_tz("UTC")
+
+        end_date = kwargs.get("interval_end")
+
+        if end_date is not None:
+            end_date = ensure_pendulum_datetime(end_date).in_tz("UTC")
+
+        from ingestr.src.solidgate import solidgate_source
+
+        try:
+            return solidgate_source(
+                public_key=public_key[0],
+                secret_key=secret_key[0],
+                start_date=start_date,
+                end_date=end_date,
+            ).with_resources(table_name)
+        except ResourcesNotFoundError:
+            raise UnsupportedResourceError(table_name, "Solidgate")

ingestr-0.13.46/docs/supported-sources/frankfurter.md

@@ -1,173 +0,0 @@
-# **Frankfurter Source Documentation**
-
-The `frankfurter` source in the `ingestr` pipeline is designed to fetch exchange rate data from the [Frankfurter API](https://www.frankfurter.dev/). This source supports fetching the latest exchange rates, historical exchange rates, and currency metadata. The data can be ingested into a specified destination database, such as DuckDB.
-
----
-
-## **Command Overview**
-
-The `ingestr` command to use the `frankfurter` source is as follows:
-
-```bash
-ingestr ingest \
---source-uri 'frankfurter://?base=IDR' \
---interval-start '2025-03-27' \     # Optional. See 'exchange_rates'.
---interval-end '2025-03-28' \       # Optional.
---source-table '<table_name>' \     # E.g 'currencies', 'latest', 'exchange_rates'. See below.
---dest-uri '<your-destination-uri>' \
---dest-table '<your-schema>.<your-table_name>'
-```
-
----
-
-## **Command Parameters**
-
-### **`--source-uri`**
-- **Description**: Specifies the source URI for the Frankfurter API.
-- **Value**: `'frankfurter://'`
-- **Purpose**: Indicates that the data will be fetched from the Frankfurter API. 
-  - An optional base currency can be added `?base={base_currency}`.
-  - If no base currency is included, base currency defaults to USD.
-
----
-
-### **`--interval-start` (Optional)**
-- **Description**: The start date for fetching historical exchange rates.
-- **Value**: A date in the format `YYYY-MM-DD` (e.g., `'2025-03-27'`).
-- **Purpose**: Defines the starting point for fetching historical data.
-  - For `latest` and `currencies` this parameter is ignored.
-  - For `exchange_rates`, it defaults to the current date if not provided.
-
----
-
-### **`--interval-end` (Optional)**
-- **Description**: The end date for fetching historical exchange rates.
-- **Value**: A date in the format `YYYY-MM-DD` (e.g., `'2025-03-28'`).
-- **Purpose**: Defines the end point for fetching historical data. 
-    - If `--interval-start` is provided without `--interval-end`, `--interval-end` defaults to the current date and retrieves up until the latest published data.
-    - If `--interval-end` is provided without `--interval-start`, it will be ignored and the call will retrieve the last published data.
-    - For `latest` and `currencies` this parameter is ignored.
-
----
-
-### **`--source-table`**
-- **Description**: Specifies the table to fetch data from.
-- **Value**: One of the following:
-  - **`currencies`**: Fetches a list of the available currencies and their ISO 4217 currency code (e.g. `USD`, `EUR`, `GBP`).
-  - **`latest`**: Fetches the latest exchange rates.
-  - **`exchange_rates`**: Fetches historical exchange rates for a specified date range.
-- **Purpose**: Determines the type of data to fetch from the Frankfurter API.
-
----
-
-### **`--dest-uri`**
-- **Description**: Specifies the destination database URI.
-- **Value**: The path to the database file (e.g., `'duckdb.db'`).
-- **Purpose**: Defines where the fetched data will be stored.
-
----
-
-### **`--dest-table`**
-- **Description**: Specifies the destination table name.
-- **Value**: A string in the format `{schema}.{table_name}` (e.g., `'schema.my_table'`).
-- **Purpose**: Defines the schema and table name where the data will be written.
-- **Notes**:
-    - If the destination table does not yet exist in your database, ingestr will automatically create the table with name that is provided in this argument. The table will be structured according to the source table (see [Core Concepts](https://bruin-data.github.io/ingestr/getting-started/core-concepts.html)).
-
----
-
-## **Supported Source Tables**
-
-### **`currencies`**
-- **Description**: Fetches a list of available currencies.
-- **Columns**:
-  - `currency_code`: The ISO 4217 currency code (e.g., `USD`, `EUR`).
-  - `currency_name`: The name of the currency (e.g., `US Dollar`, `Euro`).
-- **Primary Key**: `currency_code`
-
----
-
-### **`latest`**
-- **Description**: Fetches the latest exchange rates.
-- **Columns**:
-  - `date`: The date of the exchange rates.
-  - `currency_code`: The ISO 4217 currency code (e.g., `USD`, `EUR`).
-  - `rate`: The exchange rate relative to the base currency.
-  - `base_currency`: The base currency used to calculate the exchange rate.
-- **Primary Key**: Composite key of `date`, `currency_code` and `base_currency`.
-- **Notes**:
-  - The base currency (e.g., `EUR`) is included with a rate of `1.0`.
-
----
-
-### **`exchange_rates`**
-- **Description**: Fetches historical exchange rates for a specified date range.
-- **Columns**:
-  - `date`: The date of the exchange rates.
-  - `currency_code`: The ISO 4217 currency code (e.g., `USD`, `EUR`).
-  - `rate`: The exchange rate relative to the base currency.
-  - `base_currency`: The base currency used to calculate the exchange rate.
-- **Primary Key**: Composite key of `date`, `currency_code` and `base_currency`.
-- **Notes**:
-  - An optional start and end date can be added via the arguments `--interval-start` and optionally `--interval-end` to define the date range (see examples below). If no start date is specified, the date will default today's date (and thus return the latest exchange rates).
-  - If a start date but no end date is specified, then the end date will default to today's date and ingestr will retrieve data up until the latest published data.
-  - Note that the [Frankfurter API](https://www.frankfurter.dev/) only publishes updates Monday-Friday. If the given date is on the weekend, the date will default to the previous Friday. The source is however implemented in ingestr in such a way as to avoid duplicating rows of data in this case (see [Incremental Loading - Replace](https://bruin-data.github.io/ingestr/getting-started/incremental-loading.html)).
-
-#### **Example Table: Handling Weekend Dates**
-Here `--interval-start` is set to a weekend date (e.g., `2025-03-29` -- a Saturday). `--interval-end` is set to a the following Monday (`2025-03-31`). 
-
-`--interval-start` defaults to the previous Friday (`2025-03-28`) and the next data is from the following Monday (for simplicity, only a subset of currencies is shown below):
-
-| **date**     | **currency_code** | **rate** | **base_currency** |
-|--------------|-------------------|----------|-------------------|
-| 2025-03-28   | EUR               | 1.0      | EUR               |
-| 2025-03-28   | USD               | 1.0783   | EUR               |
-| 2025-03-28   | GBP               | 0.8571   | EUR               |
-| 2025-03-31   | EUR               | 1.0      | EUR               |
-| 2025-03-31   | USD               | 1.0783   | EUR               |
-| 2025-03-31   | GBP               | 0.8571   | EUR               |
-
-
----
-
-## **Examples**
-
-### **1. Fetch the Latest Exchange Rates with GBP as Base Currency**
-```bash
-ingestr ingest \
---source-uri 'frankfurter://?base=GBP' \
---source-table 'latest' \
---dest-uri 'duckdb.db' \
---dest-table 'schema.latest_new_scheme'
-```
-
----
-
-### **2. Fetch Historical Exchange Rates with USD as Default Base Currency**
-```bash
-ingestr ingest \
---source-uri 'frankfurter://' \
---interval-start '2025-03-01' \
---interval-end '2025-03-10' \
---source-table 'exchange_rates' \
---dest-uri 'duckdb.db' \
---dest-table 'schema.exchange_rates'
-```
-
----
-
-### **3. Fetch Currency Metadata**
-```bash
-ingestr ingest \
---source-uri 'frankfurter://' \
---source-table 'currencies' \
---dest-uri 'duckdb.db' \
---dest-table 'schema.currencies'
-```
-
----
-
-## **Notes**
-- Ensure that the destination database (`--dest-uri`) is accessible and writable.
-- The `--interval-start` and `--interval-end` parameters are only applicable for the `exchange_rates` table.
-- The `latest` table always fetches the most recent exchange rates and ignores date parameters.