sfeos-helpers 6.5.1__py3-none-any.whl → 6.7.0__py3-none-any.whl

sfeos_helpers-6.5.1.dist-info/METADATA

@@ -1,714 +0,0 @@
-Metadata-Version: 2.1
-Name: sfeos-helpers
-Version: 6.5.1
-Summary: Helper library for the Elasticsearch and Opensearch stac-fastapi backends.
-Home-page: https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch
-License: MIT
-Classifier: Intended Audience :: Developers
-Classifier: Intended Audience :: Information Technology
-Classifier: Intended Audience :: Science/Research
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Programming Language :: Python :: 3.13
-Classifier: License :: OSI Approved :: MIT License
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-Requires-Dist: stac-fastapi.core==6.5.1
-
-# stac-fastapi-elasticsearch-opensearch
-
-<!-- markdownlint-disable MD033 MD041 -->
-
-
-<p align="left">
-  <img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/sfeos.png" width=1000>
-</p>
-
-**Jump to:** [Project Introduction](#project-introduction---what-is-sfeos) | [Quick Start](#quick-start) | [Table of Contents](#table-of-contents)
-
-  [![Downloads](https://static.pepy.tech/badge/stac-fastapi-core?color=blue)](https://pepy.tech/project/stac-fastapi-core)
-  [![GitHub contributors](https://img.shields.io/github/contributors/stac-utils/stac-fastapi-elasticsearch-opensearch?color=blue)](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/graphs/contributors)
-  [![GitHub stars](https://img.shields.io/github/stars/stac-utils/stac-fastapi-elasticsearch-opensearch.svg?color=blue)](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/stargazers)
-  [![GitHub forks](https://img.shields.io/github/forks/stac-utils/stac-fastapi-elasticsearch-opensearch.svg?color=blue)](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/network/members)
-   [![PyPI version](https://img.shields.io/pypi/v/stac-fastapi-elasticsearch.svg?color=blue)](https://pypi.org/project/stac-fastapi-elasticsearch/)
-  [![STAC](https://img.shields.io/badge/STAC-1.1.0-blue.svg)](https://github.com/radiantearth/stac-spec/tree/v1.1.0)
-  [![stac-fastapi](https://img.shields.io/badge/stac--fastapi-6.0.0-blue.svg)](https://github.com/stac-utils/stac-fastapi)
-
-## Sponsors & Supporters
-
-The following organizations have contributed time and/or funding to support the development of this project:
-
-<p align="left">
-  <a href="https://healy-hyperspatial.github.io/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/hh-logo-blue.png" alt="Healy Hyperspatial" height="100" hspace="20"></a>
-  <a href="https://atomicmaps.io/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/am-logo-black.png" alt="Atomic Maps" height="100" hspace="20"></a>
-  <a href="https://remotesensing.vito.be/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/VITO.png" alt="VITO Remote Sensing" height="100" hspace="20"></a>
-</p>
-
-## Project Introduction - What is SFEOS?
-
-SFEOS (stac-fastapi-elasticsearch-opensearch) is a high-performance, scalable API implementation for serving SpatioTemporal Asset Catalog (STAC) data - an enhanced GeoJSON format designed specifically for geospatial assets like satellite imagery, aerial photography, and other Earth observation data. This project enables organizations to:
-
-- **Efficiently catalog and search geospatial data** such as satellite imagery, aerial photography, DEMs, and other geospatial assets using Elasticsearch or OpenSearch as the database backend
-- **Implement standardized STAC APIs** that support complex spatial, temporal, and property-based queries across large collections of geospatial data
-- **Scale to millions of geospatial assets** with fast search performance through optimized spatial indexing and query capabilities
-- **Support OGC-compliant filtering** including spatial operations (intersects, contains, etc.) and temporal queries
-- **Perform geospatial aggregations** to analyze data distribution across space and time
-- **Enhanced collection search capabilities** with support for sorting and field selection
-
-This implementation builds on the STAC-FastAPI framework, providing a production-ready solution specifically optimized for Elasticsearch and OpenSearch databases. It's ideal for organizations managing large geospatial data catalogs who need efficient discovery and access capabilities through standardized APIs.
-
-## Common Deployment Patterns
-
-stac-fastapi-elasticsearch-opensearch can be deployed in several ways depending on your needs:
-
-- **Containerized Application**: Run as a Docker container with connections to Elasticsearch/OpenSearch databases
-- **Serverless Function**: Deploy as AWS Lambda or similar serverless function with API Gateway
-- **Traditional Server**: Run on virtual machines or bare metal servers in your infrastructure
-- **Kubernetes**: Deploy as part of a larger microservices architecture with container orchestration
-
-The implementation is flexible and can scale from small local deployments to large production environments serving millions of geospatial assets.
-
-## Technologies
-
-This project is built on the following technologies: STAC, stac-fastapi, FastAPI, Elasticsearch, Python, OpenSearch
-
-<p align="left">
-  <a href="https://stacspec.org/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/STAC-01.png" alt="STAC" height="100" hspace="10"></a>
-  <a href="https://www.python.org/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/python.png" alt="Python" height="80" hspace="10"></a>
-  <a href="https://fastapi.tiangolo.com/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/fastapi.svg" alt="FastAPI" height="80" hspace="10"></a>
-  <a href="https://www.elastic.co/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/elasticsearch.png" alt="Elasticsearch" height="80" hspace="10"></a>
-  <a href="https://opensearch.org/"><img src="https://raw.githubusercontent.com/stac-utils/stac-fastapi-elasticsearch-opensearch/refs/heads/main/assets/opensearch.svg" alt="OpenSearch" height="80" hspace="10"></a>
-</p>
-
-## Table of Contents
-
-- [stac-fastapi-elasticsearch-opensearch](#stac-fastapi-elasticsearch-opensearch)
-  - [Sponsors & Supporters](#sponsors--supporters)
-  - [Project Introduction - What is SFEOS?](#project-introduction---what-is-sfeos)
-  - [Common Deployment Patterns](#common-deployment-patterns)
-  - [Technologies](#technologies)
-  - [Table of Contents](#table-of-contents)
-  - [Collection Search Extensions](#collection-search-extensions)
-  - [Documentation & Resources](#documentation--resources)
-  - [Package Structure](#package-structure)
-  - [Examples](#examples)
-  - [Performance](#performance)
-    - [Direct Response Mode](#direct-response-mode)
-  - [Quick Start](#quick-start)
-    - [Installation](#installation)
-    - [Running Locally](#running-locally)
-      - [Using Pre-built Docker Images](#using-pre-built-docker-images)
-      - [Using Docker Compose](#using-docker-compose)
-  - [Configuration Reference](#configuration-reference)
-  - [Datetime-Based Index Management](#datetime-based-index-management)
-    - [Overview](#overview)
-    - [When to Use](#when-to-use)
-    - [Configuration](#configuration)
-      - [Enabling Datetime-Based Indexing](#enabling-datetime-based-indexing)
-    - [Related Configuration Variables](#related-configuration-variables)
-  - [How Datetime-Based Indexing Works](#how-datetime-based-indexing-works)
-    - [Index and Alias Naming Convention](#index-and-alias-naming-convention)
-    - [Index Size Management](#index-size-management)
-  - [Interacting with the API](#interacting-with-the-api)
-  - [Configure the API](#configure-the-api)
-  - [Collection Pagination](#collection-pagination)
-  - [Ingesting Sample Data CLI Tool](#ingesting-sample-data-cli-tool)
-  - [Elasticsearch Mappings](#elasticsearch-mappings)
-  - [Managing Elasticsearch Indices](#managing-elasticsearch-indices)
-    - [Snapshots](#snapshots)
-    - [Reindexing](#reindexing)
-  - [Auth](#auth)
-  - [Aggregation](#aggregation)
-  - [Rate Limiting](#rate-limiting)
-
-## Documentation & Resources
-
-- **Online Documentation**: [https://stac-utils.github.io/stac-fastapi-elasticsearch-opensearch](https://stac-utils.github.io/stac-fastapi-elasticsearch-opensearch/)
-- **Source Code**: [https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch)
-- **API Examples**: [Postman Documentation](https://documenter.getpostman.com/view/12888943/2s8ZDSdRHA) - Examples of how to use the API endpoints
-- **Community**:
-  - [Gitter Chat](https://app.gitter.im/#/room/#stac-fastapi-elasticsearch_community:gitter.im) - For real-time discussions
-  - [GitHub Discussions](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/discussions) - For longer-form questions and answers
-
-## Collection Search Extensions
-
-SFEOS provides enhanced collection search capabilities through two primary routes:
-- **GET/POST `/collections`**: The standard STAC endpoint with extended query parameters
-- **GET/POST `/collections-search`**: A custom endpoint that supports the same parameters, created to avoid conflicts with the STAC Transactions extension if enabled (which uses POST `/collections` for collection creation)
-
-These endpoints support advanced collection discovery features including:
-
-- **Sorting**: Sort collections by sortable fields using the `sortby` parameter
-  - Example: `/collections?sortby=+id` (ascending sort by ID)
-  - Example: `/collections?sortby=-id` (descending sort by ID)
-  - Example: `/collections?sortby=-temporal` (descending sort by temporal extent)
-
-- **Field Selection**: Request only specific fields to be returned using the `fields` parameter
-  - Example: `/collections?fields=id,title,description`
-  - This helps reduce payload size when only certain fields are needed
-
-- **Free Text Search**: Search across collection text fields using the `q` parameter
-  - Example: `/collections?q=landsat`
-  - Searches across multiple text fields including title, description, and keywords
-  - Supports partial word matching and relevance-based sorting
-
-- **Structured Filtering**: Filter collections using CQL2 expressions
-  - JSON format: `/collections?filter={"op":"=","args":[{"property":"id"},"sentinel-2"]}&filter-lang=cql2-json`
-  - Text format: `/collections?filter=id='sentinel-2'&filter-lang=cql2-text` (note: string values must be quoted)
-  - Advanced text format: `/collections?filter=id LIKE '%sentinel%'&filter-lang=cql2-text` (supports LIKE, BETWEEN, etc.)
-  - Supports both CQL2 JSON and CQL2 text formats with various operators
-  - Enables precise filtering on any collection property
-
-- **Datetime Filtering**: Filter collections by their temporal extent using the `datetime` parameter
-  - Example: `/collections?datetime=2020-01-01T00:00:00Z/2020-12-31T23:59:59Z` (finds collections with temporal extents that overlap this range)
-  - Example: `/collections?datetime=2020-06-15T12:00:00Z` (finds collections whose temporal extent includes this specific time)
-  - Example: `/collections?datetime=2020-01-01T00:00:00Z/..` (finds collections with temporal extents that extend to or beyond January 1, 2020)
-  - Example: `/collections?datetime=../2020-12-31T23:59:59Z` (finds collections with temporal extents that begin on or before December 31, 2020)
-  - Collections are matched if their temporal extent overlaps with the provided datetime parameter
-  - This allows for efficient discovery of collections based on time periods
-
-These extensions make it easier to build user interfaces that display and navigate through collections efficiently.
-
-> **Configuration**: Collection search extensions (sorting, field selection, free text search, structured filtering, and datetime filtering) for the `/collections` endpoint can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.
-> 
-> **Configuration**: The custom `/collections-search` endpoint can be enabled by setting the `ENABLE_COLLECTIONS_SEARCH_ROUTE` environment variable to `true`. By default, this endpoint is **disabled**.
-
-> **Note**: Sorting is only available on fields that are indexed for sorting in Elasticsearch/OpenSearch. With the default mappings, you can sort on:
-> - `id` (keyword field)
-> - `extent.temporal.interval` (date field)
-> - `temporal` (alias to extent.temporal.interval)
->
-> Text fields like `title` and `description` are not sortable by default as they use text analysis for better search capabilities. Attempting to sort on these fields will result in a user-friendly error message explaining which fields are sortable and how to make additional fields sortable by updating the mappings.
->
-> **Important**: Adding keyword fields to make text fields sortable can significantly increase the index size, especially for large text fields. Consider the storage implications when deciding which fields to make sortable.
-
-
-## Package Structure
-
-This project is organized into several packages, each with a specific purpose:
-
-- **stac_fastapi_core**: Core functionality that's database-agnostic, including API models, extensions, and shared utilities. This package provides the foundation for building STAC API implementations with any database backend. See [stac-fastapi-mongo](https://github.com/Healy-Hyperspatial/stac-fastapi-mongo) for a working example.
-
-- **sfeos_helpers**: Shared helper functions and utilities used by both the Elasticsearch and OpenSearch backends. This package includes:
-  - `database`: Specialized modules for index, document, and database utility operations
-  - `aggregation`: Elasticsearch/OpenSearch-specific aggregation functionality
-  - Shared logic and utilities that improve code reuse between backends
-
-- **stac_fastapi_elasticsearch**: Complete implementation of the STAC API using Elasticsearch as the backend database. This package depends on both `stac_fastapi_core` and `sfeos_helpers`.
-
-- **stac_fastapi_opensearch**: Complete implementation of the STAC API using OpenSearch as the backend database. This package depends on both `stac_fastapi_core` and `sfeos_helpers`.
-
-## Examples
-
-The `/examples` directory contains several useful examples and reference implementations:
-
-- **pip_docker**: Examples of running stac-fastapi-elasticsearch from PyPI in Docker without needing any code from the repository
-- **auth**: Authentication examples including:
-  - Basic authentication
-  - OAuth2 with Keycloak
-  - Route dependencies configuration
-- **rate_limit**: Example of implementing rate limiting for API requests
-- **postman_collections**: Postman collection files you can import for testing API endpoints
-
-These examples provide practical reference implementations for various deployment scenarios and features.
-
-## Performance
-
-### Direct Response Mode
-
-- The `enable_direct_response` option is provided by the stac-fastapi core library (introduced in stac-fastapi 5.2.0) and is available in this project starting from v4.0.0.
-- **Control via environment variable**: Set `ENABLE_DIRECT_RESPONSE=true` to enable this feature.
-- **How it works**: When enabled, endpoints return Starlette Response objects directly, bypassing FastAPI's default serialization for improved performance.
-- **Important limitation**: All FastAPI dependencies (including authentication, custom status codes, and validation) are disabled for all routes when this mode is enabled.
-- **Best use case**: This mode is best suited for public or read-only APIs where authentication and custom logic are not required.
-- **Default setting**: `false` for safety.
-- **More information**: See [issue #347](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/issues/347) for background and implementation details.
-
-## Quick Start
-
-This section helps you get up and running with stac-fastapi-elasticsearch-opensearch quickly.
-
-### Installation
-
-- **For versions 4.0.0a1 and newer** (PEP 625 compliant naming):
-  ```bash
-  pip install stac-fastapi-elasticsearch  # Elasticsearch backend
-  pip install stac-fastapi-opensearch    # Opensearch backend
-  pip install stac-fastapi-core          # Core library
-  ```
-
-- **For versions 4.0.0a0 and older**:
-  ```bash
-  pip install stac-fastapi.elasticsearch  # Elasticsearch backend
-  pip install stac-fastapi.opensearch    # Opensearch backend
-  pip install stac-fastapi.core          # Core library
-  ```
-
-> **Important Note:** Starting with version 4.0.0a1, package names have changed from using periods (e.g., `stac-fastapi.core`) to using hyphens (e.g., `stac-fastapi-core`) to comply with PEP 625. The internal package structure uses underscores, but users should install with hyphens as shown above. Please update your requirements files accordingly.
-
-### Running Locally
-
-There are two main ways to run the API locally:
-
-#### Using Pre-built Docker Images
-
-- We provide ready-to-use Docker images through GitHub Container Registry:
-  - [ElasticSearch backend](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pkgs/container/stac-fastapi-es)
-  - [OpenSearch backend](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pkgs/container/stac-fastapi-os)
-
-- **Pull and run the images**:
-  ```shell
-  # For Elasticsearch backend
-  docker pull ghcr.io/stac-utils/stac-fastapi-es:latest
-
-  # For OpenSearch backend
-  docker pull ghcr.io/stac-utils/stac-fastapi-os:latest
-  ```
-
-#### Using Docker Compose
-
-- **Prerequisites**: Ensure [Docker Compose](https://docs.docker.com/compose/install/) or [Podman Compose](https://podman-desktop.io/docs/compose) is installed on your machine.
-
-- **Start the API**:
-  ```shell
-  docker compose up elasticsearch app-elasticsearch
-  ```
-
-- **Configuration**: By default, Docker Compose uses Elasticsearch 8.x and OpenSearch 2.11.1. To use different versions, create a `.env` file:
-  ```shell
-  ELASTICSEARCH_VERSION=8.11.0
-  OPENSEARCH_VERSION=2.11.1
-  ENABLE_DIRECT_RESPONSE=false
-  ```
-
-- **Compatibility**: The most recent Elasticsearch 7.x versions should also work. See the [opensearch-py docs](https://github.com/opensearch-project/opensearch-py/blob/main/COMPATIBILITY.md) for compatibility information.
-
-
-
-## Configuration Reference
-
-You can customize additional settings in your `.env` file:
-
-| Variable                     | Description                                                                          | Default                  | Required                                                                                     |
-|------------------------------|--------------------------------------------------------------------------------------|--------------------------|---------------------------------------------------------------------------------------------|
-| `ES_HOST`                    | Hostname for external Elasticsearch/OpenSearch.                                      | `localhost`              | Optional                                                                                    |
-| `ES_PORT`                    | Port for Elasticsearch/OpenSearch.                                                   | `9200` (ES) / `9202` (OS)| Optional                                                                                    |
-| `ES_USE_SSL`                 | Use SSL for connecting to Elasticsearch/OpenSearch.                                  | `true`                   | Optional                                                                                    |
-| `ES_VERIFY_CERTS`            | Verify SSL certificates when connecting.                                             | `true`                   | Optional                                                                                    |
-| `ES_API_KEY`                 | API Key for external Elasticsearch/OpenSearch.                                       | N/A                      | Optional                                                                                    |
-| `ES_TIMEOUT`                 | Client timeout for Elasticsearch/OpenSearch.                                         | DB client default        | Optional                                                                                    |
-| `STAC_FASTAPI_TITLE`         | Title of the API in the documentation.                                               | `stac-fastapi-<backend>` | Optional                                                                                    |
-| `STAC_FASTAPI_DESCRIPTION`   | Description of the API in the documentation.                                         | N/A                      | Optional                                                                                    |
-| `STAC_FASTAPI_VERSION`       | API version.                                                                         | `2.1`                    | Optional                                                                                    |
-| `STAC_FASTAPI_LANDING_PAGE_ID` | Landing page ID                                                                    | `stac-fastapi`           | Optional                                                                                    |
-| `APP_HOST`                   | Server bind address.                                                                 | `0.0.0.0`                | Optional                                                                                    |
-| `APP_PORT`                   | Server port.                                                                         | `8000`                   | Optional                                                                                    |
-| `ENVIRONMENT`                | Runtime environment.                                                                 | `local`                  | Optional                                                                                    |
-| `WEB_CONCURRENCY`            | Number of worker processes.                                                          | `10`                     | Optional                                                                                    |
-| `RELOAD`                     | Enable auto-reload for development.                                                  | `true`                   | Optional                                                                                    |
-| `STAC_FASTAPI_RATE_LIMIT`    | API rate limit per client.                                                           | `200/minute`             | Optional                                                                                    |
-| `BACKEND`                    | Tests-related variable                                                               | `elasticsearch` or `opensearch` based on the backend | Optional                                                        |
-| `ELASTICSEARCH_VERSION`      | Version of Elasticsearch to use.                                                     | `8.11.0`                 | Optional                                                                                    |
-| `OPENSEARCH_VERSION`         | OpenSearch version                                                                   | `2.11.1`                 | Optional                                                                                    |
-| `ENABLE_DIRECT_RESPONSE`     | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false`                  | Optional                       |
-| `RAISE_ON_BULK_ERROR`        | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` | Optional |
-| `DATABASE_REFRESH`           | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
-| `ENABLE_COLLECTIONS_SEARCH`  | Enable collection search extensions (sort, fields, free text search, structured filtering, and datetime filtering) on the core `/collections` endpoint. | `true`                   | Optional                                                                                    |
-| `ENABLE_COLLECTIONS_SEARCH_ROUTE` | Enable the custom `/collections-search` endpoint (both GET and POST methods). When disabled, the custom endpoint will not be available, but collection search extensions will still be available on the core `/collections` endpoint if `ENABLE_COLLECTIONS_SEARCH` is true. | `false` | Optional |
-| `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. This is useful for deployments where mutating the catalog via the API should be prevented. If set to `true`, the POST `/collections` route for search will be unavailable in the API. | `true` | Optional |
-| `STAC_ITEM_LIMIT` | Sets the environment variable for result limiting to SFEOS for the number of returned items and STAC collections. | `10` | Optional |
-| `STAC_INDEX_ASSETS` | Controls if Assets are indexed when added to Elasticsearch/Opensearch. This allows asset fields to be included in search queries. | `false` | Optional |
-| `ENV_MAX_LIMIT` | Configures the environment variable in SFEOS to override the default `MAX_LIMIT`, which controls the limit parameter for returned items and STAC collections. | `10,000` | Optional |
-| `USE_DATETIME` | Configures the datetime search behavior in SFEOS. When enabled, searches both datetime field and falls back to start_datetime/end_datetime range for items with null datetime. When disabled, searches only by start_datetime/end_datetime range. | `true` | Optional |
-
-> [!NOTE]
-> The variables `ES_HOST`, `ES_PORT`, `ES_USE_SSL`, `ES_VERIFY_CERTS` and `ES_TIMEOUT` apply to both Elasticsearch and OpenSearch backends, so there is no need to rename the key names to `OS_` even if you're using OpenSearch.
-
-## Datetime-Based Index Management
-
-### Overview
-
-SFEOS supports two indexing strategies for managing STAC items:
-
-1. **Simple Indexing** (default) - One index per collection
-2. **Datetime-Based Indexing** - Time-partitioned indexes with automatic management
-
-The datetime-based indexing strategy is particularly useful for large temporal datasets. When a user provides a datetime parameter in a query, the system knows exactly which index to search, providing **multiple times faster searches** and significantly **reducing database load**.
-
-### When to Use
-
-**Recommended for:**
-- Systems with large collections containing millions of items
-- Systems requiring high-performance temporal searching
-
-**Pros:**
-- Multiple times faster queries with datetime filter
-- Reduced database load - only relevant indexes are searched
-
-**Cons:**
-- Slightly longer item indexing time (automatic index management)
-- Greater management complexity
-
-### Configuration
-
-#### Enabling Datetime-Based Indexing
-
-Enable datetime-based indexing by setting the following environment variable:
-
-```bash
-ENABLE_DATETIME_INDEX_FILTERING=true
-```
-
-### Related Configuration Variables
-
-| Variable | Description | Default | Example |
-|----------|-------------|---------|---------|
-| `ENABLE_DATETIME_INDEX_FILTERING` | Enables time-based index partitioning | `false` | `true` |
-| `DATETIME_INDEX_MAX_SIZE_GB` | Maximum size limit for datetime indexes (GB) - note: add +20% to target size due to ES/OS compression | `25` | `50` |
-| `STAC_ITEMS_INDEX_PREFIX` | Prefix for item indexes | `items_` | `stac_items_` |
-
-## How Datetime-Based Indexing Works
-
-### Index and Alias Naming Convention
-
-The system uses a precise naming convention:
-
-**Physical indexes:**
-```
-{ITEMS_INDEX_PREFIX}{collection-id}_{uuid4}
-```
-
-**Aliases:**
-```
-{ITEMS_INDEX_PREFIX}{collection-id}                                  # Main collection alias
-{ITEMS_INDEX_PREFIX}{collection-id}_{start-datetime}                 # Temporal alias
-{ITEMS_INDEX_PREFIX}{collection-id}_{start-datetime}_{end-datetime}  # Closed index alias
-```
-
-**Example:**
-
-*Physical indexes:*
-- `items_sentinel-2-l2a_a1b2c3d4-e5f6-7890-abcd-ef1234567890`
-
-*Aliases:*
-- `items_sentinel-2-l2a` - main collection alias
-- `items_sentinel-2-l2a_2024-01-01` - active alias from January 1, 2024
-- `items_sentinel-2-l2a_2024-01-01_2024-03-15` - closed index alias (reached size limit)
-
-### Index Size Management
-
-**Important - Data Compression:** Elasticsearch and OpenSearch automatically compress data. The configured `DATETIME_INDEX_MAX_SIZE_GB` limit refers to the compressed size on disk. It is recommended to add +20% to the target size to account for compression overhead and metadata.
-
-## Interacting with the API
-
-- **Creating a Collection**:
-  ```shell
-  curl -X "POST" "http://localhost:8080/collections" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "id": "my_collection"
-  }'
-  ```
-
-- **Adding an Item to a Collection**:
-  ```shell
-  curl -X "POST" "http://localhost:8080/collections/my_collection/items" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d @item.json
-  ```
-
-- **Searching for Items**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/search" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "collections": ["my_collection"],
-    "limit": 10
-  }'
-  ```
-
-- **Filtering by Bbox**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/search" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "collections": ["my_collection"],
-    "bbox": [-180, -90, 180, 90]
-  }'
-  ```
-
-- **Filtering by Datetime**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/search" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "collections": ["my_collection"],
-    "datetime": "2020-01-01T00:00:00Z/2020-12-31T23:59:59Z"
-  }'
-  ```
-
-## Configure the API
-
-- **API Title and Description**: By default set to `stac-fastapi-<backend>`. Customize these by setting:
-  - `STAC_FASTAPI_TITLE`: Changes the API title in the documentation
-  - `STAC_FASTAPI_DESCRIPTION`: Changes the API description in the documentation
-
-- **Database Indices**: By default, the API reads from and writes to:
-  - `collections` index for collections
-  - `items_<collection name>` indices for items
-  - Customize with `STAC_COLLECTIONS_INDEX` and `STAC_ITEMS_INDEX_PREFIX` environment variables
-
-- **Root Path Configuration**: The application root path is the base URL by default.
-  - For AWS Lambda with Gateway API: Set `STAC_FASTAPI_ROOT_PATH` to match the Gateway API stage name (e.g., `/v1`)
-
-- **Feature Configuration**: Control which features are enabled:
-  - `ENABLE_COLLECTIONS_SEARCH`: Set to `true` (default) to enable collection search extensions (sort, fields). Set to `false` to disable.
-  - `ENABLE_TRANSACTIONS_EXTENSIONS`: Set to `true` (default) to enable transaction extensions. Set to `false` to disable.
-
-## Collection Pagination
-
-- **Overview**: The collections route supports pagination through optional query parameters.
-- **Parameters**:
-  - `limit`: Controls the number of collections returned per page
-  - `token`: Used to retrieve subsequent pages of results
-- **Response Structure**: The `links` field in the response contains a `next` link with the token for the next page of results.
-- **Example Usage**:
-  ```shell
-  curl -X "GET" "http://localhost:8080/collections?limit=1&token=example_token"
-  ```
-
-## Ingesting Sample Data CLI Tool
-
-- **Overview**: The `data_loader.py` script provides a convenient way to load STAC items into the database.
-
-- **Usage**:
-  ```shell
-  python3 data_loader.py --base-url http://localhost:8080
-  ```
-
-- **Options**:
-  ```
-  --base-url TEXT       Base URL of the STAC API  [required]
-  --collection-id TEXT  ID of the collection to which items are added
-  --use-bulk            Use bulk insert method for items
-  --data-dir PATH       Directory containing collection.json and feature
-                        collection file
-  --help                Show this message and exit.
-  ```
-
-- **Example Workflows**:
-  - **Loading Sample Data**: 
-    ```shell
-    python3 data_loader.py --base-url http://localhost:8080
-    ```
-  - **Loading Data to a Specific Collection**:
-    ```shell
-    python3 data_loader.py --base-url http://localhost:8080 --collection-id my-collection
-    ```
-  - **Using Bulk Insert for Performance**:
-    ```shell
-    python3 data_loader.py --base-url http://localhost:8080 --use-bulk
-    ```
-
-## Elasticsearch Mappings
-
-- **Overview**: Mappings apply to search index, not source data. They define how documents and their fields are stored and indexed.
-- **Implementation**: 
-  - Mappings are stored in index templates that are created on application startup
-  - These templates are automatically applied when creating new Collection and Item indices
-  - The `sfeos_helpers` package contains shared mapping definitions used by both Elasticsearch and OpenSearch backends
-- **Customization**: Custom mappings can be defined by extending the base mapping templates.
-
-## Managing Elasticsearch Indices
-
-### Snapshots
-
-- **Overview**: Snapshots provide a way to backup and restore your indices.
-
-- **Creating a Snapshot Repository**:
-  ```shell
-  curl -X "PUT" "http://localhost:9200/_snapshot/my_fs_backup" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-               "type": "fs",
-               "settings": {
-                   "location": "/usr/share/elasticsearch/snapshots/my_fs_backup"
-               }
-  }'
-  ```
-  - This creates a snapshot repository that stores files in the elasticsearch/snapshots directory in this git repo clone
-  - The elasticsearch.yml and compose files create a mapping from that directory to /usr/share/elasticsearch/snapshots within the Elasticsearch container and grant permissions for using it
-
-- **Creating a Snapshot**:
-  ```shell
-  curl -X "PUT" "http://localhost:9200/_snapshot/my_fs_backup/my_snapshot_2?wait_for_completion=true" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "metadata": {
-      "taken_because": "dump of all items",
-      "taken_by": "pvarner"
-    },
-    "include_global_state": false,
-    "ignore_unavailable": false,
-    "indices": "items_my-collection"
-  }'
-  ```
-  - This creates a snapshot named my_snapshot_2 and waits for the action to be completed before returning
-  - This can also be done asynchronously by omitting the wait_for_completion parameter, and queried for status later
-  - The indices parameter determines which indices are snapshotted, and can include wildcards
-
-- **Viewing Snapshots**:
-  ```shell
-  # View a specific snapshot
-  curl http://localhost:9200/_snapshot/my_fs_backup/my_snapshot_2
-  
-  # View all snapshots
-  curl http://localhost:9200/_snapshot/my_fs_backup/_all
-  ```
-  - These commands allow you to check the status and details of your snapshots
-
-- **Restoring a Snapshot**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/_snapshot/my_fs_backup/my_snapshot_2/_restore?wait_for_completion=true" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-    "include_aliases": false,
-    "include_global_state": false,
-    "ignore_unavailable": true,
-    "rename_replacement": "items_$1-copy",
-    "indices": "items_*",
-    "rename_pattern": "items_(.+)"
-  }'
-  ```
-  - This specific command will restore any indices that match items_* and rename them so that the new index name will be suffixed with -copy
-  - The rename_pattern and rename_replacement parameters allow you to restore indices under new names
-
-- **Updating Collection References**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/items_my-collection-copy/_update_by_query" \
-       -H 'Content-Type: application/json; charset=utf-8' \
-       -d $'{
-      "query": {
-          "match_all": {}
-  },
-    "script": {
-      "lang": "painless",
-      "params": {
-        "collection": "my-collection-copy"
-      },
-      "source": "ctx._source.collection = params.collection"
-    }
-  }'
-  ```
-  - After restoring, the item documents have been restored in the new index (e.g., my-collection-copy), but the value of the collection field in those documents is still the original value of my-collection
-  - This command updates these values to match the new collection name using Elasticsearch's Update By Query feature
-
-- **Creating a New Collection**:
-  ```shell
-  curl -X "POST" "http://localhost:8080/collections" \
-       -H 'Content-Type: application/json' \
-       -d $'{
-    "id": "my-collection-copy"
-  }'
-  ```
-  - The final step is to create a new collection through the API with the new name for each of the restored indices
-  - This gives you a copy of the collection that has a resource URI (/collections/my-collection-copy) and can be correctly queried by collection name
-
-### Reindexing
-
-- **Overview**: Reindexing allows you to copy documents from one index to another, optionally transforming them in the process.
-
-- **Use Cases**:
-  - Apply changes to documents
-  - Correct dynamically generated mappings
-  - Transform data (e.g., lowercase identifiers)
-  - The index templates will make sure that manually created indices will also have the correct mappings and settings
-
-- **Example: Reindexing with Transformation**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/_reindex" \
-    -H 'Content-Type: application/json' \
-    -d $'{
-      "source": {
-        "index": "items_my-collection-lower_my-collection-hex-000001"
-      }, 
-      "dest": {
-        "index": "items_my-collection-lower_my-collection-hex-000002"
-      },
-      "script": {
-        "source": "ctx._source.id = ctx._source.id.toLowerCase()",
-        "lang": "painless"
-      }
-    }'
-  ```
-  - In this example, we make a copy of an existing Item index but change the Item identifier to be lowercase
-  - The script parameter allows you to transform documents during the reindexing process
-
-- **Updating Aliases**:
-  ```shell
-  curl -X "POST" "http://localhost:9200/_aliases" \
-    -H 'Content-Type: application/json' \
-    -d $'{
-      "actions": [
-        {
-          "remove": {
-            "index": "*",
-            "alias": "items_my-collection"
-          }
-        },
-        {
-          "add": {
-            "index": "items_my-collection-lower_my-collection-hex-000002",
-            "alias": "items_my-collection"
-          }
-        }
-      ]
-    }'
-  ```
-  - If you are happy with the data in the newly created index, you can move the alias items_my-collection to the new index
-  - This makes the modified Items with lowercase identifiers visible to users accessing my-collection in the STAC API
-  - Using aliases allows you to switch between different index versions without changing the API endpoint
-
-## Auth
-
-- **Overview**: Authentication is an optional feature that can be enabled through Route Dependencies.
-- **Implementation Options**:
-  - Basic authentication
-  - OAuth2 with Keycloak
-  - Custom route dependencies
-- **Configuration**: Authentication can be configured using the `STAC_FASTAPI_ROUTE_DEPENDENCIES` environment variable.
-- **Examples and Documentation**: Detailed examples and implementation guides can be found in the [examples/auth](examples/auth) directory.
-
-## Aggregation
-
-- **Supported Aggregations**:
-  - Spatial aggregations of points and geometries
-  - Frequency distribution aggregation of any property including dates
-  - Temporal distribution of datetime values
-
-- **Endpoint Locations**:
-  - Root Catalog level: `/aggregations`
-  - Collection level: `/<collection_id>/aggregations`
-
-- **Implementation Details**: The `sfeos_helpers.aggregation` package provides specialized functionality for both Elasticsearch and OpenSearch backends.
-
-- **Documentation**: Detailed information about supported aggregations can be found in [the aggregation docs](./docs/src/aggregation.md).
-
-
-## Rate Limiting
-
-- **Overview**: Rate limiting is an optional security feature that controls API request frequency on a remote address basis.
-
-- **Configuration**: Enabled by setting the `STAC_FASTAPI_RATE_LIMIT` environment variable:
-  ```
-  STAC_FASTAPI_RATE_LIMIT=500/minute
-  ```
-
-- **Functionality**: 
-  - Limits each client to a specified number of requests per time period (e.g., 500 requests per minute)
-  - Helps prevent API abuse and maintains system stability
-  - Ensures fair resource allocation among all clients
-  
-- **Examples**: Implementation examples are available in the [examples/rate_limit](examples/rate_limit) directory.