api-dock 0.4.6__tar.gz → 0.4.8__tar.gz

{api_dock-0.4.6 → api_dock-0.4.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: api_dock
-Version: 0.4.6
+Version: 0.4.8
 Summary: A flexible API gateway that allows you to proxy requests to multiple remote APIs and Databases
 Author-email: Brookie Guzder-Williams <bguzder-williams@berkeley.edu>
 License: BSd 3-clause
@@ -39,21 +39,21 @@ Dynamic: license-file
 
 # API Dock
 
-API Dock is a flexible API gateway that allows you to proxy requests to multiple remote APIs and Databases through a single endpoint. Using API doc's CLI, the proxy can easily be launched as a FastAPI or Flask app, or integrated into any existing python based API.
+API Dock allows users to quickly build API-s that proxy requests to multiple remote APIs and Databases through a single endpoint. Configuration is handled with simple YAML files. Using API Dock's CLI, an API can easily be launched as a FastAPI or Flask app, or integrated into any existing python based API.
 
 ## Table of Contents
 
-- [Features](#features)
 - [Install](#install)
-- [Quick Example](#quick-example)
-- [CLI](#cli)
-  - [Commands](#commands)
-  - [Examples](#examples)
-- [Configuration and Syntax](#configuration-and-syntax)
-  - [Main Configuration](#main-configuration-api_dock_configconfigyaml)
+- [File Structure](#file-structure)
+- [Simple Example](#simple-example)
+- [Configuration Syntax](#configuration-syntax)
+  - [Main Configuration](#main-configuration)
   - [Remote Configurations](#remote-configurations)
   - [SQL Database Support](#sql-database-support)
   - [URL Query Parameters](#url-query-parameters)
+- [CLI](#cli)
+  - [Commands](#commands)
+  - [Examples](#examples)
 - [Cookies and Authentication](#cookies-and-authentication)
 - [Using RouteMapper in Your Own Projects](#using-routemapper-in-your-own-projects)
   - [Basic Integration](#basic-integration)
@@ -83,12 +83,53 @@ pip install api_dock
  conda install -c conda-forge api_dock
 ```
 
-## Quick Example
+---
+
+## File Structure
+
+The main configuration files are stored in the top level of the CWD's `api_dock_config/` directory. Multiple configurations, with both versioned and unversioned remote-apis and databases are possible.
+
+Here is an example:
+
+```bash
+api_dock_config
+├── config.yaml               # The default main-config file
+├── databases
+│    ├── unversioned_db.yaml  # Database config without versioning
+│    └── versioned_db         # Folder containing database configs for different versions
+│        ├── 0.1.yaml
+│        ├── 0.5.yaml
+│        └── 1.1.yaml
+└── remotes
+    ├── service1.yaml         # Remote-Api config without versioning
+    ├── service2.yaml         # Remote-Api config without versioning
+    └── versioned_service     # Folder containing remote-api configs for different versions
+        ├── 0.1.yaml
+        ├── 0.2.yaml
+        └── 0.3.yaml
+```
+
+ By default api-dock expects there to be one called `config.yaml`, however configs with different names (such as `config_v2`) can be added and launched as shown in the CLI Examples section.
+
+---
+
+## Simple Example
 
 Configuration consists of a global config (`api_dock_config/config.yaml`), as well as a config file for each remote-api or database you'd like to proxy. 
 
 Here is a simple example of a configuration serving a single remote-api and database:
 
+```bash
+api_dock_config
+├── config.yaml
+├── databases
+│    └── db_example
+│        ├── 0.1.yaml
+│        └── 0.5.yaml
+└── remotes
+    └── service1.yaml
+```
+
 ```yaml 
 # api_dock_config/config.yaml
 name: "My API Dock"
@@ -105,22 +146,22 @@ databases:
 ```
 
 ```yaml
-# api_dock_config/remotes/service1/0.5.0.yaml
+# api_dock_config/remotes/service1.yaml
 name: service1
-description: "Service1 [Version 0.5.0]"
-url: https://existing_api_url.com
+description: "API Service1"
+url: https://remote.api.com
 ```
 
 ```yaml
-# api_dock_config/databases/db_example/0.1.0.yaml
+# api_dock_config/databases/db_example/0.5.yaml
 name: db_example
-description: "Example DB Version 0.1"
+description: "Example DB Version 0.5"
 authors:
   - "API Team"
 
 tables:
   users:
-    uri: s3://path/to/users-database/folder/**/*.parquet
+    uri: s3://path/to/users-database/partitioned-db_example/**/*.parquet
     region: us-west-2
 
 routes:
@@ -131,115 +172,55 @@ routes:
     sql: SELECT [[users]].* FROM [[users]] WHERE [[users]].user_id = {{user_id}}
 ```
 
-This will create an "api-dock" with the following endpoints
+Note: the use of wildcards `**/*` for the partioned parquet file. If there was no partitioning you would give the direct uri `s3://path/to/users-database/db_example.parquet`. 
+
+This will create an "api-dock" with the following endpoints. 
 
 ```
-- `/service1/0.5.0/*`: maps directly onto `https://existing_api_url.com/*`
-- `/db_example/0.1.0/users`: queries all users in the "users-database"
-- `/db_example/0.1.0/users/{user_id}`: queries all users in the "users-database" with `user.user_id = user_id`
+- `/service1/*`: maps directly onto `https://remote.api.com/*`
+- `/db_example/0.5/users`: queries all users in the "users-database"
+- `/db_example/0.5/users/{user_id}`: queries all users in the "users-database" with `user.user_id = user_id`
 ```
 
-Note: the filename is being used for versioning. An endpoint with "latest" is also generated that will numerically order versions by name and serve the most recent version. For example,
-`/service1/0.5.0` uses the config in `/service1/0.5.0.yaml` and `/service1/latest` will use the most recent version in the `/service1` folder.
+Note: the filename is being used for versioning. An endpoint with "latest" is also generated that will numerically order versions by name and serve the most recent version. In this example, the route
+`/db_example/0.1` uses the config in `/db_example/0.1.yaml`,  while both `/db_example/0.5` and `/db_example/latest` will use `/db_example/0.5.yaml`, the most recent version in the `/db_example` folder.
 
 These basic configurations can be expanded to include a number of use cases: [restricting routes/methods](#route-restrictions), [custom mapping of remote-api routes](#custom-route-mapping), [accepting query parameters to filter data](#query-parameter-filtering), [limiting and sorting results](#sorting-and-pagination), [authentication](#authentication-setup), and [accessing data stored in cookies](#cookie-access).
 
 ---
 
-# CLI
-
-## Commands
-
-API Dock provides a modern Click-based CLI:
-
-- **pixi run api-dock** (default): List all available configurations
-- **pixi run api-dock init [--force]**: Initialize `api_dock_config/` directory with default configs
-- **pixi run api-dock start [config_name]**: Start API Dock server with optional config name
-- **pixi run api-dock describe [config_name]**: Display formatted configuration with expanded SQL queries
-
-**Note**: All commands shown use `pixi run` for the pixi environment. If not using pixi, drop the `pixi run` prefix (e.g., `api-dock start` instead of `pixi run api-dock start`).
+# Configuration Syntax
 
+## Main Configuration
 
-## Examples
-
-```bash
-# Initialize local configuration directory
-pixi run api-dock init
-
-# List available configurations, and available commands
-pixi run api-dock
-
-# Start API server
-# - default configuration (api_dock_config/config.yaml) with FastAPI
-pixi run api-dock start
-# - default configuration with Flask (backbone options: fastapi (default) or flask)
-pixi run api-dock start --backbone flask
-# - specify with host and/or port
-pixi run api-dock start --host 0.0.0.0 --port 9000
-
-
-# these commands also work for alternative configurations (example: api_dock_config/config_v2.yaml)
-pixi run api-dock start config_v2
-pixi run api-dock describe config_v2
-```
-
-**For more details**, see the [Configuration Wiki](https://github.com/yourusername/api_dock/wiki/Configuration).
-
----
-
-# CONFIGURATION AND SYNTAX
-
-Assume our file structure is:
-
-```bash
-api_dock_config
-├── config.yaml
-├── config_v2.yaml
-├── databases
-│    ├── analytics_db.yaml
-│    └── versioned_db
-│        ├── 0.1.yaml
-│        ├── 0.5.yaml
-│        └── 1.1.yaml
-└── remotes
-    ├── service1.yaml
-    ├── service2.yaml
-    └── versioned_service
-        ├── 0.1.yaml
-        ├── 0.2.yaml
-        └── 0.3.yaml
-```
-
----
-
-## Main Configuration (`api_dock_config/config.yaml`)
-
-The main configuration files are stored in the top level of the CWD's `api_dock_config/` directory. By default api-dock expects there to be one called `config.yaml`, however configs with different names (such as `config_v2`) can be added and launched as shown in the CLI Examples section.
+The main configuration file tells api-dock which remote-api and database configuration files to connect to.  Additionally there are adds meta-data (returned by the base-api-route) and optional-settings:
 
 ```yaml
 # api_dock_config/config.yaml
-name: "My API Dock"
-description: "API proxy for multiple services"
-authors: ["Your Name"]
 
-# Remote APIs to proxy
+# meta-data: this is returned as the base api-route by default
+name: # API Name
+description: # Description of API
+authors: # list of Authors
+
+# sql-databases to query
+databases:
+  - "unversioned_db"     # adds database configuration in  "api_dock_config/databases/unversioned_db.yaml"
+  - "versioned_db"       # adds database configurations in  "api_dock_config/databases/versioned_db/"
+
+# remote APIs to proxy
 remotes:
   - "service1"           # add configuration in "api_dock_config/remotes/service1.yaml"
   - "service2"           # add configuration in "api_dock_config/remotes/service2.yaml"
   - "versioned_service"  # add configurations in versions in "api_dock_config/remotes/versioned_service/"
 
-# SQL databases to query
-databases:
-  - "analytics_db"       # adds database configuration in  "api_dock_config/databases/analytics_db.yaml"
-  - "versioned_db"       # adds database configurations in  "api_dock_config/databases/versioned_db/"
-
 # Optional HTTP behavior settings
 settings:
   add_trailing_slash: true              # Auto-add trailing slash to paths (default: true)
   follow_protocol_downgrades: false     # Allow HTTPS->HTTP redirects (default: false)
 ```
 
-### Settings
+### HTTP behavior Settings
 
 The optional `settings` section controls HTTP behavior:
 
@@ -247,38 +228,30 @@ The optional `settings` section controls HTTP behavior:
 
 - **`follow_protocol_downgrades`** (default: `false`): Control how HTTP redirects are handled. When `false` (recommended), HTTPS→HTTP redirects are blocked for security. When `true`, allows following redirects that downgrade from HTTPS to HTTP (not recommended for production).
 
-**Example:**
-```yaml
-settings:
-  add_trailing_slash: true              # Avoids redirects by adding trailing slash
-  follow_protocol_downgrades: false     # Blocks insecure HTTPS->HTTP redirects
-```
-
 ---
 
 ## Remote Configurations
 
-The example below is a remote configuration. 
+Remote Configurations allow you to proxy existing apis.  In the simple example above, 
+
+```yaml
+# api_dock_config/remotes/service1.yaml
+name: service1
+description: "API Service1"
+url: https://remote.api.com
+```
+
+`name` defines the slug and `url` points to the existing api. So any route on `https://remote.api.com/*` may also be reached by at `service1/*`. However, the configuration file offers much more control over what endpoints may or may not be served through the api-proxy.  In particular, specific endpoints may be added or blocked, methods such as `DELETE` may be blocked, and routes with different signatures may be parsed. The structure is as follows:
 
 ```yaml 
 # api_dock_config/remotes/service1.yaml
-name: service1                 # this is the slug that goes in the url (ie: /service1/users)
-url: http://api.example.com    # the base-url of the api being proxied
-description: This is an api    # included in response for /service1 route
+name:        # <str> this is the slug that goes in the url
+url:         # <str> the base-url of the api being proxied
+description: # (optional) <str> included in response for /service1 base route
 
-# Here is where we define the routing
-routes:
-  # routes with identical signatures
-  - health                                  # GET  http://api.example.com/health
-  - route: users                            # GET  http://api.example.com/users (using explicit method)
-    method: get
-  - users/{{user_id}}                       # GET  http://api.example.com/users/{{user_id}}
-  - route: users/{{user_id}}/posts          # POST http://api.example.com/users/{{user_id}}/posts
-    method: post
-  # route with a different signature
-  - route: users/{{user_id}}/permissions    # GET  http://api.example.com/user-permissions/{{user_id}}
-    remote_route: user-permissions/{{user_id}}
-    method: get
+routes:      # (optional) <list[str, dict]> constrain available-routes or map between route-signatures
+
+restricted:  # (optional) <list[str, dict]> block specific endpoints or methods
 ```
 
 ### Variable Placeholders
@@ -290,7 +263,9 @@ Routes use double curly braces `{{}}` for variable placeholders:
 - `users/{{user_id}}/profile` - Matches "users/123/profile"
 - `{{}}` - Anonymous variable (matches any single path segment)
 
-### String Routes (Simple GET Routes)
+### Routes
+
+#### String Routes (Simple GET Routes)
 
 ```yaml
 routes:
@@ -300,7 +275,7 @@ routes:
   - posts/{{post_id}}              # GET /posts/456
 ```
 
-### Dictionary Routes (Custom Methods and Mappings)
+#### Dictionary Routes (Custom Methods and Mappings)
 
 ```yaml
 routes:
@@ -327,20 +302,18 @@ routes:
 
 You can restrict access to specific routes using the `restricted` section. Restrictions support wildcards and method-specific filtering:
 
-```yaml
-name: restricted_config
-
-...
-
-routes:
-  ...
+#### String Routes Restrictions
 
+```yaml
 # Simple route restrictions (string format)
 restricted:
   - admin/{{}}                       # Block all admin routes (single segment wildcard)
   - users/{{user_id}}/private        # Block private user data
   - system/*                         # Block all routes starting with system/ (prefix wildcard)
+```
 
+#### Dictionary Routes Restrictions
+```yaml
 # Method-aware restrictions (dict format)
 restricted:
   - route: "*"
@@ -351,25 +324,27 @@ restricted:
     method: patch                    # Block PATCH requests to user routes
 ```
 
-**Wildcard Patterns:**
-- `{{}}` or `*` - Matches any single path segment (e.g., `users/{{}}` matches `users/123`)
-- `prefix/*` - Matches all routes starting with prefix/ (e.g., `admin/*` matches `admin/dashboard`, `admin/users/123`, etc.)
-- `*` - When used alone (string format), matches any single-segment route
-- `{route: "*", method: "X"}` - When used with a method (dict format), matches ALL routes regardless of path length
-
-**Method-Specific Restrictions:**
-- Use dict format with `route` and `method` fields to restrict specific HTTP methods
-- When `{route: "*", method: "X"}` is used, it blocks the specified method on ALL routes
-- Omit `method` field to restrict all methods for a route
-- Methods are case-insensitive (DELETE, delete, Delete all work)
-
-**For more details**, see the [Routing and Restrictions Wiki](https://github.com/yourusername/api_dock/wiki/Routing-and-Restrictions).
 
 ---
 
 ## SQL Database Support
 
-API Dock can also be used to query Databases. For now only parquet support is working but we will be adding other Databases in the future.
+Adding databases is similar to adding remote-apis, however now the `routes` section is required and maps directly to specifc `SQL` queries. Database routes support declarative URL query parameters via the `query_params` section. This lets you add filtering, sorting, pagination, conditional logic, and direct responses — all driven by the URL query string. Here's the structure
+
+
+```yaml 
+# api_dock_config/databases/versioned_db/0.1.yaml
+name:        # <str> this is the slug that goes in the url
+description: # (optional) <str> included in response for /versioned_db/0.1 base route
+authors:     # (optional) <str,list> included in response for /versioned_db/0.1 base route
+
+tables:      # <list[dict(name: uri)]> table definitions
+queries:     # (optional) named-queries used for complex sql queries for readability 
+
+routes:      # <list[dict]> maps routes to sql queries
+```
+
+For now only parquet support is working but we will be adding other Databases in the future.
 
 
 ### Database Configuration
@@ -414,7 +389,7 @@ queries:
 
 routes:
   - route: users/{{user_id}}/permissions
-    sql: "[[get_permissions]]"
+    sql: "[[get_user_permissions]]"
 ```
 
 
@@ -455,15 +430,12 @@ routes:
     sql: "[[get_permissions]]"
 ```
 
-**For more details**, see the [SQL Database Support Wiki](https://github.com/yourusername/api_dock/wiki/SQL-Database-Support).
+**For more details**, see the [SQL Database Support Wiki](https://github.com/SchmidtDSE/api_dock/wiki/SQL-Database-Support).
 
 ---
 
 ## URL Query Parameters
 
-Database routes support declarative URL query parameters via the `query_params` section. This lets you add filtering, sorting, pagination, conditional logic, and direct responses — all driven by the URL query string.
-
-Routes without a `query_params` section work exactly as before (full backward compatibility).
 
 ### Basic Filtering with `sql`
 
@@ -667,6 +639,63 @@ Parameters are processed in this order (first match wins for early returns):
 5. `sql_append` parameters — append post-WHERE clauses (ORDER BY, LIMIT, etc.)
 6. Execute final SQL query
 
+---
+
+# CLI
+
+## Commands
+
+API Dock provides a modern Click-based CLI:
+
+- **pixi run api-dock** (default): List all available configurations and commands
+- **pixi run api-dock init [--force]**: Initialize `api_dock_config/` directory with default configs
+- **pixi run api-dock start [config_name]**: Start API Dock server with optional config name
+- **pixi run api-dock describe [config_name]**: Display formatted configuration with expanded SQL queries
+- **pixi run api-dock encrypt <plaintext>**: Encrypt values using local/AWS KMS encryption
+- **pixi run api-dock decrypt <ciphertext>**: Decrypt encrypted values (for testing/debugging)
+- **pixi run api-dock generate-key**: Generate new Fernet encryption key for local encryption
+
+**Note**: All commands shown use `pixi run` for the pixi environment. If not using pixi, drop the `pixi run` prefix (e.g., `api-dock start` instead of `pixi run api-dock start`).
+
+
+## Examples
+
+```bash
+# Initialize local configuration directory
+pixi run api-dock init
+
+# List available configurations, and available commands
+pixi run api-dock
+
+# Start API server
+# - default configuration (api_dock_config/config.yaml) with FastAPI
+pixi run api-dock start
+# - default configuration with Flask (backbone options: fastapi (default) or flask)
+pixi run api-dock start --backbone flask
+# - specify with host and/or port
+pixi run api-dock start --host 0.0.0.0 --port 9000
+
+# Alternative configurations (example: api_dock_config/config_v2.yaml)
+pixi run api-dock start config_v2
+pixi run api-dock describe config_v2
+
+# Encryption commands
+pixi run api-dock generate-key                                    # Generate new encryption key
+pixi run api-dock encrypt "my-secret-token"                      # Encrypt using local key
+pixi run api-dock encrypt --method aws_kms --key-id arn:aws:... "secret"  # Encrypt using AWS KMS
+pixi run api-dock decrypt "gAAAAABh..."                          # Decrypt encrypted value
+```
+
+**For more details**, see the [Configuration Wiki](https://github.com/SchmidtDSE/api_dock/wiki/Configuration).
+
+---
+
+
+
+
+
+---
+
 # Cookies and Authentication
 
 API Dock supports cookie extraction and authentication for both remote APIs and database routes. Cookies can be passed through to remote APIs or used for authentication validation.
@@ -762,7 +791,7 @@ authentication:
 
 - `encrypted: true/false` - Whether stored values are encrypted and need decryption
 
-Note: Authentication currently only supports token extraction from cookies, not Authorization headers.
+Note: Authentication extracts tokens from cookies and supports multiple backend sources including AWS Secrets Manager, AWS KMS encryption, GCP Secret Manager, and file-based authentication.
 
 For detailed setup instructions and examples, see the complete authentication documentation.
 
@@ -775,7 +804,7 @@ The core functionality is available as a standalone `RouteMapper` class that can
 ## Basic Integration
 
 ```python
-from api_dock.route_mapper import RouteMapper
+from api_dock import RouteMapper
 import asyncio
 
 # Initialize with optional config path
@@ -863,7 +892,9 @@ route_mapper = RouteMapper(config_path="path/to/config.yaml")
 async def query_database():
     success, data, status, error = await route_mapper.map_database_route(
         database_name="db_example",
-        path="users/123"
+        path="users/123",
+        query_params={},
+        cookies={}
     )
 
     if success:
@@ -928,7 +959,7 @@ def database_proxy(database_name, path):
 
 # Advanced Configuration Examples
 
-This section provides examples for advanced API Dock features mentioned in the [Quick Example](#quick-example).
+This section provides examples for advanced API Dock features mentioned in the [Simple Example](#simple-example).
 
 ## Route Restrictions
 
@@ -1088,26 +1119,6 @@ pixi run jupyter lab .
 pixi run python scripts/hello_world.py
 ```
 
-The first time `pixi run` is executed the project will be installed (note this means the first run will be a bit slower). Any changes to the project will be updated on the subsequent `pixi run`.  It is unnecessary, but you can run `pixi install` after changes - this will update your local environment, so that it does not need to be updated on the next `pixi run`.
-
-Note, the repo's `pyproject.toml`, and `pixi.lock` files ensure `pixi run` will just work. No need to recreate an environment. Additionally, the `pyproject.toml` file includes `api_dock = { path = ".", editable = true }`. This line is equivalent to `pip install -e .`, so there is no need to pip install this module.
-
-The project was initially created using a `package_names.txt` and the following steps. Note that this should **NOT** be re-run as it will create a new project (potentially changing package versions).
-
-```bash
-#
-# IMPORTANT: Do NOT run this unless you explicity want to create a new pixi project
-#
-# 1. initialize pixi project (in this case the pyproject.toml file had already existed)
-pixi init . --format pyproject
-# 2. add specified python version
-pixi add python=3.11
-# 3. add packages (note this will use pixi magic to determine/fix package version ranges)
-pixi add $(cat package_names.txt)
-# 4. add pypi-packages, if any (note this will use pixi magic to determine/fix package version ranges)
-pixi add --pypi $(cat pypi_package_names.txt)
-```
-
 ---
 
 # License