api-dock 0.7.1__tar.gz → 0.8.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. {api_dock-0.7.1 → api_dock-0.8.0}/PKG-INFO +220 -1
  2. {api_dock-0.7.1 → api_dock-0.8.0}/README.md +219 -0
  3. api_dock-0.8.0/api_dock/database_config.py +972 -0
  4. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/example_api_dock_config/config.yaml +8 -0
  5. api_dock-0.8.0/api_dock/example_api_dock_config/databases/config.yaml +82 -0
  6. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/fast_api.py +33 -0
  7. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/flask_api.py +31 -0
  8. api_dock-0.8.0/api_dock/listings.py +346 -0
  9. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/route_mapper.py +53 -13
  10. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/sql_builder.py +109 -12
  11. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/storage_auth.py +133 -36
  12. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/types.py +59 -2
  13. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock.egg-info/PKG-INFO +220 -1
  14. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock.egg-info/SOURCES.txt +4 -0
  15. {api_dock-0.7.1 → api_dock-0.8.0}/pyproject.toml +1 -1
  16. api_dock-0.8.0/tests/test_listings.py +221 -0
  17. api_dock-0.8.0/tests/test_shared_database_config.py +835 -0
  18. api_dock-0.7.1/api_dock/database_config.py +0 -441
  19. {api_dock-0.7.1 → api_dock-0.8.0}/LICENSE.md +0 -0
  20. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/__init__.py +0 -0
  21. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/auth.py +0 -0
  22. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/cli.py +0 -0
  23. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/config.py +0 -0
  24. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/config_discovery.py +0 -0
  25. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/encryption.py +0 -0
  26. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/example_api_dock_config/databases/example_db.yaml +0 -0
  27. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock/example_api_dock_config/remotes/example_remote.yaml +0 -0
  28. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock.egg-info/dependency_links.txt +0 -0
  29. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock.egg-info/entry_points.txt +0 -0
  30. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock.egg-info/requires.txt +0 -0
  31. {api_dock-0.7.1 → api_dock-0.8.0}/api_dock.egg-info/top_level.txt +0 -0
  32. {api_dock-0.7.1 → api_dock-0.8.0}/setup.cfg +0 -0
  33. {api_dock-0.7.1 → api_dock-0.8.0}/tests/test_inject_cookies.py +0 -0
  34. {api_dock-0.7.1 → api_dock-0.8.0}/tests/test_proxy_pipeline.py +0 -0
  35. {api_dock-0.7.1 → api_dock-0.8.0}/tests/test_sql_builder.py +0 -0
  36. {api_dock-0.7.1 → api_dock-0.8.0}/tests/test_sql_selector.py +0 -0
  37. {api_dock-0.7.1 → api_dock-0.8.0}/tests/test_types.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: api_dock
3
- Version: 0.7.1
3
+ Version: 0.8.0
4
4
  Summary: A flexible API gateway that allows you to proxy requests to multiple remote APIs and Databases
5
5
  Author-email: Brookie Guzder-Williams <bguzder-williams@berkeley.edu>
6
6
  License-Expression: BSD-3-Clause
@@ -95,6 +95,7 @@ Here is an example:
95
95
  api_dock_config
96
96
  ├── config.yaml # The default main-config file
97
97
  ├── databases
98
+ │ ├── config.yaml # (optional) shared tables/schemas for all databases
98
99
  │ ├── unversioned_db.yaml # Database config without versioning
99
100
  │ └── versioned_db # Folder containing database configs for different versions
100
101
  │ ├── 0.1.yaml
@@ -231,6 +232,58 @@ The optional `settings` section controls HTTP behavior:
231
232
 
232
233
  - **`timeout`** (default: `10`): Upstream request timeout in seconds, applied to both the streaming and buffered proxy paths. Raise it for slow upstreams (e.g. large aggregation queries) that would otherwise return a 502 on timeout. Set to `null` or `false` to disable the timeout entirely (not recommended — a stalled upstream can hold the connection open indefinitely).
233
234
 
235
+ ### Catalog Endpoints (`expose`)
236
+
237
+ The optional `expose` section adds read-only endpoints that list the models and versions of your configured databases, remotes, or both ("sources"). Listings are **opt-in** — with no `expose` key nothing is added.
238
+
239
+ ```yaml
240
+ # Enable all three defaults: /databases, /remotes, /sources
241
+ expose: true
242
+ ```
243
+
244
+ ```yaml
245
+ GET /databases
246
+ # [{"model": "birdnet", "version": "2.4"},
247
+ # {"model": "birdnet", "version": "3.0"},
248
+ # {"model": "owl", "version": "0.5"}]
249
+
250
+ GET /sources # databases + remotes, combined
251
+ # [{"model": "birdnet", "version": "2.4"}, ..., {"model": "core", "version": "0.5.0"}]
252
+ ```
253
+
254
+ Versions are the config filename stems, so semver like `0.5.0` is preserved; unversioned sources report `version: null`. Enable only what you want, and control the output shape with `dict`:
255
+
256
+ ```yaml
257
+ expose:
258
+ dict: false # return "model/version" strings instead of {model, version} dicts
259
+ databases: true # add /databases
260
+ remotes: true # add /remotes
261
+ # sources omitted → not added
262
+
263
+ # GET /databases -> ["birdnet/2.4", "birdnet/3.0", "owl/0.5"]
264
+ ```
265
+
266
+ Each of `databases` / `remotes` / `sources` accepts several forms:
267
+
268
+ ```yaml
269
+ expose:
270
+ databases: false # do not add the endpoint
271
+
272
+ remotes: "list/remotes" # custom route path (same as true, but at /list/remotes)
273
+
274
+ databases: # explicit list of models (optionally version-filtered)
275
+ - birdnet
276
+ - owl:
277
+ versions: [4.0, 5.0] # ints/floats/strings all match the "4.0"/"5.0" stems
278
+
279
+ sources: # most explicit form
280
+ route: "list/sources"
281
+ include: [birdnet, core] # true (default) | false | list of models
282
+ dict: true # per-endpoint override of the top-level `dict`
283
+ ```
284
+
285
+ Each listing route is served both with and without a trailing slash (e.g. `/sources` and `/sources/` both work), so it doesn't matter which convention your clients use. Custom multi-segment routes (e.g. `list/databases`) take precedence over the `/{remote}/{path}` proxy. If a listing route would shadow a configured remote/database, or an `include` names something that doesn't exist, API Dock emits a startup warning. The exposed routes are also reflected in the root (`/`) metadata's `endpoints`.
286
+
234
287
  ---
235
288
 
236
289
  ## Remote Configurations
@@ -356,6 +409,7 @@ Database configurations are stored in `api_dock_config/databases/` directory. Ea
356
409
  - **tables**: Mapping of table names to file paths (supports S3, GCS, HTTPS, local paths)
357
410
  - **queries**: Named SQL queries for reuse
358
411
  - **routes**: REST endpoints mapped to SQL queries
412
+ - **schema** (optional): the shared schema (from `databases/config.yaml`) this config's `[[table]]` references fall back to. See [Shared Tables and Schemas](#shared-tables-and-schemas-databasesconfigyaml)
359
413
 
360
414
  ### Syntax
361
415
 
@@ -433,6 +487,171 @@ routes:
433
487
  sql: "[[get_permissions]]"
434
488
  ```
435
489
 
490
+ ### Shared Tables and Schemas (`databases/config.yaml`)
491
+
492
+ When several databases or versions read the same tables, define them once in the optional `api_dock_config/databases/config.yaml`. Everything lives under a `database` key: `meta` holds default table metadata, `schema` holds named groups of tables, and every other key is a global table.
493
+
494
+ ```yaml
495
+ # api_dock_config/databases/config.yaml
496
+ database:
497
+ # global tables, available as [[table1]] in any database config
498
+ table1:
499
+ uri: s3://your-bucket/table1.parquet
500
+ table3:
501
+ uri: s3://your-other-bucket/table3.parquet
502
+ region: us-west-1 # a table's own keys override `meta`
503
+ public: false
504
+
505
+ # defaults applied to every table (shared tables and the tables in each version config)
506
+ meta:
507
+ region: us-west-2
508
+ public: true
509
+
510
+ # schemas, available as [[birdnet_2p4.detections]] in any route
511
+ schema:
512
+ birdnet_2p4:
513
+ detections:
514
+ uri: s3://your-bucket/birdnet/2.4/detections.parquet
515
+ birdnet_3p0:
516
+ detections:
517
+ uri: s3://your-bucket/birdnet/3.0/detections.parquet
518
+ public: false
519
+ ```
520
+
521
+ A version config can name the schema it uses with `schema:`. An unqualified `[[name]]` is then looked up in order, first match wins:
522
+
523
+ 1. the version config's own `tables`
524
+ 2. its `schema:` in the shared config
525
+ 3. the shared config's global tables
526
+
527
+ ```yaml
528
+ # api_dock_config/databases/birdnet/2.4.yaml
529
+ name: birdnet
530
+ schema: birdnet_2p4
531
+ tables:
532
+ revisions: s3://your-bucket/birdnet/2.4/revisions.parquet # local to this version
533
+
534
+ routes:
535
+ # [[detections]] isn't in `tables`, so it comes from the birdnet_2p4 schema
536
+ - route: recordings/{{recording_id}}/detections
537
+ sql: SELECT [[detections]].* FROM [[detections]] WHERE [[detections]].recording_id = {{recording_id}}
538
+ ```
539
+
540
+ Any route can reference any schema directly as `[[schema.table]]`, so a different database (e.g. `owl/5.0`) can query `[[birdnet_2p4.detections]]`. To keep a table private to one database/version, define it in that version's `tables` instead. Qualified references are exposed to DuckDB as real views, so you can also use the full name or your own alias in plain SQL:
541
+
542
+ ```yaml
543
+ - route: detections/
544
+ sql: >
545
+ SELECT detections.common_name, COUNT(revisions.id) AS revcount
546
+ FROM [[birdnet_2p4.detections]]
547
+ LEFT JOIN [[revisions]] ON revisions.observation_id = birdnet_2p4.detections.id
548
+ GROUP BY birdnet_2p4.detections.common_name
549
+ ```
550
+
551
+ expands to
552
+
553
+ ```sql
554
+ SELECT detections.common_name, COUNT(revisions.id) AS revcount
555
+ FROM birdnet_2p4.detections
556
+ LEFT JOIN 's3://your-bucket/birdnet/2.4/revisions.parquet' AS revisions ON revisions.observation_id = birdnet_2p4.detections.id
557
+ GROUP BY birdnet_2p4.detections.common_name
558
+ ```
559
+
560
+ Notes:
561
+ - After `FROM`/`JOIN`, `[[schema.table]]` becomes `schema.table` with no alias, so `FROM [[birdnet_2p4.detections]] o` works. Elsewhere it becomes the bare table name (`detections`), because DuckDB doesn't accept `schema.table.*`.
562
+ - Schema and table names used as `[[schema.table]]` must be plain identifiers (letters, digits, underscores).
563
+ - Storage credentials are set per table. Tables whose `region`/`public` differ from the rest get their own S3 secret scoped to their path, so one query can mix regions and public/private buckets.
564
+ - Views are created only for the `[[schema.table]]` tables a query actually references.
565
+
566
+ #### Inline database configs (`slugs`)
567
+
568
+ Simple database/version configs (often just a description and a `schema`) can live in the shared file instead of in their own files. Config files keep working, and the two can be mixed, even for the same database:
569
+
570
+ ```yaml
571
+ # api_dock_config/databases/config.yaml
572
+ slugs:
573
+ - name: birdnet-bullfrog # the database slug in the URL
574
+ version: "2.5" # one version...
575
+ description: American Bullfrog Classifier from Birdnet 2.4
576
+ schema: birdnet_bullfrog_2p5v0p5
577
+ - name: birdnet-apple
578
+ authors: [API Team] # ...or several; keys here are defaults for each version
579
+ versions:
580
+ - version: "1.0"
581
+ description: Apple Classifier 1.0
582
+ schema: birdnet_apple_1p0
583
+ - version: "12.0"
584
+ description: Apple Classifier 12.0
585
+ schema: birdnet_apple_12p0
586
+ - name: notes # no version/versions = an unversioned database
587
+ tables:
588
+ notes: s3://your-bucket/notes.parquet
589
+ ```
590
+
591
+ - Each entry (or each `versions` item) takes the same keys as a database config file: `description`, `authors`, `schema`, `tables`, `routes`, `query_params`, and so on. Shared `routes`/`query_params` (below), including `include`/`exclude`, apply to them like any other database/version.
592
+ - Like file-based databases, a slug is only served if it's listed under `databases:` in the main `config.yaml`.
593
+ - A database's versions are the union of its version files and its `slugs` versions, so `latest`, the `/{database}` versions listing, and the `expose` catalog endpoints all see both. If a file and a slug define the same database/version, the file wins.
594
+ - Quote versions (`version: "2.10"`). Unquoted YAML numbers are floats, so `2.10` would become `"2.1"`.
595
+ - A malformed `slugs` section (missing `name`, both `version` and `versions`, a duplicate version, or a mix of versioned and unversioned entries for one name) returns a 500 "Shared database configuration error".
596
+
597
+ #### Shared routes and query params
598
+
599
+ The shared file can also define top-level `routes` and `query_params`. These are added to **every** database/version, which is handy when each model/version serves the same endpoints over its own `schema`:
600
+
601
+ ```yaml
602
+ # api_dock_config/databases/config.yaml
603
+ database:
604
+ ...
605
+
606
+ routes:
607
+ - route: recordings/{{recording_id}}/detections/
608
+ sql: SELECT [[detections]].* FROM [[detections]] WHERE [[detections]].recording_id = {{recording_id}}
609
+ - route: detections/{{id}}
610
+ sql: SELECT [[detections]].* FROM [[detections]] WHERE [[detections]].id = {{id}}
611
+ - route: not_for_everyone/{{id}}
612
+ sql: SELECT [[other]].* FROM [[other]] WHERE [[other]].id = {{id}}
613
+ exclude: # don't add this route to these slug/versions
614
+ - 'slug1/3.0'
615
+ - slug: slug2
616
+ version: 2.3
617
+ - slug: slug3
618
+ version: '*' # '*' = every version
619
+
620
+ # the same route defined twice: one for everything except birdnet/2.4, one only for it
621
+ - route: detections/
622
+ exclude: ['birdnet/2.4']
623
+ sql: SELECT [[detections]].* FROM [[detections]]
624
+ - route: detections/
625
+ include: ['birdnet/2.4'] # ONLY add this route to these slug/versions
626
+ sql: SELECT [[detections]].*, [[revisions]].id AS revision_id FROM [[detections]] LEFT JOIN [[revisions]] ON [[revisions]].observation_id = [[detections]].id
627
+
628
+ query_params:
629
+ - confidence:
630
+ sql: "[[detections]].confidence >= {{confidence}}"
631
+ - start_time:
632
+ sql: "[[detections]].start_time >= {{start_time}}"
633
+ exclude: ['slug1/3.9']
634
+ - limit:
635
+ sql_append: LIMIT {{limit}}
636
+
637
+ # limit ALL shared routes / query params to these slug/versions
638
+ route_inclusions: ['birdnet', 'owl/5.0']
639
+ query_inclusions: [] # empty or missing = no restriction
640
+
641
+ # opt slug/versions out of ALL shared routes / query params
642
+ route_exclusions: ['legacy_db']
643
+ query_exclusions:
644
+ - slug: slug4
645
+ version: 1.0
646
+ ```
647
+
648
+ Rules:
649
+ - **The version config wins.** Its own routes come first and replace any shared route with the same shape. Shape means the same path segments; `{{param}}` names and leading/trailing slashes are ignored, so `detections/{{id}}` and `/detections/{{detection_id}}/` are the same route. Routes the version config adds on top are kept.
650
+ - Shared `query_params` behave like a version config's top-level `query_params`. They apply to every route, and a param with the same name in the version config (or on a route) overrides the shared one.
651
+ - `include` and `route_inclusions`/`query_inclusions` are the opposite of `exclude` and `route_exclusions`/`query_exclusions`. When given (non-empty), the route or query param is added **only** to the listed slug/versions. A shared item is added only if it passes both the top-level lists and its own `include`/`exclude`.
652
+ - The same route (by shape) or query param (by name) can appear more than once in the shared file. Each database/version gets the first one whose `include`/`exclude` select it, so complementary `include`/`exclude` lists give different databases different versions of an endpoint.
653
+ - `include`, `exclude` and the four top-level lists take a list of `'<slug>/<version>'` strings or `{slug: <slug>, version: <version>}` mappings. `'<slug>'`, `'<slug>/*'` or a missing/`'*'` version match every version, including unversioned databases. Versions compare numerically when possible (`2.3`, `"2.3"`), and `latest` is resolved before matching.
654
+
436
655
  **For more details**, see the [SQL Database Support Wiki](https://github.com/SchmidtDSE/api_dock/wiki/SQL-Database-Support).
437
656
 
438
657
  ---
@@ -56,6 +56,7 @@ Here is an example:
56
56
  api_dock_config
57
57
  ├── config.yaml # The default main-config file
58
58
  ├── databases
59
+ │ ├── config.yaml # (optional) shared tables/schemas for all databases
59
60
  │ ├── unversioned_db.yaml # Database config without versioning
60
61
  │ └── versioned_db # Folder containing database configs for different versions
61
62
  │ ├── 0.1.yaml
@@ -192,6 +193,58 @@ The optional `settings` section controls HTTP behavior:
192
193
 
193
194
  - **`timeout`** (default: `10`): Upstream request timeout in seconds, applied to both the streaming and buffered proxy paths. Raise it for slow upstreams (e.g. large aggregation queries) that would otherwise return a 502 on timeout. Set to `null` or `false` to disable the timeout entirely (not recommended — a stalled upstream can hold the connection open indefinitely).
194
195
 
196
+ ### Catalog Endpoints (`expose`)
197
+
198
+ The optional `expose` section adds read-only endpoints that list the models and versions of your configured databases, remotes, or both ("sources"). Listings are **opt-in** — with no `expose` key nothing is added.
199
+
200
+ ```yaml
201
+ # Enable all three defaults: /databases, /remotes, /sources
202
+ expose: true
203
+ ```
204
+
205
+ ```yaml
206
+ GET /databases
207
+ # [{"model": "birdnet", "version": "2.4"},
208
+ # {"model": "birdnet", "version": "3.0"},
209
+ # {"model": "owl", "version": "0.5"}]
210
+
211
+ GET /sources # databases + remotes, combined
212
+ # [{"model": "birdnet", "version": "2.4"}, ..., {"model": "core", "version": "0.5.0"}]
213
+ ```
214
+
215
+ Versions are the config filename stems, so semver like `0.5.0` is preserved; unversioned sources report `version: null`. Enable only what you want, and control the output shape with `dict`:
216
+
217
+ ```yaml
218
+ expose:
219
+ dict: false # return "model/version" strings instead of {model, version} dicts
220
+ databases: true # add /databases
221
+ remotes: true # add /remotes
222
+ # sources omitted → not added
223
+
224
+ # GET /databases -> ["birdnet/2.4", "birdnet/3.0", "owl/0.5"]
225
+ ```
226
+
227
+ Each of `databases` / `remotes` / `sources` accepts several forms:
228
+
229
+ ```yaml
230
+ expose:
231
+ databases: false # do not add the endpoint
232
+
233
+ remotes: "list/remotes" # custom route path (same as true, but at /list/remotes)
234
+
235
+ databases: # explicit list of models (optionally version-filtered)
236
+ - birdnet
237
+ - owl:
238
+ versions: [4.0, 5.0] # ints/floats/strings all match the "4.0"/"5.0" stems
239
+
240
+ sources: # most explicit form
241
+ route: "list/sources"
242
+ include: [birdnet, core] # true (default) | false | list of models
243
+ dict: true # per-endpoint override of the top-level `dict`
244
+ ```
245
+
246
+ Each listing route is served both with and without a trailing slash (e.g. `/sources` and `/sources/` both work), so it doesn't matter which convention your clients use. Custom multi-segment routes (e.g. `list/databases`) take precedence over the `/{remote}/{path}` proxy. If a listing route would shadow a configured remote/database, or an `include` names something that doesn't exist, API Dock emits a startup warning. The exposed routes are also reflected in the root (`/`) metadata's `endpoints`.
247
+
195
248
  ---
196
249
 
197
250
  ## Remote Configurations
@@ -317,6 +370,7 @@ Database configurations are stored in `api_dock_config/databases/` directory. Ea
317
370
  - **tables**: Mapping of table names to file paths (supports S3, GCS, HTTPS, local paths)
318
371
  - **queries**: Named SQL queries for reuse
319
372
  - **routes**: REST endpoints mapped to SQL queries
373
+ - **schema** (optional): the shared schema (from `databases/config.yaml`) this config's `[[table]]` references fall back to. See [Shared Tables and Schemas](#shared-tables-and-schemas-databasesconfigyaml)
320
374
 
321
375
  ### Syntax
322
376
 
@@ -394,6 +448,171 @@ routes:
394
448
  sql: "[[get_permissions]]"
395
449
  ```
396
450
 
451
+ ### Shared Tables and Schemas (`databases/config.yaml`)
452
+
453
+ When several databases or versions read the same tables, define them once in the optional `api_dock_config/databases/config.yaml`. Everything lives under a `database` key: `meta` holds default table metadata, `schema` holds named groups of tables, and every other key is a global table.
454
+
455
+ ```yaml
456
+ # api_dock_config/databases/config.yaml
457
+ database:
458
+ # global tables, available as [[table1]] in any database config
459
+ table1:
460
+ uri: s3://your-bucket/table1.parquet
461
+ table3:
462
+ uri: s3://your-other-bucket/table3.parquet
463
+ region: us-west-1 # a table's own keys override `meta`
464
+ public: false
465
+
466
+ # defaults applied to every table (shared tables and the tables in each version config)
467
+ meta:
468
+ region: us-west-2
469
+ public: true
470
+
471
+ # schemas, available as [[birdnet_2p4.detections]] in any route
472
+ schema:
473
+ birdnet_2p4:
474
+ detections:
475
+ uri: s3://your-bucket/birdnet/2.4/detections.parquet
476
+ birdnet_3p0:
477
+ detections:
478
+ uri: s3://your-bucket/birdnet/3.0/detections.parquet
479
+ public: false
480
+ ```
481
+
482
+ A version config can name the schema it uses with `schema:`. An unqualified `[[name]]` is then looked up in order, first match wins:
483
+
484
+ 1. the version config's own `tables`
485
+ 2. its `schema:` in the shared config
486
+ 3. the shared config's global tables
487
+
488
+ ```yaml
489
+ # api_dock_config/databases/birdnet/2.4.yaml
490
+ name: birdnet
491
+ schema: birdnet_2p4
492
+ tables:
493
+ revisions: s3://your-bucket/birdnet/2.4/revisions.parquet # local to this version
494
+
495
+ routes:
496
+ # [[detections]] isn't in `tables`, so it comes from the birdnet_2p4 schema
497
+ - route: recordings/{{recording_id}}/detections
498
+ sql: SELECT [[detections]].* FROM [[detections]] WHERE [[detections]].recording_id = {{recording_id}}
499
+ ```
500
+
501
+ Any route can reference any schema directly as `[[schema.table]]`, so a different database (e.g. `owl/5.0`) can query `[[birdnet_2p4.detections]]`. To keep a table private to one database/version, define it in that version's `tables` instead. Qualified references are exposed to DuckDB as real views, so you can also use the full name or your own alias in plain SQL:
502
+
503
+ ```yaml
504
+ - route: detections/
505
+ sql: >
506
+ SELECT detections.common_name, COUNT(revisions.id) AS revcount
507
+ FROM [[birdnet_2p4.detections]]
508
+ LEFT JOIN [[revisions]] ON revisions.observation_id = birdnet_2p4.detections.id
509
+ GROUP BY birdnet_2p4.detections.common_name
510
+ ```
511
+
512
+ expands to
513
+
514
+ ```sql
515
+ SELECT detections.common_name, COUNT(revisions.id) AS revcount
516
+ FROM birdnet_2p4.detections
517
+ LEFT JOIN 's3://your-bucket/birdnet/2.4/revisions.parquet' AS revisions ON revisions.observation_id = birdnet_2p4.detections.id
518
+ GROUP BY birdnet_2p4.detections.common_name
519
+ ```
520
+
521
+ Notes:
522
+ - After `FROM`/`JOIN`, `[[schema.table]]` becomes `schema.table` with no alias, so `FROM [[birdnet_2p4.detections]] o` works. Elsewhere it becomes the bare table name (`detections`), because DuckDB doesn't accept `schema.table.*`.
523
+ - Schema and table names used as `[[schema.table]]` must be plain identifiers (letters, digits, underscores).
524
+ - Storage credentials are set per table. Tables whose `region`/`public` differ from the rest get their own S3 secret scoped to their path, so one query can mix regions and public/private buckets.
525
+ - Views are created only for the `[[schema.table]]` tables a query actually references.
526
+
527
+ #### Inline database configs (`slugs`)
528
+
529
+ Simple database/version configs (often just a description and a `schema`) can live in the shared file instead of in their own files. Config files keep working, and the two can be mixed, even for the same database:
530
+
531
+ ```yaml
532
+ # api_dock_config/databases/config.yaml
533
+ slugs:
534
+ - name: birdnet-bullfrog # the database slug in the URL
535
+ version: "2.5" # one version...
536
+ description: American Bullfrog Classifier from Birdnet 2.4
537
+ schema: birdnet_bullfrog_2p5v0p5
538
+ - name: birdnet-apple
539
+ authors: [API Team] # ...or several; keys here are defaults for each version
540
+ versions:
541
+ - version: "1.0"
542
+ description: Apple Classifier 1.0
543
+ schema: birdnet_apple_1p0
544
+ - version: "12.0"
545
+ description: Apple Classifier 12.0
546
+ schema: birdnet_apple_12p0
547
+ - name: notes # no version/versions = an unversioned database
548
+ tables:
549
+ notes: s3://your-bucket/notes.parquet
550
+ ```
551
+
552
+ - Each entry (or each `versions` item) takes the same keys as a database config file: `description`, `authors`, `schema`, `tables`, `routes`, `query_params`, and so on. Shared `routes`/`query_params` (below), including `include`/`exclude`, apply to them like any other database/version.
553
+ - Like file-based databases, a slug is only served if it's listed under `databases:` in the main `config.yaml`.
554
+ - A database's versions are the union of its version files and its `slugs` versions, so `latest`, the `/{database}` versions listing, and the `expose` catalog endpoints all see both. If a file and a slug define the same database/version, the file wins.
555
+ - Quote versions (`version: "2.10"`). Unquoted YAML numbers are floats, so `2.10` would become `"2.1"`.
556
+ - A malformed `slugs` section (missing `name`, both `version` and `versions`, a duplicate version, or a mix of versioned and unversioned entries for one name) returns a 500 "Shared database configuration error".
557
+
558
+ #### Shared routes and query params
559
+
560
+ The shared file can also define top-level `routes` and `query_params`. These are added to **every** database/version, which is handy when each model/version serves the same endpoints over its own `schema`:
561
+
562
+ ```yaml
563
+ # api_dock_config/databases/config.yaml
564
+ database:
565
+ ...
566
+
567
+ routes:
568
+ - route: recordings/{{recording_id}}/detections/
569
+ sql: SELECT [[detections]].* FROM [[detections]] WHERE [[detections]].recording_id = {{recording_id}}
570
+ - route: detections/{{id}}
571
+ sql: SELECT [[detections]].* FROM [[detections]] WHERE [[detections]].id = {{id}}
572
+ - route: not_for_everyone/{{id}}
573
+ sql: SELECT [[other]].* FROM [[other]] WHERE [[other]].id = {{id}}
574
+ exclude: # don't add this route to these slug/versions
575
+ - 'slug1/3.0'
576
+ - slug: slug2
577
+ version: 2.3
578
+ - slug: slug3
579
+ version: '*' # '*' = every version
580
+
581
+ # the same route defined twice: one for everything except birdnet/2.4, one only for it
582
+ - route: detections/
583
+ exclude: ['birdnet/2.4']
584
+ sql: SELECT [[detections]].* FROM [[detections]]
585
+ - route: detections/
586
+ include: ['birdnet/2.4'] # ONLY add this route to these slug/versions
587
+ sql: SELECT [[detections]].*, [[revisions]].id AS revision_id FROM [[detections]] LEFT JOIN [[revisions]] ON [[revisions]].observation_id = [[detections]].id
588
+
589
+ query_params:
590
+ - confidence:
591
+ sql: "[[detections]].confidence >= {{confidence}}"
592
+ - start_time:
593
+ sql: "[[detections]].start_time >= {{start_time}}"
594
+ exclude: ['slug1/3.9']
595
+ - limit:
596
+ sql_append: LIMIT {{limit}}
597
+
598
+ # limit ALL shared routes / query params to these slug/versions
599
+ route_inclusions: ['birdnet', 'owl/5.0']
600
+ query_inclusions: [] # empty or missing = no restriction
601
+
602
+ # opt slug/versions out of ALL shared routes / query params
603
+ route_exclusions: ['legacy_db']
604
+ query_exclusions:
605
+ - slug: slug4
606
+ version: 1.0
607
+ ```
608
+
609
+ Rules:
610
+ - **The version config wins.** Its own routes come first and replace any shared route with the same shape. Shape means the same path segments; `{{param}}` names and leading/trailing slashes are ignored, so `detections/{{id}}` and `/detections/{{detection_id}}/` are the same route. Routes the version config adds on top are kept.
611
+ - Shared `query_params` behave like a version config's top-level `query_params`. They apply to every route, and a param with the same name in the version config (or on a route) overrides the shared one.
612
+ - `include` and `route_inclusions`/`query_inclusions` are the opposite of `exclude` and `route_exclusions`/`query_exclusions`. When given (non-empty), the route or query param is added **only** to the listed slug/versions. A shared item is added only if it passes both the top-level lists and its own `include`/`exclude`.
613
+ - The same route (by shape) or query param (by name) can appear more than once in the shared file. Each database/version gets the first one whose `include`/`exclude` select it, so complementary `include`/`exclude` lists give different databases different versions of an endpoint.
614
+ - `include`, `exclude` and the four top-level lists take a list of `'<slug>/<version>'` strings or `{slug: <slug>, version: <version>}` mappings. `'<slug>'`, `'<slug>/*'` or a missing/`'*'` version match every version, including unversioned databases. Versions compare numerically when possible (`2.3`, `"2.3"`), and `latest` is resolved before matching.
615
+
397
616
  **For more details**, see the [SQL Database Support Wiki](https://github.com/SchmidtDSE/api_dock/wiki/SQL-Database-Support).
398
617
 
399
618
  ---