tileserver-gl-light 5.4.0 → 5.5.0-pre.0

package/CHANGELOG.md

@@ -1,5 +1,8 @@
 # tileserver-gl changelog
 
+## 5.5.0-pre.0
+*  Add S3 support for PMTiles with multiple AWS credential profiles (https://github.com/maptiler/tileserver-gl/pull/1779) by @acalcutt
+
 ## 5.4.0
 * Fix the issue where the tile URL cannot be correctly parsed with the HTTPS protocol when using an nginx proxy service (https://github.com/maptiler/tileserver-gl/pull/1578) by @dakanggo
 * Use jemalloc as memory allocator in the docker image (https://github.com/maptiler/tileserver-gl/pull/1574) by @MichielMortier

package/docs/config.rst

@@ -225,7 +225,7 @@ Each item in this object defines one style (map). It can have the following opti
 Each item specifies one data source which should be made accessible by the server. It has to have one of the following options:
 
 * ``mbtiles`` -- name of the mbtiles file
-* ``pmtiles`` -- name of the pmtiles file or url.
+* ``pmtiles`` -- name of the pmtiles file, url, or S3 path.
 
 For example::
 
@@ -238,6 +238,9 @@ For example::
     },
     "source3": {
       "pmtiles": "https://foo.lan/source3.pmtiles"
+    },
+    "source4": {
+      "pmtiles": "s3://my-bucket/tiles/terrain.pmtiles"
     }
   }
 
@@ -260,6 +263,10 @@ For example::
     "sparse_vector_tiles": {
       "pmtiles": "custom_osm.pmtiles",
       "sparse": true
+    },
+    "production-s3-tiles": {
+      "pmtiles": "s3://prod-bucket/tiles.pmtiles",
+      "s3Profile": "production"
     }
   }
 
@@ -281,6 +288,29 @@ Here are the available options for each data source:
     When ``false`` (default), *tileserver-gl* returns a ``204 No Content`` for missing tiles, which typically signals the client to stop trying to load a substitute.
     Default: ``false``.
 
+``s3Profile`` (string)
+    Specifies the AWS credential profile to use for S3 PMTiles sources. The profile must be defined in your ``~/.aws/credentials`` file.
+    This is useful when you need to access multiple S3 buckets with different credentials.
+    Alternatively, you can specify the profile in the URL using ``?profile=profile-name``.
+    If both are specified, the configuration ``s3Profile`` takes precedence.
+    Optional, only applicable to PMTiles sources using S3 URLs.
+
+``requestPayer`` (boolean)
+    Enables support for "requester pays" S3 buckets where the requester (not the bucket owner) pays for data transfer costs.
+    Set to ``true`` if accessing a requester pays bucket.
+    Can be specified in the URL using ``?requestPayer=true`` or in the configuration.
+    If both are specified, the configuration value takes precedence.
+    Default: ``false``.
+    Optional, only applicable to PMTiles sources using S3 URLs.
+
+``s3Region`` (string)
+    Specifies the AWS region for the S3 bucket.
+    Important for optimizing performance and reducing data transfer costs when accessing AWS S3 buckets.
+    Can be specified in the URL using ``?region=region-name`` or in the configuration.
+    If both are specified, the configuration value takes precedence.
+    If not specified, uses ``AWS_REGION`` environment variable or defaults to ``us-east-1``.
+    Optional, only applicable to PMTiles sources using S3 URLs.
+
 .. note::
     These configuration options will be overridden by metadata in the MBTiles or PMTiles file. if corresponding properties exist in the file's metadata, you do not need to specify them in the data configuration.
 
@@ -331,7 +361,133 @@ For example::
 
 Alternatively, you can use ``pmtiles://{source2}`` to reference existing data object from the config.
 In this case, the server will look into the ``config.json`` to determine what file to use by data id.
-For the config above, this is equivalent to ``pmtiles://source2.mbtiles``.
+For the config above, this is equivalent to ``pmtiles://source2.pmtiles``.
+
+S3 and S3-Compatible Storage
+-----------------------------
+
+PMTiles files can be accessed directly from AWS S3 or S3-compatible storage services (such as Contabo, DigitalOcean Spaces, MinIO, etc.) using S3 URLs. This provides better performance and eliminates HTTP rate limiting issues.
+
+**Supported URL Formats:**
+
+1. **AWS S3 (default):**
+   ``s3://bucket-name/path/to/file.pmtiles``
+
+2. **S3-compatible storage with custom endpoint:**
+   ``s3://endpoint-url/bucket-name/path/to/file.pmtiles``
+
+**AWS Credentials:**
+
+S3 sources require AWS credentials to be configured. The server will automatically use credentials from:
+
+* Environment variables: ``AWS_ACCESS_KEY_ID``, ``AWS_SECRET_ACCESS_KEY``, ``AWS_REGION``
+* AWS credentials file: ``~/.aws/credentials`` on Linux/macOS or ``C:\Users\USERNAME\.aws\credentials`` on Windows
+* IAM role (when running on AWS EC2, ECS, or Lambda)
+
+For S3-compatible storage providers, use the same AWS credential format with your provider's access keys.
+
+Example using environment variables::
+
+  export AWS_ACCESS_KEY_ID=your_access_key
+  export AWS_SECRET_ACCESS_KEY=your_secret_key
+  export AWS_REGION=us-west-2
+
+**Multiple AWS Credential Profiles:**
+
+If you need to access S3 buckets with different credentials, you can use AWS credential profiles. Profiles are defined in your AWS credentials file (``~/.aws/credentials`` on Linux/macOS or ``C:\Users\USERNAME\.aws\credentials`` on Windows)::
+
+  [default]
+  aws_access_key_id=YOUR_DEFAULT_KEY
+  aws_secret_access_key=YOUR_DEFAULT_SECRET
+
+  [production]
+  aws_access_key_id=YOUR_PRODUCTION_KEY
+  aws_secret_access_key=YOUR_PRODUCTION_SECRET
+
+  [staging]
+  aws_access_key_id=YOUR_STAGING_KEY
+  aws_secret_access_key=YOUR_STAGING_SECRET
+
+**S3 Configuration Options (Main Config Data Section):**
+
+When configuring S3 sources in the main configuration file's ``data`` section, you can use URL query parameters or configuration properties. Configuration properties take precedence over URL parameters.
+
+*Profile* - Specifies which AWS credential profile to use::
+
+  # URL parameter
+  "pmtiles": "s3://bucket/tiles.pmtiles?profile=production"
+  
+  # Configuration property
+  "pmtiles": "s3://bucket/tiles.pmtiles",
+  "s3Profile": "production"
+
+Precedence order (highest to lowest): Configuration property ``s3Profile``, URL parameter ``?profile=...``, default AWS credential chain.
+
+*Region* - Specifies the AWS region (important for performance and cost optimization)::
+
+  # URL parameter
+  "pmtiles": "s3://bucket/tiles.pmtiles?region=us-west-2"
+  
+  # Configuration property
+  "pmtiles": "s3://bucket/tiles.pmtiles",
+  "s3Region": "us-west-2"
+
+Precedence order (highest to lowest): Configuration property ``s3Region``, URL parameter ``?region=...``, Environment variable ``AWS_REGION``, Default: ``us-east-1``.
+
+*RequestPayer* - Enables "requester pays" buckets where you pay for data transfer::
+
+  # URL parameter
+  "pmtiles": "s3://bucket/tiles.pmtiles?requestPayer=true"
+  
+  # Configuration property
+  "pmtiles": "s3://bucket/tiles.pmtiles",
+  "requestPayer": true
+
+Precedence order (highest to lowest): Configuration property ``requestPayer``, URL parameter ``?requestPayer=true``, Default: ``false``.
+
+**Complete Configuration Examples:**
+
+Using URL parameters::
+
+  "data": {
+    "us-west-tiles": {
+      "pmtiles": "s3://prod-bucket/tiles.pmtiles?profile=production&region=us-west-2"
+    },
+    "eu-requester-pays": {
+      "pmtiles": "s3://bucket/tiles.pmtiles?profile=prod&region=eu-central-1&requestPayer=true"
+    }
+  }
+
+Using configuration properties (recommended)::
+
+  "data": {
+    "us-west-tiles": {
+      "pmtiles": "s3://prod-bucket/tiles.pmtiles",
+      "s3Profile": "production",
+      "s3Region": "us-west-2"
+    },
+    "eu-requester-pays": {
+      "pmtiles": "s3://bucket/tiles.pmtiles",
+      "s3Profile": "production",
+      "s3Region": "eu-central-1",
+      "requestPayer": true
+    }
+  }
+
+**Using S3 in Style JSON Sources:**
+
+When referencing S3 sources from within a style JSON file, use the ``pmtiles://`` prefix with S3 URLs. You can only specify profile, region, and requestPayer using URL query parameters (configuration properties are not available in style JSON)::
+
+  "sources": {
+    "aws-tiles": {
+      "url": "pmtiles://s3://my-bucket/tiles.pmtiles?profile=production",
+      "type": "vector"
+    },
+    "spaces-tiles": {
+      "url": "pmtiles://s3://example-storage.com/my-bucket/tiles.pmtiles?region=nyc3",
+      "type": "vector"
+    }
+  }
 
 Sprites
 -------

package/docs/usage.rst

@@ -6,10 +6,10 @@ Getting started
 ======
 ::
 
-  Usage: main.js tileserver-gl [file] [options]
+  Usage: tileserver-gl [file] [options]
 
   Options:
-    --file <file>             MBTiles or PMTiles file
+    --file <file>             MBTiles or PMTiles file (local path, http(s)://, s3://, pmtiles://, or mbtiles:// URL)
                                 ignored if the configuration file is also specified
     --mbtiles <file>          (DEPRECIATED) MBTiles file
                                 ignored if file is also specified
@@ -19,7 +19,10 @@ Getting started
     -p, --port <port>         Port [8080] (default: 8080)
     -C|--no-cors              Disable Cross-origin resource sharing headers
     -u|--public_url <url>     Enable exposing the server on subpaths, not necessarily the root of the domain
-    -V, --verbose             More verbose output
+    -V, --verbose [level]     More verbose output (level 1-3)
+                                -V, --verbose, -V 1, or --verbose 1: Important operations
+                                -V 2 or --verbose 2: Detailed operations  
+                                -V 3 or --verbose 3: All requests and debug info
     -s, --silent              Less verbose output
     -l|--log_file <file>      output log file (defaults to standard out)
     -f|--log_format <format>  define the log format:  https://github.com/expressjs/morgan#morganformat-options
@@ -27,6 +30,58 @@ Getting started
     -h, --help                display help for command
 
 
+File Source Options
+======
+
+The `--file` option supports multiple source types:
+
+**Local files:**
+::
+
+  tileserver-gl --file ./data/zurich.mbtiles
+  tileserver-gl --file ./data/terrain.pmtiles
+
+**HTTP/HTTPS URLs:**
+::
+
+  tileserver-gl --file https://example.com/tiles.pmtiles
+
+**S3 URLs:**
+::
+
+  # Basic AWS S3
+  tileserver-gl --file s3://my-bucket/tiles.pmtiles
+
+  # With AWS credential profile
+  tileserver-gl --file "s3://my-bucket/tiles.pmtiles?profile=production"
+
+  # With specific region
+  tileserver-gl --file "s3://my-bucket/tiles.pmtiles?region=us-west-2"
+
+  # With profile and region
+  tileserver-gl --file "s3://my-bucket/tiles.pmtiles?profile=production&region=eu-central-1"
+
+  # Requester-pays bucket
+  tileserver-gl --file "s3://bucket/tiles.pmtiles?requestPayer=true"
+
+  # All options combined
+  tileserver-gl --file "s3://bucket/tiles.pmtiles?profile=prod&region=us-west-2&requestPayer=true"
+
+  # S3-compatible storage (e.g., DigitalOcean Spaces, Contabo)
+  tileserver-gl --file "s3://example-storage.com/my-bucket/tiles.pmtiles?profile=dev"
+
+**Protocol prefixes:**
+
+You can also use `pmtiles://` or `mbtiles://` prefixes to explicitly specify the file type:
+::
+
+  tileserver-gl --file pmtiles://https://example.com/tiles.pmtiles
+  tileserver-gl --file "pmtiles://s3://my-bucket/tiles.pmtiles?profile=production"
+  tileserver-gl --file mbtiles://./data/zurich.mbtiles
+
+.. note::
+   For S3 sources, AWS credentials must be configured via environment variables, AWS credentials file (`~/.aws/credentials` on Linux/macOS or `C:\Users\USERNAME\.aws\credentials` on Windows), or IAM roles. See the Configuration documentation for details on using AWS credential profiles.
+
 Default preview style and configuration
 ======
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tileserver-gl-light",
-  "version": "5.4.0",
+  "version": "5.5.0-pre.0",
   "description": "Map tile server for JSON GL styles - serving vector tiles",
   "main": "src/main.js",
   "bin": "src/main.js",
@@ -34,21 +34,22 @@
     "docker": "docker build . && docker run --rm -i -p 8080:8080 $(docker build -q .)"
   },
   "dependencies": {
+    "@aws-sdk/client-s3": "^3.922.0",
     "@jsse/pbfont": "^0.3.0",
     "@mapbox/mapbox-gl-rtl-text": "0.3.0",
     "@mapbox/mbtiles": "0.12.1",
     "@mapbox/polyline": "^1.2.1",
-    "@mapbox/sphericalmercator": "2.0.1",
+    "@mapbox/sphericalmercator": "2.0.2",
     "@mapbox/vector-tile": "2.0.4",
     "@maplibre/maplibre-gl-inspect": "1.7.1",
-    "@maplibre/maplibre-gl-style-spec": "23.3.0",
+    "@maplibre/maplibre-gl-style-spec": "24.2.0",
     "@sindresorhus/fnv1a": "3.1.0",
     "advanced-pool": "0.3.3",
-    "axios": "^1.11.0",
+    "axios": "^1.13.1",
     "chokidar": "4.0.3",
     "clone": "2.1.2",
     "color": "5.0.0",
-    "commander": "14.0.0",
+    "commander": "14.0.2",
     "copyfiles": "2.4.1",
     "cors": "2.8.5",
     "express": "5.1.0",
@@ -56,7 +57,7 @@
     "http-shutdown": "1.2.2",
     "leaflet": "1.9.4",
     "leaflet-hash": "0.2.1",
-    "maplibre-gl": "5.6.2",
+    "maplibre-gl": "5.8.0",
     "morgan": "1.10.1",
     "pbf": "4.0.1",
     "pmtiles": "4.3.0",