@versatiles/container 1.0.0

package/LICENSE.md

@@ -0,0 +1,24 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to [unlicense.org](https://unlicense.org/)

package/README.md

@@ -0,0 +1,245 @@
+A client library for [VersaTiles containers](https://github.com/versatiles-org/versatiles-spec).
+
+# Table of Content
+
+* [Install](#install)
+* [Usage Example](#usage-example)
+* [API](#api)
+  * [Class: VersaTiles](#class_versatiles)
+    * [constructor: new VersaTiles(source, options)](#constructor-new-versatilessource-options)
+    * [async getHeader()](#async-getheader)
+    * [async getMetadata()](#async-getmetadata)
+    * [async getTileFormat()](#async-gettileformat)
+    * [async getBlockIndex()](#async-getblockindex)
+    * [async getTileIndex(block)](#async-gettileindexblock)
+    * [async getTile(z, x, y)](#async-gettilez-x-y)
+    * [async getTileUncompressed(z, x, y)](#async-gettileuncompressedz-x-y)
+  * [Interface: Block](#interface_block)
+  * [Interface: Header](#interface_header)
+  * [Interface: Options](#interface_options)
+  * [Interface: TileIndex](#interface_tileindex)
+  * [Type: Compression](#type_compression)
+  * [Type: Format](#type_format)
+  * [Type: Reader](#type_reader)
+* [License](#license)
+* [Future work](#future-work)
+
+# Install
+
+`npm i versatiles`
+
+# Usage Example
+
+```js
+import VersaTiles from 'versatiles';
+import fs from 'fs';
+
+const container = new VersaTiles('https://example.org/planet.versatiles');
+const header = await container.getHeader();
+const tile = await container.getTileUncompressed(z,x,y);
+fs.writeFileSync('tile.' + header.tile_format, tile);
+```
+
+# API
+
+## Class: `VersaTiles`<a id="class_versatiles"></a>
+
+The `VersaTiles` class is a wrapper around a `.versatiles` container file. It provides methods\
+to access tile data, metadata, and other properties within the container. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L27">\[src]</a></sup>
+
+### constructor: `new VersaTiles(source, options)`
+
+Constructs a new instance of the VersaTiles class. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L50">\[src]</a></sup>
+
+**Parameters:**
+
+* `source: string | `[`Reader`](#type_reader)\
+  The data source for the tiles. This can be a URL starting with `http://` or `https://`,
+  a path to a local file, or a custom `Reader` function that reads data chunks based on offset and length.
+* `options: `[`Options`](#interface_options) (optional)\
+  Optional settings that configure tile handling.
+
+**Returns:** [`VersaTiles`](#class_versatiles)
+**Methods**
+
+### `async getHeader()`
+
+Asynchronously retrieves the header information from the `.versatiles` container.\
+This method is primarily for internal use. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L75">\[src]</a></sup>
+
+**Returns:** [`Header`](#interface_header)
+
+### `async getMetadata()`
+
+Asynchronously retrieves the metadata associated with the `.versatiles` container.\
+Metadata typically includes information about `vector_layers` for vector tiles.
+If the container does not include metadata, this method returns `null`. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L133">\[src]</a></sup>
+
+**Returns:** `null | object`
+
+### `async getTileFormat()`
+
+Asynchronously determines the tile format, such as "png" or "pbf", based on the header information. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L153">\[src]</a></sup>
+
+**Returns:** [`Format`](#type_format)
+
+### `async getBlockIndex()`
+
+Asynchronously retrieves a mapping of tile block indices. The map's keys are formatted as "{z},{x},{y}".\
+This method is for internal use to manage tile lookup within the container. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L165">\[src]</a></sup>
+
+**Returns:** `Map<string,`[`Block`](#interface_block)`>`
+
+### `async getTileIndex(block)`
+
+Asynchronously retrieves the tile index for a specified block. This is an internal method used to\
+maintain a lookup for every tile within a block. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L244">\[src]</a></sup>
+
+**Parameters:**
+
+* `block: `[`Block`](#interface_block)\
+  The block for which to retrieve the tile index.
+
+**Returns:** [`TileIndex`](#interface_tileindex)
+
+### `async getTile(z, x, y)`
+
+Asynchronously retrieves a specific tile's data as a Buffer. If the tile data is compressed as\
+defined in the container header, the returned Buffer will contain the compressed data.
+To obtain uncompressed data, use the `getTileUncompressed` method.
+If the specified tile does not exist, the method returns `null`. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L274">\[src]</a></sup>
+
+**Parameters:**
+
+* `z: number`\
+  The zoom level of the tile.
+* `x: number`\
+  The x coordinate of the tile within its zoom level.
+* `y: number`\
+  The y coordinate of the tile within its zoom level.
+
+**Returns:** `null | Buffer`
+
+### `async getTileUncompressed(z, x, y)`
+
+Asynchronously retrieves a specific tile's uncompressed data as a Buffer. This method first\
+retrieves the compressed tile data using `getTile` and then decompresses it based on the
+compression setting in the container header.
+If the specified tile does not exist, the method returns `null`. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/index.ts#L322">\[src]</a></sup>
+
+**Parameters:**
+
+* `z: number`\
+  The zoom level of the tile.
+* `x: number`\
+  The x coordinate of the tile within its zoom level.
+* `y: number`\
+  The y coordinate of the tile within its zoom level.
+
+**Returns:** `null | Buffer`
+
+## Interface: `Block`<a id="interface_block"></a>
+
+Interface for a block of tiles including necessary metadata.\
+For more details, refer to [spec v02](https://github.com/versatiles-org/versatiles-spec/blob/main/v02/readme.md#block_index). <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L85">\[src]</a></sup>
+**Properties**
+
+* `level: number`\
+  The zoom level for this block.
+* `column: number`\
+  The column position of this block at the current zoom level.
+* `row: number`\
+  The row position of this block at the current zoom level.
+* `colMin: number`\
+  Minimum column index for tiles stored in this block (range: 0-255).
+* `rowMin: number`\
+  Minimum row index for tiles stored in this block (range: 0-255).
+* `colMax: number`\
+  Maximum column index for tiles stored in this block (range: 0-255).
+* `rowMax: number`\
+  Maximum row index for tiles stored in this block (range: 0-255).
+* `blockOffset: number`\
+  Byte position where this block starts in the file container.
+* `tileIndexOffset: number`\
+  Byte position where the tile index starts within the container.
+* `tileIndexLength: number`\
+  Length of the tile index in bytes.
+* `tileCount: number`\
+  The number of tiles contained in this block.
+* `tileIndex: `[`TileIndex`](#interface_tileindex) (optional)\
+  Optional tile index if it has been fetched.
+
+## Interface: `Header`<a id="interface_header"></a>
+
+Interface for the metadata header of a `*.Versatiles` container. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L52">\[src]</a></sup>
+**Properties**
+
+* `magic: string`\
+  Identifier for the container format, usually "versatiles\_v02".
+* `version: string`\
+  Version of the container format, typically "v02".
+* `tileFormat: `[`Format`](#type_format)\
+  The format used for storing tiles.
+* `tileCompression: `[`Compression`](#type_compression)\
+  The type of compression applied to tiles.
+* `zoomMin: number`\
+  The minimum zoom level.
+* `zoomMax: number`\
+  The maximum zoom level.
+* `bbox: [number, number, number, number]`\
+  Bounding box coordinates as \[lon\_min, lat\_min, lon\_max, lat\_max].
+* `metaOffset: number`\
+  The byte offset for metadata within the container.
+* `metaLength: number`\
+  The byte size of the metadata. A value of 0 means no metadata.
+* `blockIndexOffset: number`\
+  The byte offset for the block index within the container.
+* `blockIndexLength: number`\
+  The byte size of the block index. A value of 0 indicates no tiles in the container.
+
+## Interface: `Options`<a id="interface_options"></a>
+
+Interface for defining the options available for reading a container. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L119">\[src]</a></sup>
+**Properties**
+
+* `tms: boolean`\
+  If set to true, uses the [TMS (Tile Map Service) tile ordering](https://wiki.openstreetmap.org/wiki/TMS) where y=0 is the southernmost point.
+
+## Interface: `TileIndex`<a id="interface_tileindex"></a>
+
+Interface for the index structure used for tiles within a block. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L108">\[src]</a></sup>
+**Properties**
+
+* `offsets: Float64Array`\
+  Array indicating the start byte positions of tiles within the block.
+* `lengths: Float64Array`\
+  Array specifying the byte lengths of the tiles. A length of 0 means the tile is not stored.
+
+## Type: `Compression`<a id="type_compression"></a>
+
+Supported compression.\
+`null` signifies that the data is uncompressed. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L5">\[src]</a></sup>
+
+**Type:** `"br" | "gzip" | null`
+
+## Type: `Format`<a id="type_format"></a>
+
+Supported tile formats. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L11">\[src]</a></sup>
+
+**Type:** `"avif" | "bin" | "geojson" | "jpeg" | "json" | "pbf" | "png" | "svg" | "topojson" | "webp" | null`
+
+## Type: `Reader`<a id="type_reader"></a>
+
+Type definition for an asynchronous function to read a container's content.
+
+This can be useful for accessing data stored in various storage mediums. <sup><a href="https://github.com/versatiles-org/node-versatiles/blob/575beea/container/src/interfaces.ts#L33">\[src]</a></sup>
+
+**Type:** `(position: number, length: number) => Promise<Buffer>`
+
+# License
+
+[Unlicense](./LICENSE.md)
+
+# Future work
+
+This library could be extended to run in a web browser to read VersaTiles containers via `fetch`.

package/dist/decompress.d.ts

@@ -0,0 +1,14 @@
+/// <reference types="node" resolution-mode="require"/>
+import type { Compression } from './interfaces.js';
+/**
+ * Decompresses a buffer using the specified compression algorithm. Currently supports 'br' for Brotli and 'gzip' for GZIP.
+ * If the specified algorithm is not supported, the function resolves the promise with the original buffer.
+ *
+ * @param {Buffer} buffer - The buffer to be decompressed.
+ * @param {Compression} compression - The compression algorithm to use. Supported values are 'br' for Brotli and 'gzip' for GZIP.
+ * @returns {Promise<Buffer>} A promise that, when resolved, provides the decompressed buffer. If the compression type is not recognized,
+ * the promise will resolve with the original buffer. If decompression fails, the promise will be rejected with an error message.
+ * @throws {Error} Throws an error if the decompression process encounters an error. The error includes the buffer's length and
+ * the compression algorithm that was attempted.
+ */
+export declare function decompress(buffer: Buffer, compression: Compression): Promise<Buffer>;

package/dist/decompress.js

@@ -0,0 +1,45 @@
+import zlib from 'zlib';
+/**
+ * Decompresses a buffer using the specified compression algorithm. Currently supports 'br' for Brotli and 'gzip' for GZIP.
+ * If the specified algorithm is not supported, the function resolves the promise with the original buffer.
+ *
+ * @param {Buffer} buffer - The buffer to be decompressed.
+ * @param {Compression} compression - The compression algorithm to use. Supported values are 'br' for Brotli and 'gzip' for GZIP.
+ * @returns {Promise<Buffer>} A promise that, when resolved, provides the decompressed buffer. If the compression type is not recognized,
+ * the promise will resolve with the original buffer. If decompression fails, the promise will be rejected with an error message.
+ * @throws {Error} Throws an error if the decompression process encounters an error. The error includes the buffer's length and
+ * the compression algorithm that was attempted.
+ */
+export async function decompress(buffer, compression) {
+    return new Promise((resolve, reject) => {
+        switch (compression) {
+            case 'br':
+                zlib.brotliDecompress(buffer, handle);
+                break;
+            case 'gzip':
+                zlib.gunzip(buffer, handle);
+                break;
+            default:
+                resolve(buffer);
+                break;
+        }
+        /**
+         * Handles the result of the decompression operation. If decompression is successful, the promise is resolved
+         * with the decompressed buffer. If an error occurs during decompression, the promise is rejected with a descriptive
+         * error message.
+         *
+         * @param {Error | null} error - The error object, if any, returned from the decompression operation.
+         * @param {Buffer} result - The decompressed buffer returned from the decompression operation.
+         * @throws {Error} Throws an error if decompression fails, including the buffer length and the attempted compression method.
+         */
+        function handle(error, result) {
+            if (error) {
+                reject(`Can not decompress buffer (length=${buffer.length}) with "${compression}"`);
+                return;
+            }
+            else {
+                resolve(result);
+            }
+        }
+    });
+}

package/dist/index.d.ts

@@ -0,0 +1,82 @@
+/// <reference types="node" resolution-mode="require"/>
+import type { Block, Format, Header, Options, Reader, TileIndex } from './interfaces.js';
+export type { Block, Compression, Format, Header, Options, Reader, TileIndex } from './interfaces.js';
+/**
+ * The `VersaTiles` class is a wrapper around a `.versatiles` container file. It provides methods
+ * to access tile data, metadata, and other properties within the container.
+ */
+export declare class VersaTiles {
+    #private;
+    /**
+     * Constructs a new instance of the VersaTiles class.
+     *
+     * @param source - The data source for the tiles. This can be a URL starting with `http://` or `https://`,
+     * a path to a local file, or a custom `Reader` function that reads data chunks based on offset and length.
+     * @param options - Optional settings that configure tile handling.
+     * @throws Will throw an error if the provided source is neither a string nor a Reader function.
+     */
+    constructor(source: Reader | string, options?: Options);
+    /**
+     * Asynchronously retrieves the header information from the `.versatiles` container.
+     * This method is primarily for internal use.
+     *
+     * @returns A promise that resolves with the header object.
+     * @throws Will throw an error if the container does not start with expected magic bytes indicating a valid format.
+     */
+    getHeader(): Promise<Header>;
+    /**
+     * Asynchronously retrieves the metadata associated with the `.versatiles` container.
+     * Metadata typically includes information about `vector_layers` for vector tiles.
+     * If the container does not include metadata, this method returns `null`.
+     *
+     * @returns A promise that resolves with an object representing the metadata, or null if no metadata is present.
+     */
+    getMetadata(): Promise<object | null>;
+    /**
+     * Asynchronously determines the tile format, such as "png" or "pbf", based on the header information.
+     *
+     * @returns A promise that resolves with the tile format string, or null if the format is undefined.
+     */
+    getTileFormat(): Promise<Format | null>;
+    /**
+     * Asynchronously retrieves a mapping of tile block indices. The map's keys are formatted as "{z},{x},{y}".
+     * This method is for internal use to manage tile lookup within the container.
+     *
+     * @returns A promise that resolves with the block index map.
+     * @internal
+     */
+    getBlockIndex(): Promise<Map<string, Block>>;
+    /**
+     * Asynchronously retrieves the tile index for a specified block. This is an internal method used to
+     * maintain a lookup for every tile within a block.
+     *
+     * @param block - The block for which to retrieve the tile index.
+     * @returns A promise that resolves with the tile index.
+     * @internal
+     */
+    getTileIndex(block: Block): Promise<TileIndex>;
+    /**
+     * Asynchronously retrieves a specific tile's data as a Buffer. If the tile data is compressed as
+     * defined in the container header, the returned Buffer will contain the compressed data.
+     * To obtain uncompressed data, use the `getTileUncompressed` method.
+     * If the specified tile does not exist, the method returns `null`.
+     *
+     * @param z - The zoom level of the tile.
+     * @param x - The x coordinate of the tile within its zoom level.
+     * @param y - The y coordinate of the tile within its zoom level.
+     * @returns A promise that resolves with the tile data as a Buffer, or null if the tile cannot be found.
+     */
+    getTile(z: number, x: number, y: number): Promise<Buffer | null>;
+    /**
+     * Asynchronously retrieves a specific tile's uncompressed data as a Buffer. This method first
+     * retrieves the compressed tile data using `getTile` and then decompresses it based on the
+     * compression setting in the container header.
+     * If the specified tile does not exist, the method returns `null`.
+     *
+     * @param z - The zoom level of the tile.
+     * @param x - The x coordinate of the tile within its zoom level.
+     * @param y - The y coordinate of the tile within its zoom level.
+     * @returns A promise that resolves with the uncompressed tile data as a Buffer, or null if the tile cannot be found.
+     */
+    getTileUncompressed(z: number, x: number, y: number): Promise<Buffer | null>;
+}

package/dist/index.js

@@ -0,0 +1,292 @@
+import HttpReader from './reader_http.js';
+import FileReader from './reader_file.js';
+import { decompress } from './decompress.js';
+const FORMATS = {
+    'c01': ['png', 'jpeg', 'webp', null, null, null, null, null, null, null, null, null, null, null, null, null, 'pbf'],
+    'v01': ['png', 'jpeg', 'webp', 'pbf'],
+    'v02': [
+        'bin', null, null, null, null, null, null, null, null, null, null, null, null, null, null, null,
+        'png', 'jpeg', 'webp', 'avif', 'svg', null, null, null, null, null, null, null, null, null, null, null,
+        'pbf', 'geojson', 'topojson', 'json',
+    ],
+};
+const COMPRESSIONS = [null, 'gzip', 'br'];
+/**
+ * The `VersaTiles` class is a wrapper around a `.versatiles` container file. It provides methods
+ * to access tile data, metadata, and other properties within the container.
+ */
+export class VersaTiles {
+    #options = {
+        tms: false,
+    };
+    #reader;
+    #decompress;
+    #header;
+    #metadata;
+    #blockIndex;
+    /**
+     * Constructs a new instance of the VersaTiles class.
+     *
+     * @param source - The data source for the tiles. This can be a URL starting with `http://` or `https://`,
+     * a path to a local file, or a custom `Reader` function that reads data chunks based on offset and length.
+     * @param options - Optional settings that configure tile handling.
+     * @throws Will throw an error if the provided source is neither a string nor a Reader function.
+     */
+    constructor(source, options) {
+        if (options)
+            Object.assign(this.#options, options);
+        if (typeof source === 'string') {
+            if (source.startsWith('https://') || source.startsWith('http://')) {
+                this.#reader = HttpReader(source);
+            }
+            else {
+                this.#reader = FileReader(source);
+            }
+        }
+        else if (typeof source === 'function') {
+            this.#reader = source;
+        }
+        else {
+            throw new Error('source must be a string or a Reader');
+        }
+        this.#decompress = decompress;
+    }
+    /**
+     * Asynchronously retrieves the header information from the `.versatiles` container.
+     * This method is primarily for internal use.
+     *
+     * @returns A promise that resolves with the header object.
+     * @throws Will throw an error if the container does not start with expected magic bytes indicating a valid format.
+     */
+    async getHeader() {
+        // deliver if known
+        if (this.#header)
+            return this.#header;
+        const data = await this.#read(0, 66);
+        // check magic bytes
+        if (!/^versatiles_v0[12]$/.test(data.toString('utf8', 0, 14))) {
+            throw new Error('Invalid Container');
+        }
+        const version = data.toString('utf8', 11, 14);
+        switch (version) {
+            case 'v01':
+                this.#header = {
+                    magic: data.toString('utf8', 0, 14),
+                    version: version,
+                    tileFormat: FORMATS[version][data.readUInt8(14)] ?? 'bin',
+                    tileCompression: COMPRESSIONS[data.readUInt8(15)] ?? null,
+                    zoomMin: data.readUInt8(16),
+                    zoomMax: data.readUInt8(17),
+                    bbox: [data.readFloatBE(18), data.readFloatBE(22), data.readFloatBE(26), data.readFloatBE(30)],
+                    metaOffset: Number(data.readBigUInt64BE(34)),
+                    metaLength: Number(data.readBigUInt64BE(42)),
+                    blockIndexOffset: Number(data.readBigUInt64BE(50)),
+                    blockIndexLength: Number(data.readBigUInt64BE(58)),
+                };
+                break;
+            case 'v02':
+                this.#header = {
+                    magic: data.toString('utf8', 0, 14),
+                    version: version,
+                    tileFormat: FORMATS[version][data.readUInt8(14)] ?? 'bin',
+                    tileCompression: COMPRESSIONS[data.readUInt8(15)] ?? null,
+                    zoomMin: data.readUInt8(16),
+                    zoomMax: data.readUInt8(17),
+                    bbox: [data.readInt32BE(18) / 1e7, data.readInt32BE(22) / 1e7, data.readInt32BE(26) / 1e7, data.readInt32BE(30) / 1e7],
+                    metaOffset: Number(data.readBigUInt64BE(34)),
+                    metaLength: Number(data.readBigUInt64BE(42)),
+                    blockIndexOffset: Number(data.readBigUInt64BE(50)),
+                    blockIndexLength: Number(data.readBigUInt64BE(58)),
+                };
+                break;
+            default:
+                throw new Error('Invalid Container');
+        }
+        return this.#header;
+    }
+    /**
+     * Asynchronously retrieves the metadata associated with the `.versatiles` container.
+     * Metadata typically includes information about `vector_layers` for vector tiles.
+     * If the container does not include metadata, this method returns `null`.
+     *
+     * @returns A promise that resolves with an object representing the metadata, or null if no metadata is present.
+     */
+    async getMetadata() {
+        if (this.#metadata !== undefined)
+            return this.#metadata;
+        const header = await this.getHeader();
+        if (header.metaLength === 0) {
+            this.#metadata = null;
+        }
+        else {
+            let buffer = await this.#read(header.metaOffset, header.metaLength);
+            buffer = await this.#decompress(buffer, header.tileCompression);
+            this.#metadata = JSON.parse(buffer.toString());
+        }
+        return this.#metadata;
+    }
+    /**
+     * Asynchronously determines the tile format, such as "png" or "pbf", based on the header information.
+     *
+     * @returns A promise that resolves with the tile format string, or null if the format is undefined.
+     */
+    async getTileFormat() {
+        return (await this.getHeader()).tileFormat;
+    }
+    /**
+     * Asynchronously retrieves a mapping of tile block indices. The map's keys are formatted as "{z},{x},{y}".
+     * This method is for internal use to manage tile lookup within the container.
+     *
+     * @returns A promise that resolves with the block index map.
+     * @internal
+     */
+    async getBlockIndex() {
+        if (this.#blockIndex)
+            return this.#blockIndex;
+        const header = await this.getHeader();
+        // read block_index buffer
+        let data = await this.#read(header.blockIndexOffset, header.blockIndexLength);
+        data = await this.#decompress(data, 'br');
+        // read index from buffer
+        const blocks = [];
+        switch (header.version) {
+            case 'c01':
+            case 'v01':
+                // check blog index length
+                if (data.length % 29 !== 0)
+                    throw new Error('invalid block index');
+                for (let i = 0; i < data.length; i += 29) {
+                    const slice = data.subarray(i, i + 29);
+                    addBlock(slice, 0n, slice.readBigUInt64BE(13), slice.readBigUInt64BE(21));
+                }
+                break;
+            case 'v02':
+                // check blog index length
+                if (data.length % 33 !== 0)
+                    throw new Error('invalid block index');
+                for (let i = 0; i < data.length; i += 33) {
+                    const slice = data.subarray(i, i + 33);
+                    addBlock(slice, slice.readBigUInt64BE(13), slice.readBigUInt64BE(13) + slice.readBigUInt64BE(21), // block_offset + tile_blobs_length
+                    slice.readUInt32BE(29));
+                }
+                break;
+        }
+        // build map
+        this.#blockIndex = new Map(blocks.map(b => [`${b.level},${b.column},${b.row}`, b]));
+        return this.#blockIndex;
+        // eslint-disable-next-line @typescript-eslint/max-params
+        function addBlock(buffer, blockOffset, tileIndexOffset, tileIndexLength) {
+            const block = {
+                level: buffer.readUInt8(0),
+                column: buffer.readUInt32BE(1),
+                row: buffer.readUInt32BE(5),
+                colMin: buffer.readUInt8(9),
+                rowMin: buffer.readUInt8(10),
+                colMax: buffer.readUInt8(11),
+                rowMax: buffer.readUInt8(12),
+                blockOffset: Number(blockOffset),
+                tileIndexOffset: Number(tileIndexOffset),
+                tileIndexLength: Number(tileIndexLength),
+                tileCount: 0,
+            };
+            block.tileCount = (block.colMax - block.colMin + 1) * (block.rowMax - block.rowMin + 1);
+            blocks.push(block);
+        }
+    }
+    /**
+     * Asynchronously retrieves the tile index for a specified block. This is an internal method used to
+     * maintain a lookup for every tile within a block.
+     *
+     * @param block - The block for which to retrieve the tile index.
+     * @returns A promise that resolves with the tile index.
+     * @internal
+     */
+    async getTileIndex(block) {
+        if (block.tileIndex)
+            return block.tileIndex;
+        let buffer = await this.#read(block.tileIndexOffset, block.tileIndexLength);
+        buffer = await this.#decompress(buffer, 'br');
+        const offsets = new Float64Array(block.tileCount);
+        const lengths = new Float64Array(block.tileCount);
+        for (let i = 0; i < block.tileCount; i++) {
+            offsets[i] = Number(buffer.readBigUInt64BE(12 * i)) + block.blockOffset;
+            lengths[i] = buffer.readUInt32BE(12 * i + 8);
+        }
+        block.tileIndex = { offsets, lengths };
+        return block.tileIndex;
+    }
+    /**
+     * Asynchronously retrieves a specific tile's data as a Buffer. If the tile data is compressed as
+     * defined in the container header, the returned Buffer will contain the compressed data.
+     * To obtain uncompressed data, use the `getTileUncompressed` method.
+     * If the specified tile does not exist, the method returns `null`.
+     *
+     * @param z - The zoom level of the tile.
+     * @param x - The x coordinate of the tile within its zoom level.
+     * @param y - The y coordinate of the tile within its zoom level.
+     * @returns A promise that resolves with the tile data as a Buffer, or null if the tile cannot be found.
+     */
+    async getTile(z, x, y) {
+        // when y index is inverted
+        if (this.#options.tms)
+            y = Math.pow(2, z) - y - 1;
+        // ensure block index is loaded
+        const blockIndex = await this.getBlockIndex();
+        // block xy
+        const bx = x >> 8;
+        const by = y >> 8;
+        // tile xy (within block)
+        const tx = x & 0xFF;
+        const ty = y & 0xFF;
+        // check if block containing tile is within bounds
+        const blockKey = `${z},${bx},${by}`;
+        const block = blockIndex.get(blockKey);
+        if (!block)
+            return null;
+        // check if block contains tile
+        if (tx < block.colMin || tx > block.colMax)
+            return null;
+        if (ty < block.rowMin || ty > block.rowMax)
+            return null;
+        // calculate sequential tile number
+        const j = (ty - block.rowMin) * (block.colMax - block.colMin + 1) + (tx - block.colMin);
+        // get tile index
+        const tileIndex = await this.getTileIndex(block);
+        const offset = tileIndex.offsets[j];
+        const length = tileIndex.lengths[j];
+        if (length === 0)
+            return null;
+        return this.#read(offset, length);
+    }
+    /**
+     * Asynchronously retrieves a specific tile's uncompressed data as a Buffer. This method first
+     * retrieves the compressed tile data using `getTile` and then decompresses it based on the
+     * compression setting in the container header.
+     * If the specified tile does not exist, the method returns `null`.
+     *
+     * @param z - The zoom level of the tile.
+     * @param x - The x coordinate of the tile within its zoom level.
+     * @param y - The y coordinate of the tile within its zoom level.
+     * @returns A promise that resolves with the uncompressed tile data as a Buffer, or null if the tile cannot be found.
+     */
+    async getTileUncompressed(z, x, y) {
+        const tile = await this.getTile(z, x, y);
+        if (!tile)
+            return null;
+        const header = await this.getHeader();
+        return this.#decompress(tile, header.tileCompression);
+    }
+    /**
+     * A private method to read a chunk of data from the source based on the specified offset and length.
+     *
+     * @param offset - The offset from the start of the source data to begin reading.
+     * @param length - The number of bytes to read from the source.
+     * @returns A promise that resolves with the read data as a Buffer.
+     * @private
+     */
+    async #read(offset, length) {
+        if (length === 0)
+            return Buffer.allocUnsafe(0);
+        return this.#reader(offset, length);
+    }
+}

package/dist/interfaces.d.ts

@@ -0,0 +1,103 @@
+/// <reference types="node" resolution-mode="require"/>
+/**
+ * Supported compression.
+ * `null` signifies that the data is uncompressed.
+ */
+export type Compression = 'br' | 'gzip' | null;
+/**
+ * Supported tile formats.
+ */
+export type Format = 'avif' | 'bin' | 'geojson' | 'jpeg' | 'json' | 'pbf' | 'png' | 'svg' | 'topojson' | 'webp' | null;
+/**
+ * Type definition for a function that decompresses data.
+ *
+ * @param {Buffer} data - The compressed data.
+ * @param {Compression} compression - The type of compression used.
+ * @returns {Promise<Buffer>} - The decompressed data.
+ */
+export type Decompressor = (data: Buffer, compression: Compression) => Promise<Buffer>;
+/**
+ * Type definition for an asynchronous function to read a container's content.
+ *
+ * This can be useful for accessing data stored in various storage mediums.
+ * @param {number} position - The byte offset at which to start reading.
+ * @param {number} length - The number of bytes to read.
+ * @returns {Promise<Buffer>} - The read data.
+ */
+export type Reader = (position: number, length: number) => Promise<Buffer>;
+/**
+ * Interface for the metadata header of a `*.Versatiles` container.
+ *
+ * @property {string} magic - Identifier for the container format, usually "versatiles_v02".
+ * @property {string} version - Version of the container format, typically "v02".
+ * @property {Format} tileFormat - The format used for storing tiles.
+ * @property {Compression} tileCompression - The type of compression applied to tiles.
+ * @property {number} zoomMin - The minimum zoom level.
+ * @property {number} zoomMax - The maximum zoom level.
+ * @property {[number, number, number, number]} bbox - Bounding box coordinates as [lon_min, lat_min, lon_max, lat_max].
+ * @property {number} metaOffset - The byte offset for metadata within the container.
+ * @property {number} metaLength - The byte size of the metadata. A value of 0 means no metadata.
+ * @property {number} blockIndexOffset - The byte offset for the block index within the container.
+ * @property {number} blockIndexLength - The byte size of the block index. A value of 0 indicates no tiles in the container.
+ */
+export interface Header {
+    magic: string;
+    version: string;
+    tileFormat: Format;
+    tileCompression: Compression;
+    zoomMin: number;
+    zoomMax: number;
+    bbox: [number, number, number, number];
+    metaOffset: number;
+    metaLength: number;
+    blockIndexOffset: number;
+    blockIndexLength: number;
+}
+/**
+ * Interface for a block of tiles including necessary metadata.
+ * For more details, refer to [spec v02](https://github.com/versatiles-org/versatiles-spec/blob/main/v02/readme.md#block_index).
+ *
+ * @property {number} level - The zoom level for this block.
+ * @property {number} column - The column position of this block at the current zoom level.
+ * @property {number} row - The row position of this block at the current zoom level.
+ * @property {number} colMin - Minimum column index for tiles stored in this block (range: 0-255).
+ * @property {number} rowMin - Minimum row index for tiles stored in this block (range: 0-255).
+ * @property {number} colMax - Maximum column index for tiles stored in this block (range: 0-255).
+ * @property {number} rowMax - Maximum row index for tiles stored in this block (range: 0-255).
+ * @property {number} blockOffset - Byte position where this block starts in the file container.
+ * @property {number} tileIndexOffset - Byte position where the tile index starts within the container.
+ * @property {number} tileIndexLength - Length of the tile index in bytes.
+ * @property {number} tileCount - The number of tiles contained in this block.
+ * @property {TileIndex=} tileIndex - Optional tile index if it has been fetched.
+ */
+export interface Block {
+    level: number;
+    column: number;
+    row: number;
+    colMin: number;
+    rowMin: number;
+    colMax: number;
+    rowMax: number;
+    blockOffset: number;
+    tileIndexOffset: number;
+    tileIndexLength: number;
+    tileCount: number;
+    tileIndex?: TileIndex;
+}
+/**
+ * Interface for the index structure used for tiles within a block.
+ *
+ * @property {Float64Array} offsets - Array indicating the start byte positions of tiles within the block.
+ * @property {Float64Array} lengths - Array specifying the byte lengths of the tiles. A length of 0 means the tile is not stored.
+ */
+export interface TileIndex {
+    offsets: Float64Array;
+    lengths: Float64Array;
+}
+/**
+ * Interface for defining the options available for reading a container.
+ * @property {boolean} tms - If set to true, uses the [TMS (Tile Map Service) tile ordering](https://wiki.openstreetmap.org/wiki/TMS) where y=0 is the southernmost point.
+ */
+export interface Options {
+    tms: boolean;
+}

package/dist/interfaces.js

@@ -0,0 +1 @@
+export {};

package/dist/reader_file.d.ts

@@ -0,0 +1,19 @@
+import type { Reader } from './interfaces.js';
+/**
+ * Creates a file reader function for reading chunks of data from a file. This reader abstracts
+ * the file reading process and provides a simple async interface to read data from the specified file.
+ * The reader keeps the file open, hence it should be used in contexts where the file is not
+ * expected to be modified by other processes. It's important to manage the lifecycle of the file
+ * descriptor properly and ensure it's closed when no longer needed.
+ *
+ * @param {string} filename - The name of the file to read from. This file must exist and be readable,
+ *                            otherwise the function will throw an error when trying to open it.
+ * @returns {Reader} A reader function that when called, returns a promise. This promise resolves with a
+ *                   Buffer containing the read bytes from the file. The function reads `length` bytes of data
+ *                   from the file starting at `position`. If an error occurs during the read operation,
+ *                   the promise is rejected with the error. If `position` is beyond the end of the file,
+ *                   the promise resolves to an empty buffer. If the read operation attempts to read beyond
+ *                   the end of the file, the promise resolves with a buffer that contains only the bytes
+ *                   that could be read.
+ */
+export default function getFileReader(filename: string): Reader;

package/dist/reader_file.js

@@ -0,0 +1,54 @@
+import fs from 'fs';
+/**
+ * Creates a file reader function for reading chunks of data from a file. This reader abstracts
+ * the file reading process and provides a simple async interface to read data from the specified file.
+ * The reader keeps the file open, hence it should be used in contexts where the file is not
+ * expected to be modified by other processes. It's important to manage the lifecycle of the file
+ * descriptor properly and ensure it's closed when no longer needed.
+ *
+ * @param {string} filename - The name of the file to read from. This file must exist and be readable,
+ *                            otherwise the function will throw an error when trying to open it.
+ * @returns {Reader} A reader function that when called, returns a promise. This promise resolves with a
+ *                   Buffer containing the read bytes from the file. The function reads `length` bytes of data
+ *                   from the file starting at `position`. If an error occurs during the read operation,
+ *                   the promise is rejected with the error. If `position` is beyond the end of the file,
+ *                   the promise resolves to an empty buffer. If the read operation attempts to read beyond
+ *                   the end of the file, the promise resolves with a buffer that contains only the bytes
+ *                   that could be read.
+ */
+export default function getFileReader(filename) {
+    const fd = fs.openSync(filename, 'r');
+    /**
+     * Reads a chunk of data from the file at the specified position and length.
+     * It creates a buffer of the specified `length`, seeks to the `position` in the file,
+     * and reads `length` bytes into the buffer. This function is intended to be used with files
+     * that are not being concurrently modified, as it relies on consistent file state to
+     * function correctly.
+     *
+     * @param {number} position - The starting position in the file to read from. If this position
+     *                            is beyond the end of the file, the function will resolve to an
+     *                            empty buffer.
+     * @param {number} length   - The number of bytes to read from the file starting at `position`.
+     *                            If `length` extends beyond the end of the file, the returned buffer
+     *                            will only include the bytes that could be read up to the end of the file.
+     * @returns {Promise<Buffer>} A promise that resolves with the buffer containing the read bytes. If
+     *                            an error occurs during the read, such as if the position is invalid or
+     *                            the file descriptor becomes invalid due to the file being closed, the
+     *                            promise will be rejected with an error.
+     */
+    return async function read(position, length) {
+        return new Promise((resolve, reject) => {
+            fs.read(fd, {
+                buffer: Buffer.alloc(length),
+                position,
+                length,
+            }, (err, _, buf) => {
+                if (err) {
+                    reject(err);
+                    return;
+                }
+                resolve(buf);
+            });
+        });
+    };
+}

package/dist/reader_http.d.ts

@@ -0,0 +1,10 @@
+import type { Reader } from './interfaces.js';
+/**
+ * Creates a function capable of reading data from a specified URL, which can be used
+ * to read data chunks in an HTTP GET request. This is particularly useful for
+ * operations such as streaming or handling large data in segments.
+ *
+ * @param url - The URL from which data will be read.
+ * @returns A `Reader` function that asynchronously reads a specified chunk of data from the URL.
+ */
+export default function getHTTPReader(url: string): Reader;

package/dist/reader_http.js

@@ -0,0 +1,86 @@
+import https from 'https';
+import http from 'http';
+const DEFAULT_TIMEOUT = 10000;
+/**
+ * A collection mapping protocol names to their respective `ClientInfo`.
+ */
+const clients = {
+    https: { client: https, agent: new https.Agent({ keepAlive: true }) },
+    http: { client: http, agent: new http.Agent({ keepAlive: true }) },
+};
+/**
+ * Creates a function capable of reading data from a specified URL, which can be used
+ * to read data chunks in an HTTP GET request. This is particularly useful for
+ * operations such as streaming or handling large data in segments.
+ *
+ * @param url - The URL from which data will be read.
+ * @returns A `Reader` function that asynchronously reads a specified chunk of data from the URL.
+ */
+export default function getHTTPReader(url) {
+    /**
+     * Asynchronously reads a data chunk from the provided URL based on the specified range.
+     *
+     * @param position - The starting byte position of the data chunk to read.
+     * @param length - The number of bytes to read from the starting position.
+     * @returns A promise that resolves with a `Buffer` containing the data chunk.
+     *          If the request fails or the server responds with a non-successful status code,
+     *          the promise is rejected with an error.
+     */
+    return async function read(position, length) {
+        const headers = {
+            // eslint-disable-next-line @typescript-eslint/naming-convention
+            'user-agent': 'Mozilla/5.0 (compatible; versatiles; +https://www.npmjs.com/package/versatiles)',
+            'range': `bytes=${position}-${position + length - 1}`,
+        };
+        const protocol = new URL(url).protocol.slice(0, -1);
+        if (!(protocol in clients)) {
+            throw new Error(`Unsupported protocol: ${protocol}`);
+        }
+        /**
+         * Performs the HTTP request and retrieves the response.
+         * @type {IncomingMessage}
+         */
+        const message = await new Promise((resolve, reject) => {
+            const watchdog = setTimeout(() => {
+                reject(new Error('Request timed out'));
+            }, DEFAULT_TIMEOUT);
+            clients[protocol].client
+                .request(url, {
+                method: 'GET',
+                agent: clients[protocol].agent,
+                headers,
+                timeout: DEFAULT_TIMEOUT,
+            })
+                .on('response', response => {
+                clearTimeout(watchdog);
+                resolve(response);
+            })
+                .on('error', err => {
+                clearTimeout(watchdog);
+                reject(err);
+            })
+                .end();
+        });
+        if ((message.statusCode == null) || Math.floor(message.statusCode / 100) !== 2) {
+            message.destroy();
+            throw new Error(`Server responded with status code: ${message.statusCode}`);
+        }
+        /**
+         * Collects and concatenates response data chunks into a buffer.
+         * @type {Buffer}
+         */
+        const body = await new Promise((resolve, reject) => {
+            const buffers = [];
+            message
+                .on('data', (chunk) => buffers.push(chunk))
+                .on('error', err => {
+                message.destroy();
+                reject(err);
+            })
+                .once('end', () => {
+                resolve(Buffer.concat(buffers));
+            });
+        });
+        return body;
+    };
+}

package/package.json

@@ -0,0 +1,52 @@
+{
+  "name": "@versatiles/container",
+  "version": "1.0.0",
+  "description": "A client library for reading VersaTiles containers",
+  "scripts": {
+    "check": "npm run lint && npm run build && npm run test",
+    "build": "rm -rf dist && tsc -p tsconfig.build.json",
+    "doc": "npm run -w tsdoc2readme update_readme container/src/index.ts '# API'",
+    "lint": "eslint .",
+    "test": "jest"
+  },
+  "files": [
+    "dist/**/*.js",
+    "dist/**/*.d.ts"
+  ],
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/versatiles-org/node-versatiles.git"
+  },
+  "homepage": "https://github.com/versatiles-org/node-versatiles/blob/main/container/README.md",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "type": "module",
+  "author": "yetzt <node@yetzt.me>, Michael Kreil <versatiles@michael-kreil.de>",
+  "license": "Unlicense",
+  "engines": {
+    "node": ">= 18"
+  },
+  "keywords": [
+    "cloudtiles",
+    "geo",
+    "gis",
+    "map",
+    "maps",
+    "mapserver",
+    "maptiles",
+    "mvt",
+    "opencloudtiles",
+    "osm",
+    "pbf",
+    "tile",
+    "tiles",
+    "tileserver",
+    "vectortiles",
+    "versatiles"
+  ],
+  "devDependencies": {
+    "@types/node": "^20.9.0",
+    "jest": "^29.7.0",
+    "typescript": "^5.2.2"
+  }
+}