@ferrybig/cgeodata-reader 0.1.0 → 0.1.1

package/binary-format.md

@@ -184,6 +184,11 @@ self-contained: it can be inflated without any other block. Readers
 locate a block by the byte lengths recorded in the file, not by scanning
 for stream boundaries.
 
+In a version `2` file ([Section 5](#5-header)) chunks are not compressed
+blocks: a chunk is stored as [Section 8.7](#87-version-2-chunks) lays it
+out, with its own entropy coding. Name blocks are compressed blocks in
+every version.
+
 ## 4. File Layout
 
 A CGEODATA file consists of four consecutive sections, with no padding or
@@ -223,7 +228,7 @@ and a CGEODATA file never refers to them.
 | Field                | Type       | Description |
 |----------------------|------------|-------------|
 | `signature`          | 8 bytes    | The ASCII bytes `CGEODATA` (`43 47 45 4f 44 41 54 41`). |
-| `version`            | varint     | File kind and format version, `kind × 16 + version`. A CGEODATA file is kind `0`; this document specifies version `1`, so the field is `1`. See below. |
+| `version`            | varint     | File kind and format version, `kind × 16 + version`. A CGEODATA file is kind `0`; this document specifies version `2`, so the field is `2`. See below. |
 | `attribution_length` | varint     | Byte length of `attribution`. |
 | `attribution`        | bytes      | UTF-8 text, MAY contain Markdown, intended for display to humans, for example `[OpenStreetMap](https://openstreetmap.org/copyright)`. MAY be empty. |
 | `width_exp`          | u8         | The chunk grid has `W = 2^width_exp` columns. |
@@ -253,10 +258,12 @@ Version `0` of any kind is a pre-release version: files written under
 this document carry no compatibility promise once the version number
 changes.
 
-Version `1` of kind `0` differs from version `0` only in the layout of a
-chunk: version `0` stored every record's Morton code, which version `1`
-derives from the depths ([Section 8.6](#86-version-0-chunks)). A reader
-MAY read both.
+Versions `0`, `1` and `2` of kind `0` differ only in how a chunk is
+stored. Version `0` stored every record's Morton code, which version `1`
+derives from the depths ([Section 8.6](#86-version-0-chunks)). Version
+`2` keeps version `1`'s records but stores them range coded instead of
+in a zlib stream ([Section 8.7](#87-version-2-chunks)). A reader MAY read
+all three.
 
 Constraints:
 
@@ -641,6 +648,201 @@ million records took 3.77 million index-`0` records (3.5% more records)
 to fill, and the chunks went from 62.2 MB to 45.6 MB (-26.7%), with the
 same cell for every place.
 
+### 8.7. Version 2 Chunks
+
+In a version `2` file ([Section 5](#5-header)), a chunk's records are
+those of a version `1` chunk ([Sections 8.4](#84-records) and
+[8.5](#85-invariants)): they tile the chunk from code `0` in Morton order,
+and MAY end before the chunk does. Only their storage differs. The block
+is not a zlib stream:
+
+| Field             | Type                       | Description |
+|-------------------|----------------------------|-------------|
+| `palette_count`   | varint                     | As in Section 8.2, and below `2^26`. |
+| `palette`         | `P` varints, delta-coded   | As in Section 8.2. |
+| `record_count`    | varint                     | As in Section 8.2. |
+| `stream`          | bytes                      | The rest of the block: the records' depths and palette indices, range coded as described below. |
+
+The stream codes the chunk's quadtree as a series of binary decisions:
+whether each node splits, and, for each leaf, which palette index it has.
+Every decision is made with an adaptive probability chosen by a
+**context** from what was already decoded: chiefly the records to the
+west and south of the node, which in Morton order always come first.
+Neighbouring cells usually belong to the same place, so most decisions
+are nearly certain and cost a small fraction of a bit.
+
+#### 8.7.1. Range Decoder
+
+The decoder keeps two unsigned 32-bit integers, `range` and `code`, and a
+position `pos` in `stream`. Each probability `p` is an integer in
+`[1, 4095]`, the chance out of `4096` that a decision is `0`; every
+probability starts at `2048`.
+
+`stream[0]` MUST be `0`. `code` starts as `stream[1..4]` read as a
+big-endian integer, and MUST NOT be `0xFFFFFFFF`; `range` starts as
+`0xFFFFFFFF` and `pos` as `5`. A decision with probability `p` is decoded
+as follows, and `p` is updated in place:
+
+```
+bound = (range >> 12) × p
+if code < bound:
+    range = bound
+    p = p + ((4096 - p) >> 4)
+    bit = 0
+else:
+    code = code - bound
+    range = range - bound
+    p = p - (p >> 4)
+    bit = 1
+while range < 2^24:
+    range = range × 256
+    code = code × 256 + stream[pos]     # reading past the end is an error
+    pos = pos + 1
+```
+
+After the last record, `pos` MUST equal the length of `stream`. This is
+LZMA's range coder [LZMA] with 12-bit probabilities that adapt by a
+sixteenth each time. The matching encoder (informative):
+
+```
+low = 0  (at least 33 bits), range = 0xFFFFFFFF, cache = 0, pending = 1
+
+encode(bit, p):
+    bound = (range >> 12) × p
+    if bit == 0: range = bound;  p = p + ((4096 - p) >> 4)
+    else: low = low + bound;  range = range - bound;  p = p - (p >> 4)
+    while range < 2^24: range = range × 256;  shift_low()
+
+shift_low():
+    if low < 0xFF000000 or low >= 2^32:
+        carry = low >> 32
+        emit (cache + carry) & 0xFF, then pending - 1 bytes of (0xFF + carry) & 0xFF
+        pending = 0
+        cache = (low >> 24) & 0xFF
+    pending = pending + 1
+    low = (low & 0x00FFFFFF) × 256
+
+finish(): shift_low() five times.
+```
+
+Its first byte is always `0`, and the decoder then reads exactly the
+bytes it wrote.
+
+#### 8.7.2. Nodes and Neighbours
+
+A node is a cell of the chunk's quadtree: its depth `d` and its south-west
+corner `(x, y)`, in full-precision cells ([Section 8.1](#81-cell-grid-and-morton-codes)).
+Nodes are visited in pre-order from the root `(d = 0, x = 0, y = 0)`. A
+node at depth `d < D` first decodes a **split** decision
+([Section 8.7.3](#873-contexts)): `1` visits its four children of depth
+`d + 1` in Morton order, south-west `(x, y)`, south-east `(x + h, y)`,
+north-west `(x, y + h)` and north-east `(x + h, y + h)` with
+`h = 2^(D - d - 1)`; `0` makes it a leaf. A node at depth `D` is a leaf
+without a decision. Each leaf is the next record: its depth is `d`, its
+code is where the record before it ends ([Section 8.4](#84-records)), and
+its palette index is decoded as [Section 8.7.4](#874-palette-indices)
+describes. Decoding stops after `record_count` records, wherever the
+traversal is.
+
+A node's **west neighbour** is the record that contains the
+full-precision cell `(x - 1, y)`, and its **south neighbour** the one
+that contains `(x, y - 1)`. There is none when `x = 0` (respectively
+`y = 0`): neighbours never come from another chunk. Both cells have
+lower Morton codes than `(x, y)` and so were decoded before the node. A
+decoder can find them by remembering, for every row and every column of
+full-precision cells, the last record decoded in it: in Morton order the
+last record in row `y` before `(x, y)` is the one at `(x - 1, y)`, and
+likewise for columns.
+
+A neighbour's **category** relative to a node at depth `d` is `0` when
+there is no neighbour, `1` when its record's depth is at most `d`, `2`
+when it is `d + 1`, and `3` when it is deeper.
+
+#### 8.7.3. Contexts
+
+There are five tables of probabilities, each indexed by a context:
+
+| Table         | Entries | Used for |
+|---------------|---------|----------|
+| `split`       | 256     | split decisions, at `16 × d + 4 × cw + cs` |
+| `same_west`   | 544     | "the index is the west neighbour's", at `c` |
+| `same_south`  | 544     | "the index is the south neighbour's", at `c` |
+| `rank_length` | 128     | the unary part of a rank ([Section 8.7.4](#874-palette-indices)) |
+| `rank_bits`   | 1024    | the binary part of a rank |
+
+`cw` and `cs` are the west and south neighbours' categories. For a leaf,
+let `w` and `s` be the west and south neighbours' palette indices, or
+`-1` for a missing one, and `prev` the previous record's palette index,
+or `-1` for the first record. Then
+
+```
+c = 4 × (8 × d + 4 × [w = s] + 2 × [prev = w] + [prev = s]) + cw
+```
+
+where `[...]` is `1` when the comparison holds and `0` otherwise, and
+`-1` equals `-1`.
+
+#### 8.7.4. Palette Indices
+
+The decoder keeps a **move-to-front list** of the palette indices
+`0` to `P`, initially in that order. A leaf's palette index `v` is
+decoded as follows:
+
+1. If there is a west neighbour, a decision in `same_west[c]`: `1` means
+   `v = w`.
+2. Otherwise, or if that decision was `0`: if there is a south
+   neighbour and `s != w`, a decision in `same_south[c]`: `1` means
+   `v = s`.
+3. Otherwise, a **rank** `r`: `v` is the entry of the move-to-front list
+   that has `r` entries before it, not counting entries equal to `w` or
+   `s`. `r + 1` is coded in Elias gamma: let `a` be `1` if there is a
+   west neighbour plus `2` if there is a south neighbour. Decisions in
+   `rank_length[32 × a + k]`, for `k = 0, 1, ...`, are `1` until one is
+   `0`; their number of `1`s is `n`, and MUST be below `32`. Then `n`
+   decisions in `rank_bits[32 × n + k]`, for `k = n - 1` down to `0`,
+   are the bits of `r + 1` below its leading `1`, most significant
+   first. The list MUST have such an entry.
+
+Then `v` is moved to the front of the list.
+
+#### 8.7.5. Bounds and Errors
+
+A decoder MUST reject a block where `record_count` exceeds `4^D` (more
+records than cells), or exceeds `2048` times the length of `stream`, and
+MUST check both before allocating for the records. No stream reaches the
+second bound: every record takes at least one decision, a decision
+takes at least `log2(4096 / 4081)` bits, as no probability passes
+`4081` or drops below `15`, so a byte codes at most about 1500 records.
+
+It MUST also reject a block whose traversal ends (every node visited)
+before `record_count` records, and any of the conditions marked MUST in
+Sections 8.7.1 and 8.7.4. Every record is valid by construction: its
+depth is at most `D`, it is aligned, it ends within the chunk, and its
+palette index is at most `P`.
+
+#### 8.7.6. Design Notes
+
+This section is informative.
+
+The range coder replaced zlib for chunks because the data has structure
+a general-purpose compressor does not see. A version `1` chunk stores
+each depth in a byte, though the depths are just a pre-order listing of
+the quadtree's split flags, which are well predicted by the neighbours'
+depths. Its palette indices are well predicted by the west and south
+neighbours, which in Morton order are not usually the previous record:
+in the worldwide file, 74% of records have their west neighbour's index
+and 19% their south neighbour's, which zlib, looking at one byte column,
+cannot exploit. Measured on the worldwide build (7,608 chunks,
+110 million records), the chunks went from 45.6 MB as version `1` to
+18.5 MB (−59.5%), and the file from 47.6 MB to 20.5 MB.
+
+The cost is decoding speed: about 2.5 binary decisions per record, each
+data-dependent, make a chunk about 4 times as slow to decode as to
+inflate and parse in version `1`. Encoding is faster than zlib's highest
+level. Other layouts were measured on the same file: zlib over split
+flags and neighbour-predicted symbols gave 30.2 MB, and Zopfli, a better
+deflate encoder for version `1` unchanged, about 37.4 MB.
+
 ## 9. Lookup Procedure
 
 To resolve a coordinate `(lat, lon)` to a place ID, a reader performs the
@@ -667,9 +869,10 @@ following steps. `W`, `H`, and `D` come from the header.
    avoids inconsistent rounding between the two.
 4. **Fetch the chunk.** If `chunk_lengths[i]` is `0`, the result is
    unknown. Otherwise read `length(i)` bytes at `offset(i)`
-   ([Section 6.3](#63-locating-a-chunk)) and inflate them, and compute
-   the records' codes ([Section 8.4](#84-records)). Readers SHOULD cache
-   inflated chunks with their codes.
+   ([Section 6.3](#63-locating-a-chunk)) and inflate them (in a version
+   `2` file, decode them per [Section 8.7](#87-version-2-chunks)), and
+   compute the records' codes ([Section 8.4](#84-records)). Readers
+   SHOULD cache decoded chunks with their codes.
 5. **Compute the query code** `q` from `(x, y)` with the procedure of
    [Section 8.1](#81-cell-grid-and-morton-codes).
 6. **Binary search.** Find the largest `j` such that `codes[j] <= q`
@@ -738,12 +941,15 @@ record at all.
 
 **Keyframe interval.** See [Section 7.3](#73-name-blocks).
 
-**Compression.** zlib was chosen over zstd or brotli because it is a
-zero-dependency built-in in Node.js and browsers. The columnar and
-delta-coded layout captures most of the gain regardless of the
-compressor; on the prototype, zstd and brotli were only 10–20% smaller.
-Writers SHOULD use the highest zlib compression level; compression is a
-one-time cost.
+**Compression.** Versions `0` and `1` chose zlib over zstd or brotli
+because it is a zero-dependency built-in in Node.js and browsers. The
+columnar and delta-coded layout captures most of the gain regardless of
+the compressor; on the prototype, zstd and brotli were only 10–20%
+smaller. Writers of those versions SHOULD use the highest zlib
+compression level; compression is a one-time cost. Version `2`'s range
+coder ([Section 8.7](#87-version-2-chunks)) is a model of the data, not
+a general-purpose compressor, and needs no library; see
+[Section 8.7.6](#876-design-notes) for what it gains and costs.
 
 ## 11. Robustness and Security Considerations
 
@@ -765,6 +971,10 @@ assume that a file is well formed.
   verify the alignment invariant of [Section 8.5](#85-invariants), and
   in a version `0` file the ordering and non-overlap invariants of
   [Section 8.6](#86-version-0-chunks).
+- In a version `2` file, readers MUST check [Section 8.7.5](#875-bounds-and-errors)'s
+  bounds on `record_count` before allocating for it, and reject a stream
+  that runs past the end of its block or ends before it. The records'
+  depths, codes and palette indices are valid by construction.
 - Readers MUST treat the attribution text and place names as untrusted
   text. Where they are rendered as Markdown or HTML, they MUST be
   sanitised.
@@ -773,7 +983,7 @@ assume that a file is well formed.
 
 ## 12. Open Issues
 
-The following are known gaps or unresolved decisions in version `1`.
+The following are known gaps or unresolved decisions in version `2`.
 
 1. **Latitude scaling.** Because the grid is equirectangular, cells
    shrink in east-west extent towards the poles. A per-row scaling
@@ -801,8 +1011,9 @@ The following are known gaps or unresolved decisions in version `1`.
 
 ## Appendix A. Worked Example
 
-This appendix gives a complete, valid, synthetic file of 157 bytes and
-walks through several lookups. Its geography is fictional. The file uses
+This appendix gives a complete, valid, synthetic version `1` file of
+157 bytes and walks through several lookups; Section A.6 gives its chunk
+as version `2` stores it. Its geography is fictional. The file uses
 a `2 × 2` chunk grid, a quadtree depth of `2`, and name blocks of `4`
 entries.
 
@@ -963,6 +1174,46 @@ point lies after the last record: **unknown**.
 `c = 1, r = 0`, index `1`. Its length is `0`: **unknown**, with no
 decompression performed.
 
+### A.6. Chunk 3 as Version 2 (11 bytes)
+
+As a version `2` file, the example is 142 bytes: the header's `version`
+byte is `02`, the directory's last chunk length `0b`, and chunk `3` is
+stored as follows ([Section 8.7](#87-version-2-chunks)):
+
+```
+02                        palette_count P = 2
+02 03                     palette deltas 2, 3  -> IDs 2, 5
+07                        record_count M = 7
+00 a5 11 64 ba 00 00      stream
+```
+
+The stream holds these decisions, in order. `W` and `S` are the west
+and south neighbours' records, `L` the move-to-front list before the
+leaf; every probability used is still at `2048` unless it was used
+before.
+
+| Node `(d; x, y)` | Neighbours                  | Decisions                                          | Record |
+|------------------|-----------------------------|----------------------------------------------------|--------|
+| `(0; 0, 0)`      | none                        | `split[0] = 1`                                     | |
+| `(1; 0, 0)`      | none                        | `split[16] = 0`; `L = 0 1 2`, `r = 1`: `rank_length[0] = 1`, `rank_length[1] = 0`, `rank_bits[32] = 0` | 0: depth 1, index 1 |
+| `(1; 2, 0)`      | W record 0                  | `split[20] = 1`                                    | |
+| `(2; 2, 0)`      | W record 0                  | `same_west[73] = 0`; `L = 1 0 2`, skipping `1`, `r = 1`: `rank_length[32] = 1`, `rank_length[33] = 0`, `rank_bits[32] = 0` | 1: depth 2, index 2 |
+| `(2; 3, 0)`      | W record 1                  | `same_west[73] = 0`; `L = 2 1 0`, skipping `2`, `r = 1`: `rank_length[32] = 1`, `rank_length[33] = 0`, `rank_bits[32] = 0` | 2: depth 2, index 0 |
+| `(2; 2, 1)`      | W record 0, S record 1      | `same_west[65] = 0`, `same_south[65] = 0`; `L = 0 2 1`, skipping `1` and `2`, `r = 0`: `rank_length[96] = 0` | 3: depth 2, index 0 |
+| `(2; 3, 1)`      | W record 3, S record 2      | `same_west[93] = 1`                                | 4: depth 2, index 0 |
+| `(1; 0, 2)`      | S record 0                  | `split[17] = 0`; `same_south[32] = 1`              | 5: depth 1, index 1 |
+| `(1; 2, 2)`      | W record 5, S record 3      | `split[22] = 1`                                    | |
+| `(2; 2, 2)`      | W record 5, S record 3      | `same_west[73] = 0`, `same_south[73] = 0`; `L = 1 0 2`, skipping `1` and `0`, `r = 0`: `rank_length[96] = 0` | 6: depth 2, index 2 |
+
+For example, at `(2; 3, 1)` both neighbours have index `0`, as had the
+previous record, so `c = 4 × (8 × 2 + 4 + 2 + 1) + 1 = 93`, and since
+`s = w` no `same_south` decision follows. At `(1; 2, 2)` the south
+neighbour, record 3, is one level deeper than the node (category `2`),
+so `split[16 × 1 + 4 × 1 + 2] = split[22]`. Decoding stops after the
+seventh record, with three nodes of the traversal still unvisited, and
+`pos` is then `7`, the length of the stream. The records are those of
+Section A.4.
+
 ## Appendix B. Text Search Index
 
 This appendix specifies an OPTIONAL sidecar file that speeds up substring
@@ -2341,6 +2592,9 @@ A full-precision cell of this file is `45°` wide and `22.5°` high,
 
 ## Appendix D. References
 
+- [LZMA] Pavlov, I., "LZMA SDK", https://www.7-zip.org/sdk.html: the
+  range coder of `LzmaDec.c`, with `kNumBitModelTotalBits` 11 and
+  `kNumMoveBits` 5 where this format uses 12 and 4.
 - [RFC1950] Deutsch, P. and J-L. Gailly, "ZLIB Compressed Data Format
   Specification version 3.3", May 1996.
 - [RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification

package/dist/chunk.d.ts

@@ -27,8 +27,9 @@ export interface ChunkRecords {
  * (`width_exp`, `height_exp`) instead (binary-format.md Section 10).
  */
 export declare const MAX_SUPPORTED_DEPTH = 16;
-/** Parses one already-inflated chunk block (Section 8.2, or Section 8.6's
- * layout for a `version` 0 file) and checks the cheap Section 8.5
+/** Parses one chunk block: already inflated for a `version` 0 or 1 file
+ * (Section 8.2, or Section 8.6's layout for `version` 0), as stored for
+ * `version` 2, whose blocks are not zlib streams (Section 8.7). It checks the cheap Section 8.5
  * invariants a reader can verify without external context (place ids
  * are validated by the caller, which has the place table). */
 export declare function parseChunk(buf: Uint8Array, maxDepth: number, version: number): ChunkRecords;

package/dist/chunk.js

@@ -1,5 +1,6 @@
 import { decodeChunkCodes, WASM_MIN_RECORDS } from "./chunkDecoder.js";
 import { CGeoDataFormatError } from "./errors.js";
+import { decodeRangeCodedRecords } from "./rangeChunk.js";
 import { readDeltaVarintList, readDeltaVarints32Into, readVarint, } from "./varint.js";
 /**
  * The deepest `max_depth` this reader supports, though the format allows
@@ -75,8 +76,9 @@ function readLittleEndian(buf, offset, width, n) {
 function isMultipleOf(value, span) {
     return value < 0x80000000 ? (value & (span - 1)) === 0 : value % span === 0;
 }
-/** Parses one already-inflated chunk block (Section 8.2, or Section 8.6's
- * layout for a `version` 0 file) and checks the cheap Section 8.5
+/** Parses one chunk block: already inflated for a `version` 0 or 1 file
+ * (Section 8.2, or Section 8.6's layout for `version` 0), as stored for
+ * `version` 2, whose blocks are not zlib streams (Section 8.7). It checks the cheap Section 8.5
  * invariants a reader can verify without external context (place ids
  * are validated by the caller, which has the place table). */
 export function parseChunk(buf, maxDepth, version) {
@@ -90,6 +92,12 @@ export function parseChunk(buf, maxDepth, version) {
     offset += b2;
     const { value: recordCount, bytesRead: b3 } = readVarint(buf, offset);
     offset += b3;
+    if (version >= 2) {
+        return {
+            palette,
+            ...decodeRangeCodedRecords(buf, offset, paletteCount, recordCount, maxDepth),
+        };
+    }
     if (version !== 0) {
         return parseRecords(buf, offset, palette, recordCount, maxDepth);
     }

package/dist/format.d.ts

@@ -15,6 +15,7 @@ export { type Inflater, inflateBlock, inflateCeiling, streamInflater, } from "./
 export { decodeLocateEntry, encodeLocateEntry, type LocateCell, type LocateGrid, locateEntryWidth, } from "./locateIndex.ts";
 export { lowBits, mortonDecode, mortonEncode, shiftRight } from "./morton.ts";
 export { type PlaceTableInfo, parsePlaceTable, type ResolvedPlace, readNameBlock, readNameEntry, resolveIdentifier, } from "./placeTable.ts";
+export { decodeRangeCodedRecords, encodeRangeCodedRecords, type RangeCodedRecords, } from "./rangeChunk.ts";
 export { containsTokenRun, EXACT_SPELLING_BONUS, foldDiacritics, LEVEL_BONUS, type Needle, nameMatchQuality, placeMatchQuality, prepareNeedle, QUALITY_EXACT, QUALITY_MAX, QUALITY_SUBSTRING, QUALITY_WHOLE_WORD, tokenize, tokenizeFolded, } from "./relevance.ts";
 export { fnv1a32, foldSymbols, trigramListIndex, trigramListIndices, } from "./searchIndex.ts";
 export { toSource } from "./source.ts";

package/dist/format.js

@@ -15,6 +15,7 @@ export { inflateBlock, inflateCeiling, streamInflater, } from "./inflate.js";
 export { decodeLocateEntry, encodeLocateEntry, locateEntryWidth, } from "./locateIndex.js";
 export { lowBits, mortonDecode, mortonEncode, shiftRight } from "./morton.js";
 export { parsePlaceTable, readNameBlock, readNameEntry, resolveIdentifier, } from "./placeTable.js";
+export { decodeRangeCodedRecords, encodeRangeCodedRecords, } from "./rangeChunk.js";
 export { containsTokenRun, EXACT_SPELLING_BONUS, foldDiacritics, LEVEL_BONUS, nameMatchQuality, placeMatchQuality, prepareNeedle, QUALITY_EXACT, QUALITY_MAX, QUALITY_SUBSTRING, QUALITY_WHOLE_WORD, tokenize, tokenizeFolded, } from "./relevance.js";
 export { fnv1a32, foldSymbols, trigramListIndex, trigramListIndices, } from "./searchIndex.js";
 export { toSource } from "./source.js";

package/dist/header.d.ts

@@ -1,7 +1,8 @@
 /** A file's header (Section 5). */
 export interface CGeoDataHeader {
-    /** Format version: 1, or 0 for files written before chunks dropped
-     * their codes (Section 8.6). This reader reads both. */
+    /** Format version: 2 (range-coded chunks, Section 8.7), 1 (zlib
+     * chunks), or 0 for files written before chunks dropped their codes
+     * (Section 8.6). This reader reads all three. */
     version: number;
     /** The data's attribution notice, as given when the file was built.
      * Show it wherever results are shown. */

package/dist/header.js

@@ -21,7 +21,7 @@ export function parseHeader(buf) {
     let offset = SIGNATURE.length;
     const version = readVarint(buf, offset);
     offset += version.bytesRead;
-    if (version.value !== 0 && version.value !== 1) {
+    if (version.value > 2) {
         throw new CGeoDataVersionError(`unsupported CGEODATA version ${version.value}`, version.value);
     }
     const attrLen = readVarint(buf, offset);

package/dist/rangeChunk.d.ts

@@ -0,0 +1,46 @@
+import type { ChunkRecords } from "./chunk.ts";
+/**
+ * A version 2 chunk's records (binary-format.md Section 8.7): the
+ * quadtree's split flags and each leaf's palette index, coded with an
+ * adaptive binary range coder whose contexts come from the west and south
+ * neighbours of each node. This is the JavaScript decoder, the one that
+ * reports errors; `rangeDecoder.ts`'s WebAssembly one only accepts or
+ * declines, and a declined block is decoded again here.
+ */
+/** A chunk's records as `decodeRangeCodedRecords` decodes them: all of
+ * `ChunkRecords` but the palette, which precedes the stream. */
+export type RangeCodedRecords = Omit<ChunkRecords, "palette">;
+/** Context table sizes (Section 8.7.3). */
+export declare const SPLIT_CONTEXTS: number;
+export declare const VALUE_CONTEXTS: number;
+export declare const RANK_LENGTH_CONTEXTS: number;
+export declare const RANK_BIT_CONTEXTS: number;
+/** A version 2 chunk's palette_count is below this, so a palette index
+ * and a depth fit in one 32-bit integer together. */
+export declare const MAX_PALETTE_COUNT: number;
+/** At most this many records per byte of coded stream: every leaf takes at
+ * least one binary decision, which costs at least log2(4096 / 4081) bits,
+ * so 2048 records per byte can't be reached by any stream. Checked before
+ * allocating for `record_count`. */
+export declare const MAX_RECORDS_PER_BYTE = 2048;
+/** The narrowest typed array for palette indices up to `paletteCount`
+ * (Section 8.3's S, with 3 bytes rounded up to 4). */
+export declare function paletteIndexArray(paletteCount: number, n: number): Uint8Array | Uint16Array | Uint32Array;
+/**
+ * Decodes the records of a version 2 chunk from `buf[offset..]`, the
+ * range-coded stream that follows `record_count` and runs to the end of
+ * the block. Throws a `CGeoDataFormatError` for anything Section 8.7
+ * rejects.
+ */
+export declare function decodeRangeCodedRecords(buf: Uint8Array, offset: number, paletteCount: number, recordCount: number, maxDepth: number): RangeCodedRecords;
+/** Section 8.7.5's bounds on `record_count`, checked before allocating
+ * for it: at most one record per cell, and at most `MAX_RECORDS_PER_BYTE`
+ * per byte of stream. */
+export declare function checkRecordCount(recordCount: number, maxDepth: number, streamBytes: number): void;
+/**
+ * Encodes records that tile a chunk from code 0 (Section 8.4) as Section
+ * 8.7's range-coded stream: `depths[j]` and `paletteIndices[j]` (0..P)
+ * per record, in Morton order. The inverse of `decodeRangeCodedRecords`.
+ * Throws a RangeError for records that don't tile the chunk.
+ */
+export declare function encodeRangeCodedRecords(depths: ArrayLike<number>, paletteIndices: ArrayLike<number>, paletteCount: number, maxDepth: number): Uint8Array;

package/dist/rangeChunk.js

@@ -0,0 +1,401 @@
+import { CGeoDataFormatError } from "./errors.js";
+import { decodeRangeCodedWasm } from "./rangeDecoder.js";
+/** Probabilities are 12-bit: the chance of a 0, out of 4096. */
+const PROB_BITS = 12;
+const PROB_INIT = 2048;
+/** How fast a probability adapts: it moves 1/16 of the way each time. */
+const ADAPT_SHIFT = 4;
+const TOP = 2 ** 24;
+/** Context table sizes (Section 8.7.3). */
+export const SPLIT_CONTEXTS = 16 * 16;
+export const VALUE_CONTEXTS = 17 * 8 * 4;
+export const RANK_LENGTH_CONTEXTS = 4 * 32;
+export const RANK_BIT_CONTEXTS = 32 * 32;
+/** A version 2 chunk's palette_count is below this, so a palette index
+ * and a depth fit in one 32-bit integer together. */
+export const MAX_PALETTE_COUNT = 2 ** 26;
+/** At most this many records per byte of coded stream: every leaf takes at
+ * least one binary decision, which costs at least log2(4096 / 4081) bits,
+ * so 2048 records per byte can't be reached by any stream. Checked before
+ * allocating for `record_count`. */
+export const MAX_RECORDS_PER_BYTE = 2048;
+/** Section 8.7.2's neighbour category: 0 when there is no neighbour, else
+ * how much deeper than `d` its record is (1: not deeper, 2: one level, 3:
+ * more). `tag` is the neighbour's `index << 5 | depth`, or -1. */
+function category(tag, d) {
+    if (tag < 0)
+        return 0;
+    const e = tag & 31;
+    return e <= d ? 1 : e === d + 1 ? 2 : 3;
+}
+class RangeDecoder {
+    range = 0xffffffff;
+    code = 0;
+    pos;
+    buf;
+    end;
+    constructor(buf, start, end) {
+        this.buf = buf;
+        this.end = end;
+        if (end - start < 5) {
+            throw new CGeoDataFormatError("chunk range-coded stream is shorter than 5 bytes");
+        }
+        if (buf[start] !== 0) {
+            throw new CGeoDataFormatError("chunk range-coded stream does not start with a 0 byte");
+        }
+        let code = 0;
+        for (let i = 1; i <= 4; i++)
+            code = code * 256 + buf[start + i];
+        if (code === 0xffffffff) {
+            throw new CGeoDataFormatError("chunk range-coded stream starts with code 0xffffffff");
+        }
+        this.code = code;
+        this.pos = start + 5;
+    }
+    bit(probs, i) {
+        const p = probs[i];
+        const bound = (this.range >>> PROB_BITS) * p;
+        let b;
+        if (this.code < bound) {
+            this.range = bound;
+            probs[i] = p + (((1 << PROB_BITS) - p) >> ADAPT_SHIFT);
+            b = 0;
+        }
+        else {
+            this.code -= bound;
+            this.range -= bound;
+            probs[i] = p - (p >> ADAPT_SHIFT);
+            b = 1;
+        }
+        while (this.range < TOP) {
+            if (this.pos >= this.end) {
+                throw new CGeoDataFormatError("chunk range-coded stream runs past end of block");
+            }
+            this.range *= 256;
+            this.code = this.code * 256 + this.buf[this.pos++];
+        }
+        return b;
+    }
+}
+/** The narrowest typed array for palette indices up to `paletteCount`
+ * (Section 8.3's S, with 3 bytes rounded up to 4). */
+export function paletteIndexArray(paletteCount, n) {
+    if (paletteCount <= 255)
+        return new Uint8Array(n);
+    if (paletteCount <= 65535)
+        return new Uint16Array(n);
+    return new Uint32Array(n);
+}
+/**
+ * Decodes the records of a version 2 chunk from `buf[offset..]`, the
+ * range-coded stream that follows `record_count` and runs to the end of
+ * the block. Throws a `CGeoDataFormatError` for anything Section 8.7
+ * rejects.
+ */
+export function decodeRangeCodedRecords(buf, offset, paletteCount, recordCount, maxDepth) {
+    if (paletteCount >= MAX_PALETTE_COUNT) {
+        throw new CGeoDataFormatError(`chunk palette_count ${paletteCount} is 2^26 or more, past what a version 2 chunk allows`);
+    }
+    checkRecordCount(recordCount, maxDepth, buf.length - offset);
+    const fast = decodeRangeCodedWasm(buf, offset, paletteCount, recordCount, maxDepth);
+    if (fast)
+        return fast;
+    const rc = new RangeDecoder(buf, offset, buf.length);
+    const D = maxDepth;
+    const N = 2 ** D;
+    const split = new Uint16Array(SPLIT_CONTEXTS).fill(PROB_INIT);
+    const eqW = new Uint16Array(VALUE_CONTEXTS).fill(PROB_INIT);
+    const eqS = new Uint16Array(VALUE_CONTEXTS).fill(PROB_INIT);
+    const rankLength = new Uint16Array(RANK_LENGTH_CONTEXTS).fill(PROB_INIT);
+    const rankBits = new Uint16Array(RANK_BIT_CONTEXTS).fill(PROB_INIT);
+    // row[y] / col[x]: the last record decoded in row y / column x, as
+    // `index << 5 | depth`; in Morton order that is the cell west of /
+    // south of the next one to be decoded there.
+    const row = new Int32Array(N).fill(-1);
+    const col = new Int32Array(N).fill(-1);
+    // Move-to-front list of palette indices 0..P.
+    const mtf = new Int32Array(paletteCount + 1);
+    for (let k = 0; k <= paletteCount; k++)
+        mtf[k] = k;
+    const codes = new Uint32Array(recordCount);
+    const depths = new Uint8Array(recordCount);
+    const paletteIndices = paletteIndexArray(paletteCount, recordCount);
+    // Pending nodes, popped in pre-order: at most 3 per level plus one.
+    const stackD = new Int32Array(4 * (D + 1));
+    const stackX = new Int32Array(4 * (D + 1));
+    const stackY = new Int32Array(4 * (D + 1));
+    let sp = 1;
+    let j = 0;
+    let prev = -1;
+    let code = 0;
+    while (j < recordCount) {
+        if (sp === 0) {
+            throw new CGeoDataFormatError(`chunk record_count ${recordCount} is more than the ${j} records its quadtree has`);
+        }
+        sp--;
+        const d = stackD[sp];
+        const x = stackX[sp];
+        const y = stackY[sp];
+        const w = x > 0 ? row[y] : -1;
+        const s = y > 0 ? col[x] : -1;
+        const wc = category(w, d);
+        if (d < D && rc.bit(split, d * 16 + wc * 4 + category(s, d)) === 1) {
+            const h = 2 ** (D - d - 1);
+            const d1 = d + 1;
+            // Pushed in reverse, so the south-west child pops first.
+            stackD[sp] = d1;
+            stackX[sp] = x + h;
+            stackY[sp] = y + h;
+            sp++;
+            stackD[sp] = d1;
+            stackX[sp] = x;
+            stackY[sp] = y + h;
+            sp++;
+            stackD[sp] = d1;
+            stackX[sp] = x + h;
+            stackY[sp] = y;
+            sp++;
+            stackD[sp] = d1;
+            stackX[sp] = x;
+            stackY[sp] = y;
+            sp++;
+            continue;
+        }
+        const wv = w < 0 ? -1 : w >>> 5;
+        const sv = s < 0 ? -1 : s >>> 5;
+        const c = (d * 8 +
+            (wv === sv ? 4 : 0) +
+            (prev === wv ? 2 : 0) +
+            (prev === sv ? 1 : 0)) *
+            4 +
+            wc;
+        let v = -1;
+        let p = 0;
+        if (wv >= 0 && rc.bit(eqW, c) === 1)
+            v = wv;
+        else if (sv >= 0 && sv !== wv && rc.bit(eqS, c) === 1)
+            v = sv;
+        if (v < 0) {
+            const cc = (wv >= 0 ? 32 : 0) + (sv >= 0 ? 64 : 0);
+            let nb = 0;
+            while (rc.bit(rankLength, cc + nb) === 1) {
+                if (++nb === 32) {
+                    throw new CGeoDataFormatError(`chunk record ${j}'s palette rank is 2^32 or more`);
+                }
+            }
+            let value = 1;
+            for (let k = nb - 1; k >= 0; k--) {
+                value = value * 2 + rc.bit(rankBits, nb * 32 + k);
+            }
+            let rank = value - 1;
+            for (;; p++) {
+                if (p > paletteCount) {
+                    throw new CGeoDataFormatError(`chunk record ${j}'s palette rank ${value - 1} is past the palette`);
+                }
+                const t = mtf[p];
+                if (t === wv || t === sv)
+                    continue;
+                if (rank === 0)
+                    break;
+                rank--;
+            }
+            v = mtf[p];
+        }
+        else {
+            while (mtf[p] !== v)
+                p++;
+        }
+        for (let k = p; k > 0; k--)
+            mtf[k] = mtf[k - 1];
+        mtf[0] = v;
+        const size = 2 ** (D - d);
+        const tag = v * 32 + d;
+        row.fill(tag, y, y + size);
+        col.fill(tag, x, x + size);
+        codes[j] = code;
+        depths[j] = d;
+        paletteIndices[j] = v;
+        code += size * size;
+        prev = v;
+        j++;
+    }
+    if (rc.pos !== rc.end) {
+        throw new CGeoDataFormatError(`chunk block has ${rc.end - rc.pos} trailing bytes after its range-coded stream`);
+    }
+    return { codes, depths, paletteIndices };
+}
+/** Section 8.7.5's bounds on `record_count`, checked before allocating
+ * for it: at most one record per cell, and at most `MAX_RECORDS_PER_BYTE`
+ * per byte of stream. */
+export function checkRecordCount(recordCount, maxDepth, streamBytes) {
+    if (recordCount > 4 ** maxDepth) {
+        throw new CGeoDataFormatError(`chunk record_count ${recordCount} is more than the 4^${maxDepth} cells of a chunk`);
+    }
+    if (recordCount > MAX_RECORDS_PER_BYTE * Math.max(streamBytes, 0)) {
+        throw new CGeoDataFormatError(`chunk record_count ${recordCount} is more than a ${streamBytes}-byte stream can code`);
+    }
+}
+class RangeEncoder {
+    low = 0;
+    range = 0xffffffff;
+    cache = 0;
+    cacheSize = 1;
+    out = [];
+    bit(probs, i, b) {
+        const p = probs[i];
+        const bound = (this.range >>> PROB_BITS) * p;
+        if (b === 0) {
+            this.range = bound;
+            probs[i] = p + (((1 << PROB_BITS) - p) >> ADAPT_SHIFT);
+        }
+        else {
+            this.low += bound;
+            this.range -= bound;
+            probs[i] = p - (p >> ADAPT_SHIFT);
+        }
+        while (this.range < TOP) {
+            this.range *= 256;
+            this.shiftLow();
+        }
+    }
+    /** Emits the top byte of `low`, holding back runs of 0xff until a
+     * carry out of them is ruled out (LZMA's encoder). */
+    shiftLow() {
+        if (this.low < 0xff000000 || this.low >= 0x100000000) {
+            const carry = this.low >= 0x100000000 ? 1 : 0;
+            let c = this.cache;
+            do {
+                this.out.push((c + carry) & 0xff);
+                c = 0xff;
+            } while (--this.cacheSize !== 0);
+            this.cache = Math.floor(this.low / 0x1000000) & 0xff;
+        }
+        this.cacheSize++;
+        this.low = (this.low % 0x1000000) * 256;
+    }
+    finish() {
+        for (let i = 0; i < 5; i++)
+            this.shiftLow();
+        return Uint8Array.from(this.out);
+    }
+}
+/**
+ * Encodes records that tile a chunk from code 0 (Section 8.4) as Section
+ * 8.7's range-coded stream: `depths[j]` and `paletteIndices[j]` (0..P)
+ * per record, in Morton order. The inverse of `decodeRangeCodedRecords`.
+ * Throws a RangeError for records that don't tile the chunk.
+ */
+export function encodeRangeCodedRecords(depths, paletteIndices, paletteCount, maxDepth) {
+    if (paletteCount >= MAX_PALETTE_COUNT) {
+        throw new RangeError(`palette_count ${paletteCount} is 2^26 or more, past what a version 2 chunk allows`);
+    }
+    const rc = new RangeEncoder();
+    const D = maxDepth;
+    const N = 2 ** D;
+    const recordCount = depths.length;
+    const split = new Uint16Array(SPLIT_CONTEXTS).fill(PROB_INIT);
+    const eqW = new Uint16Array(VALUE_CONTEXTS).fill(PROB_INIT);
+    const eqS = new Uint16Array(VALUE_CONTEXTS).fill(PROB_INIT);
+    const rankLength = new Uint16Array(RANK_LENGTH_CONTEXTS).fill(PROB_INIT);
+    const rankBits = new Uint16Array(RANK_BIT_CONTEXTS).fill(PROB_INIT);
+    const row = new Int32Array(N).fill(-1);
+    const col = new Int32Array(N).fill(-1);
+    const mtf = new Int32Array(paletteCount + 1);
+    for (let k = 0; k <= paletteCount; k++)
+        mtf[k] = k;
+    const stackD = new Int32Array(4 * (D + 1));
+    const stackX = new Int32Array(4 * (D + 1));
+    const stackY = new Int32Array(4 * (D + 1));
+    let sp = 1;
+    let j = 0;
+    let prev = -1;
+    while (j < recordCount) {
+        if (sp === 0)
+            throw new RangeError("records run past the end of the chunk");
+        sp--;
+        const d = stackD[sp];
+        const x = stackX[sp];
+        const y = stackY[sp];
+        const depth = depths[j];
+        if (depth < d || depth > D) {
+            throw new RangeError(`record ${j} at depth ${depth} doesn't tile the chunk`);
+        }
+        const w = x > 0 ? row[y] : -1;
+        const s = y > 0 ? col[x] : -1;
+        const wc = category(w, d);
+        if (d < D) {
+            const splits = depth > d ? 1 : 0;
+            rc.bit(split, d * 16 + wc * 4 + category(s, d), splits);
+            if (splits) {
+                const h = 2 ** (D - d - 1);
+                const d1 = d + 1;
+                stackD[sp] = d1;
+                stackX[sp] = x + h;
+                stackY[sp] = y + h;
+                sp++;
+                stackD[sp] = d1;
+                stackX[sp] = x;
+                stackY[sp] = y + h;
+                sp++;
+                stackD[sp] = d1;
+                stackX[sp] = x + h;
+                stackY[sp] = y;
+                sp++;
+                stackD[sp] = d1;
+                stackX[sp] = x;
+                stackY[sp] = y;
+                sp++;
+                continue;
+            }
+        }
+        const v = paletteIndices[j];
+        if (!(v >= 0 && v <= paletteCount)) {
+            throw new RangeError(`record ${j}'s palette index ${v} is past P (${paletteCount})`);
+        }
+        const wv = w < 0 ? -1 : w >>> 5;
+        const sv = s < 0 ? -1 : s >>> 5;
+        const c = (d * 8 +
+            (wv === sv ? 4 : 0) +
+            (prev === wv ? 2 : 0) +
+            (prev === sv ? 1 : 0)) *
+            4 +
+            wc;
+        let coded = false;
+        if (wv >= 0) {
+            rc.bit(eqW, c, v === wv ? 1 : 0);
+            coded = v === wv;
+        }
+        if (!coded && sv >= 0 && sv !== wv) {
+            rc.bit(eqS, c, v === sv ? 1 : 0);
+            coded = v === sv;
+        }
+        let p = 0;
+        while (mtf[p] !== v)
+            p++;
+        if (!coded) {
+            let rank = 0;
+            for (let k = 0; k < p; k++)
+                if (mtf[k] !== wv && mtf[k] !== sv)
+                    rank++;
+            const value = rank + 1;
+            const nb = 31 - Math.clz32(value);
+            const cc = (wv >= 0 ? 32 : 0) + (sv >= 0 ? 64 : 0);
+            for (let k = 0; k < nb; k++)
+                rc.bit(rankLength, cc + k, 1);
+            rc.bit(rankLength, cc + nb, 0);
+            for (let k = nb - 1; k >= 0; k--)
+                rc.bit(rankBits, nb * 32 + k, (value >>> k) & 1);
+        }
+        for (let k = p; k > 0; k--)
+            mtf[k] = mtf[k - 1];
+        mtf[0] = v;
+        const size = 2 ** (D - d);
+        const tag = v * 32 + d;
+        row.fill(tag, y, y + size);
+        col.fill(tag, x, x + size);
+        prev = v;
+        j++;
+    }
+    return rc.finish();
+}
+//# sourceMappingURL=rangeChunk.js.map

package/dist/rangeDecoder.d.ts

@@ -0,0 +1,17 @@
+import type { RangeCodedRecords } from "./rangeChunk.ts";
+/** Turns the fast path off (or back on), for tests and benchmarks that
+ * compare it against the JavaScript decoder. */
+export declare function useWasmRangeDecoder(on: boolean): void;
+/** How many bytes of WebAssembly memory the fast path holds right now:
+ * 0 while idle. */
+export declare function wasmRangeDecoderBytes(): number;
+/** Whether this platform runs the fast path. */
+export declare function wasmRangeDecoderAvailable(): boolean;
+/**
+ * The records of the range-coded stream `buf[offset..]` (Section 8.7), or
+ * undefined when the fast path declined it or isn't available, and the
+ * caller should decode it in JavaScript. The caller has checked Section
+ * 8.7.5's bounds on `recordCount`, that `maxDepth` is at most 16 and
+ * `paletteCount` below 2^26.
+ */
+export declare function decodeRangeCodedWasm(buf: Uint8Array, offset: number, paletteCount: number, recordCount: number, maxDepth: number): RangeCodedRecords | undefined;

package/dist/rangeDecoder.js

@@ -0,0 +1,129 @@
+import { RANGE_DECODER_WASM } from "./rangeDecoderWasm.js";
+/**
+ * `decodeRangeCodedRecords`'s fast path (wasm/rangeDecoder.wat), about
+ * twice as fast as the JavaScript decoder in ./rangeChunk.ts. It only
+ * accepts or declines a stream: a declined one -- malformed, or valid
+ * but out of its reach -- is decoded again in JavaScript, which throws
+ * the error or returns the records. Platforms without WebAssembly (or
+ * that forbid compiling it, such as a page whose CSP lacks
+ * `'wasm-unsafe-eval'`) always take the JavaScript path.
+ *
+ * Memory: one instance, grown to the largest chunk decoded so far and
+ * dropped once no decode has used it for `IDLE_RELEASE_MS`, as
+ * ./chunkDecoder.ts does for version 0 chunks.
+ */
+/** How long the instance outlives its last decode. */
+const IDLE_RELEASE_MS = 1000;
+const PAGE_BYTES = 65536;
+/** Probabilities and node stack (wasm/rangeDecoder.wat's `scratch`). */
+const FIXED_SCRATCH_BYTES = 4992 + 816;
+/** The compiled module; null once compiling failed. */
+let compiled;
+let instance;
+let releaseTimer;
+let enabled = true;
+/** Turns the fast path off (or back on), for tests and benchmarks that
+ * compare it against the JavaScript decoder. */
+export function useWasmRangeDecoder(on) {
+    enabled = on;
+    if (!on)
+        release();
+}
+/** How many bytes of WebAssembly memory the fast path holds right now:
+ * 0 while idle. */
+export function wasmRangeDecoderBytes() {
+    return instance?.memory.buffer.byteLength ?? 0;
+}
+/** Whether this platform runs the fast path. */
+export function wasmRangeDecoderAvailable() {
+    return enabled && getInstance() !== undefined;
+}
+function getInstance() {
+    if (instance)
+        return instance;
+    if (compiled === undefined) {
+        try {
+            // Throws a ReferenceError where there's no WebAssembly at all.
+            compiled = new WebAssembly.Module(Uint8Array.from(atob(RANGE_DECODER_WASM), (c) => c.charCodeAt(0)));
+        }
+        catch {
+            compiled = null;
+        }
+    }
+    if (compiled === null)
+        return undefined;
+    const exports = new WebAssembly.Instance(compiled).exports;
+    instance = {
+        memory: exports.memory,
+        decode: exports.decode,
+    };
+    return instance;
+}
+function release() {
+    instance = undefined;
+    if (releaseTimer !== undefined)
+        clearTimeout(releaseTimer);
+    releaseTimer = undefined;
+}
+function releaseWhenIdle() {
+    if (releaseTimer !== undefined)
+        clearTimeout(releaseTimer);
+    releaseTimer = setTimeout(release, IDLE_RELEASE_MS);
+    // Node.js: don't keep the process alive just to free this.
+    releaseTimer.unref?.();
+}
+/**
+ * The records of the range-coded stream `buf[offset..]` (Section 8.7), or
+ * undefined when the fast path declined it or isn't available, and the
+ * caller should decode it in JavaScript. The caller has checked Section
+ * 8.7.5's bounds on `recordCount`, that `maxDepth` is at most 16 and
+ * `paletteCount` below 2^26.
+ */
+export function decodeRangeCodedWasm(buf, offset, paletteCount, recordCount, maxDepth) {
+    if (!enabled)
+        return undefined;
+    const decoder = getInstance();
+    if (!decoder)
+        return undefined;
+    const width = paletteCount <= 255 ? 1 : paletteCount <= 65535 ? 2 : 4;
+    const streamBytes = buf.length - offset;
+    // Slack past the stream: a stream that runs past its end reads a few
+    // bytes there before the fast path declines it.
+    const out = Math.ceil((streamBytes + 64) / 16) * 16;
+    const indices = out + 4 * recordCount;
+    const depthsAt = indices + width * recordCount;
+    const scratch = Math.ceil((depthsAt + recordCount) / 16) * 16;
+    const needed = scratch + FIXED_SCRATCH_BYTES + 4 * (2 * 2 ** maxDepth + paletteCount + 1);
+    const { memory } = decoder;
+    if (memory.buffer.byteLength < needed) {
+        try {
+            memory.grow(Math.ceil((needed - memory.buffer.byteLength) / PAGE_BYTES));
+        }
+        catch {
+            return undefined; // out of memory: leave it to the JavaScript path
+        }
+    }
+    new Uint8Array(memory.buffer, 0, streamBytes).set(buf.subarray(offset));
+    let ok;
+    try {
+        ok = decoder.decode(streamBytes, recordCount, paletteCount, maxDepth, width, out, scratch);
+    }
+    catch {
+        // A stream that runs far past its end can read past the memory.
+        ok = 0;
+    }
+    releaseWhenIdle();
+    if (!ok)
+        return undefined;
+    const { buffer } = memory;
+    return {
+        codes: new Uint32Array(buffer, out, recordCount).slice(),
+        depths: new Uint8Array(buffer, depthsAt, recordCount).slice(),
+        paletteIndices: width === 1
+            ? new Uint8Array(buffer, indices, recordCount).slice()
+            : width === 2
+                ? new Uint16Array(buffer, indices, recordCount).slice()
+                : new Uint32Array(buffer, indices, recordCount).slice(),
+    };
+}
+//# sourceMappingURL=rangeDecoder.js.map

package/dist/rangeDecoderWasm.d.ts

@@ -0,0 +1,2 @@
+/** rangeDecoder.wat's compiled module (1381 bytes), base64. */
+export declare const RANGE_DECODER_WASM = "AGFzbQEAAAABFwNgAX8Bf2ACf38Bf2AHf39/f39/fwF/AwQDAAECBQMBAAEGFQR/AUEAC38BQQALfwFBAAt/AUEACwcTAgZtZW1vcnkCAAZkZWNvZGUAAgqKCgOBAQEDfyAALwEAIQEjAEEMdiABbCECIwEgAkkEQCACJAAgACABQYAgIAFrQQR2ajsBAAUjASACayQBIwAgAmskACAAIAEgAUEEdms7AQBBASEDCwJAA0AjAEGAgIAITw0BIwFBCHQjAi0AAHIkASMCQQFqJAIjAEEIdCQADAALCyADCzEBAX8gAEEASARAQQAPCyAAQR9xIQIgAiABTQRAQQEPCyACIAFBAWpGBEBBAg8LQQML0ggBJH9BAC0AAARAQQAPC0EBLQAAQRh0QQItAABBEHRyQQMtAABBCHRBBC0AAHJyJAFBfyQAQQUkAiAAJAMgBiEHIAZBgARqIQggBkHADGohCSAGQYAVaiEKIAZBgBdqIQsgBkGAJ2ohDCAMQbAGaiENIA1BBCADdGohDiAOQQQgA3RqIQ8gBSABQQJ0aiERIBEgASAEbGohEEEAIRIDQCAGIBJqQYAQOwEAIBJBAmohEiASQYAnSQ0ACyANQf8BQQggA3T8CwBBACESA0AgDyASQQJ0aiASNgIAIBJBAWohEiASIAJNDQALIAxCADcDACAMQQA2AghBASETQX8hFQJAA0AgFCABTw0BIBNFBEBBAA8LIBNBAWshEyAMIBNBDGxqIRsgGygCACEXIBsoAgQhGCAbKAIIIRkgGAR/IA0gGUECdGooAgAFQX8LIRwgGQR/IA4gGEECdGooAgAFQX8LIR0gHCAXEAEhHiAXIANJBEAgByAXQQR0IB5BAnRqIB0gFxABakEBdGoQAARAQQEgAyAXa0EBa3QhGiAXQQFqIRcgGyAXNgIAIBsgGCAaajYCBCAbIBkgGmo2AgggGyAXNgIMIBsgGDYCECAbIBkgGmo2AhQgGyAXNgIYIBsgGCAaajYCHCAbIBk2AiAgGyAXNgIkIBsgGDYCKCAbIBk2AiwgE0EEaiETDAILC0F/IBxBBXYgHEEASBshH0F/IB1BBXYgHUEASBshICAXQQN0IB8gIEZBAnRqIBUgH0ZBAXRqIBUgIEZqQQJ0IB5qISFBfyEiIB9BAE4EQCAIICFBAXRqEAAEQCAfISILCyAiQQBIICBBAE4gICAfR3FxBEAgCSAhQQF0ahAABEAgICEiCwtBACEjICJBAEgEQEEgQQAgH0EAThtBwABBACAgQQBOG2ohJEEAISUCQANAIAogJCAlakEBdGoQAEUNASAlQQFqISUgJUEgRgRAQQAPCwwACwtBASEmICUhEgJAA0AgEkUNASASQQFrIRIgJkEBdCALICVBBXQgEmpBAXRqEABqISYMAAsLICZBAWshJwJAA0AgIyACSwRAQQAPCyAPICNBAnRqKAIAISggKCAfRyAoICBHcQRAICdFDQIgJ0EBayEnCyAjQQFqISMMAAsLICghIgUDQCAPICNBAnRqKAIAICJHBEAgI0EBaiEjDAELCwsgIwRAIA9BBGogDyAjQQJ0/AoAACAPICI2AgALQQEgAyAXa3QhKSAiQQV0IBdyISpBACESA0AgDSAZIBJqQQJ0aiAqNgIAIA4gGCASakECdGogKjYCACASQQFqIRIgEiApSQ0ACyAFIBRBAnRqIBY2AgAgECAUaiAXOgAAIARBAUYEQCARIBRqICI6AAAFIARBAkYEQCARIBRBAXRqICI7AQAFIBEgFEECdGogIjYCAAsLIBYgKSApbGohFiAiIRUgFEEBaiEUDAALCyMCIwNGCw==";

package/dist/rangeDecoderWasm.js

@@ -0,0 +1,4 @@
+// Generated by `npm run build:wasm -w reader` from wasm/rangeDecoder.wat; do not edit.
+/** rangeDecoder.wat's compiled module (1381 bytes), base64. */
+export const RANGE_DECODER_WASM = "AGFzbQEAAAABFwNgAX8Bf2ACf38Bf2AHf39/f39/fwF/AwQDAAECBQMBAAEGFQR/AUEAC38BQQALfwFBAAt/AUEACwcTAgZtZW1vcnkCAAZkZWNvZGUAAgqKCgOBAQEDfyAALwEAIQEjAEEMdiABbCECIwEgAkkEQCACJAAgACABQYAgIAFrQQR2ajsBAAUjASACayQBIwAgAmskACAAIAEgAUEEdms7AQBBASEDCwJAA0AjAEGAgIAITw0BIwFBCHQjAi0AAHIkASMCQQFqJAIjAEEIdCQADAALCyADCzEBAX8gAEEASARAQQAPCyAAQR9xIQIgAiABTQRAQQEPCyACIAFBAWpGBEBBAg8LQQML0ggBJH9BAC0AAARAQQAPC0EBLQAAQRh0QQItAABBEHRyQQMtAABBCHRBBC0AAHJyJAFBfyQAQQUkAiAAJAMgBiEHIAZBgARqIQggBkHADGohCSAGQYAVaiEKIAZBgBdqIQsgBkGAJ2ohDCAMQbAGaiENIA1BBCADdGohDiAOQQQgA3RqIQ8gBSABQQJ0aiERIBEgASAEbGohEEEAIRIDQCAGIBJqQYAQOwEAIBJBAmohEiASQYAnSQ0ACyANQf8BQQggA3T8CwBBACESA0AgDyASQQJ0aiASNgIAIBJBAWohEiASIAJNDQALIAxCADcDACAMQQA2AghBASETQX8hFQJAA0AgFCABTw0BIBNFBEBBAA8LIBNBAWshEyAMIBNBDGxqIRsgGygCACEXIBsoAgQhGCAbKAIIIRkgGAR/IA0gGUECdGooAgAFQX8LIRwgGQR/IA4gGEECdGooAgAFQX8LIR0gHCAXEAEhHiAXIANJBEAgByAXQQR0IB5BAnRqIB0gFxABakEBdGoQAARAQQEgAyAXa0EBa3QhGiAXQQFqIRcgGyAXNgIAIBsgGCAaajYCBCAbIBkgGmo2AgggGyAXNgIMIBsgGDYCECAbIBkgGmo2AhQgGyAXNgIYIBsgGCAaajYCHCAbIBk2AiAgGyAXNgIkIBsgGDYCKCAbIBk2AiwgE0EEaiETDAILC0F/IBxBBXYgHEEASBshH0F/IB1BBXYgHUEASBshICAXQQN0IB8gIEZBAnRqIBUgH0ZBAXRqIBUgIEZqQQJ0IB5qISFBfyEiIB9BAE4EQCAIICFBAXRqEAAEQCAfISILCyAiQQBIICBBAE4gICAfR3FxBEAgCSAhQQF0ahAABEAgICEiCwtBACEjICJBAEgEQEEgQQAgH0EAThtBwABBACAgQQBOG2ohJEEAISUCQANAIAogJCAlakEBdGoQAEUNASAlQQFqISUgJUEgRgRAQQAPCwwACwtBASEmICUhEgJAA0AgEkUNASASQQFrIRIgJkEBdCALICVBBXQgEmpBAXRqEABqISYMAAsLICZBAWshJwJAA0AgIyACSwRAQQAPCyAPICNBAnRqKAIAISggKCAfRyAoICBHcQRAICdFDQIgJ0EBayEnCyAjQQFqISMMAAsLICghIgUDQCAPICNBAnRqKAIAICJHBEAgI0EBaiEjDAELCwsgIwRAIA9BBGogDyAjQQJ0/AoAACAPICI2AgALQQEgAyAXa3QhKSAiQQV0IBdyISpBACESA0AgDSAZIBJqQQJ0aiAqNgIAIA4gGCASakECdGogKjYCACASQQFqIRIgEiApSQ0ACyAFIBRBAnRqIBY2AgAgECAUaiAXOgAAIARBAUYEQCARIBRqICI6AAAFIARBAkYEQCARIBRBAXRqICI7AQAFIBEgFEECdGogIjYCAAsLIBYgKSApbGohFiAiIRUgFEEBaiEUDAALCyMCIwNGCw==";
+//# sourceMappingURL=rangeDecoderWasm.js.map

package/dist/reader.js

@@ -929,7 +929,11 @@ export class CGeoDataReader {
     }
     async decodeChunk(index) {
         const compressed = await this.source.read(this.directory.chunkOffsets[index], this.directory.chunkLengths[index]);
-        return this.decodes.run(async () => this.checkPalette(index, parseChunk(await inflateBlock(this.inflater, compressed), this.header.maxDepth, this.header.version)));
+        return this.decodes.run(async () => this.checkPalette(index, parseChunk(
+        // Version 2 chunks are range coded, not zlib (Section 8.7).
+        this.header.version >= 2
+            ? compressed
+            : await inflateBlock(this.inflater, compressed), this.header.maxDepth, this.header.version)));
     }
     checkPalette(index, chunk) {
         // Section 11: "every palette entry [MUST be] less than N".

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@ferrybig/cgeodata-reader",
-	"version": "0.1.0",
+	"version": "0.1.1",
 	"description": "Reverse geocoding without a database: resolves a latitude/longitude to its country, region and city from OpenStreetMap boundaries, reading a compact CGEODATA file. No dependencies; Node.js and browsers.",
 	"keywords": [
 		"reverse-geocoding",