ol-load-geopackage 2.1.0 → 2.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/API.md +65 -17
- package/README.md +5 -3
- package/dist/ol-load-geopackage.js +271 -105
- package/package.json +1 -1
package/API.md
CHANGED
|
@@ -2,8 +2,8 @@
|
|
|
2
2
|
|
|
3
3
|
This page describes the 3 exported functions/constants of the [ol-load-geopackage](README.md) module:
|
|
4
4
|
|
|
5
|
-
- [initSqlJsWasm()](#initsqljswasmsqljswasmdir) - initialisation: start loading of required sql.js WASM file
|
|
6
|
-
- [loadGpkg()](#loadgpkggpkgfile-displayprojection) - start loading and data extraction of GeoPackage
|
|
5
|
+
- [initSqlJsWasm()](#initsqljswasmsqljswasmdir) - initialisation: start asynchronous loading of required sql.js WASM file
|
|
6
|
+
- [loadGpkg()](#loadgpkggpkgfile-displayprojection) - start asynchronous loading and data extraction of GeoPackage
|
|
7
7
|
- [sql_js_version](#sql_js_version) - NPM version number of underlying sql.js module
|
|
8
8
|
|
|
9
9
|
## initSqlJsWasm(sqlJsWasmDir)
|
|
@@ -31,38 +31,86 @@ initSqlJsWasm(sqlJsWasmDir);
|
|
|
31
31
|
|
|
32
32
|
Begin asynchronous loading of a single OGC GeoPackage, then extract vector data tables into OpenLayers Vector Sources, transforming the data (if necessary) to match the specified display projection. If a "layer_styles" table is found (as generated by QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) Processing Toolbox command), it will extract the constituent SLD XML styling data associated with each vector data table.
|
|
33
33
|
|
|
34
|
-
Parameters
|
|
34
|
+
### Parameters
|
|
35
35
|
|
|
36
|
-
- string `gpkgFile`: OGC GeoPackage
|
|
36
|
+
- string|File|Blob|URL `gpkgFile`: OGC GeoPackage URL string or File/URL object (URL can be path relative to document's base URI)
|
|
37
37
|
- string `displayProjection`: Map display projection for output sources (e.g. 'EPSG:3857'). Note that projections not built in to OpenLayers must be defined before calling the function. This is most easily done using the Proj4JS library - see Proj4 [Example](README.md#examples-in-github-repository).
|
|
38
|
+
- object `options`: Configuration options
|
|
39
|
+
|
|
40
|
+
### Options
|
|
41
|
+
|
|
42
|
+
`MissingDataSrsAction`: Action if missing source data SRS, one of:
|
|
43
|
+
|
|
44
|
+
- `stop` (default): Throw error, stop processing, discard all tables
|
|
45
|
+
- `discard`: Skip processing of data; put error and missing data SRS ID in table status
|
|
46
|
+
- `noProject`: Keep data but do not reproject; put error and missing data SRC in table status
|
|
38
47
|
|
|
39
|
-
|
|
48
|
+
`FailedTableLoadAction`: Action if gpkg table load fails, one of:
|
|
49
|
+
|
|
50
|
+
- `stop` (default): Throw error, stop processing, discard all tables
|
|
51
|
+
- `discard`: Skip processing of data; put error and data SRS ID in table status
|
|
52
|
+
|
|
53
|
+
### Return Values
|
|
54
|
+
|
|
55
|
+
Returns a Promise which delivers an array of 3 objects:
|
|
40
56
|
|
|
41
57
|
```javascript
|
|
42
|
-
[dataFromGpkg, sldsFromGpkg]
|
|
58
|
+
[dataFromGpkg, sldsFromGpkg, gpkgTableStatus]
|
|
43
59
|
```
|
|
44
60
|
|
|
45
61
|
- object `dataFromGpkg`: data tables (OpenLayers vector sources, indexed by table name),
|
|
46
62
|
- object `sldsFromGpkg`: styles (SLD layer_styles XML strings, indexed by layer name)
|
|
63
|
+
- object `gpkgTableStatus`: table status (indexed by table name) which is an object with 3 properties:
|
|
64
|
+
- integer `statusCode`: error code
|
|
65
|
+
- string `statusMsg`: error message
|
|
66
|
+
- integer `origSrsId`: original geopackage table SRS ID
|
|
67
|
+
|
|
68
|
+
`dataFromGpkg`:
|
|
47
69
|
|
|
48
|
-
|
|
70
|
+
If any `option` is set to `discard`, no Vector Source will be returned in `dataFromGpkg` for failing tables, but there will be a table entry in `gpkgTableStatus`.
|
|
71
|
+
|
|
72
|
+
**DEPRECATED (will be removed in later versions)**: Vector Source extension: for information only, the original data projection (SRS ID) with an "EPSG:" prefix is currently also returned as the string Property "origProjection" of each data source, so can be accessed with:
|
|
49
73
|
|
|
50
74
|
```javascript
|
|
51
|
-
|
|
75
|
+
tableSrsId = gpkgTableStatus[table].origSrsId
|
|
76
|
+
|
|
77
|
+
// DEPRECATED
|
|
78
|
+
tableEpsgSrsId = dataFromGpkg[table].getProperties()["origProjection"]
|
|
52
79
|
```
|
|
53
80
|
|
|
54
|
-
|
|
81
|
+
`gpkgTableStatus` table status values:
|
|
82
|
+
| statusCode | statusMsg |
|
|
83
|
+
| ------------- | ------------- |
|
|
84
|
+
| 0 | OK |
|
|
85
|
+
| 1 | Not reprojected as ol/proj missing table SRS |
|
|
86
|
+
| 2 | Discarded as ol/proj missing table SRS |
|
|
87
|
+
| 3 | Discarded as no (or only null geom) features |
|
|
88
|
+
| 4 | Failed GeoPackage table load: _(specific error)_ |
|
|
55
89
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
90
|
+
### Promise rejections
|
|
91
|
+
|
|
92
|
+
Promise is rejected (i.e. asynchronously) on these events (usually indicating unexpected data or network errors):
|
|
93
|
+
- Unable to load SQLite JS WebAssembly binary (sql-wasm.wasm)
|
|
94
|
+
- Unable to load requested GeoPackage
|
|
95
|
+
- Unable to extract feature table names and types from OGC GeoPackage file
|
|
96
|
+
- (Geopackage table specific) Missing a table data projection
|
|
97
|
+
- (Geopackage table specific) Failure trying to extract feature table
|
|
59
98
|
|
|
60
|
-
|
|
61
|
-
- Unable to load WebAssembly binary (sql-wasm.wasm)
|
|
62
|
-
- Unable to load requested OGC GeoPackage
|
|
63
|
-
- Missing requested display projection
|
|
64
|
-
- Missing a data projection (from any of Geopackage tables)
|
|
99
|
+
The last 2 (table-specific) promise rejections can be avoided by setting the appropriate `option` to handle the errors. The errors can then be determined in the calling routine by examining the returned table-specific `gpkgTableStatus` status codes and status messages.
|
|
65
100
|
|
|
101
|
+
### Errors Thrown
|
|
102
|
+
|
|
103
|
+
Errors are thrown immediately on these events (usually indicating programming errors):
|
|
104
|
+
- (Error) SQL WASM loading not previously started, i.e. missing a preceding call to initSqlJsWasm()
|
|
105
|
+
- (Error) ol/proj is missing the requested display projection (can be added beforehand using ol/proj/proj4.js)
|
|
106
|
+
- (TypeError) gpkgFile is not a supported type
|
|
107
|
+
|
|
108
|
+
### General Notes
|
|
109
|
+
|
|
110
|
+
1. After loading the GeoPackage file, extraction of GeoPackage data will wait (if necessary) for the loading of the sql.js WASM file to complete (as initiated by [initSqlJsWasm()](#initsqljswasmsqljswasmdir)).
|
|
111
|
+
2. Any features with null geom attributes will be silently discarded.
|
|
112
|
+
3. `sldsFromGpkg` will be an empty object if no table named "layer_styles" is found in the GeoPackage.
|
|
113
|
+
4. In the output GeoPackage from QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) the "table name" used for each vector data table will be exactly the same as the "layer name" used to index the SLD style strings in the "layer_styles" table.
|
|
66
114
|
|
|
67
115
|
## sql_js_version
|
|
68
116
|
|
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
[](https://www.npmjs.com/package/ol-load-geopackage)
|
|
4
4
|
|
|
5
|
-
A JavaScript module to load OGC GeoPackage vector data tables into OpenLayers Vector Sources, transforming the data (if necessary) to match the specified display projection.
|
|
5
|
+
A JavaScript module to load OGC GeoPackage vector data tables into [OpenLayers Vector Sources](https://openlayers.org/en/latest/apidoc/module-ol_source_Vector-VectorSource.html), transforming the data (if necessary) to match the specified display projection. Additionally, to help directly load data exported by the QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) Processing Toolbox operation it will also (if it exists) load the associated "layer_styles" table of SLD XML styling data exported by QGIS in the same GeoPackage. It is implemented as an NPM module and is a lightweight wrapper around the [sql.js](https://github.com/sql-js/sql.js) SQLite JavaScript library.
|
|
6
6
|
|
|
7
7
|
The current version was tested with OpenLayers 10.7, but should work with OpenLayers 6+.
|
|
8
8
|
|
|
@@ -69,6 +69,8 @@ gpkgPromise
|
|
|
69
69
|
|
|
70
70
|
Note that the _initSqlJsWasm()_ statement will start the asynchronous loading of the required sql.js WebAssembly binary file sql-wasm.wasm (from the current folder in this case), so is best placed early in the code.
|
|
71
71
|
|
|
72
|
+
For more advanced usage, an additional `options` parameter can be passed to loadGpkg() to control handling of problems encountered with GeoPackages, with information on the issues available as a 3rd return value (`gpkgTableStatus`). See the separate [API Specification](API.md) for details.
|
|
73
|
+
|
|
72
74
|
### Building with Webpack
|
|
73
75
|
|
|
74
76
|
The (shared) support files used to build the examples using [Webpack 5](https://webpack.js.org/) ([package.json](https://github.com/richard-thomas/ol-load-geopackage/tree/master/examples/package.json), [webpack.config.js](https://github.com/richard-thomas/ol-load-geopackage/tree/master/examples/webpack.config.js)) are in the _examples_ folder. If you clone the repository then you can (re-)build the code bundles (for both examples) with the commands:
|
|
@@ -120,8 +122,8 @@ npm run-script dev
|
|
|
120
122
|
|
|
121
123
|
The JavaScript module has 3 exported functions/constants which are described in the separate [API Specification](API.md):
|
|
122
124
|
|
|
123
|
-
- [initSqlJsWasm()](API.md#initsqljswasmsqljswasmdir) - Initialisation: start loading of required sql.js WASM file
|
|
124
|
-
- [loadGpkg()](API.md#loadgpkggpkgfile-displayprojection) - start loading and data extraction of GeoPackage
|
|
125
|
+
- [initSqlJsWasm()](API.md#initsqljswasmsqljswasmdir) - Initialisation: start asynchronous loading of required sql.js WASM file
|
|
126
|
+
- [loadGpkg()](API.md#loadgpkggpkgfile-displayprojection) - start asynchronous loading and data extraction of GeoPackage
|
|
125
127
|
- [sql_js_version](API.md#sql_js_version) - NPM version number of underlying sql.js module
|
|
126
128
|
|
|
127
129
|
## Migrating from ol-load-geopackage v1.x.x
|
|
@@ -19,7 +19,7 @@ const sql_js_version = sqlJsPkg.version;
|
|
|
19
19
|
/**
|
|
20
20
|
* Whether sql.js WASM file has been (successfully) loaded yet
|
|
21
21
|
*/
|
|
22
|
-
|
|
22
|
+
let promiseSqlWasmLoaded;
|
|
23
23
|
|
|
24
24
|
// -------- Public Functions --------
|
|
25
25
|
export { initSqlJsWasm, loadGpkg, sql_js_version };
|
|
@@ -30,8 +30,8 @@ export { initSqlJsWasm, loadGpkg, sql_js_version };
|
|
|
30
30
|
* then the returned promise can be ignored; loadGpkg() will wait if necessary
|
|
31
31
|
* and handle any errors loading the WASM file.
|
|
32
32
|
* @param {string} sqlJsWasmDir - URL of folder containing sql-wasm.wasm file to load for sql.js
|
|
33
|
-
* @returns {Promise} Promise which delivers:
|
|
34
|
-
*
|
|
33
|
+
* @returns {Promise<WebAssembly>} Promise which delivers:
|
|
34
|
+
* sql.js SQLITE database access library WebAssembly
|
|
35
35
|
*/
|
|
36
36
|
function initSqlJsWasm(sqlJsWasmDir) {
|
|
37
37
|
// If the WASM file location isn't specified look for it in the root folder
|
|
@@ -46,24 +46,67 @@ function initSqlJsWasm(sqlJsWasmDir) {
|
|
|
46
46
|
});
|
|
47
47
|
promiseSqlWasmLoaded
|
|
48
48
|
.catch(error => {
|
|
49
|
-
// Only
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
`
|
|
49
|
+
// Only reporting error to console here to simplify error handling;
|
|
50
|
+
// Error will be dealt with on later call(s) to loadGpkg()
|
|
51
|
+
console.error('initSqlJsWasm() unable to load SQLite JS binary ' +
|
|
52
|
+
`(sql-wasm.wasm) from URL folder:\n` +
|
|
53
|
+
` ${sqlJsWasmDir}/\n` +
|
|
54
|
+
error);
|
|
53
55
|
});
|
|
54
56
|
|
|
55
57
|
return promiseSqlWasmLoaded;
|
|
56
58
|
}
|
|
57
59
|
|
|
60
|
+
/**
|
|
61
|
+
* @typedef {'stop' | 'discard' | 'noProject'} MissingDataSrsAction
|
|
62
|
+
* Action if missing source data CRS, one of:
|
|
63
|
+
* * `stop` (default): Throw error, stop processing, discard all tables
|
|
64
|
+
* * `discard`: Skip processing of data; put error and missing data SRS ID in table status
|
|
65
|
+
* * `noProject`: Keep data but do not reproject; put error and missing data SRC in table status
|
|
66
|
+
*/
|
|
67
|
+
|
|
68
|
+
/**
|
|
69
|
+
* @typedef {'stop' | 'discard'} FailedTableLoadAction
|
|
70
|
+
* Action if gpkg table load fails, one of:
|
|
71
|
+
* * `stop` (default): Throw error, stop processing, discard all tables
|
|
72
|
+
* * `discard`: Skip processing of data; put error and data SRS ID in table status
|
|
73
|
+
*/
|
|
74
|
+
|
|
75
|
+
/**
|
|
76
|
+
* Configuration options for loadGpkg() function
|
|
77
|
+
* @typedef {Object} loadGpkgOptions
|
|
78
|
+
* @property {MissingDataSrsAction} [missingDataSrsAction='stop'] Missing source data CRS action
|
|
79
|
+
* @property {FailedTableLoadAction} [failedTableLoadAction='stop'] Failed gpkg table action
|
|
80
|
+
*/
|
|
81
|
+
|
|
82
|
+
/**
|
|
83
|
+
* gpkgTableStatus return value object
|
|
84
|
+
* @typedef {Object} gpkgTableStatusObj
|
|
85
|
+
* @property {integer} statusCode - error code
|
|
86
|
+
* @property {string} statusMsg - error message
|
|
87
|
+
* @property {integer} origSrsId - original geopackage table SRS ID
|
|
88
|
+
*/
|
|
89
|
+
|
|
58
90
|
/**
|
|
59
91
|
* Wrapper to load a single OGC GeoPackage
|
|
60
|
-
* @param {string} gpkgFile - OGC GeoPackage
|
|
92
|
+
* @param {(string|File|Blob|URL)} gpkgFile - OGC GeoPackage URL string or File/URL object
|
|
61
93
|
* @param {string} displayProjection - map display projection (e.g. EPSG:3857)
|
|
62
|
-
* @
|
|
63
|
-
*
|
|
64
|
-
*
|
|
94
|
+
* @param {loadGpkgOptions} options - configuration options
|
|
95
|
+
* @returns {Promise<[Object, Object, gpkgTableStatusObj]>} Promise delivering array of 3 objects:
|
|
96
|
+
* * data tables (OpenLayers vector sources, indexed by table name),
|
|
97
|
+
* * styles (SLD layer_styles XML strings, indexed by layer name),
|
|
98
|
+
* * table status (indexed by table name) which is an object with 3 properties:
|
|
99
|
+
* * (integer) `statusCode`: error code
|
|
100
|
+
* * (string) `statusMsg`: error message
|
|
101
|
+
* * (integer) `origSrsId`: original geopackage table SRS
|
|
102
|
+
* @throws {Error} If SQL WASM loading not previously started
|
|
103
|
+
* @throws {Error} If ol/proj is missing the requested display projection
|
|
104
|
+
* @throws {TypeError} If gpkgFile is not a supported type
|
|
65
105
|
*/
|
|
66
|
-
function loadGpkg(gpkgFile, displayProjection) {
|
|
106
|
+
function loadGpkg(gpkgFile, displayProjection, options) {
|
|
107
|
+
options = options || {};
|
|
108
|
+
options.missingDataSrsAction ??= 'stop';
|
|
109
|
+
options.failedTableLoadAction ??= 'stop';
|
|
67
110
|
|
|
68
111
|
// Check SQL WASM loading was initiated
|
|
69
112
|
if (promiseSqlWasmLoaded === undefined) {
|
|
@@ -78,20 +121,35 @@ function loadGpkg(gpkgFile, displayProjection) {
|
|
|
78
121
|
'] - can be added beforehand with ol/proj/proj4');
|
|
79
122
|
}
|
|
80
123
|
|
|
124
|
+
// Check if gpkgFile is valid type (string, URL, File or Blob)
|
|
125
|
+
if (!(typeof gpkgFile === 'string' || gpkgFile instanceof URL ||
|
|
126
|
+
gpkgFile instanceof Blob)) {
|
|
127
|
+
throw new TypeError('GeoPackage file specifier must be URL string ' +
|
|
128
|
+
'or URL/File/Blob object');
|
|
129
|
+
}
|
|
130
|
+
|
|
81
131
|
// Start OGC GeoPackage load and processing to extract data/SLDs
|
|
82
|
-
|
|
132
|
+
const gpkgReadPromise = readRawGpkg(gpkgFile);
|
|
83
133
|
|
|
84
134
|
return Promise.allSettled([promiseSqlWasmLoaded, gpkgReadPromise])
|
|
85
135
|
.then((results) => {
|
|
86
136
|
if (results[0].status === 'rejected') {
|
|
87
|
-
|
|
137
|
+
// Throw (initSqlJs() failure) will convert to rejected promise
|
|
138
|
+
throw new Error(
|
|
139
|
+
'Unable to load SQLite JS binary (sql-wasm.wasm).\n' +
|
|
140
|
+
results[0].reason);
|
|
88
141
|
}
|
|
89
142
|
if (results[1].status === 'rejected') {
|
|
90
|
-
|
|
143
|
+
// Propagate exact Error object from readRawGpkg().
|
|
144
|
+
// Throw will convert to rejected promise
|
|
145
|
+
throw results[1].reason;
|
|
91
146
|
}
|
|
92
147
|
const sqlWasm = results[0].value;
|
|
93
|
-
const
|
|
94
|
-
|
|
148
|
+
const gpkgByteArray = results[1].value;
|
|
149
|
+
const gpkgFileName = (gpkgFile instanceof File) ? gpkgFile.name :
|
|
150
|
+
gpkgFile.toString();
|
|
151
|
+
return processGpkgData(gpkgFileName, gpkgByteArray, sqlWasm,
|
|
152
|
+
displayProjection, options);
|
|
95
153
|
}
|
|
96
154
|
);
|
|
97
155
|
}
|
|
@@ -100,67 +158,91 @@ function loadGpkg(gpkgFile, displayProjection) {
|
|
|
100
158
|
|
|
101
159
|
/**
|
|
102
160
|
* Read raw data from source for a single OGC GeoPackage
|
|
103
|
-
* @param {string}
|
|
104
|
-
* @returns {Promise} Promise with Gpkg contents in
|
|
161
|
+
* @param {(string|File|Blob|URL)} input - OGC GeoPackage URL string or File/URL object
|
|
162
|
+
* @returns {Promise<Uint8Array>} Promise with Gpkg contents in byte array format
|
|
105
163
|
*/
|
|
106
|
-
function readRawGpkg(
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
}
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
164
|
+
async function readRawGpkg(input) {
|
|
165
|
+
|
|
166
|
+
// Fetch() or File/Blob object (both have the equivalent methods we need)
|
|
167
|
+
let gpkgSource;
|
|
168
|
+
|
|
169
|
+
if (input instanceof Blob) {
|
|
170
|
+
// File or Blob (File object is a specific type of Blob)
|
|
171
|
+
gpkgSource = input;
|
|
172
|
+
} else {
|
|
173
|
+
// URL string or URL object
|
|
174
|
+
const response = await fetch(input);
|
|
175
|
+
if (!response.ok) {
|
|
176
|
+
// Throw will convert to rejected promise (as an async function)
|
|
177
|
+
throw new Error('GPKG file could not be loaded from URL:\n' +
|
|
178
|
+
` ${input}\n` +
|
|
179
|
+
`Response: (${response.status}) ${response.statusText}`);
|
|
180
|
+
}
|
|
181
|
+
gpkgSource = response;
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
try {
|
|
185
|
+
// Use .bytes() method if available (most browsers since Jan 2025)
|
|
186
|
+
if (typeof gpkgSource.bytes === 'function') {
|
|
187
|
+
return await gpkgSource.bytes();
|
|
188
|
+
}
|
|
189
|
+
|
|
190
|
+
// Fallback approach: Convert ArrayBuffer to Uint8Array
|
|
191
|
+
//console.log('readRawGpkg(): using fallback .arrayBuffer() + Uint8Array() methods');
|
|
192
|
+
const buffer = await gpkgSource.arrayBuffer();
|
|
193
|
+
return new Uint8Array(buffer);
|
|
194
|
+
} catch (error) {
|
|
195
|
+
// Throw will convert to rejected promise (as an async function)
|
|
196
|
+
throw new Error('Unable to extract Byte Array from GPKG file.\n' + error);
|
|
197
|
+
}
|
|
127
198
|
}
|
|
128
199
|
|
|
129
200
|
/**
|
|
130
|
-
* Process OGC GeoPackage (SQLite database) once loaded
|
|
131
|
-
* @param {
|
|
132
|
-
* @param {
|
|
201
|
+
* Process OGC GeoPackage (based on SQLite database) once loaded
|
|
202
|
+
* @param {string} loadedGpkgFile - name of GeoPackage file (for diagnostics only)
|
|
203
|
+
* @param {Uint8Array} gpkgByteArray - Byte Array containing Gpkg data read
|
|
133
204
|
* @param {WebAssembly} sqlWasm - sql.js SQLITE database access library
|
|
134
205
|
* @param {string} displayProjection - map display projection (e.g. EPSG:3857)
|
|
135
|
-
* @
|
|
136
|
-
*
|
|
137
|
-
*
|
|
206
|
+
* @param {loadGpkgOptions} options - configuration options
|
|
207
|
+
* @returns {[Object, Object, gpkgTableStatusObj]} Array of 3 objects:
|
|
208
|
+
* * data tables (OpenLayers vector sources, indexed by table name),
|
|
209
|
+
* * styles (SLD layer_styles XML strings, indexed by layer name),
|
|
210
|
+
* * table status (indexed by table name) which is an object with 3 properties:
|
|
211
|
+
* * (integer) `statusCode`: error code
|
|
212
|
+
* * (string) `statusMsg`: error message
|
|
213
|
+
* * (integer) `origSrsId`: original geopackage table SRS
|
|
214
|
+
* @throws {Error} If unable to extract feature tables from OGC GeoPackage file
|
|
215
|
+
* @throws {Error} If ol/proj is missing required projection for a data table
|
|
216
|
+
* @throws {Error} Failed GeoPackage table load
|
|
138
217
|
*/
|
|
139
|
-
function processGpkgData(loadedGpkgFile,
|
|
140
|
-
displayProjection) {
|
|
141
|
-
|
|
218
|
+
function processGpkgData(loadedGpkgFile, gpkgByteArray, sqlWasm,
|
|
219
|
+
displayProjection, options) {
|
|
220
|
+
let db = null;
|
|
221
|
+
let stmt;
|
|
222
|
+
const featureTableNames = [];
|
|
142
223
|
|
|
143
224
|
// Data and associated SLD styles loaded both from GPKG
|
|
144
|
-
|
|
145
|
-
|
|
225
|
+
const dataFromGpkg = {};
|
|
226
|
+
const sldsFromGpkg = {};
|
|
146
227
|
|
|
147
|
-
|
|
148
|
-
|
|
228
|
+
/**
|
|
229
|
+
* @type {gpkgTableStatusObj}
|
|
230
|
+
* Table data extraction status
|
|
231
|
+
*/
|
|
232
|
+
const gpkgTableStatus = {};
|
|
149
233
|
|
|
150
|
-
//
|
|
151
|
-
|
|
234
|
+
// DEBUG: measure GPKG processing time
|
|
235
|
+
//let startProcessing = Date.now();
|
|
152
236
|
|
|
153
237
|
try {
|
|
154
238
|
db = new sqlWasm.Database(gpkgByteArray);
|
|
155
239
|
|
|
156
|
-
// Extract all feature
|
|
240
|
+
// Extract all feature table names, SRS IDs and their geometry types
|
|
157
241
|
// Note the following fields are not extracted:
|
|
158
242
|
// gpkg_contents.identifier - title (QGIS: same as table_name)
|
|
159
243
|
// gpkg_contents.description - human readable (QGIS: blank)
|
|
160
244
|
// gpkg_geometry_columns.geometry_type_name
|
|
161
245
|
// - e.g. LINESTRING (but info also embedded in each feature)
|
|
162
|
-
var featureTableNames = [];
|
|
163
|
-
var stmt;
|
|
164
246
|
stmt = db.prepare(`
|
|
165
247
|
SELECT gpkg_contents.table_name, gpkg_contents.srs_id,
|
|
166
248
|
gpkg_geometry_columns.column_name
|
|
@@ -169,18 +251,21 @@ function processGpkgData(loadedGpkgFile, gpkgArrayBuffer, sqlWasm,
|
|
|
169
251
|
gpkg_contents.table_name=gpkg_geometry_columns.table_name;
|
|
170
252
|
`);
|
|
171
253
|
while (stmt.step()) {
|
|
172
|
-
|
|
254
|
+
const row = stmt.get();
|
|
173
255
|
featureTableNames.push({
|
|
174
256
|
'table_name': row[0],
|
|
175
257
|
'srs_id': row[1].toString(),
|
|
176
258
|
'geometry_column_name': row[2]
|
|
177
259
|
});
|
|
178
260
|
}
|
|
261
|
+
stmt.free();
|
|
179
262
|
}
|
|
180
263
|
catch (err) {
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
264
|
+
if (db) {
|
|
265
|
+
db.close();
|
|
266
|
+
}
|
|
267
|
+
throw new Error('Unable to extract feature table names and types ' +
|
|
268
|
+
'from OGC GeoPackage file.\n' + err);
|
|
184
269
|
}
|
|
185
270
|
|
|
186
271
|
// Extract SLD styles for each layer (if styles included in the gpkg)
|
|
@@ -192,75 +277,156 @@ function processGpkgData(loadedGpkgFile, gpkgArrayBuffer, sqlWasm,
|
|
|
192
277
|
if (stmt.step()) {
|
|
193
278
|
stmt = db.prepare('SELECT f_table_name,styleSLD FROM layer_styles');
|
|
194
279
|
while (stmt.step()) {
|
|
195
|
-
|
|
280
|
+
const row = stmt.get();
|
|
196
281
|
sldsFromGpkg[row[0]] = row[1];
|
|
197
282
|
}
|
|
198
283
|
}
|
|
284
|
+
stmt.free();
|
|
199
285
|
|
|
200
286
|
// For each table, extract geometry and other properties
|
|
201
287
|
// (Note: becomes OpenLayers-specific from here)
|
|
202
|
-
|
|
203
|
-
for (
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
let
|
|
288
|
+
const formatWKB = new ol_format_WKB();
|
|
289
|
+
for (const table of featureTableNames) {
|
|
290
|
+
const table_name = table.table_name;
|
|
291
|
+
const tableDataProjection = 'EPSG:' + table.srs_id;
|
|
292
|
+
let noReProject = false;
|
|
293
|
+
let featureCount = 0;
|
|
207
294
|
|
|
208
295
|
// Check if we have a definition for the data projection (SRS)
|
|
209
296
|
if (!ol_proj_get(tableDataProjection)) {
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
297
|
+
if (options.missingDataSrsAction === 'discard') {
|
|
298
|
+
// discard: Skip processing of data; put error and
|
|
299
|
+
// missing data SRS ID in table status
|
|
300
|
+
console.warn(`Discarding table "${table_name}" from gpkg ` +
|
|
301
|
+
`"${loadedGpkgFile}" as ol/proj missing table SRS ` +
|
|
302
|
+
`${tableDataProjection}`);
|
|
303
|
+
gpkgTableStatus[table_name] = {
|
|
304
|
+
statusCode: 2,
|
|
305
|
+
statusMsg: 'Discarded as ol/proj missing table SRS',
|
|
306
|
+
origSrsId: table.srs_id
|
|
307
|
+
};
|
|
308
|
+
continue;
|
|
309
|
+
|
|
310
|
+
} else if (options.missingDataSrsAction === 'noProject') {
|
|
311
|
+
// noProject: Keep data but do not reproject;
|
|
312
|
+
// put error and missing data SRC in table status
|
|
313
|
+
console.warn(`Not reprojecting table "${table_name}" from ` +
|
|
314
|
+
`gpkg "${loadedGpkgFile}" as ol/proj missing table SRS ` +
|
|
315
|
+
`${tableDataProjection}`);
|
|
316
|
+
gpkgTableStatus[table_name] = {
|
|
317
|
+
statusCode: 1,
|
|
318
|
+
statusMsg: 'Not reprojected as ol/proj missing table SRS',
|
|
319
|
+
origSrsId: table.srs_id
|
|
320
|
+
};
|
|
321
|
+
noReProject = true;
|
|
322
|
+
|
|
323
|
+
} else {
|
|
324
|
+
// stop (default): Throw error, stop processing, discard all tables
|
|
325
|
+
throw new Error('Missing data projection [' +
|
|
326
|
+
tableDataProjection + '] for table "' + table_name +
|
|
327
|
+
'" - can be added beforehand with ol/proj/proj4');
|
|
328
|
+
}
|
|
213
329
|
}
|
|
214
330
|
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
331
|
+
// For each feature in the table, extract geometry + other properties
|
|
332
|
+
// and add these to a new OpenLayers Vector Source for the table.
|
|
333
|
+
//console.log(`Extracting table "${table_name}" (SRS: ${tableDataProjection})`);
|
|
334
|
+
const vectorSource = new ol_source_Vector();
|
|
335
|
+
const geometry_column_name = table.geometry_column_name;
|
|
336
|
+
try {
|
|
337
|
+
stmt = db.prepare("SELECT * FROM `" + table_name + "`");
|
|
338
|
+
while (stmt.step()) {
|
|
339
|
+
// Extract properties & geometry for a single feature
|
|
340
|
+
const properties = stmt.getAsObject();
|
|
341
|
+
const geomProp = properties[geometry_column_name];
|
|
342
|
+
if (geomProp == null) {
|
|
343
|
+
//console.log(`Discarding feature ${featureCount + 1} with null geom in table: '${table_name}'`);
|
|
344
|
+
continue;
|
|
345
|
+
}
|
|
346
|
+
delete properties[geometry_column_name];
|
|
347
|
+
const featureWkb = parseGpkgGeom(geomProp);
|
|
348
|
+
/*
|
|
349
|
+
// DEBUG: show endianness of WKB data (can differ from header)
|
|
350
|
+
if (!vectorSource.getFeatures().length) {
|
|
351
|
+
console.log('WKB Geometry: ' +
|
|
352
|
+
(featureWkb[0] ? 'NDR (Little' : 'XDR (Big') + ' Endian)');
|
|
353
|
+
}
|
|
354
|
+
*/
|
|
355
|
+
// Put the feature into the vector source for the current table
|
|
356
|
+
// Reproject unless ol/proj is missing SRS of source data table
|
|
357
|
+
const features = formatWKB.readFeatures(featureWkb, {
|
|
358
|
+
dataProjection: tableDataProjection,
|
|
359
|
+
featureProjection: noReProject ? undefined : displayProjection
|
|
360
|
+
});
|
|
361
|
+
features[0].setProperties(properties);
|
|
362
|
+
vectorSource.addFeatures(features);
|
|
363
|
+
featureCount++;
|
|
364
|
+
}
|
|
365
|
+
stmt.free();
|
|
366
|
+
} catch (error) {
|
|
367
|
+
if (options.failedTableLoadAction === 'discard') {
|
|
368
|
+
// discard: Skip processing of data; put error and data SRS ID in table status
|
|
369
|
+
console.warn(`Discarding table "${table_name}" from gpkg ` +
|
|
370
|
+
`"${loadedGpkgFile}" as table load failed:\n${error}`);
|
|
371
|
+
gpkgTableStatus[table_name] = {
|
|
372
|
+
statusCode: 4,
|
|
373
|
+
statusMsg: 'Failed GeoPackage table load: ' + error,
|
|
374
|
+
origSrsId: table.srs_id
|
|
375
|
+
};
|
|
376
|
+
stmt.free();
|
|
377
|
+
continue;
|
|
378
|
+
} else {
|
|
379
|
+
// stop (default): Throw error, stop processing, discard all tables
|
|
380
|
+
db.close();
|
|
381
|
+
throw error;
|
|
230
382
|
}
|
|
231
|
-
*/
|
|
232
|
-
// Put the feature into the vector source for the current table
|
|
233
|
-
features = formatWKB.readFeatures(featureWkb, {
|
|
234
|
-
dataProjection: tableDataProjection,
|
|
235
|
-
featureProjection: displayProjection
|
|
236
|
-
});
|
|
237
|
-
features[0].setProperties(properties);
|
|
238
|
-
vectorSource.addFeatures(features);
|
|
239
383
|
}
|
|
240
384
|
|
|
385
|
+
if (featureCount === 0) {
|
|
386
|
+
console.warn('Discarding table with no (or only null geom) features: ' + table_name);
|
|
387
|
+
gpkgTableStatus[table_name] = {
|
|
388
|
+
statusCode: 3,
|
|
389
|
+
statusMsg: 'Discarded as no (or only null geom) features',
|
|
390
|
+
origSrsId: table.srs_id
|
|
391
|
+
};
|
|
392
|
+
continue;
|
|
393
|
+
}
|
|
241
394
|
// For information only, save details of original projection (SRS)
|
|
395
|
+
// DEPRECATED (as now included in gpkgTableStatus). TBD: remove this soon!
|
|
242
396
|
vectorSource.setProperties({'origProjection': tableDataProjection});
|
|
397
|
+
|
|
243
398
|
dataFromGpkg[table_name] = vectorSource;
|
|
399
|
+
if (!noReProject) {
|
|
400
|
+
gpkgTableStatus[table_name] = {
|
|
401
|
+
statusCode: 0,
|
|
402
|
+
statusMsg: 'OK',
|
|
403
|
+
origSrsId: table.srs_id
|
|
404
|
+
};
|
|
405
|
+
}
|
|
244
406
|
}
|
|
407
|
+
|
|
408
|
+
// Close database (gpkg) to trigger garbage collection
|
|
409
|
+
db.close();
|
|
245
410
|
/*
|
|
246
411
|
// DEBUG: measure OGC GeoPackage processing time
|
|
247
|
-
|
|
412
|
+
let processingSecs = (Date.now() - startProcessing) / 1000;
|
|
248
413
|
console.log('INFO: OGC GeoPackage file ("' + loadedGpkgFile +
|
|
249
414
|
'") processing time = ' + processingSecs + ' s');
|
|
250
415
|
*/
|
|
251
|
-
return [dataFromGpkg, sldsFromGpkg];
|
|
416
|
+
return [dataFromGpkg, sldsFromGpkg, gpkgTableStatus];
|
|
252
417
|
}
|
|
253
418
|
|
|
254
419
|
/**
|
|
255
420
|
* Extract (SRS ID &) WKB from an OGC GeoPackage feature
|
|
256
421
|
* (i.e. strip off the variable length header)
|
|
257
|
-
* @param {
|
|
258
|
-
* @returns feature geometry in WKB (Well Known Binary) format
|
|
422
|
+
* @param {Uint8Array} gpkgBinGeom - feature geometry property (includes header)
|
|
423
|
+
* @returns {Uint8Array} feature geometry in WKB (Well Known Binary) format
|
|
424
|
+
* @throws {Error} If invalid geometry envelope size flag in GeoPackage
|
|
259
425
|
*/
|
|
260
426
|
function parseGpkgGeom(gpkgBinGeom) {
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
427
|
+
const flags = gpkgBinGeom[3];
|
|
428
|
+
const eFlags = (flags >> 1) & 7;
|
|
429
|
+
let envelopeSize;
|
|
264
430
|
switch (eFlags) {
|
|
265
431
|
case 0:
|
|
266
432
|
envelopeSize = 0;
|
|
@@ -281,9 +447,9 @@ function parseGpkgGeom(gpkgBinGeom) {
|
|
|
281
447
|
/*
|
|
282
448
|
// Extract SRS (EPSG code)
|
|
283
449
|
// (not required as given for whole table in gpkg_contents table)
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
450
|
+
const littleEndian = flags & 1;
|
|
451
|
+
const srs = gpkgBinGeom.subarray(4,8);
|
|
452
|
+
let srsId;
|
|
287
453
|
if (littleEndian) {
|
|
288
454
|
srsId = srs[0] + (srs[1]<<8) + (srs[2]<<16) + (srs[3]<<24);
|
|
289
455
|
} else {
|
|
@@ -301,6 +467,6 @@ function parseGpkgGeom(gpkgBinGeom) {
|
|
|
301
467
|
console.log('gpkgBinGeom envelope size (bytes):', envelopeSize);
|
|
302
468
|
*/
|
|
303
469
|
// Extract WKB which starts after variable-size "envelope" field
|
|
304
|
-
|
|
470
|
+
const wkbOffset = envelopeSize + 8;
|
|
305
471
|
return gpkgBinGeom.subarray(wkbOffset);
|
|
306
472
|
}
|