ol-load-geopackage 2.1.0 → 2.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/API.md CHANGED
@@ -2,8 +2,8 @@
2
2
 
3
3
  This page describes the 3 exported functions/constants of the [ol-load-geopackage](README.md) module:
4
4
 
5
- - [initSqlJsWasm()](#initsqljswasmsqljswasmdir) - initialisation: start loading of required sql.js WASM file
6
- - [loadGpkg()](#loadgpkggpkgfile-displayprojection) - start loading and data extraction of GeoPackage
5
+ - [initSqlJsWasm()](#initsqljswasmsqljswasmdir) - initialisation: start asynchronous loading of required sql.js WASM file
6
+ - [loadGpkg()](#loadgpkggpkgfile-displayprojection) - start asynchronous loading and data extraction of GeoPackage
7
7
  - [sql_js_version](#sql_js_version) - NPM version number of underlying sql.js module
8
8
 
9
9
  ## initSqlJsWasm(sqlJsWasmDir)
@@ -31,38 +31,86 @@ initSqlJsWasm(sqlJsWasmDir);
31
31
 
32
32
  Begin asynchronous loading of a single OGC GeoPackage, then extract vector data tables into OpenLayers Vector Sources, transforming the data (if necessary) to match the specified display projection. If a "layer_styles" table is found (as generated by QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) Processing Toolbox command), it will extract the constituent SLD XML styling data associated with each vector data table.
33
33
 
34
- Parameters:
34
+ ### Parameters
35
35
 
36
- - string `gpkgFile`: OGC GeoPackage file URL or path relative to root
36
+ - string|File|Blob|URL `gpkgFile`: OGC GeoPackage URL string or File/URL object (URL can be path relative to document's base URI)
37
37
  - string `displayProjection`: Map display projection for output sources (e.g. 'EPSG:3857'). Note that projections not built in to OpenLayers must be defined before calling the function. This is most easily done using the Proj4JS library - see Proj4 [Example](README.md#examples-in-github-repository).
38
+ - object `options`: Configuration options
39
+
40
+ ### Options
41
+
42
+ `MissingDataSrsAction`: Action if missing source data SRS, one of:
43
+
44
+ - `stop` (default): Throw error, stop processing, discard all tables
45
+ - `discard`: Skip processing of data; put error and missing data SRS ID in table status
46
+ - `noProject`: Keep data but do not reproject; put error and missing data SRC in table status
38
47
 
39
- Returns a Promise which delivers an array of 2 objects:
48
+ `FailedTableLoadAction`: Action if gpkg table load fails, one of:
49
+
50
+ - `stop` (default): Throw error, stop processing, discard all tables
51
+ - `discard`: Skip processing of data; put error and data SRS ID in table status
52
+
53
+ ### Return Values
54
+
55
+ Returns a Promise which delivers an array of 3 objects:
40
56
 
41
57
  ```javascript
42
- [dataFromGpkg, sldsFromGpkg]
58
+ [dataFromGpkg, sldsFromGpkg, gpkgTableStatus]
43
59
  ```
44
60
 
45
61
  - object `dataFromGpkg`: data tables (OpenLayers vector sources, indexed by table name),
46
62
  - object `sldsFromGpkg`: styles (SLD layer_styles XML strings, indexed by layer name)
63
+ - object `gpkgTableStatus`: table status (indexed by table name) which is an object with 3 properties:
64
+ - integer `statusCode`: error code
65
+ - string `statusMsg`: error message
66
+ - integer `origSrsId`: original geopackage table SRS ID
67
+
68
+ `dataFromGpkg`:
47
69
 
48
- For information only, the original data projection (SRS ID) will be returned as the string Property "origProjection" of each data source, so can be accessed with:
70
+ If any `option` is set to `discard`, no Vector Source will be returned in `dataFromGpkg` for failing tables, but there will be a table entry in `gpkgTableStatus`.
71
+
72
+ **DEPRECATED (will be removed in later versions)**: Vector Source extension: for information only, the original data projection (SRS ID) with an "EPSG:" prefix is currently also returned as the string Property "origProjection" of each data source, so can be accessed with:
49
73
 
50
74
  ```javascript
51
- dataFromGpkg[table].getProperties()["origProjection"]
75
+ tableSrsId = gpkgTableStatus[table].origSrsId
76
+
77
+ // DEPRECATED
78
+ tableEpsgSrsId = dataFromGpkg[table].getProperties()["origProjection"]
52
79
  ```
53
80
 
54
- Notes:
81
+ `gpkgTableStatus` table status values:
82
+ | statusCode | statusMsg |
83
+ | ------------- | ------------- |
84
+ | 0 | OK |
85
+ | 1 | Not reprojected as ol/proj missing table SRS |
86
+ | 2 | Discarded as ol/proj missing table SRS |
87
+ | 3 | Discarded as no (or only null geom) features |
88
+ | 4 | Failed GeoPackage table load: _(specific error)_ |
55
89
 
56
- 1. After loading the GeoPackage file, extraction of GeoPackage data will wait (if necessary) for the loading of the sql.js WASM file to complete (as initiated by [initSqlJsWasm()](#initsqljswasmsqljswasmdir)).
57
- 2. `sldsFromGpkg` will be an empty object if no table named "layer_styles" is found in the GeoPackage.
58
- 3. In the output GeoPackage from QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) the "table name" used for each vector data table will be exactly the same as the "layer name" used to index the SLD style strings in the "layer_styles" table.
90
+ ### Promise rejections
91
+
92
+ Promise is rejected (i.e. asynchronously) on these events (usually indicating unexpected data or network errors):
93
+ - Unable to load SQLite JS WebAssembly binary (sql-wasm.wasm)
94
+ - Unable to load requested GeoPackage
95
+ - Unable to extract feature table names and types from OGC GeoPackage file
96
+ - (Geopackage table specific) Missing a table data projection
97
+ - (Geopackage table specific) Failure trying to extract feature table
59
98
 
60
- Errors thrown on these events:
61
- - Unable to load WebAssembly binary (sql-wasm.wasm)
62
- - Unable to load requested OGC GeoPackage
63
- - Missing requested display projection
64
- - Missing a data projection (from any of Geopackage tables)
99
+ The last 2 (table-specific) promise rejections can be avoided by setting the appropriate `option` to handle the errors. The errors can then be determined in the calling routine by examining the returned table-specific `gpkgTableStatus` status codes and status messages.
65
100
 
101
+ ### Errors Thrown
102
+
103
+ Errors are thrown immediately on these events (usually indicating programming errors):
104
+ - (Error) SQL WASM loading not previously started, i.e. missing a preceding call to initSqlJsWasm()
105
+ - (Error) ol/proj is missing the requested display projection (can be added beforehand using ol/proj/proj4.js)
106
+ - (TypeError) gpkgFile is not a supported type
107
+
108
+ ### General Notes
109
+
110
+ 1. After loading the GeoPackage file, extraction of GeoPackage data will wait (if necessary) for the loading of the sql.js WASM file to complete (as initiated by [initSqlJsWasm()](#initsqljswasmsqljswasmdir)).
111
+ 2. Any features with null geom attributes will be silently discarded.
112
+ 3. `sldsFromGpkg` will be an empty object if no table named "layer_styles" is found in the GeoPackage.
113
+ 4. In the output GeoPackage from QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) the "table name" used for each vector data table will be exactly the same as the "layer name" used to index the SLD style strings in the "layer_styles" table.
66
114
 
67
115
  ## sql_js_version
68
116
 
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  [![npm](https://img.shields.io/npm/v/ol-load-geopackage)](https://www.npmjs.com/package/ol-load-geopackage)
4
4
 
5
- A JavaScript module to load OGC GeoPackage vector data tables into OpenLayers Vector Sources, transforming the data (if necessary) to match the specified display projection. This was primarily designed to directly load data exported by the QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) Processing Toolbox operation. As such, it will also (if it exists) load the associated "layer_styles" table of SLD XML styling data exported by QGIS in the same GeoPackage. It is implemented as an NPM module and is a lightweight wrapper around the [sql.js](https://github.com/sql-js/sql.js) SQLite JavaScript library.
5
+ A JavaScript module to load OGC GeoPackage vector data tables into [OpenLayers Vector Sources](https://openlayers.org/en/latest/apidoc/module-ol_source_Vector-VectorSource.html), transforming the data (if necessary) to match the specified display projection. Additionally, to help directly load data exported by the QGIS [Package Layers](https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/database.html#package-layers) Processing Toolbox operation it will also (if it exists) load the associated "layer_styles" table of SLD XML styling data exported by QGIS in the same GeoPackage. It is implemented as an NPM module and is a lightweight wrapper around the [sql.js](https://github.com/sql-js/sql.js) SQLite JavaScript library.
6
6
 
7
7
  The current version was tested with OpenLayers 10.7, but should work with OpenLayers 6+.
8
8
 
@@ -69,6 +69,8 @@ gpkgPromise
69
69
 
70
70
  Note that the _initSqlJsWasm()_ statement will start the asynchronous loading of the required sql.js WebAssembly binary file sql-wasm.wasm (from the current folder in this case), so is best placed early in the code.
71
71
 
72
+ For more advanced usage, an additional `options` parameter can be passed to loadGpkg() to control handling of problems encountered with GeoPackages, with information on the issues available as a 3rd return value (`gpkgTableStatus`). See the separate [API Specification](API.md) for details.
73
+
72
74
  ### Building with Webpack
73
75
 
74
76
  The (shared) support files used to build the examples using [Webpack 5](https://webpack.js.org/) ([package.json](https://github.com/richard-thomas/ol-load-geopackage/tree/master/examples/package.json), [webpack.config.js](https://github.com/richard-thomas/ol-load-geopackage/tree/master/examples/webpack.config.js)) are in the _examples_ folder. If you clone the repository then you can (re-)build the code bundles (for both examples) with the commands:
@@ -120,8 +122,8 @@ npm run-script dev
120
122
 
121
123
  The JavaScript module has 3 exported functions/constants which are described in the separate [API Specification](API.md):
122
124
 
123
- - [initSqlJsWasm()](API.md#initsqljswasmsqljswasmdir) - Initialisation: start loading of required sql.js WASM file
124
- - [loadGpkg()](API.md#loadgpkggpkgfile-displayprojection) - start loading and data extraction of GeoPackage
125
+ - [initSqlJsWasm()](API.md#initsqljswasmsqljswasmdir) - Initialisation: start asynchronous loading of required sql.js WASM file
126
+ - [loadGpkg()](API.md#loadgpkggpkgfile-displayprojection) - start asynchronous loading and data extraction of GeoPackage
125
127
  - [sql_js_version](API.md#sql_js_version) - NPM version number of underlying sql.js module
126
128
 
127
129
  ## Migrating from ol-load-geopackage v1.x.x
@@ -19,7 +19,7 @@ const sql_js_version = sqlJsPkg.version;
19
19
  /**
20
20
  * Whether sql.js WASM file has been (successfully) loaded yet
21
21
  */
22
- var promiseSqlWasmLoaded;
22
+ let promiseSqlWasmLoaded;
23
23
 
24
24
  // -------- Public Functions --------
25
25
  export { initSqlJsWasm, loadGpkg, sql_js_version };
@@ -30,8 +30,8 @@ export { initSqlJsWasm, loadGpkg, sql_js_version };
30
30
  * then the returned promise can be ignored; loadGpkg() will wait if necessary
31
31
  * and handle any errors loading the WASM file.
32
32
  * @param {string} sqlJsWasmDir - URL of folder containing sql-wasm.wasm file to load for sql.js
33
- * @returns {Promise} Promise which delivers:
34
- * {WebAssembly} sql.js SQLITE database access library
33
+ * @returns {Promise<WebAssembly>} Promise which delivers:
34
+ * sql.js SQLITE database access library WebAssembly
35
35
  */
36
36
  function initSqlJsWasm(sqlJsWasmDir) {
37
37
  // If the WASM file location isn't specified look for it in the root folder
@@ -46,24 +46,67 @@ function initSqlJsWasm(sqlJsWasmDir) {
46
46
  });
47
47
  promiseSqlWasmLoaded
48
48
  .catch(error => {
49
- // Only need report error here - any later calls to loadGpkg() will throw error
50
- console.error(`initSqlJsWasm() unable to load SQLite JS binary (sql-wasm.wasm) from folder:\n` +
51
- `${sqlJsWasmDir}/\n` +
52
- `[sql.js error message]: ${error}`);
49
+ // Only reporting error to console here to simplify error handling;
50
+ // Error will be dealt with on later call(s) to loadGpkg()
51
+ console.error('initSqlJsWasm() unable to load SQLite JS binary ' +
52
+ `(sql-wasm.wasm) from URL folder:\n` +
53
+ ` ${sqlJsWasmDir}/\n` +
54
+ error);
53
55
  });
54
56
 
55
57
  return promiseSqlWasmLoaded;
56
58
  }
57
59
 
60
+ /**
61
+ * @typedef {'stop' | 'discard' | 'noProject'} MissingDataSrsAction
62
+ * Action if missing source data CRS, one of:
63
+ * * `stop` (default): Throw error, stop processing, discard all tables
64
+ * * `discard`: Skip processing of data; put error and missing data SRS ID in table status
65
+ * * `noProject`: Keep data but do not reproject; put error and missing data SRC in table status
66
+ */
67
+
68
+ /**
69
+ * @typedef {'stop' | 'discard'} FailedTableLoadAction
70
+ * Action if gpkg table load fails, one of:
71
+ * * `stop` (default): Throw error, stop processing, discard all tables
72
+ * * `discard`: Skip processing of data; put error and data SRS ID in table status
73
+ */
74
+
75
+ /**
76
+ * Configuration options for loadGpkg() function
77
+ * @typedef {Object} loadGpkgOptions
78
+ * @property {MissingDataSrsAction} [missingDataSrsAction='stop'] Missing source data CRS action
79
+ * @property {FailedTableLoadAction} [failedTableLoadAction='stop'] Failed gpkg table action
80
+ */
81
+
82
+ /**
83
+ * gpkgTableStatus return value object
84
+ * @typedef {Object} gpkgTableStatusObj
85
+ * @property {integer} statusCode - error code
86
+ * @property {string} statusMsg - error message
87
+ * @property {integer} origSrsId - original geopackage table SRS ID
88
+ */
89
+
58
90
  /**
59
91
  * Wrapper to load a single OGC GeoPackage
60
- * @param {string} gpkgFile - OGC GeoPackage file path
92
+ * @param {(string|File|Blob|URL)} gpkgFile - OGC GeoPackage URL string or File/URL object
61
93
  * @param {string} displayProjection - map display projection (e.g. EPSG:3857)
62
- * @returns {Promise} Promise which delivers array of 2 objects:
63
- * data tables (OpenLayers vector sources, indexed by table name),
64
- * styles (SLD layer_styles XML strings, indexed by layer name)
94
+ * @param {loadGpkgOptions} options - configuration options
95
+ * @returns {Promise<[Object, Object, gpkgTableStatusObj]>} Promise delivering array of 3 objects:
96
+ * * data tables (OpenLayers vector sources, indexed by table name),
97
+ * * styles (SLD layer_styles XML strings, indexed by layer name),
98
+ * * table status (indexed by table name) which is an object with 3 properties:
99
+ * * (integer) `statusCode`: error code
100
+ * * (string) `statusMsg`: error message
101
+ * * (integer) `origSrsId`: original geopackage table SRS
102
+ * @throws {Error} If SQL WASM loading not previously started
103
+ * @throws {Error} If ol/proj is missing the requested display projection
104
+ * @throws {TypeError} If gpkgFile is not a supported type
65
105
  */
66
- function loadGpkg(gpkgFile, displayProjection) {
106
+ function loadGpkg(gpkgFile, displayProjection, options) {
107
+ options = options || {};
108
+ options.missingDataSrsAction ??= 'stop';
109
+ options.failedTableLoadAction ??= 'stop';
67
110
 
68
111
  // Check SQL WASM loading was initiated
69
112
  if (promiseSqlWasmLoaded === undefined) {
@@ -78,20 +121,35 @@ function loadGpkg(gpkgFile, displayProjection) {
78
121
  '] - can be added beforehand with ol/proj/proj4');
79
122
  }
80
123
 
124
+ // Check if gpkgFile is valid type (string, URL, File or Blob)
125
+ if (!(typeof gpkgFile === 'string' || gpkgFile instanceof URL ||
126
+ gpkgFile instanceof Blob)) {
127
+ throw new TypeError('GeoPackage file specifier must be URL string ' +
128
+ 'or URL/File/Blob object');
129
+ }
130
+
81
131
  // Start OGC GeoPackage load and processing to extract data/SLDs
82
- var gpkgReadPromise = readRawGpkg(gpkgFile);
132
+ const gpkgReadPromise = readRawGpkg(gpkgFile);
83
133
 
84
134
  return Promise.allSettled([promiseSqlWasmLoaded, gpkgReadPromise])
85
135
  .then((results) => {
86
136
  if (results[0].status === 'rejected') {
87
- throw new Error('Unable to load SQLite JS binary (sql-wasm.wasm)');
137
+ // Throw (initSqlJs() failure) will convert to rejected promise
138
+ throw new Error(
139
+ 'Unable to load SQLite JS binary (sql-wasm.wasm).\n' +
140
+ results[0].reason);
88
141
  }
89
142
  if (results[1].status === 'rejected') {
90
- throw new Error(`Unable to read raw GeoPackage data from file: ${gpkgFile}`);
143
+ // Propagate exact Error object from readRawGpkg().
144
+ // Throw will convert to rejected promise
145
+ throw results[1].reason;
91
146
  }
92
147
  const sqlWasm = results[0].value;
93
- const gpkgArrayBuffer = results[1].value;
94
- return processGpkgData(gpkgFile, gpkgArrayBuffer, sqlWasm, displayProjection);
148
+ const gpkgByteArray = results[1].value;
149
+ const gpkgFileName = (gpkgFile instanceof File) ? gpkgFile.name :
150
+ gpkgFile.toString();
151
+ return processGpkgData(gpkgFileName, gpkgByteArray, sqlWasm,
152
+ displayProjection, options);
95
153
  }
96
154
  );
97
155
  }
@@ -100,67 +158,91 @@ function loadGpkg(gpkgFile, displayProjection) {
100
158
 
101
159
  /**
102
160
  * Read raw data from source for a single OGC GeoPackage
103
- * @param {string} gpkgFile - OGC GeoPackage file path
104
- * @returns {Promise} Promise with Gpkg contents in ArrayBuffer format
161
+ * @param {(string|File|Blob|URL)} input - OGC GeoPackage URL string or File/URL object
162
+ * @returns {Promise<Uint8Array>} Promise with Gpkg contents in byte array format
105
163
  */
106
- function readRawGpkg(gpkgFile) {
107
- return new Promise(function(succeed, fail) {
108
- var oReq = new XMLHttpRequest();
109
- oReq.responseType = 'arraybuffer';
110
- oReq.onreadystatechange = function() {
111
-
112
- // When request finished and response is ready
113
- if (this.readyState == 4) {
114
- var gpkgArrayBuffer = this.response;
115
- if (this.status === 200 && gpkgArrayBuffer) {
116
- succeed(gpkgArrayBuffer);
117
- } else {
118
- fail(new Error(
119
- 'Requested GPKG file could not be loaded: ' +
120
- gpkgFile));
121
- }
122
- }
123
- };
124
- oReq.open('GET', gpkgFile);
125
- oReq.send();
126
- });
164
+ async function readRawGpkg(input) {
165
+
166
+ // Fetch() or File/Blob object (both have the equivalent methods we need)
167
+ let gpkgSource;
168
+
169
+ if (input instanceof Blob) {
170
+ // File or Blob (File object is a specific type of Blob)
171
+ gpkgSource = input;
172
+ } else {
173
+ // URL string or URL object
174
+ const response = await fetch(input);
175
+ if (!response.ok) {
176
+ // Throw will convert to rejected promise (as an async function)
177
+ throw new Error('GPKG file could not be loaded from URL:\n' +
178
+ ` ${input}\n` +
179
+ `Response: (${response.status}) ${response.statusText}`);
180
+ }
181
+ gpkgSource = response;
182
+ }
183
+
184
+ try {
185
+ // Use .bytes() method if available (most browsers since Jan 2025)
186
+ if (typeof gpkgSource.bytes === 'function') {
187
+ return await gpkgSource.bytes();
188
+ }
189
+
190
+ // Fallback approach: Convert ArrayBuffer to Uint8Array
191
+ //console.log('readRawGpkg(): using fallback .arrayBuffer() + Uint8Array() methods');
192
+ const buffer = await gpkgSource.arrayBuffer();
193
+ return new Uint8Array(buffer);
194
+ } catch (error) {
195
+ // Throw will convert to rejected promise (as an async function)
196
+ throw new Error('Unable to extract Byte Array from GPKG file.\n' + error);
197
+ }
127
198
  }
128
199
 
129
200
  /**
130
- * Process OGC GeoPackage (SQLite database) once loaded
131
- * @param {*} loadedGpkgFile - name of GeoPackage file (for diagnostics only)
132
- * @param {ArrayBuffer} gpkgArrayBuffer - ArrayBuffer containing Gpkg data read
201
+ * Process OGC GeoPackage (based on SQLite database) once loaded
202
+ * @param {string} loadedGpkgFile - name of GeoPackage file (for diagnostics only)
203
+ * @param {Uint8Array} gpkgByteArray - Byte Array containing Gpkg data read
133
204
  * @param {WebAssembly} sqlWasm - sql.js SQLITE database access library
134
205
  * @param {string} displayProjection - map display projection (e.g. EPSG:3857)
135
- * @returns {object[]} array of 2 objects: [<data tables>, <slds>]
136
- * <data tables>: OpenLayers vector sources, indexed by table name
137
- * <slds>: SLD XML strings, indexed by layer name
206
+ * @param {loadGpkgOptions} options - configuration options
207
+ * @returns {[Object, Object, gpkgTableStatusObj]} Array of 3 objects:
208
+ * * data tables (OpenLayers vector sources, indexed by table name),
209
+ * * styles (SLD layer_styles XML strings, indexed by layer name),
210
+ * * table status (indexed by table name) which is an object with 3 properties:
211
+ * * (integer) `statusCode`: error code
212
+ * * (string) `statusMsg`: error message
213
+ * * (integer) `origSrsId`: original geopackage table SRS
214
+ * @throws {Error} If unable to extract feature tables from OGC GeoPackage file
215
+ * @throws {Error} If ol/proj is missing required projection for a data table
216
+ * @throws {Error} Failed GeoPackage table load
138
217
  */
139
- function processGpkgData(loadedGpkgFile, gpkgArrayBuffer, sqlWasm,
140
- displayProjection) {
141
- var db;
218
+ function processGpkgData(loadedGpkgFile, gpkgByteArray, sqlWasm,
219
+ displayProjection, options) {
220
+ let db = null;
221
+ let stmt;
222
+ const featureTableNames = [];
142
223
 
143
224
  // Data and associated SLD styles loaded both from GPKG
144
- var dataFromGpkg = {};
145
- var sldsFromGpkg = {};
225
+ const dataFromGpkg = {};
226
+ const sldsFromGpkg = {};
146
227
 
147
- // DEBUG: measure GPKG processing time
148
- //var startProcessing = Date.now();
228
+ /**
229
+ * @type {gpkgTableStatusObj}
230
+ * Table data extraction status
231
+ */
232
+ const gpkgTableStatus = {};
149
233
 
150
- // Convert Array Buffer to Byte Array for SQLite
151
- var gpkgByteArray = new Uint8Array(gpkgArrayBuffer);
234
+ // DEBUG: measure GPKG processing time
235
+ //let startProcessing = Date.now();
152
236
 
153
237
  try {
154
238
  db = new sqlWasm.Database(gpkgByteArray);
155
239
 
156
- // Extract all feature tables, SRS IDs and their geometry types
240
+ // Extract all feature table names, SRS IDs and their geometry types
157
241
  // Note the following fields are not extracted:
158
242
  // gpkg_contents.identifier - title (QGIS: same as table_name)
159
243
  // gpkg_contents.description - human readable (QGIS: blank)
160
244
  // gpkg_geometry_columns.geometry_type_name
161
245
  // - e.g. LINESTRING (but info also embedded in each feature)
162
- var featureTableNames = [];
163
- var stmt;
164
246
  stmt = db.prepare(`
165
247
  SELECT gpkg_contents.table_name, gpkg_contents.srs_id,
166
248
  gpkg_geometry_columns.column_name
@@ -169,18 +251,21 @@ function processGpkgData(loadedGpkgFile, gpkgArrayBuffer, sqlWasm,
169
251
  gpkg_contents.table_name=gpkg_geometry_columns.table_name;
170
252
  `);
171
253
  while (stmt.step()) {
172
- let row = stmt.get();
254
+ const row = stmt.get();
173
255
  featureTableNames.push({
174
256
  'table_name': row[0],
175
257
  'srs_id': row[1].toString(),
176
258
  'geometry_column_name': row[2]
177
259
  });
178
260
  }
261
+ stmt.free();
179
262
  }
180
263
  catch (err) {
181
- throw new Error(
182
- 'Unable to extract feature tables from OGC GeoPackage file "' +
183
- loadedGpkgFile + '":\n' + err);
264
+ if (db) {
265
+ db.close();
266
+ }
267
+ throw new Error('Unable to extract feature table names and types ' +
268
+ 'from OGC GeoPackage file.\n' + err);
184
269
  }
185
270
 
186
271
  // Extract SLD styles for each layer (if styles included in the gpkg)
@@ -192,75 +277,156 @@ function processGpkgData(loadedGpkgFile, gpkgArrayBuffer, sqlWasm,
192
277
  if (stmt.step()) {
193
278
  stmt = db.prepare('SELECT f_table_name,styleSLD FROM layer_styles');
194
279
  while (stmt.step()) {
195
- let row = stmt.get();
280
+ const row = stmt.get();
196
281
  sldsFromGpkg[row[0]] = row[1];
197
282
  }
198
283
  }
284
+ stmt.free();
199
285
 
200
286
  // For each table, extract geometry and other properties
201
287
  // (Note: becomes OpenLayers-specific from here)
202
- var formatWKB = new ol_format_WKB();
203
- for (let table of featureTableNames) {
204
- let features;
205
- let table_name = table.table_name;
206
- let tableDataProjection = 'EPSG:' + table.srs_id;
288
+ const formatWKB = new ol_format_WKB();
289
+ for (const table of featureTableNames) {
290
+ const table_name = table.table_name;
291
+ const tableDataProjection = 'EPSG:' + table.srs_id;
292
+ let noReProject = false;
293
+ let featureCount = 0;
207
294
 
208
295
  // Check if we have a definition for the data projection (SRS)
209
296
  if (!ol_proj_get(tableDataProjection)) {
210
- throw new Error('Missing data projection [' +
211
- tableDataProjection + '] for table "' + table_name +
212
- '" - can be added beforehand with ol/proj/proj4');
297
+ if (options.missingDataSrsAction === 'discard') {
298
+ // discard: Skip processing of data; put error and
299
+ // missing data SRS ID in table status
300
+ console.warn(`Discarding table "${table_name}" from gpkg ` +
301
+ `"${loadedGpkgFile}" as ol/proj missing table SRS ` +
302
+ `${tableDataProjection}`);
303
+ gpkgTableStatus[table_name] = {
304
+ statusCode: 2,
305
+ statusMsg: 'Discarded as ol/proj missing table SRS',
306
+ origSrsId: table.srs_id
307
+ };
308
+ continue;
309
+
310
+ } else if (options.missingDataSrsAction === 'noProject') {
311
+ // noProject: Keep data but do not reproject;
312
+ // put error and missing data SRC in table status
313
+ console.warn(`Not reprojecting table "${table_name}" from ` +
314
+ `gpkg "${loadedGpkgFile}" as ol/proj missing table SRS ` +
315
+ `${tableDataProjection}`);
316
+ gpkgTableStatus[table_name] = {
317
+ statusCode: 1,
318
+ statusMsg: 'Not reprojected as ol/proj missing table SRS',
319
+ origSrsId: table.srs_id
320
+ };
321
+ noReProject = true;
322
+
323
+ } else {
324
+ // stop (default): Throw error, stop processing, discard all tables
325
+ throw new Error('Missing data projection [' +
326
+ tableDataProjection + '] for table "' + table_name +
327
+ '" - can be added beforehand with ol/proj/proj4');
328
+ }
213
329
  }
214
330
 
215
- stmt = db.prepare("SELECT * FROM '" + table_name + "'");
216
- let vectorSource = new ol_source_Vector();
217
- let geometry_column_name = table.geometry_column_name;
218
- let properties = {};
219
- while (stmt.step()) {
220
- // Extract properties & geometry for a single feature
221
- properties = stmt.getAsObject();
222
- let geomProp = properties[geometry_column_name];
223
- delete properties[geometry_column_name];
224
- let featureWkb = parseGpkgGeom(geomProp);
225
- /*
226
- // DEBUG: show endianness of WKB data (can differ from header)
227
- if (!vectorSource.getFeatures().length) {
228
- console.log('WKB Geometry: ' +
229
- (featureWkb[0] ? 'NDR (Little' : 'XDR (Big') + ' Endian)');
331
+ // For each feature in the table, extract geometry + other properties
332
+ // and add these to a new OpenLayers Vector Source for the table.
333
+ //console.log(`Extracting table "${table_name}" (SRS: ${tableDataProjection})`);
334
+ const vectorSource = new ol_source_Vector();
335
+ const geometry_column_name = table.geometry_column_name;
336
+ try {
337
+ stmt = db.prepare("SELECT * FROM `" + table_name + "`");
338
+ while (stmt.step()) {
339
+ // Extract properties & geometry for a single feature
340
+ const properties = stmt.getAsObject();
341
+ const geomProp = properties[geometry_column_name];
342
+ if (geomProp == null) {
343
+ //console.log(`Discarding feature ${featureCount + 1} with null geom in table: '${table_name}'`);
344
+ continue;
345
+ }
346
+ delete properties[geometry_column_name];
347
+ const featureWkb = parseGpkgGeom(geomProp);
348
+ /*
349
+ // DEBUG: show endianness of WKB data (can differ from header)
350
+ if (!vectorSource.getFeatures().length) {
351
+ console.log('WKB Geometry: ' +
352
+ (featureWkb[0] ? 'NDR (Little' : 'XDR (Big') + ' Endian)');
353
+ }
354
+ */
355
+ // Put the feature into the vector source for the current table
356
+ // Reproject unless ol/proj is missing SRS of source data table
357
+ const features = formatWKB.readFeatures(featureWkb, {
358
+ dataProjection: tableDataProjection,
359
+ featureProjection: noReProject ? undefined : displayProjection
360
+ });
361
+ features[0].setProperties(properties);
362
+ vectorSource.addFeatures(features);
363
+ featureCount++;
364
+ }
365
+ stmt.free();
366
+ } catch (error) {
367
+ if (options.failedTableLoadAction === 'discard') {
368
+ // discard: Skip processing of data; put error and data SRS ID in table status
369
+ console.warn(`Discarding table "${table_name}" from gpkg ` +
370
+ `"${loadedGpkgFile}" as table load failed:\n${error}`);
371
+ gpkgTableStatus[table_name] = {
372
+ statusCode: 4,
373
+ statusMsg: 'Failed GeoPackage table load: ' + error,
374
+ origSrsId: table.srs_id
375
+ };
376
+ stmt.free();
377
+ continue;
378
+ } else {
379
+ // stop (default): Throw error, stop processing, discard all tables
380
+ db.close();
381
+ throw error;
230
382
  }
231
- */
232
- // Put the feature into the vector source for the current table
233
- features = formatWKB.readFeatures(featureWkb, {
234
- dataProjection: tableDataProjection,
235
- featureProjection: displayProjection
236
- });
237
- features[0].setProperties(properties);
238
- vectorSource.addFeatures(features);
239
383
  }
240
384
 
385
+ if (featureCount === 0) {
386
+ console.warn('Discarding table with no (or only null geom) features: ' + table_name);
387
+ gpkgTableStatus[table_name] = {
388
+ statusCode: 3,
389
+ statusMsg: 'Discarded as no (or only null geom) features',
390
+ origSrsId: table.srs_id
391
+ };
392
+ continue;
393
+ }
241
394
  // For information only, save details of original projection (SRS)
395
+ // DEPRECATED (as now included in gpkgTableStatus). TBD: remove this soon!
242
396
  vectorSource.setProperties({'origProjection': tableDataProjection});
397
+
243
398
  dataFromGpkg[table_name] = vectorSource;
399
+ if (!noReProject) {
400
+ gpkgTableStatus[table_name] = {
401
+ statusCode: 0,
402
+ statusMsg: 'OK',
403
+ origSrsId: table.srs_id
404
+ };
405
+ }
244
406
  }
407
+
408
+ // Close database (gpkg) to trigger garbage collection
409
+ db.close();
245
410
  /*
246
411
  // DEBUG: measure OGC GeoPackage processing time
247
- var processingSecs = (Date.now() - startProcessing) / 1000;
412
+ let processingSecs = (Date.now() - startProcessing) / 1000;
248
413
  console.log('INFO: OGC GeoPackage file ("' + loadedGpkgFile +
249
414
  '") processing time = ' + processingSecs + ' s');
250
415
  */
251
- return [dataFromGpkg, sldsFromGpkg];
416
+ return [dataFromGpkg, sldsFromGpkg, gpkgTableStatus];
252
417
  }
253
418
 
254
419
  /**
255
420
  * Extract (SRS ID &) WKB from an OGC GeoPackage feature
256
421
  * (i.e. strip off the variable length header)
257
- * @param {object} gpkgBinGeom feature geometry property (includes header)
258
- * @returns feature geometry in WKB (Well Known Binary) format
422
+ * @param {Uint8Array} gpkgBinGeom - feature geometry property (includes header)
423
+ * @returns {Uint8Array} feature geometry in WKB (Well Known Binary) format
424
+ * @throws {Error} If invalid geometry envelope size flag in GeoPackage
259
425
  */
260
426
  function parseGpkgGeom(gpkgBinGeom) {
261
- var flags = gpkgBinGeom[3];
262
- var eFlags = (flags >> 1) & 7;
263
- var envelopeSize;
427
+ const flags = gpkgBinGeom[3];
428
+ const eFlags = (flags >> 1) & 7;
429
+ let envelopeSize;
264
430
  switch (eFlags) {
265
431
  case 0:
266
432
  envelopeSize = 0;
@@ -281,9 +447,9 @@ function parseGpkgGeom(gpkgBinGeom) {
281
447
  /*
282
448
  // Extract SRS (EPSG code)
283
449
  // (not required as given for whole table in gpkg_contents table)
284
- var littleEndian = flags & 1;
285
- var srs = gpkgBinGeom.subarray(4,8);
286
- var srsId;
450
+ const littleEndian = flags & 1;
451
+ const srs = gpkgBinGeom.subarray(4,8);
452
+ let srsId;
287
453
  if (littleEndian) {
288
454
  srsId = srs[0] + (srs[1]<<8) + (srs[2]<<16) + (srs[3]<<24);
289
455
  } else {
@@ -301,6 +467,6 @@ function parseGpkgGeom(gpkgBinGeom) {
301
467
  console.log('gpkgBinGeom envelope size (bytes):', envelopeSize);
302
468
  */
303
469
  // Extract WKB which starts after variable-size "envelope" field
304
- var wkbOffset = envelopeSize + 8;
470
+ const wkbOffset = envelopeSize + 8;
305
471
  return gpkgBinGeom.subarray(wkbOffset);
306
472
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ol-load-geopackage",
3
- "version": "2.1.0",
3
+ "version": "2.3.0",
4
4
  "description": "Loads OGC GeoPackage vector data tables into OpenLayers Vector Sources",
5
5
  "files": [
6
6
  "dist/",