@iebh/reflib 2.0.5 → 2.1.1

package/LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2021 Institute for Evidence-Based Healthcare
+Copyright (c) 2022 Bond University Institute for Evidence-Based Healthcare
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of
 this software and associated documentation files (the "Software"), to deal in

package/LICENSES.md

@@ -0,0 +1,97 @@
+# @iebh/reflib (MIT)
+
+## [JSONStream](https://github.com/dominictarr/JSONStream)
+```
+Apache License, Version 2.0
+
+Copyright (c) 2011 Dominic Tarr
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+```
+
+## [htmlparser2](https://github.com/fb55/htmlparser2)
+```
+Copyright 2010, 2011, Chris Winberry <chris@winberry.net>. All rights reserved.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to
+deal in the Software without restriction, including without limitation the
+rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+sell copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+ 
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+ 
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+IN THE SOFTWARE.
+```
+
+## [mitt](https://github.com/developit/mitt)
+```
+MIT License
+
+Copyright (c) 2021 Jason Miller
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+```
+
+## [vite-plugin-replace](https://github.com/leanupjs/leanup)
+```
+MIT License
+
+Copyright (c) 2021 leanupjs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+```
+
+

package/README.md

@@ -4,8 +4,7 @@ Reference library processing for Node.
 
 This library provides various read/write functionality to process citation libraries and handle individual references (henceforth "Refs").
 
-**NOTE: THIS LIBRARY IS STILL UNDER CONSTRUCTION**
-**Please use [Version 1](https://github.com/hash-bang/Reflib-Node) ([NPM](https://www.npmjs.com/package/reflib)) until this message is removed**
+This module forms part of the [Systematic Review Accelerator](https://sr-accelerator.com)
 
 
 Compatibility
@@ -193,6 +192,19 @@ reflib.readFile('./data/json/json1.json')
 	.then(refs => /* Do something with Ref collection */)
 ```
 
+An emitter is available to track progress while reading. Note that due to the chainable nature of promises the first return contains the `emitter` key only:
+
+```javascript
+let reader = reflib.readFile('./data/json/json1.json');
+
+reader.emitter
+	.on('progress', ({readBytes, totalSize, refsFound}) => /* Report progress somehow */);
+	.on('end', ({refsFound}) => /* Report progress somehow */);
+
+reader
+	.then(refs => /* Do something with Ref collection */)
+```
+
 uploadFile(options)
 ===================
 Available: Browser
@@ -257,12 +269,3 @@ Credits
 =======
 Developed for the [Bond University Institute for Evidence-Based Healthcare](https://iebh.bond.edu.au).
 Please contact [the author](mailto:matt_carter@bond.edu.au) with any issues.
-
-
-TODO
-====
-- [x] Basic parsing iterfaces
-- [ ] "progress" emitter for files
-- [x] `.uploadFile()` browser compatibility
-- [ ] `.downloadFile()` browser compatibility
-- [x] `setup()` functions per module to avoid things like map calculations unless the module is actually needed

package/lib/readFile.js

@@ -1,4 +1,5 @@
 import {createReadStream} from 'node:fs';
+import Emitter from '../shared/emitter.js';
 import {stat} from 'node:fs/promises';
 import {identifyFormat} from './identifyFormat.js';
 import {readStream} from './readStream.js';
@@ -9,16 +10,26 @@ import {readStream} from './readStream.js';
 * @param {string} path The file path to parse
 * @param {Object} [options] Additional options to pass to the parser
 * @param {string} [options.module] The module to use if overriding from the file path
+*
 * @returns {Promise<Array>} An eventual array of all references parsed from the file
+* @property {EventEmitter} emitter An event emitter which will fire the below events
+*
+* @fires progress Emitted as `({readBytes: Number, totalSize: Number, refsFound: Number})`
+* @fires end Emitted as `({refsFound: Number})` when the reading operation has completed
 */
 export function readFile(path, options) {
-	let settings = {...options};
+	let settings = {
+		progressTotal: false,
+		...options,
+	};
 	let module = options?.module || identifyFormat(path)?.id;
 	if (!module) throw new Error(`Unable to identify reference library format for file path "${path}"`);
 
-	return stat(path)
+	let promiseEmitter = Promise.resolve()
+		.then(()=> stat(path))
 		.then(stats => new Promise((resolve, reject) => {
-			let refs = [];
+			let refs = []; // eslint-disable-line no-unused-vars
+
 			readStream(
 				module,
 				createReadStream(path),
@@ -30,5 +41,23 @@ export function readFile(path, options) {
 				.on('end', ()=> resolve(refs))
 				.on('error', reject)
 				.on('ref', ref => refs.push(ref))
+				.on('progress', readBytes => promiseEmitter.emitter.emit('progress', {
+					readBytes,
+					totalSize: stats.size,
+					refsFound: refs.length,
+				}))
 		}))
+		.then(refs => {
+			promiseEmitter.emitter.emit('end', {refsFound: refs.length})
+			return refs;
+		})
+
+	// Extend our base promise with an emitter subkey
+	return Object.defineProperties(promiseEmitter, {
+		emitter: {
+			value: Emitter(),
+			enumerable: true,
+			writable: false,
+		},
+	});
 }

package/modules/endnoteXml.js

@@ -3,6 +3,7 @@ import Emitter from '../shared/emitter.js';
 
 // TODO: CF: Don't need to import both, it depends if we are on browser or node
 import * as htmlparser2 from "htmlparser2";
+// FIXME: CF: Browsers freak out without pollyfills if this is imported
 import {WritableStream as XMLParser} from 'htmlparser2/lib/WritableStream';
 
 /**
@@ -108,6 +109,7 @@ export function readStream(stream) {
 		if (typeof stream.pipe === 'function') {
 			// We are on the node.js client
 			let parser = new XMLParser(parserOptions);
+			stream.on('data', ()=> emitter.emit('progress', stream.bytesRead))
 			stream.pipe(parser)
 			return;
 		}

package/modules/interface.js

@@ -14,6 +14,7 @@
 * @emits ref Emitted with a single ref object when found
 * @emits end Emitted when parsing has completed
 * @emits error Emitted when an error has been raised
+* @emits progress Emitted as (bytesRead) when reading a stream
 */
 export function readStream(stream, options) {
 	// Stub

package/modules/json.js

@@ -1,4 +1,5 @@
 import Emitter from '../shared/emitter.js';
+// FIXME: CF: Browsers freak out without pollyfills if this is imported
 import JSONStream from 'JSONStream';
 
 /**
@@ -9,7 +10,8 @@ export function readStream(stream) {
 	let emitter = Emitter();
 
 	// Queue up the parser in the next tick (so we can return the emitter first)
-	setTimeout(()=>
+	setTimeout(()=> {
+		stream.on('data', ()=> emitter.emit('progress', stream.bytesRead));
 		stream.pipe(
 			JSONStream.parse('*')
 				.on('data', ref => emitter.emit('ref', {
@@ -19,7 +21,7 @@ export function readStream(stream) {
 				.on('end', ()=> emitter.emit('end'))
 				.on('error', emitter.emit.bind('error'))
 		)
-	);
+	});
 
 	return emitter;
 }

package/modules/medline.js

@@ -122,6 +122,7 @@ export function readStream(stream, options) {
 	setTimeout(()=> {
 		stream
 			.on('data', chunkBuffer => {
+				emitter.emit('progress', stream.bytesRead);
 				buffer += chunkBuffer.toString(); // Append incomming data to the partial-buffer we're holding in memory
 
 				let bufferCrop = 0; // How many bytes to shift off the front of the buffer based on the last full reference we saw, should end up at the last byte offset of buffer that is valid to shift-truncate to

package/modules/ris.js

@@ -21,6 +21,7 @@ export function readStream(stream, options) {
 	setTimeout(()=> {
 		stream
 			.on('data', chunkBuffer => {
+				emitter.emit('progress', stream.bytesRead);
 				buffer += chunkBuffer.toString(); // Append incomming data to the partial-buffer we're holding in memory
 
 				let bufferCrop = 0; // How many bytes to shift off the front of the buffer based on the last full reference we saw, should end up at the last byte offset of buffer that is valid to shift-truncate to

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@iebh/reflib",
-  "version": "2.0.5",
+  "version": "2.1.1",
   "description": "Reference / Citation reference library utilities",
   "scripts": {
     "lint": "eslint lib modules shared test",
@@ -17,7 +17,10 @@
     "citations",
     "library"
   ],
-  "author": "Matt Carter <m@ttcarter.com>",
+  "author": [
+    "Matt Carter <m@ttcarter.com> (https://github.com/hash-bang)",
+    "Connor Forbes <cforbes.software@gmail.com> (https://github.com/connorf25)"
+  ],
   "license": "MIT",
   "bugs": {
     "url": "https://github.com/IEBH/RefLib/issues"
@@ -40,13 +43,13 @@
     "eslint": "^8.15.0",
     "mocha": "^10.0.0",
     "mocha-logger": "^1.0.8",
-    "temp": "^0.9.4"
+    "temp": "^0.9.4",
+    "vite-plugin-replace": "^0.1.1"
   },
   "dependencies": {
     "htmlparser2": "^8.0.1",
     "JSONStream": "^1.3.5",
-    "mitt": "^3.0.0",
-    "vite-plugin-replace": "^0.1.1"
+    "mitt": "^3.0.0"
   },
   "eslintConfig": {
     "extends": "eslint:recommended",