npm - officeparser - Versions diffs - 5.0.0 → 5.2.0 - Mend

officeparser 5.0.0 → 5.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (7) hide show

package/README.md +42 -0
package/officeParser.js +47 -28
package/package.json +2 -1
package/pdfjs-dist-build/pdf.js +0 -17644
package/pdfjs-dist-build/pdf.js.map +0 -1
package/pdfjs-dist-build/pdf.worker.js +0 -45163
package/pdfjs-dist-build/pdf.worker.js.map +0 -1

package/README.md CHANGED Viewed

@@ -13,6 +13,7 @@ A Node.js library to parse text out of any office file.
 #### Update
+* 2024/11/12 - Added ArrayBuffer as a type of file input. Generating bundle files now which exposes namespace officeParser to be able to access parseOffice and parseOfficeAsync directly on the browser. Extracting text out of pdf files does not work currently in browser bundles.
 * 2024/10/21 - Replaced extracting zip files from decompress to yauzl. This means that we now extract files in memory and we no longer need to write them to disk. Removed config flags related to extracted files. Added flags for CLI execution.
 * 2024/10/15 - Fixed erroring out while deleting temp files when multiple worker threads make parallel executions resulting in same file name for multiple files. Fixed erroring out when multiple executions are made without waiting for the previous execution to finish which resulted in deleting the file from other execution. Upgraded dependencies.
 * 2024/10/13 - Fixed parsing text from xlsx files which contain no shared strings file and files which have inlineStr based strings.
@@ -185,10 +186,51 @@ function searchForTermInOfficeFile(searchterm: string, filepath: string): Promis
 \
 **Please take note: I have breached convention in placing err as second argument in my callback but please understand that I had to do it to not break other people's existing modules.**
+## Browser Usage
+Download the bundle file available as part of the release asset.
+Include this bundle file in your browser html file and access `parseOffice` and `parseOfficeAsync` under the **`officeParser`** namespace.
+**Example**
+```html
+<head>
+    ...
+    <!-- Include bundle file in the script tag. -->
+    <script src="officeParserBundle@5.1.0.js"></script>
+</head>
+<body>
+    ...
+    <input type="file" id="fileInput" />
+    ...
+    <script>
+        document.getElementById('fileInput').addEventListener('change', async function(event) {
+            const outputDiv = document.getElementById('output');
+            const file = event.target.files[0];
+            try {
+                // Your configuration options for officeParser
+                const config = {
+                    outputErrorToConsole: false,
+                    newlineDelimiter: '\n',
+                    ignoreNotes: false,
+                    putNotesAtLast: false
+                };
+                const arrayBuffer = await file.arrayBuffer();
+                const result = await officeParser.parseOfficeAsync(arrayBuffer, config);
+                // result contains the extracted text.
+            }
+            catch (error) {
+                // Handle error
+            }
+        });
+    </script>
+</body>
+```
 ## Known Bugs
 1. Inconsistency and incorrectness in the positioning of footnotes and endnotes in .docx files where the footnotes and endnotes would end up at the end of the parsed text whereas it would be positioned exactly after the referenced word in .odt files.
 2. The charts and objects information of .odt files are not accurate and may end up showing a few NaN in some cases.
+3. Extracting texts in browser bundles does not work for pdf files.
 ----------
 **npm**

package/officeParser.js CHANGED Viewed

@@ -6,9 +6,11 @@ const concat        = require('concat-stream');
 const { DOMParser } = require('@xmldom/xmldom');
 const fileType      = require('file-type');
 const fs            = require('fs');
-const pdfjs         = require('./pdfjs-dist-build/pdf.js');
 const yauzl         = require('yauzl');
+/** Load pdfjs-dist once at module scope. This returns a Promise that resolves to the module. */
+const pdfjsPromise = import('pdfjs-dist/legacy/build/pdf.mjs');
 /** Header for error messages */
 const ERRORHEADER = "[OfficeParser]: ";
 /** Error messages */
@@ -271,8 +273,10 @@ function parseExcel(file, callback, config) {
                                         ? sharedStrings[value]
                                         : value;
                             }
-                            // TODO: Add debug asserts for if we reach here which would mean we are filtering more items than we are processing.
-                            // Not the case now but it could happen and it is better to be safe.
+                            // Should not reach here. If we do, it means we are not filtering out items that we are not ready to process.
+                            // Not the case now but it could happen if we change the filtering logic without updating the processing logic.
+                            // So, it is better to error out here.
+                            handleError(`Invalid c node found in sheet xml content: ${cNode}`, callback, config.outputErrorToConsole);
                             return '';
                         })
                         // Join each cell text within a sheet with a space.
@@ -442,14 +446,17 @@ function parseOpenOffice(file, callback, config) {
  * @param {string | Buffer}    file     File path or Buffers
  * @param {function}           callback Callback function that returns value or error
  * @param {OfficeParserConfig} config   Config Object for officeParser
- * @returns {void}
+ * @returns {Promise<void>}
  */
-function parsePdf(file, callback, config) {
-    // Get the pdfjs document for the filepath or buffers.
-    // @ts-ignore
-    pdfjs.getDocument(file).promise
+async function parsePdf(file, callback, config) {
+    // Wait for pdfjs module to be loaded once
+    const pdfjs = await pdfjsPromise;
+    // Get the pdfjs document for the filepath or Uint8Array buffers.
+    // pdfjs does not accept Buffers directly, so we convert them to Uint8Array.
+    pdfjs.getDocument(file instanceof Buffer ? new Uint8Array(file) : file).promise
         // We go through each page and build our text content promise array.
-        .then(document => Promise.all(Array.from({ length: document.numPages }, (_, index) => index + 1).map(pageNr => document.getPage(pageNr).then(page => page.getTextContent()))))
+        .then(document => Promise.all(Array.from({ length: document.numPages }, (_, index) => document.getPage(index + 1).then(page => page.getTextContent()))))
         // Each textContent item has property 'items' which is an array of objects.
         // Each object element in the array has text stored in their 'str' key.
         // The concatenation of str is what makes our pdf content.
@@ -462,29 +469,35 @@ function parsePdf(file, callback, config) {
             const responseText = textContentArray
                                     .map(textContent => textContent.items)      // Get all the items
                                     .flat()                                     // Flatten all the items object
-                                    .filter(item => item.str != '')             // Ignore the empty string items.
-                                    .reduce((a, v) => (
-                                        {
-                                            text: a.text + (v.transform[5] != a.transform5 ? (config.newlineDelimiter ?? "\n") : '') + v.str,
-                                            transform5: v.transform[5]
-                                        }),
-                                        {
-                                            text: '',
-                                            transform5: undefined
-                                        }).text;
+                                    .reduce((a, v) =>  (
+                                        // the items could be TextItem or a TextMarkedContent.
+                                        // We are only interested in the TextItem which has a str property.
+                                        'str' in v && v.str != ''
+                                            ? {
+                                                text: a.text + (v.transform[5] != a.transform5 ? (config.newlineDelimiter ?? "\n") : '') + v.str,
+                                                transform5: v.transform[5]
+                                            } : {
+                                                text: a.text,
+                                                transform5: a.transform5
+                                            }
+                                    ),
+                                    {
+                                        text: '',
+                                        transform5: undefined
+                                    }).text;
             callback(responseText, undefined);
         })
         .catch(e => callback(undefined, e));
 }
 /** Main async function with callback to execute parseOffice for supported files
- * @param {string | Buffer}    file        File path or file buffers
- * @param {function}           callback    Callback function that returns value or error
- * @param {OfficeParserConfig} [config={}] [OPTIONAL]: Config Object for officeParser
+ * @param {string | Buffer | ArrayBuffer} srcFile      File path or file buffers or Javascript ArrayBuffer
+ * @param {function}                      callback     Callback function that returns value or error
+ * @param {OfficeParserConfig}            [config={}]  [OPTIONAL]: Config Object for officeParser
  * @returns {void}
  */
-function parseOffice(file, callback, config = {}) {
+function parseOffice(srcFile, callback, config = {}) {
     // Make a clone of the config with default values such that none of the config flags are undefined.
     /** @type {OfficeParserConfig} */
     const internalConfig = {
@@ -494,6 +507,12 @@ function parseOffice(file, callback, config = {}) {
         outputErrorToConsole: false,
         ...config
     };
+    // Our internal code can process regular node Buffers or file path.
+    // So, if the src file was presented as ArrayBuffers, we create Buffers from them.
+    let file = srcFile instanceof ArrayBuffer ? Buffer.from(srcFile)
+                                              : srcFile;
     /**
      * Prepare file for processing
      * @type {Promise<{ file:string | Buffer, ext: string}>}
@@ -559,13 +578,13 @@ function parseOffice(file, callback, config = {}) {
 }
 /** Main async function that can be used with await to execute parseOffice. Or it can be used with promises.
- * @param {string | Buffer}    file        File path or file buffers
- * @param {OfficeParserConfig} [config={}] [OPTIONAL]: Config Object for officeParser
+ * @param {string | Buffer | ArrayBuffer} srcFile     File path or file buffers or Javascript ArrayBuffer
+ * @param {OfficeParserConfig}            [config={}] [OPTIONAL]: Config Object for officeParser
  * @returns {Promise<string>}
  */
-function parseOfficeAsync(file, config = {}) {
+function parseOfficeAsync(srcFile, config = {}) {
     return new Promise((res, rej) => {
-        parseOffice(file, function (data, err) {
+        parseOffice(srcFile, function (data, err) {
             if (err)
                 return rej(err);
             return res(data);

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "officeparser",
-  "version": "5.0.0",
+  "version": "5.2.0",
   "description": "A Node.js library to parse text out of any office file. Currently supports docx, pptx, xlsx, odt, odp, ods, pdf files.",
   "main": "officeParser.js",
   "files": [
@@ -47,6 +47,7 @@
     "concat-stream": "^2.0.0",
     "file-type": "^16.5.4",
     "node-ensure": "^0.0.0",
+    "pdfjs-dist": "^5.3.31",
     "yauzl": "^3.1.3"
   },
   "devDependencies": {