mindee 1.3.2 → 2.0.0

package/CHANGELOG.md

@@ -1,21 +1,50 @@
 # CHANGELOG
 
+## v2.0.0 (2022-08-26)
+### Note
+This is a complete rewrite in TypeScript!
+
+It's **not** backwards compatible with previous versions.
+
+### Changes
+* :sparkles: Add Passport API support
+* :sparkles: Add custom document support (API builder)
+* :sparkles: Add basic geometry functions
+* :label: everything is typed, in particular response objects.
+  Allows for much easier auto-completion in your IDE.
+* :arrow_up: all major dependencies updated
+
+## v1.4.0
+### Fixes
+* :bug: cut/merge pages of an encrypted PDF leads to unexpected results
+* :bug: logger not activating in some cases
+* :bug: align PDF cut/merge with other client libs
+
+### Changes
+* :sparkles: Add TIFF and HEIC support
+* :white_check_mark: fully test PDF cut function
+
+## v1.3.3
+
+### Fixes
+* :bug: reject errors instead of printing them only to the console
+
 ## v1.3.2
 
 ### Fixes
-* 🐛 use PDF length instead of cut PDF size limit
+* :bug: use PDF length instead of cut PDF size limit
 
 ## v1.3.1
 
 ### Fixes
-* 🐛 forward api error to stderr instead of stdout
+* :bug: forward api error to stderr instead of stdout
 
 ## v1.3.0
 
 ### Changes
 
 * :sparkles: allow sending the filename for streams and base64
-* fixes :bug: not able to specify MIME type
+* :bug: not able to specify MIME type
 * :page_facing_up: change license to MIT
 * :white_check_mark: add node18 to tests
 * various package updates
@@ -24,43 +53,43 @@
 
 ### Fixes
 
-* fix: 🐛 Get `words` values when `includeWords` parameter is `true`
+* :bug: Get `words` values when `includeWords` parameter is `true`
 
 ## v1.1.2
 
 ### Fixes
 
-* fix: 🐛 Fix returned probability value
-* fix: 🐛 Fix returned bbox value
+* :bug: Fix returned probability value
+* :bug: Fix returned bbox value
 
 ### new
 
-* new: ✨ New Mindee Invoice API V3 support
-* new: ✨ Add new supplier and customer fields
+* :sparkles: New Mindee Invoice API V3 support
+* :sparkles: Add new supplier and customer fields
 
 ## v1.1.0 (2022-01-03)
 
 ### new
 
-* chg: :sparkles: New Mindee API V2 support
-* chg: :zap: Server side invoice reconstruction from multi-page pdfs
-* new: :sparkles: Added OS in User-Agent headers
+* :sparkles: New Mindee API V2 support
+* :zap: Server side invoice reconstruction from multi-page pdfs
+* :sparkles: Added OS in User-Agent headers
 
 
 ## v1.0.9 (2021-12-09)
 
 ### Fixes
 
-* fix: 🐛 handle attribute of type array when merging pages
+* :bug: handle attribute of type array when merging pages
 
 ## v1.0.8 (2021-11-29)
 
 ### Fixes
 
-* 🐛 fix cutPDF for ReadableStream + add it for base64 file string
-* 🐛 Updated api Input initialization to specifically declare parameters
-* 🐛 prevent error when the mime type isn't detectable
-* 🐛 raise proper error when the APi doesn't return a valid JSON
+* :bug: fix cutPDF for ReadableStream + add it for base64 file string
+* :bug: Updated api Input initialization to specifically declare parameters
+* :bug: prevent error when the mime type isn't detectable
+* :bug: raise proper error when the APi doesn't return a valid JSON
 
 ### new
 
@@ -70,7 +99,7 @@
 
 ### New
 
-* ✨ Added pdf page number parameter for multi-pages pdfs with file
+* :sparkles: Added pdf page number parameter for multi-pages pdfs with file
 
 ### Changes
 
@@ -109,4 +138,4 @@
 
 ## v1.0.1 (2021-01-11)
 
-* 🎉 First release
+* :tada: First release

package/README.md

@@ -1,150 +1,66 @@
-# Mindee Client Library for Node.js
+# Mindee API Helper Library for Node.js
+Quickly and easily connect to Mindee's API services using Node.js.
 
-# Installation
+## Quick Start
+Here's the TL;DR of getting started.
 
-## Requirements
+First, get an [API Key](https://developers.mindee.com/docs/create-api-key)
 
-The SDK is officially supported on Node.js LTS _active_ and _maintenance_
-[versions](https://nodejs.org/en/about/releases/).
-
-## From source code
-
-Download and decompress or clone the source code from GitHub and then do in your Node.js project:
-
-```sh
+Then, install this library:
+```shell
 npm install mindee
 ```
 
-# Usage
-
-You can use the SDK easily by creating a Client like this:
-
-```js
-const { Client } = require("mindee");
+Finally, Node.js away!
 
-const mindeeClient = new Client({
-  invoiceToken: "invoiceApiToken",
-  receiptToken: "receiptExpenseApiToken",
-});
-
-mindeeClient.invoice.parse({ input: "path/to/file", inputType: "file" });
-mindeeClient.receipt.parse({ input: base64String, inputType: "base64" });
-mindeeClient.financialDocument.parse({
-  input: base64String,
-  inputType: "base64",
-});
-```
+### Off-the-Shelf Document
 
-Three APIs are actually supported: invoice (`ìnvoice`), receipt (`receipt`) and financial document (`financialDocument`)
+```ts
+import { Client, InvoiceResponse } from "mindee";
 
-You can find more examples on how to use the SDK into the folder `examples`.
+// Init a new client
+const mindeeClient = new Client({apiKey: "my-api-key"});
 
-## MIME Types
-When using `stream` or `base64` inputs, the MIME type is guessed using magic numbers.
-In some cases, for example with non-standard files, it will not be possible to detect the MIME type in this way.
+// Load a file from disk and parse it
+const invoiceResponse = mindeeClient.docFromPath("/path/to/the/invoice.pdf")
+  .parse(InvoiceResponse);
 
-If you are getting MIME type errors, you will need to specify a file name having the correct extension, for example:
-```js
-mindeeClient.invoice.parse({
-  input: base64String,
-  inputType: "base64",
-  filename: "myfile.pdf",
+// Print a brief summary of the parsed data
+invoiceResponse.then((resp) => {
+  console.log(resp.document);
 });
 ```
 
-## Client
-
-The Mindee client can take multiple parameters to be initialize.
-Some this parameters can also be set with env variables.
-If the env variable and the parameter are both set, the parameter will be the one chosen by the client.
-This is a list of them:
-
-- invoiceToken (env variable : `MINDEE_INVOICE_TOKEN`) The expense api token
-- receiptToken (env variable : `MINDEE_RECEIPT_TOKEN`) The invoice api token
-- throwOnError (`true` by default) If `true`, the SDK will throw on error
-- debug (env variable : `MINDEE_DEBUG`, `false` by default) If `true`, logging will be in debug mode
-
-Depending on what type of document you want to parse, you need to add specifics auth token for each endpoint.
-We suggest storing your credentials as environment variables for security reasons.
-
-## Parsing Invoices
-
-```js
-const { Client } = require("mindee");
-const fs = require("fs");
-
-// Invoice token can be set by Env (MINDEE_INVOICE_TOKEN) or via params (Client({invoiceToken: "token"}))
-const mindeeClient = new Client();
-
-// parsing invoice from pdf
-mindeeClient.invoice
-  .parse({ input: "./documents/invoices/invoice.pdf" }) // see examples for more input types
-  .then((res) => {
-    console.log("Success !");
-    console.log(res.invoices);
-    console.log(res.invoice);
-  })
-  .catch((err) => {
-    console.error(err);
-  });
-```
-
-## Parsing receipts
-
-```js
-const { Client } = require("mindee");
-const fs = require("fs");
-
-// Receipt token can be set by Env (MINDEE_RECEIPT_TOKEN) or via params (Client({receiptToken: "token"}))
-const mindeeClient = new Client();
-
-// parsing receipt from picture
-mindeeClient.receipt
-  .parse({ input: "./documents/receipts/receipt.jpg" }) // see examples for more input types
-  .then((res) => {
-    console.log("Success !");
-    console.log(res.receipts);
-    console.log(res.receipt);
-  })
-  .catch((err) => {
-    console.error(err);
-  });
-```
-
-## Parsing a mix of invoices and receipts: Financial document
+### Custom Document (API Builder)
 
-The Financial document API optimizes the parsing results when you don't know if your file is a receipt or an invoice.
+```ts
+import { Client, CustomResponse } from "mindee";
 
-```js
-const { Client } = require("mindee");
-
-// Invoice token and Receipt token must be set
-// Receipt token can be set by Env (MINDEE_RECEIPT_TOKEN) or via params (Client({receiptToken: "token"}))
-// Invoice token can be set by Env (MINDEE_INVOICE_TOKEN) or via params (Client({invoiceToken: "token"}))
+// Init a new client and add your document endpoint
+const mindeeClient = new Client({apiKey: "my-api-key"})
+  .addEndpoint({
+    accountName: "john",
+    documentType: "wsnine",
+});
 
-const mindeeClient = new Client();
+// Load a file from disk and parse it
+const customResponse = mindeeClient.docFromPath("/path/to/the/wsnine.jpg")
+  .parse(CustomResponse, {docType: "wsnine"});
 
-// parsing receipt from picture
-mindeeClient.financialDocument
-  .parse({ input: "./documents/receipts/receipt.jpg" })
-  .then((res) => {
-    console.log("Success !");
-    console.log(res.financialDocuments);
-    console.log(res.financialDocument);
-  })
-  .catch((err) => {
-    console.error(err);
-  });
+// Print a brief summary of the parsed data
+customResponse.then((resp) => {
+  console.log(resp.document);
+});
 ```
 
-# Tests
-
-To run the tests, use the command `npm run test`
+## Further Reading
+There's more to it than that for those that need more features, or want to
+customize the experience.
 
-# Documentation
+All the juicy details are described in the
+**[Official Documentation](https://developers.mindee.com/docs/nodejs-sdk)**.
 
-A basic documentation of the sdk has been done with the help of JSDoc.
-If you want to generate it, you can use the command `npm run documentation`.
-A folder `docs` will be created with generated documentation.
-Open the `index.html` in your browser to read the documentation.
+## License
+Copyright © Mindee
 
+Available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

package/bin/mindee

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+
+require('../dist/src/cli').cli();

package/dist/package.json

@@ -0,0 +1,70 @@
+{
+    "name": "mindee",
+    "version": "2.0.0",
+    "description": "Mindee Client Library for Node.js",
+    "main": "src/index.js",
+    "license": "MIT",
+    "scripts": {
+        "build": "tsc --build",
+        "clean": "rm -rf ./dist",
+        "test": "mocha 'tests/**/*.spec.ts'",
+        "lint-fix": "eslint 'src/**/*.ts' --fix",
+        "lint": "eslint 'src/**/*.ts' --report-unused-disable-directives && echo 'Your .ts files look good.'",
+        "docs": "jsdoc -r src -d docs"
+    },
+    "files": [
+        "dist/",
+        "bin/",
+        "LICENSE",
+        "README.md",
+        "CHANGELOG.md"
+    ],
+    "engines": {
+        "node": ">= 14.0.0"
+    },
+    "repository": {
+        "type": "git",
+        "url": "git+https://github.com/mindee/mindee-api-nodejs.git"
+    },
+    "author": {
+        "name": "Mindee",
+        "email": "devrel@mindee.com",
+        "url": "https://mindee.com/"
+    },
+    "bugs": {
+        "url": "https://github.com/mindee/mindee-api-nodejs/issues"
+    },
+    "homepage": "https://mindee.com/",
+    "devDependencies": {
+        "@tsconfig/node16": "^1.0.3",
+        "@types/chai": "^4.3.1",
+        "@types/mocha": "^9.1.1",
+        "@types/node": "^17.0.38",
+        "@typescript-eslint/eslint-plugin": "^5.30.0",
+        "@typescript-eslint/parser": "^5.30.0",
+        "chai": "^4.3.6",
+        "eslint": "^8.18.0",
+        "eslint-config-prettier": "^8.5.0",
+        "eslint-plugin-prettier": "^4.1.0",
+        "jsdoc": "^3.6.10",
+        "mocha": "^10.0.0",
+        "prettier": "^2.7.1",
+        "ts-node": "^10.8.1",
+        "typescript": "^4.7.4"
+    },
+    "dependencies": {
+        "commander": "^9.4.0",
+        "file-type": "^16.5.4",
+        "form-data": "^3.0.1",
+        "mrz": "^3.1.4",
+        "pdf-lib": "^1.17.1"
+    },
+    "keywords": [
+        "typescript",
+        "mindee",
+        "api",
+        "client",
+        "client library",
+        "nodejs"
+    ]
+}

package/dist/src/api/endpoint.d.ts

@@ -0,0 +1,29 @@
+import { Input } from "../inputs";
+export declare const STANDARD_API_OWNER = "mindee";
+export declare const API_KEY_ENVVAR_NAME = "MINDEE_API_KEY";
+export declare class Endpoint {
+    apiKey: string;
+    urlName: string;
+    keyName: string;
+    owner: string;
+    version: string;
+    urlRoot: string;
+    constructor(owner: string, urlName: string, version: string, apiKey: string, keyName?: string);
+    /**
+     * Make a prediction request.
+     */
+    predictRequest(input: Input, includeWords?: boolean): Promise<unknown>;
+    protected apiKeyFromEnv(): string;
+}
+export declare class InvoiceEndpoint extends Endpoint {
+    constructor(apiKey: string);
+}
+export declare class ReceiptEndpoint extends Endpoint {
+    constructor(apiKey: string);
+}
+export declare class PassportEndpoint extends Endpoint {
+    constructor(apiKey: string);
+}
+export declare class CustomEndpoint extends Endpoint {
+    constructor(documentType: string, accountName: string, version: string, apiKey: string);
+}

package/dist/src/api/endpoint.js

@@ -0,0 +1,148 @@
+"use strict";
+var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    var desc = Object.getOwnPropertyDescriptor(m, k);
+    if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
+      desc = { enumerable: true, get: function() { return m[k]; } };
+    }
+    Object.defineProperty(o, k2, desc);
+}) : (function(o, m, k, k2) {
+    if (k2 === undefined) k2 = k;
+    o[k2] = m[k];
+}));
+var __setModuleDefault = (this && this.__setModuleDefault) || (Object.create ? (function(o, v) {
+    Object.defineProperty(o, "default", { enumerable: true, value: v });
+}) : function(o, v) {
+    o["default"] = v;
+});
+var __importStar = (this && this.__importStar) || function (mod) {
+    if (mod && mod.__esModule) return mod;
+    var result = {};
+    if (mod != null) for (var k in mod) if (k !== "default" && Object.prototype.hasOwnProperty.call(mod, k)) __createBinding(result, mod, k);
+    __setModuleDefault(result, mod);
+    return result;
+};
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.CustomEndpoint = exports.PassportEndpoint = exports.ReceiptEndpoint = exports.InvoiceEndpoint = exports.Endpoint = exports.API_KEY_ENVVAR_NAME = exports.STANDARD_API_OWNER = void 0;
+const https = __importStar(require("https"));
+const os = __importStar(require("os"));
+const package_json_1 = require("../../package.json");
+const url_1 = require("url");
+const form_data_1 = __importDefault(require("form-data"));
+const logger_1 = require("../logger");
+const MINDEE_API_URL = "https://api.mindee.net/v1";
+const USER_AGENT = `mindee-api-nodejs@v${package_json_1.version} nodejs-${process.version} ${os.type().toLowerCase()}`;
+exports.STANDARD_API_OWNER = "mindee";
+exports.API_KEY_ENVVAR_NAME = "MINDEE_API_KEY";
+class Endpoint {
+    constructor(owner, urlName, version, apiKey, keyName) {
+        this.owner = owner;
+        this.urlName = urlName;
+        this.version = version;
+        this.keyName = keyName || urlName;
+        this.apiKey = apiKey || this.apiKeyFromEnv();
+        this.urlRoot = `${MINDEE_API_URL}/products/${owner}/${urlName}/v${version}`;
+    }
+    /**
+     * Make a prediction request.
+     */
+    predictRequest(input, includeWords = false) {
+        return new Promise((resolve, reject) => {
+            const form = new form_data_1.default();
+            let body;
+            let headers = {
+                "User-Agent": USER_AGENT,
+                Authorization: `Token ${this.apiKey}`,
+            };
+            // TODO: redo this section given there should only
+            //       be a single way of reading the input doc
+            if (["path", "stream"].includes(input.inputType)) {
+                const fileParams = { filename: input.filename };
+                form.append("document", input.fileObject, fileParams);
+                if (includeWords) {
+                    form.append("include_mvision", "true");
+                }
+                headers = Object.assign(Object.assign({}, headers), form.getHeaders());
+            }
+            else if (input.inputType === "base64") {
+                const bodyObj = { document: input.fileObject };
+                if (includeWords) {
+                    bodyObj["include_mvision"] = "true";
+                }
+                body = JSON.stringify(bodyObj);
+                headers["Content-Type"] = "application/json";
+                headers["Content-Length"] = body.length;
+            }
+            const uri = new url_1.URL(`${this.urlRoot}/predict`);
+            const options = {
+                method: "POST",
+                headers: headers,
+                hostname: uri.hostname,
+                path: `${uri.pathname}${uri.search}`,
+            };
+            logger_1.logger.debug(`Prediction request: ${uri}`);
+            const req = https.request(options, function (res) {
+                // when the encoding is set, data chunks will be strings
+                res.setEncoding("utf-8");
+                let responseBody = "";
+                res.on("data", function (chunk) {
+                    responseBody += chunk;
+                });
+                res.on("end", function () {
+                    try {
+                        resolve(Object.assign(Object.assign({}, res), { data: JSON.parse(responseBody) }));
+                    }
+                    catch (error) {
+                        reject(error);
+                    }
+                });
+            });
+            req.on("error", (err) => {
+                reject(err);
+            });
+            if (["path", "stream"].includes(input.inputType)) {
+                form.pipe(req);
+            }
+            if (input.inputType === "base64") {
+                req.write(body);
+                req.end();
+            }
+        });
+    }
+    apiKeyFromEnv() {
+        const envVarValue = process.env[exports.API_KEY_ENVVAR_NAME];
+        if (envVarValue) {
+            logger_1.logger.debug(`Set '${this.keyName}' API key from environment: ${exports.API_KEY_ENVVAR_NAME}`);
+            return envVarValue;
+        }
+        return "";
+    }
+}
+exports.Endpoint = Endpoint;
+class InvoiceEndpoint extends Endpoint {
+    constructor(apiKey) {
+        super(exports.STANDARD_API_OWNER, "invoices", "3", apiKey, "invoice");
+    }
+}
+exports.InvoiceEndpoint = InvoiceEndpoint;
+class ReceiptEndpoint extends Endpoint {
+    constructor(apiKey) {
+        super(exports.STANDARD_API_OWNER, "expense_receipts", "3", apiKey, "receipt");
+    }
+}
+exports.ReceiptEndpoint = ReceiptEndpoint;
+class PassportEndpoint extends Endpoint {
+    constructor(apiKey) {
+        super(exports.STANDARD_API_OWNER, "passport", "1", apiKey);
+    }
+}
+exports.PassportEndpoint = PassportEndpoint;
+class CustomEndpoint extends Endpoint {
+    constructor(documentType, accountName, version, apiKey) {
+        super(accountName, documentType, version, apiKey);
+    }
+}
+exports.CustomEndpoint = CustomEndpoint;

package/dist/src/api/index.d.ts

@@ -0,0 +1,2 @@
+export { Response, InvoiceResponse, ReceiptResponse, FinancialDocResponse, PassportResponse, CustomResponse, } from "./response";
+export { Endpoint, InvoiceEndpoint, ReceiptEndpoint, PassportEndpoint, CustomEndpoint, STANDARD_API_OWNER, API_KEY_ENVVAR_NAME, } from "./endpoint";

package/dist/src/api/index.js

@@ -0,0 +1,18 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.API_KEY_ENVVAR_NAME = exports.STANDARD_API_OWNER = exports.CustomEndpoint = exports.PassportEndpoint = exports.ReceiptEndpoint = exports.InvoiceEndpoint = exports.Endpoint = exports.CustomResponse = exports.PassportResponse = exports.FinancialDocResponse = exports.ReceiptResponse = exports.InvoiceResponse = exports.Response = void 0;
+var response_1 = require("./response");
+Object.defineProperty(exports, "Response", { enumerable: true, get: function () { return response_1.Response; } });
+Object.defineProperty(exports, "InvoiceResponse", { enumerable: true, get: function () { return response_1.InvoiceResponse; } });
+Object.defineProperty(exports, "ReceiptResponse", { enumerable: true, get: function () { return response_1.ReceiptResponse; } });
+Object.defineProperty(exports, "FinancialDocResponse", { enumerable: true, get: function () { return response_1.FinancialDocResponse; } });
+Object.defineProperty(exports, "PassportResponse", { enumerable: true, get: function () { return response_1.PassportResponse; } });
+Object.defineProperty(exports, "CustomResponse", { enumerable: true, get: function () { return response_1.CustomResponse; } });
+var endpoint_1 = require("./endpoint");
+Object.defineProperty(exports, "Endpoint", { enumerable: true, get: function () { return endpoint_1.Endpoint; } });
+Object.defineProperty(exports, "InvoiceEndpoint", { enumerable: true, get: function () { return endpoint_1.InvoiceEndpoint; } });
+Object.defineProperty(exports, "ReceiptEndpoint", { enumerable: true, get: function () { return endpoint_1.ReceiptEndpoint; } });
+Object.defineProperty(exports, "PassportEndpoint", { enumerable: true, get: function () { return endpoint_1.PassportEndpoint; } });
+Object.defineProperty(exports, "CustomEndpoint", { enumerable: true, get: function () { return endpoint_1.CustomEndpoint; } });
+Object.defineProperty(exports, "STANDARD_API_OWNER", { enumerable: true, get: function () { return endpoint_1.STANDARD_API_OWNER; } });
+Object.defineProperty(exports, "API_KEY_ENVVAR_NAME", { enumerable: true, get: function () { return endpoint_1.API_KEY_ENVVAR_NAME; } });

package/dist/src/api/response.d.ts

@@ -0,0 +1,52 @@
+import { Document, Passport, Receipt, Invoice, FinancialDocument, CustomDocument, DocumentConstructorProps } from "../documents";
+import { FullText } from "../fields";
+import { Input } from "../inputs";
+export interface ResponseProps {
+    httpResponse: any;
+    documentType: string;
+    input: Input;
+    error: boolean;
+}
+/**
+ * Base class for all responses.
+ */
+export declare class Response<DocType extends Document> {
+    httpResponse: any;
+    readonly documentType: string;
+    inputFile: Input;
+    document?: DocType;
+    pages: Array<DocType>;
+    constructor(params: ResponseProps);
+    protected getPageText(httpDataDocument: any, pageId: number): FullText;
+}
+/**
+ * Class for all constructed (API Builder) endpoint responses.
+ */
+export declare class CustomResponse extends Response<CustomDocument> {
+    constructor(params: ResponseProps);
+    protected formatResponse(): void;
+}
+declare type StandardDocumentSig<DocType extends Document> = {
+    new ({ apiPrediction, inputFile, pageId, fullText, }: DocumentConstructorProps): DocType;
+};
+/**
+ * Generic class for all standard (Off-the-Shelf) endpoint responses.
+ */
+export declare class StandardProductResponse<DocType extends Document> extends Response<DocType> {
+    documentClass: StandardDocumentSig<DocType>;
+    constructor(documentClass: StandardDocumentSig<DocType>, params: ResponseProps);
+    protected formatResponse(): void;
+}
+export declare class InvoiceResponse extends StandardProductResponse<Invoice> {
+    constructor(params: ResponseProps);
+}
+export declare class ReceiptResponse extends StandardProductResponse<Receipt> {
+    constructor(params: ResponseProps);
+}
+export declare class FinancialDocResponse extends StandardProductResponse<FinancialDocument> {
+    constructor(params: ResponseProps);
+}
+export declare class PassportResponse extends StandardProductResponse<Passport> {
+    constructor(params: ResponseProps);
+}
+export {};