@patrickdk77/pdf-defuse 0.2.1 → 0.2.3

package/README.md

@@ -69,7 +69,7 @@ Removed by default:
 - Actions that reach outside the document: launching programs, links to other or embedded files, form submit and import, show and hide toggles, layer toggles, menu commands and media.
 - Links other than http, https and mailto, and links built to mislead. That covers credentials before the host, IP hosts, punycode or mixed-script hosts, encoded hosts, tooltips or alternate text that name a different site, links and push buttons that cover most of a page, and relative links with no base address.
 - Attached files, portfolios, XFA forms, slideshows, media and 3D annotations, and annotation types outside an allowlist.
-- Encryption with an empty password, or with the password you pass.
+- Encryption with an empty password, or with a password you pass.
 - Earlier revisions, unreferenced objects and damaged structure. The output is one clean revision, unless it is a signed file that keeps its bytes, as described below.
 
 Kept: text, images, fonts, layout, bookmarks, form fields and their values, safe links and metadata.
@@ -121,7 +121,7 @@ A run that stops before it has seen the whole file rejects the file with a null
 
 | Option | Default | Purpose |
 |---|---|---|
-| `password` | `''` | Tried as the user password, then as the owner password, then as the empty password. See Passwords below |
+| `password` | `''` | One password or a list. pdf-defuse tries each in order, as the user password and then as the owner password, and the empty password last. See Passwords below |
 | `limits` | none | `fileSize`, `objects`, `decompressedBytes`, `nestingDepth` and `timeMs`. A limit left out is not enforced |
 | `stripMetadata` | `false` | Remove the document information dictionary wherever it is referenced, and the XMP metadata of every object: the catalog, pages, images and the rest |
 | `actionOverrides` | none | Change the action for a category, or for a category and detail |
@@ -139,11 +139,13 @@ await disarmPdf(bytes, { actionOverrides: [{ category: 'JAVASCRIPT', action: 're
 
 ### Passwords
 
+`password` takes one password or a list, because one upload can hold files with different passwords, such as a ZIP with one password and a PDF inside it with another. pdf-defuse tries each entry in order, as the user password and then as the owner password, and stops at the first that opens the file. The empty password comes after the list, so a file that opens without a password opens whatever the list holds. ENCRYPTED/USER_PASSWORD or ENCRYPTED/OWNER_PASSWORD says which kind of password opened the file. `pdfPlugin()` gives every attached PDF the list of the run that found it, and `pdfPlugin(options)` gives it the list in `options`.
+
 A revision 6 password, the AES-256 scheme of ISO 32000-2, is tried three ways: prepared with SASLprep, as that standard requires, prepared the way pdf.js prepares it, and as typed. The last opens files from tools that skip SASLprep. Other revisions use the password as typed.
 
 An empty owner password opens a file, as in qpdf, and ENCRYPTED/OWNER_PASSWORD reports it. pdf.js asks for a password there.
 
-A file that encrypts only its attached files, through a crypt filter with `/AuthEvent /EFOpen`, opens without the password, as in pdf.js, and ENCRYPTED/ATTACHMENTS_ONLY reports it. pdf-defuse removes the attached files the password does not decrypt and any other stream under their key, which ENCRYPTED/NO_KEY reports. pdf.js draws such a page blank.
+A file that encrypts only its attached files, through a crypt filter with `/AuthEvent /EFOpen`, opens without a password, as in pdf.js, and ENCRYPTED/ATTACHMENTS_ONLY reports it. pdf-defuse removes the attached files that no password you pass decrypts, and any other stream under their key, which ENCRYPTED/NO_KEY reports. pdf.js draws such a page blank.
 
 ## Plugins
 
@@ -252,6 +254,8 @@ const result = await router.process(upload, sink, { depth: 0, deadline: Date.now
 
 `pdfPlugin(options)` runs every PDF it gets with those options. Built without options, it takes the options of the pdf-defuse run that found the file, which pdf-defuse passes under a symbol only it can read. A `pdfPlugin()` that another package calls with a context of its own runs with no options at all, so a master builds it with options.
 
+`detectPdf(bytes, options)` and `detectPdfSource(source, options)` tell a router whether pdf-defuse reads a file. They return `{ mimeType, extension, offset }`, or undefined. A header in the first 1024 bytes makes the file a PDF at the header's offset. A file without one is a PDF only when `options.name` or `options.declaredType` says PDF and pdf-defuse opens it, the way pdf.js opens a file without a header. That open reads the whole file, and the answer then carries `heuristic`. A file that opens but is too broken to inspect still counts, and the inspection rejects it. `pdfPlugin` takes what detection finds. `detectPdfSource` leaves the source open, and throws with the `DEFUSE_LIMIT` code once `options.deadline` passes.
+
 The context holds `depth` and `deadline` and nothing PDF-specific, so any package can make one. `depth` is the depth of the file itself, the same number as `file.depth`. The upload is 0 and a file inside it is 1, whatever the formats. A container gives each file it finds its own depth plus one. `pdfPlugin` runs a PDF at the depth in its context and checks `limits.nestingDepth` against it, so a PDF in a ZIP in a PDF counts as depth 2. `deadline` is the `Date.now()` value the whole upload must finish by, and a container passes it on unchanged.
 
 A plugin returns its findings as `DefuseFinding` objects, whose `category` and `detail` are plain strings. The string enums of two packages never match each other. `PdfFinding`, with pdf-defuse's enums, fits `DefuseFinding`. pdf-defuse adds a contained file's findings to its own report with `attachment` set, and keeps the action each came with, except that `reject` becomes `strip`. `actionOverrides` apply to the findings a pdf-defuse run makes, never to those a plugin returns.
@@ -268,7 +272,7 @@ The `fileSize`, `objects` and `decompressedBytes` limits apply to one file and h
 pdf-defuse inspect <file> [options]
 pdf-defuse defuse <in> <out> [options]
 
---password-file <path>   read the password from a file, or set PDF_DEFUSE_PASSWORD
+--password-file <path>   read passwords from a file, one per line; PDF_DEFUSE_PASSWORD holds one
 --plugin <module>        load plugins from a file or an installed package, repeatable
 --config <module>        load a whole options object, repeatable; the other options apply on top
 --keep-type <type>       keep attached files of this MIME type or file extension unchanged, repeatable
@@ -286,6 +290,8 @@ Use `-` for stdin or stdout. The tool copies stdin to a temporary file in `tempD
 
 The text report shows each control character as an escape such as `\u001b`. The JSON report escapes DEL and the C1 range too, which JSON itself does not require. A field name or a title from the PDF can hold an escape sequence that rewrites what the terminal shows, the verdict included.
 
+`--password-file` reads one password per line. The tool drops a carriage return at the end of a line and skips blank lines, since the empty password is always tried. Spaces belong to the password. `PDF_DEFUSE_PASSWORD` holds one password, and a line break in it is part of that password. `--password-file` replaces it, and either one replaces a `password` that a `--config` module sets.
+
 `objects`, `nestingDepth` and `timeMs` take whole numbers. `fileSize` and `decompressedBytes` take sizes. Any other value is a usage error.
 
 A `--plugin` module exports `plugins`, an array, or `plugin`, a single plugin. Either can also sit on the default export, and a default export that is itself a plugin or an array of them works too. ES modules, CommonJS files and installed packages all load. Package names resolve from the working directory, and a package that exports only an `import` condition loads as well, through a subpath pattern such as `"./*"` too. `--plugin keep.js` loads `./keep.js` when the working directory has that file, and the installed package `keep.js` when it does not.

package/dist/cli.js

@@ -50,7 +50,7 @@ const USAGE = `Usage:
   pdf-defuse defuse <in> <out> [options]
 
 Options:
-  --password-file <path>   read the password from a file, or set PDF_DEFUSE_PASSWORD
+  --password-file <path>   read passwords from a file, one per line; PDF_DEFUSE_PASSWORD holds one
   --plugin <module>        load plugins from a file or an installed package, repeatable
   --config <module>        load a whole options object, repeatable; the other options apply on top
   --keep-type <type>       keep contained files of this MIME type or file extension unchanged, repeatable
@@ -181,7 +181,11 @@ async function parse(argv) {
         };
         switch (a) {
             case '--password-file':
-                password = (await fsp.readFile(next(), 'utf8')).replace(/\r?\n$/, '');
+                // The empty password is always tried, so a blank line adds nothing.
+                password = (await fsp.readFile(next(), 'utf8'))
+                    .split('\n')
+                    .map(line => line.replace(/\r$/, ''))
+                    .filter(line => line !== '');
                 break;
             case '--plugin':
                 ordered.push(...pluginsFrom(await loadModule(next())));

package/dist/detect.d.ts

@@ -0,0 +1,9 @@
+import type { ByteSource, DetectOptions, FormatDetection } from './types';
+/**
+ * Whether pdf-defuse reads `source` as a PDF. A header in the first 1024 bytes makes it one, at the header's offset.
+ * A file without one is a PDF only when its name or declared type says PDF and it opens as one, the way pdf.js opens
+ * a file without a header. That open reads the whole file. The source stays open.
+ */
+export declare function detectPdfSource(source: ByteSource, options?: DetectOptions): Promise<FormatDetection | undefined>;
+/** `detectPdfSource` over bytes in memory. */
+export declare function detectPdf(bytes: Uint8Array, options?: DetectOptions): Promise<FormatDetection | undefined>;

package/dist/detect.js

@@ -0,0 +1,50 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.detectPdfSource = detectPdfSource;
+exports.detectPdf = detectPdf;
+const document_1 = require("./document");
+const io_1 = require("./io");
+const sniff_1 = require("./sniff");
+const walker_1 = require("./walker");
+/** PDF readers look for the header in the first 1024 bytes, and so does PdfDocument when it opens a file. */
+const HEADER_WINDOW = 1024;
+const HEADER = Buffer.from('%PDF-', 'latin1');
+const isIoError = (e) => typeof e?.code === 'string' && /^E[A-Z]+$/.test(e.code);
+/**
+ * Whether pdf-defuse reads `source` as a PDF. A header in the first 1024 bytes makes it one, at the header's offset.
+ * A file without one is a PDF only when its name or declared type says PDF and it opens as one, the way pdf.js opens
+ * a file without a header. That open reads the whole file. The source stays open.
+ */
+async function detectPdfSource(source, options = {}) {
+    const size = await source.size();
+    const head = await source.read(0, Math.min(size, HEADER_WINDOW));
+    const at = Buffer.from(head.buffer, head.byteOffset, head.byteLength).indexOf(HEADER);
+    if (at >= 0)
+        return { mimeType: 'application/pdf', extension: 'pdf', offset: at };
+    const claimed = [options.declaredType, (0, sniff_1.typeFromName)(options.name)].some(t => t !== undefined && (0, sniff_1.family)(t) === 'application/pdf');
+    if (!claimed)
+        return undefined;
+    const temp = new io_1.TempDir(options.tempDir);
+    try {
+        const doc = await document_1.PdfDocument.open(temp.watch(source), { deadline: options.deadline, temp, memoryThreshold: options.memoryThreshold });
+        await doc.release();
+        if (temp.ioError !== undefined)
+            throw temp.ioError;
+        return { mimeType: 'application/pdf', extension: 'pdf', offset: 0, heuristic: 'no PDF header, but the name or declared type says PDF and the file opens as one' };
+    }
+    catch (e) {
+        if (temp.ioError !== undefined)
+            throw temp.ioError;
+        if ((0, walker_1.stopsTree)(e) !== undefined || isIoError(e))
+            throw e;
+        // Any other failure to open, as in an inspection, means pdf-defuse does not read the file as a PDF.
+        return undefined;
+    }
+    finally {
+        await temp.cleanup();
+    }
+}
+/** `detectPdfSource` over bytes in memory. */
+function detectPdf(bytes, options = {}) {
+    return detectPdfSource((0, io_1.bufferSource)(bytes), options);
+}

package/dist/document.d.ts

@@ -61,7 +61,7 @@ interface DocumentIssues {
     memoryFallback: string[];
 }
 interface OpenOptions {
-    password?: string;
+    password?: string | readonly string[];
     decompressedBytes?: number;
     objectLimit?: number;
     deadline?: number;

package/dist/document.js

@@ -681,11 +681,16 @@ class PdfDocument {
             this.encryptRefs.add(num);
         const id = this.trailer.get('ID');
         const id0 = Array.isArray(id) && id[0] instanceof objects_1.PdfString ? id[0].bytes : new Uint8Array(0);
-        const password = this.opts.password ?? '';
-        let r = crypto_1.SecurityHandler.open(dict, id0, password);
-        // A password meant for other files must not lock out one that opens without a password.
-        if (r.status === 'password-required' && password !== '')
-            r = crypto_1.SecurityHandler.open(dict, id0, '');
+        const given = this.opts.password ?? '';
+        // A password meant for other files must not lock out one that opens without a password, so the empty one comes last.
+        const passwords = new Set([...(typeof given === 'string' ? [given] : given), '']);
+        let r = { status: 'password-required' };
+        for (const password of passwords) {
+            this.checkTime();
+            r = crypto_1.SecurityHandler.open(dict, id0, password);
+            if (r.status !== 'password-required')
+                break;
+        }
         this.securityResult = r;
         if (r.status === 'ok')
             this.security = r.handler;

package/dist/index.d.ts

@@ -1,5 +1,6 @@
 export type { DelimitedOptions, FormulaHandling } from './data-plugins';
 export { csvPlugin, jsonPlugin, tsvPlugin } from './data-plugins';
+export { detectPdf, detectPdfSource } from './detect';
 export { disarmPdf, disarmPdfSource, inspectPdf, inspectPdfSource } from './engine';
 export { allFindingSpecs } from './findings';
 export { bufferSink, bufferSource, fileSink, fileSource, writableSink } from './io';

package/dist/index.js

@@ -14,11 +14,14 @@ var __exportStar = (this && this.__exportStar) || function(m, exports) {
     for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
 };
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.scoreFindings = exports.DEFAULT_BANDS = exports.COMBINATIONS = exports.pdfPlugin = exports.passThrough = exports.writableSink = exports.fileSource = exports.fileSink = exports.bufferSource = exports.bufferSink = exports.allFindingSpecs = exports.inspectPdfSource = exports.inspectPdf = exports.disarmPdfSource = exports.disarmPdf = exports.tsvPlugin = exports.jsonPlugin = exports.csvPlugin = void 0;
+exports.scoreFindings = exports.DEFAULT_BANDS = exports.COMBINATIONS = exports.pdfPlugin = exports.passThrough = exports.writableSink = exports.fileSource = exports.fileSink = exports.bufferSource = exports.bufferSink = exports.allFindingSpecs = exports.inspectPdfSource = exports.inspectPdf = exports.disarmPdfSource = exports.disarmPdf = exports.detectPdfSource = exports.detectPdf = exports.tsvPlugin = exports.jsonPlugin = exports.csvPlugin = void 0;
 var data_plugins_1 = require("./data-plugins");
 Object.defineProperty(exports, "csvPlugin", { enumerable: true, get: function () { return data_plugins_1.csvPlugin; } });
 Object.defineProperty(exports, "jsonPlugin", { enumerable: true, get: function () { return data_plugins_1.jsonPlugin; } });
 Object.defineProperty(exports, "tsvPlugin", { enumerable: true, get: function () { return data_plugins_1.tsvPlugin; } });
+var detect_1 = require("./detect");
+Object.defineProperty(exports, "detectPdf", { enumerable: true, get: function () { return detect_1.detectPdf; } });
+Object.defineProperty(exports, "detectPdfSource", { enumerable: true, get: function () { return detect_1.detectPdfSource; } });
 var engine_1 = require("./engine");
 Object.defineProperty(exports, "disarmPdf", { enumerable: true, get: function () { return engine_1.disarmPdf; } });
 Object.defineProperty(exports, "disarmPdfSource", { enumerable: true, get: function () { return engine_1.disarmPdfSource; } });

package/dist/index.mjs

@@ -1,3 +1,3 @@
 import cjs from './index.js';
-export const { csvPlugin, jsonPlugin, tsvPlugin, disarmPdf, disarmPdfSource, inspectPdf, inspectPdfSource, allFindingSpecs, bufferSink, bufferSource, fileSink, fileSource, writableSink, passThrough, pdfPlugin, COMBINATIONS, DEFAULT_BANDS, scoreFindings, PdfCategory, PdfDetail, PdfRisk } = cjs;
+export const { csvPlugin, jsonPlugin, tsvPlugin, detectPdf, detectPdfSource, disarmPdf, disarmPdfSource, inspectPdf, inspectPdfSource, allFindingSpecs, bufferSink, bufferSource, fileSink, fileSource, writableSink, passThrough, pdfPlugin, COMBINATIONS, DEFAULT_BANDS, scoreFindings, PdfCategory, PdfDetail, PdfRisk } = cjs;
 export default cjs;

package/dist/plugins.js

@@ -2,6 +2,7 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.passThrough = passThrough;
 exports.pdfPlugin = pdfPlugin;
+const detect_1 = require("./detect");
 const engine_1 = require("./engine");
 const score_1 = require("./score");
 const sniff_1 = require("./sniff");
@@ -36,8 +37,9 @@ function pdfPlugin(options) {
     return {
         kind: 'file',
         name: 'pdf',
-        // It can only read PDF bytes, so the content decides, whatever the name says.
-        accepts: f => f.sniffedType === 'application/pdf',
+        // It takes what detectPdf finds to be a PDF: a header in the first 1024 bytes, or no header, a name or declared
+        // type that says PDF, and content that opens as one.
+        accepts: async (f) => f.sniffedType === 'application/pdf' || (await (0, detect_1.detectPdfSource)(f.source, { name: f.name, declaredType: f.declaredType })) !== undefined,
         async process(file, sink, context) {
             const run = options ?? context[walker_1.RUN_OPTIONS] ?? {};
             const r = await (0, engine_1.disarmAtDepth)(file.source, sink, run, context.depth, context.deadline);

package/dist/sniff.js

@@ -95,7 +95,14 @@ function family(mime) {
  * of "page.html." is html.
  */
 function extensionOf(name) {
-    return /\.([^./\\]+)$/.exec((name ?? '').replace(/[. ]+$/, ''))?.[1].toLowerCase();
+    // A loop and lastIndexOf, since /[. ]+$/ backtracks: a name of 40,000 dots took seconds to match.
+    const n = name ?? '';
+    let end = n.length;
+    while (end > 0 && (n[end - 1] === '.' || n[end - 1] === ' '))
+        end--;
+    const dot = n.lastIndexOf('.', end - 1);
+    const ext = dot < 0 ? '' : n.slice(dot + 1, end);
+    return ext && !/[/\\]/.test(ext) ? ext.toLowerCase() : undefined;
 }
 function typeFromName(name) {
     const ext = extensionOf(name);

package/dist/types.d.ts

@@ -269,8 +269,11 @@ export interface PdfScoreBands {
     critical: number;
 }
 export interface PdfOptions {
-    /** Default '', tried as the user password and then as the owner password. */
-    password?: string;
+    /**
+     * Default ''. Each password is tried in order, as the user password and then as the owner password. The empty
+     * password comes after them, so a file that opens without a password opens whatever the list holds.
+     */
+    password?: string | string[];
     /** A limit left out is not enforced. */
     limits?: PdfLimits;
     /** Default false. */
@@ -295,3 +298,30 @@ export interface PdfOptions {
      */
     preserveSignatures?: boolean;
 }
+/**
+ * A file's format as a package of the family finds it, among the formats that package reads. The family's router asks
+ * each package and routes the file by the answers.
+ */
+export interface FormatDetection {
+    /** The format's MIME type, such as application/pdf. */
+    mimeType: string;
+    /** The format's usual extension, lowercase and without the dot. */
+    extension: string;
+    /** Where the format starts in the file: 0, or the offset of its data when other data comes first. */
+    offset: number;
+    /** Set when the match rests on more than the content, and says what it rests on. */
+    heuristic?: string;
+}
+/** What a detect function may know of a file besides its content. */
+export interface DetectOptions {
+    /** The file's name, as the upload or the container that holds it gives it. */
+    name?: string;
+    /** The MIME type the upload or the container declares. */
+    declaredType?: string;
+    /** The Date.now() value detection must finish by, as `PluginContext.deadline`. */
+    deadline?: number;
+    /** Folder for temporary files. Defaults to `os.tmpdir()`. */
+    tempDir?: string;
+    /** Bytes held in memory before a temporary file takes over. */
+    memoryThreshold?: number;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@patrickdk77/pdf-defuse",
-  "version": "0.2.1",
+  "version": "0.2.3",
   "description": "Sanitize and clean PDF files. Detects and removes JavaScript, attached files, launch actions, unsafe links and empty-password encryption, scores the risk, and writes a disarmed copy with bounded memory and no runtime dependencies.",
   "license": "MIT",
   "author": "patrickdk77",