@patrickdk77/pdf-defuse 0.1.2 → 0.2.0

package/README.md

@@ -8,7 +8,7 @@ It has no runtime dependencies, keeps memory bounded, and runs the same code for
 npm install @patrickdk77/pdf-defuse
 ```
 
-Node 22 or later.
+Node 18 or later.
 
 ## Quick start
 
@@ -31,7 +31,7 @@ ES modules work too: `import { disarmPdf } from '@patrickdk77/pdf-defuse'`.
 
 The engine reads through a source with `size()` and `read(offset, length)`, and writes to a sink that only appends, with `write(chunk)` and `close()`. A local file, a buffer and an S3 object all fit, so large files never need to sit in memory. `disarmPdfSource` reads the source once, start to end, into its own copy and works from that. For S3 that is one pass of ranged reads.
 
-`inspectPdfSource` and `disarmPdfSource` call the source's `close()`, if it has one, when the run ends. `fileSource` opens its file again if something reads it after that. A `read` must return exactly the bytes asked for, or fewer only at the end of the source. A read that returns any other length fails the run with an error whose `code` is `EIO`, because it would shift every byte after it.
+`inspectPdfSource` and `disarmPdfSource` call the source's `close()`, if it has one, when the run ends. `fileSource` opens its file again if something reads it after that. A `read` must return exactly the bytes asked for, or fewer only at the end of the source. A read that returns any other length fails the run with an error whose `code` is `EIO`, because it would shift every byte after it. pdf-defuse never asks a source for a read that starts outside it, even when a damaged file points there. `bufferSource`, `fileSource` and the source a file plugin receives return no bytes for a read that starts outside the data or at an offset that is not a whole number.
 
 ```js
 const { disarmPdfSource, fileSource, fileSink, writableSink } = require('@patrickdk77/pdf-defuse');
@@ -76,6 +76,12 @@ Kept: text, images, fonts, layout, bookmarks, form fields and their values, safe
 
 Every file that is not rejected is rewritten, so the output holds only what pdf-defuse read. That includes a file with nothing to remove, whose status stays `clean`.
 
+pdf-defuse reads damaged files the way pdf.js reads them, and a CORRUPTED finding reports each repair. That covers a file with no `%PDF-` header in its first 1024 bytes, which CORRUPTED/MISSING_HEADER reports, a catalog written in the trailer, a stream written inside another object or stored in an object stream, broken object streams, and a cross-reference table rebuilt by scanning. The rebuild ignores anything inside a comment and picks the trailer pdf.js picks. The output holds each repaired object as an object of its own.
+
+Where pdf.js opens nothing, pdf-defuse searches the whole file for objects and trailers, as other readers do, and rejects the file only when that finds none. It reads a stream pdf.js gives up on, such as one with no `endstream`, the way qpdf reads it. pdf.js draws nothing for such a stream. When one object in an object stream fails to parse, pdf.js can lose the whole stream, depending on the order it looks objects up in. pdf-defuse reads each object that parses.
+
+pdf-defuse decodes FlateDecode, LZWDecode, ASCIIHexDecode, ASCII85Decode, RunLengthDecode and BrotliDecode, so the objects, scripts and attached files inside them get the same checks as the rest of the file. Like pdf.js, it refuses Brotli data that is truncated, corrupt or followed by more bytes. It also refuses a BrotliDecode stream with a predictor, which pdf.js ignores and PDFium applies, and a filter chain with two BrotliDecode filters.
+
 A rewrite invalidates a signature, so a signed file can keep its bytes instead. That happens when the file is clean and a signature field reachable from the catalog holds a signature whose `/ByteRange` covers every byte of the file except that signature's `/Contents` value. Bytes added after signing fall outside that range, so a file updated after signing is rewritten. The file must also have no CORRUPTED finding, whatever its action, and no second definition of any object, not even one inside another object, which only a reader that rebuilds the cross-reference table by scanning would take. pdf-defuse does not verify the signature. Anyone can sign a file, with a self-signed certificate too, so this exception trusts whoever signed it. `preserveSignatures: false` rewrites signed files as well.
 
 Before anything reaches your sink, pdf-defuse inspects its own output. It rejects the file if anything removable is left, if the page count changed, or if any page's content changed length.
@@ -91,7 +97,7 @@ Every finding has two enums and a string. `category` is the kind, such as `ENCRY
 
 `action` says what happens: `reject` refuses the file, `strip` removes the content and `info` only reports it. An override to `strip` removes the content for METADATA/XMP, METADATA/INFO_DICTIONARY, LINK/SAFE and EMBEDDED_FILE/TYPE_MISMATCH. Some `info` findings name content that every rewrite drops anyway, such as SIGNATURE/USAGE_RIGHTS, STRUCTURE/ESCAPED_NAMES and the ENCRYPTED findings that name the algorithm. A `strip` override on one of those changes only the status. The other `info` findings, such as FORM/FIELDS, SIGNATURE/SIGNED, STRUCTURE/VERSION_UPGRADED and the PLUGIN_PASSED and PLUGIN_SCRUBBED findings, name nothing that can be removed. A `strip` override on one of them leaves the finding in the output, so the output fails its own inspection and the upload is rejected.
 
-Some content goes whatever an override says. Its finding then shows `strip`, or `reject` if you override it to that. This covers a script reference an action loses under the re-checking rules described under Plugins, a reference to an object stream or cross-reference stream, and an attached file a plugin removed or failed on. It also covers a reference in `/Annots` to an object that already has another role, and a reference to a file specification that is the catalog, a page, or the names or information dictionary and carries an embedded file. Such an object keeps its role and its own entries, `/EF` included, so that embedded file stays in the output. Only the reference that treated the object as a file specification goes.
+Some content goes whatever an override says. Its finding then shows `strip`, or `reject` if you override it to that. This covers a script reference an action loses under the re-checking rules described under Plugins, a reference to an object stream or cross-reference stream, and an attached file a plugin removed or failed on, or that the password does not decrypt. It also covers any other stream encrypted with a key the password does not give, which ENCRYPTED/NO_KEY reports. It also covers a reference in `/Annots` to an object that already has another role, and a reference to a file specification that is the catalog, a page, or the names or information dictionary and carries an embedded file. Such an object keeps its role and its own entries, `/EF` included, so that embedded file stays in the output. Only the reference that treated the object as a file specification goes.
 
 pdf-defuse compares what it would write with what it read. If it would remove something that no other finding names, the report gets PROCESSING/CONTENT_REMOVED, which follows the same rule. The status then shows the removal, and a signed file does not keep its bytes.
 
@@ -115,14 +121,14 @@ A run that stops before it has seen the whole file rejects the file with a null
 
 | Option | Default | Purpose |
 |---|---|---|
-| `password` | `''` | Tried as the user password and then as the owner password. If it opens neither, the empty password is tried, so a file that needs no password still opens |
+| `password` | `''` | Tried as the user password, then as the owner password, then as the empty password. See Passwords below |
 | `limits` | none | `fileSize`, `objects`, `decompressedBytes`, `nestingDepth` and `timeMs`. A limit left out is not enforced |
 | `stripMetadata` | `false` | Remove the document information dictionary wherever it is referenced, and the XMP metadata of every object: the catalog, pages, images and the rest |
 | `actionOverrides` | none | Change the action for a category, or for a category and detail |
 | `scoreWeights`, `scoreBands` | built in | Change the scoring |
 | `filePlugins`, `scriptPlugins` | none | Keep chosen attached files or scripts |
 | `tempDir` | `os.tmpdir()` | Where temporary files go |
-| `memoryThreshold` | 8 MiB | `disarmPdfSource` keeps its copy of the upload in memory up to this size and in a temporary file above it. Each decoded attachment and object stream above this size goes through a temporary file. A decoded attachment's file is deleted once its plugins have decided it. Scrubbed attachment copies waiting for the write, and an attachment decoded again so a plugin can check its further names, share this much memory between them. One that does not fit goes to a temporary file. A script that decodes larger than this is removed without going to the script plugins |
+| `memoryThreshold` | 8 MiB | `disarmPdfSource` keeps its copy of the upload in memory up to this size and in a temporary file above it. Each decoded attachment above this size goes through a temporary file. Decoded object streams share this much memory, and the ones that do not fit go to temporary files, so each is decoded once. A decoded attachment's file is deleted once its plugins have decided it. Scrubbed attachment copies waiting for the write, and an attachment decoded again so a plugin can check its further names, share this much memory between them. One that does not fit goes to a temporary file. A script that decodes larger than this is removed without going to the script plugins |
 | `preserveSignatures` | `true` | Keep the bytes of a clean file that is signed over all of it, as described under What it removes and what it keeps. `false` rewrites it like any other file, which invalidates the signature |
 
 To reject every file that contains JavaScript instead of removing the scripts:
@@ -131,6 +137,14 @@ To reject every file that contains JavaScript instead of removing the scripts:
 await disarmPdf(bytes, { actionOverrides: [{ category: 'JAVASCRIPT', action: 'reject' }] });
 ```
 
+### Passwords
+
+A revision 6 password, the AES-256 scheme of ISO 32000-2, is tried three ways: prepared with SASLprep, as that standard requires, prepared the way pdf.js prepares it, and as typed. The last opens files from tools that skip SASLprep. Other revisions use the password as typed.
+
+An empty owner password opens a file, as in qpdf, and ENCRYPTED/OWNER_PASSWORD reports it. pdf.js asks for a password there.
+
+A file that encrypts only its attached files, through a crypt filter with `/AuthEvent /EFOpen`, opens without the password, as in pdf.js, and ENCRYPTED/ATTACHMENTS_ONLY reports it. pdf-defuse removes the attached files the password does not decrypt and any other stream under their key, which ENCRYPTED/NO_KEY reports. pdf.js draws such a page blank.
+
 ## Plugins
 
 Attached files and scripts are removed unless a plugin keeps them. A plugin says which files or scripts it accepts, then scrubs them, passes them through unchanged, or removes them after all. The first plugin that accepts handles it. If a plugin throws, or scrubs and writes nothing, the item is removed. An error with a `DEFUSE_LIMIT` or `DEFUSE_IO` code stops the whole run instead, as described under The defuse family. A file plugin that passes a file keeps its original bytes, or the bytes it wrote to the sink if it wrote any.
@@ -280,11 +294,15 @@ A `--config` module's default export is the options object. For CommonJS that is
 
 Exit codes are 0 for clean, 1 for stripped or strippable, 2 for rejected and 3 for a usage or I/O error. An output or a report that could not be written is an I/O error, and so is a report whose reader has gone away. There is no password flag, because other users can read command-line arguments from the process list. An unknown option shows in the error without the text after its `=`, in case that text is a password typed in the wrong place.
 
-## Memory
+## Memory and time
 
 `disarmPdfSource` first copies the upload, in memory up to `memoryThreshold` and into a temporary file above it. It analyzes and writes from that copy only, so a source that changes during the run cannot get anything past the inspection. It rejects an upload over `limits.fileSize` without reading it. `inspectPdfSource` reads the source directly.
 
-The engine reads the cross-reference table from the end of the file, then reads only the objects reachable from the document root, one at a time. Stream bodies are copied and decrypted in chunks and are never loaded whole. Apart from the copy of the upload, memory grows with the number of objects, at roughly 250 bytes each, and not with the size of the file. A parsed value takes far more memory than its text, so pdf-defuse reads an object as damaged when its values would take more than about 16 MB, not counting the text of names and strings. It keeps at most about 32 MB of parsed objects for reuse.
+The engine reads the cross-reference table from the end of the file, then reads only the objects reachable from the document root, one at a time. Stream bodies are copied and decrypted in chunks and are never loaded whole. A stream body inside an object stream stays in the decoded object stream. A Brotli decoder can take 16 MB for its window, which is why a chain with two BrotliDecode filters is refused. Apart from the copy of the upload, memory grows with the number of objects, at roughly 250 bytes each, and not with the size of the file. A parsed value takes far more memory than its text, so pdf-defuse reads an object as damaged when its values would take more than about 16 MB, not counting the text of names and strings. It keeps at most about 32 MB of parsed objects for reuse. The dictionaries of streams written inside other objects stay for the whole run, up to about 32 MB more, and an object whose streams would pass that reads as damaged.
+
+pdf-defuse parses an object in a window that starts at 4 KB and grows as the parser needs more. Runs of whitespace and comments, stream bodies inside the object and the line after a `stream` keyword are skipped in the source instead of read into the window, so padding costs time but not memory. After 64 skips in one object, the window takes in the rest of it, up to 256 MB.
+
+Each object stream is decoded once, the first time an object in it is read. Decoded object streams share `memoryThreshold` of memory. The least recently used move to a temporary file when they need more, and a stream larger than `memoryThreshold` gets a file of its own. Time and temporary space grow with the decoded size of the object streams read. `limits.decompressedBytes` caps each stream, and `limits.timeMs` the run.
 
 Free rows in a cross-reference section cost a few numbers per run of them, since a few kilobytes of xref stream can free millions of object numbers. Only rows that an older section could use are held one by one. When those outnumber both 65,536 and the live objects, pdf-defuse rebuilds the table by scanning and reports CORRUPTED/XREF_REBUILT. `limits.objects` counts live rows only, in a table and in a stream alike.
 
@@ -293,9 +311,13 @@ Measured with Node 24. The heap cap is `--max-old-space-size`, and the semi-spac
 | File | Size | Objects | Heap cap | Result |
 |---|---|---|---|---|
 | Generated, one large stream | 100 MB | 4 | 48 MB | Defused |
-| ISO 32000-1 specification | 22 MB | 127,000 | 48 MB | Defused, about 200 MB resident |
-| ISO 32000-1 specification | 22 MB | 127,000 | 48 MB, 2 MB semi-space | Defused, about 125 MB resident |
-| ISO 32000-1 specification | 22 MB | 127,000 | Node default | Defused, about 350 MB resident |
+| A 756-page manual | 22 MB | 127,000 | 48 MB | Defused, about 200 MB resident |
+| A 756-page manual | 22 MB | 127,000 | 48 MB, 2 MB semi-space | Defused, about 125 MB resident |
+| A 756-page manual | 22 MB | 127,000 | Node default | Defused, about 350 MB resident |
+| Generated, objects alternating across 9 object streams of 1 MB each decoded | 400 KB | 18,000 | Node default | Defused in 1 s |
+| Generated, the same across 64 object streams | 760 KB | 32,000 | 48 MB | Defused in 3 s, about 190 MB resident |
+| Generated, 16 object streams of 32 MB each decoded | 560 KB | 1,600 | 48 MB | Defused in 5 s, about 100 MB resident, with 512 MB of temporary files |
+| Generated, an object stream whose last object never closes, then 128 MB of spaces decoded | 130 KB | 8 | 48 MB | Defused in 1 s, about 95 MB resident |
 
 ## What it does not protect against
 
@@ -305,7 +327,7 @@ Measured with Node 24. The heap cap is `--max-old-space-size`, and the semi-spac
 
 ## Development
 
-Development needs Node 22.12 or later. mocha 12 requires it, and the CommonJS test build loads chai 6, an ES module, with `require()`, which Node 22 supports only from 22.12. The pre-commit hook needs Node 22.22.1 or later for lint-staged. The published package has no runtime dependencies and runs on any Node 22.
+Development needs Node 22.12 or later. mocha 12 requires it, and the CommonJS test build loads chai 6, an ES module, with `require()`, which Node 22 supports only from 22.12. The pre-commit hook needs Node 22.22.1 or later for lint-staged. The published package has no runtime dependencies and runs on Node 18 or later.
 
 ```sh
 npm install --ignore-scripts
@@ -317,6 +339,8 @@ npm run format
 
 `npm run build` compiles `src` into `dist`. `npm test` runs the lint, builds the package and the tests, then runs mocha, so a lint error fails the tests. `npm run lint` runs `biome check` over the whole tree and changes nothing. `npm run format` rewrites files in Biome's format.
 
+`test/fixtures/cases` holds one small PDF for each problem a review found. `node scripts/make-fixtures.js` writes every fixture from scratch, and a second run writes the same bytes. It needs qpdf and the test build from `npm run build:test`. `manifest.json` in that folder gives each fixture's password and options, the verdicts inspect and disarm must return, and what release 0.1.2 returned. `test/cases.test.ts` checks those verdicts for every entry. It also checks with pdf.js that no script, attached file or outside action is left in the output, and that the output's pages hold the text pdf.js finds in the input. Set `PDF_DEFUSE_BEFORE` to the `index.js` of a 0.1.2 build and it checks what 0.1.2 returned too. For a new problem, add a fixture to the script and an entry to the manifest.
+
 `npx husky` activates the pre-commit hook. A plain `npm install` does the same through the `prepare` script. The hook runs Biome on the staged files, applying its safe fixes, then runs `npm audit`. The commit stops if either one fails.
 
 ## License

package/dist/crypto.d.ts

@@ -9,10 +9,12 @@ export declare class Rc4 {
 }
 export declare const rc4: (key: Uint8Array, data: Uint8Array) => Buffer<ArrayBufferLike>;
 type CryptMethod = 'RC4' | 'AES128' | 'AES256' | 'Identity';
-export type OpenResult = {
+export type OpenResult = 
+/** `none` opened a file that encrypts only its attached files, whose key the password did not give. */
+{
     status: 'ok';
     handler: SecurityHandler;
-    password: 'empty' | 'user' | 'owner';
+    password: 'empty' | 'user' | 'owner' | 'none';
 } | {
     status: 'password-required' | 'certificate-handler' | 'unknown-handler' | 'unknown-filter';
     detail?: string;
@@ -35,6 +37,8 @@ export declare class SecurityHandler {
     private readonly unknownFilters;
     private constructor();
     static open(enc: PdfDict, id0: Uint8Array, password: string): OpenResult;
+    /** False for a stream whose crypt filter needs the key a file that encrypts only its attached files keeps back. */
+    canDecrypt(stream: PdfStream): boolean;
     private readonly keyCache;
     private objectKey;
     decryptWith(method: CryptMethod, data: Uint8Array, num: number, gen: number): Uint8Array;

package/dist/crypto.js

@@ -36,6 +36,7 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.SecurityHandler = exports.rc4 = exports.Rc4 = void 0;
 const crypto = __importStar(require("node:crypto"));
 const objects_1 = require("./objects");
+const saslprep_1 = require("./saslprep");
 const PAD = Buffer.from('28BF4E5E4E758A4164004E56FFFA01082E2E00B6D0683E802F0CA9FE6453697A', 'hex');
 const md5 = (...parts) => {
     const h = crypto.createHash('md5');
@@ -205,25 +206,35 @@ class SecurityHandler {
         let which = password === '' ? 'empty' : 'user';
         if (R === 5 || R === 6) {
             keyBits = 256;
-            const pw = Buffer.from(password, 'utf8').subarray(0, 127);
-            const uKey = () => {
+            const uKey = (pw) => {
                 if (!hash2B(pw, U.subarray(32, 40), Buffer.alloc(0), R).equals(Buffer.from(U.subarray(0, 32))))
                     return undefined;
                 return aes256NoIv(hash2B(pw, U.subarray(40, 48), Buffer.alloc(0), R), bytesOf(enc.get('UE')));
             };
-            const oKey = () => {
+            const oKey = (pw) => {
                 const u48 = U.subarray(0, 48);
                 if (!hash2B(pw, O.subarray(32, 40), u48, R).equals(Buffer.from(O.subarray(0, 32))))
                     return undefined;
                 return aes256NoIv(hash2B(pw, O.subarray(40, 48), u48, R), bytesOf(enc.get('OE')));
             };
-            fileKey = uKey();
-            if (!fileKey && password !== '') {
-                fileKey = oKey();
-                which = 'owner';
+            // ISO 32000-2 prepares a revision 6 password with SASLprep. pdf.js prepares it its own way and then tries it as
+            // typed, which opens files from writers that skip SASLprep, so every form a reader tries is tried here too.
+            // Revision 5, Adobe's extension, takes it as typed, as pdf.js does.
+            const forms = R === 6 ? [(0, saslprep_1.saslprep)(password), (0, saslprep_1.pdfjsPrep)(password), password] : [password];
+            for (const form of new Set(forms)) {
+                if (form === undefined)
+                    continue;
+                const pw = Buffer.from(form, 'utf8').subarray(0, 127);
+                fileKey = uKey(pw);
+                if (fileKey)
+                    break;
+                // The empty password is tried as the owner password too, as qpdf tries it. pdf.js does not.
+                fileKey = oKey(pw);
+                if (fileKey) {
+                    which = 'owner';
+                    break;
+                }
             }
-            if (!fileKey)
-                return { status: 'password-required' };
         }
         else if (R >= 2 && R <= 4) {
             const n = V === 1 ? 5 : V >= 4 ? 16 : (enc.number('Length') ?? 40) / 8;
@@ -252,7 +263,7 @@ class SecurityHandler {
             const asUser = keyFor(padded);
             if (checkUser(asUser))
                 fileKey = asUser;
-            else if (password !== '') {
+            else {
                 // Owner password: derive the RC4 key from it, decrypt O to recover the padded user password.
                 let h = md5(padded);
                 if (R >= 3)
@@ -273,16 +284,38 @@ class SecurityHandler {
                     which = 'owner';
                 }
             }
-            if (!fileKey)
-                return { status: 'password-required' };
         }
         else {
             return { status: 'unknown-filter', detail: `R=${R}` };
         }
+        if (!fileKey) {
+            // Without a password, pdf.js opens a file whose /AuthEvent /EFOpen filter encrypts only the attached files, and
+            // asks for the password when one is opened. Nothing here can read those files.
+            const plain = (key) => enc.get(key) === undefined || enc.name(key) === 'Identity';
+            const efFilter = cf instanceof objects_1.PdfDict ? cf.get(enc.name('EFF') ?? '') : undefined;
+            const onOpen = efFilter instanceof objects_1.PdfDict && efFilter.name('AuthEvent') === 'EFOpen';
+            if (password !== '' || V < 4 || !plain('StmF') || !plain('StrF') || !onOpen)
+                return { status: 'password-required' };
+            which = 'none';
+        }
         return { status: 'ok', password: which, handler: new SecurityHandler(V, keyBits, fileKey, stm, str, eff, encryptMetadata, cryptFilters, unknownFilters) };
     }
+    /** False for a stream whose crypt filter needs the key a file that encrypts only its attached files keeps back. */
+    canDecrypt(stream) {
+        if (this.fileKey)
+            return true;
+        try {
+            return this.methodForStream(stream) === 'Identity';
+        }
+        catch {
+            // An unknown crypt filter fails where the stream is decoded, as it always has.
+            return true;
+        }
+    }
     keyCache = new Map();
     objectKey(num, gen, method) {
+        if (!this.fileKey)
+            throw new Error('No key for an encrypted stream');
         if (method === 'AES256')
             return Buffer.from(this.fileKey);
         const ck = `${num} ${gen} ${method}`;
@@ -341,11 +374,10 @@ class SecurityHandler {
         const full = 16 + Math.floor((len - 16) / 16) * 16;
         const tail = await readRaw(full - 32, 32);
         const key = this.objectKey(num, gen, method);
-        const d = crypto.createDecipheriv(key.length === 16 ? 'aes-128-ecb' : 'aes-256-ecb', key, null);
+        // In CBC the block before the last is the last block's IV, so the last block decrypts on its own.
+        const d = crypto.createDecipheriv(key.length === 16 ? 'aes-128-cbc' : 'aes-256-cbc', key, tail.subarray(0, 16));
         d.setAutoPadding(false);
         const last = Buffer.concat([d.update(tail.subarray(16, 32)), d.final()]);
-        for (let i = 0; i < 16; i++)
-            last[i] ^= tail[i];
         const pad = last[15];
         const valid = pad >= 1 && pad <= 16 && last.subarray(16 - pad).every(b => b === pad);
         return full - 16 - (valid ? pad : 0);

package/dist/document.d.ts

@@ -2,6 +2,7 @@ import { type OpenResult, SecurityHandler } from './crypto';
 import { Reader, TempDir } from './io';
 import { PdfDict, type PdfObject, PdfRef, PdfStream } from './objects';
 import type { ByteSource } from './types';
+/** A compressed object's entry names its object stream and its index there, as pdf.js reads it. */
 type XrefEntry = {
     type: 1;
     offset: number;
@@ -9,10 +10,11 @@ type XrefEntry = {
 } | {
     type: 2;
     stream: number;
+    index: number;
 };
 /** Entries are packed into one number each, so the index costs a few bytes per object. */
 export declare const packOffset: (offset: number, gen: number) => number;
-export declare const packStream: (stream: number) => number;
+export declare const packStream: (stream: number, index: number) => number;
 export declare function unpack(v: number | undefined): XrefEntry | undefined;
 export declare class OpenError extends Error {
     readonly kind: 'unparseable' | 'truncated' | 'not-pdf';
@@ -45,8 +47,16 @@ interface DocumentIssues {
     xrefStmOverFree: number;
     /** Object streams whose header pdf.js refuses, offsets that do not increase or numbers that are not whole. */
     badObjStm: Set<number>;
+    /** Object streams an entry names whose /Type is not /ObjStm, which pdf.js reads and qpdf warns about. */
+    objStmType: Set<number>;
+    /** Streams written inside other objects, by the object and where the body starts. Only indirect objects may be streams. */
+    directStreams: Set<string>;
+    /** A catalog written in the trailer, which pdf.js reads and qpdf does not. */
+    directRoot: boolean;
     /** Objects referenced with a generation their entry does not have, which pdf.js refuses to resolve. */
     refGen: Set<number>;
+    /** Streams stored in an object stream, which pdf.js reads from the decoded data. */
+    objStmStreams: Set<number>;
     /** Parts held in a temporary file instead of memory. */
     memoryFallback: string[];
 }
@@ -65,6 +75,8 @@ export declare class PdfDocument {
     private readonly opts;
     headerOffset: number;
     headerVersion: string;
+    /** No "%PDF-" in the first 1024 bytes, where pdf.js looks for one before it parses the file all the same. */
+    missingHeader: boolean;
     /** Added to every offset when the file has bytes before its header and the xref ignores them. */
     private base;
     /** Object number to packed entry. */
@@ -78,12 +90,46 @@ export declare class PdfDocument {
     securityResult?: OpenResult;
     encryptRef?: PdfRef;
     readonly issues: DocumentIssues;
-    /** Body offsets of the streams already counted in issues.streamLengthWrong, so a second read adds nothing. */
+    /**
+     * Objects with no entry of their own: a catalog written in the trailer, and streams written inside other objects.
+     * Their numbers are negative, so no object of the file can take one, and the writer gives each a number of its own.
+     * `key` names the object whose key decrypts a stream, and is null for one that is never encrypted.
+     */
+    private readonly synthetic;
+    /** Synthetic numbers by the object that holds the stream and where its body starts, so a second read reuses one. */
+    private readonly syntheticAt;
+    /** The walker numbers the name trees it writes -1 and -2. */
+    private nextSynthetic;
+    /** Streams whose bodies sit in a decoded object stream, by object or synthetic number, and where they sit there. */
+    private readonly heldBodies;
+    /** What the streams inside each object hold, by that object, and the total, which MAX_SYNTHETIC_COST bounds. */
+    private readonly syntheticCost;
+    private syntheticTotal;
+    /** Values of the indirect /Length entries of streams written inside objects. */
+    private directLengths;
+    /** The streams already counted in issues.streamLengthWrong, so a second read adds nothing. */
     private readonly lengthWrongAt;
     private objCache;
-    private objStmCache;
-    /** Object streams that spilled to a temporary file. Kept for the life of the document, so each is decoded once. */
-    private readonly spilledObjStm;
+    /**
+     * The object streams decoded since the map or the key last changed, or null for one that does not read. Each is
+     * decoded once: memory holds them up to memoryThreshold, and the rest go to temporary files.
+     */
+    private readonly objStms;
+    /** The object streams held in memory, least recently used first, and the bytes they take. */
+    private readonly inMemory;
+    private inMemoryBytes;
+    /** The temporary file that takes the object streams memory cannot hold, its length, and one handle that reads it. */
+    private pool?;
+    /** Temporary files of object streams too large for memory. */
+    private objStmFiles;
+    /**
+     * Readers over decoded object streams, so each keeps its blocks between lookups. Readers over a file of their own
+     * hold a file handle each, so fewer of them stay open.
+     */
+    private objStmReaders;
+    private objStmFileReaders;
+    /** By object stream, the objects whose entries point into it and the index each entry gives. */
+    private wanted?;
     /** Object streams found by scan(), indexed once the key exists. */
     private pendingObjStm;
     /** Streams whose indirect /Length or filter entries are being resolved, so a cycle ends after one lap. */
@@ -101,6 +147,12 @@ export declare class PdfDocument {
     /** Where the objects whose bad tokens are already in issues.malformed were read, so a second read adds nothing. */
     private readonly badTokensCounted;
     private scanned;
+    /** The first trailer any cross-reference section gave, which pdf.js falls back on when its recovery finds none. */
+    private topDict?;
+    /** Set once recovery has looked past what pdf.js finds. */
+    private extended;
+    /** What a read of every object of the rebuilt map found, kept until the map is rebuilt again. */
+    private survey?;
     /** Offsets of the xref sections and xref streams that read, which are not ordinary object definitions. */
     readonly structuralOffsets: Set<number>;
     /** The definitions that older xref sections use and newer ones replaced or freed, as object number then offset. */
@@ -111,6 +163,8 @@ export declare class PdfDocument {
     /** The extent of the live object insideLiveObject() read last, empty when it does not parse. */
     private lastLive?;
     private unboundedScanBudget;
+    /** The entries the page lookups found broken or sound, while they run. */
+    private checkedEntries?;
     /** Bytes parseWindowed() may still read past the next known object. */
     private parseBudget;
     private constructor();
@@ -118,17 +172,97 @@ export declare class PdfDocument {
     checkTime(): void;
     private checkObjectLimit;
     private init;
+    private load;
+    private catalog;
+    /**
+     * Whether pdf.js rebuilds the map when it checks the first and the last page, as its checkFirstPage and checkLastPage
+     * do: an entry it reads on the way points at another object. When the last page is not where /Count puts it, pdf.js
+     * reads every page.
+     */
+    private pageEntriesBroken;
+    /**
+     * Looks for page `index` as pdf.js's getPageDict does, which skips a node whose /Count says the page is not under it.
+     * False where pdf.js fails for another reason, such as a node it already visited.
+     */
+    private findPage;
+    /** Reads every page as pdf.js's getAllPageDicts does, which stops at the first node it cannot read. */
+    private readAllPages;
+    /** A node typed /Page, or any node without /Kids, is a page to pdf.js. */
+    private isPageNode;
+    /** Reads a value one level deep, as pdf.js reads one on its way to a page. Throws BrokenEntry where pdf.js would. */
+    private fetch;
+    /**
+     * Whether an entry points at another object or at nothing that reads as an object header. pdf.js refuses such an
+     * entry, and an entry whose generation differs from the reference, before it reads the object. A missing entry
+     * reads as null instead. An object stream's entry is left out: pdf.js finds the object by its index there.
+     */
+    private entryBroken;
     private initSecurity;
     private loadXrefChain;
-    /** Reads one section; for hybrid files its XRefStm entries add the objects the table does not give. */
+    /**
+     * Reads one section; for hybrid files its XRefStm entries add the objects the table does not give. Without `rebase`,
+     * an offset that misses is not tried again past bytes before the header.
+     */
     private readXrefSection;
     private readXrefTable;
     /** `at` is where the stream's object header starts, which can be past whitespace at `offset`. */
     private readXrefStream;
-    /** Rebuilds the object map by scanning every byte. Used when the xref is missing or wrong. */
+    /**
+     * Rebuilds the object map by scanning the file, as pdf.js's indexObjects does when the cross-reference data is
+     * missing or wrong, and picks the trailer pdf.js picks.
+     */
     private scan;
+    /**
+     * pdf.js's indexObjects. It reads the file token by token, a token running to the next LF, CR or "<", and skips
+     * comments. A token that starts "N G obj" defines an object, and the scan jumps from it to the next "endobj", "N G
+     * obj", "xref" or "trailer<<" and a letter. From "xref" it jumps to "trailer" and then to "startxref", and from
+     * "trailer" to the next "startxref" or "N G obj". So a "trailer" inside an object, a comment or right after
+     * another one is no candidate. Returns the candidates, and the objects that look like xref streams.
+     */
+    private indexObjects;
+    /** pdf.js's /^(\d+)\s+(\d+)\s+obj\b/ against the token [pos, end): the numbers, or undefined. */
+    private objHead;
+    /**
+     * The first match from `from` on of pdf.js's /\b(endobj|\d+\s+\d+\s+obj|xref|trailer\s*<<)\b/, or with `endobj`
+     * false of /\b(startxref|\d+\s+\d+\s+obj)\b/. The "N G obj" pattern is followed as states, so a long run of digits or
+     * spaces costs one pass: 1 in N, 2 the spaces after it, 3 in G, 4 the spaces after G, 5 digits that cannot start N.
+     */
+    private mark;
+    /**
+     * Reads the xref streams the scan found, and the sections their /Prev chains lead to, as pdf.js does: each fills
+     * only the numbers nothing before it gave, and a free row counts as given.
+     */
+    private readRecovered;
+    /**
+     * pdf.js's choice among the trailers its recovery finds: the first, in file order, whose /Root and /Pages are
+     * dictionaries, whose /Count is a whole number and that has an /ID, and an /Encrypt if any candidate has one.
+     * Failing that, the last whose /Root and /Pages are dictionaries. A reference whose generation is higher than its
+     * entry's counts in a second pass, once any lookup failed.
+     */
+    private pickTrailer;
+    /** Reads a value one level deep, throwing BrokenEntry where pdf.js's lookup throws. */
+    private fetchStrict;
+    /** With no trailer at all, pdf.js takes the lowest-numbered object, or stream dictionary, that holds /Root. */
+    private rootHolder;
+    /**
+     * Reads every object of the rebuilt map once, in number order, for the readings recovery falls back on: the first
+     * that holds /Root, the object streams, the newest xref stream and the earliest catalog in the file.
+     */
+    private surveyObjects;
+    /** Runs `fn` with the issues as they were before it, for lookups that only decide how to read the file. */
+    private quietly;
+    /**
+     * Where pdf.js's recovery gives no catalog and page tree, as other readers do: every object is read, the objects in
+     * object streams it finds are added, and a catalog is taken from an xref stream dictionary or a /Type /Catalog.
+     */
+    private scanFurther;
     /** Adds the objects held in the object streams scan() found. Their bodies may be encrypted, so this runs after initSecurity(). */
     private indexObjStms;
+    /**
+     * Forgets parsed objects, the stream lengths read for them and the streams lifted out of them, once the map or the
+     * key changes. Only the catalog written in the trailer, lifted once the map is final, outlives this.
+     */
+    private forgetParsed;
     /**
      * Parses the indirect object ("N G obj ...") at a source offset. `at` is where its header starts, past any
      * whitespace, and `end` where its value or its stream body ends.
@@ -136,28 +270,85 @@ export declare class PdfDocument {
     private parseIndirectAt;
     /** Finds a stream body's true length, trusting /Length only when "endstream" follows it. */
     private locateStream;
-    /** Position just before the EOL that precedes "endstream" in [from, to), or -1. */
+    /**
+     * Outside xref streams the filter entries may be indirect. They are resolved one level, as viewers do, so every
+     * reader of this dictionary decodes what a viewer decodes. A reference to a stream or to nothing stays in place.
+     */
+    private resolveFilters;
+    /** Position just before the EOL that precedes "endstream", or a misspelling pdf.js takes, in [from, to), or -1. */
     private findEndstream;
     /** The smallest known object or xref offset after `pos`, or the end of the file. */
     private nextOffsetAfter;
     /** Loads an object by reference, decrypted. Missing and free objects are null. */
     getObject(ref: PdfRef, raw?: boolean): Promise<PdfObject>;
     private decryptObject;
+    /**
+     * Gives each stream written inside `obj` a synthetic number and puts a reference to it in its place, so the walk
+     * and the writer meet it as they meet any stream. `key` names the object whose key decrypts it. `owner` names
+     * where `obj` came from, so a second read of it reuses the numbers. `stm` names the object stream whose decoded
+     * data holds the bodies, when `obj` sits in one.
+     */
+    private lift;
+    /** Numbers an object that has no entry of its own, the same number each time the same object is read. */
+    private addSynthetic;
+    /** Records what the streams inside the object `owner` hold. Throws ParseError when that would pass the budget. */
+    private chargeSynthetic;
+    /** The object whose key decrypts a stream: the stream's own, or for a synthetic one the object that held it. */
+    private keyOf;
+    /**
+     * Reads object `num` from object stream `stmNum`. pdf.js finds an object by the number the stream's header gives it,
+     * and by `index`, the index its entry gives, when the header does not name it. Each object ends where the next one
+     * starts, and the last at the end of the stream, as pdf.js reads them.
+     */
     private getFromObjStm;
-    /** Decodes an object stream in chunks; large ones go to a temporary file instead of memory. */
+    /** The decoded object stream `stmNum`, decoded on first use, or null when it does not read. */
+    private objStm;
+    /** Counts a decoded object stream against memoryThreshold, and moves the least recently used ones out past it. */
+    private holdInMemory;
+    private tempDir;
+    /** A reader over a decoded object stream. Only a few stay open, so a file handle is given up when its reader goes. */
+    private objStmReader;
+    /**
+     * Decodes an object stream in chunks; one larger than memoryThreshold goes to a temporary file. Of its header it
+     * keeps the pairs that the entries pointing into it can reach, or with `all` every pair, for indexing it.
+     */
     private loadObjStm;
+    /** The objects whose entries point into object stream `stmNum`, with the index each entry gives. */
+    private wantedIn;
     private ownTemp?;
     /**
-     * Parses at an offset in any reader, growing the window as needed. A window that reaches `bound`, the next known
-     * object, grows further only while a budget for the whole file lasts, so damaged files still parse and crafted
-     * ones stay linear. `key` names the place read, so its bad tokens count once however often it is parsed.
+     * Parses at an offset in any reader. The window starts small and grows while the parser asks for more. A run of
+     * whitespace and comments that reaches the end of the window is skipped in the source and stands in the window as one
+     * space, and the body of a stream written inside the object is found in the source and left out, so the window holds
+     * the object's tokens and not its padding. A window that reaches `bound`, the next known object, grows further only
+     * while a budget for the whole file lasts, unless `hard` makes the bound the end of the data. `key` names the place
+     * read, so its bad tokens count once however often it is parsed.
      */
     private parseWindowed;
+    /**
+     * Where the line after a "stream" keyword ends, as pdf.js finds it: after the first CR, LF or CRLF from `start` on, or
+     * at `limit`. `text` says whether anything but spaces came first. Bytes past `bound` come out of the parse budget.
+     */
+    private lineEnd;
+    /**
+     * Where a run of whitespace, and of comments when `comments` is set, ends: the first byte at or after `from` that
+     * belongs to neither, or `limit`. `comment` says whether `from` is inside a comment. Bytes past `bound` come out of
+     * the parse budget.
+     */
+    private skipRun;
+    /**
+     * Finds the body of a stream written inside an object, starting at `start`, as the parser would in a window that held
+     * it: at its /Length when "endstream" follows after whitespace, and otherwise at the first "endstream". Returns its
+     * length and where the object goes on after "endstream", or null when no "endstream" comes before `limit`.
+     */
+    private findBody;
     resolve(v: PdfObject | undefined): Promise<PdfObject | undefined>;
     /** The generation of a live object's entry, 0 for one in an object stream or none. */
     genOf(num: number): number;
     /** Length of the stream once decrypted, which is what gets written out. */
     plainLength(stream: PdfStream, num: number): Promise<number>;
+    /** False for a stream encrypted with a key the password did not give, as attached files can be. */
+    canDecrypt(stream: PdfStream, num: number): boolean;
     /** Yields the decrypted, still-encoded body in chunks. */
     plainChunks(stream: PdfStream, num: number): AsyncGenerator<Uint8Array>;
     /** Decrypts and decodes a whole stream body into memory, subject to the decompression limit and an optional cap. */
@@ -185,7 +376,10 @@ export declare class PdfDocument {
     liveNumbers(): IterableIterator<number>;
     /** Drops caches and the object index once the document is no longer read, and deletes spilled object streams. */
     release(): Promise<void>;
-    /** Forgets every decoded object stream, closing and deleting the ones that spilled to a temporary file. */
+    /**
+     * Forgets every decoded object stream, closing and deleting the temporary files, once the map or the key changes. The
+     * streams whose bodies they held are forgotten with the parsed objects.
+     */
     private dropObjStms;
 }
 export {};