@patrickdk77/pdf-defuse 0.2.4 → 0.3.0

package/README.md

@@ -140,6 +140,8 @@ The score runs from 0 to 100. Each kind of finding has a weight: a launch action
 
 `scoreWeights` comes first, then the built-in weights. A kind of finding pdf-defuse does not know, such as one another package's plugin returned, scores at its `weight`, or 0 without one.
 
+A script or file a plugin answered for scores only by the findings the plugin returned, at their weights. pdf-defuse adds nothing of its own for it, and the PLUGIN_PASSED and PLUGIN_SCRUBBED findings that record the answer weigh 0, in the upload's score and in the output's. A form whose scripts a plugin keeps, and in which the plugin finds nothing, scores 0. With no plugin to judge them, the same scripts score at pdf-defuse's own weights.
+
 The bands are NONE at 0, LOW from 1, MEDIUM from 25, HIGH from 50 and CRITICAL from 80.
 
 A run that stops before it has seen the whole file rejects the file with a null score and the UNKNOWN level. Runs stop at a limit, at encryption they cannot open, and at a file they cannot parse or that ends before its trailer. A file with no page to read gets the same result. This holds whatever `actionOverrides` say, because an override can make the finding that stopped the run look harmless. A file rejected for what it holds, for example JavaScript under an override that rejects it, keeps its score. So does a file whose output failed its own inspection.
@@ -178,7 +180,9 @@ A file that encrypts only its attached files, through a crypt filter with `/Auth
 
 ## Plugins
 
-Attached files and scripts are removed unless a plugin keeps them. A plugin says which files or scripts it accepts, then scrubs them, passes them through unchanged, or removes them after all. The first plugin that accepts handles it. If a plugin throws, or scrubs and writes nothing, the item is removed. An error with a `DEFUSE_LIMIT` or `DEFUSE_IO` code stops the whole run instead, as described under The defuse family. A file plugin that passes a file keeps its original bytes, or the bytes it wrote to the sink if it wrote any.
+Attached files and scripts are removed unless a plugin keeps them. A plugin says which files or scripts it accepts, then scrubs them, passes them through unchanged, or removes them after all. The first plugin that accepts handles it. If a plugin throws, or scrubs and writes nothing, the item is removed.
+
+The plugin's answer is final. For a script, the report then holds JAVASCRIPT/PLUGIN_PASSED, PLUGIN_SCRUBBED or PLUGIN_REMOVED and the plugin's findings, and none of pdf-defuse's own, such as JAVASCRIPT/OPEN_ACTION. An attached file gets EMBEDDED_FILE/PLUGIN_PASSED, PLUGIN_SCRUBBED or PLUGIN_REMOVED and the plugin's findings. pdf-defuse reports an item itself only when no plugin answered: a script no plugin accepts gets its JAVASCRIPT finding, and one whose plugin failed gets that finding and JAVASCRIPT/PLUGIN_FAILED. A file no plugin accepts is EMBEDDED_FILE/NO_PLUGIN, and a file whose plugin failed is EMBEDDED_FILE/PLUGIN_FAILED, which weighs what NO_PLUGIN does. An error with a `DEFUSE_LIMIT` or `DEFUSE_IO` code stops the whole run instead, as described under The defuse family. A file plugin that passes a file keeps its original bytes, or the bytes it wrote to the sink if it wrote any.
 
 Five plugins are bundled:
 
@@ -250,7 +254,7 @@ const scriptPlugin = {
 };
 ```
 
-A script plugin can also return `findings` with its result, saying what it found in the script. They go into the report at the script's location, unless a finding gives a finer one, such as the line. A `reject` among them is recorded as `strip`, since the plugin decides one script, not the file. [script-defuse](https://github.com/patrickdk77/script-defuse)'s `formScriptPlugin` keeps form calculations, formats and checks, and reports why it removed every other script.
+A script plugin can also return `findings` with its result, saying what it found in the script. They go into the report at the script's location, unless a finding gives a finer one, such as the line. A `reject` among them is recorded as `strip`, since the plugin decides one script, not the file. Their `weight` is what they add to the score. For a script the plugin passed, the output's report holds them again. [script-defuse](https://github.com/patrickdk77/script-defuse)'s `formScriptPlugin` keeps form calculations, formats and checks, and reports why it removed every other script.
 
 ## The defuse family
 

package/dist/engine.js

@@ -62,7 +62,6 @@ function encryptionFindings(doc, factory) {
         out.push(factory.make(types_1.PdfCategory.Encrypted, types_1.PdfDetail.UnknownCryptFilter, undefined, r.detail ? { filter: r.detail } : undefined));
     return out;
 }
-const PLUGIN_KEPT = new Set([types_1.PdfDetail.PluginPassed, types_1.PdfDetail.PluginScrubbed]);
 /**
  * `incomplete` marks a run that could not inspect the file: it stopped early, or found no page to read. finish()
  * then rejects it without a score, whatever the actions say, because an override can make the finding that stopped
@@ -75,8 +74,9 @@ function finish(findings, options, incomplete, version = '', pages, isOutput = f
     // In an upload, a plugin rewrite or a metadata strip changes the file, so the output must be written.
     // In an output being verified, the change is already there.
     const strippable = findings.some(f => f.action === 'strip' || (!isOutput && (f.detail === types_1.PdfDetail.PluginScrubbed || f.detail === types_1.PdfDetail.Stripped)));
-    // The upload's score leaves out what plugins kept; the output's score counts it.
-    const { score, risk } = (0, score_1.scoreFindings)(isOutput ? findings : findings.filter(f => !PLUGIN_KEPT.has(f.detail)), options);
+    // What a plugin decided counts at the weights of the plugin's own findings; the PLUGIN_* findings that record its
+    // answer weigh nothing.
+    const { score, risk } = (0, score_1.scoreFindings)(findings, options);
     return { status: rejected ? 'rejected' : strippable ? 'strippable' : 'clean', score, risk, findings, version, pages };
 }
 /**

package/dist/findings.js

@@ -46,8 +46,8 @@ def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.Rendition, 'strip', 50, 'A
 def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.Url, 'strip', 50, 'A link uses a javascript: URL');
 def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.Unattached, 'strip', 50, 'A JavaScript entry sits outside any action');
 def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.Xfa, 'strip', 50, 'An XFA form script, in <language>, runs on a form event');
-def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginPassed, 'info', 40, 'JavaScript kept unchanged by plugin <plugin>');
-def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginScrubbed, 'info', 20, 'JavaScript rewritten and kept by plugin <plugin>');
+def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginPassed, 'info', 0, 'JavaScript kept unchanged by plugin <plugin>');
+def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginScrubbed, 'info', 0, 'JavaScript rewritten and kept by plugin <plugin>');
 def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginFailed, 'strip', 0, 'Plugin <plugin> failed, so the JavaScript was removed');
 def(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginRemoved, 'strip', 0, 'JavaScript removed by plugin <plugin>');
 // ACTION
@@ -78,9 +78,9 @@ def(types_1.PdfCategory.Link, types_1.PdfDetail.PathTrick, 'strip', 40, 'A relat
 // EMBEDDED_FILE
 def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.NoPlugin, 'strip', 30, '<Attached file "{name}"|An attached file> (<{sniffed}|{type}|unknown type>) removed. No plugin accepts this type');
 def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.TypeMismatch, 'info', 40, 'The types of <attached file "{name}"|an unnamed attached file> disagree: <its name says {nameType}|its name has no known extension>, <it is declared {declared}|it has no declared type>, and <its content looks like {sniffed}|its content is not recognized>');
-def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginPassed, 'info', 20, 'Attached file kept by plugin <plugin> with nothing removed');
-def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginScrubbed, 'info', 10, 'Attached file cleaned and kept by plugin <plugin>');
-def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginFailed, 'strip', 0, 'Plugin <plugin> failed, so the attached file was removed');
+def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginPassed, 'info', 0, 'Attached file kept by plugin <plugin> with nothing removed');
+def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginScrubbed, 'info', 0, 'Attached file cleaned and kept by plugin <plugin>');
+def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginFailed, 'strip', 30, 'Plugin <plugin> failed, so the attached file was removed');
 def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.PluginRemoved, 'strip', 0, 'Attached file removed by plugin <plugin>');
 def(types_1.PdfCategory.EmbeddedFile, types_1.PdfDetail.Portfolio, 'strip', 30, 'PDF is a portfolio of files');
 // MEDIA
@@ -103,9 +103,9 @@ def(types_1.PdfCategory.Form, types_1.PdfDetail.XfaConnection, 'strip', 40, 'An
 def(types_1.PdfCategory.Form, types_1.PdfDetail.XfaExternalObject, 'strip', 60, 'An XFA form embeds an external object');
 def(types_1.PdfCategory.Form, types_1.PdfDetail.XfaExternalReference, 'strip', 40, 'An XFA form names something outside the PDF to load');
 def(types_1.PdfCategory.Form, types_1.PdfDetail.XfaImage, 'strip', 20, 'An XFA form image (<type|unknown type>) removed. <reason|No plugin accepts this type>');
-def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginPassed, 'info', 10, 'XFA form image kept by plugin <plugin> with nothing removed');
-def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginScrubbed, 'info', 5, 'XFA form image cleaned and kept by plugin <plugin>');
-def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginFailed, 'strip', 0, 'Plugin <plugin> failed, so the XFA form image was removed');
+def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginPassed, 'info', 0, 'XFA form image kept by plugin <plugin> with nothing removed');
+def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginScrubbed, 'info', 0, 'XFA form image cleaned and kept by plugin <plugin>');
+def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginFailed, 'strip', 20, 'Plugin <plugin> failed, so the XFA form image was removed');
 def(types_1.PdfCategory.Form, types_1.PdfDetail.PluginRemoved, 'strip', 0, 'XFA form image removed by plugin <plugin>');
 def(types_1.PdfCategory.Form, types_1.PdfDetail.CalculationOrder, 'strip', 20, 'Form calculation order removed with the calculation scripts it ran');
 // SIGNATURE

package/dist/walker.d.ts

@@ -6,7 +6,10 @@ import { type ByteSink, type DefuseFinding, type PdfOptions, type PluginContext
 interface KeptItem {
     result: 'passed' | 'scrubbed';
     plugin: string;
-    /** For a file: the findings inside the kept bytes, which go into the output's report. */
+    /**
+     * What goes into the output's report for it: for a file, the findings inside the kept bytes, and for a script the
+     * plugin passed, what the plugin found in it.
+     */
     findings?: DefuseFinding[];
 }
 export interface VerifyExpectations {
@@ -103,6 +106,8 @@ export declare class Walker {
     private againBudget;
     /** A script's findings from visits of removed content, held until the walk ends so a later real visit that keeps it can correct them. */
     private readonly shadowScripts;
+    /** Scripts a plugin answered for. Its answer is final, so what a shadow visit made of them is not reported. */
+    private readonly answeredScripts;
     /** Information dictionaries and XMP streams removed for the caller. */
     private readonly strippedMeta;
     /** Bytes of scrubbed attachments held in memory until the write. */

package/dist/walker.js

@@ -336,6 +336,8 @@ class Walker {
     againBudget = 16 * 1024 * 1024;
     /** A script's findings from visits of removed content, held until the walk ends so a later real visit that keeps it can correct them. */
     shadowScripts = new Map();
+    /** Scripts a plugin answered for. Its answer is final, so what a shadow visit made of them is not reported. */
+    answeredScripts = new Set();
     /** Information dictionaries and XMP streams removed for the caller. */
     strippedMeta = new Set();
     /** Bytes of scrubbed attachments held in memory until the write. */
@@ -1052,9 +1054,10 @@ class Walker {
                 d.types.decoded = undefined;
             await (await decoded?.catch(() => undefined))?.close?.();
         }
-        for (const list of this.shadowScripts.values())
-            for (const f of list)
-                this.push(f);
+        for (const [key, list] of this.shadowScripts)
+            if (!this.answeredScripts.has(key))
+                for (const f of list)
+                    this.push(f);
         if (this.dropped.size || this.refusedSlots.size)
             this.removedAt ??= 'document';
         if (this.coPresent && this.calcRemoved && this.keptCalcFields.size === 0)
@@ -1898,6 +1901,7 @@ class Walker {
         const verify = sc.language ? this.cfg.verify?.scripts.get(sc.text) : undefined;
         if (verify) {
             this.emit(types_1.PdfCategory.JavaScript, verify.result === 'passed' ? types_1.PdfDetail.PluginPassed : types_1.PdfDetail.PluginScrubbed, location, { plugin: verify.plugin });
+            this.pushScript(verify.findings, location);
             return { keep: true };
         }
         const language = sc.language === 'formcalc' ? 'FormCalc' : sc.language === 'javascript' ? 'JavaScript' : short(sc.contentType || 'an unreadable form', 40);
@@ -1905,6 +1909,7 @@ class Walker {
         let decision = { keep: f.action !== 'strip' && sc.language !== undefined };
         let answer;
         let found;
+        let answered = false;
         if (f.action === 'strip' && sc.language) {
             for (const plugin of this.options.scriptPlugins ?? []) {
                 const script = {
@@ -1923,10 +1928,11 @@ class Walker {
                     const r = await plugin.process(script);
                     if (Array.isArray(r.findings))
                         found = r.findings;
+                    answered = r.result === 'passed' || r.result === 'removed' || (r.result === 'scrubbed' && typeof r.text === 'string' && r.text.length > 0);
                     if (r.result === 'passed') {
                         decision = { keep: true };
                         answer = this.cfg.factory.make(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginPassed, location, data);
-                        this.kept.scripts.set(sc.text, { result: 'passed', plugin: plugin.name });
+                        this.kept.scripts.set(sc.text, { result: 'passed', plugin: plugin.name, findings: found });
                     }
                     else if (r.result === 'scrubbed' && typeof r.text === 'string' && r.text.length) {
                         decision = { keep: true, text: r.text };
@@ -1939,20 +1945,23 @@ class Walker {
                 }
                 catch (e) {
                     this.passOn(e);
+                    answered = false;
                     answer = this.cfg.factory.make(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginFailed, location, data);
                 }
                 break;
             }
-            if (decision.keep)
-                f.action = 'info';
         }
         // A script in a language no plugin reads goes, whatever the override says.
         if (!sc.language && f.action === 'info')
             f.action = 'strip';
-        this.push(f);
+        // A plugin's answer is final: the report holds the answer and what the plugin found, at the plugin's weights, and
+        // nothing of this package's own. Its own finding stands when no plugin answered.
+        if (!answered)
+            this.push(f);
         if (answer)
             this.push(answer);
-        this.pushScript(found, location);
+        if (answered)
+            this.pushScript(found, location);
         return decision;
     }
     /**
@@ -2570,6 +2579,7 @@ class Walker {
         const verify = src !== undefined ? this.cfg.verify?.scripts.get(src) : undefined;
         if (verify) {
             this.emit(types_1.PdfCategory.JavaScript, verify.result === 'passed' ? types_1.PdfDetail.PluginPassed : types_1.PdfDetail.PluginScrubbed, ctx.location, { plugin: verify.plugin });
+            this.pushScript(verify.findings, ctx.location);
             if (first)
                 return first;
             const d = { keep: true };
@@ -2581,6 +2591,7 @@ class Walker {
         let decision = { keep: f.action !== 'strip' };
         let answer;
         let found;
+        let answered = false;
         if (f.action === 'strip' && !ctx.shadow && src !== undefined) {
             for (const plugin of this.options.scriptPlugins ?? []) {
                 const script = {
@@ -2599,11 +2610,12 @@ class Walker {
                     const r = await plugin.process(script);
                     if (Array.isArray(r.findings))
                         found = r.findings;
+                    answered = r.result === 'passed' || r.result === 'removed' || (r.result === 'scrubbed' && typeof r.text === 'string' && r.text.length > 0);
                     if (r.result === 'passed') {
                         decision = { keep: true };
                         answer = this.cfg.factory.make(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginPassed, ctx.location, data);
                         if (!first)
-                            this.kept.scripts.set(src, { result: 'passed', plugin: plugin.name });
+                            this.kept.scripts.set(src, { result: 'passed', plugin: plugin.name, findings: found });
                     }
                     else if (r.result === 'scrubbed' && typeof r.text === 'string' && r.text.length) {
                         decision = { keep: true, text: r.text };
@@ -2619,12 +2631,11 @@ class Walker {
                 }
                 catch (e) {
                     this.passOn(e);
+                    answered = false;
                     answer = this.cfg.factory.make(types_1.PdfCategory.JavaScript, types_1.PdfDetail.PluginFailed, ctx.location, data);
                 }
                 break;
             }
-            if (decision.keep)
-                f.action = 'info';
         }
         if (ctx.shadow) {
             // Recorded when the walk ends, after any real visit that keeps the script has corrected it.
@@ -2635,10 +2646,16 @@ class Walker {
                 this.shadowScripts.set(key, [f]);
             return { keep: false };
         }
-        this.push(f);
+        // A plugin's answer is final: the report holds the answer and what the plugin found, at the plugin's weights, and
+        // nothing of this package's own. Its own finding stands when no plugin answered.
+        if (answered)
+            this.answeredScripts.add(key);
+        else
+            this.push(f);
         if (answer)
             this.push(answer);
-        this.pushScript(found, ctx.location);
+        if (answered)
+            this.pushScript(found, ctx.location);
         if (first) {
             // One object holds one text, so a different rewrite is a refusal too, and only the slot can report it.
             if (first.keep && (!decision.keep || decision.text !== first.text)) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@patrickdk77/pdf-defuse",
-  "version": "0.2.4",
+  "version": "0.3.0",
   "description": "Sanitize and clean PDF files. Detects and removes JavaScript, attached files, launch actions, unsafe links and empty-password encryption, scores the risk, and writes a disarmed copy with bounded memory and no runtime dependencies.",
   "license": "MIT",
   "author": "patrickdk77",