pi-quiver 6.10.0 → 6.11.0

package/CHANGELOG.md

@@ -8,6 +8,14 @@ Published to npm as `pi-quiver` (`pi install npm:pi-quiver`). Pushing a
 via OIDC trusted publishing. The release helper at
 `.agents/skills/release/scripts/release.sh` cuts the tag; CI publishes.
 
+## v6.11.0 - 2026-10-02
+
+- `doc_to_md`: textless PDF pages holding one eligible full-page image deliver the embedded stream without re-encoding; the handle prints `Native-Images:` and tool details / CLI JSON carry `nativeImages` (#27).
+- `doc_to_md`: native probing, PDF page renders, and textless-page inline OCR run in a spawned raster worker with per-job budgets; a toxic page becomes a `Failed pages` note instead of failing the conversion when other pages succeed (#27).
+- `doc_to_md`: the render ceiling rises from 16 to 50 Mpx; clamped PDF renders add Notes and report effective `pageImages[].dpi` plus `requestedDpi` (#27).
+- `doc_to_md`: `hideAnnotations` / `--hide-annotations` hides PDF annotations and form widgets on renders and lets annotated scans use native delivery; default runs keep their painted render (#27).
+- `doc_to_md`: `.done` markers retain native-image and clamp metadata for completed pages across primary-tier failure and fallback; corrupt metadata does not discard completed images (#27).
+
 ## v6.10.0 - 2026-10-02
 
 - `doc_to_md`: new per-call `words` option (`--words`; not settable) writes `<stem>.words.json` - per selected PDF/image page, every text-layer word and every word inline OCR recognized in the same run, with display-space bbox (points; source pixels for images) and `source` `text`/`ocr`. Under `--ocr-mode all` each sidecar gets `ocr/<stem>-pNNN.words.json`. Never triggers OCR; Markdown, page stats and OCR outcome are unchanged. Handle gains `Words:`, `--json` gains `wordsPath`, `wordsReason`, `wordsErrors`, `ocr.wordSidecars`; `--info --words` is a usage error (#28).

package/README.md

@@ -65,7 +65,7 @@ A 300 KB changelog page never touches your context window - you get a preview an
 | Extension | Tool | What it does |
 | --- | --- | --- |
 | `extensions/fetch.ts` | `fetch` | Retrieve URLs over HTTP(S). HTML -> Markdown (Readability extraction, Turndown conversion). Binary saved untouched to a temp file. GitHub issue/PR/repo/actions-run/actions-job URLs auto-route through `gh` (falls back to HTTP); failed runs/jobs include failed-step logs (best-effort, summary-only otherwise). Same size gate as `fetch`. Behavior lives in `lib/fetch-core.ts`; also exposed as the `pi-quiver fetch` CLI (see [Claude Code support](#claude-code-support)). |
-| `extensions/doc_to_md.ts` | `doc_to_md` | Convert a local PDF/DOCX/DOC/PPTX/XLSX/XLSM/XLS, MSG/EML, HTML, or image file to a Markdown bundle on disk (`<stem>.md` + `images/`, spreadsheet `sheets/`, optional `pageImages` in `pages/`, and email `attachments/`) and return a handle (paths, page count, outline, diagnostics) - never inline Markdown. `info` inspects PDF/Office inputs; `pages` selects 1-based PDF/Office pages (DOCX: explicit-page-break segments); HTML, image, and email inputs reject both; every selected PDF/PPTX page or DOCX segment when `pageCount > 1` ends with `--- end of page.page_number=N ---`. Tiers: pymupdf4llm -> PyMuPDF text (degraded) -> unpdf worker (no Python only). Excel -> sheet inventory, full CSVs, bounded previews, and optional rendered views. HTML keeps local and `data:` images in the bundle and remote images as links; image inputs keep the original image. Scanned PDF pages keep a page picture on both Python tiers; opt-in OCR needs Tesseract language data. DOCX converts directly (mammoth, python-docx fallback); LibreOffice pagination marks the degraded route. The Outline lists `L<line>` and `p<page>` per heading. Settings under `quiver.docToMd`. Every conversion that goes through the Python PDF tiers (PDF, PPTX, `.doc`, DOCX via LibreOffice) writes `<stem>.pages.json` (per-page chars, image count, image coverage; handle line `Page-Stats:`); unpdf writes no stats. `ocrMode: "all"` with `ocr: true` and an explicit `pages` forces OCR on those pages into `ocr/<stem>-pNNN.md` sidecars, leaving the Markdown byte-identical to the same selected-page call without OCR. `words: true` (CLI `--words`, per-call only - no settings key) writes `<stem>.words.json` with every word's bbox per selected PDF/image page (`source` `text` or `ocr`), plus `ocr/<stem>-pNNN.words.json` beside each forced-OCR sidecar; it never triggers OCR. |
+| `extensions/doc_to_md.ts` | `doc_to_md` | Convert a local PDF/DOCX/DOC/PPTX/XLSX/XLSM/XLS, MSG/EML, HTML, or image file to a Markdown bundle on disk (`<stem>.md` + `images/`, spreadsheet `sheets/`, optional `pageImages` in `pages/`, and email `attachments/`) and return a handle (paths, page count, outline, diagnostics) - never inline Markdown. `info` inspects PDF/Office inputs; `pages` selects 1-based PDF/Office pages (DOCX: explicit-page-break segments); HTML, image, and email inputs reject both; every selected PDF/PPTX page or DOCX segment when `pageCount > 1` ends with `--- end of page.page_number=N ---`. Tiers: pymupdf4llm -> PyMuPDF text (degraded) -> unpdf worker (no Python only). Excel -> sheet inventory, full CSVs, bounded previews, and optional rendered views. HTML keeps local and `data:` images in the bundle and remote images as links; image inputs keep the original image. Scanned PDF pages keep a page picture on both Python tiers; eligible single full-page images are delivered as their embedded stream, without re-encoding or render DPI; opt-in OCR needs Tesseract language data. DOCX converts directly (mammoth, python-docx fallback); LibreOffice pagination marks the degraded route. The Outline lists `L<line>` and `p<page>` per heading. Settings under `quiver.docToMd`. Every conversion that goes through the Python PDF tiers (PDF, PPTX, `.doc`, DOCX via LibreOffice) writes `<stem>.pages.json` (per-page chars, image count, image coverage; handle line `Page-Stats:`); unpdf writes no stats. `ocrMode: "all"` with `ocr: true` and an explicit `pages` forces OCR on those pages into `ocr/<stem>-pNNN.md` sidecars, leaving the Markdown byte-identical to the same selected-page call without OCR. `words: true` (CLI `--words`, per-call only - no settings key) writes `<stem>.words.json` with every word's bbox per selected PDF/image page (`source` `text` or `ocr`), plus `ocr/<stem>-pNNN.words.json` beside each forced-OCR sidecar; it never triggers OCR. |
 | `extensions/session-name.ts` | `/session-name` | Manual + opt-in automatic session naming, naming rules and deny list, long-session revisits, and Ghostty/Herdr tab rename. OFF by default. |
 | `extensions/sword-header.ts` | `/builtin-header` | Themed ASCII startup header replacing pi's default logo. OFF by default. |
 | `extensions/fast-mode.ts` | `/fast` | Inject Anthropic fast-mode (`speed: "fast"` + `anthropic-beta: fast-mode-2026-02-01`) into every Claude Opus 4.8 / Opus 5 request, any thinking level. `--fast` flag + `/fast [on\|off\|status]`. OFF by default. |
@@ -303,8 +303,9 @@ Each setting can also be overridden per-process via `PI_QUIVER_SLACK_ENABLED`, `
 | `excelTimeoutMs` | `60000` | Excel child and Excel info deadline. |
 | `warmTimeoutMs` | `120000` | Absolute first-call backend discovery/bootstrap deadline. |
 | `pymupdfVersion` | `1.27.2.3` | pymupdf4llm pin, minimum `1.27.0`. |
-| `imageDpi` | `150` | Render DPI for page images and Excel rendered views (capped by a 16 Mpx budget). |
+| `imageDpi` | `150` | Render DPI for page images and Excel rendered views (capped by a 50 Mpx budget). |
 | `imageFormat` | `png` | Rendered image format: `png` or `jpg`; embedded images retain their extension. |
+| `hideAnnotations` | `false` | Hide annotations on PDF textless-page and `pages/` renders, including form-field widgets (filled values disappear). Default paints them. Does not affect OCR text or embedded images; also lets an annotated scan be delivered as its embedded image. |
 | `maxOutputBytes` | `20000000` | Child stdout cap in bytes. |
 | `outlineMaxEntries` | `40` | Heading outline, TOC, or sheet inventory entries in the handle. |
 | `ocr` | `false` | Opt-in OCR for scanned pages and images when Tesseract language data is installed. |
@@ -314,7 +315,7 @@ Each setting can also be overridden per-process via `PI_QUIVER_SLACK_ENABLED`, `
 
 A bundle is `<outputDir>/<stem>.md` plus `<outputDir>/<stem>.pages.json` on Python PDF tiers, optional OCR sidecars under `<outputDir>/ocr/`, and `<outputDir>/images/` and, for spreadsheets with data, `<outputDir>/sheets/`; without `outputDir`, the tool creates a per-call temp root. A conversion owns `<stem>.md.lock` until it atomically publishes the Markdown. An existing `<stem>.md` fails the call unless `overwrite` is set; `overwrite` replaces that Markdown and the files it owns (`images/<stem>-p<N>-<n>.*`, `images/<stem>-s<idx>[-<n>].*`, `sheets/<stem>-s<idx>-<slug>.csv`, `<stem>.pages.json`, `ocr/<stem>-pNNN.md`), nothing else. Temp bundles are caller-owned - the tool never deletes a bundle it produced.
 
-Excel needs a Python backend with openpyxl, xlrd and pillow - otherwise the call fails with `Remedy: install uv, or pip install openpyxl xlrd pillow`. The Markdown opens with a `## Sheets` table listing every sheet in workbook order (0-based `#`, `worksheet`/`chartsheet`, size, hidden, chart and image counts, rendered view, CSV link for non-empty worksheets), then one section per sheet: a `Data:` line linking the full-content CSV under `sheets/` for non-empty worksheets, chart metadata from the workbook model, embedded images, an optional rendered view, a preview of at most the first 100 rows x 50 columns, and - only when the preview is truncated - a `Columns:` profile (type, non-empty count, min/max, distinct up to 50). Sizes are the extent of non-empty cells (the `info` handle reports the raw worksheet dimensions instead, which may be larger). Rendered views (`images/<stem>-s<idx>.<fmt>`) are produced for sheets carrying charts or images when LibreOffice is on `PATH`: the workbook is exported one PDF page per sheet and rasterized under a 16 Mpx budget. Any LibreOffice or rasterization failure degrades to `Rendered view: unavailable (<reason>)` and a handle note; it never fails the conversion. Workbooks whose chartsheet drawings carry a zero-size anchor (openpyxl-authored files; Excel-authored files are unaffected) render as a degenerate page and are reported as such. `.xls` gets the inventory, CSVs and previews but no visual detection or rendering.
+Excel needs a Python backend with openpyxl, xlrd and pillow - otherwise the call fails with `Remedy: install uv, or pip install openpyxl xlrd pillow`. The Markdown opens with a `## Sheets` table listing every sheet in workbook order (0-based `#`, `worksheet`/`chartsheet`, size, hidden, chart and image counts, rendered view, CSV link for non-empty worksheets), then one section per sheet: a `Data:` line linking the full-content CSV under `sheets/` for non-empty worksheets, chart metadata from the workbook model, embedded images, an optional rendered view, a preview of at most the first 100 rows x 50 columns, and - only when the preview is truncated - a `Columns:` profile (type, non-empty count, min/max, distinct up to 50). Sizes are the extent of non-empty cells (the `info` handle reports the raw worksheet dimensions instead, which may be larger). Rendered views (`images/<stem>-s<idx>.<fmt>`) are produced for sheets carrying charts or images when LibreOffice is on `PATH`: the workbook is exported one PDF page per sheet and rasterized under a 50 Mpx budget. Any LibreOffice or rasterization failure degrades to `Rendered view: unavailable (<reason>)` and a handle note; it never fails the conversion. Workbooks whose chartsheet drawings carry a zero-size anchor (openpyxl-authored files; Excel-authored files are unaffected) render as a degenerate page and are reported as such. `.xls` gets the inventory, CSVs and previews but no visual detection or rendering.
 
 Worst-case wall time: PDF `warmTimeoutMs (first call) + primaryTimeoutMs + fallbackTimeoutMs`; PPTX adds `sofficeTimeoutMs`; DOCX on the Python path `warmTimeoutMs + primaryTimeoutMs` (success or a terminal child failure), DOCX child exit 1 then LibreOffice `warmTimeoutMs + primaryTimeoutMs + sofficeTimeoutMs + primaryTimeoutMs + fallbackTimeoutMs`, DOCX without a DOCX-capable backend `warmTimeoutMs + sofficeTimeoutMs + primaryTimeoutMs + fallbackTimeoutMs`; Excel `warmTimeoutMs + excelTimeoutMs + sofficeTimeoutMs + fallbackTimeoutMs`. Add `KILL_GRACE_MS` (2000 ms) per kill. There is no cap on image count, image bytes, cell count or workbook memory - deliberately; the per-tier timeouts, the rendered-view pixel budget and `maxOutputBytes` are the bounds.
 

package/dist/bin/pi-quiver.js

@@ -680,6 +680,13 @@ function publishStaged(b) {
       continue;
     }
     const page = Number(m[1]);
+    const raw = readFileSync(join2(pageDir, ".done"), "utf8").trim();
+    let meta = {};
+    try {
+      const v = raw ? JSON.parse(raw) : {};
+      if (v && typeof v === "object" && !Array.isArray(v)) meta = v;
+    } catch {
+    }
     const files = readdirSync(pageDir).filter((f) => f !== ".done" && statSync(join2(pageDir, f)).isFile()).sort();
     const names = [];
     files.forEach((f, i) => {
@@ -689,8 +696,9 @@ function publishStaged(b) {
       b.sourceMap.set(`${dir}/${f}`, `images/${name}`);
       names.push(name);
     });
+    if (meta.native) meta.native = { ...meta.native, file: names[files.indexOf(meta.native.file)] ?? meta.native.file };
     rmSync(pageDir, { recursive: true, force: true });
-    out.set(page, names);
+    out.set(page, { files: names, meta });
   }
   return out;
 }
@@ -969,6 +977,7 @@ function formatHandle(h) {
   if (h.pagesDir && h.pageImageCount > 0) lines.push(`Pages-Dir: ${h.pagesDir} (${h.pageImageCount} pages)`);
   else if (h.pageImagesReason) lines.push(`Pages-Dir: none - ${h.pageImagesReason}`);
   if (h.pageStatsPath) lines.push(`Page-Stats: ${h.pageStatsPath}`);
+  if (h.nativeImages.length) lines.push(`Native-Images: ${h.nativeImages.length === 1 ? "page" : "pages"} ${compactRanges(h.nativeImages.map((n) => n.page))} (embedded image streams, no render DPI)`);
   if (h.wordsPath) {
     const bad = Object.keys(h.wordsErrors).map(Number).sort((a, b) => a - b);
     lines.push(`Words: ${h.wordsPath}${bad.length ? ` (extraction failed for pages ${compactRanges(bad)}: ${h.wordsErrors[bad[0]]})` : ""}`);
@@ -1036,7 +1045,8 @@ var DOC_TO_MD_OPTIONS = [
   { key: "maxOutputBytes", flag: "--max-output-bytes", type: "int", default: 2e7, settable: true, help: "Child stdout cap in bytes" },
   { key: "outlineMaxEntries", flag: "--outline-max-entries", type: "int", default: 40, settable: true, help: "Heading outline / TOC / sheet inventory cap in the handle" },
   { key: "ocr", flag: "--ocr", type: "bool", default: false, settable: true, help: "Run OCR on pages without a text layer and on image inputs when Tesseract language data is installed; off by default (--no-ocr turns a settings-level true off)" },
-  { key: "ocrLanguage", flag: "--ocr-language", type: "lang", default: "eng", settable: true, help: "Tesseract language code(s), +-joined, e.g. deu+eng" }
+  { key: "ocrLanguage", flag: "--ocr-language", type: "lang", default: "eng", settable: true, help: "Tesseract language code(s), +-joined, e.g. deu+eng" },
+  { key: "hideAnnotations", flag: "--hide-annotations", type: "bool", default: false, settable: true, help: "Render PDF pages without annotations (sticky notes, highlights, stamps - and form-field widgets, so filled form values disappear); default paints them, as PyMuPDF does. Applies to pages/ renders and textless-page renders, not to OCR text or embedded images; also lets an annotated scan be delivered as its embedded image." }
 ];
 var TUNABLE_DEFAULTS = Object.fromEntries(
   DOC_TO_MD_OPTIONS.filter((d) => d.settable).map((d) => [d.key, d.default])
@@ -1706,6 +1716,22 @@ function handleOcr(tier, type, o, json) {
   if (!x || type !== "image" && !x.textless.length && !x.pages.length && !x.ocrFailed.length) return null;
   return { ...emptyOcr(o.ocrLanguage), ...x };
 }
+var nativeFromChild = (b, json, notes) => (json.nativeImages ?? []).flatMap((e) => {
+  const file = b.sourceMap.get(`p${e.page}/${e.file}`);
+  if (!file) {
+    notes.push(`Native image p${e.page}/${e.file} not published`);
+    return [];
+  }
+  return [{ ...e, file: join3(b.root, file) }];
+});
+function retainedNative(b, kept, notes) {
+  const out = [];
+  for (const [page, k] of kept) {
+    if (k.meta.native) out.push({ page, ...k.meta.native, file: join3(b.imagesDir, k.meta.native.file) });
+    if (k.meta.dpi !== void 0 && k.meta.requestedDpi !== void 0 && k.meta.dpi < k.meta.requestedDpi) notes.push(`Page ${page} rendered at ${k.meta.dpi} dpi (requested ${k.meta.requestedDpi}; 50 Mpx ceiling)`);
+  }
+  return out;
+}
 async function convertDocument(o, signal, seams) {
   const s = { backend: (c) => getBackend(c, void 0, signal), runTier: runTierReal, office: tryConvertOffice, ...seams };
   const inputPath = resolve2(o.path);
@@ -1732,10 +1758,11 @@ async function convertDocument(o, signal, seams) {
   let office = null;
   try {
     let pdfPath = inputPath;
-    const base = { path: inputPath, pages: o.pages, ...o.words && (type === "pdf" || type === "image") ? { words: true } : {}, stagingDir: b.stagingDir, sheetsStagingDir: b.sheetsStagingDir, pageImages: o.pageImages, pagesStagingDir: b.pagesStagingDir, imageDpi: o.imageDpi, imageFormat: o.imageFormat, maxOutputBytes: o.maxOutputBytes, pymupdfVersion: o.pymupdfVersion, ocr: o.ocr && !forced, ocrLanguage: o.ocrLanguage, ocrBudgetMs: o.primaryTimeoutMs };
+    const base = { path: inputPath, pages: o.pages, ...o.words && (type === "pdf" || type === "image") ? { words: true } : {}, stagingDir: b.stagingDir, sheetsStagingDir: b.sheetsStagingDir, pageImages: o.pageImages, pagesStagingDir: b.pagesStagingDir, imageDpi: o.imageDpi, imageFormat: o.imageFormat, maxOutputBytes: o.maxOutputBytes, pymupdfVersion: o.pymupdfVersion, ocr: o.ocr && !forced, ocrLanguage: o.ocrLanguage, ocrBudgetMs: o.primaryTimeoutMs, hideAnnotations: o.hideAnnotations };
     let tier, engine, json, degraded = null, fallbackReason = null;
     let explicitBreaks = null;
     let notes = [];
+    let nativeImages = [];
     let officeRoute = null;
     let copyReason = null;
     if (type === "html") {
@@ -1890,11 +1917,12 @@ async function convertDocument(o, signal, seams) {
           tier = "primary";
           engine = "pymupdf4llm";
           json = p.json;
+          nativeImages = nativeFromChild(b, json, notes);
           if (json.pageImages?.length) publishPageImages(b, json.pageCount ?? 0);
         } else if ("userError" in p) throw new Error(p.userError);
         else {
           if (signal?.aborted) throw new Error("aborted");
-          const keepPages = Object.fromEntries([...kept.entries()].map(([k, v]) => [String(k), v]));
+          const keepPages = Object.fromEntries([...kept.entries()].map(([k, v]) => [String(k), v.files]));
           rmSync2(b.pagesStagingDir, { recursive: true, force: true });
           const f = await s.runTier("pdf-fallback", { ...pdfBase, keepPages }, b, signal, o.fallbackTimeoutMs, backend);
           publishStaged(b);
@@ -1905,6 +1933,7 @@ async function convertDocument(o, signal, seams) {
           json = f.json;
           degraded = DEGRADED_TEXT;
           fallbackReason = `primary ${p.reason}`;
+          nativeImages = [...retainedNative(b, kept, notes), ...nativeFromChild(b, json, notes)].sort((x, y) => x.page - y.page);
         }
       }
     }
@@ -1957,7 +1986,7 @@ async function convertDocument(o, signal, seams) {
 ` : "") + body;
     commitBundle(b, markdown);
     const outline = scanOutline(markdown, o.outlineMaxEntries);
-    const details = { path: inputPath, backend: backend.kind, pymupdfVersion: o.pymupdfVersion, inputType: type, file: b.mdPath, outputDir: b.root, savedTo: b.mdPath, imagesDir: b.imagesDir, sheetsDir: b.csvManifest.size ? b.sheetsDir : null, pagesDir: b.pageManifest.size ? b.pagesDir : null, pageImageCount: b.pageManifest.size, pageImagesReason, type, engine, tier, pageCount: json.pageCount ?? null, pages: o.pages, explicitBreaks, imageCount: b.manifest.size, bytes: Buffer.byteLength(markdown, "utf8"), lines: markdown.split("\n").length, degraded, fallbackReason, failedPages: (json.failedPages ?? []).map((f) => f.page), emptyPages: json.emptyPages ?? [], notes, outline: outline.entries, outlineTotal: outline.total, ocr, pageStats, pageStatsPath: pageStats ? b.pageStatsPath : null, ocrDir: b.ocrManifest.size ? b.ocrDir : null, wordsPath, wordsReason, wordsErrors };
+    const details = { path: inputPath, backend: backend.kind, pymupdfVersion: o.pymupdfVersion, inputType: type, file: b.mdPath, outputDir: b.root, savedTo: b.mdPath, imagesDir: b.imagesDir, sheetsDir: b.csvManifest.size ? b.sheetsDir : null, pagesDir: b.pageManifest.size ? b.pagesDir : null, pageImageCount: b.pageManifest.size, pageImagesReason, type, engine, tier, pageCount: json.pageCount ?? null, pages: o.pages, explicitBreaks, imageCount: b.manifest.size, bytes: Buffer.byteLength(markdown, "utf8"), lines: markdown.split("\n").length, degraded, fallbackReason, failedPages: (json.failedPages ?? []).map((f) => f.page), emptyPages: json.emptyPages ?? [], notes, outline: outline.entries, outlineTotal: outline.total, ocr, pageStats, pageStatsPath: pageStats ? b.pageStatsPath : null, ocrDir: b.ocrManifest.size ? b.ocrDir : null, nativeImages, wordsPath, wordsReason, wordsErrors };
     return { output: formatHandle(details), details };
   } catch (e) {
     abortBundle(b);

package/lib/doc-to-md-bundle.ts

@@ -109,9 +109,12 @@ export function openBundle(root: string, requested: string, overwrite: boolean):
 	} catch (e) { rmSync(lockPath, { force: true }); throw e; }
 }
 
-/** Publish every `p<N>/` staging dir carrying `.done`; discard partial ones. Returns page -> published filenames. */
-export function publishStaged(b: Bundle): Map<number, string[]> {
-	const out = new Map<number, string[]>();
+export interface DoneMeta { native?: { file: string; width: number; height: number }; dpi?: number; requestedDpi?: number }
+export interface StagedPage { files: string[]; meta: DoneMeta }
+
+/** Publish completed pages, retaining metadata with native filenames renamed; discard partial pages. */
+export function publishStaged(b: Bundle): Map<number, StagedPage> {
+	const out = new Map<number, StagedPage>();
 	if (!existsSync(b.stagingDir)) return out;
 	for (const dir of readdirSync(b.stagingDir).sort()) {
 		const m = dir.match(/^p(\d+)$/);
@@ -119,6 +122,12 @@ export function publishStaged(b: Bundle): Map<number, string[]> {
 		const pageDir = join(b.stagingDir, dir);
 		if (!existsSync(join(pageDir, ".done"))) { rmSync(pageDir, { recursive: true, force: true }); continue; }
 		const page = Number(m[1]);
+		const raw = readFileSync(join(pageDir, ".done"), "utf8").trim();
+		let meta: DoneMeta = {};
+		try {
+			const v = raw ? JSON.parse(raw) : {};
+			if (v && typeof v === "object" && !Array.isArray(v)) meta = v;
+		} catch { /* Completed images survive truncated metadata. */ }
 		const files = readdirSync(pageDir).filter((f) => f !== ".done" && statSync(join(pageDir, f)).isFile()).sort();
 		const names: string[] = [];
 		files.forEach((f, i) => {
@@ -128,8 +137,9 @@ export function publishStaged(b: Bundle): Map<number, string[]> {
 			b.sourceMap.set(`${dir}/${f}`, `images/${name}`);
 			names.push(name);
 		});
+		if (meta.native) meta.native = { ...meta.native, file: names[files.indexOf(meta.native.file)] ?? meta.native.file };
 		rmSync(pageDir, { recursive: true, force: true });
-		out.set(page, names);
+		out.set(page, { files: names, meta });
 	}
 	return out;
 }

package/lib/doc-to-md-core.ts

@@ -12,8 +12,8 @@ import { type ChildProcess, spawn } from "node:child_process";
 import { homedir, tmpdir } from "node:os";
 import { basename, dirname, extname, isAbsolute, join, resolve } from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
-import { type Bundle, abortBundle, commitBundle, openBundle, publishAttachments, publishSidecars, publishWords, writePageStats, publishPageImages, publishSheetCsvs, publishSheetImages, publishStaged, rewriteLinks, tempBundleRoot, validateImageLinks } from "./doc-to-md-bundle.ts";
-import { type Engine, type HandleData, type InfoData, type OcrInfo, type PageStat, type Tier, formatHandle, formatInfoHandle, scanOutline, type SheetInfo, type TocEntry } from "./doc-to-md-handle.ts";
+import { type StagedPage, type Bundle, abortBundle, commitBundle, openBundle, publishAttachments, publishSidecars, publishWords, writePageStats, publishPageImages, publishSheetCsvs, publishSheetImages, publishStaged, rewriteLinks, tempBundleRoot, validateImageLinks } from "./doc-to-md-bundle.ts";
+import { type NativeImage, type Engine, type HandleData, type InfoData, type OcrInfo, type PageStat, type Tier, formatHandle, formatInfoHandle, scanOutline, type SheetInfo, type TocEntry } from "./doc-to-md-handle.ts";
 import { type DocToMdOptions, type InputType, IMAGE_EXTS, TUNABLE_DEFAULTS, UsageError, classifyInput, sanitizeStem } from "./doc-to-md-options.ts";
 
 export * from "./doc-to-md-options.ts";
@@ -467,7 +467,7 @@ const clearStaging = (b: Pick<Bundle, "stagingDir">) => { for (const f of readdi
 export const EXCEL_REMEDY = "Remedy: install uv, or pip install openpyxl xlrd pillow";
 
 export type Mode = "html" | "image" | "info" | "pdf-primary" | "pdf-fallback" | "xlsx" | "pdf-text" | "render-pages" | "docx" | "email" | "ocr-pages";
-export interface TierJson { words?: boolean; wordsErrors?: Record<string, string>; pageStats?: PageStat[]; status?: string; written?: number[]; noText?: number[]; ocrFailed?: number[]; ocrErrors?: Record<string, string>; budgetStopped?: number[]; pageImages?: { page: number; file: string }[]; ocr?: OcrInfo; markdown?: string; pages?: number[]; pageCount?: number; emptyPages?: number[]; failedPages?: { page: number; error: string }[]; notes?: string[]; images?: { sheetIndex: number; file: string }[]; metadata?: Record<string, string>; toc?: [number, string, number | null][]; explicitBreaks?: number; engine?: string; degraded?: boolean; fallbackReason?: string | null; sheets?: SheetInfo[]; renderPages?: number[]; sheetCount?: number; ok?: boolean; reason?: string; rendered?: { idx: number; file: string; dpi: number }[]; failed?: { idx: number; reason: string }[]; }
+export interface TierJson { words?: boolean; wordsErrors?: Record<string, string>; pageStats?: PageStat[]; status?: string; written?: number[]; noText?: number[]; ocrFailed?: number[]; ocrErrors?: Record<string, string>; budgetStopped?: number[]; pageImages?: { page: number; file: string; dpi?: number; requestedDpi?: number }[]; nativeImages?: NativeImage[]; ocr?: OcrInfo; markdown?: string; pages?: number[]; pageCount?: number; emptyPages?: number[]; failedPages?: { page: number; error: string }[]; notes?: string[]; images?: { sheetIndex: number; file: string }[]; metadata?: Record<string, string>; toc?: [number, string, number | null][]; explicitBreaks?: number; engine?: string; degraded?: boolean; fallbackReason?: string | null; sheets?: SheetInfo[]; renderPages?: number[]; sheetCount?: number; ok?: boolean; reason?: string; rendered?: { idx: number; file: string; dpi: number }[]; failed?: { idx: number; reason: string }[]; }
 export type TierResult = { ok: true; json: TierJson } | { ok: false; reason: string; detail?: string } | { ok: false; userError: string; pageCount?: number };
 
 export interface PipelineSeams {
@@ -586,6 +586,25 @@ function handleOcr(tier: Tier, type: InputType, o: DocToMdOptions, json: TierJso
 	return { ...emptyOcr(o.ocrLanguage), ...x };
 }
 
+const nativeFromChild = (b: Bundle, json: TierJson, notes: string[]): NativeImage[] => (json.nativeImages ?? []).flatMap((e) => {
+	const file = b.sourceMap.get(`p${e.page}/${e.file}`);
+	if (!file) {
+		notes.push(`Native image p${e.page}/${e.file} not published`);
+		return [];
+	}
+	return [{ ...e, file: join(b.root, file) }];
+});
+
+// A dead primary has no JSON response; retained pages carry their image and clamp facts in .done.
+function retainedNative(b: Bundle, kept: Map<number, StagedPage>, notes: string[]): NativeImage[] {
+	const out: NativeImage[] = [];
+	for (const [page, k] of kept) {
+		if (k.meta.native) out.push({ page, ...k.meta.native, file: join(b.imagesDir, k.meta.native.file) });
+		if (k.meta.dpi !== undefined && k.meta.requestedDpi !== undefined && k.meta.dpi < k.meta.requestedDpi) notes.push(`Page ${page} rendered at ${k.meta.dpi} dpi (requested ${k.meta.requestedDpi}; 50 Mpx ceiling)`);
+	}
+	return out;
+}
+
 export async function convertDocument(o: DocToMdOptions, signal?: AbortSignal, seams?: Partial<PipelineSeams>): Promise<ConvertOutcome> {
 	const s: PipelineSeams = { backend: (c) => getBackend(c, undefined, signal), runTier: runTierReal, office: tryConvertOffice, ...seams };
 	const inputPath = resolve(o.path);
@@ -612,10 +631,11 @@ export async function convertDocument(o: DocToMdOptions, signal?: AbortSignal, s
 	let office: { pdfPath: string; cleanup: () => void } | null = null;
 	try {
 		let pdfPath = inputPath;
-		const base = { path: inputPath, pages: o.pages, ...(o.words && (type === "pdf" || type === "image") ? { words: true } : {}), stagingDir: b.stagingDir, sheetsStagingDir: b.sheetsStagingDir, pageImages: o.pageImages, pagesStagingDir: b.pagesStagingDir, imageDpi: o.imageDpi, imageFormat: o.imageFormat, maxOutputBytes: o.maxOutputBytes, pymupdfVersion: o.pymupdfVersion, ocr: o.ocr && !forced, ocrLanguage: o.ocrLanguage, ocrBudgetMs: o.primaryTimeoutMs };
+		const base = { path: inputPath, pages: o.pages, ...(o.words && (type === "pdf" || type === "image") ? { words: true } : {}), stagingDir: b.stagingDir, sheetsStagingDir: b.sheetsStagingDir, pageImages: o.pageImages, pagesStagingDir: b.pagesStagingDir, imageDpi: o.imageDpi, imageFormat: o.imageFormat, maxOutputBytes: o.maxOutputBytes, pymupdfVersion: o.pymupdfVersion, ocr: o.ocr && !forced, ocrLanguage: o.ocrLanguage, ocrBudgetMs: o.primaryTimeoutMs, hideAnnotations: o.hideAnnotations };
 		let tier: Tier | undefined, engine: Engine | undefined, json: TierJson | undefined, degraded: string | null = null, fallbackReason: string | null = null;
 		let explicitBreaks: number | null = null;
 		let notes: string[] = [];
+		let nativeImages: NativeImage[] = [];
 		let officeRoute: string | null = null;
 		let copyReason: string | null = null;
 		if (type === "html") {
@@ -728,17 +748,18 @@ export async function convertDocument(o: DocToMdOptions, signal?: AbortSignal, s
 			} else {
 				const p = await s.runTier("pdf-primary", pdfBase, b, signal, o.primaryTimeoutMs, backend);
 				const kept = publishStaged(b);
-				if (p.ok) { tier = "primary"; engine = "pymupdf4llm"; json = p.json; if (json.pageImages?.length) publishPageImages(b, json.pageCount ?? 0); }
+				if (p.ok) { tier = "primary"; engine = "pymupdf4llm"; json = p.json; nativeImages = nativeFromChild(b, json, notes); if (json.pageImages?.length) publishPageImages(b, json.pageCount ?? 0); }
 				else if ("userError" in p) throw new Error(p.userError);
 				else {
 					if (signal?.aborted) throw new Error("aborted");
-					const keepPages = Object.fromEntries([...kept.entries()].map(([k, v]) => [String(k), v]));
+					const keepPages = Object.fromEntries([...kept.entries()].map(([k, v]) => [String(k), v.files]));
 					rmSync(b.pagesStagingDir, { recursive: true, force: true });
 					const f = await s.runTier("pdf-fallback", { ...pdfBase, keepPages }, b, signal, o.fallbackTimeoutMs, backend);
 					publishStaged(b);
 					if (f.ok && f.json.pageImages?.length) publishPageImages(b, f.json.pageCount ?? 0);
 					if (!f.ok) throw new Error("userError" in f ? f.userError : `Conversion failed: primary ${p.reason}; fallback ${f.reason}${detailSuffix(f)}`);
 					tier = "fallback"; engine = "pymupdf-text"; json = f.json; degraded = DEGRADED_TEXT; fallbackReason = `primary ${p.reason}`;
+					nativeImages = [...retainedNative(b, kept, notes), ...nativeFromChild(b, json, notes)].sort((x, y) => x.page - y.page);
 				}
 			}
 		}
@@ -789,7 +810,7 @@ export async function convertDocument(o: DocToMdOptions, signal?: AbortSignal, s
 		const markdown = (head.length ? `${head.join("\n")}\n\n` : "") + body;
 		commitBundle(b, markdown);
 		const outline = scanOutline(markdown, o.outlineMaxEntries);
-		const details: DocToMdDetails = { path: inputPath, backend: backend.kind, pymupdfVersion: o.pymupdfVersion, inputType: type, file: b.mdPath, outputDir: b.root, savedTo: b.mdPath, imagesDir: b.imagesDir, sheetsDir: b.csvManifest.size ? b.sheetsDir : null, pagesDir: b.pageManifest.size ? b.pagesDir : null, pageImageCount: b.pageManifest.size, pageImagesReason, type, engine, tier, pageCount: json.pageCount ?? null, pages: o.pages, explicitBreaks, imageCount: b.manifest.size, bytes: Buffer.byteLength(markdown, "utf8"), lines: markdown.split("\n").length, degraded, fallbackReason, failedPages: (json.failedPages ?? []).map((f) => f.page), emptyPages: json.emptyPages ?? [], notes, outline: outline.entries, outlineTotal: outline.total, ocr, pageStats, pageStatsPath: pageStats ? b.pageStatsPath : null, ocrDir: b.ocrManifest.size ? b.ocrDir : null, wordsPath, wordsReason, wordsErrors };
+		const details: DocToMdDetails = { path: inputPath, backend: backend.kind, pymupdfVersion: o.pymupdfVersion, inputType: type, file: b.mdPath, outputDir: b.root, savedTo: b.mdPath, imagesDir: b.imagesDir, sheetsDir: b.csvManifest.size ? b.sheetsDir : null, pagesDir: b.pageManifest.size ? b.pagesDir : null, pageImageCount: b.pageManifest.size, pageImagesReason, type, engine, tier, pageCount: json.pageCount ?? null, pages: o.pages, explicitBreaks, imageCount: b.manifest.size, bytes: Buffer.byteLength(markdown, "utf8"), lines: markdown.split("\n").length, degraded, fallbackReason, failedPages: (json.failedPages ?? []).map((f) => f.page), emptyPages: json.emptyPages ?? [], notes, outline: outline.entries, outlineTotal: outline.total, ocr, pageStats, pageStatsPath: pageStats ? b.pageStatsPath : null, ocrDir: b.ocrManifest.size ? b.ocrDir : null, nativeImages, wordsPath, wordsReason, wordsErrors };
 		return { output: formatHandle(details), details };
 	} catch (e) { abortBundle(b); throw e; }
 	finally { office?.cleanup(); }

package/lib/doc-to-md-handle.ts

@@ -39,12 +39,15 @@ export interface OcrInfo {
 	childError: string | null;
 }
 
+export interface NativeImage { page: number; file: string; width: number; height: number }
+
 export interface HandleData {
 	savedTo: string; imagesDir: string | null; sheetsDir: string | null; pagesDir: string | null; type: InputType; engine: Engine; tier: Tier;
 	pageCount: number | null; pages: number[] | null; explicitBreaks: number | null; imageCount: number; pageImageCount: number; pageImagesReason: string | null; bytes: number; lines: number;
 	degraded: string | null; fallbackReason: string | null; failedPages: number[]; emptyPages: number[];
 	notes: string[]; outline: OutlineEntry[]; outlineTotal: number; ocr: OcrInfo | null;
 	pageStats: PageStat[] | null; pageStatsPath: string | null; ocrDir: string | null;
+	nativeImages: NativeImage[];
 	wordsPath: string | null; wordsReason: string | null; wordsErrors: Record<number, string>;
 }
 
@@ -171,6 +174,7 @@ export function formatHandle(h: HandleData): string {
 	if (h.pagesDir && h.pageImageCount > 0) lines.push(`Pages-Dir: ${h.pagesDir} (${h.pageImageCount} pages)`);
 	else if (h.pageImagesReason) lines.push(`Pages-Dir: none - ${h.pageImagesReason}`);
 	if (h.pageStatsPath) lines.push(`Page-Stats: ${h.pageStatsPath}`);
+	if (h.nativeImages.length) lines.push(`Native-Images: ${h.nativeImages.length === 1 ? "page" : "pages"} ${compactRanges(h.nativeImages.map((n) => n.page))} (embedded image streams, no render DPI)`);
 	if (h.wordsPath) {
 		const bad = Object.keys(h.wordsErrors).map(Number).sort((a, b) => a - b);
 		lines.push(`Words: ${h.wordsPath}${bad.length ? ` (extraction failed for pages ${compactRanges(bad)}: ${h.wordsErrors[bad[0]]})` : ""}`);

package/lib/doc-to-md-options.ts

@@ -21,6 +21,7 @@ export interface Tunables {
 	outlineMaxEntries: number;
 	ocr: boolean;
 	ocrLanguage: string;
+	hideAnnotations: boolean;
 }
 
 export interface DocToMdOptions extends Tunables {
@@ -87,6 +88,7 @@ export const DOC_TO_MD_OPTIONS: readonly OptionDescriptor[] = [
 	{ key: "outlineMaxEntries", flag: "--outline-max-entries", type: "int", default: 40, settable: true, help: "Heading outline / TOC / sheet inventory cap in the handle" },
 	{ key: "ocr", flag: "--ocr", type: "bool", default: false, settable: true, help: "Run OCR on pages without a text layer and on image inputs when Tesseract language data is installed; off by default (--no-ocr turns a settings-level true off)" },
 	{ key: "ocrLanguage", flag: "--ocr-language", type: "lang", default: "eng", settable: true, help: "Tesseract language code(s), +-joined, e.g. deu+eng" },
+	{ key: "hideAnnotations", flag: "--hide-annotations", type: "bool", default: false, settable: true, help: "Render PDF pages without annotations (sticky notes, highlights, stamps - and form-field widgets, so filled form values disappear); default paints them, as PyMuPDF does. Applies to pages/ renders and textless-page renders, not to OCR text or embedded images; also lets an annotated scan be delivered as its embedded image." },
 ];
 
 export const TUNABLE_DEFAULTS: Tunables = Object.fromEntries(

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-quiver",
-  "version": "6.10.0",
+  "version": "6.11.0",
   "description": "Personal pack of Pi coding-agent extensions: context-safe fetch, doc_to_md PDF/DOCX/PPTX-to-Markdown conversion, session naming, a themed ASCII startup header, Opus 4.8 fast mode, and a provider-stall watchdog.",
   "author": "Jacek Juraszek",
   "license": "MIT",

package/scripts/doc_to_md.py

@@ -90,8 +90,9 @@ def page_dir(staging, n):
     return d
 
 
-def mark_done(d):
-    open(os.path.join(d, ".done"), "w").close()
+def mark_done(d, meta=None):
+    with open(os.path.join(d, ".done"), "w", encoding="utf-8") as fh:
+        json.dump(meta or {}, fh)
 
 
 OCR_SENTINEL = "\x00OCR {}\x00"
@@ -150,16 +151,232 @@ def clamped_dpi(w, h, dpi):
     return eff if eff >= MIN_RENDER_DPI else None
 
 
-def render_textless_page(page, d, o):
+RASTER_BUDGET_S, OCR_JOB_BUDGET_S, OCR_DPI = 20, 30, 300
+RASTER_SPAWN_BUDGET_S = 30
+MIN_OCR_JOB_S = 1
+NATIVE_MIN_COVERAGE, NATIVE_MAX_COVERAGE = 0.9, 1.1
+NATIVE_EXTS = ("png", "jpeg", "jpg")
+JOB_VERB = {"native": "render", "render": "render", "images": "render", "ocr": "OCR"}
+RASTER_INLINE = False  # tests only: run raster jobs in-process so monkeypatches reach them
+
+
+class RasterError(Exception):
+    """A raster job did not complete; str(exc) is the failedPages message."""
+
+
+class RasterUnavailable(Exception):
+    pass
+
+
+def raster_budget(default):
+    return float(os.environ.get("DOC_TO_MD_RASTER_BUDGET_S", default))
+
+
+def clamp_note(n, eff, requested):
+    return f"Page {n} rendered at {eff} dpi (requested {requested}; {MAX_RENDER_PX // 1_000_000} Mpx ceiling)"
+
+
+def native_page_image(doc, page, target_dir, allow_annots=False):
+    """One full-page, unrotated, unmasked PNG/JPEG stream with no vector overlay -> written untouched as page.<ext>."""
+    import pymupdf
+    if not allow_annots and (page.first_annot is not None or page.first_widget is not None):
+        return None
+    infos = page.get_image_info(xrefs=True)
+    if len(infos) != 1 or infos[0].get("xref", 0) <= 0 or page.rotation != 0:
+        return None
+    drawings = page.get_drawings()
+    if allow_annots:
+        # MuPDF includes appearance streams, clipped to annotation/widget rectangles.
+        annot_rects = [a.rect + (-1, -1, 1, 1) for items in (page.annots(), page.widgets()) for a in items]
+        drawings = [drawing for drawing in drawings if not any(drawing["rect"] in rect for rect in annot_rects)]
+    if drawings:
+        return None
+    info = infos[0]
+    a, b, c, d = info["transform"][:4]
+    if b != 0 or c != 0 or a <= 0 or d <= 0:
+        return None
+    area, bbox = page.rect.get_area(), pymupdf.Rect(info["bbox"])
+    if area <= 0 or (bbox & page.rect).get_area() / area < NATIVE_MIN_COVERAGE or bbox.get_area() / area > NATIVE_MAX_COVERAGE:
+        return None
+    img = doc.extract_image(info["xref"])
+    ext, cs = img["ext"].lower(), img.get("cs-name")
+    plain = cs in ("DeviceRGB", "DeviceGray") or (cs == "DeviceCMYK" and ext in ("jpeg", "jpg"))
+    if img.get("smask", 0) or ext not in NATIVE_EXTS or not img["image"] or not plain:
+        return None
+    name = f"page.{ext}"
+    with open(os.path.join(target_dir, name), "wb") as fh:
+        fh.write(img["image"])
+    return {"file": name, "width": img["width"], "height": img["height"]}
+
+
+def run_raster_job(doc, job):
+    import time
+    import pymupdf
+    n, kind = job["page"], job["kind"]
+    try:
+        if os.environ.get("DOC_TO_MD_RASTER_STALL_PAGE") in (str(n), f"{kind}:{n}"):  # tests only: a wedged page
+            time.sleep(3600)
+        if os.environ.get("DOC_TO_MD_RASTER_CRASH_PAGE") == str(n):  # tests only: partial output, stdout chatter, hard exit
+            if kind == "render":
+                with open(job["target"], "wb") as fh:
+                    fh.write(b"\x89PNG partial")
+            print("raster worker crash hook", flush=True)
+            os._exit(1)
+        page = doc[n - 1]
+        if kind == "native":
+            r = native_page_image(doc, page, job["target"], job.get("allowAnnots", False))
+            return {"ok": True, "eligible": r is not None, **(r or {})}
+        if kind == "render":
+            kw = {"dpi": job["dpi"], "annots": job["annots"]}
+            if job.get("clip") is not None:
+                kw["clip"] = pymupdf.Rect(job["clip"])
+            pix = page.get_pixmap(**kw)
+            pix.save(job["target"])
+            return {"ok": True, "width": pix.width, "height": pix.height}
+        if kind == "images":
+            files = []
+            for i, image in enumerate(page.get_image_info(xrefs=True), 1):
+                xref = image.get("xref", 0)
+                if xref > 0:
+                    img = doc.extract_image(xref)
+                    name = f"img{i}.{img['ext'].lower()}"
+                    with open(os.path.join(job["target"], name), "wb") as fh:
+                        fh.write(img["image"])
+                else:
+                    if job["dpi"] is None:
+                        continue
+                    name = f"img{i}.{job['format']}"
+                    pix = page.get_pixmap(clip=pymupdf.Rect(image["bbox"]), dpi=job["dpi"], annots=job["annots"])
+                    pix.save(os.path.join(job["target"], name))
+                files.append(name)
+            return {"ok": True, "files": files}
+        import pymupdf4llm
+        md = pymupdf4llm.to_markdown(doc, pages=[n - 1], write_images=False, page_separators=False, use_ocr=True, force_ocr=True,
+                                     ocr_language=job["lang"], ocr_dpi=job["ocrDpi"])
+        result = {"ok": True, "markdown": md}
+        if job.get("wordsSnapshot"):
+            # OCR mutates only the worker's document; carry that text layer back for parent-side word extraction.
+            try:
+                snapshot = pymupdf.open()
+                snapshot.insert_pdf(doc, from_page=n - 1, to_page=n - 1)
+                snapshot.save(job["wordsSnapshot"])
+                snapshot.close()
+            except Exception as exc:  # noqa: BLE001 - geometry never changes the OCR outcome
+                result["wordsSnapshotError"] = words_error(exc)
+        return result
+    except Exception as exc:  # noqa: BLE001 - the parent decides what a failed job costs
+        return {"ok": False, "error": f"{type(exc).__name__}: {exc}"[:300]}
+
+
+def redirect_worker_stdout():
+    # redirect_stdout in the parent is a Python-object swap the spawned process does not inherit; pymupdf4llm prints to stdout.
+    try:
+        os.dup2(sys.stderr.fileno(), sys.stdout.fileno())
+    except (AttributeError, OSError, ValueError):
+        sys.stdout = sys.stderr
+
+
+def raster_worker(conn, path):
+    redirect_worker_stdout()
+    import pymupdf
+    doc = pymupdf.open(path)
+    conn.send({"ready": True})
+    while True:
+        job = conn.recv()
+        if job["kind"] == "stop":
+            break
+        conn.send(run_raster_job(doc, job))
+    doc.close()
+
+
+class RasterWorker:
+    """One spawned PyMuPDF process per conversion. MuPDF spins at C level, so only an OS kill ends a toxic page; a job that overruns its budget or takes the process down costs that job, and the next job respawns."""
+
+    def __init__(self, path, doc):
+        self.path, self.doc, self.proc, self.conn = path, doc, None, None
+
+    def run(self, job, budget_s):
+        if RASTER_INLINE:
+            return run_raster_job(self.doc, job)
+        if self.proc is None:
+            try:
+                import multiprocessing
+                ctx = multiprocessing.get_context("spawn")
+                self.conn, child = ctx.Pipe()
+                self.proc = ctx.Process(target=raster_worker, args=(child, self.path))
+                try:
+                    self.proc.start()
+                finally:
+                    child.close()
+            except Exception as exc:
+                if self.conn is not None:
+                    self.conn.close()
+                self.proc = self.conn = None
+                raise RasterUnavailable(f"raster worker unavailable: {exc}") from exc
+            try:
+                ready = self.conn.poll(RASTER_SPAWN_BUDGET_S) and self.conn.recv() == {"ready": True}
+            except (EOFError, OSError):
+                ready = False
+            if not ready:
+                self.kill()
+                return {"ok": False, "error": "renderer failed to start"}
+        try:
+            self.conn.send(job)
+            if self.conn.poll(budget_s):
+                return self.conn.recv()
+            error = f"{JOB_VERB[job['kind']]} timed out after {budget_s:g}s"
+        except (EOFError, OSError):
+            error = "renderer crashed"
+        self.kill()
+        if job["kind"] == "render":
+            with contextlib.suppress(OSError):
+                os.remove(job["target"])
+        if job["kind"] == "ocr" and job.get("wordsSnapshot"):
+            with contextlib.suppress(OSError):
+                os.remove(job["wordsSnapshot"])
+        return {"ok": False, "error": error}
+
+    def kill(self):
+        if self.proc is not None:
+            if self.proc.is_alive():
+                self.proc.kill()
+            self.proc.join()
+            self.conn.close()
+            self.proc = self.conn = None
+
+    def stop(self):
+        if self.proc is None:
+            return
+        with contextlib.suppress(OSError):
+            self.conn.send({"kind": "stop"})
+        self.proc.join(2)
+        self.kill()
+
+
+def textless_picture(worker, page, n, d, o, notes, render):
+    """Native stream when the page is one full-page image, else (when render) a worker render at the clamped DPI. Returns (file name or None, .done metadata)."""
+    r = worker.run({"kind": "native", "page": n, "target": d, "allowAnnots": bool(o.get("hideAnnotations"))}, raster_budget(RASTER_BUDGET_S))
+    if r.get("ok") and r.get("eligible"):
+        return r["file"], {"native": {"file": r["file"], "width": r["width"], "height": r["height"]}}
+    for f in os.listdir(d):
+        if f.startswith("page."):  # a failed native job may have left a partial stream
+            os.remove(os.path.join(d, f))
+    if not render:
+        return None, {}
     eff = clamped_dpi(page.rect.width, page.rect.height, o["imageDpi"])
     if eff is None:
-        return None
+        return None, {}
     name = f"page.{o['imageFormat']}"
-    page.get_pixmap(dpi=eff).save(os.path.join(d, name))
-    return name
+    r = worker.run({"kind": "render", "page": n, "dpi": eff, "annots": not o.get("hideAnnotations"), "target": os.path.join(d, name)}, raster_budget(RASTER_BUDGET_S))
+    if not r["ok"]:
+        raise RasterError(r["error"])
+    if eff < o["imageDpi"]:
+        notes.append(clamp_note(n, eff, o["imageDpi"]))
+        return name, {"dpi": eff, "requestedDpi": o["imageDpi"]}
+    return name, {}
 
 
-def render_page_image(page, n, o, page_images):
+def render_page_image(worker, page, n, o, page_images, notes):
     d = o["pagesStagingDir"]
     eff = clamped_dpi(page.rect.width, page.rect.height, o["imageDpi"])
     if eff is None:
@@ -167,18 +384,24 @@ def render_page_image(page, n, o, page_images):
     os.makedirs(d, exist_ok=True)
     name = f"p{n}.{o['imageFormat']}"
     target = os.path.join(d, name)
-    try:
-        page.get_pixmap(dpi=eff).save(target)
-    except Exception:
-        try:
+    r = worker.run({"kind": "render", "page": n, "dpi": eff, "annots": not o.get("hideAnnotations"), "target": target}, raster_budget(RASTER_BUDGET_S))
+    if not r["ok"]:
+        notes.append(f"Page {n} render unavailable: {r['error']}")
+        with contextlib.suppress(OSError):
             os.remove(target)
-        except OSError:
-            pass
         return None
-    page_images.append({"page": n, "file": name})
+    entry = {"page": n, "file": name, "dpi": eff}
+    if eff < o["imageDpi"]:
+        entry["requestedDpi"] = o["imageDpi"]
+        notes.append(clamp_note(n, eff, o["imageDpi"]))
+    page_images.append(entry)
     return name
 
 
+def page_failure(n, exc):
+    return {"page": n, "error": str(exc) if isinstance(exc, RasterError) else f"{type(exc).__name__}: {exc}"[:300]}
+
+
 def page_ocr_kwargs(textless, lang):
     if textless:
         return {"use_ocr": True, "force_ocr": True, "ocr_language": lang}
@@ -359,72 +582,113 @@ def mode_pdf_primary(o):
     pages = check_pages(o.get("pages"), doc.page_count)
     staging, out, empty, failed, notes = o["stagingDir"], [], [], [], []
     page_images = []
+    native_images = []
+    worker = RasterWorker(o["path"], doc)
     stats = []
     lang, budget = o.get("ocrLanguage", "eng"), o.get("ocrBudgetMs", 60000)
     info = new_ocr(lang)
     status = ocr_status(True, lang) if o.get("ocr") else None
     ocr_ms, plain_ms = [], []
     word_pages, words_errors = [], {}
-    for i, n in enumerate(pages):
-        stats.append(page_stats(doc, n))
-        d = page_dir(staging, n)
-        try:
-            page = doc[n - 1]
-            rotation = page.rotation
-            textless = not page.get_text("text").strip()
-            if textless:
-                info["textless"].append(n)
-                if status is None:
-                    status = ocr_status(False, lang)
-            kw = page_ocr_kwargs(textless, lang) if status and status["status"] == "ready" else {"use_ocr": False}
-            if kw["use_ocr"]:
-                elapsed = (time.monotonic() - start) * 1000
-                est_ocr = max(ocr_ms) if ocr_ms else OCR_EST_INITIAL_MS
-                est_page = sum(plain_ms) / len(plain_ms) if plain_ms else PAGE_EST_INITIAL_MS
-                if not ocr_admit(elapsed, est_ocr, len(pages) - i - 1, est_page, budget):
-                    info["budgetStopped"].append(n)
-                    kw = {"use_ocr": False}
-            t0 = time.monotonic()
+    try:
+        for i, n in enumerate(pages):
+            stats.append(page_stats(doc, n))
+            d = page_dir(staging, n)
             try:
-                md = primary_page_markdown(doc, n, d, o, kw, not textless) if kw["use_ocr"] or not textless else ""
-            except Exception:  # noqa: BLE001
-                if not kw["use_ocr"]:
-                    raise
-                kw, t0, md = {"use_ocr": False}, time.monotonic(), ""
-                info["ocrFailed"].append(n)
-            (ocr_ms if kw["use_ocr"] else plain_ms).append((time.monotonic() - t0) * 1000)
-            if textless:
-                pic = None if o.get("pageImages") else render_textless_page(page, d, o)
-                text = md.strip()
-                if kw["use_ocr"] and text:
-                    info["pages"].append(n)
-                else:
+                page = doc[n - 1]
+                rotation = page.rotation
+                textless = not page.get_text("text").strip()
+                if textless:
+                    info["textless"].append(n)
+                    if status is None:
+                        status = ocr_status(False, lang)
+                kw = page_ocr_kwargs(textless, lang) if status and status["status"] == "ready" else {"use_ocr": False}
+                if kw["use_ocr"]:
+                    elapsed = (time.monotonic() - start) * 1000
+                    est_ocr = max(ocr_ms) if ocr_ms else OCR_EST_INITIAL_MS
+                    est_page = sum(plain_ms) / len(plain_ms) if plain_ms else PAGE_EST_INITIAL_MS
+                    if not ocr_admit(elapsed, est_ocr, len(pages) - i - 1, est_page, budget):
+                        info["budgetStopped"].append(n)
+                        kw = {"use_ocr": False}
+                text, meta = "", {}
+                ocr_words = None
+                if textless:
+                    pic, meta = textless_picture(worker, page, n, d, o, notes, render=not o.get("pageImages"))
+                    t0 = time.monotonic()
                     if kw["use_ocr"]:
-                        info["noText"].append(n)
+                        remaining = (budget - (time.monotonic() - start) * 1000 - OCR_BUDGET_RESERVE_MS) / 1000
+                        if remaining < MIN_OCR_JOB_S:
+                            info["budgetStopped"].append(n)
+                            kw = {"use_ocr": False}
+                        else:
+                            ocr_dpi = clamped_dpi(page.rect.width, page.rect.height, OCR_DPI)
+                            words_snapshot = os.path.join(staging, f".ocr-words-p{n}.pdf") if o.get("words") else None
+                            try:
+                                r = worker.run({"kind": "ocr", "page": n, "lang": lang, "ocrDpi": ocr_dpi, **({"wordsSnapshot": words_snapshot} if words_snapshot else {})}, min(raster_budget(OCR_JOB_BUDGET_S), remaining)) if ocr_dpi is not None else {"ok": False}
+                                if words_snapshot and r["ok"]:
+                                    if r.get("wordsSnapshotError"):
+                                        words_errors[str(n)] = r["wordsSnapshotError"]
+                                    else:
+                                        try:
+                                            with open_pdf(words_snapshot) as snapshot:
+                                                ocr_words = words_page(snapshot[0], n, rotation, page_words(snapshot[0], True))
+                                        except Exception as exc:  # noqa: BLE001
+                                            words_errors[str(n)] = words_error(exc)
+                            finally:
+                                if words_snapshot:
+                                    with contextlib.suppress(OSError):
+                                        os.remove(words_snapshot)
+                            if r["ok"]:
+                                text = r["markdown"].strip()
+                            else:
+                                kw = {"use_ocr": False}
+                                info["ocrFailed"].append(n)
+                    md = ""
+                else:
+                    t0 = time.monotonic()
+                    md = primary_page_markdown(doc, n, d, o, kw, True)
+                if not (textless and not kw["use_ocr"]):
+                    (ocr_ms if kw["use_ocr"] else plain_ms).append((time.monotonic() - t0) * 1000)
+                if textless:
+                    if kw["use_ocr"] and text:
+                        info["pages"].append(n)
+                    else:
+                        if kw["use_ocr"]:
+                            info["noText"].append(n)
+                        empty.append(n)
+                elif not md.strip():
                     empty.append(n)
-            elif not md.strip():
+                mark_done(d, meta)
+                page_pic = render_page_image(worker, page, n, o, page_images, notes) if o.get("pageImages") else None
+                if textless:
+                    parts = [f"![page {n}](p{n}/{pic})"] if pic else []
+                    if kw["use_ocr"] and text:
+                        parts.append(ocr_block(f"p{n}/{pic}" if pic else f"pages/{page_pic}" if page_pic else "-", text))
+                    md = "\n\n".join(parts)
+                if page_pic:
+                    md = "\n\n".join(x for x in [md.rstrip(), f"![page {n}](pages/{page_pic})"] if x)
+                out.append(md.rstrip())
+                if o.get("words") and not (textless and o.get("ocr") and not kw["use_ocr"]):
+                    try:
+                        if textless and kw["use_ocr"]:
+                            if ocr_words is not None:
+                                word_pages.append(ocr_words)
+                        else:
+                            word_pages.append(words_page(page, n, rotation, page_words(page, kw["use_ocr"])))
+                    except Exception as exc:  # noqa: BLE001
+                        words_errors[str(n)] = words_error(exc)
+                if meta.get("native"):
+                    native_images.append({"page": n, **meta["native"]})
+            except RasterUnavailable:
+                raise
+            except Exception as exc:  # noqa: BLE001
+                shutil.rmtree(d, ignore_errors=True)
+                failed.append(page_failure(n, exc))
                 empty.append(n)
-            mark_done(d)
-            page_pic = render_page_image(page, n, o, page_images) if o.get("pageImages") else None
-            if textless:
-                parts = [f"![page {n}](p{n}/{pic})"] if pic else []
-                if kw["use_ocr"] and text:
-                    parts.append(ocr_block(f"pages/{page_pic}" if page_pic else f"p{n}/{pic}" if pic else "-", text))
-                md = "\n\n".join(parts)
-            if page_pic:
-                md = "\n\n".join(x for x in [md.rstrip(), f"![page {n}](pages/{page_pic})"] if x)
-            out.append(md.rstrip())
-            if o.get("words") and not (textless and o.get("ocr") and not kw["use_ocr"]):
-                try:
-                    word_pages.append(words_page(page, n, rotation, page_words(page, kw["use_ocr"])))
-                except Exception as exc:  # noqa: BLE001
-                    words_errors[str(n)] = words_error(exc)
-        except Exception as exc:  # noqa: BLE001
-            shutil.rmtree(d, ignore_errors=True)
-            failed.append({"page": n, "error": f"{type(exc).__name__}: {exc}"[:300]})
-            empty.append(n)
-            out.append("")
-        out.append(SEP.format(n=n).strip("\n"))
+                out.append("")
+            out.append(SEP.format(n=n).strip("\n"))
+    finally:
+        worker.stop()
     if failed and len(failed) == len(pages):
         raise RuntimeError("every selected page failed: " + failed[0]["error"])
     if status is not None:
@@ -433,7 +697,7 @@ def mode_pdf_primary(o):
     if missing:
         notes.append(f"Page images: {missing} of {len(pages)} unavailable")
     result = {"markdown": "\n\n".join(out) + "\n", "pages": pages, "pageCount": doc.page_count,
-              "emptyPages": empty, "failedPages": failed, "notes": notes, "ocr": info, "pageImages": page_images, "pageStats": stats}
+              "emptyPages": empty, "failedPages": failed, "notes": notes, "ocr": info, "pageImages": page_images, "nativeImages": native_images, "pageStats": stats}
     if o.get("words"):
         result.update(write_words(staging, "pt", word_pages, words_errors))
     return result
@@ -446,72 +710,75 @@ def mode_pdf_fallback(o):
     keep = {int(k): v for k, v in (o.get("keepPages") or {}).items()}
     staging, out, empty, failed = o["stagingDir"], [], [], []
     page_images = []
+    native_images = []
+    worker = RasterWorker(o["path"], doc)
     stats = []
     lang = o.get("ocrLanguage", "eng")
     ocr_info = new_ocr(lang)
     word_pages, words_errors = [], {}
     status = {"status": "unavailable", "reason": "fallback tier", "tesseract": None} if o.get("ocr") else None
-    for n in pages:
-        stats.append(page_stats(doc, n))
-        links = [f"![](images/{f})" for f in keep.get(n, [])]
-        text = ""
-        page_pic = None
-        page_ok = False
-        try:
-            page = doc[n - 1]
-            rotation = page.rotation
-            text = page.get_text("text").strip()
-            if not text:
-                ocr_info["textless"].append(n)
-                if status is None:
-                    status = ocr_status(False, lang)
-            if n not in keep:
-                d = page_dir(staging, n)
-                if not text and not o.get("pageImages"):
-                    pic = render_textless_page(page, d, o)
-                    if pic:
-                        links.append(f"![page {n}](p{n}/{pic})")
-                elif text:
-                    i = 0
-                    for image in page.get_image_info(xrefs=True):
-                        i += 1
-                        xref = image.get("xref", 0)
-                        if xref > 0:
-                            img = doc.extract_image(xref)
-                            name = f"img{i}.{img['ext'].lower()}"
-                            with open(os.path.join(d, name), "wb") as fh:
-                                fh.write(img["image"])
-                        else:
-                            name = f"img{i}.{o['imageFormat']}"
-                            page.get_pixmap(clip=pymupdf.Rect(image["bbox"]), dpi=o["imageDpi"]).save(os.path.join(d, name))
-                        links.append(f"![](p{n}/{name})")
-                mark_done(d)
-            page_pic = render_page_image(page, n, o, page_images) if o.get("pageImages") else None
-            page_ok = True
-        except Exception as exc:  # noqa: BLE001
-            shutil.rmtree(os.path.join(staging, f"p{n}"), ignore_errors=True)
-            text = ""
+    notes = [DEGRADED_NOTE]
+    try:
+        for n in pages:
+            stats.append(page_stats(doc, n))
             links = [f"![](images/{f})" for f in keep.get(n, [])]
-            failed.append({"page": n, "error": f"{type(exc).__name__}: {exc}"[:300]})
-        if o.get("words") and page_ok and not (not text and o.get("ocr")):
+            text = ""
+            page_pic = None
+            meta = {}
+            page_ok = False
             try:
-                word_pages.append(words_page(page, n, rotation, page_words(page, False)))
+                page = doc[n - 1]
+                rotation = page.rotation
+                text = page.get_text("text").strip()
+                if not text:
+                    ocr_info["textless"].append(n)
+                    if status is None:
+                        status = ocr_status(False, lang)
+                if n not in keep:
+                    d = page_dir(staging, n)
+                    if not text:
+                        pic, meta = textless_picture(worker, page, n, d, o, notes, render=not o.get("pageImages"))
+                        if pic:
+                            links.append(f"![page {n}](p{n}/{pic})")
+                    else:
+                        eff = clamped_dpi(page.rect.width, page.rect.height, o["imageDpi"])
+                        r = worker.run({"kind": "images", "page": n, "dpi": eff, "annots": not o.get("hideAnnotations"), "format": o["imageFormat"], "target": d}, raster_budget(RASTER_BUDGET_S))
+                        if not r["ok"]:
+                            raise RasterError(r["error"])
+                        links.extend(f"![](p{n}/{name})" for name in r["files"])
+                    mark_done(d, meta)
+                page_pic = render_page_image(worker, page, n, o, page_images, notes) if o.get("pageImages") else None
+                page_ok = True
+            except RasterUnavailable:
+                raise
             except Exception as exc:  # noqa: BLE001
-                words_errors[str(n)] = words_error(exc)
-        if not text:
-            empty.append(n)
-        out.append("\n\n".join(x for x in [text, "\n".join(links), f"![page {n}](pages/{page_pic})" if page_pic else ""] if x))
-        out.append(SEP.format(n=n).strip("\n"))
+                shutil.rmtree(os.path.join(staging, f"p{n}"), ignore_errors=True)
+                text = ""
+                meta = {}
+                links = [f"![](images/{f})" for f in keep.get(n, [])]
+                failed.append(page_failure(n, exc))
+            if o.get("words") and page_ok and not (not text and o.get("ocr")):
+                try:
+                    word_pages.append(words_page(page, n, rotation, page_words(page, False)))
+                except Exception as exc:  # noqa: BLE001
+                    words_errors[str(n)] = words_error(exc)
+            if not text:
+                empty.append(n)
+            out.append("\n\n".join(x for x in [text, "\n".join(links), f"![page {n}](pages/{page_pic})" if page_pic else ""] if x))
+            if meta.get("native"):
+                native_images.append({"page": n, **meta["native"]})
+            out.append(SEP.format(n=n).strip("\n"))
+    finally:
+        worker.stop()
     if failed and len(failed) == len(pages):
         raise RuntimeError("every selected page failed: " + failed[0]["error"])
     if status is not None:
         apply_status(ocr_info, status)
-    notes = [DEGRADED_NOTE]
     missing = len(pages) - len(page_images) if o.get("pageImages") else 0
     if missing:
         notes.append(f"Page images: {missing} of {len(pages)} unavailable")
     result = {"markdown": "\n\n".join(out) + "\n", "pages": pages, "pageCount": doc.page_count,
-              "emptyPages": empty, "failedPages": failed, "notes": notes, "ocr": ocr_info, "pageImages": page_images, "pageStats": stats}
+              "emptyPages": empty, "failedPages": failed, "notes": notes, "ocr": ocr_info, "pageImages": page_images, "nativeImages": native_images, "pageStats": stats}
     if o.get("words"):
         result.update(write_words(staging, "pt", word_pages, words_errors))
     return result
@@ -614,7 +881,7 @@ def col_letter(i):
 
 PREVIEW_ROWS, PREVIEW_COLS = 100, 50
 PROFILE_MAJORITY, DISTINCT_CAP, SLUG_MAX = 0.6, 50, 40
-MAX_RENDER_PX, MIN_RENDER_DPI, MIN_PAGE_PT = 16_000_000, 36, 72
+MAX_RENDER_PX, MIN_RENDER_DPI, MIN_PAGE_PT = 50_000_000, 36, 72
 INV_HEADER = "| # | name | kind | size | hidden | charts | images | rendered | data |\n|---|---|---|---|---|---|---|---|---|\n"
 XLS_NOTE = "Rendered views: unavailable (visual detection not supported for .xls)"
 
@@ -1424,7 +1691,6 @@ def mode_info_excel(o):
     return {"sheets": sheets}
 
 def mode_render_pages(o):
-    import math
     import pymupdf
     doc = pymupdf.open(o["path"])
     expected = o["expectedPages"]
@@ -1439,8 +1705,8 @@ def mode_render_pages(o):
             if w < MIN_PAGE_PT or h < MIN_PAGE_PT:
                 failed.append({"idx": idx, "reason": f"rendered view degenerate (page {w:.0f} x {h:.0f} pt)"})
                 continue
-            eff = min(dpi, math.floor(math.sqrt(MAX_RENDER_PX / (w * h / 72 ** 2))))
-            if eff < MIN_RENDER_DPI:
+            eff = clamped_dpi(w, h, dpi)
+            if eff is None:
                 failed.append({"idx": idx, "reason": f"rendered view too large (page {w:.0f} x {h:.0f} pt)"})
                 continue
             name = f"s{idx}.{fmt}"