dsh-office-kit 0.2.0 → 0.3.1

package/assets/skills/office-pdf/references/limits.md

@@ -0,0 +1,58 @@
+# When a PDF job is beyond this pack
+
+Every entry names the missing capability, what you can still deliver, and how the user resolves it. Never answer with a bare "unsupported": say what is missing, what the floor is, and give the next step.
+
+## Scanned pages with no text layer
+
+The words exist only as pixels; extraction and search cannot see them. `render_pages.py` reports `chars: 0` for such a page, and the render shows the text as an image.
+
+- Still possible: render, crop with `read_image`, and transcribe the visible text; redact the image pixels with `--images pixels`; rebuild the page as a created PDF from the transcription.
+- Resolution: the user supplies a text-based copy of the file, or runs OCR where they have it (this runtime has no OCR engine, so do not promise the words will be extracted here).
+
+## Encrypted or password-protected files
+
+pypdf and PyMuPDF can open with a password only if it is provided; every script here exits with "encrypted" and does no work.
+
+- Resolution: ask the user to decrypt the file first, then re-run. Never write a password into a command line that will be logged.
+
+## Signing and certificates
+
+Creating a digital signature needs a certificate, a signer, and a signature field; the runtime has none of them. Editing a signed document invalidates its signature.
+
+- Still possible: fill unsigned fields, rebuild the content, deliver an unsigned file — and say clearly that the signature is gone.
+- Resolution: signing must happen in a tool with a certificate; the user applies it after the content is final.
+
+## Paragraph-level re-editing
+
+PDF has no paragraph model: the content stream is positioned glyphs and paths. "Rewrite this paragraph" becomes a block-level operation — redact the old block, stamp the new text, and check the reflow yourself. Truly reflowing a page means rebuilding it from extracted content.
+
+- Still possible: replace a line or a block in place (see `tasks/annotations.md`), remove a block, stamp a note.
+- Resolution for reflow: rebuild the document from a spec with the corrected text, and say that it was rebuilt rather than patched.
+
+## Lexical repair of broken files
+
+A file with a damaged cross-reference table may still render (viewers rebuild the xref) while strict parsers refuse it. If PyMuPDF or pypdf cannot open the file, do not silently produce an empty document.
+
+- Still possible: report the parse error, ask the user to open and re-save the file in a reader (which repairs it), then re-run.
+- Resolution: whichever tool originally produced the file should be told about the corruption.
+
+## Tagged PDF and accessibility remediation
+
+Reading order, alternative text, and tags live in a structure tree that the libraries here can read but not author reliably. A visual edit does not add tags.
+
+- Still possible: keep the visual result correct and say the file is untagged.
+- Resolution: accessibility remediation needs an authoring tool that writes the structure tree.
+
+## PDF/A and archival conformance
+
+Conformance requires font embedding rules, metadata schemas, and colour profiles that these libraries do not validate.
+
+- Still possible: many PDF/A inputs will survive a page-level edit and remain close, but claiming conformance would be a guess.
+- Resolution: validate with a dedicated PDF/A checker before delivery.
+
+## Creating or restructuring form fields
+
+pypdf can fill existing fields; creating a new interactive field with a correct `/AP` state map is much more than a fill. reportlab can draw a form-like appearance, but without a widget tree it is not a form.
+
+- Still possible: stamp labels and values at coordinates so the printed result looks filled.
+- Resolution: an authoring tool for interactive forms, or accept a non-interactive output and say so.

package/assets/skills/office-pdf/scripts/extract_text.py

@@ -0,0 +1,172 @@
+#!/usr/bin/env python3
+"""Extract per-page text and tables from a PDF with pdfplumber.
+
+Use this when you need the words, not the picture: page text, word counts, and
+detected tables as row arrays. Visual fidelity is not part of the contract -
+layout, wrapping, and column order may differ from what the rendered page
+shows - so always render before judging a document on this output alone.
+
+Exit codes: 0 success, 1 bad arguments, 2 unreadable input or write failure.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+for _stream in (sys.stdout, sys.stderr):
+    try:
+        _stream.reconfigure(encoding="utf-8", errors="replace")
+    except (AttributeError, ValueError):
+        pass
+
+
+def emit(value, out=None):
+    text = json.dumps(value, ensure_ascii=False, indent=2, default=str)
+    if out:
+        target = Path(out).expanduser()
+        target.parent.mkdir(parents=True, exist_ok=True)
+        target.write_text(text + "\n", encoding="utf-8")
+    print(text)
+
+
+def fail(message, code=1, detail=None):
+    payload = {"ok": False, "error": message}
+    if detail is not None:
+        payload["detail"] = detail
+    print(json.dumps(payload, ensure_ascii=False, indent=2, default=str), file=sys.stderr)
+    raise SystemExit(code)
+
+
+def parse_pages(spec, page_count):
+    if not spec or spec.strip().lower() == "all":
+        return list(range(1, page_count + 1))
+    pages = []
+    try:
+        for part in spec.split(","):
+            part = part.strip()
+            if not part:
+                continue
+            if "-" in part:
+                start_text, end_text = part.split("-", 1)
+                start, end = int(start_text), int(end_text)
+            else:
+                start = end = int(part)
+            if start < 1 or end > page_count or start > end:
+                fail(f"page range out of bounds: {part} (document has {page_count} pages)")
+            pages.extend(range(start, end + 1))
+    except ValueError:
+        fail(f"cannot parse page selection: {spec!r}")
+    return sorted(dict.fromkeys(pages))
+
+
+def clean(cell):
+    return "" if cell is None else str(cell).replace("\n", " ").strip()
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Extract page text and detected tables from a PDF.",
+        epilog="Example: extract_text.py --input report.pdf --pages 1-2 --layout",
+    )
+    parser.add_argument("--input", required=True, help="PDF file to read")
+    parser.add_argument("--pages", default="all", help='1-based selection such as "1-3,7" (default: all)')
+    parser.add_argument("--layout", action="store_true", help="Keep rough horizontal layout in the text output")
+    parser.add_argument("--no-tables", action="store_true", help="Skip table detection (faster on long documents)")
+    parser.add_argument("--text-out", help="Also save the joined page text to this file")
+    parser.add_argument("--json-out", help="Also write the JSON report to this path")
+    args = parser.parse_args()
+
+    source = Path(args.input).expanduser()
+    if not source.is_file():
+        fail(f"input not found: {source}", code=2)
+
+    try:
+        import pdfplumber
+    except ImportError:
+        fail("pdfplumber is not installed; run the plugin postinstall again", code=2)
+
+    try:
+        pdf = pdfplumber.open(str(source))
+    except Exception as exc:
+        fail(f"cannot open PDF: {exc}", code=2)
+
+    with pdf:
+        page_count = len(pdf.pages)
+        selected = parse_pages(args.pages, page_count)
+        reports = []
+        warnings = []
+        joined = []
+        total_chars = 0
+        total_tables = 0
+        for index in selected:
+            page = pdf.pages[index - 1]
+            try:
+                text = page.extract_text(layout=args.layout) or ""
+            except Exception as exc:
+                fail(f"page {index} text extraction failed: {exc}", code=2)
+            report = {
+                "page": index,
+                "width_pt": round(float(page.width), 2),
+                "height_pt": round(float(page.height), 2),
+                "rotation": int(page.rotation or 0),
+                "chars": len(text),
+                "words": len(text.split()),
+                "text": text,
+                "tables": [],
+            }
+            if not args.no_tables:
+                try:
+                    tables = page.find_tables()
+                except Exception as exc:
+                    tables = []
+                    warnings.append(f"page {index}: table detection failed: {exc}")
+                for order, table in enumerate(tables, start=1):
+                    rows = [[clean(cell) for cell in row] for row in table.extract()]
+                    report["tables"].append(
+                        {
+                            "index": order,
+                            "bbox": [round(float(value), 2) for value in table.bbox],
+                            "row_count": len(rows),
+                            "column_count": max((len(row) for row in rows), default=0),
+                            "rows": rows,
+                        }
+                    )
+                total_tables += len(report["tables"])
+            if report["chars"] == 0 and len(page.images) > 0:
+                warnings.append(
+                    f"page {index} has images but no extractable text: it is probably a scan, "
+                    "and this runtime has no OCR"
+                )
+            total_chars += report["chars"]
+            joined.append(f"===== page {index} =====\n{text}")
+            reports.append(report)
+
+    if args.text_out:
+        try:
+            target = Path(args.text_out).expanduser()
+            target.parent.mkdir(parents=True, exist_ok=True)
+            target.write_text("\n\n".join(joined).rstrip() + "\n", encoding="utf-8")
+        except OSError as exc:
+            fail(f"cannot write text output: {exc}", code=2)
+
+    emit(
+        {
+            "ok": True,
+            "input": str(source),
+            "page_count": page_count,
+            "pages_read": len(selected),
+            "text_out": str(Path(args.text_out).expanduser()) if args.text_out else None,
+            "total_chars": total_chars,
+            "total_tables": total_tables,
+            "pages": reports,
+            "warnings": warnings,
+        },
+        args.json_out,
+    )
+
+
+if __name__ == "__main__":
+    main()