skilldrop-cli 0.16.32 → 0.16.33

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "skilldrop-cli",
-  "version": "0.16.32",
+  "version": "0.16.33",
   "description": "Portable AI-agent skills, measured against the agent without them. ADRs, PRDs, runbooks, decks and reviews, installable into Claude Code, Cursor, Kiro, and any IDE that takes custom instructions. One command per skill or per role-based pack.",
   "author": {
     "name": "Sanjay Ananthanarayan",

package/packs/converters/skills/file-to-markdown/SKILL.md

@@ -47,12 +47,16 @@ or a rewrite that goes back out through `md-to-docx` or `md-to-html`.
    | `.csv` / `.tsv` | One table, delimiter detected |
    | `.json` | An array of objects becomes a table; an object becomes a key/value table plus a table per array |
    | `.txt` / `.md` | Passed through |
-   | `.pdf` | Through `pdftotext`. Tables from aligned columns, lists from indentation, short standalone lines as `###` headings (guessed, and said so); pages with side-by-side columns are read in column order with nothing inferred. No images |
+   | `.pdf` | Through `pdftotext`. Tables from aligned columns, lists from indentation, short standalone lines as `###` headings (guessed, and said so); a table that runs onto the next page is joined into one, without the header the PDF repeats; pages with side-by-side columns are read in column order with nothing inferred. Scanned pages are read by OCR when `tesseract` is installed. No images |
 
 3. **PDF without `pdftotext`.** The script stops with install steps (`brew install poppler`
    on macOS, `sudo apt install poppler-utils` on Debian/Ubuntu). Pass those on; don't try to
-   read the PDF bytes yourself. A PDF with no text layer is a scan and needs OCR, which this
-   skill does not do.
+   read the PDF bytes yourself. A page with no text layer is a scan. The script reads it by
+   OCR when `tesseract` is installed (`brew install tesseract`, `sudo apt install tesseract-ocr`),
+   and otherwise stops with those steps. For a scan that isn't in English, pass `--ocr-lang`
+   with tesseract's code (`deu`, `fra`, `eng+fra`). OCR misreads characters, so the reply says
+   which pages were OCR'd and asks the user to check every number, name and date in them
+   against the PDF.
 
 4. **Old binary formats** (`.doc`, `.ppt`, `.xls`, `.rtf`, `.pages`, `.key`): the script
    refuses and names the format to save as. On macOS, `textutil -convert docx file.doc`
@@ -73,7 +77,7 @@ missing, emit `BLOCKED: need <the file | pdftotext on PATH>` and stop.
 
 ## Useful references in this skill
 
-- [`scripts/to_markdown.py`](scripts/to_markdown.py): the converter (stdlib only; `pdftotext` optional for PDF)
+- [`scripts/to_markdown.py`](scripts/to_markdown.py): the converter (stdlib only; `pdftotext` optional for PDF, `tesseract` for scanned pages)
 - [`examples/mixed-inputs.md`](examples/mixed-inputs.md): a Word brief, a deck, a workbook and a web page converted, with the real Markdown and stderr report
 
 ## Quality bar

package/packs/converters/skills/file-to-markdown/manifest.json

@@ -1,6 +1,6 @@
 {
   "name": "file-to-markdown",
-  "version": "0.1.4",
+  "version": "0.2.0",
   "description": "Convert a Word document, PowerPoint deck, Excel workbook, web page, CSV, JSON, text file or PDF into clean Markdown that keeps headings, lists, tables, links, bold and italic, slide titles and speaker notes, and reports everything it dropped (images, charts, comments, formulas, tracked changes), so the content can be read, reviewed or fed to another skill. Use when the user hands over a .docx, .pptx, .xlsx, .html, .csv, .json or .pdf and says \"read this\", \"turn this into markdown\", \"extract the text\", or wants it critiqued or briefed.",
   "entrypoint": "SKILL.md",
   "deps": {
@@ -41,7 +41,16 @@
     "pdf",
     "extract"
   ],
-  "permissions": {"network": [], "commands": ["pdftotext"], "files": "named-paths", "notes": "pdftotext (Poppler) is used only for .pdf input, and only if it is installed."},
+  "permissions": {
+    "network": [],
+    "commands": [
+      "pdftotext",
+      "pdftoppm",
+      "tesseract"
+    ],
+    "files": "named-paths",
+    "notes": "pdftotext (Poppler) is used only for .pdf input, and only if it is installed. A page with no text layer is rendered with pdftoppm (Poppler) into a temporary folder that is deleted afterwards, and read with tesseract, only if tesseract is installed."
+  },
   "model": {
     "tier": "light",
     "rationale": "A deterministic stdlib script does the extraction; the model runs it, relays the dropped-content report and hands off. No generation beyond a short summary of losses."

package/packs/converters/skills/file-to-markdown/scripts/to_markdown.py

@@ -2,7 +2,8 @@
 """Convert a document into clean Markdown that another skill can read.
 
 Stdlib only, Python 3.9+. Handles .docx, .pptx, .xlsx, .html/.htm, .csv/.tsv,
-.json, .txt/.md, and .pdf (through `pdftotext` when it is installed).
+.json, .txt/.md, and .pdf (through `pdftotext` when it is installed). A PDF page with no
+text layer (a scan) is read by OCR when `tesseract` and `pdftoppm` are installed.
 Everything the conversion could not carry (images, charts, comments, formulas,
 tracked changes) is counted and reported on stderr, so nobody mistakes the
 Markdown for the whole document.
@@ -12,6 +13,7 @@ Usage:
     python3 to_markdown.py deck.pptx -o deck.md
     python3 to_markdown.py model.xlsx -o model.md --max-rows 200
     python3 to_markdown.py page.html -o -          # write to stdout
+    python3 to_markdown.py scan.pdf -o scan.md --ocr-lang deu   # OCR a German scan
 """
 
 import argparse
@@ -22,6 +24,7 @@ import re
 import shutil
 import subprocess
 import sys
+import tempfile
 import zipfile
 from collections import OrderedDict
 from datetime import datetime, timedelta
@@ -1122,8 +1125,9 @@ def _pdf_page_md(page, rep):
                     region.append(line); rows += 1; cols |= _col_starts(line); j += 1
                 elif line.strip() and _col_starts(line) and _col_starts(line) <= _near(cols):
                     region.append(line); j += 1
-                elif not line.strip() and j + 1 < n and lines[j + 1].strip() and (
-                        len(_cells(lines[j + 1])) >= 2 or _col_starts(lines[j + 1]) <= _near(cols)):
+                elif not line.strip() and j + 1 < n and len(_cells(lines[j + 1])) >= 2:
+                    # After a blank line only a full row continues the table. A one-cell line
+                    # there ("End of report.") is the next paragraph, not a wrapped cell.
                     j += 1
                 else:
                     break
@@ -1193,7 +1197,83 @@ def _pdf_page_md(page, rep):
     return out
 
 
-def pdf_to_md(path, rep):
+_FURNITURE = re.compile(r"^\s*(page\s+)?\d{1,4}(\s*(of|/)\s*\d{1,4})?\s*$|^\s*[-–—]\s*\d{1,4}\s*[-–—]\s*$", re.I)
+
+
+def _join_split_tables(pages, rep):
+    """A table that runs off the foot of one page continues at the top of the next, where it
+    would otherwise start a second table whose first data row reads as its header. When a page
+    ends in a table and the next page's first rows (after page numbers and any running header)
+    have the same number of cells, those rows are moved up to finish the table. A header row
+    the PDF repeats on the new page is dropped. Returns the pages, rewritten."""
+    pages = [p.split("\n") for p in pages]
+    first_lines = [next((l.strip() for l in p if l.strip()), "") for p in pages]
+    running = {t for t in first_lines if t and first_lines.count(t) > 1}  # a header on several pages
+    holder = None
+    for k in range(len(pages) - 1):
+        a_idx = k if holder is None else holder
+        a, b = pages[a_idx], pages[k + 1]
+        # the last table row on the holder page, before trailing page furniture
+        end = len(a) - 1
+        while end >= 0 and (not a[end].strip() or _FURNITURE.match(a[end])):
+            end -= 1
+        holder = None
+        if end < 0 or len(_cells(a[end])) < 2:
+            continue
+        width = len(_cells(a[end]))
+        start = end
+        while start - 1 >= 0 and (len(_cells(a[start - 1])) == width or
+                                  (not a[start - 1].strip() and start - 2 >= 0 and len(_cells(a[start - 2])) == width)):
+            start -= 1
+        header = " ".join(a[start].split())
+        # the first content on the next page, past page numbers and a running header
+        i = 0
+        while i < len(b) and (not b[i].strip() or _FURNITURE.match(b[i]) or b[i].strip() in running):
+            i += 1
+        if i >= len(b) or len(_cells(b[i])) != width:
+            continue
+        j = i
+        rows = []
+        while j < len(b) and (len(_cells(b[j])) == width or
+                              (not b[j].strip() and j + 1 < len(b) and len(_cells(b[j + 1])) == width)):
+            if b[j].strip():
+                rows.append(b[j])
+            j += 1
+        dropped = bool(rows) and " ".join(rows[0].split()) == header
+        if dropped:
+            rows = rows[1:]
+        if not rows:
+            continue
+        a[end + 1:end + 1] = rows
+        del b[i:j]
+        rep.notes.append("a table continued from page %d onto page %d was joined into one%s" % (
+            a_idx + 1, k + 2, ", and the header row repeated on page %d was dropped" % (k + 2) if dropped else ""))
+        rep.keep("pdf tables joined across pages", 1)
+        if not any(l.strip() and not _FURNITURE.match(l) and l.strip() not in running for l in b):
+            holder = a_idx  # the whole page was table: the next page may continue it too
+    return ["\n".join(p) for p in pages]
+
+
+def _ocr_page(path, n, ppm, tess, lang):
+    """Page n of the PDF as text, by rendering it at 300 dpi and reading it with tesseract.
+    Page segmentation mode 6 reads a uniform block and keeps the spacing between words, which
+    is what the table and list inference reads columns from."""
+    with tempfile.TemporaryDirectory(prefix="to-md-ocr-") as tmp:
+        stem = str(Path(tmp) / "page")
+        r = subprocess.run([ppm, "-r", "300", "-png", "-f", str(n), "-l", str(n), str(path), stem],
+                           stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False)
+        pngs = sorted(Path(tmp).glob("page*.png"))
+        if r.returncode != 0 or not pngs:
+            sys.exit("error: pdftoppm could not render page %d: %s" % (n, r.stderr.decode("utf-8", "replace").strip()))
+        r = subprocess.run([tess, str(pngs[0]), "stdout", "-l", lang, "--psm", "6",
+                            "-c", "preserve_interword_spaces=1"],
+                           stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False)
+        if r.returncode != 0:
+            sys.exit("error: tesseract failed on page %d: %s" % (n, r.stderr.decode("utf-8", "replace").strip()))
+        return r.stdout.decode("utf-8", "replace").replace("\f", "")
+
+
+def pdf_to_md(path, rep, ocr=True, ocr_lang="eng"):
     exe = shutil.which("pdftotext")
     if not exe:
         sys.exit("error: reading PDF needs `pdftotext` (part of Poppler), which is not on PATH.\n"
@@ -1211,9 +1291,33 @@ def pdf_to_md(path, rep):
         return pages[:-1] if pages and not pages[-1].strip() else pages
 
     layout, reading = run(["-layout"]), run([])
-    if not "".join(layout).strip():
-        sys.exit("error: %s has no text layer (probably scanned). It needs OCR first; "
-                 "this script does not do OCR." % path)
+    blank = [n for n, p in enumerate(layout, 1) if not p.strip()]
+    if blank and ocr is not False:
+        tess, ppm = shutil.which("tesseract"), shutil.which("pdftoppm")
+        if not (tess and ppm):
+            if len(blank) == len(layout):
+                sys.exit("error: %s has no text layer (probably scanned), so it needs OCR, which uses "
+                         "`tesseract`%s.\n"
+                         "  macOS:          brew install tesseract   (pdftoppm comes with poppler)\n"
+                         "  Debian/Ubuntu:  sudo apt install tesseract-ocr poppler-utils\n"
+                         "  Windows:        install Tesseract (UB Mannheim build) and add it to PATH\n"
+                         "Then rerun. Add --ocr-lang deu (or another installed language) for a non-English scan."
+                         % (path, "" if ppm else " and `pdftoppm`"))
+            rep.notes.append("page(s) %s have no text layer and were left empty: install tesseract to read them by OCR"
+                             % ", ".join(map(str, blank)))
+        else:
+            for n in blank:
+                text = _ocr_page(path, n, ppm, tess, ocr_lang)
+                layout[n - 1] = text
+                if n <= len(reading):
+                    reading[n - 1] = text
+            rep.keep("pages read by OCR", len(blank))
+            rep.notes.append("page(s) %s are scanned, so their text was read by OCR (tesseract, language %s, 300 dpi). "
+                             "OCR misreads characters: check every number, name and date against the PDF"
+                             % (", ".join(map(str, blank)), ocr_lang))
+    elif blank and len(blank) == len(layout):
+        sys.exit("error: %s has no text layer (probably scanned), and --no-ocr was given." % path)
+    layout = _join_split_tables(layout, rep)
     out, plain_pages = [], []
     for n, page in enumerate(layout, 1):
         out += ["<!-- page %d -->" % n, ""]
@@ -1250,7 +1354,12 @@ def main(argv=None):
     ap.add_argument("-o", "--output", required=True, help="Markdown file to write, or - for stdout")
     ap.add_argument("--max-rows", type=int, default=0,
                     help="xlsx only: keep at most this many data rows per sheet (0 = all)")
+    ap.add_argument("--ocr-lang", default="eng",
+                    help="pdf only: tesseract language for scanned pages, e.g. eng, deu, fra, eng+fra (default eng)")
+    ap.add_argument("--no-ocr", action="store_true", help="pdf only: never run OCR on scanned pages")
     a = ap.parse_args(argv)
+    if not re.fullmatch(r"[A-Za-z_]{3,}(\+[A-Za-z_]{3,})*", a.ocr_lang):
+        sys.exit("error: --ocr-lang takes tesseract language codes such as eng or eng+fra")
 
     path = Path(a.input)
     if not path.is_file():
@@ -1267,7 +1376,8 @@ def main(argv=None):
             md += "\n"
     elif ext in HANDLERS:
         fn = HANDLERS[ext]
-        md = fn(path, rep, a.max_rows) if fn is xlsx_to_md else fn(path, rep)
+        md = (fn(path, rep, a.max_rows) if fn is xlsx_to_md else
+              fn(path, rep, ocr=not a.no_ocr, ocr_lang=a.ocr_lang) if fn is pdf_to_md else fn(path, rep))
     else:
         sys.exit("error: unsupported file type %r. Supported: %s, .txt, .md"
                  % (ext, ", ".join(sorted(HANDLERS))))

package/packs/converters/skills/md-to-docx/examples/q3-support-review.md

@@ -66,6 +66,13 @@ has a shaded header row and right-aligned numbers, and the 480 × 240 px chart (
 DPI) is centred at 3.3 × 1.7 in. The numbered proposals run 1 to 3, the "support dashboard" and
 "the alias section" links are blue and underlined, and the quote is indented and italic.
 
+**Opened in Microsoft Word** for Mac 16.114 (2026-10-20), through AppleScript: it opened with
+no repair prompt, the first paragraph has the built-in Heading 1 style, and it holds one table
+and one inline picture. Word's own PDF export of it shows the nested bullets, both task boxes,
+the table with its numbers right-aligned, the code block and the quote, embeds the chart at
+480 × 240, and carries both links as live link annotations: the dashboard URL and the jump to
+"Retiring the alias".
+
 ---
 
 ## Reply to the user

package/packs/converters/skills/md-to-xlsx/examples/vendor-review.md

@@ -76,7 +76,12 @@ all well-formed. Row 2 of the first sheet reads back as:
 
 The frozen pane is `ySplit="1"` and the autofilter is `A1:G5`. A Quick Look preview shows
 `$1,240,500.00`, `12.5%` and `2027-03-31` as formatted values. Quick Look leaves boolean cells
-blank, which is a limitation of the previewer. Excel and LibreOffice show them as TRUE/FALSE.
+blank, which is a limitation of the previewer.
+
+**Opened in Microsoft Excel** for Mac 16.114 (2026-10-20), read through AppleScript without
+saving: three sheets; D2 is the number 1240500 shown as `$1,240,500.00`, E2 the number 0.125
+shown as `12.5%`, F2 a date shown as `2027-03-31`, G2 the boolean `TRUE`, and A2 the text
+`00412` with its zeros. The first sheet has its filter on and its header row frozen.
 
 ---