rsc-universal 0.1.1

package/skills/docker/references/compose-recipes.md

@@ -0,0 +1,109 @@
+# Compose recipes
+
+Fuller `compose.yaml` patterns. Canonical filename is `compose.yaml`; omit `version:`; use the
+`docker compose` v2 subcommand.
+
+## Watch per stack
+
+| Stack | `sync` paths | `rebuild` triggers |
+|---|---|---|
+| Node/Vite/Next | `./src`, `./public` | `package-lock.json` |
+| Python/uv | `./app` | `uv.lock`, `pyproject.toml` |
+| Go (live reload via `air`) | `./` | `go.mod`, `go.sum` |
+
+`sync` copies changed files into the running container (the process must itself hot-reload).
+`rebuild` rebuilds the image when a dependency manifest changes. `sync+restart` syncs then restarts
+PID 1 — use it for config files the process reads only at startup. Use `ignore:` under a watch
+rule to exclude noisy paths (e.g. `node_modules`).
+
+## Healthcheck graph: postgres + redis + app
+
+```yaml
+services:
+  app:
+    build: .
+    depends_on:
+      db:
+        condition: service_healthy
+      cache:
+        condition: service_healthy
+    environment:
+      DATABASE_URL: postgres://postgres:dev@db:5432/app
+      REDIS_URL: redis://cache:6379
+    develop:
+      watch:
+        - { action: sync, path: ./app, target: /app/app }
+        - { action: rebuild, path: ./uv.lock }
+  db:
+    image: postgres:17-bookworm
+    environment: { POSTGRES_PASSWORD: dev, POSTGRES_DB: app }
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U postgres -d app"]
+      interval: 5s
+      timeout: 3s
+      retries: 5
+    volumes: ["pgdata:/var/lib/postgresql/data"]
+  cache:
+    image: redis:7-bookworm
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 5s
+      timeout: 3s
+      retries: 5
+
+volumes:
+  pgdata:
+```
+
+`condition: service_healthy` makes `app` wait until both backing services report healthy, instead
+of merely "container started" — which avoids the classic "connection refused on boot" race.
+
+## Profiles
+
+```yaml
+services:
+  app: { build: . }
+  seed:
+    profiles: ["tools"]
+    build: .
+    command: ["python", "-m", "app.seed"]
+  mailpit:
+    profiles: ["tools"]
+    image: axllent/mailpit:latest
+```
+
+Services with a `profiles:` key are inert unless their profile is requested:
+`docker compose --profile tools up`. Use it for seeders, mail catchers, admin UIs.
+
+## env_file vs secrets
+
+- `env_file: [.env]` — convenient for non-sensitive dev config; values become container env.
+- `secrets:` — mounts a file at `/run/secrets/<name>`; the value is not in the process environment
+  and not in `docker inspect`. Prefer it for anything sensitive even in dev.
+
+```yaml
+services:
+  app:
+    build: .
+    secrets: [db_password]
+secrets:
+  db_password:
+    file: ./secrets/db_password.txt
+```
+
+## Override files
+
+`docker compose` automatically merges `compose.override.yaml` on top of `compose.yaml`. Keep dev-only
+bind mounts and `develop.watch` in the base for local use; create an explicit
+`-f compose.yaml -f compose.prod.yaml` pair if you must describe a prod-ish stack — but that is the
+edge of this skill's scope.
+
+## When to graduate
+
+Compose is a single-host orchestrator. The moment you need rolling deploys, multi-host scheduling,
+autoscaling, or a registry/CI pipeline, stop here:
+
+- registry push, CI build, deploy to a host, rollback → `../deployment/SKILL.md`
+- horizontal autoscaling, k8s, HPA, ingress → `scaling`
+- running Compose on a managed host / self-hosted PaaS → `../coolify/SKILL.md`,
+  `../railway/SKILL.md`, `../fly-io/SKILL.md`

package/skills/docker/scripts/verify.sh

@@ -0,0 +1,149 @@
+#!/usr/bin/env bash
+# verify.sh — read-only static checks for docker artifacts (Dockerfile, compose.yaml, .dockerignore)
+#
+# Usage:
+#   ./verify.sh [DIR] [IMAGE_TAG]
+#     DIR        directory to scan (default: current dir)
+#     IMAGE_TAG  optional built image to scan with trivy/dockle and size-check
+#
+# Behavior:
+#   - Read-only: never builds, never writes, never pulls.
+#   - Degrades gracefully: a missing tool is a SKIP (warning), not a failure — CI-safe.
+#   - Exits 0 on a clean or empty target (no Dockerfile/compose present => nothing to fail).
+#   - Exits 1 only on a real banned pattern or an error-level lint/parse failure.
+
+set -uo pipefail
+
+DIR="${1:-.}"
+IMAGE_TAG="${2:-}"
+SIZE_LIMIT_BYTES="${DOCKER_VERIFY_SIZE_LIMIT:-524288000}" # 500 MB default, override via env
+
+fail=0
+have() { command -v "$1" >/dev/null 2>&1; }
+warn() { printf 'SKIP: %s\n' "$1" >&2; }
+err()  { printf 'FAIL: %s\n' "$1" >&2; fail=1; }
+ok()   { printf 'OK:   %s\n' "$1"; }
+
+# Locate a Dockerfile / Containerfile.
+DOCKERFILE=""
+for cand in "$DIR/Dockerfile" "$DIR/Containerfile"; do
+  [ -f "$cand" ] && DOCKERFILE="$cand" && break
+done
+
+# Locate a compose file.
+COMPOSE=""
+for cand in "$DIR/compose.yaml" "$DIR/compose.yml" "$DIR/docker-compose.yaml" "$DIR/docker-compose.yml"; do
+  [ -f "$cand" ] && COMPOSE="$cand" && break
+done
+
+if [ -z "$DOCKERFILE" ] && [ -z "$COMPOSE" ]; then
+  ok "no Dockerfile or compose file in '$DIR' — nothing to verify"
+  exit 0
+fi
+
+# ---- Dockerfile checks ----
+if [ -n "$DOCKERFILE" ]; then
+  ok "found $DOCKERFILE"
+
+  # Ban: :latest base images.
+  if grep -nEi '^[[:space:]]*FROM[[:space:]]+[^[:space:]]+:latest([[:space:]]|$)' "$DOCKERFILE" >/dev/null \
+     || grep -nEi '^[[:space:]]*FROM[[:space:]]+[^[:space:]@:]+([[:space:]]+AS|[[:space:]]*$)' "$DOCKERFILE" >/dev/null; then
+    err "$DOCKERFILE: FROM uses :latest or an unpinned image (pin a tag or digest)"
+  else
+    ok "all FROM lines are pinned (no :latest)"
+  fi
+
+  # Ban: secrets in ARG/ENV.
+  if grep -nEi '^[[:space:]]*(ARG|ENV)[[:space:]]+.*(TOKEN|SECRET|PASSWORD|PASSWD|API[_-]?KEY|PRIVATE[_-]?KEY|ACCESS[_-]?KEY)' "$DOCKERFILE" >/dev/null; then
+    err "$DOCKERFILE: secret-looking value in ARG/ENV (use RUN --mount=type=secret)"
+  else
+    ok "no secret-looking ARG/ENV"
+  fi
+
+  # Require: a USER instruction (non-root).
+  if grep -nEi '^[[:space:]]*USER[[:space:]]+' "$DOCKERFILE" >/dev/null; then
+    if grep -nEi '^[[:space:]]*USER[[:space:]]+(root|0)[[:space:]]*$' "$DOCKERFILE" >/dev/null; then
+      err "$DOCKERFILE: final USER is root/0 (run as a non-root user)"
+    else
+      ok "non-root USER present"
+    fi
+  else
+    err "$DOCKERFILE: no USER instruction (container will run as root)"
+  fi
+
+  # hadolint (optional).
+  if have hadolint; then
+    if hadolint --no-fail "$DOCKERFILE" | grep -qiE ' error| DL[0-9]+ error'; then
+      err "$DOCKERFILE: hadolint reported error-level findings"
+    else
+      hadolint --failure-threshold error "$DOCKERFILE" && ok "hadolint clean (error level)" || err "$DOCKERFILE: hadolint error-level findings"
+    fi
+  else
+    warn "hadolint not installed — skipping Dockerfile lint"
+  fi
+fi
+
+# ---- compose checks ----
+if [ -n "$COMPOSE" ]; then
+  ok "found $COMPOSE"
+
+  # Ban: obsolete version: key.
+  if grep -nE '^[[:space:]]*version[[:space:]]*:' "$COMPOSE" >/dev/null; then
+    err "$COMPOSE: obsolete top-level 'version:' key (remove it)"
+  else
+    ok "no obsolete version: key"
+  fi
+
+  # Validate with docker compose if available.
+  if have docker && docker compose version >/dev/null 2>&1; then
+    if docker compose -f "$COMPOSE" config -q >/dev/null 2>&1; then
+      ok "docker compose config -q parses"
+    else
+      err "$COMPOSE: docker compose config -q failed to parse/resolve"
+    fi
+  else
+    warn "docker compose v2 not available — skipping compose validation"
+  fi
+fi
+
+# ---- optional image scans (only when a tag is provided) ----
+if [ -n "$IMAGE_TAG" ]; then
+  if have trivy; then
+    trivy image --scanners vuln --severity HIGH,CRITICAL --exit-code 0 "$IMAGE_TAG" \
+      && ok "trivy scanned $IMAGE_TAG (review HIGH/CRITICAL above)" \
+      || warn "trivy could not scan $IMAGE_TAG"
+  else
+    warn "trivy not installed — skipping image CVE scan"
+  fi
+
+  if have dockle; then
+    dockle --exit-code 0 "$IMAGE_TAG" && ok "dockle checked $IMAGE_TAG" || warn "dockle could not check $IMAGE_TAG"
+  else
+    warn "dockle not installed — skipping image hygiene check"
+  fi
+
+  if have docker; then
+    size="$(docker image inspect -f '{{.Size}}' "$IMAGE_TAG" 2>/dev/null || echo '')"
+    if [ -n "$size" ]; then
+      if [ "$size" -gt "$SIZE_LIMIT_BYTES" ]; then
+        err "image $IMAGE_TAG is ${size} bytes (> ${SIZE_LIMIT_BYTES}); consider a smaller base / multi-stage"
+      else
+        ok "image $IMAGE_TAG size ${size} bytes is within limit ${SIZE_LIMIT_BYTES}"
+      fi
+    else
+      warn "could not inspect size of $IMAGE_TAG (not built locally?)"
+    fi
+  fi
+elif have trivy && [ -n "$DOCKERFILE" ]; then
+  # No tag: still do a build-free config scan if trivy is present.
+  trivy config "$DIR" --severity HIGH,CRITICAL --exit-code 0 \
+    && ok "trivy config scanned $DIR (review findings above)" \
+    || warn "trivy config could not scan $DIR"
+fi
+
+if [ "$fail" -ne 0 ]; then
+  printf '\nverify.sh: FAILED\n' >&2
+  exit 1
+fi
+printf '\nverify.sh: PASSED\n'
+exit 0

package/skills/document-processing/SKILL.md

@@ -0,0 +1,214 @@
+---
+name: document-processing
+description: "Use when you need to get content out of a document or build a document out of data — extracting text/tables from PDFs, filling interactive PDF forms (AcroForm), generating PDF/DOCX from templates, or OCR'ing scanned/image-only files into Markdown/JSON. Triggers: 'extract the line items from this invoice PDF', 'fill this PDF form and flatten it', 'generate a contract DOCX from this template', 'this scan has no text layer', 'the PDF copies out as garbage symbols', 'merge/split/rotate these PDFs', 'OCR 300 scanned pages', 'rellena este formulario PDF y fusiónalos', 'extreu les taules d'aquesta factura escanejada'. NOT pulling schema-typed fields out of text (that is structured-extraction), NOT sending it for signature (that is e-signature), NOT reading spreadsheet cells/formulas (that is spreadsheet-ops)."
+tags: [pdf, ocr, docx, forms, extraction, document-ai]
+recommends: [structured-extraction, e-signature, spreadsheet-ops, rag, data-scraper]
+origin: risco
+---
+
+# Document processing
+
+File in, content out — or data in, file out. You open a byte stream (PDF, DOCX, scan) and either pull the content out, or you build a new document from a template and a data dict. That is the whole job: the deliverable is **bytes of a document or the literal content of one**.
+
+The boundary test, apply it first:
+
+- Deliverable is **raw text / Markdown / table cells / a generated file** → you are in the right place.
+- Deliverable is **a typed object matching a schema** (`{parties: [...], total: 1234.50}`) → that is `structured-extraction`. This skill stops at "clean Markdown out of the file"; the schema-constrained extraction runs on that Markdown.
+
+Everything else routes too: signing → `e-signature`, spreadsheet grids/formulas → `spreadsheet-ops`, indexing for Q&A → `rag`, downloading the files off a site → `data-scraper`.
+
+## Step 0 — does the PDF have a text layer?
+
+The most expensive mistake in this skill is OCR'ing a PDF that already has a text layer. A digital PDF (exported from Word, a browser, a report tool) carries selectable text — extracting it is free, instant, and lossless. OCR is slow, costs money or GPU, and *introduces* errors. **Never OCR a PDF you can extract.**
+
+Check before you pick an engine:
+
+```python
+import pdfplumber
+
+with pdfplumber.open("doc.pdf") as pdf:
+    txt = pdf.pages[0].extract_text() or ""
+
+if len(txt.strip()) > 20:
+    print("text layer present -> extract directly (pdfplumber / pypdf)")
+else:
+    print("image-only or empty -> this is an OCR job")
+```
+
+If `extract_text()` returns empty (or near-empty) across the first few pages, it is a scan or image-only PDF and you go to the OCR branch. Symptom from the user's side: *"the text copies out as garbage / random symbols"* usually means a broken/embedded font, not a missing text layer — try `pypdf` extraction too before assuming OCR.
+
+## Engine selection
+
+| Goal | Use | Why |
+|---|---|---|
+| Extract text + tables with layout | **pdfplumber** | Layout-aware; `extract_tables()` returns rows/cols as Python lists → pandas/CSV. |
+| Raw text, merge, split, rotate, page ops | **pypdf** (6.12.2) | Pure-Python, no C deps, runs in Lambda/containers; the maintained core. |
+| Fill an interactive PDF form | **pypdf** | `update_page_form_field_values` writes AcroForm fields; can flatten. |
+| Generate a Word/DOCX from a template | **docxtpl** (0.20.x) | A real `.docx` becomes a Jinja2 template; author in Word, tag, render. |
+| Generate a PDF from scratch | **ReportLab** | Canvas / Platypus flowables for laid-out PDFs. |
+| OCR a scan, local / no API budget | **Docling** (or Marker) | Layout + reading order + table structure, fully local, wraps Tesseract/RapidOCR. |
+| OCR messy scans / handwriting / hard tables, API ok | **Mistral OCR** | `mistral-ocr-2512` (OCR 3), ~$2 / 1,000 pages, tuned for forms + handwriting. |
+| Fastest extract / easiest page→PNG raster | **PyMuPDF** ⚠️ **AGPL** | Fast, but AGPL: shipping it imposes an open-source obligation or needs a paid license. Flag this before recommending. |
+
+Rule: use the maintained **`pypdf`** import, never the dead `PyPDF2` — `PyPDF2` is unmaintained and was merged back into `pypdf`. Importing it is a signal of stale code.
+
+## Extraction recipes
+
+Text + tables with `pdfplumber`, straight to CSV:
+
+```python
+import csv
+import pdfplumber
+
+rows = []
+with pdfplumber.open("invoice.pdf") as pdf:
+    for page in pdf.pages:
+        for table in page.extract_tables():
+            rows.extend(table)
+
+with open("out.csv", "w", newline="") as f:
+    csv.writer(f).writerows(rows)
+```
+
+Raw text, merge, split, rotate with `pypdf`:
+
+```python
+from pypdf import PdfReader, PdfWriter
+
+# raw text
+text = "\n".join(p.extract_text() or "" for p in PdfReader("doc.pdf").pages)
+
+# merge two files
+w = PdfWriter()
+for src in ("a.pdf", "b.pdf"):
+    w.append(src)
+with open("merged.pdf", "wb") as f:
+    w.write(f)
+
+# split first 3 pages + rotate one
+w2 = PdfWriter()
+reader = PdfReader("doc.pdf")
+for page in reader.pages[:3]:
+    w2.add_page(page)
+w2.pages[0].rotate(90)
+with open("first3.pdf", "wb") as f:
+    w2.write(f)
+```
+
+## Form filling (AcroForm)
+
+Dump the field names first — guessing them is the #1 reason a fill silently does nothing:
+
+```python
+from pypdf import PdfReader
+
+fields = PdfReader("form.pdf").get_fields() or {}
+for name, f in fields.items():
+    print(name, "->", f.get("/FT"))  # /Tx text, /Btn checkbox/radio, /Ch choice
+```
+
+Then write the values. Set `auto_regenerate=False` and bake with `flatten=True` if it must not be editable:
+
+```python
+from pypdf import PdfReader, PdfWriter
+
+reader = PdfReader("form.pdf")
+writer = PdfWriter()
+writer.append(reader)
+
+for page in writer.pages:
+    writer.update_page_form_field_values(
+        page,
+        {"applicant_name": "Eric Risco", "agree": "/Yes"},  # checkbox = its on-state
+        auto_regenerate=False,  # else a spurious "save changes?" prompt fires on open
+    )
+
+# flatten=True bakes the values and drops the editable widgets
+with open("filled.pdf", "wb") as f:
+    writer.write(f)
+```
+
+Why `auto_regenerate=False`: it defaults to `True` for legacy reasons, which marks the AcroForm dirty and triggers a "you have unsaved changes" prompt when the user opens the PDF. You almost never want that. Checkbox/radio values are the field's `/V` on-state (often `/Yes`), not `True` — read the field to find it.
+
+## Generation
+
+DOCX from a Word template you authored and tagged with Jinja2 (`{{ client }}`, `{% tr for row in items %}` on a table row, `InlineImage` for pictures):
+
+```python
+from docxtpl import DocxTemplate
+
+doc = DocxTemplate("contract_template.docx")
+doc.render({
+    "client": "Acme SL",
+    "date": "2026-06-02",
+    "items": [{"desc": "Audit", "amount": "1.200,00 €"}],
+})
+doc.save("contract_2026-06-02.docx")
+```
+
+PDF from scratch with ReportLab Platypus:
+
+```python
+from reportlab.lib.pagesizes import A4
+from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer
+from reportlab.lib.styles import getSampleStyleSheet
+
+styles = getSampleStyleSheet()
+doc = SimpleDocTemplate("report.pdf", pagesize=A4)
+doc.build([
+    Paragraph("Quarterly Report", styles["Title"]),
+    Spacer(1, 12),
+    Paragraph("Generated automatically from the data dict.", styles["BodyText"]),
+])
+```
+
+## OCR
+
+Branch on cost and privacy. **Local, no API budget, or data must not leave the machine → Docling/Marker.** **Messy scans, handwriting, brutal tables, and an API budget is fine → Mistral OCR.**
+
+Local with Docling (wraps Tesseract / RapidOCR, exports Markdown preserving tables):
+
+```python
+from docling.document_converter import DocumentConverter
+
+result = DocumentConverter().convert("scan.pdf")
+markdown = result.document.export_to_markdown()
+open("scan.md", "w").write(markdown)
+```
+
+Hosted with Mistral OCR (~$2 / 1,000 pages, 50% off via Batch API; outputs interleaved text+images as Markdown):
+
+```python
+from mistralai import Mistral
+
+client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
+resp = client.ocr.process(
+    model="mistral-ocr-2512",
+    document={"type": "document_url", "document_url": signed_url},
+)
+markdown = "\n\n".join(p.markdown for p in resp.pages)
+```
+
+**Never trust OCR output blind.** OCR confuses `0/O`, `1/l/I`, and drops or shifts decimal points — a `1.234,50` can come back as `1234,50` or `1,234.50`. Always spot-check totals, dates, and ID numbers against the rendered page before you hand the text downstream. For clean scans with no budget, plain `pytesseract` is the zero-cost baseline, but it is weak on layout/tables versus the pipelines above.
+
+## Scale and handoff
+
+Batch jobs: parallelize per-file, cap concurrency on the hosted API (rate limits + cost), and use Mistral's Batch API for the 50% discount on large runs. Engine install matrix, exact version pins, the full licensing table, the Docling-vs-Marker-vs-Mistral feature/cost comparison, and troubleshooting (encrypted PDFs, mangled AcroForm field names, multi-column reading order, CJK/handwriting) live in `references/engines.md` — read it before a non-trivial install.
+
+Handoffs:
+- Need typed fields (`{total, due_date, parties}`) out of the Markdown you produced → `structured-extraction`.
+- Need to route the finished PDF for signature with an audit trail → `e-signature`.
+- The grid is really a spreadsheet (cells, formulas, XLSX as data) → `spreadsheet-ops`.
+- Need to index the extracted text for cross-document Q&A → `rag`. This skill *produces* the text `rag` ingests; it does not index it.
+
+## Anti-patterns
+
+| Anti-pattern | Why it is wrong | Do instead |
+|---|---|---|
+| Pipe every PDF straight to OCR | OCR'ing a digital PDF is slow, costs money, and *adds* errors to text you could extract losslessly | Step 0: check the text layer first; OCR only image-only PDFs |
+| `import PyPDF2` | Unmaintained; merged into `pypdf` years ago — a stale-code smell | `from pypdf import PdfReader, PdfWriter` |
+| Recommend PyMuPDF without a word about its license | PyMuPDF is **AGPL**; shipping it silently creates an open-source obligation | Flag AGPL; prefer pdfplumber/pypdf, or get a commercial license knowingly |
+| Leave `auto_regenerate=True` on a form fill | Marks the AcroForm dirty → a spurious "save changes?" prompt for every user | Pass `auto_regenerate=False` |
+| Trust OCR'd totals/numbers as-is | `0/O`, `1/l`, shifted decimals silently corrupt amounts | Spot-check totals/dates/IDs against the page image |
+| Hand-roll a regex to pull typed fields from the Markdown | Brittle, re-implements a sibling, breaks on layout drift | Output clean Markdown, hand it to `structured-extraction` |
+| Use Mistral OCR when the user said "no cloud / local only" | Sends documents off-machine, violating the privacy constraint | Use Docling/Marker + Tesseract/RapidOCR locally |

package/skills/document-processing/evals/README.md

@@ -0,0 +1,3 @@
+# Evals — document-processing
+
+`cases.yaml` is a routing-and-capability fixture, not an automated runner. To use it, take each prompt under `should_trigger` and confirm a router/agent picks `document-processing` for the stated `why`; take each `should_not_trigger` prompt and confirm it routes instead to the named `route_to` sibling (`structured-extraction`, `e-signature`, `spreadsheet-ops`, `rag`, `data-scraper`) — a miss here is a boundary leak, the most common failure for this skill. For `capability`, give the model the `scenario` with only the SKILL.md loaded and grade the response against every line in `must_include` (it must detect the missing text layer, choose a *local* OCR pipeline because the user said no cloud, hand the Markdown to structured-extraction rather than regexing fields, add an OCR spot-check, and use current libs not PyPDF2). If the repo has a top-level eval harness, point it at this file; otherwise read the cases by hand.

package/skills/document-processing/evals/cases.yaml

@@ -0,0 +1,65 @@
+skill: document-processing
+
+# Prompts that MUST load `document-processing`. The skill owns the byte-level
+# work of turning documents into content and data into documents: extract
+# text/tables from PDFs, fill AcroForm fields, generate PDF/DOCX from
+# templates, and OCR image-only files into Markdown/JSON. The deliverable is
+# bytes of a document or the literal content of one.
+should_trigger:
+  - prompt: "Extract all the tables from this scanned invoice PDF into CSV."
+    why: "Table extraction is core; 'scanned' means it must first detect no text layer and OCR, then emit CSV cells — file-in/content-out."
+
+  - prompt: "Fill this PDF application form with these values and flatten it so it can't be edited."
+    why: "AcroForm fill via pypdf update_page_form_field_values + flatten — the canonical form-filling job this skill owns."
+
+  - prompt: "Generate a Word contract from this .docx template and a dict of client fields."
+    why: "docxtpl: author-in-Word template rendered with a context dict — data-in/file-out generation."
+
+  - prompt: "These PDFs copy out as garbled symbols — get me clean text."
+    why: "Non-obvious: the symptom describes a broken/embedded font or image-only PDF, not a feature request; the skill diagnoses the text-layer problem and OCRs or re-extracts."
+
+  - prompt: "Rellena este formulario PDF y luego fusiona los tres documentos en uno solo."
+    why: "Spanish: fill a PDF form then merge three documents into one — form fill + pypdf merge, both owned here."
+
+  - prompt: "Convert 200 scanned pages to Markdown keeping the table structure, locally, with no API."
+    why: "Local batch OCR with table structure preserved -> Docling/Marker; the 'no API / local' constraint forces the local pipeline, not a hosted one."
+
+  - prompt: "Merge these five PDFs, rotate page 2, and rasterize the cover to a PNG."
+    why: "Pure page-level byte ops (merge/rotate/raster) are squarely this skill's pypdf/PyMuPDF territory."
+
+# NEAR-MISS prompts that must NOT load `document-processing`. The recurring
+# trap: anything where the real deliverable is a typed object, a signature
+# workflow, a spreadsheet grid, a retrieval index, or the acquisition of files.
+should_not_trigger:
+  - prompt: "Parse this contract text into a typed JSON schema of parties, dates, and amounts."
+    route_to: "structured-extraction"
+    why: "Deliverable is a schema-typed object, not raw content; this skill produces the Markdown that structured-extraction then types."
+
+  - prompt: "Send this signed PDF to the client for signature with an audit trail."
+    route_to: "e-signature"
+    why: "Signer workflow, routing, and audit trail are e-signature; document-processing only produces or edits the bytes."
+
+  - prompt: "Read the values from B2:D40 of this spreadsheet and sum them."
+    route_to: "spreadsheet-ops"
+    why: "Cells, ranges, and formulas are a spreadsheet grid — spreadsheet-ops, not document bytes."
+
+  - prompt: "Index these PDFs so I can ask questions across all of them."
+    route_to: "rag"
+    why: "Retrieval/embeddings index for Q&A is rag; this skill produces the text rag ingests but does not index it."
+
+  - prompt: "Crawl this site and download every PDF it links to."
+    route_to: "data-scraper"
+    why: "Acquiring documents off the web is data-scraper; processing them once they are on disk is this skill."
+
+# Capability: graded against the must_include rubric.
+capability:
+  - scenario: >
+      The user has a folder of scanned, image-only invoice PDFs (a few pages
+      are handwritten) and wants the line items as CSV. Hard constraint: fully
+      local, no cloud API may be used. Outline the pipeline you would run.
+    must_include:
+      - "Detects or states the PDFs are image-only (no text layer), e.g. via pdfplumber extract_text() returning empty, so OCR is required."
+      - "Picks a LOCAL pipeline (Docling or Marker + Tesseract/RapidOCR), explicitly NOT a hosted API like Mistral OCR, honoring the 'no cloud' constraint."
+      - "Produces Markdown/tables from the OCR, then notes the handoff to structured-extraction for typed line-item fields rather than regexing them here."
+      - "Adds a verification step: spot-check OCR'd totals/numbers against the page image (0/O, 1/l, shifted decimals)."
+      - "Uses current libraries (pypdf / pdfplumber / docling), never PyPDF2."

package/skills/document-processing/references/engines.md

@@ -0,0 +1,67 @@
+# Engines: install, version pins, licensing, troubleshooting
+
+Offloaded depth for `document-processing`. Read before a non-trivial install or when a job hits a quirk.
+
+## Install matrix + version pins
+
+| Library | Install | Pin (current stable, 2026-06) | Notes |
+|---|---|---|---|
+| pypdf | `pip install pypdf` | `pypdf==6.12.2` | Pure-Python, no C deps. Runs in Lambda / slim containers. Core: read/write/merge/split/rotate/forms. |
+| pdfplumber | `pip install pdfplumber` | latest | Layout-aware text + `extract_tables()`. Built on pdfminer.six. |
+| docxtpl | `pip install docxtpl` | `docxtpl==0.20.*` | python-docx-template. Jinja2 over a real `.docx`. Pulls in python-docx + Jinja2. |
+| python-docx | `pip install python-docx` | latest | Programmatic DOCX when you build the doc from code, not from a template. |
+| reportlab | `pip install reportlab` | latest | PDF generation: low-level `canvas` and high-level Platypus flowables. |
+| docling | `pip install docling` | latest | IBM / LF AI local pipeline. First run downloads models. CPU works; GPU is faster. |
+| marker | `pip install marker-pdf` | latest | Datalab pipeline on the Surya engine. PDF/image → Markdown/JSON/HTML. GPU recommended. |
+| pytesseract | `pip install pytesseract` + system Tesseract | latest | Needs the Tesseract binary: macOS `brew install tesseract`, Debian `apt install tesseract-ocr`. Add `tesseract-ocr-<lang>` packs for non-English. |
+| mistralai | `pip install mistralai` | latest | Hosted OCR client. Needs `MISTRAL_API_KEY`. No local model download. |
+| PyMuPDF | `pip install pymupdf` | latest | ⚠️ **AGPL** — see licensing. Fastest extract + easiest page→PNG raster + annotations. |
+
+## Licensing — read before you ship
+
+| Library | License | Implication |
+|---|---|---|
+| pypdf | BSD-3 | Permissive. Ship freely. |
+| pdfplumber | MIT | Permissive. |
+| python-docx / docxtpl | MIT / LGPL-ish | Permissive enough for commercial use. |
+| reportlab | BSD (open-source edition) | Open edition is BSD; "ReportLab PLUS" is the paid product. The `reportlab` PyPI package is the open one. |
+| docling | MIT | Permissive, local, no per-page cost. |
+| marker / surya | check current repo license | Historically had usage-revenue conditions on Surya weights — verify before commercial deployment. |
+| **PyMuPDF** | **AGPL-3.0** | **Network/SaaS use can trigger source-disclosure obligations.** Either comply with AGPL (open your source) or buy Artifex's commercial license. Do not slip it into a closed product silently. |
+| Mistral OCR | Hosted, paid | ~$2 / 1,000 pages; 50% off via Batch API. No code license issue — it is an API. Data leaves the machine. |
+| Tesseract | Apache-2.0 | Permissive, free, local. |
+
+## Local OCR vs hosted: Docling vs Marker vs Mistral
+
+| Dimension | Docling | Marker | Mistral OCR |
+|---|---|---|---|
+| Runs | Local | Local | Hosted API |
+| Cost | Free (compute only) | Free (compute only) | ~$2 / 1k pages (50% off batch) |
+| Best at | Layout + reading order, table structure, many input formats (PDF/DOCX/PPTX/XLSX/HTML/images) | Clean Markdown from PDFs/images, equations, code blocks | Messy scans, handwriting, complex tables, forms |
+| OCR backend | Wraps Tesseract / EasyOCR / RapidOCR | Surya recognition engine | Proprietary (`mistral-ocr-2512`, OCR 3) |
+| Output | Markdown / JSON | Markdown / JSON / HTML | Markdown / JSON, interleaved text+images |
+| Pick when | Data must stay local; batch conversion; varied formats | Local + want deterministic layout parsing | Quality matters more than per-page cost; handwriting; API budget fine |
+
+Decision: **data must stay local → Docling (varied formats) or Marker (PDF→Markdown).** **Handwriting / messy / API budget fine → Mistral OCR.** **Clean scan, zero budget, simple layout → plain `pytesseract`.**
+
+## Troubleshooting
+
+**Encrypted / password PDFs.** `PdfReader` raises on encrypted files. Decrypt first:
+
+```python
+from pypdf import PdfReader
+
+reader = PdfReader("locked.pdf")
+if reader.is_encrypted:
+    reader.decrypt("the-password")  # empty string for owner-locked-but-readable PDFs
+```
+
+Some PDFs are owner-locked (printing/editing disabled) but openable with `reader.decrypt("")`.
+
+**AcroForm fields fill but show blank, or names look mangled.** Dump `get_fields()` and inspect `/FT` and the exact keys — field names are case-sensitive and often have a hierarchy (`section1.name`). For checkbox/radio, the value is the on-state from `/V` (commonly `/Yes`), not `True`. If a fill writes but the viewer renders nothing, the AcroForm may lack `/NeedAppearances`; setting `auto_regenerate=False` and flattening sidesteps appearance-stream issues.
+
+**Multi-column reading order is scrambled.** pdfplumber reads in PDF object order, which can interleave columns. For complex layouts (academic papers, newspapers) prefer Docling/Marker, which do explicit reading-order analysis, over raw pdfplumber text.
+
+**CJK / accented text comes out as boxes or mojibake.** For OCR, install the matching Tesseract language pack (`tesseract-ocr-jpn`, `-chi-sim`, `-cat`, ...) and pass `lang=`. For extraction, the font may lack a ToUnicode map — that is a broken text layer; fall back to OCR. Handwriting: Tesseract is poor; use Mistral OCR.
+
+**Lambda / container cold-start size.** pypdf + pdfplumber are small and pure-Python — fine in a slim image. Docling/Marker pull large model weights — bake them into the image or mount a cache; do not download on every cold start.