devendra-skill-package 1.0.2

package/.github/skills/md-file-converter/SKILL.md

@@ -0,0 +1,291 @@
+---
+name: md-file-converter
+description: Convert any supported file (PDF, DOCX, XLSX, PPTX, images, code, CSV, HTML, JSON, YAML, archives, logs, etc.) into a Markdown (.md) document with zero information loss. Use when asked to convert, transform, or export a file to Markdown format.
+---
+
+# Universal File to Markdown Conversion Skill
+
+## Role
+
+You are a **Universal File-to-Markdown Conversion Engine**.
+
+Convert any supported file into a `.md` document with **zero information loss**.
+You are NOT a summarizer, editor, or formatter — you are a document preservation engine.
+
+---
+
+## Skill Directory Layout
+
+All supporting files for this skill live alongside this SKILL.md:
+
+```
+.github/skills/md-file-generator/
+├── SKILL.md                         ← you are here (orchestration logic)
+├── config/
+│   └── formats.json                 ← format→strategy mapping, chunk sizes, tool priority
+├── extractors/
+│   ├── extract-pdf.sh               ← PDF (pdftotext → pandoc → PyPDF2)
+│   ├── extract-docx.sh              ← DOCX/ODT/RTF (pandoc → unzip+XML)
+│   ├── extract-xlsx.py              ← Excel (openpyxl → xlrd)
+│   ├── extract-pptx.py              ← PowerPoint (python-pptx)
+│   └── extract-eml.py               ← Email (.eml, stdlib only)
+├── scripts/
+│   ├── check-tools.sh               ← detect all available CLI tools
+│   ├── extract-archive.sh           ← ZIP/TAR/7Z/GZ/RAR extraction
+│   ├── get-metadata.sh              ← universal file metadata
+│   └── extract-ocr.sh              ← tesseract → easyocr OCR
+└── templates/
+    ├── output.md                    ← output structure with {{placeholders}}
+    └── unsupported-object.md        ← block template for unreadable objects
+```
+
+---
+
+## Step 1 — Determine Input File
+
+If the user provided a file path, upload, or pasted content → use it directly.
+
+If no file was provided, ask exactly:
+
+> **"Which file would you like me to convert into Markdown? Please upload the file or provide its path."**
+
+Never guess. Never search the project automatically.
+
+---
+
+## Step 2 — Detect Available Tools
+
+Run this first. It determines which extraction strategy to use.
+
+```bash
+bash .github/skills/md-file-generator/scripts/check-tools.sh
+```
+
+Read `config/formats.json` to understand which extractor maps to which format and what fallback order applies.
+
+If Bash is unavailable and the file is binary, tell the user:
+
+> "I cannot read `<filename>` (binary `<format>`) without Bash tool access.
+> Please either enable Bash or paste the file contents directly."
+
+---
+
+## Step 3 — Validate the File
+
+```bash
+test -f "<path>" && echo "EXISTS" || echo "NOT FOUND"
+file --mime-type "<path>"
+wc -c "<path>"
+```
+
+Check if it is an archive (`.zip`, `.tar`, `.gz`, `.7z`, `.rar`).
+Check if it is password-protected (attempt read; catch error).
+
+---
+
+## Step 4 — Extract Metadata
+
+```bash
+bash .github/skills/md-file-generator/scripts/get-metadata.sh "<path>"
+```
+
+Use the output to fill the Metadata section in `templates/output.md`.
+
+---
+
+## Step 5 — Choose Extraction Strategy
+
+Consult `config/formats.json` to classify the file:
+
+| Category      | Strategy                                  | Tool needed     |
+|---------------|-------------------------------------------|-----------------|
+| Text formats  | Read tool directly                        | Read tool       |
+| Binary docs   | Run the matching extractor script         | Bash            |
+| Images        | See OCR Strategy below                    | Vision or Bash  |
+| Archives      | Extract first, then convert each file     | Bash            |
+
+### Text formats (Read tool)
+
+Extensions listed under `text_formats.extensions` in `formats.json`.
+
+Read the file directly with the Read tool. No extractor script needed.
+
+### Binary documents (Bash extractors)
+
+| Format             | Extractor script                         |
+|--------------------|------------------------------------------|
+| PDF                | `extractors/extract-pdf.sh <file>`       |
+| DOCX / DOC / ODT / RTF | `extractors/extract-docx.sh <file>` |
+| XLSX / XLS / ODS   | `python3 extractors/extract-xlsx.py <file>` |
+| PPTX / PPT         | `python3 extractors/extract-pptx.py <file>` |
+| EML                | `python3 extractors/extract-eml.py <file>`  |
+
+Each extractor tries its primary tool first, then falls through its fallback chain automatically. Check `config/formats.json` for the priority order per format.
+
+### Archives
+
+```bash
+bash .github/skills/md-file-generator/scripts/extract-archive.sh "<archive>" "<output_dir>"
+```
+
+After extraction, list all files and convert each one independently using its own strategy. Generate `index.md` mapping the archive tree to converted files.
+
+---
+
+## Step 6 — OCR Strategy
+
+**Never claim OCR was performed when it was not.**
+
+| Situation                              | Action                                              |
+|----------------------------------------|-----------------------------------------------------|
+| Image uploaded directly into chat      | Use Claude vision — extract all visible text        |
+| Image file path + Bash available       | `bash scripts/extract-ocr.sh "<image>" [lang]`      |
+| Neither vision nor Bash available      | Document limitation; do not invent text             |
+
+When vision is used, output:
+
+```markdown
+![Image description](image001.png)
+
+**Extracted Text (Vision):**
+<full text exactly as seen>
+
+**Visual Description:**
+<describe charts, diagrams, layout, colours, icons>
+```
+
+When OCR is not possible, output:
+
+```markdown
+![Image](image001.png)
+
+> **OCR Not Possible:** No vision input and no OCR tool in this session.
+> Upload the image directly into chat, or enable Bash with tesseract installed.
+```
+
+---
+
+## Step 7 — Large File Chunking
+
+Check size before reading:
+
+```bash
+wc -c "<file>"   # bytes
+wc -l "<file>"   # lines (text files)
+```
+
+If the file exceeds **100 KB** of extracted text, process in chunks using thresholds from `config/formats.json`:
+
+| Format      | Default chunk size     |
+|-------------|------------------------|
+| Text files  | 500 lines per chunk    |
+| PDF         | 20 pages per chunk     |
+| Slide decks | 10 slides per chunk    |
+| Spreadsheets| 1 worksheet per chunk  |
+
+Name output files: `document.part01.md`, `document.part02.md`, …
+
+After each part, tell the user:
+
+> "Part N written to `document.partNN.md` (range: X–Y). Type **continue** to process the next chunk."
+
+Wait for confirmation before processing the next chunk.
+
+If context pressure hits mid-conversion, stop at a clean boundary, save the part, and report the resume point. **Never drop content silently.**
+
+---
+
+## Step 8 — Convert to Markdown
+
+Use `templates/output.md` as the structure. Replace all `{{placeholders}}` with real values.
+
+### Formatting rules
+
+| Element      | Markdown syntax                          |
+|--------------|------------------------------------------|
+| Headings     | `# H1` `## H2` `### H3`                  |
+| Bold         | `**bold**`                               |
+| Italic       | `*italic*`                               |
+| Underline    | `<u>text</u>`                            |
+| Strike       | `~~text~~`                               |
+| Blockquote   | `> quote`                                |
+| Hyperlink    | Markdown link syntax — never plain text  |
+| Equation     | `$inline$` or `$$block$$` LaTeX         |
+| Code         | Fenced block with language identifier    |
+
+### Tables
+
+Use Markdown tables where possible. Use HTML `<table>` for merged/spanned cells. Never flatten to plain text.
+
+### Images
+
+Every image must appear. Apply OCR Strategy (Step 6). Never discard images.
+
+### Code / Source files
+
+Never modify, format, lint, or remove comments. Wrap in fenced blocks with the correct language identifier only.
+
+### Unsupported objects
+
+For any object that cannot be converted, use `templates/unsupported-object.md`:
+
+```markdown
+## Unsupported Object
+- **Type:** <type>
+- **Location:** <page/slide/sheet/position>
+- **Reason:** <why>
+- **Original Metadata:** <any available>
+```
+
+---
+
+## Step 9 — Output File Location
+
+Write the `.md` file next to the source file, preserving folder structure:
+
+```
+docs/api/specification.pdf  →  docs/api/specification.md
+```
+
+If `output.md` already exists, generate `output_v2.md`, `output_v3.md`, etc., unless the user explicitly asks to overwrite.
+
+---
+
+## Step 10 — Validation Checklist
+
+Before completing, verify:
+
+- [ ] Tool availability detected and recorded
+- [ ] All pages / slides / sheets processed (or chunked with user confirmation)
+- [ ] Every heading preserved
+- [ ] Every table preserved (not flattened)
+- [ ] Every image represented (OCR or documented limitation)
+- [ ] Archives extracted via Bash (or user notified)
+- [ ] Metadata captured from `get-metadata.sh`
+- [ ] Hyperlinks preserved as Markdown link syntax — never plain text
+- [ ] Code unchanged
+- [ ] Reading order maintained
+- [ ] All technical limitations documented in Conversion Report
+- [ ] No content silently omitted
+
+---
+
+## Zero Information Loss Policy
+
+Never intentionally: summarize, shorten, paraphrase, simplify, rewrite, remove duplicates, skip pages, skip sections, skip images, skip tables, skip metadata.
+
+If content cannot be extracted due to a technical limitation, document it in the Conversion Report. **Never silently drop it.**
+
+---
+
+## Success Message
+
+After conversion, report:
+
+- Source file(s) processed and output file(s) location
+- Extraction method used (Read / Bash extractor / Vision)
+- OCR method used (Vision / tesseract / easyocr / None)
+- Whether archive extraction ran
+- Whether chunking was used and how many parts
+- Any unsupported objects found
+- Any technical limitations encountered

package/.github/skills/md-file-converter/config/formats.json

@@ -0,0 +1,114 @@
+{
+  "text_formats": {
+    "description": "Readable directly with the Read tool — no binary parsing needed",
+    "tool": "read",
+    "chunk_lines": 500,
+    "extensions": [
+      ".txt", ".md", ".markdown",
+      ".csv", ".tsv",
+      ".json", ".yaml", ".yml", ".toml", ".ini", ".properties",
+      ".xml", ".html", ".htm", ".xhtml",
+      ".css", ".scss", ".less", ".sass",
+      ".svg",
+      ".log",
+      ".sql",
+      ".sh", ".bash", ".zsh", ".fish",
+      ".ps1", ".psm1", ".psd1",
+      ".js", ".mjs", ".cjs",
+      ".ts", ".tsx", ".jsx",
+      ".java", ".kt", ".kts",
+      ".swift", ".m", ".mm",
+      ".c", ".h", ".cpp", ".cc", ".cxx", ".hpp",
+      ".cs",
+      ".go",
+      ".rs",
+      ".py", ".pyw",
+      ".php",
+      ".rb",
+      ".dart",
+      ".r", ".rmd",
+      ".lua",
+      ".pl", ".pm",
+      ".ex", ".exs",
+      ".clj", ".cljs",
+      ".hs",
+      ".erl",
+      ".scala",
+      ".groovy",
+      ".tf", ".tfvars",
+      ".dockerfile",
+      ".makefile",
+      ".gitignore", ".env", ".editorconfig",
+      ".bat", ".cmd"
+    ]
+  },
+  "binary_formats": {
+    "description": "Require Bash tool + CLI extractors. Fallback chain tried in order.",
+    "pdf": {
+      "extractor": "extractors/extract-pdf.sh",
+      "chunk_pages": 20,
+      "primary_tool": "pdftotext",
+      "fallbacks": ["pandoc", "python-pypdf2"],
+      "metadata_tool": "pdfinfo"
+    },
+    "docx": {
+      "extractor": "extractors/extract-docx.sh",
+      "primary_tool": "pandoc",
+      "fallbacks": ["unzip+xml-parse"],
+      "extensions": [".docx", ".doc", ".odt", ".rtf"]
+    },
+    "xlsx": {
+      "extractor": "extractors/extract-xlsx.py",
+      "primary_tool": "openpyxl",
+      "fallbacks": ["xlrd"],
+      "chunk_by": "worksheet",
+      "extensions": [".xlsx", ".xls", ".ods"]
+    },
+    "pptx": {
+      "extractor": "extractors/extract-pptx.py",
+      "primary_tool": "python-pptx",
+      "fallbacks": ["unzip+xml-parse"],
+      "chunk_slides": 10,
+      "extensions": [".pptx", ".ppt", ".odp"]
+    },
+    "eml": {
+      "extractor": "extractors/extract-eml.py",
+      "primary_tool": "python-email",
+      "fallbacks": [],
+      "extensions": [".eml", ".msg"]
+    }
+  },
+  "image_formats": {
+    "description": "OCR required. Strategy depends on how the image is provided.",
+    "ocr_extractor": "scripts/extract-ocr.sh",
+    "extensions": [".png", ".jpg", ".jpeg", ".gif", ".bmp", ".tiff", ".tif", ".webp", ".heic"],
+    "strategies": {
+      "vision_upload": "User uploaded image into chat — use Claude vision directly",
+      "file_path_bash": "Image at path + Bash available — run tesseract or easyocr",
+      "no_tools": "Document limitation in Conversion Report, do not invent text"
+    },
+    "ocr_tools_priority": ["tesseract", "easyocr"]
+  },
+  "archive_formats": {
+    "description": "Bash tool required. Extract first, then convert each file.",
+    "extractor": "scripts/extract-archive.sh",
+    "extensions": [".zip", ".tar", ".tar.gz", ".tgz", ".gz", ".7z", ".rar", ".tar.bz2", ".tar.xz"],
+    "commands": {
+      ".zip":     "unzip -o <file> -d <output_dir>",
+      ".tar":     "tar -xf <file> -C <output_dir>",
+      ".tar.gz":  "tar -xzf <file> -C <output_dir>",
+      ".tgz":     "tar -xzf <file> -C <output_dir>",
+      ".tar.bz2": "tar -xjf <file> -C <output_dir>",
+      ".tar.xz":  "tar -xJf <file> -C <output_dir>",
+      ".gz":      "gunzip -k <file>",
+      ".7z":      "7z x <file> -o<output_dir>",
+      ".rar":     "unrar x <file> <output_dir>"
+    }
+  },
+  "chunk_thresholds": {
+    "text_lines_per_chunk": 500,
+    "pdf_pages_per_chunk": 20,
+    "slides_per_chunk": 10,
+    "max_bytes_before_chunking": 102400
+  }
+}

package/.github/skills/md-file-converter/extractors/extract-docx.sh

@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+# DOCX / DOC / ODT / RTF extractor. Tries pandoc → unzip+XML in order.
+# Usage: bash extractors/extract-docx.sh <file>
+
+FILE="$1"
+
+if [ -z "$FILE" ]; then
+  echo "Usage: extract-docx.sh <file>"
+  exit 1
+fi
+
+if [ ! -f "$FILE" ]; then
+  echo "Error: File not found: $FILE"
+  exit 1
+fi
+
+EXT_LOWER=$(echo "${FILE##*.}" | tr '[:upper:]' '[:lower:]')
+
+echo "=== Document Extraction ==="
+echo "File   : $FILE"
+echo "Format : $EXT_LOWER"
+echo ""
+
+# Strategy 1: pandoc (best quality — preserves headings, tables, lists)
+if command -v pandoc >/dev/null 2>&1; then
+  echo "Method: pandoc"
+  echo ""
+  echo "--- Content ---"
+  pandoc "$FILE" -t markdown --wrap=none --extract-media=./extracted-media 2>/dev/null
+  EXIT_CODE=$?
+  if [ $EXIT_CODE -eq 0 ]; then
+    echo ""
+    echo "Extraction Status: SUCCESS (pandoc)"
+    # List any extracted media
+    if [ -d "./extracted-media" ]; then
+      echo ""
+      echo "=== Extracted Media Files ==="
+      find ./extracted-media -type f | sort
+    fi
+    exit 0
+  fi
+  echo "pandoc failed (exit $EXIT_CODE), trying fallback..."
+fi
+
+# Strategy 2: unzip + XML parsing (DOCX/PPTX/XLSX are ZIP archives)
+if [ "$EXT_LOWER" = "docx" ] || [ "$EXT_LOWER" = "odt" ]; then
+  if command -v unzip >/dev/null 2>&1 && command -v python3 >/dev/null 2>&1; then
+    echo "Method: unzip + XML parse"
+    echo ""
+    TMPDIR_XML=$(mktemp -d)
+    unzip -q "$FILE" -d "$TMPDIR_XML" 2>/dev/null
+    echo "--- Content (raw XML text extraction) ---"
+    python3 - "$TMPDIR_XML" <<'PYEOF'
+import os, sys, re
+from xml.etree import ElementTree as ET
+
+base = sys.argv[1]
+
+# DOCX uses word/document.xml; ODT uses content.xml
+candidates = [
+    os.path.join(base, "word", "document.xml"),
+    os.path.join(base, "content.xml"),
+]
+
+ns = {
+    'w':   'http://schemas.openxmlformats.org/wordprocessingml/2006/main',
+    'text':'urn:oasis:names:tc:opendocument:xmlns:text:1.0',
+}
+
+def extract_docx(path):
+    tree = ET.parse(path)
+    root = tree.getroot()
+    for para in root.iter('{http://schemas.openxmlformats.org/wordprocessingml/2006/main}p'):
+        texts = [t.text or '' for t in para.iter('{http://schemas.openxmlformats.org/wordprocessingml/2006/main}t')]
+        line = ''.join(texts).strip()
+        if line:
+            print(line)
+
+def extract_odt(path):
+    tree = ET.parse(path)
+    root = tree.getroot()
+    for elem in root.iter():
+        if elem.text and elem.text.strip():
+            print(elem.text.strip())
+
+for c in candidates:
+    if os.path.exists(c):
+        try:
+            if 'document.xml' in c:
+                extract_docx(c)
+            else:
+                extract_odt(c)
+            break
+        except Exception as e:
+            print(f"XML parse failed: {e}")
+PYEOF
+    rm -rf "$TMPDIR_XML"
+    echo ""
+    echo "Extraction Status: PARTIAL (unzip+XML — formatting not preserved)"
+    exit 0
+  fi
+fi
+
+# Strategy 3: RTF via unrtf
+if [ "$EXT_LOWER" = "rtf" ] && command -v unrtf >/dev/null 2>&1; then
+  echo "Method: unrtf"
+  echo ""
+  echo "--- Content ---"
+  unrtf --text "$FILE" 2>/dev/null
+  echo ""
+  echo "Extraction Status: SUCCESS (unrtf)"
+  exit 0
+fi
+
+echo "Extraction Status: FAILED"
+echo "No document extractor available."
+echo "Install options:"
+echo "  pandoc : https://pandoc.org/installing.html"
+echo "  unrtf  : sudo apt install unrtf"
+exit 1

package/.github/skills/md-file-converter/extractors/extract-eml.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+"""
+Email extractor for .eml files. Uses Python stdlib — no dependencies needed.
+Captures: headers, plain text body, HTML body (converted), attachments list.
+Usage: python3 extractors/extract-eml.py <file.eml>
+"""
+
+import sys
+import os
+import email
+import email.policy
+import html
+import re
+
+FILE = sys.argv[1] if len(sys.argv) > 1 else None
+
+if not FILE:
+    print("Usage: extract-eml.py <file.eml>")
+    sys.exit(1)
+
+if not os.path.exists(FILE):
+    print(f"Error: File not found: {FILE}")
+    sys.exit(1)
+
+print(f"=== Email Extraction ===")
+print(f"File : {FILE}")
+print("")
+
+with open(FILE, "rb") as f:
+    msg = email.message_from_binary_file(f, policy=email.policy.default)
+
+# ── Headers ──────────────────────────────────────────────────────────────────
+print("## Headers\n")
+important_headers = [
+    "From", "To", "Cc", "Bcc", "Reply-To",
+    "Subject", "Date", "Message-ID",
+    "Content-Type", "MIME-Version",
+    "X-Mailer", "User-Agent",
+]
+
+for header in important_headers:
+    val = msg.get(header)
+    if val:
+        print(f"**{header}:** {val}")
+
+# Any additional headers not in the list above
+for key, val in msg.items():
+    if key not in important_headers:
+        print(f"**{key}:** {val}")
+
+print("")
+
+# ── Body ─────────────────────────────────────────────────────────────────────
+plain_body = None
+html_body  = None
+attachments = []
+
+def html_to_text(html_content):
+    """Minimal HTML → plain text strip."""
+    text = re.sub(r'<br\s*/?>', '\n', html_content, flags=re.IGNORECASE)
+    text = re.sub(r'<p\s*/?>', '\n\n', text, flags=re.IGNORECASE)
+    text = re.sub(r'</p>', '', text, flags=re.IGNORECASE)
+    text = re.sub(r'<[^>]+>', '', text)
+    text = html.unescape(text)
+    return text.strip()
+
+if msg.is_multipart():
+    for part in msg.walk():
+        ct   = part.get_content_type()
+        disp = str(part.get("Content-Disposition") or "")
+
+        if "attachment" in disp:
+            filename = part.get_filename() or "unnamed"
+            size     = len(part.get_payload(decode=True) or b"")
+            attachments.append((filename, ct, size))
+        elif ct == "text/plain" and plain_body is None:
+            payload = part.get_payload(decode=True)
+            if payload:
+                plain_body = payload.decode(part.get_content_charset() or "utf-8", errors="replace")
+        elif ct == "text/html" and html_body is None:
+            payload = part.get_payload(decode=True)
+            if payload:
+                html_body = payload.decode(part.get_content_charset() or "utf-8", errors="replace")
+else:
+    payload = msg.get_payload(decode=True)
+    if payload:
+        plain_body = payload.decode(msg.get_content_charset() or "utf-8", errors="replace")
+
+# Prefer plain text; fall back to HTML converted to text
+print("## Body\n")
+if plain_body:
+    print(plain_body)
+elif html_body:
+    print("> *Note: No plain-text body. Converted from HTML.*\n")
+    print(html_to_text(html_body))
+else:
+    print("> No readable body content found.")
+
+print("")
+
+# ── Attachments ───────────────────────────────────────────────────────────────
+if attachments:
+    print("## Attachments\n")
+    for name, content_type, size in attachments:
+        print(f"- **{name}** ({content_type}, {size} bytes)")
+    print("")
+
+print("Extraction Status: SUCCESS (python stdlib email)")