@sitar_fiercer4c/skills 0.1.0

package/skills/murtaza-breif/SKILL.md

@@ -0,0 +1,143 @@
+---
+name: murtaza-breif
+description: Explain what a source file does and how its code actually works, as a two-row Markdown table — one row plain English, one row Hinglish (Hindi in Roman script, casual dev tone) — capped at 50 words per language, then save it into FILE_BRIEFS.md at the project root. Use this skill whenever the user says "murtaza breif", "brief this file", "/murtaza-breif", or points at a file or directory and asks for a quick/short/one-line explanation, a summary, a TL;DR, "what does this file do", "explain this in Hinglish", or "samjhao". Use it even when they only name a path and say "brief" with no other instruction. It explains files and records the briefs; it does not review, refactor, or change code.
+---
+
+# Brief a file, in English and Hinglish
+
+Someone has landed on an unfamiliar file and wants to know, in the time it takes to read a
+tweet, what it is for and how it pulls that off. Two rows: English, then Hinglish. Fifty words
+each, hard cap. Every brief is also recorded in `FILE_BRIEFS.md` at the project root, so the
+document slowly becomes a map of the codebase rather than a one-off answer.
+
+## Pick the file
+
+Use the path the user gave. If they said "this file" with no path, use the file most recently
+opened or discussed in the conversation. If nothing is in play at all, ask which file — guessing
+wastes everyone's time.
+
+If they point at a **directory**, brief every source file in it, one table each, in the order the
+files are listed. Don't ask which one they meant — they asked for the directory.
+
+Read the **whole** file before writing, not the first screenful. Skimming the top and inferring
+the rest from the filename is exactly how these summaries go wrong. If the file imports project
+modules whose behaviour you'd be guessing at, open them too — you are describing what this file
+actually does, and that often lives one hop away.
+
+## What goes in the 50 words
+
+Two things, and roughly this split:
+
+- **~20 words — what it's for.** Why the file exists, who calls it, what it guards or produces.
+- **~30 words — how the code works.** The actual mechanism: the key step, the branch that matters,
+  what it returns or hands off, the failure path.
+
+The "how" gets the bigger share on purpose. Someone can guess "what" from the filename;
+they cannot guess that every auth failure returns the same 401 on purpose. Spend the words on
+what's genuinely non-obvious — a surprising decision, a gotcha, a deliberate trade-off — because
+that is the part that saves the reader from opening the file themselves.
+
+Skip the ceremony: imports, exports, boilerplate, and "this file contains a function that..."
+Lead with the verb.
+
+## Tone
+
+Write like a colleague explaining it at your desk, not like generated documentation. Concrete
+over abstract — "reads the Bearer token from the header" beats "handles authentication concerns."
+
+Keep jargon low. Framework nouns everyone knows (route, token, middleware, database, request) are
+fine and stay in English. But if the file leans on an idiom the reader may not have met — a
+promise chain, a curried wrapper, a stream pipe — spend three words saying what it does in plain
+language rather than naming the pattern and moving on.
+
+## Hinglish is a retelling, not a translation
+
+The most common failure here is translating the English row word by word. That produces stiff,
+robotic Hindi that no developer speaks and nobody enjoys reading.
+
+Instead, explain the same file again from scratch, the way you'd say it out loud to an Indian dev
+sitting next to you. Hindi carries the connective tissue — *matlab, phir, agar, toh, karta hai,
+milta hai, sambhal leta hai*. English carries the technical nouns — *token, request, middleware,
+database, header*. Nobody translates those and translating them makes the sentence unreadable.
+
+Roman script only, never Devanagari. Casual is right; the reader is a peer, not a client. The two
+rows will not be sentence-for-sentence parallel, and that's the point — each should read naturally
+in its own register.
+
+## Count the words
+
+Fifty is a cap, not a target — 40 clear words beat 50 padded ones. Count whitespace-separated
+tokens, ignoring standalone punctuation: a clause-separating em-dash is not a word. A path or
+identifier like `req.user` or `auth.middleware.js` counts as one. Count before you output, and if
+you're over, cut a clause rather than shaving articles into a telegram.
+
+## Save the brief
+
+After showing each table, record it by running the bundled script from the project root:
+
+```bash
+python3 .claude/skills/murtaza-breif/scripts/save_brief.py \
+  --file <path/to/file.ext> \
+  --english "<the English row>" \
+  --hinglish "<the Hinglish row>"
+```
+
+Run it once per briefed file. It writes to `FILE_BRIEFS.md` at the root of whichever git repo you
+are working in, creating the file on first use. Re-briefing a file **replaces** its existing entry
+rather than appending a second one, so the document stays a current index instead of a changelog —
+which means you can safely re-brief a file after it changes.
+
+Use the script rather than editing `FILE_BRIEFS.md` by hand. It enforces the 50-word cap (and
+refuses the write if you're over, which is your cue to cut a clause and rerun), rejects summaries
+containing a `|` that would break the table, and finds the repo root correctly even though
+`.claude/skills` is a symlink into a shared skills repo.
+
+If the script reports an error, fix the summary and rerun it — don't fall back to hand-editing the
+file, because that is how duplicate entries and broken tables get in.
+
+## Output format
+
+Heading naming the file, the table, then a single line confirming where it was saved. Nothing
+else — no preamble, no analysis, no follow-up commentary. When briefing several files, repeat the
+heading-and-table pair for each, and put one save confirmation at the end covering all of them.
+
+```markdown
+## Brief: `<path/to/file.ext>`
+
+| Language | Summary |
+| --- | --- |
+| English | ... |
+| Hinglish | ... |
+
+Saved to `FILE_BRIEFS.md`.
+```
+
+## Examples
+
+**Example 1** — `Backend/src/utils/asyncHandler.js`, a five-line wrapper.
+
+## Brief: `Backend/src/utils/asyncHandler.js`
+
+| Language | Summary |
+| --- | --- |
+| English | Wraps an async route handler so its rejected promises don't vanish. Instead of every controller needing its own try/catch, asyncHandler calls the function, catches any error, and passes it to Express's `next`, which hands it to the central error middleware. Five lines, used everywhere. |
+| Hinglish | Async controller ko wrap karta hai taaki error kahin gum na ho jaaye. Har controller mein alag try/catch likhne ki zaroorat nahi — ye function chalata hai, error aaye toh seedha Express ke `next` ko de deta hai, aur central error middleware sambhal leta hai. |
+
+Saved to `FILE_BRIEFS.md`.
+
+Note what the "how" half buys the reader: the point of the file is the try/catch they now don't
+have to write. That's the insight, so it gets the words.
+
+**Example 2** — `Backend/src/middleware/auth.middleware.js`, a JWT gate.
+
+## Brief: `Backend/src/middleware/auth.middleware.js`
+
+| Language | Summary |
+| --- | --- |
+| English | Gatekeeper for protected routes. Reads the `Bearer` token from the Authorization header, verifies its signature with `JWT_SECRET`, then loads that user from MongoDB. Any failure — missing token, bad token, deleted user — returns the same 401 so attackers learn nothing. On success it attaches `req.user` and continues. |
+| Hinglish | Protected routes ka darban. Authorization header se `Bearer` token nikaalta hai, `JWT_SECRET` se verify karta hai, user ko MongoDB se laata hai. Token missing ho, galat ho ya user delete ho gaya ho — teeno mein same 401, taaki hacker ko hint na mile. Sahi nikla toh `req.user` set hota hai. |
+
+Saved to `FILE_BRIEFS.md`.
+
+The identical-401 detail is a deliberate design decision that is invisible from the filename —
+worth more than restating that it's authentication middleware.

package/skills/murtaza-breif/scripts/save_brief.py

@@ -0,0 +1,128 @@
+#!/usr/bin/env python3
+"""Upsert one file's brief into FILE_BRIEFS.md at the project root.
+
+Re-briefing a file replaces its existing entry rather than appending a
+duplicate, so the document stays a current index instead of a changelog.
+
+Usage:
+    python3 save_brief.py --file Backend/src/config/db.js \
+        --english "..." --hinglish "..."
+"""
+import argparse
+import re
+import sys
+from pathlib import Path
+
+def find_repo_root(start=None):
+    """Walk up from the working directory to the enclosing git repo.
+
+    Deliberately not derived from __file__: .claude/skills is a symlink into a
+    shared skills repo, so resolving the script's own path lands in a different
+    project entirely. The brief belongs to the repo being briefed, which is the
+    one we are running in.
+    """
+    here = (start or Path.cwd()).resolve()
+    for d in [here, *here.parents]:
+        if (d / ".git").exists():
+            return d
+    return here
+
+HEADER = """# File briefs
+
+Short English + Hinglish summaries of individual files, written by the
+`murtaza-breif` skill. Each entry is capped at 50 words per language.
+Re-briefing a file replaces its entry in place.
+"""
+
+WORD_CAP = 50
+
+
+def count_words(s):
+    """Words, ignoring standalone punctuation.
+
+    A bare em-dash between clauses is whitespace-separated but is not a word,
+    and counting it makes a 50-word summary read as 51.
+    """
+    return len([w for w in re.split(r"\s+", s.strip())
+                if w and re.search(r"[A-Za-z0-9]", w)])
+
+
+def render(rel_path, english, hinglish):
+    return (
+        f"## `{rel_path}`\n\n"
+        "| Language | Summary |\n"
+        "| --- | --- |\n"
+        f"| English | {english} |\n"
+        f"| Hinglish | {hinglish} |\n"
+    )
+
+
+def upsert(BRIEFS, rel_path, english, hinglish):
+    entry = render(rel_path, english, hinglish)
+
+    if not BRIEFS.exists():
+        BRIEFS.write_text(HEADER + "\n" + entry)
+        return "created"
+
+    text = BRIEFS.read_text()
+
+    # Match this file's section up to the next "## " heading or end of file.
+    pattern = re.compile(
+        r"^##\s+`" + re.escape(rel_path) + r"`\s*$.*?(?=^##\s|\Z)",
+        re.M | re.S,
+    )
+    if pattern.search(text):
+        text = pattern.sub(entry + "\n", text, count=1)
+        BRIEFS.write_text(text)
+        return "updated"
+
+    if not text.endswith("\n"):
+        text += "\n"
+    BRIEFS.write_text(text + "\n" + entry)
+    return "added"
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--file", required=True,
+                    help="Path to the briefed file, absolute or repo-relative")
+    ap.add_argument("--english", required=True)
+    ap.add_argument("--hinglish", required=True)
+    ap.add_argument("--repo-root", default=None,
+                    help="Override the project root (defaults to the enclosing git repo)")
+    args = ap.parse_args()
+
+    REPO_ROOT = Path(args.repo_root).resolve() if args.repo_root else find_repo_root()
+    BRIEFS = REPO_ROOT / "FILE_BRIEFS.md"
+
+    p = Path(args.file)
+    if p.is_absolute():
+        try:
+            rel = p.resolve().relative_to(REPO_ROOT).as_posix()
+        except ValueError:
+            sys.exit(f"error: {p} is outside the project at {REPO_ROOT}")
+    else:
+        rel = p.as_posix()
+
+    if not (REPO_ROOT / rel).exists():
+        sys.exit(f"error: no such file in project: {rel}")
+
+    english = " ".join(args.english.split())
+    hinglish = " ".join(args.hinglish.split())
+
+    if "|" in english or "|" in hinglish:
+        sys.exit("error: summaries cannot contain '|' — it breaks the table")
+
+    ew, hw = count_words(english), count_words(hinglish)
+    over = [f"{name} is {n} words" for name, n in
+            (("English", ew), ("Hinglish", hw)) if n > WORD_CAP]
+    if over:
+        sys.exit(f"error: over the {WORD_CAP}-word cap ({'; '.join(over)}). "
+                 "Cut a clause and rerun.")
+
+    action = upsert(BRIEFS, rel, english, hinglish)
+    print(f"{action}: {rel} ({ew}/{hw} words) -> {BRIEFS}")
+
+
+if __name__ == "__main__":
+    main()

package/skills/pdf-to-json/SKILL.md

@@ -0,0 +1,42 @@
+---
+
+name: pdf-to-json
+description: Use and invoke this skill whenever a PDF must be converted into structured JSON/Markdown, including catalogues, price lists, quotes, invoices, BOQs, tables, or product images.
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+
+# PDF to JSON
+
+**Goal:** Convert PDF → JSON + Markdown using Datalab with reliable tables, categories, and optional images.
+
+**Workflow**
+
+1. Submit PDF to Datalab and poll until complete.
+2. Process typed blocks recursively: Page, Text, SectionHeader, ListGroup, Table, Picture, Figure, Equation, Code.
+3. Preserve pixel `bbox` as `[x0,y0,x1,y1]`; never treat it as spreadsheet coordinates.
+4. Render Markdown locally from block HTML; do not request a second conversion.
+5. Repair fragmented/misaligned tables, carry valid headers, remove repeated header rows.
+6. Infer categories only from document evidence: cell, rowspan, margin label, then carry-forward.
+7. Use `--page-images` for embedded/in-cell images; map images by physical SKU position, not equal row bands.
+8. Validate and report missing, inferred, unanchored, ambiguous, and rejected data.
+
+**Commands**
+`python scripts/pdf_extract.py "<pdf-or-dir>" [--out-dir DIR] [--mode balanced]`
+`python scripts/load_mongo.py "<abs-path>/<stem>.json" --collection <name>`
+
+**Rules**
+
+* Datalab `block_type` is authoritative; block IDs are unstable.
+* Keep Datalab-generated image descriptions marked `datalab-generated`.
+* Never assume table fragments share schemas or that equal widths share meaning.
+* Carry headers only from tables containing headers + rows; use `none` when alignment is uncertain.
+* Preserve `category_raw` and `category_fill`; never blindly forward-fill categories.
+* Use rendered-page crops for images to preserve masks/backgrounds.
+* Never interpolate rows whose SKU cannot be located; count them as unanchored.
+* Cache raw Datalab responses; use `--refresh` when source/options change.
+* Use `--force-ocr` for scanned/garbled PDFs; `--use-llm` only for difficult merged-cell tables.
+* Never run MongoDB writes automatically; output the command with an explicit collection placeholder.
+* Use `--dry-run`/`--check` before writes and verify the collection afterward.
+
+**Validation:** Report tables/rows, stated vs carried categories, failed pages, dropped headers, alignment decisions, image coverage, unanchored rows, duplicates/rejects, and judgement calls.
+
+**Wrong Tool:** XLSX/XLS/CSV → `datalab-api`; plain text-only PDF → `pdftotext`; use Datalab when structure, tables, or images matter.

package/skills/pdf-to-json/references/output-schema.md

@@ -0,0 +1,168 @@
+# Shape of the generated `<stem>.json`
+
+Read this when you need to consume the JSON programmatically rather than just
+hand it to someone. Everything here is produced by `scripts/pdf_extract.py`.
+
+```jsonc
+{
+  "source": {
+    "file": "price-list.pdf",
+    "path": "/abs/path/price-list.pdf",
+    "bytes": 1776420,
+    "sha256": "…",                  // identifies the exact input across re-runs
+    "extracted_at": "2026-09-05T06:39:41+00:00"
+  },
+
+  "extraction": {
+    "provider": "datalab.to/api/v1/convert",
+    "output_format": "json",
+    "mode": "fast",                 // fast | balanced | accurate
+    "page_range": null,             // whatever --pages was given
+    "max_pages": 6,
+    "force_ocr": false,
+    "use_llm": false,
+    "request_id": "…",              // quote this when reporting a bad extraction
+    "runtime_seconds": 24.7,        // ~0.2 if Datalab served it from its own cache
+    "cost_cents": 3.0,              // 0 on a server-side cache hit
+    "page_count": 6,
+    "failed_pages": []              // non-empty means data is missing, not just messy
+  },
+
+  "summary": {
+    "pages": 6,
+    "blocks": 26,
+    "tables": 9,
+    "figures": 7,
+    "text_characters": 998,
+    "block_types": { "Page": 6, "Table": 9, "Picture": 7, "Text": 8 },
+    "table_rows": 213,                        // data rows across every table
+    "header_sources": { "own": 3, "carried-trimmed": 7 },
+    // The next two keys are absent when no table has a grouping column.
+    "category_fill": { "own": 7, "rowspan": 124, "margin-label": 53,
+                       "carried-forward": 5, "carried-back": 24 },
+    "categories": { "Screw Drivers": 86, "Levels": 32 },  // rows per group
+    // Present only with --page-images. `rows_without_image` are rows the
+    // document itself gives no photo; `rows_unanchored` are rows whose SKU
+    // could not be found on the page, so they were skipped rather than guessed.
+    "page_images": { "extracted": 328, "dpi": 200, "rows_with_image": 785,
+                     "rows_without_image": 27, "rows_anchored": 801,
+                     "rows_unanchored": 11, "unplaced_images": 9 }
+  },
+
+  "pages": [
+    {
+      "page": 1,                            // 1-based, unlike Datalab's 0-based ids
+      "size": { "width": 1064, "height": 1484 },
+      "blocks": [
+        { "id": "/page/0/SectionHeader/0", "type": "SectionHeader",
+          "bbox": [122, 77, 929, 163], "text": "…" },
+        { "id": "/page/0/Table/1", "type": "Table",
+          "bbox": [74, 246, 995, 310], "rows": 64, "columns": 9,
+          "has_header": true },
+        { "id": "/page/0/Picture/2", "type": "Picture",
+          "bbox": [104, 136, 2023, 1307],
+          "images": ["2dfa…_img.jpg"],
+          "description": "Stanley logo on yellow",   // NOT text from the PDF
+          "description_source": "datalab-generated" }
+      ]
+    }
+  ],
+
+  "tables": [
+    {
+      "id": "/page/1/Table/0",
+      "page": 2,
+      "bbox": [74, 246, 995, 310],
+      "columns": 9,                 // after realignment — matches headers
+      "columns_raw": 8,             // what the block itself had
+      "row_count": 64,              // data rows only — never includes the header
+      "has_header": true,           // did this block carry its own <th> row
+      "header_source": "own",       // own | carried:<id> | carried-trimmed:<id> | none
+                                    // `carried` means the fragment fit unchanged;
+                                    // both are content-checked before being accepted
+      // present only when columns had to be moved; `dropped_columns` indexes
+      // into the block's own grid and is only ever entirely-empty columns
+      "alignment": { "mode": "pad", "left": 1, "right": 0,
+                     "dropped_columns": [], "confidence": 0.9 },
+      "dropped_header_rows": 1,     // reprinted header lines removed from the body
+      "group_column": { "index": 0, "name": "Category" },
+      // --page-images only, positional like `categories`: the photos beside
+      // row i. A photo merged over a run of products repeats on each row.
+      "row_images": [["p02_STHT33463-8.png"], []],
+      "image_column": 8,            // which column the PDF drew the photo in
+
+      "headers": ["Category", "Sku", "…", ""],
+      "rows_data": [["Short Tapes", "STHT33203-8", "…"]],   // realigned, as printed
+      "categories":    ["Short Tapes", "Short Tapes"],      // filled, one per row
+      "category_raw":  ["Short Tapes", ""],                 // what the cell printed
+      "category_fill": ["own", "rowspan"],
+      "records": [{ "Category": "Short Tapes", "Sku": "STHT33203-8",
+                    "category": "Short Tapes", "category_raw": "Short Tapes",
+                    "category_fill": "own" }]
+    }
+  ],
+
+  "figures": [
+    // Datalab's own Picture/Figure blocks. `file` appears with --page-images,
+    // which renders them from the page so the Markdown link always resolves.
+    { "name": "2dfa…_img.jpg", "page": 1, "block_id": "/page/0/Picture/2",
+      "bbox": [104, 136, 2023, 1307], "source": "datalab",
+      "file": "price-list_images/2dfa….png", "file_source": "pdf-render" },
+
+    // --page-images only: cut out of the rendered page, not from Datalab.
+    { "name": "p02_STHT33463-8.png", "page": 2, "source": "pdf-render",
+      "file": "price-list_images/p02_STHT33463-8.png",
+      "bbox_pdf": [422.1, 175.7, 470.9, 212.8],   // PDF points
+      "bbox": [733, 305, 818, 370],               // same box in Datalab pixels
+      "rows": [{ "table": "/page/1/Table/0", "row": 3, "sku": "STHT33463-8" }] }
+  ]
+}
+```
+
+## Reading it without getting caught out
+
+`rows_data` is the authoritative row list; `records` is a convenience that is
+**absent whenever headers are duplicated**, so branch on its presence rather
+than assuming it. Write `table.get("records") or rows_via(table["headers"],
+table["rows_data"])`, never `table["records"]`.
+
+To count line items across a document, use `summary.table_rows`, or sum
+`row_count` over the tables — the header row is already excluded everywhere, and
+reprinted header lines have been removed, so there is no off-by-one to correct.
+Tables whose `header_source` starts with `carried` are continuations of an
+earlier table, so a document with one visual table can still list ten entries
+here. After realignment every table in a chain has the same `columns`, so
+grouping consecutive tables by `columns` recovers the logical table.
+
+`rows_data` is realigned to match `headers`, not a verbatim copy of the block's
+own grid — `columns_raw` and `alignment` record what was changed. The transform
+is lossless: padding inserts empty cells for columns the fragment never had, and
+trimming only ever removes columns that were entirely empty.
+
+The three category arrays are positional — index `i` of `categories`,
+`category_raw` and `category_fill` describes row `i` of `rows_data`. They are
+absent on tables with no grouping column. Filter on `category_fill` when the
+distinction matters: `own`, `rowspan` and `margin-label` come from the document,
+while `carried-forward` and `carried-back` are inferred from neighbouring rows.
+
+`bbox` is `[x0, y0, x1, y1]` in page pixels, top-left origin. Divide by the
+page's `size` to get fractions if you need coordinates that survive a different
+render resolution.
+
+Only prose blocks have `text`. Picture and Figure blocks have `description`
+instead, and it is model-generated rather than read off the page — that naming
+split is deliberate, so a pass that collects `text` across blocks cannot
+accidentally sweep generated content into the document body.
+
+Figure `name` values match files written under `<stem>_images/` when the script
+ran with `--keep-images`; without that flag the names are still recorded, the
+pixels just aren't on disk. Branch on `file`, not `name`: an entry with `file`
+has bytes you can open, and `source` says whether Datalab found the figure or
+the picture was cut out of the page by `--page-images`.
+
+`figures[].rows` and `tables[].row_images` are the same relation read from
+either end, so a catalogue import can walk products and ask for their photos, or
+walk photos and ask which products share them. `rows_without_image` in the
+summary counts rows the *document* gives no picture — it is a fact about the
+PDF, not a failure, and it is the number to quote before someone assumes every
+line item arrived with an image.