srm-report 0.0.0-stage → 1.0.0

package/references/rules.md

@@ -0,0 +1,105 @@
+# Rules and lessons
+
+These come from real report sessions and the signed reports compared against them. The attached template overrides the formatting defaults in section 1. The corrections in section 2 came from the user and apply to every report.
+
+## 1. Formatting defaults (21CSP302L / 21CSE734P template)
+
+| Item | Rule |
+|---|---|
+| Font | Times New Roman throughout (code may use Courier New) |
+| Chapter number and title | 16pt bold caps, centred, every chapter on a new page |
+| Sub-heading (1.1) | 16pt bold Title Case |
+| Sub-sub-heading (1.1.1) | 12pt bold capitals |
+| Body | 12pt, 1.5 spacing, justified, space between paragraphs, at most 12 lines a paragraph |
+| Page | A4, left margin 1 inch, right 0.5 inch |
+| Figures | "Fig X.Y" centred, caption below |
+| Tables | "Table X.Y" centred, caption above |
+| Page numbers | Bottom centre. Lower roman on front matter, arabic from Chapter 1 (page 1) to the end |
+| Citations | [1], [4,5], [7-10], numbered by first use, listed in the same order, at least 15 |
+| Length | At least 30 pages, excluding code and screenshots |
+| Plagiarism | Turnitin through the guide, 10% or less; publication proofs where publications are claimed |
+
+Front order: cover, Own Work Declaration, Bonafide, Acknowledgements, Abstract, Table of Contents, List of Figures, List of Tables, Abbreviations.
+
+Older departmental sheets online mention 15% plagiarism and other margins, and SRM NCR uses different margins again. They are not sources for this campus. Trust the template.
+
+### Reading template notes
+
+- A note such as `<Font Size 18> <1.5 line spacing>` is an instruction for the text above it. Apply it, then delete it. `apply_cover_notes()` does this for the cover. `<Font Style ... Bold>` at the top of the cover covers the whole cover.
+- `<Font Style Times New Roman, Font Size 14>` near the abstract is not followed in the signed report or in the accepted report: abstract heading 16pt, body 12pt. Keep 12pt unless the user says otherwise.
+- A note's paragraph may also hold the logo, so check for an image before deleting it.
+- Negative character spacing in template runs squeezes words together on the page ("requirementsfor"). The text is fine, so the fix is removing the spacing.
+
+### Template errors to fix quietly, and mention
+
+Chapter numbering jumps from 3 to 6; "2.5" used twice; "EXECTION"; "compiles with" and "Compiled with" (should be "complies" and "Complied"); no full stop on the last acknowledgement line.
+
+## 2. Corrections from the user
+
+**Fixed pages**
+- Template pages must be exact ("ye wala page exact bana"). Fill a copy; never recreate them.
+- Bonafide fits on one page. Signature block centred in its box. Each name has its designation, with the same line count in each column.
+- Footer is the page number only, no rule line.
+- A page border only if asked.
+
+**Names and labels**
+- The label is GUIDE / CO-GUIDE, not SUPERVISOR. The senior designation is GUIDE and goes first (Associate before Assistant).
+- Get faculty spelling and designation right, and confirm them with the user. All faculty are named everywhere.
+- Faculty Advisors are named individually, in the same order as the students.
+- Names are bold in the acknowledgements.
+- The template's Vice-Chancellor, Dean and Chairperson names and its "his/her" may be out of date, or wrong for the person now in the post. Ask the user to confirm them.
+
+**Voice**
+- "We", never "I", everywhere, including the declaration and bonafide ("under our supervision", "best of our knowledge"). User stories in a backlog stay in the first person.
+
+**Course and degree**
+- The course code from the user or title sheet beats the template (21CSE734P vs 21CSP302L), and must be identical on cover and bonafide.
+- Core CSE drops the "with specialization in" line. Never print `[SPECIALIZATION NAME]`; a signed report did, so don't copy mistakes from examples.
+- Replace degree and department everywhere the same way. The template says Computational Intelligence, which may not be the user's department.
+
+**Tables and contents**
+- No table crosses a page ("pages mei ek bhi table katna nahi"). White space at a page end is acceptable.
+- TOC filled with real page numbers. Chapter lines bold, sub-topics not.
+
+**Appendices**
+- Remove placeholder appendices when asked ("appendix b hata de"). With no publications, drop Conference and Journal and re-letter the rest.
+- Code listings longer than a page are split at logical boundaries into A.2(a), (b), (c). Keep each to about 55 lines; 74 was still too long.
+
+**Truth**
+- "saare rules updated hai na? real hai na sab?" Everything must be real. See `content.md`.
+
+**Images**
+- Don't crop preparation screenshots ("crop nahi kar").
+- A photo goes in only after checking it matches the session and date.
+- Take screenshots from a separate fixture-mode instance, never the user's live backend or quota.
+- Figures: white background, Times New Roman, print-friendly.
+
+**Before printing**
+- Print from the PDF, not the .docx.
+- Checklist for the user: Turnitin report, HoD name, month on the cover, hand signatures.
+- Compare against a signed report if the user has one.
+
+## 3. Failures already hit, and what prevents them
+
+| Symptom | Cause | Prevention |
+|---|---|---|
+| Word says the file is corrupt | XML children out of schema order | `schema_order()` runs in `save()` |
+| Corrupt file after copying a code box | Copied after trimming it | Keep a pristine deepcopy first |
+| A word jumps to the next page | Cloned paragraph carried a `w:sectPr` | `clone_after` strips it |
+| "DepartmentAssistant Professor," | Stale runs left behind after editing one run | `set_text` removes other runs |
+| Three page numbers in one footer | Looping over a linked footer | Unlink and clear the footer first |
+| TOC lists cover lines | Cover uses Heading styles, and a `\o` TOC picks them up | Custom styles and `TOC \t` |
+| Bold TOC chapter lines also bold the List of Figures | Lists share toc 1 | Lists use toc 4 |
+| Title printed twice on the cover | A broad `replace()` hit the already filled cover | Edit the one paragraph, not the whole document |
+| Logo missing from the cover | Logo sat in a note paragraph that got deleted | Check for images first; the checker now tests this |
+| Blank page before Abstract | Empty lines before the cut spilled over | `cut_from` trims them |
+| Bonafide spills to a second page | Note lines kept as blanks | Delete them; the checker now tests this |
+| One name alone on a page | The author list overflowed the acknowledgements | Trim blank lines above it; the checker now flags stray-line pages |
+| Acknowledgements starts on the bonafide page | Template reaches the next page with blank lines alone | `start_new_page()` gives a real break; the checker tests main headings |
+| Paragraphs on one page have different widths | Removed `w:ind`; the page depends on left and right indents | `tidy_paragraphs()` removes only first-line offsets |
+| Checker reports results for the previous build | Word pass failed, and the old PDF was still there | `word_pass.ps1` deletes the old PDF first |
+| "Device or resource busy", `~$` file | The user has the file open in Word | Work on a scratch copy; never kill Word |
+| cp1252 errors in the console | Windows default encoding | Run Python with `-X utf8` |
+| Shell errors on long inline scripts | Quoting in heredocs | Write the script to a file first |
+| Can't read a source PDF through a web fetch | Binary file | Save it and read it with PyMuPDF |
+| Word windows popping up | Visible automation | Keep `Visible=false`; never kill by window title |

package/scripts/check_report.py

@@ -0,0 +1,205 @@
+#!/usr/bin/env python3
+"""Check a built report against the rules. Exit code 1 if anything FAILs.
+
+usage: python -X utf8 check_report.py report.docx report.pdf [checks.json]
+
+checks.json (all optional; take the values from the template's rules, not from memory):
+  {"must_contain": ["names, reg numbers, course code"], "must_not_contain": ["SUPERVISOR", "old spelling"],
+   "min_refs": 15, "min_pages": 30, "body_pt": 12, "layout": "layout.json", "font": "Times New Roman",
+   "max_para_chars": 1250, "allow_first_person": false}
+"""
+import json, re, sys
+from collections import Counter
+import docx, fitz
+from docx.oxml.ns import qn
+
+LEFTOVER = re.compile(r"<<[^<>]*>>|<\s*Font[^<>]*>|<\s*[lI1]\.5 line spacing>|<Italic>|\.{6,}(?=\s*$)|STUDENT ?\d|REG NUM|\(GUIDE NAME|\(Designation, Department\)|"
+                      r"SPECIALIZATION NAME|PROJECT TITLE|TITLE OF THE (?:REPORT|COMMUNITY)|Name of the [Ss]tudent|Dr\. ?ABCD|Specimen|Update this field|Error! |No table of (?:contents|figures) entries|Subtitle 1")
+FIRST = re.compile(r"(?<![\w/])(I|my|My|me|mine|myself)(?![\w/.-])")
+ROMAN_LABEL = re.compile(r"(Sprint|SPRINT|Phase|Part|Type|Review|Table|Class|Stage|Level|Tier|Unit|Module)\s+I\b")
+results = []
+
+
+def report(level, name, detail=""):
+    results.append(level)
+    print(f"{level:4s} {name}{': ' + detail if detail else ''}")
+
+
+def spans(pdf):
+    for p in pdf:
+        for b in p.get_text("dict")["blocks"]:
+            for l in b.get("lines", []):
+                for sp in l["spans"]:
+                    if sp["text"].strip():
+                        yield sp
+
+
+def main(docx_path, pdf_path, cfg):
+    d = docx.Document(docx_path)
+    body = d.element.body
+    pdf = fitz.open(pdf_path)
+    paras = [("".join(t.text or "" for t in p.iter(qn("w:t"))), p) for p in body.iter(qn("w:p"))]
+    style = lambda p: (p.xpath("./w:pPr/w:pStyle/@w:val") or [""])[0]
+    alltext = "\n".join(t for t, _ in paras)
+    pdf_text = [p.get_text(sort=True) for p in pdf]   # sort: Word writes the footer first in the content stream
+    pdf_all = "\n".join(pdf_text)
+
+    hits = sorted({m.group(0) for m in LEFTOVER.finditer(pdf_all)})
+    report("FAIL" if hits else "ok", "template placeholders and specimen text left", ", ".join(hits[:12]))
+
+    flat = pdf_all.replace("\n", " ").lower()
+    for s in cfg.get("must_contain", []):
+        if s.lower() not in flat and s.lower() not in alltext.lower():
+            report("FAIL", "missing from report", s)
+    for s in cfg.get("must_not_contain", []):
+        if s.lower() in alltext.lower() or s.lower() in pdf_all.lower():
+            report("FAIL", "must not appear", s)
+
+    if not cfg.get("allow_first_person"):
+        fp = []
+        for t, p in paras:
+            if style(p) in ("RptCode", "RptRef") or t.strip() == "I":   # a lone I is a Sprint I table cell
+                continue
+            for m in FIRST.finditer(t):
+                ctx = t[max(0, m.start() - 25):m.end() + 25]
+                if m.group(1) == "I" and ROMAN_LABEL.search(ctx):
+                    continue
+                if re.match(r"\s*As an? ", t):   # user stories are written in the first person on purpose
+                    continue
+                fp.append(ctx.strip())
+        report("WARN" if fp else "ok", "first person ('I', 'my') outside code: team reports say 'We'", " | ".join(fp[:6]))
+
+    want = cfg.get("font", "Times New Roman").replace(" ", "").lower()
+    fonts = Counter(); sizes = Counter()
+    for sp in spans(pdf):
+        fonts[sp["font"].split("+")[-1].split(",")[0].split("-")[0]] += len(sp["text"])
+        if len(sp["text"].strip()) > 3:
+            sizes[round(sp["size"] * 2) / 2] += len(sp["text"])
+    other = {k: v for k, v in fonts.items() if want not in k.lower().replace("psmt", "").replace("ps", "") and not re.search("courier|symbol|wingdings", k, re.I)}
+    total = sum(fonts.values()) or 1
+    report("WARN" if sum(other.values()) > total * 0.002 else "ok", f"font is {cfg.get('font', 'Times New Roman')} (code may use Courier)", str(other) if other else "")
+    top = sizes.most_common(1)[0][0] if sizes else 0
+    if "body_pt" in cfg:
+        report("ok" if top == cfg["body_pt"] else "FAIL", f"body text size is {cfg['body_pt']}pt", f"most common size {top}pt")
+
+    # figures and tables
+    fig, tab, prob = [], [], []
+    kids = list(body.iterchildren())
+    has_pic = lambda e: e.find(".//{http://schemas.openxmlformats.org/drawingml/2006/main}blip") is not None
+    for i, e in enumerate(kids):
+        if e.tag != qn("w:p"):
+            continue
+        t = "".join(x.text or "" for x in e.iter(qn("w:t")))
+        m = re.match(r"(?:Fig|Figure)\.?\s*([A-Z0-9]+)\.(\d+)\s*:", t)
+        if m and style(e) == "FigCaption":
+            fig.append((m.group(1), int(m.group(2))))
+            if i == 0 or not has_pic(kids[i - 1]):
+                prob.append(f"{t[:30]} is not directly below a figure")
+        m = re.match(r"Table\s*([A-Z0-9]+)\.(\d+)\s*:", t)
+        if m and style(e) == "TabCaption":
+            tab.append((m.group(1), int(m.group(2))))
+            if i + 1 >= len(kids) or kids[i + 1].tag != qn("w:tbl"):
+                prob.append(f"{t[:30]} is not directly above a table")
+    for name, seq in (("Fig", fig), ("Table", tab)):
+        last = {}
+        for ch, n in seq:
+            if n != last.get(ch, 0) + 1:
+                prob.append(f"{name} {ch}.{n} breaks the numbering")
+            last[ch] = n
+    report("FAIL" if prob else "ok", f"captions: {len(fig)} figures below, {len(tab)} tables above, numbered per chapter", "; ".join(prob[:6]))
+
+    # citations
+    ref_at = max((i for i, (t, _) in enumerate(paras) if t.strip().upper() == "REFERENCES"), default=len(paras))
+    ref_paras = [t for t, _ in paras[ref_at:] if re.match(r"\[\d+\]", t)]
+    ref_nums = [int(re.match(r"\[(\d+)\]", t).group(1)) for t in ref_paras]
+    order, seen = [], set()
+    for t, p in paras[:ref_at]:
+        if style(p) == "RptCode":
+            continue
+        for m in re.finditer(r"\[(\d+(?:\s*[,-]\s*\d+)*)\]", t):
+            for part in m.group(1).split(","):
+                part = part.strip()
+                a, b = part.split("-") if "-" in part else (part, part)
+                for n in range(int(a), int(b) + 1):
+                    if n not in seen:
+                        seen.add(n); order.append(n)
+    prob = []
+    if ref_nums != list(range(1, len(ref_nums) + 1)):
+        prob.append("reference list is not numbered 1..N")
+    if order != sorted(order):
+        prob.append("citations do not first appear in order 1, 2, 3...")
+    un = sorted(set(ref_nums) - seen); no = sorted(seen - set(ref_nums))
+    if un:
+        prob.append(f"never cited: {un[:10]}")
+    if no:
+        prob.append(f"cited but not listed: {no[:10]}")
+    mn = cfg.get("min_refs", 0)
+    if len(ref_nums) < mn:
+        prob.append(f"only {len(ref_nums)} references, template asks for {mn}")
+    report("FAIL" if prob else "ok", f"citations: {len(ref_nums)} references", "; ".join(prob))
+
+    limit = cfg.get("max_para_chars", 1250)
+    long_p = [t[:50] for t, p in paras if style(p) in ("RptBody", "") and len(t) > limit]
+    report("WARN" if long_p else "ok", f"paragraphs near the 12-line limit (over {limit} characters)", " | ".join(long_p[:5]))
+
+    # layout from the PDF
+    def foot(pg):
+        lines = [x.strip() for x in pg.get_text(sort=True).splitlines() if x.strip()]
+        return lines[-1] if lines else ""
+
+    chap = [i for i, t in enumerate(pdf_text) if re.search(r"^\s*CHAPTER\s+\d+\s*$", t, re.M) and "TABLE OF CONTENTS" not in t and not re.search(r"\.{5,}", t)]
+    late = [i + 1 for i in chap if not re.match(r"\s*CHAPTER\s+\d+", pdf_text[i])]
+    report("WARN" if late else "ok", f"{len(chap)} chapters each start at the top of a new page", f"physical pages {late}" if late else "")
+    if chap:
+        first = chap[0]
+        seq = [foot(pdf[i]) for i in range(first, len(pdf))]
+        bad = [i for i, s in enumerate(seq) if s != str(i + 1)]
+        report("FAIL" if bad else "ok", "arabic page numbers start at 1 on Chapter 1 and run to the end", f"at position {bad[0] + 1} the footer reads {seq[bad[0]]!r}" if bad else "")
+        rom = [i + 1 for i in range(1, first) if not re.fullmatch(r"[ivxlc]+", foot(pdf[i]))]
+        report("WARN" if rom else "ok", "front pages show roman numerals", f"physical pages without one: {rom}" if rom else "")
+    blank = [i + 1 for i, p in enumerate(pdf) if len(pdf_text[i].strip()) < 5 and not p.get_images()]
+    report("FAIL" if blank else "ok", "no blank pages", str(blank) if blank else "")
+    # One stray line on its own page: a fixed page overflowed, as an author list did in testing.
+    stub = [i + 1 for i, p in enumerate(pdf) if 5 <= len(re.sub(r"\s+", "", pdf_text[i])) < 60 and not p.get_images()]
+    report("FAIL" if stub else "ok", "no page holding only a stray line", str(stub) if stub else "")
+    if cfg.get("min_pages"):
+        report("ok" if len(pdf) >= cfg["min_pages"] else "FAIL", f"at least {cfg['min_pages']} pages", f"{len(pdf)} pages (the template excludes code and screenshots from this count)")
+
+    if cfg.get("layout"):
+        lay = json.load(open(cfg["layout"], encoding="utf-8-sig"))
+        tabs = lay.get("tables") or []
+        tabs = tabs if isinstance(tabs, list) else [tabs]
+        sp = [t for t in tabs if t["start"] != t["end"]]
+        report("FAIL" if sp else "ok", f"no table breaks across pages ({len(tabs)} tables)", "; ".join(f"pages {t['start']}-{t['end']}: {t['text'][:30]}" for t in sp[:6]))
+
+    # Main headings open a page. A template that paginates with blank lines loses this silently when text moves.
+    misplaced = []
+    for h in ("ACKNOWLEDGEMENTS", "ABSTRACT", "TABLE OF CONTENTS", "LIST OF FIGURES", "LIST OF TABLES", "ABBREVIATIONS", "REFERENCES"):
+        for i, t in enumerate(pdf_text):
+            lines = [x.strip() for x in t.splitlines() if x.strip()]
+            if h in lines and lines[0] != h:
+                misplaced.append(f"{h} on physical page {i + 1}")
+    report("FAIL" if misplaced else "ok", "main headings start at the top of a page", "; ".join(misplaced))
+
+    # Fixed pages that must stay whole. Text can overflow silently, so look at the rendered pages.
+    pairs = [["BONAFIDE CERTIFICATE", "EXAMINER 2"]] + cfg.get("same_page", [])
+    for a, b in pairs:
+        on_a = [i for i, t in enumerate(pdf_text) if a in t]; on_b = [i for i, t in enumerate(pdf_text) if b in t]
+        if on_a or on_b:
+            ok = bool(set(on_a) & set(on_b))
+            report("ok" if ok else "FAIL", f"'{a}' and '{b}' are on one page", "" if ok else f"physical pages {[x + 1 for x in on_a]} and {[x + 1 for x in on_b]}")
+    if not pdf[0].get_images():
+        report("FAIL", "cover has its logo", "no image on page 1")
+    else:
+        report("ok", "cover has its logo")
+    in_cells = [s for s in body.iter(qn("w:sectPr")) if any(a.tag == qn("w:tc") for a in s.iterancestors())]
+    report("FAIL" if in_cells else "ok", "no stray section breaks inside tables", str(len(in_cells)) if in_cells else "")
+    print(f"\n{results.count('FAIL')} fail, {results.count('WARN')} warn, {results.count('ok')} ok, {len(pdf)} pages")
+    return 1 if "FAIL" in results else 0
+
+
+if __name__ == "__main__":
+    if len(sys.argv) < 3:
+        sys.exit(__doc__)
+    cfg = json.load(open(sys.argv[3], encoding="utf-8")) if len(sys.argv) > 3 else {}
+    sys.exit(main(sys.argv[1], sys.argv[2], cfg))

package/scripts/inspect_template.py

@@ -0,0 +1,124 @@
+#!/usr/bin/env python3
+"""Read an SRM report template (or a finished report) and write down everything a builder must know.
+
+usage: python -X utf8 inspect_template.py <template.docx> <outdir>
+
+Writes to <outdir>:
+  structure.txt  every body element: index, type, style, size, bold, align, markers, text
+  rules.txt      sentences that state a rule, inline notes like <Font Size 18>, and placeholders to fill
+  sections.txt   each section's page size, margins, numbering and footer text
+Prints a short summary. Read all three files before planning the report.
+"""
+import os, re, sys
+from collections import Counter
+import docx
+from docx.oxml.ns import qn
+from docx.table import Table
+from docx.text.paragraph import Paragraph
+
+RULE_WORDS = re.compile(
+    r"\b(must|should|shall|minimum|maximum|mandatory|at least|not more than|font|margin|spacing|"
+    r"justify|similarity|turnitin|plagiarism|caption|binding|pages?\b.*\b(excluding|minimum|maximum)|"
+    r"every chapter|page numbers?|references? (must|to be)|hard ?bound|spiral)\b", re.I)
+NOTE = re.compile(r"<(?!<)[^<>\n]{2,80}>(?!>)")
+PLACEHOLDER = re.compile(
+    r"<<[^<>]+>>|\[[A-Z][A-Za-z .]*(NAME|NUM|NO)[^\]]*\]|\.{5,}|_{5,}|\(GUIDE NAME\s*\)|\(Designation[^)]*\)|"
+    r"\(SPECIALIZATION NAME\)|STUDENT ?\d ?NAME|PROJECT TITLE|TITLE OF THE [A-Z]+|Name of the [Ss]tudent|Dr\. ?ABCD")
+
+
+def ptext(el):
+    out = []
+    for x in el.iter():
+        if x.tag == qn("w:t") and x.text: out.append(x.text)
+        elif x.tag == qn("w:tab"): out.append("\t")
+        elif x.tag == qn("w:br"): out.append(" / ")
+    return "".join(out)
+
+
+def describe(p):
+    size = bold = None
+    for r in p.runs:
+        if r.text.strip():
+            size = r.font.size.pt if r.font.size else None
+            bold = r.bold
+            break
+    if size is None and p.style is not None and p.style.font.size:
+        size = p.style.font.size.pt
+    if bold is None and p.style is not None:
+        bold = p.style.font.bold
+    al = p.alignment
+    return size, bold, (al.name if al is not None else "")
+
+
+def main(path, out):
+    os.makedirs(out, exist_ok=True)
+    d = docx.Document(path)
+    body = d.element.body
+    lines, rules, notes, holders = [], [], Counter(), Counter()
+
+    def scan(t, where):
+        for m in NOTE.finditer(t): notes[m.group(0)] += 1
+        for m in PLACEHOLDER.finditer(t): holders[m.group(0).strip()] += 1
+        if RULE_WORDS.search(t) and len(t) > 25: rules.append(f"[{where}] {t.strip()}")
+
+    for i, el in enumerate(body.iterchildren()):
+        if el.tag == qn("w:sectPr"):
+            lines.append(f"{i:4d} FINAL-SECTPR")
+            continue
+        if el.tag == qn("w:tbl"):
+            t = Table(el, d)
+            n_rows, n_cols = len(t.rows), len(t.columns)
+            first = " | ".join(c.text.strip().replace("\n", " / ")[:70] for c in t.rows[0].cells)
+            lines.append(f"{i:4d} TABLE {n_rows}x{n_cols}: {first}")
+            for tr in el.iter(qn("w:p")):
+                scan(ptext(tr), f"{i} table")
+            continue
+        if el.tag != qn("w:p"):
+            lines.append(f"{i:4d} {el.tag.split('}')[1]}")
+            continue
+        p = Paragraph(el, d)
+        t = ptext(el)
+        size, bold, al = describe(p)
+        marks = []
+        if el.find(".//" + qn("w:sectPr")) is not None: marks.append("SECT")
+        if el.find(".//" + qn("w:pageBreakBefore")) is not None: marks.append("PBB")
+        if any(b.get(qn("w:type")) == "page" for b in el.iter(qn("w:br"))): marks.append("PB")
+        n_img = len(list(el.iter("{http://schemas.openxmlformats.org/drawingml/2006/main}blip")))
+        if n_img: marks.append(f"IMG{n_img}")
+        if el.find(".//" + qn("w:txbxContent")) is not None: marks.append("TEXTBOX")
+        if el.find(".//" + qn("w:numPr")) is not None: marks.append("LIST")
+        if el.find(".//" + qn("w:instrText")) is not None: marks.append("FIELD")
+        style = p.style.name if p.style is not None else "-"
+        if t.strip() or marks:
+            lines.append(f"{i:4d} P [{style}|{size or ''}|{'B' if bold else ''}|{al}]{' {' + ','.join(marks) + '}' if marks else ''} {t[:170]}")
+        scan(t, str(i))
+
+    secs = []
+    cm = lambda v: round(v / 360000, 2) if v is not None else None
+    for i, s in enumerate(d.sections):
+        pg = s._sectPr.find(qn("w:pgNumType"))
+        num = dict((k.split("}")[1], v) for k, v in pg.attrib.items()) if pg is not None else {}
+        foot = " ".join(p.text for p in s.footer.paragraphs).strip() if not s.footer.is_linked_to_previous else "(linked)"
+        fld = bool(s.footer._element.findall(".//" + qn("w:instrText")) or s.footer._element.findall(".//" + qn("w:fldSimple"))) if not s.footer.is_linked_to_previous else None
+        secs.append(f"{i}: page {cm(s.page_width)}x{cm(s.page_height)} cm  margins L {cm(s.left_margin)} R {cm(s.right_margin)} "
+                    f"T {cm(s.top_margin)} B {cm(s.bottom_margin)} cm  numbering {num}  footer {foot!r} pagefield={fld}")
+
+    with open(os.path.join(out, "structure.txt"), "w", encoding="utf-8") as f:
+        f.write("\n".join(lines))
+    with open(os.path.join(out, "sections.txt"), "w", encoding="utf-8") as f:
+        f.write("\n".join(secs))
+    with open(os.path.join(out, "rules.txt"), "w", encoding="utf-8") as f:
+        f.write("# RULE SENTENCES (template's own words)\n" + "\n".join(rules))
+        f.write("\n\n# INLINE NOTES (specimen hints: apply them to the text next to them, then delete them)\n")
+        f.write("\n".join(f"{v}x {k}" for k, v in notes.most_common()))
+        f.write("\n\n# PLACEHOLDERS TO FILL\n" + "\n".join(f"{v}x {k}" for k, v in holders.most_common()))
+
+    print(f"{os.path.basename(path)}: {len(lines)} body elements, {len(d.sections)} sections, {len(d.tables)} tables, "
+          f"{len(rules)} rule sentences, {sum(notes.values())} inline notes, {sum(holders.values())} placeholders")
+    print("wrote structure.txt, rules.txt, sections.txt to", out)
+
+
+if __name__ == "__main__":
+    if len(sys.argv) != 3:
+        sys.exit(__doc__)
+    main(sys.argv[1], sys.argv[2])

package/scripts/render_pages.py

@@ -0,0 +1,34 @@
+#!/usr/bin/env python3
+"""Render a report PDF so the pages can be looked at.
+usage: python -X utf8 render_pages.py report.pdf outdir            contact sheets of every page (sheet0.png, ...)
+       python -X utf8 render_pages.py report.pdf outdir 1,2,3 90   the listed physical pages at the given dpi (p1.png, ...)
+"""
+import os, sys
+import fitz
+from PIL import Image
+
+if len(sys.argv) < 3:
+    sys.exit(__doc__)
+pdf, out = sys.argv[1], sys.argv[2]
+os.makedirs(out, exist_ok=True)
+d = fitz.open(pdf)
+if len(sys.argv) > 3:
+    dpi = int(sys.argv[4]) if len(sys.argv) > 4 else 90
+    for n in (int(x) for x in sys.argv[3].split(",")):
+        d[n - 1].get_pixmap(dpi=dpi).save(os.path.join(out, f"p{n}.png"))
+    print("wrote", sys.argv[3], "to", out)
+else:
+    ims = []
+    for p in d:
+        px = p.get_pixmap(dpi=45)
+        ims.append(Image.frombytes("RGB", (px.width, px.height), px.samples))
+    w, h = ims[0].size
+    cols = 10
+    for k in range(0, len(ims), 30):
+        sub = ims[k:k + 30]
+        rows = (len(sub) + cols - 1) // cols
+        c = Image.new("RGB", (w * cols, h * rows), "white")
+        for j, im in enumerate(sub):
+            c.paste(im, ((j % cols) * w, (j // cols) * h))
+        c.save(os.path.join(out, f"sheet{k // 30}.png"))
+    print(len(ims), "pages,", (len(ims) + 29) // 30, "sheets in", out)