ctrl-kd 4.5.0__tar.gz → 4.5.1__tar.gz

{ctrl_kd-4.5.0/src/ctrl_kd.egg-info → ctrl_kd-4.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ctrl-kd
-Version: 4.5.0
+Version: 4.5.1
 Summary: Convert WordStar 4-7 documents and print-to-disk files to text, Markdown, HTML, RTF, or PDF. ^KD: save and done.
 Author: Jon Michaels
 License: MIT
@@ -66,6 +66,7 @@ More information: **[FAQ.md](FAQ.md)**, and **[ERAS.md](ERAS.md)**.
 $ brew install jonmichaels/tap/ctrl-kd     # macOS / Linuxbrew
 $ pipx install ctrl-kd                     # or: pip install ctrl-kd
 ```
+Download Windows x86_64: [Latest Version](https://github.com/jonmichaels/ctrl-kd/releases/latest/download/ctrl-kd-windows-x86_64.zip)
 
 Python ≥ 3.9, no dependencies. Library API: `ctrlkd.convert(data, to='html')`.
 
@@ -97,7 +98,12 @@ My own test files are personal and are not distributed — this repo's tests use
 synthetic fixtures and some public domain docs I retyped in WordStar 4 and 
 WordStar 7 in DOSBox-X, plus Robert J. Sawyer's public WS7 archive (opt-in, 
 `pytest -m sawyer`; see `tests/SAWYER-CORPUS.md`) you can run your own tests
-against that if you have a copy.
+against that if you have a copy: download the archive, point
+`CTRLKD_SAWYER_ARCHIVE` at its top-level directory (the one holding
+`CONVERT.WS`, `INSET/`, `ARTICLES/`), and verify the path before arming —
+`tests/SAWYER-CORPUS.md` has a one-line smoke check. A `CTRLKD_PRIVATE_CORPUS`
+variable also exists, for my own private regression fixtures — it has no
+effect unless you're me.
 
 ## Credits
 

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1}/README.md

@@ -47,6 +47,7 @@ More information: **[FAQ.md](FAQ.md)**, and **[ERAS.md](ERAS.md)**.
 $ brew install jonmichaels/tap/ctrl-kd     # macOS / Linuxbrew
 $ pipx install ctrl-kd                     # or: pip install ctrl-kd
 ```
+Download Windows x86_64: [Latest Version](https://github.com/jonmichaels/ctrl-kd/releases/latest/download/ctrl-kd-windows-x86_64.zip)
 
 Python ≥ 3.9, no dependencies. Library API: `ctrlkd.convert(data, to='html')`.
 
@@ -78,7 +79,12 @@ My own test files are personal and are not distributed — this repo's tests use
 synthetic fixtures and some public domain docs I retyped in WordStar 4 and 
 WordStar 7 in DOSBox-X, plus Robert J. Sawyer's public WS7 archive (opt-in, 
 `pytest -m sawyer`; see `tests/SAWYER-CORPUS.md`) you can run your own tests
-against that if you have a copy.
+against that if you have a copy: download the archive, point
+`CTRLKD_SAWYER_ARCHIVE` at its top-level directory (the one holding
+`CONVERT.WS`, `INSET/`, `ARTICLES/`), and verify the path before arming —
+`tests/SAWYER-CORPUS.md` has a one-line smoke check. A `CTRLKD_PRIVATE_CORPUS`
+variable also exists, for my own private regression fixtures — it has no
+effect unless you're me.
 
 ## Credits
 

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1}/pyproject.toml

@@ -48,8 +48,11 @@ ctrlkd = ["samples/*.WS"]
 # DESELECTED here, not skipped: it never even collects into the report, so
 # a bare run's summary line is an honest denominator.
 #
-# Arming (see tests/sawyer_fixture.py, conftest.py):
-#   sawyer   CTRLKD_SAWYER_ARCHIVE=/path   (legacy alias: CTRLKD_CORPUS_SOURCE)
+# Arming (see tests/sawyer_fixture.py, conftest.py, tools/pcl_tolerance.py):
+#   sawyer   CTRLKD_SAWYER_ARCHIVE=/path
+#   pcl      CTRLKD_PRIVATE_CORPUS=/path (2026-09-05, planning #197)
+#   paper    CTRLKD_PRIVATE_CORPUS=/path (2026-09-06, planning #200; verdicts
+#            file path itself overridable with CTRLKD_PAPER_VERDICTS)
 #
 # tools/run-full-suite.sh overrides this filter (`-o addopts=""`) for an
 # armed run, where an unarmed gate FAILS instead of silently not running --
@@ -57,5 +60,7 @@ ctrlkd = ["samples/*.WS"]
 testpaths = ["tests"]
 markers = [
     "sawyer: opt-in tier 2 -- an explicit, committed list of documents from Robert J. Sawyer's public WordStar 7 archive (arm with CTRLKD_SAWYER_ARCHIVE)",
+    "pcl: opt-in -- coordinate-level fidelity gate against real WordStar 7 LaserJet PCL captures (arm with CTRLKD_PRIVATE_CORPUS); see tools/pcl_tolerance.py and tests/test_pcl_fidelity.py",
+    "paper: opt-in -- machine-readable verdicts for the 69 M479fdw paper-scan pages (arm with CTRLKD_PRIVATE_CORPUS); see tools/PAPER-VERDICTS.md and tests/test_paper_verdicts.py",
 ]
-addopts = "-m \"not sawyer\""
+addopts = "-m \"not sawyer and not pcl and not paper\""

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1/src/ctrl_kd.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ctrl-kd
-Version: 4.5.0
+Version: 4.5.1
 Summary: Convert WordStar 4-7 documents and print-to-disk files to text, Markdown, HTML, RTF, or PDF. ^KD: save and done.
 Author: Jon Michaels
 License: MIT
@@ -66,6 +66,7 @@ More information: **[FAQ.md](FAQ.md)**, and **[ERAS.md](ERAS.md)**.
 $ brew install jonmichaels/tap/ctrl-kd     # macOS / Linuxbrew
 $ pipx install ctrl-kd                     # or: pip install ctrl-kd
 ```
+Download Windows x86_64: [Latest Version](https://github.com/jonmichaels/ctrl-kd/releases/latest/download/ctrl-kd-windows-x86_64.zip)
 
 Python ≥ 3.9, no dependencies. Library API: `ctrlkd.convert(data, to='html')`.
 
@@ -97,7 +98,12 @@ My own test files are personal and are not distributed — this repo's tests use
 synthetic fixtures and some public domain docs I retyped in WordStar 4 and 
 WordStar 7 in DOSBox-X, plus Robert J. Sawyer's public WS7 archive (opt-in, 
 `pytest -m sawyer`; see `tests/SAWYER-CORPUS.md`) you can run your own tests
-against that if you have a copy.
+against that if you have a copy: download the archive, point
+`CTRLKD_SAWYER_ARCHIVE` at its top-level directory (the one holding
+`CONVERT.WS`, `INSET/`, `ARTICLES/`), and verify the path before arming —
+`tests/SAWYER-CORPUS.md` has a one-line smoke check. A `CTRLKD_PRIVATE_CORPUS`
+variable also exists, for my own private regression fixtures — it has no
+effect unless you're me.
 
 ## Credits
 

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1}/src/ctrl_kd.egg-info/SOURCES.txt

@@ -42,10 +42,14 @@ tests/test_lj6dtp_legend_line_spacing.py
 tests/test_lj6dtp_pcl_rectangles.py
 tests/test_lj6dtp_shading_table_rules.py
 tests/test_lj6dtp_table_rule_weight.py
+tests/test_load_plugins.py
 tests/test_modern_box_regions.py
 tests/test_modern_line_spacing.py
 tests/test_modern_lint.py
 tests/test_note_rulings_20260824.py
+tests/test_paper_verdicts.py
+tests/test_pcl_fidelity.py
+tests/test_pcl_tolerance.py
 tests/test_pictures.py
 tests/test_pix.py
 tests/test_polarity_gate.py

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1}/src/ctrlkd/__init__.py

@@ -8,4 +8,4 @@ from .pdf import emit_pdf                        # registers the 'pdf' format
 from .convert import convert, select_notes, DEFAULT_NOTE_KINDS, ALL_NOTE_KINDS
 from .info import document_info
 
-__version__ = '4.5.0'
+__version__ = '4.5.1'

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1}/src/ctrlkd/core.py

@@ -1260,6 +1260,37 @@ class Document:
     # FONT (not just its text) mid-file keeps only the last one seen.
     header_fonts: dict = field(default_factory=dict)
     footer_fonts: dict = field(default_factory=dict)
+    # A `.h#`/`.f#` line's own RIGHT/CENTER/DECIMAL-align tab (WSFORMAT type-9
+    # symmetric sequence), when its typed text carries one: (char_idx, cols,
+    # abs_hmi) -- char_idx is where the tab's own BAKED padding run (already
+    # expanded to `cols` space characters when the file was last saved,
+    # exactly like a body span's tab, see `_symmetric_blocks`' `_tab_columns`)
+    # sits within the FINAL decoded `headers[line]`/`footers[line]` string;
+    # `abs_hmi` is the tab's own absolute target column (content[2:4], same
+    # field a body span's `tabN` mark carries), from the document's own left
+    # reference, in HMI (1/1800in).
+    #
+    # WHY THIS EXISTS (planning #202, -README's own running head): a body
+    # span's tab gets this same absolute target as a MARK a Printed-mode
+    # renderer can consult live (`_decode_spans`' `tab_target_at`) -- but
+    # `.h#`/`.f#` text is stored as a flat, already-`#`-unaware STRING, so the
+    # `cols` count baked into it was fixed FOREVER at whatever the page
+    # number's width was assumed to be WHEN THE FILE WAS SAVED (always 1
+    # digit -- WordStar's own screen shows the LITERAL '#' token, one column,
+    # never the eventual printed number). Real WS7 re-evaluates the tab at
+    # PRINT TIME, using THAT page's own actual page-number width, so a
+    # right-aligned "WordStar 7.0 Archive / #" header shifts one column
+    # LEFT the moment the page number grows a second digit (measured,
+    # -README.WS pages 9->10: WS7's own "WordStar" moves from x=345.6pt to
+    # x=338.4pt, exactly 7.2pt = one Courier column, while the text's own
+    # RIGHT edge -- "9" ends at 518.4pt, "10" also ends at 518.4pt -- stays
+    # utterly fixed). Carrying `(char_idx, cols, abs_hmi)` lets the Printed
+    # emitter redo that same per-page arithmetic instead of trusting the
+    # baked, page-1-shaped `cols`. None when this line's text has no tab of
+    # its own (the common case, and the only case before this existed) --
+    # `_hf_line_ops` falls back to the baked text unchanged, byte-identical.
+    header_tabs: dict = field(default_factory=dict)
+    footer_tabs: dict = field(default_factory=dict)
     # Every .he/.h1-.h5/.fo/.f1-.f5 IN DOCUMENT ORDER, with the block it
     # precedes: ('H'|'F', line 1-5, text, block_index). WordStar applies a
     # running head from the page where it is defined -- on that page itself
@@ -2538,7 +2569,8 @@ def _parse_collect_dot(cmd: bytes, doc, encoding: str, block_index: int):
             doc.meta['line_numbering'] = value if value > 0 else None
 
 
-def _parse_head_foot(cmd: bytes, doc, encoding: str, anchor=None, font_idx=None):
+def _parse_head_foot(cmd: bytes, doc, encoding: str, anchor=None, font_idx=None,
+                     tab_mark=None):
     """Record `.he`/`.h1`-`.h5` and `.fo`/`.f1`-`.f5` text on the Document.
 
     `.HE` and `.FO` are line 1; the numbered forms select their own line, so a
@@ -2554,6 +2586,12 @@ def _parse_head_foot(cmd: bytes, doc, encoding: str, anchor=None, font_idx=None)
     Font block exactly like body text can, and that block contributes no
     bytes of its own to the cleaned stream, so nothing but the caller's own
     line_marks lookup can recover it once we are down here working on bytes.
+
+    `tab_mark` (planning #202, -README's own running head) is the caller's
+    own `line_marks` 'tab' entry for this line, if any: `(rel, abs_hmi,
+    leader, cols)` -- `rel` a BYTE offset from the start of `cmd` (the SAME
+    coordinate space `_symmetric_blocks` recorded it in), the rest exactly
+    what a body span's own tab mark carries. See `Document.header_tabs`.
     """
     m = _HEAD_FOOT_RE.match(cmd)
     if not m:
@@ -2565,22 +2603,38 @@ def _parse_head_foot(cmd: bytes, doc, encoding: str, anchor=None, font_idx=None)
     # body uses -- control-range middles are chart glyphs, the rest are the
     # byte's own cp437 character.
     raw_txt = m.group(2)
+    tab_byte_idx = tab_mark[0] - m.start(2) if tab_mark is not None else None
+    tab_char_idx = None
     parts, pos = [], 0
     for t in re.finditer(rb'\x1b(.)\x1c', raw_txt, re.S):
+        if (tab_char_idx is None and tab_byte_idx is not None
+                and tab_byte_idx <= t.start()):
+            tab_char_idx = sum(len(p) for p in parts) + (tab_byte_idx - pos)
         parts.append(raw_txt[pos:t.start()].decode(encoding, 'replace'))
         x = t.group(1)[0]
         parts.append(CP437_GRAPHICS[x] if x < 0x20 or x == 0x7F
                      else bytes([x]).decode(encoding, 'replace'))
         pos = t.end()
+    if tab_char_idx is None and tab_byte_idx is not None:
+        tab_char_idx = sum(len(p) for p in parts) + (tab_byte_idx - pos)
     parts.append(raw_txt[pos:].decode(encoding, 'replace'))
     text = ''.join(parts).rstrip()
     kind = 'H' if tag.startswith(b'H') else 'F'
     which = doc.headers if kind == 'H' else doc.footers
     which_fonts = doc.header_fonts if kind == 'H' else doc.footer_fonts
+    which_tabs = doc.header_tabs if kind == 'H' else doc.footer_tabs
     second = tag[1:2]
     line = 1 if second in (b'E', b'O') else int(second)
     which[line] = text
     which_fonts[line] = font_idx
+    # A tab whose own byte offset landed at or past the '#'-bearing text (or
+    # whose target char offset ended up past what rstrip() kept) has nothing
+    # left to reposition -- None, same as a line with no tab at all.
+    if (tab_mark is not None and tab_char_idx is not None
+            and 0 <= tab_char_idx <= len(text)):
+        which_tabs[line] = (tab_char_idx, tab_mark[3], tab_mark[1])
+    else:
+        which_tabs[line] = None
     if anchor is not None:
         doc.hf_events.append((kind, line, text, anchor))
 
@@ -4251,11 +4305,18 @@ def parse_ws(data: bytes, encoding: str = 'cp437') -> Document:
             # dot-command check) is the only place left to find it.
             hf_font_idx = next((m[1] for _rel, m in line_marks if m[0] == 'font'),
                                None)
+            # planning #202: a right/center/decimal-align tab typed into a
+            # `.h#`/`.f#` argument (-README's own running head) is the SAME
+            # 'tab' mark a body span reads via tab_target_at -- found here the
+            # same way hf_font_idx is, since _parse_head_foot only ever sees
+            # bytes, never this line's own marks.
+            hf_tab_mark = next(((rel, m[1], m[2], m[3]) for rel, m in line_marks
+                               if m[0] == 'tab'), None)
             _parse_head_foot(cmd if strip_hibit else raw.rstrip(), doc,
                              encoding,
                              anchor=len(doc.blocks) + (1 if cur.lines or
                                                        cur_line.spans else 0),
-                             font_idx=hf_font_idx)
+                             font_idx=hf_font_idx, tab_mark=hf_tab_mark)
             # The index of the block this entry POINTS AT -- the one that follows it,
             # which is the block still open (if it has content) or the next to open.
             # "This heading is in the table of contents" refers forward, not back.

{ctrl_kd-4.5.0 → ctrl_kd-4.5.1}/src/ctrlkd/emit.py

@@ -72,7 +72,17 @@ def formats():
 def load_plugins():
     """Discover third-party emitters via the 'ctrlkd.emitters' entry-point group."""
     from importlib.metadata import entry_points
-    for ep in entry_points(group='ctrlkd.emitters'):
+    eps = entry_points()
+    # Python 3.10+: entry_points() returns a SelectableGroups object with a
+    # .select(group=...) method (and accepts group= directly). Python 3.9:
+    # it returns a plain dict keyed by group name, with no select/group=
+    # support at all -- pyproject's requires-python is >=3.9, so both paths
+    # must work (planning #203; CI run 34011057595 failed 19 tests on 3.9).
+    if hasattr(eps, 'select'):
+        group = eps.select(group='ctrlkd.emitters')
+    else:
+        group = eps.get('ctrlkd.emitters', [])
+    for ep in group:
         if ep.name not in _REGISTRY:
             fn = ep.load()
             _REGISTRY[ep.name] = {'fn': fn, 'ext': getattr(fn, 'ext', '.' + ep.name)}
@@ -1000,7 +1010,8 @@ def _html_img(r, pictures, image_links, idx):
 
 
 def _html_line(spans, refs, keep, keep_ws=False, shown_map=None, inline_styling=True,
-               pix_map=None, pictures='off', image_links=None, sentence_spacing=False):
+               pix_map=None, pictures='off', image_links=None, sentence_spacing=False,
+               merge_tab_position_tags=False):
     """Render one already-decided list of Spans (a logical line, or one
     Line's worth of a paragraph unit -- callers choose). Coalesces adjacent
     identically-styled spans unconditionally: cheap, idempotent for a
@@ -1016,9 +1027,27 @@ def _html_line(spans, refs, keep, keep_ws=False, shown_map=None, inline_styling=
     raw incoming spans -- before this is the SINGLE choke point every
     HTML render path (Printed physical lines, Modern paragraph units,
     _html_slice's structure-row slices alike) funnels through, so one
-    application here covers all of them."""
+    application here covers all of them.
+
+    `merge_tab_position_tags` (issue #204, centred structure rows only --
+    see `_html_centered_row`'s own docstring for why it is the ONE caller
+    that passes it): strips `tabhmi<N>`/`tableader<N>` tags (Printed-only
+    positioning data; no HTML `_TAG`/class exists for either) from a
+    COPY of each span's style set before coalescing, purely so an
+    adjacent same-styled run merges across a tab-run instead of being
+    fenced off by it. Defaults False -- every other caller (ordinary
+    Modern paragraphs, bullet/def-list slices) keeps the existing
+    behaviour, where a tab-run stays its own span."""
     if sentence_spacing:
         spans = sentence_spacing_spans(spans)
+    if merge_tab_position_tags:
+        spans = [Span(s.text, frozenset(t for t in s.styles
+                                        if not (t.startswith('tabhmi')
+                                                or t.startswith('tableader'))))
+                if any(t.startswith('tabhmi') or t.startswith('tableader')
+                       for t in s.styles)
+                else s
+                for s in spans]
     out = []
     for s in split_graphic_spans(coalesce_spans(spans)):
         pctl = next((t for t in s.styles if t.startswith('pctl')), None)
@@ -1447,7 +1476,7 @@ def _classify_modern_blocks(doc):
 
 def _html_slice(line, start, end, refs, keep, shown_map, inline_styling=True,
                 pix_map=None, pictures='off', image_links=None,
-                sentence_spacing=False):
+                sentence_spacing=False, merge_tab_position_tags=False):
     # Round 13 (main-merge reconciliation): `_html_line` now takes a bare
     # spans LIST directly (the b23 overhaul's own signature -- it runs
     # `coalesce_spans`/`split_graphic_spans` over its argument), not a
@@ -1463,7 +1492,8 @@ def _html_slice(line, start, end, refs, keep, shown_map, inline_styling=True,
     return _html_line(_slice_spans(line.spans, start, end),
                       refs, keep, shown_map=shown_map, inline_styling=inline_styling,
                       pix_map=pix_map, pictures=pictures, image_links=image_links,
-                      sentence_spacing=sentence_spacing)
+                      sentence_spacing=sentence_spacing,
+                      merge_tab_position_tags=merge_tab_position_tags)
 
 
 def _html_centered_row(line, s, refs, keep, shown_map, inline_styling=True,
@@ -1472,12 +1502,36 @@ def _html_centered_row(line, s, refs, keep, shown_map, inline_styling=True,
     """The centred line's own text with its alignment padding sliced off
     (both mechanisms: a real align=center tag already had the M3 strip
     upstream, so lead/trail are 0 and this is a no-op; spaces-only
-    centering strips the padding here for the first time)."""
+    centering strips the padding here for the first time).
+
+    Issue #204: `merge_tab_position_tags=True` here, and ONLY here -- a
+    centred structure row is exactly the shape WSFORMAT.WS's own
+    Symmetric-Sequences table row is (a short "label <tab-run> value"
+    line the spaces-centering heuristic pulled out of the flow, see this
+    module's own `classify_rows` caller): a WordStar "tabs and dot-leader"
+    symmetrical sequence expands to literal leader characters already,
+    but the Printed-only `tabhmi`/`tableader` tags riding on that Span
+    fence it into its own `<span>` -- all-spaces text that then trips
+    `_html_span`'s "typescript indent" heuristic into a run of `&nbsp;`
+    entities, which a browser renders as a wide, uncollapsed gap. sr's
+    own Modern HTML instead merges the tab-run into ONE span with the
+    surrounding text, literal spaces intact (`<span
+    ...>4              Endnote</span>`) -- a browser's own default
+    whitespace collapsing then renders that as a single visible space,
+    matching real WS7's own plain-gap look (paper-scan-verified,
+    planning #204's own review pack). Scoped to centred rows only:
+    trying the SAME merge in `_html_line` generally (bullet/def-list
+    slices, ordinary paragraphs) additionally collapsed unrelated
+    label/long-prose rows sr does NOT collapse (e.g. WSFORMAT.WS's own
+    "00h ^@ <tab> Fix the print position...") -- reverted; this is not a
+    general "tab-runs never get their own span" rule, only a centred
+    structure row's own."""
     raw = ''.join(sp.text for sp in line.spans)
     lead = len(raw) - len(raw.lstrip(' '))
     trail = len(raw) - len(raw.rstrip(' '))
     return _html_slice(line, lead, len(raw) - trail, refs, keep, shown_map, inline_styling,
-                       pix_map, pictures, image_links, sentence_spacing)
+                       pix_map, pictures, image_links, sentence_spacing,
+                       merge_tab_position_tags=True)
 
 
 def _html_toc_index(doc):