ctrl-kd 1.1.2__py3-none-any.whl

ctrl_kd-1.1.2.dist-info/METADATA

@@ -0,0 +1,120 @@
+Metadata-Version: 2.4
+Name: ctrl-kd
+Version: 1.1.2
+Summary: Convert WordStar 4-7 documents and print-to-disk files to text, Markdown, HTML, RTF, or PDF. ^KD: save and done.
+Author: Jon Michaels
+License: MIT
+Project-URL: Homepage, https://github.com/jonmichaels/ctrl-kd
+Keywords: wordstar,converter,retrocomputing,archive,dos
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Text Processing :: Filters
+Classifier: Topic :: System :: Archiving
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# ctrl-kd
+
+Convert WordStar-era files to modern formats. **^KD: save and done.**
+
+`ctrl-kd` reads WordStar 4 documents, WordStar 5–7 documents, and WordStar
+**print-to-disk files** (the printer byte stream, captured to a file — a distinct
+format most converters mangle), and writes plain text, Markdown, HTML, RTF, or PDF (typewriter-set on the
+built-in Courier fonts — no dependencies, the page as it would have printed).
+
+```console
+$ ctrl-kd ESSAY.WS                      # -> ESSAY.md
+$ ctrl-kd ESSAY.WS -t html -t rtf       # multiple formats
+$ ctrl-kd ESSAY.WS -t pdf --mode printed # a facsimile of the 1990 printout
+$ ctrl-kd --mode printed LETTER.WS      # line-for-line, as it printed in 1990
+$ ctrl-kd --diagnose MYSTERY.FIL        # what IS this file?
+```
+
+## Why another converter?
+
+Existing tools each lose something. Fed a WordStar 4 file, converters written for
+WS7 delete the last letter of every word (WS4 set bit 7 on it). Most delete soft
+returns outright — `Jon Michaels` + `March 6, 1992` becomes
+`Jon MichaelsMarch 6, 1992` — which also destroys every poem, because poem lines
+end in soft returns too. And print-to-disk files aren't WordStar documents at all,
+so feeding them to a WordStar converter produces stray superscripts and garbage.
+
+`ctrl-kd` was built by converting a real 1987–1992 corpus (high-school and college
+papers, poems, stories — WordStar 4 on DOS, dot-matrix printer) and verifying
+against surviving period printouts of the same documents. Its rules are empirical:
+
+* **Detection by content, never by extension.** WS4 vs WS5+ vs print stream vs
+  plain text vs binary, with the evidence shown in `--diagnose`.
+* **The wrap test.** WordStar wrapped only when the next word didn't fit. So a
+  soft return where the next word *would* have fit (strictly — WordStar wrapped
+  even on an exact-margin fit) is a deliberate break: a poem line, a heading.
+  Everything else is word wrap and joins with a space. The margin is estimated
+  from the 90th percentile of soft-wrapped line lengths (floor 65, the default).
+* **Break runs.** Soft/hard return runs containing a hard return and a blank line
+  are paragraph breaks; a lone hard return is the author's deliberate line break.
+  Double-spaced documents (blank soft lines between every line) collapse
+  automatically.
+* **Ruler lines mean columns.** A `.rr----!----` dot line defines tab stops; the
+  document's alignment is space-built and only survives fixed-width. Such
+  documents render `printed` in every mode.
+* **Print streams render verbatim** — they ARE the printed page — with printer
+  style codes decoded (superscript/underline/italic/bold pairs; table in
+  `core.PRINT_CODES`, derived from a late-80s dot-matrix driver and overridable).
+* **WS5+ symmetric blocks** (`0x1D`: real footnotes/endnotes, headings, page
+  breaks — machinery added in WS5) are parsed with their nested structure,
+  verified against the 86 WordStar 7 documents in Robert J. Sawyer's public
+  WordStar archive: footnotes extract with in-text references (`[^n]` in
+  Markdown), paragraph styles become headings, and 82/86 convert with zero
+  mojibake. More WS5–7 corpora still welcome.
+
+## Modes
+
+* `--mode modern` (default): reflowed paragraphs, semantic markup, deliberate
+  line breaks kept.
+* `--mode printed`: every line as laid out, fixed-width, `.pa`/form-feed page
+  breaks honored — how it came off the printer.
+
+## Install
+
+Straight from GitHub (not yet on PyPI):
+
+```console
+$ pipx install git+https://github.com/jonmichaels/ctrl-kd
+```
+
+or `pip install git+https://github.com/jonmichaels/ctrl-kd` into an environment
+of your choice. Python ≥ 3.9, no dependencies.
+Library API: `ctrlkd.convert(data, to='html')`.
+
+## Adding an output format
+
+An output format is one function over the parsed document — register it with the
+`@ctrlkd.emitter` decorator, or ship it as a pip-installable plugin via the
+`ctrlkd.emitters` entry-point group and it appears in the CLI automatically.
+**[EXTENDING.md](EXTENDING.md)** has the IR contract, a complete worked example
+(BBCode in ~40 lines), and a checklist.
+
+## Lineage
+
+Standing on the shoulders of the tools and documentation that kept WordStar
+readable: Yohanes Nugroho's WS-CON, Michael Petrie's English port, the `wsconvert`
+project, Robert J. Sawyer's WordStar archive, and the WordStar format
+documentation community. Behaviors were studied and reimplemented; no code was
+copied. The development corpus is personal and is not distributed — tests use
+synthetic fixtures that encode the same behaviors.
+
+## Credits
+
+Written by Jon Michaels — whose 1987–1992 WordStar files, and the need to read
+them again, are the reason this exists — with Athena (Claude, Anthropic) as
+co-author: the byte archaeology, the wrap test, and the implementation grew out
+of a joint effort to recover those disks. Every commit carries the co-author
+trailer.
+
+## License
+
+MIT © Jon Michaels

ctrl_kd-1.1.2.dist-info/RECORD

@@ -0,0 +1,11 @@
+ctrl_kd-1.1.2.dist-info/licenses/LICENSE,sha256=BEHxdhJXtQzaKyykQunEdL8p67aK6x9fdDYkbeBlRpY,1069
+ctrlkd/__init__.py,sha256=Zgb2ShkQMe_sdGKtFM_5xuAipytBH1I4xpR5I9DQ56I,669
+ctrlkd/cli.py,sha256=RcQkcC3etnZhA3rPJXQlqgkYIH4CsTUJFmWyKxHUkxg,4114
+ctrlkd/core.py,sha256=aPuaCl0y76_pn7_ZPtL-2akhcZmlQAFgtQmZBczU7YM,15807
+ctrlkd/emit.py,sha256=Y8TszTHA74r4l18NFxI28D5Lcv5B9s9SOaJ4BO748fg,9305
+ctrlkd/pdf.py,sha256=uGsGNtWIZWF3w-IAn4XUQryXIL335xf0IY-GCEB3lj8,8493
+ctrl_kd-1.1.2.dist-info/METADATA,sha256=KzD6IGjeHjnYqIMhnBxla9qjrb-rUGBgBN_1W3PqU7Q,5648
+ctrl_kd-1.1.2.dist-info/WHEEL,sha256=K260EYznzXsJYBQGqmI8VTxEdiZYNvDZwW9cBh9-_MA,91
+ctrl_kd-1.1.2.dist-info/entry_points.txt,sha256=K8MXIMbfy87rE2ef0ll7MgbgFMVONpr4IAHDhIwGi2w,44
+ctrl_kd-1.1.2.dist-info/top_level.txt,sha256=ohMOsb7bEutCwzkdhY43QEv1ydDAKXH2hArckeKKBSY,7
+ctrl_kd-1.1.2.dist-info/RECORD,,

ctrl_kd-1.1.2.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (83.0.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

ctrl_kd-1.1.2.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+ctrl-kd = ctrlkd.cli:main

ctrl_kd-1.1.2.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jon Michaels
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ctrl_kd-1.1.2.dist-info/top_level.txt

@@ -0,0 +1 @@
+ctrlkd

ctrlkd/__init__.py

@@ -0,0 +1,12 @@
+"""ctrl-kd — convert WordStar-era files to modern formats. ^KD: save and done."""
+from .core import detect, parse, parse_ws, parse_printstream, Document, Block, Line, Span
+from .emit import (emit_text, emit_markdown, emit_html, emit_rtf,
+                   emitter, get_emitter, formats, load_plugins)
+from .pdf import emit_pdf                        # registers the 'pdf' format
+
+__version__ = '1.1.2'
+
+def convert(data: bytes, to: str = 'markdown', mode: str = 'modern',
+            encoding: str = 'cp437', **options) -> str:
+    """One-call API: bytes in, converted string out."""
+    return get_emitter(to)['fn'](parse(data, encoding=encoding), mode, **options)

ctrlkd/cli.py

@@ -0,0 +1,90 @@
+"""ctrl-kd command line: convert WordStar-era files to modern formats.
+
+    ctrl-kd PAPER.WS                       # -> PAPER.md, modern reflow
+    ctrl-kd PAPER.WS -t html -o out.html
+    ctrl-kd --mode printed LETTER          # as it came off the printer
+    ctrl-kd --diagnose MYSTERY.FIL         # what IS this file?
+    ctrl-kd -t text -t html -d out/ *.WS   # batch, multiple formats
+"""
+import argparse, json, os, sys
+from . import core, emit
+
+def diagnose(path, data):
+    det = core.detect(data)
+    info = {'file': path, **det}
+    if det['variant'] in ('ws4', 'ws5+'):
+        doc = core.parse_ws(data)
+        info.update({k: doc.meta[k] for k in
+                     ('margin_estimate', 'dot_commands', 'unknown_codes', 'columnar')})
+        info['paragraphs'] = sum(1 for b in doc.blocks if b.kind == 'para')
+        info['footnotes'] = len(doc.footnotes)
+    return info
+
+def main(argv=None):
+    emit.load_plugins()          # third-party emitters (ctrlkd.emitters entry points)
+    ap = argparse.ArgumentParser(
+        prog='ctrl-kd',
+        description='Convert WordStar 4-7 documents and print-to-disk files to '
+                    'text, Markdown, HTML, or RTF (extensible: see EXTENDING.md). '
+                    '^KD: save and done.')
+    ap.add_argument('files', nargs='+', help='input file(s)')
+    ap.add_argument('-t', '--to', action='append', choices=emit.formats(),
+                    help='output format (repeatable; default: markdown)')
+    ap.add_argument('-o', '--output', help='output file (single input only)')
+    ap.add_argument('-d', '--outdir', help='output directory for batch conversion')
+    ap.add_argument('--mode', choices=('modern', 'printed'), default='modern',
+                    help='modern: reflowed paragraphs. printed: line-for-line, '
+                         'fixed-width, as it printed in 1990 (default: modern; '
+                         'print streams and ruler-line documents always render printed)')
+    ap.add_argument('--variant', choices=('ws4', 'ws5+', 'printstream', 'text'),
+                    help='override detection')
+    ap.add_argument('--encoding', default='cp437',
+                    help='byte encoding of the source (default: cp437)')
+    ap.add_argument('--diagnose', action='store_true',
+                    help='report what the file is (variant, margin, dot commands, '
+                         'unknown codes) as JSON; no conversion')
+    a = ap.parse_args(argv)
+    formats = a.to or ['markdown']
+    if a.output and (len(a.files) > 1 or len(formats) > 1):
+        ap.error('-o works with a single input and a single format; use -d for batch')
+
+    status = 0
+    for path in a.files:
+        try:
+            data = open(path, 'rb').read()
+        except OSError as e:
+            print(f'ctrl-kd: {e}', file=sys.stderr)
+            status = 1
+            continue
+        if a.diagnose:
+            print(json.dumps(diagnose(path, data), indent=2))
+            continue
+        try:
+            doc = core.parse(data, encoding=a.encoding, variant=a.variant)
+        except ValueError as e:
+            print(f'ctrl-kd: {path}: {e} (use --diagnose to inspect, '
+                  f'--variant to force)', file=sys.stderr)
+            status = 1
+            continue
+        base = os.path.splitext(os.path.basename(path))[0]
+        for fmt in formats:
+            reg = emit.get_emitter(fmt)
+            out = reg['fn'](doc, a.mode, title=base)
+            if a.output:
+                dest = a.output
+            else:
+                dest = os.path.join(a.outdir or os.path.dirname(path) or '.',
+                                    base + reg['ext'])
+                if a.outdir:
+                    os.makedirs(a.outdir, exist_ok=True)
+            if isinstance(out, bytes):           # binary formats (e.g. pdf)
+                with open(dest, 'wb') as f:
+                    f.write(out)
+            else:
+                with open(dest, 'w', encoding='utf-8', newline='\n') as f:
+                    f.write(out)
+            print(f'{path} -> {dest}')
+    return status
+
+if __name__ == '__main__':
+    sys.exit(main())

ctrlkd/core.py

@@ -0,0 +1,392 @@
+"""ctrl-kd core: detection, parsing, and the intermediate representation.
+
+Everything parses into one IR (a Document of Blocks of Lines of Spans), and every
+export format is a small emitter over that IR — the architectural lesson from
+wsconvert and WS-CON, which each go bytes->output in a single pass and each lose
+something on the way.
+
+WordStar background this code encodes:
+
+* WS4 and earlier set bit 7 on the LAST character of each word ("microjustify"
+  flags). WS5+ dropped that; high bytes there are extended (cp437) characters.
+* Soft returns (8D 0A) mark where WordStar word-wrapped; hard returns (0D 0A) are
+  the author pressing Return. WS4 stores the on-screen layout, so recovering
+  intent takes the wrap test (see lines_pass).
+* Print-to-disk files are not WordStar documents at all: they are the byte stream
+  sent to the printer, captured to a file. They ARE the printed page.
+"""
+from __future__ import annotations
+import re
+from dataclasses import dataclass, field
+
+# ---------------------------------------------------------------- IR
+
+@dataclass
+class Span:
+    text: str
+    styles: frozenset = frozenset()      # subset of {'b','i','u','sup','sub','strike'}
+
+@dataclass
+class Line:
+    spans: list = field(default_factory=list)
+
+    def text(self):
+        return ''.join(s.text for s in self.spans)
+
+@dataclass
+class Block:
+    kind: str                            # 'para' | 'pagebreak' | 'softpage'
+    lines: list = field(default_factory=list)
+    heading: int = 0                     # 0 = body text; 1-3 = WS5+ title/header/subheading
+
+@dataclass
+class Document:
+    blocks: list = field(default_factory=list)
+    footnotes: list = field(default_factory=list)     # list[list[Span]] (WS5+)
+    meta: dict = field(default_factory=dict)          # detection + diagnose info
+
+    def iter_lines(self):
+        for b in self.blocks:
+            yield from b.lines
+
+# ---------------------------------------------------------------- detection
+
+def detect(data: bytes) -> dict:
+    """Classify a file by CONTENT (names and extensions lie).
+
+    Returns dict with 'variant': ws4 | ws5+ | printstream | text | binary
+    plus the evidence, suitable for --diagnose output.
+    """
+    core = data[:data.index(0x1A)] if 0x1A in data else data
+    if not core:
+        return {'variant': 'binary', 'reason': 'empty (or ^Z at start)'}
+    soft = core.count(b'\x8d\x0a')
+    hard = core.count(b'\x0d\x0a')
+    hi = sum(1 for x in core if x >= 0x80)
+    blocks_1d = core.count(b'\x1d')
+    txt = sum(1 for x in core if 0x20 <= (x & 0x7F) < 0x7F or x in (0x0D, 0x0A, 0x09)) * 100 // len(core)
+    ev = {'soft_returns': soft, 'hard_returns': hard, 'high_bit_bytes': hi,
+          'text_pct': txt, 'symmetric_blocks_1d': blocks_1d, 'size': len(core)}
+    if txt < 40:
+        return {'variant': 'binary', 'reason': f'only {txt}% text-like', **ev}
+    if blocks_1d >= 2:
+        # 1D symmetric blocks are WS5+ machinery regardless of anything else
+        return {'variant': 'ws5+', **ev}
+    # soft returns are strong WS evidence on their own; high-bit density alone is
+    # not — binaries are full of high bytes — unless the file is mostly text
+    if soft >= 3 or (hi >= max(1, len(core) // 20) and txt >= 70):
+        # WS5+ kept soft returns but dropped the bit-7-on-last-letter convention:
+        # a wordstar file with many soft returns and near-zero high bits is WS5+,
+        # as is one using 1D symmetric blocks (footnotes etc., WS5+ only)
+        if blocks_1d >= 2 or (soft >= 3 and hi < soft // 4):
+            return {'variant': 'ws5+', **ev}
+        return {'variant': 'ws4', **ev}
+    if txt >= 90 and hard >= 2:
+        return {'variant': 'printstream', **ev}
+    if txt >= 90:
+        return {'variant': 'text', **ev}
+    return {'variant': 'binary', 'reason': f'{txt}% text but no structure', **ev}
+
+# ---------------------------------------------------------------- line engine
+
+def _visible(text: bytes) -> bytes:
+    return bytes(b & 0x7F for b in text if 0x20 <= (b & 0x7F) < 0x7F)
+
+def lines_pass(data: bytes):
+    """Split into physical lines and classify every break.
+
+    Yields (line_bytes, sep) with sep in {'wrap','line','para','eof'}:
+      para  a break run with >=1 hard return and >=2 breaks total (blank line)
+      line  a single hard return (the author's Return), or a soft return where
+            the next word WOULD have fit — WS4 wrapped only when it didn't fit,
+            so breaking early was a choice (poem line, heading). Strict <:
+            WS4 wrapped even when the word would land exactly at the margin.
+      wrap  a soft return that is just word wrap: join with a space
+    Margin is the 90th percentile of soft-wrapped line lengths (outliers from
+    hanging punctuation sit 1-2 past the true margin), floor 65 (WS4 default).
+    """
+    cut = data.find(b'\x1a')
+    if cut != -1:
+        data = data[:cut]
+    parts = re.split(rb'(\x8d\x0a|\x0d\x0a|\x8d|\x0d|\x0a)', data)
+    lines = []
+    for i in range(0, len(parts), 2):
+        text = parts[i]
+        brk = parts[i + 1] if i + 1 < len(parts) else b''
+        kind = 'eof' if not brk else ('soft' if brk[0] == 0x8D else 'hard')
+        if text or kind != 'eof':
+            lines.append((text, kind))
+
+    softlens = sorted(len(_visible(t).rstrip()) for t, k in lines
+                      if k == 'soft' and _visible(t).strip())
+    margin = max(65, softlens[int(len(softlens) * 0.9)] if softlens else 0)
+
+    out = []
+    i = 0
+    while i < len(lines):
+        text, kind = lines[i]
+        if not _visible(text).strip():
+            i += 1
+            continue
+        n_hard = 1 if kind == 'hard' else 0
+        n_total = 0 if kind == 'eof' else 1
+        j = i + 1
+        while j < len(lines) and not _visible(lines[j][0]).strip():
+            k = lines[j][1]
+            if k != 'eof':
+                n_total += 1
+                n_hard += 1 if k == 'hard' else 0
+            j += 1
+        if j >= len(lines):
+            out.append((text, 'eof'))
+            break
+        if n_hard >= 1 and n_total >= 2:
+            sep = 'para'
+        elif n_hard == 1:
+            sep = 'line'
+        else:
+            nxt_vis = _visible(lines[j][0])
+            if nxt_vis[:1] == b' ':
+                sep = 'line'                      # indented continuation = deliberate
+            else:
+                L = len(_visible(text).rstrip())
+                W = len(nxt_vis.split(b' ', 1)[0])
+                sep = 'line' if L + 1 + W < margin else 'wrap'
+        out.append((text, sep))
+        i = j
+    return out, margin
+
+# ---------------------------------------------------------------- WS documents
+
+# WordStar inline control codes (same core set WS4 through WS7)
+WS_TOGGLES = {0x02: 'b', 0x13: 'u', 0x19: 'i', 0x14: 'sup', 0x16: 'sub',
+              0x18: 'strike', 0x04: 'b'}         # ^D doublestrike -> bold
+WS_DROP = {0x01, 0x03, 0x08, 0x0B, 0x0E, 0x10, 0x11, 0x12, 0x15, 0x17, 0x1C}
+
+DOT_PAGEBREAK = {b'PA', b'CP'}
+
+def _decode_spans(raw: bytes, strip_hibit: bool, encoding: str, active: set,
+                  unknown: dict, fn_counter: list = None) -> list:
+    """One physical line of bytes -> list of Span. `active` persists across lines
+    (WordStar styles span line breaks). fn_counter (ws5+ only) numbers the
+    footnote-reference sentinels injected by _symmetric_blocks."""
+    spans, buf = [], bytearray()
+
+    def flush():
+        if buf:
+            spans.append(Span(buf.decode(encoding, 'replace'), frozenset(active)))
+            buf.clear()
+
+    i = 0
+    while i < len(raw):
+        # WS4's bit-7-on-last-letter applies to CONTROL TOGGLES too (a word ending
+        # at a style boundary yields e.g. 0x94 = ^T|0x80) — so mask BEFORE dispatch,
+        # or high-bit toggles leak into text and styles never close.
+        b = raw[i] & 0x7F if strip_hibit and raw[i] >= 0x80 else raw[i]
+        if b == 0x1B and i + 1 < len(raw):        # extended char escape
+            buf.append(raw[i + 1]); i += 2; continue
+        if fn_counter is not None and b == SENT_FNREF:
+            flush()
+            fn_counter[0] += 1
+            spans.append(Span(str(fn_counter[0]), frozenset(active | {'sup', 'fnref'})))
+        elif b in WS_TOGGLES:
+            flush()
+            style = WS_TOGGLES[b]
+            (active.remove if style in active else active.add)(style)
+        elif b == 0x0F:
+            buf.append(0x20)                      # binding space
+        elif b == 0x1E:
+            pass                                  # inactive soft hyphen
+        elif b == 0x1F:
+            buf.append(0x2D)                      # active soft hyphen
+        elif b == 0x09:
+            buf.append(b)
+        elif b < 0x20 or b == 0x7F:
+            if b not in WS_DROP:
+                unknown[b] = unknown.get(b, 0) + 1
+        else:
+            buf.append(b)
+        i += 1
+    flush()
+    return spans
+
+SENT_FNREF = 0x07      # sentinels injected into the cleaned stream; these bytes
+SENT_SOFTPAGE = 0x0B   # cannot appear as text in a WS5+ document body
+SENT_HEADING = 0x11
+
+def _note_text(block: bytes, encoding: str) -> str:
+    """Note content is NESTED: header, then an inner 1D, the text, then a 2-byte
+    length + 1D tail (verified on the Sawyer WS7 archive: 'Footnote\\r\\n,\\x00')."""
+    inner = block.split(b'\x1d')
+    text = inner[1][:-2] if len(inner) > 1 and len(inner[1]) > 2 else block[20:]
+    clean = bytes(c for c in text if 0x20 <= c < 0x7F or c >= 0x80 or c == 0x09)
+    return clean.decode(encoding, 'replace').strip()
+
+def _symmetric_blocks(data: bytes, encoding: str):
+    """Strip WS5+ 1D symmetric sequences (2-byte LE length, command type at +2),
+    collecting footnotes/endnotes and injecting sentinels for the block types that
+    carry document structure. Verified against the 86 WS7 documents in Robert J.
+    Sawyer's WordStar archive."""
+    out = bytearray()
+    footnotes = []
+    i = 0
+    while i < len(data):
+        if data[i] == 0x1D and i + 3 <= len(data):
+            jump = int.from_bytes(data[i + 1:i + 3], 'little')
+            block = data[i + 1:i + 3 + jump]
+            cmd = block[2] if len(block) > 2 else -1
+            if cmd in (0x03, 0x04):                               # foot/endnote
+                footnotes.append(_note_text(block, encoding))
+                out.append(SENT_FNREF)
+            elif cmd == 0x09:                                     # tab
+                out += b'    '
+            elif cmd == 0x0B:                                     # end of page
+                out.append(SENT_SOFTPAGE)
+            elif cmd == 0x11 and len(block) > 3:                  # paragraph style
+                level = {0x05: 1, 0x02: 2, 0x03: 3}.get(block[3], 0)
+                if level:
+                    out += bytes([SENT_HEADING, 0x30 + level])
+            i += jump + 3
+        else:
+            out.append(data[i])
+            i += 1
+    return bytes(out), footnotes
+
+def parse_ws(data: bytes, encoding: str = 'cp437') -> Document:
+    doc = Document()
+    det = detect(data)
+    doc.meta.update(det)
+    strip_hibit = det['variant'] == 'ws4'
+
+    ws5 = det['variant'] == 'ws5+'
+    if ws5:
+        data, notes = _symmetric_blocks(data, encoding)
+        doc.footnotes = [[Span(n)] for n in notes]
+
+    physical, margin = lines_pass(data)
+    doc.meta['margin_estimate'] = margin
+
+    active, unknown, dots = set(), {}, []
+    fn_counter = [0] if ws5 else None
+    cur = Block('para')
+    cur_line = Line()
+    ruler = False
+
+    def close_line():
+        nonlocal cur_line
+        if cur_line.spans:
+            cur.lines.append(cur_line)
+        cur_line = Line()
+
+    def close_block():
+        nonlocal cur
+        close_line()
+        if cur.lines:
+            doc.blocks.append(cur)
+        cur = Block('para')
+
+    for raw, sep in physical:
+        stripped = bytes(b & 0x7F for b in raw)
+        if stripped[:1] == b'.':                   # dot command line
+            cmd = stripped.rstrip()
+            dots.append(cmd.decode(encoding, 'replace'))
+            if cmd[1:3].upper() in DOT_PAGEBREAK:
+                close_block()
+                doc.blocks.append(Block('pagebreak'))
+            if cmd[1:2].lower() == b'r' and b'!' in cmd:
+                ruler = True
+            continue
+        if ws5:                                    # sentinels from _symmetric_blocks
+            if raw.count(SENT_SOFTPAGE):
+                close_block()
+                doc.blocks.append(Block('softpage'))
+                raw = raw.replace(bytes([SENT_SOFTPAGE]), b'')
+            if raw[:1] == bytes([SENT_HEADING]) and len(raw) > 1:
+                close_block()
+                cur.heading = raw[1] - 0x30
+                raw = raw[2:]
+            raw = raw.replace(bytes([SENT_HEADING]), b'')
+        spans = _decode_spans(raw, strip_hibit, encoding, active, unknown, fn_counter)
+        for s in spans:
+            cur_line.spans.append(s)
+        if sep == 'wrap':
+            t = cur_line.spans[-1].text if cur_line.spans else ''
+            if t and not t.endswith((' ', '-')):
+                cur_line.spans.append(Span(' ', cur_line.spans[-1].styles))
+        elif sep == 'line':
+            close_line()
+        else:                                      # para / eof
+            close_block()
+    close_block()
+
+    doc.meta['dot_commands'] = dots
+    doc.meta['unknown_codes'] = {f'0x{k:02x}': v for k, v in sorted(unknown.items())}
+    doc.meta['columnar'] = ruler
+    return doc
+
+# ---------------------------------------------------------------- print streams
+
+# Empirically derived from a late-80s dot-matrix driver (see README); pass a
+# custom table if your printer differed.
+PRINT_CODES = {0x18: ('sup', True), 0x12: ('sup', False),
+               0x10: ('u', True), 0x11: ('u', False),
+               0x13: ('i', True), 0x15: ('i', False),
+               0x05: ('i', True), 0x06: ('i', False),
+               0x1E: ('b', True), 0x1F: ('b', False)}
+
+def parse_printstream(data: bytes, encoding: str = 'cp437',
+                      codes: dict = None) -> Document:
+    """A print-to-disk capture IS the printed page: every line verbatim, printer
+    style codes decoded, everything else below 0x20 stripped."""
+    codes = PRINT_CODES if codes is None else codes
+    doc = Document(meta={'variant': 'printstream', 'columnar': True})
+    cut = data.find(b'\x1a')
+    if cut != -1:
+        data = data[:cut]
+    active = set()
+    cur = Block('para')
+    line = Line()
+    buf = bytearray()
+
+    def flush():
+        if buf:
+            line.spans.append(Span(buf.decode(encoding, 'replace'), frozenset(active)))
+            buf.clear()
+
+    def endline():
+        nonlocal line
+        flush()
+        cur.lines.append(line)                    # blank lines are page geometry: keep
+        line = Line()
+
+    for b in data:
+        c = b & 0x7F
+        if c in codes:
+            flush()
+            style, on = codes[c]
+            (active.add if on else active.discard)(style)
+        elif c == 0x0A:
+            endline()
+        elif c == 0x0C:
+            endline()
+            doc.blocks.append(cur)
+            doc.blocks.append(Block('pagebreak'))
+            cur = Block('para')
+        elif c == 0x0D or (c < 0x20 and c != 0x09):
+            continue
+        else:
+            buf.append(c)
+    endline()
+    doc.blocks.append(cur)
+    return doc
+
+# ---------------------------------------------------------------- front door
+
+def parse(data: bytes, encoding: str = 'cp437', variant: str = None) -> Document:
+    """Detect (unless told) and parse. This is the library's main entry."""
+    v = variant or detect(data)['variant']
+    if v in ('ws4', 'ws5+'):
+        return parse_ws(data, encoding)
+    if v in ('printstream', 'text'):
+        return parse_printstream(data, encoding)
+    raise ValueError(f'not a convertible file (detected: {v})')

ctrlkd/emit.py

@@ -0,0 +1,236 @@
+"""ctrl-kd emitters: Document IR -> text / markdown / html / rtf.
+
+Two rendering philosophies, chosen by the caller:
+  modern   reflowed paragraphs, semantic markup (the IR already joined word
+           wraps and kept deliberate breaks — emitters just express it)
+  printed  every line as laid out, fixed-width — how it came off the printer.
+           Print streams and columnar documents (WordStar ruler lines) force
+           this: their alignment only exists in a fixed-width world.
+"""
+import html as _html
+
+# ---------------------------------------------------------------- registry
+#
+# The extension point. An emitter is any callable (doc, mode='modern', **options)
+# -> str, registered under a name. Two ways in:
+#
+#   @ctrlkd.emitter('latex', ext='.tex')            # in your own code
+#   def emit_latex(doc, mode='modern', **options): ...
+#
+#   [project.entry-points."ctrlkd.emitters"]        # in an installable plugin's
+#   docx = "ctrlkd_docx:emit_docx"                  # pyproject.toml
+#
+# Entry-point plugins are discovered at CLI startup; `pip install ctrl-kd-docx`
+# is all a user needs. See EXTENDING.md for the IR contract and a worked example.
+
+_REGISTRY = {}          # name -> {'fn': callable, 'ext': '.xyz'}
+_ALIASES = {'txt': 'text', 'md': 'markdown'}
+
+def emitter(name, ext=None, aliases=()):
+    """Register an output format. Usable as a decorator."""
+    def deco(fn):
+        _REGISTRY[name] = {'fn': fn, 'ext': ext or '.' + name}
+        for a in aliases:
+            _ALIASES[a] = name
+        return fn
+    return deco
+
+def get_emitter(name):
+    return _REGISTRY[_ALIASES.get(name, name)]
+
+def formats():
+    """All registered format names (canonical + aliases), for CLI choices."""
+    return sorted(set(_REGISTRY) | set(_ALIASES))
+
+def load_plugins():
+    """Discover third-party emitters via the 'ctrlkd.emitters' entry-point group."""
+    from importlib.metadata import entry_points
+    for ep in entry_points(group='ctrlkd.emitters'):
+        if ep.name not in _REGISTRY:
+            fn = ep.load()
+            _REGISTRY[ep.name] = {'fn': fn, 'ext': getattr(fn, 'ext', '.' + ep.name)}
+
+def _printed(doc):
+    return doc.meta.get('variant') == 'printstream' or doc.meta.get('columnar')
+
+# ---------------------------------------------------------------- text
+
+def emit_text(doc, mode='modern', **_options):
+    out = []
+    for b in doc.blocks:
+        if b.kind == 'softpage':                 # WordStar's own pagination:
+            if mode == 'printed':                # meaningful only line-for-line
+                out.append('\f')
+            continue
+        if b.kind == 'pagebreak':
+            out.append('\f' if mode == 'printed' else '\n' + '-' * 20 + '\n')
+            continue
+        para = '\n'.join(line.text() for line in b.lines)
+        if para.strip() or mode == 'printed':
+            out.append(para)
+    text = ('\n'.join(out) if mode == 'printed' or _printed(doc)
+            else '\n\n'.join(o for o in out if o.strip()))
+    if doc.footnotes:
+        text += '\n\n' + '\n'.join(f'[{i+1}] {"".join(s.text for s in n)}'
+                                   for i, n in enumerate(doc.footnotes))
+    return text + '\n'
+
+# ---------------------------------------------------------------- markdown
+
+_MD = {'b': '**', 'i': '*', 'strike': '~~'}
+_MD_HTML = {'u': 'u', 'sup': 'sup', 'sub': 'sub'}
+
+def _md_span(s):
+    text = s.text
+    if 'fnref' in s.styles:
+        return f'[^{text}]'
+    if not text.strip():
+        return text
+    esc = text.replace('\\', '\\\\')
+    for ch in '*_#`[]':
+        esc = esc.replace(ch, '\\' + ch)
+    lead = esc[:len(esc) - len(esc.lstrip())]
+    trail = esc[len(esc.rstrip()):]
+    core = esc.strip()
+    for st in s.styles:
+        if st in _MD:
+            core = f'{_MD[st]}{core}{_MD[st]}'
+        elif st in _MD_HTML:
+            t = _MD_HTML[st]
+            core = f'<{t}>{core}</{t}>'
+    return lead + core + trail
+
+def emit_markdown(doc, mode='modern', **_options):
+    if mode == 'printed' or _printed(doc):
+        # alignment is the content: a fenced block is the honest representation
+        body = emit_text(doc, 'printed')
+        return '```\n' + body.rstrip('\n') + '\n```\n'
+    out = []
+    for b in doc.blocks:
+        if b.kind == 'softpage':
+            continue
+        if b.kind == 'pagebreak':
+            out.append('---')
+            continue
+        lines = [''.join(_md_span(s) for s in line.spans) for line in b.lines]
+        para = '\\\n'.join(l for l in lines)          # hard breaks: trailing backslash
+        if b.heading and para.strip():
+            para = '#' * b.heading + ' ' + para.strip()
+        if para.strip():
+            out.append(para)
+    md = '\n\n'.join(out)
+    if doc.footnotes:
+        md += '\n\n' + '\n'.join(f'[^{i+1}]: {"".join(s.text for s in n)}'
+                                 for i, n in enumerate(doc.footnotes))
+    return md + '\n'
+
+# ---------------------------------------------------------------- html
+
+_CSS = """body{max-width:42rem;margin:2rem auto;padding:0 1rem;
+font:17px/1.6 Georgia,serif;color:#222}p{margin:0 0 1em}
+pre{font:14px/1.5 ui-monospace,Menlo,Consolas,monospace;overflow-x:auto}
+hr.pb{border:none;border-top:1px dashed #bbb;margin:2rem 0}
+@media(prefers-color-scheme:dark){body{background:#161616;color:#ddd}
+hr.pb{border-top-color:#444}}"""
+
+_TAG = {'b': 'strong', 'i': 'em', 'u': 'u', 'sup': 'sup', 'sub': 'sub', 'strike': 's'}
+
+def _html_span(s, keep_ws=False):
+    text = _html.escape(s.text)
+    if keep_ws:
+        pass
+    elif text.startswith('     '):                    # typescript indent -> keep visible
+        n = len(text) - len(text.lstrip())
+        text = '&nbsp;' * n + text.lstrip()
+    for st in sorted(s.styles):
+        t = _TAG.get(st)                              # e.g. 'fnref' has no tag of its own
+        if t:
+            text = f'<{t}>{text}</{t}>'
+    return text
+
+def emit_html(doc, mode='modern', title='', **_options):
+    parts = []
+    printed = mode == 'printed' or _printed(doc)
+    for b in doc.blocks:
+        if b.kind == 'softpage':
+            if printed:
+                parts.append('<hr class="pb">')
+            continue
+        if b.kind == 'pagebreak':
+            parts.append('<hr class="pb">')
+            continue
+        if b.heading:
+            txt = ' '.join(''.join(_html_span(s) for s in line.spans)
+                           for line in b.lines).strip()
+            if txt:
+                parts.append(f'<h{b.heading}>{txt}</h{b.heading}>')
+            continue
+        if printed:
+            body = '\n'.join(''.join(_html_span(s, keep_ws=True) for s in line.spans)
+                             for line in b.lines)
+            if body.strip():
+                parts.append(f'<pre>{body}</pre>')
+        else:
+            lines = [''.join(_html_span(s) for s in line.spans) for line in b.lines]
+            para = '<br>\n'.join(lines)
+            if para.strip():
+                parts.append(f'<p>{para}</p>')
+    if doc.footnotes:
+        notes = ''.join(f'<li>{"".join(_html.escape(s.text) for s in n)}</li>'
+                        for n in doc.footnotes)
+        parts.append(f'<hr><ol class="footnotes">{notes}</ol>')
+    return ('<!doctype html><html><head><meta charset="utf-8">'
+            f'<meta name="viewport" content="width=device-width,initial-scale=1">'
+            f'<title>{_html.escape(title)}</title><style>{_CSS}</style></head>\n'
+            f'<body>\n' + '\n'.join(parts) + '\n</body></html>\n')
+
+# ---------------------------------------------------------------- rtf
+
+_RTF_ON = {'b': r'\b ', 'i': r'\i ', 'u': r'\ul ', 'sup': r'\super ',
+           'sub': r'\sub ', 'strike': r'\strike '}
+
+def _rtf_escape(text):
+    out = []
+    for ch in text:
+        if ch in '\\{}':
+            out.append('\\' + ch)
+        elif ord(ch) < 128:
+            out.append(ch)
+        else:
+            out.append(f'\\u{ord(ch)}?')
+    return ''.join(out)
+
+def emit_rtf(doc, mode='modern', **_options):
+    printed = mode == 'printed' or _printed(doc)
+    font = r'\f1' if printed else r'\f0'
+    parts = []
+    for b in doc.blocks:
+        if b.kind == 'softpage':
+            if printed:
+                parts.append(r'\page ')
+            continue
+        if b.kind == 'pagebreak':
+            parts.append(r'\page ')
+            continue
+        lines = []
+        for line in b.lines:
+            seg = ''.join('{' + ''.join(_RTF_ON.get(s, '') for s in sorted(sp.styles))
+                          + _rtf_escape(sp.text) + '}' for sp in line.spans)
+            lines.append(seg)
+        if b.heading:
+            lines = ['{' + r'\b\fs28 ' + l + '}' for l in lines]
+        joiner = r'\line ' if not printed else r'\line '
+        para = joiner.join(lines)
+        if para.strip() or printed:
+            parts.append(para + r'\par ')
+        if not printed:
+            parts.append(r'\par ')                    # blank line between paragraphs
+    body = '\n'.join(parts)
+    return (r'{\rtf1\ansi\deff0{\fonttbl{\f0 Times New Roman;}{\f1 Courier New;}}'
+            + '\n' + font + r'\fs24 ' + '\n' + body + '\n}\n')
+
+# built-ins register through the same door plugins use
+emitter('text', ext='.txt')(emit_text)
+emitter('markdown', ext='.md')(emit_markdown)
+emitter('html', ext='.html')(emit_html)
+emitter('rtf', ext='.rtf')(emit_rtf)

ctrlkd/pdf.py

@@ -0,0 +1,200 @@
+"""ctrl-kd PDF emitter — the page as it would have printed.
+
+Hand-written PDF 1.4, zero dependencies: the base-14 Courier family needs no font
+embedding and its fixed metrics make layout exact. That fits the tool's soul — a
+WordStar document rendered as the typescript it was, on Letter pages:
+
+  printed mode   line-for-line, form feeds / .pa / WordStar's own page breaks
+                 honored — a facsimile of the 1990 printout
+  modern mode    reflowed paragraphs wrapped to the text column, headings bold,
+                 footnotes at the end — still typewriter-set, still Courier
+
+Styles: bold/italic map to the Courier variants, underline is drawn, superscript
+is raised and reduced. Non-Latin-1 characters degrade to '?'.
+"""
+import re as _re
+from .emit import emitter, _printed
+
+PAGE_W, PAGE_H = 612, 792            # US Letter, points
+MARGIN = 72                          # 1 inch
+SIZE, LEAD = 12, 12                  # 10 CPI pica x 6 LPI — the dot-matrix standard;
+                                     # a 65-col WordStar line is exactly 6.5in
+TOP_MODERN, TOP_PRINTED = 72, 36     # print streams carry their own top-margin blanks
+LINES_MODERN = (PAGE_H - 2 * 72) // LEAD                 # 54
+LINES_PRINTED = (PAGE_H - 2 * 36) // LEAD                # 60
+MAX_COLS = int((PAGE_W - 2 * MARGIN) / (SIZE * 0.6))     # 65 — WordStar's own margin
+
+FONTS = {(False, False): 'F1', (True, False): 'F2',
+         (False, True): 'F3', (True, True): 'F4'}
+FONT_NAMES = {'F1': 'Courier', 'F2': 'Courier-Bold',
+              'F3': 'Courier-Oblique', 'F4': 'Courier-BoldOblique'}
+
+def _esc(text):
+    raw = text.encode('latin-1', 'replace')
+    return raw.replace(b'\\', b'\\\\').replace(b'(', b'\\(').replace(b')', b'\\)')
+
+def _wrap_line(spans, width):
+    """Wrap one IR line's spans to `width` columns, preserving styles.
+    Returns a list of segment-lines: [[(text, styles), ...], ...]."""
+    tokens = []                                   # words and space-runs, styled
+    for text, styles in spans:
+        for piece in _re.split(r'( +)', text):
+            if piece:
+                tokens.append((piece, styles))
+    lines, line, col = [], [], 0
+    for text, styles in tokens:
+        if not text.isspace() and col and col + len(text) > width:
+            while line and line[-1][0].isspace():          # no trailing spaces
+                col -= len(line.pop()[0])
+            lines.append(line); line, col = [], 0
+        line.append((text, styles)); col += len(text)
+    while line and line[-1][0].isspace():
+        line.pop()
+    if line or not lines:
+        lines.append(line)
+    return lines
+
+def _doc_to_pagelines(doc, printed):
+    """IR -> list of pages, each a list of segment-lines."""
+    lines = []                                            # None = forced page break
+    for b in doc.blocks:
+        if b.kind == 'pagebreak' or (b.kind == 'softpage' and printed):
+            lines.append(None)
+            continue
+        if b.kind == 'softpage':
+            continue
+        for line in b.lines:
+            spans = [(s.text, s.styles) for s in line.spans]
+            if printed:
+                lines.append(spans)                       # verbatim, no wrap
+            else:
+                lines.extend(_wrap_line(spans, MAX_COLS))
+        if not printed and b.lines:
+            lines.append([])                              # blank line between paragraphs
+    if doc.footnotes:
+        lines += [[], [('-' * 20, frozenset())], []]
+        for i, n in enumerate(doc.footnotes):
+            note = f'[{i + 1}] ' + ''.join(s.text for s in n)
+            lines.extend(_wrap_line([(note, frozenset())], MAX_COLS))
+    cap = LINES_PRINTED if printed else LINES_MODERN
+    pages, page = [], []
+    for l in lines:
+        if l is None or len(page) >= cap:
+            if page or l is None:
+                pages.append(page); page = []
+            if l is None:
+                continue
+        page.append(l)
+    if page:
+        pages.append(page)
+    # We supply the paper margins, so WordStar's own margin blanks in a print
+    # stream would double up. But deliberate spacing (a chapter-drop on page 1)
+    # must survive: the MACHINE margin is uniform on every page, so strip only
+    # the minimum leading-blank count seen on pages 2+ — anything beyond it on
+    # any page is the author's layout. Trailing blanks are always machine.
+    def leading(pg):
+        n = 0
+        while n < len(pg) and not any(t.strip() for t, _ in pg[n]):
+            n += 1
+        return n
+    if printed and pages:
+        machine = min(leading(pg) for pg in pages[1:]) if len(pages) > 1 \
+                  else leading(pages[0])
+        for pg in pages:
+            del pg[:min(machine, leading(pg))]
+            while pg and not any(t.strip() for t, _ in pg[-1]):
+                pg.pop()
+    else:
+        for pg in pages:
+            del pg[:leading(pg)]
+            while pg and not any(t.strip() for t, _ in pg[-1]):
+                pg.pop()
+    return pages or [[]]
+
+def _coalesce(line):
+    """Merge adjacent same-style segments into single text runs."""
+    out = []
+    for text, styles in line:
+        if out and out[-1][1] == styles:
+            out[-1][0] += text
+        else:
+            out.append([text, styles])
+    return out
+
+def _page_stream(pagelines, top):
+    ops = []
+    y = PAGE_H - top - SIZE
+    for line in pagelines:
+        x = MARGIN
+        for text, styles in _coalesce(line):
+            if not text:
+                continue
+            sup = 'sup' in styles or 'sub' in styles
+            size = 8 if sup else SIZE
+            rise = 3 if 'sup' in styles else (-2 if 'sub' in styles else 0)
+            font = FONTS[('b' in styles, 'i' in styles)]
+            ops.append(b'BT /%s %d Tf %d Ts %.1f %.1f Td (%s) Tj ET' %
+                       (font.encode(), size, rise, x, y, _esc(text)))
+            w = len(text) * size * 0.6
+            if 'u' in styles and text.strip():
+                ops.append(b'0.6 w %.1f %.1f m %.1f %.1f l S' % (x, y - 1.5, x + w, y - 1.5))
+            if 'strike' in styles and text.strip():
+                ops.append(b'0.6 w %.1f %.1f m %.1f %.1f l S' % (x, y + 3, x + w, y + 3))
+            x += w
+        y -= LEAD
+    return b'\n'.join(ops)
+
+@emitter('pdf')
+def emit_pdf(doc, mode='modern', **options):
+    """Assemble the PDF: catalog, page tree, four Courier fonts, one content
+    stream per page, xref. Returns bytes — PDF is a binary format."""
+    printed = mode == 'printed' or _printed(doc)
+    pages = _doc_to_pagelines(doc, printed)
+    top = TOP_PRINTED if printed else TOP_MODERN
+    objs = []                                             # (obj_number, bytes)
+
+    n_pages = len(pages)
+    font_objs = {}                                        # F1..F4 -> obj num
+    next_num = 3
+    for f in ('F1', 'F2', 'F3', 'F4'):
+        font_objs[f] = next_num
+        objs.append((next_num,
+                     b'<< /Type /Font /Subtype /Type1 /BaseFont /%s >>'
+                     % FONT_NAMES[f].encode()))
+        next_num += 1
+    font_dict = b' '.join(b'/%s %d 0 R' % (f.encode(), n) for f, n in font_objs.items())
+
+    page_nums, content_nums = [], []
+    for _ in range(n_pages):
+        page_nums.append(next_num); next_num += 1
+        content_nums.append(next_num); next_num += 1
+
+    kids = b' '.join(b'%d 0 R' % n for n in page_nums)
+    objs.insert(0, (1, b'<< /Type /Catalog /Pages 2 0 R >>'))
+    objs.insert(1, (2, b'<< /Type /Pages /Kids [%s] /Count %d >>' % (kids, n_pages)))
+
+    for pnum, cnum, pl in zip(page_nums, content_nums, pages):
+        objs.append((pnum,
+                     b'<< /Type /Page /Parent 2 0 R /MediaBox [0 0 %d %d] '
+                     b'/Resources << /Font << %s >> >> /Contents %d 0 R >>'
+                     % (PAGE_W, PAGE_H, font_dict, cnum)))
+        stream = _page_stream(pl, top)
+        objs.append((cnum, b'<< /Length %d >>\nstream\n%s\nendstream'
+                     % (len(stream), stream)))
+
+    objs.sort()
+    out = bytearray(b'%PDF-1.4\n')
+    offsets = {}
+    for num, body in objs:
+        offsets[num] = len(out)
+        out += b'%d 0 obj\n%s\nendobj\n' % (num, body)
+    xref_at = len(out)
+    count = max(offsets) + 1
+    out += b'xref\n0 %d\n0000000000 65535 f \n' % count
+    for n in range(1, count):
+        out += b'%010d 00000 n \n' % offsets[n]
+    out += (b'trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n'
+            % (count, xref_at))
+    return bytes(out)
+
+emit_pdf.ext = '.pdf'