quantized-lab 0.8.0__py3-none-any.whl

quantized/io/origin_project/opju_curves_allcols.py

@@ -0,0 +1,258 @@
+"""The 0x01-subtype ``.opju`` curve token — the second, all-columns curve
+encoding for ordinary, single-curve *default-dialog* graphs (item 35, the
+"third encoding" search — this closes it, 2026-07-04).
+
+The shipped ``opju_curves.py`` decodes one curve-token family (subtype
+``0x03``), reverse-engineered from custom-styled/multi-curve/Select-Data
+graphs. A prior session searched specifically for how *ordinary* single-
+curve default-dialog graphs (``RockingCurve`` ``Graph1``/``Graph2``, all of
+``XAS``, all of ``UnpolPlots``, most of ``"Fixed Lambdas SI"``) encode their
+Y column and came up empty — three hypotheses chased, none validated (see
+``opju_curves.py``'s docstring, "The third encoding search — negative
+result"). This module is the found fourth hypothesis, now validated end to
+end against the real ``plots.json`` oracle on every one of those files.
+
+**The token.** A whole-file scan for ``[[byte]] 01 01 01 80 01 [[byte]]``
+(the discovery regex ``rb"[\\x00-\\xff]\\x01\\x01\\x01\\x80([\\x01\\x03])(.)"``
+with the subtype fixed to ``0x01``) finds exactly the shape the earlier
+session already isolated in ``RockingCurve``'s ``Graph1``/``Graph2`` objects
+and confirmed *was* a real Origin field (the 4.3811 re-save of the same
+project rewrites the identical slot into the canonical ``0x03`` shape while
+preserving the same numeric value) — but could not decode, because it was
+being read through the ``0x03`` family's ordinal map:
+
+```
+<flag:1> 01 01 01 80 01 <val:1>
+```
+
+Same family as the shipped token (``<flag> 01 01 01 80 03 <y_ord> 00``),
+same fixed ``01 01`` "konst" pair, but subtype byte ``0x01`` instead of
+``0x03``, and **no fixed ``0x00`` terminator** — confirmed by dumping every
+hit's context across the full real corpus (below): the 7 bytes above are
+followed by a short, subtype-specific tail (``83 01 <byte> 82 01 01 80 01``
+for XAS; ``83 07 <2 bytes> 00 00 0b 00`` for RockingCurve; several other
+shapes for UnpolPlots/"Fixed Lambdas SI") that never repeats the fixed
+``0x00`` the ``0x03`` family always carries. No consistent terminator or
+tail structure was found *across files* (each file's tail shape differs),
+so none is folded into the matcher — the 7-byte prefix above is already
+100% precise by itself (see "Validation" below), and adding an unproven
+tail constraint would only risk *losing* matches, never gaining precision.
+The ``<flag>`` byte was checked across every hit and, like the ``0x03``
+family's own flag byte, shows no correlation with column choice — it
+clusters loosely in ``0xae``-``0xc4`` (a per-curve creation-order/style
+counter, consistent with the shipped family's documented flag semantics)
+but is never used as a match constraint.
+
+**The counting convention — the actual bug in every earlier refutation.**
+`opju_curves.py`'s ``_global_column_map`` (the shipped ordinal map) counts
+*only* columns ``opju_codec.scan_columns`` actually FPC-decoded, skipping
+any book with zero decodable columns entirely (e.g. an unused default
+"Book1"). Decoding the 0x01 token's ``<val>`` through that map is exactly
+what the earlier session tried and rejected (``RockingCurve``'s ``Nb!B``
+decoded to ``Nb!C`` — wrong). The real convention is different: ``<val>``
+is a 1-based ordinal counted cumulatively across **every allocated column
+of every workbook, including empty/undecoded books and empty/undecoded
+columns**, in file book-appearance order. Re-decoding through this all-
+columns map (below) resolves every hit exactly:
+
+* ``XAS`` (``Book1``=2 cols, an EMPTY/undecoded default book, then ``Co``=3,
+  ``bl11YIGPy032``=3, ``bl11YIGPy033``=3): ``<val>`` = 5, 8, 11 resolve to
+  exactly ``Co!C``, ``bl11YIGPy032!C``, ``bl11YIGPy033!C`` — the file's
+  full ``plots.json`` oracle, 3/3.
+* ``RockingCurve`` (``NbAu``=7 cols incl. an undecoded ``G``, ``Nb``=5,
+  ``NbAl``=3): ``<val>`` = 9, 14 (each found twice — the composite
+  ``Graph3`` re-references) resolve to ``Nb!B``, ``NbAl!B`` — exactly the
+  2 oracle pairs the shipped ``0x03`` path could not reach (it already
+  recovers ``NbAu!D``/``NbAu!F`` from the multi-curve ``NbAuRocking``
+  layer), bringing this file to 4/4.
+* ``UnpolPlots`` (``Book1``=2, ``J315NdNiO3STO``=3, ``J315NdNiO3ST1``=9,
+  ``PrNiO3STOprof``=3, ``PrNiO3STOrefl``=9): 16 hits collapse to the 8
+  unique oracle pairs (each doubled by a composite-window re-reference),
+  8/8.
+* ``"Fixed Lambdas SI"`` (``Book1``=2, then 10 PNR books x 11 cols each):
+  28 hits collapse to exactly the file's 14 unique oracle pairs (each
+  doubled), 14/14.
+
+No book in this corpus needed anything beyond the FPC-decoded-only map's
+*complement* — the all-columns map — to resolve every hit; every value
+that decodes lands on a real, contiguous, allocated column.
+
+**Building the all-columns map without ground truth.** ``opju_codec._NAME``
+(``rb"[A-Za-z][\\w ]{0,40}_[A-Za-z0-9]{1,4}(?:@\\d{1,2})?"``) matches a
+length-prefixed dataset name for *every* allocated column — including
+empty ones (``XAS``'s ``Book1_A``/``Book1_B``, never decoded by
+``scan_columns`` because ``Book1`` carries no data) — but also binary
+noise (arbitrary byte runs that happen to look like ``t_R``, ``G_L``,
+``U_5``). :func:`_allocated_column_map` filters this down to a clean,
+GT-matching book/column inventory with three checks, all confirmed
+necessary and sufficient across the four real-corpus files (see
+``test_realdata_allocated_column_map_matches_index`` for the exact
+book/count comparison against each file's independently-exported
+``index.json``):
+
+1. Reuse ``scan_columns``'s own length-prefix anchor (``b[m.start()-1] ==
+   len(match)``) — the same check that makes ``_NAME`` usable at all.
+2. Keep only matches whose column suffix is **pure letters, 1-2 chars**
+   (``[A-Z]{1,2}``) and drop any ``@N`` sheet-suffixed match entirely (this
+   map does not track the extra-sheet pseudo-books the FPC-decoded map
+   does — no multi-sheet book in this corpus needed one).
+3. Group by book (everything before the last ``_``) and require the
+   resulting column-letter *set* to be an exact contiguous run starting at
+   ``A`` (``{A}``, ``{A,B}``, ``{A,B,C}``, …) — this is what rejects the
+   noise matches, which never land on a clean contiguous run.
+
+Book order is first-appearance order of a *surviving* book's name records
+in the byte stream (matches every stem's ``index.json`` book order
+exactly). Column letters use the standard spreadsheet base-26 scheme
+(``A``=1..``Z``=26, ``AA``=27, … — length-then-lexicographic).
+
+**No designation gate — a deliberate, checked difference from the ``0x03``
+path.** The shipped path drops any resolved column unless
+``windows_opju``'s independently-validated designation is exactly ``"Y"``
+(never ``"X"``/``"Y-error"``), because that path's whole-file regex alone
+is not curve-exclusive enough (the ``__BCO`` boilerplate false positive).
+Applying the same gate here was checked against every one of the 27 unique
+oracle-confirmed bindings this token resolves and would **wrongly reject
+four of them**: ``UnpolPlots``' ``J315NdNiO3ST1!H``/``PrNiO3STOrefl!H``
+("dR Fresnel") and ``"Fixed Lambdas SI"``'s ``PNRNbAl80nm!J``/
+``PNRNbAu100nm!J`` ("dSA") are genuinely plotted curves per ``plots.json``
+whose column is independently designated ``"Y-error"``, not ``"Y"`` — the
+project plots an uncertainty column as its own curve, which is a legitimate
+Origin usage the designation gate cannot distinguish from the ``__BCO``
+artifact. Since this token's raw 7-byte match is *already* 100% precise
+file-wide with zero designation cross-check (confirmed by scanning every
+``.opju`` in the corpus, both real-corpus files and every specimen —
+zero hits anywhere except the four files that need them, see
+``tools/origin_trial/score_curve_bindings.py``), adding the ``0x03`` path's
+designation gate here would only lose true positives for no precision
+gain. The only safety check applied is structural: an ordinal that exceeds
+the map's total column count for its file, or that the cumulative map
+simply has no entry for, is dropped — never guessed.
+
+**Attribution.** Reuses the same ``[anchor, next_anchor)`` per-figure
+window scoping the ``0x03`` path uses (see ``figures_opju.py`` /
+``opju_curves.extract_curves``). Every hit in this corpus falls inside
+some figure's window (none needed to be dropped for falling outside every
+window), though — like the shipped path's own documented attribution gap
+— a composite/last window can absorb tokens that structurally belong to
+an earlier, already-closed window (``"Fixed Lambdas SI"``'s last anchor's
+span runs to EOF and physically contains all 28 hits for both of its
+book families). This is a known, pre-existing class of imprecision
+(*which* figure a curve is attributed to), not a soundness one (the
+``(book, column)`` pair itself is never wrong) — see
+``opju_curves.py``'s "Known gap — per-figure attribution".
+
+**Validation.** ``tools/origin_trial/score_curve_bindings.py`` re-run
+after wiring this module in: precision 100% (0 wrong) on every stem,
+aggregate recall 36/36 (100%), up from 11/36 (30.6%) before this change —
+see ``docs/origin_project_format.md`` §6.2.1 for the full updated table.
+"""
+
+from __future__ import annotations
+
+import re
+from collections import OrderedDict
+from collections.abc import Mapping
+
+from quantized.io.origin_project.opju_codec import _NAME, curve_plot_style
+
+__all__ = ["extract_curves_allcols"]
+
+# <flag:1> 01 01 01 80 01 <val:1> -- see module docstring. Unlike the shipped
+# 0x03 family, no fixed terminator byte was found, so none is required here.
+_CURVE_RE = re.compile(rb"[\x00-\xff]\x01\x01\x01\x80\x01(.)")
+
+# A column suffix eligible for the all-columns map: pure letters, any width,
+# no sheet suffix (that case is dropped by the caller before this check runs).
+# No length cap: the old {1,2} silently discarded AAA+ columns of a >702-col
+# book while the survivors still formed a contiguous A..ZZ run, shifting every
+# later book's cumulative ordinal base (2026-07-06 genericity audit) —
+# ``_letter_index`` already handles arbitrary-width bijective base-26.
+_PURE_COLUMN = re.compile(r"[A-Z]+")
+
+
+def _letter_index(letters: str) -> int:
+    """Standard spreadsheet column lettering: A=1, .., Z=26, AA=27, .."""
+    idx = 0
+    for ch in letters:
+        idx = idx * 26 + (ord(ch) - ord("A") + 1)
+    return idx
+
+
+def _index_to_letters(n: int) -> str:
+    """Inverse of :func:`_letter_index`."""
+    out = ""
+    while n > 0:
+        n, rem = divmod(n - 1, 26)
+        out = chr(ord("A") + rem) + out
+    return out
+
+
+def _allocated_column_map(b: bytes) -> OrderedDict[str, int]:
+    """``{book: total allocated column count}``, book-appearance order.
+
+    Counts EVERY allocated column of every workbook (including empty/
+    undecoded books and columns) -- the universe the 0x01 token's ``val``
+    counts over, unlike ``opju_curves._global_column_map``'s FPC-decoded-
+    only universe. See the module docstring's "Building the all-columns
+    map" section for the filter rule and its validation.
+    """
+    names = [
+        m.group(0).decode("latin1")
+        for m in _NAME.finditer(b)
+        if m.start() > 0 and b[m.start() - 1] == len(m.group(0))
+    ]
+    seen: OrderedDict[str, set[str]] = OrderedDict()
+    for name in names:
+        base, _, sheet = name.partition("@")
+        if sheet:
+            continue  # drop sheet-suffixed matches -- see module docstring
+        book, _, col = base.rpartition("_")
+        if not book or not _PURE_COLUMN.fullmatch(col):
+            continue
+        seen.setdefault(book, set()).add(col)
+    out: OrderedDict[str, int] = OrderedDict()
+    for book, cols in seen.items():
+        n = len(cols)
+        if cols == {_index_to_letters(i) for i in range(1, n + 1)}:
+            out[book] = n
+    return out
+
+
+def _cumulative_ordinals(book_counts: Mapping[str, int]) -> dict[int, tuple[str, str]]:
+    """1-based cumulative ordinal -> ``(book, column letter)`` over EVERY
+    allocated column of EVERY book (contrast ``opju_curves._global_column_map``,
+    which only counts FPC-decoded columns) -- see module docstring."""
+    out: dict[int, tuple[str, str]] = {}
+    cum = 0
+    for book, n in book_counts.items():
+        for i in range(1, n + 1):
+            cum += 1
+            out[cum] = (book, _index_to_letters(i))
+    return out
+
+
+def extract_curves_allcols(
+    b: bytes, start: int, end: int, book_counts: Mapping[str, int]
+) -> list[dict[str, str | float]]:
+    """Every curve's ``{book, x, y}`` binding found via the 0x01-subtype
+    token in ``b[start:end)``.
+
+    No designation gate is applied (see module docstring — it would reject
+    genuine Y-error-plotted-as-curve bindings this token legitimately
+    finds); the only safety check is structural: a ``val`` with no entry in
+    the cumulative map (out of range, or the book/column doesn't exist) is
+    dropped, never guessed. ``x`` is the resolved book's first column
+    ("A") -- always present by construction, since :func:`_allocated_column_map`
+    only keeps books whose columns form a contiguous run starting at A.
+    """
+    ordmap = _cumulative_ordinals(book_counts)
+    out: list[dict[str, str | float]] = []
+    for m in _CURVE_RE.finditer(b, start, end):
+        info = ordmap.get(m.group(1)[0])
+        if info is None:
+            continue
+        book, y_col = info
+        style = curve_plot_style(b, m.start())
+        out.append({"book": book, "x": "A", "y": y_col, **({"style": style} if style else {})})
+    return out

quantized/io/origin_project/opju_figure_curves.py

@@ -0,0 +1,302 @@
+"""``.opju`` (CPYUA) figure->curve binding via the global column-id table.
+
+**The discovery (2026-07-05, the Hc2 per-graph rework).** The two curve-token
+"families" ``opju_curves.py`` / ``opju_curves_allcols.py`` decode by *counting
+columns* (``<flag> 01 01 01 80 03 <y_ord> 00`` and ``… 80 01 <val>``) are in
+fact ONE encoding, and the value is not an ordinal to count at all: it is the
+plotted column's own **global, project-wide, creation-order serial id** — the
+exact CPYUA analogue of the ``.opj`` curve-anchor id ``opj_curves.py`` decodes.
+The token's id field is a tagged variable-width little-endian integer:
+
+```
+<flag:1> 01 01 01 80 <width:01|03> <payload>
+    width 0x01 -> payload = <id:u8>
+    width 0x03 -> payload = <id:u16 LE> <flag:1>   (3rd byte varies: 01/09/0b/
+                                                    0c/11/21 observed; not id)
+```
+
+The old ``0x03``-family regex required the byte after the id byte to be
+``0x00`` — that byte is really the u16 id's HIGH byte, so the old reader
+worked only on projects with < 256 columns and silently aliased ids mod 256
+on larger ones (measured on ``Hc2 data.opju``: 1251+ allocated columns, 12 of
+14 decoded bindings wrong before this rework). The counting conventions the
+old modules validated were an artifact: in a never-edited project, creation
+order equals current layout order, so the cumulative all-columns ordinal
+happens to equal the stored id. ``Hc2 data`` (heavily edited: columns added
+to early books after later books existed, e.g. the ``Derivative Y1`` column
+``AH`` of the first Lockin book carrying id 132) breaks the count and proves
+the id semantics.
+
+**The id table.** Every worksheet column's own windows-section record stores
+that id. Two record forms exist (both validated against the per-column
+designation-marker runs ``windows_opju.py`` independently decodes):
+
+```
+form A:  80 <serial> 01 10 80 03 <id:u16 LE> <pb> <fields…>
+form B:  80 <serial> 07 10 01 00 00 <id:u16 LE> <pb> <fields…>
+```
+
+``<pb>`` is one uninterpreted byte (``03/09/0b/0c`` observed). ``<fields…>``
+is a run of tagged fields ``<tag:0x80-0x9f> <len:u8> <payload:len>``:
+
+* the column's **short name**: payload = optional designation prefix byte
+  (``0x03`` on X columns, ``0x02`` on Y-error columns) + the ASCII name;
+  identified as the first alnum-payload field whose *next* field's payload
+  opens with ``0x09``;
+* a fixed ``<tag> 01 09`` separator field;
+* the field carrying the 2-byte **plot-designation marker** as its payload
+  tail — ``21 51`` X / ``21 61`` Y / ``30 61`` Y-error, the same markers
+  ``windows_opju.py`` anchors label runs on; a Y column's marker field is
+  exactly 4 bytes — ``<x_partner_id:u16 LE> 21 61`` — giving the column's own
+  designated **X partner column id** (validated: Hc2's ``Derivative Y1``
+  column AH, id 132, carries partner id 131 = its sibling ``Derivative X1``
+  column AG, not the sheet's column A).
+
+Records are attributed to their book by the containing **page span**: the
+``0a``-framed page headers ``tree_opju._OPJU_WIN_RE`` already enumerates
+(byte-exact vs live COM on 5 corpus files) mark every window's start; a
+column record belongs to the page (workbook) whose span it falls in.
+
+**Form B semantics are unknown** beyond carrying the same id+fields: it is
+rare (RockingCurve ``NbAu!D``/``NbAl!B``, UnpolPlots ``PrNiO3STOprof!B``/
+``PrNiO3STOrefl!I`` — exactly the four bindings the form-A-only table
+missed) and never overlaps form A ids (checked corpus-wide: 0 overlaps in
+28 files). Both forms parse identically after the id.
+
+**Validation (2026-07-05).**
+
+* File-level ``plots.json`` oracle (7 stems, 36 unique pairs): 36/36 decoded,
+  0 wrong — same aggregate as the shipped counting decoders, now via ids.
+* Per-graph ``index.json`` oracle (``Hc2 data``, the first stem whose export
+  populated ``graphs[].layers[].plots``): decoded figures are matched to
+  oracle graphs by their *page name* (see ``figures_opju.py``); 7 of the 8
+  oracle graphs that exist as real graph pages bind exactly (Graph1/2/4/6/8/
+  10/11 — every ``(book, column)`` set identical to the oracle, including
+  u16 ids > 255 and the id-132 later-added column). 0 wrong bindings.
+* ``Graph5`` (single-curve, [A6221LockinD3]!I) is a **documented negative**:
+  its page span contains no curve token at all — its two DataPlot-magic
+  objects carry no recognisable id field (a diffed/duplicate-window form,
+  byte-dumped in the RE log). It decodes with ``curves == []`` (missing,
+  never guessed). Do NOT be tempted by the ``90 00 80 <tag> 01 89`` bytes in
+  its ``_202``/``_232`` sub-objects: that shape occurs in EVERY graph page of
+  the corpus with the constant 0x89 (and 0x02) regardless of what is plotted
+  — a style-boilerplate coincidence that happens to equal Graph5's true
+  column id 137, chased and refuted during this rework.
+* Embedded fit-report graphs (the oracle's ``FitLine*``/``Residual*`` pages,
+  which are NOT ``0a``-framed pages): their report-sheet spans (Book2/Book3)
+  carry one canonical token per embedded layer anchor, resolving to the
+  fitted source column (Book2!C/D/E) — in-oracle, but per-graph attribution
+  for them is unverifiable (no page name), so they ship on unnamed figures
+  only. The fit-CURVE overlays (``FitNLCurveN!B`` etc.) are not
+  token-encoded anywhere and are honestly missing.
+
+The old counting decoders stay in place as the fallback for byte streams
+with no id table at all (synthetic fixtures; degraded files): see
+``figures_opju.extract_figures_opju``.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import NamedTuple
+
+from quantized.io.origin_project.curve_style_color import (
+    apply_increment_colors,
+    opju_style_record,
+    style_fields,
+)
+from quantized.io.origin_project.opju_codec import curve_plot_style
+from quantized.io.origin_project.tree_opju import iter_opju_windows
+
+__all__ = ["ColumnIdTable", "column_id_table", "extract_curves_by_id", "opju_pages"]
+
+# Column-record id field, form A / form B (see module docstring).
+_ID_FORM_A = re.compile(rb"\x01\x10\x80\x03(..)", re.DOTALL)
+_ID_FORM_B = re.compile(rb"\x07\x10\x01\x00\x00(..)", re.DOTALL)
+
+# The unified curve token: <flag> 01 01 01 80 <width> <id…> (module docstring).
+_CURVE_TOKEN = re.compile(rb"\x01\x01\x01\x80([\x01\x03])", re.DOTALL)
+
+# Per-column plot-designation markers (same bytes windows_opju.py validates).
+_MARKS = {b"\x21\x51": "X", b"\x21\x61": "Y", b"\x30\x61": "Y-error"}
+
+_MAX_FIELDS = 10  # field-walk runaway backstop; real records resolve in <= 6
+# Column short-name gate: no 16-char ceiling (the old {0,15} was a corpus
+# maximum — a longer user-renamed short name made the column's id unresolvable
+# and silently dropped every curve plotting it; 2026-07-06 genericity audit).
+# The structural guards (the 09-separator field walk + designation marker)
+# carry the precision; digit-led short names are legal in Origin.
+_NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9_]*\Z")
+
+
+class ColumnIdTable(NamedTuple):
+    """The decoded global column-id table of one ``.opju`` byte stream.
+
+    ``ids``: column id -> ``(book page name, column short name)``.
+    ``x_partner``: a Y column's id -> its designated X partner column's id.
+    ``book_x``: book -> its first X-designated column's short name.
+    ``book_pages``: page names that own >= 1 column record (i.e. workbook /
+    report-table pages — used by ``figures_opju`` to tell a graph page from
+    a book page when attaching window names to figures).
+    """
+
+    ids: dict[int, tuple[str, str]]
+    x_partner: dict[int, int]
+    book_x: dict[str, str]
+    book_pages: frozenset[str]
+
+
+def opju_pages(b: bytes) -> list[tuple[int, str]]:
+    """Every ``0a``-framed page header as ``(offset, name)``, file order.
+
+    Reuses the exact enumeration ``tree_opju`` validated byte-exact against
+    live COM (first occurrence per name; the ``namelen+2`` length prefix
+    drives the name span, so arbitrary-length window names enumerate — a
+    missed header here would attribute every column in its span to the
+    previous page).
+    """
+    return iter_opju_windows(b)
+
+
+def _walk_fields(b: bytes, p: int) -> list[bytes]:
+    """The tagged-field run at ``p``: payload per ``<tag 0x80-0x9f> <len>
+    <payload>`` field, stopping at the first byte that isn't a field tag."""
+    out: list[bytes] = []
+    n = len(b)
+    for _ in range(_MAX_FIELDS):
+        if p + 2 > n or not 0x80 <= b[p] <= 0x9F:
+            break
+        ln = b[p + 1]
+        payload = b[p + 2 : p + 2 + ln]
+        if len(payload) < ln:
+            break
+        out.append(payload)
+        p += 2 + ln
+    return out
+
+
+def _parse_column_record(b: bytes, after_id: int) -> tuple[str, str | None, int | None] | None:
+    """Decode one column record's field run (starting just past the id).
+
+    Returns ``(short_name, designation, x_partner_id)`` or ``None`` when the
+    record doesn't resolve — dropped, never guessed. The name field must be
+    followed by the fixed ``09`` separator field AND a designation-marker
+    field within the next 3 fields (both checks together are what reject
+    coincidental byte runs; see module docstring).
+    """
+    fields = _walk_fields(b, after_id + 1)  # +1: skip the uninterpreted <pb> byte
+    for i in range(len(fields) - 1):
+        payload = fields[i]
+        body = payload[1:] if payload[:1] in (b"\x02", b"\x03") else payload
+        try:
+            name = body.decode("ascii")
+        except UnicodeDecodeError:
+            continue
+        if not _NAME_RE.match(name) or fields[i + 1][:1] != b"\x09":
+            continue
+        for later in fields[i + 1 : i + 4]:
+            if len(later) >= 2 and later[-2:] in _MARKS:
+                desig = _MARKS[later[-2:]]
+                partner: int | None = None
+                if desig == "Y" and len(later) == 4:
+                    partner = int.from_bytes(later[:2], "little")
+                return name, desig, partner
+        return None  # name shape found but no designation marker: not a column record
+    return None
+
+
+def column_id_table(b: bytes, pages: list[tuple[int, str]]) -> ColumnIdTable:
+    """Scan every page span for column records and build the global id table.
+
+    A record whose fields don't resolve is skipped; an id claimed twice with
+    *different* ``(book, column)`` is poisoned (removed entirely) — fail
+    closed, never guess. Records outside any page span cannot be attributed
+    to a book and are ignored.
+    """
+    ids: dict[int, tuple[str, str]] = {}
+    poisoned: set[int] = set()
+    x_partner: dict[int, int] = {}
+    book_x: dict[str, str] = {}
+    book_pages: set[str] = set()
+    bounds = [*pages, (len(b), "")]
+    for (start, book), (end, _next) in zip(bounds, bounds[1:], strict=False):
+        for pattern in (_ID_FORM_A, _ID_FORM_B):
+            for m in pattern.finditer(b, start, end):
+                cid = int.from_bytes(m.group(1), "little")
+                parsed = _parse_column_record(b, m.end())
+                if parsed is None:
+                    continue
+                name, desig, partner = parsed
+                if cid in ids and ids[cid] != (book, name):
+                    poisoned.add(cid)
+                    continue
+                ids[cid] = (book, name)
+                book_pages.add(book)
+                if partner is not None:
+                    x_partner[cid] = partner
+                if desig == "X" and book not in book_x:
+                    book_x[book] = name
+    for cid in poisoned:
+        ids.pop(cid, None)
+        x_partner.pop(cid, None)
+    return ColumnIdTable(ids, x_partner, book_x, frozenset(book_pages))
+
+
+def extract_curves_by_id(
+    b: bytes, start: int, end: int, table: ColumnIdTable
+) -> list[dict[str, str | float]]:
+    """Every curve binding in ``b[start:end)`` via the unified id token.
+
+    ``y`` resolves through ``table.ids`` (a token whose id is unknown is
+    dropped, never guessed). ``x`` is the Y column's own stored X-partner
+    when its record carried one, else the book's first X-designated column,
+    else ``"A"`` (the documented structural fallback). Deduped on
+    ``(book, y)`` in first-seen (token) order; ``style`` is attached when
+    ``opju_codec.curve_plot_style`` finds the ``8f 01 <style> 83`` tag
+    (falling back to the reconstructed record's own style byte), and
+    ``color``/``symbol`` when the token's sparse style record decodes
+    (``curve_style_color.py`` -- oracle-verified on the ``Hc2 data``/
+    ``RockingCurve``/``UnpolPlots`` ``curve_style.json`` oracle; auto/
+    undecodable fields stay absent, never defaulted).
+    """
+    out: list[dict[str, str | float]] = []
+    records: list[bytes | None] = []
+    seen: set[tuple[str, str]] = set()
+    for m in _CURVE_TOKEN.finditer(b, start, end):
+        width = m.group(1)[0]
+        if m.end() + width > len(b):
+            continue
+        if width == 1:
+            cid = b[m.end()]
+        else:
+            cid = int.from_bytes(b[m.end() : m.end() + 2], "little")
+        info = table.ids.get(cid)
+        if info is None:
+            continue
+        book, y_col = info
+        key = (book, y_col)
+        if key in seen:
+            continue
+        seen.add(key)
+        partner = table.x_partner.get(cid)
+        partner_info = table.ids.get(partner) if partner is not None else None
+        if partner_info is not None and partner_info[0] == book:
+            x_col = partner_info[1]
+        else:
+            x_col = table.book_x.get(book, "A")
+        curve: dict[str, str | float] = {"book": book, "x": x_col, "y": y_col}
+        # the token IS the sparse form of the .opj curve-anchor record; the
+        # id chunk's 0x80 tag sits 3 bytes into the regex match
+        record = opju_style_record(b, m.start() + 3)
+        if record is not None:
+            curve.update(style_fields(record))
+        # the shipped 8f-tag reader stays authoritative for line/scatter
+        style = curve_plot_style(b, m.start())
+        if style:
+            curve["style"] = style
+        out.append(curve)
+        records.append(record)
+    # auto/increment placeholders resolve by group role + plot order
+    # (curve_style_color.apply_increment_colors, pixel-oracle-verified)
+    apply_increment_colors(out, records)
+    return out