mostlyright-data 0.9.0__py3-none-any.whl

mostlyright/data_harness/readers/grib2/admission.py

@@ -0,0 +1,722 @@
+"""Admission for weather-model messages: the bounded header walk and the closed pair allowlist.
+
+This module decides whether a GRIB2 message may be opened at all, and it decides it without
+reading a single packed value.  That is possible because a GRIB2 message states its geometry
+and its packing method in its headers, ahead of the data, so both decisions are available for
+the price of a few hundred bytes.  It is also necessary, because the expensive part of a
+weather Reader is not the arithmetic -- it is knowing which files to refuse.
+
+What a message is made of
+-------------------------
+A message is a chain of numbered sections between a sixteen-byte indicator and a four-byte
+trailer.  Six of them matter here.
+
+* **Section 0**, the indicator, carries the magic ``GRIB``, the discipline, the edition, and
+  the message's own total length.  Edition 2 only; edition 1 is a different format wearing the
+  same first four bytes.
+* **Section 1** identifies the producing centre and the reference time.  Nothing here reads it;
+  its span is recorded so the weather family can read the reference time without a second walk.
+* **Section 2** is optional local use, and is walked over.
+* **Section 3** defines the grid: how the values are laid out on the earth.  It carries the
+  number of data points the grid holds and the grid definition template number, and those two
+  numbers are the whole of this module's interest in it.
+* **Section 4** describes the product -- which variable, at which level, valid at what time.
+  Nothing here reads it either: which variable a message carries is not an admission question.
+  Its span is recorded for the same reason section 1's is.
+* **Section 5** describes the packing: how the values were compressed into bits.  It carries
+  the packing template number, the reference value, the binary and decimal scale factors, and
+  the bit width.
+* **Section 6** is the bitmap, which says whether some points have no value.
+* **Section 7** holds the packed values.  **This module never reads it.**  It records where it
+  begins and where it ends, and stops.
+* **Section 8** is the four-byte trailer ``7777`` at the declared end.
+
+Why the allowlist is a set of pairs
+-----------------------------------
+``ADMITTED_TEMPLATE_PAIRS`` holds ``(grid template number, packing template number)`` tuples
+and is not two independent sets, because the risk is combinatorial.  A grid we can locate and
+a packing we can unpack are each necessary and neither is sufficient: unpacking values
+correctly and then placing them on a grid we have never checked produces plausible numbers at
+wrong coordinates, which is the worst failure this Reader has available to it.  So the unit of
+approval is the combination, and every pair on the list was read out of a real message that is
+vendored in ``../messages``.  A pair nobody has a message for is not on the list, and that
+rule is what stops the list growing by optimism.
+
+The allowlist is narrower than the decoder. Complex packing without spatial differencing (5.2)
+and run-length packing (5.200) are not admitted because no vendored sample declares them.
+
+Why nothing here decodes
+------------------------
+``admit`` returns the offsets of the data section and never reads inside them, so "refused
+before decode" is a property of the shape of the code rather than of the order of its
+statements. The decoder takes a ``MessageAdmission`` as its argument. A
+message nobody admitted therefore has no way to reach a decoder: it is not that we remember to
+check, it is that there is nothing to pass.
+
+The walk is bounded twice over
+------------------------------
+Section lengths are attacker-controlled integers, and a length of zero is the classic
+unbounded walk.  Every length is validated against the bytes remaining before it is followed:
+a length too small to hold its own five-byte header is refused, and so is one that runs past
+the declared end.  Section numbers must also strictly ascend, which both refuses a multi-field
+message -- out of scope for this Reader -- and caps the walk at seven steps independently of
+the length arithmetic.  Two bounds, either of which alone would be sufficient, because this is
+the loop an attacker most wants to own.
+
+The declared-geometry budget
+----------------------------
+A message can declare a grid of two million points in under two hundred bytes: set the bit
+width to zero and every value is the reference value.  One of the vendored messages does
+exactly that, and it is a real operational product rather than a contrivance.  So the ceiling
+that matters is on the **declared** point count, checked against ``max_declared_cells`` from
+the Reader budgets, and it is checked here where the declaration is read.  A budget expressed
+as a ratio of output bytes to input bytes cannot see this class at all.
+
+Lossy codecs are not an allowlist matter
+----------------------------------------
+JPEG 2000 packing is refused with its own wording and its own code, deliberately not as an
+unknown template. Adding it to the allowlist does not bypass the refusal.
+
+That refusal is the outer of two independent layers.  The inner one is that the vendored
+decoder is compiled with its JPEG 2000 feature disabled, so the codec is absent from the binary.
+
+Import discipline: ``readers.contracts`` and the standard library.  Nothing from
+``acquisition``, ``sources``, ``recipe``, or ``pipeline``.
+"""
+
+from __future__ import annotations
+
+import struct
+from collections.abc import Mapping
+from dataclasses import dataclass
+from typing import NoReturn
+
+from mostlyright.data_harness.readers.contracts import ReaderBudgets, ReaderError
+
+__all__ = [
+    "ADMISSION_ALLOWLIST_VERSION",
+    "ADMITTED_TEMPLATE_PAIRS",
+    "ALLOWLIST_CONSTANT",
+    "ALLOWLIST_VERSION_CONSTANT",
+    "LOSSY_PACKING_TEMPLATES",
+    "MessageAdmission",
+    "admit",
+]
+
+# The closed allowlist.  Every pair here was read out of a message vendored in ``../messages``:
+#
+#   (3.0,  5.3)  GFS at one degree, regular latitude/longitude, complex packing with spatial
+#                differencing
+#   (3.30, 5.0)  HRRR categorical rain, Lambert conformal, simple packing
+#   (3.30, 5.3)  HRRR 2 m temperature, Lambert conformal, complex packing with spatial
+#                differencing -- the flagship's own field
+#
+# Note what is absent and note that it is absent on purpose: (3.0, 5.0) is every member of the
+# list recombined, and it is refused, because a combination is admitted by evidence and not by
+# arithmetic on the members.
+ADMITTED_TEMPLATE_PAIRS: frozenset[tuple[int, int]] = frozenset(
+    {
+        (0, 3),
+        (30, 0),
+        (30, 3),
+    }
+)
+
+# The name a refusal tells an operator to edit.  Held as a constant so the message and the
+# thing it names cannot drift apart under a rename.
+ALLOWLIST_CONSTANT = "ADMITTED_TEMPLATE_PAIRS in readers/grib2/admission.py"
+
+# Which allowlist a recipe was written against.  **Bump this whenever the pair set above
+# changes**, in the same edit, because a recipe seals this number and a Reader that has learned a
+# new template refuses a recipe pinned to an older one.
+#
+# The reason is the direction the refusals above do not cover.  Those handle a message this
+# product has never checked.  This handles the opposite case: the recipe was frozen when the list
+# held three pairs, the Toolbox has since learned a fourth, and the same recipe would now decode
+# a file it would previously have refused.  That is a change in what the recipe *means*, arriving
+# with no change to the recipe, which is exactly what a sealed recipe exists to prevent.  So the
+# recipe states the version it was written against and a mismatch halts, naming both.
+#
+# Held to the pair set by test: ``tests/h3/test_readers_grib2.py`` pins the version and the pairs
+# together, so widening one without the other fails.
+ADMISSION_ALLOWLIST_VERSION = 1
+
+# The name a version-mismatch refusal tells an operator to read, carrying the current value so a
+# person reading the message knows what this build admits without opening the source.
+ALLOWLIST_VERSION_CONSTANT = (
+    f"ADMISSION_ALLOWLIST_VERSION, which is {ADMISSION_ALLOWLIST_VERSION} in this build, "
+    f"beside {ALLOWLIST_CONSTANT}"
+)
+
+# JPEG 2000 code-stream identifiers are always refused, independently of the allowlist.
+LOSSY_PACKING_TEMPLATES: frozenset[int] = frozenset({40, 40000})
+
+_MAGIC = b"GRIB"
+_TRAILER = b"7777"
+_EDITION = 2
+_INDICATOR_LENGTH = 16
+_TRAILER_LENGTH = 4
+_SECTION_HEADER_LENGTH = 5
+
+# The sections edition 2 defines between the indicator and the trailer, and the ones a
+# single-field message must carry.  Section 2 is local use and optional; the rest are not.
+_SECTION_NUMBERS = frozenset({1, 2, 3, 4, 5, 6, 7})
+_REQUIRED_SECTIONS = (1, 3, 4, 5, 6, 7)
+
+# Bitmap indicators this module understands.  255 means every point has a value; 0 means the
+# bitmap is carried in section 6 itself.  254 means "the bitmap defined in an earlier message",
+# which cannot be resolved when exactly one message is admitted at a time, and 1 to 253 are
+# reserved and therefore mean nothing.
+_NO_BITMAP = 255
+_BITMAP_PRESENT = 0
+
+# Packing octets 12 to 21 are shared by the simple and complex families and by the codec
+# templates, which is why they can be read before knowing which of them applies.
+_PACKING_PARAMETERS_END = 21
+
+
+@dataclass(frozen=True)
+class MessageAdmission:
+    """What one admitted message declares about itself, and where its values sit.
+
+    Every field is a number.  There are deliberately no bytes on this record: it locates the
+    data section rather than carrying it, so a caller that wants the values has to go back to
+    the message with these offsets in hand, and the act of doing so is visible.
+
+    The grid section is located the same way and for the same reason.  This module reads two
+    numbers out of it -- the point count and the template number -- because those two are the
+    whole of admission's interest in the geometry; the projection parameters are read by
+    ``geometry`` from an admitted message, and it locates them from ``grid_start`` and
+    ``grid_length`` rather than by walking the sections a second time.  One walk, bounded once,
+    in the module that refuses.
+
+    The identification and product sections are located for the same reason and read by nobody
+    here.  Which variable a message carries, at which level, and at which time are not admission
+    questions -- admission refuses on geometry and packing -- but they are questions the weather
+    family must answer, and answering them by walking the sections a second time would put a
+    second bounded walk in a module that does not own one.
+    """
+
+    edition: int
+    discipline: int
+    total_length: int
+    grid_template: int
+    packing_template: int
+    declared_points: int
+    packed_points: int
+    reference_value: float
+    binary_scale: int
+    decimal_scale: int
+    bits_per_value: int
+    original_field_type: int
+    bitmap_indicator: int
+    identification_start: int
+    identification_length: int
+    grid_start: int
+    grid_length: int
+    product_start: int
+    product_length: int
+    data_start: int
+    data_end: int
+
+    @property
+    def template_pair(self) -> tuple[int, int]:
+        """The admitted combination, as the allowlist spells it."""
+
+        return (self.grid_template, self.packing_template)
+
+    @property
+    def data_length(self) -> int:
+        """How many bytes of packed values the message holds."""
+
+        return self.data_end - self.data_start
+
+    @property
+    def has_bitmap(self) -> bool:
+        """Whether some grid points carry no value."""
+
+        return self.bitmap_indicator == _BITMAP_PRESENT
+
+
+def admit(
+    content: bytes,
+    budgets: ReaderBudgets,
+    *,
+    origin: str | None = None,
+) -> MessageAdmission:
+    """Read one message's headers and decide whether this product will open it.
+
+    Returns a description of the message, or raises ``ReaderError``.
+
+    ``origin`` is for a caller that already knows what it is looking at -- a sample check, a
+    tool run over a directory of messages -- and it is absent on the Reader contract's own
+    path by design.  ``ReaderFamily.decode`` is handed bytes, a pin and budgets and no
+    provenance whatever, and that is the property that makes a Reader refusal safe to carry
+    out of the confinement: a decoder that never learns a path cannot name one.  On the
+    production path the recipe's own name for the source is attached one layer up, by
+    ``sources.adapters._decode_in_clean_room``, which is the layer that knows it.
+
+    Nothing here reads the data section, and nothing here allocates in proportion to what the
+    message declares.
+    """
+
+    where = _origin_clause(origin)
+    if not isinstance(content, (bytes, bytearray, memoryview)):
+        _refuse("READER_ADMISSION", "reader.grib2.message", "must be exact bytes")
+    if not isinstance(budgets, ReaderBudgets):
+        _refuse(
+            "READER_BUDGET",
+            "reader.grib2.budgets",
+            "must be the Reader contract's budgets, so a cap cannot be invented at a call site",
+        )
+
+    held = len(content)
+    if held > budgets.max_input_bytes:
+        _refuse(
+            "READER_BUDGET",
+            "reader.grib2.message",
+            f"holds {held} bytes{where}, over the {budgets.max_input_bytes}-byte input budget",
+        )
+    if held < _INDICATOR_LENGTH + _TRAILER_LENGTH:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"holds {held} bytes{where}, too few to be a weather message at all",
+        )
+    if content[:4] != _MAGIC:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"does not begin with the GRIB magic{where}, so it is not a weather message",
+        )
+    edition = content[7]
+    if edition != _EDITION:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"declares edition {edition}{where}; this Reader reads edition {_EDITION}, and "
+            f"edition 1 is a different format that happens to share the first four bytes",
+        )
+
+    total = int.from_bytes(content[8:16], "big")
+    if total != held:
+        _refuse_a_length_that_is_not_the_file(content, total=total, held=held, where=where)
+    if content[total - _TRAILER_LENGTH : total] != _TRAILER:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"has no trailer at its declared end{where}, so the message it declares is not the "
+            f"message it holds",
+        )
+
+    sections = _walk(content, total, where)
+    grid = _read_grid_section(content, sections[3], where)
+    packing_template = _read_packing_template(content, sections[5], where)
+    _refuse_a_lossy_codec(packing_template, where)
+    _admit_the_pair(grid.template, packing_template, where)
+
+    declared_points = grid.points
+    if declared_points > budgets.max_declared_cells:
+        _refuse(
+            "READER_BUDGET",
+            "reader.grib2.declared_points",
+            f"declares a grid of {declared_points} points{where}, over the "
+            f"{budgets.max_declared_cells}-point budget; refused from the header, with nothing "
+            f"read and nothing allocated",
+        )
+
+    packing = _read_packing_parameters(content, sections[5], packing_template, where)
+    bitmap = _read_bitmap_indicator(content, sections[6], where)
+    _agree_on_the_point_count(declared_points, packing.points, bitmap, where)
+    data_start, data_end = _locate_data(sections[7])
+
+    return MessageAdmission(
+        edition=edition,
+        discipline=content[6],
+        total_length=total,
+        grid_template=grid.template,
+        packing_template=packing_template,
+        declared_points=declared_points,
+        packed_points=packing.points,
+        reference_value=packing.reference_value,
+        binary_scale=packing.binary_scale,
+        decimal_scale=packing.decimal_scale,
+        bits_per_value=packing.bits_per_value,
+        original_field_type=packing.original_field_type,
+        bitmap_indicator=bitmap,
+        identification_start=sections[1][0],
+        identification_length=sections[1][1],
+        grid_start=sections[3][0],
+        grid_length=sections[3][1],
+        product_start=sections[4][0],
+        product_length=sections[4][1],
+        data_start=data_start,
+        data_end=data_end,
+    )
+
+
+def _agree_on_the_point_count(declared: int, packed: int, bitmap: int, where: str) -> None:
+    """Hold the grid's point count and the packing's against each other, on the bitmap's terms.
+
+    Section 3 says how many points the grid holds.  Section 5 says how many points a value was
+    packed *for*.  With no bitmap those are the same question and the two numbers must agree,
+    because picking one of them when they disagree is how a field ends up read shifted -- which
+    produces real numbers at the wrong coordinates and raises nothing.
+
+    With a bitmap they are different questions: the bitmap marks points that carry no value, so
+    fewer values are packed than the grid holds.  Requiring equality there would refuse every
+    message that marks a point absent, which is most published fields outside a full-globe
+    analysis.  What stays refused is a packing covering more points than the grid has, which
+    cannot be placed under any reading of either number.
+    """
+
+    if bitmap == _BITMAP_PRESENT:
+        if packed <= declared:
+            return
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"says its grid holds {declared} points and packs values for {packed} of them{where}; "
+            f"a bitmap marks points that carry no value, so a message may pack fewer than its "
+            f"grid holds and can never pack more",
+        )
+    if packed != declared:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"says its grid holds {declared} points and its packing covers {packed}{where}; with "
+            f"no bitmap the two must agree, because choosing one of them is how values end up at "
+            f"the wrong coordinates",
+        )
+
+
+# --- The walk ---------------------------------------------------------------------------
+
+
+def _walk(content: bytes, total: int, where: str) -> Mapping[int, tuple[int, int]]:
+    """Every section of the message as ``number -> (start, length)``, or a refusal.
+
+    Bounded twice: each declared length is checked against the bytes remaining before it is
+    followed, and the section numbers must strictly ascend, which caps the walk at seven steps
+    whatever the lengths say.
+    """
+
+    end = total - _TRAILER_LENGTH
+    offset = _INDICATOR_LENGTH
+    previous = 0
+    found: dict[int, tuple[int, int]] = {}
+    while offset < end:
+        if end - offset < _SECTION_HEADER_LENGTH:
+            _refuse(
+                "READER_ADMISSION",
+                "reader.grib2.section",
+                f"leaves {end - offset} bytes at byte {offset}{where}, too few for a section "
+                f"header, so the message does not account for all of its own bytes",
+            )
+        length = int.from_bytes(content[offset : offset + 4], "big")
+        number = content[offset + 4]
+        if length < _SECTION_HEADER_LENGTH:
+            _refuse(
+                "READER_ADMISSION",
+                "reader.grib2.section",
+                f"section {number} at byte {offset} declares a length of {length}{where}, which "
+                f"is smaller than the {_SECTION_HEADER_LENGTH}-byte header it must contain; a "
+                f"walk that followed it would not advance",
+            )
+        if offset + length > end:
+            _refuse(
+                "READER_ADMISSION",
+                "reader.grib2.section",
+                f"section {number} at byte {offset} declares a length of {length}{where}, which "
+                f"runs past the end of the message",
+            )
+        if number not in _SECTION_NUMBERS:
+            _refuse(
+                "READER_ADMISSION",
+                "reader.grib2.section",
+                f"section number {number} at byte {offset}{where} is not one edition "
+                f"{_EDITION} defines",
+            )
+        if number <= previous:
+            _refuse(
+                "READER_ADMISSION",
+                "reader.grib2.section",
+                f"section {number} at byte {offset} follows section {previous}{where}; sections "
+                f"ascend, and a message carrying more than one field is not one this Reader "
+                f"opens",
+            )
+        found[number] = (offset, length)
+        previous = number
+        offset += length
+
+    if offset != end:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.section",
+            f"has a section chain ending at byte {offset} rather than at the trailer at "
+            f"{end}{where}",
+        )
+    missing = [number for number in _REQUIRED_SECTIONS if number not in found]
+    if missing:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.section",
+            f"does not carry section {', '.join(str(number) for number in missing)}{where}, "
+            f"which a single-field message must",
+        )
+    return found
+
+
+# --- The sections this module actually reads ---------------------------------------------
+
+
+@dataclass(frozen=True)
+class _Grid:
+    points: int
+    template: int
+
+
+@dataclass(frozen=True)
+class _Packing:
+    points: int
+    reference_value: float
+    binary_scale: int
+    decimal_scale: int
+    bits_per_value: int
+    original_field_type: int
+
+
+def _read_grid_section(content: bytes, span: tuple[int, int], where: str) -> _Grid:
+    """The declared point count and the grid template number, and nothing else."""
+
+    start, length = span
+    _require_length(length, 14, 3, where)
+    source = content[start + 5]
+    if source != 0:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.grid",
+            f"defines its grid from source {source}{where} rather than from a template in the "
+            f"message, so the geometry is somewhere this Reader cannot see it",
+        )
+    optional_octets = content[start + 10]
+    if optional_octets != 0:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.grid",
+            f"carries a {optional_octets}-octet list of point counts per row{where}, which is a "
+            f"quasi-regular grid; this Reader opens grids whose shape is in the template",
+        )
+    return _Grid(
+        points=int.from_bytes(content[start + 6 : start + 10], "big"),
+        template=int.from_bytes(content[start + 12 : start + 14], "big"),
+    )
+
+
+def _read_packing_template(content: bytes, span: tuple[int, int], where: str) -> int:
+    """The packing template number alone, read before anything is decided about it."""
+
+    start, length = span
+    _require_length(length, 11, 5, where)
+    return int.from_bytes(content[start + 9 : start + 11], "big")
+
+
+def _read_packing_parameters(
+    content: bytes,
+    span: tuple[int, int],
+    template: int,
+    where: str,
+) -> _Packing:
+    """The reference value, the two scale factors, and the bit width of an admitted packing."""
+
+    start, length = span
+    _require_length(length, _PACKING_PARAMETERS_END, 5, where)
+    return _Packing(
+        points=int.from_bytes(content[start + 5 : start + 9], "big"),
+        reference_value=struct.unpack(">f", content[start + 11 : start + 15])[0],
+        binary_scale=_sign_magnitude(content[start + 15 : start + 17]),
+        decimal_scale=_sign_magnitude(content[start + 17 : start + 19]),
+        bits_per_value=content[start + 19],
+        original_field_type=content[start + 20],
+    )
+
+
+def _read_bitmap_indicator(content: bytes, span: tuple[int, int], where: str) -> int:
+    """Whether every point carries a value, refusing an indicator that resolves to nothing."""
+
+    start, length = span
+    _require_length(length, 6, 6, where)
+    indicator = content[start + 5]
+    if indicator not in {_NO_BITMAP, _BITMAP_PRESENT}:
+        detail = (
+            "refers to a bitmap defined in an earlier message, and this Reader admits one "
+            "message at a time, so there is no earlier message to refer to"
+            if indicator == 254
+            else f"uses bitmap indicator {indicator}, which edition {_EDITION} reserves and "
+            f"therefore does not define"
+        )
+        _refuse("READER_ADMISSION", "reader.grib2.bitmap", f"{detail}{where}")
+    return indicator
+
+
+def _locate_data(span: tuple[int, int]) -> tuple[int, int]:
+    """Where the packed values begin and end.  This is as close as this module gets to them."""
+
+    start, length = span
+    return start + _SECTION_HEADER_LENGTH, start + length
+
+
+# --- The allowlist ------------------------------------------------------------------------
+
+
+def _refuse_a_lossy_codec(packing: int, where: str) -> None:
+    """Refuse JPEG 2000 packing, in wording that offers no fix, because there is not one.
+
+    Checked before the allowlist and reported under its own subject, so nothing downstream can
+    read this as "a template we have not got round to".  It has not got round to nothing: a
+    sealed dataset states that its numbers are the publisher's numbers, and a codec free to
+    return values the publisher never wrote cannot support that statement at any bit rate.
+    """
+
+    if packing not in LOSSY_PACKING_TEMPLATES:
+        return
+    _refuse(
+        "READER_ADMISSION",
+        "reader.grib2.packing.lossy",
+        f"is packed with the JPEG 2000 code stream, template 5.{packing}{where}. This product "
+        f"does not build sealed data from a lossy codec, so this is not a matter of approving "
+        f"one more packing method: approving it would not help, because the objection is to "
+        f"the codec and not to the paperwork. A file published this way has to be republished "
+        f"in a lossless packing before it can be read here.",
+    )
+
+
+def _admit_the_pair(grid: int, packing: int, where: str) -> None:
+    """Refuse a grid, a packing, or a combination, each naming the edit that would admit it."""
+
+    if (grid, packing) in ADMITTED_TEMPLATE_PAIRS:
+        return
+    admitted_grids = {number for number, _ in ADMITTED_TEMPLATE_PAIRS}
+    admitted_packings = {number for _, number in ADMITTED_TEMPLATE_PAIRS}
+    pair = _pair_text(grid, packing)
+    fix = (
+        f"Approving it means adding the pair ({grid}, {packing}) to {ALLOWLIST_CONSTANT}, "
+        f"vendoring a message that declares it, and re-certifying the Reader."
+    )
+    if grid not in admitted_grids:
+        _refuse(
+            "READER_GRID_UNKNOWN",
+            "reader.grib2.grid_template",
+            f"is on grid template 3.{grid}{where}, a layout this product has never been checked "
+            f"against, so it will not guess where the values belong. {pair} {fix}",
+        )
+    if packing not in admitted_packings:
+        _refuse(
+            "READER_PACKING_UNKNOWN",
+            "reader.grib2.packing_template",
+            f"is packed with template 5.{packing}{where}, a method this product does not "
+            f"unpack. {pair} {fix}",
+        )
+    _refuse(
+        "READER_PACKING_UNKNOWN",
+        "reader.grib2.packing_template",
+        f"pairs grid template 3.{grid} with packing template 5.{packing}{where}, and this "
+        f"product has opened each of those but never the two together. {pair} {fix}",
+    )
+
+
+def _pair_text(grid: int, packing: int) -> str:
+    """The one sentence that says this is policy rather than breakage."""
+
+    return (
+        f"The combination this file declares, grid 3.{grid} with packing 5.{packing}, is not on "
+        f"the list of combinations this product opens. That list is closed on purpose: a "
+        f"combination nobody has checked against a real file is not on it."
+    )
+
+
+# --- Shared refusals ----------------------------------------------------------------------
+
+
+def _refuse_a_length_that_is_not_the_file(
+    content: bytes,
+    *,
+    total: int,
+    held: int,
+    where: str,
+) -> NoReturn:
+    """Say which of the two things a length mismatch means, because the fixes differ.
+
+    A message whose declared length ends before the file does is almost never a damaged
+    download: it is a **multi-message file**, which is what every published GFS and HRRR file
+    is.  This Reader opens exactly one message and has no way to be told which one, so such a
+    file is refused -- but it has to be refused as what it is.  Reporting "it claims fewer
+    bytes than arrived" sends a person to check their download, which is the wrong repair for
+    a file that is intact and correct.
+
+    The two are told apart from the bytes themselves: a message that ends with the edition-2
+    trailer at its own declared end and is followed by another ``GRIB`` indicator is a
+    well-formed message inside a well-formed collection.  Anything else -- a short read, a
+    truncated transfer, a length field that points into the middle of nothing -- keeps the
+    original refusal, because that one really is the file disagreeing with itself.
+    """
+
+    ends_cleanly = (
+        _INDICATOR_LENGTH + _TRAILER_LENGTH <= total < held
+        and content[total - _TRAILER_LENGTH : total] == _TRAILER
+    )
+    another_follows = content[total : total + len(_MAGIC)] == _MAGIC
+    if ends_cleanly and another_follows:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.message",
+            f"holds more than one weather message{where}: the first declares {total} bytes, "
+            f"ends properly at that point, and another begins straight after it in the "
+            f"{held} bytes that arrived. Published model files are collections of records and "
+            f"this is what one looks like. This Reader opens exactly one message and its "
+            f"settings have no way to say which record of a collection to open, so a "
+            f"collection is refused rather than opened at its first record. The single record "
+            f"you want has to be extracted before the recipe points at it; opening a "
+            f"collection is separate, unbuilt work and the file is not damaged",
+        )
+    direction = "more" if total > held else "fewer"
+    _refuse(
+        "READER_ADMISSION",
+        "reader.grib2.message",
+        f"declares {total} bytes{where} but holds {held}, so it claims {direction} bytes "
+        f"than arrived; a message is complete or it is not read",
+    )
+
+
+def _origin_clause(origin: str | None) -> str:
+    """The recipe's own name for the source, so a halted refresh names the file."""
+
+    if origin is None:
+        return ""
+    if not isinstance(origin, str) or not origin.strip():
+        _refuse("READER_ADMISSION", "reader.grib2.origin", "must be a nonempty name or absent")
+    return f" in {origin.strip()}"
+
+
+def _require_length(length: int, needed: int, section: int, where: str) -> None:
+    """Refuse a section too short to hold the field about to be read from it."""
+
+    if length < needed:
+        _refuse(
+            "READER_ADMISSION",
+            "reader.grib2.section",
+            f"section {section} declares {length} bytes{where} but must hold at least {needed} "
+            f"to state what it is for",
+        )
+
+
+def _sign_magnitude(raw: bytes) -> int:
+    """A GRIB2 signed integer: a sign bit and a magnitude, never two's complement."""
+
+    value = int.from_bytes(raw, "big")
+    sign = 1 << (8 * len(raw) - 1)
+    return -(value & ~sign) if value & sign else value
+
+
+def _refuse(code: str, subject: str, detail: str) -> NoReturn:
+    raise ReaderError(code, subject, detail)