mostlyright-data 0.9.0__py3-none-any.whl

mostlyright/data_harness/readers/containers.py

@@ -0,0 +1,963 @@
+"""The container Readers: an archive opened one layer deep, in memory, never to disk.
+
+A container Reader takes the bytes of an archive and returns the rows of exactly one member
+of it.  Four clauses govern every family in this module, and each one deletes a bug class
+rather than mitigating one.  They are written here as a list a reviewer can check the code
+against, in the order the code applies them.
+
+1. **Depth 1.**  After the member is selected, its first bytes are checked against the one
+   archive-and-executable prefix table in ``formats``.  A member that is itself an archive is
+   refused as ``READER_NESTED_CONTAINER``.  Nothing in this module recurses, under any
+   option, ever.  The recipe layer is the first of the two layers: ``max_nesting_depth`` is
+   fixed at 1 by the Reader contract and a recipe that asks for 2 is refused where the budget
+   is built, so depth cannot be raised by configuration.  This module is the second layer,
+   and it catches the different thing -- an archive whose *contents* are nested rather than a
+   recipe that asked to descend.
+2. **Exactly one recipe-named member.**  The recipe supplies the member name in its sealed
+   decode options.  No family here picks a member by heuristic: "the only CSV in the archive"
+   is non-deterministic by construction, because an archive that later gains a second CSV
+   would silently change the dataset while the recipe digest stayed the same.  A missing name
+   is ``READER_OPTIONS``; a name matching no member is ``READER_MEMBER_MISSING``; a name
+   matching more than one member -- which zip and tar both permit -- is
+   ``READER_MEMBER_AMBIGUOUS``; a member that is not a regular file is
+   ``READER_MEMBER_TYPE``, naming the type found.
+3. **Streamed in memory.**  The input is exact bytes, the archive is opened over
+   ``io.BytesIO``, and the member is read in bounded chunks through ``ZipFile.open``,
+   ``TarFile.extractfile``, or a bounded inflate.  Those three plus nothing else: no API
+   taking a filesystem path is named anywhere in this module, which is how zip-slip and path
+   traversal are deleted rather than defended against.
+4. **Never extracted to disk.**  A decode creates no file, no directory, and no temporary
+   anything.  ``tests/h3/test_readers_container.py`` asserts it by observation -- a watched
+   working directory and a watched temporary directory, empty before and empty after --
+   because the absence of a spill is not provable by reading the source.
+
+Where a clause degenerates for a format, it is said here rather than left for a reader to
+notice.  A gzip stream has exactly one member and no member name, so "exactly one
+recipe-named member" becomes "exactly one stream, whose format the recipe still names": the
+``member`` option is refused for that family rather than ignored, because silently accepting
+a meaningless option is how two different recipes come to look identical, and a concatenated
+second stream -- which is legal gzip -- is refused rather than appended to the first.  The
+optional filename field a gzip header may carry is never read: it is attacker-controlled text
+that names nothing this family selects on.
+
+A member is decoded by the same helper the delimited family uses, and it is decoded under the
+same rules: the member's first row is its heading row, and ``header: false`` is refused here in
+the delimited family's own words rather than answered with positional column names.  These
+families call ``decode_delimited_stream`` directly rather than through ``DelimitedTextReader``, so a
+rule stated only on that class is a rule the wrapped route does not have -- which is how one
+product came to give two answers to "are columns ever named by position", depending on whether
+the file arrived inside a wrapper.
+
+Budgets run before allocation, and the order is visible in the code because that is the only
+way the claim is checkable.  Zip and tar declare their members' uncompressed sizes in a
+directory, so those families decide against the declaration before a member is opened and a
+bomb is refused with nothing allocated.  Gzip declares nothing at all, so its inflate is
+bounded by a byte counter and abandoned mid-stream rather than after.  A declaration is only
+a claim either way, so the read itself is bounded independently of it.
+
+The tar member-type check is the one that matters most and the easiest to get subtly wrong.
+``tarfile``'s ``filter="data"`` is a second layer and is not the first: this module never
+extracts, so that filter never runs on our path.  The explicit ``isfile()`` check is the real
+gate, and the refusal enumerates every type it refuses so the message is actionable.
+
+The decoder pin is the CPython standard library -- ``zipfile``, ``tarfile``, ``zlib`` -- and
+that is deliberate.  We never extract, so the memory-safety argument is about the
+decompressor, and CPython's zlib is the same battle-tested C whichever route we take.  A
+third-party or Rust decompressor here would buy nothing and would import the whole
+supply-chain problem on Reader number one.  These families therefore carry no third-party
+dependency at all; the Certification record states the pin as the interpreter range the
+project supports.
+
+Import discipline: this module reads ``formats`` and ``readers.tabular`` and nothing from
+``acquisition``.  The archive prefix table lives in the leaf precisely so both the tabular
+parser (which refuses an archive outright) and these families (which refuse a nested one) can
+read one declaration; a ``readers -> acquisition`` edge would close an import cycle.
+"""
+
+from __future__ import annotations
+
+import dataclasses
+import io
+import tarfile
+import zipfile
+import zlib
+from collections.abc import Iterator, Mapping
+from dataclasses import dataclass, field
+from types import MappingProxyType
+from typing import Any, BinaryIO
+
+from mostlyright.data_harness.formats import (
+    ARCHIVE_OR_EXECUTABLE_MAGICS,
+    FORMAT_MEDIA_TYPES,
+    FORMAT_SUFFIXES,
+    READER_CONTRACT_VERSION,
+    READER_OUTPUT_FORMATS,
+)
+from mostlyright.data_harness.readers.contracts import (
+    ReaderBudgets,
+    ReaderError,
+    ReaderPin,
+    ReaderResult,
+    bulk_default_budgets,
+)
+from mostlyright.data_harness.readers.tabular import (
+    FALLBACK_STEM,
+    decode_delimited_stream,
+    encode_canonical_csv,
+    sealed_filename,
+)
+
+__all__ = [
+    "CHUNK_BYTES",
+    "GZIP_MAGICS",
+    "ZIP_MAGICS",
+    "GzipReader",
+    "TarReader",
+    "ZipReader",
+    "inflate_one_gzip_stream",
+    "read_bounded",
+]
+
+# The one encoding a Reader may seal, unpacked rather than restated.  A second member in the
+# leaf's table makes this line raise at import, which is the loud failure that forces a
+# family to choose its output encoding deliberately instead of inheriting whichever name
+# happened to sort first.
+(_OUTPUT_FORMAT,) = READER_OUTPUT_FORMATS
+_OUTPUT_MEDIA_TYPE = sorted(FORMAT_MEDIA_TYPES[_OUTPUT_FORMAT])[0]
+_OUTPUT_SUFFIX = sorted(FORMAT_SUFFIXES[_OUTPUT_FORMAT])[0]
+
+# The admission prefixes, derived from the one leaf table rather than copied out of it.  A
+# second literal list would let this module and the tabular parser drift into disagreeing
+# about what an archive is, which is a security-relevant disagreement and not a cosmetic one.
+ZIP_MAGICS: tuple[bytes, ...] = tuple(
+    magic for magic in ARCHIVE_OR_EXECUTABLE_MAGICS if magic.startswith(b"PK")
+)
+GZIP_MAGICS: tuple[bytes, ...] = tuple(
+    magic for magic in ARCHIVE_OR_EXECUTABLE_MAGICS if magic.startswith(b"\x1f\x8b")
+)
+
+# Tar is the one family with no leading magic: a tar file begins with its first member's
+# name.  The POSIX ``ustar`` marker sits at offset 257 of the first header block, and the two
+# spellings below are the two this family admits -- POSIX writes ``ustar\0`` and GNU writes
+# ``ustar``-space.  A pre-POSIX v7 tar carries no marker at all and is therefore not admitted;
+# that is a residual recorded in the Certification record, not an oversight.
+_TAR_MAGIC_OFFSET = 257
+_TAR_MAGICS: tuple[bytes, ...] = (b"ustar\x00", b"ustar ")
+
+# The window read at an admission offset.  Bounded so a 16 MiB input is not copied to compare
+# six bytes.
+_MAGIC_WINDOW_BYTES = 16
+
+# The compression methods a zip member may use, as a closed list rather than as whatever the
+# running interpreter happens to support.  ``zipfile`` decodes bzip2 and lzma only when the
+# interpreter was built with those modules and raises an untyped ``NotImplementedError`` when
+# it was not, so admitting them would make a member's decodability a property of the host.  A
+# Reader that answers differently on two hosts cannot promise the same rows forever.
+_ZIP_METHODS: Mapping[int, str] = MappingProxyType(
+    {zipfile.ZIP_STORED: "stored", zipfile.ZIP_DEFLATED: "deflated"}
+)
+
+# Bit 0 of the general-purpose flag: the member is encrypted.  Read from the directory so an
+# encrypted member is refused with a typed refusal rather than escaping as the ``RuntimeError``
+# ``ZipFile.open`` raises for one.
+_ZIP_ENCRYPTED_FLAG = 0x01
+
+# A zip member made on a unix host stores its st_mode in the high 16 bits of ``external_attr``.
+# ``ZipInfo.is_dir()`` only reads the trailing-slash naming convention, so it says nothing about
+# a symbolic link, a fifo, or a device node -- and a symlink entry's *content* is the link
+# target text, which without this check would be decoded as if it were the member's data and
+# sealed as rows.  Tar has had this gate since it shipped (``_tar_member_type``); clause 2 says
+# the rule governs every family in this module, so zip reads the mode the same way.
+_ZIP_CREATE_SYSTEM_UNIX = 3
+_UNIX_MODE_TYPE_MASK = 0o170000
+_UNIX_MODE_REGULAR = 0o100000
+_UNIX_MODE_DIRECTORY = 0o040000
+_ZIP_MEMBER_TYPES: Mapping[int, str] = MappingProxyType(
+    {
+        _UNIX_MODE_DIRECTORY: "directory",
+        0o120000: "symbolic link",
+        0o010000: "fifo",
+        0o020000: "character device",
+        0o060000: "block device",
+        0o140000: "socket",
+    }
+)
+
+# gzip, and gzip only: 16 selects the gzip wrapper rather than raw or zlib-wrapped deflate,
+# so a stream that is not gzip fails in the decompressor rather than being read as something
+# else.
+_GZIP_WBITS = 16 + zlib.MAX_WBITS
+
+# Which delimiter each member form uses.  The delimited decoder takes the delimiter as an
+# argument, so this table is the whole of "which form is this member": a second delimited
+# decoder would be a second implementation of one thing.
+_DELIMITER_BY_MEMBER_FORM: Mapping[str, str] = MappingProxyType(
+    {"csv": ",", "semicolon": ";", "tsv": "\t"}
+)
+_V1_MEMBER_FORMS = frozenset({"csv", "tsv"})
+_V1_1_MEMBER_FORMS = frozenset(_DELIMITER_BY_MEMBER_FORM)
+
+# The settings every container family takes, and the one more that a named-member family adds.
+_SHARED_OPTION_KEYS = ("header", "member_format")
+_MEMBER_OPTION_KEY = "member"
+
+# A member name is bounded text.  It is never joined to a path and never opened as one, so
+# this bound is about keeping a refusal message readable rather than about traversal.
+_MAX_MEMBER_NAME_BYTES = 1024
+
+# The read granularity.  Public because it is the tolerance the gzip bomb assertion is
+# written against: "abandoned mid-stream" means no more than one chunk came out past the
+# ceiling, and a test that invented its own tolerance would be asserting a different claim.
+# Small enough that a bomb is abandoned after one chunk rather than after one member, large
+# enough that an ordinary member costs a handful of calls.
+CHUNK_BYTES = 64 * 1024
+
+
+def read_bounded(stream: BinaryIO, *, declared: int, subject: str) -> bytes:
+    """Read exactly the declared number of bytes, and refuse a stream that is not that long.
+
+    Public because it is the second layer of the size invariant and a second layer nobody can
+    exercise is a second layer nobody knows works.  The declared size is a claim made by the
+    archive's own directory, so the read is bounded independently of it: one byte past the
+    declaration is read deliberately, and getting it is the refusal.
+    """
+
+    if type(declared) is not int or declared < 0:
+        raise ReaderError("READER_BUDGET", subject, "a declared member size must be a count")
+    chunks: list[bytes] = []
+    remaining = declared + 1
+    while remaining > 0:
+        chunk = stream.read(min(CHUNK_BYTES, remaining))
+        if not chunk:
+            break
+        remaining -= len(chunk)
+        chunks.append(chunk)
+    payload = b"".join(chunks)
+    if len(payload) > declared:
+        raise ReaderError(
+            "READER_BUDGET",
+            subject,
+            f"holds more bytes than the {declared} its directory declares",
+        )
+    if len(payload) < declared:
+        raise ReaderError(
+            "READER_ADMISSION",
+            subject,
+            f"ends after {len(payload)} bytes, short of the {declared} its directory declares",
+        )
+    return payload
+
+
+def screen_zip_entry(entry: zipfile.ZipInfo, *, subject: str, described: str) -> None:
+    """Refuse from the central directory anything ``ZipFile.open`` would not refuse typed.
+
+    Three screens, all read from the directory record, all before the entry is opened:
+
+    * a member type that is not a regular file -- see ``_zip_member_type``;
+    * the encryption flag, because ``ZipFile.open`` raises a bare ``RuntimeError`` for one,
+      and a refusal with no code is a refusal the remediation map cannot bind a fix to;
+    * a compression method outside the closed two, because ``zipfile`` decodes bzip2 and
+      lzma only when the interpreter was built with those modules and raises an untyped
+      ``NotImplementedError`` when it was not, which would make a file's decodability a
+      property of the host.
+
+    Public and shared rather than inlined per caller.  ``spreadsheet`` opens the parts of a
+    workbook package with the same standard-library engine and had none of these screens, so
+    an encrypted or oddly-compressed part escaped as an untyped ``RuntimeError``: the same
+    class this module had already learned once, re-entering through a second call site.  One
+    implementation means a third call site cannot re-enter it a third time.
+
+    ``described`` is how the refusal names the entry, because "the member the recipe named"
+    and "a part of the workbook package" are different things to a person reading a halt.
+    """
+
+    found = _zip_member_type(entry)
+    if found is not None:
+        raise ReaderError(
+            "READER_MEMBER_TYPE",
+            subject,
+            f"{described}, which is a {found}. A container Reader opens a regular file only; "
+            "a directory, symbolic link, hard link, fifo, character device, block device, or "
+            "socket is refused. The bytes a symbolic link entry stores are the path it points "
+            "at, so reading one as data would seal that path as the dataset's rows",
+        )
+    if entry.flag_bits & _ZIP_ENCRYPTED_FLAG:
+        raise ReaderError(
+            "READER_ADMISSION",
+            subject,
+            f"{described}, which is encrypted; a recipe seals no secret and a Reader holds "
+            "none, so an encrypted member cannot be opened",
+        )
+    if entry.compress_type not in _ZIP_METHODS:
+        raise ReaderError(
+            "READER_ADMISSION",
+            subject,
+            f"{described}, whose compression method {entry.compress_type} is outside the "
+            f"admitted list: {', '.join(sorted(_ZIP_METHODS.values()))}",
+        )
+
+
+def _admit_container_bytes(
+    content: Any,
+    *,
+    magics: tuple[bytes, ...],
+    budgets: ReaderBudgets,
+    subject: str,
+    described: str,
+    offset: int = 0,
+) -> bytes:
+    """Admit the archive's own bytes, before any decoder object is constructed.
+
+    ``offset`` exists for tar, whose marker sits inside the first header block rather than at
+    the front.  The window compared is bounded, so admission never copies the input.
+    """
+
+    if not isinstance(content, (bytes, bytearray)):
+        raise ReaderError("READER_ADMISSION", subject, "must be exact bytes")
+    raw = bytes(content)
+    if len(raw) > budgets.max_input_bytes:
+        raise ReaderError("READER_BUDGET", subject, "exceeds the input byte budget")
+    window = raw[offset : offset + _MAGIC_WINDOW_BYTES]
+    if not any(window.startswith(magic) for magic in magics):
+        raise ReaderError("READER_ADMISSION", subject, f"does not begin as {described}")
+    return raw
+
+
+def inflate_one_gzip_stream(raw: bytes, *, ceiling: int, subject: str) -> bytes:
+    """Inflate exactly one gzip stream, bounded by a counter and abandoned when it is passed.
+
+    A gzip stream declares nothing about how large it will become, so there is no declaration
+    to decide against and the counter is the whole defence.  Output is taken one chunk at a
+    time and the total is checked after every chunk, so a bomb is refused with at most one
+    chunk of it in memory rather than after the whole thing has been produced.
+
+    Exactly one stream: a gzip file may legally carry a second stream concatenated onto the
+    first, and reading both would silently append one table to another.  Trailing bytes after
+    the first stream ends are therefore a refusal, not a continuation.
+    """
+
+    decompressor = zlib.decompressobj(wbits=_GZIP_WBITS)
+    chunks: list[bytes] = []
+    produced = 0
+    offset = 0
+    # A progress bound rather than a trust in the loop's own termination: every pass either
+    # consumes input or produces output, so a pass that does neither is a decompressor
+    # behaving in a way this loop did not foresee, and spinning is the one outcome a bounded
+    # decode may not have.
+    steps = 0
+    max_steps = 4 * (len(raw) // CHUNK_BYTES + ceiling // CHUNK_BYTES + 4)
+
+    while not decompressor.eof:
+        steps += 1
+        if steps > max_steps:
+            raise ReaderError(
+                "READER_BUDGET",
+                subject,
+                "made no progress inflating; the decode is abandoned rather than retried",
+            )
+        if decompressor.unconsumed_tail:
+            piece = decompressor.unconsumed_tail
+        elif offset < len(raw):
+            piece = raw[offset : offset + CHUNK_BYTES]
+            offset += len(piece)
+        else:
+            raise ReaderError(
+                "READER_ADMISSION",
+                subject,
+                "ends before its gzip stream is complete",
+            )
+        try:
+            produced_chunk = decompressor.decompress(piece, CHUNK_BYTES)
+        except zlib.error as error:
+            raise ReaderError("READER_ADMISSION", subject, "is not a readable gzip stream") from (
+                error
+            )
+        if produced_chunk:
+            produced += len(produced_chunk)
+            if produced > ceiling:
+                raise ReaderError(
+                    "READER_BUDGET",
+                    subject,
+                    f"inflates past the {ceiling} bytes this stream may produce -- the "
+                    "smaller of the uncompressed byte budget and the expansion ratio applied "
+                    "to its compressed size -- and is abandoned mid-stream",
+                )
+            chunks.append(produced_chunk)
+
+    if decompressor.unused_data or offset < len(raw):
+        raise ReaderError(
+            "READER_ADMISSION",
+            subject,
+            "carries more than one gzip stream; exactly one stream is opened, because a "
+            "concatenated second one would silently append a second table to the first",
+        )
+    return b"".join(chunks)
+
+
+def _validate_container_options(
+    options: Any,
+    *,
+    names_a_member: bool,
+    subject: str,
+    admitted_member_forms: frozenset[str] = _V1_MEMBER_FORMS,
+) -> dict[str, Any]:
+    """Admit this family's closed option set, refusing an unknown key by name.
+
+    Strict on purpose.  The options digest is sealed into the recipe forever, so a key that
+    was accepted and ignored would be an unreviewable difference between two recipes that
+    look identical to everyone who reads them.
+    """
+
+    if not isinstance(options, Mapping):
+        raise ReaderError("READER_OPTIONS", subject, "must be an object")
+    allowed = set(_SHARED_OPTION_KEYS)
+    if names_a_member:
+        allowed.add(_MEMBER_OPTION_KEY)
+    unknown = sorted(str(key) for key in options if key not in allowed)
+    if unknown:
+        raise ReaderError(
+            "READER_OPTIONS",
+            subject,
+            f"names no such setting: {', '.join(unknown)}; this family takes "
+            f"{', '.join(sorted(allowed))}",
+        )
+
+    member_format = options.get("member_format")
+    if member_format not in admitted_member_forms or not isinstance(member_format, str):
+        raise ReaderError(
+            "READER_OPTIONS",
+            f"{subject}.member_format",
+            "must name the delimited form the member's bytes use, one of "
+            f"{', '.join(sorted(admitted_member_forms))}",
+        )
+    header = options.get("header", True)
+    if not isinstance(header, bool):
+        raise ReaderError("READER_OPTIONS", f"{subject}.header", "must be true or false")
+    if not header:
+        # The same ruling the delimited family gives, in the same words, because it is the
+        # same question about the same bytes.  These families hand their member to
+        # ``decode_delimited_stream`` directly rather than through ``DelimitedTextReader``, so the
+        # rule has to be stated on both routes or the product answers "are columns ever
+        # named by position" two ways depending on whether the file arrived in a wrapper.
+        raise ReaderError(
+            "READER_OPTIONS",
+            f"{subject}.header",
+            "must be true: a headerless source needs the recipe to supply its column names, "
+            "this family's settings have no way to state them, and naming columns by position "
+            "would put names into the dataset that nobody wrote down. Add a header row to the "
+            "source, or widen this family with a reviewed columns option",
+        )
+
+    admitted: dict[str, Any] = {"member_format": member_format, "header": header}
+    if names_a_member:
+        member = options.get(_MEMBER_OPTION_KEY)
+        if (
+            not isinstance(member, str)
+            or not member
+            or "\x00" in member
+            or len(member.encode("utf-8", errors="replace")) > _MAX_MEMBER_NAME_BYTES
+        ):
+            raise ReaderError(
+                "READER_OPTIONS",
+                f"{subject}.member",
+                "must name exactly one member of the archive; this family never picks a "
+                "member by heuristic",
+            )
+        admitted[_MEMBER_OPTION_KEY] = member
+    return admitted
+
+
+def _check_member_count(count: int, budgets: ReaderBudgets, subject: str) -> None:
+    """Refuse an archive with too many members, while the directory is scanned."""
+
+    if count > budgets.max_container_members:
+        raise ReaderError(
+            "READER_BUDGET",
+            subject,
+            f"holds {count} members, over the member budget of {budgets.max_container_members}",
+        )
+
+
+def _check_declared_member_size(
+    *,
+    declared: int,
+    compressed: int,
+    budgets: ReaderBudgets,
+    subject: str,
+) -> None:
+    """Decide against the directory's declaration, before the member is opened.
+
+    This is the whole reason zip and tar are the safe two of the three: the size a member
+    claims is readable without decompressing a byte, so a bomb is refused with nothing
+    allocated rather than detected once it is too late.
+    """
+
+    if type(declared) is not int or declared < 0:
+        raise ReaderError("READER_BUDGET", subject, "declares a size that is not a count")
+    if declared > budgets.max_uncompressed_bytes:
+        raise ReaderError(
+            "READER_BUDGET",
+            subject,
+            f"declares {declared} uncompressed bytes, over the uncompressed byte budget",
+        )
+    if declared > budgets.max_output_bytes:
+        raise ReaderError(
+            "READER_BUDGET",
+            subject,
+            f"declares {declared} uncompressed bytes, over the decode output byte budget "
+            "the extracted member becomes",
+        )
+    if compressed <= 0:
+        if declared > 0:
+            raise ReaderError(
+                "READER_BUDGET",
+                subject,
+                "declares content it stores no compressed bytes for",
+            )
+        return
+    if declared > compressed * budgets.max_expansion_ratio:
+        raise ReaderError(
+            "READER_BUDGET",
+            subject,
+            f"declares an expansion of {declared} from {compressed} bytes, over the "
+            f"expansion ratio budget of {budgets.max_expansion_ratio}",
+        )
+
+
+def _refuse_a_nested_container(payload: bytes, subject: str) -> None:
+    """Clause 1, as its own refusal: a member that is itself an archive is never opened."""
+
+    for magic in ARCHIVE_OR_EXECUTABLE_MAGICS:
+        if payload.startswith(magic):
+            raise ReaderError(
+                "READER_NESTED_CONTAINER",
+                subject,
+                "is itself an archive or an executable; containers are opened one layer "
+                "deep and nesting is refused rather than followed",
+            )
+
+
+def _emit(
+    payload: bytes,
+    *,
+    options: Mapping[str, Any],
+    budgets: ReaderBudgets,
+    stem_source: str,
+) -> ReaderResult:
+    """Decode the member's bytes through the shared delimited reader and canonical emitter.
+
+    One emitter, shared by every family in the package.  Byte-stability of the sealed
+    artifact is a Reader-contract obligation, so it is not re-implemented per family.
+    """
+
+    # The payload is container OUTPUT: it was admitted against the declared-member-size
+    # check, which bounds it by the output byte budget. The inner delimited stage would
+    # otherwise re-judge it as INPUT and refuse any member larger than the fetched-bytes
+    # budget, which for a compressed archive is almost every legitimate member.
+    member_budgets = dataclasses.replace(budgets, max_input_bytes=budgets.max_output_bytes)
+    # Parsed rows feed the canonical encoder one at a time rather than being drained into a
+    # table first.  A year-scale member is millions of rows, and the table form would hold
+    # every cell as a live object at once -- the decode's memory ceiling is the row, not the
+    # member.
+    columns, rows = decode_delimited_stream(
+        payload,
+        delimiter=_DELIMITER_BY_MEMBER_FORM[options["member_format"]],
+        header=bool(options["header"]),
+        budgets=member_budgets,
+    )
+    row_count = 0
+
+    def counted() -> Iterator[tuple[str, ...]]:
+        nonlocal row_count
+        for row in rows:
+            row_count += 1
+            yield row
+
+    payload_csv = encode_canonical_csv(columns, counted(), budgets=budgets)
+    return ReaderResult(
+        content=payload_csv,
+        data_format=_OUTPUT_FORMAT,
+        media_type=_OUTPUT_MEDIA_TYPE,
+        filename=sealed_filename(stem_source, suffix=_OUTPUT_SUFFIX),
+        row_count=row_count,
+        column_names=columns,
+        declared_cell_count=row_count * len(columns),
+    )
+
+
+@dataclass(frozen=True)
+class ZipReader:
+    """``archive.zip``: one named member of a zip archive, at depth one, in memory.
+
+    The order of the work is admission, budget, selection, nesting, decode, emit, and the
+    function below is written in that order because "budgets before allocation" is only true
+    if the code says so.  ``ZipInfo`` comes from the central directory, which is what makes
+    every size decision here a decision taken before a byte is decompressed.
+
+    Two further refusals are taken from the same directory, both found by adversarial review
+    and both before the member is opened: an encrypted member (``ZipFile.open`` raises an
+    untyped ``RuntimeError`` for one, and a refusal with no code is a refusal the remediation
+    map cannot bind a fix to), and a member stored by a compression method outside the closed
+    admitted list.
+    """
+
+    family_id: str = "archive.zip"
+    family_version: str = "1.0.0"
+    contract_version: str = READER_CONTRACT_VERSION
+    output_format: str = _OUTPUT_FORMAT
+    accepted_media_types: tuple[str, ...] = ("application/x-zip-compressed", "application/zip")
+    default_budgets: ReaderBudgets = field(default_factory=bulk_default_budgets)
+
+    def validate_options(self, options: Mapping[str, Any]) -> Mapping[str, Any]:
+        return _validate_container_options(
+            options, names_a_member=True, subject="reader.archive.zip.decode_options"
+        )
+
+    def decode(self, content: bytes, pin: ReaderPin, budgets: ReaderBudgets) -> ReaderResult:
+        subject = "reader.archive.zip"
+        options = self.validate_options(pin.decode_options)
+        raw = _admit_container_bytes(
+            content,
+            magics=ZIP_MAGICS,
+            budgets=budgets,
+            subject=subject,
+            described="a zip archive",
+        )
+        member = str(options[_MEMBER_OPTION_KEY])
+        where = f"{subject}.member"
+
+        try:
+            archive = zipfile.ZipFile(io.BytesIO(raw))
+        except (zipfile.BadZipFile, EOFError, zlib.error) as error:
+            raise ReaderError(
+                "READER_ADMISSION", subject, "is not a readable zip archive"
+            ) from error
+
+        with archive:
+            entries = archive.infolist()
+            _check_member_count(len(entries), budgets, subject)
+
+            matches = [entry for entry in entries if entry.filename == member]
+            if not matches:
+                raise ReaderError(
+                    "READER_MEMBER_MISSING",
+                    where,
+                    f"names {member!r}, which this archive does not contain",
+                )
+            if len(matches) > 1:
+                raise ReaderError(
+                    "READER_MEMBER_AMBIGUOUS",
+                    where,
+                    f"names {member!r}, which this archive holds {len(matches)} times; a "
+                    "recipe names exactly one member and this one names more",
+                )
+            entry = matches[0]
+            if entry.is_dir():
+                raise ReaderError(
+                    "READER_MEMBER_TYPE",
+                    where,
+                    f"names {member!r}, which is not a regular file",
+                )
+            screen_zip_entry(entry, subject=where, described=f"names {member!r}")
+            _check_declared_member_size(
+                declared=entry.file_size,
+                compressed=entry.compress_size,
+                budgets=budgets,
+                subject=where,
+            )
+
+            try:
+                with archive.open(entry, "r") as stream:
+                    payload = read_bounded(stream, declared=entry.file_size, subject=where)
+            except (zipfile.BadZipFile, EOFError, zlib.error) as error:
+                raise ReaderError(
+                    "READER_ADMISSION",
+                    where,
+                    f"names {member!r}, whose stored bytes do not match what the archive "
+                    "records for it",
+                ) from error
+
+        _refuse_a_nested_container(payload, where)
+        return _emit(payload, options=options, budgets=budgets, stem_source=member)
+
+
+@dataclass(frozen=True)
+class ZipReaderV1_1(ZipReader):
+    """``archive.zip@1.1.0`` adds explicit semicolon-delimited members.
+
+    Version 1.0.0 remains closed to CSV and TSV, so an already-approved recipe cannot silently
+    acquire new parsing behavior.  The new coordinate exists for publisher archives such as DWD
+    historical observations, whose selected text member uses a semicolon delimiter.
+    """
+
+    family_version: str = "1.1.0"
+
+    def validate_options(self, options: Mapping[str, Any]) -> Mapping[str, Any]:
+        return _validate_container_options(
+            options,
+            names_a_member=True,
+            subject="reader.archive.zip.decode_options",
+            admitted_member_forms=_V1_1_MEMBER_FORMS,
+        )
+
+
+@dataclass(frozen=True)
+class ZipReaderV1_2(ZipReaderV1_1):
+    """``archive.zip@1.2.0`` admits the byte-stream label, and opens nothing new.
+
+    ``application/octet-stream`` is a weak label and is treated as one -- ``acquisition/http``
+    records the rule for the slice path: it says only that a server declined to say anything,
+    and it is what an object store or a plain file server answers for a zip it was told
+    nothing about.  ``weather.grib2`` already admits it for exactly that reason.  The refusal
+    it removes landed at the fetch, before this family's own admission ever ran -- and that
+    admission is decisive where the label is a guess: the first bytes must be a zip prefix
+    from the one leaf table, so a response that is not a zip is refused by signature whatever
+    its label said.
+
+    A new coordinate in the ``archive.zip@1.1.0`` lineage, so the semicolon member form is
+    inherited and ``1.1.0`` stays closed to the weak label: an already-approved recipe cannot
+    silently begin admitting responses its review never saw.  ``decode`` is inherited
+    untouched; what moves is only which responses reach it.
+    """
+
+    family_version: str = "1.2.0"
+    accepted_media_types: tuple[str, ...] = (
+        "application/octet-stream",
+        "application/x-zip-compressed",
+        "application/zip",
+    )
+
+
+@dataclass(frozen=True)
+class GzipReader:
+    """``archive.gzip``: one gzip stream, inflated under a counter, in memory.
+
+    The degenerate case of clause 2, and the reason it is stated rather than left implicit: a
+    gzip stream has exactly one member and no name for it, so "exactly one recipe-named
+    member" becomes "exactly one stream, whose format the recipe still names".  A ``member``
+    option is therefore refused rather than ignored, and the header's optional filename field
+    is never read -- it is attacker-controlled text naming nothing this family selects on.
+    """
+
+    family_id: str = "archive.gzip"
+    family_version: str = "1.0.0"
+    contract_version: str = READER_CONTRACT_VERSION
+    output_format: str = _OUTPUT_FORMAT
+    accepted_media_types: tuple[str, ...] = ("application/gzip", "application/x-gzip")
+    default_budgets: ReaderBudgets = field(default_factory=bulk_default_budgets)
+
+    def validate_options(self, options: Mapping[str, Any]) -> Mapping[str, Any]:
+        return _validate_container_options(
+            options, names_a_member=False, subject="reader.archive.gzip.decode_options"
+        )
+
+    def decode(self, content: bytes, pin: ReaderPin, budgets: ReaderBudgets) -> ReaderResult:
+        subject = "reader.archive.gzip"
+        options = self.validate_options(pin.decode_options)
+        raw = _admit_container_bytes(
+            content,
+            magics=GZIP_MAGICS,
+            budgets=budgets,
+            subject=subject,
+            described="a gzip stream",
+        )
+        # The ceiling is the narrowest of the three bounds that apply: what the budgets allow
+        # uncompressed, what the decode output budget allows the stream to become, and what
+        # this many compressed bytes are allowed to expand into.  The last is what turns a
+        # 1 KiB stream claiming to be 128 MiB into a refusal rather than a permitted
+        # allocation.
+        ceiling = min(
+            budgets.max_uncompressed_bytes,
+            budgets.max_output_bytes,
+            max(len(raw), 1) * budgets.max_expansion_ratio,
+        )
+        payload = inflate_one_gzip_stream(raw, ceiling=ceiling, subject=subject)
+        _refuse_a_nested_container(payload, subject)
+        return _emit(payload, options=options, budgets=budgets, stem_source=FALLBACK_STEM)
+
+
+def _zip_member_type(entry: zipfile.ZipInfo) -> str | None:
+    """Name the non-regular type a zip directory entry declares, or ``None`` for a file.
+
+    The unix mode lives in the high 16 bits of ``external_attr``.  Three cases, and the
+    middle one is why this returns ``None`` rather than a name for everything it does not
+    recognise: a zip written by a DOS or Windows toolchain -- and one written by CPython's
+    own ``ZipFile.writestr``, which sets ``0o600 << 16`` -- carries no type bits at all, so
+    zero means "this entry does not state a type" and is read as a regular file exactly as
+    it was before this check existed.  A stated type is either regular, or one of the named
+    kinds, or a value no format names; the last of those is refused rather than admitted,
+    because a type nobody can name is not a type this Reader has been checked against.
+
+    ``create_system`` is deliberately not consulted.  It is attacker-controlled text in the
+    same directory record, so gating on it would let a crafted archive turn the check off by
+    claiming to have been made on DOS.
+    """
+
+    mode = (entry.external_attr >> 16) & 0xFFFF
+    stated = mode & _UNIX_MODE_TYPE_MASK
+    if stated in (0, _UNIX_MODE_REGULAR):
+        return None
+    return _ZIP_MEMBER_TYPES.get(stated, "member of a type this archive format does not name")
+
+
+def _tar_member_type(entry: tarfile.TarInfo) -> str:
+    """Name the member type found, in the words the refusal message uses."""
+
+    for holds, described in (
+        (entry.isdir, "directory"),
+        (entry.issym, "symbolic link"),
+        (entry.islnk, "hard link"),
+        (entry.isfifo, "fifo"),
+        (entry.ischr, "character device"),
+        (entry.isblk, "block device"),
+    ):
+        if holds():
+            return described
+    return "member of a type this archive format does not name"
+
+
+@dataclass(frozen=True)
+class GzipReaderV1_1(GzipReader):
+    """``archive.gzip@1.1.0`` admits the byte-stream label, and inflates nothing new.
+
+    The same one-label delta as ``archive.zip@1.2.0``, for the same reason, and the same
+    decisive admission behind it: the first bytes must be the gzip prefix from the leaf
+    table, so the weak label buys a response nothing but the signature check it then fails.
+    ``1.0.0`` stays closed to the weak label.
+    """
+
+    family_version: str = "1.1.0"
+    accepted_media_types: tuple[str, ...] = (
+        "application/gzip",
+        "application/octet-stream",
+        "application/x-gzip",
+    )
+
+
+@dataclass(frozen=True)
+class TarReader:
+    """``archive.tar``: one named regular-file member of an uncompressed tar, in memory.
+
+    Opened with ``mode="r:"`` and never ``mode="r"``.  The difference is the whole of clause
+    1 for this family: the transparent modes make ``tarfile`` decompress a gzip, bzip2, or xz
+    wrapper on our behalf, which is a second layer opened without anyone deciding to.  A
+    compressed tar is therefore refused at admission, and a recipe that wants one states the
+    wrapper's own Reader instead.
+    """
+
+    family_id: str = "archive.tar"
+    family_version: str = "1.0.0"
+    contract_version: str = READER_CONTRACT_VERSION
+    output_format: str = _OUTPUT_FORMAT
+    accepted_media_types: tuple[str, ...] = ("application/x-tar",)
+    default_budgets: ReaderBudgets = field(default_factory=bulk_default_budgets)
+
+    def validate_options(self, options: Mapping[str, Any]) -> Mapping[str, Any]:
+        return _validate_container_options(
+            options, names_a_member=True, subject="reader.archive.tar.decode_options"
+        )
+
+    def decode(self, content: bytes, pin: ReaderPin, budgets: ReaderBudgets) -> ReaderResult:
+        subject = "reader.archive.tar"
+        options = self.validate_options(pin.decode_options)
+        raw = _admit_container_bytes(
+            content,
+            magics=_TAR_MAGICS,
+            budgets=budgets,
+            subject=subject,
+            described="a POSIX tar archive",
+            offset=_TAR_MAGIC_OFFSET,
+        )
+        member = str(options[_MEMBER_OPTION_KEY])
+        where = f"{subject}.member"
+
+        try:
+            archive = tarfile.open(fileobj=io.BytesIO(raw), mode="r:")
+        except tarfile.TarError as error:
+            raise ReaderError(
+                "READER_ADMISSION", subject, "is not a readable tar archive"
+            ) from error
+
+        with archive:
+            matches: list[tarfile.TarInfo] = []
+            scanned = 0
+            try:
+                for entry in archive:
+                    scanned += 1
+                    _check_member_count(scanned, budgets, subject)
+                    if entry.name == member:
+                        matches.append(entry)
+            except tarfile.TarError as error:
+                raise ReaderError(
+                    "READER_ADMISSION", subject, "holds a member header it cannot read"
+                ) from error
+
+            if not matches:
+                raise ReaderError(
+                    "READER_MEMBER_MISSING",
+                    where,
+                    f"names {member!r}, which this archive does not contain",
+                )
+            if len(matches) > 1:
+                raise ReaderError(
+                    "READER_MEMBER_AMBIGUOUS",
+                    where,
+                    f"names {member!r}, which this archive holds {len(matches)} times; a "
+                    "recipe names exactly one member and this one names more",
+                )
+            entry = matches[0]
+            if not entry.isfile():
+                raise ReaderError(
+                    "READER_MEMBER_TYPE",
+                    where,
+                    f"names {member!r}, which is a {_tar_member_type(entry)}. A container "
+                    "Reader opens a regular file only; a directory, symbolic link, hard "
+                    "link, fifo, character device, or block device is refused",
+                )
+            # Tar stores its members uncompressed, so the declared size is both the size and
+            # the stored size and the expansion ratio is one by construction.
+            _check_declared_member_size(
+                declared=entry.size,
+                compressed=entry.size,
+                budgets=budgets,
+                subject=where,
+            )
+
+            try:
+                stream = archive.extractfile(entry)
+            except tarfile.TarError as error:
+                raise ReaderError(
+                    "READER_ADMISSION", where, f"names {member!r}, which could not be read"
+                ) from error
+            if stream is None:
+                raise ReaderError(
+                    "READER_MEMBER_TYPE",
+                    where,
+                    f"names {member!r}, which is a {_tar_member_type(entry)}. A container "
+                    "Reader opens a regular file only; a directory, symbolic link, hard "
+                    "link, fifo, character device, or block device is refused",
+                )
+            with stream:
+                payload = read_bounded(stream, declared=entry.size, subject=where)
+
+        _refuse_a_nested_container(payload, where)
+        return _emit(payload, options=options, budgets=budgets, stem_source=member)
+
+
+@dataclass(frozen=True)
+class TarReaderV1_1(TarReader):
+    """``archive.tar@1.1.0`` admits the byte-stream label, and extracts nothing new.
+
+    The same one-label delta as the zip and gzip coordinates above.  A tar's signature sits
+    at offset 257 rather than at the first byte, and this family checks that POSIX ``ustar``
+    marker itself, before ``tarfile`` parses anything -- so a response that is not a tar is
+    refused by signature whatever its label said, and the nested-container refusal still
+    reads every selected member's own first bytes against the leaf table.  ``1.0.0`` stays
+    closed to the weak label.
+    """
+
+    family_version: str = "1.1.0"
+    accepted_media_types: tuple[str, ...] = (
+        "application/octet-stream",
+        "application/x-tar",
+    )