@moqtap/codec 0.10.0 → 0.12.0

package/CHANGELOG.md

@@ -7,6 +7,181 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 This file starts at 0.9.0. Earlier releases are in the git history.
 
+## [0.12.0] - 2026-09-17
+
+### Added
+
+- **draft-21, as `@moqtap/codec/draft21` and `createCodec({ draft: '21' })`.**
+  Draft-21 restructures draft-20 and changes nothing on the wire: every code
+  point, every table row and every wire diagram is identical, so the codec, the
+  message and data-stream tables, the wire rules, the error registries and the
+  session FSM are draft-20's. The two are told apart only by what they
+  negotiate, `moqt-21` against `moqt-20`.
+
+  `message-type-names.test.ts` records that as a fact rather than a claim:
+  `20/21` joins the list of draft pairs whose whole name table coincides, which
+  until now held only drafts 08, 09 and 10.
+
+  What did move is where the draft says things -- 200 numbered sections became
+  214 and only 11 keep both their title and their number -- so every citation
+  in the new module is retargeted, two figure references with them. Requires
+  `@moqtap/test-vectors` 0.17.0, which adds the draft-21 corpus.
+
+### Fixed
+
+- **A private path no longer ships in the package.** Three source files carried
+  a "Spec of record" reference to an internal planning document. `files`
+  includes `src`, and the generated `dist/*.d.ts` reproduced it, so the path was
+  in the published tarball. The comments now say why the code is as it is
+  without naming the document.
+
+## [0.11.0] - 2026-09-17
+
+### Added
+
+- **`redactAuthTokens`, on all fourteen drafts.** Every draft entry point now
+  exports `redactAuthTokens`, with the `AuthRedaction` result type and the
+  `REDACTION_FILL` byte. It decodes a control frame for one purpose -- learning
+  where the credentials sat -- discards the message and returns the frame with
+  those bytes overwritten, so a caller never has to hold a frame containing one.
+
+  There are two credential shapes and the difference is not cosmetic. Drafts 07
+  through 10 have no Authorization Token: they carry AUTHORIZATION_INFO, a bare
+  UTF-8 string which *is* the bearer credential, so the whole parameter value
+  goes. From draft-11 only the Token Value is overwritten, and the Alias Type,
+  Token Alias and Token Type stay -- those are structure rather than secret, and
+  they are what distinguishes a session that failed to authenticate from one
+  that never tried.
+
+  A frame that fails to decode is still redacted as far as the decode reached,
+  which is the useful direction to fail in: a value that was read is a value
+  that was exposed, whatever went wrong after it. The result reports that case
+  as `incomplete`.
+
+- **Every draft's error-code registries are reachable from its entry point.**
+  draft-17's were written, committed and left unexported, so the constants sat
+  in the published package with no way to name them from outside their own file.
+
+### Fixed
+
+- **draft-16 Setup Parameters are delta-encoded, and were read as absolute.**
+  draft-16 Section 1.4.2 introduced Key-Value-Pair Types carried as "a delta
+  from the previous Type value", with the Length-present-when-odd parity decided
+  by the *resolved* type. `decodeParams` had always done this; `decodeSetupParams`
+  never did. Drafts 07-15 are unaffected because delta encoding did not exist
+  yet, and 17-20 route their setup through the delta-aware decoder -- 16 was the
+  only hole.
+
+  **This leaked credentials rather than merely misreading them.** With absolute
+  types an AUTHORIZATION_TOKEN in any position but the first resolves to the
+  wrong type -- 0x03 reads as 0x01, PATH, which is also odd and whose Length
+  happens to cover the token exactly -- so the decode *completes*, no span is
+  recorded, and the frame comes back unredacted with `redacted: 0` and nothing
+  to notice. Found by pointing a real session at a public relay, whose
+  SERVER_SETUP ends where the frame does only under the delta reading.
+
+- **draft-17's Token Value is not length-prefixed.** The encoder wrote, and the
+  decoder read, a varint length in front of the Token Value that draft-17 does
+  not define. Figure 5 writes the field as `Token Value (..)` -- the rest of the
+  value -- and the only length genuinely present is the Key-Value-Pair's own.
+  Draft-18 repeats that figure and its prose verbatim and already encoded none.
+  Because the redactor locates the credential from the outer length, the stray
+  byte also left draft-17 frames unredacted. Requires `@moqtap/test-vectors`
+  0.16.0, which drops the same byte from ten vectors.
+
+- **A status Object no longer desynchronises the streaming decoders.** draft-20
+  Section 11.4.2: "The Object Status field is only sent if the Object Payload
+  Length is zero." The streaming subgroup transforms read the length, saw zero
+  and went to the next Object, leaving the status varint to be taken for the
+  next Object's Object ID Delta -- so one status Object silently decoded every
+  Object after it as garbage. Drafts 15, 17, 18, 19 and 20 were affected, and
+  draft-15's fetch decoder with them; each draft's one-shot decoder always had
+  the branch.
+
+- **`bun run typecheck` no longer overwrites the published bundle.** tsup owns
+  `dist`, and every package `tsconfig.json` pointed `outDir` at the same
+  directory, so a local `build` followed by `typecheck` replaced the bundle with
+  an unbundled compilation plus a nested tree -- and `.npmignore` excludes
+  `dist/*/`, so the tarball kept the entry point and dropped the subtree it
+  imported. The typecheck program now emits nothing. Published tarballs were
+  unaffected only because both CI workflows happen to run `typecheck` before
+  `build`; any local `build -> typecheck -> publish` shipped it.
+
+- **The streaming decoders and the one-shot decoders now agree.** Every draft
+  from 14 to 20 ships two ways to read a data stream, and nothing had ever
+  compared them. They had drifted, in six ways, none of which corrupted a
+  payload -- which is why it survived. A consumer that switched paths got
+  objects that looked right and carried a different shape.
+
+  On subgroup streams: the incremental decoder returned Properties as opaque
+  `extensionData` where the one-shot parsed them into `objectProperties`
+  (drafts 17-20); it dropped `headerType`, `endOfGroup` and `firstObject`
+  entirely; it left the Subgroup ID at zero under the mode that derives it from
+  the first Object's ID, rather than at that ID; and it reported every Object
+  as `byteOffset: 0`.
+
+  On fetch streams it was worse. The incremental decoder returned a bare
+  `ObjectPayload` with no `serializationFlags`, no `groupId`, no `subgroupId`,
+  no `publisherPriority` and no deltas, and because it carried no state between
+  Objects, **DATAGRAM-mode Object IDs were simply wrong** -- 0, 0, 0 where the
+  one-shot resolved 1, 2, 3.
+
+  Both decoders now share one header reader and one object reader per draft,
+  lifted out of the one-shot unchanged, so there is no second copy left to
+  drift. A new cross-draft test sweeps every header type and every
+  Serialization Flags combination each draft accepts and requires the two paths
+  to agree, whole-buffer and one byte at a time.
+
+- **`createDataStreamDecoder` streams.** It picked an inner decoder and then
+  never wrote a byte to it: chunks accumulated in a local buffer and `flush`
+  decoded the whole stream in one shot, so every event arrived at end of
+  stream. On a subscription that stays open that is no events at all. It now
+  pumps the inner decoder's output as the bytes arrive. Which first bytes
+  select which decoder is unchanged.
+
+### Changed
+
+- **Breaking, at the type level.** `SubgroupStreamHeader` gains a required
+  `headerType`, and `endOfGroup` and `firstObject` where the draft has them.
+  Code that reads the header event is unaffected; code that constructs one has
+  a field to add. The value is the same one `SubgroupStream.headerType`
+  carries.
+
+  Under the Subgroup ID mode that takes the ID from the first Object, the
+  header event is now emitted after that Object has been read rather than
+  before, because that is the first moment its Subgroup ID is known. It still
+  arrives ahead of every Object in the output.
+
+### Performance
+
+- **The streaming decoders stop recopying their buffer on every chunk.** Both
+  allocated an array of `unread + chunk.length` and copied both halves into it
+  per chunk, which is quadratic in chunks per stream, and both built a fresh
+  subarray and `BufferReader` per *object*. Capacity now grows until reclaiming
+  the consumed prefix would leave the buffer at least half free, and never
+  shrinks; the reader is hoisted out of the object loop.
+
+  The buffer never compacts in place, and that is load-bearing rather than
+  incidental: payloads are handed out as views into it, so moving bytes within
+  it would corrupt an Object already emitted. The consumed prefix is dropped
+  only when a fresh array is being allocated anyway, which keeps the old array
+  alive exactly as long as the views into it.
+
+  Four megabytes, best of five, previous strategy against this one:
+
+  | objects | chunks | before | after |       |
+  | ------- | ------ | ------ | ----- | ----- |
+  | 6 KB    | 16 KB  | 0.6 ms | 0.7 ms | 0.8x  |
+  | 6 KB    | 1 KB   | 2.5 ms | 1.2 ms | 2.0x  |
+  | 120 KB  | 16 KB  | 2.5 ms | 0.7 ms | 3.5x  |
+  | 120 KB  | 1 KB   | 27.0 ms | 0.8 ms | 32.1x |
+
+  The first row is the shape the T10 captures have, and it is why a browser
+  measurement of this moved not at all: at 6 KB objects against 16 KB chunks
+  almost no chunk has an unread remainder to recopy. The last row is the same
+  code on objects those captures do contain -- they reach 120 KB -- arriving in
+  small chunks.
+
 ## [0.10.0] - 2026-09-03
 
 ### Changed

package/README.md

@@ -71,6 +71,7 @@ Each draft is available as a subpath import with its own codec and session state
 | `@moqtap/codec/draft18`          | Draft-18 codec                                    |
 | `@moqtap/codec/draft19`          | Draft-19 codec                                    |
 | `@moqtap/codec/draft20`          | Draft-20 codec                                    |
+| `@moqtap/codec/draft21`          | Draft-21 codec                                    |
 | `@moqtap/codec/draft{N}/session` | Session state machine for draft N                 |
 
 > **Note:** A default (versionless) codec will be available once the MoQT specification reaches RFC status. Until then, always specify a draft version.
@@ -78,11 +79,11 @@ Each draft is available as a subpath import with its own codec and session state
 `DRAFT_VERSIONS` is a table of numeric keys, and only the entries up to `'14'` are values a peer
 ever puts on the wire. From draft-15 the version is negotiated by ALPN (raw QUIC) or
 `WT-Available-Protocols` (WebTransport) as the string `moqt-NN`, and no version number is sent at
-all — so `DRAFT_VERSIONS['20']` is a derived identifier, not something observed. Each draft module
+all — so `DRAFT_VERSIONS['21']` is a derived identifier, not something observed. Each draft module
 from 15 on exports the real one:
 
 ```typescript
-import { PROTOCOL_STRING } from '@moqtap/codec/draft20' // 'moqt-20'
+import { PROTOCOL_STRING } from '@moqtap/codec/draft21' // 'moqt-21'
 ```
 
 ## Draft-Specific Imports

package/dist/auth-redaction-B9JJOlIY.d.cts

@@ -0,0 +1,100 @@
+/**
+ * Overwriting Authorization Token values at the point of parse.
+ *
+ * A collector that ships a customer's control plane verbatim is useful — the
+ * control plane is the part of a session a player library cannot misreport —
+ * right up until it also ships the bearer token sitting in a SETUP option or a
+ * SUBSCRIBE parameter. At that point a debugging tool has become a
+ * credential-exfiltration path.
+ *
+ * The fix has to happen **where the parse happens**, not on the way out. Bytes
+ * that were merely *masked before upload* still sat in memory in a form a later
+ * feature — a dump, an export, a bug — could reach. So this module hangs off
+ * the decoders themselves: the one pass that reads a Token Value is the pass
+ * that records where it was, and the value is overwritten before the frame is
+ * handed to anything that retains it.
+ *
+ * ── What is overwritten, and what deliberately is not
+ *
+ * Only the **Token Value** (draft-20 Section 10.2.2, Figure 5). The Alias Type,
+ * the Token Alias and the Token Type all stay:
+ *
+ *  - the parameter's *presence* distinguishes a session that failed to
+ *    authenticate from one that never tried, and losing that distinction blinds
+ *    the tool at exactly the moment it is most wanted;
+ *  - the Token Alias is how you see the same credential being reused across
+ *    messages, which is a real debugging signal and is not itself a secret;
+ *  - `USE_ALIAS` and `DELETE` carry no value at all, so those frames pass
+ *    through untouched and it is visible that they did.
+ *
+ * The overwrite is **length-preserving**. That is not an aesthetic choice: a
+ * control frame carries a fixed 16-bit Message Length, and every offset a
+ * caller has already computed into the frame stays valid only if the frame does
+ * not change size. Preserving the length also preserves the fact that a token
+ * of *some* size was sent.
+ *
+ * ── Why a span sink and not a return value
+ *
+ * A Token Value is read in `decodeParams` and `decodeSetupOptions`, which sit
+ * under a dozen per-message payload decoders per draft. Threading a redaction
+ * channel out through all of them would mean changing every one of those
+ * signatures, in every draft, to carry something almost none of them use.
+ *
+ * The sink is module state, which is safe here for one specific reason and it
+ * is worth being explicit about it: **decoding is synchronous and single
+ * threaded, and no decoder re-enters the redactor.** {@link redactWith} opens
+ * the capture, runs one decode to completion, and closes it in a `finally`.
+ * There is no `await` between those points, so no second capture can interleave
+ * with the first.
+ */
+/**
+ * The byte written over a redacted Token Value.
+ *
+ * Zero, so that a redacted frame is obviously not a token rather than
+ * plausibly one. A caller who needs to tell "redacted" from "the peer really
+ * sent zeros" has {@link AuthRedaction.redacted}, which counts what was
+ * overwritten.
+ */
+declare const REDACTION_FILL = 0;
+/**
+ * The result of a redaction pass over one control frame.
+ */
+interface AuthRedaction {
+    /**
+     * The frame with every Token Value overwritten, exactly as long as the input.
+     *
+     * When nothing needed redacting this is the **same reference** as the input,
+     * so the common case costs no copy. Callers that retain it must copy for the
+     * same reason they would have had to copy the input.
+     */
+    readonly bytes: Uint8Array;
+    /** How many Token Values were overwritten. */
+    readonly redacted: number;
+    /**
+     * A span was recorded that could not be applied to the frame.
+     *
+     * This means the offset arithmetic and the frame disagree, which is a defect
+     * rather than a wire condition. **A caller must not treat {@link bytes} as
+     * clean when this is set** — the safe response is to drop the frame's raw
+     * bytes rather than ship them, because the one thing that is certain is that
+     * a token was seen and its position is not trusted.
+     */
+    readonly incomplete: boolean;
+    /**
+     * Whether the message parsed all the way through.
+     *
+     * A frame can be perfectly legitimate and still not decode: an unknown or
+     * extension message codepoint is something a peer is allowed to send, and a
+     * collector ships those raw on purpose rather than losing the whole control
+     * plane over one of them. But a message that cannot be parsed is a message
+     * whose parameters cannot be walked, so `redacted === 0` on such a frame
+     * means "nothing was found", not "there is nothing there".
+     *
+     * That residual is narrow and it is real, and the only dishonest thing to do
+     * with it is hide it. Count the frames where this is false, so the gap in the
+     * guarantee is a number somebody can look at.
+     */
+    readonly decoded: boolean;
+}
+
+export { type AuthRedaction as A, REDACTION_FILL as R };

package/dist/auth-redaction-B9JJOlIY.d.ts

@@ -0,0 +1,100 @@
+/**
+ * Overwriting Authorization Token values at the point of parse.
+ *
+ * A collector that ships a customer's control plane verbatim is useful — the
+ * control plane is the part of a session a player library cannot misreport —
+ * right up until it also ships the bearer token sitting in a SETUP option or a
+ * SUBSCRIBE parameter. At that point a debugging tool has become a
+ * credential-exfiltration path.
+ *
+ * The fix has to happen **where the parse happens**, not on the way out. Bytes
+ * that were merely *masked before upload* still sat in memory in a form a later
+ * feature — a dump, an export, a bug — could reach. So this module hangs off
+ * the decoders themselves: the one pass that reads a Token Value is the pass
+ * that records where it was, and the value is overwritten before the frame is
+ * handed to anything that retains it.
+ *
+ * ── What is overwritten, and what deliberately is not
+ *
+ * Only the **Token Value** (draft-20 Section 10.2.2, Figure 5). The Alias Type,
+ * the Token Alias and the Token Type all stay:
+ *
+ *  - the parameter's *presence* distinguishes a session that failed to
+ *    authenticate from one that never tried, and losing that distinction blinds
+ *    the tool at exactly the moment it is most wanted;
+ *  - the Token Alias is how you see the same credential being reused across
+ *    messages, which is a real debugging signal and is not itself a secret;
+ *  - `USE_ALIAS` and `DELETE` carry no value at all, so those frames pass
+ *    through untouched and it is visible that they did.
+ *
+ * The overwrite is **length-preserving**. That is not an aesthetic choice: a
+ * control frame carries a fixed 16-bit Message Length, and every offset a
+ * caller has already computed into the frame stays valid only if the frame does
+ * not change size. Preserving the length also preserves the fact that a token
+ * of *some* size was sent.
+ *
+ * ── Why a span sink and not a return value
+ *
+ * A Token Value is read in `decodeParams` and `decodeSetupOptions`, which sit
+ * under a dozen per-message payload decoders per draft. Threading a redaction
+ * channel out through all of them would mean changing every one of those
+ * signatures, in every draft, to carry something almost none of them use.
+ *
+ * The sink is module state, which is safe here for one specific reason and it
+ * is worth being explicit about it: **decoding is synchronous and single
+ * threaded, and no decoder re-enters the redactor.** {@link redactWith} opens
+ * the capture, runs one decode to completion, and closes it in a `finally`.
+ * There is no `await` between those points, so no second capture can interleave
+ * with the first.
+ */
+/**
+ * The byte written over a redacted Token Value.
+ *
+ * Zero, so that a redacted frame is obviously not a token rather than
+ * plausibly one. A caller who needs to tell "redacted" from "the peer really
+ * sent zeros" has {@link AuthRedaction.redacted}, which counts what was
+ * overwritten.
+ */
+declare const REDACTION_FILL = 0;
+/**
+ * The result of a redaction pass over one control frame.
+ */
+interface AuthRedaction {
+    /**
+     * The frame with every Token Value overwritten, exactly as long as the input.
+     *
+     * When nothing needed redacting this is the **same reference** as the input,
+     * so the common case costs no copy. Callers that retain it must copy for the
+     * same reason they would have had to copy the input.
+     */
+    readonly bytes: Uint8Array;
+    /** How many Token Values were overwritten. */
+    readonly redacted: number;
+    /**
+     * A span was recorded that could not be applied to the frame.
+     *
+     * This means the offset arithmetic and the frame disagree, which is a defect
+     * rather than a wire condition. **A caller must not treat {@link bytes} as
+     * clean when this is set** — the safe response is to drop the frame's raw
+     * bytes rather than ship them, because the one thing that is certain is that
+     * a token was seen and its position is not trusted.
+     */
+    readonly incomplete: boolean;
+    /**
+     * Whether the message parsed all the way through.
+     *
+     * A frame can be perfectly legitimate and still not decode: an unknown or
+     * extension message codepoint is something a peer is allowed to send, and a
+     * collector ships those raw on purpose rather than losing the whole control
+     * plane over one of them. But a message that cannot be parsed is a message
+     * whose parameters cannot be walked, so `redacted === 0` on such a frame
+     * means "nothing was found", not "there is nothing there".
+     *
+     * That residual is narrow and it is real, and the only dishonest thing to do
+     * with it is hide it. Count the frames where this is false, so the gap in the
+     * guarantee is a number somebody can look at.
+     */
+    readonly decoded: boolean;
+}
+
+export { type AuthRedaction as A, REDACTION_FILL as R };