@moqtap/trace 0.3.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/SPEC.md CHANGED
@@ -32,11 +32,19 @@ The `"detail"` field (see Part 2 header) declares what was recorded. Each level
32
32
  | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------- |
33
33
  | `"control"` | Control messages only (setup, subscribe, publish, goaway, etc.). No data stream events, no object payloads. | Lightweight protocol flow analysis. DevTools default. |
34
34
  | `"headers"` | Control messages + data stream headers (subgroup/fetch/datagram headers, object metadata: group, object ID, priority, status). No payload bytes. | Delivery pattern analysis, timing. |
35
- | `"headers+sizes"` | Everything in `"headers"` + payload byte lengths for each object. | Bandwidth analysis without storing media. |
35
+ | `"headers+sizes"` | Everything in `"headers"` + payload byte lengths for each object, and the size of the bytes behind an error ([Event 6](#event-6-error) `"rawlen"`). | Bandwidth analysis without storing media. |
36
36
  | `"headers+data"` | Everything in `"headers"` + full payload bytes for each object. | Full session replay, debugging media corruption. |
37
- | `"full"` | Everything above + raw wire bytes for every message (pre-decode). | Wire-level debugging, compliance testing. |
37
+ | `"full"` | Everything above + raw wire bytes for every *control* message (pre-decode), and the bytes behind an error event ([Event 6](#event-6-error) `"raw"`). See the note below on data-stream framing. | Wire-level debugging, compliance testing. |
38
38
 
39
- Levels `"headers+data"` and `"full"`, and the `"raw"` field on `"control"`-level events, are **payload-bearing** — see [Privacy Considerations](#privacy-considerations).
39
+ Levels `"headers+data"` and `"full"`, and the `"raw"` field on Event 0 and [Event 6](#event-6-error), are **payload-bearing** — see [Privacy Considerations](#privacy-considerations). So is [Event 6](#event-6-error)'s `"raw"`, which is why it is gated at `"full"` and not at Event 6's own `control`+: an error naming a *data* stream has subgroup framing and object payload behind it, which is media, and inheriting the event's level would have put it in traces whose declared level excludes payloads entirely.
40
+
41
+ **No level carries data-stream framing bytes.** `"raw"` exists only on Event 0,
42
+ and Event 4's `"pl"` begins after the object header, so the bytes of a
43
+ `SUBGROUP_HEADER`, a fetch header or a datagram header are recorded at no level
44
+ including `"full"`. What those headers carry is recorded as decoded fields
45
+ instead — see [Event 1](#event-1-stream-opened). Closing that gap needs a new
46
+ key and is not in this revision; the level's description no longer implies
47
+ otherwise.
40
48
 
41
49
  At any detail level, the recorder MAY be configured to **mask payloads** — replacing payload bytes with zeroes before writing. This preserves payload sizes for bandwidth analysis while stripping media content. Sources MUST advertise masking via `"payloadMasked": true` in the header's `"custom"` map when active.
42
50
 
@@ -96,8 +104,8 @@ Because `"p"` is source-local, a collector correlating traces from multiple sour
96
104
  | ------- | ----------- | ------------ | ------------------------------------------------- |
97
105
  | `"d"` | integer | `control`+ | Direction: `0` = sent (tx), `1` = received (rx) |
98
106
  | `"mt"` | integer | `control`+ | Wire message type ID (e.g., `0x03` for SUBSCRIBE) |
99
- | `"msg"` | map | `control`+ | Decoded message fields (draft-specific structure) |
100
- | `"sid"` | integer | `control`+ | QUIC stream ID the message travelled on. Optional (see below). |
107
+ | `"msg"` | map | `control`+ | Decoded message fields, keyed in snake_case (see below) |
108
+ | `"sid"` | unsigned integer | `control`+ | QUIC stream ID the message travelled on. Optional (see below). |
101
109
  | `"raw"` | byte string | `full` only | Raw wire bytes (including type and length prefix). Payload-bearing — see [Privacy Considerations](#privacy-considerations). |
102
110
 
103
111
  `"sid"` is optional: a recorder that sits at the session level rather than the
@@ -114,6 +122,52 @@ track names, aliases, request outcomes — is lost to the reader. Through
114
122
  draft-16 all control messages share one stream and the field is merely
115
123
  informative.
116
124
 
125
+ #### `"msg"` field naming and shape
126
+
127
+ `"msg"` MUST be a CBOR map. Its keys are the message's field names **in
128
+ snake_case** — `request_id`, `track_alias`, `group_order` — which is what the
129
+ drafts themselves use and what the shared codec vectors carry, so a reader can
130
+ address a field without knowing which implementation wrote the file.
131
+
132
+ A writer that has no decoded fields — the message type is one it cannot parse,
133
+ or the recorder is not decoding bodies at all — MUST write an empty map `{}`
134
+ rather than omit the key. Omission costs a reader more than the one byte it
135
+ saves: Event 0 is one of the types [sampling](#sampling) MUST NOT drop, so a
136
+ reader that treats a missing `"msg"` as a malformed event discards exactly the
137
+ events the format promises to keep.
138
+
139
+ Readers MUST be more tolerant than writers, because files predating this rule
140
+ exist:
141
+
142
+ - An **absent** `"msg"` MUST be read as an empty map, not as an error.
143
+ - A `"msg"` that is **not a map** MUST be preserved verbatim and offered to the
144
+ caller unchanged. It MUST NOT cause the event to be rejected. Recordings
145
+ written before this revision carry a text rendering of the decoded message
146
+ here — every `capture-*` case in the conformance corpus is such a file — and
147
+ they remain readable, with the field simply not addressable by key.
148
+
149
+ CBOR `null` is a value, not an absence: a writer put it there, so it falls
150
+ under "not a map" and MUST be preserved. CBOR `undefined` (major type 7,
151
+ simple value 23) is **not** preservable here, and writers MUST NOT emit it.
152
+ Neither reference implementation can represent it distinctly *as a `"msg"`* —
153
+ the TypeScript reader cannot tell it from a field the caller never set, and the
154
+ Rust reader's CBOR library decodes it to `null` before the reader sees it — so
155
+ a reader MAY normalise it to an empty map or to `null`, and the two will
156
+ disagree about which. There is no file this matters for unless one is written
157
+ deliberately; the rule exists so that nobody writes one expecting it to
158
+ survive. It is one row of a general table — see [Shapes a CBOR library may
159
+ normalise before you see them](#shapes-a-cbor-library-may-normalise-before-you-see-them),
160
+ which also explains why the answer differs by *where* in a trace the value sits:
161
+ the same `undefined` inside an unrecognised-key store is preserved by one
162
+ implementation and not the other.
163
+
164
+ **A tool that reads a trace and writes it back MUST preserve a non-map `"msg"`
165
+ as it found it**, and MUST NOT replace it with an empty map to satisfy the
166
+ writer rule above. Replacing it would discard the only record of a message that
167
+ will never be seen again. This is one instance of a general rule stated under
168
+ [Versioning and Compatibility](#versioning-and-compatibility): on a rewrite the
169
+ reader rule outranks the writer rule.
170
+
117
171
  ### Event 1: Stream Opened
118
172
 
119
173
  | Key | Type | Detail Level | Description |
@@ -121,6 +175,73 @@ informative.
121
175
  | `"sid"` | integer | `headers`+ | QUIC stream ID |
122
176
  | `"d"` | integer | `headers`+ | Direction: `0` = outgoing, `1` = incoming |
123
177
  | `"st"` | integer | `headers`+ | Stream type: `0` = subgroup, `1` = datagram, `2` = fetch |
178
+ | `"ta"` | integer | `headers`+ | Track alias the stream carries. Optional. |
179
+ | `"sg"` | integer | `headers`+ | Subgroup ID. Optional; only meaningful when `st == 0`. |
180
+ | `"fri"` | integer | `headers`+ | Fetch request ID. Optional; only meaningful when `st == 2`. |
181
+ | `"g"` | integer | `headers`+ | Group ID. Optional; only meaningful when `st == 1`. |
182
+
183
+ #### Why these four keys exist
184
+
185
+ A `SUBGROUP_HEADER` carries a track alias, a group, a subgroup and a publisher
186
+ priority. A fetch header carries a request ID. A datagram header carries a track
187
+ alias, a group, an object and a priority. Before this revision the model could
188
+ express group, object, priority and status and nothing else — **track alias,
189
+ subgroup ID and fetch request ID had nowhere to live**, and at `"headers"` there
190
+ are no payload bytes to re-parse them from. A `"headers"` recording therefore
191
+ could not answer which track a stream belonged to, which is most of what the
192
+ level exists for.
193
+
194
+ **A writer MUST write `"fri"` on a fetch stream** (`st == 2`), where it is the
195
+ only correlation between the stream and the FETCH that asked for it. A reader
196
+ MUST NOT reject a fetch stream that lacks it: recordings predating this revision
197
+ have none, and Event 1 is not a type a reader may discard on a missing optional
198
+ key. The same asymmetry as [`"msg"`](#msg-field-naming-and-shape) — writers
199
+ conform, readers tolerate.
200
+
201
+ The other three are written when known. `"sg"`, `"fri"` and `"g"` are each
202
+ scoped to one stream type because on the others they have no source; a writer
203
+ MUST NOT write one outside its scope, and a reader that meets one there MUST
204
+ **keep** it — read it into the field it names, and write it back — rather than
205
+ reject the event or route it elsewhere.
206
+
207
+ "Keep", not "ignore": a reader is free to disregard the *meaning* of an
208
+ out-of-scope key, but it does not get to drop the value. Dropping would violate
209
+ [the preservation rule](#versioning-and-compatibility), and routing it to the
210
+ unrecognised-key store would be wrong twice over — the key *is* recognised, its
211
+ value *is* usable, and a key held in both places is written back twice.
212
+
213
+ Wrong *place* is not wrong *shape*, and they part company here. A `"sg"` on a
214
+ fetch stream is a usable value in a position that means nothing: it reaches the
215
+ field. A `"sg"` carrying a text string is not a usable value at all, wherever it
216
+ sits, and goes to the unrecognised-key store under [the rule for unusable
217
+ types](#versioning-and-compatibility). A reader has to answer both questions,
218
+ in that order.
219
+
220
+ **Event 3 wins.** Where a value appears both here and on an Event 3 for the same
221
+ stream — a group ID, most obviously — **Event 3 is authoritative** and this copy
222
+ is a convenience for readers that have not yet seen an object. A reader MUST NOT
223
+ treat a disagreement as corruption; it MUST prefer Event 3.
224
+
225
+ `"g"` is scoped to datagrams for that reason: on a subgroup stream every object
226
+ carries the same group by construction, so a copy here would be a second field
227
+ with no independent source. `"pp"` is deliberately **not** added for the same
228
+ reason — Event 3's `"pp"` is the stream header's publisher priority copied per
229
+ object, and a second copy would have no tiebreak.
230
+
231
+ #### Integer keys decode to the same type as their Event 3 counterparts
232
+
233
+ `"ta"`, `"sg"`, `"fri"` and `"g"` MUST decode to whatever language type an
234
+ implementation gives Event 3's `"g"`, `"o"` and `"sid"` — not to whatever it
235
+ gives Event 1's existing `"st"` and `"d"`.
236
+
237
+ Those two groups differ today, for a good reason that does not extend here:
238
+ `"st"` and `"d"` are small enumerations, and an implementation may reasonably
239
+ hold them in a narrow type. The keys added above are wire identifiers with the
240
+ same range as Event 3's. An implementer following the enclosing event's local
241
+ convention would type Event 1's `"g"` as a JavaScript `number` while Event 3's
242
+ is a `bigint`, and `event1.g === event3.groupId` would then evaluate `42 === 42n`
243
+ as `false` — silently, and for every reader that made the same reasonable
244
+ choice.
124
245
 
125
246
  ### Event 2: Stream Closed
126
247
 
@@ -162,10 +283,160 @@ Object header events and object payload events (event 4) for the same object sha
162
283
 
163
284
  ### Event 6: Error
164
285
 
165
- | Key | Type | Detail Level | Description |
166
- | ---------- | ----------- | ------------ | --------------------- |
167
- | `"ec"` | integer | `control`+ | Error code |
168
- | `"reason"` | text string | `control`+ | Human-readable reason |
286
+ | Key | Type | Detail Level | Description |
287
+ | ---------- | ---------------- | ------------ | --------------------- |
288
+ | `"ec"` | integer | `control`+ | Error code |
289
+ | `"reason"` | text string | `control`+ | Human-readable reason |
290
+ | `"sid"` | unsigned integer | `control`+ | QUIC stream ID the error was observed on. **Optional**, on the same terms as [Event 0's `"sid"`](#event-0-control-message): absent means there was no stream or none is known, and readers MUST distinguish that from stream `0`. |
291
+ | `"ek"` | text string | `control`+ | Error kind. **Optional.** See below. |
292
+ | `"rawlen"` | unsigned integer | `headers+sizes`+ | Byte length of the input the recorder held for this error, before any truncation. **Optional.** See below. |
293
+ | `"raw"` | byte string | `full` only | The offending bytes, truncated to the cap below. **Optional.** Payload-bearing — see [Privacy Considerations](#privacy-considerations). |
294
+
295
+ #### Why an error event carries bytes
296
+
297
+ A peer that sends something malformed is one of the few things a shared trace
298
+ is uniquely good for: the recording party can see it and the sending party
299
+ cannot. A report saying "your SUBSCRIBE_OK did not parse" is an assertion; the
300
+ same report carrying the bytes that did not parse is evidence, and the other
301
+ party can run it against their own encoder without reproducing the session.
302
+
303
+ Before these keys the only field able to hold bytes was Event 0's `"raw"`, and
304
+ a recorder wanting to keep the evidence had to claim the violation was a
305
+ control message in order to have somewhere to put it. That is a worse record
306
+ than none: it asserts a decodable message where there was a protocol
307
+ violation, and no reader can tell the two apart. One recorder in this project
308
+ declined the trade and left the note behind in its source — *"the offending
309
+ bytes stay behind: Event 6 has no field to carry them"* — which is the gap
310
+ these keys close.
311
+
312
+ #### `"ek"` — error kind
313
+
314
+ An open vocabulary. This revision names `"protocol"` (the peer violated the
315
+ protocol), `"transport"` (the QUIC or WebTransport layer failed), and
316
+ `"decode"` (bytes that would not parse as any message this recorder knows).
317
+ Other values MAY be used and MAY be added without a version bump; readers MUST
318
+ preserve a value they do not recognise and MUST NOT reject the event, as with
319
+ [`"perspective"`](#perspective).
320
+
321
+ `"ek"` is spelled in full rather than as `"kind"` because `"kind"` is already a
322
+ top-level key on [Event 10](#event-10-subscription-derivation) with the
323
+ vocabulary `"created"` / `"shared"`. Two vocabularies under one key at the same
324
+ map level survive in a typed reader and break in every flat projection of a
325
+ trace — a JSONL export, a warehouse load, a column-oriented viewer — where
326
+ `kind` becomes one column holding both.
327
+
328
+ #### `"raw"` is capped, and `"rawlen"` says whether the cap bit
329
+
330
+ **A recorder MUST NOT write more than 4096 bytes in `"raw"`.** Where it holds
331
+ more, it writes the first 4096 and records the full length in `"rawlen"`.
332
+
333
+ The cap is a MUST and a fixed number rather than a SHOULD and a suggestion,
334
+ because Event 6 is [non-droppable](#non-droppable-event-types). Every other
335
+ high-volume field in this format is bounded by sampling; this one cannot be,
336
+ so if it is not bounded here it is not bounded at all. A peer opening ten
337
+ thousand streams and sending garbage on each is not a hypothetical — it is
338
+ what a fuzzer does to a relay, and a trace of it should not be larger than the
339
+ attack. A number a reader can test also means the corpus can hold a case
340
+ proving the cap was applied rather than merely described; "SHOULD keep it
341
+ small" cannot be tested by anyone.
342
+
343
+ 4096 was chosen against two known quantities. It is far above what control
344
+ messages actually measure: the largest encoded control message across every
345
+ draft in the shared codec vectors is 95 bytes, and each of the four real
346
+ third-party captures in the conformance corpus is under 3 kB *in total*,
347
+ control messages, object headers and all. And it sits well below
348
+ `MAX_MESSAGE_LENGTH`, the 65535-byte ceiling the protocol itself puts on a
349
+ control message from draft-11 onward — deliberately below, because `"raw"`
350
+ exists for diagnosis and not for replay. A message can legitimately be larger
351
+ than the cap; the cap is not a claim that it cannot. A fault that leaves no trace in the
352
+ first 4096 bytes is a real possibility, and it is what `"rawlen"` is for: the
353
+ reader learns that the record is partial and how much is missing, which is
354
+ enough to go and ask for the rest.
355
+
356
+ **`"rawlen"` sits at `headers+sizes`, not at `full` and not at `control`.** It
357
+ is available in every trace where `"raw"` MUST NOT appear, which is the point
358
+ of having it: how large a malformed message was is often enough on its own to
359
+ separate a truncated message from a mistyped one, and it carries no content.
360
+
361
+ It is nonetheless a **size**, and this format gates sizes deliberately —
362
+ `headers+sizes` exists precisely to mean "how much, but not what". Putting
363
+ `"rawlen"` at `control` would let a `control`-level trace carry a fact about
364
+ data-stream volume that its own declared level excludes, because Event 6's
365
+ `"sid"` may name a data stream: the length would then be the size of a run of
366
+ media framing. One number is a small leak, but the latch below bounds `"raw"`
367
+ and not `"rawlen"`, so on a repeatedly failing stream it is a *sequence* of
368
+ sizes, which is the shape traffic analysis wants. Gating it with the other
369
+ sizes costs a `control`-level trace one diagnostic and keeps each level's
370
+ promise exactly what it says.
371
+
372
+ A recorder at `headers+sizes` or above SHOULD write it whether or not it is
373
+ also writing the bytes.
374
+
375
+ When `"raw"` is present, `"rawlen"` SHOULD be present too, and a reader MAY
376
+ take `rawlen > len(raw)` as the definition of a truncated capture. If
377
+ `"rawlen"` is absent, a `"raw"` of exactly 4096 bytes MUST be treated as
378
+ possibly truncated, since that is the one length the cap makes ambiguous.
379
+
380
+ **A recorder that does not know the true length MUST omit `"rawlen"` rather
381
+ than write the length it happens to hold.** This is the case the key is most
382
+ easily got wrong in, and it defeats the key entirely. A recorder that stops
383
+ *reading* at the cap — which is the sensible thing to do with a stream that
384
+ will not parse, and what the recorder cited below does — never holds more than
385
+ 4096 bytes. Writing `rawlen` as 4096 there says `rawlen == len(raw)`, which is
386
+ this section's own signal for **not truncated**: the field would assert
387
+ completeness precisely when the recorder has the least idea whether the input
388
+ was complete. Omitting it falls through to the rule above, where a `"raw"` of
389
+ exactly the cap is treated as possibly truncated, which is the truth.
390
+
391
+ Where the length *is* knowable without keeping the bytes, write it. A
392
+ length-prefixed control message supplies it from its own length field, and a
393
+ recorder that read that prefix and then gave up on the body knows exactly how
394
+ much it did not keep. That is the case `"rawlen"` exists for, and it is common:
395
+ the fault is often the body, and the prefix is what got the recorder as far as
396
+ noticing.
397
+
398
+ **The cap binds the recorder, and nothing else.** It is addressed to the party
399
+ deciding what to say about traffic it has just observed. It is *not* addressed
400
+ to a tool that reads an event and serializes it again: a reader meeting a
401
+ `"raw"` longer than 4096 bytes MUST NOT reject the event, and a rewrite MUST
402
+ NOT shorten it. That is [the reader rule outranking the writer
403
+ rule](#versioning-and-compatibility), and this is the case it exists for.
404
+ Re-truncating someone else's file destroys evidence in order to make it conform
405
+ to a rule that was never addressed to the tool doing the truncating. Report the
406
+ non-conformance if it is worth reporting; do not repair it.
407
+
408
+ The distinction has a consequence worth stating for implementers, because
409
+ getting it wrong is easy and the result is silent. **A serializer is the wrong
410
+ place to enforce this cap.** A serializer cannot tell a freshly recorded event
411
+ from one that arrived by being read, so a cap applied there either truncates
412
+ evidence on rewrite or rejects a file the reader was required to accept —
413
+ whichever it does, it does to the wrong events. The cap belongs where the event
414
+ is *constructed from observed bytes*.
415
+
416
+ #### One `"raw"` per flow
417
+
418
+ **A recorder MUST NOT emit `"raw"` more than once per flow** — per stream where
419
+ the error names one, per peer where it does not, and once for the whole
420
+ recording where it writes no peer identifier either. That last case is every
421
+ recorder but a relay tap, since `"p"` is only required of one: the three
422
+ collapse to "the narrowest scope the recorder can name", and a recorder that
423
+ can name none has one flow. Later errors on a flow that
424
+ has already carried its bytes are still recorded; they simply carry no `"raw"`.
425
+
426
+ The latch is on the field, not on the event, and the distinction is the whole
427
+ point. Event 6 is non-droppable: suppressing the *event* would discard the
428
+ causal record this document elsewhere forbids discarding, and would do it
429
+ precisely when a peer is misbehaving repeatedly, which is when the record
430
+ matters most. What actually grows without bound is the bytes, so that is what
431
+ is bounded. The first failure on a flow is the diagnostic one in any case —
432
+ once a stream will not parse, a recorder is no longer interpreting what
433
+ follows, so the "bytes" of the second error are the same undifferentiated run
434
+ as the first.
435
+
436
+ This matches what a recorder in this project already does, one level up: its
437
+ control-stream reader latches on the first undecodable run and stops reading
438
+ that stream, capping the kept bytes at 4096. This section takes that behaviour,
439
+ which was chosen for a panel display, and makes it the format's rule.
169
440
 
170
441
  ### Event 7: Annotation
171
442
 
@@ -271,11 +542,16 @@ Trace recording ranges from protocol-metadata-only (low sensitivity) to full med
271
542
  1. **Payload-bearing events and fields.** These carry or reveal user content:
272
543
  - Event 4 `"pl"` (object payload bytes, `"headers+data"`+)
273
544
  - Event 0 `"raw"` (raw control-message bytes, `"full"`)
545
+ - [Event 6](#event-6-error) `"raw"` (the bytes behind an error, `"full"`).
546
+ Easy to overlook, because an error event sounds like metadata: its
547
+ `"sid"` may name a *data* stream, in which case the bytes behind the
548
+ error are subgroup framing and object payload — media, arriving through
549
+ the one event type sampling may never drop.
274
550
  - Detail levels `"headers+data"` and `"full"` in aggregate
275
551
 
276
552
  Capturing them is **optional** for all conformant recorders. Observer- and tap-based capture of these fields is equivalent to intercepting user media; recorders MUST NOT enable it by default and SHOULD require explicit operator opt-in per session or per track. The default detail level for general-purpose tools SHOULD be `"control"` or `"headers"`.
277
553
 
278
- 2. **Payload size leakage.** Even without payload bytes, `"headers+sizes"` reveals object sizes, which can fingerprint media content. Operators handling sensitive traffic SHOULD evaluate size leakage as part of their threat model.
554
+ 2. **Payload size leakage.** Even without payload bytes, `"headers+sizes"` reveals object sizes, which can fingerprint media content. [Event 6](#event-6-error)'s `"rawlen"` belongs to this class rather than to the one above — it is gated with the sizes for that reason — and it leaks a little differently: the one-`"raw"`-per-flow latch does not bind it, so a stream that fails repeatedly yields a *sequence* of sizes rather than a single one. Operators handling sensitive traffic SHOULD evaluate size leakage as part of their threat model.
279
555
 
280
556
  3. **Trace ID correlation.** `MOQTAP_TRACE_ID` enables cross-operator correlation of subscription behavior. See [Trace ID Propagation](#trace-id-propagation) for egress mitigations.
281
557
 
@@ -326,9 +602,9 @@ A single CBOR map with the following keys:
326
602
  | `"protocol"` | text string | Yes | MoQT version identifier (e.g., `"moq-transport-17"`, `"moq-transport-rfc9999"`) |
327
603
  | `"perspective"` | text string | Yes | Recording viewpoint: `"client"`, `"server"`, `"observer"`, or `"relay-tap"` |
328
604
  | `"detail"` | text string | Yes | Detail level (see [Detail Levels](#detail-levels)) |
329
- | `"startTime"` | integer | Yes | Recording start time (Unix epoch milliseconds) |
605
+ | `"startTime"` | unsigned integer | Yes | Recording start time (Unix epoch milliseconds). Unsigned: a trace from before 1970 is a clock fault, not a recording, and a reader that accepts one has no way to say so. |
330
606
  | `"transport"` | text string | No | Transport type (e.g., `"webtransport"`, `"raw-quic"`) |
331
- | `"endTime"` | integer | No | Recording end time (Unix epoch milliseconds). Written when trace is finalized. |
607
+ | `"endTime"` | unsigned integer | No | Recording end time (Unix epoch milliseconds). Written when trace is finalized. |
332
608
  | `"source"` | text string | No | Software that produced the trace (e.g., `"moqtap-devtools/0.1.0"`, `"my-relay/2.3.1"`). Also serves as the namespace for source-local `"p"` values. |
333
609
  | `"endpoint"` | text string | No | Remote peer URI (e.g., `"https://relay.example.com/moq"`) |
334
610
  | `"sessionId"` | text string | No | Capture-correlation identifier — see [Identifier Scoping](#identifier-scoping). |
@@ -336,6 +612,186 @@ A single CBOR map with the following keys:
336
612
  | `"sampling"` | map | No | Sampling/filter metadata; present iff events were sampled or filtered at the source. See [Sampling](#sampling). |
337
613
  | `"custom"` | map | No | User-defined metadata (arbitrary key-value pairs). `"payloadMasked": true` here indicates payload masking is active. |
338
614
 
615
+ ### Unrecognised keys in the header
616
+
617
+ [Versioning and Compatibility](#versioning-and-compatibility) requires a reader
618
+ that writes a trace back out to preserve the unrecognised keys it read, in
619
+ "header maps or event maps". This section says what that means for the header,
620
+ because the header is where the rule was least obviously binding and where both
621
+ reference implementations broke it: each read a header, dropped every key it
622
+ did not know, and wrote out a file that looked as though it had never carried
623
+ them.
624
+
625
+ **Three maps in the header have keys this document names**: the header map
626
+ itself, `"segment"`, and `"sampling"`. Each MUST carry its own
627
+ unrecognised-key store, and each store MUST be written back into the map it
628
+ came from. One store for the whole header would not do — a private key on
629
+ `"segment"` and a key of the same name at the top level are different keys, and
630
+ re-emitting either in the other's map changes what the file says.
631
+
632
+ Everything the [key-preservation rules](#versioning-and-compatibility) say
633
+ about events applies here unchanged, reading "header" for "event" and "the
634
+ segment" for "the event":
635
+
636
+ - A key this document does not define goes to the store of the map it appeared
637
+ in, and is written back from there.
638
+ - A key this document *does* define, carrying a value the reader cannot use,
639
+ goes to that same store. Knowing more about a key must not mean preserving it
640
+ less. `"transport": 42` is not a transport; it is also not nothing.
641
+ - Which values are usable is settled by [the same
642
+ list](#versioning-and-compatibility) — integral floats yes, fractional no,
643
+ negative no for a key defined as unsigned, and no rounding of a value the
644
+ reader cannot represent.
645
+ - **The range clause binds only where this document states a range.** In the
646
+ header that is one key: `"effectiveRate"`, defined as lying in `(0.0, 1.0]`.
647
+ A rate of `1.5`, of `0.0`, or of NaN is unusable and goes to the sampling
648
+ store. This is a semantic check rather than a type check and it is the only
649
+ one in the header, deliberately — every other key here is bounded by its type
650
+ alone, and a reader inventing bounds this document does not state would start
651
+ rejecting values a future revision means to allow. The reason `"effectiveRate"`
652
+ earns the check is that consumers divide by it: a rate of `0.0` handed to a
653
+ caller reconstructing true event counts is a division by zero, and `1.5` is a
654
+ count larger than what was recorded.
655
+ - **A value under a CBOR tag is unusable**, tag 64 excepted. This format uses
656
+ no tags; a writer MUST NOT emit one. Tag 64 is the exception readers already
657
+ MUST accept for byte strings (see [Interoperability](#interoperability)), and
658
+ a reader that unwraps it and re-emits the bytes as major type 2 has done what
659
+ that rule asks — so "written back unchanged" binds the value, not its
660
+ encoding. Any other tag is a shape this document gives no meaning, and it
661
+ goes to the store with its tag intact where the reader can hold one.
662
+
663
+ #### A composite value is usable only as a whole
664
+
665
+ `"sampling.appliesTo"` is an array of event type IDs. If one element is not
666
+ one, the array is unusable and goes to the store **entire**. A reader MUST NOT
667
+ keep the elements it liked and discard the rest.
668
+
669
+ This is the shape where discarding is not merely lossy but wrong. An
670
+ `"appliesTo"` of `[3, "x", 5]` read as `[3, 5]` is not a partial answer: the
671
+ key names the event types the drop policy touched, and every type absent from
672
+ it may be treated as complete. Shortening the array therefore reports a sampled
673
+ event type as fully recorded — the opposite of what the file said, stated with
674
+ the same confidence. The same reasoning governs `"ns"` on Event 10, and any
675
+ array or map a future key defines.
676
+
677
+ #### `"custom"`, and values a reader cannot hold
678
+
679
+ `"custom"` is not one of the three maps above: every key in it belongs to
680
+ whoever wrote the trace, so there is no such thing as an unrecognised key
681
+ there and it needs no store. It is a passthrough. A reader MUST hand back what
682
+ the file carried, key for key and value for value, and MUST write back what it
683
+ was handed.
684
+
685
+ A reader whose representation cannot hold that map exactly MUST treat
686
+ `"custom"` as unusable and route the whole value to the *header's* store,
687
+ rather than hand back a version of it that lost something. Both reference
688
+ implementations declare `"custom"` as a string-keyed map, so for them a
689
+ `"custom"` that is not a map at all is unusable — and so is one carrying a
690
+ non-text key, *where the reader can see that it does*; the next subsection
691
+ explains why that qualifier is not a loophole. Losing typed access is the
692
+ smaller harm: nothing in this document gives `"custom"` keys meaning, so there
693
+ is nothing to lose but convenience, and the bytes survive.
694
+
695
+ A reader whose decoder happens to hand back a shape its own declared type
696
+ cannot describe — a map object where a string-keyed record was promised — is
697
+ not thereby conformant. It is preserving the bytes by accident, through a type
698
+ that lies to every caller reading the field, and the first caller to iterate
699
+ the keys finds a shape it was told could not occur. Route the value to the
700
+ store and keep the field's type true.
701
+
702
+ That generalises to a test worth applying to every key in this section:
703
+ **can this reader write back exactly what it read?** Where it can, the value
704
+ belongs in the field. Where it cannot, the value belongs in the store. A
705
+ reader that answers "no" and keeps the field anyway is the defect this section
706
+ exists to name.
707
+
708
+ #### Keys in these maps are text
709
+
710
+ Every key this document defines — in the header map, in `"segment"`, in
711
+ `"sampling"`, and by the nature of the field in `"custom"` — is a text string,
712
+ and **a writer MUST NOT emit a key of any other kind.**
713
+
714
+ What a reader does with one is deliberately left undefined, and nothing may
715
+ depend on two readers agreeing about it. The reason is worth stating, because
716
+ the obvious rule is one that cannot be implemented: **some CBOR decoders
717
+ cannot tell.** A decoder that returns maps as language-native objects coerces
718
+ the integer key `1` into the text key `"1"` before any code written against
719
+ this document runs, at which point it is indistinguishable from a text `"1"`
720
+ the same map might also carry — and the two collide. One of the two reference
721
+ implementations decodes exactly this way. Requiring it to report the
722
+ difference would be requiring something it cannot observe, and a specification
723
+ that mandates the impossible teaches implementers which of its rules to skip.
724
+
725
+ This is one instance of a general problem, not a special case: see [Shapes a
726
+ CBOR library may normalise before you see them](#shapes-a-cbor-library-may-normalise-before-you-see-them)
727
+ for the full list and the rule that governs all of it. A non-text key is the
728
+ most obvious member; the text key `"__proto__"`, a duplicate key, and CBOR
729
+ `undefined` are others, and each is invisible to at least one of the two
730
+ reference implementations.
731
+
732
+ So the requirement lands on the writer, where it can be met. What a reader
733
+ MUST NOT do is **re-key** — turn an integer key into a text key and write that
734
+ back — because the result is a file asserting a key the original never carried,
735
+ possibly colliding with a real one. A reader that can hold the key as it found
736
+ it SHOULD do exactly that and preserve it in the store; a reader whose decoder
737
+ has already re-keyed it before it was asked is not thereby non-conformant,
738
+ having never had the choice.
739
+
740
+ Rejecting the header is *not* the recommended answer here, which is worth
741
+ saying because an earlier draft of this section recommended it. Failing the
742
+ header loses every event behind it, and this subsection would then be
743
+ prescribing, for the most capable reader, the most destructive outcome — the
744
+ same harm the `"segment"`/`"sampling"` subsection below forbids for a value it
745
+ merely cannot use. Preserve it and move on. This is the one place in this
746
+ section where the two implementations are permitted to differ, and it is
747
+ permitted only because no conformant file reaches it.
748
+
749
+ #### `"segment"` and `"sampling"` are optional keys
750
+
751
+ A `"segment"` or `"sampling"` that is not a map MUST NOT fail the file. It goes
752
+ to the store like any other unusable optional value. A reader that rejects the
753
+ file instead has turned one unreadable metadata value into the loss of every
754
+ event behind it.
755
+
756
+ **But an unusable `"segment"` is not an absent one.** [Segmented
757
+ Traces](#segmented-traces) requires a reader to treat an absent `"segment"` as
758
+ "this is a complete, single-segment file", and that assertion MUST NOT be made
759
+ here: the file said it was a segment and the reader merely could not place it.
760
+ Making it would be the same invention as defaulting `"sequence"` to `0`, and
761
+ worse in a segmented stream, where a segment read as non-segmented has its
762
+ `"n"` and `"t"` taken as file-global when they are segment-local, silently
763
+ misordering every event in it.
764
+
765
+ No new field is needed to tell the two apart: the unusable value is sitting in
766
+ the header's store under the key `"segment"`, so a reader that finds it there
767
+ knows exactly which case it is in. What it MUST NOT do is answer "complete
768
+ single-segment file" to a caller that asks.
769
+
770
+ `"segment.sequence"` is the exception, and the only key in the header whose
771
+ absence or unusability makes the header itself malformed. It is the sole
772
+ ordering key of a segmented stream: a reader that cannot read it cannot place
773
+ the segment, and a default invents an order the file never had. The four
774
+ required top-level keys — `"protocol"`, `"perspective"`, `"detail"`,
775
+ `"startTime"` — are malformed on the same terms, there being no header to
776
+ construct without them.
777
+
778
+ Malformed here means malformed *for that segment*. A reader MUST report it and
779
+ MUST NOT present the segment as read; it MAY continue to the next segment, on
780
+ the same terms as [a malformed event](#versioning-and-compatibility). What it
781
+ MUST NOT do is return a header it invented — a required key filled in with a
782
+ default, an empty string, or a language's null standing in for a value the file
783
+ never carried. That reader reports a fabricated trace as a real one, which is
784
+ worse than reporting nothing.
785
+
786
+ #### The store is written last, and never twice
787
+
788
+ A store entry whose key the surrounding map already writes MUST NOT be written
789
+ a second time. A reader never produces such an entry — a key it recognised and
790
+ used is by definition not unrecognised — but a caller assembling a header by
791
+ hand can, and a CBOR map carrying one key twice is a map no two readers need
792
+ agree on. The field wins and the store entry is dropped. Event serialization
793
+ already resolves the collision this way.
794
+
339
795
  ### Protocol Identifier
340
796
 
341
797
  The `"protocol"` field identifies the MoQT version without assuming lifecycle stage. Use the IETF document name minus the `draft-ietf-` prefix for drafts, and the RFC number for published standards:
@@ -436,10 +892,87 @@ If the stream ends part-way through a CBOR item, the file was truncated — a cr
436
892
  - **Version 2** is defined by this document. **Version 1** is defined by the previous revision.
437
893
  - **Writers SHOULD write version 2.**
438
894
  - **Readers MUST accept both version 1 and version 2, and MUST reject any other version.** A version-1 file is exactly a non-segmented version-2 trace without the keys this revision adds, and every added key is optional. Accepting version 1 therefore costs a reader nothing and keeps every previously recorded capture readable.
439
- - Unknown keys in header maps or event maps MUST be ignored (forward-compatible).
895
+ - Unknown keys in header maps or event maps MUST be ignored (forward-compatible). For the header specifically, see [Unrecognised keys in the header](#unrecognised-keys-in-the-header) — the header has three maps with named keys, not one, and each keeps its own.
896
+ - **A reader that writes a trace back out MUST preserve the unrecognised keys it read**, on a recognised event type as much as on an unrecognised one, and in the header as much as in an event. "Ignore" above means read past, not discard. Skipping a key costs a reader nothing; dropping it costs every reader downstream, because the output is a valid file that looks like it never carried the key — and the tools that read and rewrite traces are exactly the ones a trace passes through on its way to someone else: a redaction pass, a filter, a re-segmentation, a download with annotations applied.
440
897
  - Optional keys MAY be added to an existing event type without a version bump — `"sid"` on event 0 was added this way. A reader must therefore treat any optional key as absent-by-default rather than assuming files of a given version all carry the same keys.
441
898
  - New event types (`"e"` values) MAY be added without a version bump; readers MUST skip unknown event types rather than failing. A reader that rejects an unknown `"e"` value turns every future addition into a breaking change.
442
899
  - New `"perspective"` and `"detail"` values MAY likewise be added; see [Perspective](#perspective).
900
+ - **A defined key whose value has an unusable type is treated as unrecognised.**
901
+ If this document says `"ta"` is an integer and a file carries `"ta": "hello"`,
902
+ a reader cannot use the value — but it MUST NOT delete it. The key goes to the
903
+ unrecognised-key store, is ignored for meaning, and is written back unchanged,
904
+ exactly as a key the reader had never heard of would be. A reader that knows
905
+ more about a key must not therefore preserve it less. "Unchanged" binds the
906
+ value, not its position: such a key is written wherever the store is emitted,
907
+ which is generally after the keys the event type owns rather than where the
908
+ key originally sat. CBOR maps are unordered, so this loses nothing — but a
909
+ byte-for-byte diff against the input will show it.
910
+
911
+ This matters most where it is least expected: adding a key to a reader's
912
+ vocabulary must never *reduce* what that reader preserves. Before a key is
913
+ defined, a wrong-typed value survives as an unrecognised key; after, a reader
914
+ that simply type-checks and moves on drops it. Both reference implementations
915
+ did precisely that when Event 1's keys were added, and lost values they had
916
+ previously kept.
917
+
918
+ Two bounds on the rule. A **required** key with an unusable type is a
919
+ malformed event, indistinguishable from that key being absent, because there
920
+ is no event to construct without it. And an unusable **optional** key MUST NOT fail
921
+ the event at all — it is the case this whole rule exists for, and the value
922
+ goes to the store.
923
+
924
+ That bound is about the event, not the API. Whether a malformed event ends the
925
+ read is a property of how a reader is driven: a streaming reader whose
926
+ idiomatic use collects into a fallible result stops at the first one, so for
927
+ that caller failing an event *is* failing the file. Readers MAY stop and MAY
928
+ continue, but MUST make which one is happening visible to the caller, and MUST
929
+ NOT report a partial event list as a complete one. Two conformant readers can
930
+ return different event counts for the same malformed file; nothing may depend
931
+ on them agreeing.
932
+
933
+ **What "unusable" means**, because two readers that draw this line differently
934
+ disagree about which values reach a field and which reach the store — and that
935
+ disagreement is invisible until someone diffs their output:
936
+
937
+ - A CBOR integer and a float with an integral value are both usable for an
938
+ integer key. Files carrying the float form exist and readers already MUST
939
+ accept them (see [Interoperability](#interoperability)); this rule does not
940
+ take that back.
941
+ - A number with a fractional part is **unusable** for an integer key.
942
+ - A negative value is **unusable** for a key this document defines as
943
+ unsigned. Every identifier here is a wire varint and has no negative form.
944
+ - A value outside the range the key's meaning allows is **unusable** where the
945
+ key is optional — `"sg"` as `-1`. Where the key is **required**, a reader
946
+ that can represent the value MUST still build the event: `"pp": 999` is a
947
+ number no conformant writer produces, but discarding the event loses far
948
+ more than the odd priority does. Report it if you like; do not delete it.
949
+ A required value the reader genuinely cannot represent remains malformed,
950
+ which is the same outcome as the key being absent.
951
+ - An implementation that holds a key in a type narrower than the key's range
952
+ MUST treat an unrepresentable value as unusable rather than rounding it.
953
+ Rounding invents a value the file never carried. This is why the identifier
954
+ keys must be held in a 64-bit-capable type: in a language where the ordinary
955
+ number type stops being exact at 2^53, using it for a `u64` identifier makes
956
+ that reader disagree with every other one above 2^53 — silently, and only
957
+ for large values, which is to say only in production.
958
+ - **On a rewrite, the reader rule outranks the writer rule.** A tool that reads
959
+ a trace and writes it back is bound by what it read, not by what a recorder
960
+ should have written. Where this document tells a writer MUST NOT emit
961
+ something — a `"msg"` that is not a map, a key outside its stream-type scope —
962
+ and tells a reader to keep it, a rewrite **keeps it**. The writer rules bind a
963
+ recorder deciding what to say about traffic it just saw; they do not license a
964
+ redaction pass, a filter or a format converter to quietly correct someone
965
+ else's file into one that no longer says what happened. A tool that wants to
966
+ report non-conformance should report it, not silently repair it.
967
+ - **Keys beginning `"x-"` are reserved for private use.** No revision of this
968
+ specification will define one, so a tool may add its own without ever
969
+ colliding with a future key — and a conformance test can rely on such a key
970
+ staying unrecognised. Every rule above applies to them unchanged: ignore them,
971
+ and preserve them on a rewrite. Absent this reservation there is no key an
972
+ implementation can be *sure* stays unknown, which matters more than it sounds:
973
+ a test fixture built from keys borrowed from a live proposal silently stops
974
+ testing anything the moment that proposal ships, and this specification did
975
+ exactly that to its own corpus once.
443
976
  - The detail level in the header is informational — readers MUST handle missing fields gracefully regardless of declared level.
444
977
 
445
978
  ### Why version 2 exists
@@ -462,10 +995,168 @@ Two encoding choices are **normative**, because a CBOR library's defaults are no
462
995
  - **An integral value MUST be written as a CBOR integer (major type 0 or 1), not as a float.** CBOR permits a float carrying an integral value, and some encoders emit one for any number beyond 32 bits — which every epoch-millisecond timestamp is. A reader MUST nevertheless accept the float form for any integer field, rejecting only a value that is fractional or outside the range a float represents exactly, since files written that way exist.
463
996
  - **A byte string MUST be written as major type 2, not wrapped in the typed-array tag 64 of RFC 8746.** A reader MUST nevertheless accept tag 64 wrapping a byte string, for the same reason.
464
997
 
998
+ The first rule is about the **value**, not about the type this document gives
999
+ the key. `"effectiveRate"` is defined as a float and its most common value is
1000
+ `1.0` — "no rate-based dropping" — which is integral, so it is written as the
1001
+ CBOR integer `1`. Conversely a reader MUST accept a CBOR integer for a key this
1002
+ document types as a float, which is the mirror of the sentence above and was
1003
+ left implicit for one revision too long: the two reference implementations
1004
+ disagreed on exactly this value, one writing a float64 and one an integer, for
1005
+ a key whose commonest setting triggers it.
1006
+
1007
+ Both directions are byte-level differences that preserve the number exactly.
1008
+ That is the general shape of these rules: they exist so that two encoders
1009
+ produce the same bytes for the same trace, not because either encoding loses
1010
+ anything.
1011
+
465
1012
  Both rules exist because both were broken in opposite directions by the two implementations, and neither test suite could see it: each read only bytes it had written itself, so each agreed with a convention the other did not implement.
466
1013
 
467
1014
  This claim is maintained by a shared corpus of `.moqtrace` files that both implementations read and write as part of their test suites, covering at minimum: a version-1 file, a non-segmented version-2 file, a segmented version-2 stream, a file carrying an unknown event type, a file carrying an unknown perspective, and a truncated file. An implementation that cannot round-trip the corpus is not conformant, whatever this document says.
468
1015
 
1016
+ The corpus is [`moqtrace/` in the `test-traces` repository](https://github.com/moqtap/test-traces/tree/master/moqtrace), published as `@moqtap/test-traces`. Its `manifest.json` indexes every case with the format version, segment count, header fields, event count and event-type histogram a conformant reader must agree on, so a third implementation can use it without reading either reference implementation. Two files there carry the non-canonical encodings the rules above require readers to accept — integers as floats, and byte strings under tag 64 — because a rule with no file exercising it is a rule nobody is held to.
1017
+
1018
+ ### Shapes a CBOR library may normalise before you see them
1019
+
1020
+ Every rule in this document about preserving a value assumes the reader is
1021
+ handed what the file carried. Sometimes it is not. A CBOR library may normalise
1022
+ a decoded value before any code written against this specification runs, and in
1023
+ each case below the normalisation is invisible: the reader cannot report a
1024
+ difference it was never shown.
1025
+
1026
+ The two reference implementations are known to do the following, and neither
1027
+ can be configured out of it without changing decoders:
1028
+
1029
+ | Shape in the file | What one library hands back |
1030
+ | ----------------- | --------------------------- |
1031
+ | An integer map key, `1` | The text key `"1"` — which then collides with a real `"1"` |
1032
+ | The text key `"__proto__"` | The text key `"__proto_"` — which then collides with a real `"__proto_"` |
1033
+ | The same key twice † | One entry — `cbor-x` keeps the last. **`ciborium` keeps both**, so a Rust reader *can* see this one |
1034
+ | CBOR `undefined` (major type 7, value 23) | CBOR `null` |
1035
+ | A byte string under RFC 8746 tag 64 | A bare byte string |
1036
+ | A float carrying an integral value | An integer |
1037
+
1038
+ † **The duplicate-key row is asymmetric, and that asymmetry is the point.**
1039
+ `ciborium` models a map as a list of pairs and hands back both entries, so part
1040
+ 2 binds a Rust reader there: it can see the duplicate and must not delete
1041
+ either entry. `cbor-x` has already collapsed the pair before its caller runs,
1042
+ so part 3 applies to a JavaScript reader on the same file. A row in this table
1043
+ is not automatically excused for every implementation — read parts 2 and 3
1044
+ against the library actually in use, and expect a shape to be observable to one
1045
+ reader and invisible to the other.
1046
+
1047
+ **The rule, in three parts.**
1048
+
1049
+ 1. **A writer MUST NOT *choose* any shape in that table.** Nothing this format
1050
+ defines needs one, so a recorder composing a trace out of what it observed
1051
+ has no reason to produce one, and this is where the requirement can actually
1052
+ be met.
1053
+
1054
+ This binds what a writer **originates**, not what it relays. A store may
1055
+ legitimately hold one of these shapes — the Rust reader hands back
1056
+ `"__proto__"` and an integer map key intact — and writing that store back
1057
+ offers exactly three options: drop the entry, rename it, or emit it as it
1058
+ stands. The first loses a value and the second invents one, and part 2
1059
+ forbids both. So the third is what a conformant rewrite does, and this rule
1060
+ does not stand in its way: **a stored entry is written as it stands.**
1061
+ Anything else would make a reader non-conformant for faithfully preserving
1062
+ what it was given, which is the opposite of what this section is for.
1063
+
1064
+ **Two rows are exceptions, for opposite reasons.** The last two — an
1065
+ integral float, and a byte string under tag 64 — are exceptions because a
1066
+ value-preserving re-encoding *exists*: rewriting them costs nothing but
1067
+ bytes, and the paragraph after this one requires it. The duplicate-key row
1068
+ is an exception because no such re-encoding exists at all: a map cannot
1069
+ carry one key twice and still be a map any two readers agree on, so a
1070
+ rewrite must drop an entry, and dropping it loses a value. That is the one
1071
+ place this document knowingly accepts a loss, and it is set out at the end
1072
+ of this section rather than waved at here.
1073
+ 2. **Where a reader can observe the shape, it MUST NOT substitute a different
1074
+ one.** Route it to the unrecognised-key store and preserve it, or report the
1075
+ value as unusable — but do not hand back a value the file did not carry. A
1076
+ reader that turns `undefined` into `null`, or `"__proto__"` into
1077
+ `"__proto_"`, has invented data, and it has done so in the one code path
1078
+ whose entire purpose is preservation.
1079
+ 3. **Where a reader cannot observe it, the reader is not non-conformant, and
1080
+ nothing may depend on the outcome.** A rule that requires reporting a
1081
+ difference the reader was never shown is a rule no one can implement, and
1082
+ this document does not issue one.
1083
+
1084
+ Part 3 is not an escape hatch for the merely inconvenient. It applies only
1085
+ where the normalisation happens *below* the implementation — where the value
1086
+ reaching this specification's code is already the normalised one. Anything the
1087
+ reader can still see, part 2 governs.
1088
+
1089
+ **The last two rows are different from the rest**, and are the exception that
1090
+ proves the rule: this document already requires readers to accept them and
1091
+ writers not to produce them (see the two normative encoding choices above).
1092
+ They are listed here because a reader that *preserves* one — a byte string
1093
+ still wrapped in tag 64, sitting in a store — is preserving a shape its own
1094
+ writer rule forbids, and would then emit it.
1095
+
1096
+ **So the two encoding rules apply to stored values too.** When a store is
1097
+ written back, an integral float in it is written as an integer and a tag-64
1098
+ byte string as major type 2, at any depth. That is a change of encoding and not
1099
+ of value, which is exactly the line this document already draws elsewhere:
1100
+ "written back unchanged" binds the value, not its encoding. The alternative —
1101
+ verbatim bytes — cannot be implemented by a reader whose decoder already
1102
+ normalised them on the way in, so requiring it would make one implementation
1103
+ permanently non-conformant while the other did nothing useful with its
1104
+ compliance.
1105
+
1106
+ One value does change under this rule: negative zero, whose sign is lost when
1107
+ `-0.0` is written as `0`. It is called out here rather than carved out, because
1108
+ a trace has no field where negative zero means anything, and an exception would
1109
+ cost more than it buys.
1110
+
1111
+ **Duplicate keys.** A writer MUST NOT emit a map carrying the same key twice.
1112
+ Readers are not required to agree about which entry of a duplicate pair wins,
1113
+ and no file may depend on it. RFC 8949 already calls such a map invalid; this
1114
+ document adds that a conformant tool must not *emit* one, having read one.
1115
+
1116
+ Two places produce one without anybody meaning to. A store built by hand is an
1117
+ ordered list of pairs rather than a map, so nothing stops a caller putting the
1118
+ same key in twice. And a map **nested inside** a stored value can carry a
1119
+ duplicate that came from the file itself — which is the more likely of the two,
1120
+ since it needs no mistake at all, only a non-conformant peer. The rule reaches
1121
+ both: a tool that read one must not emit one, at any depth.
1122
+
1123
+ **This is the one rule in this document that loses data, and it does so
1124
+ knowingly.** Collapsing a duplicate pair to a single entry discards the other
1125
+ entry's value. The alternative is emitting a map RFC 8949 calls invalid, which
1126
+ every downstream reader would then resolve differently — so the choice is not
1127
+ between losing a value and keeping it, but between losing it once, here, and
1128
+ losing it unpredictably at each reader thereafter. The input was already
1129
+ invalid; no rewrite of it can be lossless. Where the loss matters, report the
1130
+ duplicate rather than passing it on silently.
1131
+
1132
+ **And the two implementations will not lose the same one.** The distinction
1133
+ worth keeping straight, because the table above makes it and this paragraph
1134
+ once contradicted it: `ciborium` keeps **both** entries and makes no choice at
1135
+ all, so the Rust *reader* is the thing that keeps the first, by the first-match
1136
+ lookup it uses for every key. `cbor-x` has kept only the last before its caller
1137
+ runs. So the same input file rewritten through each yields two files differing
1138
+ in **value**, not merely in encoding.
1139
+
1140
+ That is a worse divergence than anything the two encoding rules address, and it
1141
+ is stated here rather than left to be discovered because it cannot be fixed:
1142
+ making one of them normative would require the other to report a difference its
1143
+ decoder discarded before any of this document's code ran. Keeping the first
1144
+ entry is the better behaviour where the choice exists, and it is what a
1145
+ first-match reader does anyway — but it is a SHOULD only the implementation
1146
+ that can still see both entries can be held to.
1147
+
1148
+ That asymmetry is exactly why this is worth a rule. A reader handed both
1149
+ entries can *delete* one silently, which a reader handed one cannot; the
1150
+ capable implementation is the one at risk, and both reference implementations
1151
+ did lose a value here — the Rust one on its read path, for a key a field had
1152
+ already taken — until it was found by inspection rather than by any test.
1153
+
1154
+ What keeps this from mattering is the rule at the top of this paragraph: **no
1155
+ conformant writer produces a duplicate key**, so a file carrying one came from
1156
+ a non-conformant peer, and a trace of a non-conformant peer is exactly the
1157
+ thing [Event 6](#event-6-error) exists to record. Record it there; do not rely
1158
+ on the rewrite.
1159
+
469
1160
  ---
470
1161
 
471
1162
  # Appendix A: MoQT Carriage Convention (informative)
@@ -505,6 +1196,9 @@ Implementations MAY batch multiple events into one object for efficiency at high
505
1196
  - **Event 10 carries track identity and per-hop timestamps** — `"ns"`, `"tn"`, `"tdr"`, `"tus"`, `"tuo"`, `"tdo"` — making per-hop latency attribution possible from the event alone, plus explicit update semantics for events emitted before all four timestamps are known.
506
1197
  - **Event 10's trace ID is `"traceId"`, a 16-byte byte string.** An earlier revision of this proposal spelled it `"trace"` and left the type open; a text-string spelling invites per-implementation encodings, which defeats the cross-operator stitching the field exists for.
507
1198
  - **Unknown event types, perspectives and detail levels must be tolerated, not rejected.** Version 1 said readers "SHOULD skip unknown event types"; neither reference implementation did, which is why the purely additive parts of this revision would have broken them anyway.
1199
+ - **Unrecognised keys must survive a read-modify-write.** Version 1 said only that unknown keys must be ignored, which both implementations read as permission to drop them — so every optional key either revision adds was safe to read past and silently destroyed by any tool that rewrote the file.
1200
+ - **The key-preservation rule now says what it means for the header**, in a section of its own. "Header maps" was plural and neither implementation read it that way: both dropped every unknown header key outright, which is the same data loss as on an event and strictly larger, since it applies to every unknown key rather than only a wrong-typed one. The header has three maps with named keys — the header itself, `"segment"` and `"sampling"` — and each keeps its own store. See [Unrecognised keys in the header](#unrecognised-keys-in-the-header).
1201
+ - **Event 6 can carry the bytes behind the error** — `"sid"`, `"ek"`, a `"rawlen"` gated at `"headers+sizes"` and a `"raw"` gated at `"full"`, with a normative 4096-byte cap and a one-per-flow latch. Version 1's error event held a code and a sentence, so a recorder that wanted to keep the offending bytes had to record the violation as a control message to have a field to put them in. See [Event 6](#event-6-error).
508
1202
  - Event `"n"` and `"t"` clarified as **segment-local** in segmented traces.
509
1203
  - **Non-droppable event types rule** added to the Sampling section.
510
1204
  - **Trace ID Propagation section** defining the role of the `MOQTAP_TRACE_ID` extension parameter.