@cosyte/dicom 0.0.10 → 0.0.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -6,13 +6,391 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
6
6
 
7
7
  ### Fixed
8
8
 
9
+ - **🩺 Repeated de-identification was not a fixed point: `(0012,0063)` grew by the whole method
10
+ string on every pass, for any `deidentificationMethod` ending in a SPACE or a NUL**
11
+ (`DICOM-DEIDENT-NOT-A-FIXED-POINT`). **`INTRODUCED` by the entry below and never published** - it
12
+ was found by a fourth graded pass while this release was still held, so no consumer ever saw it.
13
+ Measured in memory over four passes: `"ACME Anonymizer v3 "` read **19 -> 38 -> 57 -> 76** bytes
14
+ and `"Pass A\Pass B "` read **14 -> 21 -> 28 -> 35**, against a flat **19** and **14** on the
15
+ commit before. Over a real `parse -> deidentify -> serializeDicom -> parse` round trip a 16-byte
16
+ method read **16 -> 32 -> 48 -> 64 -> 80 -> 96** over six cycles. Growth continued to the
17
+ 65,534-byte ceiling, where the guard **replaces the entire prior provenance chain** - the exact
18
+ loss the entry below exists to prevent, reached from a benign caller string.
19
+
20
+ One asymmetry caused it: the prior value was right-trimmed of `0x20`/`0x00` and the added one was
21
+ not, so a freshly supplied value never equalled its own prior copy - the library wrote the method,
22
+ the serializer's even-length pad folded the trailing byte in, the next parse trimmed it off, and
23
+ the next pass appended the method again. PS3.5 2026c **Table 6.2-1**, `LO` row: "A character string
24
+ that **may be padded with leading and/or trailing spaces**" - trailing spaces are padding, not
25
+ content, so the comparison cannot honour that on one side only.
26
+
27
+ **The trim is per VALUE at the comparison, and a graded pass is why.** A first remedy trimmed each
28
+ operand as a whole Value Field, which reaches only its last value, so a pad byte on an interior
29
+ value of a `1-n` method still regrew the attribute byte-identically: `"Pass A \Pass B"` beside a
30
+ prior `"Pass B "` read **14 -> 21 -> 28 -> 35 -> 42**, and the chain was still replaced at the
31
+ ceiling. Table 6.2-1 describes a **Value** and `LO` is `1-n`, so every value's trailing pad is
32
+ padding; **§6.4**'s "single padding character ... to the end of the Value Field (**to the last
33
+ Value**)" is about where the encoder writes its pad, which is why the value **written** is trimmed
34
+ once over the field. Leading spaces a caller wrote are still written through untouched. With both,
35
+ `deidentify` is a fixed point **from the first pass**. A `deidentificationMethod` that is padding
36
+ only records nothing.
37
+
38
+ - **🩺 A prior `(0012,0063)` value surviving into de-identified output is no longer silent**, via
39
+ the new Tier-2 code **`DICOM_DEIDENT_METHOD_PRIOR_RETAINED`** on `report.warnings`. Keeping it is
40
+ what PS3.15 E.1.1 requires and nothing about the retention changes; the silence was the defect. A
41
+ name a sender wrote into `(0012,0063)` reached output stamped `(0012,0062) Patient Identity
42
+ Removed = YES` with `report.warnings` empty **and** `report.retained` `[]` - an audit reading as a
43
+ scrub it had not performed, on an attribute Table E.1-1 does not list and no rule in the run
44
+ inspected. The code carries **no value, no length and no VR**: the retained text is the file's
45
+ own, the tag in the message is a constant of the code, and `position.byteOffset` locates the
46
+ element. It is deliberately **not** on `report.retained`, which lists the Annex E option sets
47
+ active for the run - a kept attribute is not an option set, and widening that type would break
48
+ every consumer switching over the nine names. Emitted by `deidentify()` only, so it never reaches
49
+ the parser's `{ strict: true }` escalation.
50
+
51
+ - **🩺 A second copy of a MODELED `(0002,xxxx)` element left the parsed object with no warning and
52
+ no residue** (`DICOM-FILE-META-DROPS-DUPLICATE`, raised by `#70`'s gate). **`PRE-EXISTING`**, on
53
+ every released version including the current published `0.0.10`. This is `#70`'s shape one group
54
+ over, and **`(0002,xxxx)` is the group that decides how every following byte is read**, which
55
+ makes it the more dangerous of the two. Disclosed now by the new Tier-2 code
56
+ **`DICOM_DUPLICATE_FILE_META_ELEMENT`**, raised at the moment the projection drops the copy.
57
+
58
+ **It is lossy by a different route from the Data Set case, and the difference is the whole
59
+ entry.** The File Meta group is collected into an **array**, so nothing is overwritten and
60
+ `DICOM_DUPLICATE_TAG_IN_DATA_SET` never fired here - `#70`'s own JSDoc said exactly that, and
61
+ reading it as an all-clear was the defect. The eight tags `parseFileMeta` projects into typed
62
+ `FileMeta` fields are answered by a **first-match** search and are **excluded** from
63
+ `FileMeta.extraElements`, the verbatim residue that gives the group its byte-exact round trip. A
64
+ second copy of one is therefore in neither.
65
+
66
+ **So the two codes resolve a repeat the opposite way round, deliberately, because the two readings
67
+ do: the FIRST copy wins in the File Meta group, the LAST read wins in a Data Set.** Neither
68
+ reading moves in this release, **no value is guessed for the copy that lost**, and no residue is
69
+ invented for it - inventing one would make the serializer write a group it should not. A repeated
70
+ `(0002,xxxx)` tag this library does **not** model stays silent, because every copy of one is
71
+ already kept verbatim in `extraElements` and nothing is dropped. **Two `PRE-EXISTING` bounds a graded pass named, neither closed here**: `serializeDicom` re-emits BOTH copies of such a non-modeled repeat, which is where this package's round-trip promise and its spec-clean promise disagree; and the disclosure covers the group **as the parser delimits it**, so a copy an intermediary appended past an honest `(0002,0000)` group length is never a File Meta element to this parser at all and is relocated into the main Data Set, silently, on this tree and every earlier one.
72
+
73
+ **Measured on the published tarball rather than inferred from a version number.**
74
+ `npm pack @cosyte/dicom@0.0.10` (the registry's current `latest`; there is no `0.0.9`), a file
75
+ carrying `(0002,0010)` twice with two **different** Transfer Syntax UIDs:
76
+ `fileMeta.transferSyntaxUID` reads the first, `fileMeta.extraElements` is `[]`, `ds.warnings` is
77
+ `[]`, and `{ strict: true }` does **not** throw. Silent on every channel, on the released package.
78
+ **And the stakes are not hypothetical**: the same dataset bytes with only the ORDER of those two
79
+ UIDs swapped parse to two different objects, one of which raises `INVALID_FILE_META` out of the
80
+ dataset parser, because a length field read in the wrong encoding declares 1,199,696 bytes.
81
+
82
+ **The message names no tag, and the bound is the factory signature** -
83
+ `duplicateFileMetaElement(position)` - as for `DICOM_DUPLICATE_TAG_IN_DATA_SET`,
84
+ `DICOM_NONZERO_RESERVED_BYTES`, `DICOM_ITEM_CROSSES_SEQUENCE_END` and
85
+ `DICOM_DEIDENT_UNDEFINED_VR_NOT_AUDITABLE`. **Unlike the Data Set code, `position.byteOffset` here
86
+ is unambiguously file-absolute** - `parseFileMeta` is called once per parse with the whole buffer
87
+ and the group is never nested, so none of `Element.byteOffset`'s frame ambiguity can reach it - and
88
+ it locates the copy that was **dropped**, not the survivor.
89
+
90
+ **The citation, and the one that is deliberately absent.** PS3.5 2026c section 7.1, read from the
91
+ SHA-pinned `vendor/nema/part05/` and occurring **exactly once** in that document: "The Data
92
+ Elements in a Data Set shall be ordered by increasing Data Element Tag Number and shall occur at
93
+ most once in a Data Set." **PS3.10, which governs the File Meta Information group, is NOT vendored
94
+ in this repo, so no PS3.10 sentence is cited here and no conformance verdict about a repeated
95
+ `(0002,xxxx)` is claimed.** The code's trigger is narrower and needs neither: it fires exactly when
96
+ a value the file carried does not reach the parsed object, and a `{ strict: true }` caller has
97
+ asked to be thrown at rather than handed a lenient reading.
98
+
99
+ - **🩺 `deidentify()` REPLACED `(0012,0063)` De-identification Method where PS3.15 says it is
100
+ "inserted in or added to" it** (same item, second half). **`PRE-EXISTING`**, measured on `0.0.10`:
101
+ a file recording `"ACME Anonymizer v3 Basic Profile"` came out of `deidentify()` recording only
102
+ this library's own method, with the earlier one gone and nothing saying so. What that destroyed is
103
+ the **provenance chain** the attribute exists to carry, on a file whose earlier pass may be the one
104
+ a recipient was relying on.
105
+
106
+ PS3.15 2026c section **E.1.1 "De-identifier"**, read from the SHA-pinned `vendor/nema/part15/` and
107
+ occurring **exactly once**: "a text string describing the method used shall be **inserted in or
108
+ added to** De-identification Method (0012,0063)." Replacing is neither verb. This release appends
109
+ its own text as a further value of the `1-n` attribute after a `\`, copying the prior bytes through
110
+ **verbatim** so a value encoded under a `(0008,0005)` repertoire survives byte for byte. A value
111
+ that already records this method is left alone rather than growing the attribute. **The comparison
112
+ is per VALUE on both sides**: the method string is itself a `1-n` value, and a graded pass refuted
113
+ the draft that compared the whole string against each prior value - a caller method carrying a `\`
114
+ never matched one, and every pass appended a whole further copy (29 -> 59 -> 89 -> 119 bytes over
115
+ four passes, against a flat 29 on base). **Per value was necessary and not sufficient, and the
116
+ fixed-point claim as first written did not hold** - see the trailing-pad entry below, which is what
117
+ makes it true from the first pass.
118
+
119
+ **The join is bounded, and the bound is not cosmetic.** `LO` is a short-form VR, so
120
+ `encodeDatasetElement` writes its Value Length with a 16-bit field. A `(0012,0063)` carrying a legal
121
+ 65,534-byte chain of `1-n` values - exactly the provenance chain this feature exists to build -
122
+ parses with no warnings, and an unbounded append produced a 65,611-byte value that `serializeDicom`
123
+ could not encode: a raw `RangeError` out of Node's `Buffer` internals, outside the documented
124
+ `DicomSerializeError` surface, taking the whole de-identified object down. When the join would
125
+ exceed the ceiling the prior value is **replaced** instead, which is what every released version did
126
+ on every file, and `report.warnings` carries the new `DICOM_DEIDENT_METHOD_NOT_ADDED`, so THAT
127
+ fallback is disclosed. Read the code as "the length ceiling was reached", never as "every fallback
128
+ is disclosed": the `PRE-EXISTING` non-`LO` replacement below is a second shape where `deidentify`
129
+ cannot add, and it is still silent. Truncating the sender's earlier records instead was refused: choosing
130
+ which to drop is a policy the standard does not state.
131
+
132
+ **`(0012,0062)` Patient Identity Removed is still REPLACED with `YES`, and the asymmetry is the
133
+ standard's own**: the sentence immediately above it, in the same list, says it "shall be **replaced
134
+ or added to** the Data Set with a value of YES". Different verbs, different attributes.
135
+
136
+ **The cost is disclosed rather than glossed, and a residual test asserts it.** `(0012,0063)` is
137
+ **not in Table E.1-1**, so the Basic Profile never acted on it and the incoming value reached the
138
+ insertion point untouched - the replacement was the only thing removing it, and removing it was an
139
+ action no profile asked for. A sender who wrote something identifying into `(0012,0063)` now sees
140
+ that text in de-identified output, which is the retained-by-omission posture every other unlisted
141
+ attribute already has. **Closing that direction is a product call about unlisted attributes, not a
142
+ fix to this insertion.** A `(0012,0063)` a file encoded under a VR other than `LO` is replaced
143
+ rather than appended to.
144
+
145
+ **The evidence, with its sha, because the base moves.** Re-measured at `e75fb38` after the last
146
+ test was added - and again after each conformance-gate remedy added more, which is the rule
147
+ and not a courtesy: **8 of the 12** tests in `test/integration/file-meta-duplicate.test.ts` and
148
+ **11 of the 19** in `test/deident/deident-method-add.test.ts` run red against that base, 19 of 31
149
+ in all. **Neither file can link against base `src/`** - none of
150
+ `WARNING_CODES.DICOM_DUPLICATE_FILE_META_ELEMENT`, `duplicateFileMetaElement`,
151
+ `WARNING_CODES.DICOM_DEIDENT_METHOD_NOT_ADDED` or `deidentMethodNotAdded` exists there - so both
152
+ figures are measured with those symbols substituted for their literals; unmodified they are 12 of
153
+ 12 and 19 of 19 by construction, which is a fact about linking rather than about behaviour. The
154
+ ones that stay green on base are the controls that make the rest non-vacuous. **A figure taken this way is substitution-sensitive and the third pass read 9 rather than 8**, the delta being the factory-signature row, which is red or green purely by how the absent factory is stood in for. Read it as a floor, and re-derive it rather than quoting it. Quote such a figure
155
+ only with the sha it was run on.
156
+
157
+ - **🩺 A Data Set that carries one tag twice DESTROYED the first element's value at parse time, and
158
+ said nothing** (`DICOM-TAG-COLLISION-DESTROYS-ELEMENT`). **`PRE-EXISTING`**, on every released
159
+ version including the current published `0.0.10`. A parsed Data Set is a `Map<Tag, Element>`, so
160
+ `Map.set` on a tag the map already holds **overwrites in place**: the earlier element's value is
161
+ gone from the object, and the survivor is indistinguishable from an element the sender wrote once.
162
+ No warning, no report entry, nothing a reader could query and nothing a round trip could reveal -
163
+ which is the same harm as a leak, in the other direction. It is now disclosed by the new Tier-2
164
+ code **`DICOM_DUPLICATE_TAG_IN_DATA_SET`**, raised at the moment of the replacement, in every Data
165
+ Set at every depth.
166
+
167
+ **Measured on the published tarball rather than inferred from a version number.**
168
+ `npm pack @cosyte/dicom@0.0.10` (the registry's current `latest`; `npm view` is the only source
169
+ for that, and there is no `0.0.9`), an Explicit VR LE file carrying `(0010,0020)` twice:
170
+ `ds.get("00100020")` reads the **second** value, `ds.warnings` is **empty**, and `{ strict: true }`
171
+ does **not** throw. Silent on every channel, on the released package.
172
+
173
+ **The remedy is the disclosure and nothing else.** The reading does not move: the last element read
174
+ still wins, exactly as in every released version, and **no value is invented for the one that was
175
+ replaced**. That is deliberate rather than modest. A file whose Item over-declares and a file whose
176
+ Sequence under-declares are byte-identical (pinned by `Buffer.equals` in
177
+ `test/integration/explicit-sq-item-bound.test.ts`), so a reader cannot choose the "right" element
178
+ from the bytes; what it can do is say that it had to choose.
179
+
180
+ **Citations, traced rather than stated.** PS3.5 2026c section 7.1 "Data Elements": "The Data
181
+ Elements in a Data Set shall be ordered by increasing Data Element Tag Number and shall occur at
182
+ most once in a Data Set." Section 7.5.1 "Item Encoding Rules", one level down: "Within the context
183
+ of each Item, these Data Elements shall be ordered by increasing Data Element Tag value and appear
184
+ only once." Both read from the SHA-pinned `vendor/nema/part05/`, each occurring exactly once in
185
+ that document. **So this code cannot fire on a conformant file**, which is what makes it safe to add
186
+ under the `{ strict: true }` escalation every Tier-2 code takes.
187
+
188
+ **The message names no tag, and that is specific to this code.** A sender writing the same tag twice
189
+ is the rare route; the ordinary one is a length field that lies, so the second header's four tag
190
+ bytes are read out of the middle of some element's value. `renderTag` shape-checks a tag and
191
+ therefore cannot refuse one, so the bound is the factory signature, as it is for
192
+ `DICOM_NONZERO_RESERVED_BYTES` and `DICOM_ITEM_CROSSES_SEQUENCE_END`. `position.byteOffset` is the
193
+ offset of the header that replaced, which is the surviving element's own `Element.byteOffset` -
194
+ so at the **root** the tag is read off the model rather than out of a message. **Inside a
195
+ defined-length Sequence Item it is not a lookup at all**: `Element.byteOffset` is item-relative
196
+ there, so the same number can name an untouched root element, and no parser warning populates
197
+ `position.contextPath` to tell them apart. Read the code as "an element in this object was
198
+ destroyed" unless you have established the frame.
199
+
200
+ **What this cost, measured on `scripts/measure-sq-bound-grid.ts` against `0ead071`, because a
201
+ disclosure that changes a `{ strict: true }` parse is a behaviour change.** Of 83,037 cells, **349
202
+ differ and every difference is confined to the warning channels and `{ strict: true }`**: 0 cells
203
+ differ in the element tree, the `DeidentifyReport`, the de-identified bytes, which marker values
204
+ survive, or the root `(0010,0020)`; 0 new lenient fatals; leaking cells 11 -> 11; conformant tiling
205
+ controls 7 -> 7. **345 cells now report a collision that was silent on base** (295 Implicit VR LE,
206
+ 25 Explicit VR LE, 25 Explicit VR BE), all of them in the grid's two hoist-collision families -
207
+ a fact about those fixtures, not a rate for real files. **9 cells that parsed under
208
+ `{ strict: true }` now throw** - element trees, reports, root identifiers and surviving markers
209
+ identical on both trees, taken from those per-field counters and **not** from the harness's
210
+ `...on an UNCHANGED lenient reading` line, which reads 0 by construction for any slice that adds a
211
+ code. A further **4** were already fatal there and now carry this code instead of
212
+ `INVALID_FILE_META`, because the escalation happens earlier in the parse. The shipped
213
+ `profiles.strict` preset is unchanged.
214
+
215
+ **Three more things a `{ strict: true }` caller should know, none of them fixed here.** (1) On the
216
+ one shape where the parse rolls a descent back - an Implicit VR LE defined-length `SQ` whose value
217
+ holds a duplicate and then a non-Item tag - `{ strict: true }` threw `DICOM_SQ_NOT_DESCENDED`
218
+ before and throws this code now, on a file where **nothing was destroyed**: both values survive in
219
+ `Element.rawBytes`. Both refuse the file, so no object is lost, but the older code was the more
220
+ accurate diagnosis. (2) The same shape streams the code to `onWarning` while `ds.warnings` never
221
+ receives it, which is the pre-existing D-03 ordering reached by one more code. (3) **"Names no tag"
222
+ is about the message.** Strict mode throws a `DicomParseError` whose `snippet` is 16 raw source
223
+ bytes at that offset, rendered as hex - measured on a plain duplicate:
224
+ `10 00 20 00 4c 4f 0e 00 53 4d 49 54 48 53 4f 4e`, the withheld tag plus eight bytes of the value.
225
+ That is the package-wide D-10 design. The same file does not throw at all on `0ead071` (this code is
226
+ what refuses it), but the same file with its even-length padding removed produces the identical 16
227
+ bytes there through `DICOM_ODD_LENGTH_VALUE_PADDED`, one length byte apart - which is why the
228
+ guarantee is worded about the message rather than about the channel.
229
+
230
+ **What the grid is NOT evidence for here, stated because it was read that way once.** It reaches
231
+ the collision only through a length lie, and it swept nothing that carries a tag twice honestly.
232
+ The plain duplicate, the duplicate inside an Item, the collision that lands on a sequence's own tag
233
+ and takes the whole `SQ` and everything nested in it out of the object, the frame the byte offset
234
+ is in, and the `{ strict: true }` behaviour are pinned in
235
+ `test/integration/tag-collision.test.ts` or nowhere: **14 of its 15 tests run red against
236
+ `origin/main` at `0ead071`** - re-measured after each gate pass added tests, and written with its sha
237
+ because the figure moves with every test added - the fifteenth being the no-duplicate
238
+ control, which is green there by design.
239
+
240
+ **Still open, and not touched here:** the private-`SQ` carve-out (`DICOM-PRIVATE-SQ-CARVE-OUT`),
241
+ and the structural relocation itself under `DICOM-EXPLICIT-VR-UNBOUNDED-ITEM-READ` - an
242
+ over-declaring Item still moves the element that follows the sequence. This slice makes the loss
243
+ observable; it does not make the file readable.
244
+
245
+ - **🩺 The mirror of the above: a private value ejected OUT of a Sequence Item was retained on a
246
+ reservation it borrowed from the Data Set it landed in, in output stamped
247
+ `PatientIdentityRemoved = YES` with `report.removedPrivateTags` reading `[]`**
248
+ (`DICOM-ITEM-EJECT-ROUTE`). **`PRE-EXISTING`**, found by `#66`'s pass-2 `conformance-refuter`,
249
+ pinned by `#66` as residual tests that asserted the leaking behaviour, and re-measured on
250
+ `300af87` before anything changed. The **private-`SQ` carve-out** is still open (below). Read this
251
+ as "the eject direction is refused", never as "the class is closed".
252
+
253
+ **Measured on the published tarball, not inferred from a version number.**
254
+ `npm pack @cosyte/dicom@0.0.10` and the Explicit VR LE eject fixture:
255
+ `report.removedPrivateTags` is `[]`, `SECRET-PRIVATE-PHI` is in the serialized output,
256
+ `(0012,0062)` is `YES`, **`ds.warnings` is empty and `{ strict: true }` does not throw**. So on the
257
+ released package this route is silent on every channel, exactly as the item filed it.
258
+ **`0.0.9` is NOT on the registry** - `package.json` carried it, the publish never happened
259
+ (npm `E404`), and the current published version is `0.0.10`. Earlier entries in this file that say
260
+ "live on the published `0.0.9`" name a version that does not exist.
261
+
262
+ **On `main` it is no longer silent, and that difference is `#51` being unreleased.** With
263
+ `DICOM_ITEM_CROSSES_SEQUENCE_END` present, the Explicit VR shapes announce themselves on both
264
+ channels and throw under `{ strict: true }`; the Implicit VR LE shape raises the older
265
+ `DICOM_SQ_NOT_DESCENDED` and throws too. The leak was live beside those warnings, which is the
266
+ point: a warning about structure is not an all-clear about what was retained.
267
+
268
+ **The shape.** An `(FFFE,E000)` Item that declares **fewer** bytes than its content occupies ends
269
+ early, so the elements the sender encoded as Item content are read as elements of the **enclosing**
270
+ Data Set. A Private Creator that lands there reserves a `(gggg,00EE)` block for elements the sender
271
+ never put beside it, and the next private element is kept verbatim on it. **It is not
272
+ root-specific**: an inner sequence ejecting a creator into the still-usable Item that encloses it
273
+ reproduces the same leak one level down, and a root-scoped remedy would read green on it.
274
+
275
+ **Two predicates, because the parser records the same contradiction two different ways, and this is
276
+ why it is its own slice rather than a widening of the absorb rule.** Under **Explicit VR** the item
277
+ stream is bounded against the buffer, so it is read past the sequence's declared end and the span
278
+ shows up as `Element.rawBytes.length` exceeding `Element.length`. Under **Implicit VR LE** that path
279
+ slices the item stream to the declared Value Length, so **nothing over-runs at all**
280
+ (`rawBytes.length === length`); the item does not fit the slice, the descent is refused, and the
281
+ element carries `items === undefined` with `DICOM_SQ_NOT_DESCENDED`. Both are facts the parser had
282
+ already recorded: no re-parse, no scan, no cost that follows an attacker-chosen length. **The second
283
+ predicate is broader than the ejection it is here for, deliberately and in the fail-safe direction**
284
+ - it says the parser could not walk the sequence, which another unwalkable item stream also reaches.
285
+
286
+ **The remedy is positional, inside every Data Set, and it removes rather than downgrading the
287
+ stamp.** `settledBound` finds where a Data Set stops accounting for its own membership: the first
288
+ sequence whose own contents contradict its declared extent. Both the reservation map and the
289
+ retention decision are taken from the settled run, so a creator ejected into a Data Set reserves
290
+ nothing there and a private element read after that point is removed and named in
291
+ `report.removedPrivateTags`. An element read **before** the offending sequence cannot have come out
292
+ of it and is untouched. `processElements` derives this at every depth. **No parser file is touched
293
+ and no reading changes.**
294
+
295
+ **🛑 TWO BOUNDS, NOT ONE, BECAUSE A DATA SET IS A `Map<Tag, Element>`.** When the ejected element
296
+ carries a tag the Data Set **already holds**, `Map.set` overwrites in place and the newcomer
297
+ inherits the earlier element's **position**, ahead of the sequence it came out of. An index cut
298
+ alone reads it as settled and retains it: measured on a root holding a genuine
299
+ `(0009,0010)` + `(0009,1001)` reservation ahead of a sequence whose item ejects a second
300
+ `(0009,1001)`, which lands at index 2 with `byteOffset` 274 while the sequence sits at index 3 with
301
+ `byteOffset` 238. `Element.byteOffset` is the position the parser counted and the overwrite cannot
302
+ move it, so it is checked beside the index; the two are conjunctive because a hand-built
303
+ `Dataset` may carry no meaningful offsets at all, and there the index bound is what still
304
+ bites. **The grid is blind to this: the index-only and two-bound remedies differ on 0 of 83,037
305
+ cells**, because no `priv|` fixture collides tags. **3 tests, counted over the full suite, are the
306
+ whole pin**: the two collision rows and the creator-flip. Both offsets above are measured on the fixture pinned in `test/integration/deident-private-reservation.test.ts`, whose File Meta is the minimum **this parser** requires rather than PS3.10's - a fixture with more File Meta shifts every offset, and a pass-2 grade read 292 off one that did.
307
+
308
+ **🩺 AND THE COLLISION DESTROYS THE ROOT'S OWN VALUE ON THE WAY IN, WHICH THIS SLICE DOES NOT FIX.**
309
+ The overwrite replaces the reservation's genuine root element with the Item's, silently, at parse
310
+ time, with no warning naming it and no report entry - the `Map<Tag, Element>` substitution already
311
+ recorded for `(0010,0020)`, now reached on the private-retention path. `PRE-EXISTING`, identical on
312
+ both trees, **its own item**, and asserted in the tests so it cannot be mistaken for something this
313
+ remedy handled.
314
+
315
+ **The spec does the work, and both sentences were re-located in the pins rather than carried
316
+ forward.** PS3.5 2026c §7.8.1, one occurrence in `vendor/nema/part05/4dfd7b8c…`: "Items within a
317
+ sequence are self contained Data Sets ... The scope of the reservation is just within the Item.
318
+ Items do not inherit the Private Data Element reservations made by Private Creator Data Elements in
319
+ the Data Set in which the Item is nested." So the reservation's scope **is** the Item's boundary,
320
+ and a file that contradicts itself about where that boundary falls establishes no knowledge of which
321
+ Data Set an element is in. PS3.15 2026c §E.3.10, one occurrence in
322
+ `vendor/nema/part15/77d60b85…`: "Private Attributes that are known by the de-identifier to be safe
323
+ from identity leakage shall be retained, together with the Private Creator IDs that are required to
324
+ fully define the retained Private Attributes; all other Private Attributes shall be removed **or
325
+ processed in the element-specific manner recommended by Deidentification Action (0008,0307), if
326
+ present within Private Data Element Characteristics Sequence (0008,0300)**" - a two-branch clause,
327
+ quoted whole; this library does not implement `(0008,0307)`, so removal is the branch available to
328
+ it, and **"known"** is the load-bearing word.
329
+
330
+ **The price is measured, not asserted.** `scripts/measure-sq-bound-grid.ts` over **83,037** cells
331
+ against `300af87`: `priv: kept at ROOT, file CONTRADICTS` **78 -> 0**, of which the eject leaks
332
+ (`...whose honest control does NOT keep it`) are **22 -> 0** and the remaining **56 are the cost** -
333
+ root retentions on self-contradicting files whose honest control does keep the value. **That cost
334
+ has its own column and it is named rather than folded in: `de-identified OUTPUT lost a marker
335
+ (cost)` reads 78**, which is the de-identify-boundary column; the `LOST` and `GAINED a marker value`
336
+ counters beside it are **parse**-tree columns and both read 0, and quoting those two without this
337
+ one is exactly the confusion this repo has paid for before. Retention on files that do **not**
338
+ contradict themselves is unchanged (**9 -> 9** at the root, **6 -> 6** inside an Item),
339
+ `no-creator` rows stay **0 -> 0**, `LEAKING a source value` is **11 -> 11** (the binary-carrier
340
+ residual, untouched), `conformant tiling control emptied` **7 -> 7**, and **0 cells differ in any
341
+ PARSE respect, 0 cells whose READING differs**, 0 new lenient fatals, 0 new strict fatals, 0 wrong
342
+ root `(0010,0020)`, 0 reports losing an attribute. **118 cells changed, every one of them in the
343
+ `priv|` family: 74 Implicit VR LE and 44 Explicit VR.** (`structural` also reads 118 by construction
344
+ - it counts any record difference, including the de-identify columns, so it is not a reading claim.)
345
+
346
+ **🛑 THE HARNESS COULD NOT REPORT THAT SPLIT AND NOW CAN.** `--diff` classified a cell's transfer
347
+ syntax by testing whether its key _starts with_ the syntax, which is true only of the sequence
348
+ sweep; the `carrier|`, `legit|` and `priv|` families put their own prefix in that position, so
349
+ **not one of their rows could ever be counted as Implicit VR LE**. This slice's 74 Implicit VR LE
350
+ cells were reported as `Implicit VR LE 0`. `transferSyntaxOf` fixes it, which also moves
351
+ `onWarning != ds.warnings, Explicit VR` from 43 to **0** - those 43 were Implicit VR LE rows in
352
+ other families, i.e. the pre-existing D-03 ordering residual, misattributed. **Any syntax split
353
+ quoted off that line for a slice touching those three families predates this fix; re-run the diff
354
+ rather than carrying the figure forward.**
355
+
356
+ **The whole-Data-Set variant was built and measured rather than argued about, and the grid cannot
357
+ tell it apart.** Refusing every private element in a Data Set that _contains_ a disputed sequence
358
+ produces a **byte-identical** grid diff (0 of 83,037 cells differ from what shipped), because every
359
+ `priv|` fixture writes its private block after the sequence. **It differs on exactly 5 tests,
360
+ counted over the full suite rather than one file**, and they are why it is not taken: it reds "a
361
+ root reservation the sender wrote at the root survives an over-running sequence" and both collision
362
+ tests, all three of which are genuine root reservations the sender wrote ahead of the disputed
363
+ sequence, and it also closes **both** private-`SQ` carve-out residuals, which belong to a different
364
+ item - the second of them in `test/integration/deident-unauditable-sequence.test.ts`, which a
365
+ per-file enumeration misses. **The tests are the pin for the positional cut, not the grid.**
366
+
367
+ **What is NOT closed, and is not asserted away.** The **private-`SQ` carve-out**: `keepsPrivate`
368
+ decides before `descendSequence`, so a private `SQ` inside the settled run that the profile vouches
369
+ for is kept **verbatim** and nothing inside it is examined. `PRE-EXISTING`, still pinned as a
370
+ residual test that asserts the leaking behaviour. And the `Map<Tag, Element>` substitution above.
371
+
372
+ **The base measurement, taken fresh rather than carried forward:** **8 of the 37** tests in
373
+ `test/integration/deident-private-reservation.test.ts` run red against `origin/main`'s `src/` at
374
+ `300af87`, and they are exactly the eight eject tests. The remaining private-`SQ` residual is
375
+ **green** on that base, by design. **The "10 of the 31" figure and the "both residual tests are red
376
+ against base" correction beside it are both retired**: the first was measured against a pre-`#66`
377
+ tree, and the second is false against today's `main`, because `DICOM_ITEM_CROSSES_SEQUENCE_END` has
378
+ been on `main` since `79e9f34`.
379
+
380
+ **No new public surface, deliberately** - no Tier-2 code, no report field, no snapshot change, the
381
+ same choice `#66` made. `report.removedPrivateTags` already names every removed tag and is the
382
+ channel this defect read `[]` on.
383
+
9
384
  - **🩺 A private value crossed the PS3.5 §7.8.1 reservation boundary and was retained under
10
385
  `RetainSafePrivate`, in output stamped `PatientIdentityRemoved = YES`.** The **absorb** direction
11
- is closed; the **eject** direction and the private-`SQ` carve-out are measured, pinned and **not**
12
- closed (see below)
13
- (`DICOM-PRIVATE-CREATOR-RESERVATION-LEAK`). **`PRE-EXISTING` and live on the published `0.0.9`**;
14
- found by `#51`'s pass-6 `conformance-refuter` and measured **identical on `origin/main`**, so it
15
- was never introduced by that branch.
386
+ is closed; the **eject** direction and the private-`SQ` carve-out were measured, pinned and **not**
387
+ closed by this entry. (The **eject** direction has since been closed by `DICOM-ITEM-EJECT-ROUTE`,
388
+ the entry above; the private-`SQ` carve-out is still open.)
389
+ (`DICOM-PRIVATE-CREATOR-RESERVATION-LEAK`). **`PRE-EXISTING`**; found by `#51`'s pass-6
390
+ `conformance-refuter` and measured **identical on `origin/main`**, so it was never introduced by
391
+ that branch. (This entry said "live on the published `0.0.9`". **`0.0.9` was never published** -
392
+ `package.json` carried it and the publish never happened, npm `E404`. The leak is measured on the
393
+ published `0.0.10` tarball.)
16
394
 
17
395
  **The shape.** An `(FFFE,E000)` Item that declares more bytes than its enclosing `SQ` element's
18
396
  Value Length allows reads past the end of its sequence and absorbs the element that followed it.
@@ -86,8 +464,14 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
86
464
  a block for elements the sender never gave it. **It is NOT root-specific and an earlier draft of
87
465
  this entry said it was** - pass 2 reproduced it one level down, where an inner sequence ejects a
88
466
  creator into the still-usable enclosing Item. The item filed for this must be scoped to every
89
- still-usable Data Set or it will be built to the wrong bar. Silent on every channel
90
- and under `{ strict: true }`, Explicit VR LE and BE. **22 grid cells.** The widening was **built
467
+ still-usable Data Set or it will be built to the wrong bar. **It was silent on every channel and
468
+ under `{ strict: true }` when this shipped, and `DICOM-EXPLICIT-VR-UNBOUNDED-ITEM-READ` changed
469
+ that on the Explicit VR shapes and nowhere else, so do not carry the old sentence forward.**
470
+ Re-measured on this residual's own fixture: `DICOM_ITEM_CROSSES_SEQUENCE_END` on both channels and
471
+ a throw under `{ strict: true }`, beside an entirely unchanged leak (`removedPrivateTags: []`, the
472
+ value in the serialized output, `(0012,0062) = YES`). That warning is now the only signal on a file
473
+ whose de-identification audit is false, and it is never an all-clear. The 20 Implicit VR LE cells
474
+ stay silent: that path slices the item stream, so no over-run is recorded. **22 grid cells.** The widening was **built
91
475
  and measured**: narrowing the flag whenever a Data Set _contains_ an over-running sequence costs
92
476
  **24** root retentions and closes **2** of the 22, because the other 20 are Implicit VR LE where
93
477
  the sequence records no over-run at all. A different mechanism, and its own item.
@@ -327,6 +711,134 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
327
711
 
328
712
  ### Security
329
713
 
714
+ - **A Sequence Item under an Explicit VR transfer syntax can read past the end its own sequence
715
+ declared and swallow the element that followed the sequence, and did so silently. It now says so**
716
+ (`DICOM-EXPLICIT-VR-UNBOUNDED-ITEM-READ`, pre-existing and live on the published `0.0.10`, on both
717
+ Explicit VR Little Endian and Explicit VR Big Endian). New Tier-2 code
718
+ `DICOM_ITEM_CROSSES_SEQUENCE_END` (`WARNING_CODES` is **29, was 28**; the locked snapshot is the
719
+ pin and was updated deliberately), promoted to a throw under the `{ strict: true }` parse option.
720
+ **No reading changes, on any file.** That is the fix's whole shape, and the next paragraph is why.
721
+
722
+ **THE FACT THAT DECIDED THIS, AND IT IS PROVEN BY A TEST RATHER THAN ARGUED.** A file whose
723
+ **item over-declares** and a file whose **sequence under-declares** are **the same bytes**.
724
+ `test/integration/explicit-sq-item-bound.test.ts` builds both from two contradictory intentions and
725
+ asserts `Buffer.equals`. So "which of the two length fields is the lie?" has no answer on the wire,
726
+ and any bound that prefers section 7.5.2's extent for the first file imposes it on the second one
727
+ too, because there is no second file. **Five graded attempts at such a bound were refused on
728
+ `#51`.** This slice adds no bound, and that one fact is the entire reason.
729
+
730
+ **🛑 THE FAIL-SAFE-DIRECTION ARGUMENT IS DELETED, NOT REWORDED.** Five artifacts on this branch
731
+ said that following the Item's length field is the safe half of the ambiguity, because a Private
732
+ Creator swallowed **into** an item leaves the enclosing block unclaimed and an unclaimed block is
733
+ removed. **That is false, and it was the sixth refusal.** Which direction leaks is a property of
734
+ **where the sender put the Private Creator**, not of which length field a reader follows: with the
735
+ creator as genuine Item content, the absorb direction leaks too
736
+ (`DICOM-PRIVATE-CREATOR-RESERVATION-LEAK`, measured on `164eb39`, closed at the de-identify
737
+ boundary by `#66` and never in the parser). The eject direction is still open. Neither reading is
738
+ safe by construction, which is precisely why this code reports rather than decides. Per this
739
+ repo's own rule, a claim refuted twice is deleted rather than given a third wording.
740
+
741
+ Measured, on `0.0.10`: `(0008,1115)` holding one item that over-declares its length by 18 bytes,
742
+ followed by a root `(0010,0020)` Patient ID that is 18 bytes on the wire (an 8-byte Explicit VR
743
+ short-form header plus a 9-character value padded to 10). The Patient ID is **absent from the root**
744
+ and present instead as an attribute of the item; the `DeidentifyReport` names it with a
745
+ `contextPath` pointing at a sequence item it was never in. It is swallowed once and relocated, not
746
+ read twice: the parser resumes where the descent actually ended. The parse was silent about all of
747
+ it, including under `{ strict: true }`, and that silence is what is fixed. The mis-structure itself
748
+ is **not repaired here** and is pinned as a residual by a test.
749
+
750
+ **▶ AND IT IS THE ONLY SIGNAL ON A FILE WHOSE DE-IDENTIFICATION AUDIT IS FALSE.** `#66` recorded
751
+ the eject direction of `DICOM-PRIVATE-CREATOR-RESERVATION-LEAK` as silent on every channel. On the
752
+ **Explicit VR** shapes that is no longer true: re-measured on that residual's own fixture, this
753
+ code fires on both channels and throws under `{ strict: true }`, beside an unchanged leak
754
+ (`report.removedPrivateTags` `[]`, the private value in the serialized output, the object stamped
755
+ `(0012,0062) Patient Identity Removed = YES`). The leak is not closed here and the warning is not
756
+ an all-clear - **the troubleshooting row says so explicitly**, because the earlier draft of that
757
+ row told an operator "nothing is retained that would not be", which is false on exactly this file.
758
+ Implicit VR LE stays silent: that path slices the item stream, so no over-run is recorded.
759
+
760
+ PS3.5 2026c section **7.5.2** "Delimitation of The Sequence of Items" makes the `SQ` element's own
761
+ Value Length the exact extent of the item stream: "This length shall include the total length
762
+ resulting from the sequence of zero or more items conveyed by this Data Element." Section **7.5.1**
763
+ "Item Encoding Rules" governs each `(FFFE,E000)` Item's own length field. Both traced to the
764
+ SHA-pinned `vendor/nema/part05/`, each sentence unique in the document. **Neither clause says what
765
+ a decoder does when the two disagree**, so no reading is derived from them; where they disagree
766
+ this reader follows 7.5.1, which is what every released version does.
767
+
768
+ **Where the disclosure fires, exactly.** A defined-length `(FFFE,E000)` item, inside a
769
+ defined-length `SQ`, whose declared end is **not** the end of the buffer it is being read in -
770
+ which is what says the sequence sits inside a larger Data Set whose bytes are there to be taken.
771
+ A sequence handed a slice cut at its declared end (Implicit VR LE's `tryParseDefinedLengthSQ`,
772
+ CP-246's `tryParseUnAsSQ`), an undefined-length sequence, an undefined-length item, and a sequence
773
+ that ends its own buffer all have nothing to reach into and stay silent. Each is a test.
774
+
775
+ **🩺 The Item's declared length is WITHHELD from the message, and the bound is the factory
776
+ signature.** A diagnostic about a length field that lies is itself a PHI surface: the condition
777
+ that raises this code is exactly "these length fields are not what they claim to be", so the Item's
778
+ 32-bit Value Length can be four bytes of somebody's value. Measured, an item header fabricated over
779
+ the payload `"SMITHSON"` rendered it as `1414090067`, `"SMIT"` in wire order, reversible with one
780
+ `readUInt32LE` - and it is emitted **above** the truncation guard, so the message reaches
781
+ `onWarning` on a file the parse then refuses. `itemCrossesSequenceEnd` takes no parameter for it,
782
+ the same remedy `#64` applied to `DICOM_NONZERO_RESERVED_BYTES` and `#55` to
783
+ `DICOM_DEIDENT_UNDEFINED_VR_NOT_AUDITABLE`; `position.byteOffset` locates the item. The bytes that
784
+ remained inside the sequence are still reported, and that asymmetry is structural rather than a
785
+ judgement call: the emit site's own `endLimit < buffer.length` conjunct bounds that count by the
786
+ buffer. Measured against the identical attack - fabricating the **`SQ`**'s length field over the
787
+ same name puts `endLimit` past the buffer, so the code does not fire at all. Both pinned with a
788
+ name-bearing payload and a mutation control; the shipped message said "no bytes off the wire", then
789
+ carried the length itself, and both are gone.
790
+
791
+ **The measurement, on `scripts/measure-sq-bound-grid.ts` against `2f0abd9`, 83,037 cells.** Both
792
+ sequence length fields and an element's own swept independently, across ten item shapes, both
793
+ Explicit VR syntaxes, an Implicit VR LE control, both `strict` postures, and `#66`'s `priv|`
794
+ private-reservation family:
795
+
796
+ | | |
797
+ | ---------------------------------------------------------- | ------------------------- |
798
+ | cells whose **reading** differs | **0** |
799
+ | cells newly emitting `DICOM_ITEM_CROSSES_SEQUENCE_END` | 616 |
800
+ | ...cells that LOSE it | **0** |
801
+ | cells that newly fail under `{ strict: true }` (the price) | **576** |
802
+ | new lenient fatals | **0** |
803
+ | values lost, values gained, wrong root `(0010,0020)` | **0 / 0 / 0** |
804
+ | PHI regressions | **0** |
805
+ | reports that lose an attribute | **0** |
806
+ | Implicit VR LE cells changed | **0** |
807
+ | every `priv\|` column from `#66` | unchanged |
808
+ | cells leaking a source value, base -> now | 11 -> 11 (`PRE-EXISTING`) |
809
+
810
+ The other **16,396** of the 17,012 differing cells are **strict-fatal on both trees** and differ
811
+ only in the _class_ of the `{ strict: true }` throw, because the new Tier-2 code escalates before
812
+ the Tier-3 fatal those files already had. All 576 new strict fatals carry the new code, so none is
813
+ collateral. **Quote the reading count and the strict count together or neither** - a new warning is
814
+ a real behaviour change, and 576 files that parsed under `{ strict: true }` now do not. Every one
815
+ is a file whose two normative length fields contradict each other, which is what the strict posture
816
+ exists to refuse; the lenient reading of all 576 is byte-identical to before. `cells whose READING
817
+ differs` is the counter that expresses this and it already exists on `main` (`#66` added it); the
818
+ earlier draft of this entry claimed to add it.
819
+
820
+ **Disclosed and NOT fixed, deliberately:**
821
+ - **The mis-structure.** An over-declaring item still relocates the element that follows the
822
+ sequence, and `deidentify()` still reports it under a `contextPath` naming an item it was never
823
+ in. Pinned by a test so the claim cannot drift from the code.
824
+ - **`ds.warnings` is uncapped, and "at most one per sequence" is not an amplification bound.** The
825
+ shape holds, but a file may carry as many sequences as it can encode. Pinned by a test that
826
+ asserts the growth. This is `#48`'s pre-existing, package-wide posture for parser warnings.
827
+ - **`profiles.strict` does not escalate this code.** The `{ strict: true }` option does. Adding a
828
+ code to a shipped preset moves every `profiles.strict` consumer's parse and is its own measured
829
+ change. Pinned by a test.
830
+ - **`position.byteOffset` is frame-dependent**, like `Element.byteOffset`: file-absolute for a
831
+ root-level sequence, slice-relative inside an enclosing item. Measured and pinned.
832
+ - **The undefined-length item with no `(FFFE,E00D)`** still runs to the end of the buffer. There is
833
+ no declared item length, so there is no disagreement to disclose.
834
+ - **`Element.byteOffset` inside a sequence item disagrees with itself between the two item forms,
835
+ and always has.** Measured identically on this branch and on `origin/main`: on a 210-byte file
836
+ with the `SQ` at 172, an element inside a **defined-length** item reads `0` (the item slice is its
837
+ own frame) and the same element inside an **undefined-length** item reads `192` (file-absolute,
838
+ because `parseSequence` hands that branch the outer buffer). `PRE-EXISTING`.
839
+ - **11 grid cells still leak a source value** - the over-declare swallow into an `OB`/`OW`/`US`/`UN`
840
+ leaf carrier, `PRE-EXISTING`, identical on both trees, its own item.
841
+
330
842
  - **A symbolic link under `test/fixtures/` pointing at a PHI-bearing file scanned CLEAN on BOTH of
331
843
  `scripts/phi-scan.ts`'s enumerating routes, so the commit gate passed it twice over**
332
844
  (`PHI-SCAN-SYMLINK-BLIND-ON-BOTH-ROUTES`; pre-existing). Reproduced on `0f898be` with a synthetic