@cosyte/dicom 0.0.10 → 0.0.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +578 -7
- package/README.md +9 -3
- package/dist/index.cjs +212 -35
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +107 -5
- package/dist/index.d.ts +107 -5
- package/dist/index.mjs +212 -35
- package/dist/index.mjs.map +1 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -6,13 +6,450 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
|
|
|
6
6
|
|
|
7
7
|
### Fixed
|
|
8
8
|
|
|
9
|
+
- **🩺 The `(0012,0063)` De-identification Method this library writes for itself exceeded the 64-character
|
|
10
|
+
maximum PS3.5 gives an `LO` Value, on every file it ever de-identified**
|
|
11
|
+
(`DICOM-LO-LENGTH-AND-SILENT-REPLACE`). `PRE-EXISTING`; measured through the built package on
|
|
12
|
+
`da1f209`: **76** characters with no options, **130** with `RetainUIDs + RetainSafePrivate +
|
|
13
|
+
RetainDeviceIdentity`, **272** with all nine, and **all 512 option subsets over the maximum**. PS3.5
|
|
14
|
+
2026c **Table 6.2-1**, `LO` row, is "64 chars maximum" and that row describes a **Value**;
|
|
15
|
+
`(0012,0063)` is `1-n`, so the bound falls per value. Reading it as a bound on the Value Field is the
|
|
16
|
+
same misreading that left the fixed-point hole above (where §6.4, a clause about the **encoder**, was
|
|
17
|
+
read as a bound on a **comparison**), and it matters because the attribute concerned is the one a
|
|
18
|
+
receiver reads to decide whether an object was de-identified at all.
|
|
19
|
+
|
|
20
|
+
The default method text is now **multi-valued**: one Value naming the Profile
|
|
21
|
+
(`@cosyte/dicom Basic Application Level Confidentiality Profile`, **61** characters) and one Value per
|
|
22
|
+
active Annex E Option, joined with `\`. No option name exceeds **28** characters, so no subset can
|
|
23
|
+
breach the maximum - proved by sweeping **all 512 subsets / 2,816 value cells** rather than by
|
|
24
|
+
argument: **0 over the maximum**, against **512 of 512** on the commit before. The Value Field is
|
|
25
|
+
still 111 and 247 bytes for those two option sets, which is legal and is asserted alongside the
|
|
26
|
+
per-value figure so a remedy that merely shortened the text could not pass. Options are emitted in
|
|
27
|
+
`DEIDENTIFY_OPTIONS` order rather than the caller's, so the same option set always writes the same
|
|
28
|
+
bytes. The fixed point is unaffected and re-measured over six real wire round trips: **62** bytes flat
|
|
29
|
+
by default and **248** flat with all nine.
|
|
30
|
+
|
|
31
|
+
**Your own `deidentificationMethod` is not bounded for you** and neither is a prior value the source
|
|
32
|
+
file wrote: splitting or truncating either would invent a de-identification record nobody made. Both
|
|
33
|
+
are pinned as residual tests rather than left to be rediscovered. Split your string on `\` yourself if
|
|
34
|
+
a strict receiver is in your path.
|
|
35
|
+
|
|
36
|
+
- **🩺 A `(0012,0063)` a file encoded under a VR other than `LO` was replaced silently**, and now raises
|
|
37
|
+
the new Tier-2 code **`DICOM_DEIDENT_METHOD_NOT_LO`** on `report.warnings`. The replacement itself is
|
|
38
|
+
deliberate and unchanged: the provenance chain is built by concatenating `LO` values with the `5CH`
|
|
39
|
+
delimiter, which is a text operation and not defined over the arbitrary octets an `OB` or `UN` value
|
|
40
|
+
holds, so guessing an encoding for them was refused. What was wrong was that the sender's earlier
|
|
41
|
+
de-identification record left the object with `report.warnings` **empty**, under
|
|
42
|
+
`(0012,0062) = YES`. It is a separate code from `DICOM_DEIDENT_METHOD_NOT_ADDED` because the causes
|
|
43
|
+
are unrelated - the chain outgrew the VR, or the bytes were never in that VR at all - and a consumer
|
|
44
|
+
that has to tell them apart now can. A prior value that is empty or padding only raises neither,
|
|
45
|
+
because nothing was lost. The code carries **no value, no length and no VR**: two bytes read out of a
|
|
46
|
+
fabricated header are document content, which is how four letters of a surname once reached
|
|
47
|
+
`DICOM_NONZERO_RESERVED_BYTES`.
|
|
48
|
+
|
|
49
|
+
- **🩺 `{ strict: true }` renders source bytes the warning it replaces deliberately withholds.**
|
|
50
|
+
`PRE-EXISTING`, **disclosed rather than changed**: the escalation raises a `DicomParseError` whose
|
|
51
|
+
`snippet` is 16 raw source bytes read from the file at the warning's own `byteOffset` (D-10), while
|
|
52
|
+
`err.message` stays the frozen registry string. Measured on
|
|
53
|
+
`DICOM_DUPLICATE_FILE_META_ELEMENT`, whose group is never nested and whose offset is therefore
|
|
54
|
+
unambiguously the dropped copy's header: a `(0002,0016)` Source AE Title of `AE-SMITHSON` comes
|
|
55
|
+
back as `02 00 16 00 41 45 0c 00 41 45 2d 53 4d 49 54 48`, five letters of the surname. **Nothing
|
|
56
|
+
more general is claimed, and two graded passes are why**: which element a snippet's bytes belong to
|
|
57
|
+
is **not contracted** anywhere in this package: that offset's frame follows where the element was
|
|
58
|
+
read (file-absolute at the root, relative to the enclosing slice inside a defined-length Sequence or
|
|
59
|
+
Item, into the inflated stream under Deflated Explicit VR LE), while the snippet is cut from
|
|
60
|
+
whichever buffer the parse is holding, so the two can disagree. Do not reason from a code's message to what its snippet holds;
|
|
61
|
+
measure it, and treat every snippet as document content. Redacting `snippet` is a decision about every Tier-3
|
|
62
|
+
fatal in the library, not a rider on a File Meta disclosure, so the **claim** was corrected instead:
|
|
63
|
+
the test block asserting "the diagnostic is not itself a PHI surface" read `warning.message` and
|
|
64
|
+
nothing else, and is now titled for what it proves, with the strict path pinned beside it as the
|
|
65
|
+
residual it is. `ParseOptions.strict` and the "Keeping PHI out of logs" guide now say so where the
|
|
66
|
+
decision is made. Review the two paths separately.
|
|
67
|
+
|
|
68
|
+
- **🩺 Repeated de-identification was not a fixed point: `(0012,0063)` grew by the whole method
|
|
69
|
+
string on every pass, for any `deidentificationMethod` ending in a SPACE or a NUL**
|
|
70
|
+
(`DICOM-DEIDENT-NOT-A-FIXED-POINT`). **`INTRODUCED` by the entry below and never published** - it
|
|
71
|
+
was found by a fourth graded pass while this release was still held, so no consumer ever saw it.
|
|
72
|
+
Measured in memory over four passes: `"ACME Anonymizer v3 "` read **19 -> 38 -> 57 -> 76** bytes
|
|
73
|
+
and `"Pass A\Pass B "` read **14 -> 21 -> 28 -> 35**, against a flat **19** and **14** on the
|
|
74
|
+
commit before. Over a real `parse -> deidentify -> serializeDicom -> parse` round trip a 16-byte
|
|
75
|
+
method read **16 -> 32 -> 48 -> 64 -> 80 -> 96** over six cycles. Growth continued to the
|
|
76
|
+
65,534-byte ceiling, where the guard **replaces the entire prior provenance chain** - the exact
|
|
77
|
+
loss the entry below exists to prevent, reached from a benign caller string.
|
|
78
|
+
|
|
79
|
+
One asymmetry caused it: the prior value was right-trimmed of `0x20`/`0x00` and the added one was
|
|
80
|
+
not, so a freshly supplied value never equalled its own prior copy - the library wrote the method,
|
|
81
|
+
the serializer's even-length pad folded the trailing byte in, the next parse trimmed it off, and
|
|
82
|
+
the next pass appended the method again. PS3.5 2026c **Table 6.2-1**, `LO` row: "A character string
|
|
83
|
+
that **may be padded with leading and/or trailing spaces**" - trailing spaces are padding, not
|
|
84
|
+
content, so the comparison cannot honour that on one side only.
|
|
85
|
+
|
|
86
|
+
**The trim is per VALUE at the comparison, and a graded pass is why.** A first remedy trimmed each
|
|
87
|
+
operand as a whole Value Field, which reaches only its last value, so a pad byte on an interior
|
|
88
|
+
value of a `1-n` method still regrew the attribute byte-identically: `"Pass A \Pass B"` beside a
|
|
89
|
+
prior `"Pass B "` read **14 -> 21 -> 28 -> 35 -> 42**, and the chain was still replaced at the
|
|
90
|
+
ceiling. Table 6.2-1 describes a **Value** and `LO` is `1-n`, so every value's trailing pad is
|
|
91
|
+
padding; **§6.4**'s "single padding character ... to the end of the Value Field (**to the last
|
|
92
|
+
Value**)" is about where the encoder writes its pad, which is why the value **written** is trimmed
|
|
93
|
+
once over the field. Leading spaces a caller wrote are still written through untouched. With both,
|
|
94
|
+
`deidentify` is a fixed point **from the first pass**. A `deidentificationMethod` that is padding
|
|
95
|
+
only records nothing.
|
|
96
|
+
|
|
97
|
+
- **🩺 A prior `(0012,0063)` value surviving into de-identified output is no longer silent**, via
|
|
98
|
+
the new Tier-2 code **`DICOM_DEIDENT_METHOD_PRIOR_RETAINED`** on `report.warnings`. Keeping it is
|
|
99
|
+
what PS3.15 E.1.1 requires and nothing about the retention changes; the silence was the defect. A
|
|
100
|
+
name a sender wrote into `(0012,0063)` reached output stamped `(0012,0062) Patient Identity
|
|
101
|
+
Removed = YES` with `report.warnings` empty **and** `report.retained` `[]` - an audit reading as a
|
|
102
|
+
scrub it had not performed, on an attribute Table E.1-1 does not list and no rule in the run
|
|
103
|
+
inspected. The code carries **no value, no length and no VR**: the retained text is the file's
|
|
104
|
+
own, the tag in the message is a constant of the code, and `position.byteOffset` locates the
|
|
105
|
+
element. It is deliberately **not** on `report.retained`, which lists the Annex E option sets
|
|
106
|
+
active for the run - a kept attribute is not an option set, and widening that type would break
|
|
107
|
+
every consumer switching over the nine names. Emitted by `deidentify()` only, so it never reaches
|
|
108
|
+
the parser's `{ strict: true }` escalation.
|
|
109
|
+
|
|
110
|
+
- **🩺 A second copy of a MODELED `(0002,xxxx)` element left the parsed object with no warning and
|
|
111
|
+
no residue** (`DICOM-FILE-META-DROPS-DUPLICATE`, raised by `#70`'s gate). **`PRE-EXISTING`**, on
|
|
112
|
+
every released version including the current published `0.0.10`. This is `#70`'s shape one group
|
|
113
|
+
over, and **`(0002,xxxx)` is the group that decides how every following byte is read**, which
|
|
114
|
+
makes it the more dangerous of the two. Disclosed now by the new Tier-2 code
|
|
115
|
+
**`DICOM_DUPLICATE_FILE_META_ELEMENT`**, raised at the moment the projection drops the copy.
|
|
116
|
+
|
|
117
|
+
**It is lossy by a different route from the Data Set case, and the difference is the whole
|
|
118
|
+
entry.** The File Meta group is collected into an **array**, so nothing is overwritten and
|
|
119
|
+
`DICOM_DUPLICATE_TAG_IN_DATA_SET` never fired here - `#70`'s own JSDoc said exactly that, and
|
|
120
|
+
reading it as an all-clear was the defect. The eight tags `parseFileMeta` projects into typed
|
|
121
|
+
`FileMeta` fields are answered by a **first-match** search and are **excluded** from
|
|
122
|
+
`FileMeta.extraElements`, the verbatim residue that gives the group its byte-exact round trip. A
|
|
123
|
+
second copy of one is therefore in neither.
|
|
124
|
+
|
|
125
|
+
**So the two codes resolve a repeat the opposite way round, deliberately, because the two readings
|
|
126
|
+
do: the FIRST copy wins in the File Meta group, the LAST read wins in a Data Set.** Neither
|
|
127
|
+
reading moves in this release, **no value is guessed for the copy that lost**, and no residue is
|
|
128
|
+
invented for it - inventing one would make the serializer write a group it should not. A repeated
|
|
129
|
+
`(0002,xxxx)` tag this library does **not** model stays silent, because every copy of one is
|
|
130
|
+
already kept verbatim in `extraElements` and nothing is dropped. **Two `PRE-EXISTING` bounds a graded pass named, neither closed here**: `serializeDicom` re-emits BOTH copies of such a non-modeled repeat, which is where this package's round-trip promise and its spec-clean promise disagree; and the disclosure covers the group **as the parser delimits it**, so a copy an intermediary appended past an honest `(0002,0000)` group length is never a File Meta element to this parser at all and is relocated into the main Data Set, silently, on this tree and every earlier one.
|
|
131
|
+
|
|
132
|
+
**Measured on the published tarball rather than inferred from a version number.**
|
|
133
|
+
`npm pack @cosyte/dicom@0.0.10` (the registry's current `latest`; there is no `0.0.9`), a file
|
|
134
|
+
carrying `(0002,0010)` twice with two **different** Transfer Syntax UIDs:
|
|
135
|
+
`fileMeta.transferSyntaxUID` reads the first, `fileMeta.extraElements` is `[]`, `ds.warnings` is
|
|
136
|
+
`[]`, and `{ strict: true }` does **not** throw. Silent on every channel, on the released package.
|
|
137
|
+
**And the stakes are not hypothetical**: the same dataset bytes with only the ORDER of those two
|
|
138
|
+
UIDs swapped parse to two different objects, one of which raises `INVALID_FILE_META` out of the
|
|
139
|
+
dataset parser, because a length field read in the wrong encoding declares 1,199,696 bytes.
|
|
140
|
+
|
|
141
|
+
**The message names no tag, and the bound is the factory signature** -
|
|
142
|
+
`duplicateFileMetaElement(position)` - as for `DICOM_DUPLICATE_TAG_IN_DATA_SET`,
|
|
143
|
+
`DICOM_NONZERO_RESERVED_BYTES`, `DICOM_ITEM_CROSSES_SEQUENCE_END` and
|
|
144
|
+
`DICOM_DEIDENT_UNDEFINED_VR_NOT_AUDITABLE`. **Unlike the Data Set code, `position.byteOffset` here
|
|
145
|
+
is unambiguously file-absolute** - `parseFileMeta` is called once per parse with the whole buffer
|
|
146
|
+
and the group is never nested, so none of `Element.byteOffset`'s frame ambiguity can reach it - and
|
|
147
|
+
it locates the copy that was **dropped**, not the survivor.
|
|
148
|
+
|
|
149
|
+
**The citation, and the one that is deliberately absent.** PS3.5 2026c section 7.1, read from the
|
|
150
|
+
SHA-pinned `vendor/nema/part05/` and occurring **exactly once** in that document: "The Data
|
|
151
|
+
Elements in a Data Set shall be ordered by increasing Data Element Tag Number and shall occur at
|
|
152
|
+
most once in a Data Set." **PS3.10, which governs the File Meta Information group, is NOT vendored
|
|
153
|
+
in this repo, so no PS3.10 sentence is cited here and no conformance verdict about a repeated
|
|
154
|
+
`(0002,xxxx)` is claimed.** The code's trigger is narrower and needs neither: it fires exactly when
|
|
155
|
+
a value the file carried does not reach the parsed object, and a `{ strict: true }` caller has
|
|
156
|
+
asked to be thrown at rather than handed a lenient reading.
|
|
157
|
+
|
|
158
|
+
- **🩺 `deidentify()` REPLACED `(0012,0063)` De-identification Method where PS3.15 says it is
|
|
159
|
+
"inserted in or added to" it** (same item, second half). **`PRE-EXISTING`**, measured on `0.0.10`:
|
|
160
|
+
a file recording `"ACME Anonymizer v3 Basic Profile"` came out of `deidentify()` recording only
|
|
161
|
+
this library's own method, with the earlier one gone and nothing saying so. What that destroyed is
|
|
162
|
+
the **provenance chain** the attribute exists to carry, on a file whose earlier pass may be the one
|
|
163
|
+
a recipient was relying on.
|
|
164
|
+
|
|
165
|
+
PS3.15 2026c section **E.1.1 "De-identifier"**, read from the SHA-pinned `vendor/nema/part15/` and
|
|
166
|
+
occurring **exactly once**: "a text string describing the method used shall be **inserted in or
|
|
167
|
+
added to** De-identification Method (0012,0063)." Replacing is neither verb. This release appends
|
|
168
|
+
its own text as a further value of the `1-n` attribute after a `\`, copying the prior bytes through
|
|
169
|
+
**verbatim** so a value encoded under a `(0008,0005)` repertoire survives byte for byte. A value
|
|
170
|
+
that already records this method is left alone rather than growing the attribute. **The comparison
|
|
171
|
+
is per VALUE on both sides**: the method string is itself a `1-n` value, and a graded pass refuted
|
|
172
|
+
the draft that compared the whole string against each prior value - a caller method carrying a `\`
|
|
173
|
+
never matched one, and every pass appended a whole further copy (29 -> 59 -> 89 -> 119 bytes over
|
|
174
|
+
four passes, against a flat 29 on base). **Per value was necessary and not sufficient, and the
|
|
175
|
+
fixed-point claim as first written did not hold** - see the trailing-pad entry below, which is what
|
|
176
|
+
makes it true from the first pass.
|
|
177
|
+
|
|
178
|
+
**The join is bounded, and the bound is not cosmetic.** `LO` is a short-form VR, so
|
|
179
|
+
`encodeDatasetElement` writes its Value Length with a 16-bit field. A `(0012,0063)` carrying a legal
|
|
180
|
+
65,534-byte chain of `1-n` values - exactly the provenance chain this feature exists to build -
|
|
181
|
+
parses with no warnings, and an unbounded append produced a 65,611-byte value that `serializeDicom`
|
|
182
|
+
could not encode: a raw `RangeError` out of Node's `Buffer` internals, outside the documented
|
|
183
|
+
`DicomSerializeError` surface, taking the whole de-identified object down. When the join would
|
|
184
|
+
exceed the ceiling the prior value is **replaced** instead, which is what every released version did
|
|
185
|
+
on every file, and `report.warnings` carries the new `DICOM_DEIDENT_METHOD_NOT_ADDED`, so THAT
|
|
186
|
+
fallback is disclosed. Read the code as "the length ceiling was reached", never as "every fallback
|
|
187
|
+
is disclosed": the `PRE-EXISTING` non-`LO` replacement below is a second shape where `deidentify`
|
|
188
|
+
cannot add, and it is still silent. Truncating the sender's earlier records instead was refused: choosing
|
|
189
|
+
which to drop is a policy the standard does not state.
|
|
190
|
+
|
|
191
|
+
**`(0012,0062)` Patient Identity Removed is still REPLACED with `YES`, and the asymmetry is the
|
|
192
|
+
standard's own**: the sentence immediately above it, in the same list, says it "shall be **replaced
|
|
193
|
+
or added to** the Data Set with a value of YES". Different verbs, different attributes.
|
|
194
|
+
|
|
195
|
+
**The cost is disclosed rather than glossed, and a residual test asserts it.** `(0012,0063)` is
|
|
196
|
+
**not in Table E.1-1**, so the Basic Profile never acted on it and the incoming value reached the
|
|
197
|
+
insertion point untouched - the replacement was the only thing removing it, and removing it was an
|
|
198
|
+
action no profile asked for. A sender who wrote something identifying into `(0012,0063)` now sees
|
|
199
|
+
that text in de-identified output, which is the retained-by-omission posture every other unlisted
|
|
200
|
+
attribute already has. **Closing that direction is a product call about unlisted attributes, not a
|
|
201
|
+
fix to this insertion.** A `(0012,0063)` a file encoded under a VR other than `LO` is replaced
|
|
202
|
+
rather than appended to.
|
|
203
|
+
|
|
204
|
+
**The evidence, with its sha, because the base moves.** Re-measured at `e75fb38` after the last
|
|
205
|
+
test was added - and again after each conformance-gate remedy added more, which is the rule
|
|
206
|
+
and not a courtesy: **8 of the 12** tests in `test/integration/file-meta-duplicate.test.ts` and
|
|
207
|
+
**11 of the 19** in `test/deident/deident-method-add.test.ts` run red against that base, 19 of 31
|
|
208
|
+
in all. **Neither file can link against base `src/`** - none of
|
|
209
|
+
`WARNING_CODES.DICOM_DUPLICATE_FILE_META_ELEMENT`, `duplicateFileMetaElement`,
|
|
210
|
+
`WARNING_CODES.DICOM_DEIDENT_METHOD_NOT_ADDED` or `deidentMethodNotAdded` exists there - so both
|
|
211
|
+
figures are measured with those symbols substituted for their literals; unmodified they are 12 of
|
|
212
|
+
12 and 19 of 19 by construction, which is a fact about linking rather than about behaviour. The
|
|
213
|
+
ones that stay green on base are the controls that make the rest non-vacuous. **A figure taken this way is substitution-sensitive and the third pass read 9 rather than 8**, the delta being the factory-signature row, which is red or green purely by how the absent factory is stood in for. Read it as a floor, and re-derive it rather than quoting it. Quote such a figure
|
|
214
|
+
only with the sha it was run on.
|
|
215
|
+
|
|
216
|
+
- **🩺 A Data Set that carries one tag twice DESTROYED the first element's value at parse time, and
|
|
217
|
+
said nothing** (`DICOM-TAG-COLLISION-DESTROYS-ELEMENT`). **`PRE-EXISTING`**, on every released
|
|
218
|
+
version including the current published `0.0.10`. A parsed Data Set is a `Map<Tag, Element>`, so
|
|
219
|
+
`Map.set` on a tag the map already holds **overwrites in place**: the earlier element's value is
|
|
220
|
+
gone from the object, and the survivor is indistinguishable from an element the sender wrote once.
|
|
221
|
+
No warning, no report entry, nothing a reader could query and nothing a round trip could reveal -
|
|
222
|
+
which is the same harm as a leak, in the other direction. It is now disclosed by the new Tier-2
|
|
223
|
+
code **`DICOM_DUPLICATE_TAG_IN_DATA_SET`**, raised at the moment of the replacement, in every Data
|
|
224
|
+
Set at every depth.
|
|
225
|
+
|
|
226
|
+
**Measured on the published tarball rather than inferred from a version number.**
|
|
227
|
+
`npm pack @cosyte/dicom@0.0.10` (the registry's current `latest`; `npm view` is the only source
|
|
228
|
+
for that, and there is no `0.0.9`), an Explicit VR LE file carrying `(0010,0020)` twice:
|
|
229
|
+
`ds.get("00100020")` reads the **second** value, `ds.warnings` is **empty**, and `{ strict: true }`
|
|
230
|
+
does **not** throw. Silent on every channel, on the released package.
|
|
231
|
+
|
|
232
|
+
**The remedy is the disclosure and nothing else.** The reading does not move: the last element read
|
|
233
|
+
still wins, exactly as in every released version, and **no value is invented for the one that was
|
|
234
|
+
replaced**. That is deliberate rather than modest. A file whose Item over-declares and a file whose
|
|
235
|
+
Sequence under-declares are byte-identical (pinned by `Buffer.equals` in
|
|
236
|
+
`test/integration/explicit-sq-item-bound.test.ts`), so a reader cannot choose the "right" element
|
|
237
|
+
from the bytes; what it can do is say that it had to choose.
|
|
238
|
+
|
|
239
|
+
**Citations, traced rather than stated.** PS3.5 2026c section 7.1 "Data Elements": "The Data
|
|
240
|
+
Elements in a Data Set shall be ordered by increasing Data Element Tag Number and shall occur at
|
|
241
|
+
most once in a Data Set." Section 7.5.1 "Item Encoding Rules", one level down: "Within the context
|
|
242
|
+
of each Item, these Data Elements shall be ordered by increasing Data Element Tag value and appear
|
|
243
|
+
only once." Both read from the SHA-pinned `vendor/nema/part05/`, each occurring exactly once in
|
|
244
|
+
that document. **So this code cannot fire on a conformant file**, which is what makes it safe to add
|
|
245
|
+
under the `{ strict: true }` escalation every Tier-2 code takes.
|
|
246
|
+
|
|
247
|
+
**The message names no tag, and that is specific to this code.** A sender writing the same tag twice
|
|
248
|
+
is the rare route; the ordinary one is a length field that lies, so the second header's four tag
|
|
249
|
+
bytes are read out of the middle of some element's value. `renderTag` shape-checks a tag and
|
|
250
|
+
therefore cannot refuse one, so the bound is the factory signature, as it is for
|
|
251
|
+
`DICOM_NONZERO_RESERVED_BYTES` and `DICOM_ITEM_CROSSES_SEQUENCE_END`. `position.byteOffset` is the
|
|
252
|
+
offset of the header that replaced, which is the surviving element's own `Element.byteOffset` -
|
|
253
|
+
so at the **root** the tag is read off the model rather than out of a message. **Inside a
|
|
254
|
+
defined-length Sequence Item it is not a lookup at all**: `Element.byteOffset` is item-relative
|
|
255
|
+
there, so the same number can name an untouched root element, and no parser warning populates
|
|
256
|
+
`position.contextPath` to tell them apart. Read the code as "an element in this object was
|
|
257
|
+
destroyed" unless you have established the frame.
|
|
258
|
+
|
|
259
|
+
**What this cost, measured on `scripts/measure-sq-bound-grid.ts` against `0ead071`, because a
|
|
260
|
+
disclosure that changes a `{ strict: true }` parse is a behaviour change.** Of 83,037 cells, **349
|
|
261
|
+
differ and every difference is confined to the warning channels and `{ strict: true }`**: 0 cells
|
|
262
|
+
differ in the element tree, the `DeidentifyReport`, the de-identified bytes, which marker values
|
|
263
|
+
survive, or the root `(0010,0020)`; 0 new lenient fatals; leaking cells 11 -> 11; conformant tiling
|
|
264
|
+
controls 7 -> 7. **345 cells now report a collision that was silent on base** (295 Implicit VR LE,
|
|
265
|
+
25 Explicit VR LE, 25 Explicit VR BE), all of them in the grid's two hoist-collision families -
|
|
266
|
+
a fact about those fixtures, not a rate for real files. **9 cells that parsed under
|
|
267
|
+
`{ strict: true }` now throw** - element trees, reports, root identifiers and surviving markers
|
|
268
|
+
identical on both trees, taken from those per-field counters and **not** from the harness's
|
|
269
|
+
`...on an UNCHANGED lenient reading` line, which reads 0 by construction for any slice that adds a
|
|
270
|
+
code. A further **4** were already fatal there and now carry this code instead of
|
|
271
|
+
`INVALID_FILE_META`, because the escalation happens earlier in the parse. The shipped
|
|
272
|
+
`profiles.strict` preset is unchanged.
|
|
273
|
+
|
|
274
|
+
**Three more things a `{ strict: true }` caller should know, none of them fixed here.** (1) On the
|
|
275
|
+
one shape where the parse rolls a descent back - an Implicit VR LE defined-length `SQ` whose value
|
|
276
|
+
holds a duplicate and then a non-Item tag - `{ strict: true }` threw `DICOM_SQ_NOT_DESCENDED`
|
|
277
|
+
before and throws this code now, on a file where **nothing was destroyed**: both values survive in
|
|
278
|
+
`Element.rawBytes`. Both refuse the file, so no object is lost, but the older code was the more
|
|
279
|
+
accurate diagnosis. (2) The same shape streams the code to `onWarning` while `ds.warnings` never
|
|
280
|
+
receives it, which is the pre-existing D-03 ordering reached by one more code. (3) **"Names no tag"
|
|
281
|
+
is about the message.** Strict mode throws a `DicomParseError` whose `snippet` is 16 raw source
|
|
282
|
+
bytes at that offset, rendered as hex - measured on a plain duplicate:
|
|
283
|
+
`10 00 20 00 4c 4f 0e 00 53 4d 49 54 48 53 4f 4e`, the withheld tag plus eight bytes of the value.
|
|
284
|
+
That is the package-wide D-10 design. The same file does not throw at all on `0ead071` (this code is
|
|
285
|
+
what refuses it), but the same file with its even-length padding removed produces the identical 16
|
|
286
|
+
bytes there through `DICOM_ODD_LENGTH_VALUE_PADDED`, one length byte apart - which is why the
|
|
287
|
+
guarantee is worded about the message rather than about the channel.
|
|
288
|
+
|
|
289
|
+
**What the grid is NOT evidence for here, stated because it was read that way once.** It reaches
|
|
290
|
+
the collision only through a length lie, and it swept nothing that carries a tag twice honestly.
|
|
291
|
+
The plain duplicate, the duplicate inside an Item, the collision that lands on a sequence's own tag
|
|
292
|
+
and takes the whole `SQ` and everything nested in it out of the object, the frame the byte offset
|
|
293
|
+
is in, and the `{ strict: true }` behaviour are pinned in
|
|
294
|
+
`test/integration/tag-collision.test.ts` or nowhere: **14 of its 15 tests run red against
|
|
295
|
+
`origin/main` at `0ead071`** - re-measured after each gate pass added tests, and written with its sha
|
|
296
|
+
because the figure moves with every test added - the fifteenth being the no-duplicate
|
|
297
|
+
control, which is green there by design.
|
|
298
|
+
|
|
299
|
+
**Still open, and not touched here:** the private-`SQ` carve-out (`DICOM-PRIVATE-SQ-CARVE-OUT`),
|
|
300
|
+
and the structural relocation itself under `DICOM-EXPLICIT-VR-UNBOUNDED-ITEM-READ` - an
|
|
301
|
+
over-declaring Item still moves the element that follows the sequence. This slice makes the loss
|
|
302
|
+
observable; it does not make the file readable.
|
|
303
|
+
|
|
304
|
+
- **🩺 The mirror of the above: a private value ejected OUT of a Sequence Item was retained on a
|
|
305
|
+
reservation it borrowed from the Data Set it landed in, in output stamped
|
|
306
|
+
`PatientIdentityRemoved = YES` with `report.removedPrivateTags` reading `[]`**
|
|
307
|
+
(`DICOM-ITEM-EJECT-ROUTE`). **`PRE-EXISTING`**, found by `#66`'s pass-2 `conformance-refuter`,
|
|
308
|
+
pinned by `#66` as residual tests that asserted the leaking behaviour, and re-measured on
|
|
309
|
+
`300af87` before anything changed. The **private-`SQ` carve-out** is still open (below). Read this
|
|
310
|
+
as "the eject direction is refused", never as "the class is closed".
|
|
311
|
+
|
|
312
|
+
**Measured on the published tarball, not inferred from a version number.**
|
|
313
|
+
`npm pack @cosyte/dicom@0.0.10` and the Explicit VR LE eject fixture:
|
|
314
|
+
`report.removedPrivateTags` is `[]`, `SECRET-PRIVATE-PHI` is in the serialized output,
|
|
315
|
+
`(0012,0062)` is `YES`, **`ds.warnings` is empty and `{ strict: true }` does not throw**. So on the
|
|
316
|
+
released package this route is silent on every channel, exactly as the item filed it.
|
|
317
|
+
**`0.0.9` is NOT on the registry** - `package.json` carried it, the publish never happened
|
|
318
|
+
(npm `E404`), and the current published version is `0.0.10`. Earlier entries in this file that say
|
|
319
|
+
"live on the published `0.0.9`" name a version that does not exist.
|
|
320
|
+
|
|
321
|
+
**On `main` it is no longer silent, and that difference is `#51` being unreleased.** With
|
|
322
|
+
`DICOM_ITEM_CROSSES_SEQUENCE_END` present, the Explicit VR shapes announce themselves on both
|
|
323
|
+
channels and throw under `{ strict: true }`; the Implicit VR LE shape raises the older
|
|
324
|
+
`DICOM_SQ_NOT_DESCENDED` and throws too. The leak was live beside those warnings, which is the
|
|
325
|
+
point: a warning about structure is not an all-clear about what was retained.
|
|
326
|
+
|
|
327
|
+
**The shape.** An `(FFFE,E000)` Item that declares **fewer** bytes than its content occupies ends
|
|
328
|
+
early, so the elements the sender encoded as Item content are read as elements of the **enclosing**
|
|
329
|
+
Data Set. A Private Creator that lands there reserves a `(gggg,00EE)` block for elements the sender
|
|
330
|
+
never put beside it, and the next private element is kept verbatim on it. **It is not
|
|
331
|
+
root-specific**: an inner sequence ejecting a creator into the still-usable Item that encloses it
|
|
332
|
+
reproduces the same leak one level down, and a root-scoped remedy would read green on it.
|
|
333
|
+
|
|
334
|
+
**Two predicates, because the parser records the same contradiction two different ways, and this is
|
|
335
|
+
why it is its own slice rather than a widening of the absorb rule.** Under **Explicit VR** the item
|
|
336
|
+
stream is bounded against the buffer, so it is read past the sequence's declared end and the span
|
|
337
|
+
shows up as `Element.rawBytes.length` exceeding `Element.length`. Under **Implicit VR LE** that path
|
|
338
|
+
slices the item stream to the declared Value Length, so **nothing over-runs at all**
|
|
339
|
+
(`rawBytes.length === length`); the item does not fit the slice, the descent is refused, and the
|
|
340
|
+
element carries `items === undefined` with `DICOM_SQ_NOT_DESCENDED`. Both are facts the parser had
|
|
341
|
+
already recorded: no re-parse, no scan, no cost that follows an attacker-chosen length. **The second
|
|
342
|
+
predicate is broader than the ejection it is here for, deliberately and in the fail-safe direction**
|
|
343
|
+
- it says the parser could not walk the sequence, which another unwalkable item stream also reaches.
|
|
344
|
+
|
|
345
|
+
**The remedy is positional, inside every Data Set, and it removes rather than downgrading the
|
|
346
|
+
stamp.** `settledBound` finds where a Data Set stops accounting for its own membership: the first
|
|
347
|
+
sequence whose own contents contradict its declared extent. Both the reservation map and the
|
|
348
|
+
retention decision are taken from the settled run, so a creator ejected into a Data Set reserves
|
|
349
|
+
nothing there and a private element read after that point is removed and named in
|
|
350
|
+
`report.removedPrivateTags`. An element read **before** the offending sequence cannot have come out
|
|
351
|
+
of it and is untouched. `processElements` derives this at every depth. **No parser file is touched
|
|
352
|
+
and no reading changes.**
|
|
353
|
+
|
|
354
|
+
**🛑 TWO BOUNDS, NOT ONE, BECAUSE A DATA SET IS A `Map<Tag, Element>`.** When the ejected element
|
|
355
|
+
carries a tag the Data Set **already holds**, `Map.set` overwrites in place and the newcomer
|
|
356
|
+
inherits the earlier element's **position**, ahead of the sequence it came out of. An index cut
|
|
357
|
+
alone reads it as settled and retains it: measured on a root holding a genuine
|
|
358
|
+
`(0009,0010)` + `(0009,1001)` reservation ahead of a sequence whose item ejects a second
|
|
359
|
+
`(0009,1001)`, which lands at index 2 with `byteOffset` 274 while the sequence sits at index 3 with
|
|
360
|
+
`byteOffset` 238. `Element.byteOffset` is the position the parser counted and the overwrite cannot
|
|
361
|
+
move it, so it is checked beside the index; the two are conjunctive because a hand-built
|
|
362
|
+
`Dataset` may carry no meaningful offsets at all, and there the index bound is what still
|
|
363
|
+
bites. **The grid is blind to this: the index-only and two-bound remedies differ on 0 of 83,037
|
|
364
|
+
cells**, because no `priv|` fixture collides tags. **3 tests, counted over the full suite, are the
|
|
365
|
+
whole pin**: the two collision rows and the creator-flip. Both offsets above are measured on the fixture pinned in `test/integration/deident-private-reservation.test.ts`, whose File Meta is the minimum **this parser** requires rather than PS3.10's - a fixture with more File Meta shifts every offset, and a pass-2 grade read 292 off one that did.
|
|
366
|
+
|
|
367
|
+
**🩺 AND THE COLLISION DESTROYS THE ROOT'S OWN VALUE ON THE WAY IN, WHICH THIS SLICE DOES NOT FIX.**
|
|
368
|
+
The overwrite replaces the reservation's genuine root element with the Item's, silently, at parse
|
|
369
|
+
time, with no warning naming it and no report entry - the `Map<Tag, Element>` substitution already
|
|
370
|
+
recorded for `(0010,0020)`, now reached on the private-retention path. `PRE-EXISTING`, identical on
|
|
371
|
+
both trees, **its own item**, and asserted in the tests so it cannot be mistaken for something this
|
|
372
|
+
remedy handled.
|
|
373
|
+
|
|
374
|
+
**The spec does the work, and both sentences were re-located in the pins rather than carried
|
|
375
|
+
forward.** PS3.5 2026c §7.8.1, one occurrence in `vendor/nema/part05/4dfd7b8c…`: "Items within a
|
|
376
|
+
sequence are self contained Data Sets ... The scope of the reservation is just within the Item.
|
|
377
|
+
Items do not inherit the Private Data Element reservations made by Private Creator Data Elements in
|
|
378
|
+
the Data Set in which the Item is nested." So the reservation's scope **is** the Item's boundary,
|
|
379
|
+
and a file that contradicts itself about where that boundary falls establishes no knowledge of which
|
|
380
|
+
Data Set an element is in. PS3.15 2026c §E.3.10, one occurrence in
|
|
381
|
+
`vendor/nema/part15/77d60b85…`: "Private Attributes that are known by the de-identifier to be safe
|
|
382
|
+
from identity leakage shall be retained, together with the Private Creator IDs that are required to
|
|
383
|
+
fully define the retained Private Attributes; all other Private Attributes shall be removed **or
|
|
384
|
+
processed in the element-specific manner recommended by Deidentification Action (0008,0307), if
|
|
385
|
+
present within Private Data Element Characteristics Sequence (0008,0300)**" - a two-branch clause,
|
|
386
|
+
quoted whole; this library does not implement `(0008,0307)`, so removal is the branch available to
|
|
387
|
+
it, and **"known"** is the load-bearing word.
|
|
388
|
+
|
|
389
|
+
**The price is measured, not asserted.** `scripts/measure-sq-bound-grid.ts` over **83,037** cells
|
|
390
|
+
against `300af87`: `priv: kept at ROOT, file CONTRADICTS` **78 -> 0**, of which the eject leaks
|
|
391
|
+
(`...whose honest control does NOT keep it`) are **22 -> 0** and the remaining **56 are the cost** -
|
|
392
|
+
root retentions on self-contradicting files whose honest control does keep the value. **That cost
|
|
393
|
+
has its own column and it is named rather than folded in: `de-identified OUTPUT lost a marker
|
|
394
|
+
(cost)` reads 78**, which is the de-identify-boundary column; the `LOST` and `GAINED a marker value`
|
|
395
|
+
counters beside it are **parse**-tree columns and both read 0, and quoting those two without this
|
|
396
|
+
one is exactly the confusion this repo has paid for before. Retention on files that do **not**
|
|
397
|
+
contradict themselves is unchanged (**9 -> 9** at the root, **6 -> 6** inside an Item),
|
|
398
|
+
`no-creator` rows stay **0 -> 0**, `LEAKING a source value` is **11 -> 11** (the binary-carrier
|
|
399
|
+
residual, untouched), `conformant tiling control emptied` **7 -> 7**, and **0 cells differ in any
|
|
400
|
+
PARSE respect, 0 cells whose READING differs**, 0 new lenient fatals, 0 new strict fatals, 0 wrong
|
|
401
|
+
root `(0010,0020)`, 0 reports losing an attribute. **118 cells changed, every one of them in the
|
|
402
|
+
`priv|` family: 74 Implicit VR LE and 44 Explicit VR.** (`structural` also reads 118 by construction
|
|
403
|
+
- it counts any record difference, including the de-identify columns, so it is not a reading claim.)
|
|
404
|
+
|
|
405
|
+
**🛑 THE HARNESS COULD NOT REPORT THAT SPLIT AND NOW CAN.** `--diff` classified a cell's transfer
|
|
406
|
+
syntax by testing whether its key _starts with_ the syntax, which is true only of the sequence
|
|
407
|
+
sweep; the `carrier|`, `legit|` and `priv|` families put their own prefix in that position, so
|
|
408
|
+
**not one of their rows could ever be counted as Implicit VR LE**. This slice's 74 Implicit VR LE
|
|
409
|
+
cells were reported as `Implicit VR LE 0`. `transferSyntaxOf` fixes it, which also moves
|
|
410
|
+
`onWarning != ds.warnings, Explicit VR` from 43 to **0** - those 43 were Implicit VR LE rows in
|
|
411
|
+
other families, i.e. the pre-existing D-03 ordering residual, misattributed. **Any syntax split
|
|
412
|
+
quoted off that line for a slice touching those three families predates this fix; re-run the diff
|
|
413
|
+
rather than carrying the figure forward.**
|
|
414
|
+
|
|
415
|
+
**The whole-Data-Set variant was built and measured rather than argued about, and the grid cannot
|
|
416
|
+
tell it apart.** Refusing every private element in a Data Set that _contains_ a disputed sequence
|
|
417
|
+
produces a **byte-identical** grid diff (0 of 83,037 cells differ from what shipped), because every
|
|
418
|
+
`priv|` fixture writes its private block after the sequence. **It differs on exactly 5 tests,
|
|
419
|
+
counted over the full suite rather than one file**, and they are why it is not taken: it reds "a
|
|
420
|
+
root reservation the sender wrote at the root survives an over-running sequence" and both collision
|
|
421
|
+
tests, all three of which are genuine root reservations the sender wrote ahead of the disputed
|
|
422
|
+
sequence, and it also closes **both** private-`SQ` carve-out residuals, which belong to a different
|
|
423
|
+
item - the second of them in `test/integration/deident-unauditable-sequence.test.ts`, which a
|
|
424
|
+
per-file enumeration misses. **The tests are the pin for the positional cut, not the grid.**
|
|
425
|
+
|
|
426
|
+
**What is NOT closed, and is not asserted away.** The **private-`SQ` carve-out**: `keepsPrivate`
|
|
427
|
+
decides before `descendSequence`, so a private `SQ` inside the settled run that the profile vouches
|
|
428
|
+
for is kept **verbatim** and nothing inside it is examined. `PRE-EXISTING`, still pinned as a
|
|
429
|
+
residual test that asserts the leaking behaviour. And the `Map<Tag, Element>` substitution above.
|
|
430
|
+
|
|
431
|
+
**The base measurement, taken fresh rather than carried forward:** **8 of the 37** tests in
|
|
432
|
+
`test/integration/deident-private-reservation.test.ts` run red against `origin/main`'s `src/` at
|
|
433
|
+
`300af87`, and they are exactly the eight eject tests. The remaining private-`SQ` residual is
|
|
434
|
+
**green** on that base, by design. **The "10 of the 31" figure and the "both residual tests are red
|
|
435
|
+
against base" correction beside it are both retired**: the first was measured against a pre-`#66`
|
|
436
|
+
tree, and the second is false against today's `main`, because `DICOM_ITEM_CROSSES_SEQUENCE_END` has
|
|
437
|
+
been on `main` since `79e9f34`.
|
|
438
|
+
|
|
439
|
+
**No new public surface, deliberately** - no Tier-2 code, no report field, no snapshot change, the
|
|
440
|
+
same choice `#66` made. `report.removedPrivateTags` already names every removed tag and is the
|
|
441
|
+
channel this defect read `[]` on.
|
|
442
|
+
|
|
9
443
|
- **🩺 A private value crossed the PS3.5 §7.8.1 reservation boundary and was retained under
|
|
10
444
|
`RetainSafePrivate`, in output stamped `PatientIdentityRemoved = YES`.** The **absorb** direction
|
|
11
|
-
is closed; the **eject** direction and the private-`SQ` carve-out
|
|
12
|
-
closed (
|
|
13
|
-
|
|
14
|
-
found by `#51`'s pass-6
|
|
15
|
-
was never introduced by
|
|
445
|
+
is closed; the **eject** direction and the private-`SQ` carve-out were measured, pinned and **not**
|
|
446
|
+
closed by this entry. (The **eject** direction has since been closed by `DICOM-ITEM-EJECT-ROUTE`,
|
|
447
|
+
the entry above; the private-`SQ` carve-out is still open.)
|
|
448
|
+
(`DICOM-PRIVATE-CREATOR-RESERVATION-LEAK`). **`PRE-EXISTING`**; found by `#51`'s pass-6
|
|
449
|
+
`conformance-refuter` and measured **identical on `origin/main`**, so it was never introduced by
|
|
450
|
+
that branch. (This entry said "live on the published `0.0.9`". **`0.0.9` was never published** -
|
|
451
|
+
`package.json` carried it and the publish never happened, npm `E404`. The leak is measured on the
|
|
452
|
+
published `0.0.10` tarball.)
|
|
16
453
|
|
|
17
454
|
**The shape.** An `(FFFE,E000)` Item that declares more bytes than its enclosing `SQ` element's
|
|
18
455
|
Value Length allows reads past the end of its sequence and absorbs the element that followed it.
|
|
@@ -86,8 +523,14 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
|
|
|
86
523
|
a block for elements the sender never gave it. **It is NOT root-specific and an earlier draft of
|
|
87
524
|
this entry said it was** - pass 2 reproduced it one level down, where an inner sequence ejects a
|
|
88
525
|
creator into the still-usable enclosing Item. The item filed for this must be scoped to every
|
|
89
|
-
still-usable Data Set or it will be built to the wrong bar.
|
|
90
|
-
|
|
526
|
+
still-usable Data Set or it will be built to the wrong bar. **It was silent on every channel and
|
|
527
|
+
under `{ strict: true }` when this shipped, and `DICOM-EXPLICIT-VR-UNBOUNDED-ITEM-READ` changed
|
|
528
|
+
that on the Explicit VR shapes and nowhere else, so do not carry the old sentence forward.**
|
|
529
|
+
Re-measured on this residual's own fixture: `DICOM_ITEM_CROSSES_SEQUENCE_END` on both channels and
|
|
530
|
+
a throw under `{ strict: true }`, beside an entirely unchanged leak (`removedPrivateTags: []`, the
|
|
531
|
+
value in the serialized output, `(0012,0062) = YES`). That warning is now the only signal on a file
|
|
532
|
+
whose de-identification audit is false, and it is never an all-clear. The 20 Implicit VR LE cells
|
|
533
|
+
stay silent: that path slices the item stream, so no over-run is recorded. **22 grid cells.** The widening was **built
|
|
91
534
|
and measured**: narrowing the flag whenever a Data Set _contains_ an over-running sequence costs
|
|
92
535
|
**24** root retentions and closes **2** of the 22, because the other 20 are Implicit VR LE where
|
|
93
536
|
the sequence records no over-run at all. A different mechanism, and its own item.
|
|
@@ -327,6 +770,134 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
|
|
|
327
770
|
|
|
328
771
|
### Security
|
|
329
772
|
|
|
773
|
+
- **A Sequence Item under an Explicit VR transfer syntax can read past the end its own sequence
|
|
774
|
+
declared and swallow the element that followed the sequence, and did so silently. It now says so**
|
|
775
|
+
(`DICOM-EXPLICIT-VR-UNBOUNDED-ITEM-READ`, pre-existing and live on the published `0.0.10`, on both
|
|
776
|
+
Explicit VR Little Endian and Explicit VR Big Endian). New Tier-2 code
|
|
777
|
+
`DICOM_ITEM_CROSSES_SEQUENCE_END` (`WARNING_CODES` is **29, was 28**; the locked snapshot is the
|
|
778
|
+
pin and was updated deliberately), promoted to a throw under the `{ strict: true }` parse option.
|
|
779
|
+
**No reading changes, on any file.** That is the fix's whole shape, and the next paragraph is why.
|
|
780
|
+
|
|
781
|
+
**THE FACT THAT DECIDED THIS, AND IT IS PROVEN BY A TEST RATHER THAN ARGUED.** A file whose
|
|
782
|
+
**item over-declares** and a file whose **sequence under-declares** are **the same bytes**.
|
|
783
|
+
`test/integration/explicit-sq-item-bound.test.ts` builds both from two contradictory intentions and
|
|
784
|
+
asserts `Buffer.equals`. So "which of the two length fields is the lie?" has no answer on the wire,
|
|
785
|
+
and any bound that prefers section 7.5.2's extent for the first file imposes it on the second one
|
|
786
|
+
too, because there is no second file. **Five graded attempts at such a bound were refused on
|
|
787
|
+
`#51`.** This slice adds no bound, and that one fact is the entire reason.
|
|
788
|
+
|
|
789
|
+
**🛑 THE FAIL-SAFE-DIRECTION ARGUMENT IS DELETED, NOT REWORDED.** Five artifacts on this branch
|
|
790
|
+
said that following the Item's length field is the safe half of the ambiguity, because a Private
|
|
791
|
+
Creator swallowed **into** an item leaves the enclosing block unclaimed and an unclaimed block is
|
|
792
|
+
removed. **That is false, and it was the sixth refusal.** Which direction leaks is a property of
|
|
793
|
+
**where the sender put the Private Creator**, not of which length field a reader follows: with the
|
|
794
|
+
creator as genuine Item content, the absorb direction leaks too
|
|
795
|
+
(`DICOM-PRIVATE-CREATOR-RESERVATION-LEAK`, measured on `164eb39`, closed at the de-identify
|
|
796
|
+
boundary by `#66` and never in the parser). The eject direction is still open. Neither reading is
|
|
797
|
+
safe by construction, which is precisely why this code reports rather than decides. Per this
|
|
798
|
+
repo's own rule, a claim refuted twice is deleted rather than given a third wording.
|
|
799
|
+
|
|
800
|
+
Measured, on `0.0.10`: `(0008,1115)` holding one item that over-declares its length by 18 bytes,
|
|
801
|
+
followed by a root `(0010,0020)` Patient ID that is 18 bytes on the wire (an 8-byte Explicit VR
|
|
802
|
+
short-form header plus a 9-character value padded to 10). The Patient ID is **absent from the root**
|
|
803
|
+
and present instead as an attribute of the item; the `DeidentifyReport` names it with a
|
|
804
|
+
`contextPath` pointing at a sequence item it was never in. It is swallowed once and relocated, not
|
|
805
|
+
read twice: the parser resumes where the descent actually ended. The parse was silent about all of
|
|
806
|
+
it, including under `{ strict: true }`, and that silence is what is fixed. The mis-structure itself
|
|
807
|
+
is **not repaired here** and is pinned as a residual by a test.
|
|
808
|
+
|
|
809
|
+
**▶ AND IT IS THE ONLY SIGNAL ON A FILE WHOSE DE-IDENTIFICATION AUDIT IS FALSE.** `#66` recorded
|
|
810
|
+
the eject direction of `DICOM-PRIVATE-CREATOR-RESERVATION-LEAK` as silent on every channel. On the
|
|
811
|
+
**Explicit VR** shapes that is no longer true: re-measured on that residual's own fixture, this
|
|
812
|
+
code fires on both channels and throws under `{ strict: true }`, beside an unchanged leak
|
|
813
|
+
(`report.removedPrivateTags` `[]`, the private value in the serialized output, the object stamped
|
|
814
|
+
`(0012,0062) Patient Identity Removed = YES`). The leak is not closed here and the warning is not
|
|
815
|
+
an all-clear - **the troubleshooting row says so explicitly**, because the earlier draft of that
|
|
816
|
+
row told an operator "nothing is retained that would not be", which is false on exactly this file.
|
|
817
|
+
Implicit VR LE stays silent: that path slices the item stream, so no over-run is recorded.
|
|
818
|
+
|
|
819
|
+
PS3.5 2026c section **7.5.2** "Delimitation of The Sequence of Items" makes the `SQ` element's own
|
|
820
|
+
Value Length the exact extent of the item stream: "This length shall include the total length
|
|
821
|
+
resulting from the sequence of zero or more items conveyed by this Data Element." Section **7.5.1**
|
|
822
|
+
"Item Encoding Rules" governs each `(FFFE,E000)` Item's own length field. Both traced to the
|
|
823
|
+
SHA-pinned `vendor/nema/part05/`, each sentence unique in the document. **Neither clause says what
|
|
824
|
+
a decoder does when the two disagree**, so no reading is derived from them; where they disagree
|
|
825
|
+
this reader follows 7.5.1, which is what every released version does.
|
|
826
|
+
|
|
827
|
+
**Where the disclosure fires, exactly.** A defined-length `(FFFE,E000)` item, inside a
|
|
828
|
+
defined-length `SQ`, whose declared end is **not** the end of the buffer it is being read in -
|
|
829
|
+
which is what says the sequence sits inside a larger Data Set whose bytes are there to be taken.
|
|
830
|
+
A sequence handed a slice cut at its declared end (Implicit VR LE's `tryParseDefinedLengthSQ`,
|
|
831
|
+
CP-246's `tryParseUnAsSQ`), an undefined-length sequence, an undefined-length item, and a sequence
|
|
832
|
+
that ends its own buffer all have nothing to reach into and stay silent. Each is a test.
|
|
833
|
+
|
|
834
|
+
**🩺 The Item's declared length is WITHHELD from the message, and the bound is the factory
|
|
835
|
+
signature.** A diagnostic about a length field that lies is itself a PHI surface: the condition
|
|
836
|
+
that raises this code is exactly "these length fields are not what they claim to be", so the Item's
|
|
837
|
+
32-bit Value Length can be four bytes of somebody's value. Measured, an item header fabricated over
|
|
838
|
+
the payload `"SMITHSON"` rendered it as `1414090067`, `"SMIT"` in wire order, reversible with one
|
|
839
|
+
`readUInt32LE` - and it is emitted **above** the truncation guard, so the message reaches
|
|
840
|
+
`onWarning` on a file the parse then refuses. `itemCrossesSequenceEnd` takes no parameter for it,
|
|
841
|
+
the same remedy `#64` applied to `DICOM_NONZERO_RESERVED_BYTES` and `#55` to
|
|
842
|
+
`DICOM_DEIDENT_UNDEFINED_VR_NOT_AUDITABLE`; `position.byteOffset` locates the item. The bytes that
|
|
843
|
+
remained inside the sequence are still reported, and that asymmetry is structural rather than a
|
|
844
|
+
judgement call: the emit site's own `endLimit < buffer.length` conjunct bounds that count by the
|
|
845
|
+
buffer. Measured against the identical attack - fabricating the **`SQ`**'s length field over the
|
|
846
|
+
same name puts `endLimit` past the buffer, so the code does not fire at all. Both pinned with a
|
|
847
|
+
name-bearing payload and a mutation control; the shipped message said "no bytes off the wire", then
|
|
848
|
+
carried the length itself, and both are gone.
|
|
849
|
+
|
|
850
|
+
**The measurement, on `scripts/measure-sq-bound-grid.ts` against `2f0abd9`, 83,037 cells.** Both
|
|
851
|
+
sequence length fields and an element's own swept independently, across ten item shapes, both
|
|
852
|
+
Explicit VR syntaxes, an Implicit VR LE control, both `strict` postures, and `#66`'s `priv|`
|
|
853
|
+
private-reservation family:
|
|
854
|
+
|
|
855
|
+
| | |
|
|
856
|
+
| ---------------------------------------------------------- | ------------------------- |
|
|
857
|
+
| cells whose **reading** differs | **0** |
|
|
858
|
+
| cells newly emitting `DICOM_ITEM_CROSSES_SEQUENCE_END` | 616 |
|
|
859
|
+
| ...cells that LOSE it | **0** |
|
|
860
|
+
| cells that newly fail under `{ strict: true }` (the price) | **576** |
|
|
861
|
+
| new lenient fatals | **0** |
|
|
862
|
+
| values lost, values gained, wrong root `(0010,0020)` | **0 / 0 / 0** |
|
|
863
|
+
| PHI regressions | **0** |
|
|
864
|
+
| reports that lose an attribute | **0** |
|
|
865
|
+
| Implicit VR LE cells changed | **0** |
|
|
866
|
+
| every `priv\|` column from `#66` | unchanged |
|
|
867
|
+
| cells leaking a source value, base -> now | 11 -> 11 (`PRE-EXISTING`) |
|
|
868
|
+
|
|
869
|
+
The other **16,396** of the 17,012 differing cells are **strict-fatal on both trees** and differ
|
|
870
|
+
only in the _class_ of the `{ strict: true }` throw, because the new Tier-2 code escalates before
|
|
871
|
+
the Tier-3 fatal those files already had. All 576 new strict fatals carry the new code, so none is
|
|
872
|
+
collateral. **Quote the reading count and the strict count together or neither** - a new warning is
|
|
873
|
+
a real behaviour change, and 576 files that parsed under `{ strict: true }` now do not. Every one
|
|
874
|
+
is a file whose two normative length fields contradict each other, which is what the strict posture
|
|
875
|
+
exists to refuse; the lenient reading of all 576 is byte-identical to before. `cells whose READING
|
|
876
|
+
differs` is the counter that expresses this and it already exists on `main` (`#66` added it); the
|
|
877
|
+
earlier draft of this entry claimed to add it.
|
|
878
|
+
|
|
879
|
+
**Disclosed and NOT fixed, deliberately:**
|
|
880
|
+
- **The mis-structure.** An over-declaring item still relocates the element that follows the
|
|
881
|
+
sequence, and `deidentify()` still reports it under a `contextPath` naming an item it was never
|
|
882
|
+
in. Pinned by a test so the claim cannot drift from the code.
|
|
883
|
+
- **`ds.warnings` is uncapped, and "at most one per sequence" is not an amplification bound.** The
|
|
884
|
+
shape holds, but a file may carry as many sequences as it can encode. Pinned by a test that
|
|
885
|
+
asserts the growth. This is `#48`'s pre-existing, package-wide posture for parser warnings.
|
|
886
|
+
- **`profiles.strict` does not escalate this code.** The `{ strict: true }` option does. Adding a
|
|
887
|
+
code to a shipped preset moves every `profiles.strict` consumer's parse and is its own measured
|
|
888
|
+
change. Pinned by a test.
|
|
889
|
+
- **`position.byteOffset` is frame-dependent**, like `Element.byteOffset`: file-absolute for a
|
|
890
|
+
root-level sequence, slice-relative inside an enclosing item. Measured and pinned.
|
|
891
|
+
- **The undefined-length item with no `(FFFE,E00D)`** still runs to the end of the buffer. There is
|
|
892
|
+
no declared item length, so there is no disagreement to disclose.
|
|
893
|
+
- **`Element.byteOffset` inside a sequence item disagrees with itself between the two item forms,
|
|
894
|
+
and always has.** Measured identically on this branch and on `origin/main`: on a 210-byte file
|
|
895
|
+
with the `SQ` at 172, an element inside a **defined-length** item reads `0` (the item slice is its
|
|
896
|
+
own frame) and the same element inside an **undefined-length** item reads `192` (file-absolute,
|
|
897
|
+
because `parseSequence` hands that branch the outer buffer). `PRE-EXISTING`.
|
|
898
|
+
- **11 grid cells still leak a source value** - the over-declare swallow into an `OB`/`OW`/`US`/`UN`
|
|
899
|
+
leaf carrier, `PRE-EXISTING`, identical on both trees, its own item.
|
|
900
|
+
|
|
330
901
|
- **A symbolic link under `test/fixtures/` pointing at a PHI-bearing file scanned CLEAN on BOTH of
|
|
331
902
|
`scripts/phi-scan.ts`'s enumerating routes, so the commit gate passed it twice over**
|
|
332
903
|
(`PHI-SCAN-SYMLINK-BLIND-ON-BOTH-ROUTES`; pre-existing). Reproduced on `0f898be` with a synthetic
|