@cosyte/dicom 0.0.11 → 0.0.12

package/CHANGELOG.md

@@ -6,6 +6,65 @@ All notable changes to `@cosyte/dicom` will be documented in this file. The form
 
 ### Fixed
 
+- **🩺 The `(0012,0063)` De-identification Method this library writes for itself exceeded the 64-character
+  maximum PS3.5 gives an `LO` Value, on every file it ever de-identified**
+  (`DICOM-LO-LENGTH-AND-SILENT-REPLACE`). `PRE-EXISTING`; measured through the built package on
+  `da1f209`: **76** characters with no options, **130** with `RetainUIDs + RetainSafePrivate +
+RetainDeviceIdentity`, **272** with all nine, and **all 512 option subsets over the maximum**. PS3.5
+  2026c **Table 6.2-1**, `LO` row, is "64 chars maximum" and that row describes a **Value**;
+  `(0012,0063)` is `1-n`, so the bound falls per value. Reading it as a bound on the Value Field is the
+  same misreading that left the fixed-point hole above (where §6.4, a clause about the **encoder**, was
+  read as a bound on a **comparison**), and it matters because the attribute concerned is the one a
+  receiver reads to decide whether an object was de-identified at all.
+
+  The default method text is now **multi-valued**: one Value naming the Profile
+  (`@cosyte/dicom Basic Application Level Confidentiality Profile`, **61** characters) and one Value per
+  active Annex E Option, joined with `\`. No option name exceeds **28** characters, so no subset can
+  breach the maximum - proved by sweeping **all 512 subsets / 2,816 value cells** rather than by
+  argument: **0 over the maximum**, against **512 of 512** on the commit before. The Value Field is
+  still 111 and 247 bytes for those two option sets, which is legal and is asserted alongside the
+  per-value figure so a remedy that merely shortened the text could not pass. Options are emitted in
+  `DEIDENTIFY_OPTIONS` order rather than the caller's, so the same option set always writes the same
+  bytes. The fixed point is unaffected and re-measured over six real wire round trips: **62** bytes flat
+  by default and **248** flat with all nine.
+
+  **Your own `deidentificationMethod` is not bounded for you** and neither is a prior value the source
+  file wrote: splitting or truncating either would invent a de-identification record nobody made. Both
+  are pinned as residual tests rather than left to be rediscovered. Split your string on `\` yourself if
+  a strict receiver is in your path.
+
+- **🩺 A `(0012,0063)` a file encoded under a VR other than `LO` was replaced silently**, and now raises
+  the new Tier-2 code **`DICOM_DEIDENT_METHOD_NOT_LO`** on `report.warnings`. The replacement itself is
+  deliberate and unchanged: the provenance chain is built by concatenating `LO` values with the `5CH`
+  delimiter, which is a text operation and not defined over the arbitrary octets an `OB` or `UN` value
+  holds, so guessing an encoding for them was refused. What was wrong was that the sender's earlier
+  de-identification record left the object with `report.warnings` **empty**, under
+  `(0012,0062) = YES`. It is a separate code from `DICOM_DEIDENT_METHOD_NOT_ADDED` because the causes
+  are unrelated - the chain outgrew the VR, or the bytes were never in that VR at all - and a consumer
+  that has to tell them apart now can. A prior value that is empty or padding only raises neither,
+  because nothing was lost. The code carries **no value, no length and no VR**: two bytes read out of a
+  fabricated header are document content, which is how four letters of a surname once reached
+  `DICOM_NONZERO_RESERVED_BYTES`.
+
+- **🩺 `{ strict: true }` renders source bytes the warning it replaces deliberately withholds.**
+  `PRE-EXISTING`, **disclosed rather than changed**: the escalation raises a `DicomParseError` whose
+  `snippet` is 16 raw source bytes read from the file at the warning's own `byteOffset` (D-10), while
+  `err.message` stays the frozen registry string. Measured on
+  `DICOM_DUPLICATE_FILE_META_ELEMENT`, whose group is never nested and whose offset is therefore
+  unambiguously the dropped copy's header: a `(0002,0016)` Source AE Title of `AE-SMITHSON` comes
+  back as `02 00 16 00 41 45 0c 00 41 45 2d 53 4d 49 54 48`, five letters of the surname. **Nothing
+  more general is claimed, and two graded passes are why**: which element a snippet's bytes belong to
+  is **not contracted** anywhere in this package: that offset's frame follows where the element was
+  read (file-absolute at the root, relative to the enclosing slice inside a defined-length Sequence or
+  Item, into the inflated stream under Deflated Explicit VR LE), while the snippet is cut from
+  whichever buffer the parse is holding, so the two can disagree. Do not reason from a code's message to what its snippet holds;
+  measure it, and treat every snippet as document content. Redacting `snippet` is a decision about every Tier-3
+  fatal in the library, not a rider on a File Meta disclosure, so the **claim** was corrected instead:
+  the test block asserting "the diagnostic is not itself a PHI surface" read `warning.message` and
+  nothing else, and is now titled for what it proves, with the strict path pinned beside it as the
+  residual it is. `ParseOptions.strict` and the "Keeping PHI out of logs" guide now say so where the
+  decision is made. Review the two paths separately.
+
 - **🩺 Repeated de-identification was not a fixed point: `(0012,0063)` grew by the whole method
   string on every pass, for any `deidentificationMethod` ending in a SPACE or a NUL**
   (`DICOM-DEIDENT-NOT-A-FIXED-POINT`). **`INTRODUCED` by the entry below and never published** - it

package/README.md

@@ -373,7 +373,7 @@ Thrown by `serializeDicom` for `MISSING_TRANSFER_SYNTAX` (the dataset names no t
 - **`RetainSafePrivate` retains nothing inside ANY Item of a sequence whose item stream over-runs its own declared length.** An Item that declares more bytes than its enclosing Sequence does absorbs whatever follows the sequence, so a private data element the sender wrote at the **root** - with no Private Creator reserving its block there - can land beside a creator that is genuine Item content and be kept verbatim. Measured on the published `0.0.8` tarball it was, with `report.removedPrivateTags` reading `[]`, the object stamped `(0012,0062) Patient Identity Removed = YES` and `ds.warnings` empty: a stamp that outran the redaction. (**There is no `0.0.9` on the registry** - `package.json` carried it and the publish never happened, an npm `E404` - so earlier wording here that said "through `0.0.9`" named a version that does not exist. Measured on the current published `0.0.10`, this direction is fixed.) PS3.5 2026c §7.8.1 scopes a private block reservation to the Item ("The scope of the reservation is just within the Item"), so a file that contradicts itself about where the Item ends does not determine which reservation covers an element; PS3.15 2026c §E.3.10 retains only what is **known** safe and requires that "all other Private Attributes shall be removed **or processed in the element-specific manner recommended by Deidentification Action (0008,0307), if present within Private Data Element Characteristics Sequence (0008,0300)**" (a two-branch clause; this library does not implement `(0008,0307)`, so removal is the branch available to it). So every private element the recursion **reaches** inside such a sequence, at every depth below it, is removed and named in `report.removedPrivateTags`. **The unit is the SEQUENCE, not the individual Item**: `descendSequence` decides once and applies it to every item, so a two-item sequence where only item 1 over-runs also loses a genuine block sitting in the honest item 0 (measured; the grid sweeps single-item fixtures, so that extra cost is NOT inside the priced 20). **One route is not closed and still leaks, measured and pinned:** a **private `SQ`** the profile vouches for is kept verbatim before the descent runs, which is the carve-out named two bullets above. **The direction is not fail-safe on its own and the opposite claim is retracted**: it depends on where the sender put the Private Creator. Both **absorb** placements are pinned by tests, and so is the **eject** placement (below). Treat `DICOM_ITEM_CROSSES_SEQUENCE_END` as "this file's structure is in dispute", never as a statement about what was retained. **The price is measured, not asserted**: on `scripts/measure-sq-bound-grid.ts`, 58 cells lose a private value and **20 of them were not leaking anything** (the creator and the data element were both genuine Item content). Retention on a file that does not contradict itself is unchanged, and **no reading changes** - no parser file is touched. This is not a parser bound: an over-declaring Item and an under-declaring Sequence are byte-identical, so nothing on the wire can separate them.
 - **`RetainSafePrivate` also retains nothing a Data Set holds AFTER a sequence whose own contents contradict the extent it declared.** The mirror direction: an Item that _under_-declares **ejects** its trailing elements out into the enclosing Data Set, and a Private Creator that lands there reserves a block for elements the sender never put beside it - the same false attestation. **Measured on the published `0.0.10` tarball, which is the current registry version**: `report.removedPrivateTags` `[]`, the value in the serialized output, `(0012,0062) = YES`, `ds.warnings` empty and no throw under `{ strict: true }`. It applies **in every Data Set, not only the root**: an inner sequence ejecting a creator into the still-usable Item that encloses it is measured and pinned. The cut is **positional**, so a reservation the sender wrote **ahead of** the offending sequence is untouched, and the same reservation written **after** it is refused. **Two predicates, because the parser records the same contradiction two ways**: under Explicit VR the item stream over-runs and the span shows up as `rawBytes.length > length`; under Implicit VR LE that path slices the item stream, so nothing over-runs at all and the descent is refused instead (`DICOM_SQ_NOT_DESCENDED`, `Element.items === undefined`). The second predicate is **broader than the ejection it is here for, deliberately**: it says the parser could not walk the sequence, which other unwalkable item streams also reach. **And two BOUNDS, not one, because a Data Set is a `Map<Tag, Element>`**: an ejected element whose tag the Data Set already holds overwrites in place and inherits the earlier element's position, ahead of the sequence it came out of, so `Element.byteOffset` is checked beside the index. **That collision also destroys the root's own value at parse time** - `PRE-EXISTING`, unchanged here, and pinned by a test so it is not mistaken for something this rule handles. It was silent when that was written; `DICOM_DUPLICATE_TAG_IN_DATA_SET` reports it now (below), and the destruction itself is still unchanged. **The price is measured**: on `scripts/measure-sq-bound-grid.ts` all **22** cells that retained a private value at the root on a self-contradicting file whose honest control removes it go to **0**, at the cost of **56** further root retentions on self-contradicting files whose honest control does keep the value (the column is `de-identified OUTPUT lost a marker (cost)`, reading 78; the parse-tree `LOST`/`GAINED a marker value` counters beside it read 0). Retention on a file that does not contradict itself is unchanged (9 at the root, 6 in an Item), and **no reading changes**: 0 cells differ in any parse respect, 0 cells whose reading differs, and no parser file is touched.
 - **An over-declared Value Length into a binary carrier still leaks, and it is a known trade.** The mirror case - an _over_-declared length that swallows the following element into an `OB`/`OW`/`US`/`UN` value - is **not** detected, and a `(0010,0020)` inside it reaches de-identified output with no warning and no report entry. Arbitrary bytes are exactly what those VRs are for, so no content test can decide it; the only candidate remedy was measured and empties conformant binary values (a legal LUT or blob deleted because 8 of its bytes read as a zero-length `(0010,0020)`). String carriers **are** covered, because there the same bytes are provably outside the VR's repertoire. If you accept files from a sender you do not control, treat a binary attribute's length as untrusted.
-- **`(0012,0063)` De-identification Method is ADDED TO, so a prior de-identifier's text is carried into the output.** PS3.15 2026c §E.1.1 says a text string describing the method "shall be inserted in or added to" that attribute, and this library used to **replace** it - destroying the provenance chain the attribute exists to carry, silently, on a file whose earlier pass may be the one a recipient was relying on. It now appends its own text as a further value of the `1-n` attribute, after a `\`, copying the prior bytes through verbatim. **The method string is itself a `1-n` value**: it is split on `\` and only the values not already recorded are added. **Trailing SPACE and NUL are padding rather than content** (PS3.5 2026c Table 6.2-1's `LO` row, which describes a **Value**, and `LO` is `1-n`), so they are ignored **per value, on both sides** when your string is matched against what is already recorded, and trimmed from the value written. Repeated de-identification is therefore a fixed point **from the first pass**, for every string: with or without a delimiter, and with a pad byte on any value, last or not. Leading spaces are yours and are written through untouched; a method string that is padding only records nothing. **One bound, and it is not cosmetic**: `LO` is a short-form VR whose Value Length field is 16 bits, so when the join would not fit, the prior value is **replaced** rather than added to and `report.warnings` carries `DICOM_DEIDENT_METHOD_NOT_ADDED`. Read that code as "the length ceiling was reached", never as "every fallback is disclosed": a `(0012,0063)` a file encoded under a VR other than `LO` is also replaced, and that one is silent (`PRE-EXISTING`). An element the serializer cannot encode would take the whole de-identified object down, and truncating the sender's earlier records instead would be a policy the standard does not state. A `deidentificationMethod` longer than that ceiling on its own is unchanged from every earlier release and still fails to serialize. **The cost is stated rather than glossed**: `(0012,0063)` is **not in Table E.1-1**, so the Basic Profile keeps it exactly as it keeps every other unlisted attribute, and a sender who wrote something identifying into it now sees that text in de-identified output where the replacement used to remove it as a side effect. That is the retained-by-omission posture of every unlisted attribute, not a rule this insertion states; it is pinned by a residual test that asserts the cost. **It is not silent**: `report.warnings` carries `DICOM_DEIDENT_METHOD_PRIOR_RETAINED` whenever prior bytes from the source file survive into `(0012,0063)`, because an object stamped `(0012,0062) = YES` carrying text nothing in the run inspected is a stamp that outran the redaction. It is **not** on `report.retained` - that field lists the Annex E option sets active for the run, and a kept attribute is not one of them. `(0012,0062)` Patient Identity Removed is still **replaced** with `YES` - the same paragraph of §E.1.1 uses the other verb for it ("shall be replaced or added to"), and the asymmetry is the standard's. A `(0012,0063)` a file encoded under a VR other than `LO` is replaced rather than appended to.
+- **`(0012,0063)` De-identification Method is ADDED TO, so a prior de-identifier's text is carried into the output.** PS3.15 2026c §E.1.1 says a text string describing the method "shall be inserted in or added to" that attribute, and this library used to **replace** it - destroying the provenance chain the attribute exists to carry, silently, on a file whose earlier pass may be the one a recipient was relying on. It now appends its own text as a further value of the `1-n` attribute, after a `\`, copying the prior bytes through verbatim. **The method string is itself a `1-n` value**: it is split on `\` and only the values not already recorded are added. **Trailing SPACE and NUL are padding rather than content** (PS3.5 2026c Table 6.2-1's `LO` row, which describes a **Value**, and `LO` is `1-n`), so they are ignored **per value, on both sides** when your string is matched against what is already recorded, and trimmed from the value written. Repeated de-identification is therefore a fixed point **from the first pass**, for every string: with or without a delimiter, and with a pad byte on any value, last or not. Leading spaces are yours and are written through untouched; a method string that is padding only records nothing. **One bound, and it is not cosmetic**: `LO` is a short-form VR whose Value Length field is 16 bits, so when the join would not fit, the prior value is **replaced** rather than added to and `report.warnings` carries `DICOM_DEIDENT_METHOD_NOT_ADDED`. Read that code as "the length ceiling was reached", never as "every fallback is disclosed": a `(0012,0063)` a file encoded under a VR other than `LO` is also replaced, and that one has its own code, `DICOM_DEIDENT_METHOD_NOT_LO`. An element the serializer cannot encode would take the whole de-identified object down, and truncating the sender's earlier records instead would be a policy the standard does not state. A `deidentificationMethod` longer than that ceiling on its own is unchanged from every earlier release and still fails to serialize. **The cost is stated rather than glossed**: `(0012,0063)` is **not in Table E.1-1**, so the Basic Profile keeps it exactly as it keeps every other unlisted attribute, and a sender who wrote something identifying into it now sees that text in de-identified output where the replacement used to remove it as a side effect. That is the retained-by-omission posture of every unlisted attribute, not a rule this insertion states; it is pinned by a residual test that asserts the cost. **It is not silent**: `report.warnings` carries `DICOM_DEIDENT_METHOD_PRIOR_RETAINED` whenever prior bytes from the source file survive into `(0012,0063)`, because an object stamped `(0012,0062) = YES` carrying text nothing in the run inspected is a stamp that outran the redaction. It is **not** on `report.retained` - that field lists the Annex E option sets active for the run, and a kept attribute is not one of them. `(0012,0062)` Patient Identity Removed is still **replaced** with `YES` - the same paragraph of §E.1.1 uses the other verb for it ("shall be replaced or added to"), and the asymmetry is the standard's. A `(0012,0063)` a file encoded under a VR other than `LO` is replaced rather than appended to, and `report.warnings` now carries `DICOM_DEIDENT_METHOD_NOT_LO` when it is: the join concatenates `LO` values with the `5CH` delimiter, which is not defined over arbitrary octets, so replacing is deliberate and only the silence was the defect. **And the method text this library writes for itself is now multi-valued, one Value for the Profile and one per active Option.** PS3.5 2026c Table 6.2-1 caps an `LO` at **64 characters per Value** and `(0012,0063)` is `1-n`; the single-value string it wrote before measured **76** characters with no options, **130** with three and **272** with all nine, so every object de-identified without a caller-supplied method, in every release up to and including this one, carried a value no `LO` may legally hold - in the one attribute a strict receiver reads to decide whether the object was de-identified at all. **Re-de-identifying such an object keeps that 76-character Value**, because a prior record is added to rather than rewritten (measured: a flat 138 bytes over four passes, values of 76 and 61, `DICOM_DEIDENT_METHOD_PRIOR_RETAINED` raised for the retention and nothing said about the length). Your own `deidentificationMethod` is **not** bounded for you either: split it on `\` yourself if a strict receiver is in your path.
 - **Networking & web.** No DIMSE (C-STORE/FIND/MOVE, MWL, MPPS); no DICOMweb (QIDO/WADO/STOW). → `@cosyte/dicom-net`, `@cosyte/dicomweb`.
 - **Transcoding.** No transfer-syntax conversion. The serializer re-emits in the dataset's source syntax only.
 - **Terminology resolution.** Coded values are surfaced (designator + canonical source) but not validated against SNOMED/LOINC/etc.

package/dist/index.cjs

@@ -11,7 +11,7 @@ var __export = (target, all) => {
 };
 
 // src/version.ts
-var VERSION = "0.0.11";
+var VERSION = "0.0.12";
 
 // src/dictionary/index.ts
 var dictionary_exports = {};
@@ -11748,6 +11748,8 @@ var WARNING_CODES = {
   // emitted by deidentify(), never by the parser
   DICOM_DEIDENT_METHOD_NOT_ADDED: "DICOM_DEIDENT_METHOD_NOT_ADDED",
   // emitted by deidentify(), never by the parser
+  DICOM_DEIDENT_METHOD_NOT_LO: "DICOM_DEIDENT_METHOD_NOT_LO",
+  // emitted by deidentify(), never by the parser
   DICOM_DEIDENT_METHOD_PRIOR_RETAINED: "DICOM_DEIDENT_METHOD_PRIOR_RETAINED",
   // emitted by deidentify(), never by the parser
   DICOM_DEIDENT_SEQUENCE_NOT_AUDITABLE: "DICOM_DEIDENT_SEQUENCE_NOT_AUDITABLE",
@@ -11797,6 +11799,11 @@ var WARNING_MESSAGES = Object.freeze({
   // constant of this code rather than composed from input, as in the two
   // (0002,0000) messages above.
   DICOM_DEIDENT_METHOD_NOT_ADDED: "The De-identification Method this run recorded could not be added beside the value (0012,0063) already carried without exceeding the largest Value Length that VR can encode, so the earlier value was replaced (PS3.15 E.1.1). The replaced text is not in the output.",
+  // No value, no length AND NO VR. The VR is two bytes read from the file, and a
+  // fabricated header makes those two bytes document content - the shape that
+  // put "ITHS" into DICOM_NONZERO_RESERVED_BYTES. Naming the VR here would read
+  // as harmless and is exactly the same defect.
+  DICOM_DEIDENT_METHOD_NOT_LO: "The (0012,0063) value the source file carried is encoded under a Value Representation other than LO, which is not a De-identification Method this run can add its own text to (PS3.15 E.1.1), so the earlier value was replaced. The replaced text is not in the output. The text and the VR are both withheld from this message.",
   // No value and no length, for the same reason as the code above: the retained
   // text is the file's own. The tag is a constant of this code.
   DICOM_DEIDENT_METHOD_PRIOR_RETAINED: "A De-identification Method (0012,0063) value the source file already carried was kept beside the one this run recorded, as PS3.15 E.1.1 requires. That attribute is not in Table E.1-1, so no rule in this run inspected, audited or redacted those bytes: if the sender wrote identifying text there it is in the de-identified output, under (0012,0062) = YES. The text is withheld from this message.",
@@ -11915,6 +11922,9 @@ function burnedInAnnotationNotRemoved(position) {
 function deidentMethodNotAdded(position) {
   return build(WARNING_CODES.DICOM_DEIDENT_METHOD_NOT_ADDED, position);
 }
+function deidentMethodNotLo(position) {
+  return build(WARNING_CODES.DICOM_DEIDENT_METHOD_NOT_LO, position);
+}
 function deidentMethodPriorRetained(position) {
   return build(WARNING_CODES.DICOM_DEIDENT_METHOD_PRIOR_RETAINED, position);
 }
@@ -15351,19 +15361,23 @@ function rebuildFileMeta(fileMeta, ctx) {
     mediaStorageSOPInstanceUID: ctx.remap.map(fileMeta.mediaStorageSOPInstanceUID)
   };
 }
+var DEFAULT_METHOD_PROFILE = "@cosyte/dicom Basic Application Level Confidentiality Profile";
 function defaultMethod(active) {
-  const base = "Cosyte @cosyte/dicom: PS3.15 Basic Application Level Confidentiality Profile";
-  return active.size === 0 ? base : `${base} + ${[...active].join(", ")}`;
+  const options = DEIDENTIFY_OPTIONS.filter((option) => active.has(option));
+  return [DEFAULT_METHOD_PROFILE, ...options].join(String.fromCharCode(VALUE_DELIMITER));
 }
 var VALUE_DELIMITER = 92;
 var MAX_SHORT_FORM_VALUE_BYTES = 65534;
 function addDeidentificationMethod(existing, method) {
   const added = trimTrailingPad(buffer.Buffer.from(method, "latin1"));
-  if (existing === void 0 || existing.vr !== "LO") {
-    return { value: added, replacedPrior: false, retainedPrior: false };
+  const none = { replacedPrior: false, retainedPrior: false, replacedNonLoPrior: false };
+  if (existing === void 0) return { value: added, ...none };
+  if (existing.vr !== "LO") {
+    const lost = trimTrailingPad(existing.rawBytes).length > 0;
+    return { value: added, ...none, replacedNonLoPrior: lost };
   }
   const kept = buffer.Buffer.from(trimTrailingPad(existing.rawBytes));
-  if (kept.length === 0) return { value: added, replacedPrior: false, retainedPrior: false };
+  if (kept.length === 0) return { value: added, ...none };
   const keptValues = splitValues(kept).map(trimTrailingPad);
   const missing = added.length === 0 ? [] : splitValues(added).filter((v) => !keptValues.some((k) => k.equals(trimTrailingPad(v))));
   const parts = [kept];
@@ -15372,9 +15386,9 @@ function addDeidentificationMethod(existing, method) {
   }
   const value = trimTrailingPad(buffer.Buffer.concat(parts));
   if (value.length > MAX_SHORT_FORM_VALUE_BYTES) {
-    return { value: added, replacedPrior: true, retainedPrior: false };
+    return { value: added, ...none, replacedPrior: true };
   }
-  return { value, replacedPrior: false, retainedPrior: true };
+  return { value, ...none, retainedPrior: true };
 }
 function trimTrailingPad(value) {
   let end = value.length;
@@ -15446,6 +15460,11 @@ function deidentify(ds, options = {}) {
       deidentMethodPriorRetained({ byteOffset: priorMethod?.byteOffset ?? 0, fileMeta: false })
     );
   }
+  if (deidentMethod.replacedNonLoPrior) {
+    warnings.push(
+      deidentMethodNotLo({ byteOffset: priorMethod?.byteOffset ?? 0, fileMeta: false })
+    );
+  }
   if (hasUncleanedBurnedIn(ds)) {
     const offset = ds.get(TAG_PIXEL_DATA)?.byteOffset ?? 0;
     warnings.push(burnedInAnnotationNotRemoved({ byteOffset: offset, fileMeta: false }));