dicom-dre 1.0.0.dev2__tar.gz → 1.0.0.dev3__tar.gz

dicom_dre-1.0.0.dev3/CHANGELOG.md

@@ -0,0 +1,23 @@
+# CHANGELOG
+
+<!-- version list -->
+
+## v1.0.0-dev.2 (2026-07-24)
+
+### Documentation
+
+- Initial rewrite from source repo
+  ([`80e26c9`](https://github.com/susom/dicom-dre/commit/80e26c99e5f270528dfa2ddb8551b0baf3fdc7b7))
+
+- Scaffolding
+  ([`c1db56a`](https://github.com/susom/dicom-dre/commit/c1db56a224050b8704f82ac99b857d6c0c43690f))
+
+### Performance Improvements
+
+- Redactor caching
+  ([`ce14989`](https://github.com/susom/dicom-dre/commit/ce14989512d0d688d6b726444c84f91cafc28757))
+
+
+## v1.0.0-dev.1 (2026-07-24)
+
+- Initial Release

{dicom_dre-1.0.0.dev2 → dicom_dre-1.0.0.dev3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dicom-dre
-Version: 1.0.0.dev2
+Version: 1.0.0.dev3
 Summary: DICOM De-identification & Redaction Engine
 Project-URL: repository, https://github.com/susom/dicom-dre
 Author-email: Stanford <people@stanford.edu>
@@ -25,7 +25,7 @@ Description-Content-Type: text/markdown
 
 # DICOM-DRE
 
-A fast, deterministic DICOM de-identification and redaction engine.
+A fast, reproducible DICOM de-identification and redaction engine.
 
 `dicom-dre` removes protected health information (PHI) from DICOM instances in
 two places where it commonly hides:
@@ -39,22 +39,28 @@ two places where it commonly hides:
 - **Free-text metadata PHI.** Description fields that frequently carry PHI
   (`SeriesDescription`, `StudyDescription`, `ProtocolName`) are redacted
   token-by-token against an allowlist: any token not on the allowlist is masked,
-  while known clinical terms pass through.
+  while known clinical terms pass through. This approach works because the
+  distinct values in these short description fields are overwhelmingly derived
+  from structured entries in the source PACS, so a finite allowlist covers the
+  legitimate vocabulary. It is not intended for paragraphs of prose, general
+  clinical notes, or other entirely free-form fields with no discernible
+  pattern, where the allowlist model cannot bound the vocabulary or reliably
+  de-identify the text.
 
 Instance metadata is also scrubbed against a configurable de-identification
 profile, and instance/study/series UIDs are deterministically re-derived, so the
 same input plus the same parameters always yields the same output.
 
-## Deterministic by design
+## Reproducible by design
 
 The engine performs no hashing lookups, no network calls, and no randomization
 of its own. De-identification parameters (patient ID, accession number, UID
-root, salt, etc.) are consumed exactly as supplied. Given identical inputs and
-parameters, output is byte-reproducible. Callers are responsible for supplying
-already-hashed or already-mapped identifier values.
+root, salt, jitter, etc.) are consumed exactly as supplied. Given identical
+inputs and parameters, output is byte-for-byte reproducible. Callers are
+responsible for supplying already-hashed or already-mapped identifier values.
 
-This determinism is guarded by a regression test suite built from sampled DICOM
-studies. Each fixture records the technical matching tags and the expected
+This reproducibility is guarded by a regression test suite built from sampled
+DICOM studies. Each fixture records the technical matching tags and the expected
 catalog filtering and pixel-scrub decisions (with no PHI or pixel data), and the
 tests assert that the engine still reaches the recorded decision for every case,
 so catalog or profile changes that alter existing outcomes are caught.
@@ -151,13 +157,14 @@ site's vocabulary.
 ## Library usage
 
 ```python
-from dicom_dre import deidentify_paths, ProfileSpec
+from dicom_dre import DeidParameters, ProfileSpec, deidentify_paths
 
 for item in deidentify_paths(
     sources=["studies/"],
     output_dir="out/",
     recursive=True,
-    profile_spec=ProfileSpec(name="default", parameters={"PATIENT_ID": "TEST"}),
+    profile_spec=ProfileSpec(name="default"),
+    parameters=DeidParameters(patient_id="TEST"),
 ):
     print(item.input_file, item.result.outcome)
 ```

{dicom_dre-1.0.0.dev2 → dicom_dre-1.0.0.dev3}/README.md

@@ -1,6 +1,6 @@
 # DICOM-DRE
 
-A fast, deterministic DICOM de-identification and redaction engine.
+A fast, reproducible DICOM de-identification and redaction engine.
 
 `dicom-dre` removes protected health information (PHI) from DICOM instances in
 two places where it commonly hides:
@@ -14,22 +14,28 @@ two places where it commonly hides:
 - **Free-text metadata PHI.** Description fields that frequently carry PHI
   (`SeriesDescription`, `StudyDescription`, `ProtocolName`) are redacted
   token-by-token against an allowlist: any token not on the allowlist is masked,
-  while known clinical terms pass through.
+  while known clinical terms pass through. This approach works because the
+  distinct values in these short description fields are overwhelmingly derived
+  from structured entries in the source PACS, so a finite allowlist covers the
+  legitimate vocabulary. It is not intended for paragraphs of prose, general
+  clinical notes, or other entirely free-form fields with no discernible
+  pattern, where the allowlist model cannot bound the vocabulary or reliably
+  de-identify the text.
 
 Instance metadata is also scrubbed against a configurable de-identification
 profile, and instance/study/series UIDs are deterministically re-derived, so the
 same input plus the same parameters always yields the same output.
 
-## Deterministic by design
+## Reproducible by design
 
 The engine performs no hashing lookups, no network calls, and no randomization
 of its own. De-identification parameters (patient ID, accession number, UID
-root, salt, etc.) are consumed exactly as supplied. Given identical inputs and
-parameters, output is byte-reproducible. Callers are responsible for supplying
-already-hashed or already-mapped identifier values.
+root, salt, jitter, etc.) are consumed exactly as supplied. Given identical
+inputs and parameters, output is byte-for-byte reproducible. Callers are
+responsible for supplying already-hashed or already-mapped identifier values.
 
-This determinism is guarded by a regression test suite built from sampled DICOM
-studies. Each fixture records the technical matching tags and the expected
+This reproducibility is guarded by a regression test suite built from sampled
+DICOM studies. Each fixture records the technical matching tags and the expected
 catalog filtering and pixel-scrub decisions (with no PHI or pixel data), and the
 tests assert that the engine still reaches the recorded decision for every case,
 so catalog or profile changes that alter existing outcomes are caught.
@@ -126,13 +132,14 @@ site's vocabulary.
 ## Library usage
 
 ```python
-from dicom_dre import deidentify_paths, ProfileSpec
+from dicom_dre import DeidParameters, ProfileSpec, deidentify_paths
 
 for item in deidentify_paths(
     sources=["studies/"],
     output_dir="out/",
     recursive=True,
-    profile_spec=ProfileSpec(name="default", parameters={"PATIENT_ID": "TEST"}),
+    profile_spec=ProfileSpec(name="default"),
+    parameters=DeidParameters(patient_id="TEST"),
 ):
     print(item.input_file, item.result.outcome)
 ```

dicom_dre-1.0.0.dev3/docs/about/changelog.md

@@ -0,0 +1,8 @@
+# Changelog
+
+python-semantic-release generates the changelog at the repository root from
+Conventional Commit messages.
+
+```{include} ../../CHANGELOG.md
+:start-line: 2
+```

{dicom_dre-1.0.0.dev2 → dicom_dre-1.0.0.dev3}/docs/about/history.md

@@ -2,43 +2,43 @@
 
 `dicom-dre` grew out of a de-identification workflow built on the MIRC CTP
 (Clinical Trial Processor) engine. This page records that background. It is not
-a reference for configuring the engine; the behavior described here has been
-replaced by the device catalog and profiles documented elsewhere in these docs.
+a reference for configuring the engine; the device catalog and profiles
+documented elsewhere in these docs have replaced the behavior described here.
 
 ## From CTP scripts to a device catalog
 
 The original workflow controlled DICOM image handling with two CTP script files
 and an XML metadata template:
 
-- **filter.script** decided which images were accepted for processing. Images
-  matching a filter rule passed through; images matching no rule were
-  quarantined.
+- **filter.script** decided which images to accept for processing. Images
+  matching a filter rule passed through; the engine quarantined images matching
+  no rule.
 - **pixel.script** defined rectangular regions to blank for images with
   burned-in PHI annotations.
 
-Both scripts used a Java boolean expression syntax over DICOM attributes, with
-filter rules keyed on manufacturer, model, modality, software version, and image
-dimensions, and pixel rules pairing a matching signature with one or more
+Both scripts used a Java boolean expression syntax over DICOM attributes. Filter
+rules keyed on manufacturer, model, modality, software version, and image
+dimensions. Pixel rules paired a matching signature with one or more
 `(x, y, width, height)` scrub regions.
 
 Two problems motivated the rewrite:
 
 - The filter and pixel scripts encoded the same device signatures twice, once to
   decide acceptance and again to place scrub regions, so the two files drifted
-  apart as devices were added.
+  apart as maintainers added devices.
 - The expression syntax was code rather than data, which made the rule set hard
   to inspect, test, and validate.
 
 `dicom-dre` replaces both script files with a single Python device catalog. Each
 catalog entry describes one device and binds its acquisition variants directly
-to the pixel regions that must be blanked, so a device's filter decision and its
-scrub regions are defined together. See [Device catalog](../concepts/device-catalog.md).
+to the pixel regions to blank, so each device defines its filter decision and
+its scrub regions together. See [Device catalog](../concepts/device-catalog.md).
 
 ## From an XML template to profiles
 
-Metadata handling that was previously expressed in a CTP anonymizer template is
-now expressed as de-identification profiles. See [Profiles](../concepts/profiles.md)
-for the profiles shipped in this repository.
+De-identification profiles now express the metadata handling that a CTP
+anonymizer template previously expressed. See
+[Profiles](../concepts/profiles.md) for the profiles shipped in this repository.
 
 ## Acknowledgement
 

{dicom_dre-1.0.0.dev2 → dicom_dre-1.0.0.dev3}/docs/about/provenance.md

@@ -1,10 +1,10 @@
 # Provenance and Portability
 
-The bundled device catalog and free-text allowlist were derived from studies on
-a single PACS at one medical research center. The catalog's device rules, pixel
-scrub regions, and allowlisted vocabulary reflect the scanner fleet and
-reporting conventions observed there. They are unlikely to be complete or
-correct for a different site.
+The bundled device catalog and free-text allowlist come from studies on a single
+PACS at one medical research center. The catalog's device rules, pixel scrub
+regions, and allowlisted vocabulary reflect the scanner fleet and reporting
+conventions seen there. They are unlikely to be complete or correct for a
+different site.
 
 ## What is site-specific
 
@@ -12,21 +12,22 @@ The following ship with `dicom-dre` but encode assumptions about one site:
 
 - **Device rules.** Each catalog entry matches a specific device by
   manufacturer, model, modality, software version, and image dimensions. Devices
-  absent from the observed fleet have no entry and will not be recognized.
+  absent from the observed fleet have no entry, and the engine will not
+  recognize them.
 - **Pixel scrub regions.** Blanking coordinates are fixed per device and
-  acquisition variant. A device that burns in text at a different location, or a
-  firmware revision that moves the overlay, will not be scrubbed correctly.
+  acquisition variant. If a device burns in text at a different location, or a
+  firmware revision moves the overlay, the engine will not scrub it correctly.
 - **Free-text allowlist.** The allowlisted vocabulary reflects the reporting
-  conventions and clinical terms seen in one site's description fields. Local
-  vocabulary that is absent from the allowlist is masked; local PHI patterns not
-  anticipated by the allowlist may pass through.
+  conventions and clinical terms seen in one site's description fields. The
+  engine masks local vocabulary that is absent from the allowlist; local PHI
+  patterns the allowlist does not anticipate may pass through.
 
 ## Consequences of an unvalidated deployment
 
 - **Unmatched or mismatched devices** do not have their burned-in text blanked.
-  An instance from an unknown device is filtered rather than emitted with
-  unscrubbed PHI, but a partially matching rule with wrong coordinates can emit
-  an instance whose overlay is not fully covered.
+  The engine filters an instance from an unknown device rather than emitting it
+  with unscrubbed PHI, but a partially matching rule with wrong coordinates can
+  emit an instance whose overlay is not fully covered.
 - **Over-redaction** masks legitimate terms when the allowlist omits vocabulary
   used at your site, degrading the usefulness of description fields.
 - **Under-redaction** leaks PHI when tokens that should be masked resemble
@@ -35,12 +36,12 @@ The following ship with `dicom-dre` but encode assumptions about one site:
 ## Before relying on `dicom-dre` elsewhere
 
 - Validate pixel scrubbing against your own devices. Confirm that each device
-  you process either matches a catalog entry with correct scrub regions or is
-  intentionally filtered.
+  you process either matches a catalog entry with correct scrub regions or that
+  the engine intentionally filters it.
 - Review and extend the allowlist for your local vocabulary to reduce
   over-redaction and under-redaction.
 - Run the regression suite after any catalog or allowlist change to confirm that
-  existing decisions are preserved.
+  existing decisions still hold.
 
 The [Local validation](../guides/local-validation.md) guide describes these
 steps in detail. The [Extending the catalog](../guides/extending-the-catalog.md)

{dicom_dre-1.0.0.dev2 → dicom_dre-1.0.0.dev3}/docs/about/testing.md

@@ -51,12 +51,12 @@ package, runs the suite, and restores the extension afterward.
 reaches the recorded filtering and pixel-scrub decision for each sampled case.
 The fixtures record only the technical matching tags and the expected decision,
 with no PHI or pixel data. Run this suite after any catalog or profile change so
-that alterations to existing outcomes are caught.
+that it catches alterations to existing outcomes.
 
 ## Writing tests
 
 - Place tests in `tests/unit/`, mirroring the source structure.
-- Use descriptive test names that state what is being verified.
+- Use descriptive test names that state what each test verifies.
 - Mock external services in unit tests, especially any that would make network
   calls.
 - Use fixtures for common setup.

{dicom_dre-1.0.0.dev2 → dicom_dre-1.0.0.dev3}/docs/concepts/architecture.md

@@ -8,18 +8,18 @@ it produces one of three outcomes by running three stages in sequence.
 
 For each input DICOM file the engine runs three stages:
 
-1. **Filter** — a device catalog decides whether the image is kept or rejected.
-2. **Pixel scrub** — burned-in text regions identified by the catalog are
-   blanked.
-3. **Metadata** — DICOM tags are rewritten (PHI removed, UIDs re-derived, dates
-   shifted or kept).
+1. **Filter** — a device catalog decides whether to keep or reject the image.
+2. **Pixel scrub** — the engine blanks the burned-in text regions the catalog
+   identified.
+3. **Metadata** — the engine rewrites DICOM tags: it removes PHI, re-derives
+   UIDs, and shifts or keeps dates.
 
 Each run returns a {py:class}`~dicom_dre.result.DeidentifyResult` whose `outcome`
 is one of three {py:class}`~dicom_dre.result.Outcome` values:
 
 | Outcome | Cause |
 |---------|-------|
-| `DEIDENTIFIED` | The file passed the filter and was written out. |
+| `DEIDENTIFIED` | The file passed the filter, and the engine wrote it out. |
 | `FILTERED` | A device or exclusion rule rejected the file; the result carries the matched reason. |
 | `QUARANTINED` | Processing raised an exception; the result carries the error text. |
 
@@ -39,8 +39,8 @@ flowchart LR
     F -.->|exception| Q["QUARANTINED"]
 ```
 
-An exception raised in any stage is caught by `deidentify_file` and returned as
-a `QUARANTINED` result rather than propagating to the caller.
+An exception raised in any stage does not propagate to the caller.
+`deidentify_file` catches it and returns a `QUARANTINED` result instead.
 
 ## Stages
 
@@ -57,23 +57,27 @@ See [Device Catalog](device-catalog.md).
 ### Pixel scrub
 
 When the decision carries scrub regions,
-{py:func}`dicom_dre.pixel_blanker.blank_regions` blanks them. JPEG Baseline images
-are edited directly in the compressed bitstream with no re-encoding; all other
-transfer syntaxes are decoded with pydicom and numpy, blanked, and written as
-uncompressed Explicit VR Little Endian.
+{py:func}`dicom_dre.pixel_blanker.blank_regions` blanks them. The engine edits
+JPEG Baseline images directly in the compressed bitstream with no re-encoding.
+For all other transfer syntaxes, it decodes with pydicom and numpy, blanks the
+regions, and writes uncompressed Explicit VR Little Endian.
 
 See [JPEG DCT Scrubbing](jpeg-dct.md).
 
 ### Metadata
 
 The bound {py:class}`~dicom_dre.profile.DeidProfile` runs its tag rules on the
-dataset with `DeidProfile.apply(ds)`. Rules are small closures defined in
+dataset with `DeidProfile.apply(ds, params)`, where `params` is a
+{py:class}`~dicom_dre.parameters.DeidParameters` carrying the per-patient values.
+Rules are small closures defined in
 {py:mod}`dicom_dre.actions` (for example `remove`, `empty`, `hash_uid`,
-`jitter_date`, `keep`); the tag sets each profile applies them to are composed
-by the profile factories in {py:mod}`dicom_dre.profiles`.
+`jitter_date`, `keep`). The profile factories in {py:mod}`dicom_dre.profiles`
+compose the tag sets each profile applies them to.
 
-Each factory in `actions.py` closes over its arguments at profile-construction
-time and returns a `(Dataset, BaseTag) -> None` callable. The available actions:
+Each factory in `actions.py` closes over its build-time arguments at
+profile-construction time and returns a
+`(Dataset, BaseTag, DeidParameters) -> None` callable that reads per-patient
+values from the supplied parameters. The available actions:
 
 | Action | Effect |
 |--------|--------|
@@ -81,9 +85,9 @@ time and returns a `(Dataset, BaseTag) -> None` callable. The available actions:
 | `remove()` | Delete the element from the dataset. |
 | `empty()` | Replace the value with a zero-length value (empty list for `SQ`). |
 | `set_value(value, create_if_missing=False)` | Replace the value with a literal string, optionally creating the element when absent. |
-| `redact_text(redactor)` | Redact a free-text element token-by-token against an allowlist. |
-| `hash_uid(root, salt="")` | Re-derive a UID as an MD5 hash under `root`, optionally salted. |
-| `jitter_date(days)` | Shift the `DA`/`DT` date component forward by `days`, preserving any time/timezone remainder. |
+| `set_param(field, default=None, fallback_field=None, create_if_missing=False)` | Write a per-patient value read from `DeidParameters`, with fallbacks. |
+| `hash_uid(root, use_study_salt=False)` | Re-derive a UID as an MD5 hash under `root`, optionally salted with the study identifier. |
+| `jitter_date()` | Shift the `DA`/`DT` date component forward by `params.jitter` days, preserving any time/timezone remainder. |
 | `append_value(text, create_if_missing=False)` | Append `text` to a multi-valued element using `\` as separator. |
 | `cap_age(threshold, replacement)` | Replace an `AS` age when its numeric part exceeds `threshold`. |
 | `if_exists(inner)` | Apply `inner` only when the element is present. |
@@ -97,12 +101,12 @@ By default the engine removes every private data element that has no explicit
 tag rule. A device rule may name a small, reviewer-approved set of private
 elements to retain verbatim through its `preserved_private_tags` field (a tuple
 of `PrivateTagSpec`). When the catalog decision carries specs, the pipeline
-attaches them to the `DeidProfile`, and `DeidProfile.apply(ds)` resolves each
-spec's private-creator block at apply time (the block is runtime-assigned, not
-fixed) and keeps both the resolved data elements and their creator element. All
-other private elements are removed as usual. When preservation is active the
-engine also stamps the De-identification Method Code Sequence `(0012,0064)` with
-the Retain Safe Private Option; the sequence is emitted only for files that
+attaches them to the `DeidProfile`. At apply time, `DeidProfile.apply(ds, params)`
+resolves each spec's private-creator block (the block is runtime-assigned, not
+fixed) and keeps both the resolved data elements and their creator element. The
+engine removes all other private elements as usual. When preservation is active,
+the engine also stamps the De-identification Method Code Sequence `(0012,0064)`
+with the Retain Safe Private Option. It emits the sequence only for files that
 actually preserve private tags.
 
 See [Device Catalog](device-catalog.md#preserved-private-tags) and
@@ -111,16 +115,16 @@ See [Device Catalog](device-catalog.md#preserved-private-tags) and
 ## Profiles
 
 The `default`, `lds`, `lds-no-dob`, and `pixels-only` profiles each map to a
-factory function in {py:mod}`dicom_dre.profiles`. A profile name selects which tags
-are kept, removed, or shifted. The behavior of each profile is described in
-[Profiles](profiles.md).
+factory function in {py:mod}`dicom_dre.profiles`. A profile name selects which
+tags the engine keeps, removes, or shifts. [Profiles](profiles.md) describes the
+behavior of each profile.
 
 ## Regression validation
 
 A regression fixture suite guards catalog and metadata behavior against known
 inputs. Each fixture pins the expected filter decision, scrub decision, scrub
-regions, and de-identified metadata for a representative instance, so a change
-that alters any of them is caught in the test run. The fixtures live in
+regions, and de-identified metadata for a representative instance, so the test
+run catches a change that alters any of them. The fixtures live in
 `tests/unit/test_catalog_regression_fixtures.py`.
 
 See [Testing](../about/testing.md).