@radicalimaging/static-wado-creator 1.2.8 → 1.3.0

package/README.md

@@ -1,287 +1,56 @@
-# `@radicalimaging/static-wado-creator`
+# Static DICOMweb Creator
+The Static DICOMweb Creator package converts DICOM Part 10 files to and from files in a DICOMweb like format.  
 
-# static-wado-js
+# Installation
+Using npm, install the package:
 
-Status: Beta - in development (as of Oct 25, 2021)
-
-The scope of this project is to convert DICOM P10 into a format into a DICOMweb compatible format - specifically JSON metadata, frames and bulkdata.  This library will enable the following:
-* Building of DICOMweb compliant services
-  * Ability to stream the data for "on the fly" use cases
-* Ability to pregenerate DICOMweb responses and store them so they can be served up with a standard HTTP server to implement a subset of DICOMweb WADO-RS, specifically:
-  * Retrieve instance metadata 
-  * Retrieve series level metadata
-  * Retrieve frames in an instance
-  * Retrieve bulk data
-  * Retrieve study, series or instance query
-* To explore an alternative archive format that is web friendly (no need for a DICOM parser)
-  * Should be able to recreate DICOM P10 file from this data (semantic equivalence or possibly bit for bit equivalent)
-  * Will make updates more efficient (e.g. updating patient name just requires updating metadata)
-  * More efficient image access (no need to scan through DICOM P10 to access frames, or store the offsets of each frame separately and seek to them) 
-
-The scope of this library is limited to:
-* Taking one or more DICOM files as input, writing Bulkdata and Pixeldata
-* Creating either deduplicated representations or instance level metadata
-* Combining deduplicated representations into sets of deduplicated representations
-* Using the deduplicated representations to generate study/series/instance query results and series metadata
-
-View the [design rationale](docs/design.md) for more information on the scope
-
-View [frequently asked questions](docs/faq.md)
-
-View [specification](docs/spec.md)
-
-## Pre-requisites
-View root pre-requisites section [pre-requisites](../../README.md#pre-requisites)
-
-## Development
-View root development section [development](../../README.md#development)
-
-## Running the CLI
-Install the CLI with either npm install -g .  or npm install -g @radicalimaging/static-wado-js
-
-## Usage
-Once you install the package the executables (i.e static-wado commands) are installed into to your PATH system, then you can use anywhere on your command line program.
-
-### Api
-There are 5 different commands: mkdicomweb, mkdicomwebinstances, mkdicomwebstudy, mkdicomwebdeduplicated, mkdicomwebdeduplicatedgroup that can be used (see usage bellow). It can be run as:
-
-```
-commnadName [options] [<input...>]
-```
-
-Arguments are:
-
-input
- : List of directories/files separated by space.
- Required for commands: **mkdicomweb**, **mkdicomwebdeduplicated**, **mkdicomwebinstances**. Optional for the others. This argument should be placed at end of command invoking.
-
-The options are:
-
-  -c, --clean
-  : Clean the study output directory for these instances (default: false)
-  
-  -C, --remove-deduplicated-instances
-  : Remove single instance deduplicated files after writing group files (default: false)
-  
-  -d, --deduplicate
-  : Write deduplicate instance level data (default: false)
-  
-  -g, --group 
-  : Write combined deduplicate data (default: true)
-  
-  -i, --instances
-  : Write instance metadata (default: false)
-  
-  -m, --maximum-inline-public-length <value>
-  : Maximum length of public binary data (default: 131074)
-  
-  -M, --maximum-inline-private-length <value>
-  : Maximum length of private binary data (default: 64)
-  
-  --no-recompress
-  : Force no recompression
-  [see](#to-recompress)
-
-  --no-recompress-thumb
-  : Force no recompression for thumbnail
-  [see](#to-recompress-thumb)
-  
-  -o, --dir <value>
-  : Set output directory (default: "~/dicomweb")
-  
-  --path-deduplicated <path>
-  : Set the deduplicate data directory path (relative to dir) (default: "deduplicated")
-  
-  --path-instances <path>
-  : Set the instances directory path (relative to dir) (default: "instances")
-  
-  -r, --recompress <listvalue...> 
-  : List of types to recompress separated by space (choices: "uncompressed", "jp2", "jpeglossless", "rle", default: "uncompressed jp2")
-  [see](#to-recompress)
-
-  --recompress-thumb <listvalue...> 
-  : List of types to recompress thumb separated by space (choices: "uncompressed", "jp2", "jpeglossless", "rle", default: "uncompressed jp2")
-  [see](#to-recompress-thumb)
-  
-  -s, --study
-  : Write study metadata - on provided instances only (TO FIX), (default: true)
-  
-  -t, --content-type <type>
-  : Destination type to compress to (choices: "jls", "lei", "jls-lossy" or DICOM Transfer Syntax UID - default: "jls")
-  
-  -T, --colour-content-type <type>
-  : Colour content type (default: null)
-  
-  -v, --verbose
-  : Write verbose output (default: false)
-  
-  -V, --version
-  : output the version number
-
-### Help command
-You can output help information by:
-```
-commandName help
-```
-
-```
-commandName -h
-```
-
-```
-commandName --help
-```
-
-### To create instances
-Run the tool:
-```
-mkdicomwebinstances <directoryOfP10Files>
-```
-
-### To create a full DICOMweb output structure
-Run the tool:
-```
-mkdicomweb <directoryOfP10Files>
-```
-
-### To run separated stages
-The mkdicomweb tool runs the three stages all together, on just the studies references.  This can instead be done on separate files by running:
+```bash
+npm install -g @radicalimaging/static-wado-creator
 ```
-mkdicomwebdeduplicated  <directoryOfP10Files>
-mkdicomwebdeduplicatedgroup
-mkdicomwebstudy
-```
-which creates a full study directory.  The first stage writes to ~/dicomweb/instances/<studyUID>/ data about each instance as it is read in.  The second stage then groups these files for more efficient compression into the ~/dicomweb/deduplicated/<studyUID>/   The last stage then creates the actual DICOMweb files.
 
-There is currently no notification of what studies have been updated between stages.  The intent is to write notifications to ~/dicomweb/notifications/<studyUID> of what operations need to be applied/updated.
 
-### To Serve Instances As a Web Server
-```
-cd ~/dicomweb
-npx http-server -p 5000 -g --cors -e json
-```
+## Additional Installs:
 
-The -g option serves up compressed files ending in .gz as compressed http streams.
+* To render thumbnail/rendered responses, install the dcm2jpg command from [dcm4che](www.dcm4che.org)
+* To generate thumbnails for video, instal ffmpeg.
 
-The -e option will mark json files as the default extension, so when requesting `GET dicomweb/studies/` the `./studies/index.json.gz` file will be served.
 
-### To Create DICOM part 10 from DICOMweb files
-TODO
+# Tools Usage
+The tools will write to a dicomweb folder created by default in the users home directory (~) in the `~/dicomweb/` location in the locations as specified in [File Structure](../../file-structure.md).
 
-Run the tool mkdicomwebpart10 on the studyUID, and optionally on the series/instance UID's of interest to generate a local set of part 10 files.
+## Converting DICOM Part 10 to DICOMweb
 
-### To Update DICOM Metadata
-TODO
+DICOM part 10 files can be converted to DICOMweb.  By default, LEI grayscale images will be converted to JPEG-LS compressed images and a thumbnail will be generated.  This can be done with:
 
-Run the tool
+```bash
+mkdicomweb create <DICOM-FILES-DIR>
 ```
-mkdicomwebupdate -<delete/anonymize/patient/study/series/instance> <studyInstanceUID> (tag=newValue)* 
-```
-to delete the given item or to update the specified attribute contained in the given level.  Multiple mkdicomwebupdate commands may be run to perform updates on different attribute sets, or they may be grouped into a single file for bulk application.
 
-### To recompress
-It allows commands to recompress data/metadata prior writing it to local. Currently, if recompress is active the designed data types will be transcoded [see](#recompress-aliases-and-input-transfer-syntaxes).
+Note there is no glob support on the input parameter, so either directories or files need to be fully specified.
 
-By default the recompression occurs for incoming types: uncompressed, jp2
-```
-mkdicomweb ./folderName
-```
-It will recompress any existing data whose transfer syntaxes are 1.2.840.10008.1.2.4.90, 1.2.840.10008.1.2.4.91, 1.2.840.10008.1.2, 1.2.840.10008.1.2.1 and 1.2.840.10008.1.2.2.
-
-Define incoming types for recompression of: uncompressed,jp2,rle
-```
-mkdicomweb  -r uncompressed jp2 rle ./folderName
-```
-It will recompress any existing data whose transfer syntaxes: 1.2.840.10008.1.2.4.90, 1.2.840.10008.1.2.4.91, 1.2.840.10008.1.2, 1.2.840.10008.1.2.1, 1.2.840.10008.1.2.2 and 1.2.840.10008.1.2.5 to transfer syntax 1.2.840.10008.1.2.4.80.
+## Converting Static DICOMweb to Part 10
 
-By default the destination compression type for recompression: jls
-```
-mkdicomweb ./folderName
-```
-It will recompress any existing data whose transfer syntaxes are 1.2.840.10008.1.2.4.90, 1.2.840.10008.1.2.4.91, 1.2.840.10008.1.2, 1.2.840.10008.1.2.1 and 1.2.840.10008.1.2.2 to jls (transfer syntax 1.2.840.10008.1.2.4.80)
+Static DICOMweb files can be converted back into Part 10, provided:
 
-Define destination type for uncompressed encoding: lei
-```
-mkdicomweb -t lei ./folderName
-```
-It will decode any existing data whose transfer syntaxes are 1.2.840.10008.1.2.4.90, 1.2.840.10008.1.2.4.91, 1.2.840.10008.1.2, 1.2.840.10008.1.2.1 and 1.2.840.10008.1.2.2 and store uncompressed as LEI (transfer syntax 1.2.840.10008.1.2.1)
+* Total DICOM Part 10 size is under 2 gb
+* DICOMweb data is stored in a compressed transfer syntax
 
-Force no recompression
-```
-mkdicomweb  -r uncompressed jp2 rle --no-recompress ./folderName
-```
-It will NOT recompress any existing data but instead it will keep them in the original encoding.
+The commands is:
 
-Force no recompression
-```
-mkdicomweb --no-recompress ./folderName
+```bash
+mkdicomweb part10 <studyUID>
 ```
-It will NOT recompress any existing data but instead it will keep them in the original encoding.
-
-Obs: --no-recompress forces no compression at all. Use it if you want to disable even the default behavior.
-
 
-### Recompress aliases and input transfer syntaxes
-The table below shows the support for transfer syntax recompression(i.e use "... -r jp2" if you want to recompress from 1.2.840.10008.1.2.4.90 to 1.2.840.10008.1.2.4.80)
-
-| TransferSyntaxUID      	| Name                                                    	| Recompress alias 	|  Target TransferSyntaxUID |
-|------------------------	|---------------------------------------------------------	|------------------	|--------------------------	|
-| 1.2.840.10008.1.2      	| Implicit VR Endian                                      	| uncompressed     	| 1.2.840.10008.1.2         |
-| 1.2.840.10008.1.2.1    	| Explicit VR Little Endian                               	| uncompressed     	| 1.2.840.10008.1.2.1       |
-| 1.2.840.10008.1.2.2    	| Explicit VR Big Endian                                  	| uncompressed     	| 1.2.840.10008.1.2.1       |
-| 1.2.840.10008.1.2.4.57 	| JPEG Lossless, Nonhierarchical (Processes 14)           	| jpeglossless     	| 1.2.840.10008.1.2.4.80    |
-| 1.2.840.10008.1.2.4.70 	| JPEG Lossless, Nonhierarchical, First- Order Prediction 	| jpeglossless     	| 1.2.840.10008.1.2.4.80    |
-| 1.2.840.10008.1.2.4.90 	| JPEG 2000 Image Compression (Lossless Only)             	| jp2              	| 1.2.840.10008.1.2.4.80    |
-| 1.2.840.10008.1.2.4.91 	| JPEG 2000 Image Compression                             	| jp2              	| 1.2.840.10008.1.2.4.80    |
-| 1.2.840.10008.1.2.5    	| RLE Lossless                                            	| rle              	| 1.2.840.10008.1.2.4.80    |
-
-
-### To recompress thumb
-It tells thumbnail creation to use recompressed data or not.
-The list of types MUST be necessarily a subset of recompress list.
-
-Obs: --no-recompress-thumb forces to not use compression at all. Use it if you want to disable even the default behavior.
-
-By default the recompression occurs for incoming types: uncompressed, jp2. This will recompress the types mentioned and will generate the thumbnails with the result of data recompression.
+## Delete A Study
+A study can be deleted using:
 
 ```
-mkdicomweb ./folderName
+mkdicomweb delete <studyUID>
 ```
 
-Force recompression thumb. This will recompress the types: uncompressed jp2 rle, thumbnails creation will use original data for all types except jp2 (which will use recompress data).
-```
-mkdicomweb  -r uncompressed jp2 rle --recompress-thumb jp2 ./folderName
-```
+## Reject A Series
+A series can be "deleted", that is rejected, which removed it only from the indices by running:
 
-Skipping not recompressed data. This will recompress the types: uncompressed jp2, thumbnails creation will use original data for all types (since recompress-thumb list does not intersect with recompress list).
 ```
-mkdicomweb  -r uncompressed jp2 --recompress-thumb rle ./folderName
-```
-
-Force no recompression thumb. This will recompress the types: uncompressed jp2 rle, but for thumbnails creation will use original data.
+mkdicomweb reject <studyUID>/seres/<seriesUID>
 ```
-mkdicomweb  -r uncompressed jp2 rle --no-recompress-thumb ./folderName
-```
-
-
-
-
-## Development
-
-## TODO (Looking for help here!!)
-
-* Create docker container to encapsulate build environment
-* Create test data (DICOM P10 and expected DICOMweb results) - Jordan working on this
-* Document API
-* Create CI + Publish to NPM
-* Fix Bugs
-    * Get bulkdata refs written properly
-* Enhance cli
-    * Add support for specifying bulkDataMinSize
-    * Add support for writing out DICOM P10 file
-* Enhance library
-    * Write out "info" file
-        * P10 Header
-        * Data needed to recreate original P10 instance
-* Create DICOMweb -> DICOM P10 tool
-

package/lib/StaticWado.js

@@ -9,12 +9,13 @@ const getDataSet = require("./operation/getDataSet");
 const InstanceDeduplicate = require("./operation/InstanceDeduplicate");
 const DeduplicateWriter = require("./writer/DeduplicateWriter");
 const ImageFrameWriter = require("./writer/ImageFrameWriter");
+const WriteStream = require("./writer/WriteStream");
 const CompleteStudyWriter = require("./writer/CompleteStudyWriter");
 const IdCreator = require("./util/IdCreator");
 const ScanStudy = require("./operation/ScanStudy");
 const HashDataWriter = require("./writer/HashDataWriter");
 const VideoWriter = require("./writer/VideoWriter");
-const { transcodeImageFrame, transcodeId, transcodeMetadata } = require("./operation/adapter/transcodeImage");
+const { transcodeImageFrame, generateLossyImage, transcodeId, transcodeMetadata } = require("./operation/adapter/transcodeImage");
 const ThumbnailWriter = require("./writer/ThumbnailWriter");
 const decodeImage = require("./operation/adapter/decodeImage");
 const ThumbnailService = require("./operation/ThumbnailService");
@@ -168,14 +169,30 @@ class StaticWado {
       bulkdata: async (bulkData, options) => {
         const _bulkDataIndex = bulkDataIndex;
         bulkDataIndex += 1;
+        // TODO - handle other types here too as single part rendered
+        if (options?.mimeType === "application/pdf") {
+          console.log("Writing rendered mimeType", options.mimeType);
+          const writeStream = WriteStream(id.sopInstanceRootPath, "rendered.pdf", {
+            gzip: false,
+            mkdir: true,
+          });
+          await writeStream.write(bulkData);
+          await writeStream.close();
+        }
         return this.callback.bulkdata(targetId, _bulkDataIndex, bulkData, options);
       },
       imageFrame: async (originalImageFrame) => {
-        const { imageFrame: transcodedImageFrame, id: transcodedId } = await transcodeImageFrame(id, targetId, originalImageFrame, dataSet, this.options);
+        const { imageFrame: transcodedImageFrame, decoded, id: transcodedId } = await transcodeImageFrame(id, targetId, originalImageFrame, dataSet, this.options);
+
+        const lossyImage = await generateLossyImage(id, decoded, this.options);
 
         const currentImageFrameIndex = imageFrameIndex;
         imageFrameIndex += 1;
 
+        if (lossyImage) {
+          await this.callback.imageFrame(lossyImage.id, currentImageFrameIndex, lossyImage.imageFrame);
+        }
+
         thumbnailService.queueThumbnail(
           {
             imageFrame: originalImageFrame,

package/lib/createPart10.js

@@ -0,0 +1,114 @@
+/* eslint-disable no-param-reassign */
+const { Tags, readBulkData } = require("@radicalimaging/static-wado-util");
+const fs = require("fs");
+const dcmjs = require("dcmjs");
+
+const StaticWado = require("./StaticWado");
+const adaptProgramOpts = require("./util/adaptProgramOpts");
+const WriteStream = require("./writer/WriteStream");
+
+const UncompressedLEIExplicit = "1.2.840.10008.1.2.1";
+const { DicomDict, DicomMetaDictionary } = dcmjs.data;
+
+const fileMetaInformationVersionArray = new Uint8Array(2);
+fileMetaInformationVersionArray[1] = 1;
+
+const createFmi = (instance) => {
+  // Assume the TSUID is in the value 0
+  const TransferSyntaxUID = Tags.getValue(instance, Tags.AvailableTransferSyntaxUID) || UncompressedLEIExplicit;
+  const MediaStorageSOPClassUID = Tags.getValue(instance, Tags.MediaStorageSOPClassUID);
+  const SOPInstanceUID = Tags.getValue(instance, Tags.SOPInstanceUID);
+  const naturalFmi = {
+    MediaStorageSOPClassUID,
+    MediaStorageSOPInstanceUID: SOPInstanceUID,
+    ImplementationVersionName: "mkdicomweb",
+    TransferSyntaxUID,
+    ImplementationClassUID: "2.25.984723498557234098.001",
+    FileMetaInformationVersion: fileMetaInformationVersionArray.buffer,
+  };
+  const denaturalized = DicomMetaDictionary.denaturalizeDataset(naturalFmi);
+  return denaturalized;
+};
+
+const readBulkDataValue = async (studyDir, instance, value) => {
+  const { BulkDataURI } = value;
+  value.vr = "OB";
+  const seriesUid = Tags.getValue(instance, Tags.SeriesInstanceUID);
+  const numberOfFrames = Tags.getValue(instance, Tags.NumberOfFrames) || 1;
+  const dir = `${studyDir}/series/${seriesUid}`;
+  if (BulkDataURI.indexOf("/frames") !== -1) {
+    // Really only support a limited number of frames based on memory size
+    value.Value = [];
+    for (let frame = 1; frame <= numberOfFrames; frame++) {
+      const bulk = await readBulkData(dir, BulkDataURI, frame);
+      if (!bulk) break;
+      value.Value.push(bulk);
+    }
+  } else {
+    value.Value = [new ArrayBuffer(await readBulkData(dir, BulkDataURI))];
+  }
+};
+
+const readBinaryData = async (dir, instance) => {
+  for (const tag of Object.keys(instance)) {
+    const v = instance[tag];
+    if (v.BulkDataURI) {
+      await readBulkDataValue(dir, instance, v);
+      continue;
+    }
+    if (!v.vr) {
+      const value0 = v.Value?.[0];
+      if (typeof value0 === "string") {
+        v.vr = "LT";
+      } else {
+        console.log("Deleting", tag, v.Value, v);
+        delete instance[tag];
+      }
+      continue;
+    }
+    if (v.vr === "SQ") {
+      await Promise.all(v.Values.map(readBinaryData));
+      continue;
+    }
+  }
+};
+
+const writeBuffer = (dir, fileName, buffer) => {
+  const writeStream = WriteStream(dir, fileName, {
+    mkdir: true,
+    gzip: false,
+  });
+  writeStream.write(buffer);
+  return writeStream.close();
+};
+
+module.exports = async function createMain(options, program) {
+  const finalOptions = adaptProgramOpts(options, {
+    ...this,
+    // Instance metadata is the instances/<sopUID>/metadata.gz files
+    isInstance: false,
+    // Deduplicated data is single instance deduplicated data
+    isDeduplicate: false,
+    // Group data is the group file directories
+    isGroup: false,
+    isStudyData: false,
+    isDeleteInstances: false,
+  });
+  const importer = new StaticWado(finalOptions);
+  const studyInstanceUid = program.args[0];
+  const dir = `${importer.options.rootDir}/studies/${studyInstanceUid}`;
+  console.log("Creating part 10 for", studyInstanceUid, dir);
+  const study = await importer.callback.scanStudy(studyInstanceUid);
+  const outputPath = `./studies/${studyInstanceUid}`;
+  for (const sop of study.getSopUids()) {
+    const instance = await study.recombine(sop);
+    const fmi = createFmi(instance);
+
+    console.log("Processing sop", sop);
+    const dicomDict = new DicomDict(fmi);
+    await readBinaryData(dir, instance);
+    dicomDict.dict = instance;
+    const buffer = Buffer.from(dicomDict.write());
+    writeBuffer(outputPath, sop, buffer);
+  }
+};

package/lib/mkdicomwebConfig.js

@@ -1,6 +1,7 @@
 const ConfigPoint = require("config-point");
 const { staticWadoConfig } = require("@radicalimaging/static-wado-util");
 const createMain = require("./createMain");
+const createPart10 = require("./createPart10");
 const deleteMain = require("./deleteMain");
 const rejectMain = require("./rejectMain");
 const instanceMain = require("./instanceMain");
@@ -21,7 +22,12 @@ const { mkdicomwebConfig } = ConfigPoint.register({
     isGroup: true,
     options: [
       {
-        key: "-c, --clean",
+        key: "-c <configFile.json5>",
+        description: "Use an alternate configuration file",
+        defaultValue: "~/static-wado.json5",
+      },
+      {
+        key: "--clean",
         description: "Clean the outputs before generating/starting to write new values.",
         defaultValue: false,
       },
@@ -39,6 +45,25 @@ const { mkdicomwebConfig } = ConfigPoint.register({
         description: "Delete the imported instances",
         defaultValue: false,
       },
+      {
+        key: "--lossy",
+        description: "Generate lossy instances instead of lossless",
+        defaultValue: false,
+      },
+      {
+        key: "--alternate <type>",
+        description: "Generates an alternate representaton of the image generally in the /lossy sub-directory",
+        choices: ["jhc", "jls", "jhcLossless", "jlsLossless"],
+      },
+      {
+        key: "--alternate-thumbnail",
+        description: "Generates a thumbnail for the alternate representation",
+      },
+      {
+        key: "--alternate-name <dir>",
+        description: "Uses a given sub directory name",
+        defaultValue: "lossy",
+      },
       {
         key: "-v, --verbose",
         description: "Write verbose output",
@@ -47,7 +72,7 @@ const { mkdicomwebConfig } = ConfigPoint.register({
       {
         key: "-t, --content-type <type>",
         description: 'Destination type to compress to (choices: "jpeg", "jls", "lei", "jls-lossy", "jhc", "jxl" or DICOM Transfer Syntax UID - default: "jls")',
-        defaultValue: "jls",
+        defaultValue: "jhc",
         customParser: compressionOptionParser,
       },
       {
@@ -170,6 +195,12 @@ const { mkdicomwebConfig } = ConfigPoint.register({
           "of deduplicated files each time a study instance UID is found, and only updates\n" +
           "those studies having at least one ",
       },
+      {
+        command: "part10",
+        arguments: ["studyUID"],
+        main: createPart10,
+        helpDescription: "Store the specified study as part 10 data",
+      },
       {
         command: "index",
         main: indexMain,

package/lib/model/uids.js

@@ -33,7 +33,11 @@ const uids = {
   "1.2.840.10008.1.2.4.81": { contentType: jls, lossy: true },
   "1.2.840.10008.1.2.4.90": { contentType: jp2, extension: ".jp2" },
   "1.2.840.10008.1.2.4.91": { contentType: jp2 },
-  "3.2.840.10008.1.2.4.96": { contentType: htj2k, extension: ".jhc" },
+  // Private/test transfer syntax for HTJ2K
+  "3.2.840.10008.1.2.4.96": { contentType: htj2k },
+  "1.2.840.10008.1.2.4.201": { contentType: htj2k },
+  "1.2.840.10008.1.2.4.202": { contentType: htj2k, extension: ".jhc" },
+  "1.2.840.10008.1.2.4.203": { contentType: htj2k, lossy: true },
   "1.2.840.10008.1.2.5": { contentType: rle },
   default: {},
 };

package/lib/operation/StudyData.js

@@ -158,7 +158,8 @@ class StudyData {
   /**
    * Create a full study instance data for instance at index
    */
-  async recombine(index) {
+  async recombine(indexOrSop) {
+    const index = typeof indexOrSop === "string" ? this.sopInstances[indexOrSop] : indexOrSop;
     const deduplicated = this.deduplicated[index];
     if (index < 0 || index >= this.deduplicated.length) {
       throw new Error(`Can't read index ${index}, out of bounds [0..${this.deduplicated.length})`);
@@ -461,6 +462,10 @@ class StudyData {
     await JSONWriter(this.deduplicatedPath, hashValue, deduplicatedList, { gzip: true, index: false });
     console.log("Wrote naturalized dataset");
   }
+
+  getSopUids() {
+    return Object.keys(this.sopInstances);
+  }
 }
 
 module.exports = StudyData;

package/lib/operation/ThumbnailService.js

@@ -120,9 +120,10 @@ class ThumbnailService {
       if (pixelData) {
         const { BulkDataURI } = pixelData;
         if (BulkDataURI?.indexOf("mp4")) {
+          fs.mkdirSync(`${itemId.sopInstanceRootPath}/rendered`, { recursive: true });
           const mp4Path = path.join(itemId.sopInstanceRootPath, "rendered/index.mp4");
           // Generate as rendered, as more back ends support that.
-          const thumbPath = path.join(itemId.sopInstanceRootPath, "rendered");
+          const thumbPath = path.join(itemId.sopInstanceRootPath, "rendered/1.jpg");
           console.log("MP4 - converting video format", mp4Path);
           this.ffmpeg(mp4Path, thumbPath);
           return thumbPath;
@@ -132,26 +133,23 @@ class ThumbnailService {
       }
       return null;
     }
-    if( options.dcm2jpg ) {
-      return this.dcm2jpg(id.filename, id.imageFrameRootPath.replace(/frames/,"thumbnail"), {});
+    if (options.dcm2jpg) {
+      return this.dcm2jpg(id.filename, id.imageFrameRootPath.replace(/frames/, "thumbnail"), {});
     }
 
-    console.log("Using CS2 to generate thumbnails", id, options);
-
     await callback.internalGenerateImage(imageFrame, dataSet, metadata, id.transferSyntaxUid, async (thumbBuffer) => {
-        try {
-          if (thumbBuffer) {
-            await callback.thumbWriter(id.sopInstanceRootPath, this.thumbFileName, thumbBuffer);
-
-            this.copySyncThumbnail(id.sopInstanceRootPath, id.seriesRootPath);
-            this.copySyncThumbnail(id.seriesRootPath, id.studyPath);
-            Stats.StudyStats.add("Thumbnail Write", `Write thumbnail ${this.thumbFileName}`, 100);
-          }
-          return this.thumbFileName;
-        } catch(e) {
-          console.log("Couldn't generate thumbnail", this.thumbFileName, e);
-        }
+      try {
+        if (thumbBuffer) {
+          await callback.thumbWriter(id.sopInstanceRootPath, this.thumbFileName, thumbBuffer);
 
+          this.copySyncThumbnail(id.sopInstanceRootPath, id.seriesRootPath);
+          this.copySyncThumbnail(id.seriesRootPath, id.studyPath);
+          Stats.StudyStats.add("Thumbnail Write", `Write thumbnail ${this.thumbFileName}`, 100);
+        }
+        return this.thumbFileName;
+      } catch (e) {
+        console.log("Couldn't generate thumbnail", this.thumbFileName, e);
+      }
     });
   }
 

package/lib/operation/adapter/transcodeImage.js

@@ -1,5 +1,5 @@
 const dicomCodec = require("@cornerstonejs/dicom-codec");
-const { Tags } = require("@radicalimaging/static-wado-util");
+const { Tags, replicate } = require("@radicalimaging/static-wado-util");
 const getImageInfo = require("./getImageInfo");
 
 dicomCodec.setConfig("verbose: false");
@@ -36,7 +36,7 @@ const transcodeDestinationMap = {
     transcodeOp: transcodeOp.encode,
   },
   jhc: {
-    transferSyntaxUid: "3.2.840.10008.1.2.4.96",
+    transferSyntaxUid: "1.2.840.10008.1.2.4.202",
     transcodeOp: transcodeOp.encode,
   },
   jpeg: {
@@ -67,6 +67,10 @@ const transcodeSourceMap = {
     alias: "jls",
   },
   "1.2.840.10008.1.2.4.70": {
+    transcodeOp: transcodeOp.decode,
+    alias: "jpll",
+  },
+  "1.2.840.10008.1.2.4.80": {
     transcodeOp: transcodeOp.decode,
     alias: "jls",
   },
@@ -82,6 +86,10 @@ const transcodeSourceMap = {
     transcodeOp: transcodeOp.decode,
     alias: "jhc",
   },
+  "1.2.840.10008.1.2.4.202": {
+    transcodeOp: transcodeOp.decode,
+    alias: "jhc",
+  },
   "1.2.840.10008.1.2.5": {
     transcodeOp: transcodeOp.decode,
     alias: "rle",
@@ -136,17 +144,20 @@ function shouldTranscodeImageFrame(id, options, samplesPerPixel) {
     return false;
   }
 
-  function isValidTranscoder() {
-    const { transferSyntaxUid } = id;
-    const transcoder = getTranscoder(transferSyntaxUid, options, samplesPerPixel);
-    return transcoder && transcoder.transferSyntaxUid && (recompress.includes("true") || recompress.includes(transcoder.alias));
+  const { transferSyntaxUid } = id;
+  const transcoder = getTranscoder(transferSyntaxUid, options, samplesPerPixel);
+  const validTranscoder = transcoder && transcoder.transferSyntaxUid;
+  if (!validTranscoder) {
+    console.verbose("Not transcoding because no decoder found for", transferSyntaxUid, samplesPerPixel);
+    return false;
   }
-
-  const ret = isValidTranscoder();
-  if (!ret) {
-    console.verbose("Not transcoding");
+  const validRecompress = recompress.includes("true") || recompress.includes(transcoder.alias);
+  if (!validRecompress) {
+    console.verbose("Not transcoding because recompress", recompress, "does not include", transcoder.alias);
+    return false;
   }
-  return ret;
+
+  return true;
 }
 
 /**
@@ -179,6 +190,102 @@ function transcodeLog(options, msg, error = "") {
   }
 }
 
+const beforeEncode = (options, encoder) => {
+  const lossy = !!options.lossy;
+  const quality = lossy ? 0.002 : -1;
+  const delta = lossy ? 3 : 0;
+  // First value is to encode as reversible lossless colour
+  encoder.setQuality?.(!lossy, quality);
+  encoder.setNearLossless?.(delta);
+};
+
+function scale(imageFrame, imageInfo) {
+  const { rows, columns, bitsPerPixel, pixelRepresentation, samplesPerPixel } = imageInfo;
+  let arrayConstructor = Float32Array;
+  if (bitsPerPixel === 8) {
+    arrayConstructor = pixelRepresentation ? Int8Array : Uint8Array;
+  } else if (bitsPerPixel > 8 && bitsPerPixel <= 16) {
+    arrayConstructor = pixelRepresentation ? Int16Array : Uint16Array;
+  }
+  const src = {
+    pixelData: new arrayConstructor(imageFrame.buffer),
+    rows,
+    columns,
+    samplesPerPixel,
+  };
+  const dest = {
+    rows: Math.round(rows / 4),
+    columns: Math.round(columns / 4),
+    samplesPerPixel,
+  };
+  dest.pixelData = new arrayConstructor(dest.rows * dest.columns * samplesPerPixel);
+  replicate(src, dest);
+
+  return {
+    imageFrame: dest.pixelData,
+    imageInfo: {
+      ...imageInfo,
+      rows: dest.rows,
+      columns: dest.columns,
+      width: dest.columns,
+      height: dest.rows,
+    },
+  };
+}
+
+/** Generates a lossy version of the image.
+ * Options for lossy are:
+ * * htj2k - generates a full resolution but high loss htj2k representation of the frames
+ * * jls - generates a +/- 2 value, maximum 128 on an edge JLS representation
+ *
+ * In either case, saves it to <sopUID>/fsiz/frameNo as multipart related
+ */
+async function generateLossyImage(id, decoded, options) {
+  if (!options.alternate) return;
+  if (!decoded?.imageFrame) return;
+
+  try {
+    let { imageFrame, imageInfo } = decoded;
+    const lossyId = {
+      ...id,
+      imageFrameRootPath: id.imageFrameRootPath.replace("frames", options.alternateName),
+      transferSyntaxUid: transcodeDestinationMap.jhc.transferSyntaxUid,
+    };
+
+    let lossy = true;
+    if (options.alternateThumbnail && imageInfo.rows >= 512) {
+      const scaled = scale(imageFrame, imageInfo);
+      if (!scaled) {
+        console.log("Couldn't scale");
+        return;
+      }
+      imageFrame = Buffer.from(scaled.imageFrame.buffer);
+      imageInfo = scaled.imageInfo;
+    }
+
+    if (options.alternate === "jls") {
+      lossyId.transferSyntaxUid = transcodeDestinationMap["jls-lossy"].transferSyntaxUid;
+    } else if (options.alternate === "jlsLossless") {
+      lossyId.transferSyntaxUid = transcodeDestinationMap.jls.transferSyntaxUid;
+      lossy = false;
+    } else if (options.alternate === "jhcLossless") {
+      lossy = false;
+    }
+
+    const encodeOptions = {
+      beforeEncode: beforeEncode.bind(null, {
+        lossy,
+      }),
+    };
+    const lossyEncoding = await dicomCodec.encode(imageFrame, imageInfo, lossyId.transferSyntaxUid, encodeOptions);
+    console.log("Encoded alternate", lossyId.transferSyntaxUid, "of size", lossyEncoding.imageFrame.length);
+    // eslint-disable-next-line consistent-return
+    return { id: lossyId, imageFrame: lossyEncoding.imageFrame };
+  } catch (e) {
+    console.warn("Unable to create alternate:", e);
+  }
+}
+
 /**
  * Transcode imageFrame. It uses transcodeMap to define the target encoding based on source encoding.
  *
@@ -206,8 +313,8 @@ async function transcodeImageFrame(id, targetIdSrc, imageFrame, dataSet, options
 
   const transcoder = getTranscoder(id.transferSyntaxUid, options, samplesPerPixel);
 
-  // last chance to prevent transcoding
-  if (targetId.transferSyntaxUid !== transcoder.transferSyntaxUid) {
+  // Don't transcode if not required
+  if (targetId.transferSyntaxUid !== transcoder.transferSyntaxUid && !options.forceTranscode) {
     console.verbose("Image is already in", targetId.transferSyntaxUid, "not transcoding");
     return {
       id,
@@ -221,20 +328,28 @@ async function transcodeImageFrame(id, targetIdSrc, imageFrame, dataSet, options
   const imageInfo = getImageInfo(dataSet);
   let done = false;
   let processResultMsg = "";
+  const encodeOptions = {
+    beforeEncode: beforeEncode.bind(null, {
+      lossy: options.lossy,
+    }),
+  };
+  let decoded;
 
   try {
     switch (transcoder.transcodeOp) {
       case transcodeOp.transcode:
         transcodeLog(options, `Full transcoding image from \x1b[43m${id.transferSyntaxUid}\x1b[0m to \x1b[43m${targetId.transferSyntaxUid}\x1b[0m`);
 
-        result = await dicomCodec.transcode(imageFrame, imageInfo, id.transferSyntaxUid, targetId.transferSyntaxUid);
+        decoded = await dicomCodec.decode(imageFrame, imageInfo, id.transferSyntaxUid);
+        result = await dicomCodec.encode(decoded.imageFrame, decoded.imageInfo, targetId.transferSyntaxUid, encodeOptions);
 
+        console.log("transcoded image to", targetId.transferSyntaxUid, "of size", result.imageFrame.length);
         processResultMsg = `Transcoding finished`;
         break;
       case transcodeOp.encode:
         transcodeLog(options, `Encoding image to \x1b[43m${targetId.transferSyntaxUid}\x1b[0m`);
 
-        result = await dicomCodec.encode(imageFrame, imageInfo, targetId.transferSyntaxUid);
+        result = await dicomCodec.encode(imageFrame, imageInfo, targetId.transferSyntaxUid, encodeOptions);
 
         processResultMsg = `Encoding finished`;
         break;
@@ -270,6 +385,7 @@ async function transcodeImageFrame(id, targetIdSrc, imageFrame, dataSet, options
     id: targetId,
     imageFrame: _imageFrame,
     done,
+    decoded,
   };
 }
 
@@ -329,3 +445,4 @@ exports.transcodeId = transcodeId;
 exports.getDestinationTranscoder = getDestinationTranscoder;
 exports.transcodeImageFrame = transcodeImageFrame;
 exports.transcodeMetadata = transcodeMetadata;
+exports.generateLossyImage = generateLossyImage;

package/lib/operation/getValue.js

@@ -151,10 +151,16 @@ const isPrivate = (attr) => {
   return chHex % 2 == 1;
 };
 
+const RawEncapsulatedDocument = "x00420011";
+
 const isValueInline = (attr, options) => {
   if (isPrivate(attr)) {
     return attr.length <= options.maximumInlinePrivateLength;
   }
+  // For encapsulated documents, almost always have them be external
+  if (attr.tag === RawEncapsulatedDocument && attr.length > 4) {
+    return false;
+  }
   return !attr.length || attr.length <= options.maximumInlinePublicLength;
 };
 
@@ -198,6 +204,7 @@ const getValue = async (dataSet, attr, vr, getDataSet, callback, options, parent
   }
   const binaryValue = dataSet.byteArray.slice(attr.dataOffset, attr.dataOffset + attr.length);
   const mimeType = attr.tag == "x00420011" && dataSet.string("x00420012");
+
   const BulkDataURI = await callback.bulkdata(binaryValue, { mimeType });
   return { BulkDataURI, vr };
 };

package/lib/writer/HashDataWriter.js

@@ -16,19 +16,18 @@ const extensions = {
 };
 
 const baseHasher = hashFactory.hasher();
-const MAX_HASH = 1024*1024;
+const MAX_HASH = 1024 * 1024;
 
 const hasher = {
   hash: (v) => {
     const type = v.constructor?.name;
     if (type === "Buffer") {
-      return createHash('sha256').update(v).digest('hex');
+      return createHash("sha256").update(v).digest("hex");
     }
     return baseHasher.hash(v);
   },
 };
 
-
 /** Writes out JSON files to the given file name.  Automatically GZips them, and adds the extension */
 const HashDataWriter =
   (options) =>

package/lib/writer/ImageFrameWriter.js

@@ -18,7 +18,7 @@ const ImageFrameWriter = (options) => {
     }
 
     if (encapsulatedImage || !extension) {
-      const writeStream = WriteStream(id.imageFrameRootPath, `${1 + index}`, {
+      const writeStream = WriteStream(id.imageFrameRootPath, `${1 + index}.mht`, {
         gzip: type.gzip,
         mkdir: true,
       });

package/lib/writer/VideoWriter.js

@@ -25,8 +25,8 @@ const VideoWriter = () =>
   async function run(id, dataSet) {
     console.log(`Writing video  ${id.sopInstanceUid}`);
     const extension = VIDEO_TYPES[dataSet.string(Tags.RawTransferSyntaxUID)];
-    const filename = `pixeldata.${extension}`;
-    const writeStream = WriteStream(id.sopInstanceRootPath, filename, {
+    const filename = `index.${extension}`;
+    const writeStream = WriteStream(`${id.sopInstanceRootPath}/rendered`, filename, {
       mkdir: true,
     });
     let length = 0;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@radicalimaging/static-wado-creator",
-  "version": "1.2.8",
+  "version": "1.3.0",
   "description": "Library to convert DICOMP10 to DICOMweb format. Original upstream: https://github.com/chafey/dicomp10-to-dicomweb-js",
   "publishConfig": {
     "access": "public"
@@ -55,10 +55,10 @@
     "verbose": true
   },
   "dependencies": {
-    "@cornerstonejs/dicom-codec": ">=0.1.6",
-    "@radicalimaging/static-cs-lite": ">=1.2.1",
-    "@radicalimaging/static-wado-util": ">=1.2.1",
-    "canvas": "^2.11.0",
+    "@cornerstonejs/dicom-codec": ">=1.0.2",
+    "@radicalimaging/static-cs-lite": ">=1.3.0",
+    "@radicalimaging/static-wado-util": ">=1.3.0",
+    "canvas": "^2.11.1",
     "config-point": ">=0.5.1",
     "dcmjs": "^0.29.5",
     "dicom-parser": "^1.8.13",