pylantir 0.2.3__tar.gz → 0.3.1__tar.gz

{pylantir-0.2.3 → pylantir-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pylantir
-Version: 0.2.3
+Version: 0.3.1
 Summary: Python - DICOM Modality WorkList with Optional API
 Author-email: Milton Camacho <miltoncamachoicc@gmail.com>
 Requires-Python: >=3.11.1
@@ -21,6 +21,8 @@ Requires-Dist: uuid
 Requires-Dist: coloredlogs
 Requires-Dist: python-dotenv
 Requires-Dist: pandas
+Requires-Dist: requests
+Requires-Dist: pytz
 Requires-Dist: fastapi>=0.104.1 ; extra == "api"
 Requires-Dist: uvicorn[standard]>=0.24.0 ; extra == "api"
 Requires-Dist: passlib[bcrypt]==1.7.4 ; extra == "api"
@@ -46,6 +48,7 @@ Requires-Dist: pytest-runner ; extra == "test"
 Requires-Dist: pytest==7.3.1 ; extra == "test"
 Requires-Dist: pytest-github-actions-annotate-failures ; extra == "test"
 Requires-Dist: shellcheck-py==0.9.0.2 ; extra == "test"
+Requires-Dist: responses>=0.23.0 ; extra == "test"
 Project-URL: Documentation, https://github.com/miltoncamacho/pylantir/tree/main#readme
 Project-URL: Source, https://github.com/miltoncamacho/pylantir
 Project-URL: Tracker, https://github.com/miltoncamacho/pylantir/issues
@@ -61,9 +64,9 @@ Provides-Extra: test
     <img src="pylantir.png" alt="Pylantir" width="50%">
 </div>
 
-This project's goal is to significantly reduce the number of human-related errors when manualy registering participants for medical imaging procedures.
+This project's goal is to significantly reduce the number of human-related errors when manually registering participants for medical imaging procedures.
 
-It effectively provides a python based DICOM Modality Worklist Server (SCP) and Modality Performed Procedure Step (SCP) able to receive requests from medical imaging equipemnt based on DICOM network comunication (e.g., a C-FIND, N-CREATE, N-SET requests).
+It effectively provides a python based DICOM Modality Worklist Server (SCP) and Modality Performed Procedure Step (SCP) able to receive requests from medical imaging equipment based on DICOM network communication (e.g., C-FIND, N-CREATE, N-SET requests).
 
 It will build/update a database based on the information entered in the study-related REDCap database using a REDCap API (You will require to have API access to the study).
 
@@ -147,7 +150,7 @@ cd pylantir/tests
 Query the worklist database to check that you have some entries using:
 
 ```bash
-python query-db.py
+python tests/query_db.py
 ```
 
 Then, you can get a StudyUID from one of the entries to test the MPPS workflow. For example: 1.2.840.10008.3.1.2.3.4.55635351412689303463019139483773956632
@@ -155,19 +158,19 @@ Then, you can get a StudyUID from one of the entries to test the MPPS workflow.
 Take this and run a create action to mark the worklist Procedure Step Status as IN_PROGRESS
 
 ```bash
-python test-mpps.py --AEtitle MWL_SERVER --mpps_action create --callingAEtitle MWL_TESTER --ip 127.0.0.1 --port 4242 --study_uid 1.2.840.10008.3.1.2.3.4.55635351412689303463019139483773956632
+python tests/mpps_tester.py --AEtitle MWL_SERVER --mpps_action create --callingAEtitle MWL_TESTER --ip 127.0.0.1 --port 4242 --study_uid 1.2.840.10008.3.1.2.3.4.55635351412689303463019139483773956632
 ```
 
-You can verify that this in fact modified your database re-running:
+You can verify that this in fact modified your database by re-running:
 
 ```bash
-python query-db.py
+python tests/query_db.py
 ```
 
-Finally, you can also simulate the pocedure completion efectively updating the Procedure Step Status to COMPLETED or DISCONTINUED:
+Finally, you can also simulate the procedure completion, effectively updating the Procedure Step Status to COMPLETED or DISCONTINUED:
 
 ```bash
-python test-mpps.py --AEtitle MWL_SERVER --mpps_action set --mpps_status COMPLETED --callingAEtitle MWL_TESTER --ip 127.0.0.1 --port 4242 --study_uid 1.2.840.10008.3.1.2.3.4.55635351412689303463019139483773956632 --sop_uid 1.2.840.10008.3.1.2.3.4.187176383255263644225774937658729238426
+python tests/mpps_tester.py --AEtitle MWL_SERVER --mpps_action set --mpps_status COMPLETED --callingAEtitle MWL_TESTER --ip 127.0.0.1 --port 4242 --study_uid 1.2.840.10008.3.1.2.3.4.55635351412689303463019139483773956632 --sop_uid 1.2.840.10008.3.1.2.3.4.187176383255263644225774937658729238426
 ```
 
 ## Usage
@@ -197,14 +200,16 @@ usage: pylantir [-h] [--AEtitle AETITLE] [--ip IP] [--port PORT] [--pylantir_con
 - **--ip IP**: IP/host address for the server
 - **--port PORT**: Port for the server
 - **--pylantir_config PYLANTIR_CONFIG**: Path to the configuration JSON file containing pylantir configs:
+  - **data_sources**: Array of data source configurations (recommended new format)
+    - Each source has: `name`, `type`, `enabled`, `sync_interval`, `operation_interval`, `config`, `field_mapping`
+  - **redcap2wl**: Legacy field mapping (deprecated, auto-converts to data_sources)
   - **allowed_aet**: List of allowed AE titles e.g. `["MRI_SCANNER", "MRI_SCANNER_2"]`
-  - **site**: Site ID:string
+  - **site**: Site ID (legacy format, deprecated)
   - **protocol**: `{"site": "protocol_name", "mapping": "HIS/RIS mapping"}`
-  - **redcap2wl**: Dictionary of REDCap fields to worklist fields mapping e.g., `{"redcap_field": "worklist_field"}`
   - **db_path**: Path to main worklist database e.g., `"/path/to/worklist.db"`
   - **users_db_path**: Optional path to users authentication database e.g., `"/path/to/users.db"`
-  - **db_update_interval**: How often to reload the database e
-  - **operation_interval**: What is the time range in a day in which the database will be updated e.g., `{"start_time":[hours,minutes],"end_time":[hours,minutes]}`
+  - **db_update_interval**: Legacy sync interval (deprecated, use sync_interval in data_sources)
+  - **operation_interval**: Legacy operation window (deprecated, use operation_interval in data_sources)
 - **--mpps_action {create,set}**: Action to perform for MPPS either create or set
 - **--mpps_status {COMPLETED,DISCONTINUED}**: Status to set for MPPS either COMPLETED or DISCONTINUED
 - **--callingAEtitle CALLINGAETITLE**: Calling AE Title for MPPS, it helps when the MWL is limited to only accept certain AE titles
@@ -213,40 +218,318 @@ usage: pylantir [-h] [--AEtitle AETITLE] [--ip IP] [--port PORT] [--pylantir_con
 
 ## Configuration JSON file
 
-As a default pylantir will try to read a JSON structured file with the following structure:
+Pylantir supports a modular data sources configuration that allows you to connect to multiple data sources simultaneously.
+
+### New Data Sources Format (Recommended)
+
+The new configuration format uses a `data_sources` array to define one or more data sources:
 
 ```json
 {
   "db_path": "/path/to/worklist.db",
   "users_db_path": "/path/to/users.db",
   "db_echo": "False",
-  "db_update_interval": 60,
-  "operation_interval": {"start_time": [0,0],"end_time": [23,59]},
   "allowed_aet": [],
+  "data_sources": [
+    {
+      "name": "main_redcap",
+      "type": "redcap",
+      "enabled": true,
+      "sync_interval": 60,
+      "operation_interval": {
+        "start_time": [0, 0],
+        "end_time": [23, 59]
+      },
+      "config": {
+        "site_id": "792",
+        "protocol": "BRAIN_MRI_3T"
+      },
+      "field_mapping": {
+        "study_id": "study_id",
+        "instrument": "redcap_repeat_instrument",
+        "session_id": "mri_instance",
+        "family_id": "family_id",
+        "youth_dob_y": "youth_dob_y",
+        "t1_date": "t1_date",
+        "demo_sex": "demo_sex",
+        "scheduled_date": "mri_date",
+        "scheduled_time": "mri_time",
+        "mri_wt_lbs": "patient_weight_lb",
+        "referring_physician": "referring_physician_name",
+        "performing_physician": "performing_physician",
+        "station_name": "station_name",
+        "status": "performed_procedure_step_status"
+      }
+    }
+  ],
+  "protocol": {
+    "792": "BRAIN_MRI_3T",
+    "mapping": "GEHC"
+  }
+}
+```
+
+**Data Source Configuration Fields:**
+
+- **`name`**: Unique identifier for this data source (used in logs and database tracking)
+- **`type`**: Data source type (currently supports `"redcap"`; extensible for future sources)
+- **`enabled`**: Boolean to enable/disable this source without removing its configuration
+- **`sync_interval`**: How often to sync data (in seconds)
+- **`operation_interval`**: Time window when sync should occur (24-hour format)
+  - `start_time`: `[hours, minutes]` - Start of operation window
+  - `end_time`: `[hours, minutes]` - End of operation window
+- **`config`**: Source-specific configuration
+  - For REDCap: `site_id`, `protocol`, and optional API credentials
+- **`field_mapping`**: Maps source fields to DICOM worklist fields
+- **`window_mode`** (Calpendo optional): `rolling` (default) or `today`
+- **`daily_window`** (Calpendo optional): `{"start_time": [h, m], "end_time": [h, m]}`
+
+### Multiple Data Sources Example
+
+You can configure multiple data sources to sync simultaneously:
+
+```json
+{
+  "db_path": "/path/to/worklist.db",
+  "data_sources": [
+    {
+      "name": "site_792_redcap",
+      "type": "redcap",
+      "enabled": true,
+      "sync_interval": 60,
+      "config": {
+        "site_id": "792",
+        "protocol": "BRAIN_MRI_3T"
+      },
+      "field_mapping": { "study_id": "study_id" }
+    },
+    {
+      "name": "site_793_redcap",
+      "type": "redcap",
+      "enabled": true,
+      "sync_interval": 120,
+      "config": {
+        "site_id": "793",
+        "protocol": "CARDIAC_MRI"
+      },
+      "field_mapping": { "patient_id": "patient_id" }
+    }
+  ]
+}
+```
+
+## Calpendo Data Source Integration
+
+Pylantir supports integration with Calpendo booking systems to automatically sync MRI/EEG scanner bookings into the DICOM worklist. This allows scanner operators to see scheduled bookings directly on their imaging equipment.
+
+### Prerequisites
+
+Set up environment variables for Calpendo authentication:
+
+```bash
+export CALPENDO_USERNAME=<your_calpendo_username>
+export CALPENDO_PASSWORD=<your_calpendo_password>
+```
+
+### Minimal Configuration
+
+```json
+{
+  "db_path": "/path/to/worklist.db",
+  "data_sources": [
+    {
+      "name": "calpendo_mri",
+      "type": "calpendo",
+      "enabled": true,
+      "sync_interval": 60,
+      "config": {
+        "base_url": "https://your-institution.calpendo.com",
+        "resources": ["3T Diagnostic", "EEG Lab"]
+      },
+      "field_mapping": {
+        "patient_id": {
+          "source_field": "title",
+          "_extract": {
+            "pattern": "^([A-Z0-9]+)_.*",
+            "group": 1
+          }
+        },
+        "patient_name": {
+          "source_field": "title",
+          "_extract": {
+            "pattern": "^[A-Z0-9]+_(.+)$",
+            "group": 1
+          }
+        },
+        "study_description": {
+          "source_field": "properties.project.formattedName",
+          "_extract": {
+            "pattern": "^([^(]+)",
+            "group": 1
+          }
+        },
+        "accession_number": "id"
+      }
+    }
+  ]
+}
+```
+
+### Full Configuration with All Options
+
+```json
+{
+  "name": "calpendo_scanners",
+  "type": "calpendo",
+  "enabled": true,
+  "sync_interval": 60,
+  "config": {
+    "base_url": "https://your-institution.calpendo.com",
+    "resources": ["3T Diagnostic", "EEG Lab", "Mock Scanner"],
+    "status_filter": "Approved",
+    "lookback_multiplier": 2,
+    "timezone": "America/Edmonton",
+    "resource_modality_mapping": {
+      "3T": "MR",
+      "EEG": "EEG",
+      "Mock": "OT"
+    }
+  },
+  "field_mapping": {
+    "patient_id": {
+      "source_field": "title",
+      "_extract": {
+        "pattern": "^([A-Z0-9]+)_.*",
+        "group": 1
+      }
+    },
+    "patient_name": {
+      "source_field": "title",
+      "_extract": {
+        "pattern": "^[A-Z0-9]+_(.+)$",
+        "group": 1
+      }
+    },
+    "study_description": {
+      "source_field": "properties.project.formattedName",
+      "_extract": {
+        "pattern": "^([^(]+)",
+        "group": 1
+      }
+    },
+    "accession_number": "id",
+    "study_instance_uid": "id"
+  }
+}
+```
+
+**Calpendo Configuration Fields:**
+
+- **`base_url`**: Calpendo server URL (e.g., `https://your-institution.calpendo.com`)
+- **`resources`**: Array of resource names to sync (e.g., scanner names)
+- **`status_filter`** (optional): Only sync bookings with this status (e.g., `"Approved"`)
+- **`lookback_multiplier`** (optional, default: 2): Rolling window multiplier for incremental sync
+- **`timezone`** (optional, default: `"America/Edmonton"`): Timezone for booking timestamps
+- **`resource_modality_mapping`** (optional): Map resource names to DICOM modality codes
+- **`field_mapping`** (at data source root): Maps Calpendo fields to worklist fields
+- **`window_mode`** (optional): `rolling` (default) or `today`
+- **`daily_window`** (optional): `{"start_time": [h, m], "end_time": [h, m]}`
+  - Use **`_extract`** for regex-based field extraction:
+    - **`pattern`**: Regular expression pattern (use `\\` for escaping in JSON)
+    - **`group`**: Capture group number (0 = full match, 1+ = capture groups)
+
+### Regex Pattern Examples
+
+Common patterns for extracting information from Calpendo booking titles:
+
+```json
+{
+  "patient_id": {
+    "source_field": "title",
+    "_extract": {
+      "pattern": "^([A-Z0-9]+)_.*",
+      "group": 1
+    }
+  },
+  "patient_name": {
+    "source_field": "title",
+    "_extract": {
+      "pattern": "^[A-Z0-9]+_(.+)$",
+      "group": 1
+    }
+  },
+  "study_description": {
+    "source_field": "properties.project.formattedName",
+    "_extract": {
+      "pattern": "^([^(]+)",
+      "group": 1
+    }
+  }
+}
+```
+
+### Troubleshooting
+
+**Authentication Errors:**
+- Verify environment variables are set correctly
+- Check Calpendo username/password are valid
+- Ensure API access is enabled for your account
+
+**No Bookings Synced:**
+- Check `resources` list matches actual Calpendo resource names
+- Verify `status_filter` isn't too restrictive
+- Check booking dates are within the rolling window (current time ± lookback window)
+
+**Regex Extraction Failures:**
+- Test regex patterns with sample data before deployment
+- Check JSON escaping (use `\\` for backslashes)
+- Set logging to DEBUG to see extraction warnings
+
+### Workflow Notes
+
+- MPPS status updates are owned by N-CREATE/N-SET; sync operations do not overwrite existing `performed_procedure_step_status` values.
+- Calpendo booking times are interpreted in the configured local timezone and stored in the legacy formats `YYYY-MM-DD` (date) and `HH:MM` (time).
+
+For more details, see the [Calpendo plugin quickstart](specs/002-calpendo-plugin/quickstart.md).
+
+### Legacy Configuration Format (Deprecated)
+
+⚠️ **The legacy configuration format is deprecated but still supported for backward compatibility.**
+
+If you're using the old format, Pylantir will automatically convert it to the new format at runtime:
+
+```json
+{
+  "db_path": "/path/to/worklist.db",
+  "db_echo": "False",
+  "db_update_interval": 60,
+  "operation_interval": {"start_time": [0,0], "end_time": [23,59]},
   "site": "792",
   "redcap2wl": {
     "study_id": "study_id",
-    "instrument": "redcap_repeat_instrument",
-    "session_id" : "mri_instance",
-    "family_id": "family_id",
-    "youth_dob_y": "youth_dob_y",
-    "t1_date": "t1_date",
-    "demo_sex": "demo_sex",
-    "scheduled_date": "mri_date",
-    "scheduled_time": "mri_time",
-    "mri_wt_lbs": "patient_weight_lb",
-    "referring_physician": "referring_physician_name",
-    "performing_physician": "performing_physician",
-    "station_name": "station_name",
-    "status": "performed_procedure_step_status"
+    "demo_sex": "demo_sex"
   },
   "protocol": {
-    "792": "BRAIN_MRI_3T",
-    "mapping": "GEHC"
+    "792": "BRAIN_MRI_3T"
   }
 }
 ```
 
+When using the legacy format, you'll see a deprecation warning:
+```
+WARNING: Legacy configuration format detected.
+Consider migrating to 'data_sources' format for better flexibility.
+```
+
+**Migration Note**: To migrate from legacy to new format:
+1. Rename `redcap2wl` → `field_mapping`
+2. Move `site` → `config.site_id`
+3. Move `protocol[site]` → `config.protocol`
+4. Move `db_update_interval` → `sync_interval`
+5. Wrap everything in a `data_sources` array with `name`, `type`, and `enabled` fields
+
+See `config/mwl_config_multi_source_example.json` for a complete example.
+
 ### Memory Management (Optional)
 
 When you install the `monitoring` optional dependency (`pip install pylantir[monitoring]`), Pylantir gains enhanced memory monitoring capabilities during REDCap synchronization:
@@ -262,7 +545,7 @@ This is particularly useful for production deployments with frequent synchroniza
 
 ## FastAPI REST API (Optional)
 
-**New in v0.2.0**: Pylantir now includes an optional REST API for programmatic access to worklist data and user management.
+Pylantir now includes an optional REST API for programmatic access to worklist data and user management.
 
 ### Installation with API Support
 
@@ -580,7 +863,7 @@ Control Cross-Origin Resource Sharing (CORS) for web frontend integration:
 
 ## Clean Stop of the MWL and Database Sync
 
-To cleanly stop the MWL server and ensure the database syncronization properly, press `Ctrl + C` (you might need to press it twice).
+To cleanly stop the MWL server and ensure the database synchronization properly, press `Ctrl + C` (you might need to press it twice).
 
 To stop the API server, use `Ctrl + C` in the terminal where it's running.