Zigong-MEDS 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- zigong_meds-0.3.0/LICENSE +21 -0
- zigong_meds-0.3.0/PKG-INFO +177 -0
- zigong_meds-0.3.0/README.md +160 -0
- zigong_meds-0.3.0/pyproject.toml +82 -0
- zigong_meds-0.3.0/setup.cfg +4 -0
- zigong_meds-0.3.0/src/Zigong_MEDS/__init__.py +12 -0
- zigong_meds-0.3.0/src/Zigong_MEDS/event_configs.yaml +570 -0
- zigong_meds-0.3.0/src/Zigong_MEDS.egg-info/PKG-INFO +177 -0
- zigong_meds-0.3.0/src/Zigong_MEDS.egg-info/SOURCES.txt +12 -0
- zigong_meds-0.3.0/src/Zigong_MEDS.egg-info/dependency_links.txt +1 -0
- zigong_meds-0.3.0/src/Zigong_MEDS.egg-info/entry_points.txt +2 -0
- zigong_meds-0.3.0/src/Zigong_MEDS.egg-info/requires.txt +1 -0
- zigong_meds-0.3.0/src/Zigong_MEDS.egg-info/top_level.txt +1 -0
- zigong_meds-0.3.0/tests/test_config.py +17 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2024 Matthew McDermott
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: Zigong-MEDS
|
|
3
|
+
Version: 0.3.0
|
|
4
|
+
Summary: An ETL pipeline to extract the Zigong Fourth People's Hospital ICU infection database into the MEDS format.
|
|
5
|
+
Author-email: Robin van de Water <rpv2114@cumc.columbia.edu>
|
|
6
|
+
Project-URL: Homepage, https://github.com/rvandewater/ZIGONG_MEDS
|
|
7
|
+
Project-URL: Issues, https://github.com/rvandewater/ZIGONG_MEDS/issues
|
|
8
|
+
Project-URL: Dataset, https://physionet.org/content/icu-infection-zigong-fourth/1.1/
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Requires-Python: <3.14,>=3.11.4
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: meds-extract[download]<0.8,>=0.7.0
|
|
16
|
+
Dynamic: license-file
|
|
17
|
+
|
|
18
|
+
# Zigong MEDS Extraction ETL
|
|
19
|
+
|
|
20
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/tests.yaml)
|
|
21
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/synthetic-etl.yaml)
|
|
22
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/code-quality-main.yaml)
|
|
23
|
+

|
|
24
|
+
[](https://github.com/mmcdermott/MEDS_extract)
|
|
25
|
+
[](LICENSE)
|
|
26
|
+
|
|
27
|
+
This pipeline extracts the
|
|
28
|
+
[Zigong Fourth People's Hospital ICU infection database v1.1](https://physionet.org/content/icu-infection-zigong-fourth/1.1/)
|
|
29
|
+
from PhysioNet into the [MEDS format](https://medical-event-data-standard.github.io/). The release holds
|
|
30
|
+
2,790 ICU patients with infection, one hospital admission each, from Zigong Fourth People's Hospital in
|
|
31
|
+
Sichuan, China. It was built from the
|
|
32
|
+
[ETL_MEDS_Template](https://github.com/Medical-Event-Data-Standard/ETL_MEDS_Template) and uses the
|
|
33
|
+
config-only layout introduced by [MEDS-Extract](https://github.com/mmcdermott/MEDS_extract) 0.7.
|
|
34
|
+
|
|
35
|
+
## Usage
|
|
36
|
+
|
|
37
|
+
Zigong is a credentialed dataset. You need a PhysioNet account that has signed the project's data use
|
|
38
|
+
agreement.
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
pip install git+https://github.com/rvandewater/ZIGONG_MEDS.git
|
|
42
|
+
export DATASET_DOWNLOAD_USERNAME=$PHYSIONET_USERNAME
|
|
43
|
+
export DATASET_DOWNLOAD_PASSWORD=$PHYSIONET_PASSWORD
|
|
44
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
When you run this, the program will:
|
|
48
|
+
|
|
49
|
+
1. Download the release (seven `dt*.csv` tables, about 840 MB) from PhysioNet into
|
|
50
|
+
`$MEDS_OUTPUT_DIR/.meds_extract_run/raw_input`. Pass `download_dest_dir=$RAW_INPUT_DIR` to keep the raw
|
|
51
|
+
data somewhere durable and reuse it across runs.
|
|
52
|
+
2. Build the MEDS cohort from those files and write it to `$MEDS_OUTPUT_DIR` (`data/` and `metadata/`).
|
|
53
|
+
All transformations and joins are declared in
|
|
54
|
+
[`src/Zigong_MEDS/event_configs.yaml`](src/Zigong_MEDS/event_configs.yaml).
|
|
55
|
+
|
|
56
|
+
If you already have the PhysioNet release on disk, skip the download and point `input_dir` at the
|
|
57
|
+
directory that contains `dtBaseline.csv`, `dtDrugs.csv`, `dtICD.csv`, `dtLab.csv`, `dtNursingChart.csv`,
|
|
58
|
+
`dtOutCome.csv` and `dtTransfer.csv`:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR \
|
|
62
|
+
do_download=false input_dir=/path/to/icu-infection-zigong-fourth-1.1
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
To parallelize the stages, pass a runner file:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
printf 'parallelize:\n n_workers: 8\n launcher: joblib\n' >runner.yaml
|
|
69
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR stage_runner_fp=runner.yaml
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Run `meds-extract-run --help` for every option.
|
|
73
|
+
|
|
74
|
+
## How this ETL is defined
|
|
75
|
+
|
|
76
|
+
The package contains no Python. The whole ETL is one MESSY (MEDS-Extract Specification Syntax YAML)
|
|
77
|
+
file, [`src/Zigong_MEDS/event_configs.yaml`](src/Zigong_MEDS/event_configs.yaml). It declares:
|
|
78
|
+
|
|
79
|
+
- **`sources:`**: where the raw data lives (the PhysioNet release).
|
|
80
|
+
- **`etl:`**: the dataset name and sharding options.
|
|
81
|
+
- **One block per raw table**: how each row becomes one or more MEDS events.
|
|
82
|
+
|
|
83
|
+
`pyproject.toml` registers the file with MEDS-Extract's `MEDS_extract.pipelines` entry-point group under
|
|
84
|
+
the name `Zigong`. That registration is what makes `spec=Zigong` work. No pre-MEDS step is needed:
|
|
85
|
+
tables without `PATIENT_ID` (`dtLab`, `dtNursingChart`) are joined to `dtBaseline` on `INP_NO` inside the
|
|
86
|
+
config.
|
|
87
|
+
|
|
88
|
+
**Times.** Every source time is an offset in hours from hospital admission. The ETL places them on a
|
|
89
|
+
shared pseudotime anchor, 1970-01-01 00:00 = admission, which preserves order and intervals without
|
|
90
|
+
claiming a real calendar date. Each patient has exactly one admission, so offsets never collide.
|
|
91
|
+
|
|
92
|
+
## Events
|
|
93
|
+
|
|
94
|
+
| Table | Event codes | Notes |
|
|
95
|
+
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
96
|
+
| `dtBaseline` | `HOSPITAL_ADMISSION`, `AGE_AT_ADMISSION`, `SEX//…`, `ADMISSION_DEPARTMENT//…`, `INFECTION_SITE//…`, `ICU_DISCHARGE`, `HOSPITAL_DISCHARGE` | Age is the `numeric_value`. |
|
|
97
|
+
| `dtTransfer` | `TRANSFER_TO//{dept}`, `TRANSFER_FROM//{dept}` | The reason is kept in `transfer_reason`. See [#3](https://github.com/rvandewater/ZIGONG_MEDS/issues/3). |
|
|
98
|
+
| `dtICD` | `DIAGNOSIS//ICD//{code}` | Placed at hospital discharge. The clinician's description is the `text_value`; `diagnosis_sequence` and `discharge_diagnosis_status` are kept. |
|
|
99
|
+
| `dtLab` | `LAB//{category}//{item}//{unit}` | A missing unit becomes `UNK`. `reference_range` is kept. |
|
|
100
|
+
| `dtDrugs` | `MEDICATION//{drug}` | Prescription time. `Formula` is the `text_value`. |
|
|
101
|
+
| `dtNursingChart` | `NURSING//{MEASUREMENT}` (vitals, ventilation, consciousness, GCS components, lines and tubes, risk and pain scores, intake and output, notes, …) | One code per charted column. Vital signs carry a `unit` column. GCS components carry the score as the `numeric_value` and the charted label as text. |
|
|
102
|
+
| `dtOutCome` | `MEDS_DEATH`, `FOLLOW_UP_VITAL_STATUS//…`, `SF36//{ITEM}//{answer}`, `DISCHARGE_DEPARTMENT//…` | Follow-up events are placed at `Follow_Date`; the discharge department at hospital discharge. |
|
|
103
|
+
|
|
104
|
+
Every event linked to a stay carries `inpatient_stay_id` (`INP_NO`). The R-written CSVs encode missing
|
|
105
|
+
values as the string `NA` or a quoted empty string; both are treated as missing, as are text entries with
|
|
106
|
+
no letters or digits.
|
|
107
|
+
|
|
108
|
+
## Data notes
|
|
109
|
+
|
|
110
|
+
- **Coarse nursing chart timestamps.** `dtNursingChart` has about 1 million rows with a usable
|
|
111
|
+
`ChartTime` but only about 23,000 distinct (patient, `ChartTime`) pairs. A typical timestamp covers 26
|
|
112
|
+
source rows, and the largest covers 266. This comes from the source release, not the ETL. In the MEDS
|
|
113
|
+
output, one timestamp therefore holds many readings of the same measurement, on average about 15
|
|
114
|
+
`NURSING//HEART_RATE` values, and similarly for blood pressure, SpO2 and other vitals. The order of events
|
|
115
|
+
within a timestamp carries no clinical meaning. When defining tasks or features:
|
|
116
|
+
- Do not rely on order within a timestamp, such as "last value before X", minute-level features, or
|
|
117
|
+
sequence models that treat event order as real time order.
|
|
118
|
+
- Choose prediction times that do not fall inside a timestamp's group of events, and aggregate values
|
|
119
|
+
within a timestamp (for example mean, min, or max) rather than picking one.
|
|
120
|
+
- **Dropped nursing rows.** 313,974 nursing rows (23%) belong to 723 stays that are not in the cohort
|
|
121
|
+
(`dtBaseline`), and all of them have `ChartTime = NA`; they are dropped. Rows charted more than 24 h before
|
|
122
|
+
admission (9,011 rows, including some corrupt offsets) or after discharge (141) are dropped too.
|
|
123
|
+
- **Dropped labs.** Results timed after hospital discharge (4,415 rows, 0.3%) and placeholder values made
|
|
124
|
+
only of dots or asterisks are dropped.
|
|
125
|
+
- **Coarse death dates.** `Death_Date` has about day precision. In-hospital deaths typically fall about 12 h
|
|
126
|
+
before the recorded discharge, so a stay's last events can follow `MEDS_DEATH`. See
|
|
127
|
+
[#4](https://github.com/rvandewater/ZIGONG_MEDS/issues/4).
|
|
128
|
+
- **Line and tube columns** (`CVC`, `PICC`, `Gastric_tube`, …) hold the length in cm, used as the
|
|
129
|
+
`numeric_value`, or a status such as "removed" kept as text. The numeric values of `Endotracheal_intubation`
|
|
130
|
+
duplicate `Endotracheal_intubation_Depth`, so only its statuses are emitted.
|
|
131
|
+
- **Bounded scores.** RASS, NRS and CPOT values outside their scales (-5..+4, 0..10, 0..8) are dropped.
|
|
132
|
+
Other vitals are not range-filtered; see [#7](https://github.com/rvandewater/ZIGONG_MEDS/issues/7).
|
|
133
|
+
- **Unmapped columns.** Five nursing columns are NA in every row. `MAP`, `PSIPAP`, `R` and `FIO2` hold only
|
|
134
|
+
unit strings. `Invasive_SBP._1` is a documented technical duplicate, and `F` is documented as having no
|
|
135
|
+
clinical relevance. See [#6](https://github.com/rvandewater/ZIGONG_MEDS/issues/6).
|
|
136
|
+
- **Local vocabularies.** Diagnoses use the release's Chinese national extension of ICD-10, and labs and
|
|
137
|
+
drugs use local names. No standard-terminology parents are attached yet; see
|
|
138
|
+
[#1](https://github.com/rvandewater/ZIGONG_MEDS/issues/1) and
|
|
139
|
+
[#8](https://github.com/rvandewater/ZIGONG_MEDS/issues/8). SF-36 answers are inconsistently coded in the
|
|
140
|
+
source; see [#5](https://github.com/rvandewater/ZIGONG_MEDS/issues/5).
|
|
141
|
+
- **No ICU admission event.** The release records ICU discharge but not ICU entry; see
|
|
142
|
+
[#2](https://github.com/rvandewater/ZIGONG_MEDS/issues/2).
|
|
143
|
+
|
|
144
|
+
## Testing
|
|
145
|
+
|
|
146
|
+
Zigong has no open demo release, so there are two synthetic checks:
|
|
147
|
+
|
|
148
|
+
- `tests/e2e_synthetic_test.py` writes a few fully synthetic rows with the release's exact headers and R
|
|
149
|
+
`write.csv` encoding, runs `meds-extract-run spec=Zigong` over them, and checks the events, times, values
|
|
150
|
+
and missing-value handling it produces.
|
|
151
|
+
- The [Synthetic ETL workflow](.github/workflows/synthetic-etl.yaml) generates raw files straight from the
|
|
152
|
+
config with [MESSY-synth](https://pypi.org/project/MESSY-synth/) and runs the ETL over them. It fails if
|
|
153
|
+
any event block produces no events.
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
uv sync --group dev
|
|
157
|
+
uv run pytest tests/ -v
|
|
158
|
+
uv run pre-commit run --all-files
|
|
159
|
+
|
|
160
|
+
# MESSY-synth needs Python >= 3.12.
|
|
161
|
+
uv run --with MESSY-synth==0.0.1 MESSY-synth ./src/Zigong_MEDS/event_configs.yaml -o raw_synthetic/
|
|
162
|
+
uv run meds-extract-run spec=Zigong output_dir=zigong_meds_synth/ \
|
|
163
|
+
do_download=false input_dir=raw_synthetic/
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
For a reusable checklist on mapping a new dataset, see [docs/MEDS_ETL_MAPPING.md](docs/MEDS_ETL_MAPPING.md).
|
|
167
|
+
|
|
168
|
+
## Citation
|
|
169
|
+
|
|
170
|
+
If you use this ETL, please cite the Zigong database and its descriptor paper:
|
|
171
|
+
|
|
172
|
+
- Xu, P., Chen, L., & Zhang, Z. (2022). *Critical care database comprising patients with infection at
|
|
173
|
+
Zigong Fourth People's Hospital* (version 1.1). PhysioNet. https://doi.org/10.13026/xpt9-z726
|
|
174
|
+
- Xu, P., Chen, L., Zhu, Y. et al. Critical Care Database Comprising Patients With Infection. *Front.
|
|
175
|
+
Public Health* 10 (2022). https://doi.org/10.3389/fpubh.2022.852410
|
|
176
|
+
|
|
177
|
+
See also [`CITATION.cff`](CITATION.cff).
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# Zigong MEDS Extraction ETL
|
|
2
|
+
|
|
3
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/tests.yaml)
|
|
4
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/synthetic-etl.yaml)
|
|
5
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/code-quality-main.yaml)
|
|
6
|
+

|
|
7
|
+
[](https://github.com/mmcdermott/MEDS_extract)
|
|
8
|
+
[](LICENSE)
|
|
9
|
+
|
|
10
|
+
This pipeline extracts the
|
|
11
|
+
[Zigong Fourth People's Hospital ICU infection database v1.1](https://physionet.org/content/icu-infection-zigong-fourth/1.1/)
|
|
12
|
+
from PhysioNet into the [MEDS format](https://medical-event-data-standard.github.io/). The release holds
|
|
13
|
+
2,790 ICU patients with infection, one hospital admission each, from Zigong Fourth People's Hospital in
|
|
14
|
+
Sichuan, China. It was built from the
|
|
15
|
+
[ETL_MEDS_Template](https://github.com/Medical-Event-Data-Standard/ETL_MEDS_Template) and uses the
|
|
16
|
+
config-only layout introduced by [MEDS-Extract](https://github.com/mmcdermott/MEDS_extract) 0.7.
|
|
17
|
+
|
|
18
|
+
## Usage
|
|
19
|
+
|
|
20
|
+
Zigong is a credentialed dataset. You need a PhysioNet account that has signed the project's data use
|
|
21
|
+
agreement.
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
pip install git+https://github.com/rvandewater/ZIGONG_MEDS.git
|
|
25
|
+
export DATASET_DOWNLOAD_USERNAME=$PHYSIONET_USERNAME
|
|
26
|
+
export DATASET_DOWNLOAD_PASSWORD=$PHYSIONET_PASSWORD
|
|
27
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
When you run this, the program will:
|
|
31
|
+
|
|
32
|
+
1. Download the release (seven `dt*.csv` tables, about 840 MB) from PhysioNet into
|
|
33
|
+
`$MEDS_OUTPUT_DIR/.meds_extract_run/raw_input`. Pass `download_dest_dir=$RAW_INPUT_DIR` to keep the raw
|
|
34
|
+
data somewhere durable and reuse it across runs.
|
|
35
|
+
2. Build the MEDS cohort from those files and write it to `$MEDS_OUTPUT_DIR` (`data/` and `metadata/`).
|
|
36
|
+
All transformations and joins are declared in
|
|
37
|
+
[`src/Zigong_MEDS/event_configs.yaml`](src/Zigong_MEDS/event_configs.yaml).
|
|
38
|
+
|
|
39
|
+
If you already have the PhysioNet release on disk, skip the download and point `input_dir` at the
|
|
40
|
+
directory that contains `dtBaseline.csv`, `dtDrugs.csv`, `dtICD.csv`, `dtLab.csv`, `dtNursingChart.csv`,
|
|
41
|
+
`dtOutCome.csv` and `dtTransfer.csv`:
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR \
|
|
45
|
+
do_download=false input_dir=/path/to/icu-infection-zigong-fourth-1.1
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
To parallelize the stages, pass a runner file:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
printf 'parallelize:\n n_workers: 8\n launcher: joblib\n' >runner.yaml
|
|
52
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR stage_runner_fp=runner.yaml
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Run `meds-extract-run --help` for every option.
|
|
56
|
+
|
|
57
|
+
## How this ETL is defined
|
|
58
|
+
|
|
59
|
+
The package contains no Python. The whole ETL is one MESSY (MEDS-Extract Specification Syntax YAML)
|
|
60
|
+
file, [`src/Zigong_MEDS/event_configs.yaml`](src/Zigong_MEDS/event_configs.yaml). It declares:
|
|
61
|
+
|
|
62
|
+
- **`sources:`**: where the raw data lives (the PhysioNet release).
|
|
63
|
+
- **`etl:`**: the dataset name and sharding options.
|
|
64
|
+
- **One block per raw table**: how each row becomes one or more MEDS events.
|
|
65
|
+
|
|
66
|
+
`pyproject.toml` registers the file with MEDS-Extract's `MEDS_extract.pipelines` entry-point group under
|
|
67
|
+
the name `Zigong`. That registration is what makes `spec=Zigong` work. No pre-MEDS step is needed:
|
|
68
|
+
tables without `PATIENT_ID` (`dtLab`, `dtNursingChart`) are joined to `dtBaseline` on `INP_NO` inside the
|
|
69
|
+
config.
|
|
70
|
+
|
|
71
|
+
**Times.** Every source time is an offset in hours from hospital admission. The ETL places them on a
|
|
72
|
+
shared pseudotime anchor, 1970-01-01 00:00 = admission, which preserves order and intervals without
|
|
73
|
+
claiming a real calendar date. Each patient has exactly one admission, so offsets never collide.
|
|
74
|
+
|
|
75
|
+
## Events
|
|
76
|
+
|
|
77
|
+
| Table | Event codes | Notes |
|
|
78
|
+
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
79
|
+
| `dtBaseline` | `HOSPITAL_ADMISSION`, `AGE_AT_ADMISSION`, `SEX//…`, `ADMISSION_DEPARTMENT//…`, `INFECTION_SITE//…`, `ICU_DISCHARGE`, `HOSPITAL_DISCHARGE` | Age is the `numeric_value`. |
|
|
80
|
+
| `dtTransfer` | `TRANSFER_TO//{dept}`, `TRANSFER_FROM//{dept}` | The reason is kept in `transfer_reason`. See [#3](https://github.com/rvandewater/ZIGONG_MEDS/issues/3). |
|
|
81
|
+
| `dtICD` | `DIAGNOSIS//ICD//{code}` | Placed at hospital discharge. The clinician's description is the `text_value`; `diagnosis_sequence` and `discharge_diagnosis_status` are kept. |
|
|
82
|
+
| `dtLab` | `LAB//{category}//{item}//{unit}` | A missing unit becomes `UNK`. `reference_range` is kept. |
|
|
83
|
+
| `dtDrugs` | `MEDICATION//{drug}` | Prescription time. `Formula` is the `text_value`. |
|
|
84
|
+
| `dtNursingChart` | `NURSING//{MEASUREMENT}` (vitals, ventilation, consciousness, GCS components, lines and tubes, risk and pain scores, intake and output, notes, …) | One code per charted column. Vital signs carry a `unit` column. GCS components carry the score as the `numeric_value` and the charted label as text. |
|
|
85
|
+
| `dtOutCome` | `MEDS_DEATH`, `FOLLOW_UP_VITAL_STATUS//…`, `SF36//{ITEM}//{answer}`, `DISCHARGE_DEPARTMENT//…` | Follow-up events are placed at `Follow_Date`; the discharge department at hospital discharge. |
|
|
86
|
+
|
|
87
|
+
Every event linked to a stay carries `inpatient_stay_id` (`INP_NO`). The R-written CSVs encode missing
|
|
88
|
+
values as the string `NA` or a quoted empty string; both are treated as missing, as are text entries with
|
|
89
|
+
no letters or digits.
|
|
90
|
+
|
|
91
|
+
## Data notes
|
|
92
|
+
|
|
93
|
+
- **Coarse nursing chart timestamps.** `dtNursingChart` has about 1 million rows with a usable
|
|
94
|
+
`ChartTime` but only about 23,000 distinct (patient, `ChartTime`) pairs. A typical timestamp covers 26
|
|
95
|
+
source rows, and the largest covers 266. This comes from the source release, not the ETL. In the MEDS
|
|
96
|
+
output, one timestamp therefore holds many readings of the same measurement, on average about 15
|
|
97
|
+
`NURSING//HEART_RATE` values, and similarly for blood pressure, SpO2 and other vitals. The order of events
|
|
98
|
+
within a timestamp carries no clinical meaning. When defining tasks or features:
|
|
99
|
+
- Do not rely on order within a timestamp, such as "last value before X", minute-level features, or
|
|
100
|
+
sequence models that treat event order as real time order.
|
|
101
|
+
- Choose prediction times that do not fall inside a timestamp's group of events, and aggregate values
|
|
102
|
+
within a timestamp (for example mean, min, or max) rather than picking one.
|
|
103
|
+
- **Dropped nursing rows.** 313,974 nursing rows (23%) belong to 723 stays that are not in the cohort
|
|
104
|
+
(`dtBaseline`), and all of them have `ChartTime = NA`; they are dropped. Rows charted more than 24 h before
|
|
105
|
+
admission (9,011 rows, including some corrupt offsets) or after discharge (141) are dropped too.
|
|
106
|
+
- **Dropped labs.** Results timed after hospital discharge (4,415 rows, 0.3%) and placeholder values made
|
|
107
|
+
only of dots or asterisks are dropped.
|
|
108
|
+
- **Coarse death dates.** `Death_Date` has about day precision. In-hospital deaths typically fall about 12 h
|
|
109
|
+
before the recorded discharge, so a stay's last events can follow `MEDS_DEATH`. See
|
|
110
|
+
[#4](https://github.com/rvandewater/ZIGONG_MEDS/issues/4).
|
|
111
|
+
- **Line and tube columns** (`CVC`, `PICC`, `Gastric_tube`, …) hold the length in cm, used as the
|
|
112
|
+
`numeric_value`, or a status such as "removed" kept as text. The numeric values of `Endotracheal_intubation`
|
|
113
|
+
duplicate `Endotracheal_intubation_Depth`, so only its statuses are emitted.
|
|
114
|
+
- **Bounded scores.** RASS, NRS and CPOT values outside their scales (-5..+4, 0..10, 0..8) are dropped.
|
|
115
|
+
Other vitals are not range-filtered; see [#7](https://github.com/rvandewater/ZIGONG_MEDS/issues/7).
|
|
116
|
+
- **Unmapped columns.** Five nursing columns are NA in every row. `MAP`, `PSIPAP`, `R` and `FIO2` hold only
|
|
117
|
+
unit strings. `Invasive_SBP._1` is a documented technical duplicate, and `F` is documented as having no
|
|
118
|
+
clinical relevance. See [#6](https://github.com/rvandewater/ZIGONG_MEDS/issues/6).
|
|
119
|
+
- **Local vocabularies.** Diagnoses use the release's Chinese national extension of ICD-10, and labs and
|
|
120
|
+
drugs use local names. No standard-terminology parents are attached yet; see
|
|
121
|
+
[#1](https://github.com/rvandewater/ZIGONG_MEDS/issues/1) and
|
|
122
|
+
[#8](https://github.com/rvandewater/ZIGONG_MEDS/issues/8). SF-36 answers are inconsistently coded in the
|
|
123
|
+
source; see [#5](https://github.com/rvandewater/ZIGONG_MEDS/issues/5).
|
|
124
|
+
- **No ICU admission event.** The release records ICU discharge but not ICU entry; see
|
|
125
|
+
[#2](https://github.com/rvandewater/ZIGONG_MEDS/issues/2).
|
|
126
|
+
|
|
127
|
+
## Testing
|
|
128
|
+
|
|
129
|
+
Zigong has no open demo release, so there are two synthetic checks:
|
|
130
|
+
|
|
131
|
+
- `tests/e2e_synthetic_test.py` writes a few fully synthetic rows with the release's exact headers and R
|
|
132
|
+
`write.csv` encoding, runs `meds-extract-run spec=Zigong` over them, and checks the events, times, values
|
|
133
|
+
and missing-value handling it produces.
|
|
134
|
+
- The [Synthetic ETL workflow](.github/workflows/synthetic-etl.yaml) generates raw files straight from the
|
|
135
|
+
config with [MESSY-synth](https://pypi.org/project/MESSY-synth/) and runs the ETL over them. It fails if
|
|
136
|
+
any event block produces no events.
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
uv sync --group dev
|
|
140
|
+
uv run pytest tests/ -v
|
|
141
|
+
uv run pre-commit run --all-files
|
|
142
|
+
|
|
143
|
+
# MESSY-synth needs Python >= 3.12.
|
|
144
|
+
uv run --with MESSY-synth==0.0.1 MESSY-synth ./src/Zigong_MEDS/event_configs.yaml -o raw_synthetic/
|
|
145
|
+
uv run meds-extract-run spec=Zigong output_dir=zigong_meds_synth/ \
|
|
146
|
+
do_download=false input_dir=raw_synthetic/
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
For a reusable checklist on mapping a new dataset, see [docs/MEDS_ETL_MAPPING.md](docs/MEDS_ETL_MAPPING.md).
|
|
150
|
+
|
|
151
|
+
## Citation
|
|
152
|
+
|
|
153
|
+
If you use this ETL, please cite the Zigong database and its descriptor paper:
|
|
154
|
+
|
|
155
|
+
- Xu, P., Chen, L., & Zhang, Z. (2022). *Critical care database comprising patients with infection at
|
|
156
|
+
Zigong Fourth People's Hospital* (version 1.1). PhysioNet. https://doi.org/10.13026/xpt9-z726
|
|
157
|
+
- Xu, P., Chen, L., Zhu, Y. et al. Critical Care Database Comprising Patients With Infection. *Front.
|
|
158
|
+
Public Health* 10 (2022). https://doi.org/10.3389/fpubh.2022.852410
|
|
159
|
+
|
|
160
|
+
See also [`CITATION.cff`](CITATION.cff).
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=64", "wheel"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "Zigong-MEDS"
|
|
7
|
+
# Bump with `uv version --bump patch|minor|major`, then tag the commit with the same version.
|
|
8
|
+
version = "0.3.0"
|
|
9
|
+
authors = [
|
|
10
|
+
{ name="Robin van de Water", email="rpv2114@cumc.columbia.edu" },
|
|
11
|
+
]
|
|
12
|
+
description = "An ETL pipeline to extract the Zigong Fourth People's Hospital ICU infection database into the MEDS format."
|
|
13
|
+
readme = "README.md"
|
|
14
|
+
# Mirrors MEDS-Extract 0.7's own interpreter range; a looser range here admits interpreters it
|
|
15
|
+
# cannot be installed on.
|
|
16
|
+
requires-python = ">=3.11.4,<3.14"
|
|
17
|
+
classifiers = [
|
|
18
|
+
"Programming Language :: Python :: 3",
|
|
19
|
+
"License :: OSI Approved :: MIT License",
|
|
20
|
+
"Operating System :: OS Independent",
|
|
21
|
+
]
|
|
22
|
+
# This package ships no Python: the whole ETL is `event_configs.yaml`, run by
|
|
23
|
+
# MEDS-Extract's own `meds-extract-run`. So there is exactly one dependency — the
|
|
24
|
+
# `[download]` extra adds `meds-extract-download` (httpx + tenacity), which the runner
|
|
25
|
+
# spawns to stage the raw data. MEDS-transforms, polars and hydra all arrive through it
|
|
26
|
+
# at the versions it fences; duplicating those pins here would only create drift.
|
|
27
|
+
#
|
|
28
|
+
# The floor is 0.7.0: this config is written in MEDS-Extract 0.7's single-file MESSY format, which
|
|
29
|
+
# earlier releases cannot read.
|
|
30
|
+
dependencies = [
|
|
31
|
+
"meds-extract[download]>=0.7.0,<0.8",
|
|
32
|
+
]
|
|
33
|
+
|
|
34
|
+
[dependency-groups]
|
|
35
|
+
# `shfmt-py` bundles the `shfmt` binary that the `mdformat-shfmt` pre-commit hook shells out to;
|
|
36
|
+
# declaring it here keeps local and CI behavior in lock step. `pyyaml` is read by the e2e test to
|
|
37
|
+
# enumerate the config's event blocks. MESSY-synth (Python >= 3.12) is installed by its own CI
|
|
38
|
+
# workflow rather than here, so the dev group still resolves on Python 3.11.
|
|
39
|
+
dev = [
|
|
40
|
+
"pre-commit~=3.8",
|
|
41
|
+
"ruff>=0.11,<1.0",
|
|
42
|
+
"pytest~=9.0",
|
|
43
|
+
"pyyaml>=6.0",
|
|
44
|
+
"shfmt-py==3.12.0.2",
|
|
45
|
+
]
|
|
46
|
+
|
|
47
|
+
[project.entry-points."MEDS_extract.pipelines"]
|
|
48
|
+
Zigong = "Zigong_MEDS:event_configs.yaml"
|
|
49
|
+
|
|
50
|
+
[project.urls]
|
|
51
|
+
Homepage = "https://github.com/rvandewater/ZIGONG_MEDS"
|
|
52
|
+
Issues = "https://github.com/rvandewater/ZIGONG_MEDS/issues"
|
|
53
|
+
Dataset = "https://physionet.org/content/icu-infection-zigong-fourth/1.1/"
|
|
54
|
+
|
|
55
|
+
[tool.setuptools.packages.find]
|
|
56
|
+
where = ["src"]
|
|
57
|
+
include = ["Zigong_MEDS*"]
|
|
58
|
+
|
|
59
|
+
[tool.setuptools.package-data]
|
|
60
|
+
Zigong_MEDS = ["*.yaml"]
|
|
61
|
+
|
|
62
|
+
[tool.ruff]
|
|
63
|
+
target-version = "py311"
|
|
64
|
+
line-length = 110
|
|
65
|
+
|
|
66
|
+
[tool.ruff.lint]
|
|
67
|
+
select = [
|
|
68
|
+
"E", # pycodestyle
|
|
69
|
+
"F", # pyflakes
|
|
70
|
+
"I", # isort
|
|
71
|
+
"UP", # pyupgrade
|
|
72
|
+
"B", # flake8-bugbear
|
|
73
|
+
"C4", # flake8-comprehensions
|
|
74
|
+
"RUF", # Ruff-specific rules
|
|
75
|
+
"SIM", # flake8-simplify
|
|
76
|
+
]
|
|
77
|
+
ignore = [
|
|
78
|
+
"SIM108", # ternary operator suggestions
|
|
79
|
+
]
|
|
80
|
+
|
|
81
|
+
[tool.ruff.lint.per-file-ignores]
|
|
82
|
+
"__init__.py" = ["F401"]
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
"""The Zigong ICU infection database MEDS ETL — a configuration package, not a library.
|
|
2
|
+
|
|
3
|
+
This distribution ships no code. Its entire content is ``Zigong_MEDS/event_configs.yaml``, a MESSY
|
|
4
|
+
(MEDS-Extract Specification Syntax YAML) document declaring where the raw Zigong data comes from and
|
|
5
|
+
how every event is extracted from it. ``pyproject.toml`` registers that file with MEDS-Extract's
|
|
6
|
+
``MEDS_extract.pipelines`` entry-point group under the name ``Zigong``, so the ETL runs as::
|
|
7
|
+
|
|
8
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR
|
|
9
|
+
|
|
10
|
+
This package exists so the YAML is addressable as package data —
|
|
11
|
+
``importlib.resources.files("Zigong_MEDS")`` — from an installed wheel. Nothing here is importable API.
|
|
12
|
+
"""
|
|
@@ -0,0 +1,570 @@
|
|
|
1
|
+
# The complete Zigong Fourth People's Hospital ICU infection database (PhysioNet v1.1) MEDS ETL.
|
|
2
|
+
# This one file *is* the package: `sources:` says where the raw data comes from, `etl:` supplies
|
|
3
|
+
# the run knobs, and every other top-level key is a raw table whose events are extracted from it.
|
|
4
|
+
# It is registered with `meds-extract-run` under the name `Zigong` (see this repo's
|
|
5
|
+
# `pyproject.toml`), so the whole ETL is one command:
|
|
6
|
+
#
|
|
7
|
+
# meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR
|
|
8
|
+
#
|
|
9
|
+
# Conventions used throughout:
|
|
10
|
+
#
|
|
11
|
+
# - All source times are deidentified hours from hospital admission. 1970-01-01 is a pseudotime
|
|
12
|
+
# anchor: it preserves ordering and intervals without asserting a real calendar date. Every
|
|
13
|
+
# patient has exactly one admission in v1.1, so per-admission offsets never collide.
|
|
14
|
+
# - The release was written by R's write.csv: a missing value is the bare string NA, and missing
|
|
15
|
+
# text in several dtNursingChart and dtLab columns is a quoted empty string. Neither is read as
|
|
16
|
+
# null, so every code and text value below guards against both. Free-text nursing fields also
|
|
17
|
+
# require a word character, which drops whitespace-, DEL- and punctuation-only entries.
|
|
18
|
+
# - Codes keep the source's own vocabulary. No standard-terminology parent is claimed where the
|
|
19
|
+
# release does not document one.
|
|
20
|
+
|
|
21
|
+
etl:
|
|
22
|
+
# `meds-extract-run` defaults the dataset name to the registered entry-point name; this fallback
|
|
23
|
+
# makes the spec self-describing when it is resolved by `pkg://` reference or filesystem path.
|
|
24
|
+
dataset_name: Zigong
|
|
25
|
+
n_subjects_per_shard: 1000
|
|
26
|
+
|
|
27
|
+
_defaults:
|
|
28
|
+
subject_id: $PATIENT_ID
|
|
29
|
+
|
|
30
|
+
dtBaseline:
|
|
31
|
+
hospital_admission:
|
|
32
|
+
code: HOSPITAL_ADMISSION
|
|
33
|
+
time: 1970-01-01 @ 00:00
|
|
34
|
+
inpatient_stay_id: $INP_NO
|
|
35
|
+
age_at_admission:
|
|
36
|
+
code: '"AGE_AT_ADMISSION" if $Age::?float64 >= 0'
|
|
37
|
+
time: 1970-01-01 @ 00:00
|
|
38
|
+
numeric_value: $Age::?float64
|
|
39
|
+
inpatient_stay_id: $INP_NO
|
|
40
|
+
sex:
|
|
41
|
+
code: f"SEX//{$SEX}"
|
|
42
|
+
time: 1970-01-01 @ 00:00
|
|
43
|
+
inpatient_stay_id: $INP_NO
|
|
44
|
+
admission_department:
|
|
45
|
+
code: f"ADMISSION_DEPARTMENT//{$AdmitDept}"
|
|
46
|
+
time: 1970-01-01 @ 00:00
|
|
47
|
+
inpatient_stay_id: $INP_NO
|
|
48
|
+
infection_site:
|
|
49
|
+
code: f"INFECTION_SITE//{$InfectionSite}"
|
|
50
|
+
time: 1970-01-01 @ 00:00
|
|
51
|
+
inpatient_stay_id: $INP_NO
|
|
52
|
+
icu_discharge:
|
|
53
|
+
code: ICU_DISCHARGE
|
|
54
|
+
time: "(1970-01-01 @ 00:00) + $ICU_discharge_time::hours"
|
|
55
|
+
inpatient_stay_id: $INP_NO
|
|
56
|
+
hospital_discharge:
|
|
57
|
+
code: HOSPITAL_DISCHARGE
|
|
58
|
+
time: "(1970-01-01 @ 00:00) + $DISCHARGE_DATE_TIME::hours"
|
|
59
|
+
inpatient_stay_id: $INP_NO
|
|
60
|
+
|
|
61
|
+
dtDrugs:
|
|
62
|
+
# dtDrugs has no INP_NO; with one admission per patient the stay is recovered from dtBaseline.
|
|
63
|
+
_table:
|
|
64
|
+
join:
|
|
65
|
+
dtBaseline:
|
|
66
|
+
key: PATIENT_ID
|
|
67
|
+
cols: [INP_NO]
|
|
68
|
+
prescription:
|
|
69
|
+
code: f"MEDICATION//{$DrugName}"
|
|
70
|
+
time: "(1970-01-01 @ 00:00) + $Drug_time::hours"
|
|
71
|
+
text_value: $Formula
|
|
72
|
+
inpatient_stay_id: $INP_NO
|
|
73
|
+
|
|
74
|
+
dtICD:
|
|
75
|
+
_table:
|
|
76
|
+
join:
|
|
77
|
+
dtBaseline:
|
|
78
|
+
key: INP_NO
|
|
79
|
+
cols: [DISCHARGE_DATE_TIME]
|
|
80
|
+
diagnosis:
|
|
81
|
+
code: 'f"DIAGNOSIS//ICD//{$ICD_Code}" if $ICD_Code != "NA"'
|
|
82
|
+
time: "(1970-01-01 @ 00:00) + $DISCHARGE_DATE_TIME::hours"
|
|
83
|
+
# ICD_desc is the clinician's free-text diagnosis, not the code's title: most codes carry
|
|
84
|
+
# several distinct descriptions, so it stays per-event and is not used as code metadata.
|
|
85
|
+
text_value: $ICD_desc
|
|
86
|
+
diagnosis_sequence: $Diagnosis_seq
|
|
87
|
+
discharge_diagnosis_status: '$Status_Discharge if $Status_Discharge != "NA"'
|
|
88
|
+
inpatient_stay_id: $INP_NO
|
|
89
|
+
|
|
90
|
+
dtLab:
|
|
91
|
+
_table:
|
|
92
|
+
join:
|
|
93
|
+
dtBaseline:
|
|
94
|
+
key: INP_NO
|
|
95
|
+
cols: [PATIENT_ID, DISCHARGE_DATE_TIME]
|
|
96
|
+
# Results timed after hospital discharge (~0.3% of rows) are dropped; pre-admission results
|
|
97
|
+
# (down to about -24 h) are kept. Placeholder values made only of dots or asterisks (".....",
|
|
98
|
+
# "****") carry no result and are dropped.
|
|
99
|
+
lab_result:
|
|
100
|
+
code: >-
|
|
101
|
+
f"LAB//{$Category}//{$Item}//{$Unit if $Unit != '' else 'UNK'}"
|
|
102
|
+
if /[^.*\s]/ in $LabValue::str
|
|
103
|
+
time: "(1970-01-01 @ 00:00) + $LabTime::hours if $LabTime <= $DISCHARGE_DATE_TIME"
|
|
104
|
+
numeric_value: $LabValue::?float64
|
|
105
|
+
text_value: $LabValue
|
|
106
|
+
reference_range: '$Ref if $Ref != ""'
|
|
107
|
+
inpatient_stay_id: $INP_NO
|
|
108
|
+
|
|
109
|
+
dtTransfer:
|
|
110
|
+
transfer_start:
|
|
111
|
+
code: f"TRANSFER_TO//{$TransferDept}"
|
|
112
|
+
time: "(1970-01-01 @ 00:00) + $StartTime::hours"
|
|
113
|
+
transfer_reason: $StartReason
|
|
114
|
+
inpatient_stay_id: $INP_NO
|
|
115
|
+
transfer_end:
|
|
116
|
+
code: f"TRANSFER_FROM//{$TransferDept}"
|
|
117
|
+
time: "(1970-01-01 @ 00:00) + $StopTime::hours"
|
|
118
|
+
transfer_reason: $StopReason
|
|
119
|
+
inpatient_stay_id: $INP_NO
|
|
120
|
+
|
|
121
|
+
# Every row with ChartTime = NA belongs to a stay that is not in dtBaseline, so the inner join
|
|
122
|
+
# drops them. Chart times more than 24 h before admission (including some corrupt offsets of
|
|
123
|
+
# millions of hours) or after discharge are dropped too. Several columns are not mapped: Rhythm,
|
|
124
|
+
# Lower_extremity_blood_pressure_monitoring, Left/Right_pupil_reflex and ETCO2 are NA in every
|
|
125
|
+
# row; MAP, PSIPAP, R and FIO2 hold only unit strings; Invasive_SBP._1 is a documented technical
|
|
126
|
+
# duplicate; F is documented as having no clinical relevance.
|
|
127
|
+
dtNursingChart:
|
|
128
|
+
_table:
|
|
129
|
+
join:
|
|
130
|
+
dtBaseline:
|
|
131
|
+
key: INP_NO
|
|
132
|
+
cols: [PATIENT_ID, DISCHARGE_DATE_TIME]
|
|
133
|
+
cols:
|
|
134
|
+
chart_time: >-
|
|
135
|
+
(1970-01-01 @ 00:00) + $ChartTime::?float64::hours
|
|
136
|
+
if $ChartTime::?float64 >= -24 and $ChartTime::?float64 <= $DISCHARGE_DATE_TIME
|
|
137
|
+
temperature:
|
|
138
|
+
code: '"NURSING//TEMPERATURE" if $temperature::?float64 >= 0'
|
|
139
|
+
time: $chart_time
|
|
140
|
+
numeric_value: $temperature::?float64
|
|
141
|
+
unit: degC
|
|
142
|
+
inpatient_stay_id: $INP_NO
|
|
143
|
+
heart_rate:
|
|
144
|
+
code: '"NURSING//HEART_RATE" if $heart_rate::?float64 >= 0'
|
|
145
|
+
time: $chart_time
|
|
146
|
+
numeric_value: $heart_rate::?float64
|
|
147
|
+
unit: "'/min'"
|
|
148
|
+
inpatient_stay_id: $INP_NO
|
|
149
|
+
respiratory_rate:
|
|
150
|
+
code: '"NURSING//RESPIRATORY_RATE" if $breathing::?float64 >= 0'
|
|
151
|
+
time: $chart_time
|
|
152
|
+
numeric_value: $breathing::?float64
|
|
153
|
+
unit: "'/min'"
|
|
154
|
+
inpatient_stay_id: $INP_NO
|
|
155
|
+
oxygen_saturation:
|
|
156
|
+
code: '"NURSING//OXYGEN_SATURATION" if $Blood_oxygen_saturation::?float64 >= 0'
|
|
157
|
+
time: $chart_time
|
|
158
|
+
numeric_value: $Blood_oxygen_saturation::?float64
|
|
159
|
+
unit: '"%"'
|
|
160
|
+
inpatient_stay_id: $INP_NO
|
|
161
|
+
systolic_blood_pressure:
|
|
162
|
+
code: '"NURSING//SYSTOLIC_BLOOD_PRESSURE" if $Blood_pressure_high::?float64 >= 0'
|
|
163
|
+
time: $chart_time
|
|
164
|
+
numeric_value: $Blood_pressure_high::?float64
|
|
165
|
+
unit: mmHg
|
|
166
|
+
inpatient_stay_id: $INP_NO
|
|
167
|
+
diastolic_blood_pressure:
|
|
168
|
+
code: '"NURSING//DIASTOLIC_BLOOD_PRESSURE" if $Blood_pressure_low::?float64 >= 0'
|
|
169
|
+
time: $chart_time
|
|
170
|
+
numeric_value: $Blood_pressure_low::?float64
|
|
171
|
+
unit: mmHg
|
|
172
|
+
inpatient_stay_id: $INP_NO
|
|
173
|
+
invasive_systolic_blood_pressure:
|
|
174
|
+
code: '"NURSING//INVASIVE_SYSTOLIC_BLOOD_PRESSURE" if $Invasive_SBP::?float64 >= 0'
|
|
175
|
+
time: $chart_time
|
|
176
|
+
numeric_value: $Invasive_SBP::?float64
|
|
177
|
+
inpatient_stay_id: $INP_NO
|
|
178
|
+
invasive_diastolic_blood_pressure:
|
|
179
|
+
code: '"NURSING//INVASIVE_DIASTOLIC_BLOOD_PRESSURE" if $Invasive_diastolic_blood_pressure::?float64 >= 0'
|
|
180
|
+
time: $chart_time
|
|
181
|
+
numeric_value: $Invasive_diastolic_blood_pressure::?float64
|
|
182
|
+
inpatient_stay_id: $INP_NO
|
|
183
|
+
central_venous_pressure:
|
|
184
|
+
code: '"NURSING//CENTRAL_VENOUS_PRESSURE" if $Central_venous_pressure::?float64 >= 0'
|
|
185
|
+
time: $chart_time
|
|
186
|
+
numeric_value: $Central_venous_pressure::?float64
|
|
187
|
+
inpatient_stay_id: $INP_NO
|
|
188
|
+
blood_sugar:
|
|
189
|
+
code: '"NURSING//BLOOD_SUGAR" if $blood_sugar::?float64 >= 0'
|
|
190
|
+
time: $chart_time
|
|
191
|
+
numeric_value: $blood_sugar::?float64
|
|
192
|
+
inpatient_stay_id: $INP_NO
|
|
193
|
+
oxygen_inhalation_mode:
|
|
194
|
+
code: '"NURSING//OXYGEN_INHALATION_MODE" if $Oxygen_inhalation_mode != "NA" and /\w/ in $Oxygen_inhalation_mode'
|
|
195
|
+
time: $chart_time
|
|
196
|
+
text_value: $Oxygen_inhalation_mode
|
|
197
|
+
inpatient_stay_id: $INP_NO
|
|
198
|
+
oxygen_flow:
|
|
199
|
+
code: '"NURSING//OXYGEN_FLOW" if $Oxygen_flow::?float64 >= 0'
|
|
200
|
+
time: $chart_time
|
|
201
|
+
numeric_value: $Oxygen_flow::?float64
|
|
202
|
+
inpatient_stay_id: $INP_NO
|
|
203
|
+
ventilation_mode:
|
|
204
|
+
code: '"NURSING//VENTILATION_MODE" if $Breathing_pattern != "NA" and /\w/ in $Breathing_pattern'
|
|
205
|
+
time: $chart_time
|
|
206
|
+
text_value: $Breathing_pattern
|
|
207
|
+
inpatient_stay_id: $INP_NO
|
|
208
|
+
tidal_volume_setting:
|
|
209
|
+
code: '"NURSING//TIDAL_VOLUME_SETTING" if $Vt_setting::?float64 >= 0'
|
|
210
|
+
time: $chart_time
|
|
211
|
+
numeric_value: $Vt_setting::?float64
|
|
212
|
+
inpatient_stay_id: $INP_NO
|
|
213
|
+
monitored_tidal_volume:
|
|
214
|
+
code: '"NURSING//MONITORED_TIDAL_VOLUME" if $Vt_Supervisor::?float64 >= 0'
|
|
215
|
+
time: $chart_time
|
|
216
|
+
numeric_value: $Vt_Supervisor::?float64
|
|
217
|
+
inpatient_stay_id: $INP_NO
|
|
218
|
+
consciousness:
|
|
219
|
+
code: '"NURSING//CONSCIOUSNESS" if $consciousness != "NA" and /\w/ in $consciousness'
|
|
220
|
+
time: $chart_time
|
|
221
|
+
text_value: $consciousness
|
|
222
|
+
inpatient_stay_id: $INP_NO
|
|
223
|
+
left_pupil_size:
|
|
224
|
+
code: '"NURSING//LEFT_PUPIL_SIZE" if $Left_pupil_size::?float64 >= 0'
|
|
225
|
+
time: $chart_time
|
|
226
|
+
numeric_value: $Left_pupil_size::?float64
|
|
227
|
+
inpatient_stay_id: $INP_NO
|
|
228
|
+
right_pupil_size:
|
|
229
|
+
code: '"NURSING//RIGHT_PUPIL_SIZE" if $Right_pupil_size::?float64 >= 0'
|
|
230
|
+
time: $chart_time
|
|
231
|
+
numeric_value: $Right_pupil_size::?float64
|
|
232
|
+
inpatient_stay_id: $INP_NO
|
|
233
|
+
# GCS components are charted as "<score>→<Chinese label>", e.g. "3→呼唤睁眼". The leading score
|
|
234
|
+
# is the numeric_value; non-scorable entries ("T" intubated, "S" motor disorder) keep only the
|
|
235
|
+
# text.
|
|
236
|
+
gcs_eye_response:
|
|
237
|
+
code: '"NURSING//GCS_EYE_RESPONSE" if $`open_one''s_eyes` != "NA" and /\w/ in $`open_one''s_eyes`'
|
|
238
|
+
time: $chart_time
|
|
239
|
+
numeric_value: '(extract group 1 of /^\s*([1-4])\s*[^\d\s]/ from $`open_one''s_eyes`)::?float64'
|
|
240
|
+
text_value: $`open_one's_eyes`
|
|
241
|
+
inpatient_stay_id: $INP_NO
|
|
242
|
+
gcs_motor_response:
|
|
243
|
+
code: '"NURSING//GCS_MOTOR_RESPONSE" if $motion != "NA" and /\w/ in $motion'
|
|
244
|
+
time: $chart_time
|
|
245
|
+
numeric_value: '(extract group 1 of /^\s*([1-6])\s*[^\d\s]/ from $motion)::?float64'
|
|
246
|
+
text_value: $motion
|
|
247
|
+
inpatient_stay_id: $INP_NO
|
|
248
|
+
gcs_verbal_response:
|
|
249
|
+
code: '"NURSING//GCS_VERBAL_RESPONSE" if $language != "NA" and /\w/ in $language'
|
|
250
|
+
time: $chart_time
|
|
251
|
+
numeric_value: '(extract group 1 of /^\s*([1-5])\s*[^\d\s]/ from $language)::?float64'
|
|
252
|
+
text_value: $language
|
|
253
|
+
inpatient_stay_id: $INP_NO
|
|
254
|
+
# Line and tube columns hold the charted length in cm, or a status such as "拔除" (removed) or
|
|
255
|
+
# "不详" (unknown); the length is the numeric_value and the raw entry stays in text_value. The
|
|
256
|
+
# `::str` casts keep the text guards valid whichever dtype the column is inferred as.
|
|
257
|
+
cvc:
|
|
258
|
+
code: '"NURSING//CVC" if $CVC::str != "NA" and /\w/ in $CVC::str'
|
|
259
|
+
time: $chart_time
|
|
260
|
+
numeric_value: $CVC::?float64
|
|
261
|
+
text_value: $CVC::str
|
|
262
|
+
inpatient_stay_id: $INP_NO
|
|
263
|
+
picc:
|
|
264
|
+
code: '"NURSING//PICC" if $PICC::str != "NA" and /\w/ in $PICC::str'
|
|
265
|
+
time: $chart_time
|
|
266
|
+
numeric_value: $PICC::?float64
|
|
267
|
+
text_value: $PICC::str
|
|
268
|
+
inpatient_stay_id: $INP_NO
|
|
269
|
+
# Every numeric Endotracheal_intubation value equals Endotracheal_intubation_Depth on the same
|
|
270
|
+
# row, so only its non-numeric statuses (e.g. "拔除", removed) are emitted here; the depth is
|
|
271
|
+
# emitted by endotracheal_intubation_depth below.
|
|
272
|
+
endotracheal_intubation:
|
|
273
|
+
code: >-
|
|
274
|
+
"NURSING//ENDOTRACHEAL_INTUBATION"
|
|
275
|
+
if $Endotracheal_intubation::str != "NA" and /\w/ in $Endotracheal_intubation::str
|
|
276
|
+
and /[^0-9.\s]/ in $Endotracheal_intubation::str
|
|
277
|
+
time: $chart_time
|
|
278
|
+
text_value: $Endotracheal_intubation::str
|
|
279
|
+
inpatient_stay_id: $INP_NO
|
|
280
|
+
gastric_tube:
|
|
281
|
+
code: '"NURSING//GASTRIC_TUBE" if $Gastric_tube::str != "NA" and /\w/ in $Gastric_tube::str'
|
|
282
|
+
time: $chart_time
|
|
283
|
+
numeric_value: $Gastric_tube::?float64
|
|
284
|
+
text_value: $Gastric_tube::str
|
|
285
|
+
inpatient_stay_id: $INP_NO
|
|
286
|
+
medium_length_catheter:
|
|
287
|
+
code: '"NURSING//MEDIUM_LENGTH_CATHETER" if $Medium_length_catheter::str != "NA" and /\w/ in $Medium_length_catheter::str'
|
|
288
|
+
time: $chart_time
|
|
289
|
+
numeric_value: $Medium_length_catheter::?float64
|
|
290
|
+
text_value: $Medium_length_catheter::str
|
|
291
|
+
inpatient_stay_id: $INP_NO
|
|
292
|
+
hemodialysis_tube:
|
|
293
|
+
code: '"NURSING//HEMODIALYSIS_TUBE" if $Hemodialysis_tube::str != "NA" and /\w/ in $Hemodialysis_tube::str'
|
|
294
|
+
time: $chart_time
|
|
295
|
+
numeric_value: $Hemodialysis_tube::?float64
|
|
296
|
+
text_value: $Hemodialysis_tube::str
|
|
297
|
+
inpatient_stay_id: $INP_NO
|
|
298
|
+
basic_nursing:
|
|
299
|
+
code: '"NURSING//BASIC_NURSING" if $basic_nursing != "NA" and /\w/ in $basic_nursing'
|
|
300
|
+
time: $chart_time
|
|
301
|
+
text_value: $basic_nursing
|
|
302
|
+
inpatient_stay_id: $INP_NO
|
|
303
|
+
skin_care:
|
|
304
|
+
code: '"NURSING//SKIN_CARE" if $Skin_care != "NA" and /\w/ in $Skin_care'
|
|
305
|
+
time: $chart_time
|
|
306
|
+
text_value: $Skin_care
|
|
307
|
+
inpatient_stay_id: $INP_NO
|
|
308
|
+
acral_circulation:
|
|
309
|
+
code: '"NURSING//ACRAL_CIRCULATION" if $Acral_circulation != "NA" and /\w/ in $Acral_circulation'
|
|
310
|
+
time: $chart_time
|
|
311
|
+
text_value: $Acral_circulation
|
|
312
|
+
inpatient_stay_id: $INP_NO
|
|
313
|
+
pressure_ulcer_risk:
|
|
314
|
+
code: '"NURSING//PRESSURE_ULCER_RISK" if $Pressure_ulcer_risk_assessment::?float64 >= 0'
|
|
315
|
+
time: $chart_time
|
|
316
|
+
numeric_value: $Pressure_ulcer_risk_assessment::?float64
|
|
317
|
+
inpatient_stay_id: $INP_NO
|
|
318
|
+
catheter_drop_risk:
|
|
319
|
+
code: '"NURSING//CATHETER_DROP_RISK" if $Catheter_drop_assessment::?float64 >= 0'
|
|
320
|
+
time: $chart_time
|
|
321
|
+
numeric_value: $Catheter_drop_assessment::?float64
|
|
322
|
+
inpatient_stay_id: $INP_NO
|
|
323
|
+
fall_risk:
|
|
324
|
+
code: '"NURSING//FALL_RISK" if $Fall_bed_assessment::?float64 >= 0'
|
|
325
|
+
time: $chart_time
|
|
326
|
+
numeric_value: $Fall_bed_assessment::?float64
|
|
327
|
+
inpatient_stay_id: $INP_NO
|
|
328
|
+
# Scores outside the scale's defined range (RASS -5..+4, NRS 0..10, CPOT 0..8) are charting
|
|
329
|
+
# errors, e.g. negative RASS-like values in the NRS column, and are dropped.
|
|
330
|
+
rass_sedation_score:
|
|
331
|
+
code: >-
|
|
332
|
+
"NURSING//RASS_SEDATION_SCORE"
|
|
333
|
+
if $RASS_sedation_score::?float64 >= -5 and $RASS_sedation_score::?float64 <= 4
|
|
334
|
+
time: $chart_time
|
|
335
|
+
numeric_value: $RASS_sedation_score::?float64
|
|
336
|
+
inpatient_stay_id: $INP_NO
|
|
337
|
+
nrs_pain_score:
|
|
338
|
+
code: '"NURSING//NRS_PAIN_SCORE" if $NRS_pain_score::?float64 >= 0 and $NRS_pain_score::?float64 <= 10'
|
|
339
|
+
time: $chart_time
|
|
340
|
+
numeric_value: $NRS_pain_score::?float64
|
|
341
|
+
inpatient_stay_id: $INP_NO
|
|
342
|
+
cpot_pain_score:
|
|
343
|
+
code: '"NURSING//CPOT_PAIN_SCORE" if $Cpot_pain_score::?float64 >= 0 and $Cpot_pain_score::?float64 <= 8'
|
|
344
|
+
time: $chart_time
|
|
345
|
+
numeric_value: $Cpot_pain_score::?float64
|
|
346
|
+
inpatient_stay_id: $INP_NO
|
|
347
|
+
airbag_pressure:
|
|
348
|
+
code: '"NURSING//AIRBAG_PRESSURE" if $Airbag_pressure::?float64 >= 0'
|
|
349
|
+
time: $chart_time
|
|
350
|
+
numeric_value: $Airbag_pressure::?float64
|
|
351
|
+
inpatient_stay_id: $INP_NO
|
|
352
|
+
milk_intake:
|
|
353
|
+
code: '"NURSING//MILK_INTAKE" if $milk::?float64 >= 0'
|
|
354
|
+
time: $chart_time
|
|
355
|
+
numeric_value: $milk::?float64
|
|
356
|
+
inpatient_stay_id: $INP_NO
|
|
357
|
+
rice_intake:
|
|
358
|
+
code: '"NURSING//RICE_INTAKE" if $rice::?float64 >= 0'
|
|
359
|
+
time: $chart_time
|
|
360
|
+
numeric_value: $rice::?float64
|
|
361
|
+
inpatient_stay_id: $INP_NO
|
|
362
|
+
rice_noodles_intake:
|
|
363
|
+
code: '"NURSING//RICE_NOODLES_INTAKE" if $Rice_noodles::?float64 >= 0'
|
|
364
|
+
time: $chart_time
|
|
365
|
+
numeric_value: $Rice_noodles::?float64
|
|
366
|
+
inpatient_stay_id: $INP_NO
|
|
367
|
+
soup_intake:
|
|
368
|
+
code: '"NURSING//SOUP_INTAKE" if $soup::?float64 >= 0'
|
|
369
|
+
time: $chart_time
|
|
370
|
+
numeric_value: $soup::?float64
|
|
371
|
+
inpatient_stay_id: $INP_NO
|
|
372
|
+
traditional_chinese_medicine_intake:
|
|
373
|
+
code: '"NURSING//TRADITIONAL_CHINESE_MEDICINE_INTAKE" if $traditional_Chinese_medicine::?float64 >= 0'
|
|
374
|
+
time: $chart_time
|
|
375
|
+
numeric_value: $traditional_Chinese_medicine::?float64
|
|
376
|
+
inpatient_stay_id: $INP_NO
|
|
377
|
+
soft_food_intake:
|
|
378
|
+
code: '"NURSING//SOFT_FOOD_INTAKE" if $Soft_food::?float64 >= 0'
|
|
379
|
+
time: $chart_time
|
|
380
|
+
numeric_value: $Soft_food::?float64
|
|
381
|
+
inpatient_stay_id: $INP_NO
|
|
382
|
+
rice_soup_intake:
|
|
383
|
+
code: '"NURSING//RICE_SOUP_INTAKE" if $the_rice_soup::?float64 >= 0'
|
|
384
|
+
time: $chart_time
|
|
385
|
+
numeric_value: $the_rice_soup::?float64
|
|
386
|
+
inpatient_stay_id: $INP_NO
|
|
387
|
+
warm_water_intake:
|
|
388
|
+
code: '"NURSING//WARM_WATER_INTAKE" if $warm_water::?float64 >= 0'
|
|
389
|
+
time: $chart_time
|
|
390
|
+
numeric_value: $warm_water::?float64
|
|
391
|
+
inpatient_stay_id: $INP_NO
|
|
392
|
+
fruit_juice_intake:
|
|
393
|
+
code: '"NURSING//FRUIT_JUICE_INTAKE" if $fruit_juice::?float64 >= 0'
|
|
394
|
+
time: $chart_time
|
|
395
|
+
numeric_value: $fruit_juice::?float64
|
|
396
|
+
inpatient_stay_id: $INP_NO
|
|
397
|
+
stool_output:
|
|
398
|
+
code: '"NURSING//STOOL_OUTPUT" if $shit::?float64 >= 0'
|
|
399
|
+
time: $chart_time
|
|
400
|
+
numeric_value: $shit::?float64
|
|
401
|
+
inpatient_stay_id: $INP_NO
|
|
402
|
+
urine_output:
|
|
403
|
+
code: '"NURSING//URINE_OUTPUT" if $Urine_volume::?float64 >= 0'
|
|
404
|
+
time: $chart_time
|
|
405
|
+
numeric_value: $Urine_volume::?float64
|
|
406
|
+
inpatient_stay_id: $INP_NO
|
|
407
|
+
gastric_juice_output:
|
|
408
|
+
code: '"NURSING//GASTRIC_JUICE_OUTPUT" if $gastric_juice::?float64 >= 0'
|
|
409
|
+
time: $chart_time
|
|
410
|
+
numeric_value: $gastric_juice::?float64
|
|
411
|
+
inpatient_stay_id: $INP_NO
|
|
412
|
+
sputum_output:
|
|
413
|
+
code: '"NURSING//SPUTUM_OUTPUT" if $Sputum_volume::?float64 >= 0'
|
|
414
|
+
time: $chart_time
|
|
415
|
+
numeric_value: $Sputum_volume::?float64
|
|
416
|
+
inpatient_stay_id: $INP_NO
|
|
417
|
+
nursing_note:
|
|
418
|
+
code: '"NURSING//NOTE" if $NURSING_DESC != "NA" and /\w/ in $NURSING_DESC'
|
|
419
|
+
time: $chart_time
|
|
420
|
+
text_value: $NURSING_DESC
|
|
421
|
+
inpatient_stay_id: $INP_NO
|
|
422
|
+
endotracheal_intubation_depth:
|
|
423
|
+
code: '"NURSING//ENDOTRACHEAL_INTUBATION_DEPTH" if $Endotracheal_intubation_Depth::?float64 >= 0'
|
|
424
|
+
time: $chart_time
|
|
425
|
+
numeric_value: $Endotracheal_intubation_Depth::?float64
|
|
426
|
+
inpatient_stay_id: $INP_NO
|
|
427
|
+
extubation:
|
|
428
|
+
code: '"NURSING//EXTUBATION" if $Extubation == true'
|
|
429
|
+
time: $chart_time
|
|
430
|
+
inpatient_stay_id: $INP_NO
|
|
431
|
+
|
|
432
|
+
# One row per patient. Follow-up answers are timed at Follow_Date; rows without one (247) produce
|
|
433
|
+
# no follow-up events.
|
|
434
|
+
dtOutCome:
|
|
435
|
+
_table:
|
|
436
|
+
join:
|
|
437
|
+
dtBaseline:
|
|
438
|
+
key: PATIENT_ID
|
|
439
|
+
cols: [INP_NO, DISCHARGE_DATE_TIME]
|
|
440
|
+
follow_up_vital_status:
|
|
441
|
+
code: 'f"FOLLOW_UP_VITAL_STATUS//{$Follow_Vital}" if $Follow_Vital != "NA" and $Follow_Vital != ""'
|
|
442
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
443
|
+
# Death_Date has about day precision: in-hospital deaths typically fall ~12 h before the recorded
|
|
444
|
+
# discharge, so a stay's last events can follow MEDS_DEATH.
|
|
445
|
+
death:
|
|
446
|
+
code: MEDS_DEATH
|
|
447
|
+
time: "(1970-01-01 @ 00:00) + $Death_Date::?float64::hours"
|
|
448
|
+
sf36_general_health:
|
|
449
|
+
code: 'f"SF36//GENERAL_HEALTH//{$SF36_GeneralHeath}" if $SF36_GeneralHeath != "NA" and $SF36_GeneralHeath != ""'
|
|
450
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
451
|
+
sf36_sicker_than_others:
|
|
452
|
+
code: 'f"SF36//SICKER_THAN_OTHERS//{$SF36_GH_SickerEasierThanOthers}" if $SF36_GH_SickerEasierThanOthers != "NA" and $SF36_GH_SickerEasierThanOthers != ""'
|
|
453
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
454
|
+
sf36_healthier_than_others:
|
|
455
|
+
code: 'f"SF36//HEALTHIER_THAN_OTHERS//{$SF36_GH_HeathyasOthers}" if $SF36_GH_HeathyasOthers != "NA" and $SF36_GH_HeathyasOthers != ""'
|
|
456
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
457
|
+
sf36_health_get_worse:
|
|
458
|
+
code: 'f"SF36//HEALTH_GET_WORSE//{$SF36_GH_GetWorse}" if $SF36_GH_GetWorse != "NA" and $SF36_GH_GetWorse != ""'
|
|
459
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
460
|
+
sf36_excellent_health:
|
|
461
|
+
code: 'f"SF36//EXCELLENT_HEALTH//{$SF36_GH_ExcellentHealth}" if $SF36_GH_ExcellentHealth != "NA" and $SF36_GH_ExcellentHealth != ""'
|
|
462
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
463
|
+
sf36_vigorous_activity:
|
|
464
|
+
code: 'f"SF36//ACTIVITY_LIMIT_VIGOROUS//{$SF36_ActivityLimit_VigorousActivity}" if $SF36_ActivityLimit_VigorousActivity != "NA" and $SF36_ActivityLimit_VigorousActivity != ""'
|
|
465
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
466
|
+
sf36_moderate_activity:
|
|
467
|
+
code: 'f"SF36//ACTIVITY_LIMIT_MODERATE//{$SF36_ActivityLimit_ModerateActivity}" if $SF36_ActivityLimit_ModerateActivity != "NA" and $SF36_ActivityLimit_ModerateActivity != ""'
|
|
468
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
469
|
+
sf36_lifting_groceries:
|
|
470
|
+
code: 'f"SF36//ACTIVITY_LIMIT_LIFTING_GROCERIES//{$SF36_ActivityLimit_LiftingGroceries}" if $SF36_ActivityLimit_LiftingGroceries != "NA" and $SF36_ActivityLimit_LiftingGroceries != ""'
|
|
471
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
472
|
+
sf36_climbing_several_flights:
|
|
473
|
+
code: 'f"SF36//ACTIVITY_LIMIT_CLIMBING_SEVERAL_FLIGHTS//{$SF36_ActivityLimit_ClimingSeveralFlights}" if $SF36_ActivityLimit_ClimingSeveralFlights != "NA" and $SF36_ActivityLimit_ClimingSeveralFlights != ""'
|
|
474
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
475
|
+
sf36_climbing_one_flight:
|
|
476
|
+
code: 'f"SF36//ACTIVITY_LIMIT_CLIMBING_ONE_FLIGHT//{$SF36_ActivityLimit_ClimingOneFlight}" if $SF36_ActivityLimit_ClimingOneFlight != "NA" and $SF36_ActivityLimit_ClimingOneFlight != ""'
|
|
477
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
478
|
+
sf36_bending_kneeling_stooping:
|
|
479
|
+
code: 'f"SF36//ACTIVITY_LIMIT_BENDING_KNEELING_STOOPING//{$SF36_ActivityLimit_BendingKneelStoop}" if $SF36_ActivityLimit_BendingKneelStoop != "NA" and $SF36_ActivityLimit_BendingKneelStoop != ""'
|
|
480
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
481
|
+
sf36_walking_more_than_1500m:
|
|
482
|
+
code: 'f"SF36//ACTIVITY_LIMIT_WALKING_MORE_THAN_1500M//{$SF36_ActivityLimit_WalkingMore1500M}" if $SF36_ActivityLimit_WalkingMore1500M != "NA" and $SF36_ActivityLimit_WalkingMore1500M != ""'
|
|
483
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
484
|
+
sf36_walking_1000m:
|
|
485
|
+
code: 'f"SF36//ACTIVITY_LIMIT_WALKING_1000M//{$SF36_ActivityLimit_Walking1000M}" if $SF36_ActivityLimit_Walking1000M != "NA" and $SF36_ActivityLimit_Walking1000M != ""'
|
|
486
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
487
|
+
sf36_walking_100m:
|
|
488
|
+
code: 'f"SF36//ACTIVITY_LIMIT_WALKING_100M//{$SF36_ActivityLimit_Walking100M}" if $SF36_ActivityLimit_Walking100M != "NA" and $SF36_ActivityLimit_Walking100M != ""'
|
|
489
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
490
|
+
sf36_bathing_dressing:
|
|
491
|
+
code: 'f"SF36//ACTIVITY_LIMIT_BATHING_DRESSING//{$SF36_ActivityLimit_BathingDressing}" if $SF36_ActivityLimit_BathingDressing != "NA" and $SF36_ActivityLimit_BathingDressing != ""'
|
|
492
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
493
|
+
sf36_body_pain:
|
|
494
|
+
code: 'f"SF36//BODY_PAIN//{$SF36_Pain_BodyPainPast4wk}" if $SF36_Pain_BodyPainPast4wk != "NA" and $SF36_Pain_BodyPainPast4wk != ""'
|
|
495
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
496
|
+
sf36_pain_interferes_housework:
|
|
497
|
+
code: 'f"SF36//PAIN_INTERFERES_HOUSEWORK//{$SF36_Pain_BodyPainPast4wkInterHousework}" if $SF36_Pain_BodyPainPast4wkInterHousework != "NA" and $SF36_Pain_BodyPainPast4wkInterHousework != ""'
|
|
498
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
499
|
+
sf36_physical_health_cut_work:
|
|
500
|
+
code: 'f"SF36//PHYSICAL_HEALTH_CUT_WORK//{$SF36_PhysicalHeath_CutWork}" if $SF36_PhysicalHeath_CutWork != "NA" and $SF36_PhysicalHeath_CutWork != ""'
|
|
501
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
502
|
+
sf36_physical_health_accomplish_less:
|
|
503
|
+
code: 'f"SF36//PHYSICAL_HEALTH_ACCOMPLISH_LESS//{$SF36_PhysicalHeath_AccomplishLess}" if $SF36_PhysicalHeath_AccomplishLess != "NA" and $SF36_PhysicalHeath_AccomplishLess != ""'
|
|
504
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
505
|
+
sf36_physical_health_limited_work:
|
|
506
|
+
code: 'f"SF36//PHYSICAL_HEALTH_LIMITED_WORK//{$SF36_PhysicalHeath_LimitedWork}" if $SF36_PhysicalHeath_LimitedWork != "NA" and $SF36_PhysicalHeath_LimitedWork != ""'
|
|
507
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
508
|
+
sf36_physical_health_difficult_work:
|
|
509
|
+
code: 'f"SF36//PHYSICAL_HEALTH_DIFFICULT_WORK//{$SF36_PhysicalHeath_DiffiPerformingWork}" if $SF36_PhysicalHeath_DiffiPerformingWork != "NA" and $SF36_PhysicalHeath_DiffiPerformingWork != ""'
|
|
510
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
511
|
+
sf36_emotional_nervous:
|
|
512
|
+
code: 'f"SF36//EMOTIONAL_NERVOUS//{$SF36_Emotional_Nervous}" if $SF36_Emotional_Nervous != "NA" and $SF36_Emotional_Nervous != ""'
|
|
513
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
514
|
+
sf36_emotional_down_nothing_cheer:
|
|
515
|
+
code: 'f"SF36//EMOTIONAL_DOWN_NOTHING_CHEER//{$SF36_Emotional_DownNothingCheer}" if $SF36_Emotional_DownNothingCheer != "NA" and $SF36_Emotional_DownNothingCheer != ""'
|
|
516
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
517
|
+
sf36_emotional_calm:
|
|
518
|
+
code: 'f"SF36//EMOTIONAL_CALM//{$SF36_Emotional_Calm}" if $SF36_Emotional_Calm != "NA" and $SF36_Emotional_Calm != ""'
|
|
519
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
520
|
+
sf36_emotional_downhearted:
|
|
521
|
+
code: 'f"SF36//EMOTIONAL_DOWNHEARTED//{$SF36_Emotional_DownHearted}" if $SF36_Emotional_DownHearted != "NA" and $SF36_Emotional_DownHearted != ""'
|
|
522
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
523
|
+
sf36_emotional_happy:
|
|
524
|
+
code: 'f"SF36//EMOTIONAL_HAPPY//{$SF36_Emotional_HappyPerson}" if $SF36_Emotional_HappyPerson != "NA" and $SF36_Emotional_HappyPerson != ""'
|
|
525
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
526
|
+
sf36_emotional_energy:
|
|
527
|
+
code: 'f"SF36//EMOTIONAL_ENERGY//{$SF36_Emotional_aLotEnergy}" if $SF36_Emotional_aLotEnergy != "NA" and $SF36_Emotional_aLotEnergy != ""'
|
|
528
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
529
|
+
sf36_emotional_pep:
|
|
530
|
+
code: 'f"SF36//EMOTIONAL_PEP//{$SF36_Emotional_FullPep}" if $SF36_Emotional_FullPep != "NA" and $SF36_Emotional_FullPep != ""'
|
|
531
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
532
|
+
sf36_emotional_worn_out:
|
|
533
|
+
code: 'f"SF36//EMOTIONAL_WORN_OUT//{$SF36_Emotional_WornOut}" if $SF36_Emotional_WornOut != "NA" and $SF36_Emotional_WornOut != ""'
|
|
534
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
535
|
+
sf36_emotional_tired:
|
|
536
|
+
code: 'f"SF36//EMOTIONAL_TIRED//{$SF36_Emotional_Tired}" if $SF36_Emotional_Tired != "NA" and $SF36_Emotional_Tired != ""'
|
|
537
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
538
|
+
sf36_social_activity_emotional_or_physical:
|
|
539
|
+
code: 'f"SF36//SOCIAL_ACTIVITY_EMOTIONAL_OR_PHYSICAL//{$SF36_4wksSocialActivityEmotionOrPhysical}" if $SF36_4wksSocialActivityEmotionOrPhysical != "NA" and $SF36_4wksSocialActivityEmotionOrPhysical != ""'
|
|
540
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
541
|
+
sf36_no_interference_social_activity:
|
|
542
|
+
code: 'f"SF36//NO_INTERFERENCE_SOCIAL_ACTIVITY//{$SF36_NoInfSocialActivity}" if $SF36_NoInfSocialActivity != "NA" and $SF36_NoInfSocialActivity != ""'
|
|
543
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
544
|
+
sf36_reduced_time_emotional_or_physical:
|
|
545
|
+
code: 'f"SF36//REDUCED_TIME_EMOTIONAL_OR_PHYSICAL//{$SF36_ReducedTime4wksSocialActivityEmotion}" if $SF36_ReducedTime4wksSocialActivityEmotion != "NA" and $SF36_ReducedTime4wksSocialActivityEmotion != ""'
|
|
546
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
547
|
+
sf36_reduced_amount_emotional_or_physical:
|
|
548
|
+
code: 'f"SF36//REDUCED_AMOUNT_EMOTIONAL_OR_PHYSICAL//{$SF36_ReducedAmount4wksSocialActivityEmotion}" if $SF36_ReducedAmount4wksSocialActivityEmotion != "NA" and $SF36_ReducedAmount4wksSocialActivityEmotion != ""'
|
|
549
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
550
|
+
sf36_reduced_quality_emotional_or_physical:
|
|
551
|
+
code: 'f"SF36//REDUCED_QUALITY_EMOTIONAL_OR_PHYSICAL//{$SF36_ReducedQuality4wksSocialActivityEmotion}" if $SF36_ReducedQuality4wksSocialActivityEmotion != "NA" and $SF36_ReducedQuality4wksSocialActivityEmotion != ""'
|
|
552
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
553
|
+
sf36_one_year_health_comparison:
|
|
554
|
+
code: 'f"SF36//ONE_YEAR_HEALTH_COMPARISON//{$SF36_OneYearCompareHealthCondition}" if $SF36_OneYearCompareHealthCondition != "NA" and $SF36_OneYearCompareHealthCondition != ""'
|
|
555
|
+
time: "(1970-01-01 @ 00:00) + $Follow_Date::?float64::hours"
|
|
556
|
+
discharge_department:
|
|
557
|
+
code: f"DISCHARGE_DEPARTMENT//{$Discharge_Dept}"
|
|
558
|
+
time: "(1970-01-01 @ 00:00) + $DISCHARGE_DATE_TIME::hours"
|
|
559
|
+
inpatient_stay_id: $INP_NO
|
|
560
|
+
|
|
561
|
+
sources:
|
|
562
|
+
# The raw release version. Reserved (never treated as a bucket), interpolatable into the entries
|
|
563
|
+
# below, and stamped by `meds-extract-run` into the output's `etl_metadata.dataset_version`.
|
|
564
|
+
dataset_version: "1.1"
|
|
565
|
+
dataset:
|
|
566
|
+
# Zigong is credentialed: a PhysioNet account with a signed DUA for this project is required.
|
|
567
|
+
- type: physionet
|
|
568
|
+
base_url: https://physionet.org/files/icu-infection-zigong-fourth/${sources.dataset_version}
|
|
569
|
+
username: ${oc.env:DATASET_DOWNLOAD_USERNAME}
|
|
570
|
+
password: ${oc.env:DATASET_DOWNLOAD_PASSWORD}
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: Zigong-MEDS
|
|
3
|
+
Version: 0.3.0
|
|
4
|
+
Summary: An ETL pipeline to extract the Zigong Fourth People's Hospital ICU infection database into the MEDS format.
|
|
5
|
+
Author-email: Robin van de Water <rpv2114@cumc.columbia.edu>
|
|
6
|
+
Project-URL: Homepage, https://github.com/rvandewater/ZIGONG_MEDS
|
|
7
|
+
Project-URL: Issues, https://github.com/rvandewater/ZIGONG_MEDS/issues
|
|
8
|
+
Project-URL: Dataset, https://physionet.org/content/icu-infection-zigong-fourth/1.1/
|
|
9
|
+
Classifier: Programming Language :: Python :: 3
|
|
10
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Requires-Python: <3.14,>=3.11.4
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: meds-extract[download]<0.8,>=0.7.0
|
|
16
|
+
Dynamic: license-file
|
|
17
|
+
|
|
18
|
+
# Zigong MEDS Extraction ETL
|
|
19
|
+
|
|
20
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/tests.yaml)
|
|
21
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/synthetic-etl.yaml)
|
|
22
|
+
[](https://github.com/rvandewater/ZIGONG_MEDS/actions/workflows/code-quality-main.yaml)
|
|
23
|
+

|
|
24
|
+
[](https://github.com/mmcdermott/MEDS_extract)
|
|
25
|
+
[](LICENSE)
|
|
26
|
+
|
|
27
|
+
This pipeline extracts the
|
|
28
|
+
[Zigong Fourth People's Hospital ICU infection database v1.1](https://physionet.org/content/icu-infection-zigong-fourth/1.1/)
|
|
29
|
+
from PhysioNet into the [MEDS format](https://medical-event-data-standard.github.io/). The release holds
|
|
30
|
+
2,790 ICU patients with infection, one hospital admission each, from Zigong Fourth People's Hospital in
|
|
31
|
+
Sichuan, China. It was built from the
|
|
32
|
+
[ETL_MEDS_Template](https://github.com/Medical-Event-Data-Standard/ETL_MEDS_Template) and uses the
|
|
33
|
+
config-only layout introduced by [MEDS-Extract](https://github.com/mmcdermott/MEDS_extract) 0.7.
|
|
34
|
+
|
|
35
|
+
## Usage
|
|
36
|
+
|
|
37
|
+
Zigong is a credentialed dataset. You need a PhysioNet account that has signed the project's data use
|
|
38
|
+
agreement.
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
pip install git+https://github.com/rvandewater/ZIGONG_MEDS.git
|
|
42
|
+
export DATASET_DOWNLOAD_USERNAME=$PHYSIONET_USERNAME
|
|
43
|
+
export DATASET_DOWNLOAD_PASSWORD=$PHYSIONET_PASSWORD
|
|
44
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
When you run this, the program will:
|
|
48
|
+
|
|
49
|
+
1. Download the release (seven `dt*.csv` tables, about 840 MB) from PhysioNet into
|
|
50
|
+
`$MEDS_OUTPUT_DIR/.meds_extract_run/raw_input`. Pass `download_dest_dir=$RAW_INPUT_DIR` to keep the raw
|
|
51
|
+
data somewhere durable and reuse it across runs.
|
|
52
|
+
2. Build the MEDS cohort from those files and write it to `$MEDS_OUTPUT_DIR` (`data/` and `metadata/`).
|
|
53
|
+
All transformations and joins are declared in
|
|
54
|
+
[`src/Zigong_MEDS/event_configs.yaml`](src/Zigong_MEDS/event_configs.yaml).
|
|
55
|
+
|
|
56
|
+
If you already have the PhysioNet release on disk, skip the download and point `input_dir` at the
|
|
57
|
+
directory that contains `dtBaseline.csv`, `dtDrugs.csv`, `dtICD.csv`, `dtLab.csv`, `dtNursingChart.csv`,
|
|
58
|
+
`dtOutCome.csv` and `dtTransfer.csv`:
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR \
|
|
62
|
+
do_download=false input_dir=/path/to/icu-infection-zigong-fourth-1.1
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
To parallelize the stages, pass a runner file:
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
printf 'parallelize:\n n_workers: 8\n launcher: joblib\n' >runner.yaml
|
|
69
|
+
meds-extract-run spec=Zigong output_dir=$MEDS_OUTPUT_DIR stage_runner_fp=runner.yaml
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Run `meds-extract-run --help` for every option.
|
|
73
|
+
|
|
74
|
+
## How this ETL is defined
|
|
75
|
+
|
|
76
|
+
The package contains no Python. The whole ETL is one MESSY (MEDS-Extract Specification Syntax YAML)
|
|
77
|
+
file, [`src/Zigong_MEDS/event_configs.yaml`](src/Zigong_MEDS/event_configs.yaml). It declares:
|
|
78
|
+
|
|
79
|
+
- **`sources:`**: where the raw data lives (the PhysioNet release).
|
|
80
|
+
- **`etl:`**: the dataset name and sharding options.
|
|
81
|
+
- **One block per raw table**: how each row becomes one or more MEDS events.
|
|
82
|
+
|
|
83
|
+
`pyproject.toml` registers the file with MEDS-Extract's `MEDS_extract.pipelines` entry-point group under
|
|
84
|
+
the name `Zigong`. That registration is what makes `spec=Zigong` work. No pre-MEDS step is needed:
|
|
85
|
+
tables without `PATIENT_ID` (`dtLab`, `dtNursingChart`) are joined to `dtBaseline` on `INP_NO` inside the
|
|
86
|
+
config.
|
|
87
|
+
|
|
88
|
+
**Times.** Every source time is an offset in hours from hospital admission. The ETL places them on a
|
|
89
|
+
shared pseudotime anchor, 1970-01-01 00:00 = admission, which preserves order and intervals without
|
|
90
|
+
claiming a real calendar date. Each patient has exactly one admission, so offsets never collide.
|
|
91
|
+
|
|
92
|
+
## Events
|
|
93
|
+
|
|
94
|
+
| Table | Event codes | Notes |
|
|
95
|
+
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
96
|
+
| `dtBaseline` | `HOSPITAL_ADMISSION`, `AGE_AT_ADMISSION`, `SEX//…`, `ADMISSION_DEPARTMENT//…`, `INFECTION_SITE//…`, `ICU_DISCHARGE`, `HOSPITAL_DISCHARGE` | Age is the `numeric_value`. |
|
|
97
|
+
| `dtTransfer` | `TRANSFER_TO//{dept}`, `TRANSFER_FROM//{dept}` | The reason is kept in `transfer_reason`. See [#3](https://github.com/rvandewater/ZIGONG_MEDS/issues/3). |
|
|
98
|
+
| `dtICD` | `DIAGNOSIS//ICD//{code}` | Placed at hospital discharge. The clinician's description is the `text_value`; `diagnosis_sequence` and `discharge_diagnosis_status` are kept. |
|
|
99
|
+
| `dtLab` | `LAB//{category}//{item}//{unit}` | A missing unit becomes `UNK`. `reference_range` is kept. |
|
|
100
|
+
| `dtDrugs` | `MEDICATION//{drug}` | Prescription time. `Formula` is the `text_value`. |
|
|
101
|
+
| `dtNursingChart` | `NURSING//{MEASUREMENT}` (vitals, ventilation, consciousness, GCS components, lines and tubes, risk and pain scores, intake and output, notes, …) | One code per charted column. Vital signs carry a `unit` column. GCS components carry the score as the `numeric_value` and the charted label as text. |
|
|
102
|
+
| `dtOutCome` | `MEDS_DEATH`, `FOLLOW_UP_VITAL_STATUS//…`, `SF36//{ITEM}//{answer}`, `DISCHARGE_DEPARTMENT//…` | Follow-up events are placed at `Follow_Date`; the discharge department at hospital discharge. |
|
|
103
|
+
|
|
104
|
+
Every event linked to a stay carries `inpatient_stay_id` (`INP_NO`). The R-written CSVs encode missing
|
|
105
|
+
values as the string `NA` or a quoted empty string; both are treated as missing, as are text entries with
|
|
106
|
+
no letters or digits.
|
|
107
|
+
|
|
108
|
+
## Data notes
|
|
109
|
+
|
|
110
|
+
- **Coarse nursing chart timestamps.** `dtNursingChart` has about 1 million rows with a usable
|
|
111
|
+
`ChartTime` but only about 23,000 distinct (patient, `ChartTime`) pairs. A typical timestamp covers 26
|
|
112
|
+
source rows, and the largest covers 266. This comes from the source release, not the ETL. In the MEDS
|
|
113
|
+
output, one timestamp therefore holds many readings of the same measurement, on average about 15
|
|
114
|
+
`NURSING//HEART_RATE` values, and similarly for blood pressure, SpO2 and other vitals. The order of events
|
|
115
|
+
within a timestamp carries no clinical meaning. When defining tasks or features:
|
|
116
|
+
- Do not rely on order within a timestamp, such as "last value before X", minute-level features, or
|
|
117
|
+
sequence models that treat event order as real time order.
|
|
118
|
+
- Choose prediction times that do not fall inside a timestamp's group of events, and aggregate values
|
|
119
|
+
within a timestamp (for example mean, min, or max) rather than picking one.
|
|
120
|
+
- **Dropped nursing rows.** 313,974 nursing rows (23%) belong to 723 stays that are not in the cohort
|
|
121
|
+
(`dtBaseline`), and all of them have `ChartTime = NA`; they are dropped. Rows charted more than 24 h before
|
|
122
|
+
admission (9,011 rows, including some corrupt offsets) or after discharge (141) are dropped too.
|
|
123
|
+
- **Dropped labs.** Results timed after hospital discharge (4,415 rows, 0.3%) and placeholder values made
|
|
124
|
+
only of dots or asterisks are dropped.
|
|
125
|
+
- **Coarse death dates.** `Death_Date` has about day precision. In-hospital deaths typically fall about 12 h
|
|
126
|
+
before the recorded discharge, so a stay's last events can follow `MEDS_DEATH`. See
|
|
127
|
+
[#4](https://github.com/rvandewater/ZIGONG_MEDS/issues/4).
|
|
128
|
+
- **Line and tube columns** (`CVC`, `PICC`, `Gastric_tube`, …) hold the length in cm, used as the
|
|
129
|
+
`numeric_value`, or a status such as "removed" kept as text. The numeric values of `Endotracheal_intubation`
|
|
130
|
+
duplicate `Endotracheal_intubation_Depth`, so only its statuses are emitted.
|
|
131
|
+
- **Bounded scores.** RASS, NRS and CPOT values outside their scales (-5..+4, 0..10, 0..8) are dropped.
|
|
132
|
+
Other vitals are not range-filtered; see [#7](https://github.com/rvandewater/ZIGONG_MEDS/issues/7).
|
|
133
|
+
- **Unmapped columns.** Five nursing columns are NA in every row. `MAP`, `PSIPAP`, `R` and `FIO2` hold only
|
|
134
|
+
unit strings. `Invasive_SBP._1` is a documented technical duplicate, and `F` is documented as having no
|
|
135
|
+
clinical relevance. See [#6](https://github.com/rvandewater/ZIGONG_MEDS/issues/6).
|
|
136
|
+
- **Local vocabularies.** Diagnoses use the release's Chinese national extension of ICD-10, and labs and
|
|
137
|
+
drugs use local names. No standard-terminology parents are attached yet; see
|
|
138
|
+
[#1](https://github.com/rvandewater/ZIGONG_MEDS/issues/1) and
|
|
139
|
+
[#8](https://github.com/rvandewater/ZIGONG_MEDS/issues/8). SF-36 answers are inconsistently coded in the
|
|
140
|
+
source; see [#5](https://github.com/rvandewater/ZIGONG_MEDS/issues/5).
|
|
141
|
+
- **No ICU admission event.** The release records ICU discharge but not ICU entry; see
|
|
142
|
+
[#2](https://github.com/rvandewater/ZIGONG_MEDS/issues/2).
|
|
143
|
+
|
|
144
|
+
## Testing
|
|
145
|
+
|
|
146
|
+
Zigong has no open demo release, so there are two synthetic checks:
|
|
147
|
+
|
|
148
|
+
- `tests/e2e_synthetic_test.py` writes a few fully synthetic rows with the release's exact headers and R
|
|
149
|
+
`write.csv` encoding, runs `meds-extract-run spec=Zigong` over them, and checks the events, times, values
|
|
150
|
+
and missing-value handling it produces.
|
|
151
|
+
- The [Synthetic ETL workflow](.github/workflows/synthetic-etl.yaml) generates raw files straight from the
|
|
152
|
+
config with [MESSY-synth](https://pypi.org/project/MESSY-synth/) and runs the ETL over them. It fails if
|
|
153
|
+
any event block produces no events.
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
uv sync --group dev
|
|
157
|
+
uv run pytest tests/ -v
|
|
158
|
+
uv run pre-commit run --all-files
|
|
159
|
+
|
|
160
|
+
# MESSY-synth needs Python >= 3.12.
|
|
161
|
+
uv run --with MESSY-synth==0.0.1 MESSY-synth ./src/Zigong_MEDS/event_configs.yaml -o raw_synthetic/
|
|
162
|
+
uv run meds-extract-run spec=Zigong output_dir=zigong_meds_synth/ \
|
|
163
|
+
do_download=false input_dir=raw_synthetic/
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
For a reusable checklist on mapping a new dataset, see [docs/MEDS_ETL_MAPPING.md](docs/MEDS_ETL_MAPPING.md).
|
|
167
|
+
|
|
168
|
+
## Citation
|
|
169
|
+
|
|
170
|
+
If you use this ETL, please cite the Zigong database and its descriptor paper:
|
|
171
|
+
|
|
172
|
+
- Xu, P., Chen, L., & Zhang, Z. (2022). *Critical care database comprising patients with infection at
|
|
173
|
+
Zigong Fourth People's Hospital* (version 1.1). PhysioNet. https://doi.org/10.13026/xpt9-z726
|
|
174
|
+
- Xu, P., Chen, L., Zhu, Y. et al. Critical Care Database Comprising Patients With Infection. *Front.
|
|
175
|
+
Public Health* 10 (2022). https://doi.org/10.3389/fpubh.2022.852410
|
|
176
|
+
|
|
177
|
+
See also [`CITATION.cff`](CITATION.cff).
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
src/Zigong_MEDS/__init__.py
|
|
5
|
+
src/Zigong_MEDS/event_configs.yaml
|
|
6
|
+
src/Zigong_MEDS.egg-info/PKG-INFO
|
|
7
|
+
src/Zigong_MEDS.egg-info/SOURCES.txt
|
|
8
|
+
src/Zigong_MEDS.egg-info/dependency_links.txt
|
|
9
|
+
src/Zigong_MEDS.egg-info/entry_points.txt
|
|
10
|
+
src/Zigong_MEDS.egg-info/requires.txt
|
|
11
|
+
src/Zigong_MEDS.egg-info/top_level.txt
|
|
12
|
+
tests/test_config.py
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
meds-extract[download]<0.8,>=0.7.0
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
Zigong_MEDS
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
from MEDS_extract.config import MessyConfig
|
|
2
|
+
|
|
3
|
+
|
|
4
|
+
def test_zigong_config_loads_and_covers_release_tables():
|
|
5
|
+
config = MessyConfig.load("Zigong")
|
|
6
|
+
|
|
7
|
+
assert config.dataset_name == "Zigong"
|
|
8
|
+
assert set(config.table_prefixes) == {
|
|
9
|
+
"dtBaseline",
|
|
10
|
+
"dtDrugs",
|
|
11
|
+
"dtICD",
|
|
12
|
+
"dtLab",
|
|
13
|
+
"dtNursingChart",
|
|
14
|
+
"dtOutCome",
|
|
15
|
+
"dtTransfer",
|
|
16
|
+
}
|
|
17
|
+
assert config.raw_version_for() == "1.1"
|