endoreg-db 0.4.5__py3-none-any.whl β 0.8.6.3__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- endoreg_db/admin.py +90 -1
- endoreg_db/api_urls.py +4 -0
- endoreg_db/apps.py +12 -0
- endoreg_db/assets/dummy_model.ckpt +1 -0
- endoreg_db/codemods/readme.md +88 -0
- endoreg_db/codemods/rename_datetime_fields.py +92 -0
- endoreg_db/config/env.py +101 -0
- endoreg_db/data/__init__.py +76 -4
- endoreg_db/data/ai_model/data.yaml +7 -0
- endoreg_db/data/{label β ai_model_label}/label/data.yaml +27 -1
- endoreg_db/data/ai_model_label/label/polyp_classification.yaml +52 -0
- endoreg_db/data/ai_model_label/label-set/data.yaml +40 -0
- endoreg_db/data/ai_model_label/label-set/polyp_classifications.yaml +25 -0
- endoreg_db/data/ai_model_meta/default_multilabel_classification.yaml +27 -0
- endoreg_db/data/ai_model_video_segmentation_label/base_segmentation.yaml +176 -0
- endoreg_db/data/ai_model_video_segmentation_labelset/data.yaml +20 -0
- endoreg_db/data/center/data.yaml +40 -9
- endoreg_db/data/center_shift/ukw.yaml +9 -0
- endoreg_db/data/contraindication/bleeding.yaml +11 -0
- endoreg_db/data/db_summary.csv +58 -0
- endoreg_db/data/db_summary.xlsx +0 -0
- endoreg_db/data/disease/misc.yaml +1 -2
- endoreg_db/data/disease_classification/chronic_kidney_disease.yaml +2 -2
- endoreg_db/data/disease_classification_choice/chronic_kidney_disease.yaml +6 -6
- endoreg_db/data/distribution/numeric/data.yaml +14 -0
- endoreg_db/data/endoscope/data.yaml +93 -0
- endoreg_db/data/endoscopy_processor/data.yaml +3 -0
- endoreg_db/data/event/cardiology.yaml +0 -13
- endoreg_db/data/examination/examinations/data.yaml +34 -28
- endoreg_db/data/examination/type/data.yaml +12 -0
- endoreg_db/data/examination_indication/endoscopy.yaml +424 -0
- endoreg_db/data/examination_indication_classification/endoscopy.yaml +160 -0
- endoreg_db/data/examination_indication_classification_choice/endoscopy.yaml +101 -0
- endoreg_db/data/examination_requirement_set/colonoscopy.yaml +15 -0
- endoreg_db/data/finding/anatomy_colon.yaml +128 -0
- endoreg_db/data/finding/colonoscopy.yaml +40 -0
- endoreg_db/data/finding/colonoscopy_bowel_prep.yaml +56 -0
- endoreg_db/data/finding/complication.yaml +16 -0
- endoreg_db/data/finding/data.yaml +105 -0
- endoreg_db/data/finding/examination_setting.yaml +16 -0
- endoreg_db/data/finding/medication_related.yaml +18 -0
- endoreg_db/data/finding/outcome.yaml +12 -0
- endoreg_db/data/finding_classification/colonoscopy_bowel_preparation.yaml +95 -0
- endoreg_db/data/finding_classification/colonoscopy_jnet.yaml +22 -0
- endoreg_db/data/finding_classification/colonoscopy_kudo.yaml +25 -0
- endoreg_db/data/finding_classification/colonoscopy_lesion_circularity.yaml +20 -0
- endoreg_db/data/finding_classification/colonoscopy_lesion_planarity.yaml +24 -0
- endoreg_db/data/finding_classification/colonoscopy_lesion_size.yaml +68 -0
- endoreg_db/data/finding_classification/colonoscopy_lesion_surface.yaml +20 -0
- endoreg_db/data/finding_classification/colonoscopy_location.yaml +80 -0
- endoreg_db/data/finding_classification/colonoscopy_lst.yaml +21 -0
- endoreg_db/data/finding_classification/colonoscopy_nice.yaml +20 -0
- endoreg_db/data/finding_classification/colonoscopy_paris.yaml +26 -0
- endoreg_db/data/finding_classification/colonoscopy_sano.yaml +22 -0
- endoreg_db/data/finding_classification/colonoscopy_summary.yaml +53 -0
- endoreg_db/data/finding_classification/complication_generic.yaml +25 -0
- endoreg_db/data/finding_classification/examination_setting_generic.yaml +40 -0
- endoreg_db/data/finding_classification/histology_colo.yaml +51 -0
- endoreg_db/data/finding_classification/intervention_required.yaml +26 -0
- endoreg_db/data/finding_classification/medication_related.yaml +23 -0
- endoreg_db/data/finding_classification/visualized.yaml +33 -0
- endoreg_db/data/finding_classification_choice/bowel_preparation.yaml +78 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_circularity_default.yaml +32 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_jnet.yaml +15 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_kudo.yaml +23 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_lst.yaml +15 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_nice.yaml +17 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_paris.yaml +57 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_planarity_default.yaml +49 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_sano.yaml +14 -0
- endoreg_db/data/finding_classification_choice/colon_lesion_surface_intact_default.yaml +36 -0
- endoreg_db/data/finding_classification_choice/colonoscopy_location.yaml +229 -0
- endoreg_db/data/finding_classification_choice/colonoscopy_not_complete_reason.yaml +19 -0
- endoreg_db/data/finding_classification_choice/colonoscopy_size.yaml +82 -0
- endoreg_db/data/finding_classification_choice/colonoscopy_summary_worst_finding.yaml +15 -0
- endoreg_db/data/finding_classification_choice/complication_generic_types.yaml +15 -0
- endoreg_db/data/finding_classification_choice/examination_setting_generic_types.yaml +15 -0
- endoreg_db/data/finding_classification_choice/histology.yaml +24 -0
- endoreg_db/data/finding_classification_choice/histology_polyp.yaml +20 -0
- endoreg_db/data/finding_classification_choice/outcome.yaml +19 -0
- endoreg_db/data/finding_classification_choice/yes_no_na.yaml +11 -0
- endoreg_db/data/finding_classification_type/colonoscopy_basic.yaml +48 -0
- endoreg_db/data/finding_intervention/endoscopy.yaml +43 -0
- endoreg_db/data/finding_intervention/endoscopy_colonoscopy.yaml +168 -0
- endoreg_db/data/finding_intervention/endoscopy_egd.yaml +128 -0
- endoreg_db/data/finding_intervention/endoscopy_ercp.yaml +32 -0
- endoreg_db/data/finding_intervention/endoscopy_eus_lower.yaml +9 -0
- endoreg_db/data/finding_intervention/endoscopy_eus_upper.yaml +36 -0
- endoreg_db/data/finding_intervention_type/endoscopy.yaml +15 -0
- endoreg_db/data/finding_morphology_classification_type/colonoscopy.yaml +79 -0
- endoreg_db/data/finding_type/data.yaml +43 -0
- endoreg_db/data/gender/data.yaml +24 -0
- endoreg_db/data/information_source/annotation.yaml +6 -0
- endoreg_db/data/information_source/endoscopy_guidelines.yaml +7 -0
- endoreg_db/data/information_source/prediction.yaml +7 -0
- endoreg_db/data/information_source_type/data.yaml +8 -0
- endoreg_db/data/lab_value/cardiac_enzymes.yaml +7 -1
- endoreg_db/data/lab_value/coagulation.yaml +6 -1
- endoreg_db/data/lab_value/electrolytes.yaml +39 -1
- endoreg_db/data/lab_value/gastrointestinal_function.yaml +12 -0
- endoreg_db/data/lab_value/hematology.yaml +17 -2
- endoreg_db/data/lab_value/hormones.yaml +6 -0
- endoreg_db/data/lab_value/lipids.yaml +12 -3
- endoreg_db/data/lab_value/misc.yaml +48 -2
- endoreg_db/data/lab_value/renal_function.yaml +2 -1
- endoreg_db/data/lx_client_tag/base.yaml +54 -0
- endoreg_db/data/lx_client_type/base.yaml +30 -0
- endoreg_db/data/lx_permission/base.yaml +24 -0
- endoreg_db/data/lx_permission/endoreg.yaml +52 -0
- endoreg_db/data/medication/anticoagulation.yaml +5 -5
- endoreg_db/data/medication/tah.yaml +5 -5
- endoreg_db/data/medication_indication/anticoagulation.yaml +48 -53
- endoreg_db/data/medication_intake_time/base.yaml +4 -4
- endoreg_db/data/names_first/first_names.yaml +54 -0
- endoreg_db/data/names_last/last_names.yaml +51 -0
- endoreg_db/data/network_device/data.yaml +30 -0
- endoreg_db/data/organ/data.yaml +29 -0
- endoreg_db/data/pdf_type/data.yaml +27 -9
- endoreg_db/data/qualification/endoscopy.yaml +36 -0
- endoreg_db/data/qualification/m2.yaml +39 -0
- endoreg_db/data/qualification/outpatient_clinic.yaml +35 -0
- endoreg_db/data/qualification/sonography.yaml +36 -0
- endoreg_db/data/qualification_type/base.yaml +29 -0
- endoreg_db/data/report_reader_flag/rkh-histology-generic.yaml +10 -0
- endoreg_db/data/report_reader_flag/ukw-examination-generic.yaml +4 -0
- endoreg_db/data/report_reader_flag/ukw-histology-generic.yaml +5 -0
- endoreg_db/data/requirement/age.yaml +26 -0
- endoreg_db/data/requirement/colonoscopy_baseline_austria.yaml +45 -0
- endoreg_db/data/requirement/disease_cardiovascular.yaml +79 -0
- endoreg_db/data/requirement/disease_classification_choice_cardiovascular.yaml +41 -0
- endoreg_db/data/requirement/disease_hepatology.yaml +12 -0
- endoreg_db/data/requirement/disease_misc.yaml +12 -0
- endoreg_db/data/requirement/disease_renal.yaml +96 -0
- endoreg_db/data/requirement/endoscopy_bleeding_risk.yaml +59 -0
- endoreg_db/data/requirement/event_cardiology.yaml +251 -0
- endoreg_db/data/requirement/event_requirements.yaml +145 -0
- endoreg_db/data/requirement/finding_colon_polyp.yaml +50 -0
- endoreg_db/data/requirement/gender.yaml +25 -0
- endoreg_db/data/requirement/lab_value.yaml +441 -0
- endoreg_db/data/requirement/medication.yaml +93 -0
- endoreg_db/data/requirement_operator/age.yaml +13 -0
- endoreg_db/data/requirement_operator/lab_operators.yaml +129 -0
- endoreg_db/data/requirement_operator/model_operators.yaml +96 -0
- endoreg_db/data/requirement_set/01_endoscopy_generic.yaml +48 -0
- endoreg_db/data/requirement_set/colonoscopy_austria_screening.yaml +57 -0
- endoreg_db/data/requirement_set/endoscopy_bleeding_risk.yaml +52 -0
- endoreg_db/data/requirement_set_type/data.yaml +20 -0
- endoreg_db/data/requirement_type/requirement_types.yaml +165 -0
- endoreg_db/data/risk/bleeding.yaml +26 -0
- endoreg_db/data/risk/thrombosis.yaml +37 -0
- endoreg_db/data/risk_type/data.yaml +27 -0
- endoreg_db/data/setup_config.yaml +38 -0
- endoreg_db/data/shift/endoscopy.yaml +21 -0
- endoreg_db/data/shift_type/base.yaml +35 -0
- endoreg_db/data/tag/requirement_set_tags.yaml +11 -0
- endoreg_db/data/unit/concentration.yaml +23 -0
- endoreg_db/data/unit/time.yaml +36 -1
- endoreg_db/exceptions.py +19 -0
- endoreg_db/forms/__init__.py +3 -1
- endoreg_db/forms/examination_form.py +11 -0
- endoreg_db/forms/patient_finding_intervention_form.py +18 -0
- endoreg_db/forms/patient_form.py +27 -0
- endoreg_db/forms/questionnaires/__init__.py +1 -1
- endoreg_db/forms/questionnaires/tto_questionnaire.py +19 -19
- endoreg_db/helpers/count_db.py +45 -0
- endoreg_db/helpers/data_loader.py +208 -0
- endoreg_db/helpers/default_objects.py +378 -0
- endoreg_db/helpers/download_segmentation_model.py +31 -0
- endoreg_db/helpers/interact.py +6 -0
- endoreg_db/helpers/test_video_helper.py +119 -0
- endoreg_db/logger_conf.py +140 -0
- endoreg_db/management/__init__.py +1 -0
- endoreg_db/management/commands/__init__.py +1 -0
- endoreg_db/management/commands/anonymize_video.py +0 -0
- endoreg_db/management/commands/check_auth.py +125 -0
- endoreg_db/management/commands/create_model_meta_from_huggingface.py +115 -0
- endoreg_db/management/commands/create_multilabel_model_meta.py +214 -0
- endoreg_db/management/commands/fix_missing_patient_data.py +172 -0
- endoreg_db/management/commands/fix_video_paths.py +165 -0
- endoreg_db/management/commands/import_fallback_video.py +203 -0
- endoreg_db/management/commands/import_report.py +298 -0
- endoreg_db/management/commands/import_video.py +423 -0
- endoreg_db/management/commands/import_video_with_classification.py +367 -0
- endoreg_db/management/commands/init_default_ai_model.py +112 -0
- endoreg_db/management/commands/load_ai_model_data.py +58 -26
- endoreg_db/management/commands/load_ai_model_label_data.py +59 -0
- endoreg_db/management/commands/load_base_db_data.py +174 -118
- endoreg_db/management/commands/load_center_data.py +46 -21
- endoreg_db/management/commands/{load_logging_data.py β load_contraindication_data.py} +4 -2
- endoreg_db/management/commands/load_disease_data.py +29 -7
- endoreg_db/management/commands/{load_endoscope_type_data.py β load_endoscope_data.py} +30 -7
- endoreg_db/management/commands/load_examination_indication_data.py +86 -0
- endoreg_db/management/commands/load_finding_data.py +128 -0
- endoreg_db/management/commands/load_green_endoscopy_wuerzburg_data.py +0 -1
- endoreg_db/management/commands/load_information_source.py +13 -7
- endoreg_db/management/commands/load_lab_value_data.py +3 -3
- endoreg_db/management/commands/load_medication_data.py +83 -21
- endoreg_db/management/commands/load_name_data.py +37 -0
- endoreg_db/management/commands/{load_medication_intake_time_data.py β load_organ_data.py} +7 -5
- endoreg_db/management/commands/load_qualification_data.py +59 -0
- endoreg_db/management/commands/load_requirement_data.py +180 -0
- endoreg_db/management/commands/load_risk_data.py +56 -0
- endoreg_db/management/commands/load_shift_data.py +60 -0
- endoreg_db/management/commands/load_tag_data.py +57 -0
- endoreg_db/management/commands/register_ai_model.py +1 -1
- endoreg_db/management/commands/setup_endoreg_db.py +381 -0
- endoreg_db/management/commands/start_filewatcher.py +106 -0
- endoreg_db/management/commands/storage_management.py +548 -0
- endoreg_db/management/commands/summarize_db_content.py +189 -0
- endoreg_db/management/commands/validate_video.py +204 -0
- endoreg_db/management/commands/validate_video_files.py +161 -0
- endoreg_db/management/commands/video_validation.py +22 -0
- endoreg_db/mermaid/Overall_flow_patient_finding_intervention.md +10 -0
- endoreg_db/mermaid/anonymized_image_annotation.md +20 -0
- endoreg_db/mermaid/binary_classification_annotation.md +50 -0
- endoreg_db/mermaid/classification.md +8 -0
- endoreg_db/mermaid/examination.md +8 -0
- endoreg_db/mermaid/findings.md +7 -0
- endoreg_db/mermaid/image_classification.md +28 -0
- endoreg_db/mermaid/interventions.md +8 -0
- endoreg_db/mermaid/morphology.md +8 -0
- endoreg_db/mermaid/patient_creation.md +14 -0
- endoreg_db/mermaid/video_segmentation_annotation.md +17 -0
- endoreg_db/migrations/0001_initial.py +1234 -944
- endoreg_db/migrations/0002_add_video_correction_models.py +52 -0
- endoreg_db/migrations/0003_add_center_display_name.py +30 -0
- endoreg_db/models/__init__.py +339 -53
- endoreg_db/models/administration/__init__.py +116 -0
- endoreg_db/models/administration/ai/__init__.py +9 -0
- endoreg_db/models/administration/ai/active_model.py +35 -0
- endoreg_db/models/administration/ai/ai_model.py +156 -0
- endoreg_db/models/{ai_model β administration/ai}/model_type.py +19 -4
- endoreg_db/models/administration/case/__init__.py +19 -0
- endoreg_db/models/administration/case/case.py +114 -0
- endoreg_db/models/{case_template β administration/case/case_template}/__init__.py +10 -1
- endoreg_db/models/{case_template β administration/case/case_template}/case_template.py +60 -16
- endoreg_db/models/{case_template β administration/case/case_template}/case_template_rule.py +6 -13
- endoreg_db/models/{case_template β administration/case/case_template}/case_template_rule_value.py +21 -8
- endoreg_db/models/{case_template β administration/case/case_template}/case_template_type.py +1 -3
- endoreg_db/models/{center β administration/center}/__init__.py +9 -0
- endoreg_db/models/administration/center/center.py +67 -0
- endoreg_db/models/administration/center/center_product.py +64 -0
- endoreg_db/models/administration/center/center_resource.py +49 -0
- endoreg_db/models/administration/center/center_shift.py +88 -0
- endoreg_db/models/administration/center/center_waste.py +30 -0
- endoreg_db/models/administration/permissions/__init__.py +44 -0
- endoreg_db/models/administration/person/__init__.py +24 -0
- endoreg_db/models/administration/person/employee/__init__.py +3 -0
- endoreg_db/models/administration/person/employee/employee.py +35 -0
- endoreg_db/models/administration/person/employee/employee_qualification.py +39 -0
- endoreg_db/models/administration/person/employee/employee_type.py +42 -0
- endoreg_db/models/administration/person/examiner/__init__.py +4 -0
- endoreg_db/models/administration/person/examiner/examiner.py +54 -0
- endoreg_db/models/administration/person/names/__init__.py +0 -0
- endoreg_db/models/{persons β administration/person/names}/first_name.py +1 -1
- endoreg_db/models/{persons β administration/person/names}/last_name.py +2 -3
- endoreg_db/models/administration/person/patient/__init__.py +5 -0
- endoreg_db/models/administration/person/patient/patient.py +460 -0
- endoreg_db/models/administration/person/profession/__init__.py +24 -0
- endoreg_db/models/administration/person/user/__init__.py +5 -0
- endoreg_db/models/administration/person/user/portal_user_information.py +37 -0
- endoreg_db/models/administration/product/__init__.py +14 -0
- endoreg_db/models/administration/product/product.py +97 -0
- endoreg_db/models/administration/product/product_group.py +39 -0
- endoreg_db/models/administration/product/product_material.py +54 -0
- endoreg_db/models/{product β administration/product}/product_weight.py +21 -0
- endoreg_db/models/{product β administration/product}/reference_product.py +44 -13
- endoreg_db/models/administration/qualification/__init__.py +7 -0
- endoreg_db/models/administration/qualification/qualification.py +37 -0
- endoreg_db/models/administration/qualification/qualification_type.py +35 -0
- endoreg_db/models/administration/shift/__init__.py +9 -0
- endoreg_db/models/administration/shift/scheduled_days.py +69 -0
- endoreg_db/models/administration/shift/shift.py +51 -0
- endoreg_db/models/administration/shift/shift_type.py +108 -0
- endoreg_db/models/label/__init__.py +24 -1
- endoreg_db/models/label/annotation/__init__.py +12 -0
- endoreg_db/models/label/annotation/image_classification.py +84 -0
- endoreg_db/models/label/annotation/video_segmentation_annotation.py +66 -0
- endoreg_db/models/label/label.py +53 -54
- endoreg_db/models/label/label_set.py +53 -0
- endoreg_db/models/label/label_type.py +29 -0
- endoreg_db/models/label/label_video_segment/__init__.py +3 -0
- endoreg_db/models/label/label_video_segment/_create_from_video.py +41 -0
- endoreg_db/models/label/label_video_segment/label_video_segment.py +511 -0
- endoreg_db/models/label/video_segmentation_label.py +31 -0
- endoreg_db/models/label/video_segmentation_labelset.py +27 -0
- endoreg_db/models/media/__init__.py +16 -0
- endoreg_db/models/media/frame/__init__.py +3 -0
- endoreg_db/models/media/frame/frame.py +111 -0
- endoreg_db/models/media/pdf/__init__.py +11 -0
- endoreg_db/models/media/pdf/raw_pdf.py +757 -0
- endoreg_db/models/media/pdf/report_file.py +162 -0
- endoreg_db/models/media/pdf/report_reader/__init__.py +7 -0
- endoreg_db/models/media/pdf/report_reader/report_reader_config.py +77 -0
- endoreg_db/models/media/video/__init__.py +8 -0
- endoreg_db/models/media/video/create_from_file.py +358 -0
- endoreg_db/models/media/video/pipe_1.py +213 -0
- endoreg_db/models/media/video/pipe_2.py +105 -0
- endoreg_db/models/media/video/refactor_plan.md +0 -0
- endoreg_db/models/media/video/video_file.py +825 -0
- endoreg_db/models/media/video/video_file_ai.py +443 -0
- endoreg_db/models/media/video/video_file_anonymize.py +349 -0
- endoreg_db/models/media/video/video_file_frames/__init__.py +47 -0
- endoreg_db/models/media/video/video_file_frames/_bulk_create_frames.py +22 -0
- endoreg_db/models/media/video/video_file_frames/_create_frame_object.py +23 -0
- endoreg_db/models/media/video/video_file_frames/_delete_frames.py +104 -0
- endoreg_db/models/media/video/video_file_frames/_extract_frames.py +174 -0
- endoreg_db/models/media/video/video_file_frames/_get_frame.py +28 -0
- endoreg_db/models/media/video/video_file_frames/_get_frame_number.py +27 -0
- endoreg_db/models/media/video/video_file_frames/_get_frame_path.py +20 -0
- endoreg_db/models/media/video/video_file_frames/_get_frame_paths.py +27 -0
- endoreg_db/models/media/video/video_file_frames/_get_frame_range.py +34 -0
- endoreg_db/models/media/video/video_file_frames/_get_frames.py +27 -0
- endoreg_db/models/media/video/video_file_frames/_initialize_frames.py +129 -0
- endoreg_db/models/media/video/video_file_frames/_manage_frame_range.py +141 -0
- endoreg_db/models/media/video/video_file_frames/_mark_frames_extracted_status.py +65 -0
- endoreg_db/models/media/video/video_file_frames.py +0 -0
- endoreg_db/models/media/video/video_file_io.py +168 -0
- endoreg_db/models/media/video/video_file_meta/__init__.py +22 -0
- endoreg_db/models/media/video/video_file_meta/get_crop_template.py +45 -0
- endoreg_db/models/media/video/video_file_meta/get_endo_roi.py +39 -0
- endoreg_db/models/media/video/video_file_meta/get_fps.py +147 -0
- endoreg_db/models/media/video/video_file_meta/initialize_video_specs.py +143 -0
- endoreg_db/models/media/video/video_file_meta/text_meta.py +134 -0
- endoreg_db/models/media/video/video_file_meta/video_meta.py +70 -0
- endoreg_db/models/media/video/video_file_segments.py +209 -0
- endoreg_db/models/media/video/video_metadata.py +65 -0
- endoreg_db/models/media/video/video_processing.py +152 -0
- endoreg_db/models/medical/__init__.py +146 -0
- endoreg_db/models/medical/contraindication/__init__.py +17 -0
- endoreg_db/models/medical/disease.py +156 -0
- endoreg_db/models/medical/event.py +137 -0
- endoreg_db/models/medical/examination/__init__.py +9 -0
- endoreg_db/models/medical/examination/examination.py +148 -0
- endoreg_db/models/medical/examination/examination_indication.py +278 -0
- endoreg_db/models/medical/examination/examination_time.py +49 -0
- endoreg_db/models/medical/examination/examination_time_type.py +41 -0
- endoreg_db/models/medical/examination/examination_type.py +48 -0
- endoreg_db/models/medical/finding/__init__.py +18 -0
- endoreg_db/models/medical/finding/finding.py +96 -0
- endoreg_db/models/medical/finding/finding_classification.py +142 -0
- endoreg_db/models/medical/finding/finding_intervention.py +52 -0
- endoreg_db/models/medical/finding/finding_type.py +35 -0
- endoreg_db/models/medical/hardware/__init__.py +8 -0
- endoreg_db/models/medical/hardware/endoscope.py +65 -0
- endoreg_db/models/{hardware β medical/hardware}/endoscopy_processor.py +68 -29
- endoreg_db/models/medical/laboratory/__init__.py +5 -0
- endoreg_db/models/medical/laboratory/lab_value.py +419 -0
- endoreg_db/models/medical/medication/__init__.py +19 -0
- endoreg_db/models/medical/medication/medication.py +31 -0
- endoreg_db/models/medical/medication/medication_indication.py +50 -0
- endoreg_db/models/medical/medication/medication_indication_type.py +39 -0
- endoreg_db/models/medical/medication/medication_intake_time.py +44 -0
- endoreg_db/models/medical/medication/medication_schedule.py +45 -0
- endoreg_db/models/medical/organ/__init__.py +35 -0
- endoreg_db/models/medical/patient/__init__.py +56 -0
- endoreg_db/models/medical/patient/medication_examples.py +38 -0
- endoreg_db/models/medical/patient/patient_disease.py +63 -0
- endoreg_db/models/medical/patient/patient_event.py +75 -0
- endoreg_db/models/medical/patient/patient_examination.py +249 -0
- endoreg_db/models/medical/patient/patient_examination_indication.py +44 -0
- endoreg_db/models/medical/patient/patient_finding.py +357 -0
- endoreg_db/models/medical/patient/patient_finding_classification.py +207 -0
- endoreg_db/models/medical/patient/patient_finding_intervention.py +40 -0
- endoreg_db/models/medical/patient/patient_lab_sample.py +148 -0
- endoreg_db/models/{persons β medical}/patient/patient_lab_value.py +68 -22
- endoreg_db/models/medical/patient/patient_medication.py +104 -0
- endoreg_db/models/medical/patient/patient_medication_schedule.py +136 -0
- endoreg_db/models/medical/risk/__init__.py +7 -0
- endoreg_db/models/medical/risk/risk.py +72 -0
- endoreg_db/models/medical/risk/risk_type.py +51 -0
- endoreg_db/models/metadata/__init__.py +19 -0
- endoreg_db/models/metadata/frame_ocr_result.py +0 -0
- endoreg_db/models/metadata/model_meta.py +206 -0
- endoreg_db/models/metadata/model_meta_logic.py +343 -0
- endoreg_db/models/{data_file/metadata β metadata}/pdf_meta.py +32 -13
- endoreg_db/models/metadata/sensitive_meta.py +288 -0
- endoreg_db/models/metadata/sensitive_meta_logic.py +1048 -0
- endoreg_db/models/metadata/video_meta.py +332 -0
- endoreg_db/models/metadata/video_prediction_logic.py +190 -0
- endoreg_db/models/metadata/video_prediction_meta.py +270 -0
- endoreg_db/models/other/__init__.py +36 -1
- endoreg_db/models/other/distribution/__init__.py +44 -0
- endoreg_db/models/other/distribution/base_value_distribution.py +20 -0
- endoreg_db/models/other/distribution/date_value_distribution.py +89 -0
- endoreg_db/models/other/distribution/multiple_categorical_value_distribution.py +32 -0
- endoreg_db/models/other/distribution/numeric_value_distribution.py +125 -0
- endoreg_db/models/other/distribution/single_categorical_value_distribution.py +22 -0
- endoreg_db/models/other/emission/__init__.py +5 -0
- endoreg_db/models/other/emission/emission_factor.py +94 -0
- endoreg_db/models/{persons β other}/gender.py +8 -3
- endoreg_db/models/other/information_source.py +159 -0
- endoreg_db/models/other/material.py +14 -2
- endoreg_db/models/other/resource.py +6 -2
- endoreg_db/models/other/tag.py +27 -0
- endoreg_db/models/other/transport_route.py +15 -3
- endoreg_db/models/{unit.py β other/unit.py} +16 -6
- endoreg_db/models/other/waste.py +10 -3
- endoreg_db/models/requirement/__init__.py +11 -0
- endoreg_db/models/requirement/requirement.py +767 -0
- endoreg_db/models/requirement/requirement_evaluation/__init__.py +6 -0
- endoreg_db/models/requirement/requirement_evaluation/get_values.py +40 -0
- endoreg_db/models/requirement/requirement_evaluation/operator_evaluation_models.py +9 -0
- endoreg_db/models/requirement/requirement_evaluation/requirement_type_parser.py +95 -0
- endoreg_db/models/requirement/requirement_operator.py +176 -0
- endoreg_db/models/requirement/requirement_set.py +287 -0
- endoreg_db/models/rule/__init__.py +13 -0
- endoreg_db/models/{rules β rule}/rule.py +6 -3
- endoreg_db/models/{rules β rule}/rule_attribute_dtype.py +0 -2
- endoreg_db/models/{rules β rule}/rule_type.py +0 -2
- endoreg_db/models/{rules β rule}/ruleset.py +0 -2
- endoreg_db/models/state/__init__.py +12 -0
- endoreg_db/models/state/abstract.py +11 -0
- endoreg_db/models/state/audit_ledger.py +150 -0
- endoreg_db/models/state/label_video_segment.py +22 -0
- endoreg_db/models/state/raw_pdf.py +187 -0
- endoreg_db/models/state/sensitive_meta.py +46 -0
- endoreg_db/models/state/video.py +232 -0
- endoreg_db/models/upload_job.py +99 -0
- endoreg_db/models/utils.py +135 -0
- endoreg_db/renames.yml +8 -0
- endoreg_db/root_urls.py +9 -0
- endoreg_db/schemas/__init__.py +0 -0
- endoreg_db/schemas/examination_evaluation.py +27 -0
- endoreg_db/serializers/Frames_NICE_and_PARIS_classifications.py +775 -0
- endoreg_db/serializers/__init__.py +118 -10
- endoreg_db/serializers/_old/raw_pdf_meta_validation.py +223 -0
- endoreg_db/serializers/_old/raw_video_meta_validation.py +179 -0
- endoreg_db/serializers/_old/video.py +71 -0
- endoreg_db/serializers/administration/__init__.py +14 -0
- endoreg_db/serializers/administration/ai/__init__.py +10 -0
- endoreg_db/serializers/administration/ai/active_model.py +10 -0
- endoreg_db/serializers/administration/ai/ai_model.py +18 -0
- endoreg_db/serializers/administration/ai/model_type.py +10 -0
- endoreg_db/serializers/administration/center.py +9 -0
- endoreg_db/serializers/administration/gender.py +9 -0
- endoreg_db/serializers/anonymization.py +69 -0
- endoreg_db/serializers/evaluation/examination_evaluation.py +1 -0
- endoreg_db/serializers/examination/__init__.py +10 -0
- endoreg_db/serializers/examination/base.py +46 -0
- endoreg_db/serializers/examination/dropdown.py +21 -0
- endoreg_db/serializers/examination_serializer.py +12 -0
- endoreg_db/serializers/finding/__init__.py +5 -0
- endoreg_db/serializers/finding/finding.py +54 -0
- endoreg_db/serializers/finding_classification/__init__.py +7 -0
- endoreg_db/serializers/finding_classification/choice.py +19 -0
- endoreg_db/serializers/finding_classification/classification.py +13 -0
- endoreg_db/serializers/label/__init__.py +7 -0
- endoreg_db/serializers/label/image_classification_annotation.py +62 -0
- endoreg_db/serializers/label/label.py +15 -0
- endoreg_db/serializers/label_video_segment/__init__.py +7 -0
- endoreg_db/serializers/label_video_segment/_lvs_create.py +149 -0
- endoreg_db/serializers/label_video_segment/_lvs_update.py +138 -0
- endoreg_db/serializers/label_video_segment/_lvs_validate.py +149 -0
- endoreg_db/serializers/label_video_segment/label_video_segment.py +344 -0
- endoreg_db/serializers/label_video_segment/label_video_segment_annotation.py +99 -0
- endoreg_db/serializers/label_video_segment/label_video_segment_update.py +163 -0
- endoreg_db/serializers/meta/__init__.py +19 -0
- endoreg_db/serializers/meta/pdf_file_meta_extraction.py +115 -0
- endoreg_db/serializers/meta/report_meta.py +53 -0
- endoreg_db/serializers/meta/sensitive_meta_detail.py +162 -0
- endoreg_db/serializers/meta/sensitive_meta_update.py +148 -0
- endoreg_db/serializers/meta/sensitive_meta_verification.py +59 -0
- endoreg_db/serializers/meta/video_meta.py +39 -0
- endoreg_db/serializers/misc/__init__.py +14 -0
- endoreg_db/serializers/misc/file_overview.py +182 -0
- endoreg_db/serializers/misc/sensitive_patient_data.py +120 -0
- endoreg_db/serializers/misc/stats.py +33 -0
- endoreg_db/serializers/misc/translatable_field_mix_in.py +44 -0
- endoreg_db/serializers/misc/upload_job.py +71 -0
- endoreg_db/serializers/patient/__init__.py +11 -0
- endoreg_db/serializers/patient/patient.py +86 -0
- endoreg_db/serializers/patient/patient_dropdown.py +27 -0
- endoreg_db/serializers/patient_examination/__init__.py +7 -0
- endoreg_db/serializers/patient_examination/patient_examination.py +141 -0
- endoreg_db/serializers/patient_finding/__init__.py +15 -0
- endoreg_db/serializers/patient_finding/patient_finding.py +31 -0
- endoreg_db/serializers/patient_finding/patient_finding_classification.py +39 -0
- endoreg_db/serializers/patient_finding/patient_finding_detail.py +53 -0
- endoreg_db/serializers/patient_finding/patient_finding_intervention.py +26 -0
- endoreg_db/serializers/patient_finding/patient_finding_list.py +41 -0
- endoreg_db/serializers/patient_finding/patient_finding_write.py +126 -0
- endoreg_db/serializers/pdf/__init__.py +5 -0
- endoreg_db/serializers/pdf/anony_text_validation.py +85 -0
- endoreg_db/serializers/report/__init__.py +9 -0
- endoreg_db/serializers/report/mixins.py +45 -0
- endoreg_db/serializers/report/report.py +105 -0
- endoreg_db/serializers/report/report_list.py +22 -0
- endoreg_db/serializers/report/secure_file_url.py +26 -0
- endoreg_db/serializers/requirements/requirement_schema.py +25 -0
- endoreg_db/serializers/requirements/requirement_sets.py +29 -0
- endoreg_db/serializers/sensitive_meta_serializer.py +282 -0
- endoreg_db/serializers/video/__init__.py +7 -0
- endoreg_db/serializers/video/segmentation.py +263 -0
- endoreg_db/serializers/video/video_file_brief.py +10 -0
- endoreg_db/serializers/video/video_file_detail.py +83 -0
- endoreg_db/serializers/video/video_file_list.py +67 -0
- endoreg_db/serializers/video/video_metadata.py +105 -0
- endoreg_db/serializers/video/video_processing_history.py +153 -0
- endoreg_db/serializers/video_examination.py +198 -0
- endoreg_db/services/__init__.py +5 -0
- endoreg_db/services/anonymization.py +223 -0
- endoreg_db/services/examination_evaluation.py +149 -0
- endoreg_db/services/finding_description_service.py +0 -0
- endoreg_db/services/lookup_service.py +411 -0
- endoreg_db/services/lookup_store.py +266 -0
- endoreg_db/services/pdf_import.py +1382 -0
- endoreg_db/services/polling_coordinator.py +288 -0
- endoreg_db/services/pseudonym_service.py +89 -0
- endoreg_db/services/requirements_object.py +147 -0
- endoreg_db/services/segment_sync.py +155 -0
- endoreg_db/services/storage_aware_video_processor.py +344 -0
- endoreg_db/services/video_import.py +1259 -0
- endoreg_db/tasks/upload_tasks.py +207 -0
- endoreg_db/tasks/video_ingest.py +157 -0
- endoreg_db/tasks/video_processing_tasks.py +327 -0
- endoreg_db/templates/admin/patient_finding_intervention.html +253 -0
- endoreg_db/templates/admin/start_examination.html +12 -0
- endoreg_db/templates/timeline.html +176 -0
- endoreg_db/urls/__init__.py +83 -0
- endoreg_db/urls/anonymization.py +32 -0
- endoreg_db/urls/auth.py +16 -0
- endoreg_db/urls/classification.py +39 -0
- endoreg_db/urls/examination.py +54 -0
- endoreg_db/urls/files.py +6 -0
- endoreg_db/urls/label_video_segment_validate.py +33 -0
- endoreg_db/urls/label_video_segments.py +46 -0
- endoreg_db/urls/media.py +227 -0
- endoreg_db/urls/patient.py +19 -0
- endoreg_db/urls/report.py +48 -0
- endoreg_db/urls/requirements.py +13 -0
- endoreg_db/urls/sensitive_meta.py +0 -0
- endoreg_db/urls/stats.py +46 -0
- endoreg_db/urls/upload.py +20 -0
- endoreg_db/urls/video.py +61 -0
- endoreg_db/urls.py +9 -0
- endoreg_db/utils/__init__.py +88 -1
- endoreg_db/utils/ai/__init__.py +9 -0
- endoreg_db/{models/ai_model/utils.py β utils/ai/get.py} +1 -4
- endoreg_db/utils/ai/inference_dataset.py +52 -0
- endoreg_db/utils/ai/multilabel_classification_net.py +159 -0
- endoreg_db/utils/ai/postprocess.py +63 -0
- endoreg_db/utils/ai/predict.py +291 -0
- endoreg_db/utils/ai/preprocess.py +68 -0
- endoreg_db/utils/calc_duration_seconds.py +24 -0
- endoreg_db/utils/case_generator/__init__.py +0 -0
- endoreg_db/utils/case_generator/case_generator.py +159 -0
- endoreg_db/utils/case_generator/lab_sample_factory.py +33 -0
- endoreg_db/utils/case_generator/utils.py +30 -0
- endoreg_db/utils/check_video_files.py +148 -0
- endoreg_db/utils/dataloader.py +118 -35
- endoreg_db/utils/dates.py +60 -0
- endoreg_db/utils/env.py +33 -0
- endoreg_db/utils/extract_specific_frames.py +72 -0
- endoreg_db/utils/file_operations.py +29 -1
- endoreg_db/utils/fix_video_path_direct.py +141 -0
- endoreg_db/utils/frame_anonymization_utils.py +463 -0
- endoreg_db/utils/hashs.py +123 -4
- endoreg_db/utils/links/__init__.py +0 -0
- endoreg_db/utils/links/requirement_link.py +193 -0
- endoreg_db/utils/mime_types.py +0 -0
- endoreg_db/utils/names.py +76 -0
- endoreg_db/utils/parse_and_generate_yaml.py +46 -0
- endoreg_db/utils/paths.py +95 -0
- endoreg_db/utils/permissions.py +143 -0
- endoreg_db/utils/pipelines/Readme.md +235 -0
- endoreg_db/utils/pipelines/__init__.py +0 -0
- endoreg_db/utils/pipelines/process_video_dir.py +120 -0
- endoreg_db/utils/product/__init__.py +0 -0
- endoreg_db/utils/product/sum_emissions.py +20 -0
- endoreg_db/utils/product/sum_weights.py +18 -0
- endoreg_db/utils/pydantic_models/__init__.py +6 -0
- endoreg_db/utils/pydantic_models/db_config.py +57 -0
- endoreg_db/utils/requirement_helpers.py +0 -0
- endoreg_db/utils/requirement_operator_logic/__init__.py +0 -0
- endoreg_db/utils/requirement_operator_logic/lab_value_operators.py +578 -0
- endoreg_db/utils/requirement_operator_logic/model_evaluators.py +368 -0
- endoreg_db/utils/setup_config.py +177 -0
- endoreg_db/utils/translation.py +27 -0
- endoreg_db/utils/validate_endo_roi.py +19 -0
- endoreg_db/utils/validate_subcategory_dict.py +91 -0
- endoreg_db/utils/validate_video_detailed.py +357 -0
- endoreg_db/utils/video/__init__.py +26 -0
- endoreg_db/utils/video/extract_frames.py +88 -0
- endoreg_db/utils/video/ffmpeg_wrapper.py +835 -0
- endoreg_db/utils/video/names.py +42 -0
- endoreg_db/utils/video/streaming_processor.py +312 -0
- endoreg_db/utils/video/video_splitter.py +94 -0
- endoreg_db/views/Frames_NICE_and_PARIS_classifications_views.py +238 -0
- endoreg_db/views/__init__.py +274 -0
- endoreg_db/views/anonymization/__init__.py +27 -0
- endoreg_db/views/anonymization/media_management.py +454 -0
- endoreg_db/views/anonymization/overview.py +216 -0
- endoreg_db/views/anonymization/validate.py +107 -0
- endoreg_db/views/auth/__init__.py +13 -0
- endoreg_db/views/auth/keycloak.py +113 -0
- endoreg_db/views/examination/__init__.py +33 -0
- endoreg_db/views/examination/examination.py +37 -0
- endoreg_db/views/examination/examination_manifest_cache.py +26 -0
- endoreg_db/views/examination/get_finding_classification_choices.py +59 -0
- endoreg_db/views/examination/get_finding_classifications.py +36 -0
- endoreg_db/views/examination/get_findings.py +41 -0
- endoreg_db/views/examination/get_instruments.py +18 -0
- endoreg_db/views/examination/get_interventions.py +14 -0
- endoreg_db/views/finding/__init__.py +9 -0
- endoreg_db/views/finding/finding.py +112 -0
- endoreg_db/views/finding/get_classifications.py +14 -0
- endoreg_db/views/finding/get_interventions.py +17 -0
- endoreg_db/views/finding_classification/__init__.py +13 -0
- endoreg_db/views/finding_classification/base.py +0 -0
- endoreg_db/views/finding_classification/finding_classification.py +42 -0
- endoreg_db/views/finding_classification/get_classification_choices.py +55 -0
- endoreg_db/views/label/__init__.py +5 -0
- endoreg_db/views/label/label.py +15 -0
- endoreg_db/views/label_video_segment/__init__.py +16 -0
- endoreg_db/views/label_video_segment/create_lvs_from_annotation.py +44 -0
- endoreg_db/views/label_video_segment/get_lvs_by_name_and_video.py +50 -0
- endoreg_db/views/label_video_segment/label_video_segment.py +77 -0
- endoreg_db/views/label_video_segment/label_video_segment_by_label.py +174 -0
- endoreg_db/views/label_video_segment/label_video_segment_detail.py +73 -0
- endoreg_db/views/label_video_segment/update_lvs_from_annotation.py +46 -0
- endoreg_db/views/label_video_segment/validate.py +226 -0
- endoreg_db/views/media/__init__.py +45 -0
- endoreg_db/views/media/pdf_media.py +388 -0
- endoreg_db/views/media/segments.py +71 -0
- endoreg_db/views/media/sensitive_metadata.py +314 -0
- endoreg_db/views/media/video_media.py +272 -0
- endoreg_db/views/media/video_segments.py +524 -0
- endoreg_db/views/meta/__init__.py +15 -0
- endoreg_db/views/meta/available_files_list.py +146 -0
- endoreg_db/views/meta/report_meta.py +53 -0
- endoreg_db/views/meta/sensitive_meta_detail.py +148 -0
- endoreg_db/views/meta/sensitive_meta_list.py +104 -0
- endoreg_db/views/meta/sensitive_meta_verification.py +71 -0
- endoreg_db/views/misc/__init__.py +63 -0
- endoreg_db/views/misc/center.py +13 -0
- endoreg_db/views/misc/csrf.py +7 -0
- endoreg_db/views/misc/gender.py +14 -0
- endoreg_db/views/misc/secure_file_serving_view.py +80 -0
- endoreg_db/views/misc/secure_file_url_view.py +84 -0
- endoreg_db/views/misc/secure_url_validate.py +79 -0
- endoreg_db/views/misc/stats.py +220 -0
- endoreg_db/views/misc/translation.py +182 -0
- endoreg_db/views/misc/upload_views.py +240 -0
- endoreg_db/views/patient/__init__.py +5 -0
- endoreg_db/views/patient/patient.py +210 -0
- endoreg_db/views/patient_examination/DEPRECATED_video_backup.py +164 -0
- endoreg_db/views/patient_examination/__init__.py +11 -0
- endoreg_db/views/patient_examination/patient_examination.py +140 -0
- endoreg_db/views/patient_examination/patient_examination_create.py +63 -0
- endoreg_db/views/patient_examination/patient_examination_detail.py +66 -0
- endoreg_db/views/patient_examination/patient_examination_list.py +68 -0
- endoreg_db/views/patient_examination/video.py +194 -0
- endoreg_db/views/patient_finding/__init__.py +7 -0
- endoreg_db/views/patient_finding/base.py +0 -0
- endoreg_db/views/patient_finding/patient_finding.py +64 -0
- endoreg_db/views/patient_finding/patient_finding_optimized.py +259 -0
- endoreg_db/views/patient_finding_classification/__init__.py +5 -0
- endoreg_db/views/patient_finding_classification/pfc_create.py +67 -0
- endoreg_db/views/patient_finding_location/__init__.py +5 -0
- endoreg_db/views/patient_finding_location/pfl_create.py +70 -0
- endoreg_db/views/patient_finding_morphology/__init__.py +5 -0
- endoreg_db/views/patient_finding_morphology/pfm_create.py +70 -0
- endoreg_db/views/pdf/__init__.py +8 -0
- endoreg_db/views/pdf/pdf_stream.py +187 -0
- endoreg_db/views/pdf/reimport.py +177 -0
- endoreg_db/views/report/__init__.py +9 -0
- endoreg_db/views/report/report_list.py +112 -0
- endoreg_db/views/report/report_with_secure_url.py +28 -0
- endoreg_db/views/report/start_examination.py +7 -0
- endoreg_db/views/requirement/__init__.py +10 -0
- endoreg_db/views/requirement/evaluate.py +279 -0
- endoreg_db/views/requirement/lookup.py +367 -0
- endoreg_db/views/requirement/lookup_store.py +252 -0
- endoreg_db/views/requirement_lookup/lookup.py +0 -0
- endoreg_db/views/requirement_lookup/lookup_store.py +0 -0
- endoreg_db/views/stats/__init__.py +13 -0
- endoreg_db/views/stats/stats_views.py +229 -0
- endoreg_db/views/video/__init__.py +59 -0
- endoreg_db/views/video/correction.py +530 -0
- endoreg_db/views/video/reimport.py +195 -0
- endoreg_db/views/video/segmentation.py +274 -0
- endoreg_db/views/video/task_status.py +49 -0
- endoreg_db/views/video/timeline.py +46 -0
- endoreg_db/views/video/video_analyze.py +52 -0
- endoreg_db/views/video/video_apply_mask.py +48 -0
- endoreg_db/views/video/video_correction.py +21 -0
- endoreg_db/views/video/video_download_processed.py +58 -0
- endoreg_db/views/video/video_examination_viewset.py +242 -0
- endoreg_db/views/video/video_meta.py +29 -0
- endoreg_db/views/video/video_processing_history.py +24 -0
- endoreg_db/views/video/video_remove_frames.py +48 -0
- endoreg_db/views/video/video_stream.py +306 -0
- endoreg_db/views.py +0 -3
- endoreg_db-0.8.6.3.dist-info/METADATA +383 -0
- endoreg_db-0.8.6.3.dist-info/RECORD +793 -0
- {endoreg_db-0.4.5.dist-info β endoreg_db-0.8.6.3.dist-info}/WHEEL +1 -1
- endoreg_db/data/active_model/data.yaml +0 -3
- endoreg_db/data/agl_service/data.yaml +0 -19
- endoreg_db/data/label/label-set/data.yaml +0 -18
- endoreg_db/management/commands/_load_model_template.py +0 -41
- endoreg_db/management/commands/delete_all.py +0 -18
- endoreg_db/management/commands/delete_legacy_images.py +0 -19
- endoreg_db/management/commands/delete_legacy_videos.py +0 -17
- endoreg_db/management/commands/extract_legacy_video_frames.py +0 -18
- endoreg_db/management/commands/fetch_legacy_image_dataset.py +0 -32
- endoreg_db/management/commands/fix_auth_permission.py +0 -20
- endoreg_db/management/commands/import_legacy_images.py +0 -94
- endoreg_db/management/commands/import_legacy_videos.py +0 -76
- endoreg_db/management/commands/load_active_model_data.py +0 -45
- endoreg_db/management/commands/load_endoscopy_processor_data.py +0 -45
- endoreg_db/management/commands/load_g_play_data.py +0 -113
- endoreg_db/management/commands/load_label_data.py +0 -67
- endoreg_db/management/commands/load_medication_indication_data.py +0 -63
- endoreg_db/management/commands/load_medication_indication_type_data.py +0 -41
- endoreg_db/management/commands/load_medication_schedule_data.py +0 -55
- endoreg_db/management/commands/load_network_data.py +0 -57
- endoreg_db/migrations/0002_anonymizedimagelabel_anonymousimageannotation_and_more.py +0 -55
- endoreg_db/migrations/0003_anonymousimageannotation_original_image_url_and_more.py +0 -39
- endoreg_db/migrations/0004_alter_rawpdffile_file.py +0 -20
- endoreg_db/migrations/0005_uploadedfile_alter_rawpdffile_file_anonymizedfile.py +0 -40
- endoreg_db/migrations/0006_alter_rawpdffile_file.py +0 -20
- endoreg_db/migrations/0007_networkdevicelogentry_datetime_and_more.py +0 -43
- endoreg_db/models/ai_model/__init__.py +0 -3
- endoreg_db/models/ai_model/active_model.py +0 -9
- endoreg_db/models/ai_model/model_meta.py +0 -24
- endoreg_db/models/annotation/__init__.py +0 -3
- endoreg_db/models/annotation/anonymized_image_annotation.py +0 -60
- endoreg_db/models/annotation/binary_classification_annotation_task.py +0 -80
- endoreg_db/models/annotation/image_classification.py +0 -27
- endoreg_db/models/center/center.py +0 -25
- endoreg_db/models/center/center_product.py +0 -34
- endoreg_db/models/center/center_resource.py +0 -19
- endoreg_db/models/center/center_waste.py +0 -11
- endoreg_db/models/data_file/__init__.py +0 -6
- endoreg_db/models/data_file/base_classes/__init__.py +0 -2
- endoreg_db/models/data_file/base_classes/abstract_frame.py +0 -51
- endoreg_db/models/data_file/base_classes/abstract_video.py +0 -201
- endoreg_db/models/data_file/frame.py +0 -45
- endoreg_db/models/data_file/import_classes/__init__.py +0 -32
- endoreg_db/models/data_file/import_classes/processing_functions/__init__.py +0 -35
- endoreg_db/models/data_file/import_classes/processing_functions/pdf.py +0 -28
- endoreg_db/models/data_file/import_classes/processing_functions/video.py +0 -260
- endoreg_db/models/data_file/import_classes/raw_pdf.py +0 -188
- endoreg_db/models/data_file/import_classes/raw_video.py +0 -343
- endoreg_db/models/data_file/metadata/__init__.py +0 -3
- endoreg_db/models/data_file/metadata/sensitive_meta.py +0 -31
- endoreg_db/models/data_file/metadata/video_meta.py +0 -133
- endoreg_db/models/data_file/report_file.py +0 -89
- endoreg_db/models/data_file/video/__init__.py +0 -7
- endoreg_db/models/data_file/video/import_meta.py +0 -25
- endoreg_db/models/data_file/video/video.py +0 -25
- endoreg_db/models/data_file/video_segment.py +0 -107
- endoreg_db/models/disease.py +0 -56
- endoreg_db/models/emission/__init__.py +0 -1
- endoreg_db/models/emission/emission_factor.py +0 -20
- endoreg_db/models/event.py +0 -22
- endoreg_db/models/examination/__init__.py +0 -4
- endoreg_db/models/examination/examination.py +0 -26
- endoreg_db/models/examination/examination_time.py +0 -27
- endoreg_db/models/examination/examination_time_type.py +0 -24
- endoreg_db/models/examination/examination_type.py +0 -18
- endoreg_db/models/hardware/__init__.py +0 -2
- endoreg_db/models/hardware/endoscope.py +0 -44
- endoreg_db/models/information_source.py +0 -29
- endoreg_db/models/laboratory/__init__.py +0 -1
- endoreg_db/models/laboratory/lab_value.py +0 -102
- endoreg_db/models/legacy_data/__init__.py +0 -3
- endoreg_db/models/legacy_data/image.py +0 -34
- endoreg_db/models/logging/__init__.py +0 -4
- endoreg_db/models/logging/agl_service.py +0 -19
- endoreg_db/models/logging/base.py +0 -22
- endoreg_db/models/logging/log_type.py +0 -23
- endoreg_db/models/logging/network_device.py +0 -24
- endoreg_db/models/medication/__init__.py +0 -1
- endoreg_db/models/medication/medication.py +0 -148
- endoreg_db/models/network/__init__.py +0 -3
- endoreg_db/models/network/agl_service.py +0 -38
- endoreg_db/models/network/network_device.py +0 -53
- endoreg_db/models/network/network_device_type.py +0 -23
- endoreg_db/models/other/distribution.py +0 -215
- endoreg_db/models/patient_examination/__init__.py +0 -35
- endoreg_db/models/permissions/__init__.py +0 -44
- endoreg_db/models/persons/__init__.py +0 -7
- endoreg_db/models/persons/examiner/__init__.py +0 -2
- endoreg_db/models/persons/examiner/examiner.py +0 -16
- endoreg_db/models/persons/examiner/examiner_type.py +0 -2
- endoreg_db/models/persons/patient/__init__.py +0 -8
- endoreg_db/models/persons/patient/case/case.py +0 -30
- endoreg_db/models/persons/patient/patient.py +0 -216
- endoreg_db/models/persons/patient/patient_disease.py +0 -16
- endoreg_db/models/persons/patient/patient_event.py +0 -22
- endoreg_db/models/persons/patient/patient_lab_sample.py +0 -106
- endoreg_db/models/persons/patient/patient_medication.py +0 -44
- endoreg_db/models/persons/patient/patient_medication_schedule.py +0 -28
- endoreg_db/models/persons/portal_user_information.py +0 -27
- endoreg_db/models/prediction/__init__.py +0 -2
- endoreg_db/models/prediction/image_classification.py +0 -37
- endoreg_db/models/prediction/video_prediction_meta.py +0 -244
- endoreg_db/models/product/__init__.py +0 -5
- endoreg_db/models/product/product.py +0 -97
- endoreg_db/models/product/product_group.py +0 -19
- endoreg_db/models/product/product_material.py +0 -24
- endoreg_db/models/questionnaires/__init__.py +0 -114
- endoreg_db/models/quiz/__init__.py +0 -2
- endoreg_db/models/quiz/quiz_answer.py +0 -41
- endoreg_db/models/quiz/quiz_question.py +0 -54
- endoreg_db/models/report_reader/__init__.py +0 -2
- endoreg_db/models/report_reader/report_reader_config.py +0 -53
- endoreg_db/models/rules/__init__.py +0 -5
- endoreg_db/queries/get/__init__.py +0 -6
- endoreg_db/queries/get/center.py +0 -42
- endoreg_db/queries/get/model.py +0 -13
- endoreg_db/queries/get/patient.py +0 -14
- endoreg_db/queries/get/patient_examination.py +0 -20
- endoreg_db/queries/get/report_file.py +0 -33
- endoreg_db/queries/get/video.py +0 -31
- endoreg_db/serializers/ai_model.py +0 -19
- endoreg_db/serializers/annotation.py +0 -17
- endoreg_db/serializers/center.py +0 -11
- endoreg_db/serializers/examination.py +0 -33
- endoreg_db/serializers/frame.py +0 -13
- endoreg_db/serializers/hardware.py +0 -21
- endoreg_db/serializers/label.py +0 -22
- endoreg_db/serializers/patient.py +0 -10
- endoreg_db/serializers/prediction.py +0 -15
- endoreg_db/serializers/report_file.py +0 -7
- endoreg_db/serializers/video.py +0 -27
- endoreg_db/tests.py +0 -3
- endoreg_db/utils/legacy_ocr.py +0 -201
- endoreg_db/utils/video_metadata.py +0 -87
- endoreg_db-0.4.5.dist-info/METADATA +0 -34
- endoreg_db-0.4.5.dist-info/RECORD +0 -316
- /endoreg_db/{data/distribution/numeric/.init β api/serializers/finding_descriptions.py} +0 -0
- /endoreg_db/{models/persons/patient/case/__init__.py β api/views/finding_descriptions.py} +0 -0
- /endoreg_db/{queries/get/annotation.py β config/__init__.py} +0 -0
- /endoreg_db/data/{label β ai_model_label}/label-type/data.yaml +0 -0
- /endoreg_db/data/{model_type β ai_model_type}/data.yaml +0 -0
- /endoreg_db/{queries/get/prediction.py β data/shift/m2.yaml} +0 -0
- /endoreg_db/{queries/get/video_import_meta.py β factories/__init__.py} +0 -0
- /endoreg_db/{queries/get/video_prediction_meta.py β helpers/__init__.py} +0 -0
- /endoreg_db/management/commands/{load_report_reader_flag.py β load_report_reader_flag_data.py} +0 -0
- /endoreg_db/models/{persons β administration/person}/person.py +0 -0
- /endoreg_db/models/{report_reader β media/pdf/report_reader}/report_reader_flag.py +0 -0
- /endoreg_db/models/{rules β rule}/rule_applicator.py +0 -0
- {endoreg_db-0.4.5.dist-info β endoreg_db-0.8.6.3.dist-info/licenses}/LICENSE +0 -0
|
@@ -0,0 +1,306 @@
|
|
|
1
|
+
"""
|
|
2
|
+
Video Streaming Views (Phase 3.2)
|
|
3
|
+
|
|
4
|
+
Separate view for streaming raw and processed video files.
|
|
5
|
+
Extracted from segmentation.py for better code organization.
|
|
6
|
+
|
|
7
|
+
Created: October 9, 2025
|
|
8
|
+
Updated: October 15, 2025 - Added HTTP 206 Range Request Support
|
|
9
|
+
"""
|
|
10
|
+
|
|
11
|
+
import os
|
|
12
|
+
import re
|
|
13
|
+
import logging
|
|
14
|
+
import mimetypes
|
|
15
|
+
from pathlib import Path
|
|
16
|
+
from typing import Tuple, Optional
|
|
17
|
+
from django.http import FileResponse, Http404, StreamingHttpResponse
|
|
18
|
+
from rest_framework.views import APIView
|
|
19
|
+
|
|
20
|
+
from ...models import VideoFile
|
|
21
|
+
from ...utils.permissions import EnvironmentAwarePermission
|
|
22
|
+
from ...utils.paths import STORAGE_DIR # Import STORAGE_DIR for path resolution
|
|
23
|
+
|
|
24
|
+
logger = logging.getLogger(__name__)
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def parse_range_header(range_header: str, file_size: int) -> Tuple[int, int]:
|
|
28
|
+
"""
|
|
29
|
+
Parse HTTP Range header and return (start, end) byte positions.
|
|
30
|
+
|
|
31
|
+
Args:
|
|
32
|
+
range_header: HTTP Range header value (e.g., "bytes=0-1023")
|
|
33
|
+
file_size: Total file size in bytes
|
|
34
|
+
|
|
35
|
+
Returns:
|
|
36
|
+
Tuple of (start_byte, end_byte) inclusive
|
|
37
|
+
|
|
38
|
+
Raises:
|
|
39
|
+
ValueError: If range header is invalid
|
|
40
|
+
"""
|
|
41
|
+
# Expected format: "bytes=start-end" or "bytes=start-"
|
|
42
|
+
match = re.match(r'bytes=(\d+)-(\d*)', range_header)
|
|
43
|
+
|
|
44
|
+
if not match:
|
|
45
|
+
raise ValueError(f"Invalid Range header format: {range_header}")
|
|
46
|
+
|
|
47
|
+
start = int(match.group(1))
|
|
48
|
+
end_str = match.group(2)
|
|
49
|
+
|
|
50
|
+
# If end is not specified, use file size - 1
|
|
51
|
+
end = int(end_str) if end_str else file_size - 1
|
|
52
|
+
|
|
53
|
+
# Validate range
|
|
54
|
+
if start >= file_size or start < 0:
|
|
55
|
+
raise ValueError(f"Start byte {start} is out of range (file size: {file_size})")
|
|
56
|
+
|
|
57
|
+
if end >= file_size:
|
|
58
|
+
end = file_size - 1
|
|
59
|
+
|
|
60
|
+
if start > end:
|
|
61
|
+
raise ValueError(f"Invalid range: start ({start}) > end ({end})")
|
|
62
|
+
|
|
63
|
+
return start, end
|
|
64
|
+
|
|
65
|
+
|
|
66
|
+
def stream_file_chunk(file_path: Path, start: int, end: int, chunk_size: int = 8192):
|
|
67
|
+
"""
|
|
68
|
+
Generator that yields chunks of a file within the specified byte range.
|
|
69
|
+
|
|
70
|
+
Args:
|
|
71
|
+
file_path: Path to the file
|
|
72
|
+
start: Start byte position (inclusive)
|
|
73
|
+
end: End byte position (inclusive)
|
|
74
|
+
chunk_size: Size of each chunk to yield
|
|
75
|
+
|
|
76
|
+
Yields:
|
|
77
|
+
Bytes chunks from the file
|
|
78
|
+
"""
|
|
79
|
+
with open(file_path, 'rb') as f:
|
|
80
|
+
f.seek(start)
|
|
81
|
+
remaining = end - start + 1 # +1 because end is inclusive
|
|
82
|
+
|
|
83
|
+
while remaining > 0:
|
|
84
|
+
chunk = f.read(min(chunk_size, remaining))
|
|
85
|
+
if not chunk:
|
|
86
|
+
break
|
|
87
|
+
yield chunk
|
|
88
|
+
remaining -= len(chunk)
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
def _stream_video_file(vf: VideoFile, frontend_origin: str, file_type: str = 'raw', range_header: Optional[str] = None) -> FileResponse | StreamingHttpResponse:
|
|
92
|
+
"""
|
|
93
|
+
Helper function to stream a video file with proper headers, CORS support, and HTTP Range Requests.
|
|
94
|
+
|
|
95
|
+
Args:
|
|
96
|
+
vf: VideoFile model instance
|
|
97
|
+
frontend_origin: Frontend origin URL for CORS headers
|
|
98
|
+
file_type: Either 'raw' (original video) or 'processed' (anonymized video)
|
|
99
|
+
range_header: HTTP Range header value (e.g., "bytes=0-1023") for partial content requests
|
|
100
|
+
|
|
101
|
+
Returns:
|
|
102
|
+
FileResponse: HTTP 200 response streaming the entire file (no range header)
|
|
103
|
+
StreamingHttpResponse: HTTP 206 response streaming partial content (with range header)
|
|
104
|
+
|
|
105
|
+
Raises:
|
|
106
|
+
Http404: If video file not found or cannot be accessed
|
|
107
|
+
|
|
108
|
+
Note:
|
|
109
|
+
Permissions are handled by the calling view, not in this helper function.
|
|
110
|
+
HTTP 206 Partial Content support is critical for video seeking in browsers.
|
|
111
|
+
"""
|
|
112
|
+
try:
|
|
113
|
+
# Determine which file to stream based on file_type
|
|
114
|
+
if file_type == 'raw':
|
|
115
|
+
if hasattr(vf, 'active_raw_file') and vf.active_raw_file and hasattr(vf.active_raw_file, 'name'):
|
|
116
|
+
file_ref = vf.active_raw_file
|
|
117
|
+
else:
|
|
118
|
+
raise Http404("No raw video file available for this entry")
|
|
119
|
+
|
|
120
|
+
elif file_type == 'processed':
|
|
121
|
+
if hasattr(vf, 'processed_file') and vf.processed_file and hasattr(vf.processed_file, 'name'):
|
|
122
|
+
file_ref = vf.processed_file
|
|
123
|
+
else:
|
|
124
|
+
raise Http404("No processed video file available for this entry")
|
|
125
|
+
else:
|
|
126
|
+
raise ValueError(f"Invalid file_type: {file_type}. Must be 'raw' or 'processed'.")
|
|
127
|
+
|
|
128
|
+
# FIX: Handle both relative and absolute paths
|
|
129
|
+
# Django FileField.path returns .name if MEDIA_ROOT is not set
|
|
130
|
+
# Import services store relative paths like "videos/UUID.mp4"
|
|
131
|
+
# We need to resolve to absolute path: STORAGE_DIR / "videos/UUID.mp4"
|
|
132
|
+
file_name = file_ref.name
|
|
133
|
+
|
|
134
|
+
if file_name.startswith('/'):
|
|
135
|
+
# Already absolute path
|
|
136
|
+
path = Path(file_name)
|
|
137
|
+
else:
|
|
138
|
+
# Relative path - make absolute by prepending STORAGE_DIR
|
|
139
|
+
path = STORAGE_DIR / file_name
|
|
140
|
+
logger.debug("Resolved relative path '%s' to absolute: %s", file_name, path)
|
|
141
|
+
|
|
142
|
+
# Validate file exists on disk
|
|
143
|
+
if not path.exists():
|
|
144
|
+
raise Http404(f"Video file not found on disk: {path}")
|
|
145
|
+
|
|
146
|
+
# Validate file size before streaming
|
|
147
|
+
try:
|
|
148
|
+
file_size = path.stat().st_size
|
|
149
|
+
if file_size == 0:
|
|
150
|
+
raise Http404("Video file is empty")
|
|
151
|
+
except OSError as e:
|
|
152
|
+
raise Http404(f"Cannot access video file: {str(e)}")
|
|
153
|
+
|
|
154
|
+
# Determine MIME type
|
|
155
|
+
mime, _ = mimetypes.guess_type(str(path))
|
|
156
|
+
content_type = mime or 'video/mp4' # Default to mp4 if detection fails
|
|
157
|
+
|
|
158
|
+
# β
NEW: HTTP Range Request support for video seeking
|
|
159
|
+
if range_header:
|
|
160
|
+
try:
|
|
161
|
+
# Parse Range header
|
|
162
|
+
start, end = parse_range_header(range_header, file_size)
|
|
163
|
+
logger.debug("Range request: bytes=%d-%d (total: %d)", start, end, file_size)
|
|
164
|
+
|
|
165
|
+
# Stream partial content (HTTP 206)
|
|
166
|
+
response = StreamingHttpResponse(
|
|
167
|
+
stream_file_chunk(path, start, end),
|
|
168
|
+
status=206, # Partial Content
|
|
169
|
+
content_type=content_type
|
|
170
|
+
)
|
|
171
|
+
|
|
172
|
+
# Set Range-specific headers
|
|
173
|
+
response['Content-Range'] = f'bytes {start}-{end}/{file_size}'
|
|
174
|
+
response['Content-Length'] = str(end - start + 1)
|
|
175
|
+
response['Accept-Ranges'] = 'bytes'
|
|
176
|
+
response['Content-Disposition'] = f'inline; filename="{path.name}"'
|
|
177
|
+
|
|
178
|
+
except ValueError as e:
|
|
179
|
+
# Invalid range header - return 416 Range Not Satisfiable
|
|
180
|
+
logger.warning("Invalid Range header: %s", str(e))
|
|
181
|
+
response = StreamingHttpResponse(
|
|
182
|
+
status=416, # Range Not Satisfiable
|
|
183
|
+
content_type=content_type
|
|
184
|
+
)
|
|
185
|
+
response['Content-Range'] = f'bytes */{file_size}'
|
|
186
|
+
|
|
187
|
+
else:
|
|
188
|
+
# No Range header - stream entire file (HTTP 200)
|
|
189
|
+
try:
|
|
190
|
+
# Open file in binary mode - FileResponse will handle closing
|
|
191
|
+
file_handle = open(path, 'rb')
|
|
192
|
+
response = FileResponse(file_handle, content_type=content_type)
|
|
193
|
+
|
|
194
|
+
# Set HTTP headers for video streaming
|
|
195
|
+
response['Content-Length'] = str(file_size)
|
|
196
|
+
response['Accept-Ranges'] = 'bytes' # Enable HTTP range requests for seeking
|
|
197
|
+
response['Content-Disposition'] = f'inline; filename="{path.name}"'
|
|
198
|
+
|
|
199
|
+
except IOError as e:
|
|
200
|
+
raise Http404(f"Cannot open video file: {str(e)}")
|
|
201
|
+
|
|
202
|
+
# CORS headers for frontend access (both HTTP 200 and 206)
|
|
203
|
+
response["Access-Control-Allow-Origin"] = frontend_origin
|
|
204
|
+
response["Access-Control-Allow-Credentials"] = "true"
|
|
205
|
+
|
|
206
|
+
return response
|
|
207
|
+
|
|
208
|
+
except Exception as e:
|
|
209
|
+
# Log unexpected errors but don't expose internal details
|
|
210
|
+
logger.error("Unexpected error in _stream_video_file: %s", str(e))
|
|
211
|
+
raise Http404("Video file cannot be streamed")
|
|
212
|
+
|
|
213
|
+
|
|
214
|
+
class VideoStreamView(APIView):
|
|
215
|
+
"""
|
|
216
|
+
Separate view for video streaming to avoid DRF content negotiation issues.
|
|
217
|
+
|
|
218
|
+
Supports streaming both raw (original) and processed (anonymized) videos.
|
|
219
|
+
|
|
220
|
+
Query Parameters:
|
|
221
|
+
type: 'raw' (default) or 'processed' - Selects which video file to stream
|
|
222
|
+
file_type: (deprecated, use 'type') - Legacy parameter for backwards compatibility
|
|
223
|
+
|
|
224
|
+
Examples:
|
|
225
|
+
GET /api/media/videos/1/?type=raw - Stream original raw video
|
|
226
|
+
GET /api/media/videos/1/?type=processed - Stream anonymized/masked video
|
|
227
|
+
GET /api/videostream/1/ - Default to raw video (legacy endpoint)
|
|
228
|
+
|
|
229
|
+
Phase 3.2 Implementation:
|
|
230
|
+
- Supports dual video comparison (raw vs processed)
|
|
231
|
+
- Backward compatible with legacy ?file_type= parameter
|
|
232
|
+
- Proper error handling with Http404
|
|
233
|
+
- CORS support for frontend access
|
|
234
|
+
- HTTP range support for video seeking
|
|
235
|
+
"""
|
|
236
|
+
permission_classes = [EnvironmentAwarePermission]
|
|
237
|
+
|
|
238
|
+
def get(self, request, pk=None):
|
|
239
|
+
"""
|
|
240
|
+
Stream raw or anonymized video file with HTTP Range Request and CORS support.
|
|
241
|
+
|
|
242
|
+
Supports HTTP 206 Partial Content for video seeking functionality.
|
|
243
|
+
|
|
244
|
+
Args:
|
|
245
|
+
request: HTTP request object
|
|
246
|
+
pk: Video ID (primary key)
|
|
247
|
+
|
|
248
|
+
Returns:
|
|
249
|
+
FileResponse: HTTP 200 streaming entire video file (no range header)
|
|
250
|
+
StreamingHttpResponse: HTTP 206 streaming partial content (with range header)
|
|
251
|
+
|
|
252
|
+
Raises:
|
|
253
|
+
Http404: If video not found or file cannot be accessed
|
|
254
|
+
"""
|
|
255
|
+
if pk is None:
|
|
256
|
+
raise Http404("Video ID is required")
|
|
257
|
+
|
|
258
|
+
# Initialize variables in outer scope
|
|
259
|
+
video_id_int = None
|
|
260
|
+
|
|
261
|
+
try:
|
|
262
|
+
# Validate video_id is numeric
|
|
263
|
+
try:
|
|
264
|
+
video_id_int = int(pk)
|
|
265
|
+
except (ValueError, TypeError):
|
|
266
|
+
raise Http404("Invalid video ID format")
|
|
267
|
+
|
|
268
|
+
# Support both 'type' (frontend standard) and 'file_type' (legacy)
|
|
269
|
+
# Priority: type > file_type > default 'raw'
|
|
270
|
+
file_type = 'raw' # Default value
|
|
271
|
+
try:
|
|
272
|
+
file_type_param = request.query_params.get('type') or request.query_params.get('file_type')
|
|
273
|
+
if file_type_param:
|
|
274
|
+
file_type = file_type_param.lower()
|
|
275
|
+
|
|
276
|
+
if file_type not in ['raw', 'processed']:
|
|
277
|
+
logger.warning("Invalid file_type '%s', defaulting to 'raw'", file_type)
|
|
278
|
+
file_type = 'raw'
|
|
279
|
+
|
|
280
|
+
except Exception as e:
|
|
281
|
+
logger.warning("Error parsing file_type parameter: %s, defaulting to 'raw'", e)
|
|
282
|
+
file_type = 'raw'
|
|
283
|
+
|
|
284
|
+
# Fetch video from database
|
|
285
|
+
vf = VideoFile.objects.get(pk=video_id_int)
|
|
286
|
+
|
|
287
|
+
# Get frontend origin for CORS
|
|
288
|
+
frontend_origin = os.environ.get('FRONTEND_ORIGIN', 'http://localhost:8000')
|
|
289
|
+
|
|
290
|
+
# β
NEW: Extract Range header for HTTP 206 support
|
|
291
|
+
range_header = request.META.get('HTTP_RANGE')
|
|
292
|
+
|
|
293
|
+
# Stream the video file with optional range support
|
|
294
|
+
return _stream_video_file(vf, frontend_origin, file_type, range_header)
|
|
295
|
+
|
|
296
|
+
except VideoFile.DoesNotExist:
|
|
297
|
+
raise Http404(f"Video with ID {pk} not found")
|
|
298
|
+
|
|
299
|
+
except Http404:
|
|
300
|
+
# Re-raise Http404 exceptions as they should bubble up
|
|
301
|
+
raise
|
|
302
|
+
|
|
303
|
+
except Exception as e:
|
|
304
|
+
# Log unexpected errors and convert to Http404
|
|
305
|
+
logger.error("Unexpected error in VideoStreamView for video_id=%s: %s", pk, str(e))
|
|
306
|
+
raise Http404("Video streaming failed")
|
endoreg_db/views.py
CHANGED
|
@@ -0,0 +1,383 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: endoreg-db
|
|
3
|
+
Version: 0.8.6.3
|
|
4
|
+
Summary: EndoReg Db Django App
|
|
5
|
+
Project-URL: Homepage, https://info.coloreg.de
|
|
6
|
+
Project-URL: Repository, https://github.com/wg-lux/endoreg-db
|
|
7
|
+
Author: Max Hild, Hamza Zahid, Peter Kowalczyk
|
|
8
|
+
Author-email: "Thomas J. Lux" <lux_t1@ukw.de>
|
|
9
|
+
Maintainer: Max Hild, Hamza Zahid, Peter Kowalczyk
|
|
10
|
+
Maintainer-email: "Thomas J. Lux" <lux_t1@ukw.de>
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
14
|
+
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
|
|
15
|
+
Requires-Python: <3.13,>=3.12
|
|
16
|
+
Requires-Dist: agl-report-reader>=0.4.0
|
|
17
|
+
Requires-Dist: build>=1.3.0
|
|
18
|
+
Requires-Dist: celery>=5.5.3
|
|
19
|
+
Requires-Dist: cryptography>=44.0.2
|
|
20
|
+
Requires-Dist: decord==0.6.0
|
|
21
|
+
Requires-Dist: django-bootstrap5>=25.2
|
|
22
|
+
Requires-Dist: django-cors-headers>=4.8.0
|
|
23
|
+
Requires-Dist: django-extensions>=3.2.3
|
|
24
|
+
Requires-Dist: django-filter>=25.1
|
|
25
|
+
Requires-Dist: django-modeltranslation>=0.19.16
|
|
26
|
+
Requires-Dist: django>=5.2.4
|
|
27
|
+
Requires-Dist: djangorestframework>=3.16.1
|
|
28
|
+
Requires-Dist: dotenv>=0.9.9
|
|
29
|
+
Requires-Dist: faker>=37.6.0
|
|
30
|
+
Requires-Dist: flake8>=7.3.0
|
|
31
|
+
Requires-Dist: gunicorn>=23.0.0
|
|
32
|
+
Requires-Dist: icecream>=2.1.4
|
|
33
|
+
Requires-Dist: librosa==0.11.0
|
|
34
|
+
Requires-Dist: llvmlite>=0.44.0
|
|
35
|
+
Requires-Dist: lx-anonymizer[llm,ocr]>=0.8.8
|
|
36
|
+
Requires-Dist: moviepy==2.2.1
|
|
37
|
+
Requires-Dist: mypy>=1.16.0
|
|
38
|
+
Requires-Dist: numpy>=2.2.3
|
|
39
|
+
Requires-Dist: opencv-python>=4.11.0.86
|
|
40
|
+
Requires-Dist: openpyxl>=3.1.5
|
|
41
|
+
Requires-Dist: optimum==1.26.1
|
|
42
|
+
Requires-Dist: pandas>=2.2.3
|
|
43
|
+
Requires-Dist: pillow>=11.3.0
|
|
44
|
+
Requires-Dist: pip>=25.2
|
|
45
|
+
Requires-Dist: pydantic>=2.10.6
|
|
46
|
+
Requires-Dist: pyjwt>=2.10.1
|
|
47
|
+
Requires-Dist: pytesseract>=0.3.13
|
|
48
|
+
Requires-Dist: pytest-cov>=6.1.1
|
|
49
|
+
Requires-Dist: pytest-django>=4.11.1
|
|
50
|
+
Requires-Dist: pytest-env>=1.1.5
|
|
51
|
+
Requires-Dist: pytest>=8.3.5
|
|
52
|
+
Requires-Dist: python-ffmpeg>=2.0.12
|
|
53
|
+
Requires-Dist: python-magic>=0.4.27
|
|
54
|
+
Requires-Dist: pytorch-lightning>=2.5.1
|
|
55
|
+
Requires-Dist: pyyaml>=6.0.2
|
|
56
|
+
Requires-Dist: regex>=2024.11.6
|
|
57
|
+
Requires-Dist: requests>=2.32.3
|
|
58
|
+
Requires-Dist: ruff>=0.14.0
|
|
59
|
+
Requires-Dist: scikit-learn>=1.5.2
|
|
60
|
+
Requires-Dist: scipy>=1.16.1
|
|
61
|
+
Requires-Dist: soundfile==0.12.1
|
|
62
|
+
Requires-Dist: sphinx-rtd-theme>=3.0.2
|
|
63
|
+
Requires-Dist: sphinx>=8.2.3
|
|
64
|
+
Requires-Dist: tesseract>=0.1.3
|
|
65
|
+
Requires-Dist: torch==2.5.1
|
|
66
|
+
Requires-Dist: torchaudio==2.5.1
|
|
67
|
+
Requires-Dist: torchvision==0.20.1
|
|
68
|
+
Requires-Dist: tqdm>=4.67.0
|
|
69
|
+
Requires-Dist: transformers==4.44.2
|
|
70
|
+
Requires-Dist: twine>=6.2.0
|
|
71
|
+
Requires-Dist: urllib3>=2.2.3
|
|
72
|
+
Requires-Dist: vector-quantize-pytorch==1.18.5
|
|
73
|
+
Requires-Dist: vocos==0.1.0
|
|
74
|
+
Requires-Dist: whitenoise>=6.10.0
|
|
75
|
+
Provides-Extra: dev
|
|
76
|
+
Description-Content-Type: text/markdown
|
|
77
|
+
|
|
78
|
+
# EndoregDB - Professional Data Infrastructure for Clinical Research
|
|
79
|
+
|
|
80
|
+
EndoregDB is a comprehensive database framework designed to manage medical and research-related data for clinical trials. This repository focuses on efficient data processing, automated deployment, security, and reproducibility, offering a flexible setup for local development environments as well as distributed systems. It supports the integration of AI/ML tools and advanced image and report processing.
|
|
81
|
+
|
|
82
|
+
This infrastructure was originally designed for clinical research studies and is optimized for handling large data volumes, including:
|
|
83
|
+
|
|
84
|
+
- Medical reports,
|
|
85
|
+
- Patient imaging and video data,
|
|
86
|
+
- Clinical product and treatment data,
|
|
87
|
+
and more.
|
|
88
|
+
|
|
89
|
+
## π Key Features
|
|
90
|
+
|
|
91
|
+
### System Architecture
|
|
92
|
+
|
|
93
|
+
- **Modular Design**: Built on scalable and reusable components to simplify integration into various environments.
|
|
94
|
+
- **Multi-System Support**: Manages configurations for local workstations and production servers.
|
|
95
|
+
- **Role-Specific Configuration**: Predefined roles for common use cases:
|
|
96
|
+
- Medical data processing systems
|
|
97
|
+
- AI/ML model deployment
|
|
98
|
+
- Research workstation configuration
|
|
99
|
+
|
|
100
|
+
### Security & Data Management
|
|
101
|
+
|
|
102
|
+
- **Data Encryption**: All sensitive data is encrypted, and privacy policies are enforced.
|
|
103
|
+
- **Impermanence**: Stateless system configuration with persistence for critical data.
|
|
104
|
+
- **Access Control**: Role-based access and identity management integration.
|
|
105
|
+
|
|
106
|
+
### Data and Processing Environment
|
|
107
|
+
|
|
108
|
+
- **Data Processing**: Optimized for processing medical datasets with preprocessing tools.
|
|
109
|
+
- **AI/ML Support**:
|
|
110
|
+
- Integration of machine learning tools for predictive analysis.
|
|
111
|
+
- TensorFlow, PyTorch, and other frameworks supported for model training.
|
|
112
|
+
- **Image/Video Processing**: Support for analyzing patient images and clinical videos.
|
|
113
|
+
|
|
114
|
+
### Development Tools & Infrastructure
|
|
115
|
+
|
|
116
|
+
- **Data Science Toolchains**: Pre-configured environments for data processing, analysis, and visualization.
|
|
117
|
+
- **Monitoring & Logging**: Setup for continuous monitoring and logging to ensure system stability and performance.
|
|
118
|
+
|
|
119
|
+
---
|
|
120
|
+
|
|
121
|
+
## π Getting Started
|
|
122
|
+
|
|
123
|
+
### Prerequisites
|
|
124
|
+
|
|
125
|
+
- A Linux-based system (Ubuntu/Debian recommended) or NixOS
|
|
126
|
+
- Hardware with sufficient storage for data processing (at least 1 TB recommended)
|
|
127
|
+
|
|
128
|
+
### Quick Start
|
|
129
|
+
|
|
130
|
+
1. Clone the repository:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
git clone https://github.com/wg-lux/endoreg-db.git
|
|
134
|
+
cd endoreg-db
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
2. Set up your Python environment
|
|
138
|
+
We need to have a `devenv.nix` file.
|
|
139
|
+
This Nix `devenv.nix` configuration sets up a Python development environment for a Django-based project using `uv` for dependency management. It defines project directories, environment variables, runtime packages, and several development tasks and scripts.
|
|
140
|
+
|
|
141
|
+
**Some available Test Shortcuts**
|
|
142
|
+
|
|
143
|
+
- `runtests`: Runs all tests β `uv run python runtests.py`
|
|
144
|
+
- `runtests-dataloader`: Runs dataloader tests β `uv run python runtests.py 'dataloader'`
|
|
145
|
+
- `runtests-other`: Runs other miscellaneous tests β `uv run python runtests.py 'other'`
|
|
146
|
+
- `runtests-helpers`: Runs helper module tests β `uv run python runtests.py 'helpers'`
|
|
147
|
+
- `runtests-administration`: Runs admin module tests β `uv run python runtests.py 'administration'`
|
|
148
|
+
- `runtests-medical`: Runs medical module tests β `uv run python runtests.py 'medical'`
|
|
149
|
+
|
|
150
|
+
3. Then run
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
direnv allow
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
4. Run tests:
|
|
157
|
+
Call Devenv Script to run tests
|
|
158
|
+
|
|
159
|
+
```bash
|
|
160
|
+
runtests
|
|
161
|
+
```
|
|
162
|
+
Tests Overview
|
|
163
|
+
- These tests ensure the functionality of different models and scenarios.
|
|
164
|
+
- After running them, you can view the results as demonstrated in the image below:
|
|
165
|
+
|
|
166
|
+

|
|
167
|
+
|
|
168
|
+
5. Run
|
|
169
|
+
```python
|
|
170
|
+
python manage.py migrate
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
- It applies database migrations and make tables.
|
|
174
|
+
- It updates your database schema to match the current state of your Django models.
|
|
175
|
+
|
|
176
|
+
6. To load the database data run
|
|
177
|
+
```
|
|
178
|
+
python manage.py load_base_db_data
|
|
179
|
+
|
|
180
|
+
```
|
|
181
|
+

|
|
182
|
+

|
|
183
|
+

|
|
184
|
+

|
|
185
|
+

|
|
186
|
+

|
|
187
|
+

|
|
188
|
+
|
|
189
|
+
7. Accessing the Django Shell
|
|
190
|
+
- To fetch or interact with data in the terminal, run the following command to run the Django shell:
|
|
191
|
+
|
|
192
|
+
```bash
|
|
193
|
+
python manage.py shell
|
|
194
|
+
```
|
|
195
|
+
- Using the Django shell, you can:
|
|
196
|
+
- Import database models
|
|
197
|
+
- Fetch data from the database
|
|
198
|
+
- Access related data through model relationships (e.g., foreign keys, one-to-many, many-to-many)
|
|
199
|
+
- Example is shown below
|
|
200
|
+
|
|
201
|
+
#### EXAMPLE # 1
|
|
202
|
+

|
|
203
|
+
- Explanation:
|
|
204
|
+
This script fetches a patient by ID and prints their related examination(s) using Django ORM. It retrieves the examination name linked to the patient from the PatientExamination table.
|
|
205
|
+
|
|
206
|
+
#### EXAMPLE # 2
|
|
207
|
+

|
|
208
|
+
- Explanation:
|
|
209
|
+
In the Django shell, a specific ExaminationIndication named "colonoscopy_screening" was fetched, and its related FindingIntervention records were accessed using the reverse relation expected_interventions. The first intervention (colon_lesion_polypectomy_cold_snare) was then queried to confirm it is also linked to multiple indications, demonstrating a many-to-many relationship between indications and interventions.
|
|
210
|
+
|
|
211
|
+
#### EXAMPLE # 3
|
|
212
|
+

|
|
213
|
+
- Explanation:
|
|
214
|
+
All required labels (polyp, instrument, digital_chromo_endoscopy, etc.) are confirmed to exist. The first available video (VideoFile) was loaded, with a valid frame_dir. Using the label "polyp", 8 labeled polyp segments were found in that video, with specific start and end frame numbers.
|
|
215
|
+
|
|
216
|
+
#### EXAMPLE # 4
|
|
217
|
+
##### Image a
|
|
218
|
+
|
|
219
|
+

|
|
220
|
+
|
|
221
|
+
##### Image b - All classifications with their choices together
|
|
222
|
+
|
|
223
|
+

|
|
224
|
+
|
|
225
|
+
- Explanation: Using the Django shell to fetch all morphology classifications (e.g., NICE, Paris) and their related choices from the database.
|
|
226
|
+
|
|
227
|
+
|
|
228
|
+
|
|
229
|
+
|
|
230
|
+
---
|
|
231
|
+
## π¦ Database Backup and Restore
|
|
232
|
+
|
|
233
|
+
This project includes two shell scripts to **export** and **import** database data in JSON format using Django's management commands.
|
|
234
|
+
|
|
235
|
+
### Setup
|
|
236
|
+
|
|
237
|
+
First, make the scripts executable:
|
|
238
|
+
|
|
239
|
+
```bash
|
|
240
|
+
chmod +x import_db.sh
|
|
241
|
+
chmod +x export_db.sh
|
|
242
|
+
```
|
|
243
|
+
### Export (Backup) the Database
|
|
244
|
+
|
|
245
|
+
To export the current database into a JSON file:
|
|
246
|
+
|
|
247
|
+
```bash
|
|
248
|
+
./export_db.sh
|
|
249
|
+
```
|
|
250
|
+
This will create a backup file such as `endoreg_db_backup.json`.
|
|
251
|
+
|
|
252
|
+
#### List of the comands in 'export_db.sh'
|
|
253
|
+
1. `python manage.py dumpdata --indent 4 --output=endoreg_db_backup.json` (if migrate comand generates and stores data in database table then wee nee dto exclude those tables from dumping)
|
|
254
|
+
|
|
255
|
+
2. `python manage.py shell < fix_endoreg_db_backup_json.py`
|
|
256
|
+
|
|
257
|
+
### Import (Restore) the Database
|
|
258
|
+
|
|
259
|
+
To load the data back into the database
|
|
260
|
+
```bash
|
|
261
|
+
./import_db.sh
|
|
262
|
+
```
|
|
263
|
+
#### List of the comands in 'import_db.sh'
|
|
264
|
+
1. `rm dev_db.sqlite3`
|
|
265
|
+
2. `python manage.py migrate`
|
|
266
|
+
3. `python manage.py shell < fix_endoreg_db_backup_json.py`
|
|
267
|
+
4. `python manage.py loaddata endoreg_db_backup_fixed.json`
|
|
268
|
+
|
|
269
|
+
---
|
|
270
|
+
|
|
271
|
+
|
|
272
|
+
|
|
273
|
+
## π Repository Structure
|
|
274
|
+
|
|
275
|
+
```
|
|
276
|
+
endoreg-db/
|
|
277
|
+
βββ endoreg_db/ # Main Django app for medical data
|
|
278
|
+
β βββ case_generator/ # Sample case generator
|
|
279
|
+
β βββ data/ # Medical knowledge base
|
|
280
|
+
β βββ management/ # Data wrangling operations
|
|
281
|
+
β βββ models/ # Data models
|
|
282
|
+
β βββ migrations/ # Database migrations
|
|
283
|
+
β βββ serializers/ # Serializers for data
|
|
284
|
+
βββ .gitignore # Git ignore file for unnecessary files
|
|
285
|
+
βββ README.md # Project description and setup instructions
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
---
|
|
289
|
+
|
|
290
|
+
## π Security Features
|
|
291
|
+
|
|
292
|
+
- **Data Encryption**: All sensitive patient data is encrypted.
|
|
293
|
+
- **Role-Based Access Control**: Configurable roles for managing access to various parts of the system.
|
|
294
|
+
- **Logging & Auditing**: Comprehensive logging system that tracks user activities and data changes.
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
## π₯οΈ Supported Systems
|
|
299
|
+
|
|
300
|
+
- **Workstations**: Local development or research workstations with low data processing demands.
|
|
301
|
+
- **Servers**: Scalable server infrastructure for processing large data volumes, integrated with cloud services for scalability.
|
|
302
|
+
|
|
303
|
+
---
|
|
304
|
+
|
|
305
|
+
## π Support
|
|
306
|
+
|
|
307
|
+
For issues and questions:
|
|
308
|
+
|
|
309
|
+
- Create an issue in the repository
|
|
310
|
+
- Review the Deployment Guide for common issues
|
|
311
|
+
|
|
312
|
+
---
|
|
313
|
+
|
|
314
|
+
## π License
|
|
315
|
+
|
|
316
|
+
MIT - see LICENSE
|
|
317
|
+
|
|
318
|
+
---
|
|
319
|
+
|
|
320
|
+
|
|
321
|
+
## π Further Documentation
|
|
322
|
+
|
|
323
|
+
All extended documentation lives in the project **Wiki** β **[Browse the Wiki Β»](https://github.com/wg-lux/endoreg-db/wiki)**
|
|
324
|
+
|
|
325
|
+
### Optimization Documentation
|
|
326
|
+
- [Complete Optimization Project Report](https://github.com/wg-lux/endoreg-db/wiki/Complete-Optimiztion-Project-Report)
|
|
327
|
+
- [Test Performance Optimization Guide](https://github.com/wg-lux/endoreg-db/wiki/Test-Performance-Optimization-Guide)
|
|
328
|
+
- [Test Performance Optimization - Success Summary](https://github.com/wg-lux/endoreg-db/wiki/Test-Performance-Optimization-β-Succes-Summary)
|
|
329
|
+
- [Test Performance Optimization: Complete Implementation Summary](https://github.com/wg-lux/endoreg-db/wiki/Test-Performance-Optimization:-Complete-Implementation-Summary)
|
|
330
|
+
- [Test Suite Optimization - Final Status Report](https://github.com/wg-lux/endoreg-db/wiki/Test-Suite-Optimization-β-Final-Status-Report)
|
|
331
|
+
- [Test Suite Analysis & Optimization Plan](https://github.com/wg-lux/endoreg-db/wiki/Test-Suite-Analysis-&-Optimization-Plan)
|
|
332
|
+
|
|
333
|
+
---
|
|
334
|
+
|
|
335
|
+
### Models and Migration Documentation
|
|
336
|
+
- [Models Documentation](https://github.com/wg-lux/endoreg-db/wiki/Models-Documentation)
|
|
337
|
+
- [Test Migration & Optimization Report](https://github.com/wg-lux/endoreg-db/wiki/Test-Migration-&-Optimization-Report)
|
|
338
|
+
- [Test Migration Success Summary](https://github.com/wg-lux/endoreg-db/wiki/Test-Migration-Success-Summary)
|
|
339
|
+
- [Test Optimization Migration Guide](https://github.com/wg-lux/endoreg-db/wiki/Test-Optimization-Migration-Guide)
|
|
340
|
+
|
|
341
|
+
---
|
|
342
|
+
|
|
343
|
+
### API Documentation
|
|
344
|
+
- [Upload API Documentation](https://github.com/wg-lux/endoreg-db/wiki/Upload-API-Documentation)
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
### Frame-Anonymisierung
|
|
349
|
+
- [Frame-Anonymisierung](https://github.com/wg-lux/endoreg-db/wiki/Frame-Anonymisierung)
|
|
350
|
+
|
|
351
|
+
---
|
|
352
|
+
|
|
353
|
+
### Tutorials Documentation
|
|
354
|
+
- [Run Production Server](https://github.com/wg-lux/endoreg-db/wiki/Run-Production-Server)
|
|
355
|
+
- [Date and Time Standardization for Models](https://github.com/wg-lux/endoreg-db/wiki/Date-and-Time-Standardization-for-Models)
|
|
356
|
+
|
|
357
|
+
---
|
|
358
|
+
|
|
359
|
+
### Keycloak
|
|
360
|
+
- [How to Create a New Account for Keycloak + Nextcloud](https://github.com/wg-lux/endoreg-db/wiki/How-to-Create-a-New-Account-for-Keycloak-+-Nextcloud)
|
|
361
|
+
- [Integration with the frontend](https://github.com/wg-lux/endoreg-db/wiki/Integration-with-the-frontend)
|
|
362
|
+
- [Merging Multi-User Accounts in Nextcloud // current options](https://github.com/wg-lux/endoreg-db/wiki/Merging-Multi-User-Accounts-in-Nextcloud-//-current-options)
|
|
363
|
+
- [New user login steps for keycloak and nextcloud](https://github.com/wg-lux/endoreg-db/wiki/New-user-login-steps-for-keycloak-and-nextcloud)
|
|
364
|
+
- [keycloak integration with backend endpoint](https://github.com/wg-lux/endoreg-db/wiki/keycloak-integration-with-backend-endpoint)
|
|
365
|
+
|
|
366
|
+
---
|
|
367
|
+
|
|
368
|
+
### Coding Principles & Practices
|
|
369
|
+
- [Timestamp Naming Standard](https://github.com/wg-lux/endoreg-db/wiki/Timestamp-Naming-Standard)
|
|
370
|
+
|
|
371
|
+
---
|
|
372
|
+
|
|
373
|
+
### Figures
|
|
374
|
+
- [Coloreg](https://github.com/wg-lux/endoreg-db/wiki/Coloreg)
|
|
375
|
+
- [EndoReg Framework](https://github.com/wg-lux/endoreg-db/wiki/EndoReg-Framework)
|
|
376
|
+
- [EndoReg Data Collection Workflow](https://github.com/wg-lux/endoreg-db/wiki/EndoReg-Data-Collection-Workflow)
|
|
377
|
+
- [Eine gemeinsame Datenplattform fΓΌr Klinik & Forschung](https://github.com/wg-lux/endoreg-db/wiki/Eine-gemeinsame-Datenplattform-fΓΌr-Klinik-&-Forschung)
|
|
378
|
+
|
|
379
|
+
---
|
|
380
|
+
|
|
381
|
+
### Miscellaneous
|
|
382
|
+
- [Requirement System Guide](https://github.com/wg-lux/endoreg-db/wiki/Requirement-System-Guide)
|
|
383
|
+
- [Official Site Link](https://github.com/wg-lux/endoreg-db/wiki/Official-Site-Link)
|