docling-pipelines 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- docling_pipelines-1.0.0/.coveragerc +32 -0
- docling_pipelines-1.0.0/.dockerignore +77 -0
- docling_pipelines-1.0.0/.env.example +187 -0
- docling_pipelines-1.0.0/.env.oauth2.example +123 -0
- docling_pipelines-1.0.0/.gitattributes +4 -0
- docling_pipelines-1.0.0/.github/ISSUE_TEMPLATE/bug_report.md +26 -0
- docling_pipelines-1.0.0/.github/ISSUE_TEMPLATE/config.yml +1 -0
- docling_pipelines-1.0.0/.github/ISSUE_TEMPLATE/feature_request.md +18 -0
- docling_pipelines-1.0.0/.github/ISSUE_TEMPLATE/question.md +14 -0
- docling_pipelines-1.0.0/.github/SECURITY.md +51 -0
- docling_pipelines-1.0.0/.github/dco.yml +2 -0
- docling_pipelines-1.0.0/.github/mergify.yml +24 -0
- docling_pipelines-1.0.0/.github/pull_request_template.md +60 -0
- docling_pipelines-1.0.0/.github/workflows/ci.yml +108 -0
- docling_pipelines-1.0.0/.github/workflows/dco-advisor.yml +185 -0
- docling_pipelines-1.0.0/.github/workflows/pypi.yml +62 -0
- docling_pipelines-1.0.0/.gitignore +86 -0
- docling_pipelines-1.0.0/.pre-commit-config.yaml +80 -0
- docling_pipelines-1.0.0/.secrets.baseline +264 -0
- docling_pipelines-1.0.0/.whitesource +3 -0
- docling_pipelines-1.0.0/AGENTS.md +255 -0
- docling_pipelines-1.0.0/ARCHITECTURE.md +4700 -0
- docling_pipelines-1.0.0/CHANGELOG.md +61 -0
- docling_pipelines-1.0.0/CODEOWNERS +6 -0
- docling_pipelines-1.0.0/COMMUNITY.md +152 -0
- docling_pipelines-1.0.0/CONTRIBUTING.md +1033 -0
- docling_pipelines-1.0.0/LICENSE +21 -0
- docling_pipelines-1.0.0/MAINTAINERS.md +19 -0
- docling_pipelines-1.0.0/PKG-INFO +245 -0
- docling_pipelines-1.0.0/QUICKSTART.md +407 -0
- docling_pipelines-1.0.0/README.md +111 -0
- docling_pipelines-1.0.0/RELEASE_PROCESS.md +256 -0
- docling_pipelines-1.0.0/TROUBLESHOOTING.md +2889 -0
- docling_pipelines-1.0.0/USER_GUIDE_PIPELINE_SETUP.md +972 -0
- docling_pipelines-1.0.0/docker/Dockerfile +50 -0
- docling_pipelines-1.0.0/docker/all-in-one-deployment/.env.example +17 -0
- docling_pipelines-1.0.0/docker/all-in-one-deployment/README.md +172 -0
- docling_pipelines-1.0.0/docker/all-in-one-deployment/docker-compose.yml +157 -0
- docling_pipelines-1.0.0/docker/docker-compose.distributed.yml +265 -0
- docling_pipelines-1.0.0/docker/docker-compose.milvus.yml +131 -0
- docling_pipelines-1.0.0/docker/docker-compose.opensearch.yml +64 -0
- docling_pipelines-1.0.0/docker/docker-compose.telemetry.yml +60 -0
- docling_pipelines-1.0.0/docling-pipelines-config.yaml +87 -0
- docling_pipelines-1.0.0/docling-pipelines-config.yaml.example +300 -0
- docling_pipelines-1.0.0/docs/README.md +153 -0
- docling_pipelines-1.0.0/docs/api/ACL_DOCUMENT_RETRIEVAL.md +467 -0
- docling_pipelines-1.0.0/docs/api/OAUTH2_AUTHENTICATION.md +333 -0
- docling_pipelines-1.0.0/docs/api/REST_API_SERVER.md +201 -0
- docling_pipelines-1.0.0/docs/deployment/OPENSHIFT.md +661 -0
- docling_pipelines-1.0.0/docs/guides/ADVANCED_CONFIGURATION.md +315 -0
- docling_pipelines-1.0.0/docs/guides/CREATE_CONNECTOR_GUIDE.md +823 -0
- docling_pipelines-1.0.0/docs/guides/CUSTOM_OPERATORS_GUIDE.md +1339 -0
- docling_pipelines-1.0.0/docs/guides/DEPRECATION_POLICY.md +201 -0
- docling_pipelines-1.0.0/docs/guides/DOCUMENTATION_STYLE_GUIDE.md +403 -0
- docling_pipelines-1.0.0/docs/guides/DOCUMENT_CLASS_UTILS.md +215 -0
- docling_pipelines-1.0.0/docs/guides/EXTERNAL_OPERATOR_INTEGRATION.md +447 -0
- docling_pipelines-1.0.0/docs/guides/FLOW_AUTHORING_FORMAT.md +630 -0
- docling_pipelines-1.0.0/docs/guides/FLOW_CONFIGURATION_GUIDE.md +693 -0
- docling_pipelines-1.0.0/docs/guides/JOB_REPORT_GENERATION.md +150 -0
- docling_pipelines-1.0.0/docs/guides/LOGGING_BEST_PRACTICES.md +436 -0
- docling_pipelines-1.0.0/docs/guides/MIGRATION_GUIDE_TEMPLATE.md +139 -0
- docling_pipelines-1.0.0/docs/guides/PYTHON_API_GUIDE.md +622 -0
- docling_pipelines-1.0.0/docs/guides/SECURITY_BEST_PRACTICES.md +312 -0
- docling_pipelines-1.0.0/docs/guides/SLIM_VARIANT.md +81 -0
- docling_pipelines-1.0.0/docs/guides/TELEMETRY_SETUP.md +507 -0
- docling_pipelines-1.0.0/docs/guides/TESTING_STANDARDS.md +176 -0
- docling_pipelines-1.0.0/docs/guides/UNIFIED_LLM_ARCHITECTURE_GUIDE.md +752 -0
- docling_pipelines-1.0.0/docs/guides/USER_GUIDE_DOCUMENT_LIBRARIES.md +603 -0
- docling_pipelines-1.0.0/docs/integrations/milvus/README.md +869 -0
- docling_pipelines-1.0.0/docs/integrations/opensearch/ENVIRONMENT_SETUP.md +46 -0
- docling_pipelines-1.0.0/docs/integrations/opensearch/OPENSEARCH_QUICKSTART.md +20 -0
- docling_pipelines-1.0.0/docs/integrations/opensearch/README.md +20 -0
- docling_pipelines-1.0.0/docs/integrations/opensearch/SCHEMA_TEMPLATES.md +836 -0
- docling_pipelines-1.0.0/docs/integrations/prefect/DISTRIBUTED_EXECUTION_GUIDE.md +1692 -0
- docling_pipelines-1.0.0/docs/internals/DOCUMENT_LIBRARIES_ARCHITECTURE.md +445 -0
- docling_pipelines-1.0.0/docs/internals/NODE_METADATA_AGGREGATION_STRATEGY.md +506 -0
- docling_pipelines-1.0.0/docs/internals/UNIFIED_ASSET_ARCHITECTURE.md +336 -0
- docling_pipelines-1.0.0/docs/operators/extract/acl_readme.md +195 -0
- docling_pipelines-1.0.0/docs/operators/extract/extract_operator_readme.md +1338 -0
- docling_pipelines-1.0.0/docs/operators/functional/branching_operator_readme.md +128 -0
- docling_pipelines-1.0.0/docs/operators/functional/chunker_readme.md +238 -0
- docling_pipelines-1.0.0/docs/operators/functional/doc_id_hash_readme.md +65 -0
- docling_pipelines-1.0.0/docs/operators/functional/embeddings_readme.md +244 -0
- docling_pipelines-1.0.0/docs/operators/functional/entity_curation_readme.md +217 -0
- docling_pipelines-1.0.0/docs/operators/functional/merge_readme.md +115 -0
- docling_pipelines-1.0.0/docs/operators/functional/noop_readme.md +72 -0
- docling_pipelines-1.0.0/docs/operators/ingest/ingest_source_readme.md +906 -0
- docling_pipelines-1.0.0/docs/operators/quality/doc_quality_readme.md +108 -0
- docling_pipelines-1.0.0/docs/operators/quality/document_classifier_readme.md +741 -0
- docling_pipelines-1.0.0/docs/operators/quality/ededup_readme.md +92 -0
- docling_pipelines-1.0.0/docs/operators/quality/language_detection_readme.md +414 -0
- docling_pipelines-1.0.0/docs/operators/quality/ml_enrichment_readme.md +127 -0
- docling_pipelines-1.0.0/docs/operators/quality/pii_and_hap_readme.md +235 -0
- docling_pipelines-1.0.0/docs/operators/quality/readability_readme.md +139 -0
- docling_pipelines-1.0.0/docs/operators/quality/redaction_readme.md +110 -0
- docling_pipelines-1.0.0/docs/operators/quality/sql_filter_readme.md +120 -0
- docling_pipelines-1.0.0/docs/operators/storage/README.md +674 -0
- docling_pipelines-1.0.0/docs/operators/storage/document_set_readme.md +174 -0
- docling_pipelines-1.0.0/docs/operators/storage/storage_output_readme.md +483 -0
- docling_pipelines-1.0.0/docs/operators/vectordb/milvus.md +513 -0
- docling_pipelines-1.0.0/docs/operators/vectordb/opensearch.md +524 -0
- docling_pipelines-1.0.0/docs/operators/vectordb/vectordb_readme.md +310 -0
- docling_pipelines-1.0.0/docs/reference/DOCUMENT_SCHEMAS.md +40 -0
- docling_pipelines-1.0.0/docs/reference/GLOBAL_CONFIG.md +789 -0
- docling_pipelines-1.0.0/docs/reference/OPERATORS.md +2911 -0
- docling_pipelines-1.0.0/examples/LDAP/.env.example +9 -0
- docling_pipelines-1.0.0/examples/LDAP/README.md +308 -0
- docling_pipelines-1.0.0/examples/LDAP/abc_users.ldif +36 -0
- docling_pipelines-1.0.0/examples/LDAP/docker-compose.yml +33 -0
- docling_pipelines-1.0.0/examples/README.md +461 -0
- docling_pipelines-1.0.0/examples/chunker_docling_serve_example.json +62 -0
- docling_pipelines-1.0.0/examples/chunker_example.py +278 -0
- docling_pipelines-1.0.0/examples/connectors/.env.example +51 -0
- docling_pipelines-1.0.0/examples/connectors/README.md +75 -0
- docling_pipelines-1.0.0/examples/connectors/test_google_drive_adapter.py +174 -0
- docling_pipelines-1.0.0/examples/connectors/test_onedrive_adapter.py +158 -0
- docling_pipelines-1.0.0/examples/connectors/test_s3_adapter.py +173 -0
- docling_pipelines-1.0.0/examples/connectors/test_sharepoint_adapter.py +158 -0
- docling_pipelines-1.0.0/examples/connectors/test_web_adapter.py +224 -0
- docling_pipelines-1.0.0/examples/custom_operators/README.md +259 -0
- docling_pipelines-1.0.0/examples/custom_operators/example_custom_operator.py +100 -0
- docling_pipelines-1.0.0/examples/custom_operators/hello_operator.py +51 -0
- docling_pipelines-1.0.0/examples/custom_operators/package_example/README.md +104 -0
- docling_pipelines-1.0.0/examples/custom_operators/package_example/my_custom_operators/__init__.py +6 -0
- docling_pipelines-1.0.0/examples/custom_operators/package_example/my_custom_operators/operators/reverse_operator.py +79 -0
- docling_pipelines-1.0.0/examples/custom_operators/package_example/my_custom_operators/operators/uppercase_operator.py +79 -0
- docling_pipelines-1.0.0/examples/custom_operators/package_example/pyproject.toml +20 -0
- docling_pipelines-1.0.0/examples/deduplication_example.py +61 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/00_complete_example.py +224 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/01_execute_from_file.py +113 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/02_execute_from_dict.py +96 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/03_list_operators.py +51 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/04_custom_configuration.py +108 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/05_notebook_usage.py +105 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/06_basic_test.py +188 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/06_basic_test_flow.json +72 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/README.md +314 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/sample_flow.json +70 -0
- docling_pipelines-1.0.0/examples/docpipe_flow_manager/sample_jupyter_notebook.ipynb +163 -0
- docling_pipelines-1.0.0/examples/document_set_example.py +189 -0
- docling_pipelines-1.0.0/examples/duckdb_job_stats/README.md +205 -0
- docling_pipelines-1.0.0/examples/duckdb_job_stats/duckdb_job_store_example.py +302 -0
- docling_pipelines-1.0.0/examples/embeddings_litellm_example.py +305 -0
- docling_pipelines-1.0.0/examples/embeddings_pipeline_example.py +426 -0
- docling_pipelines-1.0.0/examples/entity_curation_example.py +245 -0
- docling_pipelines-1.0.0/examples/external_operator_integration/README.md +446 -0
- docling_pipelines-1.0.0/examples/extract_operator_example.py +1013 -0
- docling_pipelines-1.0.0/examples/ingest_filesystem_example.py +52 -0
- docling_pipelines-1.0.0/examples/ingest_local_folder_example.py +56 -0
- docling_pipelines-1.0.0/examples/ingest_source_example.py +96 -0
- docling_pipelines-1.0.0/examples/language_detection_example.py +110 -0
- docling_pipelines-1.0.0/examples/milvus_integration_example.py +518 -0
- docling_pipelines-1.0.0/examples/ml_enrichment_example.py +122 -0
- docling_pipelines-1.0.0/examples/noop_operator_example.py +42 -0
- docling_pipelines-1.0.0/examples/notebooks/.jupyterignore +1 -0
- docling_pipelines-1.0.0/examples/notebooks/01_quickstart.ipynb +449 -0
- docling_pipelines-1.0.0/examples/notebooks/02_operator_showcase.ipynb +807 -0
- docling_pipelines-1.0.0/examples/notebooks/03_document_extraction.ipynb +1121 -0
- docling_pipelines-1.0.0/examples/notebooks/04_embeddings_vectordb.ipynb +920 -0
- docling_pipelines-1.0.0/examples/notebooks/05_quality_operators.ipynb +710 -0
- docling_pipelines-1.0.0/examples/notebooks/06_rag_pipeline.ipynb +1053 -0
- docling_pipelines-1.0.0/examples/notebooks/07_flow_authoring.ipynb +1142 -0
- docling_pipelines-1.0.0/examples/notebooks/README.md +277 -0
- docling_pipelines-1.0.0/examples/opensearch_example_README.md +299 -0
- docling_pipelines-1.0.0/examples/opensearch_integration_example.py +371 -0
- docling_pipelines-1.0.0/examples/operator_metadata_example.py +27 -0
- docling_pipelines-1.0.0/examples/pii_hap_detection_example.py +33 -0
- docling_pipelines-1.0.0/examples/readability_example.py +54 -0
- docling_pipelines-1.0.0/examples/redaction_example.py +60 -0
- docling_pipelines-1.0.0/examples/test_ingest_extract_flow.json +42 -0
- docling_pipelines-1.0.0/pyproject.toml +364 -0
- docling_pipelines-1.0.0/pytest.ini +29 -0
- docling_pipelines-1.0.0/requirements.txt +1166 -0
- docling_pipelines-1.0.0/sample_documents/1kb_file.txt +1 -0
- docling_pipelines-1.0.0/sample_documents/hello.txt +6 -0
- docling_pipelines-1.0.0/sample_flows/README.md +476 -0
- docling_pipelines-1.0.0/sample_flows/advanced/branching_dual_embeddings.json +213 -0
- docling_pipelines-1.0.0/sample_flows/advanced/branching_dual_embeddings_with_ingest_report.json +195 -0
- docling_pipelines-1.0.0/sample_flows/advanced/branching_quality_routing.json +308 -0
- docling_pipelines-1.0.0/sample_flows/advanced/hybrid_chunking.json +90 -0
- docling_pipelines-1.0.0/sample_flows/advanced/multi_stage_enrichment.json +183 -0
- docling_pipelines-1.0.0/sample_flows/advanced/quality_branching_merge_pipeline.json +293 -0
- docling_pipelines-1.0.0/sample_flows/advanced/summarization_pipeline.json +62 -0
- docling_pipelines-1.0.0/sample_flows/custom_operators/custom_operators_demo.json +60 -0
- docling_pipelines-1.0.0/sample_flows/custom_operators/hello_operator.json +30 -0
- docling_pipelines-1.0.0/sample_flows/operators/classification_ollama.json +71 -0
- docling_pipelines-1.0.0/sample_flows/operators/entity_curation_ollama.json +86 -0
- docling_pipelines-1.0.0/sample_flows/operators/entity_extraction_litellm.json +73 -0
- docling_pipelines-1.0.0/sample_flows/operators/entity_extraction_watsonx.json +102 -0
- docling_pipelines-1.0.0/sample_flows/operators/pii_hap_detection.json +72 -0
- docling_pipelines-1.0.0/sample_flows/quickstart/basic_ingest_extract.json +40 -0
- docling_pipelines-1.0.0/sample_flows/quickstart/complete_pipeline_ollama.json +129 -0
- docling_pipelines-1.0.0/sample_flows/quickstart/complete_pipeline_watsonx.json +102 -0
- docling_pipelines-1.0.0/sample_flows/storage_output/comprehensive_export_box.json +62 -0
- docling_pipelines-1.0.0/sample_flows/storage_output/comprehensive_export_google_drive.json +70 -0
- docling_pipelines-1.0.0/sample_flows/storage_output/comprehensive_export_sharepoint.json +72 -0
- docling_pipelines-1.0.0/sample_flows/storage_output/processed_content_s3.json +69 -0
- docling_pipelines-1.0.0/sample_flows/storage_output/refetch_original_filesystem.json +45 -0
- docling_pipelines-1.0.0/sample_flows/use_cases/audio_video_extraction.json +85 -0
- docling_pipelines-1.0.0/sample_flows/use_cases/document_set_management.json +58 -0
- docling_pipelines-1.0.0/sample_flows/use_cases/invoice_processing.json +154 -0
- docling_pipelines-1.0.0/sample_flows/use_cases/s3_to_opensearch.json +117 -0
- docling_pipelines-1.0.0/sample_flows/vectordb/milvus_integration.json +178 -0
- docling_pipelines-1.0.0/sample_flows/vectordb/opensearch_integration.json +146 -0
- docling_pipelines-1.0.0/scripts/check_modified_files.sh +104 -0
- docling_pipelines-1.0.0/scripts/check_pr_coverage.sh +103 -0
- docling_pipelines-1.0.0/scripts/convert_dag_to_authoring.py +304 -0
- docling_pipelines-1.0.0/scripts/deploy-openshift.sh +458 -0
- docling_pipelines-1.0.0/scripts/migrate_imports.py +263 -0
- docling_pipelines-1.0.0/scripts/setup_docling_pipelines_environment.sh +947 -0
- docling_pipelines-1.0.0/scripts/test_examples.py +505 -0
- docling_pipelines-1.0.0/slim/pyproject.toml +351 -0
- docling_pipelines-1.0.0/src/docpipe/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/api/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/api/api_router.py +33 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/dependencies.py +167 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/jwt_handler.py +91 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/ldap_auth.py +223 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/models.py +25 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/oauth2_config.py +100 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/oauth2_provider.py +363 -0
- docling_pipelines-1.0.0/src/docpipe/api/auth/oauth2_routes.py +267 -0
- docling_pipelines-1.0.0/src/docpipe/api/dependencies.py +237 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/authoring_flow_dto.py +448 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/document_class_dto.py +31 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/document_dto.py +121 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/document_library_dto.py +474 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/document_library_mapper.py +195 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/document_set_dto.py +540 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/error_dto.py +297 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/field_definitions.py +692 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/flow_dto.py +1063 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/job_run_dto.py +599 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/job_stats_dto.py +274 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/mappers/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/mappers/flow_mapper.py +228 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/mappers/job_stats_mapper.py +102 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/mappers/node_stats_mapper.py +173 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/mappers/project_flow_mapper.py +68 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/mappers/project_mapper.py +87 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/models_dto.py +169 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/node_stats_dto.py +211 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/operator_dto.py +313 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/project_dto.py +257 -0
- docling_pipelines-1.0.0/src/docpipe/api/dto/project_flow_dto.py +246 -0
- docling_pipelines-1.0.0/src/docpipe/api/main.py +278 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/__init__.py +10 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/api_logging_middleware.py +39 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/error_handler.py +319 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/payload_validation.py +123 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/rate_limit.py +41 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/security_headers.py +53 -0
- docling_pipelines-1.0.0/src/docpipe/api/middleware/transaction_middleware.py +156 -0
- docling_pipelines-1.0.0/src/docpipe/api/openapi.py +137 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/document_classes.py +117 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/document_libraries.py +833 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/document_set_utils.py +97 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/document_sets.py +396 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/documents.py +318 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/flows.py +1155 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/job_runs.py +676 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/operators.py +229 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/projects.py +471 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/providers.py +213 -0
- docling_pipelines-1.0.0/src/docpipe/api/routes/validation.py +290 -0
- docling_pipelines-1.0.0/src/docpipe/api/services/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/api/services/acl_query_builder.py +199 -0
- docling_pipelines-1.0.0/src/docpipe/api/services/opensearch_service.py +143 -0
- docling_pipelines-1.0.0/src/docpipe/cli/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/cli/docpipe_cli.py +513 -0
- docling_pipelines-1.0.0/src/docpipe/core/README.md +37 -0
- docling_pipelines-1.0.0/src/docpipe/core/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/__init__.py +21 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/huggingface/__init__.py +7 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/huggingface/huggingface_adapter.py +105 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/litellm/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/litellm/litellm_adapter.py +348 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/llm_adapter_factory.py +295 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/watsonx/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/core/adapters/watsonx/watsonx_adapter.py +598 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/adapters/__init__.py +7 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/adapters/repositories/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/adapters/repositories/duckdb_asset_repository.py +524 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/adapters/repositories/duckdb_attachment_repository.py +197 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/adapters/repositories/local_asset_repository.py +330 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/application/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/application/services/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/application/services/asset_service.py +181 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/models/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/models/asset.py +171 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/models/attachment_ref.py +65 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/ports/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/ports/asset_repository.py +210 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/domain/ports/attachment_repository.py +64 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/factories/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/factories/attachment_repository_factory.py +143 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/common/factories/repository_factory.py +267 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/__init__.py +10 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/duckdb/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/duckdb/metadata_repository.py +767 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/repositories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/repositories/duckdb_document_library_metadata_repository.py +716 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/storage/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/adapters/storage/duckdb_storage.py +221 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/application/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/application/services/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/application/services/document_library_service.py +595 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/domain/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/domain/models/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/domain/models/document_library.py +470 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/domain/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/domain/ports/document_library_repository.py +295 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/domain/types.py +39 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/factories/__init__.py +3 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_libraries/factories/document_library_repository_factory.py +182 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/adapters/__init__.py +7 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/adapters/duckdb/__init__.py +7 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/adapters/duckdb/data_store.py +294 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/adapters/duckdb/duckdb_utils.py +23 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/adapters/duckdb/metadata_repository.py +515 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/adapters/filesystem/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/application/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/application/services/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/application/services/document_set_service.py +435 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/models/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/models/data_card.py +67 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/models/document_set.py +231 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/models/storage_reference.py +57 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/ports/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/ports/data_store.py +129 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/ports/metadata_repository.py +220 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/domain/types.py +61 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/factories/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/factories/data_store_factory.py +187 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/document_sets/factories/metadata_repository_factory.py +181 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/__init__.py +3 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/config/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/config/repository_factory.py +127 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/repositories/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/repositories/flow_filesystem_utils.py +166 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/repositories/local/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/repositories/local/file_lock_manager.py +164 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/adapters/repositories/local/local_flow_repository.py +844 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/application/__init__.py +3 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/application/services/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/application/services/authoring_compiler.py +284 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/application/services/flow_enrichment_service.py +438 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/application/services/flow_service.py +801 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/application/services/validation_service.py +428 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/domain/__init__.py +3 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/domain/models/__init__.py +15 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/domain/models/authoring_flow.py +401 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/domain/models/flow.py +242 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/domain/ports/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/assets/flows/domain/ports/flow_repository.py +153 -0
- docling_pipelines-1.0.0/src/docpipe/core/constants/__init__.py +63 -0
- docling_pipelines-1.0.0/src/docpipe/core/constants/asset_constants.py +22 -0
- docling_pipelines-1.0.0/src/docpipe/core/constants/constants.py +737 -0
- docling_pipelines-1.0.0/src/docpipe/core/constants/operator_constants.py +1161 -0
- docling_pipelines-1.0.0/src/docpipe/core/data_access/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/data_access/data_access_utils.py +87 -0
- docling_pipelines-1.0.0/src/docpipe/core/data_access/docpipe_data_access_local.py +83 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/__init__.py +3 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/acord_insurance_form.json +619 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/bank_statements.json +279 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/bill_of_lading.json +265 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/business_licenses_permits.json +193 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/claimant_s_statement.json +483 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/client_success_case_study.json +309 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/credit_card_statements.json +363 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/customer_data_table.json +247 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/customerinfo.json +121 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/customs_form.json +249 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/delivery_receipt.json +201 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/diploma_certification.json +183 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/driver_license.json +365 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/expense_reports.json +287 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/federal_law_cs.json +54 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/financial_statement.json +381 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/i_9_form.json +257 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/initial_claimant_s_statement_for_income_replacement_waiver_of_premium_benefits.json +899 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/insurance_claim.json +285 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/invoice.json +886 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/life_insurance_authorization_form.json +175 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/management_quarterly_cs.json +109 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/mortgage_lending_document.json +259 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/national_id_card.json +221 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/order_request_form.json +376 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/passport.json +293 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/patient_intake_form.json +265 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/purchase_order.json +802 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/receipt.json +249 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/remittance_payment_advice.json +275 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/sales_agreements.json +58 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/schooladmissonform.json +173 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/tax_forms_w_9_1099_941_1120.json +229 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/transcripts.json +238 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/utility_bill.json +451 -0
- docling_pipelines-1.0.0/src/docpipe/core/document_classes/w_4_form.json +193 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/__init__.py +35 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/__init__.py +27 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/config/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/config/incremental_metadata_factory.py +316 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/stores/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/stores/filesystem/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/stores/filesystem/filesystem_incremental_store.py +393 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/stores/postgres/__init__.py +11 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/stores/postgres/models.py +29 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/adapters/stores/postgres/postgres_incremental_store.py +277 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/application/__init__.py +12 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/application/services/__init__.py +12 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/application/services/incremental_update_service.py +438 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/domain/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/domain/models/__init__.py +11 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/domain/models/incremental_record.py +54 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/domain/ports/__init__.py +11 -0
- docling_pipelines-1.0.0/src/docpipe/core/incremental_metadata/domain/ports/incremental_metadata_store.py +125 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/__init__.py +64 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/__init__.py +35 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/config/__init__.py +32 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/config/flow_definition_snapshot_storage_factory.py +277 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/config/job_management_factory.py +547 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/config/report_storage_factory.py +204 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/frameworks/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/frameworks/default_job_run_manager.py +175 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/services/__init__.py +17 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/services/job_tracker_service.py +1686 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/storage/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/__init__.py +26 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/duckdb/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/duckdb/duckdb_job_stats_store.py +809 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/inmemory/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/inmemory/inmemory_job_stats_store.py +461 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/json/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/json/json_job_stats_store.py +715 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/__init__.py +10 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/dal/__init__.py +11 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/dal/base_dao.py +278 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/dal/job_stats_dal.py +169 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/dal/node_stats_dal.py +240 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/database.py +212 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/mappers.py +139 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/migrations/alembic.ini +148 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/migrations/env.py +157 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/migrations/script.py.mako +30 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/migrations/versions/20260422_001_initial_schema.py +191 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/models.py +338 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/adapters/stores/postgres/postgres_job_stats_store.py +416 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/aggregation/__init__.py +16 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/aggregation/aggregator.py +261 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/aggregation/batch_aggregator.py +832 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/aggregation/strategies.py +104 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/services/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/services/job_management_service.py +514 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/services/node_stats_aggregator.py +137 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/services/report_generator.py +1104 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/application/services/report_utils.py +94 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/__init__.py +18 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/models/__init__.py +18 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/models/job_stats.py +69 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/models/node_stats.py +79 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/ports/__init__.py +15 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/ports/job_run_manager.py +121 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/ports/job_stats_service.py +645 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/ports/job_stats_store.py +222 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/utils/__init__.py +10 -0
- docling_pipelines-1.0.0/src/docpipe/core/job_management/domain/utils/normalization.py +48 -0
- docling_pipelines-1.0.0/src/docpipe/core/models/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/models/session_info.py +85 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/abstract_operator.py +304 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/__init__.py +31 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/acl_operator.py +647 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/adapters/__init__.py +15 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/adapters/outbound/__init__.py +17 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/adapters/outbound/factories/__init__.py +14 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/adapters/outbound/factories/acl_adapter_factory.py +165 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/adapters/outbound/sharepoint_adapter.py +881 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/domain/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/domain/models.py +145 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/ports/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/ports/outbound/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/acl/ports/outbound/acl_extraction.py +184 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/application/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/application/services/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/application/services/operator_metadata_service.py +152 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/application/services/provider_models_service.py +309 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/document_sets/__init__.py +12 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/document_sets/document_set_operator.py +342 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/__init__.py +27 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/__init__.py +12 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/docling_entity_adapter.py +498 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/docling_entity_config.py +26 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/litellm_entity_adapter.py +38 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/llm_entity_adapter.py +518 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/llm_entity_config.py +68 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/entity_extraction/watsonx_entity_adapter.py +40 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/factories/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/factories/entity_extraction_adapter_factory.py +229 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/factories/text_extraction_adapter_factory.py +687 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/text_extraction/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/text_extraction/docling_adapter.py +510 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/text_extraction/docling_library_config.py +115 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/text_extraction/docling_serve_adapter.py +369 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/text_extraction/docling_serve_config.py +72 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/adapters/outbound/text_extraction/ocr_config.py +56 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/domain/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/domain/models.py +173 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/extract_operator.py +1754 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/ports/outbound/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/ports/outbound/entity_extraction.py +203 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/ports/outbound/text_extraction.py +560 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/services/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/extract/services/entity_extraction_service.py +654 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/__init__.py +17 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/branching_operator.py +416 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/chunker.py +1438 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/chunker_validator.py +351 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/doc_id_hash.py +139 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/__init__.py +37 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/__init__.py +18 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/factories/llm_adapter_factory.py +100 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/huggingface_adapter.py +92 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/litellm_adapter.py +175 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/ollama_adapter.py +191 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/adapters/outbound/watsonx_adapter.py +186 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/domain/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/domain/models.py +54 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/embeddings_operator.py +1034 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/ports/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/embeddings/ports/outbound/llm_service.py +98 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/entity_curation_operator.py +264 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/schema_processor.py +420 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/transforms/__init__.py +29 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/transforms/currency_to_numeric.py +168 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/transforms/make_date_uniform.py +100 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/transforms/to_number.py +178 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/entity_curation/transforms/weight_to_numeric.py +298 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/merge.py +497 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/noop.py +94 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/functional/summarization_service.py +388 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/__init__.py +24 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/box/README.md +200 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/box/__init__.py +10 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/box/adapter.py +347 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/box/auth.py +52 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/box/config.py +90 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/factories/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/factories/source_factory.py +154 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/filesystem/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/filesystem/adapter.py +345 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/filesystem/config.py +82 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/filesystem/example_usage.py +138 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/google_drive/README.md +446 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/google_drive/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/google_drive/adapter.py +670 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/google_drive/config.py +140 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/onedrive/README.md +260 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/onedrive/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/onedrive/adapter.py +470 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/onedrive/config.py +111 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/s3/README.md +488 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/s3/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/s3/adapter.py +603 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/s3/config.py +155 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/sharepoint/README.md +296 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/sharepoint/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/sharepoint/adapter.py +477 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/sharepoint/config.py +111 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/web/README.md +213 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/web/__init__.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/web/adapter.py +283 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/adapters/outbound/sources/web/config.py +101 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/domain/models.py +137 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/ingest_source.py +1041 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/ingest_utils.py +353 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/ports/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/ingest/ports/outbound/document_source.py +153 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/models/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/models/model_info.py +37 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/operator_metadata.py +338 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/operator_registry.py +214 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/operator_utils.py +1652 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/__init__.py +2 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/classification/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/classification/classification_service.py +255 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/classification/document_classifier.py +854 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/classification/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/classification/domain/models.py +101 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/de +66 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/doc_quality.py +158 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/ededup.py +121 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/en +403 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/es +68 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/fr +91 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/ja +180 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/adapters/outbound/__init__.py +14 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/adapters/outbound/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/adapters/outbound/factories/language_adapter_factory.py +99 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/adapters/outbound/fasttext_adapter.py +115 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/adapters/outbound/langdetect_adapter.py +85 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/domain/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/domain/models.py +21 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/lang_id.py +239 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/ports/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/language_detection/ports/outbound/language_service.py +52 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/ml_enrichment.py +287 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/domain/models.py +67 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/local_pii_hap_detect.py +143 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/pii_and_hap_annotator.py +769 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/pii_and_hap_helper.py +430 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/pii_prompt_examples.json +136 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/services/__init__.py +8 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/pii_and_hap/services/pii_hap_service.py +252 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/readability/__init__.py +8 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/readability/easy_words.txt +2941 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/readability/readability_metrics.py +348 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/readability/readability_operator.py +232 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/redaction.py +192 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/quality/sql_filter.py +702 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/shared/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/shared/llm_provider_config.py +52 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/box/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/box/adapter.py +357 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/box/config.py +75 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/factories/destination_factory.py +67 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/filesystem/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/filesystem/adapter.py +124 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/filesystem/config.py +26 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/google_drive/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/google_drive/adapter.py +437 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/google_drive/config.py +141 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/s3/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/s3/adapter.py +298 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/s3/config.py +126 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/sharepoint/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/sharepoint/adapter.py +307 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/adapters/outbound/destinations/sharepoint/config.py +91 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/domain/models.py +37 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/ports/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/ports/outbound/destination_adapter.py +67 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/storage/storage_output_operator.py +764 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/__init__.py +33 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/__init__.py +25 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/factories/vector_store_factory.py +278 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/__init__.py +21 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/adapter.py +290 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/batch_processor.py +366 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/client.py +261 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/config.py +87 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/index_manager.py +606 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/milvus/resource_metadata.py +372 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/adapter.py +265 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/batch_processor.py +469 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/client.py +231 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/config.py +85 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/index_manager.py +1128 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/adapters/outbound/opensearch/resource_metadata.py +239 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/domain/__init__.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/domain/models.py +107 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/metadata_fetcher.py +261 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/ports/outbound/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/ports/outbound/vector_store.py +177 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/schemas/default_schema.v1.json +56 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/schemas/full_schema.v1.json +57 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/schemas/template_with_content_analyzer.v1.json +89 -0
- docling_pipelines-1.0.0/src/docpipe/core/operators/vectordb/vectordb_operator.py +805 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/abstract_flow_execution_event_handler.py +92 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/abstract_operator_executor.py +535 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/abstract_orchestrator.py +943 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/batch_manager.py +289 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/cmdline/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/cmdline/cmd_line_operator_executor.py +24 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/executor_pool.py +21 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/feature_propagation/__init__.py +23 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/feature_propagation/features_propagator.py +679 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/feature_propagation/models.py +274 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/flow_execution_event_handler.py +764 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/flow_executor.py +289 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/flow_validator.py +1477 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/futured_list.py +68 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/global_config_metadata.py +201 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_factory.py +369 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/adapters/__init__.py +10 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/adapters/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/adapters/factories/operator_source_factory.py +152 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/adapters/filesystem_adapter.py +240 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/adapters/package_adapter.py +322 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/adapters/s3_adapter.py +212 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/loader_service.py +289 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/ports/operator_source.py +109 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/operator_loader/validator.py +158 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/orchestrator_factory.py +111 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/ports/__init__.py +12 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/ports/flow_engine.py +119 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/__init__.py +19 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/adapters/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/adapters/factories/batch_execution_factory.py +231 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/adapters/thread_pool_adapter.py +80 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/adapters/work_pool_adapter.py +1112 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/batch_subflow.py +239 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/config/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/config/work_pool_config.py +99 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/domain/models.py +145 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/ports/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/ports/batch_execution_port.py +73 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/prefect/prefect_engine.py +790 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/python/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/python/python_operator_executor.py +220 -0
- docling_pipelines-1.0.0/src/docpipe/core/orchestration/python/python_orchestrator.py +77 -0
- docling_pipelines-1.0.0/src/docpipe/core/ports/llm_embedding_port.py +89 -0
- docling_pipelines-1.0.0/src/docpipe/core/ports/llm_inference_port.py +79 -0
- docling_pipelines-1.0.0/src/docpipe/core/ports/text_detection_port.py +74 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/adapters/repositories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/adapters/repositories/local/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/adapters/repositories/local/local_project_repository.py +114 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/application/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/application/services/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/application/services/project_service.py +435 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/domain/models/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/domain/models/flow_job_run_summary.py +40 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/domain/models/project.py +103 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/domain/ports/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/domain/ports/project_repository.py +99 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/factories/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/core/projects/factories/project_repository_factory.py +91 -0
- docling_pipelines-1.0.0/src/docpipe/exceptions/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/exceptions/docpipe_exceptions.py +911 -0
- docling_pipelines-1.0.0/src/docpipe/exceptions/error_codes.py +109 -0
- docling_pipelines-1.0.0/src/docpipe/exceptions/error_messages.py +152 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/aws/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/aws/s3_utils.py +95 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/base_llm_client.py +286 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/docling/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/docling/client.py +569 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/docling/vlm_pipeline_options_provider.py +633 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/huggingface/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/huggingface/client.py +380 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/litellm/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/litellm/client.py +415 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/ollama/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/ollama/client.py +917 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/ollama/embeddings.py +72 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/rest_client.py +467 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/secrets/__init__.py +32 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/secrets/hashicorp_vault.py +384 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/secrets/secret_provider.py +195 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/secrets/vault_initializer.py +122 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/watsonx/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/watsonx/client.py +327 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/watsonx/model_validator.py +134 -0
- docling_pipelines-1.0.0/src/docpipe/integrations/watsonx/rest_client.py +554 -0
- docling_pipelines-1.0.0/src/docpipe/lib/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/lib/docpipe_flow_manager.py +470 -0
- docling_pipelines-1.0.0/src/docpipe/storage/__init__.py +38 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/connection.py +6 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/duckdb_key_value_storage.py +287 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/duckdb_table_storage.py +488 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/key_value_storage.py +278 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/table_storage.py +499 -0
- docling_pipelines-1.0.0/src/docpipe/storage/duck_db/utils.py +51 -0
- docling_pipelines-1.0.0/src/docpipe/storage/exceptions.py +32 -0
- docling_pipelines-1.0.0/src/docpipe/storage/factory.py +84 -0
- docling_pipelines-1.0.0/src/docpipe/storage/file_system/__init__.py +9 -0
- docling_pipelines-1.0.0/src/docpipe/storage/file_system/abstract_file_system_storage.py +333 -0
- docling_pipelines-1.0.0/src/docpipe/storage/file_system/content_file_system_storage.py +132 -0
- docling_pipelines-1.0.0/src/docpipe/storage/file_system/key_value_file_system_storage.py +146 -0
- docling_pipelines-1.0.0/src/docpipe/storage/file_system/key_value_storage.py +213 -0
- docling_pipelines-1.0.0/src/docpipe/storage/file_system/utils.py +3 -0
- docling_pipelines-1.0.0/src/docpipe/storage/interfaces/__init__.py +11 -0
- docling_pipelines-1.0.0/src/docpipe/storage/interfaces/content_storage_port.py +116 -0
- docling_pipelines-1.0.0/src/docpipe/storage/interfaces/key_value_storage.py +113 -0
- docling_pipelines-1.0.0/src/docpipe/storage/interfaces/key_value_storage_port.py +113 -0
- docling_pipelines-1.0.0/src/docpipe/storage/interfaces/table_storage.py +130 -0
- docling_pipelines-1.0.0/src/docpipe/storage/interfaces/table_storage_port.py +130 -0
- docling_pipelines-1.0.0/src/docpipe/types.py +31 -0
- docling_pipelines-1.0.0/src/docpipe/utils/__init__.py +1 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/__init__.py +50 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/collections.py +128 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/datetime.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/docpipe_utils.py +25 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/memmap_file_utils.py +418 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/patterns.py +20 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/strings.py +114 -0
- docling_pipelines-1.0.0/src/docpipe/utils/core/validation.py +441 -0
- docling_pipelines-1.0.0/src/docpipe/utils/data/__init__.py +22 -0
- docling_pipelines-1.0.0/src/docpipe/utils/data/incremental_metadata_store.py +0 -0
- docling_pipelines-1.0.0/src/docpipe/utils/data/pyarrow_handler.py +173 -0
- docling_pipelines-1.0.0/src/docpipe/utils/data/schema_utils.py +93 -0
- docling_pipelines-1.0.0/src/docpipe/utils/data/transform.py +13 -0
- docling_pipelines-1.0.0/src/docpipe/utils/document_class_utils.py +561 -0
- docling_pipelines-1.0.0/src/docpipe/utils/duckdb/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/utils/duckdb/connection_manager.py +87 -0
- docling_pipelines-1.0.0/src/docpipe/utils/global_config/__init__.py +5 -0
- docling_pipelines-1.0.0/src/docpipe/utils/global_config/display.py +156 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/__init__.py +37 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/caching.py +85 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/concurrency.py +124 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/config.py +224 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/error_formatter.py +277 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/fasttext_model_manager.py +287 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/filesystem.py +54 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/flow_execution_reporter.py +658 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/iam_token_manager.py +229 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/logging.py +433 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/nltk_data_manager.py +204 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/performance.py +138 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/rate_limiter.py +329 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/retry.py +104 -0
- docling_pipelines-1.0.0/src/docpipe/utils/infrastructure/telemetry_service.py +610 -0
- docling_pipelines-1.0.0/src/docpipe/utils/llm/__init__.py +12 -0
- docling_pipelines-1.0.0/src/docpipe/utils/llm/json_parser.py +193 -0
- docling_pipelines-1.0.0/src/docpipe/utils/llm/prompt_manager.py +188 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/__init__.py +18 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/binary_content_fetcher.py +309 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/config_validation.py +57 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/display.py +222 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/logging.py +301 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/non_recoverable_utils.py +162 -0
- docling_pipelines-1.0.0/src/docpipe/utils/operators/vectordb_utils.py +174 -0
- docling_pipelines-1.0.0/src/docpipe/utils/orchestration/__init__.py +45 -0
- docling_pipelines-1.0.0/src/docpipe/utils/orchestration/dag_utils.py +208 -0
- docling_pipelines-1.0.0/src/docpipe/utils/orchestration/deleted_rows_tracker.py +145 -0
- docling_pipelines-1.0.0/src/docpipe/utils/orchestration/elyra_converter.py +1185 -0
- docling_pipelines-1.0.0/src/docpipe/utils/orchestration/flow_utils.py +114 -0
- docling_pipelines-1.0.0/src/docpipe/utils/orchestration/prefect_config.py +151 -0
- docling_pipelines-1.0.0/src/docpipe/utils/summarization_util.py +263 -0
- docling_pipelines-1.0.0/tests/README.md +226 -0
- docling_pipelines-1.0.0/tests/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/conftest.py +352 -0
- docling_pipelines-1.0.0/tests/fixtures/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/fixtures/custom_operators/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/fixtures/custom_operators/invalid_operator_no_short_name.py +24 -0
- docling_pipelines-1.0.0/tests/fixtures/custom_operators/invalid_operator_no_transform.py +19 -0
- docling_pipelines-1.0.0/tests/fixtures/custom_operators/not_an_operator.py +0 -0
- docling_pipelines-1.0.0/tests/fixtures/custom_operators/valid_operator.py +38 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/account_cancel.txt +18 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/cc_update.txt +14 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/feedback.txt +14 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/feedback_duplicate.txt +14 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/free_trial.txt +14 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/order_confirmation.txt +20 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/refund.txt +16 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/special_coupon.txt +20 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/special_occasion.txt +18 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/tech_support.txt +23 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/tech_support_duplicate.txt +23 -0
- docling_pipelines-1.0.0/tests/fixtures/customer_support_docs/travel_expense_tracker.pdf +0 -0
- docling_pipelines-1.0.0/tests/fixtures/invoices/TR-INV_001_3_2.1.pdf +0 -0
- docling_pipelines-1.0.0/tests/fixtures/invoices/TR-INV_001_3_2.1_expected.doclang +313 -0
- docling_pipelines-1.0.0/tests/fixtures/invoices/TR-INV_003_3_2.1.pdf +0 -0
- docling_pipelines-1.0.0/tests/fixtures/invoices/TR-INV_044_1_1.1.pdf +0 -0
- docling_pipelines-1.0.0/tests/fixtures/invoices/invoice.md +34 -0
- docling_pipelines-1.0.0/tests/fixtures/language_docs/English invoice.pdf +0 -0
- docling_pipelines-1.0.0/tests/fixtures/language_docs/Spanish invoice.pdf +0 -0
- docling_pipelines-1.0.0/tests/fixtures/pii_hap_docs/pii_sample.txt +17 -0
- docling_pipelines-1.0.0/tests/fixtures/purchase_orders/PurchaseOrder_PCH-2026-49507.pdf +93 -0
- docling_pipelines-1.0.0/tests/fixtures/purchase_orders/sample_purchase_order.txt +92 -0
- docling_pipelines-1.0.0/tests/fixtures/semantic_chunking_docs/usc04@119-18.pdf +0 -0
- docling_pipelines-1.0.0/tests/integration/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/README_ACCURACY_TESTING.md +1066 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/README_NL_TO_SQL_TESTS.md +372 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/document_query_evaluator.py +475 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/document_query_generator.py +486 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/nl_query_generator.py +1318 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/nl_to_sql.py +1195 -0
- docling_pipelines-1.0.0/tests/integration/accuracy_tests/sample_document_generator.py +1506 -0
- docling_pipelines-1.0.0/tests/integration/api/test_document_sets_api.py +515 -0
- docling_pipelines-1.0.0/tests/integration/api/test_documents_api.py +317 -0
- docling_pipelines-1.0.0/tests/integration/api/test_job_report_api.py +261 -0
- docling_pipelines-1.0.0/tests/integration/api/test_job_runs_api.py +190 -0
- docling_pipelines-1.0.0/tests/integration/app/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/integration/app/routes/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/integration/app/routes/conftest.py +155 -0
- docling_pipelines-1.0.0/tests/integration/app/routes/test_flows_integration.py +811 -0
- docling_pipelines-1.0.0/tests/integration/auth/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/integration/auth/test_jwt_integration.py +188 -0
- docling_pipelines-1.0.0/tests/integration/auth/test_oauth2_flow.py +269 -0
- docling_pipelines-1.0.0/tests/integration/opensearch/test_opensearch_advanced.py +501 -0
- docling_pipelines-1.0.0/tests/integration/opensearch/test_opensearch_connection.py +111 -0
- docling_pipelines-1.0.0/tests/integration/opensearch/test_opensearch_with_container.py +355 -0
- docling_pipelines-1.0.0/tests/integration/operators/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/integration/operators/entity_curation/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/integration/operators/entity_curation/test_entity_curation_integration.py +313 -0
- docling_pipelines-1.0.0/tests/integration/operators/extract/README.md +375 -0
- docling_pipelines-1.0.0/tests/integration/operators/extract/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/integration/operators/extract/adapters/outbound/entity_extraction/test_docling_entity_adapter.py +343 -0
- docling_pipelines-1.0.0/tests/integration/operators/extract/test_extract_docling_serve_integration.py +694 -0
- docling_pipelines-1.0.0/tests/integration/operators/extract/test_extract_operator_integration.py +501 -0
- docling_pipelines-1.0.0/tests/integration/operators/functional/embeddings/test_watsonx_integration.py +363 -0
- docling_pipelines-1.0.0/tests/integration/operators/storage/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/integration/operators/storage/test_storage_output_operator_integration.py +474 -0
- docling_pipelines-1.0.0/tests/integration/test_classifier_extract_content_reuse.py +244 -0
- docling_pipelines-1.0.0/tests/integration/test_embeddings_ollama_integration.py +283 -0
- docling_pipelines-1.0.0/tests/integration/test_ingest_extract_chunk_txt.py +242 -0
- docling_pipelines-1.0.0/tests/integration/test_ingest_extract_integration.py +182 -0
- docling_pipelines-1.0.0/tests/integration/test_operator_factory_integration.py +438 -0
- docling_pipelines-1.0.0/tests/integration/test_s3_ingest_extract_pipeline.py +415 -0
- docling_pipelines-1.0.0/tests/manual/test_transaction_logging.py +69 -0
- docling_pipelines-1.0.0/tests/templates/test_integration_template.py +79 -0
- docling_pipelines-1.0.0/tests/templates/test_operator_template.py +98 -0
- docling_pipelines-1.0.0/tests/test_docpipe_cli.py +403 -0
- docling_pipelines-1.0.0/tests/test_flow_service_unified.py +339 -0
- docling_pipelines-1.0.0/tests/test_local_asset_repository_integration.py +220 -0
- docling_pipelines-1.0.0/tests/test_opensearch_flow_config.py +245 -0
- docling_pipelines-1.0.0/tests/test_unified_architecture.py +105 -0
- docling_pipelines-1.0.0/tests/test_unified_flow_architecture.py +148 -0
- docling_pipelines-1.0.0/tests/unit/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/api/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/api/auth/test_oauth2_provider.py +398 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/mappers/test_job_stats_mapper.py +310 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/mappers/test_node_stats_mapper.py +353 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/test_error_dto.py +370 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/test_flow_dto.py +718 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/test_job_stats_mapper.py +259 -0
- docling_pipelines-1.0.0/tests/unit/api/dto/test_node_stats_mapper.py +490 -0
- docling_pipelines-1.0.0/tests/unit/api/middleware/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/api/middleware/test_exception_handlers.py +341 -0
- docling_pipelines-1.0.0/tests/unit/api/middleware/test_payload_validation.py +195 -0
- docling_pipelines-1.0.0/tests/unit/api/middleware/test_rate_limit.py +65 -0
- docling_pipelines-1.0.0/tests/unit/api/middleware/test_transaction_middleware.py +255 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/conftest.py +167 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_document_classes_routes.py +235 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_document_set_utils.py +200 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_document_sets_routes.py +246 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_flows_routes.py +1123 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_job_runs_routes.py +399 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_operators_routes.py +613 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_project_flows_route.py +171 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_projects_routes.py +652 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_providers_routes.py +225 -0
- docling_pipelines-1.0.0/tests/unit/api/routes/test_validation_routes.py +723 -0
- docling_pipelines-1.0.0/tests/unit/api/services/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/api/services/test_acl_query_builder.py +218 -0
- docling_pipelines-1.0.0/tests/unit/api/test_app_startup.py +129 -0
- docling_pipelines-1.0.0/tests/unit/api/test_authentication.py +310 -0
- docling_pipelines-1.0.0/tests/unit/api/test_dependencies.py +314 -0
- docling_pipelines-1.0.0/tests/unit/api/test_document_library_validation.py +351 -0
- docling_pipelines-1.0.0/tests/unit/api/test_flow_operations.py +149 -0
- docling_pipelines-1.0.0/tests/unit/api/test_openapi.py +63 -0
- docling_pipelines-1.0.0/tests/unit/api/test_vault_wiring.py +15 -0
- docling_pipelines-1.0.0/tests/unit/auth/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/auth/test_jwt_handler.py +223 -0
- docling_pipelines-1.0.0/tests/unit/auth/test_models.py +151 -0
- docling_pipelines-1.0.0/tests/unit/auth/test_oauth2_config.py +185 -0
- docling_pipelines-1.0.0/tests/unit/auth/test_oauth2_provider.py +347 -0
- docling_pipelines-1.0.0/tests/unit/auth/test_oauth2_routes.py +163 -0
- docling_pipelines-1.0.0/tests/unit/cli/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/unit/cli/test_docpipe_cli.py +549 -0
- docling_pipelines-1.0.0/tests/unit/common/clients/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/unit/common/clients/test_docling_serve_client.py +421 -0
- docling_pipelines-1.0.0/tests/unit/common/clients/test_rest_client.py +708 -0
- docling_pipelines-1.0.0/tests/unit/common/clients/test_telemetry_service.py +517 -0
- docling_pipelines-1.0.0/tests/unit/common/clients/test_vlm_pipeline_options_provider.py +313 -0
- docling_pipelines-1.0.0/tests/unit/common/test_llm_clients.py +337 -0
- docling_pipelines-1.0.0/tests/unit/common/util/core/test_validation.py +417 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_common_utils.py +446 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_config_utils.py +408 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_document_class_utils.py +222 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_error_formatter.py +459 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_filesystem_utils.py +353 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_flow_execution_reporter.py +501 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_global_config_display.py +161 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_logging_utils.py +544 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_nltk_data_manager.py +337 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_operator_display_utils.py +270 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_operator_log_details.py +457 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_operator_metadata.py +359 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_orchestrator_utils.py +462 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_performance_utils.py +328 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_pyarrow_handler.py +309 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_retry_utils.py +133 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_schema_utils.py +336 -0
- docling_pipelines-1.0.0/tests/unit/common/util/test_transform_utils.py +257 -0
- docling_pipelines-1.0.0/tests/unit/core/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/adapters/litellm/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/adapters/litellm/test_litellm_adapter.py +320 -0
- docling_pipelines-1.0.0/tests/unit/core/adapters/watsonx/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/adapters/watsonx/test_watsonx_adapter.py +487 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/common/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/common/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/common/adapters/repositories/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/common/adapters/repositories/test_duckdb_attachment_repository.py +330 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/common/factories/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/common/factories/test_attachment_repository_factory.py +304 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/adapters/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/adapters/repositories/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/adapters/repositories/test_duckdb_document_library_metadata_repository.py +266 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/application/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/application/services/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/application/services/test_document_library_service.py +268 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/conftest.py +112 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/domain/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/domain/models/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_libraries/domain/models/test_document_library_validation.py +224 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/adapters/duckdb/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/adapters/duckdb/test_duckdb_document_set_storage.py +260 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/application/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/application/services/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets/document_sets/application/services/test_document_set_service.py +208 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/adapters/config/test_repository_factory.py +175 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/adapters/repositories/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/adapters/repositories/local/conftest.py +37 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/adapters/repositories/local/test_local_flow_repository.py +1272 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/application/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/application/services/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/application/services/test_authoring_compiler.py +337 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/application/services/test_flow_service.py +1119 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/conftest.py +136 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/domain/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/domain/models/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/domain/models/test_authoring_flow.py +235 -0
- docling_pipelines-1.0.0/tests/unit/core/assets_management/domain/models/test_flow_validation.py +470 -0
- docling_pipelines-1.0.0/tests/unit/core/data_access/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/data_access/test_data_access_utils.py +88 -0
- docling_pipelines-1.0.0/tests/unit/core/flows/application/services/test_flow_enrichment_service.py +853 -0
- docling_pipelines-1.0.0/tests/unit/core/flows/application/services/test_validation_service.py +668 -0
- docling_pipelines-1.0.0/tests/unit/core/incremental_metadata/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/incremental_metadata/test_factory.py +191 -0
- docling_pipelines-1.0.0/tests/unit/core/incremental_metadata/test_filesystem_store.py +293 -0
- docling_pipelines-1.0.0/tests/unit/core/incremental_metadata/test_incremental_record.py +112 -0
- docling_pipelines-1.0.0/tests/unit/core/incremental_metadata/test_incremental_update_service.py +269 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/config/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/config/test_job_management_factory.py +190 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/services/test_job_tracker_service.py +1990 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/services/test_job_tracker_write_logs.py +253 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/storage/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/postgres/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/postgres/dal/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/postgres/dal/test_base_dao.py +346 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/postgres/test_migrations.py +253 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/test_duckdb_job_stats_store.py +711 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/test_inmemory_job_stats_store.py +451 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/test_json_job_stats_store.py +436 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/adapters/stores/test_postgres_job_stats_store.py +517 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/application/aggregation/test_batch_aggregator.py +279 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/application/aggregation/test_metadata_aggregator.py +184 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/application/services/test_job_management_service.py +442 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/application/services/test_node_stats_aggregator.py +636 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/application/services/test_report_generator.py +620 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/application/services/test_report_utils.py +74 -0
- docling_pipelines-1.0.0/tests/unit/core/job_management/domain/utils/test_normalization.py +243 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/application/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/application/services/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/application/services/test_operator_metadata_service.py +217 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/application/services/test_provider_models_service.py +489 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/document_sets/test_document_set_models.py +321 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/document_sets/test_document_set_operator.py +1 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/document_sets/test_document_set_repository.py +325 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/document_sets/test_document_set_service.py +446 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/document_sets/test_duckdb_adapters.py +386 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/document_sets/test_factories.py +173 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/ingest/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/ingest/test_ingest_utils.py +169 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/ingest/test_ingest_utils_resolve.py +117 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/test_operator_metadata.py +933 -0
- docling_pipelines-1.0.0/tests/unit/core/operators/test_operator_registry_providers.py +567 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/operator_loader/adapters/test_filesystem_adapter.py +483 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/operator_loader/adapters/test_s3_adapter.py +243 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/operator_loader/test_loader_service.py +574 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/prefect/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/prefect/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/prefect/adapters/test_work_pool_adapter.py +1448 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/prefect/test_batch_subflow.py +545 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/prefect/test_prefect_engine_helpers.py +274 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/test_abstract_orchestrator_coverage.py +301 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/test_abstract_orchestrator_methods.py +695 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/test_execute_flow_empty_ingest.py +205 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/test_flow_execution_event_handler.py +388 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestration/test_flow_execution_event_handler_report.py +348 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/conftest.py +43 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/operator_loader/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/operator_loader/test_package_adapter.py +363 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/operator_loader/test_s3_adapter.py +132 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/operator_loader/test_validator.py +139 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/prefect/test_batch_info_compatibility.py +294 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_abstract_operator_executor_coverage.py +230 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_abstract_operator_executor_empty_docs.py +283 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_abstract_orchestrator.py +407 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_batch_context_executor_integration.py +448 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_batch_manager.py +966 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_cmd_line_operator_executor.py +114 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_cmdline_orchestrator.py +138 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_features_propagator.py +676 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_flow_execution_event_handler_partial_batch_failure.py +168 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_flow_executor.py +557 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_flow_validator.py +1322 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_global_config_metadata.py +111 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_incremental_store_injection.py +89 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_ingest_source_global_config.py +92 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_operator_factory.py +381 -0
- docling_pipelines-1.0.0/tests/unit/core/orchestrator/test_terminal_path_batch_propagation.py +253 -0
- docling_pipelines-1.0.0/tests/unit/core/ports/test_llm_embedding_port.py +107 -0
- docling_pipelines-1.0.0/tests/unit/core/ports/test_text_detection_port.py +96 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/adapters/repositories/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/adapters/repositories/test_local_project_repository.py +236 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/application/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/application/services/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/application/services/test_project_service.py +538 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/conftest.py +108 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/factories/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/factories/test_project_repository_factory.py +185 -0
- docling_pipelines-1.0.0/tests/unit/core/projects/test_project_service_flows.py +183 -0
- docling_pipelines-1.0.0/tests/unit/integrations/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/integrations/conftest.py +64 -0
- docling_pipelines-1.0.0/tests/unit/integrations/ollama/test_client.py +437 -0
- docling_pipelines-1.0.0/tests/unit/integrations/ollama/test_ollama_client.py +849 -0
- docling_pipelines-1.0.0/tests/unit/integrations/secrets/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/integrations/secrets/test_hashicorp_vault.py +454 -0
- docling_pipelines-1.0.0/tests/unit/integrations/secrets/test_secret_provider.py +187 -0
- docling_pipelines-1.0.0/tests/unit/integrations/secrets/test_vault_initializer.py +130 -0
- docling_pipelines-1.0.0/tests/unit/integrations/test_base_llm_client.py +252 -0
- docling_pipelines-1.0.0/tests/unit/integrations/watsonx/test_model_validator.py +348 -0
- docling_pipelines-1.0.0/tests/unit/integrations/watsonx/test_rest_client_core.py +369 -0
- docling_pipelines-1.0.0/tests/unit/integrations/watsonx/test_rest_client_token_limits.py +373 -0
- docling_pipelines-1.0.0/tests/unit/operators/acl/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/acl/conftest.py +297 -0
- docling_pipelines-1.0.0/tests/unit/operators/acl/test_acl_adapter_factory.py +335 -0
- docling_pipelines-1.0.0/tests/unit/operators/acl/test_acl_operator.py +505 -0
- docling_pipelines-1.0.0/tests/unit/operators/acl/test_domain_models.py +416 -0
- docling_pipelines-1.0.0/tests/unit/operators/acl/test_sharepoint_adapter.py +618 -0
- docling_pipelines-1.0.0/tests/unit/operators/branching/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/branching/test_branching_operator.py +904 -0
- docling_pipelines-1.0.0/tests/unit/operators/chunker/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/chunker/conftest.py +14 -0
- docling_pipelines-1.0.0/tests/unit/operators/chunker/test_chunker.py +2054 -0
- docling_pipelines-1.0.0/tests/unit/operators/classify/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/classify/test_classification_domain.py +206 -0
- docling_pipelines-1.0.0/tests/unit/operators/classify/test_document_classifier.py +1032 -0
- docling_pipelines-1.0.0/tests/unit/operators/classify/test_llm_classification_adapter.py +415 -0
- docling_pipelines-1.0.0/tests/unit/operators/doc_id/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/doc_id/test_doc_id_hash.py +527 -0
- docling_pipelines-1.0.0/tests/unit/operators/ededup/test_ededup.py +472 -0
- docling_pipelines-1.0.0/tests/unit/operators/embeddings/__init__.py +3 -0
- docling_pipelines-1.0.0/tests/unit/operators/embeddings/conftest.py +81 -0
- docling_pipelines-1.0.0/tests/unit/operators/embeddings/test_embeddings_operator.py +1274 -0
- docling_pipelines-1.0.0/tests/unit/operators/embeddings/test_litellm_adapter.py +140 -0
- docling_pipelines-1.0.0/tests/unit/operators/embeddings/test_ollama_dimension_detection.py +336 -0
- docling_pipelines-1.0.0/tests/unit/operators/entity_curation/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/unit/operators/entity_curation/test_entity_curation_operator.py +178 -0
- docling_pipelines-1.0.0/tests/unit/operators/entity_curation/test_schema_processor.py +271 -0
- docling_pipelines-1.0.0/tests/unit/operators/entity_curation/test_transforms.py +147 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/entity_extraction/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/entity_extraction/test_docling_entity_adapter.py +702 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/entity_extraction/test_entity_extraction_adapter_factory.py +193 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/entity_extraction/test_llm_entity_adapter.py +565 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/factories/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/text_extraction/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/text_extraction/test_docling_adapter_gpu.py +1259 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/adapters/outbound/text_extraction/test_docling_serve_adapter.py +1250 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/ports/test_text_extraction_port.py +446 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/test_extract_extension_validation.py +390 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/test_extract_operator.py +3116 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/test_extract_operator_doclang.py +252 -0
- docling_pipelines-1.0.0/tests/unit/operators/extract/test_ocr_config.py +359 -0
- docling_pipelines-1.0.0/tests/unit/operators/filter/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/filter/test_sql_filter.py +1208 -0
- docling_pipelines-1.0.0/tests/unit/operators/functional/embeddings/adapters/test_watsonx_adapter.py +461 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/README.md +140 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/conftest.py +30 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_box_source_adapter.py +703 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_extension_validation.py +127 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_filesystem_source_adapter.py +300 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_google_drive_config.py +72 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_google_drive_source_adapter.py +590 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_ingest_source.py +1895 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_ingest_source_filesystem.py +135 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_ingest_source_validation.py +168 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_microsoft_source_adapters.py +890 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_s3_adapter.py +449 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_s3_source_adapter.py +916 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_source_adapter_factory.py +80 -0
- docling_pipelines-1.0.0/tests/unit/operators/ingest/test_web_source_adapter.py +586 -0
- docling_pipelines-1.0.0/tests/unit/operators/language/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/unit/operators/language/conftest.py +53 -0
- docling_pipelines-1.0.0/tests/unit/operators/language/test_lang_id_fasttext.py +483 -0
- docling_pipelines-1.0.0/tests/unit/operators/language/test_language_detection.py +488 -0
- docling_pipelines-1.0.0/tests/unit/operators/merge/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/merge/test_merge.py +728 -0
- docling_pipelines-1.0.0/tests/unit/operators/ml_enrichment/__init__.py +3 -0
- docling_pipelines-1.0.0/tests/unit/operators/ml_enrichment/test_ml_enrichment.py +674 -0
- docling_pipelines-1.0.0/tests/unit/operators/noop/test_noop.py +650 -0
- docling_pipelines-1.0.0/tests/unit/operators/pii_and_hap/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/pii_and_hap/test_domain_models.py +90 -0
- docling_pipelines-1.0.0/tests/unit/operators/pii_and_hap/test_pii_and_hap_annotator.py +781 -0
- docling_pipelines-1.0.0/tests/unit/operators/pii_and_hap/test_pii_and_hap_integration.py +231 -0
- docling_pipelines-1.0.0/tests/unit/operators/pii_and_hap/test_pii_hap_flow.py +67 -0
- docling_pipelines-1.0.0/tests/unit/operators/pii_and_hap/test_pii_hap_service.py +389 -0
- docling_pipelines-1.0.0/tests/unit/operators/quality/test_doc_quality.py +87 -0
- docling_pipelines-1.0.0/tests/unit/operators/quality/test_doc_quality_coverage.py +94 -0
- docling_pipelines-1.0.0/tests/unit/operators/quality/test_pii_hap_annotator.py +111 -0
- docling_pipelines-1.0.0/tests/unit/operators/readability/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/readability/test_readability.py +211 -0
- docling_pipelines-1.0.0/tests/unit/operators/redaction/__init__.py +2 -0
- docling_pipelines-1.0.0/tests/unit/operators/redaction/test_redaction.py +207 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/box/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/box/test_box_destination_adapter.py +530 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/box/test_box_destination_config.py +64 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/filesystem/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/filesystem/test_filesystem_destination_adapter.py +146 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/filesystem/test_filesystem_destination_config.py +36 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/google_drive/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/google_drive/test_google_drive_destination_adapter.py +410 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/google_drive/test_google_drive_destination_config.py +124 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/s3/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/s3/test_s3_destination_adapter.py +708 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/s3/test_s3_destination_config.py +123 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/sharepoint/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/sharepoint/test_sharepoint_destination_adapter.py +489 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/adapters/outbound/destinations/sharepoint/test_sharepoint_destination_config.py +98 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/test_domain_models.py +65 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/test_path_template.py +110 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage/test_storage_output_operator.py +679 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage_operator/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/operators/storage_operator/test_document_set_operator.py +481 -0
- docling_pipelines-1.0.0/tests/unit/operators/test_abstract_operator.py +858 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_mandatory_for_vector_db_propagation.py +131 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_metadata_aggregation.py +603 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_metadata_fetcher.py +1161 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_metadata_normalization.py +259 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_milvus_adapter.py +247 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_milvus_batch_processor.py +692 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_milvus_client.py +171 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_milvus_index_manager.py +934 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_multi_model_embeddings.py +359 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_opensearch_batch_processor.py +956 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_opensearch_client.py +450 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_opensearch_index_manager.py +995 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_schema_template_loading.py +775 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_schema_validation.py +383 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_vectordb_metrics.py +269 -0
- docling_pipelines-1.0.0/tests/unit/operators/vectordb/test_vectordb_operator.py +988 -0
- docling_pipelines-1.0.0/tests/unit/orchestration/adapters/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/orchestration/adapters/test_work_pool_adapter.py +1240 -0
- docling_pipelines-1.0.0/tests/unit/orchestration/test_fail_fast_downstream_skip.py +417 -0
- docling_pipelines-1.0.0/tests/unit/orchestration/test_prefect_domain_models.py +55 -0
- docling_pipelines-1.0.0/tests/unit/storage/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/storage/duck_db/test_utils.py +122 -0
- docling_pipelines-1.0.0/tests/unit/storage/test_duckdb_key_value_storage.py +251 -0
- docling_pipelines-1.0.0/tests/unit/storage/test_duckdb_table_storage.py +458 -0
- docling_pipelines-1.0.0/tests/unit/storage/test_file_system_storage.py +227 -0
- docling_pipelines-1.0.0/tests/unit/test_datasift_exceptions.py +350 -0
- docling_pipelines-1.0.0/tests/unit/test_docpipe_flow_manager.py +976 -0
- docling_pipelines-1.0.0/tests/unit/util/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/util/orchestration/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/util/orchestration/test_elyra_converter.py +594 -0
- docling_pipelines-1.0.0/tests/unit/util/test_lru_cache.py +85 -0
- docling_pipelines-1.0.0/tests/unit/util/test_memmap_file_utils.py +327 -0
- docling_pipelines-1.0.0/tests/unit/util/test_operator_utils.py +2205 -0
- docling_pipelines-1.0.0/tests/unit/utils/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/utils/infrastructure/test_concurrency.py +354 -0
- docling_pipelines-1.0.0/tests/unit/utils/infrastructure/test_rate_limiter.py +453 -0
- docling_pipelines-1.0.0/tests/unit/utils/infrastructure/test_telemetry_service.py +749 -0
- docling_pipelines-1.0.0/tests/unit/utils/llm/__init__.py +1 -0
- docling_pipelines-1.0.0/tests/unit/utils/llm/test_json_parser.py +236 -0
- docling_pipelines-1.0.0/tests/unit/utils/llm/test_json_parser_redos.py +66 -0
- docling_pipelines-1.0.0/tests/unit/utils/llm/test_prompt_manager.py +262 -0
- docling_pipelines-1.0.0/tests/unit/utils/operators/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/utils/operators/test_binary_content_fetcher.py +200 -0
- docling_pipelines-1.0.0/tests/unit/utils/orchestration/__init__.py +0 -0
- docling_pipelines-1.0.0/tests/unit/utils/orchestration/test_dag_utils.py +241 -0
- docling_pipelines-1.0.0/tests/unit/utils/test_document_class_utils.py +783 -0
- docling_pipelines-1.0.0/tests/unit/utils/test_fasttext_model_manager.py +75 -0
- docling_pipelines-1.0.0/tests/unit/utils/test_summarization_util.py +121 -0
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
[run]
|
|
2
|
+
branch = True
|
|
3
|
+
source = src/docpipe
|
|
4
|
+
omit =
|
|
5
|
+
*/tests/*
|
|
6
|
+
*/test_*.py
|
|
7
|
+
*/*_test.py
|
|
8
|
+
*/venv/*
|
|
9
|
+
*/virtualenv/*
|
|
10
|
+
*/.venv/*
|
|
11
|
+
*/site-packages/*
|
|
12
|
+
*/ui/*
|
|
13
|
+
*/examples/*
|
|
14
|
+
*/__init__.py
|
|
15
|
+
/data/*
|
|
16
|
+
/sonar/*
|
|
17
|
+
/docs/*
|
|
18
|
+
/dist/*
|
|
19
|
+
/build/*
|
|
20
|
+
/ui/*
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
[report]
|
|
24
|
+
exclude_lines =
|
|
25
|
+
pragma: no cover
|
|
26
|
+
if __name__ == "__main__":
|
|
27
|
+
def __repr__
|
|
28
|
+
raise AssertionError
|
|
29
|
+
raise NotImplementedError
|
|
30
|
+
if TYPE_CHECKING:
|
|
31
|
+
@abstractmethod
|
|
32
|
+
@abc.abstractmethod
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Git
|
|
2
|
+
.git/
|
|
3
|
+
.gitignore
|
|
4
|
+
|
|
5
|
+
# Python
|
|
6
|
+
__pycache__/
|
|
7
|
+
*.py[cod]
|
|
8
|
+
*$py.class
|
|
9
|
+
*.pyc
|
|
10
|
+
*.pyo
|
|
11
|
+
*.pyd
|
|
12
|
+
*.so
|
|
13
|
+
.Python
|
|
14
|
+
|
|
15
|
+
# Virtual environments
|
|
16
|
+
venv/
|
|
17
|
+
.venv/
|
|
18
|
+
env/
|
|
19
|
+
ENV/
|
|
20
|
+
env.bak/
|
|
21
|
+
venv.bak/
|
|
22
|
+
|
|
23
|
+
# uv
|
|
24
|
+
.uv/
|
|
25
|
+
|
|
26
|
+
# IDEs
|
|
27
|
+
.vscode/
|
|
28
|
+
.idea/
|
|
29
|
+
*.swp
|
|
30
|
+
*.swo
|
|
31
|
+
*~
|
|
32
|
+
|
|
33
|
+
# Testing
|
|
34
|
+
.pytest_cache/
|
|
35
|
+
.coverage
|
|
36
|
+
htmlcov/
|
|
37
|
+
*.cover
|
|
38
|
+
.mypy_cache
|
|
39
|
+
.ruff_cache
|
|
40
|
+
pytest.ini
|
|
41
|
+
|
|
42
|
+
# Documentation
|
|
43
|
+
*.md
|
|
44
|
+
docs/
|
|
45
|
+
|
|
46
|
+
# Examples and scripts
|
|
47
|
+
examples/
|
|
48
|
+
scripts/
|
|
49
|
+
scratch/
|
|
50
|
+
|
|
51
|
+
# Data and artifacts
|
|
52
|
+
data/
|
|
53
|
+
artifacts/
|
|
54
|
+
|
|
55
|
+
# Build artifacts
|
|
56
|
+
dist/
|
|
57
|
+
build/
|
|
58
|
+
*.egg-info/
|
|
59
|
+
|
|
60
|
+
# CI/CD
|
|
61
|
+
.github/
|
|
62
|
+
Jenkinsfile
|
|
63
|
+
sonar/
|
|
64
|
+
|
|
65
|
+
# Deployment examples
|
|
66
|
+
k8s-deployment-examples/
|
|
67
|
+
|
|
68
|
+
# Environment
|
|
69
|
+
.env
|
|
70
|
+
.env.local
|
|
71
|
+
|
|
72
|
+
# OS
|
|
73
|
+
.DS_Store
|
|
74
|
+
Thumbs.db
|
|
75
|
+
|
|
76
|
+
# Test directory
|
|
77
|
+
tests/
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
# OpenSearch Connection Configuration
|
|
2
|
+
# Copy this file to .env and update with your actual values
|
|
3
|
+
# ============================================================================
|
|
4
|
+
# PYTHON PATH CONFIGURATION
|
|
5
|
+
# ============================================================================
|
|
6
|
+
# Required for imports to work correctly
|
|
7
|
+
# This should be set relative to the project root
|
|
8
|
+
PYTHONPATH=./src
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
# ============================================================================
|
|
12
|
+
# LOCAL DOCKER SETUP (using docker-compose.opensearch.yml)
|
|
13
|
+
# ============================================================================
|
|
14
|
+
# To run OpenSearch locally with Docker:
|
|
15
|
+
# docker-compose -f docker-compose.opensearch.yml up -d
|
|
16
|
+
#
|
|
17
|
+
# Use these settings for local Docker instance:
|
|
18
|
+
# OPENSEARCH_HOST=localhost
|
|
19
|
+
# OPENSEARCH_PORT=9200
|
|
20
|
+
# OPENSEARCH_USE_SSL=false
|
|
21
|
+
# OPENSEARCH_VERIFY_CERTS=false
|
|
22
|
+
# OPENSEARCH_USERNAME=admin
|
|
23
|
+
# OPENSEARCH_PASSWORD=<your-opensearch-password>
|
|
24
|
+
#
|
|
25
|
+
# OpenSearch Dashboards will be available at: http://localhost:5601
|
|
26
|
+
# ============================================================================
|
|
27
|
+
|
|
28
|
+
# OpenSearch Server
|
|
29
|
+
OPENSEARCH_HOST=localhost
|
|
30
|
+
OPENSEARCH_PORT=9200
|
|
31
|
+
OPENSEARCH_USE_SSL=false
|
|
32
|
+
OPENSEARCH_VERIFY_CERTS=false
|
|
33
|
+
|
|
34
|
+
# Authentication
|
|
35
|
+
OPENSEARCH_USERNAME=admin
|
|
36
|
+
# OPENSEARCH_PASSWORD=<set-your-password>
|
|
37
|
+
|
|
38
|
+
# AWS Authentication (optional, set to true if using AWS OpenSearch)
|
|
39
|
+
OPENSEARCH_AWS_AUTH=false
|
|
40
|
+
# JWT Authentication (optional, for AWS OpenSearch with JWT tokens)
|
|
41
|
+
# OPENSEARCH_JWT_TOKEN=your-jwt-token-here
|
|
42
|
+
OPENSEARCH_AWS_REGION=us-east-1
|
|
43
|
+
|
|
44
|
+
# Engine Configuration
|
|
45
|
+
OPENSEARCH_ENGINE=faiss
|
|
46
|
+
OPENSEARCH_ALGORITHM=hnsw
|
|
47
|
+
OPENSEARCH_SPACE_TYPE=l2
|
|
48
|
+
OPENSEARCH_VECTOR_DIMENSION=384
|
|
49
|
+
|
|
50
|
+
# Performance Settings
|
|
51
|
+
OPENSEARCH_BATCH_SIZE=100
|
|
52
|
+
OPENSEARCH_CREATE_INDEX=true
|
|
53
|
+
|
|
54
|
+
# Index Configuration
|
|
55
|
+
OPENSEARCH_INDEX_NAME=docpipe_test
|
|
56
|
+
OPENSEARCH_DOC_ID_COLUMN=doc_id_hash
|
|
57
|
+
OPENSEARCH_EMBEDDINGS_COLUMN=embeddings
|
|
58
|
+
|
|
59
|
+
# Document Retrieval API Configuration
|
|
60
|
+
# Default index for document retrieval endpoints
|
|
61
|
+
OPENSEARCH_DEFAULT_INDEX=documents
|
|
62
|
+
# Connection timeout in seconds
|
|
63
|
+
OPENSEARCH_TIMEOUT=30
|
|
64
|
+
# Maximum number of retry attempts
|
|
65
|
+
OPENSEARCH_MAX_RETRIES=3
|
|
66
|
+
|
|
67
|
+
DOCPIPE_INDEX=docpipe_test
|
|
68
|
+
DOCPIPE_MODEL=granite4:latest
|
|
69
|
+
|
|
70
|
+
# Ollama Configuration
|
|
71
|
+
# Ollama server host URL used by OllamaClient
|
|
72
|
+
# Default: http://localhost:11434
|
|
73
|
+
OLLAMA_HOST=http://localhost:11434
|
|
74
|
+
|
|
75
|
+
# ============================================================================
|
|
76
|
+
# VLM ENGINE CREDENTIALS (for ExtractOperator)
|
|
77
|
+
# ============================================================================
|
|
78
|
+
# These credentials are used by extract_operator_example.py for VLM engines
|
|
79
|
+
# that require API access (watsonx, openai, generic)
|
|
80
|
+
|
|
81
|
+
# Watsonx AI (IBM Cloud)
|
|
82
|
+
# Required for --vlm-engine watsonx
|
|
83
|
+
# WATSONX_API_KEY=your_ibm_cloud_api_key_here
|
|
84
|
+
# WATSONX_CONTAINER_KIND=project # optional: project, space, or catalog (default: project)
|
|
85
|
+
# WATSONX_CONTAINER_ID=your_project_or_space_id_here
|
|
86
|
+
# WATSONX_MODEL=meta-llama/llama-3-2-90b-vision-instruct
|
|
87
|
+
# WATSONX_API_BASE_URL=https://us-south.ml.cloud.ibm.com # optional, has default
|
|
88
|
+
|
|
89
|
+
# OpenAI API
|
|
90
|
+
# Required for --vlm-engine openai
|
|
91
|
+
# OPENAI_API_KEY=your_openai_api_key_here
|
|
92
|
+
# OPENAI_MODEL=gpt-4-vision-preview
|
|
93
|
+
# OPENAI_API_BASE_URL=https://api.openai.com/v1 # optional, has default
|
|
94
|
+
|
|
95
|
+
# Generic API (any OpenAI-compatible API)
|
|
96
|
+
# Required for --vlm-engine generic
|
|
97
|
+
# GENERIC_API_BASE_URL=https://your-api.com/v1/chat
|
|
98
|
+
# GENERIC_API_KEY=your_api_key_here # optional, some APIs don't require auth
|
|
99
|
+
|
|
100
|
+
# ============================================================================
|
|
101
|
+
# API AUTHENTICATION
|
|
102
|
+
# ============================================================================
|
|
103
|
+
# Secret key used to sign JWT access tokens.
|
|
104
|
+
# Must be at least 32 characters. Use a strong random value in production:
|
|
105
|
+
# python -c "import secrets; print(secrets.token_hex(32))"
|
|
106
|
+
JWT_SECRET_KEY=local-dev-secret-key-change-this-in-production!!
|
|
107
|
+
|
|
108
|
+
# JWT algorithm (default: HS256)
|
|
109
|
+
# JWT_ALGORITHM=HS256
|
|
110
|
+
|
|
111
|
+
# Access token expiry in minutes (default: 30)
|
|
112
|
+
# JWT_ACCESS_TOKEN_EXPIRE_MINUTES=30
|
|
113
|
+
|
|
114
|
+
# ============================================================================
|
|
115
|
+
# CORS CONFIGURATION
|
|
116
|
+
# ============================================================================
|
|
117
|
+
# Comma-separated list of allowed origins for CORS
|
|
118
|
+
# For development: http://localhost:3000 (frontend/UI server)
|
|
119
|
+
# For production: https://yourdomain.com,https://app.yourdomain.com
|
|
120
|
+
CORS_ORIGINS=http://localhost:3000
|
|
121
|
+
|
|
122
|
+
# ============================================================================
|
|
123
|
+
# FLOW REPOSITORY CONFIGURATION
|
|
124
|
+
# ============================================================================
|
|
125
|
+
# Type of repository to use for storing flow definitions
|
|
126
|
+
# Options: local (default)
|
|
127
|
+
# If not set, defaults to 'local' repository type
|
|
128
|
+
# FLOW_REPOSITORY_TYPE=local
|
|
129
|
+
|
|
130
|
+
# Local repository settings (when FLOW_REPOSITORY_TYPE=local or not set)
|
|
131
|
+
# Directory where flow JSON files are stored
|
|
132
|
+
# If not set, defaults to:
|
|
133
|
+
# macOS/Linux: ~/Documents/pipeline/assets
|
|
134
|
+
# Windows: %USERPROFILE%\Documents\pipeline\assets
|
|
135
|
+
# Use absolute paths for production deployments
|
|
136
|
+
|
|
137
|
+
# ============================================================================
|
|
138
|
+
# TELEMETRY CONFIGURATION (Optional - OpenTelemetry)
|
|
139
|
+
# ============================================================================
|
|
140
|
+
# Enable/disable OpenTelemetry distributed tracing
|
|
141
|
+
# Set to true to enable telemetry, false to disable (default: false)
|
|
142
|
+
# When disabled, there is zero performance overhead
|
|
143
|
+
DOCLING_PIPELINES_TELEMETRY_ENABLED=false
|
|
144
|
+
|
|
145
|
+
# Service name that appears in traces
|
|
146
|
+
# This identifies your Docling Pipelines instance in the monitoring backend
|
|
147
|
+
OTEL_SERVICE_NAME=docling-pipelines
|
|
148
|
+
|
|
149
|
+
# OTLP endpoint for exporting traces
|
|
150
|
+
# Default points to local Jaeger instance
|
|
151
|
+
# For production, point to your OTLP collector or monitoring backend
|
|
152
|
+
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
|
153
|
+
|
|
154
|
+
# Service version for trace metadata
|
|
155
|
+
OTEL_SERVICE_VERSION=0.1.0
|
|
156
|
+
|
|
157
|
+
# Deployment environment name (e.g., development, staging, production)
|
|
158
|
+
OTEL_DEPLOYMENT_ENVIRONMENT=development
|
|
159
|
+
|
|
160
|
+
|
|
161
|
+
# ============================================================================
|
|
162
|
+
# QUICK START WITH JAEGER (Local Development)
|
|
163
|
+
# ============================================================================
|
|
164
|
+
# To test telemetry locally with Jaeger:
|
|
165
|
+
#
|
|
166
|
+
# 1. Install telemetry dependencies:
|
|
167
|
+
# uv pip install -e ".[telemetry]"
|
|
168
|
+
#
|
|
169
|
+
# 2. Start Jaeger (using Docker):
|
|
170
|
+
# docker run -d --name jaeger \
|
|
171
|
+
# -p 16686:16686 \
|
|
172
|
+
# -p 4317:4317 \
|
|
173
|
+
# -p 4318:4318 \
|
|
174
|
+
# jaegertracing/all-in-one:latest
|
|
175
|
+
#
|
|
176
|
+
# 3. Enable telemetry:
|
|
177
|
+
# DOCLING_PIPELINES_TELEMETRY_ENABLED=true
|
|
178
|
+
# OTEL_SERVICE_NAME=docling-pipelines-dev
|
|
179
|
+
# OTEL_SERVICE_VERSION=0.1.0
|
|
180
|
+
# OTEL_DEPLOYMENT_ENVIRONMENT=Development
|
|
181
|
+
#
|
|
182
|
+
# 4. Run your flows normally - traces will appear in Jaeger UI
|
|
183
|
+
# http://localhost:16686
|
|
184
|
+
#
|
|
185
|
+
# See docs/TELEMETRY_SETUP.md for detailed setup instructions
|
|
186
|
+
# ============================================================================
|
|
187
|
+
# LOCAL_FLOWS_DIR=/path/to/custom/flows/directory
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
# OAuth2 and OIDC Configuration Example
|
|
2
|
+
# Copy this file to .env and configure your OAuth2 settings
|
|
3
|
+
|
|
4
|
+
# ============================================================================
|
|
5
|
+
# JWT Configuration (Required for all auth methods)
|
|
6
|
+
# ============================================================================
|
|
7
|
+
JWT_SECRET_KEY=your-secret-key-here-change-this-in-production
|
|
8
|
+
JWT_ALGORITHM=HS256
|
|
9
|
+
JWT_ACCESS_TOKEN_EXPIRE_MINUTES=30
|
|
10
|
+
|
|
11
|
+
# ============================================================================
|
|
12
|
+
# OAuth2 General Settings
|
|
13
|
+
# ============================================================================
|
|
14
|
+
OAUTH2_ENABLED=true
|
|
15
|
+
OAUTH2_PROVIDER=google # Options: google, azure, generic
|
|
16
|
+
OAUTH2_CLIENT_ID=your-client-id
|
|
17
|
+
OAUTH2_CLIENT_SECRET=your-client-secret
|
|
18
|
+
OAUTH2_REDIRECT_URI=http://localhost:8000/auth/oauth2/callback
|
|
19
|
+
OAUTH2_SCOPE=openid profile email
|
|
20
|
+
OAUTH2_TOKEN_VALIDATION=true
|
|
21
|
+
OAUTH2_STATE_SECRET=your-state-secret-here
|
|
22
|
+
OAUTH2_SESSION_EXPIRE_MINUTES=60
|
|
23
|
+
|
|
24
|
+
# ============================================================================
|
|
25
|
+
# Google OAuth2 Configuration
|
|
26
|
+
# ============================================================================
|
|
27
|
+
# When using OAUTH2_PROVIDER=google, these are auto-configured:
|
|
28
|
+
# OAUTH2_DISCOVERY_URL=https://accounts.google.com/.well-known/openid-configuration
|
|
29
|
+
# OAUTH2_AUTHORIZATION_ENDPOINT=https://accounts.google.com/o/oauth2/v2/auth
|
|
30
|
+
# OAUTH2_TOKEN_ENDPOINT=https://oauth2.googleapis.com/token
|
|
31
|
+
# OAUTH2_USERINFO_ENDPOINT=https://openidconnect.googleapis.com/v1/userinfo
|
|
32
|
+
# OAUTH2_JWKS_URI=https://www.googleapis.com/oauth2/v3/certs
|
|
33
|
+
# OIDC_ISSUER=https://accounts.google.com
|
|
34
|
+
# OIDC_AUDIENCE=your-client-id
|
|
35
|
+
|
|
36
|
+
# Google OAuth2 Setup:
|
|
37
|
+
# 1. Go to https://console.cloud.google.com/
|
|
38
|
+
# 2. Create a new project or select existing
|
|
39
|
+
# 3. Enable Google+ API
|
|
40
|
+
# 4. Go to Credentials > Create Credentials > OAuth 2.0 Client ID
|
|
41
|
+
# 5. Set application type to "Web application"
|
|
42
|
+
# 6. Add authorized redirect URI: http://localhost:8000/auth/oauth2/callback
|
|
43
|
+
# 7. Copy Client ID and Client Secret to OAUTH2_CLIENT_ID and OAUTH2_CLIENT_SECRET
|
|
44
|
+
|
|
45
|
+
# ============================================================================
|
|
46
|
+
# Azure AD OAuth2 Configuration
|
|
47
|
+
# ============================================================================
|
|
48
|
+
# When using OAUTH2_PROVIDER=azure, configure:
|
|
49
|
+
AZURE_TENANT_ID=common # Or your specific tenant ID
|
|
50
|
+
|
|
51
|
+
# Azure AD Setup:
|
|
52
|
+
# 1. Go to https://portal.azure.com/
|
|
53
|
+
# 2. Navigate to Azure Active Directory > App registrations
|
|
54
|
+
# 3. Click "New registration"
|
|
55
|
+
# 4. Set redirect URI: http://localhost:8000/auth/oauth2/callback
|
|
56
|
+
# 5. Go to Certificates & secrets > New client secret
|
|
57
|
+
# 6. Copy Application (client) ID to OAUTH2_CLIENT_ID
|
|
58
|
+
# 7. Copy client secret value to OAUTH2_CLIENT_SECRET
|
|
59
|
+
# 8. Copy Directory (tenant) ID to AZURE_TENANT_ID
|
|
60
|
+
|
|
61
|
+
# ============================================================================
|
|
62
|
+
# Generic OIDC Provider Configuration
|
|
63
|
+
# ============================================================================
|
|
64
|
+
# When using OAUTH2_PROVIDER=generic, manually configure all endpoints:
|
|
65
|
+
# OAUTH2_DISCOVERY_URL=https://your-provider.com/.well-known/openid-configuration
|
|
66
|
+
# OAUTH2_AUTHORIZATION_ENDPOINT=https://your-provider.com/oauth2/authorize
|
|
67
|
+
# OAUTH2_TOKEN_ENDPOINT=https://your-provider.com/oauth2/token
|
|
68
|
+
# OAUTH2_USERINFO_ENDPOINT=https://your-provider.com/oauth2/userinfo
|
|
69
|
+
# OAUTH2_JWKS_URI=https://your-provider.com/oauth2/jwks
|
|
70
|
+
# OIDC_ISSUER=https://your-provider.com
|
|
71
|
+
# OIDC_AUDIENCE=your-client-id
|
|
72
|
+
|
|
73
|
+
# Generic OIDC providers include:
|
|
74
|
+
# - Okta: https://your-domain.okta.com/.well-known/openid-configuration
|
|
75
|
+
# - Auth0: https://your-domain.auth0.com/.well-known/openid-configuration
|
|
76
|
+
# - Keycloak: https://your-domain.com/auth/realms/{realm}/.well-known/openid-configuration
|
|
77
|
+
# - GitLab: https://gitlab.com/.well-known/openid-configuration
|
|
78
|
+
|
|
79
|
+
# ============================================================================
|
|
80
|
+
# LDAP Configuration (Optional - for LDAP authentication)
|
|
81
|
+
# ============================================================================
|
|
82
|
+
LDAP_SERVER=ldap://localhost:389
|
|
83
|
+
LDAP_BASE_DN=dc=example,dc=com
|
|
84
|
+
LDAP_USER_DN=ou=users,dc=example,dc=com
|
|
85
|
+
LDAP_BIND_DN=cn=admin,dc=example,dc=com
|
|
86
|
+
LDAP_BIND_PASSWORD=<your-ldap-bind-password>
|
|
87
|
+
LDAP_USE_SSL=false
|
|
88
|
+
LDAP_USE_ACTIVE_DIRECTORY=false
|
|
89
|
+
LDAP_AD_DOMAIN=example.com
|
|
90
|
+
|
|
91
|
+
# ============================================================================
|
|
92
|
+
# CORS Configuration
|
|
93
|
+
# ============================================================================
|
|
94
|
+
CORS_ORIGINS=http://localhost:3000,http://localhost:8080
|
|
95
|
+
|
|
96
|
+
# ============================================================================
|
|
97
|
+
# Usage Examples
|
|
98
|
+
# ============================================================================
|
|
99
|
+
|
|
100
|
+
# Example 1: Google OAuth2
|
|
101
|
+
# OAUTH2_ENABLED=true
|
|
102
|
+
# OAUTH2_PROVIDER=google
|
|
103
|
+
# OAUTH2_CLIENT_ID=123456789-abcdefg.apps.googleusercontent.com
|
|
104
|
+
# OAUTH2_CLIENT_SECRET=GOCSPX-your-secret-here
|
|
105
|
+
# OAUTH2_REDIRECT_URI=http://localhost:8000/auth/oauth2/callback
|
|
106
|
+
|
|
107
|
+
# Example 2: Azure AD OAuth2
|
|
108
|
+
# OAUTH2_ENABLED=true
|
|
109
|
+
# OAUTH2_PROVIDER=azure
|
|
110
|
+
# AZURE_TENANT_ID=your-tenant-id-here
|
|
111
|
+
# OAUTH2_CLIENT_ID=your-application-id-here
|
|
112
|
+
# OAUTH2_CLIENT_SECRET=your-client-secret-here
|
|
113
|
+
# OAUTH2_REDIRECT_URI=http://localhost:8000/auth/oauth2/callback
|
|
114
|
+
|
|
115
|
+
# Example 3: Okta OIDC
|
|
116
|
+
# OAUTH2_ENABLED=true
|
|
117
|
+
# OAUTH2_PROVIDER=generic
|
|
118
|
+
# OAUTH2_DISCOVERY_URL=https://your-domain.okta.com/.well-known/openid-configuration
|
|
119
|
+
# OAUTH2_CLIENT_ID=your-okta-client-id
|
|
120
|
+
# OAUTH2_CLIENT_SECRET=your-okta-client-secret
|
|
121
|
+
# OAUTH2_REDIRECT_URI=http://localhost:8000/auth/oauth2/callback
|
|
122
|
+
# OIDC_ISSUER=https://your-domain.okta.com
|
|
123
|
+
# OIDC_AUDIENCE=your-okta-client-id
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Bug report
|
|
3
|
+
about: Report an issue to help improve docling-pipelines
|
|
4
|
+
title: ''
|
|
5
|
+
labels: bug
|
|
6
|
+
assignees: ''
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
### Bug
|
|
11
|
+
<!-- Describe the buggy behavior you have observed. -->
|
|
12
|
+
...
|
|
13
|
+
|
|
14
|
+
### Steps to reproduce
|
|
15
|
+
<!-- Describe the sequence of steps for reproducing the bug. -->
|
|
16
|
+
...
|
|
17
|
+
|
|
18
|
+
### docling-pipelines version
|
|
19
|
+
<!-- Copy the output of `docling-pipelines --version`. -->
|
|
20
|
+
...
|
|
21
|
+
|
|
22
|
+
### Python version
|
|
23
|
+
<!-- Copy the output of `python --version`. -->
|
|
24
|
+
...
|
|
25
|
+
|
|
26
|
+
<!-- ⚠️ ATTENTION: When sharing screenshots, attachments, or other data make sure not to include any sensitive information. -->
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
blank_issues_enabled: false
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Feature request
|
|
3
|
+
about: Suggest an idea for enhancing docling-pipelines
|
|
4
|
+
title: ''
|
|
5
|
+
labels: enhancement
|
|
6
|
+
assignees: ''
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
### Requested feature
|
|
11
|
+
<!-- Describe the feature you have in mind and the user need it addresses. -->
|
|
12
|
+
...
|
|
13
|
+
|
|
14
|
+
### Alternatives
|
|
15
|
+
<!-- Describe any alternatives you have considered. -->
|
|
16
|
+
...
|
|
17
|
+
|
|
18
|
+
<!-- ⚠️ ATTENTION: When sharing screenshots, attachments, or other data make sure not to include any sensitive information. -->
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Question
|
|
3
|
+
about: Ask a question
|
|
4
|
+
title: ''
|
|
5
|
+
labels: question
|
|
6
|
+
assignees: ''
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
### Question
|
|
11
|
+
<!-- Describe what you would like to achieve and which part you need help with. -->
|
|
12
|
+
...
|
|
13
|
+
|
|
14
|
+
<!-- ⚠️ ATTENTION: When sharing screenshots, attachments, or other data make sure not to include any sensitive information. -->
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Security and Disclosure Information Policy for the Docling Project
|
|
2
|
+
|
|
3
|
+
The Docling-pipelines team and community take security bugs seriously. We appreciate your efforts to responsibly disclose your findings, and will make every effort to acknowledge your contributions.
|
|
4
|
+
|
|
5
|
+
### Security
|
|
6
|
+
|
|
7
|
+
- Participation in the [OpenSSF Best Practices Badge Program](https://bestpractices.coreinfrastructure.org/en/projects/) for Free/Libre and FLOSS projects to ensure that we follow current best practices for quality and security
|
|
8
|
+
- Use of [HTTPS](https://en.wikipedia.org/wiki/HTTPS) for network communication
|
|
9
|
+
- Use of secure protocols for network communication (through the use of HTTPS)
|
|
10
|
+
- Up-to-date support for TLS/SSL (through the use of [OpenSSL](https://www.openssl.org/))
|
|
11
|
+
- Performance of TLS certificate verification by default before sending HTTP headers with private information (through the use of OpenSSL and HTTPS)
|
|
12
|
+
- Distribution of the software via cryptographically signed releases (on the [PyPI](https://pypi.org/), [Quay.io](https://quay.io/organization/docling-project/) and [GHCR.io](https://github.com/orgs/docling-project/packages) package repositories)
|
|
13
|
+
- Use of [GitHub](https://github.com/) Issues for vulnerability reporting and tracking
|
|
14
|
+
|
|
15
|
+
### Analysis
|
|
16
|
+
|
|
17
|
+
- Use of [Ruff](https://docs.astral.sh/ruff/), [ty](https://docs.astral.sh/ty/) and [Pytest](https://docs.pytest.org/en/7.2.x/) for Python code linting (static and dynamic analysers) on pull requests and builds
|
|
18
|
+
- Use of GitHub Issues for bug reporting and tracking
|
|
19
|
+
|
|
20
|
+
## Reporting a Vulnerability
|
|
21
|
+
|
|
22
|
+
If you think you've identified a security issue in a Docling project repository, please DO NOT report the issue publicly via the GitHub issue tracker, discussions, or any other public forum.
|
|
23
|
+
|
|
24
|
+
### Preferred Method: GitHub Private Vulnerability Reporting
|
|
25
|
+
|
|
26
|
+
**We strongly encourage you to use GitHub's Private Vulnerability Reporting feature**, which provides a secure and streamlined process for disclosing security issues:
|
|
27
|
+
|
|
28
|
+
1. Navigate to the [Security tab](https://github.com/docling-project/docling/security) of the Docling repository
|
|
29
|
+
2. Click on "Report a vulnerability"
|
|
30
|
+
3. Fill out the vulnerability report form with as many details as possible
|
|
31
|
+
4. Submit the report
|
|
32
|
+
|
|
33
|
+
This method allows for:
|
|
34
|
+
- **Secure communication** directly with the maintainers team
|
|
35
|
+
- **Coordinated disclosure** through GitHub's built-in workflow
|
|
36
|
+
- **Automatic tracking** of the vulnerability lifecycle
|
|
37
|
+
- **Credit attribution** when the vulnerability is published
|
|
38
|
+
|
|
39
|
+
### Alternative Method: Email Disclosure
|
|
40
|
+
|
|
41
|
+
Alternatively, you can send an email with as many details as possible to [docling_pipeline_team-dg@ibm.com](mailto:docling_pipeline_team-dg@ibm.com). This is a distribution list for the maintainers team.
|
|
42
|
+
|
|
43
|
+
**Important:** Please do not create a public issue or discuss the vulnerability in any public channel until it has been addressed.
|
|
44
|
+
|
|
45
|
+
### Security Vulnerability Response
|
|
46
|
+
|
|
47
|
+
Each report is acknowledged and analyzed by the core maintainers within 3 working days.
|
|
48
|
+
|
|
49
|
+
Any vulnerability information shared with core maintainers stays within the Docling project and will not be disseminated to other projects unless it is necessary to get the issue fixed.
|
|
50
|
+
|
|
51
|
+
After the initial reply to your report, the security team will keep you informed of the progress towards a fix and full announcement, and may ask for additional information or guidance.
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
merge_protections:
|
|
2
|
+
- name: Enforce conventional commit
|
|
3
|
+
description: Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
|
4
|
+
if:
|
|
5
|
+
- base = main
|
|
6
|
+
success_conditions:
|
|
7
|
+
- "title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(\\(.+\\))?(!)?:"
|
|
8
|
+
- name: Require two reviewers for breaking changes
|
|
9
|
+
description: Breaking changes require additional scrutiny with 2 approvers
|
|
10
|
+
if:
|
|
11
|
+
- base = main
|
|
12
|
+
- "title ~= !:"
|
|
13
|
+
success_conditions:
|
|
14
|
+
- "#approved-reviews-by >= 2"
|
|
15
|
+
- name: Require one reviewer for small changes
|
|
16
|
+
description: All non-breaking PRs to main require at least 1 approval
|
|
17
|
+
if:
|
|
18
|
+
- base = main
|
|
19
|
+
success_conditions:
|
|
20
|
+
- "#approved-reviews-by >= 1"
|
|
21
|
+
merge_protections_settings:
|
|
22
|
+
reporting_method: check-runs
|
|
23
|
+
queue_rules:
|
|
24
|
+
- name: label queue
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
<!--
|
|
2
|
+
Note -
|
|
3
|
+
1. Ensure that an appropriate title is given to the PR in the Title bar of the PR
|
|
4
|
+
2. Headings that are not applicable to the PR can be removed from the below template
|
|
5
|
+
-->
|
|
6
|
+
|
|
7
|
+
## Issue
|
|
8
|
+
- <!-- Tag the issue link to the PR for which the changes are done -->
|
|
9
|
+
|
|
10
|
+
## Dev Tracking
|
|
11
|
+
<!-- Link to epic or parent task if this PR is part of a larger feature/epic -->
|
|
12
|
+
<!-- Example: Part of Epic #123 or Subtask of #456 -->
|
|
13
|
+
|
|
14
|
+
## Description
|
|
15
|
+
<!-- Concise Summary of the PR -->
|
|
16
|
+
|
|
17
|
+
## Basic Checklist
|
|
18
|
+
- [ ] Code follows project standards (keyword-only arguments, no unicode in logging)
|
|
19
|
+
- [ ] Tests added/updated for new functionality
|
|
20
|
+
- [ ] Documentation updated (see Documentation Update Checklist below)
|
|
21
|
+
- [ ] No breaking changes (or documented in Breaking Changes section)
|
|
22
|
+
|
|
23
|
+
## Breaking Changes
|
|
24
|
+
<!-- List any breaking changes introduced by this PR -->
|
|
25
|
+
<!-- If none, write "None" -->
|
|
26
|
+
|
|
27
|
+
## Dependencies
|
|
28
|
+
<!-- List any new dependencies added or version updates -->
|
|
29
|
+
<!-- Include package name, version, and reason for addition/update -->
|
|
30
|
+
<!-- If none, write "None" -->
|
|
31
|
+
|
|
32
|
+
## Testing Details
|
|
33
|
+
<!-- Details about how the changes were tested -->
|
|
34
|
+
<!-- Include test commands, test files, or manual testing steps -->
|
|
35
|
+
|
|
36
|
+
## Documentation Update Checklist
|
|
37
|
+
<!-- Check all documentation files that were updated or reviewed for this PR -->
|
|
38
|
+
- [ ] README.md (if adding features, changing structure, or modifying setup)
|
|
39
|
+
- [ ] ARCHITECTURE.md (if modifying operators, data flow, or system design)
|
|
40
|
+
- [ ] CONTRIBUTING.md (if changing development workflow or code standards)
|
|
41
|
+
- [ ] USER_GUIDE_PIPELINE_SETUP.md (if changing installation, setup, or execution)
|
|
42
|
+
- [ ] QUICKSTART.md (if changing quick start steps or examples)
|
|
43
|
+
- [ ] docs/reference/OPERATORS.md (if adding/modifying operators)
|
|
44
|
+
- [ ] TROUBLESHOOTING.md (if discovering new issues or solutions)
|
|
45
|
+
- [ ] Operator documentation in docs/operators/ (if operator added/modified)
|
|
46
|
+
- [ ] N/A - No documentation updates needed
|
|
47
|
+
|
|
48
|
+
## Screenshots/Videos
|
|
49
|
+
<!-- For UI changes, include screenshots or videos demonstrating the changes -->
|
|
50
|
+
<!-- If not applicable, remove this section -->
|
|
51
|
+
|
|
52
|
+
### Pre-Commit Hook Results
|
|
53
|
+
<!-- Paste the output of pre-commit hooks execution to verify all checks passed -->
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
### Note
|
|
57
|
+
1. **Keyword Arguments** - Ensure function arguments are keyword-based, not positional - [Link](https://docs.python.org/3/glossary.html#term-argument)
|
|
58
|
+
2. **Metadata Independence:** When adding or modifying operator metadata, ensure the data does not depend on instance variables.
|
|
59
|
+
3. **Method Requirements:** Ensure `get_metadata` is a static method. If `get_required_features` depends on instance variables, the operator must implement `get_static_required_features` using the default values referenced in `get_required_features`
|
|
60
|
+
4. For every PR, ensure that the pre-commit hook output has been included.
|