apache-airflow-providers-amazon 8.24.0rc1__tar.gz → 8.25.0__tar.gz

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: apache-airflow-providers-amazon
-Version: 8.24.0rc1
+Version: 8.25.0
 Summary: Provider package apache-airflow-providers-amazon for Apache Airflow
 Keywords: airflow-provider,amazon,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
@@ -22,21 +22,21 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Topic :: System :: Monitoring
 Requires-Dist: PyAthena>=3.0.10
-Requires-Dist: apache-airflow-providers-common-sql>=1.3.1rc0
+Requires-Dist: apache-airflow-providers-common-sql>=1.3.1
 Requires-Dist: apache-airflow-providers-http
-Requires-Dist: apache-airflow>=2.7.0rc0
-Requires-Dist: asgiref
-Requires-Dist: boto3>=1.33.0
-Requires-Dist: botocore>=1.33.0
+Requires-Dist: apache-airflow>=2.7.0
+Requires-Dist: asgiref>=2.3.0
+Requires-Dist: boto3>=1.34.90
+Requires-Dist: botocore>=1.34.90
 Requires-Dist: inflection>=0.5.1
-Requires-Dist: jmespath
+Requires-Dist: jmespath>=0.7.0
 Requires-Dist: jsonpath_ng>=1.5.3
 Requires-Dist: redshift_connector>=2.0.918
 Requires-Dist: sqlalchemy_redshift>=0.8.6
-Requires-Dist: watchtower>=2.0.1,<4
-Requires-Dist: aiobotocore[boto3]>=2.5.3 ; extra == "aiobotocore"
+Requires-Dist: watchtower>=3.0.0,<4
+Requires-Dist: aiobotocore[boto3]>=2.13.0 ; extra == "aiobotocore"
 Requires-Dist: apache-airflow-providers-apache-hive ; extra == "apache.hive"
-Requires-Dist: apache-airflow-providers-cncf-kubernetes>=7.2.0rc0 ; extra == "cncf.kubernetes"
+Requires-Dist: apache-airflow-providers-cncf-kubernetes>=7.2.0 ; extra == "cncf.kubernetes"
 Requires-Dist: apache-airflow-providers-common-sql ; extra == "common.sql"
 Requires-Dist: apache-airflow-providers-exasol ; extra == "exasol"
 Requires-Dist: apache-airflow-providers-ftp ; extra == "ftp"
@@ -46,14 +46,15 @@ Requires-Dist: apache-airflow-providers-imap ; extra == "imap"
 Requires-Dist: apache-airflow-providers-microsoft-azure ; extra == "microsoft.azure"
 Requires-Dist: apache-airflow-providers-mongo ; extra == "mongo"
 Requires-Dist: apache-airflow-providers-openlineage ; extra == "openlineage"
-Requires-Dist: pandas>=1.2.5,<2.2 ; extra == "pandas"
+Requires-Dist: pandas>=2.1.2,<2.2 ; extra == "pandas" and (python_version>="3.9")
+Requires-Dist: pandas>=1.5.3,<2.2 ; extra == "pandas" and (python_version<"3.9")
 Requires-Dist: python3-saml>=1.16.0 ; extra == "python3-saml"
 Requires-Dist: s3fs>=2023.10.0 ; extra == "s3fs"
 Requires-Dist: apache-airflow-providers-salesforce ; extra == "salesforce"
 Requires-Dist: apache-airflow-providers-ssh ; extra == "ssh"
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.24.0/changelog.html
-Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.24.0
+Project-URL: Changelog, https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.25.0/changelog.html
+Project-URL: Documentation, https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.25.0
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
 Project-URL: Twitter, https://twitter.com/ApacheAirflow
@@ -120,7 +121,7 @@ Provides-Extra: ssh
 
 Package ``apache-airflow-providers-amazon``
 
-Release: ``8.24.0.rc1``
+Release: ``8.25.0``
 
 
 Amazon integration (including `Amazon Web Services (AWS) <https://aws.amazon.com/>`__).
@@ -133,7 +134,7 @@ This is a provider package for ``amazon`` provider. All classes for this provide
 are in ``airflow.providers.amazon`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.24.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.25.0/>`_.
 
 Installation
 ------------
@@ -153,16 +154,16 @@ PIP package                              Version required
 ``apache-airflow``                       ``>=2.7.0``
 ``apache-airflow-providers-common-sql``  ``>=1.3.1``
 ``apache-airflow-providers-http``
-``boto3``                                ``>=1.33.0``
-``botocore``                             ``>=1.33.0``
+``boto3``                                ``>=1.34.90``
+``botocore``                             ``>=1.34.90``
 ``inflection``                           ``>=0.5.1``
-``watchtower``                           ``>=2.0.1,<4``
+``watchtower``                           ``>=3.0.0,<4``
 ``jsonpath_ng``                          ``>=1.5.3``
 ``redshift_connector``                   ``>=2.0.918``
 ``sqlalchemy_redshift``                  ``>=0.8.6``
-``asgiref``
+``asgiref``                              ``>=2.3.0``
 ``PyAthena``                             ``>=3.0.10``
-``jmespath``
+``jmespath``                             ``>=0.7.0``
 =======================================  ==================
 
 Cross provider package dependencies
@@ -197,4 +198,4 @@ Dependent package
 ======================================================================================================================  ===================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.24.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.25.0/changelog.html>`_.

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/README.rst

@@ -42,7 +42,7 @@
 
 Package ``apache-airflow-providers-amazon``
 
-Release: ``8.24.0.rc1``
+Release: ``8.25.0``
 
 
 Amazon integration (including `Amazon Web Services (AWS) <https://aws.amazon.com/>`__).
@@ -55,7 +55,7 @@ This is a provider package for ``amazon`` provider. All classes for this provide
 are in ``airflow.providers.amazon`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.24.0/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.25.0/>`_.
 
 Installation
 ------------
@@ -75,16 +75,16 @@ PIP package                              Version required
 ``apache-airflow``                       ``>=2.7.0``
 ``apache-airflow-providers-common-sql``  ``>=1.3.1``
 ``apache-airflow-providers-http``
-``boto3``                                ``>=1.33.0``
-``botocore``                             ``>=1.33.0``
+``boto3``                                ``>=1.34.90``
+``botocore``                             ``>=1.34.90``
 ``inflection``                           ``>=0.5.1``
-``watchtower``                           ``>=2.0.1,<4``
+``watchtower``                           ``>=3.0.0,<4``
 ``jsonpath_ng``                          ``>=1.5.3``
 ``redshift_connector``                   ``>=2.0.918``
 ``sqlalchemy_redshift``                  ``>=0.8.6``
-``asgiref``
+``asgiref``                              ``>=2.3.0``
 ``PyAthena``                             ``>=3.0.10``
-``jmespath``
+``jmespath``                             ``>=0.7.0``
 =======================================  ==================
 
 Cross provider package dependencies
@@ -119,4 +119,4 @@ Dependent package
 ======================================================================================================================  ===================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.24.0/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-amazon/8.25.0/changelog.html>`_.

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/LICENSE

@@ -215,7 +215,7 @@ Third party Apache 2.0 licenses
 
 The following components are provided under the Apache 2.0 License.
 See project link for details. The text of each license is also included
-at licenses/LICENSE-[project].txt.
+at 3rd-party-licenses/LICENSE-[project].txt.
 
     (ALv2 License) hue v4.3.0 (https://github.com/cloudera/hue/)
     (ALv2 License) jqclock v2.3.0 (https://github.com/JohnRDOrazio/jQuery-Clock-Plugin)
@@ -227,7 +227,7 @@ MIT licenses
 ========================================================================
 
 The following components are provided under the MIT License. See project link for details.
-The text of each license is also included at licenses/LICENSE-[project].txt.
+The text of each license is also included at 3rd-party-licenses/LICENSE-[project].txt.
 
     (MIT License) jquery v3.5.1 (https://jquery.org/license/)
     (MIT License) dagre-d3 v0.6.4 (https://github.com/cpettitt/dagre-d3)
@@ -243,11 +243,11 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
 BSD 3-Clause licenses
 ========================================================================
 The following components are provided under the BSD 3-Clause license. See project links for details.
-The text of each license is also included at licenses/LICENSE-[project].txt.
+The text of each license is also included at 3rd-party-licenses/LICENSE-[project].txt.
 
     (BSD 3 License) d3 v5.16.0 (https://d3js.org)
     (BSD 3 License) d3-shape v2.1.0 (https://github.com/d3/d3-shape)
     (BSD 3 License) cgroupspy 0.2.1 (https://github.com/cloudsigma/cgroupspy)
 
 ========================================================================
-See licenses/LICENSES-ui.txt for packages used in `/airflow/www`
+See 3rd-party-licenses/LICENSES-ui.txt for packages used in `/airflow/www`

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "8.24.0"
+__version__ = "8.25.0"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.7.0"

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/aws/hooks/base_aws.py

@@ -156,7 +156,9 @@ class BaseSessionFactory(LoggingMixin):
 
         return async_get_session()
 
-    def create_session(self, deferrable: bool = False) -> boto3.session.Session:
+    def create_session(
+        self, deferrable: bool = False
+    ) -> boto3.session.Session | aiobotocore.session.AioSession:
         """Create boto3 or aiobotocore Session from connection config."""
         if not self.conn:
             self.log.info(
@@ -198,7 +200,7 @@ class BaseSessionFactory(LoggingMixin):
 
     def _create_session_with_assume_role(
         self, session_kwargs: dict[str, Any], deferrable: bool = False
-    ) -> boto3.session.Session:
+    ) -> boto3.session.Session | aiobotocore.session.AioSession:
         if self.conn.assume_role_method == "assume_role_with_web_identity":
             # Deferred credentials have no initial credentials
             credential_fetcher = self._get_web_identity_credential_fetcher()
@@ -239,7 +241,10 @@ class BaseSessionFactory(LoggingMixin):
         session._credentials = credentials
         session.set_config_variable("region", self.basic_session.region_name)
 
-        return boto3.session.Session(botocore_session=session, **session_kwargs)
+        if not deferrable:
+            return boto3.session.Session(botocore_session=session, **session_kwargs)
+
+        return session
 
     def _refresh_credentials(self) -> dict[str, Any]:
         self.log.debug("Refreshing credentials")

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/aws/hooks/comprehend.py

@@ -16,6 +16,7 @@
 # under the License.
 from __future__ import annotations
 
+from airflow.exceptions import AirflowException
 from airflow.providers.amazon.aws.hooks.base_aws import AwsBaseHook
 
 
@@ -35,3 +36,35 @@ class ComprehendHook(AwsBaseHook):
     def __init__(self, *args, **kwargs) -> None:
         kwargs["client_type"] = "comprehend"
         super().__init__(*args, **kwargs)
+
+    def validate_document_classifier_training_status(
+        self, document_classifier_arn: str, fail_on_warnings: bool = False
+    ) -> None:
+        """
+        Log the Information about the document classifier.
+
+        NumberOfLabels
+        NumberOfTrainedDocuments
+        NumberOfTestDocuments
+        EvaluationMetrics
+
+        """
+        response = self.conn.describe_document_classifier(DocumentClassifierArn=document_classifier_arn)
+
+        status = response["DocumentClassifierProperties"]["Status"]
+
+        if status == "TRAINED_WITH_WARNING":
+            self.log.info(
+                "AWS Comprehend document classifier training completed with %s, Message: %s please review the skipped files folder in the output location %s",
+                status,
+                response["DocumentClassifierProperties"]["Message"],
+                response["DocumentClassifierProperties"]["OutputDataConfig"]["S3Uri"],
+            )
+
+            if fail_on_warnings:
+                raise AirflowException("Warnings in AWS Comprehend document classifier training.")
+
+        self.log.info(
+            "AWS Comprehend document classifier metadata: %s",
+            response["DocumentClassifierProperties"]["ClassifierMetadata"],
+        )

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/aws/hooks/glue.py

@@ -20,6 +20,7 @@ from __future__ import annotations
 import asyncio
 import time
 from functools import cached_property
+from typing import Any
 
 from botocore.exceptions import ClientError
 
@@ -430,3 +431,125 @@ class GlueJobHook(AwsBaseHook):
             self.conn.create_job(**config)
 
         return self.job_name
+
+
+class GlueDataQualityHook(AwsBaseHook):
+    """
+    Interact with AWS Glue Data Quality.
+
+    Provide thick wrapper around :external+boto3:py:class:`boto3.client("glue") <Glue.Client>`.
+
+    Additional arguments (such as ``aws_conn_id``) may be specified and
+    are passed down to the underlying AwsBaseHook.
+
+    .. seealso::
+        - :class:`airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook`
+    """
+
+    def __init__(
+        self,
+        *args,
+        **kwargs,
+    ):
+        kwargs["client_type"] = "glue"
+        super().__init__(*args, **kwargs)
+
+    def has_data_quality_ruleset(self, name: str) -> bool:
+        try:
+            self.conn.get_data_quality_ruleset(Name=name)
+            return True
+        except self.conn.exceptions.EntityNotFoundException:
+            return False
+
+    def _log_results(self, result: dict[str, Any]) -> None:
+        """
+        Print the outcome of evaluation run, An evaluation run can involve multiple rulesets evaluated against a data source (Glue table).
+
+        Name    Description                                     Result        EvaluatedMetrics                                                                    EvaluationMessage
+        Rule_1    RowCount between 150000 and 600000             PASS        {'Dataset.*.RowCount': 300000.0}                                                       NaN
+        Rule_2    IsComplete "marketplace"                       PASS        {'Column.marketplace.Completeness': 1.0}                                               NaN
+        Rule_3    ColumnLength "marketplace" between 1 and 2     FAIL        {'Column.marketplace.MaximumLength': 9.0, 'Column.marketplace.MinimumLength': 3.0}     Value: 9.0 does not meet the constraint requirement!
+
+        """
+        import pandas as pd
+
+        pd.set_option("display.max_rows", None)
+        pd.set_option("display.max_columns", None)
+        pd.set_option("display.width", None)
+        pd.set_option("display.max_colwidth", None)
+
+        self.log.info(
+            "AWS Glue data quality ruleset evaluation result for RulesetName: %s RulesetEvaluationRunId: %s Score: %s",
+            result.get("RulesetName"),
+            result.get("RulesetEvaluationRunId"),
+            result.get("Score"),
+        )
+
+        rule_results = result["RuleResults"]
+        rule_results_df = pd.DataFrame(rule_results)
+        self.log.info(rule_results_df)
+
+    def get_evaluation_run_results(self, run_id: str) -> dict[str, Any]:
+        response = self.conn.get_data_quality_ruleset_evaluation_run(RunId=run_id)
+
+        return self.conn.batch_get_data_quality_result(ResultIds=response["ResultIds"])
+
+    def validate_evaluation_run_results(
+        self, evaluation_run_id: str, show_results: bool = True, verify_result_status: bool = True
+    ) -> None:
+        results = self.get_evaluation_run_results(evaluation_run_id)
+        total_failed_rules = 0
+
+        if results.get("ResultsNotFound"):
+            self.log.info(
+                "AWS Glue data quality ruleset evaluation run, results not found for %s",
+                results["ResultsNotFound"],
+            )
+
+        for result in results["Results"]:
+            rule_results = result["RuleResults"]
+
+            total_failed_rules += len(
+                list(
+                    filter(
+                        lambda result: result.get("Result") == "FAIL" or result.get("Result") == "ERROR",
+                        rule_results,
+                    )
+                )
+            )
+
+            if show_results:
+                self._log_results(result)
+
+        self.log.info(
+            "AWS Glue data quality ruleset evaluation run, total number of rules failed: %s",
+            total_failed_rules,
+        )
+
+        if verify_result_status and total_failed_rules > 0:
+            raise AirflowException(
+                "AWS Glue data quality ruleset evaluation run failed for one or more rules"
+            )
+
+    def log_recommendation_results(self, run_id: str) -> None:
+        """
+        Print the outcome of recommendation run, recommendation run generates multiple rules against a data source (Glue table) in Data Quality Definition Language (DQDL) format.
+
+        Rules = [
+        IsComplete "NAME",
+        ColumnLength "EMP_ID" between 1 and 12,
+        IsUnique "EMP_ID",
+        ColumnValues "INCOME" > 50000
+        ]
+        """
+        result = self.conn.get_data_quality_rule_recommendation_run(RunId=run_id)
+
+        if result.get("RecommendedRuleset"):
+            self.log.info(
+                "AWS Glue data quality recommended rules for DatabaseName: %s TableName: %s",
+                result["DataSource"]["GlueTable"]["DatabaseName"],
+                result["DataSource"]["GlueTable"]["TableName"],
+            )
+            self.log.info(result["RecommendedRuleset"])
+        else:
+            self.log.info("AWS Glue data quality, no recommended rules available for RunId: %s", run_id)

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/aws/hooks/redshift_sql.py

@@ -20,14 +20,19 @@ from functools import cached_property
 from typing import TYPE_CHECKING
 
 import redshift_connector
+from packaging.version import Version
 from redshift_connector import Connection as RedshiftConnection
 from sqlalchemy import create_engine
 from sqlalchemy.engine.url import URL
 
+from airflow import __version__ as AIRFLOW_VERSION
 from airflow.exceptions import AirflowException
 from airflow.providers.amazon.aws.hooks.base_aws import AwsBaseHook
 from airflow.providers.common.sql.hooks.sql import DbApiHook
 
+_IS_AIRFLOW_2_10_OR_HIGHER = Version(Version(AIRFLOW_VERSION).base_version) >= Version("2.10.0")
+
+
 if TYPE_CHECKING:
     from airflow.models.connection import Connection
     from airflow.providers.openlineage.sqlparser import DatabaseInfo
@@ -257,4 +262,6 @@ class RedshiftSQLHook(DbApiHook):
 
     def get_openlineage_default_schema(self) -> str | None:
         """Return current schema. This is usually changed with ``SEARCH_PATH`` parameter."""
-        return self.get_first("SELECT CURRENT_SCHEMA();")[0]
+        if _IS_AIRFLOW_2_10_OR_HIGHER:
+            return self.get_first("SELECT CURRENT_SCHEMA();")[0]
+        return super().get_openlineage_default_schema()

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/aws/operators/bedrock.py

@@ -20,7 +20,6 @@ import json
 from time import sleep
 from typing import TYPE_CHECKING, Any, Sequence
 
-import botocore
 from botocore.exceptions import ClientError
 
 from airflow.configuration import conf
@@ -38,7 +37,7 @@ from airflow.providers.amazon.aws.triggers.bedrock import (
     BedrockKnowledgeBaseActiveTrigger,
     BedrockProvisionModelThroughputCompletedTrigger,
 )
-from airflow.providers.amazon.aws.utils import get_botocore_version, validate_execute_complete_event
+from airflow.providers.amazon.aws.utils import validate_execute_complete_event
 from airflow.providers.amazon.aws.utils.mixins import aws_template_fields
 from airflow.utils.helpers import prune_dict
 from airflow.utils.timezone import utcnow
@@ -799,24 +798,11 @@ class BedrockRaGOperator(AwsBaseOperator[BedrockAgentRuntimeHook]):
     def execute(self, context: Context) -> Any:
         self.validate_inputs()
 
-        try:
-            result = self.hook.conn.retrieve_and_generate(
-                input={"text": self.input},
-                retrieveAndGenerateConfiguration=self.build_rag_config(),
-                **self.rag_kwargs,
-            )
-        except botocore.exceptions.ParamValidationError as error:
-            if (
-                'Unknown parameter in retrieveAndGenerateConfiguration: "externalSourcesConfiguration"'
-                in str(error)
-            ) and (self.source_type == "EXTERNAL_SOURCES"):
-                self.log.error(
-                    "You are attempting to use External Sources and the BOTO API returned an "
-                    "error message which may indicate the need to update botocore to do this.  \n"
-                    "Support for External Sources was added in botocore 1.34.90 and you are using botocore %s",
-                    ".".join(map(str, get_botocore_version())),
-                )
-            raise
+        result = self.hook.conn.retrieve_and_generate(
+            input={"text": self.input},
+            retrieveAndGenerateConfiguration=self.build_rag_config(),
+            **self.rag_kwargs,
+        )
 
         self.log.info(
             "\nPrompt: %s\nResponse: %s\nCitations: %s",

{apache_airflow_providers_amazon-8.24.0rc1 → apache_airflow_providers_amazon-8.25.0}/airflow/providers/amazon/aws/operators/comprehend.py

@@ -23,7 +23,10 @@ from airflow.configuration import conf
 from airflow.exceptions import AirflowException
 from airflow.providers.amazon.aws.hooks.comprehend import ComprehendHook
 from airflow.providers.amazon.aws.operators.base_aws import AwsBaseOperator
-from airflow.providers.amazon.aws.triggers.comprehend import ComprehendPiiEntitiesDetectionJobCompletedTrigger
+from airflow.providers.amazon.aws.triggers.comprehend import (
+    ComprehendCreateDocumentClassifierCompletedTrigger,
+    ComprehendPiiEntitiesDetectionJobCompletedTrigger,
+)
 from airflow.providers.amazon.aws.utils import validate_execute_complete_event
 from airflow.providers.amazon.aws.utils.mixins import aws_template_fields
 from airflow.utils.timezone import utcnow
@@ -190,3 +193,147 @@ class ComprehendStartPiiEntitiesDetectionJobOperator(ComprehendBaseOperator):
 
         self.log.info("Comprehend pii entities detection job `%s` complete.", event["job_id"])
         return event["job_id"]
+
+
+class ComprehendCreateDocumentClassifierOperator(AwsBaseOperator[ComprehendHook]):
+    """
+    Create a comprehend document classifier that can categorize documents.
+
+    Provide a set of training documents that are labeled with the categories.
+
+    .. seealso::
+        For more information on how to use this operator, take a look at the guide:
+        :ref:`howto/operator:ComprehendCreateDocumentClassifierOperator`
+
+    :param document_classifier_name: The name of the document classifier. (templated)
+    :param input_data_config: Specifies the format and location of the input data for the job. (templated)
+    :param mode: Indicates the mode in which the classifier will be trained. (templated)
+    :param data_access_role_arn: The Amazon Resource Name (ARN) of the IAM role that grants Amazon Comprehend
+        read access to your input data. (templated)
+    :param language_code: The language of the input documents. You can specify any of the languages supported by
+        Amazon Comprehend. All documents must be in the same language. (templated)
+    :param fail_on_warnings: If set to True, the document classifier training job will throw an error when the
+        status is TRAINED_WITH_WARNING. (default False)
+    :param output_data_config: Specifies the location for the output files from a custom classifier job.
+        This parameter is required for a request that creates a native document model. (templated)
+    :param document_classifier_kwargs: Any optional parameters to pass to the document classifier. (templated)
+
+    :param wait_for_completion: Whether to wait for job to stop. (default: True)
+    :param waiter_delay: Time in seconds to wait between status checks. (default: 60)
+    :param waiter_max_attempts: Maximum number of attempts to check for job completion. (default: 20)
+    :param deferrable: If True, the operator will wait asynchronously for the job to stop.
+        This implies waiting for completion. This mode requires aiobotocore module to be installed.
+        (default: False)
+    :param aws_conn_id: The Airflow connection used for AWS credentials.
+        If this is ``None`` or empty then the default boto3 behaviour is used. If
+        running Airflow in a distributed manner and aws_conn_id is None or
+        empty, then default boto3 configuration would be used (and must be
+        maintained on each worker node).
+    :param region_name: AWS region_name. If not specified then the default boto3 behaviour is used.
+    :param verify: Whether to verify SSL certificates. See:
+        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html
+    :param botocore_config: Configuration dictionary (key-values) for botocore client. See:
+        https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html
+    """
+
+    aws_hook_class = ComprehendHook
+
+    template_fields: Sequence[str] = aws_template_fields(
+        "document_classifier_name",
+        "input_data_config",
+        "mode",
+        "data_access_role_arn",
+        "language_code",
+        "output_data_config",
+        "document_classifier_kwargs",
+    )
+
+    template_fields_renderers: dict = {
+        "input_data_config": "json",
+        "output_data_config": "json",
+        "document_classifier_kwargs": "json",
+    }
+
+    def __init__(
+        self,
+        document_classifier_name: str,
+        input_data_config: dict[str, Any],
+        mode: str,
+        data_access_role_arn: str,
+        language_code: str,
+        fail_on_warnings: bool = False,
+        output_data_config: dict[str, Any] | None = None,
+        document_classifier_kwargs: dict[str, Any] | None = None,
+        wait_for_completion: bool = True,
+        waiter_delay: int = 60,
+        waiter_max_attempts: int = 20,
+        deferrable: bool = conf.getboolean("operators", "default_deferrable", fallback=False),
+        aws_conn_id: str | None = "aws_default",
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+        self.document_classifier_name = document_classifier_name
+        self.input_data_config = input_data_config
+        self.mode = mode
+        self.data_access_role_arn = data_access_role_arn
+        self.language_code = language_code
+        self.fail_on_warnings = fail_on_warnings
+        self.output_data_config = output_data_config
+        self.document_classifier_kwargs = document_classifier_kwargs or {}
+        self.wait_for_completion = wait_for_completion
+        self.waiter_delay = waiter_delay
+        self.waiter_max_attempts = waiter_max_attempts
+        self.deferrable = deferrable
+        self.aws_conn_id = aws_conn_id
+
+    def execute(self, context: Context) -> str:
+        if self.output_data_config:
+            self.document_classifier_kwargs["OutputDataConfig"] = self.output_data_config
+
+        document_classifier_arn = self.hook.conn.create_document_classifier(
+            DocumentClassifierName=self.document_classifier_name,
+            InputDataConfig=self.input_data_config,
+            Mode=self.mode,
+            DataAccessRoleArn=self.data_access_role_arn,
+            LanguageCode=self.language_code,
+            **self.document_classifier_kwargs,
+        )["DocumentClassifierArn"]
+
+        message_description = f"document classifier {document_classifier_arn} to complete."
+        if self.deferrable:
+            self.log.info("Deferring %s", message_description)
+            self.defer(
+                trigger=ComprehendCreateDocumentClassifierCompletedTrigger(
+                    document_classifier_arn=document_classifier_arn,
+                    waiter_delay=self.waiter_delay,
+                    waiter_max_attempts=self.waiter_max_attempts,
+                    aws_conn_id=self.aws_conn_id,
+                ),
+                method_name="execute_complete",
+            )
+        elif self.wait_for_completion:
+            self.log.info("Waiting for %s", message_description)
+
+            self.hook.get_waiter("create_document_classifier_complete").wait(
+                DocumentClassifierArn=document_classifier_arn,
+                WaiterConfig={"Delay": self.waiter_delay, "MaxAttempts": self.waiter_max_attempts},
+            )
+
+            self.hook.validate_document_classifier_training_status(
+                document_classifier_arn=document_classifier_arn, fail_on_warnings=self.fail_on_warnings
+            )
+
+        return document_classifier_arn
+
+    def execute_complete(self, context: Context, event: dict[str, Any] | None = None) -> str:
+        event = validate_execute_complete_event(event)
+        if event["status"] != "success":
+            raise AirflowException("Error while running comprehend create document classifier: %s", event)
+
+        self.hook.validate_document_classifier_training_status(
+            document_classifier_arn=event["document_classifier_arn"], fail_on_warnings=self.fail_on_warnings
+        )
+
+        self.log.info("Comprehend document classifier `%s` complete.", event["document_classifier_arn"])
+
+        return event["document_classifier_arn"]