apache-airflow-providers-edge3 1.1.2rc1__tar.gz → 1.1.3rc1__tar.gz

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/PKG-INFO

@@ -1,11 +1,11 @@
 Metadata-Version: 2.4
 Name: apache-airflow-providers-edge3
-Version: 1.1.2rc1
+Version: 1.1.3rc1
 Summary: Provider package apache-airflow-providers-edge3 for Apache Airflow
 Keywords: airflow-provider,edge3,airflow,integration
 Author-email: Apache Software Foundation <dev@airflow.apache.org>
 Maintainer-email: Apache Software Foundation <dev@airflow.apache.org>
-Requires-Python: ~=3.10
+Requires-Python: >=3.10
 Description-Content-Type: text/x-rst
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console
@@ -18,13 +18,14 @@ Classifier: License :: OSI Approved :: Apache Software License
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: System :: Monitoring
 Requires-Dist: apache-airflow>=2.10.0rc1
 Requires-Dist: pydantic>=2.11.0
 Requires-Dist: retryhttp>=1.2.0,!=1.3.0
 Project-URL: Bug Tracker, https://github.com/apache/airflow/issues
-Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.2/changelog.html
-Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.2
+Project-URL: Changelog, https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.3/changelog.html
+Project-URL: Documentation, https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.3
 Project-URL: Mastodon, https://fosstodon.org/@airflow
 Project-URL: Slack Chat, https://s.apache.org/airflow-slack
 Project-URL: Source Code, https://github.com/apache/airflow
@@ -55,8 +56,9 @@ Project-URL: YouTube, https://www.youtube.com/channel/UCSXwxpWZQ7XZ1WL3wqevChA/
 
 Package ``apache-airflow-providers-edge3``
 
-Release: ``1.1.2``
+Release: ``1.1.3``
 
+Release Date: ``|PypiReleaseDate|``
 
 Handle edge workers on remote sites via HTTP(s) connection and orchestrates work over distributed sites.
 
@@ -80,7 +82,7 @@ This is a provider package for ``edge3`` provider. All classes for this provider
 are in ``airflow.providers.edge3`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.3/>`_.
 
 Installation
 ------------
@@ -89,7 +91,7 @@ You can install this package on top of an existing Airflow 2 installation (see `
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-edge3``
 
-The package supports the following python versions: 3.10,3.11,3.12
+The package supports the following python versions: 3.10,3.11,3.12,3.13
 
 Requirements
 ------------
@@ -103,5 +105,5 @@ PIP package         Version required
 ==================  ===================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.3/changelog.html>`_.
 

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/README.rst

@@ -23,8 +23,9 @@
 
 Package ``apache-airflow-providers-edge3``
 
-Release: ``1.1.2``
+Release: ``1.1.3``
 
+Release Date: ``|PypiReleaseDate|``
 
 Handle edge workers on remote sites via HTTP(s) connection and orchestrates work over distributed sites.
 
@@ -48,7 +49,7 @@ This is a provider package for ``edge3`` provider. All classes for this provider
 are in ``airflow.providers.edge3`` python package.
 
 You can find package information and changelog for the provider
-in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.2/>`_.
+in the `documentation <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.3/>`_.
 
 Installation
 ------------
@@ -57,7 +58,7 @@ You can install this package on top of an existing Airflow 2 installation (see `
 for the minimum Airflow version supported) via
 ``pip install apache-airflow-providers-edge3``
 
-The package supports the following python versions: 3.10,3.11,3.12
+The package supports the following python versions: 3.10,3.11,3.12,3.13
 
 Requirements
 ------------
@@ -71,4 +72,4 @@ PIP package         Version required
 ==================  ===================
 
 The changelog for the provider package can be found in the
-`changelog <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.2/changelog.html>`_.
+`changelog <https://airflow.apache.org/docs/apache-airflow-providers-edge3/1.1.3/changelog.html>`_.

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/docs/architecture.rst

@@ -171,7 +171,7 @@ The following features are known missing and will be implemented in increments:
 
 - Tests
 
-  - System tests in Github, test the deployment of the worker with a Dag execution
+  - System tests in GitHub, test the deployment of the worker with a Dag execution
   - Test/Support on Windows for Edge Worker
 
 - Scaling test - Check and define boundaries of workers/jobs. Today it is known to

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/docs/changelog.rst

@@ -27,6 +27,18 @@
 Changelog
 ---------
 
+1.1.3
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix: Prevent duplicate edge_job insertions for deferrable tasks in EdgeExecutor (#53610) (#53927)``
+
+.. Below changes are excluded from the changelog. Move them to
+   appropriate section above if needed. Do not delete the lines(!):
+   * ``Remove parameter from Edge example (#53997)``
+
 1.1.2
 .....
 
@@ -34,10 +46,23 @@ Bug Fixes
 ~~~~~~~~~
 
 * ``Fix UnboundLocalError for 'edge_job_command_len' (#52328)``
+* ``Extend run detection to dev-mode to load plugin (#53576)``
+* ``Add queue and remove queue cli commands for EdgeExecutor (#53505)``
+* ``Ensure Edge Plugin for API endpoint is only loaded on API-Server and AF2 Webserver (#52952)``
+* ``Fix unreachable code mypy warnings in edge3 provider (#53430)``
+* ``Make edge3 provider compatible with mypy 1.16.1 (#53104)``
+* ``Fix task configuration defaults for AbstractOperator (#52871)``
 
 Misc
 ~~~~
 
+* ``Remove upper-binding for "python-requires" (#52980)``
+* ``Temporarily switch to use >=,< pattern instead of '~=' (#52967)``
+* ``Deprecate decorators from Core (#53629)``
+* ``Add Python 3.13 support for Airflow. (#46891)``
+* ``Cleanup type ignores in edge3 provider where possible (#53248)``
+* ``Remove type ignore across codebase after mypy upgrade (#53243)``
+* ``Remove deprecation in Edge for DEFAULT_QUEUE (#52954)``
 * ``Move 'BaseHook' implementation to task SDK (#51873)``
 * ``Force the definition of 'execution_api_server_url' based on 'api_url' (#52184)``
 * ``Drop support for Python 3.9 (#52072)``
@@ -58,6 +83,11 @@ Doc-only
    appropriate section above if needed. Do not delete the lines(!):
    * ``Revert "Revert "Remove FAB dependency from Edge3 Provider (#51995)"" (#52000)``
    * ``Revert "Remove FAB dependency from Edge3 Provider (#51995)" (#51998)``
+   * ``Make dag_version_id in TI non-nullable (#50825)``
+   * ``Fix spelling of GitHub brand name (#53735)``
+   * ``Replace mock.patch("utcnow") with time_machine in Edge Executor (#53670)``
+   * ``Prepare release for July 2025 1st provider wave (#52727)``
+
 
 1.1.1
 .....

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/docs/deployment.rst

@@ -190,3 +190,5 @@ instance. The commands are:
 - ``airflow edge remote-edge-worker-exit-maintenance``: Request a remote edge worker to exit maintenance mode
 - ``airflow edge shutdown-remote-edge-worker``: Shuts down a remote edge worker gracefully
 - ``airflow edge remove-remote-edge-worker``: Remove a worker instance from the cluster
+- ``airflow edge add-worker-queues``: Add queues to an edge worker
+- ``airflow edge remove-worker-queues``: Remove queues from an edge worker

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/docs/index.rst

@@ -90,7 +90,9 @@ Additional REST API endpoints are provided to distribute tasks and manage the ed
 are provided by the API server.
 
 
-Release: 1.1.2
+Release: 1.1.3
+
+Release Date: ``|PypiReleaseDate|``
 
 Provider package
 ----------------

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/provider.yaml

@@ -33,13 +33,14 @@ description: |
   are provided by the API server.
 
 state: ready
-source-date-epoch: 1751473321
+source-date-epoch: 1754503142
 
 # Note that those versions are maintained by release manager - do not update them manually
 # with the exception of case where other provider in sources has >= new provider version.
 # In such case adding >= NEW_VERSION and bumping to NEW_VERSION in a provider have
 # to be done in the same PR
 versions:
+  - 1.1.3
   - 1.1.2
   - 1.1.1
   - 1.1.0

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/pyproject.toml

@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
 
 [project]
 name = "apache-airflow-providers-edge3"
-version = "1.1.2rc1"
+version = "1.1.3rc1"
 description = "Provider package apache-airflow-providers-edge3 for Apache Airflow"
 readme = "README.rst"
 authors = [
@@ -47,9 +47,10 @@ classifiers = [
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
     "Topic :: System :: Monitoring",
 ]
-requires-python = "~=3.10"
+requires-python = ">=3.10"
 
 # The dependencies should be modified in place in the generated file.
 # Any change in the dependencies is preserved when the file is regenerated
@@ -95,8 +96,8 @@ apache-airflow-providers-common-sql = {workspace = true}
 apache-airflow-providers-standard = {workspace = true}
 
 [project.urls]
-"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.2"
-"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.2/changelog.html"
+"Documentation" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.3"
+"Changelog" = "https://airflow.staged.apache.org/docs/apache-airflow-providers-edge3/1.1.3/changelog.html"
 "Bug Tracker" = "https://github.com/apache/airflow/issues"
 "Source Code" = "https://github.com/apache/airflow"
 "Slack Chat" = "https://s.apache.org/airflow-slack"

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/__init__.py

@@ -14,4 +14,4 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-__path__ = __import__("pkgutil").extend_path(__path__, __name__)  # type: ignore
+__path__ = __import__("pkgutil").extend_path(__path__, __name__)

{apache_airflow_providers_edge3-1.1.2rc1/tests/unit → apache_airflow_providers_edge3-1.1.3rc1/src/airflow/providers}/__init__.py

@@ -14,4 +14,4 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-__path__ = __import__("pkgutil").extend_path(__path__, __name__)  # type: ignore
+__path__ = __import__("pkgutil").extend_path(__path__, __name__)

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/__init__.py

@@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version
 
 __all__ = ["__version__"]
 
-__version__ = "1.1.2"
+__version__ = "1.1.3"
 
 if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse(
     "2.10.0"

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/cli/edge_command.py

@@ -351,6 +351,48 @@ def remote_worker_request_shutdown(args) -> None:
     logger.info("Requested shutdown of Edge Worker host %s by %s.", args.edge_hostname, getuser())
 
 
+@cli_utils.action_cli(check_db=False)
+@providers_configuration_loaded
+def add_worker_queues(args) -> None:
+    """Add queues to an edge worker."""
+    _check_valid_db_connection()
+    _check_if_registered_edge_host(hostname=args.edge_hostname)
+    from airflow.providers.edge3.models.edge_worker import add_worker_queues
+
+    queues = args.queues.split(",") if args.queues else []
+    if not queues:
+        raise SystemExit("Error: No queues specified to add.")
+
+    try:
+        add_worker_queues(args.edge_hostname, queues)
+        logger.info("Added queues %s to Edge Worker host %s by %s.", queues, args.edge_hostname, getuser())
+    except TypeError as e:
+        logger.error(str(e))
+        raise SystemExit
+
+
+@cli_utils.action_cli(check_db=False)
+@providers_configuration_loaded
+def remove_worker_queues(args) -> None:
+    """Remove queues from an edge worker."""
+    _check_valid_db_connection()
+    _check_if_registered_edge_host(hostname=args.edge_hostname)
+    from airflow.providers.edge3.models.edge_worker import remove_worker_queues
+
+    queues = args.queues.split(",") if args.queues else []
+    if not queues:
+        raise SystemExit("Error: No queues specified to remove.")
+
+    try:
+        remove_worker_queues(args.edge_hostname, queues)
+        logger.info(
+            "Removed queues %s from Edge Worker host %s by %s.", queues, args.edge_hostname, getuser()
+        )
+    except TypeError as e:
+        logger.error(str(e))
+        raise SystemExit
+
+
 ARG_CONCURRENCY = Arg(
     ("-c", "--concurrency"),
     type=int,
@@ -380,6 +422,11 @@ ARG_REQUIRED_MAINTENANCE_COMMENT = Arg(
     help="Maintenance comments to report reason. Required if enabling maintenance",
     required=True,
 )
+ARG_QUEUES_MANAGE = Arg(
+    ("-q", "--queues"),
+    help="Comma delimited list of queues to add or remove.",
+    required=True,
+)
 ARG_WAIT_MAINT = Arg(
     ("-w", "--wait"),
     default=False,
@@ -516,4 +563,22 @@ EDGE_COMMANDS: list[ActionCommand] = [
         func=remote_worker_request_shutdown,
         args=(ARG_REQUIRED_EDGE_HOSTNAME,),
     ),
+    ActionCommand(
+        name="add-worker-queues",
+        help=add_worker_queues.__doc__,
+        func=add_worker_queues,
+        args=(
+            ARG_REQUIRED_EDGE_HOSTNAME,
+            ARG_QUEUES_MANAGE,
+        ),
+    ),
+    ActionCommand(
+        name="remove-worker-queues",
+        help=remove_worker_queues.__doc__,
+        func=remove_worker_queues,
+        args=(
+            ARG_REQUIRED_EDGE_HOSTNAME,
+            ARG_QUEUES_MANAGE,
+        ),
+    ),
 ]

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/cli/worker.py

@@ -188,8 +188,8 @@ class EdgeWorker:
 
             try:
                 api_url = conf.get("edge", "api_url")
-                execution_api_server_url = conf.get("core", "execution_api_server_url", fallback=...)
-                if execution_api_server_url is ...:
+                execution_api_server_url = conf.get("core", "execution_api_server_url", fallback="")
+                if not execution_api_server_url:
                     parsed = urlparse(api_url)
                     execution_api_server_url = f"{parsed.scheme}://{parsed.netloc}/execution/"
 

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/example_dags/integration_test.py

@@ -41,13 +41,13 @@ try:
     from airflow.sdk import DAG, Param, Variable, task, task_group
 except ImportError:
     # Airflow 2.10 compat
-    from airflow.decorators import task, task_group  # type: ignore[no-redef,attr-defined]
-    from airflow.models.dag import DAG  # type: ignore[no-redef,attr-defined,assignment]
-    from airflow.models.param import Param  # type: ignore[no-redef,attr-defined]
-    from airflow.models.variable import Variable  # type: ignore[no-redef,attr-defined]
-    from airflow.operators.bash import BashOperator  # type: ignore[no-redef,attr-defined]
-    from airflow.operators.empty import EmptyOperator  # type: ignore[no-redef,attr-defined]
-    from airflow.operators.python import PythonOperator  # type: ignore[no-redef,attr-defined]
+    from airflow.decorators import task, task_group  # type: ignore[attr-defined,no-redef]
+    from airflow.models.dag import DAG  # type: ignore[assignment]
+    from airflow.models.param import Param  # type: ignore[no-redef]
+    from airflow.models.variable import Variable
+    from airflow.operators.bash import BashOperator  # type: ignore[no-redef]
+    from airflow.operators.empty import EmptyOperator  # type: ignore[no-redef]
+    from airflow.operators.python import PythonOperator  # type: ignore[no-redef]
 
 with DAG(
     dag_id="integration_test",

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/example_dags/win_test.py

@@ -32,7 +32,11 @@ from subprocess import STDOUT, Popen
 from time import sleep
 from typing import TYPE_CHECKING, Any
 
-from airflow.decorators import task, task_group
+try:
+    from airflow.sdk import task, task_group
+except ImportError:
+    # Airflow 2 path
+    from airflow.decorators import task, task_group  # type: ignore[attr-defined,no-redef]
 from airflow.exceptions import AirflowException, AirflowNotFoundException, AirflowSkipException
 from airflow.models import BaseOperator
 from airflow.models.dag import DAG
@@ -52,11 +56,11 @@ if TYPE_CHECKING:
     try:
         from airflow.sdk.types import RuntimeTaskInstanceProtocol as TaskInstance
     except ImportError:
-        from airflow.models import TaskInstance  # type: ignore[assignment, no-redef]
+        from airflow.models import TaskInstance  # type: ignore[assignment]
     from airflow.utils.context import Context
 
 try:
-    from airflow.operators.python import PythonOperator  # type: ignore
+    from airflow.operators.python import PythonOperator
 except ImportError:
     from airflow.providers.common.compat.standard.operators import PythonOperator
 
@@ -277,7 +281,7 @@ with DAG(
 
         @task.virtualenv(requirements="numpy")
         def virtualenv():
-            import numpy  # type: ignore
+            import numpy
 
             print(f"Welcome to virtualenv with numpy version {numpy.__version__}.")
 
@@ -297,7 +301,7 @@ with DAG(
             except AirflowNotFoundException:
                 print("Connection 'integration_test' not found... but also OK.")
 
-        command = CmdOperator(task_id="command", command="echo Parameter is {{params.mapping_count}}")
+        command = CmdOperator(task_id="command", command="echo Hello World")
 
         def python_call():
             print("Hello world")

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/executors/edge_executor.py

@@ -30,7 +30,6 @@ from sqlalchemy.orm import Session
 from airflow.cli.cli_config import GroupCommand
 from airflow.configuration import conf
 from airflow.executors.base_executor import BaseExecutor
-from airflow.models.abstractoperator import DEFAULT_QUEUE
 from airflow.models.taskinstance import TaskInstance, TaskInstanceState
 from airflow.providers.edge3.cli.edge_command import EDGE_COMMANDS
 from airflow.providers.edge3.models.edge_job import EdgeJobModel
@@ -55,6 +54,7 @@ if TYPE_CHECKING:
     TaskTuple = tuple[TaskInstanceKey, CommandType, str | None, Any | None]
 
 PARALLELISM: int = conf.getint("core", "PARALLELISM")
+DEFAULT_QUEUE: str = conf.get_mandatory_value("operators", "default_queue")
 
 
 class EdgeExecutor(BaseExecutor):
@@ -140,20 +140,40 @@ class EdgeExecutor(BaseExecutor):
         del self.edge_queued_tasks[key]
 
         self.validate_airflow_tasks_run_command(command)  # type: ignore[attr-defined]
-        session.add(
-            EdgeJobModel(
+
+        # Check if job already exists with same dag_id, task_id, run_id, map_index, try_number
+        existing_job = (
+            session.query(EdgeJobModel)
+            .filter_by(
                 dag_id=key.dag_id,
                 task_id=key.task_id,
                 run_id=key.run_id,
                 map_index=key.map_index,
                 try_number=key.try_number,
-                state=TaskInstanceState.QUEUED,
-                queue=queue or DEFAULT_QUEUE,
-                concurrency_slots=task_instance.pool_slots,
-                command=str(command),
             )
+            .first()
         )
 
+        if existing_job:
+            existing_job.state = TaskInstanceState.QUEUED
+            existing_job.queue = queue or DEFAULT_QUEUE
+            existing_job.concurrency_slots = task_instance.pool_slots
+            existing_job.command = str(command)
+        else:
+            session.add(
+                EdgeJobModel(
+                    dag_id=key.dag_id,
+                    task_id=key.task_id,
+                    run_id=key.run_id,
+                    map_index=key.map_index,
+                    try_number=key.try_number,
+                    state=TaskInstanceState.QUEUED,
+                    queue=queue or DEFAULT_QUEUE,
+                    concurrency_slots=task_instance.pool_slots,
+                    command=str(command),
+                )
+            )
+
     @provide_session
     def queue_workload(
         self,
@@ -168,20 +188,40 @@ class EdgeExecutor(BaseExecutor):
 
         task_instance = workload.ti
         key = task_instance.key
-        session.add(
-            EdgeJobModel(
+
+        # Check if job already exists with same dag_id, task_id, run_id, map_index, try_number
+        existing_job = (
+            session.query(EdgeJobModel)
+            .filter_by(
                 dag_id=key.dag_id,
                 task_id=key.task_id,
                 run_id=key.run_id,
                 map_index=key.map_index,
                 try_number=key.try_number,
-                state=TaskInstanceState.QUEUED,
-                queue=task_instance.queue,
-                concurrency_slots=task_instance.pool_slots,
-                command=workload.model_dump_json(),
             )
+            .first()
         )
 
+        if existing_job:
+            existing_job.state = TaskInstanceState.QUEUED
+            existing_job.queue = task_instance.queue
+            existing_job.concurrency_slots = task_instance.pool_slots
+            existing_job.command = workload.model_dump_json()
+        else:
+            session.add(
+                EdgeJobModel(
+                    dag_id=key.dag_id,
+                    task_id=key.task_id,
+                    run_id=key.run_id,
+                    map_index=key.map_index,
+                    try_number=key.try_number,
+                    state=TaskInstanceState.QUEUED,
+                    queue=task_instance.queue,
+                    concurrency_slots=task_instance.pool_slots,
+                    command=workload.model_dump_json(),
+                )
+            )
+
     def _check_worker_liveness(self, session: Session) -> bool:
         """Reset worker state if heartbeat timed out."""
         changed = False

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/models/edge_worker.py

@@ -109,7 +109,7 @@ class EdgeWorkerModel(Base, LoggingMixin):
         super().__init__()
 
     @property
-    def sysinfo_json(self) -> dict:
+    def sysinfo_json(self) -> dict | None:
         return json.loads(self.sysinfo) if self.sysinfo else None
 
     @property
@@ -283,3 +283,37 @@ def request_shutdown(worker_name: str, session: Session = NEW_SESSION) -> None:
         EdgeWorkerState.UNKNOWN,
     ):
         worker.state = EdgeWorkerState.SHUTDOWN_REQUEST
+
+
+@provide_session
+def add_worker_queues(worker_name: str, queues: list[str], session: Session = NEW_SESSION) -> None:
+    """Add queues to an edge worker."""
+    query = select(EdgeWorkerModel).where(EdgeWorkerModel.worker_name == worker_name)
+    worker: EdgeWorkerModel = session.scalar(query)
+    if worker.state in (
+        EdgeWorkerState.OFFLINE,
+        EdgeWorkerState.OFFLINE_MAINTENANCE,
+        EdgeWorkerState.UNKNOWN,
+    ):
+        error_message = f"Cannot add queues to edge worker {worker_name} as it is in {worker.state} state!"
+        logger.error(error_message)
+        raise TypeError(error_message)
+    worker.add_queues(queues)
+
+
+@provide_session
+def remove_worker_queues(worker_name: str, queues: list[str], session: Session = NEW_SESSION) -> None:
+    """Remove queues from an edge worker."""
+    query = select(EdgeWorkerModel).where(EdgeWorkerModel.worker_name == worker_name)
+    worker: EdgeWorkerModel = session.scalar(query)
+    if worker.state in (
+        EdgeWorkerState.OFFLINE,
+        EdgeWorkerState.OFFLINE_MAINTENANCE,
+        EdgeWorkerState.UNKNOWN,
+    ):
+        error_message = (
+            f"Cannot remove queues from edge worker {worker_name} as it is in {worker.state} state!"
+        )
+        logger.error(error_message)
+        raise TypeError(error_message)
+    worker.remove_queues(queues)

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/plugins/edge_executor_plugin.py

@@ -17,6 +17,7 @@
 
 from __future__ import annotations
 
+import sys
 from typing import TYPE_CHECKING, Any
 
 from airflow.configuration import conf
@@ -213,12 +214,24 @@ try:
 except AirflowConfigException:
     EDGE_EXECUTOR_ACTIVE = False
 
+# Load the API endpoint only on api-server (Airflow 3.x) or webserver (Airflow 2.x)
+# todo(jscheffl): Remove this check when the discussion in
+#                 https://lists.apache.org/thread/w170czq6r7bslkqp1tk6bjjjo0789wgl
+#                 resulted in a proper API to selective initialize. Maybe backcompat-shim
+#                 is also needed to support Airflow-versions prior the rework.
+if AIRFLOW_V_3_0_PLUS:
+    RUNNING_ON_APISERVER = (len(sys.argv) > 1 and sys.argv[1] in ["api-server"]) or (
+        len(sys.argv) > 2 and sys.argv[2] == "airflow-core/src/airflow/api_fastapi/main.py"
+    )
+else:
+    RUNNING_ON_APISERVER = "gunicorn" in sys.argv[0] and "airflow-webserver" in sys.argv
+
 
 class EdgeExecutorPlugin(AirflowPlugin):
     """EdgeExecutor Plugin - provides API endpoints for Edge Workers in Webserver."""
 
     name = "edge_executor"
-    if EDGE_EXECUTOR_ACTIVE:
+    if EDGE_EXECUTOR_ACTIVE and RUNNING_ON_APISERVER:
         if AIRFLOW_V_3_0_PLUS:
             fastapi_apps = [_get_api_endpoint()]
         else:

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/worker_api/auth.py

@@ -58,7 +58,7 @@ if AIRFLOW_V_3_0_PLUS:
 
 else:
     # Airflow 2.10 compatibility
-    from airflow.utils.jwt_signer import JWTSigner  # type: ignore
+    from airflow.utils.jwt_signer import JWTSigner
 
     @cache
     def jwt_signer() -> JWTSigner:

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/worker_api/routes/_v2_compat.py

@@ -127,7 +127,7 @@ else:
 
     # In Airflow 3 with AIP-72 we get workload addressed by ExecuteTask
     # But in Airflow 2.10 it is a command line array
-    ExecuteTask = list[str]  # type: ignore[no-redef,assignment,misc]
+    ExecuteTask = list[str]  # type: ignore[assignment,misc]
 
     def parse_command(command: str) -> ExecuteTask:
         from ast import literal_eval

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/src/airflow/providers/edge3/worker_api/routes/_v2_routes.py

@@ -66,7 +66,7 @@ def rpcapi_v2(body: dict[str, Any]) -> APIResponse:
     # Note: Except the method map this _was_ a 100% copy of internal API module
     #       airflow.api_internal.endpoints.rpc_api_endpoint.internal_airflow_api()
     # As of rework for FastAPI in Airflow 3.0, this is updated and to be removed in the future.
-    from airflow.api_internal.endpoints.rpc_api_endpoint import (  # type: ignore[attr-defined]
+    from airflow.api_internal.endpoints.rpc_api_endpoint import (
         # Note: This is just for compatibility with Airflow 2.10, not working for Airflow 3 / main as removed
         initialize_method_map,
     )

{apache_airflow_providers_edge3-1.1.2rc1/src/airflow/providers → apache_airflow_providers_edge3-1.1.3rc1/tests/unit}/__init__.py

@@ -14,4 +14,4 @@
 # KIND, either express or implied.  See the License for the
 # specific language governing permissions and limitations
 # under the License.
-__path__ = __import__("pkgutil").extend_path(__path__, __name__)  # type: ignore
+__path__ = __import__("pkgutil").extend_path(__path__, __name__)

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/tests/unit/edge3/cli/test_worker.py

@@ -63,6 +63,7 @@ MOCK_COMMAND = (
             "dag_id": "mock",
             "run_id": "mock",
             "try_number": 1,
+            "dag_version_id": "01234567-89ab-cdef-0123-456789abcdef",
             "pool_slots": 1,
             "queue": "default",
             "priority_weight": 1,

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/tests/unit/edge3/executors/test_edge_executor.py

@@ -21,6 +21,7 @@ from datetime import datetime, timedelta
 from unittest.mock import MagicMock, patch
 
 import pytest
+import time_machine
 
 from airflow.configuration import conf
 from airflow.models.taskinstancekey import TaskInstanceKey
@@ -246,6 +247,11 @@ class TestEdgeExecutor:
 
         # Prepare some data
         with create_session() as session:
+            # Clear existing workers to avoid unique constraint violation
+            session.query(EdgeWorkerModel).delete()
+            session.commit()
+
+            # Add workers with different states
             for worker_name, state, last_heartbeat in [
                 (
                     "inactive_timed_out_worker",
@@ -275,9 +281,7 @@ class TestEdgeExecutor:
                 )
                 session.commit()
 
-        with patch(
-            "airflow.utils.timezone.utcnow", return_value=datetime(2023, 1, 1, 1, 0, 0, tzinfo=timezone.utc)
-        ):
+        with time_machine.travel(datetime(2023, 1, 1, 1, 0, 0, tzinfo=timezone.utc), tick=False):
             with conf_vars({("edge", "heartbeat_interval"): "10"}):
                 executor.sync()
 
@@ -328,13 +332,106 @@ class TestEdgeExecutor:
                 queue="default",
                 priority_weight=1,
                 start_date=timezone.utcnow(),
+                dag_version_id="4d828a62-a417-4936-a7a6-2b3fabacecab",
+            ),
+            dag_rel_path="mock.py",
+            log_path="mock.log",
+            bundle_info={"name": "n/a", "version": "no matter"},
+        )
+        executor.queue_workload(workload=workload)
+
+        with create_session() as session:
+            jobs = session.query(EdgeJobModel).all()
+            assert len(jobs) == 1
+
+    @pytest.mark.skipif(AIRFLOW_V_3_0_PLUS, reason="API only available in Airflow <3.0")
+    def test_execute_async_updates_existing_job(self):
+        executor, key = self.get_test_executor()
+
+        # First insert a job with the same key
+        with create_session() as session:
+            session.add(
+                EdgeJobModel(
+                    dag_id=key.dag_id,
+                    run_id=key.run_id,
+                    task_id=key.task_id,
+                    map_index=key.map_index,
+                    try_number=key.try_number,
+                    state=TaskInstanceState.SCHEDULED,
+                    queue="default",
+                    concurrency_slots=1,
+                    command="old-command",
+                    last_update=timezone.utcnow(),
+                )
+            )
+            session.commit()
+
+        # Trigger execute_async which should update the existing job
+        executor.edge_queued_tasks = deepcopy(executor.queued_tasks)
+        executor.execute_async(key=key, command=["airflow", "tasks", "run", "new", "command"])
+
+        with create_session() as session:
+            jobs = session.query(EdgeJobModel).all()
+            assert len(jobs) == 1
+            job = jobs[0]
+            assert job.state == TaskInstanceState.QUEUED
+            assert job.command != "old-command"
+            assert "new" in job.command
+
+    @pytest.mark.skipif(not AIRFLOW_V_3_0_PLUS, reason="API only available in Airflow 3.0+")
+    def test_queue_workload_updates_existing_job(self):
+        from uuid import uuid4
+
+        from airflow.executors.workloads import ExecuteTask, TaskInstance
+
+        executor = self.get_test_executor()[0]
+
+        key = TaskInstanceKey(dag_id="mock", run_id="mock", task_id="mock", map_index=-1, try_number=1)
+
+        # Insert an existing job
+        with create_session() as session:
+            session.add(
+                EdgeJobModel(
+                    dag_id=key.dag_id,
+                    task_id=key.task_id,
+                    run_id=key.run_id,
+                    map_index=key.map_index,
+                    try_number=key.try_number,
+                    state=TaskInstanceState.SCHEDULED,
+                    queue="default",
+                    command="old-command",
+                    concurrency_slots=1,
+                    last_update=timezone.utcnow(),
+                )
+            )
+            session.commit()
+
+        # Queue a workload with same key
+        workload = ExecuteTask(
+            token="mock",
+            ti=TaskInstance(
+                id=uuid4(),
+                task_id=key.task_id,
+                dag_id=key.dag_id,
+                run_id=key.run_id,
+                try_number=key.try_number,
+                map_index=key.map_index,
+                pool_slots=1,
+                queue="updated-queue",
+                priority_weight=1,
+                start_date=timezone.utcnow(),
+                dag_version_id=uuid4(),
             ),
             dag_rel_path="mock.py",
             log_path="mock.log",
             bundle_info={"name": "n/a", "version": "no matter"},
         )
+
         executor.queue_workload(workload=workload)
 
         with create_session() as session:
             jobs = session.query(EdgeJobModel).all()
             assert len(jobs) == 1
+            job = jobs[0]
+            assert job.queue == "updated-queue"
+            assert job.command != "old-command"

{apache_airflow_providers_edge3-1.1.2rc1 → apache_airflow_providers_edge3-1.1.3rc1}/tests/unit/edge3/plugins/test_edge_executor_plugin.py

@@ -17,6 +17,7 @@
 from __future__ import annotations
 
 import importlib
+from unittest.mock import patch
 
 import pytest
 import time_machine
@@ -44,17 +45,20 @@ def test_plugin_inactive():
 
 
 @pytest.mark.db_test
-def test_plugin_active():
-    with conf_vars({("edge", "api_enabled"): "true"}):
+def test_plugin_active_apiserver():
+    mock_cli = ["airflow", "api-server"] if AIRFLOW_V_3_0_PLUS else ["gunicorn", "airflow-webserver"]
+    with conf_vars({("edge", "api_enabled"): "true"}), patch("sys.argv", mock_cli):
         importlib.reload(edge_executor_plugin)
 
         from airflow.providers.edge3.plugins.edge_executor_plugin import (
             EDGE_EXECUTOR_ACTIVE,
+            RUNNING_ON_APISERVER,
             EdgeExecutorPlugin,
         )
 
         rep = EdgeExecutorPlugin()
         assert EDGE_EXECUTOR_ACTIVE
+        assert RUNNING_ON_APISERVER
         if AIRFLOW_V_3_0_PLUS:
             assert len(rep.appbuilder_views) == 0
             assert len(rep.flask_blueprints) == 0
@@ -64,6 +68,27 @@ def test_plugin_active():
             assert len(rep.flask_blueprints) == 2
 
 
+@patch("sys.argv", ["airflow", "some-other-command"])
+def test_plugin_active_non_apiserver():
+    with conf_vars({("edge", "api_enabled"): "true"}):
+        importlib.reload(edge_executor_plugin)
+
+        from airflow.providers.edge3.plugins.edge_executor_plugin import (
+            EDGE_EXECUTOR_ACTIVE,
+            RUNNING_ON_APISERVER,
+            EdgeExecutorPlugin,
+        )
+
+        rep = EdgeExecutorPlugin()
+        assert EDGE_EXECUTOR_ACTIVE
+        assert not RUNNING_ON_APISERVER
+        assert len(rep.appbuilder_views) == 0
+        assert len(rep.flask_blueprints) == 0
+        assert len(rep.appbuilder_views) == 0
+        if AIRFLOW_V_3_0_PLUS:
+            assert len(rep.fastapi_apps) == 0
+
+
 @pytest.fixture
 def plugin():
     from airflow.providers.edge3.plugins.edge_executor_plugin import EdgeExecutorPlugin