dt-extensions-sdk 1.1.10__tar.gz → 1.1.20__tar.gz

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: dt-extensions-sdk
-Version: 1.1.10
+Version: 1.1.20
 Project-URL: Documentation, https://github.com/dynatrace-extensions/dt-extensions-python-sdk#readme
 Project-URL: Issues, https://github.com/dynatrace-extensions/dt-extensions-python-sdk/issues
 Project-URL: Source, https://github.com/dynatrace-extensions/dt-extensions-python-sdk

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/docs/conf.py

@@ -88,7 +88,7 @@ html_theme_options = {
     "logo": "dt-sdk-logo.png",
     "logo_alt": "dt-sdk",
     "logo_url": "/",
-    "github_url": "https://github.com/dynatrace-extensions/dt-extensions-python-sdk",
+    "github_url": "https://github.com/dynatrace-extensions/dt-extensions-python-sdk/tree/main/docs/",
     "footer_links": ",".join(
         [
             "Dynatrace|https://dynatrace.com/",

dt_extensions_sdk-1.1.20/docs/guides/building.rst

@@ -0,0 +1,129 @@
+Building Extensions
+###################
+
+| This guide provides some best practices on:
+| 
+
+* Building extensions
+* Python dependencies
+* CI systems, offline installs
+
+Native dependencies
+===================
+
+| Some python libraries require "native" dependencies, they are not written in pure python and usually contain C, C++, Rust or other compiled languages code.
+| This means that they might be compiled for a very specific version of python, or for a specific operating system.
+|
+
+| Examples:
+|
+
+* **requests** requires **charset_normalizer**, a **native dependency**
+
+
+| If you navigate to the `charset-normalizer pypi page`_ you will see dozens of different wheel files.
+| Each one of these files is compiled for a different version of python, and for a different operating system.
+|
+| Your extension will run on a Dynatrace **Activegate** or **OneAgent**, which is a Linux or Windows machine, and it has a specific version of python.
+| This means that your extension must be built on a machine that has the same version of python as the Activegate.
+|
+| At this time, Dynatrace extensions run on **python 3.10**.
+|
+| When you build the extension with **dt-sdk build**, it downloads the dependencies **whl** files and places them in the lib folder of the extension.  
+| To obtain whl files for different a operating system than what the build machine is, the SDK provides the **--extra-platform** flag.
+|
+| In summary, when building from Windows, you should use:
+|
+
+.. code-block:: bash
+
+    dt-sdk build --extra-platform manylinux2014_x86_64
+
+
+| To get the correct extra wheel files for linux. Note, **manylinux2014_x86_64** works for several packages, but not all of them.
+| You need to investigate the dependencies of your extension to find the correct extra platform if that is the case.
+|
+| When building from Linux, you should use:
+|
+
+
+.. code-block:: bash
+
+     dt-sdk build --extra-platform win_amd64
+
+
+| To get the correct extra wheel files for Windows.  
+|
+
+PyPI Access
+===========
+
+| When building extensions, the SDK downloads the dependencies from PyPI.
+|
+| In some organizations, you are not allowed to access the internet from the build machine.
+| In most cases you will have either:
+| 
+
+* A local PyPI mirror
+* A directory with all the wheel files present
+
+|
+| Both of these solutions can be used with the SDK.
+|
+
+PyPI Mirror
+"""""""""""
+
+| Suppose you have a local PyPi server running on http://my-pypi-server:8080.
+|
+| To use it with the SDK, run the build command as:
+|
+
+.. code-block:: bash
+
+     PIP_INDEX_URL=http://my-pypi-server:8080/simple PIP_TRUSTED_HOST=my-pypi-server dt-sdk build
+
+
+| This will tell the SDK to use the local PyPI server to download the dependencies. 
+| The SDK uses **pip** under the covers, so all the environment variables that **pip** supports can be used with the SDK.
+|
+| Note, that assumes the build machine is a linux machine. If you are building from Windows on Powershell, you can use:
+|
+
+.. code-block:: bash
+
+    $ENV:PIP_INDEX_URL="http://my-pypi-server:8080/simple"; $ENV:PIP_TRUSTED_HOST="my-pypi-server"; dt-sdk build
+
+
+
+
+
+Local Directory
+"""""""""""""""
+
+
+| Another option is to manually download the different whl files you need, and place them in a directory on the build machine.
+| In that case, that directory can be used as the source for the dependencies.
+|
+
+.. code-block:: bash
+
+    dt-sdk build --find-links /path/to/whl/files
+
+
+| This will tell the SDK to use the directory as the source for the dependencies.
+|
+
+Musl vs libc
+============
+
+| Extensions run on `libc`_ based systems, like Ubuntu, CentOS, Windows, etc.
+| You should not use a  `musl`_ based system, like Alpine, to build extensions.
+|
+| This means that if you are using a docker container to build the extension, you should use the **python:3.10** image, or any other image that is based on a `libc`_ system.
+|
+| The reason for this is that a **musl** based system will download native whl files that are not compatible with **libc** based systems.
+
+.. _charset-normalizer pypi page: https://pypi.org/project/charset-normalizer/#files
+.. _musl: https://musl.libc.org/
+.. _libc: https://en.wikipedia.org/wiki/C_standard_library

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/docs/guides/extension_structure.rst

@@ -33,7 +33,7 @@ Here is what a sample extension definition looks like:
 
     name: custom:my-extension
     version: 0.0.1
-    minDynatraceVersion: "1.253"
+    minDynatraceVersion: "1.285"
     author:
       name: "Dynatrace"
 
@@ -41,7 +41,7 @@ Here is what a sample extension definition looks like:
       runtime:
         module: my_extension
         version:
-          min: "3.9"
+          min: "3.10"
 
       activation:
         remote:

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/docs/index.rst

@@ -273,6 +273,7 @@ Documentation
 
    guides/installation
    guides/extension_structure
+   guides/building
    guides/migration
 
 .. toctree::

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/dynatrace_extension/__about__.py

@@ -1,4 +1,5 @@
 # SPDX-FileCopyrightText: 2023-present Dynatrace LLC
 #
 # SPDX-License-Identifier: MIT
-__version__ = "1.1.10"
+
+__version__ = "1.1.20"

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/dynatrace_extension/cli/create/extension_template/extension/extension.yaml.template

@@ -1,6 +1,6 @@
 name: %extension-prefix%%extension-name%
 version: 0.0.1
-minDynatraceVersion: "1.253"
+minDynatraceVersion: "1.285"
 author:
   name: "Dynatrace"
 
@@ -8,10 +8,10 @@ python:
   runtime:
     module: %extension_name%
     version:
-      min: "3.9"
+      min: "3.10"
 
   activation:
     remote:
       path: activationSchema.json
     local:
-      path: activationSchema.json
+      path: activationSchema.json

dt_extensions_sdk-1.1.20/dynatrace_extension/cli/create/extension_template/setup.py.template

@@ -0,0 +1,28 @@
+from pathlib import Path
+from setuptools import setup, find_packages
+
+
+def find_version() -> str:
+    version = "0.0.1"
+    extension_yaml_path = Path(__file__).parent / "extension" / "extension.yaml"
+    try:
+        with open(extension_yaml_path, encoding="utf-8") as f:
+            for line in f:
+                if line.startswith("version"):
+                    version = line.split(" ")[-1].strip("\"")
+                    break
+    except Exception:
+        pass
+    return version
+
+
+setup(name="%extension_name%",
+      version=find_version(),
+      description="%Extension_Name% python EF2 extension",
+      author="Dynatrace",
+      packages=find_packages(),
+      python_requires=">=3.10",
+      include_package_data=True,
+      install_requires=["dt-extensions-sdk"],
+      extras_require={"dev": ["dt-extensions-sdk[cli]"]},
+      )

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/dynatrace_extension/cli/main.py

@@ -38,6 +38,7 @@ def run(
     fast_check: bool = typer.Option(False, "--fastcheck"),
     local_ingest: bool = typer.Option(False, "--local-ingest"),
     local_ingest_port: int = typer.Option(14499, "--local-ingest-port"),
+    print_metrics: bool = typer.Option(True),
 ):
     """
     Runs an extension, this is used during development to locally run and test an extension
@@ -47,6 +48,7 @@ def run(
     :param fast_check: If true, run a fastcheck and exits
     :param local_ingest: If true, send metrics to localhost:14499 on top of printing them
     :param local_ingest_port: The port to send metrics to, by default this is 14499
+    :param print_metrics: If true, print metrics to the console
     """
 
     # This parses the yaml, which validates it before running
@@ -58,6 +60,8 @@ def run(
         if local_ingest:
             command.append("--local-ingest")
             command.append(f"--local-ingest-port={local_ingest_port}")
+        if not print_metrics:
+            command.append("--no-print-metrics")
         run_process(command, cwd=extension_dir)
     except KeyboardInterrupt:
         console.print("\nRun interrupted with a KeyboardInterrupt, stopping", style="bold yellow")
@@ -79,6 +83,7 @@ def build(
     extra_index_url: Optional[str] = typer.Option(
         None, "--extra-index-url", "-i", help="Extra index url to use when downloading dependencies"
     ),
+    find_links: Optional[str] = typer.Option( None, "--find-links", "-f", help="Extra index url to use when downloading dependencies" ),
 ):
     """
     Builds and signs an extension using the developer fused key-certificate
@@ -90,6 +95,7 @@ def build(
     folder
     :param extra_platforms: Attempt to also download wheels for an extra platform (e.g. manylinux1_x86_64 or win_amd64)
     :param extra_index_url: Extra index url to use when downloading dependencies
+    :param find_links: Extra index url to use when downloading dependencies
     """
     console.print(f"Building and signing extension from {extension_dir} to {target_directory}", style="cyan")
     if target_directory is None:
@@ -98,7 +104,7 @@ def build(
         target_directory.mkdir()
 
     console.print("Stage 1 - Download and build dependencies", style="bold blue")
-    wheel(extension_dir, extra_platforms, extra_index_url)
+    wheel(extension_dir, extra_platforms, extra_index_url, find_links)
 
     console.print("Stage 2 - Create the extension zip file", style="bold blue")
     built_zip = assemble(extension_dir, target_directory)
@@ -163,6 +169,7 @@ def wheel(
     extra_index_url: Optional[str] = typer.Option(
         None, "--extra-index-url", "-i", help="Extra index url to use when downloading dependencies"
     ),
+    find_links: Optional[str] = typer.Option( None, "--find-links", "-f", help="Extra index url to use when downloading dependencies" ),
 ):
     """
     Builds the extension and it's dependencies into wheel files
@@ -171,6 +178,7 @@ def wheel(
     :param extension_dir: The directory of the extension, by default this is the current directory
     :param extra_platforms: Attempt to also download wheels for an extra platform (e.g. manylinux1_x86_64 or win_amd64)
     :param extra_index_url: Extra index url to use when downloading dependencies
+    :param find_links: Extra index url to use when downloading dependencies
     """
     relative_lib_folder_dir = "extension/lib"
     lib_folder: Path = extension_dir / relative_lib_folder_dir
@@ -182,6 +190,8 @@ def wheel(
     command = [sys.executable, "-m", "pip", "wheel", "-w", relative_lib_folder_dir]
     if extra_index_url is not None:
         command.extend(["--extra-index-url", extra_index_url])
+    if find_links is not None:
+        command.extend(["--find-links", find_links])
     command.append(".")
     run_process(command, cwd=extension_dir)
 
@@ -201,6 +211,8 @@ def wheel(
             ]
             if extra_index_url:
                 command.extend(["--extra-index-url", extra_index_url])
+            if find_links:
+                command.extend(["--find-links", find_links])
             command.append(".")
 
             run_process(command, cwd=extension_dir)
@@ -292,6 +304,8 @@ def upload(
         zip_file_path = Path(extension_path, "dist", zip_file_name)
 
     api_url = tenant_url or os.environ.get("DT_API_URL", "")
+    api_url = api_url.rstrip("/")
+
     if not api_url:
         console.print("Set the --tenant-url parameter or the DT_API_URL environment variable", style="bold red")
         sys.exit(1)

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/dynatrace_extension/sdk/callback.py

@@ -5,9 +5,8 @@
 import logging
 import random
 from datetime import datetime, timedelta
-from inspect import signature
 from timeit import default_timer as timer
-from typing import Callable, Optional
+from typing import Callable, Dict, Optional, Tuple
 
 from .activation import ActivationType
 from .communication import Status, StatusValue
@@ -19,8 +18,8 @@ class WrappedCallback:
         interval: timedelta,
         callback: Callable,
         logger: logging.Logger,
-        args: Optional[tuple] = None,
-        kwargs: Optional[dict] = None,
+        args: Optional[Tuple] = None,
+        kwargs: Optional[Dict] = None,
         running_in_sim=False,
         activation_type: Optional[ActivationType] = None,
     ):
@@ -47,12 +46,10 @@ class WrappedCallback:
         self.timeouts_count = 0  # counter per interval = 1 min by default
         self.exception_count = 0  # counter per interval = 1 min by default
 
-        self.callback_parameters = signature(callback).parameters
-
     def get_current_time_with_cluster_diff(self):
         return datetime.now() + timedelta(milliseconds=self.cluster_time_diff)
 
-    def __call__(self, activation_config, extension_config):
+    def __call__(self):
         self.logger.debug(f"Running scheduled callback {self}")
         self.start_timestamp = self.get_current_time_with_cluster_diff()
         self.running = True
@@ -62,11 +59,7 @@ class WrappedCallback:
         start_time = timer()
         failed = False
         try:
-            if "kwargs" in self.callback_parameters:
-                kwargs = {"activation_config": activation_config, "extension_config": extension_config}
-                self.callback(*self.callback_args, **kwargs)
-            else:
-                self.callback(*self.callback_args)
+            self.callback(*self.callback_args, **self.callback_kwargs)
             self.status = Status(StatusValue.OK)
         except Exception as e:
             failed = True

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/dynatrace_extension/sdk/communication.py

@@ -10,16 +10,21 @@ import sys
 from abc import ABC, abstractmethod
 from dataclasses import dataclass
 from enum import Enum
-from itertools import islice
 from pathlib import Path
-from typing import Any, Iterable, List, TypeVar
+from typing import Any, Generator, List, Sequence, TypeVar, Union
 
 from .vendor.mureq.mureq import HTTPException, Response, request
 
 CONTENT_TYPE_JSON = "application/json;charset=utf-8"
 CONTENT_TYPE_PLAIN = "text/plain;charset=utf-8"
 COUNT_METRIC_ITEMS_DICT = TypeVar("COUNT_METRIC_ITEMS_DICT", str, List[str])
+
+# TODO - I believe these can be adjusted via RuntimeConfig, they can't be constants
 MAX_MINT_LINES_PER_REQUEST = 1000
+MAX_LOG_EVENTS_PER_REQUEST = 50_000
+MAX_LOG_REQUEST_SIZE = 5_000_000  # actually 5_242_880
+MAX_METRIC_REQUEST_SIZE = 1_000_000  # actually 1_048_576
+
 HTTP_BAD_REQUEST = 400
 
 
@@ -92,7 +97,7 @@ class CommunicationClient(ABC):
         pass
 
     @abstractmethod
-    def send_events(self, event: dict | list[dict], eec_enrichment: bool) -> dict | None:
+    def send_events(self, event: dict | list[dict], eec_enrichment: bool) -> list[Union[dict | None]]:
         pass
 
     @abstractmethod
@@ -114,8 +119,6 @@ class HttpClient(CommunicationClient):
     """
 
     def __init__(self, base_url: str, datasource_id: str, id_token_file_path: str, logger: logging.Logger):
-        # TODO - Do we need to replace 127.0.0.1 with localhost?
-
         self._activation_config_url = f"{base_url}/userconfig/{datasource_id}"
         self._extension_config_url = f"{base_url}/extconfig/{datasource_id}"
         self._metric_url = f"{base_url}/mint/{datasource_id}"
@@ -261,41 +264,40 @@ class HttpClient(CommunicationClient):
         return self.send_status(Status())
 
     def send_metrics(self, mint_lines: list[str]) -> list[MintResponse]:
-        total_lines = len(mint_lines)
-        lines_sent = 0
-
-        self.logger.debug(f"Start sending {total_lines} metrics to the EEC")
         responses = []
 
-        # We divide into chunks of MAX_MINT_LINES_PER_REQUEST lines to avoid hitting the body size limit
-        chunks = divide_into_chunks(mint_lines, MAX_MINT_LINES_PER_REQUEST)
-
-        for chunk in chunks:
-            lines_in_chunk = len(chunk)
-            lines_sent += lines_in_chunk
-            self.logger.debug(f"Sending chunk with {lines_in_chunk} metric lines. ({lines_sent}/{total_lines})")
-            mint_data = "\n".join(chunk).encode("utf-8")
+        # We divide into batches of MAX_METRIC_REQUEST_SIZE bytes to avoid hitting the body size limit
+        batches = divide_into_batches(mint_lines, MAX_METRIC_REQUEST_SIZE, "\n")
+        for batch in batches:
             response = self._make_request(
-                self._metric_url, "POST", mint_data, extra_headers={"Content-Type": CONTENT_TYPE_PLAIN}
+                self._metric_url, "POST", batch, extra_headers={"Content-Type": CONTENT_TYPE_PLAIN}
             ).json()
             self.logger.debug(f"{self._metric_url}: {response}")
             mint_response = MintResponse.from_json(response)
             responses.append(mint_response)
         return responses
 
-    def send_events(self, events: dict | list[dict], eec_enrichment: bool = True) -> dict | None:
+    def send_events(self, events: dict | list[dict], eec_enrichment: bool = True) -> list[dict | None]:
         self.logger.debug(f"Sending log events: {events}")
-        event_data = json.dumps(events).encode("utf-8")
-        try:
-            # EEC returns empty body on success
-            return self._make_request(
-                self._events_url,
-                "POST",
-                event_data,
-                extra_headers={"Content-Type": CONTENT_TYPE_JSON, "eec-enrichment": str(eec_enrichment).lower()},
-            ).json()
-        except json.JSONDecodeError:
-            return None
+
+        responses = []
+        if isinstance(events, dict):
+            events = [events]
+        batches = divide_into_batches(events, MAX_LOG_REQUEST_SIZE)
+
+        for batch in batches:
+            try:
+                eec_response = self._make_request(
+                    self._events_url,
+                    "POST",
+                    batch,
+                    extra_headers={"Content-Type": CONTENT_TYPE_JSON, "eec-enrichment": str(eec_enrichment).lower()},
+                ).json()
+                responses.append(eec_response)
+            except json.JSONDecodeError:
+                responses.append(None)
+
+        return responses
 
     def send_sfm_metrics(self, mint_lines: list[str]) -> MintResponse:
         mint_data = "\n".join(mint_lines).encode("utf-8")
@@ -324,6 +326,7 @@ class DebugClient(CommunicationClient):
         logger: logging.Logger,
         local_ingest: bool = False,
         local_ingest_port: int = 14499,
+        print_metrics: bool = True
     ):
         self.activation_config = {}
         if activation_config_path and Path(activation_config_path).exists():
@@ -339,6 +342,7 @@ class DebugClient(CommunicationClient):
         self.logger = logger
         self.local_ingest = local_ingest
         self.local_ingest_port = local_ingest_port
+        self.print_metrics = print_metrics
 
     def get_activation_config(self) -> dict:
         return self.activation_config
@@ -386,28 +390,35 @@ class DebugClient(CommunicationClient):
         return self.send_status(Status())
 
     def send_metrics(self, mint_lines: list[str]) -> list[MintResponse]:
+        total_lines = len(mint_lines)
+        self.logger.info(f"Start sending {total_lines} metrics to the EEC")
+
         responses = []
-        for line in mint_lines:
-            self.logger.info(f"send_metric: {line}")
 
-        if self.local_ingest:
-            mint_data = "\n".join(mint_lines).encode("utf-8")
-            response = request(
-                "POST",
-                f"http://localhost:{self.local_ingest_port}/metrics/ingest",
-                body=mint_data,
-                headers={"Content-Type": CONTENT_TYPE_PLAIN},
-            ).json()
-            mint_response = MintResponse.from_json(response)
-            responses.append(mint_response)
+        batches = divide_into_batches(mint_lines, MAX_METRIC_REQUEST_SIZE)
+        for batch in batches:
+            if self.local_ingest:
+                response = request(
+                    "POST",
+                    f"http://localhost:{self.local_ingest_port}/metrics/ingest",
+                    body=batch,
+                    headers={"Content-Type": CONTENT_TYPE_PLAIN},
+                ).json()
+                mint_response = MintResponse.from_json(response)
+                responses.append(mint_response)
+            else:
+                if self.print_metrics:
+                    for line in mint_lines:
+                        self.logger.info(f"send_metric: {line}")
 
-        if not responses:
-            responses = [MintResponse(lines_invalid=0, lines_ok=len(mint_lines), error=None, warnings=None)]
         return responses
 
-    def send_events(self, events: dict | list[dict], eec_enrichment: bool = True) -> dict | None:
-        self.logger.info(f"send_events (enrichment = {eec_enrichment}): {events}")
-        return None
+    def send_events(self, events: dict | list[dict], eec_enrichment: bool = True) -> list[dict | None]:
+        self.logger.info(f"send_events (enrichment = {eec_enrichment}): {len(events)} events")
+        if self.print_metrics:
+            for event in events:
+                self.logger.info(f"send_event: {event}")
+        return []
 
     def send_sfm_metrics(self, mint_lines: list[str]) -> MintResponse:
         for line in mint_lines:
@@ -418,20 +429,33 @@ class DebugClient(CommunicationClient):
         return 0
 
 
-def divide_into_chunks(iterable: Iterable, chunk_size: int) -> Iterable:
+def divide_into_batches(items: Sequence[dict | str], max_size_bytes: int, join_with: str | None = None) -> Generator[bytes, None, None]:
     """
-    Yield successive n-sized chunks from iterable.
-    Example: _chunk([1, 2, 3, 4, 5, 6, 7, 8, 9], 3) -> [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
+    Yield successive batches from a list, according to sizing limitations
 
-    :param iterable: The iterable to chunk
-    :param chunk_size: The size of the chunks
+    :param items: The list items to divide, they myst be encodable to bytes
+    :param max_size_bytes: The maximum size of the payload in bytes
+    :param join_with: A string to join the items with before encoding
+    :return: A generator of batches of log events already encoded
     """
-    iterator = iter(iterable)
-    while True:
-        subset = list(islice(iterator, chunk_size))
-        if not subset:
-            return
-        yield subset
+
+    if not items:
+        return
+
+    if join_with is not None:
+        items = join_with.join(items)
+    encoded = f"{items}".encode(errors="replace")
+    size = len(encoded)
+    if size <= max_size_bytes:
+        yield encoded
+        return
+
+    # if we get here, the payload is too large, split it in half until we have chunks that are small enough
+    half = len(items) // 2
+    first_half = items[:half]
+    second_half = items[half:]
+    yield from divide_into_batches(first_half, max_size_bytes)
+    yield from divide_into_batches(second_half, max_size_bytes)
 
 
 @dataclass

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/dynatrace_extension/sdk/extension.py

@@ -9,6 +9,7 @@ import sys
 import threading
 import time
 from argparse import ArgumentParser
+from collections import deque
 from concurrent.futures import ThreadPoolExecutor
 from datetime import datetime, timedelta, timezone
 from enum import Enum
@@ -79,14 +80,15 @@ class DtEventType(str, Enum):
         https://docs.dynatrace.com/docs/dynatrace-api/environment-api/events-v2/post-event
     """
 
+    AVAILABILITY_EVENT = "AVAILABILITY_EVENT"
     CUSTOM_INFO = "CUSTOM_INFO"
     CUSTOM_ALERT = "CUSTOM_ALERT"
     CUSTOM_ANNOTATION = "CUSTOM_ANNOTATION"
     CUSTOM_CONFIGURATION = "CUSTOM_CONFIGURATION"
     CUSTOM_DEPLOYMENT = "CUSTOM_DEPLOYMENT"
-    MARKED_FOR_TERMINATION = "MARKED_FOR_TERMINATION"
     ERROR_EVENT = "ERROR_EVENT"
-    AVAILABILITY_EVENT = "AVAILABILITY_EVENT"
+    MARKED_FOR_TERMINATION = "MARKED_FOR_TERMINATION"
+    PERFORMANCE_EVENT = "PERFORMANCE_EVENT"
     RESOURCE_CONTENTION_EVENT = "RESOURCE_CONTENTION_EVENT"
 
 
@@ -159,8 +161,6 @@ class Extension:
         if hasattr(self, "logger"):
             return
 
-        # TODO - Move the logging implementation to its own file
-        # TODO - Add sfm logging
         self.logger = extension_logger
 
         self.extension_config: str = ""
@@ -313,7 +313,6 @@ class Extension:
         api_logger.debug(f"Scheduling callback {callback}")
 
         # These properties are updated after the extension starts
-        # TODO - These should be part of an ext singleton object instead
         callback.cluster_time_diff = self._cluster_time_diff
         callback.running_in_sim = self._running_in_sim
         self._scheduled_callbacks.append(callback)
@@ -698,14 +697,21 @@ class Extension:
         # Debug parameters, these are used when running the extension locally
         parser.add_argument("--extensionconfig", required=False, default=None)
         parser.add_argument("--activationconfig", required=False, default="activation.json")
+        parser.add_argument("--no-print-metrics", required=False, action="store_true")
 
         args, unknown = parser.parse_known_args()
         self._is_fastcheck = args.fastcheck
         if args.dsid is None:
             # DEV mode
             self._running_in_sim = True
+            print_metrics = not args.no_print_metrics
             self._client = DebugClient(
-                args.activationconfig, args.extensionconfig, api_logger, args.local_ingest, args.local_ingest_port
+                activation_config_path=args.activationconfig,
+                extension_config_path=args.extensionconfig,
+                logger=api_logger,
+                local_ingest=args.local_ingest,
+                local_ingest_port=args.local_ingest_port,
+                print_metrics=print_metrics
             )
             RuntimeProperties.set_default_log_level(args.loglevel)
         else:
@@ -769,7 +775,7 @@ class Extension:
                 current_thread_id = threading.get_ident()
                 self._running_callbacks[current_thread_id] = callback
 
-            callback(self.activation_config, self.extension_config)
+            callback()
 
             with self._sfm_metrics_lock:
                 self._callbackSfmReport[callback.name()] = callback
@@ -810,23 +816,22 @@ class Extension:
         self._scheduler.enter(SFM_METRIC_SENDING_INTERVAL.total_seconds(), 1, self._sfm_metrics_iteration)
 
     def _send_metrics(self):
-        # TODO - we might need to check size and number of lines before sending
-        # Maybe break it down into multiple packets
-        with self._metrics_lock and self._internal_callbacks_results_lock:
-            if self._metrics:
-                number_of_metrics = len(self._metrics)
-                responses = self._client.send_metrics(self._metrics)
-
-                self._internal_callbacks_results[self._send_metrics.__name__] = Status(StatusValue.OK)
-                lines_invalid = sum(response.lines_invalid for response in responses)
-                if lines_invalid > 0:
-                    message = f"{lines_invalid} invalid metric lines found"
-                    self._internal_callbacks_results[self._send_metrics.__name__] = Status(
-                        StatusValue.GENERIC_ERROR, message
-                    )
-
-                api_logger.info(f"Sent {number_of_metrics} metric lines to EEC: {responses}")
-                self._metrics = []
+        with self._metrics_lock:
+            with self._internal_callbacks_results_lock:
+                if self._metrics:
+                    number_of_metrics = len(self._metrics)
+                    responses = self._client.send_metrics(self._metrics)
+
+                    self._internal_callbacks_results[self._send_metrics.__name__] = Status(StatusValue.OK)
+                    lines_invalid = sum(response.lines_invalid for response in responses)
+                    if lines_invalid > 0:
+                        message = f"{lines_invalid} invalid metric lines found"
+                        self._internal_callbacks_results[self._send_metrics.__name__] = Status(
+                            StatusValue.GENERIC_ERROR, message
+                        )
+
+                    api_logger.info(f"Sent {number_of_metrics} metric lines to EEC: {responses}")
+                    self._metrics = []
 
     def _prepare_sfm_metrics(self) -> List[str]:
         """Prepare self monitoring metrics.
@@ -992,14 +997,21 @@ class Extension:
             self._metrics.extend(lines)
 
     def _send_events_internal(self, events: Union[dict, List[dict]]):
-        response = self._client.send_events(events, self.log_event_enrichment)
-        with self._internal_callbacks_results_lock:
-            self._internal_callbacks_results[self._send_events.__name__] = Status(StatusValue.OK)
-            if not response or "error" not in response or "message" not in response["error"]:
-                return
-            self._internal_callbacks_results[self._send_events.__name__] = Status(
-                StatusValue.GENERIC_ERROR, response["error"]["message"]
-            )
+        try:
+            responses = self._client.send_events(events, self.log_event_enrichment)
+
+            for response in responses:
+                with self._internal_callbacks_results_lock:
+                    self._internal_callbacks_results[self._send_events.__name__] = Status(StatusValue.OK)
+                    if not response or "error" not in response or "message" not in response["error"]:
+                        return
+                    self._internal_callbacks_results[self._send_events.__name__] = Status(
+                        StatusValue.GENERIC_ERROR, response["error"]["message"]
+                    )
+        except Exception as e:
+            api_logger.error(f"Error sending events: {e!r}", exc_info=True)
+            with self._internal_callbacks_results_lock:
+                self._internal_callbacks_results[self._send_events.__name__] = Status(StatusValue.GENERIC_ERROR, str(e))
 
     def _send_events(self, events: Union[dict, List[dict]]):
         self._internal_executor.submit(self._send_events_internal, events)
@@ -1008,9 +1020,8 @@ class Extension:
         self._client.send_dt_event(event)
 
     def get_version(self) -> str:
-        """Return the version of extensions sdk library."""
-
-        return __version__
+        """Return the extension version."""
+        return self.activation_config.version
 
     @property
     def techrule(self) -> str:

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/pyproject.toml

@@ -49,6 +49,7 @@ dependencies = [
     "pytest",
     "typer[all]",
     "pyyaml",
+    "dt-cli>=1.6.13"
 ]
 
 [tool.hatch.envs.default.scripts]
@@ -101,6 +102,7 @@ dependencies = [
     "dt-cli",
     "typer[all]",
     "pyyaml",
+    "tomli"
 ]
 
 [tool.hatch.envs.docs.env-vars]

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/tests/cli/test_dt_sdk.py

@@ -19,20 +19,15 @@ class TestDtSdk(TestCase):
         shutil.rmtree(self.temp_dir, ignore_errors=True)
         pass
 
-    @patch("dynatrace_extension.cli.main.subprocess.run")
+    @patch("dynatrace_extension.cli.main.dt_cli_upload")
+    @patch("dynatrace_extension.cli.main.dt_cli_validate")
     @patch("builtins.open", mock_open(read_data=SAMPLE_EXTENSION_DATA))
-    def test_dt_sdk_upload(self, mock_subprocess_run):
+    def test_dt_sdk_upload(self, mock_upload, mock_validate):
         extension_path = Path("test_extension_dir")
         tenant_url = "test_tenant_url"
         api_token = "test_api_token"
 
-        dt_sdk.upload(extension_path, tenant_url, api_token)
-        mock_subprocess_run.assert_called_once_with(
-            ["dt", "ext", "upload", "--tenant-url", tenant_url, "--api-token", api_token, f"{extension_path}"],
-            cwd=None,
-            env=None,
-            check=True,
-        )
+        dt_sdk.upload(extension_path, tenant_url, api_token, validate=False)
 
     @patch("dynatrace_extension.cli.main.subprocess.run")
     def test_dt_sdk_gen_certs(self, mock_subprocess_run: NonCallableMock):
@@ -102,6 +97,7 @@ class TestDtSdk(TestCase):
             target_directory=None,
             extra_platforms=None,
             extra_index_url=None,
+            find_links=None,
         )
 
         # Check that the built extension file exists

{dt_extensions_sdk-1.1.10 → dt_extensions_sdk-1.1.20}/tests/cli/test_types.py

@@ -8,7 +8,7 @@ from dynatrace_extension.cli.schema import ExtensionYaml
 VALID_YAML = """
 name: custom:mulesoft-cloudhub
 version: 0.0.1
-minDynatraceVersion: "1.902"
+minDynatraceVersion: "1.285"
 author:
   name: "Dynatrace"
 
@@ -16,7 +16,7 @@ python:
   runtime:
     module: mulesoft_cloudhub
     version:
-      min: "3.9"
+      min: "3.10"
 
   activation:
     remote:
@@ -38,7 +38,7 @@ class TestTypes(TestCase):
         assert extension.min_dynatrace_version == "1.902"
         assert extension.author.name == "Dynatrace"
         assert extension.python.runtime.module == "mulesoft_cloudhub"
-        assert extension.python.runtime.version.min_version == "3.9"
+        assert extension.python.runtime.version.min_version == "3.10"
         assert extension.python.activation.remote.path == "activationSchema.json"
         assert extension.python.activation.local is None
 

dt_extensions_sdk-1.1.20/tests/sdk/test_communication.py

@@ -0,0 +1,75 @@
+import unittest
+from unittest.mock import MagicMock, mock_open, patch
+
+from dynatrace_extension.sdk.communication import MAX_LOG_REQUEST_SIZE, MAX_METRIC_REQUEST_SIZE, HttpClient, divide_into_batches
+
+
+class TestCommunication(unittest.TestCase):
+    @patch("builtins.open", mock_open(read_data="test_token"))
+    @patch.object(HttpClient, "_make_request", return_value=MagicMock())
+    def test_http_client_metric_report(self, mock_make_request):
+        http_client = HttpClient("https://localhost:9999", "1", "token", MagicMock())
+        few_metrics = ["metric 1", "metric 2"]
+        responses = http_client.send_metrics(few_metrics)
+        self.assertEqual(len(responses), 1)
+
+        many_metrics = ['my.metric,dim="dim" 10'] * 500 * 100
+        responses = http_client.send_metrics(many_metrics)
+        self.assertEqual(len(responses), 2)
+
+        no_metrics = []
+        responses = http_client.send_metrics(no_metrics)
+        self.assertEqual(len(responses), 0)
+
+    def test_large_log_chunk(self):
+
+        # This is 14_660_000 bytes
+        events = []
+        for i in range(5000):
+            attributes = {}
+            for j in range(150):
+                attributes[f"attribute{j}"] = j
+            events.append(attributes)
+
+        # it needs to be divided into 4 lists, each with 3_665_000 bytes
+        chunks = list(divide_into_batches(events, MAX_LOG_REQUEST_SIZE))
+        self.assertEqual(len(chunks), 4)
+        self.assertEqual(len(chunks[0]), 3665000)
+        self.assertEqual(len(chunks[1]), 3665000)
+        self.assertEqual(len(chunks[2]), 3665000)
+        self.assertEqual(len(chunks[3]), 3665000)
+
+    def test_small_log_chunk(self):
+        events = []
+        for i in range(10):
+            attributes = {}
+            for j in range(10):
+                attributes[f"attribute{j}"] = j
+            events.append(attributes)
+
+        chunks = list(divide_into_batches(events, MAX_LOG_REQUEST_SIZE))
+        self.assertEqual(len(chunks), 1)
+        self.assertEqual(len(chunks[0]), 1720)
+
+    def test_large_metric_chunk(self):
+
+        metrics = ['my.metric,dim="dim" 10'] * 500 * 100  # 1_300_000 bytes, but becomes 1_149_999 with the newlines
+
+        # it needs to be divided into 2 lists, each with 650_000 bytes
+        chunks = list(divide_into_batches(metrics, MAX_METRIC_REQUEST_SIZE, "\n"))
+        self.assertEqual(len(chunks), 2)
+        self.assertEqual(len(chunks[0]), 574999)
+        self.assertEqual(len(chunks[1]), 575000)
+
+    def test_small_metric_chunk(self):
+        metrics = ['my.metric,dim="dim" 10'] * 100
+
+        chunks = list(divide_into_batches(metrics, MAX_METRIC_REQUEST_SIZE, "\n"))
+        self.assertEqual(len(chunks), 1)
+        self.assertEqual(len(chunks[0]), 2299)
+
+    def test_no_metrics(self):
+        metrics = []
+
+        chunks = list(divide_into_batches(metrics, MAX_METRIC_REQUEST_SIZE, "\n"))
+        self.assertEqual(len(chunks), 0)

dt_extensions_sdk-1.1.10/dynatrace_extension/cli/create/extension_template/setup.py.template

@@ -1,12 +0,0 @@
-from setuptools import setup, find_packages
-
-setup(name="%extension_name%",
-      version="0.0.1",
-      description="%Extension_Name% python EF2 extension",
-      author="Dynatrace",
-      packages=find_packages(),
-      python_requires=">=3.10",
-      include_package_data=True,
-      install_requires=["dt-extensions-sdk"],
-      extras_require={"dev": ["dt-extensions-sdk[cli]"]},
-      )

dt_extensions_sdk-1.1.10/tests/sdk/test_communication.py

@@ -1,35 +0,0 @@
-import unittest
-from unittest.mock import MagicMock, mock_open, patch
-
-from dynatrace_extension.sdk.communication import HttpClient, divide_into_chunks
-
-
-class TestCommunication(unittest.TestCase):
-    def test_large_chunk(self):
-        large_list = ["metric 1"] * 1400
-        chunks = list(divide_into_chunks(large_list, 1000))
-        self.assertEqual(len(chunks), 2)
-        self.assertEqual(len(chunks[0]), 1000)
-        self.assertEqual(len(chunks[1]), 400)
-
-    def test_small_chunk(self):
-        small_list = ["metric 1"] * 10
-        chunks = list(divide_into_chunks(small_list, 1000))
-        self.assertEqual(len(chunks), 1)
-        self.assertEqual(len(chunks[0]), 10)
-
-    @patch("builtins.open", mock_open(read_data="test_token"))
-    @patch.object(HttpClient, "_make_request", return_value=MagicMock())
-    def test_http_client_metric_report(self, mock_make_request):
-        http_client = HttpClient("https://localhost:9999", "1", "token", MagicMock())
-        few_metrics = ["metric 1", "metric 2"]
-        responses = http_client.send_metrics(few_metrics)
-        self.assertEqual(len(responses), 1)
-
-        many_metrics = ["metric 1"] * 1400
-        responses = http_client.send_metrics(many_metrics)
-        self.assertEqual(len(responses), 2)
-
-        no_metrics = []
-        responses = http_client.send_metrics(no_metrics)
-        self.assertEqual(len(responses), 0)