apitally 0.14.6__tar.gz → 0.15.0__tar.gz

{apitally-0.14.6 → apitally-0.15.0}/.pre-commit-config.yaml

@@ -8,12 +8,12 @@ repos:
       - id: trailing-whitespace
       - id: mixed-line-ending
   - repo: https://github.com/charliermarsh/ruff-pre-commit
-    rev: v0.9.9
+    rev: v0.9.10
     hooks:
       - id: ruff
         args: ["--fix", "--exit-non-zero-on-fix"]
       - id: ruff-format
   - repo: https://github.com/astral-sh/uv-pre-commit
-    rev: 0.6.3
+    rev: 0.6.5
     hooks:
       - id: uv-lock

{apitally-0.14.6 → apitally-0.15.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apitally
-Version: 0.14.6
+Version: 0.15.0
 Summary: Simple API monitoring & analytics for REST APIs built with FastAPI, Flask, Django, Starlette and Litestar.
 Project-URL: Homepage, https://apitally.io
 Project-URL: Documentation, https://docs.apitally.io
@@ -80,14 +80,13 @@ Description-Content-Type: text/markdown
 
 ---
 
-# Apitally client library for Python
+# Apitally SDK for Python
 
 [![Tests](https://github.com/apitally/apitally-py/actions/workflows/tests.yaml/badge.svg?event=push)](https://github.com/apitally/apitally-py/actions)
 [![Codecov](https://codecov.io/gh/apitally/apitally-py/graph/badge.svg?token=UNLYBY4Y3V)](https://codecov.io/gh/apitally/apitally-py)
 [![PyPI](https://img.shields.io/pypi/v/apitally?logo=pypi&logoColor=white&color=%23006dad)](https://pypi.org/project/apitally/)
 
-This client library for Apitally currently supports the following Python web
-frameworks:
+This SDK for Apitally currently supports the following Python web frameworks:
 
 - [FastAPI](https://docs.apitally.io/frameworks/fastapi)
 - [Django REST Framework](https://docs.apitally.io/frameworks/django-rest-framework)
@@ -103,19 +102,27 @@ the 📚 [documentation](https://docs.apitally.io).
 
 ### API analytics
 
-Track traffic, error and performance metrics for your API, each endpoint and individual API consumers, allowing you to make informed, data-driven engineering and product decisions.
+Track traffic, error and performance metrics for your API, each endpoint and
+individual API consumers, allowing you to make informed, data-driven engineering
+and product decisions.
 
 ### Error tracking
 
-Understand which validation rules in your endpoints cause client errors. Capture error details and stack traces for 500 error responses, and have them linked to Sentry issues automatically.
+Understand which validation rules in your endpoints cause client errors. Capture
+error details and stack traces for 500 error responses, and have them linked to
+Sentry issues automatically.
 
 ### Request logging
 
-Drill down from insights to individual requests or use powerful filtering to understand how consumers have interacted with your API. Configure exactly what is included in the logs to meet your requirements.
+Drill down from insights to individual requests or use powerful filtering to
+understand how consumers have interacted with your API. Configure exactly what
+is included in the logs to meet your requirements.
 
 ### API monitoring & alerting
 
-Get notified immediately if something isn't right using custom alerts, synthetic uptime checks and heartbeat monitoring. Notifications can be delivered via email, Slack or Microsoft Teams.
+Get notified immediately if something isn't right using custom alerts, synthetic
+uptime checks and heartbeat monitoring. Notifications can be delivered via
+email, Slack or Microsoft Teams.
 
 ## Install
 
@@ -191,8 +198,8 @@ app.wsgi_app = ApitallyMiddleware(
 
 ### Starlette
 
-This is an example of how to add the Apitally middleware to a Starlette application.
-For further instructions, see our
+This is an example of how to add the Apitally middleware to a Starlette
+application. For further instructions, see our
 [setup guide for Starlette](https://docs.apitally.io/frameworks/starlette).
 
 ```python

{apitally-0.14.6 → apitally-0.15.0}/README.md

@@ -17,14 +17,13 @@
 
 ---
 
-# Apitally client library for Python
+# Apitally SDK for Python
 
 [![Tests](https://github.com/apitally/apitally-py/actions/workflows/tests.yaml/badge.svg?event=push)](https://github.com/apitally/apitally-py/actions)
 [![Codecov](https://codecov.io/gh/apitally/apitally-py/graph/badge.svg?token=UNLYBY4Y3V)](https://codecov.io/gh/apitally/apitally-py)
 [![PyPI](https://img.shields.io/pypi/v/apitally?logo=pypi&logoColor=white&color=%23006dad)](https://pypi.org/project/apitally/)
 
-This client library for Apitally currently supports the following Python web
-frameworks:
+This SDK for Apitally currently supports the following Python web frameworks:
 
 - [FastAPI](https://docs.apitally.io/frameworks/fastapi)
 - [Django REST Framework](https://docs.apitally.io/frameworks/django-rest-framework)
@@ -40,19 +39,27 @@ the 📚 [documentation](https://docs.apitally.io).
 
 ### API analytics
 
-Track traffic, error and performance metrics for your API, each endpoint and individual API consumers, allowing you to make informed, data-driven engineering and product decisions.
+Track traffic, error and performance metrics for your API, each endpoint and
+individual API consumers, allowing you to make informed, data-driven engineering
+and product decisions.
 
 ### Error tracking
 
-Understand which validation rules in your endpoints cause client errors. Capture error details and stack traces for 500 error responses, and have them linked to Sentry issues automatically.
+Understand which validation rules in your endpoints cause client errors. Capture
+error details and stack traces for 500 error responses, and have them linked to
+Sentry issues automatically.
 
 ### Request logging
 
-Drill down from insights to individual requests or use powerful filtering to understand how consumers have interacted with your API. Configure exactly what is included in the logs to meet your requirements.
+Drill down from insights to individual requests or use powerful filtering to
+understand how consumers have interacted with your API. Configure exactly what
+is included in the logs to meet your requirements.
 
 ### API monitoring & alerting
 
-Get notified immediately if something isn't right using custom alerts, synthetic uptime checks and heartbeat monitoring. Notifications can be delivered via email, Slack or Microsoft Teams.
+Get notified immediately if something isn't right using custom alerts, synthetic
+uptime checks and heartbeat monitoring. Notifications can be delivered via
+email, Slack or Microsoft Teams.
 
 ## Install
 
@@ -128,8 +135,8 @@ app.wsgi_app = ApitallyMiddleware(
 
 ### Starlette
 
-This is an example of how to add the Apitally middleware to a Starlette application.
-For further instructions, see our
+This is an example of how to add the Apitally middleware to a Starlette
+application. For further instructions, see our
 [setup guide for Starlette](https://docs.apitally.io/frameworks/starlette).
 
 ```python

{apitally-0.14.6 → apitally-0.15.0}/apitally/client/request_logging.py

@@ -14,6 +14,12 @@ from urllib.parse import parse_qsl, urlencode, urlparse, urlunparse
 from uuid import uuid4
 
 from apitally.client.logging import get_logger
+from apitally.client.sentry import get_sentry_event_id_async
+from apitally.client.server_errors import (
+    get_exception_type,
+    get_truncated_exception_msg,
+    get_truncated_exception_traceback,
+)
 
 
 logger = get_logger(__name__)
@@ -88,6 +94,7 @@ class RequestLoggingConfig:
         log_request_body: Whether to log the request body (only if JSON or plain text)
         log_response_headers: Whether to log response header values
         log_response_body: Whether to log the response body (only if JSON or plain text)
+        log_exception: Whether to log unhandled exceptions in case of server errors
         mask_query_params: Query parameter names to mask in logs. Expects regular expressions.
         mask_headers: Header names to mask in logs. Expects regular expressions.
         mask_request_body_callback: Callback to mask the request body. Expects (method, path, body) and returns the masked body as bytes or None.
@@ -102,6 +109,7 @@ class RequestLoggingConfig:
     log_request_body: bool = False
     log_response_headers: bool = True
     log_response_body: bool = False
+    log_exception: bool = True
     mask_query_params: List[str] = field(default_factory=list)
     mask_headers: List[str] = field(default_factory=list)
     mask_request_body_callback: Optional[Callable[[RequestDict], Optional[bytes]]] = None
@@ -153,7 +161,7 @@ class RequestLogger:
         self.config = config or RequestLoggingConfig()
         self.enabled = self.config.enabled and _check_writable_fs()
         self.serialize = _get_json_serializer()
-        self.write_deque: deque[bytes] = deque([], MAX_REQUESTS_IN_DEQUE)
+        self.write_deque: deque[Dict[str, Any]] = deque([], MAX_REQUESTS_IN_DEQUE)
         self.file_deque: deque[TempGzipFile] = deque([])
         self.file: Optional[TempGzipFile] = None
         self.lock = threading.Lock()
@@ -163,7 +171,9 @@ class RequestLogger:
     def current_file_size(self) -> int:
         return self.file.size if self.file is not None else 0
 
-    def log_request(self, request: RequestDict, response: ResponseDict) -> None:
+    def log_request(
+        self, request: RequestDict, response: ResponseDict, exception: Optional[BaseException] = None
+    ) -> None:
         if not self.enabled or self.suspend_until is not None:
             return
         parsed_url = urlparse(request["url"])
@@ -215,13 +225,19 @@ class RequestLogger:
         request["headers"] = self._mask_headers(request["headers"]) if self.config.log_request_headers else []
         response["headers"] = self._mask_headers(response["headers"]) if self.config.log_response_headers else []
 
-        item = {
+        item: Dict[str, Any] = {
             "uuid": str(uuid4()),
             "request": _skip_empty_values(request),
             "response": _skip_empty_values(response),
         }
-        serialized_item = self.serialize(item)
-        self.write_deque.append(serialized_item)
+        if exception is not None and self.config.log_exception:
+            item["exception"] = {
+                "type": get_exception_type(exception),
+                "message": get_truncated_exception_msg(exception),
+                "traceback": get_truncated_exception_traceback(exception),
+            }
+            get_sentry_event_id_async(lambda event_id: item["exception"].update({"sentry_event_id": event_id}))
+        self.write_deque.append(item)
 
     def write_to_file(self) -> None:
         if not self.enabled or len(self.write_deque) == 0:
@@ -232,7 +248,7 @@ class RequestLogger:
             while True:
                 try:
                     item = self.write_deque.popleft()
-                    self.file.write_line(item)
+                    self.file.write_line(self.serialize(item))
                 except IndexError:
                     break
 

apitally-0.15.0/apitally/client/sentry.py

@@ -0,0 +1,38 @@
+import asyncio
+import contextlib
+from typing import Callable, Set
+
+
+_tasks: Set[asyncio.Task] = set()
+
+
+def get_sentry_event_id_async(cb: Callable[[str], None]) -> None:
+    try:
+        from sentry_sdk.hub import Hub
+        from sentry_sdk.scope import Scope
+    except ImportError:
+        return  # pragma: no cover
+    if not hasattr(Scope, "get_isolation_scope") or not hasattr(Scope, "_last_event_id"):
+        # sentry-sdk < 2.2.0 is not supported
+        return  # pragma: no cover
+    if Hub.current.client is None:
+        return  # sentry-sdk not initialized
+
+    scope = Scope.get_isolation_scope()
+    if event_id := scope._last_event_id:
+        cb(event_id)
+        return
+
+    async def _wait_for_sentry_event_id(scope: Scope) -> None:
+        i = 0
+        while not (event_id := scope._last_event_id) and i < 100:
+            i += 1
+            await asyncio.sleep(0.001)
+        if event_id:
+            cb(event_id)
+
+    with contextlib.suppress(RuntimeError):  # ignore no running loop
+        loop = asyncio.get_running_loop()
+        task = loop.create_task(_wait_for_sentry_event_id(scope))
+        _tasks.add(task)
+        task.add_done_callback(_tasks.discard)

apitally-0.15.0/apitally/client/server_errors.py

@@ -0,0 +1,100 @@
+from __future__ import annotations
+
+import asyncio
+import sys
+import threading
+import traceback
+from collections import Counter
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional, Set
+
+from apitally.client.sentry import get_sentry_event_id_async
+
+
+MAX_EXCEPTION_MSG_LENGTH = 2048
+MAX_EXCEPTION_TRACEBACK_LENGTH = 65536
+
+
+@dataclass(frozen=True)
+class ServerError:
+    consumer: Optional[str]
+    method: str
+    path: str
+    type: str
+    msg: str
+    traceback: str
+
+
+class ServerErrorCounter:
+    def __init__(self) -> None:
+        self.error_counts: Counter[ServerError] = Counter()
+        self.sentry_event_ids: Dict[ServerError, str] = {}
+        self._lock = threading.Lock()
+        self._tasks: Set[asyncio.Task] = set()
+
+    def add_server_error(self, consumer: Optional[str], method: str, path: str, exception: BaseException) -> None:
+        if not isinstance(exception, BaseException):
+            return  # pragma: no cover
+        with self._lock:
+            server_error = ServerError(
+                consumer=consumer,
+                method=method.upper(),
+                path=path,
+                type=get_exception_type(exception),
+                msg=get_truncated_exception_msg(exception),
+                traceback=get_truncated_exception_traceback(exception),
+            )
+            self.error_counts[server_error] += 1
+        get_sentry_event_id_async(lambda event_id: self.sentry_event_ids.update({server_error: event_id}))
+
+    def get_and_reset_server_errors(self) -> List[Dict[str, Any]]:
+        data: List[Dict[str, Any]] = []
+        with self._lock:
+            for server_error, count in self.error_counts.items():
+                data.append(
+                    {
+                        "consumer": server_error.consumer,
+                        "method": server_error.method,
+                        "path": server_error.path,
+                        "type": server_error.type,
+                        "msg": server_error.msg,
+                        "traceback": server_error.traceback,
+                        "sentry_event_id": self.sentry_event_ids.get(server_error),
+                        "error_count": count,
+                    }
+                )
+            self.error_counts.clear()
+            self.sentry_event_ids.clear()
+        return data
+
+
+def get_exception_type(exception: BaseException) -> str:
+    exception_type = type(exception)
+    return f"{exception_type.__module__}.{exception_type.__qualname__}"
+
+
+def get_truncated_exception_msg(exception: BaseException) -> str:
+    msg = str(exception).strip()
+    if len(msg) <= MAX_EXCEPTION_MSG_LENGTH:
+        return msg
+    suffix = "... (truncated)"
+    cutoff = MAX_EXCEPTION_MSG_LENGTH - len(suffix)
+    return msg[:cutoff] + suffix
+
+
+def get_truncated_exception_traceback(exception: BaseException) -> str:
+    prefix = "... (truncated) ...\n"
+    cutoff = MAX_EXCEPTION_TRACEBACK_LENGTH - len(prefix)
+    lines = []
+    length = 0
+    if sys.version_info >= (3, 10):
+        traceback_lines = traceback.format_exception(exception)
+    else:
+        traceback_lines = traceback.format_exception(type(exception), exception, exception.__traceback__)
+    for line in traceback_lines[::-1]:
+        if length + len(line) > cutoff:
+            lines.append(prefix)
+            break
+        lines.append(line)
+        length += len(line)
+    return "".join(lines[::-1]).strip()

{apitally-0.14.6 → apitally-0.15.0}/apitally/django.py

@@ -216,6 +216,7 @@ class ApitallyMiddleware:
                         "size": response_size,
                         "body": response_body,
                     },
+                    exception=getattr(request, "unhandled_exception", None),
                 )
         else:
             response = self.get_response(request)

{apitally-0.14.6 → apitally-0.15.0}/apitally/flask.py

@@ -190,6 +190,7 @@ class ApitallyMiddleware:
                     "size": response_size,
                     "body": response_body,
                 },
+                exception=g.unhandled_exception if "unhandled_exception" in g else None,
             )
 
     def get_path(self, environ: WSGIEnvironment) -> Optional[str]:

{apitally-0.14.6 → apitally-0.15.0}/apitally/litestar.py

@@ -251,6 +251,7 @@ class ApitallyPlugin(InitPluginProtocol):
                     "size": response_size,
                     "body": response_body,
                 },
+                exception=request.state["exception"] if "exception" in request.state else None,
             )
 
     def get_path(self, request: Request) -> Optional[str]:

{apitally-0.14.6 → apitally-0.15.0}/apitally/starlette.py

@@ -228,6 +228,7 @@ class ApitallyMiddleware:
                     "size": response_size,
                     "body": response_body,
                 },
+                exception=exception,
             )
 
     def get_path(self, request: Request, routes: Optional[list[BaseRoute]] = None) -> Optional[str]:

{apitally-0.14.6 → apitally-0.15.0}/tests/test_client_request_logging.py

@@ -60,7 +60,7 @@ async def test_request_logger_end_to_end(
     request_logger: RequestLogger, request_dict: RequestDict, response_dict: ResponseDict
 ):
     for _ in range(3):
-        request_logger.log_request(request_dict, response_dict)
+        request_logger.log_request(request_dict, response_dict, RuntimeError("test"))
 
     request_logger.write_to_file()
     assert request_logger.current_file_size > 0
@@ -98,6 +98,8 @@ async def test_request_logger_end_to_end(
         assert item["response"]["size"] == response_dict["size"]
         assert base64.b64decode(item["request"]["body"]) == request_dict["body"]
         assert base64.b64decode(item["response"]["body"]) == response_dict["body"]
+        assert item["exception"]["type"] == "builtins.RuntimeError"
+        assert item["exception"]["message"] == "test"
 
 
 def test_request_log_exclusion(request_logger: RequestLogger, request_dict: RequestDict, response_dict: ResponseDict):
@@ -111,7 +113,7 @@ def test_request_log_exclusion(request_logger: RequestLogger, request_dict: Requ
 
     request_logger.log_request(request_dict, response_dict)
     assert len(request_logger.write_deque) == 1
-    item = json.loads(request_logger.write_deque[0])
+    item = request_logger.write_deque[0]
     assert item["request"]["url"] == "http://localhost:8000/test"
     assert "headers" not in item["request"]
     assert "body" not in item["request"]
@@ -160,12 +162,12 @@ def test_request_log_masking(request_logger: RequestLogger, request_dict: Reques
     request_dict["headers"] += [("Authorization", "Bearer 123456"), ("X-Test", "123456")]
     request_logger.log_request(request_dict, response_dict)
 
-    item = json.loads(request_logger.write_deque[0])
+    item = request_logger.write_deque[0]
     assert item["request"]["url"] == f"http://localhost/test?secret={MASKED_QUOTED}&test={MASKED_QUOTED}&other=abcdef"
-    assert ["Authorization", "Bearer 123456"] not in item["request"]["headers"]
-    assert ["Authorization", MASKED] in item["request"]["headers"]
-    assert ["X-Test", "123456"] not in item["request"]["headers"]
-    assert ["X-Test", MASKED] in item["request"]["headers"]
-    assert ["Accept", "text/plain"] in item["request"]["headers"]
-    assert item["request"]["body"] == base64.b64encode(BODY_MASKED).decode()
-    assert item["response"]["body"] == base64.b64encode(BODY_MASKED).decode()
+    assert ("Authorization", "Bearer 123456") not in item["request"]["headers"]
+    assert ("Authorization", MASKED) in item["request"]["headers"]
+    assert ("X-Test", "123456") not in item["request"]["headers"]
+    assert ("X-Test", MASKED) in item["request"]["headers"]
+    assert ("Accept", "text/plain") in item["request"]["headers"]
+    assert item["request"]["body"] == BODY_MASKED
+    assert item["response"]["body"] == BODY_MASKED

{apitally-0.14.6 → apitally-0.15.0}/tests/test_client_server_errors.py

@@ -29,7 +29,11 @@ def test_server_error_counter():
 
 
 def test_exception_truncation(mocker: MockerFixture):
-    from apitally.client.server_errors import ServerErrorCounter
+    from apitally.client.server_errors import (
+        get_exception_type,
+        get_truncated_exception_msg,
+        get_truncated_exception_traceback,
+    )
 
     mocker.patch("apitally.client.server_errors.MAX_EXCEPTION_MSG_LENGTH", 32)
     mocker.patch("apitally.client.server_errors.MAX_EXCEPTION_TRACEBACK_LENGTH", 128)
@@ -37,9 +41,11 @@ def test_exception_truncation(mocker: MockerFixture):
     try:
         raise ValueError("a" * 88)
     except ValueError as e:
-        msg = ServerErrorCounter._get_truncated_exception_msg(e)
-        tb = ServerErrorCounter._get_truncated_exception_traceback(e)
+        type_ = get_exception_type(e)
+        msg = get_truncated_exception_msg(e)
+        tb = get_truncated_exception_traceback(e)
 
+    assert type_ == "builtins.ValueError"
     assert len(msg) == 32
     assert msg.endswith("... (truncated)")
     assert len(tb) <= 128