beacon-profiling 1.2.0__tar.gz → 1.2.2__tar.gz

{beacon_profiling-1.2.0 → beacon_profiling-1.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: beacon-profiling
-Version: 1.2.0
+Version: 1.2.2
 Summary: Beacon Profiling for OpenTelemetry Python
 Project-URL: Homepage, https://github.com/beacon-observability/beacon-python/tree/main/sdk-extension/beacon-profiling
 Project-URL: Repository, https://github.com/beacon-observability/beacon-python
@@ -18,17 +18,17 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
 Requires-Python: >=3.10
-Requires-Dist: opentelemetry-api==1.44.0
-Requires-Dist: opentelemetry-exporter-otlp==1.44.0
-Requires-Dist: opentelemetry-instrumentation==0.65b0
-Requires-Dist: opentelemetry-sdk==1.44.0
+Requires-Dist: opentelemetry-api==1.45.1
+Requires-Dist: opentelemetry-exporter-otlp==1.45.1
+Requires-Dist: opentelemetry-instrumentation==0.66b1
+Requires-Dist: opentelemetry-sdk==1.45.1
 Requires-Dist: requests>=2.20
 Description-Content-Type: text/x-rst
 
 Beacon Profiling for Python
 ===========================
 
-The current stable version is ``beacon-profiling==1.1.0``.
+The current stable version is ``beacon-profiling==1.2.1``.
 
 This package provides a Python profiling runtime for OpenTelemetry
 auto-instrumentation. The current implementation exports profiles via OTLP
@@ -44,6 +44,13 @@ or a compatible pprof endpoint and supports multiple collector types:
 Handled-exception collection requires Python ``sys.monitoring`` (Python
 3.12+); on Python 3.10/3.11 the other collectors remain available.
 
+Trace-correlated samples retain the current span ID separately from the
+local root span ID. Endpoint and span kind come from that local root, so a
+database child span does not replace the request endpoint. Root identity
+travels with the OpenTelemetry context across tasks and propagated threads.
+If profiling starts inside a pre-existing local child span whose root was
+not observed, the root and endpoint remain unknown.
+
 Usage
 -----
 
@@ -75,11 +82,17 @@ The following environment variables are supported:
 * ``OTEL_PROFILING_EXPORT_INTERVAL`` (defaults to 60 seconds and can be
   overridden by the ``Profiler(export_interval=...)`` argument)
 * ``OTEL_PROFILING_MAX_FRAMES``
+* ``OTEL_PROFILING_MAX_BUFFER_SAMPLES`` (defaults to 10000)
+* ``OTEL_PROFILING_MAX_BUFFER_BYTES`` (defaults to 8388608 accounted bytes)
 * ``OTEL_PROFILING_INCLUDE_TRACE_CONTEXT``
 * ``OTEL_PROFILING_EXCEPTION_ENABLED``
 * ``OTEL_PROFILING_EXCEPTION_SAMPLING_INTERVAL``
 * ``OTEL_PROFILING_EXCEPTION_COLLECT_MESSAGE``
 * ``OTEL_PROFILING_LOCK_ENABLED``
+* ``OTEL_PROFILING_LOCK_SAMPLING_INTERVAL`` (defaults to 100; one in N
+  short acquisitions is sampled; 1 records every operation)
+* ``OTEL_PROFILING_LOCK_MIN_DURATION_NS`` (defaults to 1000000; operations
+  at or above this duration are recorded without probabilistic sampling)
 * ``OTEL_PROFILING_MEMORY_ENABLED``
 * ``OTEL_PROFILING_MEMORY_INTERVAL`` (defaults to the profile export interval
   and can be overridden by the ``Profiler(memory_interval=...)`` argument)
@@ -94,6 +107,8 @@ The following environment variables are supported:
 * ``OTEL_EXPORTER_OTLP_PROFILES_CLIENT_KEY``
 * ``OTEL_EXPORTER_OTLP_PROFILES_CLIENT_CERTIFICATE``
 * ``OTEL_PROFILING_PPROF_PATH``
+* ``OTEL_PROFILING_PPROF_MAX_FILES`` (HTTP upload local copies only; defaults
+  to 3 retained files per exporter instance)
 * ``OTEL_PROFILING_PPROF_UPLOAD_URL``
 * ``OTEL_PROFILING_PPROF_HEADERS``
 
@@ -101,6 +116,28 @@ Service metadata should be configured with standard OpenTelemetry
 resource settings such as ``OTEL_SERVICE_NAME`` and
 ``OTEL_RESOURCE_ATTRIBUTES``.
 
+Short lock acquisition/hold pairs share one sampling decision. Slow holds
+and waits are retained even when their acquisition was not sampled. Sampling
+is decided before stack walking and trace lookup; filename normalization
+uses a bounded cache. OTLP samples carry their interval as ``period`` and
+``lock.sampling.interval``. Compatible pprof lock counts and durations are
+weighted by that interval; they are estimates when the interval exceeds 1.
+
+One profiler-wide budget covers raw samples in all collectors and the
+pending export buffer. Both limits also accept ``Profiler(max_buffer_samples=,
+max_buffer_bytes=)`` arguments. Excess samples are dropped, counted in
+``profiler.buffer_statistics``, and reported in a warning at flush. The next
+window can collect again after draining; delivery is bounded, not lossless.
+Retained samples preserve their individual timestamps, trace identity, values,
+and CPU deltas. Immutable stack frames use a bounded cache.
+
+Byte accounting conservatively includes sample metadata and frame references,
+including shared objects. It bounds queued raw data, not total process RSS;
+heap snapshots and serialization also need temporary memory. Capture/build/
+export work is excluded from the profiler's own exception and lock collection.
+``memory_interval=0`` keeps manual memory capture available on each call;
+periodic memory scheduling uses the positive export cadence instead of spinning.
+
 Examples
 --------
 
@@ -131,8 +168,19 @@ copy:
 
     export OTEL_PROFILING_PPROF_UPLOAD_URL=http://localhost:9529/profiling/v1/input
     export OTEL_PROFILING_PPROF_HEADERS="X-API-Key:xxx"
+    export OTEL_PROFILING_PPROF_PATH=/tmp/otel-profile
     beacon python app.py
 
+HTTP upload does not write local files unless ``OTEL_PROFILING_PPROF_PATH``
+is set. An empty ``pprof_path`` argument disables copies even when that
+environment variable is set. Local write/cleanup failures are logged and do
+not prevent upload. Retention removes only files written by this exporter
+instance; it never scans or removes files owned by other processes. The
+file-only ``pprof`` exporter retains its existing output behavior.
+
+Capture and export exceptions are logged; subsequent scheduled captures
+and export attempts continue. Failed batches are not replayed automatically.
+
 When ``OTEL_PROFILING_EXPORTER`` is unset and
 ``OTEL_PROFILING_PPROF_UPLOAD_URL`` is configured, the runtime
 defaults to a legacy-compatible ``pprof`` upload layout. Set

{beacon_profiling-1.2.0 → beacon_profiling-1.2.2}/README.rst

@@ -1,7 +1,7 @@
 Beacon Profiling for Python
 ===========================
 
-The current stable version is ``beacon-profiling==1.1.0``.
+The current stable version is ``beacon-profiling==1.2.1``.
 
 This package provides a Python profiling runtime for OpenTelemetry
 auto-instrumentation. The current implementation exports profiles via OTLP
@@ -17,6 +17,13 @@ or a compatible pprof endpoint and supports multiple collector types:
 Handled-exception collection requires Python ``sys.monitoring`` (Python
 3.12+); on Python 3.10/3.11 the other collectors remain available.
 
+Trace-correlated samples retain the current span ID separately from the
+local root span ID. Endpoint and span kind come from that local root, so a
+database child span does not replace the request endpoint. Root identity
+travels with the OpenTelemetry context across tasks and propagated threads.
+If profiling starts inside a pre-existing local child span whose root was
+not observed, the root and endpoint remain unknown.
+
 Usage
 -----
 
@@ -48,11 +55,17 @@ The following environment variables are supported:
 * ``OTEL_PROFILING_EXPORT_INTERVAL`` (defaults to 60 seconds and can be
   overridden by the ``Profiler(export_interval=...)`` argument)
 * ``OTEL_PROFILING_MAX_FRAMES``
+* ``OTEL_PROFILING_MAX_BUFFER_SAMPLES`` (defaults to 10000)
+* ``OTEL_PROFILING_MAX_BUFFER_BYTES`` (defaults to 8388608 accounted bytes)
 * ``OTEL_PROFILING_INCLUDE_TRACE_CONTEXT``
 * ``OTEL_PROFILING_EXCEPTION_ENABLED``
 * ``OTEL_PROFILING_EXCEPTION_SAMPLING_INTERVAL``
 * ``OTEL_PROFILING_EXCEPTION_COLLECT_MESSAGE``
 * ``OTEL_PROFILING_LOCK_ENABLED``
+* ``OTEL_PROFILING_LOCK_SAMPLING_INTERVAL`` (defaults to 100; one in N
+  short acquisitions is sampled; 1 records every operation)
+* ``OTEL_PROFILING_LOCK_MIN_DURATION_NS`` (defaults to 1000000; operations
+  at or above this duration are recorded without probabilistic sampling)
 * ``OTEL_PROFILING_MEMORY_ENABLED``
 * ``OTEL_PROFILING_MEMORY_INTERVAL`` (defaults to the profile export interval
   and can be overridden by the ``Profiler(memory_interval=...)`` argument)
@@ -67,6 +80,8 @@ The following environment variables are supported:
 * ``OTEL_EXPORTER_OTLP_PROFILES_CLIENT_KEY``
 * ``OTEL_EXPORTER_OTLP_PROFILES_CLIENT_CERTIFICATE``
 * ``OTEL_PROFILING_PPROF_PATH``
+* ``OTEL_PROFILING_PPROF_MAX_FILES`` (HTTP upload local copies only; defaults
+  to 3 retained files per exporter instance)
 * ``OTEL_PROFILING_PPROF_UPLOAD_URL``
 * ``OTEL_PROFILING_PPROF_HEADERS``
 
@@ -74,6 +89,28 @@ Service metadata should be configured with standard OpenTelemetry
 resource settings such as ``OTEL_SERVICE_NAME`` and
 ``OTEL_RESOURCE_ATTRIBUTES``.
 
+Short lock acquisition/hold pairs share one sampling decision. Slow holds
+and waits are retained even when their acquisition was not sampled. Sampling
+is decided before stack walking and trace lookup; filename normalization
+uses a bounded cache. OTLP samples carry their interval as ``period`` and
+``lock.sampling.interval``. Compatible pprof lock counts and durations are
+weighted by that interval; they are estimates when the interval exceeds 1.
+
+One profiler-wide budget covers raw samples in all collectors and the
+pending export buffer. Both limits also accept ``Profiler(max_buffer_samples=,
+max_buffer_bytes=)`` arguments. Excess samples are dropped, counted in
+``profiler.buffer_statistics``, and reported in a warning at flush. The next
+window can collect again after draining; delivery is bounded, not lossless.
+Retained samples preserve their individual timestamps, trace identity, values,
+and CPU deltas. Immutable stack frames use a bounded cache.
+
+Byte accounting conservatively includes sample metadata and frame references,
+including shared objects. It bounds queued raw data, not total process RSS;
+heap snapshots and serialization also need temporary memory. Capture/build/
+export work is excluded from the profiler's own exception and lock collection.
+``memory_interval=0`` keeps manual memory capture available on each call;
+periodic memory scheduling uses the positive export cadence instead of spinning.
+
 Examples
 --------
 
@@ -104,8 +141,19 @@ copy:
 
     export OTEL_PROFILING_PPROF_UPLOAD_URL=http://localhost:9529/profiling/v1/input
     export OTEL_PROFILING_PPROF_HEADERS="X-API-Key:xxx"
+    export OTEL_PROFILING_PPROF_PATH=/tmp/otel-profile
     beacon python app.py
 
+HTTP upload does not write local files unless ``OTEL_PROFILING_PPROF_PATH``
+is set. An empty ``pprof_path`` argument disables copies even when that
+environment variable is set. Local write/cleanup failures are logged and do
+not prevent upload. Retention removes only files written by this exporter
+instance; it never scans or removes files owned by other processes. The
+file-only ``pprof`` exporter retains its existing output behavior.
+
+Capture and export exceptions are logged; subsequent scheduled captures
+and export attempts continue. Failed batches are not replayed automatically.
+
 When ``OTEL_PROFILING_EXPORTER`` is unset and
 ``OTEL_PROFILING_PPROF_UPLOAD_URL`` is configured, the runtime
 defaults to a legacy-compatible ``pprof`` upload layout. Set

{beacon_profiling-1.2.0 → beacon_profiling-1.2.2}/pyproject.toml

@@ -25,10 +25,10 @@ classifiers = [
   "Programming Language :: Python :: 3.14",
 ]
 dependencies = [
-  "opentelemetry-api == 1.44.0",
-  "opentelemetry-sdk == 1.44.0",
-  "opentelemetry-instrumentation == 0.65b0",
-  "opentelemetry-exporter-otlp == 1.44.0",
+  "opentelemetry-api == 1.45.1",
+  "opentelemetry-sdk == 1.45.1",
+  "opentelemetry-instrumentation == 0.66b1",
+  "opentelemetry-exporter-otlp == 1.45.1",
   "requests >= 2.20",
 ]
 

{beacon_profiling-1.2.0 → beacon_profiling-1.2.2}/src/opentelemetry/sdk/extension/profiling/collector/base.py

@@ -15,6 +15,7 @@
 from __future__ import annotations
 
 from dataclasses import dataclass
+from functools import lru_cache
 from typing import Protocol
 
 
@@ -25,6 +26,17 @@ class CapturedFrame:
     lineno: int
 
 
+@lru_cache(maxsize=4096)
+def _cached_frame(function: str, filename: str, lineno: int) -> CapturedFrame:
+    return CapturedFrame(function, filename, lineno)
+
+
+def captured_frame(function: str, filename: str, lineno: int) -> CapturedFrame:
+    if len(function) <= 512 and len(filename) <= 2048:
+        return _cached_frame(function, filename, lineno)
+    return CapturedFrame(function, filename, lineno)
+
+
 @dataclass(frozen=True, slots=True)
 class CapturedAttribute:
     key: str

beacon_profiling-1.2.2/src/opentelemetry/sdk/extension/profiling/collector/buffering.py

@@ -0,0 +1,207 @@
+from __future__ import annotations
+
+import sys
+from _thread import allocate_lock
+from contextlib import contextmanager
+from contextvars import ContextVar
+from dataclasses import fields
+from functools import lru_cache
+from os import environ
+from typing import Iterator
+
+from opentelemetry.sdk.extension.profiling.collector.base import (
+    CapturedAttribute,
+    CapturedFrame,
+    CapturedSample,
+)
+
+_INTERNAL = ContextVar("beacon_profiling_internal_work", default=False)
+_FIELD_NAMES = {
+    cls: tuple(field.name for field in fields(cls))
+    for cls in (CapturedSample, CapturedAttribute)
+}
+
+
+@lru_cache(maxsize=4096)
+def _frame_size(frame: CapturedFrame) -> int:
+    return (
+        sys.getsizeof(frame)
+        + sys.getsizeof(frame.function)
+        + sys.getsizeof(frame.filename)
+        + sys.getsizeof(frame.lineno)
+    )
+
+
+def internal_work() -> bool:
+    return _INTERNAL.get()
+
+
+@contextmanager
+def suppress_collection() -> Iterator[None]:
+    token = _INTERNAL.set(True)
+    try:
+        yield
+    finally:
+        _INTERNAL.reset(token)
+
+
+def _sample_size(sample: CapturedSample, limit: int) -> int:
+    # Charge shared frame/string references conservatively per sample. The
+    # budget bounds producer-owned raw data, not application heap snapshots
+    # or temporary serialized messages. Never invoke user __sizeof__ hooks.
+    pending: list[object] = [sample]
+    seen: set[int] = set()
+    total = 96  # Buffer entry and accounting overhead.
+    while pending:
+        value = pending.pop()
+        if id(value) in seen:
+            continue
+        seen.add(id(value))
+        if len(seen) > 4096:
+            return limit + 1
+        cls = type(value)
+        if cls is CapturedFrame:
+            if (
+                type(value.function) is not str
+                or type(value.filename) is not str
+                or type(value.lineno) is not int
+            ):
+                return limit + 1
+            if len(value.function) <= 512 and len(value.filename) <= 2048:
+                total += _frame_size(value)
+            else:
+                total += sum(
+                    sys.getsizeof(item)
+                    for item in (
+                        value,
+                        value.function,
+                        value.filename,
+                        value.lineno,
+                    )
+                )
+            if total > limit:
+                return total
+            continue
+        if cls is CapturedSample or cls is CapturedAttribute:
+            pending.extend(getattr(value, name) for name in _FIELD_NAMES[cls])
+        elif cls is tuple or cls is list:
+            if len(value) > 4096:
+                return limit + 1
+            pending.extend(value)
+        elif cls is dict:
+            if len(value) > 2048:
+                return limit + 1
+            pending.extend(value.keys())
+            pending.extend(value.values())
+        elif not any(
+            cls is scalar
+            for scalar in (str, bytes, bytearray, int, float, bool, type(None))
+        ):
+            return limit + 1
+        total += sys.getsizeof(value)
+        if total > limit:
+            return total
+    return total
+
+
+class SampleBudget:
+    def __init__(
+        self, max_samples: int | None = None, max_bytes: int | None = None
+    ) -> None:
+        self.max_samples = (
+            max_samples
+            if max_samples is not None
+            else int(environ.get("OTEL_PROFILING_MAX_BUFFER_SAMPLES", "10000"))
+        )
+        self.max_bytes = (
+            max_bytes
+            if max_bytes is not None
+            else int(
+                environ.get(
+                    "OTEL_PROFILING_MAX_BUFFER_BYTES", str(8 * 1024 * 1024)
+                )
+            )
+        )
+        if self.max_samples < 1 or self.max_bytes < 1:
+            raise ValueError("profile buffer limits must be positive")
+        self.reset_after_fork()
+
+    def reset_after_fork(self) -> None:
+        self._lock = allocate_lock()
+        self.generation = getattr(self, "generation", 0) + 1
+        self._used_samples = 0
+        self._used_bytes = 0
+        self._dropped_samples = 0
+
+    def reserve(self, sample: CapturedSample, generation: int) -> int | None:
+        size = _sample_size(sample, self.max_bytes)
+        with self._lock:
+            if (
+                generation != self.generation
+                or self._used_samples >= self.max_samples
+                or self._used_bytes + size > self.max_bytes
+            ):
+                self._dropped_samples += 1
+                return None
+            self._used_samples += 1
+            self._used_bytes += size
+            return size
+
+    def release(self, count: int, size: int, generation: int) -> None:
+        with self._lock:
+            if generation != self.generation:
+                return
+            self._used_samples -= count
+            self._used_bytes -= size
+
+    def snapshot(self) -> dict[str, int]:
+        with self._lock:
+            return {
+                "max_samples": self.max_samples,
+                "max_bytes": self.max_bytes,
+                "used_samples": self._used_samples,
+                "used_bytes": self._used_bytes,
+                "dropped_samples": self._dropped_samples,
+            }
+
+
+class SampleBuffer:
+    """Caller-locked buffer sharing a single profiler-wide raw-data budget."""
+
+    def __init__(self, budget: SampleBudget) -> None:
+        self._budget = budget
+        self._generation = budget.generation
+        self._items: list[tuple[CapturedSample, int]] = []
+
+    def __len__(self) -> int:
+        return len(self._items)
+
+    def append(self, sample: CapturedSample) -> None:
+        size = self._budget.reserve(sample, self._generation)
+        if size is not None:
+            try:
+                self._items.append((sample, size))
+            except BaseException:
+                self._budget.release(1, size, self._generation)
+                raise
+
+    def extend(self, samples: list[CapturedSample]) -> None:
+        for sample in samples:
+            self.append(sample)
+
+    def drain(self) -> list[CapturedSample]:
+        result = [sample for sample, _ in self._items]
+        self.clear()
+        return result
+
+    def clear(self) -> None:
+        items, self._items = self._items, []
+        self._budget.release(
+            len(items), sum(size for _, size in items), self._generation
+        )
+
+    def __enter__(self) -> SampleBuffer:
+        return self
+
+    def __exit__(self, *_: object) -> None:
+        self.clear()

{beacon_profiling-1.2.0 → beacon_profiling-1.2.2}/src/opentelemetry/sdk/extension/profiling/collector/exception.py

@@ -27,6 +27,12 @@ from opentelemetry.sdk.extension.profiling.collector.base import (
     CapturedAttribute,
     CapturedFrame,
     CapturedSample,
+    captured_frame,
+)
+from opentelemetry.sdk.extension.profiling.collector.buffering import (
+    SampleBudget,
+    SampleBuffer,
+    internal_work,
 )
 from opentelemetry.sdk.extension.profiling.context_bridge import ContextBridge
 from opentelemetry.trace import INVALID_SPAN_CONTEXT
@@ -66,17 +72,22 @@ class ExceptionCollector:
         include_trace_context: bool = True,
         sampling_interval: int = 100,
         collect_message: bool = False,
+        sample_budget: SampleBudget | None = None,
     ) -> None:
         if sampling_interval < 1:
             raise ValueError("sampling_interval must be >= 1")
 
+        self._owns_sample_budget = sample_budget is None
+        self._sample_budget = (
+            sample_budget if sample_budget is not None else SampleBudget()
+        )
         self._context_bridge = context_bridge
         self._max_frames = max_frames
         self._include_trace_context = include_trace_context
         self._sampling_interval = sampling_interval
         self._collect_message = collect_message
 
-        self._buffer: list[CapturedSample] = []
+        self._buffer = SampleBuffer(self._sample_budget)
         self._buffer_lock = threading.Lock()
         self._sampler = _PoissonSampler()
         self._counter = 0
@@ -134,24 +145,26 @@ class ExceptionCollector:
 
     def stop(self) -> None:
         tool_id = self._monitoring_tool_id
+        with self._buffer_lock:
+            self._monitoring_tool_id = None
+            self._buffer.clear()
         if tool_id is None:
             return
 
         self._cleanup_monitoring(tool_id)
-        self._monitoring_tool_id = None
 
     def capture(self) -> list[CapturedSample]:
         with self._buffer_lock:
-            captured = self._buffer
-            self._buffer = []
-        return captured
+            return self._buffer.drain()
 
     def reset_after_fork(self) -> None:
         tool_id = self._monitoring_tool_id
         if tool_id is not None:
             self._cleanup_monitoring(tool_id)
         self._buffer_lock = threading.Lock()
-        self._buffer = []
+        if self._owns_sample_budget:
+            self._sample_budget.reset_after_fork()
+        self._buffer = SampleBuffer(self._sample_budget)
         self._counter = 0
         self._next_sample = self._sampling_interval
         self._collecting_state = threading.local()
@@ -197,7 +210,10 @@ class ExceptionCollector:
     ) -> None:
         del code, instruction_offset
 
-        if self._is_collecting():
+        if self._is_collecting() or internal_work():
+            return
+        tool_id = self._monitoring_tool_id
+        if tool_id is None:
             return
 
         self._counter += 1
@@ -213,7 +229,8 @@ class ExceptionCollector:
             if sample is None:
                 return
             with self._buffer_lock:
-                self._buffer.append(sample)
+                if self._monitoring_tool_id == tool_id:
+                    self._buffer.append(sample)
         except Exception:
             _logger.debug("Failed to capture handled exception", exc_info=True)
         finally:
@@ -287,7 +304,7 @@ class ExceptionCollector:
         while current is not None:
             frame = current.tb_frame
             frames.append(
-                CapturedFrame(
+                captured_frame(
                     function=frame.f_code.co_name,
                     filename=frame.f_code.co_filename,
                     lineno=current.tb_lineno,