sentinel-probes 0.1.0__tar.gz

sentinel_probes-0.1.0/PKG-INFO

@@ -0,0 +1,127 @@
+Metadata-Version: 2.4
+Name: sentinel-probes
+Version: 0.1.0
+Summary: Sentinel Dynamic Instrumentation for Python: metric, span and span-tag probes placed in running code without a redeploy.
+Author-email: A K M Muhibullah Nayem <nayem.drmc@gmail.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/MuhibNayem/sightglass/tree/main/clients/python
+Project-URL: Source, https://github.com/MuhibNayem/sightglass
+Project-URL: Issues, https://github.com/MuhibNayem/sightglass/issues
+Keywords: sentinel,dynamic-instrumentation,observability,opentelemetry,apm,probes
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: System :: Monitoring
+Classifier: Topic :: Software Development :: Debuggers
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+
+# sentinel-probes: Dynamic Instrumentation for Python
+
+Metric, span and span-tag probes on a running Python service, added, changed
+and removed from [Sentinel](https://github.com/MuhibNayem/sightglass) without a deploy. Standard library only.
+
+## Start it
+
+With no code change, the way `ddtrace-run` and `opentelemetry-instrument` work.
+Every Python process the command starts runs the client, including
+pre-forked workers:
+
+```sh
+pip install sentinel-probes
+
+SENTINEL_PROBES_URL=https://ingest.example.com/otlp/v1/probes \
+SENTINEL_API_KEY=<ingest key> \
+OTEL_SERVICE_NAME=checkout \
+OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production,service.version=1.4.2 \
+OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 \
+sentinel-probes-run gunicorn app:app
+```
+
+`SENTINEL_SERVICE`, `SENTINEL_ENV`, `SENTINEL_VERSION` and
+`SENTINEL_OTLP_ENDPOINT` take precedence over the OpenTelemetry variables.
+Without `SENTINEL_PROBES_URL` the client stays off. With the URL set but the
+rest of the configuration incomplete, it logs one warning and stays off.
+
+Or start it from code:
+
+```python
+import sentinel_probes
+
+sentinel_probes.start_probes(
+    service="checkout",
+    env="production",
+    api_key="<ingest key>",
+    version="1.4.2",
+    probes_url="https://ingest.example.com/otlp/v1/probes",
+    otlp_endpoint="http://otel-collector:4318",
+)
+```
+
+## How it stays cheap
+
+- **Python 3.12+:** probes are placed with `sys.monitoring` local events, only
+  on the targeted code objects. Every other function runs untouched.
+- **Rate limit:** each probe runs at most 50 times a second. Once a probe has
+  used up that allowance, its code object's events are switched off for the rest
+  of the second, so a hot method runs at full speed between its sampled calls.
+- **Before 3.12:** a method probe wraps the function. The signature is computed
+  once, the rate limit is checked before arguments are bound, and once the
+  allowance is used up the wrapper skips the lock entirely.
+- **CPU budget:** probes may use at most 2% of a second's CPU time. Beyond
+  that, they're refused until the next second.
+- **Off the application's threads:** a daemon thread polls, reports and exports.
+  It restarts in forked children with a new runtime id.
+- **Raw-socket HTTP:** the client talks over a plain socket, not urllib,
+  http.client or requests, so OpenTelemetry's and ddtrace's instrumentation
+  never see its traffic. Each call is time-boxed and responses are capped at
+  1 MiB.
+- **Capture runs no application code:**
+  - attributes are read through descriptors, never via properties,
+    `__getattr__` or `__str__`;
+  - an exception's message is read only when its class keeps
+    `BaseException.__str__`;
+  - only exact standard-library types (`datetime`, `UUID`, `Decimal`, …) are
+    turned into text.
+- **Bounded and redacted:** capture is bounded (depth 2, 20 entries, 256
+  characters, 30 variables) and redacted by name (`password`, `token`,
+  `session`, …, plus the probe's keywords).
+- **Never into the application:** a failure becomes the probe's error.
+
+Per call of a probed method (`tests/bench.py`). The probe is a gauge of the
+return value; most calls are past the 50-a-second limit:
+
+| Python                  | plain call | with the probe |    added |
+| ----------------------- | ---------: | -------------: | -------: |
+| 3.13 (`sys.monitoring`) |    0.10 µs |        0.10 µs | +0.01 µs |
+| 3.11 (function wrapper) |    0.09 µs |        0.51 µs | +0.43 µs |
+
+## What runs where
+
+| Python   | Method probes | Line probes                                                                                                             |
+| -------- | ------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| 3.12+    | yes           | yes                                                                                                                     |
+| 3.8–3.11 | yes           | no: the only way is `sys.settrace`, which taxes every call in every thread. A line probe reports an error that says so. |
+
+The client advertises what it can place (`locations=line,method` or `method`).
+
+Before 3.12, the wrapper replaces the function where it is defined. Code that
+imported it by name before the probe was placed (`from orders import total`)
+keeps calling the original, and a nested function can't be reached at all; the
+probe reports which.
+
+## Tests
+
+```sh
+python -m pytest -q                            # unit, conformance (79 cases, 3,030 patterns), hardening
+docker run --rm -v "$PWD":/w/python:ro -w /w/python/tests/e2e python:3.11-slim sh run.sh
+docker run --rm -v "$PWD":/w/python:ro -w /w/python/tests/e2e python:3.13-slim sh run.sh
+docker run --rm -v "$PWD":/w/python:ro -w /w/python python:3.13-slim python tests/bench.py
+```
+
+The end-to-end run starts an unmodified application with
+`python -m sentinel_probes.run` against a fake backend. It checks the poll,
+statuses (including the line-probe refusal on 3.11), a redacted sample and the
+OTLP metrics.

sentinel_probes-0.1.0/README.md

@@ -0,0 +1,107 @@
+# sentinel-probes: Dynamic Instrumentation for Python
+
+Metric, span and span-tag probes on a running Python service, added, changed
+and removed from [Sentinel](https://github.com/MuhibNayem/sightglass) without a deploy. Standard library only.
+
+## Start it
+
+With no code change, the way `ddtrace-run` and `opentelemetry-instrument` work.
+Every Python process the command starts runs the client, including
+pre-forked workers:
+
+```sh
+pip install sentinel-probes
+
+SENTINEL_PROBES_URL=https://ingest.example.com/otlp/v1/probes \
+SENTINEL_API_KEY=<ingest key> \
+OTEL_SERVICE_NAME=checkout \
+OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production,service.version=1.4.2 \
+OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 \
+sentinel-probes-run gunicorn app:app
+```
+
+`SENTINEL_SERVICE`, `SENTINEL_ENV`, `SENTINEL_VERSION` and
+`SENTINEL_OTLP_ENDPOINT` take precedence over the OpenTelemetry variables.
+Without `SENTINEL_PROBES_URL` the client stays off. With the URL set but the
+rest of the configuration incomplete, it logs one warning and stays off.
+
+Or start it from code:
+
+```python
+import sentinel_probes
+
+sentinel_probes.start_probes(
+    service="checkout",
+    env="production",
+    api_key="<ingest key>",
+    version="1.4.2",
+    probes_url="https://ingest.example.com/otlp/v1/probes",
+    otlp_endpoint="http://otel-collector:4318",
+)
+```
+
+## How it stays cheap
+
+- **Python 3.12+:** probes are placed with `sys.monitoring` local events, only
+  on the targeted code objects. Every other function runs untouched.
+- **Rate limit:** each probe runs at most 50 times a second. Once a probe has
+  used up that allowance, its code object's events are switched off for the rest
+  of the second, so a hot method runs at full speed between its sampled calls.
+- **Before 3.12:** a method probe wraps the function. The signature is computed
+  once, the rate limit is checked before arguments are bound, and once the
+  allowance is used up the wrapper skips the lock entirely.
+- **CPU budget:** probes may use at most 2% of a second's CPU time. Beyond
+  that, they're refused until the next second.
+- **Off the application's threads:** a daemon thread polls, reports and exports.
+  It restarts in forked children with a new runtime id.
+- **Raw-socket HTTP:** the client talks over a plain socket, not urllib,
+  http.client or requests, so OpenTelemetry's and ddtrace's instrumentation
+  never see its traffic. Each call is time-boxed and responses are capped at
+  1 MiB.
+- **Capture runs no application code:**
+  - attributes are read through descriptors, never via properties,
+    `__getattr__` or `__str__`;
+  - an exception's message is read only when its class keeps
+    `BaseException.__str__`;
+  - only exact standard-library types (`datetime`, `UUID`, `Decimal`, …) are
+    turned into text.
+- **Bounded and redacted:** capture is bounded (depth 2, 20 entries, 256
+  characters, 30 variables) and redacted by name (`password`, `token`,
+  `session`, …, plus the probe's keywords).
+- **Never into the application:** a failure becomes the probe's error.
+
+Per call of a probed method (`tests/bench.py`). The probe is a gauge of the
+return value; most calls are past the 50-a-second limit:
+
+| Python                  | plain call | with the probe |    added |
+| ----------------------- | ---------: | -------------: | -------: |
+| 3.13 (`sys.monitoring`) |    0.10 µs |        0.10 µs | +0.01 µs |
+| 3.11 (function wrapper) |    0.09 µs |        0.51 µs | +0.43 µs |
+
+## What runs where
+
+| Python   | Method probes | Line probes                                                                                                             |
+| -------- | ------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| 3.12+    | yes           | yes                                                                                                                     |
+| 3.8–3.11 | yes           | no: the only way is `sys.settrace`, which taxes every call in every thread. A line probe reports an error that says so. |
+
+The client advertises what it can place (`locations=line,method` or `method`).
+
+Before 3.12, the wrapper replaces the function where it is defined. Code that
+imported it by name before the probe was placed (`from orders import total`)
+keeps calling the original, and a nested function can't be reached at all; the
+probe reports which.
+
+## Tests
+
+```sh
+python -m pytest -q                            # unit, conformance (79 cases, 3,030 patterns), hardening
+docker run --rm -v "$PWD":/w/python:ro -w /w/python/tests/e2e python:3.11-slim sh run.sh
+docker run --rm -v "$PWD":/w/python:ro -w /w/python/tests/e2e python:3.13-slim sh run.sh
+docker run --rm -v "$PWD":/w/python:ro -w /w/python python:3.13-slim python tests/bench.py
+```
+
+The end-to-end run starts an unmodified application with
+`python -m sentinel_probes.run` against a fake backend. It checks the poll,
+statuses (including the line-probe refusal on 3.11), a redacted sample and the
+OTLP metrics.

sentinel_probes-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[build-system]
+requires = ["setuptools>=77.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "sentinel-probes"
+version = "0.1.0"
+description = "Sentinel Dynamic Instrumentation for Python: metric, span and span-tag probes placed in running code without a redeploy."
+readme = "README.md"
+requires-python = ">=3.8"
+dependencies = []
+license = "Apache-2.0"
+authors = [{ name = "A K M Muhibullah Nayem", email = "nayem.drmc@gmail.com" }]
+keywords = ["sentinel", "dynamic-instrumentation", "observability", "opentelemetry", "apm", "probes"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+    "Topic :: System :: Monitoring",
+    "Topic :: Software Development :: Debuggers",
+]
+
+[project.urls]
+Homepage = "https://github.com/MuhibNayem/sightglass/tree/main/clients/python"
+Source = "https://github.com/MuhibNayem/sightglass"
+Issues = "https://github.com/MuhibNayem/sightglass/issues"
+
+[project.scripts]
+sentinel-probes-run = "sentinel_probes.run:main"
+
+[tool.setuptools]
+packages = ["sentinel_probes", "sentinel_probes.bootstrap"]
+
+[tool.pytest.ini_options]
+pythonpath = ["."]
+testpaths = ["tests"]

sentinel_probes-0.1.0/sentinel_probes/__init__.py

@@ -0,0 +1,48 @@
+"""
+Sentinel Dynamic Instrumentation Client for Python.
+
+Zero external dependencies, ReDoS-safe linear regex engine, safe variable capture
+without descriptor/property execution, CPU governor, and PEP 669 sys.monitoring.
+"""
+
+from .runtime import start_probes, CLIENT_VERSION, Runtime
+from .evaluator import (
+    evaluate,
+    validate,
+    render_tag,
+    reads,
+    text,
+    ProbeError,
+    Capture,
+    AST_VERSION,
+)
+from .pattern import compile_pattern, compiled, run_pattern, code_points
+from .capture import capture_frame, capture_locals, value_of, scrubbed
+from .instrumenter import Instrumenter
+from .spans import active_span
+from . import http
+
+__all__ = [
+    "start_probes",
+    "CLIENT_VERSION",
+    "Runtime",
+    "evaluate",
+    "validate",
+    "render_tag",
+    "reads",
+    "text",
+    "ProbeError",
+    "Capture",
+    "AST_VERSION",
+    "compile_pattern",
+    "compiled",
+    "run_pattern",
+    "code_points",
+    "capture_frame",
+    "capture_locals",
+    "value_of",
+    "scrubbed",
+    "Instrumenter",
+    "active_span",
+    "http",
+]

sentinel_probes-0.1.0/sentinel_probes/autostart.py

@@ -0,0 +1,67 @@
+"""
+Start the client from the environment, with no code change: what
+`sentinel-probes-run` (and the sitecustomize it installs) calls.
+
+    SENTINEL_PROBES_URL      https://sightglass.example.com/otlp/v1/probes
+    SENTINEL_API_KEY         the ingest key
+    SENTINEL_SERVICE         or OTEL_SERVICE_NAME
+    SENTINEL_ENV             or deployment.environment(.name) in OTEL_RESOURCE_ATTRIBUTES
+    SENTINEL_VERSION         or service.version in OTEL_RESOURCE_ATTRIBUTES (optional)
+    SENTINEL_OTLP_ENDPOINT   or OTEL_EXPORTER_OTLP_ENDPOINT (default http://localhost:4318)
+
+Missing configuration means the client stays off, with one warning; it never
+raises into the application's start-up.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from typing import Any
+
+_runtime: Any = None
+_log = logging.getLogger("sentinel_probes")
+
+
+def _resource_attribute(*names: str) -> str:
+    raw = os.environ.get("OTEL_RESOURCE_ATTRIBUTES", "")
+    for pair in raw.split(","):
+        key, sep, value = pair.partition("=")
+        if sep and key.strip() in names:
+            return value.strip()
+    return ""
+
+
+def start_from_env() -> Any:
+    """Start once per process; None when not configured. Forked children restart
+    their own poller (Runtime registers an at-fork handler)."""
+    global _runtime
+    if _runtime is not None:
+        return _runtime
+    env = os.environ
+    url = env.get("SENTINEL_PROBES_URL", "")
+    if not url:
+        return None
+    service = env.get("SENTINEL_SERVICE") or env.get("OTEL_SERVICE_NAME", "")
+    environment = env.get("SENTINEL_ENV") or _resource_attribute("deployment.environment.name", "deployment.environment")
+    api_key = env.get("SENTINEL_API_KEY", "")
+    if not (service and environment and api_key):
+        _log.warning(
+            "[sentinel-probes] not started: set SENTINEL_SERVICE (or OTEL_SERVICE_NAME), SENTINEL_ENV and SENTINEL_API_KEY"
+        )
+        return None
+    try:
+        from .runtime import start_probes
+
+        _runtime = start_probes(
+            service=service,
+            env=environment,
+            api_key=api_key,
+            version=env.get("SENTINEL_VERSION") or _resource_attribute("service.version"),
+            probes_url=url,
+            otlp_endpoint=env.get("SENTINEL_OTLP_ENDPOINT") or env.get("OTEL_EXPORTER_OTLP_ENDPOINT") or "http://localhost:4318",
+        )
+    except Exception as e:  # never into the application
+        _log.warning("[sentinel-probes] not started: %s", e)
+        return None
+    return _runtime

sentinel_probes-0.1.0/sentinel_probes/bootstrap/sitecustomize.py

@@ -0,0 +1,29 @@
+"""
+Put on PYTHONPATH by `sentinel-probes-run`: starts the client as the interpreter
+starts, then hands over to any sitecustomize the environment already had.
+"""
+
+import importlib.util
+import os
+import sys
+
+try:
+    from sentinel_probes.autostart import start_from_env
+
+    start_from_env()
+except Exception:  # never into the application
+    pass
+
+# Chain to the next sitecustomize on the path, as if this one were not here.
+_here = os.path.dirname(os.path.abspath(__file__))
+for _entry in sys.path:
+    if _entry and os.path.abspath(_entry) != _here:
+        _candidate = os.path.join(_entry, "sitecustomize.py")
+        if os.path.isfile(_candidate):
+            _loader_spec = importlib.util.spec_from_file_location("_sentinel_chained_sitecustomize", _candidate)
+            if _loader_spec and _loader_spec.loader:
+                try:
+                    _loader_spec.loader.exec_module(importlib.util.module_from_spec(_loader_spec))
+                except Exception:
+                    pass
+            break