ko-observability 0.2.0__tar.gz

ko_observability-0.2.0/.gitignore

@@ -0,0 +1,34 @@
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+dist/
+*.egg-info/
+.venv/
+venv/
+
+node_modules/
+package-lock.json
+
+.env
+.env.otel
+.env.otel-platform
+deploy/tls/*.pem
+deploy/tls/*.key
+!deploy/tls/.gitignore
+
+.DS_Store
+.idea/
+.vscode/
+
+.claude/settings.local.json
+.codegraph/
+
+# TELEMETRY_DESTINATION=file default output + parity/Go-in-Docker caches
+telemetry/
+parity/file_mode/.out/
+.gocache/
+
+# /graphify skill output (knowledge-graph build artifacts, session-local)
+graphify-out/

ko_observability-0.2.0/CHANGELOG.md

@@ -0,0 +1,22 @@
+# Changelog
+
+## [0.2.0](https://github.com/Komus-ai/nexora-observability/compare/python-v0.1.0...python-v0.2.0) (2026-09-18)
+
+
+### ⚠ BREAKING CHANGES
+
+* telemetry is written to ./telemetry/*.jsonl by default instead of being exported to a Collector over OTLP/gRPC. Set TELEMETRY_DESTINATION=otlp to restore the previous behavior.
+
+### Features
+
+* default telemetry destination to local OTLP/JSON files ([223b5cf](https://github.com/Komus-ai/nexora-observability/commit/223b5cfc92bb31d9a3b255a4eb4c6e2dd69f1af2))
+* **go:** add Go SDK core + full-repo audit fixes (Python/JS/deploy/CI/docs) ([#5](https://github.com/Komus-ai/nexora-observability/issues/5)) ([61433d9](https://github.com/Komus-ai/nexora-observability/commit/61433d904dada98757e900b13049d801f5188085))
+* initial nexora-observability package (Python + JS/TS OTel bootstrap) ([129c2fa](https://github.com/Komus-ai/nexora-observability/commit/129c2fa69657b7de61cf652a7f120ce80af7f478))
+* **python:** add optional read-only query client for own telemetry ([b92a4bb](https://github.com/Komus-ai/nexora-observability/commit/b92a4bb23568cab30e4ee7529549d7fdcfc51104))
+
+
+### Bug Fixes
+
+* **python:** fix mypy type errors, add PEP 561 marker, cover init_otel ([85e2a19](https://github.com/Komus-ai/nexora-observability/commit/85e2a19b3d1697d0a25b853260994c26560072d2))
+* **query:** correct status field path/value from live-cluster verification ([8e5861c](https://github.com/Komus-ai/nexora-observability/commit/8e5861c9d874f97caf6c49d4317fc4e978dfede9))
+* release-readiness fixes for first 0.2.0 publish ([#8](https://github.com/Komus-ai/nexora-observability/issues/8)) ([d7dda1f](https://github.com/Komus-ai/nexora-observability/commit/d7dda1f79ab50edc3fd29b2b71348c1956c06634))

ko_observability-0.2.0/PKG-INFO

@@ -0,0 +1,110 @@
+Metadata-Version: 2.5
+Name: ko-observability
+Version: 0.2.0
+Summary: Org-wide OpenTelemetry bootstrap for Nexora Python services
+Project-URL: Repository, https://github.com/Komus-ai/nexora-observability
+Project-URL: Documentation, https://github.com/Komus-ai/nexora-observability/tree/main/docs
+Author-email: Komus-ai <dev@cinco.ai>
+License: MIT
+Keywords: observability,opentelemetry,otlp,structlog
+Classifier: Development Status :: 4 - Beta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.12
+Requires-Dist: opentelemetry-api>=1.27.0
+Requires-Dist: opentelemetry-exporter-otlp-proto-common>=1.27.0
+Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.27.0
+Requires-Dist: opentelemetry-sdk>=1.27.0
+Requires-Dist: protobuf<8.0,>=5.0
+Requires-Dist: structlog>=24.4.0
+Provides-Extra: dev
+Requires-Dist: httpx>=0.27.0; extra == 'dev'
+Requires-Dist: mypy>=1.13.0; extra == 'dev'
+Requires-Dist: opentelemetry-sdk>=1.27.0; extra == 'dev'
+Requires-Dist: pytest>=8.3.0; extra == 'dev'
+Requires-Dist: ruff>=0.8.0; extra == 'dev'
+Provides-Extra: elasticsearch
+Requires-Dist: opentelemetry-instrumentation-elasticsearch>=0.48b0; extra == 'elasticsearch'
+Provides-Extra: fastapi
+Requires-Dist: fastapi>=0.115.0; extra == 'fastapi'
+Requires-Dist: opentelemetry-instrumentation-fastapi>=0.48b0; extra == 'fastapi'
+Provides-Extra: httpx
+Requires-Dist: opentelemetry-instrumentation-httpx>=0.48b0; extra == 'httpx'
+Provides-Extra: query
+Requires-Dist: httpx>=0.27.0; extra == 'query'
+Description-Content-Type: text/markdown
+
+# ko-observability (Python)
+
+Org-wide OpenTelemetry bootstrap for Nexora Python services: one-call, idempotent
+`init_otel` for process startup, lazy `get_tracer` / `get_meter` accessors for library code
+that doesn't own process lifecycle, and a structlog processor (`otel_bridge` /
+`make_otel_bridge`) that stamps each log line with the active span's `trace_id`/`span_id`.
+
+## Install
+
+```bash
+pip install ko-observability
+```
+
+Optional extras, install alongside the base package as needed:
+
+```bash
+# Auto-instrument FastAPI request/response spans
+pip install "ko-observability[fastapi]"
+
+# Auto-instrument outbound httpx calls
+pip install "ko-observability[httpx]"
+
+# Auto-instrument the Elasticsearch client
+pip install "ko-observability[elasticsearch]"
+
+# Read your own service's telemetry back out (admin-panel style queries)
+pip install "ko-observability[query]"
+```
+
+## Quick start
+
+```python
+from ko_observability import init_otel, get_tracer
+
+init_otel(service_name="my-service", service_namespace="nexora")
+
+tracer = get_tracer(__name__)
+with tracer.start_as_current_span("do-work"):
+    ...
+```
+
+Call `init_otel` exactly once at process startup. It's safe to call again (or to let library
+code fall back to `get_tracer`/`get_meter` before it runs) — both paths are no-ops on top of an
+already-configured provider. `OTEL_SDK_DISABLED=true` turns the whole thing into a no-op, and a
+misconfigured exporter degrades to "telemetry doesn't work" rather than crashing the host process.
+
+### Where does it go?
+
+| `TELEMETRY_DESTINATION` | Output |
+|---|---|
+| `file` (default) | `$TELEMETRY_FILE_DIR/{traces,metrics,logs}.jsonl` (default `./telemetry`), one OTLP/JSON request per line |
+| `otlp` | OTLP/gRPC to `OTEL_EXPORTER_OTLP_ENDPOINT` (default `http://localhost:4317`) |
+
+Full contract: [docs/otel.md → Local file mode](../../docs/otel.md#local-file-mode-the-default).
+
+## Reading telemetry back (`query` extra)
+
+```python
+from ko_observability.query import get_query_client
+
+client = get_query_client()
+traces = client.list_traces(limit=20)
+```
+
+Scoped to the caller's own `service_name` by default. See [`docs/query.md`](../../docs/query.md).
+
+## Docs
+
+Full documentation — env vars, the logs/metrics pipelines, semantic conventions, and the
+Collector deployment this SDK ships telemetry to — lives in the repository:
+
+https://github.com/Komus-ai/nexora-observability/tree/main/docs

ko_observability-0.2.0/README.md

@@ -0,0 +1,72 @@
+# ko-observability (Python)
+
+Org-wide OpenTelemetry bootstrap for Nexora Python services: one-call, idempotent
+`init_otel` for process startup, lazy `get_tracer` / `get_meter` accessors for library code
+that doesn't own process lifecycle, and a structlog processor (`otel_bridge` /
+`make_otel_bridge`) that stamps each log line with the active span's `trace_id`/`span_id`.
+
+## Install
+
+```bash
+pip install ko-observability
+```
+
+Optional extras, install alongside the base package as needed:
+
+```bash
+# Auto-instrument FastAPI request/response spans
+pip install "ko-observability[fastapi]"
+
+# Auto-instrument outbound httpx calls
+pip install "ko-observability[httpx]"
+
+# Auto-instrument the Elasticsearch client
+pip install "ko-observability[elasticsearch]"
+
+# Read your own service's telemetry back out (admin-panel style queries)
+pip install "ko-observability[query]"
+```
+
+## Quick start
+
+```python
+from ko_observability import init_otel, get_tracer
+
+init_otel(service_name="my-service", service_namespace="nexora")
+
+tracer = get_tracer(__name__)
+with tracer.start_as_current_span("do-work"):
+    ...
+```
+
+Call `init_otel` exactly once at process startup. It's safe to call again (or to let library
+code fall back to `get_tracer`/`get_meter` before it runs) — both paths are no-ops on top of an
+already-configured provider. `OTEL_SDK_DISABLED=true` turns the whole thing into a no-op, and a
+misconfigured exporter degrades to "telemetry doesn't work" rather than crashing the host process.
+
+### Where does it go?
+
+| `TELEMETRY_DESTINATION` | Output |
+|---|---|
+| `file` (default) | `$TELEMETRY_FILE_DIR/{traces,metrics,logs}.jsonl` (default `./telemetry`), one OTLP/JSON request per line |
+| `otlp` | OTLP/gRPC to `OTEL_EXPORTER_OTLP_ENDPOINT` (default `http://localhost:4317`) |
+
+Full contract: [docs/otel.md → Local file mode](../../docs/otel.md#local-file-mode-the-default).
+
+## Reading telemetry back (`query` extra)
+
+```python
+from ko_observability.query import get_query_client
+
+client = get_query_client()
+traces = client.list_traces(limit=20)
+```
+
+Scoped to the caller's own `service_name` by default. See [`docs/query.md`](../../docs/query.md).
+
+## Docs
+
+Full documentation — env vars, the logs/metrics pipelines, semantic conventions, and the
+Collector deployment this SDK ships telemetry to — lives in the repository:
+
+https://github.com/Komus-ai/nexora-observability/tree/main/docs

ko_observability-0.2.0/pyproject.toml

@@ -0,0 +1,69 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "ko-observability"
+version = "0.2.0"
+description = "Org-wide OpenTelemetry bootstrap for Nexora Python services"
+readme = "README.md"
+requires-python = ">=3.12"
+license = { text = "MIT" }
+authors = [{ name = "Komus-ai", email = "dev@cinco.ai" }]
+keywords = ["opentelemetry", "observability", "otlp", "structlog"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+]
+dependencies = [
+    "opentelemetry-api>=1.27.0",
+    "opentelemetry-sdk>=1.27.0",
+    "opentelemetry-exporter-otlp-proto-grpc>=1.27.0",
+    "opentelemetry-exporter-otlp-proto-common>=1.27.0",
+    "protobuf>=5.0,<8.0",
+    "structlog>=24.4.0",
+]
+
+[project.optional-dependencies]
+fastapi = [
+    "opentelemetry-instrumentation-fastapi>=0.48b0",
+    "fastapi>=0.115.0",
+]
+httpx = ["opentelemetry-instrumentation-httpx>=0.48b0"]
+elasticsearch = ["opentelemetry-instrumentation-elasticsearch>=0.48b0"]
+query = ["httpx>=0.27.0"]
+dev = [
+    "pytest>=8.3.0",
+    "opentelemetry-sdk>=1.27.0",
+    "httpx>=0.27.0",
+    "ruff>=0.8.0",
+    "mypy>=1.13.0",
+]
+
+[project.urls]
+Repository = "https://github.com/Komus-ai/nexora-observability"
+Documentation = "https://github.com/Komus-ai/nexora-observability/tree/main/docs"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/ko_observability"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]
+
+[tool.ruff]
+target-version = "py312"
+line-length = 100
+src = ["src", "tests"]
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B"]
+
+[tool.mypy]
+python_version = "3.12"
+packages = ["ko_observability"]
+mypy_path = "src"
+ignore_missing_imports = true

ko_observability-0.2.0/src/ko_observability/__init__.py

@@ -0,0 +1,15 @@
+"""Nexora org-wide OpenTelemetry libraries."""
+
+from ko_observability.client import get_meter, get_tracer
+from ko_observability.otel import init_otel
+from ko_observability.structlog_bridge import make_otel_bridge, otel_bridge
+
+__all__ = [
+    "init_otel",
+    "get_tracer",
+    "get_meter",
+    "otel_bridge",
+    "make_otel_bridge",
+]
+
+__version__ = "0.2.0"

ko_observability-0.2.0/src/ko_observability/client.py

@@ -0,0 +1,67 @@
+"""Lazy OTel client helpers for library and application code.
+
+Use these when you want a Tracer/Meter without requiring the host process to
+have called ``init_otel`` first (e.g. inside a reusable library). Each
+function bootstraps a minimal provider on first use if -- and only if -- no
+provider has been registered yet by the host process.
+"""
+
+from __future__ import annotations
+
+import os
+import threading
+
+from opentelemetry import metrics as _otel_metrics
+from opentelemetry import trace
+
+from ko_observability.exporters import make_span_exporter
+from ko_observability.otlp_config import sdk_disabled
+
+_init_lock = threading.Lock()
+_initialized = False
+
+
+def get_tracer(name: str, version: str | None = None):
+    """Return a Tracer, lazily bootstrapping a minimal provider if needed."""
+    _ensure_trace_provider()
+    return trace.get_tracer(name, version)
+
+
+def get_meter(name: str, version: str | None = None) -> _otel_metrics.Meter:
+    """Return a Meter (no-op if metrics provider is not configured)."""
+    return _otel_metrics.get_meter(name, version or "")
+
+
+def _ensure_trace_provider() -> None:
+    global _initialized
+    if _initialized:
+        return
+    with _init_lock:
+        if _initialized:
+            return
+        if sdk_disabled():
+            _initialized = True
+            return
+
+        provider = trace.get_tracer_provider()
+        if "NoOp" not in type(provider).__name__ and "Proxy" not in type(provider).__name__:
+            _initialized = True
+            return
+
+        try:
+            from opentelemetry.sdk.resources import Resource
+            from opentelemetry.sdk.trace import TracerProvider
+            from opentelemetry.sdk.trace.export import BatchSpanProcessor
+
+            resource = Resource.create(
+                {
+                    "service.name": os.getenv("OTEL_SERVICE_NAME", "nexora-service"),
+                    "service.namespace": os.getenv("OTEL_SERVICE_NAMESPACE", "nexora"),
+                }
+            )
+            tp = TracerProvider(resource=resource)
+            tp.add_span_processor(BatchSpanProcessor(make_span_exporter()))
+            trace.set_tracer_provider(tp)
+        except Exception:
+            pass
+        _initialized = True

ko_observability-0.2.0/src/ko_observability/exporters.py

@@ -0,0 +1,208 @@
+"""Exporter factories: local JSONL files (default) or OTLP/gRPC, per TELEMETRY_DESTINATION.
+
+File exporters write one OTLP/JSON ``Export*ServiceRequest`` per line -- the
+same encoding the Collector's ``otlpjsonfile`` receiver replays -- with
+``traceId``/``spanId``/``parentSpanId`` rewritten from protobuf-JSON base64
+to the spec's hex form. Append-only, one ``write()`` per batch, mutex-guarded.
+"""
+
+from __future__ import annotations
+
+import base64
+import binascii
+import json
+import logging
+import os
+import threading
+from collections.abc import Sequence
+from typing import Any
+
+from opentelemetry.sdk._logs.export import LogRecordExporter, LogRecordExportResult
+from opentelemetry.sdk.metrics.export import MetricExporter, MetricExportResult, MetricsData
+from opentelemetry.sdk.trace import ReadableSpan
+from opentelemetry.sdk.trace.export import SpanExporter, SpanExportResult
+
+from ko_observability.otlp_config import (
+    otlp_endpoint,
+    otlp_exporter_insecure,
+    telemetry_destination,
+    telemetry_file_dir,
+)
+
+_logger = logging.getLogger(__name__)
+
+TRACES_FILE = "traces.jsonl"
+METRICS_FILE = "metrics.jsonl"
+LOGS_FILE = "logs.jsonl"
+
+_ID_KEYS = frozenset({"traceId", "spanId", "parentSpanId"})
+
+
+def _b64_to_hex(value: str) -> str:
+    try:
+        return base64.b64decode(value, validate=True).hex()
+    except (binascii.Error, ValueError):
+        return value
+
+
+def _hex_ids(obj: Any) -> Any:
+    if isinstance(obj, dict):
+        return {
+            k: (_b64_to_hex(v) if k in _ID_KEYS and isinstance(v, str) else _hex_ids(v))
+            for k, v in obj.items()
+        }
+    if isinstance(obj, list):
+        return [_hex_ids(v) for v in obj]
+    return obj
+
+
+def _to_otlp_json(message: Any) -> dict[str, Any]:
+    from google.protobuf.json_format import MessageToDict
+
+    return _hex_ids(MessageToDict(message, use_integers_for_enums=True))
+
+
+class _JsonlWriter:
+    def __init__(self, path: str) -> None:
+        os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
+        self._fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o644)
+        self._lock = threading.Lock()
+
+    def write_line(self, obj: dict[str, Any]) -> None:
+        data = (json.dumps(obj, separators=(",", ":")) + "\n").encode("utf-8")
+        with self._lock:
+            if self._fd < 0:
+                raise OSError("telemetry file closed")
+            view = memoryview(data)
+            written = 0
+            try:
+                while view:
+                    n = os.write(self._fd, view)
+                    written += n
+                    view = view[n:]
+            except OSError:
+                # A partial write before the failure leaves a fragment (no
+                # trailing newline) on disk -- but only if this call made any
+                # progress at all; a failure on the very first byte leaves the
+                # file exactly as the last successful write_line left it (already
+                # newline-terminated), so no recovery byte is needed there.
+                # Left alone, the next successful export would append onto the
+                # fragment and produce one unparseable JSONL line, corrupting
+                # previously-good data rather than just the failed batch.
+                # Best-effort: terminate the fragment with a newline so line
+                # framing survives, then propagate the original error (never
+                # crash the host).
+                if written > 0:
+                    try:
+                        os.write(self._fd, b"\n")
+                    except OSError:
+                        pass
+                raise
+
+    def close(self) -> None:
+        with self._lock:
+            if self._fd >= 0:
+                os.close(self._fd)
+                self._fd = -1
+
+
+def _default_path(name: str) -> str:
+    return os.path.join(telemetry_file_dir(), name)
+
+
+class FileSpanExporter(SpanExporter):
+    def __init__(self, path: str | None = None) -> None:
+        self._writer = _JsonlWriter(path or _default_path(TRACES_FILE))
+
+    def export(self, spans: Sequence[ReadableSpan]) -> SpanExportResult:
+        try:
+            from opentelemetry.exporter.otlp.proto.common.trace_encoder import encode_spans
+
+            self._writer.write_line(_to_otlp_json(encode_spans(spans)))
+            return SpanExportResult.SUCCESS
+        except Exception as exc:
+            _logger.warning("telemetry.file_export_failed: traces: %s", exc)
+            return SpanExportResult.FAILURE
+
+    def shutdown(self) -> None:
+        self._writer.close()
+
+    def force_flush(self, timeout_millis: int = 30000) -> bool:
+        return True
+
+
+class FileLogExporter(LogRecordExporter):
+    def __init__(self, path: str | None = None) -> None:
+        self._writer = _JsonlWriter(path or _default_path(LOGS_FILE))
+
+    def export(self, batch: Sequence[Any]) -> LogRecordExportResult:
+        try:
+            from opentelemetry.exporter.otlp.proto.common._log_encoder import encode_logs
+
+            self._writer.write_line(_to_otlp_json(encode_logs(batch)))
+            return LogRecordExportResult.SUCCESS
+        except Exception as exc:
+            _logger.warning("telemetry.file_export_failed: logs: %s", exc)
+            return LogRecordExportResult.FAILURE
+
+    def shutdown(self) -> None:
+        self._writer.close()
+
+    def force_flush(self, timeout_millis: int = 30000) -> bool:
+        return True
+
+
+class FileMetricExporter(MetricExporter):
+    def __init__(
+        self,
+        path: str | None = None,
+        preferred_temporality: dict[type, Any] | None = None,
+        preferred_aggregation: dict[type, Any] | None = None,
+    ) -> None:
+        super().__init__(
+            preferred_temporality=preferred_temporality,
+            preferred_aggregation=preferred_aggregation,
+        )
+        self._writer = _JsonlWriter(path or _default_path(METRICS_FILE))
+
+    def export(
+        self, metrics_data: MetricsData, timeout_millis: float = 10_000, **kwargs: Any
+    ) -> MetricExportResult:
+        try:
+            from opentelemetry.exporter.otlp.proto.common.metrics_encoder import encode_metrics
+
+            self._writer.write_line(_to_otlp_json(encode_metrics(metrics_data)))
+            return MetricExportResult.SUCCESS
+        except Exception as exc:
+            _logger.warning("telemetry.file_export_failed: metrics: %s", exc)
+            return MetricExportResult.FAILURE
+
+    def force_flush(self, timeout_millis: float = 10_000) -> bool:
+        return True
+
+    def shutdown(self, timeout_millis: float = 30_000, **kwargs: Any) -> None:
+        self._writer.close()
+
+
+def make_span_exporter() -> SpanExporter:
+    if telemetry_destination() == "file":
+        return FileSpanExporter()
+    from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
+
+    return OTLPSpanExporter(endpoint=otlp_endpoint(), insecure=otlp_exporter_insecure())
+
+
+def make_log_exporter() -> LogRecordExporter:
+    if telemetry_destination() == "file":
+        return FileLogExporter()
+    from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
+
+    return OTLPLogExporter(endpoint=otlp_endpoint(), insecure=otlp_exporter_insecure())
+
+
+def make_metric_exporter() -> MetricExporter:
+    if telemetry_destination() == "file":
+        return FileMetricExporter()
+    from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
+
+    return OTLPMetricExporter(endpoint=otlp_endpoint(), insecure=otlp_exporter_insecure())