ai-release-metadata 0.1.0__tar.gz

ai_release_metadata-0.1.0/PKG-INFO

@@ -0,0 +1,169 @@
+Metadata-Version: 2.4
+Name: ai-release-metadata
+Version: 0.1.0
+Summary: A lightweight, framework-agnostic Python SDK for standardizing AI release metadata across production applications.
+Author-email: Oura <opensource@example.com>
+License: MIT
+Project-URL: Homepage, https://github.com/ai-release-metadata/ai-release-metadata
+Project-URL: Bug Tracker, https://github.com/ai-release-metadata/ai-release-metadata/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+Provides-Extra: otel
+Requires-Dist: opentelemetry-api; extra == "otel"
+
+# AI Release Metadata (ai-release-metadata)
+
+> **A lightweight SDK for standardizing and propagating operational metadata for AI-powered services through existing observability systems.**
+
+## The Problem
+
+When a production issue occurs in an AI-powered service (e.g., regressions, prompt injection, or unexpected model behaviour), it is often extremely difficult to quickly identify:
+* Which prompt version generated the response?
+* Which model was used?
+* Which Git commit or deployment version is responsible?
+* What experiments were active?
+
+While existing observability platforms effectively capture execution metrics such as latency and token usage, manually correlating these spans back to the exact release configuration often results in fragmented and inconsistent metadata.
+
+## The Solution
+
+This SDK provides a minimal set of context managers and decorators that automatically enrich existing logs, metrics, and traces with standardized release and environment metadata.
+
+### Architecture
+
+```mermaid
+flowchart TD
+    App[Application] -->|with release_context()| Provider[MetadataProvider]
+    Provider -->|extract| Plugins["Plugins (Git, Env)"]
+    Plugins --> Context(("Release Context"))
+    Context -->|get_current_context()| Exporters["Exporters"]
+    Exporters --> OTEL["OpenTelemetry / Structlog"]
+```
+
+### Design Principles
+- **Minimal developer overhead:** Drop-in context managers and decorators.
+- **Existing Observability First:** Integrates directly into your current logging/tracing pipelines instead of requiring a new dashboard.
+- **A reusable building block:** This explores one reusable approach for improving traceability between production behaviour and deployed AI releases.
+- **Framework agnostic:** Works with FastAPI, Django, or raw scripts.
+- **Vendor agnostic:** Does not wrap or depend on specific LLM provider SDKs.
+- **Explicit over magic:** Developers explicitly define trace boundaries.
+- **Existing observability first:** Propagates metadata to your existing tools (like OpenTelemetry) rather than acting as a standalone backend.
+- **Async-safe by default:** Thread-safe and async-safe context propagation.
+
+### Features
+* **Automatic Metadata Discovery:** Detects release and runtime information from the execution environment through a pluggable metadata system.
+* **Application-level Instrumentation:** Instruments business operations instead of wrapping LLM SDKs, allowing the library to remain provider-agnostic.
+* **Integrations:** Natively exports into `structlog` (JSON logs) and OpenTelemetry.
+* **Async First:** Safe to use in high-throughput `asyncio` applications such as FastAPI.
+
+## Use Cases
+
+### 1. Diagnosing Latency Regressions by Git Commit
+If latency spikes on a specific LLM feature, you can query Datadog/Grafana for all `generate` traces grouped by `ai.git_sha`. The SDK ensures this metadata is accurately attached to every span, instantly identifying the deployment that caused the regression.
+
+### 2. A/B Testing Prompt Templates
+When rolling out a new prompt (e.g. `prompt_version="v2.1"`), you can seamlessly track the token usage, error rates, and user feedback specifically for that variant.
+
+## Documentation
+
+- [Architecture](docs/architecture.md): How context propagation works under the hood.
+- [Design Decisions](docs/design-decisions.md): The rationale behind the core architectural choices.
+- [Metadata Schema](docs/metadata-schema.md): The full list of supported fields.
+- [Plugin Development](docs/plugin-development.md): How to build custom discovery sources.
+- [Exporters](docs/exporters.md): How to export data to OpenTelemetry, structlog, and more.
+- [Roadmap](docs/roadmap.md): Future development ideas and planned integrations.
+
+## Installation
+
+```bash
+pip install ai-release-metadata
+```
+
+To enable the OpenTelemetry integration automatically, install the `otel` extra:
+```bash
+pip install "ai-release-metadata[otel]"
+```
+
+## Usage
+
+Configure the SDK once at application startup using the `MetadataProvider`. 
+
+**Plugin Precedence Rules:** The SDK resolves metadata conflicts based on the order plugins are provided. The *last* plugin evaluated takes precedence over the preceding ones. We recommend putting local plugins first, and explicit environment plugins last.
+
+```python
+from ai_release_metadata import MetadataProvider
+from ai_release_metadata.plugins import EnvPlugin, GitPlugin, GitHubActionsPlugin
+
+# 1. GitPlugin tries to guess from local filesystem
+# 2. GitHubActionsPlugin overwrites with CI variables
+# 3. EnvPlugin overwrites with explicit user-defined env vars
+MetadataProvider(plugins=[GitPlugin(), GitHubActionsPlugin(), EnvPlugin()])
+```
+
+Use the context manager (`release_context`) or decorator (`@capture_generation`) within the business logic:
+
+```python
+from ai_release_metadata import release_context, get_current_context
+
+async def generate_response(user_id: str, query: str):
+    # Start a release context block
+    with release_context(feature="customer-support", model="gpt-4o", prompt_version="v2.1"):
+        
+        # ... LLM calling logic ...
+        
+        # Dynamically append runtime information to the context
+        ctx = get_current_context()
+        if ctx:
+            ctx.add_document("doc_1")
+            ctx.add_tag("user_tier", "premium")
+```
+
+All logs emitted within the `with release_context(...)` block will automatically have the metadata appended under the `ai` key.
+
+### OpenTelemetry Integration
+
+If you use OpenTelemetry, you can automatically enrich the currently active span with the release context. It gracefully ignores spans if none are active.
+
+```python
+from ai_release_metadata.integrations.opentelemetry import enrich_span
+from opentelemetry import trace
+
+tracer = trace.get_tracer(__name__)
+
+with tracer.start_as_current_span("generate_response") as span:
+    with release_context(model="gpt-4o"):
+        # Enrich the span with git_sha, environment, model, etc.
+        enrich_span()
+        
+        # ... LLM calling logic ...
+```
+
+## Running the Demo
+
+This repository includes a FastAPI demo simulating a generation endpoint.
+
+1. Activate a virtual environment and install dependencies:
+```bash
+python3 -m venv venv
+source venv/bin/activate
+pip install structlog fastapi uvicorn pydantic
+```
+
+2. Run the application:
+```bash
+export GIT_COMMIT=a1b2c3d 
+export ENVIRONMENT=local 
+export LOG_FORMAT=console
+PYTHONPATH=. uvicorn demo.app:app
+```
+
+3. Send a request to observe the enriched telemetry in the terminal:
+```bash
+curl -X POST http://127.0.0.1:8000/generate \
+  -H "Content-Type: application/json" \
+  -d '{"model": "gpt-4o", "prompt_version": "v2.0", "user_query": "I cannot login"}'
+```

ai_release_metadata-0.1.0/README.md

@@ -0,0 +1,152 @@
+# AI Release Metadata (ai-release-metadata)
+
+> **A lightweight SDK for standardizing and propagating operational metadata for AI-powered services through existing observability systems.**
+
+## The Problem
+
+When a production issue occurs in an AI-powered service (e.g., regressions, prompt injection, or unexpected model behaviour), it is often extremely difficult to quickly identify:
+* Which prompt version generated the response?
+* Which model was used?
+* Which Git commit or deployment version is responsible?
+* What experiments were active?
+
+While existing observability platforms effectively capture execution metrics such as latency and token usage, manually correlating these spans back to the exact release configuration often results in fragmented and inconsistent metadata.
+
+## The Solution
+
+This SDK provides a minimal set of context managers and decorators that automatically enrich existing logs, metrics, and traces with standardized release and environment metadata.
+
+### Architecture
+
+```mermaid
+flowchart TD
+    App[Application] -->|with release_context()| Provider[MetadataProvider]
+    Provider -->|extract| Plugins["Plugins (Git, Env)"]
+    Plugins --> Context(("Release Context"))
+    Context -->|get_current_context()| Exporters["Exporters"]
+    Exporters --> OTEL["OpenTelemetry / Structlog"]
+```
+
+### Design Principles
+- **Minimal developer overhead:** Drop-in context managers and decorators.
+- **Existing Observability First:** Integrates directly into your current logging/tracing pipelines instead of requiring a new dashboard.
+- **A reusable building block:** This explores one reusable approach for improving traceability between production behaviour and deployed AI releases.
+- **Framework agnostic:** Works with FastAPI, Django, or raw scripts.
+- **Vendor agnostic:** Does not wrap or depend on specific LLM provider SDKs.
+- **Explicit over magic:** Developers explicitly define trace boundaries.
+- **Existing observability first:** Propagates metadata to your existing tools (like OpenTelemetry) rather than acting as a standalone backend.
+- **Async-safe by default:** Thread-safe and async-safe context propagation.
+
+### Features
+* **Automatic Metadata Discovery:** Detects release and runtime information from the execution environment through a pluggable metadata system.
+* **Application-level Instrumentation:** Instruments business operations instead of wrapping LLM SDKs, allowing the library to remain provider-agnostic.
+* **Integrations:** Natively exports into `structlog` (JSON logs) and OpenTelemetry.
+* **Async First:** Safe to use in high-throughput `asyncio` applications such as FastAPI.
+
+## Use Cases
+
+### 1. Diagnosing Latency Regressions by Git Commit
+If latency spikes on a specific LLM feature, you can query Datadog/Grafana for all `generate` traces grouped by `ai.git_sha`. The SDK ensures this metadata is accurately attached to every span, instantly identifying the deployment that caused the regression.
+
+### 2. A/B Testing Prompt Templates
+When rolling out a new prompt (e.g. `prompt_version="v2.1"`), you can seamlessly track the token usage, error rates, and user feedback specifically for that variant.
+
+## Documentation
+
+- [Architecture](docs/architecture.md): How context propagation works under the hood.
+- [Design Decisions](docs/design-decisions.md): The rationale behind the core architectural choices.
+- [Metadata Schema](docs/metadata-schema.md): The full list of supported fields.
+- [Plugin Development](docs/plugin-development.md): How to build custom discovery sources.
+- [Exporters](docs/exporters.md): How to export data to OpenTelemetry, structlog, and more.
+- [Roadmap](docs/roadmap.md): Future development ideas and planned integrations.
+
+## Installation
+
+```bash
+pip install ai-release-metadata
+```
+
+To enable the OpenTelemetry integration automatically, install the `otel` extra:
+```bash
+pip install "ai-release-metadata[otel]"
+```
+
+## Usage
+
+Configure the SDK once at application startup using the `MetadataProvider`. 
+
+**Plugin Precedence Rules:** The SDK resolves metadata conflicts based on the order plugins are provided. The *last* plugin evaluated takes precedence over the preceding ones. We recommend putting local plugins first, and explicit environment plugins last.
+
+```python
+from ai_release_metadata import MetadataProvider
+from ai_release_metadata.plugins import EnvPlugin, GitPlugin, GitHubActionsPlugin
+
+# 1. GitPlugin tries to guess from local filesystem
+# 2. GitHubActionsPlugin overwrites with CI variables
+# 3. EnvPlugin overwrites with explicit user-defined env vars
+MetadataProvider(plugins=[GitPlugin(), GitHubActionsPlugin(), EnvPlugin()])
+```
+
+Use the context manager (`release_context`) or decorator (`@capture_generation`) within the business logic:
+
+```python
+from ai_release_metadata import release_context, get_current_context
+
+async def generate_response(user_id: str, query: str):
+    # Start a release context block
+    with release_context(feature="customer-support", model="gpt-4o", prompt_version="v2.1"):
+        
+        # ... LLM calling logic ...
+        
+        # Dynamically append runtime information to the context
+        ctx = get_current_context()
+        if ctx:
+            ctx.add_document("doc_1")
+            ctx.add_tag("user_tier", "premium")
+```
+
+All logs emitted within the `with release_context(...)` block will automatically have the metadata appended under the `ai` key.
+
+### OpenTelemetry Integration
+
+If you use OpenTelemetry, you can automatically enrich the currently active span with the release context. It gracefully ignores spans if none are active.
+
+```python
+from ai_release_metadata.integrations.opentelemetry import enrich_span
+from opentelemetry import trace
+
+tracer = trace.get_tracer(__name__)
+
+with tracer.start_as_current_span("generate_response") as span:
+    with release_context(model="gpt-4o"):
+        # Enrich the span with git_sha, environment, model, etc.
+        enrich_span()
+        
+        # ... LLM calling logic ...
+```
+
+## Running the Demo
+
+This repository includes a FastAPI demo simulating a generation endpoint.
+
+1. Activate a virtual environment and install dependencies:
+```bash
+python3 -m venv venv
+source venv/bin/activate
+pip install structlog fastapi uvicorn pydantic
+```
+
+2. Run the application:
+```bash
+export GIT_COMMIT=a1b2c3d 
+export ENVIRONMENT=local 
+export LOG_FORMAT=console
+PYTHONPATH=. uvicorn demo.app:app
+```
+
+3. Send a request to observe the enriched telemetry in the terminal:
+```bash
+curl -X POST http://127.0.0.1:8000/generate \
+  -H "Content-Type: application/json" \
+  -d '{"model": "gpt-4o", "prompt_version": "v2.0", "user_query": "I cannot login"}'
+```

ai_release_metadata-0.1.0/ai_release_metadata/__init__.py

@@ -0,0 +1,13 @@
+from .core.context import (
+    release_context, 
+    capture_generation, 
+    get_current_context
+)
+from .core.sdk import MetadataProvider
+
+__all__ = [
+    "release_context",
+    "capture_generation",
+    "get_current_context",
+    "MetadataProvider"
+]

ai_release_metadata-0.1.0/ai_release_metadata/core/context.py

@@ -0,0 +1,100 @@
+import contextvars
+import asyncio
+from contextlib import contextmanager
+from functools import wraps
+from typing import Optional, Generator, Callable, Any
+import copy
+
+from .sdk import MetadataProvider
+from .models import ReleaseContext
+
+# A thread-safe, async-safe context variable to hold the current release context
+_current_context: contextvars.ContextVar[Optional[ReleaseContext]] = contextvars.ContextVar(
+    "ai_release_metadata", default=None
+)
+
+def get_current_context() -> Optional[ReleaseContext]:
+    """Retrieve the active AI release context metadata from the current execution."""
+    return _current_context.get()
+
+@contextmanager
+def release_context(
+    feature: Optional[str] = None,
+    prompt_version: Optional[str] = None,
+    model: Optional[str] = None
+) -> Generator[ReleaseContext, None, None]:
+    """Context manager to start a new AI release context block."""
+    parent_context = get_current_context()
+    
+    if parent_context:
+        # Deep copy to prevent modifying parent state
+        metadata = copy.deepcopy(parent_context)
+        
+        # Override fields if explicitly provided in this block
+        if feature:
+            metadata.feature = feature
+        if prompt_version:
+            metadata.prompt_version = prompt_version
+        if model:
+            metadata.model = model
+    else:
+        # 1. Base release metadata comes from the global MetadataProvider
+        metadata = MetadataProvider.get_global().get_base_metadata()
+        metadata.feature = feature
+        metadata.prompt_version = prompt_version
+        metadata.model = model
+    
+    token = _current_context.set(metadata)
+    try:
+        yield metadata
+    finally:
+        _current_context.reset(token)
+
+def capture_generation(
+    feature: Optional[str] = None,
+    prompt_version: Optional[str] = None,
+    model: Optional[str] = None,
+    experiment_flags: Optional[dict] = None,
+    tags: Optional[dict] = None,
+    auto_capture_args: bool = True
+):
+    """Decorator to wrap a function with an AI release context."""
+    import inspect
+    
+    def decorator(func: Callable[..., Any]) -> Callable[..., Any]:
+        sig = inspect.signature(func)
+        
+        def _capture(ctx, args, kwargs):
+            if ctx:
+                if experiment_flags:
+                    ctx.experiment_flags.update(experiment_flags)
+                if tags:
+                    ctx.tags.update(tags)
+                    
+                if auto_capture_args:
+                    try:
+                        bound = sig.bind(*args, **kwargs)
+                        bound.apply_defaults()
+                        for k, v in bound.arguments.items():
+                            if isinstance(v, (str, int, float, bool)):
+                                ctx.tags[f"arg_{k}"] = v
+                            else:
+                                ctx.tags[f"arg_{k}"] = str(v)
+                    except Exception:
+                        pass
+
+        if asyncio.iscoroutinefunction(func):
+            @wraps(func)
+            async def async_wrapper(*args, **kwargs):
+                with release_context(feature=feature, prompt_version=prompt_version, model=model) as ctx:
+                    _capture(ctx, args, kwargs)
+                    return await func(*args, **kwargs)
+            return async_wrapper
+        else:
+            @wraps(func)
+            def sync_wrapper(*args, **kwargs):
+                with release_context(feature=feature, prompt_version=prompt_version, model=model) as ctx:
+                    _capture(ctx, args, kwargs)
+                    return func(*args, **kwargs)
+            return sync_wrapper
+    return decorator

ai_release_metadata-0.1.0/ai_release_metadata/core/models.py

@@ -0,0 +1,40 @@
+import dataclasses
+from typing import Optional, Dict, Any, List
+
+@dataclasses.dataclass
+class ReleaseContext:
+    """Standardized metadata model for an AI generation trace."""
+    # Explicitly provided by developer
+    feature: Optional[str] = None
+    prompt_version: Optional[str] = None
+    model: Optional[str] = None
+    
+    # Automatically extracted
+    git_sha: Optional[str] = None
+    deployment_version: Optional[str] = None
+    environment: Optional[str] = None
+    
+    # Runtime context
+    experiment_flags: Dict[str, Any] = dataclasses.field(default_factory=dict)
+    retrieved_documents: List[str] = dataclasses.field(default_factory=list)
+    tags: Dict[str, Any] = dataclasses.field(default_factory=dict)
+    extra: Dict[str, Any] = dataclasses.field(default_factory=dict)
+    
+    def add_document(self, document: str) -> None:
+        """Helper to append a retrieved document."""
+        self.retrieved_documents.append(document)
+        
+    def add_tag(self, key: str, value: Any) -> None:
+        """Helper to attach a runtime tag."""
+        self.tags[key] = value
+    
+    def as_dict(self) -> Dict[str, Any]:
+        """Convert to dictionary, excluding None or empty values."""
+        result = {}
+        for k, v in dataclasses.asdict(self).items():
+            if v is not None:
+                # exclude empty dicts/lists for cleaner logs
+                if isinstance(v, (list, dict)) and not v:
+                    continue
+                result[k] = v
+        return result

ai_release_metadata-0.1.0/ai_release_metadata/core/sdk.py

@@ -0,0 +1,42 @@
+from typing import List, Optional, Dict, Any
+import copy
+
+from ..plugins.base import MetadataPlugin
+from .models import ReleaseContext
+
+class MetadataProvider:
+    """Instanced SDK Manager. Registers itself as the global provider upon instantiation."""
+    
+    _global_instance = None
+    
+    def __init__(self, plugins: Optional[List[MetadataPlugin]] = None):
+        self.plugins = plugins or []
+        self.base_metadata = ReleaseContext()
+        self._evaluate_plugins()
+        
+        # OpenTelemetry pattern: registering as the global instance implicitly
+        MetadataProvider._global_instance = self
+        
+    def _evaluate_plugins(self):
+        """Run all registered plugins and populate base_metadata."""
+        extra = {}
+        for plugin in self.plugins:
+            result = plugin.extract()
+            # Map known fields, dump the rest into extra
+            for k, v in result.items():
+                if hasattr(self.base_metadata, k) and k != "extra":
+                    setattr(self.base_metadata, k, v)
+                else:
+                    extra[k] = v
+        self.base_metadata.extra = extra
+        
+    @classmethod
+    def get_global(cls) -> "MetadataProvider":
+        """Get the global SDK instance. If none exists, return a dummy one to avoid crashes."""
+        if cls._global_instance is None:
+            return cls()
+        return cls._global_instance
+        
+    def get_base_metadata(self) -> ReleaseContext:
+        """Return a deep copy of the base metadata for trace injection."""
+        return copy.deepcopy(self.base_metadata)

ai_release_metadata-0.1.0/ai_release_metadata/integrations/base.py

@@ -0,0 +1,9 @@
+from typing import Protocol
+from ..core.models import ReleaseContext
+
+class Exporter(Protocol):
+    """Protocol for push-based AI release metadata exporters."""
+    
+    def export(self, context: ReleaseContext) -> None:
+        """Export the unified release context to an external system."""
+        ...

ai_release_metadata-0.1.0/ai_release_metadata/integrations/opentelemetry.py

@@ -0,0 +1,42 @@
+from opentelemetry import trace
+from ..core.context import get_current_context
+
+def enrich_span(span: trace.Span = None) -> None:
+    """
+    Appends the active AI release context to the provided OpenTelemetry span.
+    If no span is provided, it attempts to use the currently active span in the context.
+    """
+    if span is None:
+        span = trace.get_current_span()
+        
+    if not span or not span.is_recording():
+        return
+        
+    ctx = get_current_context()
+    if not ctx:
+        return
+        
+    # Serialize context and add attributes to the span
+    data = ctx.as_dict()
+    
+    # Flatten the dict for OTEL attributes (OTEL doesn't support nested dictionaries)
+    for key, value in data.items():
+        if isinstance(value, (list, tuple)):
+            # OTEL supports homogeneous arrays of primitives
+            try:
+                span.set_attribute(f"ai.{key}", list(value))
+            except Exception:
+                span.set_attribute(f"ai.{key}", [str(v) for v in value])
+                
+        elif isinstance(value, dict):
+            # Flatten one level deep (e.g. tags, experiment_flags)
+            for sub_k, sub_v in value.items():
+                if isinstance(sub_v, (str, int, float, bool)):
+                    span.set_attribute(f"ai.{key}.{sub_k}", sub_v)
+                else:
+                    span.set_attribute(f"ai.{key}.{sub_k}", str(sub_v))
+                    
+        elif isinstance(value, (str, int, float, bool)):
+            span.set_attribute(f"ai.{key}", value)
+        else:
+            span.set_attribute(f"ai.{key}", str(value))

ai_release_metadata-0.1.0/ai_release_metadata/integrations/structlog.py

@@ -0,0 +1,12 @@
+from typing import Any, Dict
+from ..core.context import get_current_context
+
+def structlog_processor(logger: Any, method_name: str, event_dict: Dict[str, Any]) -> Dict[str, Any]:
+    """Structlog processor that appends the active AI release metadata to the log event."""
+    context = get_current_context()
+    if context:
+        # Append release context under the 'ai' key
+        context_data = context.as_dict()
+        if context_data:
+            event_dict["ai"] = context_data
+    return event_dict

ai_release_metadata-0.1.0/ai_release_metadata/plugins/__init__.py

@@ -0,0 +1,6 @@
+from .base import MetadataPlugin
+from .env import EnvPlugin
+from .git import GitPlugin
+from .github import GitHubActionsPlugin
+
+__all__ = ["MetadataPlugin", "EnvPlugin", "GitPlugin", "GitHubActionsPlugin"]

ai_release_metadata-0.1.0/ai_release_metadata/plugins/base.py

@@ -0,0 +1,10 @@
+from typing import Dict, Any, Protocol
+
+class MetadataPlugin(Protocol):
+    """Protocol for all AI Release metadata discovery plugins."""
+    
+    def extract(self) -> Dict[str, Any]:
+        """Extract metadata and return it as a dictionary.
+        This dictionary will be merged into the ReleaseMetadata on startup.
+        """
+        ...

ai_release_metadata-0.1.0/ai_release_metadata/plugins/env.py

@@ -0,0 +1,19 @@
+import os
+from typing import Dict, Any
+from .base import MetadataPlugin
+
+class EnvPlugin(MetadataPlugin):
+    """Extracts metadata from environment variables."""
+    
+    def extract(self) -> Dict[str, Any]:
+        metadata = {}
+        if git_sha := os.environ.get("GIT_COMMIT") or os.environ.get("GITHUB_SHA"):
+            metadata["git_sha"] = git_sha
+            
+        if env := os.environ.get("ENVIRONMENT") or os.environ.get("APP_ENV"):
+            metadata["environment"] = env
+            
+        if version := os.environ.get("DEPLOYMENT_VERSION") or os.environ.get("APP_VERSION"):
+            metadata["deployment_version"] = version
+            
+        return metadata

ai_release_metadata-0.1.0/ai_release_metadata/plugins/git.py

@@ -0,0 +1,25 @@
+import subprocess
+from typing import Dict, Any
+from .base import MetadataPlugin
+
+class GitPlugin(MetadataPlugin):
+    """Extracts metadata from the local Git repository using subprocess."""
+    
+    def extract(self) -> Dict[str, Any]:
+        metadata = {}
+        try:
+            # Run git rev-parse HEAD to get the current commit SHA
+            result = subprocess.run(
+                ["git", "rev-parse", "HEAD"], 
+                capture_output=True, 
+                text=True, 
+                check=True
+            )
+            sha = result.stdout.strip()
+            if sha:
+                metadata["git_sha"] = sha
+        except (subprocess.CalledProcessError, FileNotFoundError):
+            # Fails gracefully if git is not installed or not in a git repo
+            pass
+            
+        return metadata

ai_release_metadata-0.1.0/ai_release_metadata/plugins/github.py

@@ -0,0 +1,23 @@
+import os
+from typing import Dict, Any
+from .base import MetadataPlugin
+
+class GitHubActionsPlugin(MetadataPlugin):
+    """Extracts metadata specific to GitHub Actions environment variables."""
+    
+    def extract(self) -> Dict[str, Any]:
+        metadata = {}
+        
+        # GITHUB_SHA is the commit that triggered the workflow
+        if git_sha := os.environ.get("GITHUB_SHA"):
+            metadata["git_sha"] = git_sha
+            
+        # GITHUB_RUN_ID is a unique number for each workflow run, great for deployment version
+        if run_id := os.environ.get("GITHUB_RUN_ID"):
+            metadata["deployment_version"] = f"gha-{run_id}"
+            
+        # GITHUB_REF_NAME is the branch or tag name
+        if ref := os.environ.get("GITHUB_REF_NAME"):
+            metadata["environment"] = ref
+            
+        return metadata

ai_release_metadata-0.1.0/ai_release_metadata.egg-info/PKG-INFO

@@ -0,0 +1,169 @@
+Metadata-Version: 2.4
+Name: ai-release-metadata
+Version: 0.1.0
+Summary: A lightweight, framework-agnostic Python SDK for standardizing AI release metadata across production applications.
+Author-email: Oura <opensource@example.com>
+License: MIT
+Project-URL: Homepage, https://github.com/ai-release-metadata/ai-release-metadata
+Project-URL: Bug Tracker, https://github.com/ai-release-metadata/ai-release-metadata/issues
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+Provides-Extra: otel
+Requires-Dist: opentelemetry-api; extra == "otel"
+
+# AI Release Metadata (ai-release-metadata)
+
+> **A lightweight SDK for standardizing and propagating operational metadata for AI-powered services through existing observability systems.**
+
+## The Problem
+
+When a production issue occurs in an AI-powered service (e.g., regressions, prompt injection, or unexpected model behaviour), it is often extremely difficult to quickly identify:
+* Which prompt version generated the response?
+* Which model was used?
+* Which Git commit or deployment version is responsible?
+* What experiments were active?
+
+While existing observability platforms effectively capture execution metrics such as latency and token usage, manually correlating these spans back to the exact release configuration often results in fragmented and inconsistent metadata.
+
+## The Solution
+
+This SDK provides a minimal set of context managers and decorators that automatically enrich existing logs, metrics, and traces with standardized release and environment metadata.
+
+### Architecture
+
+```mermaid
+flowchart TD
+    App[Application] -->|with release_context()| Provider[MetadataProvider]
+    Provider -->|extract| Plugins["Plugins (Git, Env)"]
+    Plugins --> Context(("Release Context"))
+    Context -->|get_current_context()| Exporters["Exporters"]
+    Exporters --> OTEL["OpenTelemetry / Structlog"]
+```
+
+### Design Principles
+- **Minimal developer overhead:** Drop-in context managers and decorators.
+- **Existing Observability First:** Integrates directly into your current logging/tracing pipelines instead of requiring a new dashboard.
+- **A reusable building block:** This explores one reusable approach for improving traceability between production behaviour and deployed AI releases.
+- **Framework agnostic:** Works with FastAPI, Django, or raw scripts.
+- **Vendor agnostic:** Does not wrap or depend on specific LLM provider SDKs.
+- **Explicit over magic:** Developers explicitly define trace boundaries.
+- **Existing observability first:** Propagates metadata to your existing tools (like OpenTelemetry) rather than acting as a standalone backend.
+- **Async-safe by default:** Thread-safe and async-safe context propagation.
+
+### Features
+* **Automatic Metadata Discovery:** Detects release and runtime information from the execution environment through a pluggable metadata system.
+* **Application-level Instrumentation:** Instruments business operations instead of wrapping LLM SDKs, allowing the library to remain provider-agnostic.
+* **Integrations:** Natively exports into `structlog` (JSON logs) and OpenTelemetry.
+* **Async First:** Safe to use in high-throughput `asyncio` applications such as FastAPI.
+
+## Use Cases
+
+### 1. Diagnosing Latency Regressions by Git Commit
+If latency spikes on a specific LLM feature, you can query Datadog/Grafana for all `generate` traces grouped by `ai.git_sha`. The SDK ensures this metadata is accurately attached to every span, instantly identifying the deployment that caused the regression.
+
+### 2. A/B Testing Prompt Templates
+When rolling out a new prompt (e.g. `prompt_version="v2.1"`), you can seamlessly track the token usage, error rates, and user feedback specifically for that variant.
+
+## Documentation
+
+- [Architecture](docs/architecture.md): How context propagation works under the hood.
+- [Design Decisions](docs/design-decisions.md): The rationale behind the core architectural choices.
+- [Metadata Schema](docs/metadata-schema.md): The full list of supported fields.
+- [Plugin Development](docs/plugin-development.md): How to build custom discovery sources.
+- [Exporters](docs/exporters.md): How to export data to OpenTelemetry, structlog, and more.
+- [Roadmap](docs/roadmap.md): Future development ideas and planned integrations.
+
+## Installation
+
+```bash
+pip install ai-release-metadata
+```
+
+To enable the OpenTelemetry integration automatically, install the `otel` extra:
+```bash
+pip install "ai-release-metadata[otel]"
+```
+
+## Usage
+
+Configure the SDK once at application startup using the `MetadataProvider`. 
+
+**Plugin Precedence Rules:** The SDK resolves metadata conflicts based on the order plugins are provided. The *last* plugin evaluated takes precedence over the preceding ones. We recommend putting local plugins first, and explicit environment plugins last.
+
+```python
+from ai_release_metadata import MetadataProvider
+from ai_release_metadata.plugins import EnvPlugin, GitPlugin, GitHubActionsPlugin
+
+# 1. GitPlugin tries to guess from local filesystem
+# 2. GitHubActionsPlugin overwrites with CI variables
+# 3. EnvPlugin overwrites with explicit user-defined env vars
+MetadataProvider(plugins=[GitPlugin(), GitHubActionsPlugin(), EnvPlugin()])
+```
+
+Use the context manager (`release_context`) or decorator (`@capture_generation`) within the business logic:
+
+```python
+from ai_release_metadata import release_context, get_current_context
+
+async def generate_response(user_id: str, query: str):
+    # Start a release context block
+    with release_context(feature="customer-support", model="gpt-4o", prompt_version="v2.1"):
+        
+        # ... LLM calling logic ...
+        
+        # Dynamically append runtime information to the context
+        ctx = get_current_context()
+        if ctx:
+            ctx.add_document("doc_1")
+            ctx.add_tag("user_tier", "premium")
+```
+
+All logs emitted within the `with release_context(...)` block will automatically have the metadata appended under the `ai` key.
+
+### OpenTelemetry Integration
+
+If you use OpenTelemetry, you can automatically enrich the currently active span with the release context. It gracefully ignores spans if none are active.
+
+```python
+from ai_release_metadata.integrations.opentelemetry import enrich_span
+from opentelemetry import trace
+
+tracer = trace.get_tracer(__name__)
+
+with tracer.start_as_current_span("generate_response") as span:
+    with release_context(model="gpt-4o"):
+        # Enrich the span with git_sha, environment, model, etc.
+        enrich_span()
+        
+        # ... LLM calling logic ...
+```
+
+## Running the Demo
+
+This repository includes a FastAPI demo simulating a generation endpoint.
+
+1. Activate a virtual environment and install dependencies:
+```bash
+python3 -m venv venv
+source venv/bin/activate
+pip install structlog fastapi uvicorn pydantic
+```
+
+2. Run the application:
+```bash
+export GIT_COMMIT=a1b2c3d 
+export ENVIRONMENT=local 
+export LOG_FORMAT=console
+PYTHONPATH=. uvicorn demo.app:app
+```
+
+3. Send a request to observe the enriched telemetry in the terminal:
+```bash
+curl -X POST http://127.0.0.1:8000/generate \
+  -H "Content-Type: application/json" \
+  -d '{"model": "gpt-4o", "prompt_version": "v2.0", "user_query": "I cannot login"}'
+```

ai_release_metadata-0.1.0/ai_release_metadata.egg-info/SOURCES.txt

@@ -0,0 +1,23 @@
+README.md
+pyproject.toml
+ai_release_metadata/__init__.py
+ai_release_metadata.egg-info/PKG-INFO
+ai_release_metadata.egg-info/SOURCES.txt
+ai_release_metadata.egg-info/dependency_links.txt
+ai_release_metadata.egg-info/requires.txt
+ai_release_metadata.egg-info/top_level.txt
+ai_release_metadata/core/context.py
+ai_release_metadata/core/models.py
+ai_release_metadata/core/sdk.py
+ai_release_metadata/integrations/base.py
+ai_release_metadata/integrations/opentelemetry.py
+ai_release_metadata/integrations/structlog.py
+ai_release_metadata/plugins/__init__.py
+ai_release_metadata/plugins/base.py
+ai_release_metadata/plugins/env.py
+ai_release_metadata/plugins/git.py
+ai_release_metadata/plugins/github.py
+tests/test_context.py
+tests/test_e2e_fastapi.py
+tests/test_integrations.py
+tests/test_plugins.py

ai_release_metadata-0.1.0/ai_release_metadata.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

ai_release_metadata-0.1.0/ai_release_metadata.egg-info/requires.txt

@@ -0,0 +1,3 @@
+
+[otel]
+opentelemetry-api

ai_release_metadata-0.1.0/ai_release_metadata.egg-info/top_level.txt

@@ -0,0 +1 @@
+ai_release_metadata

ai_release_metadata-0.1.0/pyproject.toml

@@ -0,0 +1,31 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ai-release-metadata"
+version = "0.1.0"
+authors = [
+    { name="Oura", email="opensource@example.com" },
+]
+description = "A lightweight, framework-agnostic Python SDK for standardizing AI release metadata across production applications."
+readme = "README.md"
+requires-python = ">=3.7"
+license = { text = "MIT" }
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "Operating System :: OS Independent",
+    "Intended Audience :: Developers",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = []
+
+[project.optional-dependencies]
+otel = ["opentelemetry-api"]
+
+[project.urls]
+"Homepage" = "https://github.com/ai-release-metadata/ai-release-metadata"
+"Bug Tracker" = "https://github.com/ai-release-metadata/ai-release-metadata/issues"
+
+[tool.setuptools.packages.find]
+include = ["ai_release_metadata*"]

ai_release_metadata-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

ai_release_metadata-0.1.0/tests/test_context.py

@@ -0,0 +1,69 @@
+import asyncio
+import pytest
+from ai_release_metadata.core.context import release_context, capture_generation, get_current_context
+
+def test_release_context_manager():
+    # Outside context, should be None
+    assert get_current_context() is None
+    
+    with release_context(feature="test-feature", prompt_version="v1"):
+        ctx = get_current_context()
+        assert ctx is not None
+        assert ctx.feature == "test-feature"
+        assert ctx.prompt_version == "v1"
+        assert ctx.model is None
+        
+        # Test updating runtime context
+        ctx.retrieved_documents.append("doc-1")
+        assert get_current_context().retrieved_documents == ["doc-1"]
+        
+    # After context, should reset to None
+    assert get_current_context() is None
+
+def test_capture_generation_decorator_sync():
+    @capture_generation(feature="decorated-feature")
+    def sync_dummy():
+        ctx = get_current_context()
+        return ctx.feature if ctx else None
+        
+    assert get_current_context() is None
+    assert sync_dummy() == "decorated-feature"
+    assert get_current_context() is None
+
+@pytest.mark.asyncio
+async def test_capture_generation_decorator_async():
+    @capture_generation(model="gpt-4o")
+    async def async_dummy():
+        ctx = get_current_context()
+        # Simulate some async work
+        await asyncio.sleep(0.01)
+        return ctx.model if ctx else None
+        
+    assert get_current_context() is None
+    assert await async_dummy() == "gpt-4o"
+    assert get_current_context() is None
+
+def test_nested_release_context():
+    with release_context(feature="parent", model="gpt-4"):
+        ctx = get_current_context()
+        ctx.tags["user_tier"] = "premium"
+        ctx.experiment_flags["new_rag"] = True
+        
+        with release_context(feature="child", prompt_version="v2"):
+            child_ctx = get_current_context()
+            # Inherited tags/experiments
+            assert child_ctx.tags["user_tier"] == "premium"
+            assert child_ctx.experiment_flags["new_rag"] is True
+            # Overridden properties
+            assert child_ctx.feature == "child"
+            assert child_ctx.prompt_version == "v2"
+            # Inherited model from parent because it wasn't specified in child
+            assert child_ctx.model == "gpt-4"
+            
+            # Modifying child shouldn't modify parent due to copy()
+            child_ctx.tags["child_specific"] = "yes"
+            
+        # Back to parent
+        parent_restored = get_current_context()
+        assert parent_restored.feature == "parent"
+        assert "child_specific" not in parent_restored.tags

ai_release_metadata-0.1.0/tests/test_e2e_fastapi.py

@@ -0,0 +1,73 @@
+import os
+import json
+import pytest
+from fastapi.testclient import TestClient
+
+@pytest.fixture
+def e2e_env(monkeypatch):
+    """Sets up the environment exactly as it would be in a CI/CD pipeline."""
+    monkeypatch.setenv("GIT_COMMIT", "e2e-test-sha")
+    monkeypatch.setenv("ENVIRONMENT", "e2e")
+    monkeypatch.setenv("LOG_FORMAT", "json") # Ensure structlog outputs raw JSON for parsing
+    
+    # Reset the global metadata provider so the new environment variables take effect
+    from ai_release_metadata.core.sdk import MetadataProvider
+    MetadataProvider._global_instance = None
+
+def test_fastapi_e2e_flow(e2e_env, caplog):
+    import logging
+    caplog.set_level(logging.INFO)
+    """
+    True E2E test:
+    1. Loads the FastAPI app
+    2. Sends a real HTTP request
+    3. Intercepts the terminal output
+    4. Asserts that the 'ai' metadata survived the async stack and printed correctly
+    """
+    # Import inside the test so environment variables are applied during module load
+    import demo.app
+    import importlib
+    
+    # Force reload in case other tests imported it first
+    importlib.reload(demo.app)
+    
+    client = TestClient(demo.app.app)
+    
+    # Send a request to the server
+    response = client.post(
+        "/generate",
+        json={
+            "model": "gpt-4o",
+            "prompt_version": "v1.0",
+            "user_query": "hello"
+        }
+    )
+    
+    assert response.status_code == 200
+    
+    # Capture the structlog output pushed to standard logging
+    # Structlog serializes the dictionary into a JSON string in record.message
+    found_target_log = False
+    
+    for record in caplog.records:
+        try:
+            log_data = json.loads(record.message)
+        except json.JSONDecodeError:
+            continue
+            
+        if log_data.get("message") == "Deep research complete" or log_data.get("event") == "Deep research complete":
+            found_target_log = True
+            ai_meta = log_data.get("ai", {})
+            
+            # Assert Global Context propagated (from EnvPlugin)
+            assert ai_meta.get("git_sha") == "e2e-test-sha"
+            
+            # Assert Top-Level Request Context propagated (from app.post)
+            assert ai_meta.get("model") == "gpt-4o"
+            assert ai_meta.get("prompt_version") == "v1.0"
+            
+            # Assert Decorator Context propagated (from @capture_generation)
+            assert ai_meta.get("feature") == "comprehensive-answer"
+            assert ai_meta.get("experiment_flags", {}).get("use_deep_research") is True
+            
+    assert found_target_log, f"Did not find the expected log event. Captured messages: {[r.message for r in caplog.records]}"

ai_release_metadata-0.1.0/tests/test_integrations.py

@@ -0,0 +1,61 @@
+import pytest
+from opentelemetry import trace
+from ai_release_metadata.core.context import release_context
+from ai_release_metadata.integrations.opentelemetry import enrich_span
+
+class MockSpan(trace.Span):
+    def __init__(self):
+        self.attributes = {}
+        self._is_recording = True
+        
+    def set_attribute(self, key: str, value) -> None:
+        self.attributes[key] = value
+        
+    def set_attributes(self, attributes) -> None:
+        for k, v in attributes.items():
+            self.attributes[k] = v
+            
+    def is_recording(self) -> bool:
+        return self._is_recording
+        
+    def end(self, end_time=None) -> None:
+        pass
+        
+    def get_span_context(self) -> trace.SpanContext:
+        return trace.SpanContext(trace_id=1, span_id=1, is_remote=False)
+        
+    def set_status(self, status, description=None) -> None:
+        pass
+        
+    def update_name(self, name) -> None:
+        pass
+        
+    def record_exception(self, exception, attributes=None, timestamp=None, escaped=False) -> None:
+        pass
+        
+    def add_event(self, name, attributes=None, timestamp=None) -> None:
+        pass
+        
+    def add_link(self, context, attributes=None) -> None:
+        pass
+
+def test_otel_enrich_span():
+    span = MockSpan()
+    
+    with release_context(feature="search", model="gpt-4o") as ctx:
+        ctx.git_sha = "abc1234"
+        ctx.tags["user_type"] = "admin"
+        ctx.retrieved_documents = ["docA", "docB"]
+        
+        enrich_span(span)
+        
+    assert span.attributes["ai.feature"] == "search"
+    assert span.attributes["ai.model"] == "gpt-4o"
+    assert span.attributes["ai.git_sha"] == "abc1234"
+    assert span.attributes["ai.tags.user_type"] == "admin"
+    assert span.attributes["ai.retrieved_documents"] == ["docA", "docB"]
+
+def test_otel_enrich_span_no_active_context():
+    span = MockSpan()
+    enrich_span(span)
+    assert not span.attributes # Should do nothing without crashing

ai_release_metadata-0.1.0/tests/test_plugins.py

@@ -0,0 +1,76 @@
+import os
+from ai_release_metadata.plugins.env import EnvPlugin
+
+def test_env_plugin():
+    os.environ["GIT_COMMIT"] = "abcdef"
+    os.environ["ENVIRONMENT"] = "staging"
+    
+    extractor = EnvPlugin()
+    data = extractor.extract()
+    assert data["git_sha"] == "abcdef"
+    assert data["environment"] == "staging"
+
+def test_config_merging():
+    os.environ["GIT_COMMIT"] = "123456"
+    from ai_release_metadata.core.sdk import MetadataProvider
+    MetadataProvider(plugins=[EnvPlugin()])
+    
+    # Re-evaluate logic to ensure context propagation still works under new config
+    from ai_release_metadata.core.context import release_context, get_current_context
+    with release_context(feature="demo"):
+        ctx = get_current_context()
+        assert ctx.feature == "demo"
+        assert ctx.git_sha == "123456"
+
+def test_github_actions_plugin(monkeypatch):
+    monkeypatch.setenv("GITHUB_SHA", "gha123")
+    monkeypatch.setenv("GITHUB_RUN_ID", "9876")
+    monkeypatch.setenv("GITHUB_REF_NAME", "main")
+    
+    from ai_release_metadata.plugins.github import GitHubActionsPlugin
+    plugin = GitHubActionsPlugin()
+    data = plugin.extract()
+    
+    assert data["git_sha"] == "gha123"
+    assert data["deployment_version"] == "gha-9876"
+    assert data["environment"] == "main"
+
+def test_git_plugin_mocked(monkeypatch):
+    import subprocess
+    
+    # Mock subprocess.run to simulate a valid git repo
+    def mock_run(*args, **kwargs):
+        class MockResult:
+            stdout = "mock-sha-890\n"
+        return MockResult()
+        
+    monkeypatch.setattr(subprocess, "run", mock_run)
+    
+    from ai_release_metadata.plugins.git import GitPlugin
+    plugin = GitPlugin()
+    data = plugin.extract()
+    
+    assert data["git_sha"] == "mock-sha-890"
+
+def test_custom_plugin_extra_fields():
+    from typing import Dict, Any
+    
+    class CustomCompanyPlugin:
+        def extract(self) -> Dict[str, Any]:
+            return {
+                "git_sha": "custom123",
+                "company_org": "finance",
+                "cost_center": "4455"
+            }
+            
+    from ai_release_metadata.core.sdk import MetadataProvider
+    provider = MetadataProvider(plugins=[CustomCompanyPlugin()])
+    
+    base_meta = provider.get_base_metadata()
+    
+    # Core field should be mapped
+    assert base_meta.git_sha == "custom123"
+    
+    # Unknown fields should be dumped into 'extra'
+    assert base_meta.extra["company_org"] == "finance"
+    assert base_meta.extra["cost_center"] == "4455"