runtime-labs-sdk 0.1.0__tar.gz

runtime_labs_sdk-0.1.0/.gitignore

@@ -0,0 +1,22 @@
+.venv/
+.env
+.env.*
+!.env.example
+.pytest_cache/
+.ruff_cache/
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+*.jsonl
+.scanner-recordings/
+.scanner-cache/
+.scanner-checkpoints/
+.scanner-runs/
+
+# Local app state and frontend build artifacts
+.runtime-labs/
+node_modules/
+*.tsbuildinfo
+playwright-report/
+test-results/

runtime_labs_sdk-0.1.0/PKG-INFO

@@ -0,0 +1,382 @@
+Metadata-Version: 2.5
+Name: runtime-labs-sdk
+Version: 0.1.0
+Summary: OpenTelemetry tracing with eval payloads and verified source identities
+Project-URL: Repository, https://github.com/arpandesai0/runtime-labs
+Project-URL: Documentation, https://github.com/arpandesai0/runtime-labs/blob/main/packages/sdk/README.md
+Requires-Python: <3.14,>=3.11
+Requires-Dist: httpx<1,>=0.28
+Requires-Dist: opentelemetry-api==1.45.1
+Requires-Dist: opentelemetry-exporter-otlp-proto-http==1.45.1
+Requires-Dist: opentelemetry-sdk==1.45.1
+Requires-Dist: python-dotenv<2,>=1.1
+Requires-Dist: runtime-labs-contracts
+Provides-Extra: google
+Requires-Dist: google-genai==2.27.0; extra == 'google'
+Provides-Extra: langgraph
+Requires-Dist: langgraph==1.2.18; extra == 'langgraph'
+Provides-Extra: otlp
+Requires-Dist: opentelemetry-exporter-otlp-proto-http==1.45.1; extra == 'otlp'
+Description-Content-Type: text/markdown
+
+# Runtime Labs local Python SDK
+
+The SDK emits OpenTelemetry spans with full eval payloads for LangGraph execution
+and Google Gen AI non-streaming model calls. The scanner's strict `DeploymentManifest`
+adds `runtime_labs.node_id` when source proof succeeds. Tracing also works with
+missing or stale manifests: those operations remain unattributed and the handle
+reports fixed diagnostic codes.
+
+## Install and initialize
+
+From this workspace:
+
+```bash
+uv sync
+uv run python examples/sdk_offline/demo.py
+```
+
+For an application with its own environment, install both local distributions
+(substitute the checkout directory):
+
+```bash
+uv pip install /path/to/runtime-labs/packages/contracts \
+  '/path/to/runtime-labs/packages/sdk[google,langgraph]'
+```
+
+`runtime-labs-sdk` is independently packaged and has no scanner dependency. The
+Google and LangGraph adapters are optional and loaded at initialization; importing
+`runtime_labs` does not import either integration. Unsupported dependency versions
+are diagnosed and their adapters are skipped.
+
+Connect and scan the application checkout in the web UI first. Run the application
+inside its Git checkout with an `origin` remote. No manifest file is downloaded or
+configured by the customer. The SDK reads that remote and asks the backend for the
+successful scan's manifest at startup. SSH and HTTPS remotes normalize to the same
+host/owner/repository identity.
+
+Optional application `.env` (the default is shown):
+
+```dotenv
+RUNTIME_LABS_API_URL=http://127.0.0.1:8000
+```
+
+At startup, before constructing provider clients or building/executing graphs:
+
+```python
+import runtime_labs
+
+sdk = runtime_labs.init(api_key="runtime-labs-local-dev-key")
+
+try:
+    # Normal application startup and execution.
+    ...
+finally:
+    sdk.flush()
+    sdk.shutdown()
+```
+
+The API key is sent to the backend as an Authorization bearer header for manifest
+lookup and automatic telemetry delivery. Authentication and workspace lookup are
+currently deferred: the backend uses the default tenant/workspace. The key is
+hidden from configuration representations and is never included in span payloads
+or provider credentials.
+
+Configuration rules:
+
+- `api_url=` overrides process `RUNTIME_LABS_API_URL`, then the nearest `.env` value,
+  then the local default above. SDK settings are read without interpolation or
+  mutating the application's environment.
+- By default the SDK resolves the checkout's `origin` and HEAD, requests
+  `POST /api/sdk/manifest`, validates the strict deployment manifest, and holds it
+  in memory. No manifest cache file or customer download step is required.
+- The backend resolves the complete canonical Git identity, reads the successful
+  scan's saved artifact, and returns the manifest and repository ingestion route.
+  Duplicate matching checkouts, unavailable artifacts, and revision mismatches
+  are rejected rather than guessed.
+- Existing source-path, source-digest, and loaded-code proofs still apply.
+- For explicit offline/test use only, `RUNTIME_LABS_MANIFEST_PATH` remains a legacy
+  override. Paths retain their existing `~` and nearest-`.env` resolution rules.
+  An empty override enables normal runtime fetching instead of disabling attribution.
+- A failed lookup never breaks application execution. Inspect `sdk.diagnostics`
+  for `repository_identity_unavailable`, `manifest_fetch_failed`, or
+  `manifest_revision_mismatch`. Unsupported or unverified spans stay unattributed.
+- Deployed environments must retain Git metadata for automatic remote discovery.
+  Initialization fetches once with a five-second HTTP timeout; retry after fixing
+  backend/scan availability by shutting down and initializing again.
+
+## Public API and local output
+
+`init(*, api_key, exporter=None, tracer_provider=None, capture_payloads=True, api_url=None)`
+returns an SDK handle with:
+
+- `diagnostics`: a sorted tuple of sanitized, deduplicated diagnostic codes.
+- `sources.manifest`: a detached strict-contract snapshot; editing it cannot change
+  the immutable attribution indexes. Repository/revision metadata stay here.
+- `flush(timeout_millis=30000) -> bool`: flush only this SDK's exporter.
+- `shutdown()`: flush, disable instrumentation, restore owned hooks, and release
+  SDK-owned resources. It is safe to call repeatedly.
+
+Identical initialization returns the same handle. A different key, resolved path,
+exporter, explicit provider or payload setting returns that handle with
+`initialization_conflict`,
+without installing more hooks. Shut down and initialize again to change settings;
+construct new graphs after reinitialization. Shutdown should follow completion or
+cancellation of application work; it does not await running requests.
+
+After successful runtime bootstrap, the SDK automatically exports OTLP/HTTP spans
+to the repository route supplied by the backend. The OTLP exporter is included in
+the base SDK dependency set. Explicit exporters and standard `OTEL_*` settings
+remain supported; `OTEL_TRACES_EXPORTER=none` disables delivery. Offline manifest
+mode retains the console default. Console records contain trace/span/parent IDs,
+start/end timestamps, duration, status, and allowed attributes including application
+payloads. HTTP authentication, environment dumps, and source bodies are not captured.
+Errors use only `error.type` values `cancelled`, `timeout` or `_OTHER`; automatic
+exception events and status descriptions are disabled. Application code and other
+instrumentation retain their own logging/export policies.
+
+## Payloads for evals
+
+Full payload capture is enabled by default. The MVP performs **no redaction or SDK
+size limiting**. Each payload attribute is a JSON string, supported by standard
+OTel exporters. Decode it with `json.loads(span.attributes[attribute_name])`.
+
+| Span | Payload attributes |
+| --- | --- |
+| Workflow / agent | `runtime_labs.input`, `runtime_labs.output`: input state and returned value |
+| Google model call | `runtime_labs.request`: original model, contents and supplied generation config; `runtime_labs.response`: returned response body, candidates, usage, parsed output and automatic tool-calling history |
+| Google model call | `gen_ai.input.messages`, `gen_ai.output.messages`: standard message/part representation; `gen_ai.system_instructions` when supplied |
+| Tool execution | `gen_ai.tool.call.arguments`, `gen_ai.tool.call.result`: bound arguments and returned value |
+
+The `runtime_labs.*` payload attributes carry application data, not additional
+source identity fields. `runtime_labs.node_id` remains the only custom source ID.
+Prompts, completions, all returned candidates and tool payloads are stored in full.
+Raw provider request/response attributes retain provider-specific body fields;
+HTTP options and `sdk_http_response` transport envelopes are not body payloads.
+The SDK does not inspect client credentials or automatically export its API key.
+
+Request/input snapshots are taken before execution, so in-place application or
+Google automatic function-calling mutations cannot rewrite the original input.
+The returned automatic function-calling history retains its intermediate tool
+calls/results. Workflow outputs contain the final graph state, including the
+application's answer, enabling complete input/output evaluation examples. Runtime
+trace and span IDs join these payloads to the manifest's source node IDs.
+
+JSON-compatible values, Pydantic models, dataclasses, enums, dates and bytes are
+supported; bytes are encoded as base64. Callable tool definitions store callable
+names and qualified names rather than executable code. Unsupported or cyclic
+values produce `payload_capture_failed` and omit that payload attribute while
+preserving application results/exceptions and other span data. A failed or
+cancelled operation keeps its input; it does not invent a response.
+
+To select metadata-only output explicitly:
+
+```python
+sdk = runtime_labs.init(api_key="runtime-labs-local-dev-key", capture_payloads=False)
+```
+
+OTel/application processors and downstream collectors can have their own limits;
+the Runtime Labs SDK does not introduce payload truncation. Local JSON and OTLP
+export receive the same payload attributes. Evaluation execution/dataset storage
+are separate later components; this change supplies their runtime examples.
+
+For assertions, supply the standard OTel in-memory exporter:
+
+```python
+from runtime_labs.export import InMemorySpanExporter
+
+exporter = InMemorySpanExporter()
+sdk = runtime_labs.init(api_key="runtime-labs-local-dev-key", exporter=exporter)
+# Execute work; then inspect exporter.get_finished_spans().
+```
+
+The explicitly selected in-memory exporter retains spans until `clear()` or
+`shutdown()`; applications should bound its use. Custom exporters use the standard
+OTel `SpanExporter` interface. Export/flush/shutdown exceptions become diagnostic
+codes and do not replace application results or exceptions. Local export is
+synchronous, so exporter work contributes latency.
+
+An application-owned global `TracerProvider` is respected. An optional explicit
+`tracer_provider` also works. The SDK adds one reusable processor scoped to
+`runtime_labs`, never shuts down an application provider, and does not export its
+other instrumentation scopes through the local exporter. If no application provider
+exists, the SDK creates a private OTel provider without taking the global slot.
+Active OTel context still establishes parentage across both providers. Application
+processors on a shared provider can also observe SDK spans.
+
+Remote OTLP/HTTP export requires **both** explicit standard settings:
+
+```bash
+uv pip install '/path/to/runtime-labs/packages/sdk[otlp]'
+export OTEL_TRACES_EXPORTER=otlp
+export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces
+export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
+```
+
+`OTEL_EXPORTER_OTLP_ENDPOINT`, generic protocol/timeout/header settings and the
+trace-specific settings are handled by the standard exporter. HTTP/protobuf is the
+supported protocol. An endpoint alone never enables remote export; missing extras,
+missing endpoint or unsupported protocol produce diagnostics and local fallback.
+The pinned exporter's per-instance transport logger is silenced so server reasons
+and transport exceptions cannot echo private text; failures use sanitized SDK
+diagnostics. Its optional internal metrics use a no-op provider.
+`OTEL_TRACES_EXPORTER=none` discards SDK output. No collector/server is included.
+
+## Correlation identifiers and trace context
+
+The SDK records the identifiers a framework uses to tie runs together, verbatim,
+and never decides what they mean. On every LangGraph `invoke`/`ainvoke` it copies
+allowlisted keys from the run config onto the workflow span as
+`runtime_labs.ctx.<framework>.<key>`, for example
+`runtime_labs.ctx.langgraph.thread_id`. The defaults are LangGraph's `thread_id`,
+`checkpoint_id`, `checkpoint_ns` and `run_id`. Only allowlisted keys leave the
+process, because applications often keep user IDs and secrets in the same config;
+only non-empty strings, integers and UUIDs are copied, truncated to 256 characters.
+
+Extend the allowlist without code changes:
+
+```bash
+export RUNTIME_LABS_CORRELATION_KEYS=langgraph.tenant,langgraph.session
+```
+
+or with `runtime_labs.init(api_key=..., correlation_keys={"langgraph": ["tenant"]})`.
+Runtime Labs pipelines map these raw identifiers to concepts such as conversations
+with rules that are data, so new frameworks need a rule, not an SDK release.
+
+Within one process, span parents follow OpenTelemetry context, which LangGraph and
+asyncio carry into their own threads and tasks. Work that crosses a process or
+service boundary (an HTTP call, a queue message, an outbox row) must carry the
+context itself, as a standard W3C `traceparent`:
+
+```python
+headers = runtime_labs.inject_context()        # producer
+with runtime_labs.continue_trace(headers):     # consumer
+    handle(message)
+```
+
+Code that starts raw threads should run work with
+`contextvars.copy_context().run(...)` to keep the parent.
+
+## Supported interfaces and conventions
+
+| Component | Pinned/tested version | Coverage |
+| --- | --- | --- |
+| CPython | 3.11–3.13 | Code identity and UTF-8 call positions |
+| Google Gen AI | 2.27.0 | `Models.generate_content`, `AsyncModels.generate_content` |
+| LangGraph | 1.2.18 | `StateGraph` construction, `add_node`, `compile`; compiled `invoke`/`ainvoke` |
+| LangChain Core | 1.7.0 (workspace lock) | LangGraph callback signature and context propagation |
+| OTel API/SDK | 1.45.1 | Context, tracing, processors and standard exporters |
+| OTel OTLP/HTTP exporter | 1.45.1 (optional) | Explicit HTTP/protobuf export |
+| OTel Python semantic-conventions package | 0.66b1 (SDK dependency) | SDK resource conventions |
+| GenAI conventions | `51a0f580f8a6ad804ca4de5e033683c6fbbe5906` | Development conventions, mapping in `semconv.py` |
+
+Verified official references: [OTel Python instrumentation](https://opentelemetry.io/docs/languages/python/instrumentation/),
+[GenAI inference conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/51a0f580f8a6ad804ca4de5e033683c6fbbe5906/docs/gen-ai/client-inference.md),
+[agent/workflow/tool conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/51a0f580f8a6ad804ca4de5e033683c6fbbe5906/docs/gen-ai/gen-ai-agent-spans.md),
+[Google SDK](https://github.com/googleapis/python-genai/tree/v2.27.0),
+[LangGraph](https://docs.langchain.com/oss/python/langgraph/overview), and
+[streaming boundaries](https://docs.langchain.com/oss/python/langgraph/streaming).
+GenAI conventions remain in development; their revision is explicit rather than
+silently adopting future changes.
+
+Provider spans use `generate_content`, `CLIENT`, and `gcp.vertex_ai`, `gcp.gemini`
+or `gcp.gen_ai` according to runtime backend metadata. Request/response models come
+from actual request/response metadata. Tokens come only from provider usage:
+input = prompt count, output = candidate count plus reasoning count when reported;
+reasoning and cache-read counts are separately reported when available. Missing
+counts remain absent. For automatic function calling, the final SDK response's
+reported usage is recorded; counts from hidden HTTP rounds are not estimated or
+aggregated. One span covers the whole public logical call, including these rounds.
+Return identity, arguments, exceptions and async cancellation are preserved.
+
+Workflow, agent and tool spans use `INTERNAL` and standard `invoke_workflow`,
+`invoke_agent` and `execute_tool` operations and names. The only custom source
+attribute is `runtime_labs.node_id`. No extra source IDs or static parent IDs are
+attached. The Google adapter honors and sets standard OTel instrumentation
+suppression, and deduplicates reentry. Uncooperative third-party instrumentors that
+ignore suppression cannot be deduplicated reliably.
+
+## Source-binding policy
+
+The SDK consumes `DeploymentManifest` directly and validates all references using
+the shared strict contract. It neither scans nor executes repository source, calls
+an LLM, generates IDs, or imports modules selected by manifest data.
+
+Attribution requires a unique kind and qualified symbol match, normalized for
+Python `<locals>` names, the complete canonical manifest-relative path suffix of
+the actual module/frame file, the verified full-file SHA-256, and agreement between
+the executing code and code compiled from those bytes. Compilation is never
+execution. Checkout filenames alone are normalized out of the code comparison;
+bytecode, constants, nested code, closure layout, flags and position tables remain.
+Different absolute checkout directories therefore work without scanner paths or
+basename-only guessing.
+Direct scripts use their module spec or a canonical module name derived from the
+manifest path; source proof is still required. Verification suppresses compiler
+warnings because those warnings can otherwise echo source lines.
+
+For model calls, the executed CALL instruction must match the exact manifest
+range, including UTF-8 byte columns; same-line sync/async calls remain distinct.
+Workflow construction must fall within its unique verified constructor range.
+Registered callback objects identify nested agents. Known module-level Python
+tools and their already-loaded aliases receive wrappers; registration by itself
+never emits a tool span. Ordinary functions such as `finalize_response` receive
+no span. Stale, missing, ambiguous or unprovable code produces generic spans and
+diagnostics, with no guessed source identity.
+
+Files are checked again at each attribution, including after construction or a
+previous successful match. Preloaded code is accepted only if its executable code
+and positions agree with the current verified file. This rejects a matching disk
+file paired with different loaded bytecode/constants. It does **not** reconstruct
+historical import-time bytes: comment-only changes with identical code/positions,
+changes outside the executing code unit, mutable globals/closures, or deliberate
+code-object forgery are not a deployment attestation. The current whole-file digest
+must still match the manifest. IDs alone do not validate a deployed revision or
+promise identity reconciliation across commits.
+
+Runtime parents always come from OTel context. A direct shared-tool call is under
+the executing agent. A Google automatic tool call is under that logical provider
+span, itself under the executing agent. Static registration/containment edges do
+not override these parents. LangGraph's public runtime events primarily describe
+streaming and do not expose this example's exact Python callback identities, so
+this adapter uses narrowly tested construction/registration/execution wrappers
+without callback arguments or customer decorators.
+
+## Offline acceptance harness
+
+`examples/sdk_offline` contains an unchanged source copy of the sibling simple
+example and its saved verified manifest. The sibling application's source and
+configuration are untouched. The demo denies socket connections and uses real
+pinned clients/graphs with HTTPX mock transports and fake Vertex credentials.
+
+Each of billing, technical and general, through both sync and async graph entry
+points, asserts this six-span tree (the selected agent varies):
+
+```text
+invoke_workflow build_workflow
+  invoke_agent triage_agent
+    generate_content offline-request-model
+  invoke_agent billing_agent | technical_agent | general_agent
+    execute_tool lookup_policy
+    generate_content offline-request-model
+```
+
+All six spans in each run must have IDs in the loaded manifest. Across the three
+branches, the harness observes all ten distinct nodes, including all four model
+sites. It checks the actual tool parent on every run and prints node IDs with the
+tree. This is an observation of the offline runs, not a source coverage estimate.
+Tests additionally exercise true async callbacks/provider calls using a newly
+compiled scanner manifest, automatic tool execution, parallel requests, stale
+loaded code and failure/privacy/lifecycle cases. Tests deny network globally.
+
+## Limitations
+
+Streaming graph execution has no workflow-lifetime adapter; registered callbacks
+may still emit leaf spans. Google streaming, chats and embeddings are not adapted.
+Provider-driven tools are covered when they are already-loaded, manifest-matched
+Python functions passed to Google; arbitrary callable objects, local tool closures,
+cached pre-init callable references, decorators with replaced code, packaged/zip or
+unavailable source, and other framework/provider versions are not guaranteed.
+Graphs must be constructed after initialization. No whole-program tracing or
+instrumentation of ordinary application functions is performed. Hosted lookup,
+authentication, collectors, other providers, evaluation execution and cross-commit
+reconciliation are outside this milestone.