@saguiitay/eventhouse-ai-observability 0.1.2 → 0.2.0

package/docs/implementation-report.md

@@ -3,18 +3,21 @@
 ## Delivered
 
 A single TypeScript ESM npm package, `@saguiitay/eventhouse-ai-observability`
-(personal project; 0.1.1 published to GitHub Packages, 0.1.2 prepared in this
-checkout), with root and `/copilot` exports and ordinary internal modules in `src`.
+(personal project; 0.1.1 published and registry-verified on GitHub Packages,
+0.1.2 published and registry-verified on npmjs, **0.2.0 prepared but not
+published** in this checkout), with root, `/copilot`, `/openai`, and `/vercel` exports and ordinary internal modules in `src`.
 The former private workspace packages, OTLP wrapper, custom batching queue, and
 declaration-rewriting build step have been removed. OpenTelemetry's
-`BatchSpanProcessor`, standard `SpanExporter` interface, and official
-`OTLPTraceExporter` replace that plumbing.
+`BatchSpanProcessor` and the standard `SpanExporter` interface replace that
+plumbing. Fabric is the only built-in transport; applications can supply the
+official `OTLPTraceExporter` or another standard exporter themselves.
 
-The implementation includes bounded asynchronous batching, parent-based sampling, explicit/ambient trace context, resource attributes, privacy filtering/redaction hooks, diagnostic counters, idempotent flush/shutdown, actual SDK event integration, three examples, Fabric schema/mapping/retention assets, analytics queries, native dashboard definitions, provisioning, local receiver tests, benchmarks, and CI.
+The implementation includes bounded asynchronous batching, parent-based sampling, explicit/ambient trace context, resource attributes, privacy filtering/redaction hooks, diagnostic counters, idempotent flush/shutdown, actual SDK event integration, five runnable examples (three Copilot, one OpenAI, one Vercel), Fabric schema/mapping/retention assets, analytics queries, native dashboard definitions, provisioning, local receiver tests, benchmarks, and CI.
 
 ## Install and configure
 
-Build and pack from source, then use npm to install the generated tarball. The one-call entry point is:
+Install the published npmjs package, or build/pack this checkout and install its
+root tarball to evaluate unreleased 0.2.0. The one-call entry point is:
 
 ```typescript
 const observer = observeCopilot(existingSession, {
@@ -24,20 +27,33 @@ const observer = observeCopilot(existingSession, {
 
 Only supply `entityName` if the Fabric source's connection string omits `EntityPath`. Metadata-only capture is the default. Shutdown the observer explicitly without transferring ownership of the session/client. See the [quickstart](quickstart.md) for all options.
 
-The repository routes `@saguiitay` packages to GitHub Packages. The basic
-example's `.npmrc` uses the organization-approved Microsoft npm proxy for its
-other dependencies. Tokens belong in user-level configuration or CI secrets.
+The repository routes `@saguiitay` packages to GitHub Packages. Consumer
+dependency registries remain organization-controlled; private registry
+configuration must not be copied into tracked files or distributed assets.
+Tokens belong in user-level configuration or CI secrets.
 The earlier Azure Artifacts authentication blocker is historical: frozen installs
 and package smoke checks subsequently passed on Node 22/24 and Windows/Linux CI,
 and the operator successfully installed the published package from GitHub Packages
 into the basic example. Packed-artifact smoke tests still reuse project runtime
 dependencies and are not themselves a clean registry install.
 
+The prepared 0.2.0 npm tarball includes exactly one generated
+`fabric-setup.zip`, not raw tenant-bound Fabric source files. Install first,
+resolve it beneath `npm root`, and extract its `README.md`, `manifest.json`,
+and `fabric\` tree using the [README commands](../README.md#fabric-setup-bundle-and-ingestion-check).
+Its manifest matches the npm version and records schema/version/hash metadata.
+Public setup distribution uses npm because the source repository stays private;
+GitHub Release ZIPs and Actions artifacts remain collaborator-only alternatives.
+This policy permits only this project's authored generic public-API templates
+and synthetic samples, not Microsoft-private code/docs/configuration, tenant
+content, credentials, real telemetry, or copied external-repository files.
+Fresh neutral generation is required but does not replace provenance review.
+
 ## Protocol and framework evidence
 
 Fabric ingestion uses the **Custom endpoint source's Event Hub producer interface**, through the official Azure Event Hubs client over AMQP/TLS. This is the documented direct-ingestion path and requires neither a separate Event Hubs resource nor a Collector. It is not Eventstream accepting OTLP, nor an outbound custom endpoint destination.
 
-The transport emits versioned framework-neutral JSON derived from standard OpenTelemetry spans. The optional OTLP transport uses the official HTTP exporter for an existing endpoint. See the [producer research](research-eventstream.md), [Copilot event research](research-copilot.md), and [Fabric provisioning/dashboard research](research-fabric.md) for source references.
+The transport emits versioned framework-neutral JSON derived from standard OpenTelemetry spans. OTLP support is provided by a caller-supplied official HTTP exporter, not built into this package. See the [producer research](research-eventstream.md), [Copilot event research](research-copilot.md), and [Fabric provisioning/dashboard research](research-fabric.md) for source references.
 
 Supported Copilot events are `session.start`/`session.resume`, `assistant.turn_start`/`assistant.turn_end`, `tool.execution_start`/`tool.execution_complete`, `subagent.started`/`subagent.completed`/`subagent.failed`, `assistant.usage`, `model.call_failure`, `user.message`/`assistant.message` (opt-in content), and `abort`/`session.error`/`session.idle`/`session.shutdown`. Request usage remains request-scoped; separate subagent aggregate usage is not added to request totals. Correlation is explicit-only and bounded; unsupported or missing metadata is not invented.
 
@@ -47,12 +63,28 @@ Fabric assets include a schema-validated native **version-75 KQLDashboard** defi
 
 SDK, adapter, exporter, and Fabric asset tests run in Vitest. The Fabric checks preserve the original fixture analytics, all twenty query contracts, complete native dashboard schema validation, and source/destination routing checks using Ajv. PowerShell handles provisioning/updates and twenty mocked REST scenarios (twelve provisioning, eight updates). See [actual benchmark results](performance.md) for event handling, serialization, memory, throughput, burst pressure, and sustained queue behavior.
 
-After the dependency and example updates, **96 Vitest tests across 8 files passed**,
-including the Fabric asset regression suites. The installed development
+The completed local validation passed **194 Vitest tests across 15 files**,
+including SDK adapters and Fabric asset regression suites. The installed development
 dependencies are AJV 8.18.0 and fast-uri 3.1.9, resolving the reported dependency
-advisories. Formatting checks, root package build, declaration/ESM smoke checks
-against the rebuilt tarball, strict typechecking, and twenty mocked REST scenarios
-are release checks. The original silent-export-failure reproduction rejects
+advisories. Strict typechecking, Prettier, eight Fabric contract tests, and twenty
+mocked REST scenarios (twelve provisioning, eight updates) passed. npm and pnpm
+root build/pack checks, ESM/declaration/CLI smoke checks, optional-peer isolation,
+privacy smoke checks, and neutral versioned standalone ZIP generation also
+passed locally. Repeated real root pnpm and npm packs regenerated identical
+setup ZIP bytes; packed inventory, bindings, synthetic samples, version, and
+per-file SHA-256 checks passed on both tarballs. The standalone ZIP matched the
+package archive byte-for-byte. These completed checks are offline evidence, not a fresh online
+consumer install, public/latest CI, real provider requests, or latest live Fabric.
+The npm-embedded ZIP change additionally passed fifteen focused bundle tests,
+including three repeated isolated fixture npm packs, stale ZIP exclusion,
+neutral bindings, per-file hashes, safe paths, manual no-overwrite behavior,
+and fail-closed content/source checks. The complete generator, three provisioning
+scripts, and bundle README were reviewed: project-authored source, public Fabric
+API/schema references, placeholder endpoints, and synthetic samples only; no
+foreign source was imported. The fixture checks supplement, rather than replace,
+the completed real root npm/pnpm pack and package-smoke validation. Content
+checks cannot independently certify source provenance.
+The original silent-export-failure reproduction rejects
 after full cleanup, reports a sanitized authentication category and failed-export
 counts, and exposes no raw input. The earlier in-memory benchmark completed;
 its deliberate overload cases retain bounded queues and report drops, not
@@ -64,6 +96,13 @@ guaranteed network throughput.
 - Fabric offline checks validate schemas, fixture contracts, query assets, and mocked REST provisioning, including polling and orchestration. These are **not KQL engine execution or dashboard service rendering**.
 - The package builds with tsup plus ordinary TypeScript 7 CLI declaration emission. There are no workspace imports or custom declaration-rewriting steps.
 
+Latest observed CI run **43727442386** failed on 287 registry-specific lockfile
+tarball URLs. Those routes were removed without changing versions or integrity
+hashes. A registry-agnostic pnpm hook prevents their regeneration while preserving
+explicit URL dependencies, custom archives, and signed routes. Frozen offline
+installation and three hook regression tests passed. This report does not claim
+that the failing run or subsequent CI has passed.
+
 **Live Fabric compatibility: verified for the operator's 0.1.0 deployment on
 2026-11-12.** The operator installed the published package, ran the basic Copilot
 example with Eventstream transport, and reported three generated/acknowledged
@@ -73,8 +112,9 @@ operator-reported, not a live run performed by the offline suite.
 
 `tools/live-fabric.ts` remains an explicit opt-in test of producer acceptance,
 ingestion, KQL lookup, and parent-child preservation; its execution has not been
-reported. Automated provisioning, update APIs, all three examples, parent-child
-assertions, performance at scale, and a live run of 0.1.1 remain unverified.
+reported. Automated provisioning, update APIs, all five examples as a complete
+set, parent-child assertions, performance at scale, and live runs of 0.1.1,
+0.1.2, and the prepared 0.2.0 remain unverified.
 
 ## Reliability and limitations
 
@@ -94,8 +134,12 @@ Pattern redaction is defense in depth, not a general PII/secret detector. Conten
 
 ## Required external validation
 
-Run CI on Node 22.15+ and 24 for each release. After publishing 0.1.2, reinstall
+Run CI on Node 22.15+ and 24 for each release. After publishing 0.2.0, reinstall
 the basic example's pinned dependency and repeat live ingestion/dashboard checks.
+Before publication, observe a passing CI run after lockfile remediation and
+complete a fresh consumer installation through an approved registry.
+Publication, uploads, and source-repository visibility changes are outside this
+documentation update.
 Run the opt-in parent-child validation and the remaining examples, and validate
 automated provisioning/updates and interactive KQL performance before claiming
 those additional capabilities as service-verified.

package/docs/openai-adapter.md

@@ -0,0 +1,137 @@
+# OpenAI Node SDK adapter
+
+The `/openai` entry point observes the official `openai` Node SDK's direct
+`client.chat.completions.create()` and `client.responses.create()` calls.
+The compatibility target is **OpenAI SDK 7.53.0**, Node.js 22.15 or later.
+It uses public SDK APIs, not provider-wide request hooks or application-global
+OpenTelemetry configuration.
+
+```ts
+import OpenAI from "openai";
+import { observeOpenAI } from "@saguiitay/eventhouse-ai-observability/openai";
+
+const sdk = new OpenAI(); // Configure authentication in the application.
+const observer = observeOpenAI(sdk, {
+  connectionString: process.env.FABRIC_EVENTSTREAM_CONNECTION_STRING!,
+});
+
+try {
+  const response = await observer.client.responses.create({
+    model: "gpt-4.1-mini",
+    input: "Hello",
+  });
+  console.log(response.output_text);
+} finally {
+  await observer.shutdown();
+}
+```
+
+Use the returned `client`. The original client remains uninstrumented and
+unmodified. Endpoint overloads and the client type are preserved; SDK results and
+raw SDK exceptions reach the caller. `then`, `catch`, `finally`, `withResponse`
+and `asResponse` remain available. SDK retries, authentication, request options
+and abort signals are unchanged. Observation never retries requests.
+
+## Scope and telemetry
+
+Each supported call produces one client span, with `gen_ai.operation.name:
+"chat"` for both APIs, `gen_ai.provider.name: "openai"`, requested/returned model,
+response ID, finite nonnegative input/output token usage when available, finish
+reasons for Chat Completions, and SDK-call-to-completion duration.
+`openai.api` distinguishes `chat.completions` from `responses`.
+
+Errors use fixed `error.type` categories and a numeric HTTP status when available.
+Raw exception messages, headers, credentials and endpoint URLs are not captured.
+Failed/incomplete Responses results are returned unchanged and marked as errors.
+Completed request/response failures are failed spans, not incomplete observations.
+`observability.incomplete` is reserved for unfinished work, cancellation/abort,
+incomplete responses/streams, unsupported results, and observer shutdown.
+Background responses that are still queued/in progress are marked incomplete;
+the adapter does not poll them.
+
+Other endpoints and SDK helpers (`parse`, `stream`, `runTools`, retrieval,
+pagination, realtime, embeddings, audio, batches, files, and helpers on `beta`)
+are **not intercepted**. A client produced by `withOptions()` must be observed
+separately. Tool execution is not traced. Only direct calls through the returned
+client's two supported `create` methods are observed.
+
+## Streaming and promise consumption
+
+Supported `create({ stream: true })` calls remain lazy: no background consumer or
+extra tee is created. The public SDK Stream constructor decorates iteration
+without changing the original stream. Async iteration, `controller.abort()`,
+`tee()` and `toReadableStream()` retain SDK behavior. Chunks and SDK errors are
+returned unchanged. The SDK's own `_request_id` property descriptor is preserved
+on decorated streams. Usage is taken from observed chunks/terminal Responses
+events, never estimated. For Chat Completions, request
+`stream_options: { include_usage: true }` if token reporting is needed; the
+adapter does not change request parameters.
+
+Stream spans finish once on exhaustion, error, early return, abort or observer
+shutdown. Missing terminal events/finish reasons are marked incomplete.
+Chat streams require every requested choice index to finish, not just the first
+choice. Tracking is bounded to `n` values from 1 to 32; higher/invalid counts
+remain usable but cannot be certified complete and are marked incomplete.
+Duplicate, invalid and out-of-range choice indices do not satisfy completion.
+`flush()` exports ended spans, without consuming or ending live streams.
+`tee()` uses the SDK's own buffers; callers must consume/cancel both branches
+responsibly. The adapter adds no chunk queue.
+
+Promises are not eagerly parsed: that would consume bodies intended for
+`asResponse()`. Await or consume a supported promise to observe its result.
+`asResponse()` finishes an HTTP-only observation without reading/cloning its
+body, so token/content telemetry is unavailable on that path. If mixed with
+parsed consumption, the first terminal observation wins. Unconsumed promises
+and streams stay tracked until observer shutdown; they are then marked
+`observability.incomplete` and `openai.lifecycle: "observer_shutdown"`.
+Shutdown does **not** abort application requests/streams, and late results
+continue to reach the caller without reopening spans.
+
+## Privacy and bounds
+
+Metadata-only is the default. With an engine configured for `captureContent:
+true`, the adapter captures text-only input/output using `gen_ai.input.messages`
+and `gen_ai.output.messages`. The engine sanitizer/redactor applies before any
+exporter sees content. Images, audio, tool arguments/results and arbitrary object
+serialization are excluded. Chat output capture is limited to the first choice.
+Structured text traversal is bounded to 32 entries per array and three levels.
+
+`maxContentChars` defaults to 4096 (range 1–4096), independently bounding input and
+output text, including streaming text accumulation. This is truncated text, not
+a complete conversation transcript. `maxTrackedRequests` defaults to 1024 (range
+1–65536) and is capped by the engine's active-span limit. At capacity, new calls
+pass through unchanged/unobserved; `requestsSkipped` reports adapter admissions
+skipped. `trackedRequests` includes unconsumed SDK promises/streams.
+
+## Shared engines and lifecycle
+
+```ts
+import { createObservability } from "@saguiitay/eventhouse-ai-observability";
+import { attachOpenAI } from "@saguiitay/eventhouse-ai-observability/openai";
+
+const engine = createObservability({ exporter: myExporter });
+const observer = attachOpenAI(sdk, engine); // External ownership by default.
+// Equivalent: observeOpenAI(sdk, { engine })
+try {
+  await observer.client.chat.completions.create({
+    model: "gpt-4.1-mini",
+    messages: [{ role: "user", content: "Hello" }],
+  });
+} finally {
+  await observer.shutdown(); // End adapter work and flush, keep engine alive.
+  await engine.shutdown(); // Once, after every shared adapter shuts down.
+}
+```
+
+`observeOpenAI` owns engines it creates and shuts them down by default.
+Supplied engines are externally owned by default. `shutdownEngine: true` explicitly
+takes ownership of a supplied engine; do not set it to true on independently shutting-down adapters
+that share an engine. Shutdown is awaited and idempotent. The SDK client is never
+closed. `shutdownEngine: false` requires an external engine so newly constructed
+engines cannot be left without an owner. Engine diagnostics are returned alongside
+the two adapter counters.
+
+Public SDK references:
+[7.53.0 README](https://github.com/openai/openai-node/blob/v7.53.0/README.md),
+[APIPromise](https://github.com/openai/openai-node/blob/v7.53.0/src/core/api-promise.ts),
+[Stream](https://github.com/openai/openai-node/blob/v7.53.0/src/core/streaming.ts).

package/docs/quickstart.md

@@ -1,6 +1,22 @@
 # Quickstart
 
-Use the [Fabric setup](fabric-setup.md) first. There is no Collector or Azure Event Hubs resource to deploy.
+Install the package first, then follow [Fabric setup](fabric-setup.md).
+There is no Collector or Azure Event Hubs resource to deploy.
+
+## Install a published version
+
+```sh
+npm install @saguiitay/eventhouse-ai-observability
+```
+
+npmjs **0.1.2** and GitHub Packages **0.1.1** are published and registry-verified.
+This checkout's **0.2.0 is prepared, not published**. Its generated
+`fabric-setup.zip` will be available from the installed package root, without
+access to this private source repository; see the
+[PowerShell extraction instructions](../README.md#fabric-setup-bundle-and-ingestion-check).
+The earlier published versions do not include it; use manual Fabric setup for
+those versions. GitHub Packages additionally requires
+[registry authentication](../README.md#installation-from-github-packages).
 
 ## Build and install locally
 
@@ -10,9 +26,13 @@ pnpm build
 pnpm pack
 ```
 
-Install the resulting tarball into an existing Node ESM application using `npm install <tarball>`. For published versions, follow the [GitHub Packages installation instructions](../README.md#installation-from-github-packages). Keep your existing Copilot session creation, permissions, model configuration, and execution unchanged; attach `observeCopilot` immediately after creating the session and before sending the first prompt.
+Install the resulting tarball into an existing Node ESM application using `npm install <tarball>`.
+The prepared package includes the generated setup ZIP; extract it after installing.
+Keep your existing Copilot session creation, permissions, model configuration, and execution unchanged; attach `observeCopilot` immediately after creating the session and before sending the first prompt.
 
-`pnpm test:package` checks the packed root and `/copilot` exports, disposal, parent-child IDs, and public TypeScript declarations. It reuses already-installed project runtime dependencies and does not claim a fresh online install.
+`pnpm test:package` checks all packed SDK exports, the validator CLI, disposal,
+parent-child IDs, and public TypeScript declarations. It reuses already-installed
+project runtime dependencies and does not claim a fresh online install.
 
 ```typescript
 import { observeCopilot } from "@saguiitay/eventhouse-ai-observability/copilot";
@@ -38,6 +58,7 @@ Shutdown closes the observer, **not your application-owned Copilot session/clien
 | ---------------------- | ------------------------------------------------------------------------- |
 | `connectionString`     | `FABRIC_EVENTSTREAM_CONNECTION_STRING`                                    |
 | `entityName`           | Needed only if `EntityPath` is absent; `FABRIC_EVENTSTREAM_ENTITY_NAME`   |
+| `exporter`             | Optional standard `SpanExporter`; replaces built-in Fabric export         |
 | `serviceName`          | `OTEL_SERVICE_NAME`, then `ai-application`                                |
 | `serviceVersion`       | `OTEL_SERVICE_VERSION`                                                    |
 | `environment`          | `DEPLOYMENT_ENVIRONMENT`                                                  |
@@ -79,14 +100,62 @@ into the application. Unknown third-party error codes remain `unknown`.
 
 Native OTel batches wait for a concurrency slot within the existing queue admission budget and callback deadline, so healthy flush backlogs drain without saturation drops. There is no second span queue. New admissions beyond `maxQueueSize` and batches exceeding their deadline are dropped. Third-party exporters remain standard OTel `SpanExporter`s; a callback deadline cannot forcibly stop their I/O, so a hung call keeps its slot until actual completion. Engine shutdown still releases local context within its deadline even if exporter cleanup hangs. Eventstream cancels Azure operations with abort signals and separately bounds pending transport calls. Known acknowledged spans count as exported; oversized, unsent, or uncertain spans count as dropped. See [transport failure semantics](eventstream-transport.md#failure-semantics).
 
-For an **existing** OTLP service, explicitly set `transport: "otlp"` and `endpoint`, or use `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`. `OTEL_EXPORTER_OTLP_ENDPOINT` is a base URL to which `/v1/traces` is appended. Headers support the corresponding standard traces/general environment variables. Without a Fabric connection string or an explicit OTLP endpoint, configuration fails early instead of silently sending to localhost.
+Fabric is the only built-in transport. Without a Fabric connection string or
+an explicit `exporter`, configuration fails early. The root package does not
+read OTLP environment variables.
+
+For an existing OTLP service, install
+`@opentelemetry/exporter-trace-otlp-http` separately:
+
+```typescript
+import { createObservability } from "@saguiitay/eventhouse-ai-observability";
+import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
+
+const observer = createObservability({
+  exporter: new OTLPTraceExporter({
+    url: "https://your-collector.example/v1/traces",
+    headers: { authorization: `Bearer ${process.env.COLLECTOR_TOKEN}` },
+  }),
+});
+```
+
+Endpoint, header, and standard OTLP environment handling belong to that exporter.
+Use HTTPS outside local development. The engine still sanitizes spans and owns
+the supplied exporter's flush/shutdown lifecycle.
+
+In 0.2.0, remove `transport`, `endpoint`, and `headers` from observability options
+and stop importing `OTLPTraceExporter` from this package. Use `exporter` as above.
 
 ## Live Fabric validation
 
 Set `FABRIC_EVENTSTREAM_CONNECTION_STRING`, optionally `FABRIC_EVENTSTREAM_ENTITY_NAME`, `FABRIC_KQL_QUERY_ENDPOINT` (database query URI, HTTPS), `FABRIC_KQL_DATABASE`, and `FABRIC_KQL_ACCESS_TOKEN` (a short-lived, authorized Kusto query token for that URI).
 
+Run the CLI from the application where this SDK is installed; no source checkout
+or separate CLI package is needed:
+
 ```sh
-pnpm exec tsx tools/live-fabric.ts
+npx --no-install eventhouse-ai-observability-validate
 ```
 
-The opt-in test sends two spans, waits up to three minutes for ingestion, queries `AIObservabilityTraces`, and checks their trace and parent IDs. It writes real telemetry to your database. It neither provisions resources nor generates tokens. A producer send acknowledgement alone is not a successful live validation.
+From a checkout, run `pnpm build` then `pnpm validate:ingestion`.
+`pnpm exec tsx tools/live-fabric.ts` remains a compatible source command.
+`--help` performs no network I/O. No other arguments are accepted; keep
+credentials out of command-line arguments, shell history, and committed files.
+
+The producer needs Eventstream source Send credentials. The separate query token
+needs Kusto database read permission and the correct Kusto audience; the
+`FABRIC_ACCESS_TOKEN` used by provisioning is not a substitute.
+
+The opt-in check reports producer acknowledgement, Eventhouse ingestion, and
+trace/parent-child correlation separately. It sends two metadata-only spans and
+queries `AIObservabilityTraces` until both appear. The polling timeout defaults
+to three minutes after producer cleanup; set `FABRIC_VALIDATION_TIMEOUT_MS` to
+a positive integer up to `900000` to change it. Export and query requests have
+bounded deadlines. Query authorization failures, producer failures, ingestion
+timeouts, and mapping/correlation failures use distinct safe error messages and
+exit code 1; successful verification exits 0. No endpoint, token, query response
+body, or raw SDK error is printed.
+
+This writes real telemetry and can incur Fabric ingestion/query charges. It
+neither provisions resources, acquires tokens, nor calls an AI provider. A
+producer acknowledgement alone is not a successful validation.

package/docs/research-eventstream.md

@@ -162,15 +162,17 @@ or schema-registry headers.
 
 [Official OTel exporter documentation][otel] identifies
 `@opentelemetry/exporter-trace-otlp-http` as **HTTP/JSON**, not HTTP/protobuf
-(`exporter-trace-otlp-proto`). The OTLP path uses the official
-`OTLPTraceExporter` directly through its standard `export(spans, callback)`
-interface, without a separate transport wrapper or private package.
+(`exporter-trace-otlp-proto`). Applications can supply the official
+`OTLPTraceExporter` directly through the standard `exporter` option, without
+a separate transport wrapper or private package. It is not a runtime
+dependency or public export of this package.
 The [official implementation][otlp-source] supplies the serializer, HTTP sender,
 and exporter lifecycle.
 
-`endpoint`, if supplied, is the full traces URL, normally ending in `/v1/traces`.
-If omitted, the SDK owns environment/default resolution. Headers are copied and
-validated against invalid names/newlines, and exporter errors are sanitized.
+The exporter's `url`, if supplied, is the full traces URL, normally ending in
+`/v1/traces`. The official exporter owns environment/default resolution and
+header configuration. The observability engine sanitizes exported spans and
+diagnostic errors, not arbitrary transport configuration.
 Use HTTPS outside trusted local development. This is an optional external OTLP
 destination, **not an assertion that Fabric Eventstream accepts OTLP**.
 

package/docs/research.md

@@ -31,6 +31,6 @@ on 2026-11-12.** The published package was installed and the basic Copilot examp
 ran successfully, with three acknowledged spans and no drops/failures. The
 operator confirmed rows in `AIObservabilityTraces` and dashboard visibility.
 This is not automated validation of all tenants, provisioning APIs, all examples,
-or 0.1.1. The opt-in live validation script's execution and parent-child assertions
+or 0.1.1, 0.1.2, or the prepared unpublished 0.2.0. The opt-in live validation script's execution and parent-child assertions
 have not been reported; it requires a real Custom endpoint source, Eventhouse
 destination, KQL query URI/database, and a user-supplied query access token.

package/docs/security.md

@@ -19,7 +19,7 @@ Use the Fabric Custom endpoint source's producer credential with **Send** capabi
 
 Rotate using Fabric's supported source key regeneration/credential controls: update the application secret, replace the observer/transport, verify ingestion, then revoke the old key. Existing clients retain their original credential; changing an environment variable does not rotate a running producer. No credential is embedded in the provisioned dashboard or sample data.
 
-AMQP uses TLS through the official Azure SDK. OTLP requires HTTPS except localhost/loopback for local tests. The SDK does not register global logging or print raw exporter errors. Diagnostics contain aggregate counts and an optional latest failure with an allowlisted category and fixed troubleshooting hint, never the original exception. `onDiagnostic` and opt-in `debug` logging expose only those safe fields.
+AMQP uses TLS through the official Azure SDK. For a caller-supplied OTLP exporter, configure HTTPS except localhost/loopback for local tests; endpoint/header security belongs to the application and its exporter. The SDK does not register global logging or print raw exporter errors. Diagnostics contain aggregate counts and an optional latest failure with an allowlisted category and fixed troubleshooting hint, never the original exception. `onDiagnostic` and opt-in `debug` logging expose only those safe fields.
 
 ## Access and retention
 

package/docs/telemetry-schema.md

@@ -26,6 +26,24 @@ Copilot correlation uses the `copilot.*` namespace, including `copilot.session.i
 
 `copilot.span.type` distinguishes `session`, `turn`, `tool`, `subagent`, and `model_request`. Execution metrics count turns, not the enclosing session observation or the separate subagent lifecycle span. Generic `invoke_agent` spans without Copilot-specific types remain supported. Agent identity is preserved as `copilot.agent.id`; names are added only when explicitly correlated metadata is available.
 
+## Request token details
+
+SDK adapters normalize explicitly reported request usage into:
+
+| Attribute                                  | Meaning                                             |
+| ------------------------------------------ | --------------------------------------------------- |
+| `gen_ai.usage.cache_read.input_tokens`     | Input tokens read from the provider's prompt cache  |
+| `gen_ai.usage.cache_creation.input_tokens` | Input tokens written to the provider's prompt cache |
+| `gen_ai.usage.reasoning.output_tokens`     | Reasoning tokens within the reported output total   |
+
+These are breakdowns, not additional totals: do not add reasoning tokens to `gen_ai.usage.output_tokens` or cached tokens to `gen_ai.usage.input_tokens`. Missing details remain absent; explicitly reported zero remains zero. Counts must be nonnegative safe integers. Adapters do not estimate, split totals, infer cache writes from reads, or copy aggregate turn/session usage onto requests.
+
+OpenAI Chat Completions uses `prompt_tokens_details.cached_tokens`, `prompt_tokens_details.cache_write_tokens`, and `completion_tokens_details.reasoning_tokens`. Responses uses `input_tokens_details.cached_tokens`, `input_tokens_details.cache_write_tokens`, and `output_tokens_details.reasoning_tokens`. Normal and terminal streaming usage follow the same mapping; cache creation is recorded only if the provider reports it.
+
+Vercel AI SDK 7 uses each `onLanguageModelCallEnd` event's `usage.inputTokenDetails.cacheReadTokens`, `cacheWriteTokens`, and `usage.outputTokenDetails.reasoningTokens` on its native model-request span. Native cache attributes and optional supplemental reasoning attributes normalize to the same keys. This also works when native supplemental usage recording is disabled; step/root aggregate usage is not duplicated.
+
+Copilot `assistant.usage` uses `cacheReadTokens`, `cacheWriteTokens`, and `reasoningTokens`. The compatibility attribute `copilot.usage.reasoning_tokens` remains alongside the standard reasoning key. Session shutdown metrics and subagent aggregates are not converted into request token details.
+
 ## Privacy and evolution
 
 Content is disabled by default. Sanitized resources, span attributes, events, and links are bounded before export. A custom transport receives the same privacy-filtered model. Error status remains available without exposing raw error text.