@saguiitay/eventhouse-ai-observability 0.1.2 → 0.2.0

package/CHANGELOG.md

@@ -1,7 +1,44 @@
 # Changelog
 
+## 0.2.0 - prepared 2026-11-15 (unpublished)
+
+- Add optional `/openai` and `/vercel` integrations sharing the existing
+  OpenTelemetry engine, privacy filtering, and Fabric/custom exporters.
+- Keep SDK-specific dependencies optional and isolate each public entry point.
+- Remove 287 machine-specific registry tarball routes from the lockfile without
+  changing versions or integrity hashes. Add a registry-agnostic pnpm hook and
+  regression tests to keep registry resolutions portable.
+- Add runnable OpenAI and Vercel samples with per-example environment templates,
+  streaming, metadata-only diagnostics, and offline lifecycle tests.
+- Remove built-in OTLP configuration and the runtime OTLP dependency/export.
+  Supply a standard exporter through `exporter` instead of `transport`,
+  `endpoint`, or `headers`. Root configuration no longer reads OTLP environment
+  variables; checkout examples explicitly construct the official exporter.
+- Upgrade development Zod to 4.6.7 to fix GHSA-mrhx-c7xc-m3w5
+  (CVE-2023-54404); remove affected Zod resolutions from the lockfile.
+- Normalize provider-reported cache and reasoning token details on model-request
+  spans without adding them again to input/output totals.
+- Change the Fabric distribution policy: include one generated
+  `fabric-setup.zip` at the npm package root for public setup downloads, without
+  access to the private source repository. Keep raw checkout Fabric assets and
+  tenant-bound files out of the tarball. Preserve standalone versioned ZIPs and
+  Release/Actions assets for repository collaborators.
+- Generate setup assets in isolation with version/schema/SHA-256 metadata
+  matching the package. Permit only this personal project's authored generic
+  public-API templates and synthetic samples; exclude Microsoft-private content,
+  tenant identifiers/bindings, credentials, private registry configuration,
+  real telemetry, and copied external-repository files. Neutrality alone does
+  not establish safe source provenance.
+- Add the installed `eventhouse-ai-observability-validate` CLI for opt-in producer,
+  ingestion, and trace/parent verification, with environment-only credentials
+  and offline tests.
+- Remove low-level root exports `createEngine`, `serializeSpan`, and
+  `redactSecrets`. Use `createObservability`, SDK adapters, and the standard
+  exporter interface; `EventstreamExporter` remains public.
+
 ## 0.1.2 - 2026-11-14
 
+- Published and registry-verified on npmjs; does not include the new setup ZIP.
 - Publish to npmjs using GitHub Actions OIDC trusted publishing without a stored
   npm token. Provenance requires a public source repository and is not generated
   for this private repository.
@@ -12,6 +49,7 @@
 
 ## 0.1.1 - 2026-11-12
 
+- Published and registry-verified on GitHub Packages.
 - Fix the reported development-dependency advisories by upgrading AJV to 8.18.0
   and fast-uri to 3.1.9.
 - Publish with public access and require verified public GitHub package visibility

package/README.md

@@ -2,14 +2,15 @@
 
 Instrument an **existing GitHub Copilot SDK session** and send OpenTelemetry spans directly to Microsoft Fabric Eventstream. No Collector, gateway, or custom frontend.
 
-1. In Fabric, add an Eventstream **Custom endpoint source**, then connect its output to an Eventhouse KQL database. Use the source's **Event Hub** producer credentials, not an outward destination. Apply the [schema and setup](docs/fabric-setup.md).
-2. Set `FABRIC_EVENTSTREAM_CONNECTION_STRING` to the Fabric-provided connection string. If it omits `EntityPath`, also set `FABRIC_EVENTSTREAM_ENTITY_NAME` to the provided event hub name.
-3. Install from npmjs (or your organization's approved npm mirror) and instrument:
+1. Install from npmjs (or your organization's approved npm mirror):
 
 ```sh
 npm install @saguiitay/eventhouse-ai-observability
 ```
 
+2. Obtain the [Fabric setup assets](#fabric-setup-bundle-and-ingestion-check). In Fabric, add an Eventstream **Custom endpoint source**, then connect its output to an Eventhouse KQL database. Use the source's **Event Hub** producer credentials, not an outward destination. Apply the [schema and setup](docs/fabric-setup.md).
+3. Set `FABRIC_EVENTSTREAM_CONNECTION_STRING` to the Fabric-provided connection string. If it omits `EntityPath`, also set `FABRIC_EVENTSTREAM_ENTITY_NAME` to the provided event hub name. Instrument your existing session:
+
 ```typescript
 import { observeCopilot } from "@saguiitay/eventhouse-ai-observability/copilot";
 
@@ -26,9 +27,11 @@ try {
 }
 ```
 
-Version 0.1.1 is published to **GitHub Packages**, and its first npmjs release
-was approved by the maintainer. A separate npmjs workflow publishes future
-versions directly using GitHub Actions OIDC trusted publishing.
+Version **0.1.1 is published and registry-verified on GitHub Packages**;
+**0.1.2 is published and registry-verified on npmjs**. This checkout prepares
+**0.2.0, not yet published**, including the setup ZIP and new SDK entry points.
+A separate npmjs workflow publishes future versions directly using GitHub
+Actions OIDC trusted publishing.
 To test an unpublished release,
 run `pnpm install` and `pnpm pack`; install the root tarball with npm.
 Node.js **22.15+** (including Node 24 LTS), ESM, Copilot SDK **1.0.31** / compatible
@@ -36,27 +39,53 @@ Node.js **22.15+** (including Node 24 LTS), ESM, Copilot SDK **1.0.31** / compat
 The repository's `.npmrc` routes only `@saguiitay` packages to GitHub Packages;
 other dependencies continue to use your configured, organization-approved registry.
 
-Packing with `pnpm pack` or `npm pack` automatically rebuilds the ESM runtime and
-TypeScript declarations through `prepack`; a failed build stops packaging. Do not
+Packing with `pnpm pack` or `npm pack` automatically rebuilds the ESM runtime,
+TypeScript declarations, and neutral setup ZIP through `prepack`; a failed build stops packaging. Do not
 disable lifecycle scripts when preparing a release.
 
 The tarball includes these documentation pages and example sources/templates, so
-relative links work from the installed package. Run example/provisioning commands
-from the source checkout; Fabric assets remain there and are not shipped with
-tenant-specific bindings.
+relative links work from the installed package. Run examples from the source
+checkout. Starting with the prepared 0.2.0 package, the tarball also includes
+one generated `fabric-setup.zip`; raw working-tree Fabric assets and tenant
+content remain excluded.
 
 Metadata only is captured by default. Prompts, responses, tool arguments, tool results, and error messages are not collected unless explicitly enabled. Credentials are redacted; review [security](docs/security.md) before enabling content.
 
 ## What is included
 
-One npm package with root and `/copilot` exports and ordinary internal modules under `src`. OpenTelemetry supplies batching, sampling, and OTLP export; our code supplies the Fabric exporter, privacy filtering, and Copilot event adapter. Includes versioned JSON, Eventhouse schema and twenty analytics queries, native dashboard assets, provisioning, three [examples](examples), local integration tests, and an opt-in live check.
+One npm package with root, `/copilot`, `/openai`, and `/vercel` exports and ordinary internal modules under `src`. OpenTelemetry supplies batching and sampling; our code supplies the Fabric exporter, privacy filtering, and SDK adapters. Standard OpenTelemetry exporters can be supplied by the application. Includes versioned JSON, Eventhouse schema and twenty analytics queries, native dashboard assets, provisioning, [runnable examples for every adapter](examples), local integration tests, and an opt-in live check.
+
+### Other SDKs
+
+The source checkout also supports [OpenAI](docs/openai-adapter.md) Chat Completions
+and Responses, and [Vercel AI SDK](docs/vercel-adapter.md) native telemetry.
+These adapters share the same privacy controls, sampling, Fabric export, custom exporters,
+and diagnostics; each SDK is an optional peer dependency. Install only the SDKs
+you use. The new entry points are included in version 0.2.0; until it is
+published, build or pack this checkout to use them.
+
+OpenAI request instrumentation does not create agent or tool-execution spans
+from model-generated tool calls. Vercel integrates with the SDK's existing
+tracing rather than adding a second set of spans. See the adapter docs for
+supported versions, streaming behavior, and complete lifecycle examples.
+Runnable checkout samples are `pnpm example:openai` and `pnpm example:vercel`;
+see [sample setup](examples/README.md#openai-and-vercel-ai-sdks).
+
+For an existing OTLP destination, install OpenTelemetry's official exporter
+separately and supply it through the standard `SpanExporter` interface:
+
+```sh
+npm install @opentelemetry/exporter-trace-otlp-http
+```
 
 ```typescript
 import { createObservability } from "@saguiitay/eventhouse-ai-observability";
+import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
 
 const observer = createObservability({
-  transport: "otlp",
-  endpoint: "https://your-existing-otlp-service.example/v1/traces",
+  exporter: new OTLPTraceExporter({
+    url: "https://your-existing-otlp-service.example/v1/traces",
+  }),
 });
 const span = observer.startSpan("invoke_agent my-framework", {
   attributes: { "gen_ai.operation.name": "invoke_agent" },
@@ -66,7 +95,11 @@ await observer.flush();
 await observer.shutdown();
 ```
 
-An OTLP endpoint is optional and **not required for Fabric**. Eventstream receives JSON events over the official Event Hubs AMQP producer protocol, not OTLP.
+Fabric is the only built-in transport. This package does not depend on or
+re-export an OTLP exporter, or read `OTEL_EXPORTER_OTLP_*` configuration.
+The supplied exporter owns its endpoint, headers, and environment settings.
+Eventstream receives JSON events over the official Event Hubs AMQP producer
+protocol, not OTLP.
 
 Examples print their selected transport and final export counters, and fail after
 cleanup if telemetry failed or was dropped. In applications, use
@@ -96,11 +129,72 @@ operator's 0.1.0 deployment on 2026-11-12**, based on the operator's reported
 published-package installation, successful Copilot execution, three acknowledged
 spans with no drops/failures, rows in `AIObservabilityTraces`, and dashboard
 visibility. This does not establish support for all tenants, automated
-provisioning, or a live run of 0.1.1. The separate [live validation
+provisioning, or a live run of 0.1.1, 0.1.2, or the prepared 0.2.0. The separate [live validation
 script](docs/quickstart.md#live-fabric-validation) checks producer acceptance,
 Eventhouse ingestion, KQL lookup, and parent-child relationships; its execution
 has not been reported.
 
+## Fabric setup bundle and ingestion check
+
+The public download channel is the **npm tarball**, not GitHub Releases or
+Actions artifacts: the source repository stays private. Starting with 0.2.0,
+install the package first, then locate its root `fabric-setup.zip`. For a local
+application install in PowerShell:
+
+```powershell
+npm install @saguiitay/eventhouse-ai-observability
+$npmRoot = (npm root).Trim()
+$packageRoot = Join-Path $npmRoot '@saguiitay\eventhouse-ai-observability'
+$archive = Join-Path $packageRoot 'fabric-setup.zip'
+if (-not (Test-Path -LiteralPath $archive)) {
+    throw 'This installed version has no setup ZIP. Use a package containing the 0.2.0 bundle.'
+}
+$destination = Join-Path (Get-Location) 'fabric-setup'
+Expand-Archive -LiteralPath $archive -DestinationPath $destination
+Set-Location $destination
+Get-Content .\README.md
+```
+
+**0.2.0 is not yet published**; today's npmjs 0.1.2 install does not contain this
+ZIP. To evaluate it before publication, collaborators can install the prepared
+root tarball instead of the registry package. Existing published users can use
+the [manual setup instructions](docs/fabric-setup.md#manual-setup--unsupported-tenant-fallback).
+
+The archive root contains `README.md`, `manifest.json`, and `fabric\` with
+schema, queries, dashboard, Eventstream template, provisioning/update scripts,
+and synthetic samples. Keep that directory structure intact and follow the
+extracted README. Its manifest's package version must match the installed
+package version; schema version and SHA-256 hashes describe the generated bytes.
+Hashes detect drift, not publisher authenticity.
+
+Only this personal project's authored generic templates, public-API definitions,
+and synthetic samples belong in the archive, not Microsoft-private code,
+documentation, endpoints, registry configuration, tenant identifiers/bindings,
+credentials, real telemetry, or copied external-repository files. Assets are
+generated in isolation; tenant-neutral bindings alone are not a provenance check.
+Raw checkout Fabric files are not added to the npm tarball.
+
+For repository collaborators, `pnpm bundle:fabric` still creates
+`release-artifacts/eventhouse-ai-observability-fabric-<version>.zip` without
+overwriting an existing archive. Release/Actions ZIPs remain collaborator
+channels, not public prerequisites.
+
+After installing the SDK and configuring the Fabric route, set
+`FABRIC_EVENTSTREAM_CONNECTION_STRING`, `FABRIC_KQL_QUERY_ENDPOINT`,
+`FABRIC_KQL_DATABASE`, and `FABRIC_KQL_ACCESS_TOKEN` through your secret provider.
+The producer needs source Send credentials; the query token needs separate
+Kusto database read permission, not a Fabric management API token.
+
+```sh
+npx --no-install eventhouse-ai-observability-validate
+```
+
+The installed CLI sends two metadata-only spans, reports producer acknowledgement,
+polls Eventhouse, and verifies their trace and parent IDs. It makes no AI calls,
+but writes real telemetry and can incur Fabric ingestion/query charges.
+Use `--help` without credentials or network access. See
+[validation configuration](docs/quickstart.md#live-fabric-validation).
+
 ## Installation from GitHub Packages
 
 GitHub's npm registry requires authentication even for public packages. Your
@@ -129,7 +223,7 @@ This scope mapping does not change the registry for dependencies outside `@sagui
 
 Run **Publish to GitHub Packages** from the repository's Actions tab on `main`,
 or publish a GitHub Release whose tag matches the package version, such as
-`v0.1.2`. Update `package.json`, the instrumentation scope version in `src/engine.ts`,
+`v0.2.0`. Update `package.json`, the instrumentation scope version in `src/engine.ts`,
 and the basic example's dependency pin before releasing another version; published versions
 cannot be overwritten. Prerelease versions use the `next` dist-tag instead of `latest`.
 
@@ -140,6 +234,16 @@ artifact with `--access public`, and checks its registry version. It uses the re
 is needed for publishing. Repository/organization Actions policy must allow this
 permission.
 
+CI and both publication workflows also upload the matching Fabric ZIP as a
+`fabric-setup` Actions artifact. The separate **Fabric setup assets** workflow
+attaches it to published GitHub Releases after bundle tests pass and the release
+tag matches the package version. Only its attachment job requests
+`contents: write`; registry publication jobs retain read-only contents access.
+Running that assets workflow manually on `main` builds a downloadable Actions
+artifact without publishing a package or attaching anything to a release.
+These private-repository assets require collaborator access; public consumers
+obtain `fabric-setup.zip` from the installed npm package.
+
 GitHub controls visibility separately from npm's access flag. Before the next
 publication, open the existing package's **Package settings → Change visibility**
 and select **Public**. This is a one-time, irreversible setting; all subsequent
@@ -158,6 +262,9 @@ GitHub Packages registry configuration and its publishing workflow are unchanged
 Packed metadata is checked against the selected publication settings.
 OIDC works with this private source repository, but npm provenance attestations
 require a public source repository and are not generated here.
+Registry verification retries for up to five minutes while npm processes a
+release. If publishing succeeded but verification failed, rerun with
+**verify_only** enabled to check availability without publishing again.
 
 Configure the npm package's **Settings > Trusted publishing** with:
 
@@ -189,4 +296,4 @@ or your approved mirror instead; GitHub authentication is not needed for npmjs.
 
 ## Documentation
 
-[Quickstart](docs/quickstart.md) · [Architecture](docs/architecture.md) · [Research](docs/research.md) · [Fabric setup](docs/fabric-setup.md) · [Transport](docs/eventstream-transport.md) · [Schema](docs/telemetry-schema.md) · [Copilot adapter](docs/copilot-adapter.md) · [Dashboards](docs/dashboards.md) · [Security](docs/security.md) · [Extending](docs/extending.md) · [Troubleshooting](docs/troubleshooting.md) · [Performance](docs/performance.md) · [Implementation report](docs/implementation-report.md).
+[Quickstart](docs/quickstart.md) · [Architecture](docs/architecture.md) · [Research](docs/research.md) · [Fabric setup](docs/fabric-setup.md) · [Transport](docs/eventstream-transport.md) · [Schema](docs/telemetry-schema.md) · [Copilot adapter](docs/copilot-adapter.md) · [OpenAI adapter](docs/openai-adapter.md) · [Vercel AI adapter](docs/vercel-adapter.md) · [Dashboards](docs/dashboards.md) · [Security](docs/security.md) · [Extending](docs/extending.md) · [Troubleshooting](docs/troubleshooting.md) · [Performance](docs/performance.md) · [Implementation report](docs/implementation-report.md).

package/dist/adapters/openai.d.ts

@@ -0,0 +1,25 @@
+import type OpenAI from "openai";
+import type { Observability } from "../engine.js";
+export interface OpenAIAttachOptions {
+    /** Shared engines are externally owned by default. */
+    shutdownEngine?: boolean;
+    /** Maximum requests retained until consumption or observer shutdown. */
+    maxTrackedRequests?: number;
+    /** Text capture limit per input/output, when the engine allows content. */
+    maxContentChars?: number;
+}
+export interface OpenAIObserver<T extends OpenAI = OpenAI> {
+    readonly client: T;
+    flush(): Promise<void>;
+    shutdown(): Promise<void>;
+    getDiagnostics(): ReturnType<Observability["getDiagnostics"]> & {
+        trackedRequests: number;
+        requestsSkipped: number;
+    };
+}
+export declare function validateOpenAIAttachOptions(options: OpenAIAttachOptions): void;
+/**
+ * Observe only direct Chat Completions/Responses create calls. The original SDK
+ * client, resources, promises, streams and application OTel globals are untouched.
+ */
+export declare function attachOpenAI<T extends OpenAI>(client: T, engine: Observability, options?: OpenAIAttachOptions): OpenAIObserver<T>;

package/dist/adapters/vercel.d.ts

@@ -0,0 +1,25 @@
+import { type Tracer } from "@opentelemetry/api";
+import type { TelemetryOptions, Telemetry } from "ai";
+import { type Observability } from "../engine.js";
+export type VercelTelemetrySettings = TelemetryOptions;
+export interface VercelAdapterOptions {
+    /** Supply the SDK's native integration: tracer => new OpenTelemetry({ tracer }). */
+    integration: (tracer: Tracer) => Telemetry;
+    /** Also requires captureContent on the engine. Defaults to false. */
+    recordInputs?: boolean;
+    /** Also requires captureContent on the engine. Defaults to false. */
+    recordOutputs?: boolean;
+    /** Explicitly transfers engine shutdown ownership. Defaults to false. */
+    shutdownEngine?: boolean;
+}
+/**
+ * Native SDK instrumentation calls this tracer, not a global provider. Every
+ * mutation is filtered before reaching the private engine, including errors
+ * and attributes that the SDK's recording flags do not protect.
+ */
+export declare function attachVercelAI(engine: Observability, options: VercelAdapterOptions): {
+    telemetry: VercelTelemetrySettings;
+    flush: () => Promise<void>;
+    getDiagnostics: () => Readonly<import("../engine.js").Diagnostics>;
+    shutdown(): Promise<void>;
+};

package/dist/{chunk-XQFUPGYU.js → chunk-XGCHW72B.js}

@@ -480,7 +480,7 @@ function createEngine(options) {
   });
   const tracer = provider.getTracer(
     "@saguiitay/eventhouse-ai-observability",
-    "0.1.2"
+    "0.2.0"
   );
   const localContext = new AsyncLocalStorage();
   const active = /* @__PURE__ */ new Set();
@@ -844,8 +844,6 @@ var EventstreamExporter = class {
 };
 
 // src/index.ts
-import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
-import { OTLPTraceExporter as OTLPTraceExporter2 } from "@opentelemetry/exporter-trace-otlp-http";
 function envNumber(name) {
   const raw = process.env[name];
   if (raw === void 0) return void 0;
@@ -854,33 +852,20 @@ function envNumber(name) {
   }
   return Number(raw);
 }
-function envHeaders() {
-  const raw = process.env.OTEL_EXPORTER_OTLP_TRACES_HEADERS ?? process.env.OTEL_EXPORTER_OTLP_HEADERS;
-  if (!raw) return void 0;
-  const result = {};
-  for (const entry of raw.split(",")) {
-    const equals = entry.indexOf("=");
-    if (equals < 1)
-      throw new Error("Invalid OTLP header environment configuration");
-    try {
-      result[entry.slice(0, equals).trim()] = decodeURIComponent(
-        entry.slice(equals + 1).trim()
-      );
-    } catch {
-      throw new Error("Invalid OTLP header environment configuration");
-    }
-  }
-  return result;
-}
 function createObservability(options = {}) {
+  if ("transport" in options || "endpoint" in options || "headers" in options)
+    throw new Error(
+      "Configure custom transports with exporter; transport, endpoint, and headers options are not supported."
+    );
   const connectionString = options.connectionString ?? process.env.FABRIC_EVENTSTREAM_CONNECTION_STRING;
-  const selection = options.transport ?? (connectionString ? "eventstream" : "otlp");
   let exporter;
   if (options.exporter) {
     exporter = options.exporter;
-  } else if (selection === "eventstream") {
+  } else {
     if (!connectionString)
-      throw new Error("Fabric Eventstream connectionString is required");
+      throw new Error(
+        "Supply a Fabric Eventstream connectionString or a custom SpanExporter."
+      );
     const entityName = options.entityName ?? process.env.FABRIC_EVENTSTREAM_ENTITY_NAME;
     exporter = new EventstreamExporter({
       connectionString,
@@ -890,32 +875,6 @@ function createObservability(options = {}) {
       ...options.shutdownTimeoutMs !== void 0 ? { shutdownTimeoutMs: options.shutdownTimeoutMs } : {},
       ...options.maxConcurrentExports !== void 0 ? { maxConcurrentExports: options.maxConcurrentExports } : {}
     });
-  } else if (selection === "otlp") {
-    const base = process.env.OTEL_EXPORTER_OTLP_ENDPOINT;
-    const endpoint = options.endpoint ?? process.env.OTEL_EXPORTER_OTLP_TRACES_ENDPOINT ?? (base ? `${base.replace(/\/$/, "")}/v1/traces` : void 0);
-    if (!endpoint)
-      throw new Error(
-        "Supply a Fabric connection string or an explicit OTLP traces endpoint"
-      );
-    const headers = options.headers ?? envHeaders();
-    try {
-      const url = new URL(endpoint);
-      if (!["http:", "https:"].includes(url.protocol) || url.username || url.password)
-        throw new Error();
-    } catch {
-      throw new Error("Invalid OTLP endpoint configuration.");
-    }
-    for (const [name, value] of Object.entries(headers ?? {})) {
-      if (!/^[!#$%&'*+.^_`|~0-9A-Za-z-]+$/u.test(name) || /[\r\n\0]/u.test(value)) {
-        throw new Error("Invalid OTLP headers configuration.");
-      }
-    }
-    exporter = new OTLPTraceExporter({
-      url: endpoint,
-      ...headers ? { headers } : {}
-    });
-  } else {
-    throw new Error("Unsupported telemetry transport");
   }
   const serviceName = options.serviceName ?? process.env.OTEL_SERVICE_NAME;
   const serviceVersion = options.serviceVersion ?? process.env.OTEL_SERVICE_VERSION;
@@ -937,10 +896,7 @@ function createObservability(options = {}) {
 
 export {
   redactSecrets,
-  createEngine,
-  serializeSpan,
   EventstreamExporter,
-  createObservability,
-  OTLPTraceExporter2 as OTLPTraceExporter
+  createObservability
 };
-//# sourceMappingURL=chunk-XQFUPGYU.js.map
+//# sourceMappingURL=chunk-XGCHW72B.js.map