opentel-mcp 0.7.0 → 0.8.0

package/CHANGELOG.md

@@ -1,5 +1,214 @@
 # Changelog
 
+## 0.8.0
+
+Three features. **Tool schema drift detection**: a server that silently
+changes a tool's `inputSchema` between deployments — a parameter renamed, a
+type tightened, a `required` field added — currently breaks agents with no
+signal pointing at the actual cause. Full investigation and design: ADR 010
+(`docs/adr/010-schema-drift.md`). **Two-axis observation contract**:
+prompted by external review (Massimiliano Brighindi), who first raised that
+`instrumentMcpServer()` with no `TracerProvider`/`MeterProvider` registered
+silently no-ops — a failed tool call in that state produces zero telemetry,
+indistinguishable from one that never failed — and then supplied the reframe
+that shaped what shipped: not "detect a broken pipeline," but "stop implying
+health by omission." Full investigation and design: ADR 008
+(`docs/adr/008-observation-liveness.md`, "Update (2026-08-05)" section).
+**Cost-aware trace sampling**: investigated whether traces that were
+expensive or thrashed could be kept regardless of the head sampler's
+decision. Found that this cannot be an in-process library feature — a
+`Sampler` decides at span start, cost/thrash are only known at span end, and
+this package doesn't own the `Sampler`/`SpanProcessor` chain in its default
+configuration anyway. Ships a marker attribute plus a documented Collector
+recipe instead of a sampler. Full investigation and design: ADR 011
+(`docs/adr/011-cost-aware-sampling.md`).
+
+**⚠️ Read "Changed — behavior change on upgrade" immediately below before
+updating.** One of the three features above changes when
+`instrumentMcpServer()` throws, for a subset of low-level `Server` users,
+purely from the new default-on config — no code change of your own
+required to hit it.
+
+### Changed — behavior change on upgrade, read before updating
+
+- **The instrument-first ordering requirement now also covers `tools/list`,
+  for low-level `Server` users specifically, because `schemaDrift.enabled`
+  defaults to `true`.** `instrumentMcpServer()` has always required being
+  called before any `tools/call` handler is registered; as of this release,
+  with schema drift enabled (the default), it also requires being called
+  before any `tools/list` handler is registered. If your low-level `Server`
+  code registers `server.setRequestHandler(ListToolsRequestSchema, ...)`
+  before calling `instrumentMcpServer()` — never previously an error, since
+  this library was blind to `tools/list` entirely before this release —
+  upgrading will make that throw `INSTRUMENT_FIRST_ERROR` where it didn't
+  before, with no other code changes on your part.
+  - **`McpServer` users are unaffected.** `McpServer` registers `tools/list`
+    and `tools/call` together, atomically, the first time `.tool()` or
+    `.registerTool()` is called — so anyone already following the
+    documented instrument-before-registration rule for `tools/call`
+    automatically satisfies it for `tools/list` too.
+  - **Migration**: either reorder your `tools/list` registration to after
+    `instrumentMcpServer()`, or pass `schemaDrift: { enabled: false }` to
+    opt out and keep your existing registration order — both fully
+    restore v0.7.0 behavior. There is no change if you don't use a
+    low-level `Server` with an independently-registered `tools/list`
+    handler.
+  - Only `schemaDrift` (schema drift detection, below) causes this — the
+    two-axis observation contract and cost-aware sampling features in this
+    same release are purely additive, with no effect on when
+    `instrumentMcpServer()` throws.
+
+### Added
+
+- **Tool schema drift detection.** Every `tools/list` response is captured,
+  canonicalized, and hashed per tool (`inputSchema` only — `description` is
+  a deliberately separate, not-yet-built dimension; see ADR 010). When a
+  previously-observed tool's schema hash changes, this emits:
+  - An `mcp.tool.schema_drift.detected` counter, labeled `gen_ai.tool.name`
+    and `mcp.tool.schema_drift.type` (both bounded — see
+    `METRIC_SAFE_ATTRIBUTES`, `src/schema-drift/attributes.js`).
+  - An `mcp.tool.schema_drift.detected` span event on the new `tools/list`
+    span (see Changed, below), carrying the drift `type`, the previous/
+    current schema hash, and — only when non-empty, never set to `[]` —
+    which field names were added/removed/changed. Field names are
+    span-only, never a metric label (unbounded across tools/deployments,
+    same reasoning as `mcp.failure.validation_paths`, ADR 009).
+  - `type` is one of `field_added` \| `field_removed` \| `type_changed` \|
+    `required_changed` \| `multiple` (more than one at once, never guessed
+    down to a single answer) \| `unknown` (a change this differ can't
+    confidently characterize — e.g. a top-level `oneOf`/`anyOf`/`allOf`
+    composition, or a change hidden behind a `$ref`/`$defs` indirection
+    that doesn't touch the referencing property's own value — still
+    reported as drift, just not attributable to a specific field).
+  - The **first** observation of any given tool is never reported as
+    drift (nothing to compare against yet — cold start). A tool that
+    stops appearing in `tools/list` responses for a while and later
+    reappears is compared against its last-seen schema, not treated as a
+    fresh cold start — see the README's "Tool schema drift detection"
+    section for why this is the correct, and possibly counter-intuitive,
+    behavior.
+  - State is scoped **per instrumented server instance, not per session**
+    (ADR 010, Q4) — every client session sees the same tool registry, so
+    session-keyed state would produce false cold-starts per new session
+    and could silently swallow drift that happened between sessions.
+  - New `schemaDrift` option on `instrumentMcpServer()`, following the
+    exact `thrashDetection`/`costTracking` partial-overrides-individual-
+    defaults pattern: `enabled` (default `true`) and `maxTrackedTools`
+    (default `1000`, an LRU cap — defense-in-depth, not a response to an
+    expected failure mode; a server's own tool count is normally small).
+    Each independently overridable via `OTEL_MCP_SCHEMA_DRIFT_ENABLED` /
+    `OTEL_MCP_SCHEMA_DRIFT_MAX_TRACKED_TOOLS`, following the existing
+    `OTEL_MCP_THRASH_*` env-var convention.
+    **Unlike** `thrashDetection`/`costTracking`, `schemaDrift.enabled: false`
+    is a true no-op: `tools/list` is not wrapped at all (no span, no
+    capture, no detector/emitter construction) — the `tools/list` span
+    this feature introduces exists purely for schema drift, unlike the
+    `tools/call` span, which already serves other purposes regardless of
+    sub-feature flags.
+  - `SchemaDriftConfig`, `SchemaDriftKind`, `SchemaDriftEvent` types,
+    exported from the package root.
+
+### Added — Two-axis observation contract
+
+- **`getObservationState()`**, a new accessor attached to the object
+  `instrumentMcpServer()` returns — unconditional (not gated behind
+  `setupNodeSdk`), same additive pattern as `shutdown()`/`getThrashSummary()`,
+  omitted entirely when `options.enabled` is `false`. Returns:
+  ```
+  {
+    toolOutcome: { success, failure, unknown },
+    observationIntegrity: 'DEGRADED' | 'UNKNOWN',
+  }
+  ```
+  No OTel emission — purely in-process, nothing sent anywhere, safe to
+  call from application code (a health-check endpoint, a periodic
+  `console.log`, a debugger). See the README's "Two-axis observation
+  contract" section for the full design rationale.
+  - **`toolOutcome`**: cumulative tool-call outcome counts since
+    instrumentation, from a **new counter that increments on every tool
+    call unconditionally** — independent of `fingerprinting`,
+    `thrashDetection`, and `enableMetrics`. Deliberately NOT read off
+    `ThrashDetector`/`getThrashSummary()`: that bookkeeping only runs
+    when a fingerprint was computed, so with `fingerprinting: false` (a
+    fully supported configuration) it would silently report zero
+    failures regardless of how many actually occurred — the exact
+    silent-success failure mode this feature exists to close. A
+    malformed, unrecognizable tool result (not a real `CallToolResult`
+    shape) increments `unknown` rather than silently defaulting to
+    `success`.
+  - **`observationIntegrity`**: `'DEGRADED' | 'UNKNOWN'` — note there is
+    no `'HEALTHY'` value, and this is not an oversight. Investigated and
+    found structurally unreachable in every configuration: the one lead
+    (OTel SDK self-observability metrics) is a write-only `Counter` with
+    no synchronous read-back API in `@opentelemetry/api`, so this
+    library's own code can never positively confirm telemetry is
+    flowing, no matter how it's wired up. `HEALTHY` is therefore absent
+    from the type entirely, not merely never returned — enforced by
+    TypeScript, not just documentation (see the new type-level tests in
+    `test/index.exports.test-d.ts`).
+    - `DEGRADED` is detected via a fragile `ProxyTracerProvider`
+      reference-equality check, and is reachable **only** under
+      `setupNodeSdk: false` (the default) — when no `TracerProvider` has
+      been registered globally at all.
+    - Under `setupNodeSdk: true`, this library registers the provider
+      itself, so absence can never be confirmed — `observationIntegrity`
+      is **always** `'UNKNOWN'` in that configuration, without even
+      attempting the check.
+    - Recomputed fresh on **every call** to `getObservationState()`,
+      never cached from instrument time — a host may register a
+      `TracerProvider` asynchronously after `instrumentMcpServer()`
+      already ran, and a value cached at startup would go stale the
+      moment that happens.
+  - `ToolOutcome`, `ToolOutcomeCounts`, `ObservationIntegrity`,
+    `ObservationState` types, exported from the package root.
+
+### Added — Cost-aware trace sampling (marker attribute + Collector recipe)
+
+- **`mcp.tool.thrash_detected`, a new boolean span attribute**, set
+  alongside (never instead of) the existing `mcp.loop.detected` span
+  event, in the same `thrash/emitter.js` call site — set only when
+  `thrashDetection` is enabled and a loop was actually detected on this
+  call, same reachability as the existing event, no new failure mode.
+  Exists specifically so an OpenTelemetry Collector's
+  `tailsamplingprocessor` has an unambiguous, attribute-level signal to
+  key on: whether a `boolean_attribute` policy can also match span-*event*
+  data was investigated and left genuinely unverified (the processor is
+  Go source in a separate repository, not installed here), so this
+  attribute removes that uncertainty entirely rather than leaving tail
+  sampling dependent on an unconfirmed answer. **Named deliberately
+  differently** from the pre-existing `mcp.tool.loop.detected` **metric**
+  counter, not reusing its string as ADR 011 originally specified — see
+  that ADR's "Update" note. A metric name and a span attribute key are
+  unrelated OTel namespaces with no technical conflict, but reusing the
+  name left the one reader who most needs it to be unambiguous — someone
+  writing a Collector tail-sampling policy — unable to tell, from the
+  name alone, which of the two same-named signals they were keying on.
+- **No new cost-threshold attribute or config.** `mcp.tool.cost.usd` and
+  `mcp.tool.cost.budget_exceeded` (both already shipped, v0.5.0) already
+  fully suffice for a Collector `numeric_attribute` / `boolean_attribute`
+  policy — the numeric threshold itself lives entirely in the
+  Collector's own policy config (the YAML), not in this package's
+  `InstrumentOptions`. No new env var, no new `instrumentMcpServer()`
+  option.
+- **A documented, pasteable OpenTelemetry Collector `tailsamplingprocessor`
+  config** (README's "Cost-aware trace sampling" section) keeping any
+  trace with an expensive call, a budget-exceeded call, or a detected
+  thrash loop, alongside an ordinary probabilistic sample for everything
+  else.
+- **No in-process sampler or buffering `SpanProcessor` was built, and none
+  is planned** — investigated and rejected on two independent grounds
+  (ADR 011): this package doesn't own the `Sampler`/`SpanProcessor` chain
+  in its default configuration (no public API to inject either into a
+  host-owned `TracerProvider`), and even where a custom processor could
+  theoretically be installed, an in-process decision can only ever rescue
+  the one span this package itself creates — never an already-finished
+  child span from other instrumentation, never an upstream span in a
+  different process. "Keep the trace" is not achievable in-process; at
+  best, "keep this one span" is, which is a materially smaller guarantee
+  than the stated goal. Real cross-span, cross-process trace buffering is
+  what the Collector's `tailsamplingprocessor` already does correctly —
+  not something to partially re-implement inside this package.
+
 ## 0.7.0
 
 Origin-aware failure classification for Agent Thrash Detection. Prompted by