@danieljvdm/dev-kit 1.0.2 → 2.0.0

package/skills/sentry/references/sdks/index.md

@@ -0,0 +1,41 @@
+# Choose a Sentry SDK reference
+
+Inspect the package and runtime being changed. Prefer the existing framework SDK
+when it fits: TanStack Start, Next.js, NestJS, React Router Framework, and React
+Native have integrations beyond generic React or Node. A monorepo can contain
+several targets; a Wrangler file elsewhere does not determine a React screen's SDK.
+
+For an existing SDK, read the requested signal file directly under its directory:
+`tracing.md`, `error-monitoring.md`, `logging.md`, `metrics.md`, `crons.md`,
+`profiling.md`, `session-replay.md`, `user-feedback.md`, or `ai-monitoring.md`,
+when present. Use the platform index for new setup, feature support, or an
+initialization question. It is not a prerequisite to every signal change.
+
+The retained platform references contain multiple runtimes and feature recipes.
+Inspect headings and read the relevant section, not the whole manual. Apply
+installation, provisioning, and optional-feature recipes only within the requested
+scope; preserve installed versions and consult matching official docs for APIs.
+
+| Platform                                              | Setup and feature index                                   |
+| ----------------------------------------------------- | --------------------------------------------------------- |
+| Android                                               | [android](android/index.md)                               |
+| browser JavaScript                                    | [browser](browser/index.md)                               |
+| Cloudflare Workers and Pages                          | [cloudflare](cloudflare/index.md)                         |
+| Apple platforms (iOS, macOS, tvOS, watchOS, visionOS) | [cocoa](cocoa/index.md)                                   |
+| .NET                                                  | [dotnet](dotnet/index.md)                                 |
+| Elixir                                                | [elixir](elixir/index.md)                                 |
+| Go                                                    | [go](go/index.md)                                         |
+| NestJS                                                | [nestjs](nestjs/index.md)                                 |
+| Next.js                                               | [nextjs](nextjs/index.md)                                 |
+| Node.js, Bun, and Deno                                | [node](node/index.md)                                     |
+| PHP                                                   | [php](php/index.md)                                       |
+| Python                                                | [python](python/index.md)                                 |
+| Flutter and Dart                                      | [flutter](flutter/index.md)                               |
+| React Native and Expo                                 | [react-native](react-native/index.md)                     |
+| React                                                 | [react](react/index.md)                                   |
+| React Router Framework                                | [react-router-framework](react-router-framework/index.md) |
+| TanStack Start React                                  | [tanstack-start](tanstack-start/index.md)                 |
+| Ruby                                                  | [ruby](ruby/index.md)                                     |
+| Svelte and SvelteKit                                  | [svelte](svelte/index.md)                                 |
+
+For an unsupported platform, use the [official SDK documentation](https://docs.sentry.io/platforms/).

package/skills/sentry/references/sdks/nestjs/ai-monitoring.md

@@ -0,0 +1,410 @@
+# AI Monitoring — Sentry NestJS SDK
+
+> OpenAI integration: `@sentry/nestjs` ≥10.53.0+ Vercel AI SDK integration: ≥10.53.0+
+> Anthropic integration: ≥10.53.0+ Google GenAI integration: ≥10.53.0+
+
+> ⚠️ **Tracing must be enabled.** AI monitoring piggybacks on tracing infrastructure.
+> `tracesSampleRate` must be > 0.
+
+---
+
+## Overview
+
+Sentry AI Agents Monitoring automatically tracks:
+
+- Agent runs and error rates
+- LLM calls (model, token counts, estimated cost)
+- Tool calls and outputs
+- Agent handoffs
+- Full prompt/completion data (opt-in)
+- Performance bottlenecks across the AI pipeline
+
+All integrations listed below are **auto-enabled** when the corresponding AI library is
+detected at startup.
+Explicit configuration is only needed to customize `recordInputs`/`recordOutputs`.
+
+---
+
+## Supported AI Libraries
+
+| Library                                        | Integration API                                          | Auto-enabled? | Min SDK Version |
+| ---------------------------------------------- | -------------------------------------------------------- | ------------- | --------------- |
+| **OpenAI** (`openai`)                          | `openAIIntegration` / `instrumentOpenAiClient`           | ✅ Yes        | **10.53.0**     |
+| **Vercel AI SDK** (`ai`)                       | `vercelAIIntegration`                                    | ✅ Yes        | **10.53.0**     |
+| **Anthropic** (`@anthropic-ai/sdk`)            | `anthropicAIIntegration` / `instrumentAnthropicAiClient` | ✅ Yes        | **10.53.0**     |
+| **Google GenAI** (`@google/generative-ai`)     | —                                                        | ✅ Yes        | **10.53.0**     |
+| **LangChain** (`langchain`, `@langchain/core`) | `langchainIntegration`                                   | ✅ Yes        | **10.53.0**     |
+
+---
+
+## OpenAI Integration
+
+### Auto-Enabled Setup
+
+OpenAI is auto-instrumented — no changes to `instrument.ts` needed:
+
+```typescript
+// instrument.ts
+import * as Sentry from "@sentry/nestjs";
+
+Sentry.init({
+  dsn: process.env.SENTRY_DSN,
+  tracesSampleRate: 1.0,
+  dataCollection: {
+    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
+    // https://docs.sentry.io/platforms/javascript/guides/nestjs/configuration/options/#dataCollection
+    // userInfo: false,
+    // httpBodies: [],
+  },
+  integrations: [Sentry.openAIIntegration()],
+});
+```
+
+### Manual Wrapping (Alternative)
+
+If auto-instrumentation doesn’t capture your client (e.g., custom transport), wrap it
+manually:
+
+```typescript
+import OpenAI from "openai";
+import * as Sentry from "@sentry/nestjs";
+
+const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+
+// Wrap once at module level — reuse this client everywhere.
+// Input/output recording is on by default (governed by dataCollection.genAI) unless explicitly overridden.
+const client = Sentry.instrumentOpenAiClient(openai);
+```
+
+### Streaming — Important
+
+For streamed responses, you **must** pass `stream_options: { include_usage: true }`.
+Without this, OpenAI does not include token counts in streamed responses, so Sentry
+cannot capture usage metrics:
+
+```typescript
+@Injectable()
+export class ChatService {
+  constructor(private readonly openai: OpenAI) {}
+
+  async streamChat(messages: Array<{ role: string; content: string }>) {
+    const stream = await this.openai.chat.completions.create({
+      model: "gpt-4o",
+      messages,
+      stream: true,
+      stream_options: { include_usage: true }, // ← REQUIRED for token tracking
+    });
+    return stream;
+  }
+}
+```
+
+### OpenAI Configuration Options
+
+| Option          | Type      | Default                                     | Description                             |
+| --------------- | --------- | ------------------------------------------- | --------------------------------------- |
+| `recordInputs`  | `boolean` | `true` (governed by `dataCollection.genAI`) | Capture prompts/messages sent to OpenAI |
+| `recordOutputs` | `boolean` | `true` (governed by `dataCollection.genAI`) | Capture generated text/responses        |
+
+**Supported versions:** `openai` ≥4.0.0
+
+---
+
+## Vercel AI SDK Integration
+
+### Setup
+
+The integration is **auto-enabled** when the `ai` package is detected:
+
+```typescript
+// instrument.ts — customize if needed
+import * as Sentry from "@sentry/nestjs";
+
+Sentry.init({
+  dsn: process.env.SENTRY_DSN,
+  tracesSampleRate: 1.0,
+  dataCollection: {
+    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
+    // https://docs.sentry.io/platforms/javascript/guides/nestjs/configuration/options/#dataCollection
+    // userInfo: false,
+    // httpBodies: [],
+  },
+  integrations: [Sentry.vercelAIIntegration()],
+});
+```
+
+### Per-Call Telemetry (Required)
+
+You **must** pass `experimental_telemetry: { isEnabled: true }` to every AI SDK function
+call you want traced:
+
+```typescript
+import { Injectable } from "@nestjs/common";
+import { generateText, streamText } from "ai";
+import { openai } from "@ai-sdk/openai";
+
+@Injectable()
+export class AiService {
+  async generate(prompt: string) {
+    const result = await generateText({
+      model: openai("gpt-4o"),
+      prompt,
+      experimental_telemetry: {
+        isEnabled: true,
+        functionId: "my-text-generation",
+        recordInputs: true,
+        recordOutputs: true,
+      },
+    });
+    return result.text;
+  }
+
+  async *stream(prompt: string) {
+    const { textStream } = await streamText({
+      model: openai("gpt-4o"),
+      prompt,
+      experimental_telemetry: {
+        isEnabled: true,
+        functionId: "my-stream",
+      },
+    });
+    yield* textStream;
+  }
+}
+```
+
+### Vercel AI SDK Configuration Options
+
+| Option          | Type      | Default | Min SDK | Description                                                                |
+| --------------- | --------- | ------- | ------- | -------------------------------------------------------------------------- |
+| `recordInputs`  | `boolean` | `true`* | 9.27.0  | Capture inputs. *Defaults to `true` (governed by `dataCollection.genAI`).  |
+| `recordOutputs` | `boolean` | `true`* | 9.27.0  | Capture outputs. *Defaults to `true` (governed by `dataCollection.genAI`). |
+
+**Supported versions:** `ai` ≥3.0.0
+
+---
+
+## Anthropic Integration
+
+### Setup
+
+```typescript
+// instrument.ts — customize if needed
+import * as Sentry from "@sentry/nestjs";
+
+Sentry.init({
+  dsn: process.env.SENTRY_DSN,
+  tracesSampleRate: 1.0,
+  dataCollection: {
+    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
+    // https://docs.sentry.io/platforms/javascript/guides/nestjs/configuration/options/#dataCollection
+    // userInfo: false,
+    // httpBodies: [],
+  },
+  integrations: [Sentry.anthropicAIIntegration()],
+});
+```
+
+### Manual Wrapping
+
+```typescript
+import Anthropic from "@anthropic-ai/sdk";
+import * as Sentry from "@sentry/nestjs";
+
+const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
+
+// Input/output recording is on by default (governed by dataCollection.genAI) unless explicitly overridden.
+const client = Sentry.instrumentAnthropicAiClient(anthropic);
+
+const response = await client.messages.create({
+  model: "claude-sonnet-4-20250514",
+  max_tokens: 1024,
+  messages: [{ role: "user", content: "Hello, Claude!" }],
+});
+```
+
+### Supported Anthropic Operations
+
+| Operation       | Method                          |
+| --------------- | ------------------------------- |
+| Create messages | `client.messages.create()`      |
+| Stream messages | `client.messages.stream()`      |
+| Count tokens    | `client.messages.countTokens()` |
+| Beta messages   | `client.beta.messages.create()` |
+
+**Supported versions:** `@anthropic-ai/sdk` ≥0.19.2
+
+---
+
+## Token Usage Tracking
+
+Sentry automatically captures token usage following OpenTelemetry GenAI semantic
+conventions:
+
+| Span Attribute                         | Description                                                                                                     |
+| -------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
+| `gen_ai.request.model`                 | Model name                                                                                                      |
+| `gen_ai.request.reasoning_effort`      | Reasoning effort level for reasoning models (e.g., `low`, `medium`, `high`). Supported values vary by provider. |
+| `gen_ai.usage.input_tokens`            | Prompt/input token count                                                                                        |
+| `gen_ai.usage.output_tokens`           | Completion/output token count                                                                                   |
+| `gen_ai.usage.input_tokens.cached`     | Cached input tokens                                                                                             |
+| `gen_ai.usage.output_tokens.reasoning` | Reasoning tokens (e.g., o1 models)                                                                              |
+
+**Cost estimates** are sourced from models.dev and OpenRouter.
+Unrecognized models show no estimate.
+
+---
+
+## Prompt/Completion Capture & PII
+
+`recordInputs` captures prompts sent to the AI API. `recordOutputs` captures the
+generated text/completions returned.
+
+With `dataCollection`, genAI input/output capture is **on by default**. To disable it,
+set `dataCollection: { genAI: { inputs: false, outputs: false } }`:
+
+```typescript
+Sentry.init({
+  dsn: process.env.SENTRY_DSN,
+  tracesSampleRate: 1.0,
+  dataCollection: {
+    genAI: { inputs: false, outputs: false },
+  },
+});
+```
+
+> ⚠️ **PII warning:** Prompts often contain user-supplied text.
+> If users include personal data in prompts, enabling `recordInputs` will send that data
+> to Sentry. Review your privacy policy before enabling.
+
+---
+
+## Complete NestJS Example
+
+```typescript
+// instrument.ts
+import * as Sentry from "@sentry/nestjs";
+
+Sentry.init({
+  dsn: process.env.SENTRY_DSN,
+  tracesSampleRate: 1.0,
+  dataCollection: {
+    // To disable sending user data and HTTP bodies, uncomment the lines below. For more info visit:
+    // https://docs.sentry.io/platforms/javascript/guides/nestjs/configuration/options/#dataCollection
+    // userInfo: false,
+    // httpBodies: [],
+  },
+  enableLogs: true,
+  integrations: [
+    Sentry.openAIIntegration(),
+    Sentry.vercelAIIntegration(),
+    Sentry.anthropicAIIntegration(),
+  ],
+});
+```
+
+```typescript
+// ai.controller.ts
+import { Controller, Post, Body } from "@nestjs/common";
+import { AiService } from "./ai.service";
+
+@Controller("ai")
+export class AiController {
+  constructor(private readonly aiService: AiService) {}
+
+  @Post("chat")
+  async chat(@Body() body: { prompt: string }) {
+    return this.aiService.chat(body.prompt);
+  }
+}
+```
+
+```typescript
+// ai.service.ts
+import { Injectable } from "@nestjs/common";
+import OpenAI from "openai";
+import * as Sentry from "@sentry/nestjs";
+
+@Injectable()
+export class AiService {
+  private readonly openai: OpenAI;
+
+  constructor() {
+    this.openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+  }
+
+  async chat(prompt: string): Promise<string> {
+    const completion = await this.openai.chat.completions.create({
+      model: "gpt-4o",
+      messages: [{ role: "user", content: prompt }],
+    });
+    return completion.choices[0].message.content ?? "";
+  }
+}
+```
+
+---
+
+## AI Agents Dashboard
+
+Access at **Sentry → AI → Agents** (or **Insights → AI**).
+
+| Tab          | What you see                                                              |
+| ------------ | ------------------------------------------------------------------------- |
+| **Overview** | Agent runs, error rates, duration, LLM calls, tokens used, tool calls     |
+| **Models**   | Per-model cost estimates, token breakdown (input/output/cached), duration |
+| **Tools**    | Per-tool call counts, error rates, input/output for each invocation       |
+| **Traces**   | Full pipeline from user request to final response with all spans          |
+
+---
+
+## Sampling Strategy
+
+If your `tracesSampleRate` is below 1.0, you may be losing entire agent runs.
+Use `tracesSampler` patterns that keep 100% of gen_ai-related transactions while
+sampling other traffic at a lower rate.
+
+---
+
+## Conversation Tracking
+
+Link AI spans across turns into a chat-style timeline at **Explore > Conversations**.
+
+**Prerequisites:** `streamGenAiSpans` defaults to `true` (SDK >=10.61.0, so AI spans
+stream as standalone items) and genAI input/output capture enabled (on by default via
+`dataCollection`) — Conversations reconstructs the chat from input/output attributes, so
+if you’ve disabled genAI capture the view will be empty.
+
+```typescript
+import * as Sentry from "@sentry/nestjs";
+
+// Set at the start of a conversation
+Sentry.setConversationId("conv_abc123");
+
+// All subsequent AI calls carry gen_ai.conversation.id: "conv_abc123"
+```
+
+A single conversation can span multiple traces, and a single trace can contain multiple
+conversations.
+
+### User Attribution
+
+To populate the **User** column in Conversations, call `setUser` once per request or
+session before any AI calls:
+
+```typescript
+Sentry.setUser({ id: "user_123", email: "jane@example.com", username: "jane" });
+```
+
+## Troubleshooting
+
+| Issue                                        | Solution                                                                                                                                                                                  |
+| -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| No AI spans appearing                        | Verify `tracesSampleRate` > 0; AI monitoring requires tracing                                                                                                                             |
+| Token counts missing in streams              | Add `stream_options: { include_usage: true }` to all OpenAI streaming calls                                                                                                               |
+| `recordInputs`/`recordOutputs` not capturing | genAI capture is on by default; ensure you haven’t set `dataCollection: { genAI: { inputs: false } }`, or explicitly pass `recordInputs: true` / `recordOutputs: true` to the integration |
+| Anthropic spans missing                      | Check SDK version; add `anthropicAIIntegration()` explicitly                                                                                                                              |
+| Cost estimates not showing                   | Model name must match models.dev/OpenRouter pricing data; custom models may show no estimate                                                                                              |
+| Vercel AI spans not tracked                  | Pass `experimental_telemetry: { isEnabled: true }` to every AI SDK call                                                                                                                   |
+| No data in AI Agents dashboard               | Ensure traces are being sent; check DSN and `tracesSampleRate`                                                                                                                            |
+| User column shows “Unknown”                  | Call `Sentry.setUser()` once per request or session                                                                                                                                       |