@assistant-ui/mcp-docs-server 0.1.30 → 0.1.32

package/.docs/raw/docs/integrations/observability/langfuse.mdx

@@ -0,0 +1,163 @@
+---
+title: Langfuse
+description: Trace AI SDK calls into Langfuse via OpenTelemetry for tracing, evals, and prompt management.
+---
+
+import { HeliconeIcon } from "@/components/icons/helicone";
+import { LangChainIcon } from "@/components/icons/langchain";
+import { VercelIcon } from "@/components/icons/vercel";
+
+[Langfuse](https://langfuse.com/) is an open-source LLM observability platform. It gives you a hierarchical trace per request (planner → tool calls → final LLM step), prompt-level analytics, datasets, and LLM-as-judge evals. Self-hostable, OpenTelemetry-native.
+
+Pick Langfuse when you want to see the agent's full call tree inside a single turn. It's complementary to [Helicone](/docs/integrations/observability/helicone), which proxies and logs individual provider calls; many teams run both, with Helicone capturing the request log and Langfuse capturing the trace.
+
+## How it works
+
+```
+your route  ──►  AI SDK streamText (with experimental_telemetry)
+                  │
+                  ▼
+                  OpenTelemetry SDK  ──►  LangfuseSpanProcessor  ──►  Langfuse
+```
+
+Langfuse subscribes to OpenTelemetry spans the AI SDK already emits when telemetry is enabled. No proxy, no wrapping; the SDK ships spans and Langfuse renders them.
+
+## Setup
+
+<Steps>
+<Step>
+
+### Get Langfuse credentials
+
+Sign up at [langfuse.com](https://langfuse.com/) (or self-host) and create a project. Copy the public and secret keys from project settings.
+
+```sh title=".env.local"
+LANGFUSE_PUBLIC_KEY=pk-lf-...
+LANGFUSE_SECRET_KEY=sk-lf-...
+LANGFUSE_BASE_URL=https://cloud.langfuse.com
+```
+
+The base URL is region-specific: `https://cloud.langfuse.com` (EU), `https://us.cloud.langfuse.com`, or your self-hosted URL.
+
+</Step>
+<Step>
+
+### Install the OTel and Langfuse packages
+
+<InstallCommand npm={["@langfuse/tracing", "@langfuse/otel", "@opentelemetry/sdk-node"]} />
+
+`@langfuse/tracing` provides the helpers that label traces with user, session, and trace name. `@langfuse/otel` provides the span processor. `@opentelemetry/sdk-node` is the OTel SDK.
+
+</Step>
+<Step>
+
+### Initialize OpenTelemetry once at startup
+
+Create an instrumentation file that boots the OTel SDK and registers the Langfuse span processor. In Next.js this goes in `instrumentation.ts` so it runs once per server process. Export the processor at module scope so other code can call `forceFlush()` on it.
+
+```ts title="instrumentation.ts"
+import { NodeSDK } from "@opentelemetry/sdk-node";
+import { LangfuseSpanProcessor } from "@langfuse/otel";
+
+export const langfuseSpanProcessor = new LangfuseSpanProcessor();
+
+export async function register() {
+  if (process.env.NEXT_RUNTIME !== "nodejs") return;
+
+  const sdk = new NodeSDK({
+    spanProcessors: [langfuseSpanProcessor],
+  });
+  sdk.start();
+}
+```
+
+`NEXT_RUNTIME !== "nodejs"` skips the edge runtime, where OTel doesn't run. On Next.js 14 or earlier, also set `experimental.instrumentationHook = true` in `next.config.mjs`:
+
+```js title="next.config.mjs"
+const nextConfig = {
+  experimental: { instrumentationHook: true },
+};
+export default nextConfig;
+```
+
+</Step>
+<Step>
+
+### Wrap AI SDK calls with `propagateAttributes`
+
+Enable telemetry with `experimental_telemetry: { isEnabled: true }`, and wrap each call in `propagateAttributes` from `@langfuse/tracing` to set the trace name and group traces by user / session.
+
+```ts title="app/api/chat/route.ts"
+import { openai } from "@ai-sdk/openai";
+import { streamText, convertToModelMessages } from "ai";
+import type { UIMessage } from "ai";
+import { propagateAttributes } from "@langfuse/tracing";
+
+export async function POST(req: Request) {
+  const { messages }: { messages: UIMessage[] } = await req.json();
+
+  const userId = "<resolve from your session>";
+  const sessionId = "<resolve from your thread state>";
+
+  const result = await propagateAttributes(
+    { traceName: "chat-completion", userId, sessionId },
+    async () =>
+      streamText({
+        model: openai("gpt-5.4-nano"),
+        messages: await convertToModelMessages(messages),
+        experimental_telemetry: { isEnabled: true },
+      }),
+  );
+
+  return result.toUIMessageStreamResponse();
+}
+```
+
+`traceName` becomes the trace label in the Langfuse dashboard. `userId` and `sessionId` are the canonical Langfuse filter dimensions; pass real values from your auth and thread state, not literal strings.
+
+</Step>
+<Step>
+
+### Run and verify
+
+Send a message through your assistant. Within a few seconds, a trace should appear in your Langfuse dashboard with:
+
+- The trace name set by `traceName`.
+- A child span per LLM call and per tool call.
+- Full prompt, completion, and token usage on each span.
+- The metadata you passed (user, session, custom keys) as filters.
+
+If nothing appears, check the server logs for OTel errors and confirm `LANGFUSE_PUBLIC_KEY` / `LANGFUSE_SECRET_KEY` are loaded in the runtime that handles the request.
+
+</Step>
+</Steps>
+
+## Notes
+
+- **Serverless flush.** On serverless platforms the function exits before OTel flushes its buffer, dropping traces. Import the processor exported in the previous step and call `await langfuseSpanProcessor.forceFlush()` before responding, or use the runtime's `waitUntil` API. Langfuse's docs cover the deployment-specific patterns.
+- **Self-hosting.** Point `LANGFUSE_BASE_URL` at your self-hosted instance. The integration is otherwise identical.
+- **Sampling.** For high-traffic apps, configure OTel sampling on `NodeSDK` to keep cost predictable. Langfuse can also sample at the project level.
+- **Pairing with Helicone.** They are complementary: Helicone proxies and logs every request; Langfuse traces the agent. Many teams use both.
+
+## Related
+
+<Cards>
+  <Card
+    icon={<LangChainIcon width={20} height={20} className="text-[#7FC8FF]" />}
+    title="LangSmith"
+    description="LangChain ecosystem alternative, uses wrapAISDK instead of OpenTelemetry."
+    href="/docs/integrations/observability/langsmith"
+  />
+  <Card
+    icon={<HeliconeIcon width={20} height={20} />}
+    title="Helicone"
+    description="Proxy-based request logging with cost, latency, and prompt diffs per call."
+    href="/docs/integrations/observability/helicone"
+  />
+  <Card
+    icon={<VercelIcon width={20} height={20} />}
+    title="AI SDK runtime"
+    description="The runtime that emits the telemetry Langfuse consumes."
+    href="/docs/runtimes/ai-sdk/v6"
+  />
+</Cards>

package/.docs/raw/docs/integrations/observability/langsmith.mdx

@@ -0,0 +1,150 @@
+---
+title: LangSmith
+description: Trace AI SDK calls into LangSmith with the wrapAISDK helper.
+---
+
+import { LangfuseIcon } from "@/components/icons/langfuse";
+import { LangGraphIcon } from "@/components/icons/langgraph";
+import { VercelIcon } from "@/components/icons/vercel";
+
+[LangSmith](https://www.langchain.com/langsmith) is LangChain's observability and eval platform. If you are already in the LangChain or LangGraph ecosystem, LangSmith is the natural pairing: traces, datasets, prompt versioning, and LLM-as-judge evals share state with the rest of the LangChain stack.
+
+This page covers the **AI SDK** path. If you use [`@assistant-ui/react-langgraph`](/docs/runtimes/langgraph), tracing flows through LangGraph Cloud automatically; you only need this guide when your route handler talks to AI SDK directly.
+
+## How it works
+
+LangSmith provides a wrapper around the `ai` namespace. You call `wrapAISDK(ai)`, get back the same exports (`generateText`, `streamText`, `generateObject`, `streamObject`), and use those in place of the originals. Every call is then traced.
+
+```
+your route  ──►  wrapped streamText  ──►  LangSmith client  ──►  LangSmith
+```
+
+## Setup
+
+<Steps>
+<Step>
+
+### Get LangSmith credentials
+
+Sign up at [smith.langchain.com](https://smith.langchain.com/) and copy an API key from settings.
+
+```sh title=".env.local"
+LANGSMITH_TRACING=true
+LANGSMITH_API_KEY=lsv2_pt_...
+LANGSMITH_PROJECT=assistant-ui
+```
+
+`LANGSMITH_PROJECT` controls which project receives traces; the default project applies if you omit it. See LangSmith's [environment variable reference](https://docs.langchain.com/langsmith/how_to_environment_variables) for the full list.
+
+</Step>
+<Step>
+
+### Install the LangSmith SDK
+
+<InstallCommand npm={["langsmith"]} />
+
+</Step>
+<Step>
+
+### Wrap the AI SDK
+
+`wrapAISDK` re-exports `generateText`, `streamText`, `generateObject`, and `streamObject` with tracing enabled. Use the wrapped versions in your route.
+
+```ts title="app/api/chat/route.ts"
+import * as ai from "ai";
+import { wrapAISDK } from "langsmith/experimental/vercel";
+import { openai } from "@ai-sdk/openai";
+import type { UIMessage } from "ai";
+
+const { streamText } = wrapAISDK(ai);
+
+export async function POST(req: Request) {
+  const { messages }: { messages: UIMessage[] } = await req.json();
+
+  const result = streamText({
+    model: openai("gpt-5.4-nano"),
+    messages: await ai.convertToModelMessages(messages),
+  });
+
+  return result.toUIMessageStreamResponse();
+}
+```
+
+`convertToModelMessages` is not part of the wrapper, so import it from `ai` directly.
+
+</Step>
+<Step>
+
+### Add metadata for grouping (optional)
+
+Pass a `langsmith` provider option to tag traces with user, session, or run identifiers. Use `createLangSmithProviderOptions` to build the value:
+
+```ts
+import { createLangSmithProviderOptions } from "langsmith/experimental/vercel";
+
+const result = streamText({
+  model: openai("gpt-5.4-nano"),
+  messages: await ai.convertToModelMessages(messages),
+  providerOptions: {
+    langsmith: createLangSmithProviderOptions({
+      name: "chat-completion",
+      metadata: { userId, threadId },
+    }),
+  },
+});
+```
+
+`name` becomes the run name in LangSmith. Traces filter by the metadata fields you pass; resolve `userId` and `threadId` from your auth and thread state, don't ship literal strings.
+
+</Step>
+<Step>
+
+### Run and verify
+
+Send a message. The trace should appear in your LangSmith project within seconds. Confirm:
+
+- A new run named according to `name` (or the default `streamText`).
+- Inputs (messages), outputs (completion), token usage, and latency are populated.
+- Metadata fields appear as filters.
+
+</Step>
+</Steps>
+
+## Notes
+
+- **Serverless flush.** Serverless functions exit before LangSmith flushes batched traces. Before returning, force the flush with the `Client` from `langsmith`:
+
+  ```ts
+  import { Client } from "langsmith";
+  const client = new Client();
+  // ...inside your route handler, after streamText:
+  await client.awaitPendingTraceBatches();
+  ```
+
+  Without this you will lose traces on Vercel, AWS Lambda, and similar platforms.
+- **`experimental_telemetry` vs `wrapAISDK`.** The AI SDK has a generic `experimental_telemetry` flag that emits OpenTelemetry spans (used by [Langfuse](/docs/integrations/observability/langfuse)). `wrapAISDK` is LangSmith's own path; you do not need to set `experimental_telemetry` when using it.
+- **LangGraph users.** If your backend is LangGraph Cloud, prefer the LangGraph runtime; tracing is built in. Use `wrapAISDK` only when calling AI SDK directly outside of LangGraph.
+- **Version requirements.** LangSmith documents AI SDK v5 as the minimum and `langsmith >= 0.3.63`. The wrapper continues to work against v6 in practice; if you hit an incompatibility, check LangSmith's release notes.
+
+## Related
+
+<Cards>
+  <Card
+    icon={<LangGraphIcon width={20} height={20} className="text-[#1C3C3C] dark:text-[#5b9595]" />}
+    title="LangGraph runtime"
+    description="If your backend is LangGraph, tracing flows through LangGraph Cloud automatically."
+    href="/docs/runtimes/langgraph"
+  />
+  <Card
+    icon={<LangfuseIcon width={20} height={20} />}
+    title="Langfuse"
+    description="OpenTelemetry-based alternative; OSS and self-hostable."
+    href="/docs/integrations/observability/langfuse"
+  />
+  <Card
+    icon={<VercelIcon width={20} height={20} />}
+    title="AI SDK runtime"
+    description="The runtime that ferries traces from the route to the chat UI."
+    href="/docs/runtimes/ai-sdk/v6"
+  />
+</Cards>