@raindrop-ai/deep-agents 0.0.1

package/README.md

@@ -0,0 +1,123 @@
+# @raindrop-ai/deep-agents
+
+Raindrop integration for [LangChain Deep Agents](https://docs.langchain.com/oss/javascript/deepagents/overview). Automatically captures LLM calls, tool usage (planning, filesystem, shell, subagent delegation), chains, and agent actions via LangChain's callback system.
+
+## Installation
+
+```bash
+npm install @raindrop-ai/deep-agents @langchain/core deepagents
+```
+
+## Usage
+
+```typescript
+import { createDeepAgent } from "deepagents";
+import { ChatAnthropic } from "@langchain/anthropic";
+import { createRaindropDeepAgents } from "@raindrop-ai/deep-agents";
+
+const raindrop = createRaindropDeepAgents({
+  writeKey: "your-write-key",
+  userId: "user-123",
+  convoId: "session-abc",
+});
+
+const agent = createDeepAgent({
+  model: new ChatAnthropic({ model: "claude-sonnet-4-20250514" }),
+  tools: [/* filesystem, shell, etc. */],
+});
+
+const result = await agent.invoke(
+  { messages: [{ role: "user", content: "Create a hello world file" }] },
+  { callbacks: [raindrop.handler] },
+);
+
+await raindrop.shutdown();
+```
+
+## What gets captured
+
+- **LLM calls**: model name, input, output, token usage (`prompt_tokens`, `completion_tokens`)
+- **Extended token categories**: `ai.usage.cached_tokens` (OpenAI prompt cache hits, Anthropic `cache_read`), `ai.usage.thoughts_tokens` (o1/o3 reasoning tokens) — when reported by the provider
+- **Finish reason**: captured as `ai.finish_reason` in event properties (`"stop"`, `"length"`, `"tool_calls"`, `"content_filter"`, etc.)
+- **Tool calls**: tool name (`write_todos`, `read_file`, `write_file`, `edit_file`, `ls`, `glob`, `grep`, `execute`, `task`), input, output
+- **Chains**: execution spans with parent-child nesting
+- **Agent actions**: tool selection and finish events
+- **Multi-modal messages**: typed-parts content (`[{type: "text", text: "..."}, ...]`) is extracted cleanly — text-type parts are concatenated; non-text parts (images, audio) are skipped
+- **Errors**: captured with `error.type` + `error.message` properties and `[Error] ...` output on the event, plus OTLP error status on the span
+
+## Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `writeKey` | `string` | - | Raindrop API write key (omit to disable telemetry) |
+| `endpoint` | `string` | `https://api.raindrop.ai/v1/` | API endpoint |
+| `userId` | `string` | - | Associate all events with a user |
+| `convoId` | `string` | - | Group events into a conversation |
+| `debug` | `boolean` | `false` | Enable verbose logging |
+| `traceChains` | `boolean` | `true` | Create spans for chain execution |
+
+## LangGraph Support
+
+Deep Agents is built on LangGraph. The handler automatically:
+- Filters LangGraph-internal chain events (`__start__`, `__end__`, `ChannelWrite:*`, `ChannelRead:*`, `Branch:*`) — the colon is part of LangGraph's naming convention, user-defined chains whose names happen to start with "Branch" (e.g. `BranchRouter`) are NOT filtered
+- Keeps the root `LangGraph` chain so the OTel association context is in place before any inner span fires (filtering it orphans all downstream spans)
+- Deduplicates LLM callbacks that LangGraph fires multiple times with the same `runId`
+- Links parent-child spans for chain→LLM→tool hierarchies
+
+## API Surface
+
+The `createRaindropDeepAgents()` factory returns:
+
+- `handler` — LangChain `BaseCallbackHandler` to pass via `config.callbacks`
+- `lastEventId` — `event_id` of the most recently finalized root event (or `undefined` if none has landed). Use this to attach a follow-up `signals.track()` to the agent invocation you just ran without polling the dashboard. Example:
+  ```typescript
+  await agent.invoke({ messages: [...] }, { callbacks: [raindrop.handler] });
+  if (raindrop.lastEventId) {
+    await raindrop.signals.track({
+      eventId: raindrop.lastEventId,
+      name: "thumbs_up",
+      type: "feedback",
+      sentiment: "POSITIVE",
+    });
+  }
+  ```
+- `events.patch(eventId, patch)` — update event properties
+- `events.finish(eventId, patch)` — finalize an event
+- `events.addAttachments(eventId, attachments)` — attach files/text to an event
+- `events.setProperties(eventId, properties)` — set custom properties
+- `users.identify(users)` — identify users with traits
+- `signals.track(signal)` — track user signals/feedback
+- `flush()` — flush pending telemetry
+- `shutdown()` — flush and close connections
+
+## Known Limitations
+
+- **Beta status**: API surface may change in future releases.
+- **Deep Agents version**: Requires `deepagents >= 0.5.0`.
+- **Streaming**: Token-by-token streaming events are not captured individually; only the final aggregated response is tracked.
+- **Subagent isolation**: When using the `task` tool for subagent delegation, each subagent's callbacks fire independently.
+- **Concurrent invocations on a shared handler**: A single handler instance keeps one in-flight span map at a time. Running multiple `agent.invoke(...)` calls **concurrently with the same handler** (e.g. `Promise.all([agent.invoke(...), agent.invoke(...)])` with the same `raindrop.handler`) can scramble the linkage between events and traces. Instantiate one `createRaindropDeepAgents()` per concurrent request; sequential invocations on a shared handler are fully supported.
+- **Long chain inputs/outputs are truncated**: Chain-level `input` and `output` captured on the root event are truncated to ~8 KB to stay within the SDK's payload-size limit. Per-LLM child events still carry full prompts.
+
+## Testing
+
+```bash
+# Unit tests (no external services; use MSW to intercept HTTP)
+pnpm test -- tests/handler.test.ts
+```
+
+**E2E tests** run REAL Deep Agents workflows via `createDeepAgent()` against a REAL LLM (OpenAI gpt-4o-mini) and verify events / tool calls / traces against the production dashboard TRPC API. They skip automatically when any of `RAINDROP_WRITE_KEY`, `OPENAI_API_KEY`, or `RAINDROP_DASHBOARD_TOKEN` is missing.
+
+```bash
+# E2E — requires all three keys
+RAINDROP_WRITE_KEY=xxx \
+OPENAI_API_KEY=sk-... \
+RAINDROP_DASHBOARD_TOKEN=eyJ... \
+  pnpm test -- tests/e2e.test.ts
+```
+
+The dashboard token comes from `app.raindrop.ai` → DevTools → Network → any `backend.raindrop.ai` request → `Authorization: Bearer ...` (expires every ~30 min, so grab a fresh one immediately before running).
+
+## Python
+
+A Python version of this integration is available as `raindrop-deep-agents` on PyPI. See [the Python package](https://github.com/invisible-tools/dawn/tree/main/packages/deep-agents-python) for details.

package/dist/index.d.mts

@@ -0,0 +1,365 @@
+import { BaseCallbackHandler } from '@langchain/core/callbacks/base';
+import { Serialized } from '@langchain/core/load/serializable';
+import { LLMResult } from '@langchain/core/outputs';
+import { BaseMessage } from '@langchain/core/messages';
+import { AgentAction, AgentFinish } from '@langchain/core/agents';
+import { ChainValues } from '@langchain/core/utils/types';
+
+type OtlpAnyValue = {
+    stringValue?: string;
+    intValue?: string;
+    doubleValue?: number;
+    boolValue?: boolean;
+    arrayValue?: {
+        values: OtlpAnyValue[];
+    };
+};
+type OtlpKeyValue = {
+    key: string;
+    value: OtlpAnyValue;
+};
+declare const SpanStatusCode: {
+    readonly UNSET: 0;
+    readonly OK: 1;
+    readonly ERROR: 2;
+};
+type OtlpSpanStatus = {
+    code: (typeof SpanStatusCode)[keyof typeof SpanStatusCode] | number;
+    message?: string;
+};
+type OtlpSpan = {
+    traceId: string;
+    spanId: string;
+    parentSpanId?: string;
+    name: string;
+    startTimeUnixNano: string;
+    endTimeUnixNano: string;
+    attributes?: OtlpKeyValue[];
+    status?: OtlpSpanStatus;
+};
+type SpanIds = {
+    traceIdB64: string;
+    spanIdB64: string;
+    parentSpanIdB64?: string;
+};
+
+type Attachment = {
+    type: string;
+    role: string;
+    name?: string;
+    value: string;
+};
+type IdentifyInput = {
+    userId: string;
+    traits?: Record<string, unknown>;
+};
+type Patch = {
+    eventName?: string;
+    userId?: string;
+    convoId?: string;
+    input?: string;
+    output?: string;
+    model?: string;
+    properties?: Record<string, unknown>;
+    attachments?: Attachment[];
+    isPending?: boolean;
+    timestamp?: string;
+};
+type SignalInput = {
+    eventId: string;
+    name: string;
+    type?: "default" | "feedback" | "edit" | "standard" | "agent" | "agent_internal";
+    sentiment?: "POSITIVE" | "NEGATIVE";
+    timestamp?: string;
+    properties?: Record<string, unknown>;
+    attachmentId?: string;
+    comment?: string;
+    after?: string;
+};
+type EventShipperOptions = {
+    writeKey?: string;
+    endpoint?: string;
+    enabled?: boolean;
+    debug: boolean;
+    partialFlushMs?: number;
+    sdkName?: string;
+    libraryName?: string;
+    libraryVersion?: string;
+    defaultEventName?: string;
+};
+declare class EventShipper {
+    private baseUrl;
+    private writeKey?;
+    private enabled;
+    private debug;
+    private partialFlushMs;
+    private sdkName;
+    private prefix;
+    private defaultEventName;
+    private context;
+    private buffers;
+    private sticky;
+    private timers;
+    private inFlight;
+    constructor(opts: EventShipperOptions);
+    isDebugEnabled(): boolean;
+    private authHeaders;
+    patch(eventId: string, patch: Patch): Promise<void>;
+    finish(eventId: string, patch: {
+        output?: string;
+        model?: string;
+        properties?: Record<string, unknown>;
+        userId?: string;
+    }): Promise<void>;
+    flush(): Promise<void>;
+    shutdown(): Promise<void>;
+    trackSignal(signal: SignalInput): Promise<void>;
+    identify(users: IdentifyInput | IdentifyInput[]): Promise<void>;
+    private flushOne;
+}
+
+type InternalSpan = {
+    ids: SpanIds;
+    name: string;
+    startTimeUnixNano: string;
+    endTimeUnixNano?: string;
+    attributes: Array<OtlpKeyValue | undefined>;
+};
+type TraceShipperOptions = {
+    writeKey?: string;
+    endpoint?: string;
+    enabled?: boolean;
+    debug: boolean;
+    debugSpans?: boolean;
+    flushIntervalMs?: number;
+    maxBatchSize?: number;
+    maxQueueSize?: number;
+    sdkName?: string;
+    serviceName?: string;
+    serviceVersion?: string;
+};
+declare class TraceShipper {
+    private baseUrl;
+    private writeKey?;
+    private enabled;
+    private debug;
+    private debugSpans;
+    private sdkName;
+    private prefix;
+    private serviceName;
+    private serviceVersion;
+    private flushIntervalMs;
+    private maxBatchSize;
+    private maxQueueSize;
+    private queue;
+    private timer;
+    private inFlight;
+    /** URL of the local debugger (from RAINDROP_LOCAL_DEBUGGER env var). */
+    private localDebuggerUrl;
+    constructor(opts: TraceShipperOptions);
+    isDebugEnabled(): boolean;
+    private authHeaders;
+    startSpan(args: {
+        name: string;
+        parent?: {
+            traceIdB64: string;
+            spanIdB64: string;
+        };
+        eventId: string;
+        operationId?: string;
+        attributes?: Array<OtlpKeyValue | undefined>;
+        startTimeUnixNano?: string;
+    }): InternalSpan;
+    endSpan(span: InternalSpan, extra?: {
+        attributes?: InternalSpan["attributes"];
+        error?: unknown;
+        status?: OtlpSpanStatus;
+        endTimeUnixNano?: string;
+    }): void;
+    createSpan(args: {
+        name: string;
+        parent?: {
+            traceIdB64: string;
+            spanIdB64: string;
+        };
+        eventId: string;
+        startTimeUnixNano: string;
+        endTimeUnixNano: string;
+        attributes?: Array<OtlpKeyValue | undefined>;
+        status?: OtlpSpanStatus;
+    }): void;
+    enqueue(span: OtlpSpan): void;
+    flush(): Promise<void>;
+    shutdown(): Promise<void>;
+}
+
+type ParentSpanContext = {
+    traceIdB64: string;
+    spanIdB64: string;
+    eventId: string;
+};
+interface ContextSpan {
+    readonly traceIdB64: string;
+    readonly spanIdB64: string;
+    readonly eventId: string;
+    log?(data: Record<string, unknown>): void;
+}
+interface AsyncLocalStorageLike<T> {
+    getStore(): T | undefined;
+    run<R>(store: T, callback: () => R): R;
+    enterWith?(store: T): void;
+}
+declare abstract class ContextManager {
+    abstract getParentSpanIds(): ParentSpanContext | undefined;
+    abstract runInContext<R>(span: ContextSpan, callback: () => R): R;
+    abstract getCurrentSpan(): ContextSpan | undefined;
+    abstract isReady(): boolean;
+}
+declare global {
+    var RAINDROP_CONTEXT_MANAGER: (new () => ContextManager) | undefined;
+    var RAINDROP_ASYNC_LOCAL_STORAGE: (new <T>() => AsyncLocalStorageLike<T>) | undefined;
+}
+
+interface RaindropDeepAgentsHandlerOptions {
+    eventShipper: EventShipper;
+    traceShipper: TraceShipper;
+    userId?: string;
+    convoId?: string;
+    traceChains?: boolean;
+}
+declare class RaindropDeepAgentsHandler extends BaseCallbackHandler {
+    name: string;
+    private eventShipper;
+    private traceShipper;
+    private userId?;
+    private convoId?;
+    private traceChains;
+    private spans;
+    private rootRunIds;
+    private eventIds;
+    private forwardedSpans;
+    /**
+     * event_id of the most recently finalized ROOT event.  Exposed via
+     * ``RaindropDeepAgentsClient.lastEventId`` so callers can attach a
+     * follow-up signal to the agent invocation they just ran without
+     * polling the dashboard.  Persists across runs; not reset on cleanup.
+     */
+    _lastRootEventId: string | undefined;
+    constructor(opts: RaindropDeepAgentsHandlerOptions);
+    private getEventId;
+    private getParent;
+    private cleanup;
+    private finalizeEventIfRoot;
+    handleLLMStart(llm: Serialized, prompts: string[], runId: string, parentRunId?: string, _extraParams?: Record<string, unknown>, tags?: string[], metadata?: Record<string, unknown>, runName?: string): Promise<void>;
+    handleChatModelStart(llm: Serialized, messages: BaseMessage[][], runId: string, parentRunId?: string, _extraParams?: Record<string, unknown>, tags?: string[], metadata?: Record<string, unknown>, runName?: string): Promise<void>;
+    handleLLMEnd(output: LLMResult, runId: string, _parentRunId?: string): Promise<void>;
+    handleLLMError(err: unknown, runId: string): Promise<void>;
+    handleChainStart(chain: Serialized, _inputs: ChainValues, runId: string, parentRunId?: string, _tags?: string[], _metadata?: Record<string, unknown>, _runType?: string, runName?: string): Promise<void>;
+    handleChainEnd(_outputs: ChainValues, runId: string): Promise<void>;
+    handleChainError(err: unknown, runId: string): Promise<void>;
+    handleToolStart(tool: Serialized, input: string, runId: string, parentRunId?: string, _tags?: string[], _metadata?: Record<string, unknown>, runName?: string): Promise<void>;
+    handleToolEnd(output: unknown, runId: string): Promise<void>;
+    handleToolError(err: unknown, runId: string): Promise<void>;
+    handleAgentAction(action: AgentAction, runId: string, parentRunId?: string): Promise<void>;
+    handleAgentEnd(_action: AgentFinish, runId: string): Promise<void>;
+}
+
+interface DeepAgentsOptions {
+    /**
+     * API write key. If omitted, telemetry shipping is disabled but the handler
+     * is still available for a consistent integration surface.
+     */
+    writeKey?: string;
+    endpoint?: string;
+    debug?: boolean;
+    userId?: string;
+    convoId?: string;
+    traceChains?: boolean;
+}
+type RaindropDeepAgentsClient = {
+    /**
+     * LangChain callback handler to pass to Deep Agents via `config.callbacks`.
+     *
+     * @example
+     * ```typescript
+     * const raindrop = createRaindropDeepAgents({ writeKey: "..." });
+     * const agent = createDeepAgent({ tools: [...] });
+     * const result = await agent.invoke(
+     *   { messages: [{ role: "user", content: "Hello" }] },
+     *   { callbacks: [raindrop.handler] },
+     * );
+     * await raindrop.shutdown();
+     * ```
+     */
+    handler: RaindropDeepAgentsHandler;
+    /**
+     * `event_id` of the most recently finalized ROOT event, or `undefined`
+     * if no root event has landed yet.  Use this to attach a feedback
+     * `signals.track()` to the agent invocation you just ran, without
+     * having to poll the dashboard for the real id.
+     *
+     * @example
+     * ```typescript
+     * const raindrop = createRaindropDeepAgents({ writeKey: "..." });
+     * await agent.invoke({ messages: [...] }, { callbacks: [raindrop.handler] });
+     * if (raindrop.lastEventId) {
+     *   await raindrop.signals.track({
+     *     eventId: raindrop.lastEventId,
+     *     name: "thumbs_up",
+     *     type: "feedback",
+     *     sentiment: "POSITIVE",
+     *   });
+     * }
+     * ```
+     */
+    readonly lastEventId: string | undefined;
+    events: {
+        patch(eventId: string, patch: Patch): Promise<void>;
+        finish(eventId: string, patch: {
+            output?: string;
+            model?: string;
+            properties?: Record<string, unknown>;
+        }): Promise<void>;
+        addAttachments(eventId: string, attachments: Attachment[]): Promise<void>;
+        setProperties(eventId: string, properties: Record<string, unknown>): Promise<void>;
+    };
+    users: {
+        identify(users: IdentifyInput | IdentifyInput[]): Promise<void>;
+    };
+    signals: {
+        track(signal: SignalInput): Promise<void>;
+    };
+    flush(): Promise<void>;
+    shutdown(): Promise<void>;
+};
+/**
+ * Create a Raindrop client for instrumenting LangChain Deep Agents.
+ *
+ * Deep Agents is built on LangChain and LangGraph. This integration provides
+ * a callback handler that captures LLM calls, tool usage (write_todos,
+ * read_file, write_file, edit_file, task, execute, etc.), chains, and agent
+ * actions — shipping them to Raindrop for observability.
+ *
+ * @example
+ * ```typescript
+ * import { createDeepAgent } from "deepagents";
+ * import { createRaindropDeepAgents } from "@raindrop-ai/deep-agents";
+ *
+ * const raindrop = createRaindropDeepAgents({
+ *   writeKey: process.env.RAINDROP_API_KEY,
+ *   userId: "user-123",
+ *   convoId: "convo-456",
+ * });
+ *
+ * const agent = createDeepAgent({ tools: [myTool] });
+ * const result = await agent.invoke(
+ *   { messages: [{ role: "user", content: "Research LangGraph" }] },
+ *   { callbacks: [raindrop.handler] },
+ * );
+ *
+ * await raindrop.shutdown();
+ * ```
+ */
+declare function createRaindropDeepAgents(opts: DeepAgentsOptions): RaindropDeepAgentsClient;
+
+export { type DeepAgentsOptions, type RaindropDeepAgentsClient, RaindropDeepAgentsHandler, createRaindropDeepAgents };