observa-sdk 0.0.7 → 0.0.9

package/README.md

@@ -41,18 +41,58 @@ const observa = init({
 
 ## Quick Start
 
-### JWT-based API Key (Recommended)
+### Auto-Capture with OpenAI (Recommended)
 
-After signing up, you'll receive a JWT-formatted API key that automatically encodes your tenant and project context:
+The easiest way to track LLM calls is using the `observeOpenAI()` wrapper - it automatically captures 90%+ of your LLM interactions:
 
 ```typescript
 import { init } from "observa-sdk";
+import OpenAI from "openai";
 
-// Initialize with JWT API key from signup (automatically extracts tenant/project context)
+// Initialize Observa
 const observa = init({
   apiKey: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", // Your API key from signup
 });
 
+// Initialize OpenAI client
+const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+
+// Wrap with Observa (automatic tracing)
+const wrappedOpenAI = observa.observeOpenAI(openai, {
+  name: 'my-app',
+  userId: 'user_123',
+  redact: (data) => {
+    // Optional: Scrub sensitive data before sending to Observa
+    if (data?.messages) {
+      return { ...data, messages: '[REDACTED]' };
+    }
+    return data;
+  }
+});
+
+// Use wrapped client - automatically tracked!
+const response = await wrappedOpenAI.chat.completions.create({
+  model: 'gpt-4',
+  messages: [{ role: 'user', content: 'Hello!' }],
+});
+
+// Streaming also works automatically
+const stream = await wrappedOpenAI.chat.completions.create({
+  model: 'gpt-4',
+  messages: [{ role: 'user', content: 'Hello!' }],
+  stream: true,
+});
+
+for await (const chunk of stream) {
+  process.stdout.write(chunk.choices[0]?.delta?.content || '');
+}
+```
+
+### Legacy Manual Tracking
+
+For more control, you can still use the manual `track()` method:
+
+```typescript
 // Track AI interactions with simple wrapping
 const response = await observa.track({ query: "What is the weather?" }, () =>
   fetch("https://api.openai.com/v1/chat/completions", {
@@ -67,18 +107,26 @@ const response = await observa.track({ query: "What is the weather?" }, () =>
 );
 ```
 
-### Legacy API Key Format
+### Manual Tracking (Advanced)
+
+For more control over what gets tracked, use the manual tracking methods:
 
 ```typescript
-// For backward compatibility, you can still provide tenantId/projectId explicitly
-const observa = init({
-  apiKey: "your-api-key",
-  tenantId: "acme_corp",
-  projectId: "prod_app",
-  environment: "prod", // optional, defaults to "dev"
+// Use trackLLMCall for fine-grained control
+const spanId = observa.trackLLMCall({
+  model: 'gpt-4',
+  input: 'Hello!',
+  output: 'Hi there!',
+  inputTokens: 10,
+  outputTokens: 5,
+  latencyMs: 1200,
+  operationName: 'chat',
+  providerName: 'openai',
 });
 ```
 
+See the [API Reference](#api-reference) section for all available methods.
+
 ## Multi-Tenant Architecture
 
 Observa SDK uses a **multi-tenant shared runtime architecture** for optimal cost, scalability, and operational simplicity.
@@ -141,6 +189,9 @@ interface ObservaInitConfig {
   projectId?: string;
   environment?: "dev" | "prod";
 
+  // Observa backend URL (optional, defaults to https://api.observa.ai)
+  apiUrl?: string;
+
   // SDK behavior
   mode?: "development" | "production";
   sampleRate?: number; // 0..1, default: 1.0
@@ -153,6 +204,7 @@ interface ObservaInitConfig {
 - **apiKey**: Your Observa API key (JWT format recommended)
 - **tenantId** / **projectId**: Required only for legacy API keys
 - **environment**: `"dev"` or `"prod"` (defaults to `"dev"`)
+- **apiUrl**: Observa backend URL (optional, defaults to `https://api.observa.ai`)
 - **mode**: SDK mode - `"development"` logs traces to console, `"production"` sends to Observa
 - **sampleRate**: Fraction of traces to record (0.0 to 1.0)
 - **maxResponseChars**: Maximum response size to capture (prevents huge payloads)
@@ -163,9 +215,246 @@ interface ObservaInitConfig {
 
 Initialize the Observa SDK instance.
 
+**Example:**
+```typescript
+import { init } from "observa-sdk";
+
+const observa = init({
+  apiKey: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...", // Your JWT API key
+  apiUrl: "https://api.observa.ai", // Optional, defaults to https://api.observa.ai
+  environment: "prod", // Optional, defaults to "dev"
+  mode: "production", // Optional, "development" or "production"
+  sampleRate: 1.0, // Optional, 0.0 to 1.0, default: 1.0
+  maxResponseChars: 50000, // Optional, default: 50000
+});
+```
+
+### `observa.observeOpenAI(client, options?)`
+
+Wrap an OpenAI client with automatic tracing. This is the **recommended** way to track LLM calls.
+
+**Parameters:**
+- `client` (required): OpenAI client instance
+- `options` (optional):
+  - `name` (optional): Application/service name
+  - `tags` (optional): Array of tags
+  - `userId` (optional): User identifier
+  - `sessionId` (optional): Session identifier
+  - `redact` (optional): Function to sanitize data before sending to Observa
+
+**Returns**: Wrapped OpenAI client (use it exactly like the original client)
+
+**Example:**
+```typescript
+import OpenAI from 'openai';
+
+const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
+const wrapped = observa.observeOpenAI(openai, {
+  name: 'my-app',
+  userId: 'user_123',
+  redact: (data) => {
+    // Sanitize sensitive data
+    if (data?.messages) {
+      return { ...data, messages: '[REDACTED]' };
+    }
+    return data;
+  }
+});
+
+// Use wrapped client - automatically tracked!
+const response = await wrapped.chat.completions.create({
+  model: 'gpt-4',
+  messages: [{ role: 'user', content: 'Hello!' }],
+});
+```
+
+### `observa.observeAnthropic(client, options?)`
+
+Wrap an Anthropic client with automatic tracing. Same API as `observeOpenAI()`.
+
+**Example:**
+```typescript
+import Anthropic from '@anthropic-ai/sdk';
+
+const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
+const wrapped = observa.observeAnthropic(anthropic, {
+  name: 'my-app',
+  redact: (data) => ({ ...data, messages: '[REDACTED]' })
+});
+
+// Use wrapped client - automatically tracked!
+const response = await wrapped.messages.create({
+  model: 'claude-3-opus-20240229',
+  max_tokens: 1024,
+  messages: [{ role: 'user', content: 'Hello!' }],
+});
+```
+
+### `observa.startTrace(options)`
+
+Start a new trace for manual trace management. Returns the trace ID.
+
+**Parameters:**
+- `options.name` (optional): Trace name
+- `options.metadata` (optional): Custom metadata object
+- `options.conversationId` (optional): Conversation identifier
+- `options.sessionId` (optional): Session identifier
+- `options.userId` (optional): User identifier
+
+**Returns**: `string` - The trace ID
+
+**Example:**
+```typescript
+const traceId = observa.startTrace({
+  name: "RAG Query",
+  conversationId: "conv-123",
+  userId: "user-456",
+  metadata: { feature: "chat", version: "2.0" }
+});
+```
+
+### `observa.endTrace(options)`
+
+End the current trace and send all buffered events. Must be called after `startTrace()`.
+
+**Parameters:**
+- `options.outcome` (optional): `"success"` | `"error"` | `"timeout"` (default: `"success"`)
+
+**Returns**: `Promise<string>` - The trace ID
+
+**Example:**
+```typescript
+await observa.endTrace({ outcome: "success" });
+```
+
+### `observa.trackLLMCall(options)` ⭐ NEW - Full OTEL Support
+
+Track an LLM call with complete OTEL compliance. **This is the recommended method** for tracking LLM calls.
+
+**Parameters:**
+- `model` (required): Model name
+- `input`, `output`: Input/output text
+- `inputTokens`, `outputTokens`, `totalTokens`: Token counts
+- `latencyMs` (required): Latency in milliseconds
+- `operationName`: OTEL operation name ("chat", "text_completion", "generate_content")
+- `providerName`: Provider name ("openai", "anthropic", etc.) - auto-inferred from model if not provided
+- `responseModel`: Actual model used (vs requested)
+- `topK`, `topP`, `frequencyPenalty`, `presencePenalty`, `stopSequences`, `seed`: Sampling parameters
+- `inputCost`, `outputCost`: Structured cost tracking
+- `inputMessages`, `outputMessages`, `systemInstructions`: Structured message objects
+- `serverAddress`, `serverPort`: Server metadata
+- `conversationIdOtel`: OTEL conversation ID
+- And more... (see SDK_SOTA_IMPLEMENTATION.md for complete list)
+
+**Example:**
+```typescript
+const spanId = observa.trackLLMCall({
+  model: "gpt-4-turbo",
+  input: "What is AI?",
+  output: "AI is...",
+  inputTokens: 10,
+  outputTokens: 50,
+  latencyMs: 1200,
+  operationName: "chat",
+  providerName: "openai", // Auto-inferred if not provided
+  temperature: 0.7,
+  topP: 0.9,
+  inputCost: 0.00245,
+  outputCost: 0.01024
+});
+```
+
+### `observa.trackEmbedding(options)` ⭐ NEW
+
+Track an embedding operation with full OTEL support.
+
+**Example:**
+```typescript
+const spanId = observa.trackEmbedding({
+  model: "text-embedding-ada-002",
+  dimensionCount: 1536,
+  inputTokens: 10,
+  outputTokens: 1536,
+  latencyMs: 45,
+  cost: 0.0001
+});
+```
+
+### `observa.trackVectorDbOperation(options)` ⭐ NEW
+
+Track vector database operations (Pinecone, Weaviate, Qdrant, etc.).
+
+**Example:**
+```typescript
+const spanId = observa.trackVectorDbOperation({
+  operationType: "vector_search",
+  indexName: "documents",
+  vectorDimensions: 1536,
+  resultsCount: 10,
+  latencyMs: 30,
+  cost: 0.0005,
+  providerName: "pinecone"
+});
+```
+
+### `observa.trackCacheOperation(options)` ⭐ NEW
+
+Track cache hit/miss operations.
+
+**Example:**
+```typescript
+const spanId = observa.trackCacheOperation({
+  cacheBackend: "redis",
+  hitStatus: "hit",
+  latencyMs: 2,
+  savedCost: 0.01269
+});
+```
+
+### `observa.trackAgentCreate(options)` ⭐ NEW
+
+Track agent creation.
+
+**Example:**
+```typescript
+const spanId = observa.trackAgentCreate({
+  agentName: "Customer Support Agent",
+  toolsBound: ["web_search", "database_query"],
+  modelConfig: { model: "gpt-4-turbo", temperature: 0.7 }
+});
+```
+
+### `observa.trackToolCall(options)` - Enhanced
+
+Track a tool call with OTEL standardization.
+
+**New Parameters:**
+- `toolType`: "function" | "extension" | "datastore"
+- `toolDescription`: Tool description
+- `toolCallId`: Unique tool invocation ID
+- `errorType`, `errorCategory`: Structured error classification
+
+### `observa.trackRetrieval(options)` - Enhanced
+
+Track retrieval operations with vector metadata.
+
+**New Parameters:**
+- `embeddingModel`: Model used for embeddings
+- `embeddingDimensions`: Vector dimensions
+- `vectorMetric`: Similarity metric
+- `rerankScore`, `fusionMethod`, `qualityScore`: Quality metrics
+
+### `observa.trackError(options)` - Enhanced
+
+Track errors with structured classification.
+
+**New Parameters:**
+- `errorCategory`: Error category
+- `errorCode`: Error code
+
 ### `observa.track(event, action)`
 
-Track an AI interaction.
+Track an AI interaction (legacy method, still supported).
 
 **Parameters**:
 
@@ -201,6 +490,166 @@ const response = await observa.track(
 );
 ```
 
+### `observa.trackFeedback(options)`
+
+Track user feedback (likes, dislikes, ratings, corrections) for AI interactions.
+
+**Parameters**:
+
+- `options.type` (required): Feedback type - `"like"` | `"dislike"` | `"rating"` | `"correction"`
+- `options.rating` (optional): Rating value (1-5 scale, automatically clamped). Required for `"rating"` type.
+- `options.comment` (optional): User comment/feedback text
+- `options.outcome` (optional): Outcome classification - `"success"` | `"failure"` | `"partial"`
+- `options.conversationId` (optional): Conversation identifier for context
+- `options.sessionId` (optional): Session identifier for context
+- `options.userId` (optional): User identifier for context
+- `options.messageIndex` (optional): Position in conversation (1, 2, 3...)
+- `options.parentMessageId` (optional): For threaded conversations
+- `options.agentName` (optional): Agent/application name
+- `options.version` (optional): Application version
+- `options.route` (optional): API route/endpoint
+- `options.parentSpanId` (optional): Attach feedback to a specific span (e.g., LLM call span)
+- `options.spanId` (optional): Custom span ID for feedback (auto-generated if not provided)
+
+**Returns**: `string` - The span ID of the feedback event
+
+**Examples**:
+
+#### Basic Like/Dislike Feedback
+
+```typescript
+// User clicks "like" button after receiving AI response
+const feedbackSpanId = observa.trackFeedback({
+  type: "like",
+  outcome: "success",
+  conversationId: "conv-123",
+  userId: "user-456",
+});
+
+// User clicks "dislike" button
+observa.trackFeedback({
+  type: "dislike",
+  outcome: "failure",
+  comment: "The answer was incorrect",
+  conversationId: "conv-123",
+  userId: "user-456",
+});
+```
+
+#### Rating Feedback (1-5 Scale)
+
+```typescript
+// User provides a 5-star rating
+observa.trackFeedback({
+  type: "rating",
+  rating: 5, // Automatically clamped to 1-5 range
+  comment: "Excellent response!",
+  outcome: "success",
+  conversationId: "conv-123",
+  userId: "user-456",
+});
+
+// Rating is automatically validated (e.g., 10 becomes 5, -1 becomes 1)
+observa.trackFeedback({
+  type: "rating",
+  rating: 10, // Will be clamped to 5
+  conversationId: "conv-123",
+});
+```
+
+#### Correction Feedback
+
+```typescript
+// User provides correction/feedback
+observa.trackFeedback({
+  type: "correction",
+  comment: "The capital of France is Paris, not Lyon",
+  outcome: "partial",
+  conversationId: "conv-123",
+  userId: "user-456",
+});
+```
+
+#### Linking Feedback to Specific Spans
+
+```typescript
+// Start a trace and track LLM call
+const traceId = observa.startTrace({
+  conversationId: "conv-123",
+  userId: "user-456",
+});
+
+const llmSpanId = observa.trackLLMCall({
+  model: "gpt-4",
+  input: "What is the capital of France?",
+  output: "The capital of France is Paris.",
+  // ... other LLM call data
+});
+
+// Link feedback directly to the LLM call span
+observa.trackFeedback({
+  type: "like",
+  parentSpanId: llmSpanId, // Attach feedback to the specific LLM call
+  conversationId: "conv-123",
+  userId: "user-456",
+});
+```
+
+#### Full Context Feedback
+
+```typescript
+// Track feedback with complete context for analytics
+observa.trackFeedback({
+  type: "rating",
+  rating: 4,
+  comment: "Good answer, but could be more detailed",
+  outcome: "partial",
+  conversationId: "conv-123",
+  sessionId: "session-789",
+  userId: "user-456",
+  messageIndex: 3,
+  agentName: "customer-support-bot",
+  version: "v2.1.0",
+  route: "/api/chat",
+});
+```
+
+#### Feedback in Conversation Flow
+
+```typescript
+// Track feedback as part of a conversation
+const traceId = observa.startTrace({
+  conversationId: "conv-123",
+  sessionId: "session-789",
+  userId: "user-456",
+  messageIndex: 1,
+});
+
+// ... perform AI operations ...
+
+// User provides feedback after message 1
+observa.trackFeedback({
+  type: "like",
+  conversationId: "conv-123",
+  sessionId: "session-789",
+  userId: "user-456",
+  messageIndex: 1, // Link to specific message in conversation
+});
+
+await observa.endTrace();
+```
+
+**Best Practices**:
+
+1. **Always include context**: Provide `conversationId`, `userId`, and `sessionId` when available for better analytics
+2. **Link to spans**: Use `parentSpanId` to attach feedback to specific LLM calls or operations
+3. **Use appropriate types**: 
+   - `"like"` / `"dislike"` for binary feedback
+   - `"rating"` for 1-5 star ratings
+   - `"correction"` for user corrections or detailed feedback
+4. **Include comments**: Comments provide valuable qualitative feedback for improving AI responses
+5. **Set outcome**: Use `outcome` to classify feedback (`"success"` for positive, `"failure"` for negative, `"partial"` for mixed)
+
 ## Data Captured
 
 The SDK automatically captures: