graphlit-client 1.0.20250615001 → 1.0.20250615003

package/README.md

@@ -16,16 +16,25 @@ Graphlit is a cloud platform that handles the complex parts of building AI appli
 
 ## ✨ What's New in v1.1.0
 
-- **Real-time streaming** - Watch AI responses appear word-by-word
+- **Real-time streaming** - Watch AI responses appear word-by-word across 9 different providers
 - **Tool calling** - Let AI execute functions and retrieve data
-- **Better performance** - Native integration with OpenAI, Anthropic, and Google
+- **Extended provider support** - Native streaming integration with OpenAI, Anthropic, Google, Groq, Cerebras, Cohere, Mistral, AWS Bedrock, and Deepseek
+- **Better performance** - Optimized streaming with provider-specific SDKs
+- **Network resilience** - Automatic retry logic for transient failures
 
 ## 📋 Table of Contents
 
 - [Quick Start](#quick-start)
+- [Installation](#installation)
+- [Setting Up](#setting-up)
+- [Network Resilience](#network-resilience-new-in-v111)
+- [Streaming Provider Support](#streaming-provider-support)
 - [Basic Examples](#basic-examples)
 - [Common Use Cases](#common-use-cases)
+- [Advanced Agent Features](#advanced-agent-features)
+- [Advanced Workflows](#advanced-workflows)
 - [API Reference](#api-reference)
+- [Testing & Examples](#testing--examples)
 - [Support](#support)
 
 ## Quick Start
@@ -93,7 +102,7 @@ npm install @google/generative-ai
 # For Groq streaming (OpenAI-compatible)
 npm install groq-sdk
 
-# For Cerebras streaming (uses OpenAI SDK with custom base URL)
+# For Cerebras streaming (OpenAI-compatible)
 npm install openai
 
 # For Cohere streaming
@@ -105,7 +114,7 @@ npm install @mistralai/mistralai
 # For AWS Bedrock streaming (Claude models)
 npm install @aws-sdk/client-bedrock-runtime
 
-# For Deepseek streaming (uses OpenAI SDK with custom base URL)
+# For Deepseek streaming (OpenAI-compatible)
 npm install openai
 ```
 
@@ -118,22 +127,22 @@ GRAPHLIT_ORGANIZATION_ID=your_org_id
 GRAPHLIT_ENVIRONMENT_ID=your_env_id
 GRAPHLIT_JWT_SECRET=your_secret
 
-# Optional: For streaming
+# Optional: For streaming with specific providers
 OPENAI_API_KEY=your_key
 ANTHROPIC_API_KEY=your_key
 GOOGLE_API_KEY=your_key
-GROQ_API_KEY=your_key
-CEREBRAS_API_KEY=your_key
-COHERE_API_KEY=your_key
-MISTRAL_API_KEY=your_key
 
-# For AWS Bedrock (requires AWS credentials)
+# Additional streaming providers
+GROQ_API_KEY=your_key          # For Groq models (Llama, Mixtral)
+CEREBRAS_API_KEY=your_key      # For Cerebras models  
+COHERE_API_KEY=your_key        # For Cohere Command models
+MISTRAL_API_KEY=your_key       # For Mistral models
+DEEPSEEK_API_KEY=your_key      # For Deepseek models
+
+# For AWS Bedrock streaming (requires AWS credentials)
 AWS_REGION=us-east-2
 AWS_ACCESS_KEY_ID=your_key
 AWS_SECRET_ACCESS_KEY=your_secret
-
-# For Deepseek streaming
-DEEPSEEK_API_KEY=your_key
 ```
 
 ## Network Resilience (New in v1.1.1)
@@ -206,6 +215,69 @@ const client = new Graphlit({
 });
 ```
 
+## Streaming Provider Support
+
+The Graphlit SDK supports real-time streaming responses from 9 different LLM providers. Each provider requires its specific SDK and API key:
+
+### Supported Providers
+
+| Provider | Models | SDK Required | API Key |
+|----------|--------|--------------|---------|
+| **OpenAI** | GPT-4, GPT-4o, GPT-4.1, O1, O3, O4 | `openai` | `OPENAI_API_KEY` |
+| **Anthropic** | Claude 3, Claude 3.5, Claude 3.7, Claude 4 | `@anthropic-ai/sdk` | `ANTHROPIC_API_KEY` |
+| **Google** | Gemini 1.5, Gemini 2.0, Gemini 2.5 | `@google/generative-ai` | `GOOGLE_API_KEY` |
+| **Groq** | Llama 4, Llama 3.3, Mixtral, Deepseek R1 | `groq-sdk` | `GROQ_API_KEY` |
+| **Cerebras** | Llama 3.3, Llama 3.1 | `openai` | `CEREBRAS_API_KEY` |
+| **Cohere** | Command R+, Command R, Command R7B, Command A | `cohere-ai` | `COHERE_API_KEY` |
+| **Mistral** | Mistral Large, Medium, Small, Nemo, Pixtral | `@mistralai/mistralai` | `MISTRAL_API_KEY` |
+| **AWS Bedrock** | Nova Premier/Pro, Claude 3.7, Llama 4 | `@aws-sdk/client-bedrock-runtime` | AWS credentials |
+| **Deepseek** | Deepseek Chat, Deepseek Reasoner | `openai` | `DEEPSEEK_API_KEY` |
+
+### Setting Up Streaming
+
+Each provider requires both the SDK installation and proper client setup:
+
+```typescript
+import { Graphlit } from "graphlit-client";
+
+const client = new Graphlit();
+
+// Example: Set up multiple streaming providers
+if (process.env.OPENAI_API_KEY) {
+  const { OpenAI } = await import("openai");
+  client.setOpenAIClient(new OpenAI());
+}
+
+if (process.env.COHERE_API_KEY) {
+  const { CohereClient } = await import("cohere-ai");
+  client.setCohereClient(new CohereClient({ token: process.env.COHERE_API_KEY }));
+}
+
+if (process.env.GROQ_API_KEY) {
+  const { Groq } = await import("groq-sdk");
+  client.setGroqClient(new Groq({ apiKey: process.env.GROQ_API_KEY }));
+}
+
+// Then create specifications for any provider
+const spec = await client.createSpecification({
+  name: "Multi-Provider Assistant",
+  type: Types.SpecificationTypes.Completion,
+  serviceType: Types.ModelServiceTypes.Cohere, // or any supported provider
+  cohere: { 
+    model: Types.CohereModels.CommandRPlus,
+    temperature: 0.7 
+  },
+});
+```
+
+### Provider-Specific Notes
+
+- **OpenAI-Compatible**: Groq, Cerebras, and Deepseek use OpenAI-compatible APIs
+- **AWS Bedrock**: Requires AWS credentials and uses the Converse API for streaming
+- **Cohere**: Supports both chat and tool calling with Command models
+- **Google**: Includes advanced multimodal capabilities with Gemini models
+- **Mistral**: Supports both text and vision models (Pixtral)
+
 ## Basic Examples
 
 ### 1. Chat with AI
@@ -323,7 +395,73 @@ const response = await client.promptAgent(
 console.log(response.message);
 ```
 
-### 4. Tool Calling
+### 4. Multiple Provider Streaming
+
+Compare responses from different LLM providers:
+
+```typescript
+import { Graphlit, Types } from "graphlit-client";
+
+const client = new Graphlit();
+
+// Set up multiple providers
+if (process.env.OPENAI_API_KEY) {
+  const { OpenAI } = await import("openai");
+  client.setOpenAIClient(new OpenAI());
+}
+
+if (process.env.COHERE_API_KEY) {
+  const { CohereClient } = await import("cohere-ai");
+  client.setCohereClient(new CohereClient({ token: process.env.COHERE_API_KEY }));
+}
+
+if (process.env.GROQ_API_KEY) {
+  const { Groq } = await import("groq-sdk");
+  client.setGroqClient(new Groq({ apiKey: process.env.GROQ_API_KEY }));
+}
+
+// Create specifications for different providers
+const providers = [
+  {
+    name: "OpenAI GPT-4o",
+    serviceType: Types.ModelServiceTypes.OpenAi,
+    openAI: { model: Types.OpenAiModels.Gpt4O_128K }
+  },
+  {
+    name: "Cohere Command R+",
+    serviceType: Types.ModelServiceTypes.Cohere,
+    cohere: { model: Types.CohereModels.CommandRPlus }
+  },
+  {
+    name: "Groq Llama",
+    serviceType: Types.ModelServiceTypes.Groq,
+    groq: { model: Types.GroqModels.Llama_3_3_70B }
+  }
+];
+
+// Compare responses
+for (const provider of providers) {
+  console.log(`\n🤖 ${provider.name}:`);
+  
+  const spec = await client.createSpecification({
+    ...provider,
+    type: Types.SpecificationTypes.Completion,
+  });
+
+  await client.streamAgent(
+    "Explain quantum computing in simple terms",
+    (event) => {
+      if (event.type === "message_update") {
+        process.stdout.write(event.message.message);
+      }
+    },
+    undefined,
+    { id: spec.createSpecification.id }
+  );
+}
+```
+
+### 5. Tool Calling
 
 Let AI call functions to get real-time data:
 

package/dist/model-mapping.js

@@ -106,8 +106,8 @@ const COHERE_MODEL_MAP = {
     [Types.CohereModels.CommandR_202403]: "command-r-03-2024",
     [Types.CohereModels.CommandR_202408]: "command-r-08-2024",
     [Types.CohereModels.CommandR7B_202412]: "command-r7b-12-2024",
-    [Types.CohereModels.CommandA]: "command-light",
-    [Types.CohereModels.CommandA_202503]: "command-light-nightly",
+    [Types.CohereModels.CommandA]: "command-a-03-2025",
+    [Types.CohereModels.CommandA_202503]: "command-a-03-2025",
 };
 // Mistral model mappings
 const MISTRAL_MODEL_MAP = {

package/dist/streaming/llm-formatters.d.ts

@@ -81,8 +81,23 @@ export declare function formatMessagesForGoogle(messages: ConversationMessage[])
  * Cohere message format
  */
 export interface CohereMessage {
-    role: "USER" | "CHATBOT" | "SYSTEM";
-    message: string;
+    role: "user" | "assistant" | "system" | "tool";
+    content: string;
+    tool_calls?: Array<{
+        id: string;
+        name: string;
+        parameters: Record<string, any>;
+    }>;
+    tool_results?: Array<{
+        call: {
+            id: string;
+            name: string;
+            parameters: Record<string, any>;
+        };
+        outputs: Array<{
+            text: string;
+        }>;
+    }>;
 }
 /**
  * Format GraphQL conversation messages for Cohere SDK

package/dist/streaming/llm-formatters.js

@@ -273,26 +273,59 @@ export function formatMessagesForGoogle(messages) {
 export function formatMessagesForCohere(messages) {
     const formattedMessages = [];
     for (const message of messages) {
-        if (!message.role || !message.message?.trim())
+        if (!message.role)
             continue;
-        const trimmedMessage = message.message.trim();
+        // Allow messages with tool calls even if they have no text content
+        const hasContent = message.message?.trim();
+        const hasToolCalls = message.toolCalls && message.toolCalls.length > 0;
+        if (!hasContent && !hasToolCalls)
+            continue;
+        const trimmedMessage = message.message?.trim() || "";
         switch (message.role) {
             case ConversationRoleTypes.System:
                 formattedMessages.push({
-                    role: "SYSTEM",
-                    message: trimmedMessage,
+                    role: "system",
+                    content: trimmedMessage,
                 });
                 break;
             case ConversationRoleTypes.Assistant:
+                const assistantMessage = {
+                    role: "assistant",
+                    content: trimmedMessage,
+                };
+                // Add tool calls if present
+                if (message.toolCalls && message.toolCalls.length > 0) {
+                    assistantMessage.tool_calls = message.toolCalls
+                        .filter((tc) => tc !== null)
+                        .map((toolCall) => ({
+                        id: toolCall.id,
+                        name: toolCall.name,
+                        parameters: toolCall.arguments ? JSON.parse(toolCall.arguments) : {},
+                    }));
+                }
+                formattedMessages.push(assistantMessage);
+                break;
+            case ConversationRoleTypes.Tool:
+                // Cohere expects tool results as tool messages
                 formattedMessages.push({
-                    role: "CHATBOT",
-                    message: trimmedMessage,
+                    role: "tool",
+                    content: trimmedMessage,
+                    tool_results: [{
+                            call: {
+                                id: message.toolCallId || "",
+                                name: "", // Would need to be tracked from the tool call
+                                parameters: {},
+                            },
+                            outputs: [{
+                                    text: trimmedMessage,
+                                }],
+                        }],
                 });
                 break;
             default: // User messages
                 formattedMessages.push({
-                    role: "USER",
-                    message: trimmedMessage,
+                    role: "user",
+                    content: trimmedMessage,
                 });
                 break;
         }

package/dist/streaming/providers.js

@@ -27,6 +27,15 @@ function cleanSchemaForGoogle(schema) {
         if (key === "$schema" || key === "additionalProperties") {
             continue;
         }
+        // Handle format field for string types - Google only supports 'enum' and 'date-time'
+        if (key === "format" && typeof value === "string") {
+            // Only keep supported formats
+            if (value === "enum" || value === "date-time") {
+                cleaned[key] = value;
+            }
+            // Skip unsupported formats like "date", "time", "email", etc.
+            continue;
+        }
         // Recursively clean nested objects
         cleaned[key] = cleanSchemaForGoogle(value);
     }
@@ -1215,15 +1224,21 @@ onEvent, onComplete) {
         if (process.env.DEBUG_GRAPHLIT_SDK_STREAMING) {
             console.log(`🤖 [Cohere] Model Config: Service=Cohere | Model=${modelName} | Temperature=${specification.cohere?.temperature} | Tools=${tools?.length || 0} | Spec="${specification.name}"`);
         }
-        // Prepare the latest user message and chat history
-        const lastMessage = messages[messages.length - 1];
-        const chatHistory = messages.slice(0, -1);
+        if (process.env.DEBUG_GRAPHLIT_SDK_STREAMING) {
+            console.log(`🔍 [Cohere] Messages array length: ${messages.length}`);
+            console.log(`🔍 [Cohere] All messages:`, JSON.stringify(messages, null, 2));
+        }
+        if (messages.length === 0) {
+            throw new Error("No messages found for Cohere streaming");
+        }
         const streamConfig = {
             model: modelName,
-            message: lastMessage.message,
-            chatHistory: chatHistory,
-            temperature: specification.cohere?.temperature,
+            messages: messages, // All messages in chronological order
         };
+        // Only add temperature if it's defined
+        if (specification.cohere?.temperature !== undefined) {
+            streamConfig.temperature = specification.cohere.temperature;
+        }
         // Add tools if provided
         if (tools && tools.length > 0) {
             streamConfig.tools = tools.map((tool) => {
@@ -1231,41 +1246,71 @@ onEvent, onComplete) {
                     return {
                         name: tool.name,
                         description: tool.description,
-                        parameterDefinitions: {},
+                        parameter_definitions: {},
                     };
                 }
                 // Parse the JSON schema
                 const schema = JSON.parse(tool.schema);
                 // Convert JSON Schema to Cohere's expected format
-                const parameterDefinitions = {};
+                const parameter_definitions = {};
                 if (schema.properties) {
                     for (const [key, value] of Object.entries(schema.properties)) {
                         const prop = value;
-                        parameterDefinitions[key] = {
+                        const paramDef = {
                             type: prop.type || "string",
                             description: prop.description || "",
                             required: schema.required?.includes(key) || false,
                         };
                         // Add additional properties that Cohere might expect
                         if (prop.enum) {
-                            parameterDefinitions[key].options = prop.enum;
+                            paramDef.options = prop.enum;
                         }
                         if (prop.default !== undefined) {
-                            parameterDefinitions[key].default = prop.default;
+                            paramDef.default = prop.default;
                         }
                         if (prop.items) {
-                            parameterDefinitions[key].items = prop.items;
+                            paramDef.items = prop.items;
                         }
+                        parameter_definitions[key] = paramDef;
                     }
                 }
                 return {
                     name: tool.name,
                     description: tool.description,
-                    parameterDefinitions,
+                    parameter_definitions, // Use snake_case as expected by Cohere API
                 };
             });
         }
-        const stream = await cohereClient.chatStream(streamConfig);
+        if (process.env.DEBUG_GRAPHLIT_SDK_STREAMING) {
+            console.log(`🔍 [Cohere] Final stream config:`, JSON.stringify(streamConfig, null, 2));
+            console.log(`🔍 [Cohere] Cohere client methods available:`, Object.getOwnPropertyNames(cohereClient));
+            console.log(`🔍 [Cohere] Has chatStream method:`, typeof cohereClient.chatStream === 'function');
+            console.log(`🔍 [Cohere] Has chat property:`, !!cohereClient.chat);
+            if (cohereClient.chat) {
+                console.log(`🔍 [Cohere] Chat methods:`, Object.getOwnPropertyNames(cohereClient.chat));
+            }
+            console.log(`⏱️ [Cohere] Starting stream request at: ${new Date().toISOString()}`);
+        }
+        let stream;
+        try {
+            stream = await cohereClient.chatStream(streamConfig);
+        }
+        catch (streamError) {
+            if (process.env.DEBUG_GRAPHLIT_SDK_STREAMING) {
+                console.error(`❌ [Cohere] Stream creation failed:`, streamError);
+                if (streamError.response) {
+                    console.error(`❌ [Cohere] Stream response status: ${streamError.response.status}`);
+                    console.error(`❌ [Cohere] Stream response data:`, streamError.response.data);
+                }
+                if (streamError.status) {
+                    console.error(`❌ [Cohere] Direct status: ${streamError.status}`);
+                }
+                if (streamError.body) {
+                    console.error(`❌ [Cohere] Response body:`, streamError.body);
+                }
+            }
+            throw streamError;
+        }
         for await (const chunk of stream) {
             if (chunk.eventType === "text-generation") {
                 const text = chunk.text;
@@ -1313,6 +1358,18 @@ onEvent, onComplete) {
         onComplete(fullMessage, toolCalls);
     }
     catch (error) {
+        if (process.env.DEBUG_GRAPHLIT_SDK_STREAMING) {
+            console.error(`❌ [Cohere] Stream error:`, error);
+            if (error instanceof Error) {
+                console.error(`❌ [Cohere] Error message: ${error.message}`);
+                console.error(`❌ [Cohere] Error stack: ${error.stack}`);
+            }
+            // Log additional error details if available
+            if (error.response) {
+                console.error(`❌ [Cohere] Response status: ${error.response.status}`);
+                console.error(`❌ [Cohere] Response data:`, error.response.data);
+            }
+        }
         throw error;
     }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "graphlit-client",
-  "version": "1.0.20250615001",
+  "version": "1.0.20250615003",
   "description": "Graphlit API Client for TypeScript",
   "type": "module",
   "main": "./dist/client.js",