@memberjunction/ai-groq 4.0.0 → 4.2.0

package/README.md

@@ -0,0 +1,89 @@
+# @memberjunction/ai-groq
+
+MemberJunction AI provider for Groq's ultra-fast inference platform. This package extends the OpenAI provider to work with Groq's OpenAI-compatible API, providing access to LLM inference powered by Groq's custom Language Processing Units (LPUs) for industry-leading inference speed.
+
+## Architecture
+
+```mermaid
+graph TD
+    A["GroqLLM<br/>(Provider)"] -->|extends| B["OpenAILLM<br/>(@memberjunction/ai-openai)"]
+    B -->|extends| C["BaseLLM<br/>(@memberjunction/ai)"]
+    A -->|overrides base URL| D["Groq API<br/>(api.groq.com/openai/v1)"]
+    D -->|runs on| E["Groq LPU<br/>Inference Engine"]
+    C -->|registered via| F["@RegisterClass"]
+
+    style A fill:#7c5295,stroke:#563a6b,color:#fff
+    style B fill:#2d6a9f,stroke:#1a4971,color:#fff
+    style C fill:#2d6a9f,stroke:#1a4971,color:#fff
+    style D fill:#2d8659,stroke:#1a5c3a,color:#fff
+    style E fill:#b8762f,stroke:#8a5722,color:#fff
+    style F fill:#b8762f,stroke:#8a5722,color:#fff
+```
+
+## Features
+
+- **Ultra-Fast Inference**: Groq's custom LPU hardware delivers extremely low latency
+- **OpenAI Compatible**: Inherits all features from the OpenAI provider
+- **Streaming**: Full streaming support for real-time responses
+- **Thinking/Reasoning**: Thinking block extraction for reasoning models
+- **Multiple Models**: Access to Llama, Mixtral, Gemma, and other open models optimized for Groq
+
+## Installation
+
+```bash
+npm install @memberjunction/ai-groq
+```
+
+## Usage
+
+```typescript
+import { GroqLLM } from '@memberjunction/ai-groq';
+
+const llm = new GroqLLM('your-groq-api-key');
+
+const result = await llm.ChatCompletion({
+    model: 'llama-3.1-70b-versatile',
+    messages: [
+        { role: 'system', content: 'You are a helpful assistant.' },
+        { role: 'user', content: 'Explain neural networks.' }
+    ],
+    temperature: 0.7,
+    maxOutputTokens: 1000
+});
+
+if (result.success) {
+    console.log(result.data.choices[0].message.content);
+}
+```
+
+### Streaming
+
+```typescript
+const result = await llm.ChatCompletion({
+    model: 'llama-3.1-8b-instant',
+    messages: [{ role: 'user', content: 'Write a haiku about speed.' }],
+    streaming: true,
+    streamingCallbacks: {
+        OnContent: (content) => process.stdout.write(content),
+        OnComplete: () => console.log('\nDone!')
+    }
+});
+```
+
+## How It Works
+
+`GroqLLM` is a thin subclass of `OpenAILLM` that redirects API calls to Groq's endpoint at `https://api.groq.com/openai/v1`. Since Groq implements an OpenAI-compatible API, all chat, streaming, and parameter handling logic is inherited from the OpenAI provider.
+
+## Supported Parameters
+
+All parameters supported by the OpenAI provider are available, including `temperature`, `maxOutputTokens`, `topP`, `frequencyPenalty`, `presencePenalty`, `seed`, `stopSequences`, and `responseFormat`.
+
+## Class Registration
+
+Registered as `GroqLLM` via `@RegisterClass(BaseLLM, 'GroqLLM')`.
+
+## Dependencies
+
+- `@memberjunction/ai` - Core AI abstractions
+- `@memberjunction/ai-openai` - OpenAI provider (parent class)
+- `@memberjunction/global` - Class registration

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@memberjunction/ai-groq",
   "type": "module",
-  "version": "4.0.0",
+  "version": "4.2.0",
   "description": "MemberJunction Wrapper for Groq AI LPU inference engine",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -20,8 +20,8 @@
     "typescript": "^5.9.3"
   },
   "dependencies": {
-    "@memberjunction/ai": "4.0.0",
-    "@memberjunction/global": "4.0.0",
+    "@memberjunction/ai": "4.2.0",
+    "@memberjunction/global": "4.2.0",
     "groq-sdk": "^0.37.0"
   },
   "repository": {

package/readme.md

@@ -1,18 +1,32 @@
 # @memberjunction/ai-groq
 
-A comprehensive wrapper for Groq's LPU (Language Processing Unit) inference engine, providing high-performance AI model access within the MemberJunction framework.
+MemberJunction AI provider for Groq's ultra-fast inference platform. This package extends the OpenAI provider to work with Groq's OpenAI-compatible API, providing access to LLM inference powered by Groq's custom Language Processing Units (LPUs) for industry-leading inference speed.
+
+## Architecture
+
+```mermaid
+graph TD
+    A["GroqLLM<br/>(Provider)"] -->|extends| B["OpenAILLM<br/>(@memberjunction/ai-openai)"]
+    B -->|extends| C["BaseLLM<br/>(@memberjunction/ai)"]
+    A -->|overrides base URL| D["Groq API<br/>(api.groq.com/openai/v1)"]
+    D -->|runs on| E["Groq LPU<br/>Inference Engine"]
+    C -->|registered via| F["@RegisterClass"]
+
+    style A fill:#7c5295,stroke:#563a6b,color:#fff
+    style B fill:#2d6a9f,stroke:#1a4971,color:#fff
+    style C fill:#2d6a9f,stroke:#1a4971,color:#fff
+    style D fill:#2d8659,stroke:#1a5c3a,color:#fff
+    style E fill:#b8762f,stroke:#8a5722,color:#fff
+    style F fill:#b8762f,stroke:#8a5722,color:#fff
+```
 
 ## Features
 
-- **High-Performance Integration**: Connect to Groq's ultra-fast LPU inference API
-- **Standardized Interface**: Implements MemberJunction's BaseLLM abstract class
-- **Streaming Support**: Full support for streaming responses for real-time interactions
-- **Message Formatting**: Handles conversion between MemberJunction and Groq message formats
-- **Error Handling**: Robust error handling with detailed reporting
-- **Token Usage Tracking**: Track token consumption for monitoring
-- **Chat Completion**: Interactive chat completions with various LLMs hosted on Groq
-- **Model Support**: Compatible with Llama, Mixtral, Gemma, and other models hosted on Groq
-- **Response Format Control**: Support for JSON, text, and model-specific response formats
+- **Ultra-Fast Inference**: Groq's custom LPU hardware delivers extremely low latency
+- **OpenAI Compatible**: Inherits all features from the OpenAI provider
+- **Streaming**: Full streaming support for real-time responses
+- **Thinking/Reasoning**: Thinking block extraction for reasoning models
+- **Multiple Models**: Access to Llama, Mixtral, Gemma, and other open models optimized for Groq
 
 ## Installation
 
@@ -20,273 +34,56 @@ A comprehensive wrapper for Groq's LPU (Language Processing Unit) inference engi
 npm install @memberjunction/ai-groq
 ```
 
-## Requirements
-
-- Node.js 16+
-- A Groq API key
-- MemberJunction Core libraries
-
 ## Usage
 
-### Basic Setup
-
 ```typescript
 import { GroqLLM } from '@memberjunction/ai-groq';
 
-// Initialize with your Groq API key
-const groqLLM = new GroqLLM('your-groq-api-key');
-```
-
-### Chat Completion
-
-```typescript
-import { ChatParams } from '@memberjunction/ai';
-
-// Create chat parameters
-const chatParams: ChatParams = {
-  model: 'llama2-70b-4096',  // or other models like 'mixtral-8x7b-32768'
-  messages: [
-    { role: 'system', content: 'You are a helpful AI assistant.' },
-    { role: 'user', content: 'Explain how LPUs differ from traditional GPUs for AI inference.' }
-  ],
-  temperature: 0.7,
-  maxOutputTokens: 1000
-};
-
-// Get a response
-try {
-  const response = await groqLLM.ChatCompletion(chatParams);
-  if (response.success) {
-    console.log('Response:', response.data.choices[0].message.content);
-    console.log('Token Usage:', response.data.usage);
-    console.log('Time Elapsed (ms):', response.timeElapsed);
-  } else {
-    console.error('Error:', response.errorMessage);
-  }
-} catch (error) {
-  console.error('Exception:', error);
-}
-```
-
-### Streaming Responses
-
-```typescript
-import { ChatParams, ChatResult } from '@memberjunction/ai';
-
-// Enable streaming in the chat parameters
-const streamingParams: ChatParams = {
-  model: 'llama3-70b-8192',
-  messages: [
-    { role: 'user', content: 'Write a short story about AI.' }
-  ],
-  stream: true,
-  onStream: (content: string) => {
-    // Handle each chunk of streamed content
-    process.stdout.write(content);
-  },
-  maxOutputTokens: 2000
-};
-
-// The response will stream to the onStream callback
-const response = await groqLLM.ChatCompletion(streamingParams);
-console.log('\n\nFinal response:', response.data.choices[0].message.content);
-```
-
-### Response Format Control
-
-```typescript
-// Request JSON formatted response
-const jsonParams: ChatParams = {
-  model: 'mixtral-8x7b-32768',
-  messages: [
-    { role: 'system', content: 'You are a helpful assistant that responds in JSON format.' },
-    { role: 'user', content: 'List 3 benefits of using Groq in JSON format with keys: benefit, description' }
-  ],
-  responseFormat: 'JSON',
-  maxOutputTokens: 1000
-};
+const llm = new GroqLLM('your-groq-api-key');
 
-const jsonResponse = await groqLLM.ChatCompletion(jsonParams);
-const benefits = JSON.parse(jsonResponse.data.choices[0].message.content);
-```
-
-### Direct Access to Groq Client
-
-```typescript
-// Access the underlying Groq client for advanced usage
-const groqClient = groqLLM.GroqClient;
-// or use the alias
-const client = groqLLM.client;
-
-// Use the client directly if needed
-const customResponse = await groqClient.chat.completions.create({
-  model: 'mixtral-8x7b-32768',
-  messages: [{ role: 'user', content: 'Hello!' }],
-  max_tokens: 500
+const result = await llm.ChatCompletion({
+    model: 'llama-3.1-70b-versatile',
+    messages: [
+        { role: 'system', content: 'You are a helpful assistant.' },
+        { role: 'user', content: 'Explain neural networks.' }
+    ],
+    temperature: 0.7,
+    maxOutputTokens: 1000
 });
-```
-
-## Supported Models
-
-Groq provides access to various open models with optimized inference:
-
-- **Llama Models**:
-  - `llama3-70b-8192` - Llama 3 70B with 8K context
-  - `llama3-8b-8192` - Llama 3 8B with 8K context
-  - `llama2-70b-4096` - Llama 2 70B with 4K context
-  
-- **Mixtral Models**:
-  - `mixtral-8x7b-32768` - Mixtral 8x7B with 32K context
-  
-- **Gemma Models**:
-  - `gemma-7b-it` - Gemma 7B Instruct
-  - `gemma2-9b-it` - Gemma 2 9B Instruct
-
-Check the [Groq documentation](https://console.groq.com/docs/models) for the latest list of supported models and their capabilities.
-
-## API Reference
-
-### GroqLLM Class
-
-A class that extends BaseLLM to provide Groq-specific functionality.
-
-#### Constructor
-
-```typescript
-new GroqLLM(apiKey: string)
-```
-
-#### Properties
-
-- `GroqClient`: (read-only) Returns the underlying Groq client instance
-- `client`: (read-only) Alias for GroqClient
-- `SupportsStreaming`: (read-only) Returns `true` - Groq supports streaming responses
-
-#### Methods
 
-##### ChatCompletion
-```typescript
-ChatCompletion(params: ChatParams): Promise<ChatResult>
-```
-Perform a chat completion with support for both streaming and non-streaming responses.
-
-##### SummarizeText
-```typescript
-SummarizeText(params: SummarizeParams): Promise<SummarizeResult>
-```
-*Note: Not yet implemented*
-
-##### ClassifyText
-```typescript
-ClassifyText(params: ClassifyParams): Promise<ClassifyResult>
-```
-*Note: Not yet implemented*
-
-## Performance Considerations
-
-Groq is known for its extremely fast inference times:
-
-- Response generation is typically 5-10x faster than traditional GPU-based inference
-- Lower latency means better interactive experiences
-- Benchmark different models to find the best performance/quality balance for your use case
-
-## Error Handling
-
-The wrapper provides detailed error information:
-
-```typescript
-try {
-  const response = await groqLLM.ChatCompletion(params);
-  if (!response.success) {
-    console.error('Error:', response.errorMessage);
-    console.error('Status:', response.statusText);
-    console.error('Time Elapsed:', response.timeElapsed, 'ms');
-  }
-} catch (error) {
-  console.error('Exception occurred:', error);
+if (result.success) {
+    console.log(result.data.choices[0].message.content);
 }
 ```
 
-## Integration with MemberJunction
-
-This package seamlessly integrates with the MemberJunction AI framework:
+### Streaming
 
 ```typescript
-import { RegisterClass } from '@memberjunction/global';
-import { BaseLLM } from '@memberjunction/ai';
-import { GroqLLM } from '@memberjunction/ai-groq';
-
-// The GroqLLM class is automatically registered with the MemberJunction class factory
-// You can retrieve it using the class factory pattern
-const llm = RegisterClass.GetRegisteredClass(BaseLLM, 'GroqLLM', 'your-api-key');
-```
-
-## Advanced Features
-
-### Effort Level Support
-
-For models that support reasoning effort levels (experimental):
-
-```typescript
-const params: ChatParams = {
-  model: 'llama3-70b-8192',
-  messages: [{ role: 'user', content: 'Solve this complex problem...' }],
-  effortLevel: 'high', // Experimental feature
-  maxOutputTokens: 2000
-};
-```
-
-### Handling Groq-Specific Requirements
-
-The wrapper automatically handles Groq's requirement that the last message must be from a user. If your message chain ends with an assistant message, the wrapper will automatically append a dummy user message to satisfy this requirement.
-
-## Dependencies
-
-- `groq-sdk` (0.21.0): Official Groq SDK
-- `@memberjunction/ai` (2.43.0): MemberJunction AI core framework
-- `@memberjunction/global` (2.43.0): MemberJunction global utilities
-
-## Development
-
-### Building
-
-```bash
-npm run build
-```
-
-### Development Mode
-
-```bash
-npm start
+const result = await llm.ChatCompletion({
+    model: 'llama-3.1-8b-instant',
+    messages: [{ role: 'user', content: 'Write a haiku about speed.' }],
+    streaming: true,
+    streamingCallbacks: {
+        OnContent: (content) => process.stdout.write(content),
+        OnComplete: () => console.log('\nDone!')
+    }
+});
 ```
 
-## Contributing
+## How It Works
 
-When contributing to this package:
-
-1. Follow the MemberJunction coding standards
-2. Ensure all TypeScript types are properly defined
-3. Update tests when adding new features
-4. Document any Groq-specific behaviors or limitations
+`GroqLLM` is a thin subclass of `OpenAILLM` that redirects API calls to Groq's endpoint at `https://api.groq.com/openai/v1`. Since Groq implements an OpenAI-compatible API, all chat, streaming, and parameter handling logic is inherited from the OpenAI provider.
 
 ## Supported Parameters
 
-The Groq provider supports the following LLM parameters:
+All parameters supported by the OpenAI provider are available, including `temperature`, `maxOutputTokens`, `topP`, `frequencyPenalty`, `presencePenalty`, `seed`, `stopSequences`, and `responseFormat`.
 
-**Supported:**
-- `temperature` - Controls randomness in the output (0.0-2.0)
-- `maxOutputTokens` - Maximum number of tokens to generate
-- `topP` - Nucleus sampling threshold (0.0-1.0)
-- `seed` - For deterministic outputs
-- `stopSequences` - Array of sequences where the API will stop generating
-- `frequencyPenalty` - Reduces repetition of token sequences (-2.0 to 2.0)
-- `presencePenalty` - Reduces repetition of specific tokens (-2.0 to 2.0)
-- `responseFormat` - Output format (Text, JSON, etc.)
+## Class Registration
 
-**Not Supported:**
-- `topK` - Not available in Groq API
-- `minP` - Not available in Groq API
+Registered as `GroqLLM` via `@RegisterClass(BaseLLM, 'GroqLLM')`.
 
-## License
+## Dependencies
 
-ISC
+- `@memberjunction/ai` - Core AI abstractions
+- `@memberjunction/ai-openai` - OpenAI provider (parent class)
+- `@memberjunction/global` - Class registration