bedrock-wrapper 2.4.1 → 2.4.3

package/CHANGELOG.md

@@ -1,6 +1,61 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## [2.4.3] - 2025-07-31 (Stop Sequences Fixes)
+### Fixed
+- **Critical Discovery**: Removed stop sequences support from Llama models
+  - AWS Bedrock does not support stop sequences for Llama models (confirmed via official AWS documentation)
+  - Llama models only support: `prompt`, `temperature`, `top_p`, `max_gen_len`, `images`
+  - This is an AWS Bedrock limitation, not a wrapper limitation
+- Fixed Nova model configuration conflicts that were causing stop sequence inconsistencies
+  - Removed conflicting empty `inferenceConfig: {}` from Nova model configurations
+- Improved error handling for empty responses when stop sequences trigger early
+
+### Updated
+- **Documentation corrections**
+  - Corrected stop sequences support claims (removed "all models support" language)
+  - Added accurate model-specific support matrix with sequence limits
+  - Added comprehensive stop sequences support table with AWS documentation references
+- **Model Support Matrix** now clearly documented:
+  - ✅ Claude models: Full support (up to 8,191 sequences) 
+  - ✅ Nova models: Full support (up to 4 sequences)
+  - ✅ Mistral models: Full support (up to 10 sequences)
+  - ❌ Llama models: Not supported (AWS Bedrock limitation)
+
+### Technical Details
+- Based on comprehensive research of official AWS Bedrock documentation
+- All changes maintain full backward compatibility
+- Test results show significant improvements in stop sequences reliability for supported models
+- Added detailed explanations to help users understand AWS Bedrock's actual capabilities
+
+## [2.4.2] - 2025-07-31 (Stop Sequences Support)
+### Added
+- Stop sequences support for compatible models
+  - OpenAI-compatible `stop` and `stop_sequences` parameters
+  - Automatic string-to-array conversion for compatibility
+  - Model-specific parameter mapping (stop_sequences for Claude, stopSequences for Nova, stop for Mistral)
+- Enhanced request building logic to include stop sequences in appropriate API formats
+- Comprehensive stop sequences testing and validation with `npm run test-stop`
+
+### Fixed
+- **Critical Discovery**: Removed stop sequences support from Llama models
+  - AWS Bedrock does not support stop sequences for Llama models (confirmed via official documentation)
+  - Llama models only support: `prompt`, `temperature`, `top_p`, `max_gen_len`, `images`
+  - This is an AWS Bedrock limitation, not a wrapper limitation
+- Fixed Nova model configuration conflicts that were causing stop sequence inconsistencies
+- Improved error handling for empty responses when stop sequences trigger early
+
+### Technical Details
+- **Model Support Matrix**:
+  - ✅ Claude models: Full support (up to 8,191 sequences)
+  - ✅ Nova models: Full support (up to 4 sequences)
+  - ✅ Mistral models: Full support (up to 10 sequences)
+  - ❌ Llama models: Not supported (AWS Bedrock limitation)
+- Updated request construction for both messages API and prompt-based models
+- Supports both single string and array formats for stop sequences
+- Maintains full backward compatibility with existing API usage
+- Added comprehensive documentation in README.md and CLAUDE.md explaining support limitations
+
 ## [2.4.0] - 2025-07-24 (AWS Nova Models)
 ### Added
 - Support for AWS Nova models

package/README.md

@@ -44,6 +44,7 @@ Bedrock Wrapper is an npm package that simplifies the integration of existing Op
         "stream": true,
         "temperature": LLM_TEMPERATURE,
         "top_p": LLM_TOP_P,
+        "stop_sequences": ["STOP", "END"], // Optional: sequences that will stop generation
     };
     ```
 
@@ -189,6 +190,48 @@ You can include multiple images in a single message by adding more image_url obj
 
 ---
 
+### Stop Sequences
+
+Stop sequences are custom text sequences that cause the model to stop generating text. This is useful for controlling where the model stops its response.
+
+```javascript
+const openaiChatCompletionsCreateObject = {
+    "messages": messages,
+    "model": "Claude-3-5-Sonnet",
+    "max_tokens": 100,
+    "stop_sequences": ["STOP", "END", "\n\n"], // Array of stop sequences
+    // OR use single string format:
+    // "stop": "STOP"
+};
+```
+
+**Model Support:**
+- ✅ **Claude models**: Fully supported (up to 8,191 sequences)
+- ✅ **Nova models**: Fully supported (up to 4 sequences)  
+- ✅ **Mistral models**: Fully supported (up to 10 sequences)
+- ❌ **Llama models**: Not supported (AWS Bedrock limitation)
+
+**Features:**
+- Compatible with OpenAI's `stop` parameter (single string or array)
+- Also accepts `stop_sequences` parameter for explicit usage
+- Automatic conversion between string and array formats
+- Model-specific parameter mapping handled automatically
+
+**Example Usage:**
+```javascript
+// Stop generation when model tries to output "7"
+const result = await bedrockWrapper(awsCreds, {
+    messages: [{ role: "user", content: "Count from 1 to 10" }],
+    model: "Claude-3-5-Sonnet",  // Use Claude, Nova, or Mistral models
+    stop_sequences: ["7"]
+});
+// Response: "1, 2, 3, 4, 5, 6," (stops before "7")
+
+// Note: Llama models will ignore stop sequences due to AWS Bedrock limitations
+```
+
+---
+
 ### 📢 P.S.
 
 In case you missed it at the beginning of this doc, for an even easier setup, use the 🔀 [Bedrock Proxy Endpoint](https://github.com/jparkerweb/bedrock-proxy-endpoint) project to spin up your own custom OpenAI server endpoint (using the standard `baseUrl`, and `apiKey` params).

package/bedrock-models.js

@@ -19,6 +19,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -46,6 +47,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -77,6 +79,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -104,6 +107,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -135,6 +139,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -166,6 +171,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -190,6 +196,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -213,6 +220,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -236,6 +244,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -254,6 +263,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -552,11 +562,11 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "maxTokens",
         "max_supported_response_tokens": 5000,
+        "stop_sequences_param_name":     "stopSequences",
         "response_chunk_element":        "contentBlockDelta.delta.text",
         "response_nonchunk_element":     "output.message.content[0].text",
         "special_request_schema": {
-            "schemaVersion": "messages-v1",
-            "inferenceConfig": {}
+            "schemaVersion": "messages-v1"
         },
         "image_support": {
             "max_image_size": 5242880, // 5MB per image
@@ -576,11 +586,11 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "maxTokens",
         "max_supported_response_tokens": 5000,
+        "stop_sequences_param_name":     "stopSequences",
         "response_chunk_element":        "contentBlockDelta.delta.text",
         "response_nonchunk_element":     "output.message.content[0].text",
         "special_request_schema": {
-            "schemaVersion": "messages-v1",
-            "inferenceConfig": {}
+            "schemaVersion": "messages-v1"
         },
         "image_support": {
             "max_image_size": 5242880, // 5MB per image
@@ -600,6 +610,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "maxTokens",
         "max_supported_response_tokens": 5000,
+        "stop_sequences_param_name":     "stopSequences",
         "response_chunk_element":        "contentBlockDelta.delta.text",
         "response_nonchunk_element":     "output.message.content[0].text",
         "special_request_schema": {
@@ -632,6 +643,7 @@ export const bedrock_models = [
         "display_role_names":            false,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "outputs[0].text"
     },
     {
@@ -659,6 +671,7 @@ export const bedrock_models = [
         "display_role_names":            false,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 4096,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "outputs[0].text"
     },
     {
@@ -686,6 +699,7 @@ export const bedrock_models = [
         "display_role_names":            false,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "outputs[0].text"
     },
 ];

package/bedrock-wrapper.js

@@ -64,7 +64,7 @@ async function processImage(imageInput) {
 
 export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObject, { logging = false } = {} ) {
     const { region, accessKeyId, secretAccessKey } = awsCreds;
-    let { messages, model, max_tokens, stream, temperature, top_p, include_thinking_data } = openaiChatCompletionsCreateObject;
+    let { messages, model, max_tokens, stream, temperature, top_p, include_thinking_data, stop, stop_sequences } = openaiChatCompletionsCreateObject;
 
 
   let {awsModelId, awsModel} = findAwsModelWithId(model);
@@ -269,13 +269,17 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
                 };
             });
             
+            const stopSequencesValue = stop_sequences || stop;
             const novaRequest = {
                 ...awsModel.special_request_schema,
                 messages: novaMessages,
                 inferenceConfig: {
                     [awsModel.max_tokens_param_name]: max_gen_tokens,
                     temperature: temperature,
-                    topP: top_p
+                    topP: top_p,
+                    ...(awsModel.stop_sequences_param_name && stopSequencesValue && {
+                        [awsModel.stop_sequences_param_name]: Array.isArray(stopSequencesValue) ? stopSequencesValue : [stopSequencesValue]
+                    })
                 }
             };
             
@@ -287,12 +291,16 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
             return novaRequest;
         } else {
             // Standard messages API format (Claude, etc.)
+            const stopSequencesValue = stop_sequences || stop;
             return {
                 messages: prompt,
                 ...(awsModel.system_as_separate_field && system_message && { system: system_message }),
                 [awsModel.max_tokens_param_name]: max_gen_tokens,
                 temperature: temperature,
                 top_p: top_p,
+                ...(awsModel.stop_sequences_param_name && stopSequencesValue && {
+                    [awsModel.stop_sequences_param_name]: Array.isArray(stopSequencesValue) ? stopSequencesValue : [stopSequencesValue]
+                }),
                 ...awsModel.special_request_schema
             };
         }
@@ -311,6 +319,12 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
         [awsModel.max_tokens_param_name]: max_gen_tokens,
         temperature: temperature,
         top_p: top_p,
+        ...(() => {
+            const stopSequencesValue = stop_sequences || stop;
+            return awsModel.stop_sequences_param_name && stopSequencesValue ? {
+                [awsModel.stop_sequences_param_name]: Array.isArray(stopSequencesValue) ? stopSequencesValue : [stopSequencesValue]
+            } : {};
+        })(),
         ...awsModel.special_request_schema
     };
     
@@ -392,7 +406,23 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
             }
         }
 
+        // Handle case where stop sequences cause empty content array
+        if (!text_result && decodedBodyResponse.stop_reason === "stop_sequence") {
+            // If stopped by sequence but no content, return empty string instead of undefined
+            text_result = "";
+        }
+
+        // Ensure text_result is a string to prevent 'undefined' from being part of the response
+        if (text_result === null || text_result === undefined) {
+            text_result = "";
+        }
+
         let result = thinking_result ? `<think>${thinking_result}</think>\n\n${text_result}` : text_result;
+        
+        // Ensure final result is a string, in case thinking_result was also empty
+        if (result === null || result === undefined) {
+            result = "";
+        }
         yield result;
     }    
 }
@@ -428,7 +458,10 @@ function findAwsModelWithId(model) {
 export async function listBedrockWrapperSupportedModels() {
     let supported_models = [];
     for (let i = 0; i < bedrock_models.length; i++) {
-        supported_models.push(`{"modelName": ${bedrock_models[i].modelName}, "modelId": ${bedrock_models[i].modelId}}`);
+        supported_models.push(JSON.stringify({
+            modelName: bedrock_models[i].modelName,
+            modelId: bedrock_models[i].modelId
+        }));
     }
     return supported_models;
 }

package/logs/e0b34b2c-ee9a-4813-893a-82d47d3d5141/notification.json

@@ -0,0 +1,51 @@
+[
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  },
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  },
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  },
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  },
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  },
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  },
+  {
+    "session_id": "e0b34b2c-ee9a-4813-893a-82d47d3d5141",
+    "transcript_path": "C:\\Users\\Justin.Parker\\.claude\\projects\\C--git-bedrock-wrapper\\e0b34b2c-ee9a-4813-893a-82d47d3d5141.jsonl",
+    "cwd": "C:\\git\\bedrock-wrapper",
+    "hook_event_name": "Notification",
+    "message": "Claude is waiting for your input"
+  }
+]