bedrock-wrapper 2.4.0 → 2.4.2

package/CHANGELOG.md

@@ -1,6 +1,21 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## [2.4.2] - 2025-07-31 (Stop Sequences Support)
+### Added
+- Stop sequences support for all models
+  - OpenAI-compatible `stop` and `stop_sequences` parameters
+  - Automatic string-to-array conversion for compatibility
+  - Model-specific parameter mapping (stop_sequences for Claude, stopSequences for Nova, stop for Llama/Mistral)
+- Enhanced request building logic to include stop sequences in appropriate API formats
+- Comprehensive stop sequences testing and validation
+
+### Technical Details
+- Added `stop_sequences_param_name` configuration to all 26+ model definitions
+- Updated request construction for both messages API and prompt-based models
+- Supports both single string and array formats for stop sequences
+- Maintains full backward compatibility with existing API usage
+
 ## [2.4.0] - 2025-07-24 (AWS Nova Models)
 ### Added
 - Support for AWS Nova models

package/README.md

@@ -44,6 +44,7 @@ Bedrock Wrapper is an npm package that simplifies the integration of existing Op
         "stream": true,
         "temperature": LLM_TEMPERATURE,
         "top_p": LLM_TOP_P,
+        "stop_sequences": ["STOP", "END"], // Optional: sequences that will stop generation
     };
     ```
 
@@ -103,12 +104,19 @@ Bedrock Wrapper is an npm package that simplifies the integration of existing Op
 
 | modelName                  | AWS Model Id                                 | Image |
 |----------------------------|----------------------------------------------|-------|
+| Claude-4-Opus              | us.anthropic.claude-opus-4-20250514-v1:0    |  ✅  |
+| Claude-4-Opus-Thinking     | us.anthropic.claude-opus-4-20250514-v1:0    |  ✅  |
+| Claude-4-Sonnet            | us.anthropic.claude-sonnet-4-20250514-v1:0  |  ✅  |
+| Claude-4-Sonnet-Thinking   | us.anthropic.claude-sonnet-4-20250514-v1:0  |  ✅  |
 | Claude-3-7-Sonnet-Thinking | us.anthropic.claude-3-7-sonnet-20250219-v1:0 |  ✅  |
 | Claude-3-7-Sonnet          | us.anthropic.claude-3-7-sonnet-20250219-v1:0 |  ✅  |
 | Claude-3-5-Sonnet-v2       | anthropic.claude-3-5-sonnet-20241022-v2:0    |  ✅  |
 | Claude-3-5-Sonnet          | anthropic.claude-3-5-sonnet-20240620-v1:0    |  ✅  |
 | Claude-3-5-Haiku           | anthropic.claude-3-5-haiku-20241022-v1:0     |  ❌  |
-| Claude-3-Haiku             | anthropic.claude-3-haiku-20240307-v1:0       |  ❌  |
+| Claude-3-Haiku             | anthropic.claude-3-haiku-20240307-v1:0       |  ✅  |
+| Nova-Pro                   | us.amazon.nova-pro-v1:0                     |  ✅  |
+| Nova-Lite                  | us.amazon.nova-lite-v1:0                    |  ✅  |
+| Nova-Micro                 | us.amazon.nova-micro-v1:0                   |  ❌  |
 | Llama-3-3-70b              | us.meta.llama3-3-70b-instruct-v1:0           |  ❌  |
 | Llama-3-2-1b               | us.meta.llama3-2-1b-instruct-v1:0            |  ❌  |
 | Llama-3-2-3b               | us.meta.llama3-2-3b-instruct-v1:0            |  ❌  |
@@ -136,7 +144,7 @@ Please modify the `bedrock_models.js` file and submit a PR 🏆 or create an Iss
 
 ### Image Support
 
-For models with image support (Claude 3.5 Sonnet, Claude 3.7 Sonnet, and Claude 3.7 Sonnet Thinking), you can include images in your messages using the following format:
+For models with image support (Claude 4 series, Claude 3.7 Sonnet, Claude 3.5 Sonnet, Claude 3 Haiku, Nova Pro, and Nova Lite), you can include images in your messages using the following format:
 
 ```javascript
 messages = [
@@ -182,6 +190,41 @@ You can include multiple images in a single message by adding more image_url obj
 
 ---
 
+### Stop Sequences
+
+All models support stop sequences - custom text sequences that cause the model to stop generating. This is useful for controlling where the model stops its response.
+
+```javascript
+const openaiChatCompletionsCreateObject = {
+    "messages": messages,
+    "model": "Claude-3-5-Sonnet",
+    "max_tokens": 100,
+    "stop_sequences": ["STOP", "END", "\n\n"], // Array of stop sequences
+    // OR use single string format:
+    // "stop": "STOP"
+};
+```
+
+**Features:**
+- Compatible with OpenAI's `stop` parameter (single string or array)
+- Also accepts `stop_sequences` parameter for explicit usage
+- Automatic conversion between string and array formats
+- Works with all 26+ supported models (Claude, Nova, Llama, Mistral)
+- Model-specific parameter mapping handled automatically
+
+**Example Usage:**
+```javascript
+// Stop generation when model tries to output "7"
+const result = await bedrockWrapper(awsCreds, {
+    messages: [{ role: "user", content: "Count from 1 to 10" }],
+    model: "Claude-3-5-Sonnet",
+    stop_sequences: ["7"]
+});
+// Response: "1, 2, 3, 4, 5, 6," (stops before "7")
+```
+
+---
+
 ### 📢 P.S.
 
 In case you missed it at the beginning of this doc, for an even easier setup, use the 🔀 [Bedrock Proxy Endpoint](https://github.com/jparkerweb/bedrock-proxy-endpoint) project to spin up your own custom OpenAI server endpoint (using the standard `baseUrl`, and `apiKey` params).

package/bedrock-models.js

@@ -19,6 +19,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -46,6 +47,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -77,6 +79,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -104,6 +107,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -135,6 +139,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "thinking_response_chunk_element": "delta.thinking",
@@ -166,6 +171,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 131072,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -190,6 +196,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -213,6 +220,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -236,6 +244,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -248,12 +257,13 @@ export const bedrock_models = [
         // ====================
         "modelName":                     "Claude-3-Haiku",
         "modelId":                       "anthropic.claude-3-haiku-20240307-v1:0",
-        "vision":                        false,
+        "vision":                        true,
         "messages_api":                  true,
         "system_as_separate_field":      true,
         "display_role_names":            true,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop_sequences",
         "response_chunk_element":        "delta.text",
         "response_nonchunk_element":     "content[0].text",
         "special_request_schema": {
@@ -291,6 +301,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -319,6 +330,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -347,6 +359,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -375,6 +388,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -403,6 +417,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -430,6 +445,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -457,6 +473,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -484,6 +501,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -511,6 +529,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -538,6 +557,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "max_gen_len",
         "max_supported_response_tokens": 2048,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "generation"
     },
     {
@@ -552,6 +572,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "maxTokens",
         "max_supported_response_tokens": 5000,
+        "stop_sequences_param_name":     "stopSequences",
         "response_chunk_element":        "contentBlockDelta.delta.text",
         "response_nonchunk_element":     "output.message.content[0].text",
         "special_request_schema": {
@@ -576,6 +597,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "maxTokens",
         "max_supported_response_tokens": 5000,
+        "stop_sequences_param_name":     "stopSequences",
         "response_chunk_element":        "contentBlockDelta.delta.text",
         "response_nonchunk_element":     "output.message.content[0].text",
         "special_request_schema": {
@@ -600,6 +622,7 @@ export const bedrock_models = [
         "display_role_names":            true,
         "max_tokens_param_name":         "maxTokens",
         "max_supported_response_tokens": 5000,
+        "stop_sequences_param_name":     "stopSequences",
         "response_chunk_element":        "contentBlockDelta.delta.text",
         "response_nonchunk_element":     "output.message.content[0].text",
         "special_request_schema": {
@@ -632,6 +655,7 @@ export const bedrock_models = [
         "display_role_names":            false,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "outputs[0].text"
     },
     {
@@ -659,6 +683,7 @@ export const bedrock_models = [
         "display_role_names":            false,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 4096,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "outputs[0].text"
     },
     {
@@ -686,6 +711,7 @@ export const bedrock_models = [
         "display_role_names":            false,
         "max_tokens_param_name":         "max_tokens",
         "max_supported_response_tokens": 8192,
+        "stop_sequences_param_name":     "stop",
         "response_chunk_element":        "outputs[0].text"
     },
 ];

package/bedrock-wrapper.js

@@ -64,7 +64,7 @@ async function processImage(imageInput) {
 
 export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObject, { logging = false } = {} ) {
     const { region, accessKeyId, secretAccessKey } = awsCreds;
-    let { messages, model, max_tokens, stream, temperature, top_p, include_thinking_data } = openaiChatCompletionsCreateObject;
+    let { messages, model, max_tokens, stream, temperature, top_p, include_thinking_data, stop, stop_sequences } = openaiChatCompletionsCreateObject;
 
 
   let {awsModelId, awsModel} = findAwsModelWithId(model);
@@ -269,13 +269,17 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
                 };
             });
             
+            const stopSequencesValue = stop_sequences || stop;
             const novaRequest = {
                 ...awsModel.special_request_schema,
                 messages: novaMessages,
                 inferenceConfig: {
                     [awsModel.max_tokens_param_name]: max_gen_tokens,
                     temperature: temperature,
-                    topP: top_p
+                    topP: top_p,
+                    ...(awsModel.stop_sequences_param_name && stopSequencesValue && {
+                        [awsModel.stop_sequences_param_name]: Array.isArray(stopSequencesValue) ? stopSequencesValue : [stopSequencesValue]
+                    })
                 }
             };
             
@@ -287,12 +291,16 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
             return novaRequest;
         } else {
             // Standard messages API format (Claude, etc.)
+            const stopSequencesValue = stop_sequences || stop;
             return {
                 messages: prompt,
                 ...(awsModel.system_as_separate_field && system_message && { system: system_message }),
                 [awsModel.max_tokens_param_name]: max_gen_tokens,
                 temperature: temperature,
                 top_p: top_p,
+                ...(awsModel.stop_sequences_param_name && stopSequencesValue && {
+                    [awsModel.stop_sequences_param_name]: Array.isArray(stopSequencesValue) ? stopSequencesValue : [stopSequencesValue]
+                }),
                 ...awsModel.special_request_schema
             };
         }
@@ -311,6 +319,12 @@ export async function* bedrockWrapper(awsCreds, openaiChatCompletionsCreateObjec
         [awsModel.max_tokens_param_name]: max_gen_tokens,
         temperature: temperature,
         top_p: top_p,
+        ...(() => {
+            const stopSequencesValue = stop_sequences || stop;
+            return awsModel.stop_sequences_param_name && stopSequencesValue ? {
+                [awsModel.stop_sequences_param_name]: Array.isArray(stopSequencesValue) ? stopSequencesValue : [stopSequencesValue]
+            } : {};
+        })(),
         ...awsModel.special_request_schema
     };
     

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bedrock-wrapper",
-  "version": "2.4.0",
+  "version": "2.4.2",
   "description": "🪨 Bedrock Wrapper is an npm package that simplifies the integration of existing OpenAI-compatible API objects with AWS Bedrock's serverless inference LLMs.",
   "homepage": "https://www.equilllabs.com/projects/bedrock-wrapper",
   "repository": {
@@ -32,7 +32,7 @@
   "author": "",
   "license": "ISC",
   "dependencies": {
-    "@aws-sdk/client-bedrock-runtime": "^3.848.0",
+    "@aws-sdk/client-bedrock-runtime": "^3.857.0",
     "dotenv": "^17.2.1",
     "sharp": "^0.34.3"
   },