bedrock-wrapper 2.8.0 → 2.9.0

package/AGENTS.md

@@ -0,0 +1,123 @@
+# AGENTS.md
+
+This file provides guidance to AI coding agents like Claude Code (claude.ai/code), Cursor AI, Codex, Gemini CLI, GitHub Copilot, and other AI coding assistants when working with code in this repository.
+
+## Project Purpose
+
+Bedrock Wrapper translates OpenAI-compatible API objects to AWS Bedrock's serverless inference LLMs. It acts as an adapter layer allowing applications using the OpenAI API format to seamlessly call AWS Bedrock models.
+
+## Development Commands
+
+```bash
+npm install              # Install dependencies
+npm run clean            # Clean reinstall (removes node_modules and package-lock.json)
+npm run test             # Test all models with both Invoke and Converse APIs
+npm run test:invoke      # Test with Invoke API only
+npm run test:converse    # Test with Converse API only
+npm run test-vision      # Test vision capabilities
+npm run test-stop        # Test stop sequences
+npm run interactive      # Interactive CLI for testing specific models
+```
+
+## Architecture Overview
+
+```
+bedrock-wrapper.js (main entry)
+       │
+       ├── Converse API Path (useConverseAPI: true)
+       │   └── Unified format for all models
+       │
+       └── Invoke API Path (default)
+           └── Model-specific request/response handling
+                    │
+                    └── bedrock-models.js
+                        └── Model configurations registry
+```
+
+### Key Functions in bedrock-wrapper.js
+
+| Function | Line | Purpose |
+|----------|------|---------|
+| `bedrockWrapper()` | ~501 | Main entry point, async generator |
+| `convertToConverseFormat()` | ~86 | OpenAI messages → Converse API format |
+| `processMessagesForInvoke()` | ~168 | Model-specific message processing |
+| `buildInvokePrompt()` | ~234 | Constructs model-specific prompts |
+| `buildInvokeRequest()` | ~300 | Creates model-specific request objects |
+| `executeInvokeAPI()` | ~409 | Handles streaming and non-streaming |
+| `findAwsModelWithId()` | ~763 | Model lookup by name or ID |
+
+### Model Configuration Schema (bedrock-models.js)
+
+Each model entry requires:
+- `modelName`: Consumer-facing name (e.g., "Claude-4-5-Sonnet")
+- `modelId`: AWS Bedrock identifier
+- `vision`: Boolean for image support
+- `messages_api`: Boolean (true = structured messages, false = prompt string)
+- `response_chunk_element`: JSON path for streaming response extraction
+- `response_nonchunk_element`: JSON path for non-streaming response
+
+### Two API Paths
+
+1. **Converse API** (`useConverseAPI: true`): Unified format, handles all models consistently
+2. **Invoke API** (default): Model-specific formatting required
+
+Some models (e.g., DeepSeek-V3.1) have `converse_api_only: true` and automatically use the Converse API.
+
+## Model Family Patterns
+
+| Family | API Type | Special Handling |
+|--------|----------|------------------|
+| Claude | Messages API | Thinking tags: `<think>`, anthropic_version required |
+| Nova | Messages API | Content as array `[{text: content}]`, schemaVersion: "messages-v1" |
+| Llama | Prompt-based | Role tags: `<\|begin_of_text\|>`, `<\|start_header_id\|>` |
+| Mistral | Prompt-based (older) / Messages (v3+) | `[INST]`/`[/INST]` tags for older models |
+| GPT-OSS | Messages API | Reasoning tags: `<reasoning>`, streaming not supported |
+| Qwen | Messages API | Standard messages format |
+| DeepSeek | Messages API | V3.1 requires Converse API only |
+| Gemma | Messages API | Standard messages format with vision |
+| Kimi | Messages API | preserve_reasoning for thinking models |
+
+## Adding a New Model
+
+1. Add entry to `bedrock_models` array in `bedrock-models.js`
+2. For prompt-based models, define all role prefix/suffix tokens
+3. For vision models, set `vision: true` and add `image_support` config
+4. For thinking models, add `thinking` config in `special_request_schema`
+5. Test with `npm run test` to verify both API paths
+
+## Key Implementation Details
+
+### Image Processing
+- Uses Sharp library to resize images to max 2048x2048
+- Converts all formats to JPEG for consistency
+- Handles base64, data URLs, and HTTP URLs
+
+### Thinking Mode
+- Claude: `<think>` tags, budget_tokens in special_request_schema
+- GPT-OSS: `<reasoning>` tags, preserve_reasoning flag
+- Temperature auto-set to 1.0, budget_tokens constrained to 80% of max_tokens
+
+### Stop Sequences
+- Claude: `stop_sequences` (up to 8,191)
+- Nova: `stopSequences` (up to 4)
+- Mistral: `stop` (up to 10)
+- Llama: Not supported by AWS Bedrock
+
+## Environment Setup
+
+Create `.env` file:
+```
+AWS_REGION=us-west-2
+AWS_ACCESS_KEY_ID=your_key
+AWS_SECRET_ACCESS_KEY=your_secret
+LLM_MAX_GEN_TOKENS=1024
+LLM_TEMPERATURE=0.2
+```
+
+## Test Output Files
+
+After running tests, check these files for results:
+- `test-models-output.txt`
+- `test-vision-models-output.txt`
+- `test-stop-sequences-output.txt`
+- `test-converse-api-output.txt`

package/CHANGELOG.md

@@ -2,6 +2,23 @@
 
 All notable changes to this project will be documented in this file.
 
+## [2.9.0] - 2026-01-08 (Llama 4 Models)
+
+### ✨ Added
+
+- Support for Llama 4 Scout and Maverick models
+  - Llama-4-Scout-17b (vision support, 2K max output tokens)
+  - Llama-4-Maverick-17b (vision support, 2K max output tokens)
+- First Llama models with multimodal/vision capabilities in this wrapper
+- Cross-region inference profile IDs (us.meta.llama4-*)
+
+### ⚙️ Technical Details
+
+- **Vision Support**: Both models support image inputs (first Llama models with vision)
+- **API Compatibility**: Both Invoke API and Converse API paths supported
+- **Streaming**: Full streaming and non-streaming support
+- **Stop Sequences**: Not supported (AWS Bedrock limitation for all Llama models)
+
 ## [2.8.0] - 2025-12-05 (New Models: Claude Opus 4.5, Gemma, Kimi, MiniMax, Mistral, Nova)
 
 ### ✨ Added

package/README.md

@@ -163,6 +163,8 @@ Bedrock Wrapper is an npm package that simplifies the integration of existing Op
 | Llama-3-2-11b              | us.meta.llama3-2-11b-instruct-v1:0           |  ❌  |
 | Llama-3-2-90b              | us.meta.llama3-2-90b-instruct-v1:0           |  ❌  |
 | Llama-3-3-70b              | us.meta.llama3-3-70b-instruct-v1:0           |  ❌  |
+| Llama-4-Scout-17b          | us.meta.llama4-scout-17b-instruct-v1:0       |  ✅  |
+| Llama-4-Maverick-17b       | us.meta.llama4-maverick-17b-instruct-v1:0    |  ✅  |
 | Magistral-Small-2509       | mistral.magistral-small-2509                 |  ❌  |
 | MiniMax-M2                 | minimax.minimax-m2                           |  ❌  |
 | Ministral-3-3b             | mistral.ministral-3-3b-instruct              |  ✅  |
@@ -229,7 +231,7 @@ for await (const chunk of bedrockWrapper(awsCreds, openaiChatCompletionsCreateOb
 
 ### Image Support
 
-For models with image support (Claude 4+ series including Claude 4.5 Opus, Claude 4.5 Sonnet, Claude 4.5 Haiku, Claude 3.7 Sonnet, Claude 3.5 Sonnet, Claude 3 Haiku, Nova Pro, Nova Lite, Nova 2 Lite, Mistral Large 3, Ministral 3 series, and Gemma 3 series), you can include images in your messages using the following format (not all models support system prompts):
+For models with image support (Claude 4+ series including Claude 4.5 Opus, Claude 4.5 Sonnet, Claude 4.5 Haiku, Claude 3.7 Sonnet, Claude 3.5 Sonnet, Claude 3 Haiku, Nova Pro, Nova Lite, Nova 2 Lite, Mistral Large 3, Ministral 3 series, Gemma 3 series, and Llama 4 series), you can include images in your messages using the following format (not all models support system prompts):
 
 ```javascript
 messages = [

package/bedrock-models.js

@@ -547,6 +547,62 @@ export const bedrock_models = [
         "max_supported_response_tokens": 2048,
         "response_chunk_element":        "generation"
     },
+    {
+        // ======================
+        // == Llama 4 Scout 17b ==
+        // ======================
+        "modelName":                     "Llama-4-Scout-17b",
+        // "modelId":                    "meta.llama4-scout-17b-instruct-v1:0",
+        "modelId":                       "us.meta.llama4-scout-17b-instruct-v1:0",
+        "vision":                        true,
+        "messages_api":                  false,
+        "bos_text":                      "<|begin_of_text|>",
+        "role_system_message_prefix":    "",
+        "role_system_message_suffix":    "",
+        "role_system_prefix":            "<|start_header_id|>",
+        "role_system_suffix":            "<|end_header_id|>",
+        "role_user_message_prefix":      "",
+        "role_user_message_suffix":      "",
+        "role_user_prefix":              "<|start_header_id|>",
+        "role_user_suffix":              "<|end_header_id|>",
+        "role_assistant_message_prefix": "",
+        "role_assistant_message_suffix": "",
+        "role_assistant_prefix":         "<|start_header_id|>",
+        "role_assistant_suffix":         "<|end_header_id|>",
+        "eom_text":                      "<|eot_id|>",
+        "display_role_names":            true,
+        "max_tokens_param_name":         "max_gen_len",
+        "max_supported_response_tokens": 2048,
+        "response_chunk_element":        "generation"
+    },
+    {
+        // ========================
+        // == Llama 4 Maverick 17b ==
+        // ========================
+        "modelName":                     "Llama-4-Maverick-17b",
+        // "modelId":                    "meta.llama4-maverick-17b-instruct-v1:0",
+        "modelId":                       "us.meta.llama4-maverick-17b-instruct-v1:0",
+        "vision":                        true,
+        "messages_api":                  false,
+        "bos_text":                      "<|begin_of_text|>",
+        "role_system_message_prefix":    "",
+        "role_system_message_suffix":    "",
+        "role_system_prefix":            "<|start_header_id|>",
+        "role_system_suffix":            "<|end_header_id|>",
+        "role_user_message_prefix":      "",
+        "role_user_message_suffix":      "",
+        "role_user_prefix":              "<|start_header_id|>",
+        "role_user_suffix":              "<|end_header_id|>",
+        "role_assistant_message_prefix": "",
+        "role_assistant_message_suffix": "",
+        "role_assistant_prefix":         "<|start_header_id|>",
+        "role_assistant_suffix":         "<|end_header_id|>",
+        "eom_text":                      "<|eot_id|>",
+        "display_role_names":            true,
+        "max_tokens_param_name":         "max_gen_len",
+        "max_supported_response_tokens": 2048,
+        "response_chunk_element":        "generation"
+    },
     {
         // ==================
         // == Llama 3.2 1b ==

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bedrock-wrapper",
-  "version": "2.8.0",
+  "version": "2.9.0",
   "description": "🪨 Bedrock Wrapper is an npm package that simplifies the integration of existing OpenAI-compatible API objects with AWS Bedrock's serverless inference LLMs.",
   "homepage": "https://www.equilllabs.com/projects/bedrock-wrapper",
   "repository": {

package/specs--completed/llama-4-model-support/PLAN-DRAFT-20260108-130000.md

@@ -0,0 +1,241 @@
+# Llama 4 Model Support - Implementation Plan
+
+**Created:** 2026-01-08
+**Status:** Complete (All Phases Verified)
+**Confidence:** 92% (Requirements: 25/25, Feasibility: 24/25, Integration: 23/25, Risk: 20/25)
+
+## 1. Executive Summary
+
+Add support for two new AWS Bedrock models: Llama 4 Scout 17B and Llama 4 Maverick 17B. Both models are multimodal (vision + text) and will be the first Llama models in this wrapper with vision enabled.
+
+## 2. Requirements
+
+### 2.1 Functional Requirements
+- [x] FR-1: Add `Llama-4-Scout-17b` model configuration to bedrock-models.js
+- [x] FR-2: Add `Llama-4-Maverick-17b` model configuration to bedrock-models.js
+- [x] FR-3: Enable vision/image input support for both models
+- [x] FR-4: Support both Invoke API and Converse API paths
+- [x] FR-5: Use cross-region inference profile IDs (following existing pattern)
+
+### 2.2 Non-Functional Requirements
+- [x] NFR-1: Follow existing Llama model configuration patterns
+- [x] NFR-2: Maintain backward compatibility with existing code
+
+### 2.3 Out of Scope
+- New test frameworks or test file creation
+- Changes to core wrapper logic
+- Tool use / function calling support
+
+### 2.4 Testing Strategy
+| Preference | Selection |
+|------------|-----------|
+| Test Types | Expand existing model tests |
+| Phase Testing | Run after implementation |
+| Coverage Target | Same coverage as existing Llama models |
+
+## 3. Tech Stack
+
+| Category | Technology | Version | Justification |
+|----------|------------|---------|---------------|
+| Language | JavaScript (ES Modules) | N/A | Existing project standard |
+| Runtime | Node.js | Existing | No changes |
+| Config Format | JSON objects in JS | N/A | Existing pattern |
+
+No new technologies required - configuration addition only.
+
+## 4. Architecture
+
+### 4.1 Architecture Pattern
+**Configuration Registry Pattern** - Models are defined as entries in the `bedrock_models` array in `bedrock-models.js`. New models follow the existing Llama configuration schema.
+
+### 4.2 System Context Diagram
+```
+┌─────────────────────────────────────────────────────────────┐
+│                     bedrock-wrapper.js                       │
+│                                                              │
+│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐  │
+│  │ Converse API │    │  Invoke API  │    │ Image Process│  │
+│  │    Path      │    │    Path      │    │   (Sharp)    │  │
+│  └──────┬───────┘    └──────┬───────┘    └──────┬───────┘  │
+│         │                   │                    │          │
+│         └─────────┬─────────┴────────────────────┘          │
+│                   │                                          │
+│         ┌─────────▼─────────┐                               │
+│         │  bedrock-models.js │◄── ADD NEW MODELS HERE       │
+│         │  (model registry)  │                               │
+│         └───────────────────┘                               │
+└─────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+                    ┌─────────────────┐
+                    │   AWS Bedrock   │
+                    │  - Llama 4 Scout│
+                    │  - Llama 4 Maverick
+                    └─────────────────┘
+```
+
+### 4.3 Component Overview
+
+| Component | Responsibility | Dependencies |
+|-----------|----------------|--------------|
+| bedrock-models.js | Model configuration registry | None |
+| test-models.js | General model testing | bedrock-models.js |
+| test-vision.js | Vision capability testing | bedrock-models.js |
+
+### 4.4 Data Model
+
+**Model Configuration Schema (per existing Llama pattern):**
+
+| Field | Type | Value (Scout) | Value (Maverick) |
+|-------|------|---------------|------------------|
+| modelName | string | "Llama-4-Scout-17b" | "Llama-4-Maverick-17b" |
+| modelId | string | "us.meta.llama4-scout-17b-instruct-v1:0" | "us.meta.llama4-maverick-17b-instruct-v1:0" |
+| vision | boolean | true | true |
+| messages_api | boolean | false | false |
+| bos_text | string | "<\|begin_of_text\|>" | "<\|begin_of_text\|>" |
+| role_system_prefix | string | "<\|start_header_id\|>" | "<\|start_header_id\|>" |
+| role_system_suffix | string | "<\|end_header_id\|>" | "<\|end_header_id\|>" |
+| role_user_prefix | string | "<\|start_header_id\|>" | "<\|start_header_id\|>" |
+| role_user_suffix | string | "<\|end_header_id\|>" | "<\|end_header_id\|>" |
+| role_assistant_prefix | string | "<\|start_header_id\|>" | "<\|start_header_id\|>" |
+| role_assistant_suffix | string | "<\|end_header_id\|>" | "<\|end_header_id\|>" |
+| eom_text | string | "<\|eot_id\|>" | "<\|eot_id\|>" |
+| display_role_names | boolean | true | true |
+| max_tokens_param_name | string | "max_gen_len" | "max_gen_len" |
+| max_supported_response_tokens | number | 2048 | 2048 |
+| response_chunk_element | string | "generation" | "generation" |
+
+### 4.5 API Design
+
+No new APIs. Models integrate with existing:
+- `bedrockWrapper()` - main entry point
+- `listBedrockWrapperSupportedModels()` - model listing
+
+## 5. Implementation Phases
+
+### Phase 1: Add Model Configurations
+**Goal:** Add both Llama 4 model configurations to bedrock-models.js
+**Dependencies:** None
+
+- [x] Task 1.1: Add Llama 4 Scout 17B configuration after existing Llama models (~line 520)
+- [x] Task 1.2: Add Llama 4 Maverick 17B configuration after Scout
+- [x] Task 1.3: Include commented single-region model IDs as fallback reference
+
+**Insertion Point:** After the Llama 3.3 70b model block (around line 520)
+
+**Configuration Template:**
+```javascript
+{
+    // ======================
+    // == Llama 4 Scout 17b ==
+    // ======================
+    "modelName":                     "Llama-4-Scout-17b",
+    // "modelId":                    "meta.llama4-scout-17b-instruct-v1:0",
+    "modelId":                       "us.meta.llama4-scout-17b-instruct-v1:0",
+    "vision":                        true,
+    "messages_api":                  false,
+    "bos_text":                      "<|begin_of_text|>",
+    "role_system_message_prefix":    "",
+    "role_system_message_suffix":    "",
+    "role_system_prefix":            "<|start_header_id|>",
+    "role_system_suffix":            "<|end_header_id|>",
+    "role_user_message_prefix":      "",
+    "role_user_message_suffix":      "",
+    "role_user_prefix":              "<|start_header_id|>",
+    "role_user_suffix":              "<|end_header_id|>",
+    "role_assistant_message_prefix": "",
+    "role_assistant_message_suffix": "",
+    "role_assistant_prefix":         "<|start_header_id|>",
+    "role_assistant_suffix":         "<|end_header_id|>",
+    "eom_text":                      "<|eot_id|>",
+    "display_role_names":            true,
+    "max_tokens_param_name":         "max_gen_len",
+    "max_supported_response_tokens": 2048,
+    "response_chunk_element":        "generation"
+},
+{
+    // ========================
+    // == Llama 4 Maverick 17b ==
+    // ========================
+    "modelName":                     "Llama-4-Maverick-17b",
+    // "modelId":                    "meta.llama4-maverick-17b-instruct-v1:0",
+    "modelId":                       "us.meta.llama4-maverick-17b-instruct-v1:0",
+    "vision":                        true,
+    "messages_api":                  false,
+    "bos_text":                      "<|begin_of_text|>",
+    "role_system_message_prefix":    "",
+    "role_system_message_suffix":    "",
+    "role_system_prefix":            "<|start_header_id|>",
+    "role_system_suffix":            "<|end_header_id|>",
+    "role_user_message_prefix":      "",
+    "role_user_message_suffix":      "",
+    "role_user_prefix":              "<|start_header_id|>",
+    "role_user_suffix":              "<|end_header_id|>",
+    "role_assistant_message_prefix": "",
+    "role_assistant_message_suffix": "",
+    "role_assistant_prefix":         "<|start_header_id|>",
+    "role_assistant_suffix":         "<|end_header_id|>",
+    "eom_text":                      "<|eot_id|>",
+    "display_role_names":            true,
+    "max_tokens_param_name":         "max_gen_len",
+    "max_supported_response_tokens": 2048,
+    "response_chunk_element":        "generation"
+},
+```
+
+### Phase 2: Verification Testing
+**Goal:** Verify both models work correctly with existing tests
+**Dependencies:** Phase 1 complete
+
+- [x] Task 2.1: Run `npm run test` - verify both models pass streaming and non-streaming tests
+- [x] Task 2.2: Run `npm run test-vision` - verify vision capability works for both models
+- [x] Task 2.3: Run `npm run test:converse` - verify Converse API works
+- [x] Task 2.4: Review output files for any errors or warnings
+
+## 6. Risks and Mitigations
+
+| Risk | Likelihood | Impact | Mitigation |
+|------|------------|--------|------------|
+| Cross-region profile ID not yet available | Low | Medium | Single-region ID included as commented fallback |
+| Llama 4 uses different special tokens | Low | High | Test both APIs; adjust tokens if tests fail |
+| Vision handling differs from expected | Low | Medium | Existing Llama image format in codebase; Converse API as fallback |
+| Model not available in user's region | Medium | Low | Document supported regions (us-east-1, us-east-2, us-west-1, us-west-2) |
+
+## 7. Success Criteria
+
+- [x] Both models appear in `listBedrockWrapperSupportedModels()` output
+- [x] `npm run test` passes for both Llama 4 models (streaming & non-streaming)
+- [x] `npm run test-vision` includes and passes for both models
+- [x] Both Invoke API and Converse API paths work correctly
+- [x] No regression in existing model tests
+
+## 8. Open Questions
+
+None - all requirements clarified during planning.
+
+## 9. Assumptions
+
+- AWS Bedrock cross-region inference profiles are available for Llama 4 models (following pattern `us.meta.llama4-*`)
+- Llama 4 models use the same special tokens as Llama 3.x family
+- Vision handling follows the existing Llama image format in the codebase
+- Max response token limit of 2048 is appropriate (same as other Llama models)
+
+## 10. Completion Summary
+
+**Completed:** 2026-01-08 | **Completion Rate:** 100% (7/7 tasks)
+
+### What Was Built
+Added support for two new AWS Bedrock models: Llama 4 Scout 17B and Llama 4 Maverick 17B. These are the first Llama models in the wrapper with vision/multimodal support enabled. Both models work with Invoke API and Converse API paths, streaming and non-streaming modes.
+
+### Files Modified
+| File | Changes |
+|------|---------|
+| `bedrock-models.js` | Added Llama-4-Scout-17b config (lines 550-577), Llama-4-Maverick-17b config (lines 578-605) |
+
+### Test Results
+- `npm run test --both`: Both models pass all 4 test modes (Invoke/Converse × Streaming/Non-streaming)
+- `npm run test-vision --both`: Both models successfully describe images
+- `npm run test:converse`: Both models pass Converse API tests
+
+### Known Limitations
+- Llama 4 models do not support stop sequences (AWS Bedrock limitation, consistent with other Llama models)