bedrock-wrapper 2.7.2 → 2.9.0

package/specs--completed/llama-4-model-support/PLAN-DRAFT-20260108-130000.md

@@ -0,0 +1,241 @@
+# Llama 4 Model Support - Implementation Plan
+
+**Created:** 2026-01-08
+**Status:** Complete (All Phases Verified)
+**Confidence:** 92% (Requirements: 25/25, Feasibility: 24/25, Integration: 23/25, Risk: 20/25)
+
+## 1. Executive Summary
+
+Add support for two new AWS Bedrock models: Llama 4 Scout 17B and Llama 4 Maverick 17B. Both models are multimodal (vision + text) and will be the first Llama models in this wrapper with vision enabled.
+
+## 2. Requirements
+
+### 2.1 Functional Requirements
+- [x] FR-1: Add `Llama-4-Scout-17b` model configuration to bedrock-models.js
+- [x] FR-2: Add `Llama-4-Maverick-17b` model configuration to bedrock-models.js
+- [x] FR-3: Enable vision/image input support for both models
+- [x] FR-4: Support both Invoke API and Converse API paths
+- [x] FR-5: Use cross-region inference profile IDs (following existing pattern)
+
+### 2.2 Non-Functional Requirements
+- [x] NFR-1: Follow existing Llama model configuration patterns
+- [x] NFR-2: Maintain backward compatibility with existing code
+
+### 2.3 Out of Scope
+- New test frameworks or test file creation
+- Changes to core wrapper logic
+- Tool use / function calling support
+
+### 2.4 Testing Strategy
+| Preference | Selection |
+|------------|-----------|
+| Test Types | Expand existing model tests |
+| Phase Testing | Run after implementation |
+| Coverage Target | Same coverage as existing Llama models |
+
+## 3. Tech Stack
+
+| Category | Technology | Version | Justification |
+|----------|------------|---------|---------------|
+| Language | JavaScript (ES Modules) | N/A | Existing project standard |
+| Runtime | Node.js | Existing | No changes |
+| Config Format | JSON objects in JS | N/A | Existing pattern |
+
+No new technologies required - configuration addition only.
+
+## 4. Architecture
+
+### 4.1 Architecture Pattern
+**Configuration Registry Pattern** - Models are defined as entries in the `bedrock_models` array in `bedrock-models.js`. New models follow the existing Llama configuration schema.
+
+### 4.2 System Context Diagram
+```
+┌─────────────────────────────────────────────────────────────┐
+│                     bedrock-wrapper.js                       │
+│                                                              │
+│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐  │
+│  │ Converse API │    │  Invoke API  │    │ Image Process│  │
+│  │    Path      │    │    Path      │    │   (Sharp)    │  │
+│  └──────┬───────┘    └──────┬───────┘    └──────┬───────┘  │
+│         │                   │                    │          │
+│         └─────────┬─────────┴────────────────────┘          │
+│                   │                                          │
+│         ┌─────────▼─────────┐                               │
+│         │  bedrock-models.js │◄── ADD NEW MODELS HERE       │
+│         │  (model registry)  │                               │
+│         └───────────────────┘                               │
+└─────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+                    ┌─────────────────┐
+                    │   AWS Bedrock   │
+                    │  - Llama 4 Scout│
+                    │  - Llama 4 Maverick
+                    └─────────────────┘
+```
+
+### 4.3 Component Overview
+
+| Component | Responsibility | Dependencies |
+|-----------|----------------|--------------|
+| bedrock-models.js | Model configuration registry | None |
+| test-models.js | General model testing | bedrock-models.js |
+| test-vision.js | Vision capability testing | bedrock-models.js |
+
+### 4.4 Data Model
+
+**Model Configuration Schema (per existing Llama pattern):**
+
+| Field | Type | Value (Scout) | Value (Maverick) |
+|-------|------|---------------|------------------|
+| modelName | string | "Llama-4-Scout-17b" | "Llama-4-Maverick-17b" |
+| modelId | string | "us.meta.llama4-scout-17b-instruct-v1:0" | "us.meta.llama4-maverick-17b-instruct-v1:0" |
+| vision | boolean | true | true |
+| messages_api | boolean | false | false |
+| bos_text | string | "<\|begin_of_text\|>" | "<\|begin_of_text\|>" |
+| role_system_prefix | string | "<\|start_header_id\|>" | "<\|start_header_id\|>" |
+| role_system_suffix | string | "<\|end_header_id\|>" | "<\|end_header_id\|>" |
+| role_user_prefix | string | "<\|start_header_id\|>" | "<\|start_header_id\|>" |
+| role_user_suffix | string | "<\|end_header_id\|>" | "<\|end_header_id\|>" |
+| role_assistant_prefix | string | "<\|start_header_id\|>" | "<\|start_header_id\|>" |
+| role_assistant_suffix | string | "<\|end_header_id\|>" | "<\|end_header_id\|>" |
+| eom_text | string | "<\|eot_id\|>" | "<\|eot_id\|>" |
+| display_role_names | boolean | true | true |
+| max_tokens_param_name | string | "max_gen_len" | "max_gen_len" |
+| max_supported_response_tokens | number | 2048 | 2048 |
+| response_chunk_element | string | "generation" | "generation" |
+
+### 4.5 API Design
+
+No new APIs. Models integrate with existing:
+- `bedrockWrapper()` - main entry point
+- `listBedrockWrapperSupportedModels()` - model listing
+
+## 5. Implementation Phases
+
+### Phase 1: Add Model Configurations
+**Goal:** Add both Llama 4 model configurations to bedrock-models.js
+**Dependencies:** None
+
+- [x] Task 1.1: Add Llama 4 Scout 17B configuration after existing Llama models (~line 520)
+- [x] Task 1.2: Add Llama 4 Maverick 17B configuration after Scout
+- [x] Task 1.3: Include commented single-region model IDs as fallback reference
+
+**Insertion Point:** After the Llama 3.3 70b model block (around line 520)
+
+**Configuration Template:**
+```javascript
+{
+    // ======================
+    // == Llama 4 Scout 17b ==
+    // ======================
+    "modelName":                     "Llama-4-Scout-17b",
+    // "modelId":                    "meta.llama4-scout-17b-instruct-v1:0",
+    "modelId":                       "us.meta.llama4-scout-17b-instruct-v1:0",
+    "vision":                        true,
+    "messages_api":                  false,
+    "bos_text":                      "<|begin_of_text|>",
+    "role_system_message_prefix":    "",
+    "role_system_message_suffix":    "",
+    "role_system_prefix":            "<|start_header_id|>",
+    "role_system_suffix":            "<|end_header_id|>",
+    "role_user_message_prefix":      "",
+    "role_user_message_suffix":      "",
+    "role_user_prefix":              "<|start_header_id|>",
+    "role_user_suffix":              "<|end_header_id|>",
+    "role_assistant_message_prefix": "",
+    "role_assistant_message_suffix": "",
+    "role_assistant_prefix":         "<|start_header_id|>",
+    "role_assistant_suffix":         "<|end_header_id|>",
+    "eom_text":                      "<|eot_id|>",
+    "display_role_names":            true,
+    "max_tokens_param_name":         "max_gen_len",
+    "max_supported_response_tokens": 2048,
+    "response_chunk_element":        "generation"
+},
+{
+    // ========================
+    // == Llama 4 Maverick 17b ==
+    // ========================
+    "modelName":                     "Llama-4-Maverick-17b",
+    // "modelId":                    "meta.llama4-maverick-17b-instruct-v1:0",
+    "modelId":                       "us.meta.llama4-maverick-17b-instruct-v1:0",
+    "vision":                        true,
+    "messages_api":                  false,
+    "bos_text":                      "<|begin_of_text|>",
+    "role_system_message_prefix":    "",
+    "role_system_message_suffix":    "",
+    "role_system_prefix":            "<|start_header_id|>",
+    "role_system_suffix":            "<|end_header_id|>",
+    "role_user_message_prefix":      "",
+    "role_user_message_suffix":      "",
+    "role_user_prefix":              "<|start_header_id|>",
+    "role_user_suffix":              "<|end_header_id|>",
+    "role_assistant_message_prefix": "",
+    "role_assistant_message_suffix": "",
+    "role_assistant_prefix":         "<|start_header_id|>",
+    "role_assistant_suffix":         "<|end_header_id|>",
+    "eom_text":                      "<|eot_id|>",
+    "display_role_names":            true,
+    "max_tokens_param_name":         "max_gen_len",
+    "max_supported_response_tokens": 2048,
+    "response_chunk_element":        "generation"
+},
+```
+
+### Phase 2: Verification Testing
+**Goal:** Verify both models work correctly with existing tests
+**Dependencies:** Phase 1 complete
+
+- [x] Task 2.1: Run `npm run test` - verify both models pass streaming and non-streaming tests
+- [x] Task 2.2: Run `npm run test-vision` - verify vision capability works for both models
+- [x] Task 2.3: Run `npm run test:converse` - verify Converse API works
+- [x] Task 2.4: Review output files for any errors or warnings
+
+## 6. Risks and Mitigations
+
+| Risk | Likelihood | Impact | Mitigation |
+|------|------------|--------|------------|
+| Cross-region profile ID not yet available | Low | Medium | Single-region ID included as commented fallback |
+| Llama 4 uses different special tokens | Low | High | Test both APIs; adjust tokens if tests fail |
+| Vision handling differs from expected | Low | Medium | Existing Llama image format in codebase; Converse API as fallback |
+| Model not available in user's region | Medium | Low | Document supported regions (us-east-1, us-east-2, us-west-1, us-west-2) |
+
+## 7. Success Criteria
+
+- [x] Both models appear in `listBedrockWrapperSupportedModels()` output
+- [x] `npm run test` passes for both Llama 4 models (streaming & non-streaming)
+- [x] `npm run test-vision` includes and passes for both models
+- [x] Both Invoke API and Converse API paths work correctly
+- [x] No regression in existing model tests
+
+## 8. Open Questions
+
+None - all requirements clarified during planning.
+
+## 9. Assumptions
+
+- AWS Bedrock cross-region inference profiles are available for Llama 4 models (following pattern `us.meta.llama4-*`)
+- Llama 4 models use the same special tokens as Llama 3.x family
+- Vision handling follows the existing Llama image format in the codebase
+- Max response token limit of 2048 is appropriate (same as other Llama models)
+
+## 10. Completion Summary
+
+**Completed:** 2026-01-08 | **Completion Rate:** 100% (7/7 tasks)
+
+### What Was Built
+Added support for two new AWS Bedrock models: Llama 4 Scout 17B and Llama 4 Maverick 17B. These are the first Llama models in the wrapper with vision/multimodal support enabled. Both models work with Invoke API and Converse API paths, streaming and non-streaming modes.
+
+### Files Modified
+| File | Changes |
+|------|---------|
+| `bedrock-models.js` | Added Llama-4-Scout-17b config (lines 550-577), Llama-4-Maverick-17b config (lines 578-605) |
+
+### Test Results
+- `npm run test --both`: Both models pass all 4 test modes (Invoke/Converse × Streaming/Non-streaming)
+- `npm run test-vision --both`: Both models successfully describe images
+- `npm run test:converse`: Both models pass Converse API tests
+
+### Known Limitations
+- Llama 4 models do not support stop sequences (AWS Bedrock limitation, consistent with other Llama models)