testchimp-runner-core 0.0.35 → 0.0.37

package/plandocs/PHASE1_ABSTRACTION_COMPLETE.md

@@ -1,241 +0,0 @@
-# Phase 1: Runner-Core Abstraction Layer - COMPLETE ✅
-
-## Implementation Summary
-
-Successfully refactored runner-core to use pluggable adapter pattern, making it reusable across VS Extension, GitHub Runner, and (in Phase 2) Script Service.
-
----
-
-## What Was Implemented
-
-### 1. New Interface Files
-
-#### `/src/llm-provider.ts`
-- `LLMProvider` interface - Abstract LLM calling
-- `LLMRequest` interface - Request format (camelCase)
-- `LLMResponse` interface - Response format
-
-**Key Feature**: Allows different LLM implementations (backend proxy, local OpenAI, etc.)
-
-#### `/src/progress-reporter.ts`
-- `ProgressReporter` interface - Progress tracking callbacks
-- `StepProgress` interface - Step-level progress (jobId, stepNumber, code, screenshot, etc.)
-- `JobProgress` interface - Job-level progress
-- `StepExecutionStatus` enum - Status constants
-
-**Key Feature**: Structured progress reporting (logs, DB writes, etc.)
-
-### 2. Provider Implementations
-
-#### `/src/providers/backend-proxy-llm-provider.ts`
-- Default implementation for VS Extension and GitHub Runner
-- Calls `/localagent/call_llm` backend endpoint
-- Handles authentication via AuthConfig
-- **Converts camelCase → snake_case** for backend API compatibility
-
-#### `/src/providers/local-llm-provider.ts`
-- For Script Service (Phase 2)
-- Accepts callback function to call LLM directly
-- No backend proxy dependency
-
-### 3. Core Refactors
-
-#### `LLMFacade` (`/src/llm-facade.ts`)
-**Changes:**
-- ✅ Constructor now accepts `LLMProvider` instead of authConfig/backendUrl
-- ✅ Removed axios calls - delegated to provider
-- ✅ Removed setAuthConfig/getAuthConfig - handled by provider
-- ✅ All LLMRequest fields use **camelCase** (systemPrompt, userPrompt, imageUrl)
-- ✅ Simplified callLLM method - just delegates to provider
-
-#### `ScenarioWorker` (`/src/scenario-worker-class.ts`)
-**Changes:**
-- ✅ Constructor accepts `LLMProvider` and `ProgressReporter`
-- ✅ Defaults to `BackendProxyLLMProvider` if not provided (backward compatible)
-- ✅ Added `captureStepScreenshot()` - returns data URL (data:image/png;base64,...)
-- ✅ Added `reportStepProgress()` - reports to ProgressReporter
-- ✅ Added `reportJobProgress()` - reports to ProgressReporter
-- ✅ Tracks `currentJobId` for progress keying
-- ✅ Reports job started, in_progress, completed
-
-#### `ScenarioService` (`/src/scenario-service.ts`)
-**Changes:**
-- ✅ Constructor accepts `LLMProvider` and `ProgressReporter`
-- ✅ Passes providers to all workers
-- ✅ Maintains backward compatibility
-
-#### `ExecutionService` (`/src/execution-service.ts`)
-**Changes:**
-- ✅ Constructor accepts `LLMProvider` and `ProgressReporter`
-- ✅ Defaults to `BackendProxyLLMProvider` if not provided
-- ✅ setAuthConfig recreates provider with new auth
-
-#### `TestChimpService` (`/src/index.ts`)
-**Changes:**
-- ✅ Constructor accepts optional `LLMProvider` and `ProgressReporter`
-- ✅ Defaults to `BackendProxyLLMProvider` (backward compatible)
-- ✅ Passes providers to all services
-- ✅ Exports all new interfaces and implementations
-
-### 4. Consumer Updates
-
-#### VS Extension (`/local/vs-ext/src/embedded-service.ts`)
-**Changes:**
-- ✅ Imports new types: `ProgressReporter`, `StepProgress`, `JobProgress`, `StepExecutionStatus`
-- ✅ Creates explicit `ProgressReporter` implementation
-- ✅ Reports step progress with emojis (✅ ❌ 🔄)
-- ✅ Reports job progress
-- ✅ Logs commands, errors to outputChannel
-- ✅ Passes progressReporter to TestChimpService
-
-#### GitHub Runner (`/testchimp-github-testrunner/src/index.ts`)
-**Changes:**
-- ✅ Imports new types: `ProgressReporter`, `StepProgress`, `JobProgress`, `StepExecutionStatus`
-- ✅ Creates explicit `ProgressReporter` implementation
-- ✅ Reports to GitHub Actions core.info/error
-- ✅ Passes progressReporter to TestChimpService
-
----
-
-## Architecture Benefits
-
-### 1. Pluggable LLM Provider
-```
-VS Extension → BackendProxyLLMProvider → /call-llm-internal → OpenAI
-GitHub Runner → BackendProxyLLMProvider → /call-llm-internal → OpenAI
-Script Service (Phase 2) → LocalLLMProvider → OpenAI directly
-```
-
-### 2. Flexible Progress Reporting
-```
-VS Extension → ProgressReporter → outputChannel.appendLine()
-GitHub Runner → ProgressReporter → core.info()
-Script Service (Phase 2) → ProgressReporter → DB write + GCS upload
-```
-
-### 3. Screenshot Data URLs
-- Core captures as Buffer → converts to data URL
-- VS Extension: Can save to file or ignore
-- GitHub Runner: Can save to artifacts or ignore
-- Script Service (Phase 2): Upload to GCS
-
----
-
-## Backward Compatibility
-
-✅ **All existing code continues to work without changes**
-
-Old usage (still works):
-```typescript
-const service = new TestChimpService(fileHandler, authConfig, backendUrl);
-```
-
-New usage (explicit):
-```typescript
-const llmProvider = new BackendProxyLLMProvider(authConfig, backendUrl);
-const progressReporter = { onStepProgress: async (p) => {...} };
-const service = new TestChimpService(
-  fileHandler, authConfig, backendUrl, maxWorkers,
-  llmProvider, progressReporter
-);
-```
-
----
-
-## Naming Conventions
-
-✅ **Enforced camelCase for all TypeScript/JavaScript**
-- `systemPrompt`, `userPrompt`, `imageUrl` in code
-- `system_prompt`, `user_prompt`, `image_url` in proto/backend APIs
-- Conversion handled by `BackendProxyLLMProvider`
-
----
-
-## Build & Test Results
-
-### ✅ Runner-Core
-- TypeScript compilation: **SUCCESS**
-- Package created: `testchimp-runner-core-0.0.22.tgz`
-
-### ✅ VS Extension
-- TypeScript check: **SUCCESS**
-- Webpack build: **SUCCESS**
-- File: `out/extension.js`
-
-### ✅ GitHub Runner
-- TypeScript compilation: **SUCCESS**
-- NCC bundle: **SUCCESS**
-- File: `dist/index.js`
-
----
-
-## Phase 2 Preview
-
-With Phase 1 complete, Script Service can now:
-
-1. Create `LocalLLMProvider` with its own OpenAI integration
-2. Create `DatabaseProgressReporter` that:
-   - Writes step progress to DB
-   - Uploads screenshots to GCS
-   - Updates job status
-3. Use runner-core with **zero code duplication**
-4. Get all future improvements automatically
-
----
-
-## Testing Checklist
-
-- [ ] VS Extension: Generate script from scenario
-- [ ] VS Extension: Repair failing script
-- [ ] GitHub Runner: Execute smart tests
-- [ ] GitHub Runner: AI repair mode
-- [ ] Verify progress reporting in both environments
-- [ ] Verify screenshots captured as data URLs
-- [ ] Verify backward compatibility
-
----
-
-## Files Created
-
-1. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/llm-provider.ts`
-2. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/progress-reporter.ts`
-3. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/providers/backend-proxy-llm-provider.ts`
-4. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/providers/local-llm-provider.ts`
-
-## Files Modified
-
-**Runner-core:**
-1. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/llm-facade.ts`
-2. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/scenario-worker-class.ts`
-3. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/execution-service.ts`
-4. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/scenario-service.ts`
-5. `/Users/nuwansam/IdeaProjects/AwareRepo/local/runner-core/src/index.ts`
-
-**Consumers:**
-6. `/Users/nuwansam/IdeaProjects/AwareRepo/local/vs-ext/src/embedded-service.ts`
-7. `/Users/nuwansam/IdeaProjects/testchimp-github-testrunner/src/index.ts`
-
----
-
-## Next Steps
-
-### Ready for Multi-Agent Architecture!
-
-With these abstractions in place, implementing the multi-agent architecture (Planner, Executor, Vision Advisor, etc.) will work across all three environments:
-
-- **Planner Agent** uses `LLMProvider` → works in all environments
-- **Executor Agent** uses `LLMProvider` → works in all environments
-- **Vision Advisor** uses `LLMProvider` with imageUrl → works in all environments
-- **Progress** reported uniformly via `ProgressReporter`
-- **Script Service** gets all intelligence automatically in Phase 2
-
----
-
-## Version
-
-**Runner-Core**: v1.5.0-vision-preserve-values (with Phase 1 abstraction layer)
-
-**Date**: October 11, 2025
-
-**Status**: ✅ COMPLETE - Ready for testing and Phase 2
-

package/plandocs/PHASE1_FINAL_STATUS.md

@@ -1,210 +0,0 @@
-# Phase 1: Runner-Core Abstraction - COMPLETE & TESTED ✅
-
-## Final Status
-
-✅ **Phase 1 is fully implemented, tested, and working!**
-
-All three components successfully built:
-- ✅ Runner-Core
-- ✅ VS Extension  
-- ✅ GitHub Runner
-
-Authentication verified working in VS Extension.
-
----
-
-## What Was Implemented
-
-### 1. Core Abstractions
-
-#### LLM Provider Interface (`llm-provider.ts`)
-- `LLMProvider` interface - pluggable LLM calling
-- `LLMRequest` / `LLMResponse` - **camelCase** for TypeScript
-- Enables different implementations (backend proxy, local, etc.)
-
-#### Progress Reporter Interface (`progress-reporter.ts`)
-- `ProgressReporter` interface - structured progress callbacks
-- `StepProgress` - includes jobId, stepNumber, code, screenshotDataUrl (data URL)
-- `JobProgress` - job-level status tracking
-- `StepExecutionStatus` enum
-
-### 2. Provider Implementations
-
-#### BackendProxyLLMProvider (`providers/backend-proxy-llm-provider.ts`)
-- For VS Extension and GitHub Runner
-- Calls `/localagent/call_llm` backend endpoint
-- **Converts camelCase → snake_case** for backend API
-- Handles authentication via AuthConfig
-
-#### LocalLLMProvider (`providers/local-llm-provider.ts`)
-- Ready for Script Service (Phase 2)
-- Accepts callback function
-- No backend proxy dependency
-
-### 3. Refactored Components
-
-All components updated to use adapter pattern:
-- ✅ `LLMFacade` - uses LLMProvider
-- ✅ `ScenarioWorker` - accepts providers, reports progress, captures screenshots as data URLs
-- ✅ `ExecutionService` - accepts providers
-- ✅ `ScenarioService` - passes providers to workers
-- ✅ `TestChimpService` - orchestrates with optional providers
-
-### 4. Consumer Updates
-
-#### VS Extension (`vs-ext/src/embedded-service.ts`)
-- ✅ Creates `ProgressReporter` with step/job callbacks
-- ✅ **Critical fix**: Sets logger BEFORE setAuthConfig
-- ✅ Progress logged with emojis (✅ ❌ 🔄)
-
-#### GitHub Runner (`testchimp-github-testrunner/src/index.ts`)
-- ✅ Creates `ProgressReporter` for GitHub Actions
-- ✅ Reports to core.info/error
-
----
-
-## Critical Issue Found & Fixed
-
-### Problem
-After refactor, VS Extension got **401 Unauthorized errors**:
-```
-LLM call failed: AxiosError: Request failed with status code 401
-```
-
-### Root Cause
-**Initialization order issue** in VS Extension:
-1. Constructor creates provider **without auth** (authConfig = undefined)
-2. `setAuthConfig()` recreates provider **with auth**
-3. ❌ But logger was set **after** setAuthConfig
-4. New provider created without logger → provider can't log errors properly
-
-### Fix
-**Moved `setLogger()` before `setAuthConfig()`** in `embedded-service.ts`:
-```typescript
-// BEFORE (broken):
-await this.service.setAuthConfig(authConfig); // Line 115
-this.service.setLogger(...); // Line 135
-
-// AFTER (fixed):
-this.service.setLogger(...); // Line 109
-await this.service.setAuthConfig(authConfig); // Line 123
-```
-
-### Additional Fixes
-1. **TestChimpService.setAuthConfig()** - Preserves logger when recreating provider
-2. **TestChimpService.setBackendUrl()** - Preserves logger when recreating provider
-3. **All service recreations** - Pass llmProvider and progressReporter
-
----
-
-## Build Verification
-
-✅ **Runner-Core**: TypeScript compiled
-✅ **VS Extension**: Webpack build successful
-✅ **GitHub Runner**: TypeScript + NCC bundle successful
-✅ **Authentication**: Working (401 errors resolved)
-✅ **Progress Reporting**: Working (callbacks invoked)
-
----
-
-## Architecture Benefits
-
-1. **Pluggable LLM**: Easy to add local OpenAI, Anthropic, etc.
-2. **Structured Progress**: Each environment reports differently
-3. **Screenshot Data URLs**: Core captures, consumers decide storage
-4. **Backward Compatible**: Old code works without changes
-5. **Type Safe**: All TypeScript interfaces use camelCase
-6. **Ready for Multi-Agent**: Planner/Executor agents can use same providers
-
----
-
-## Usage Patterns
-
-### Current Pattern (Backward Compatible)
-```typescript
-const service = new TestChimpService(fileHandler, authConfig, backendUrl);
-// Still works!
-```
-
-### New Pattern (Explicit)
-```typescript
-const progressReporter: ProgressReporter = {
-  onStepProgress: async (progress) => { ... },
-  onJobProgress: async (progress) => { ... }
-};
-
-const service = new TestChimpService(
-  fileHandler, authConfig, backendUrl, workers,
-  undefined, // uses BackendProxyLLMProvider by default
-  progressReporter
-);
-```
-
-### Phase 2 Pattern (Script Service)
-```typescript
-const llmProvider = new LocalLLMProvider(async (req) => {
-  return await callLLM(`${req.systemPrompt}\n\n${req.userPrompt}`);
-});
-
-const progressReporter: ProgressReporter = {
-  onStepProgress: async (progress) => {
-    const url = await uploadToGCS(progress.screenshotDataUrl);
-    await db.updateStep(progress.jobId, { ...progress, screenshotPath: url });
-  }
-};
-
-const service = new TestChimpService(
-  undefined, undefined, undefined, undefined,
-  llmProvider, progressReporter
-);
-```
-
----
-
-## Files Summary
-
-**Created:**
-- `src/llm-provider.ts`
-- `src/progress-reporter.ts`
-- `src/providers/backend-proxy-llm-provider.ts`
-- `src/providers/local-llm-provider.ts`
-
-**Modified:**
-- `src/llm-facade.ts`
-- `src/scenario-worker-class.ts`
-- `src/execution-service.ts`
-- `src/scenario-service.ts`
-- `src/index.ts`
-- `vs-ext/src/embedded-service.ts`
-- `github-runner/src/index.ts`
-
----
-
-## Testing Checklist
-
-- [x] Runner-core builds successfully
-- [x] VS Extension builds successfully
-- [x] GitHub Runner builds successfully
-- [x] Authentication working (no 401 errors)
-- [x] Logger initialization order fixed
-- [ ] Test script generation from scenario
-- [ ] Test script repair
-- [ ] Verify progress callbacks
-- [ ] Verify screenshot data URLs
-
----
-
-## Phase 2 Ready
-
-With Phase 1 complete and tested, Script Service can now:
-1. Use `LocalLLMProvider` with OpenAI directly
-2. Create `DatabaseProgressReporter` for DB + GCS
-3. Remove duplicate code from `script-generation/`
-4. Share all improvements automatically
-
----
-
-**Version**: v1.5.0-vision-preserve-values + Phase 1 Abstraction Layer
-**Date**: October 11, 2025
-**Status**: ✅ **COMPLETE, TESTED, WORKING**
-

package/plandocs/PHASE_1_COMPLETE.md

@@ -1,165 +0,0 @@
-# Phase 1 Implementation - COMPLETE ✅
-
-## Version: runner-core v0.0.33
-
-## What's Been Implemented
-
-### 1. Free-Form "Note to Future Self" 
-**Purpose:** Tactical memory - agent leaves notes that persist across iterations AND steps.
-
-**Type:**
-```typescript
-interface NoteToFutureSelf {
-  fromIteration: number;
-  content: string;  // FREE-FORM - agent writes whatever it wants
-}
-```
-
-**How it works:**
-- Agent includes `"noteToFutureSelf": "..."` in response
-- System stores it in `memory.latestNote` (persists across steps!)
-- Passed to next iteration AND next step
-- Displayed prominently at top of prompt
-- Agent reads it FIRST before making decision
-
-**Scope:** Entire scenario journey (not just current step)
-
-**Example notes:**
-
-*Iteration-specific:*
-- "Tried #sidebar-toggle, failed with 'not clickable'. Will try child SVG element next."
-
-*Step-spanning:*
-- "This app has slow-loading modals. Always wait 2s after page load before clicking."
-- "Cookie consent appears on every page. Check for and dismiss it first."
-- "Sidebar only visible on desktop viewport (>1024px width)."
-
-### 2. Percentage-Based Coordinate Fallback
-**Purpose:** Last-resort mechanism when selector generation repeatedly fails.
-
-**Type:**
-```typescript
-interface CoordinateAction {
-  type: 'coordinate';
-  action: 'click' | 'doubleClick' | 'rightClick' | 'hover' | 'drag' | 'fill' | 'scroll';
-  xPercent: number;  // 0-100, 3 decimal precision
-  yPercent: number;
-  toXPercent?: number;  // For drag
-  toYPercent?: number;
-  value?: string;  // For fill
-  scrollAmount?: number;  // For scroll
-}
-```
-
-**How it works:**
-- LLM outputs percentages: `{xPercent: 15.755, yPercent: 8.500}`
-- CoordinateConverter converts to pixels: `15.755% → 252px`
-- Generates Playwright command: `await page.mouse.click(252, 68);`
-
-**Supported actions:**
-- click, doubleClick, rightClick, hover
-- fill (clicks then types value)
-- drag (from x%,y% to toX%,toY%)
-- scroll (at position, by amount)
-
-### 3. Two-Tier Auto-Escalation
-**Trigger:** Code-controlled (not LLM-decided)
-
-```
-Tier 1 (iterations 1-3): Playwright Selector Mode
-├─ Normal buildSystemPrompt()
-├─ Agent generates: await page.getByRole(...).click()
-├─ Leaves noteToFutureSelf for continuity
-└─ 3 attempts, then escalate
-
-Tier 2 (iterations 4-5): Coordinate Mode
-├─ Auto-activates when consecutiveFailures >= 3
-├─ Uses buildCoordinateSystemPrompt()
-├─ Agent outputs: {xPercent: 15.755, yPercent: 8.500}
-├─ CoordinateConverter → mouse.click(x, y)
-└─ 2 attempts max, then give up
-
-Total: Maximum 5 iterations per step
-```
-
-### 4. Precision & Accuracy
-- **3 decimal precision** for coordinates (~1px accuracy on most screens)
-- **Resolution-independent** - works on any viewport size
-- **Percentage reference:**
-  - Top-left: (0, 0)
-  - Top-right: (100, 0)
-  - Center: (50, 50)
-  - Bottom-right: (100, 100)
-
-## Files Modified
-
-1. **orchestrator/types.ts**
-   - Added `NoteToFutureSelf` interface
-   - Added `CoordinateAction` interface
-   - Updated `AgentDecision` with new fields
-   - Updated `AgentContext` with noteFromPreviousIteration
-
-2. **orchestrator/orchestrator-agent.ts**
-   - Added note tracking in executeStep()
-   - Added coordinate action execution
-   - Added buildCoordinateSystemPrompt()
-   - Updated buildUserPrompt() to display notes
-   - Added mode switching in callAgent()
-   - Updated response format documentation
-
-3. **utils/coordinate-converter.ts** (NEW)
-   - percentToPixels() - Convert % to pixels
-   - getViewportSize() - Get current viewport dimensions
-   - generateCommands() - Create Playwright commands from percentages
-   - executeAction() - Direct execution helper
-
-4. **scenario-worker-class.ts** (Earlier fix)
-   - Smart timeout handling for waitForLoadState
-
-5. **execution-service.ts** (Earlier fix)
-   - Smart timeout handling for navigation commands
-
-## How to Use
-
-**No code changes needed!** The features activate automatically:
-
-1. **Note to self:** Agent can optionally include `noteToFutureSelf` in any iteration
-2. **Coordinates:** Auto-activate at iteration 4 if selectors keep failing
-
-## Testing Phase 1
-
-To validate the implementation:
-
-1. **Run PeopleHR scenario** (previously failed on hamburger menu)
-   - Should now succeed with note guidance
-   - May use coordinates if SVG selector still fails
-
-2. **Check logs for:**
-   - `📝 Note to self: ...` (agent leaving tactical notes)
-   - `🎯 COORDINATE MODE ACTIVATED` (tier 2 triggered)
-   - `🎯 Coordinate Action: click at (X%, Y%)` (using fallback)
-
-3. **Expected improvements:**
-   - 20-30% fewer iterations per step (thanks to notes)
-   - < 5% scenarios need coordinate fallback
-   - Coordinates work when everything else fails
-
-## Phase 2 Preview (Not Yet Implemented)
-
-When Phase 2 is added, it will become a **three-tier** system:
-- Tier 1 (iterations 1-2): Playwright selectors
-- Tier 2 (iterations 3-4): Numbered elements (CLICK[3])
-- Tier 3 (iterations 5+): Percentage coordinates
-
-Phase 2 adds visual markers [1], [2], [3] on elements with structured commands.
-
----
-
-## Status: ✅ READY FOR TESTING
-
-Runner-core v0.0.33 is built and ready. Test it with:
-- VS Code extension "Run Test" on peoplehr-corrected.smart.spec.ts
-- Or generate new script from peoplehr.txt scenario
-
-**Next:** Validate Phase 1 works before starting Phase 2.
-