testchimp-runner-core 0.0.35 → 0.0.37

package/plandocs/journey-id-tracking-addendum.md

@@ -1,227 +0,0 @@
-# Journey ID Tracking for Prompt-Based Explorations
-
-## Journey Identification Strategy
-
-### Test-Based Journeys
-- **Journey ID**: Test ID (from the test being executed)
-- **Journey Name**: Test name (from test metadata)
-- Journeys already exist in the database
-
-### Prompt-Based Journeys
-- **Journey ID**: UUID generated by AppExplorer when creating the journey
-- **Journey Name**: Derived from the journey focus prompt
-- Journeys must be created and stored before execution
-
----
-
-## Implementation Changes to AppExplorer
-
-### Add Journey Creation Methods
-
-**File**: `scriptservice/workers/app-explorer.ts`
-
-Add helper methods for creating prompt-based journeys:
-
-```typescript
-/**
- * Generate a descriptive journey name from the focus prompt
- */
-private generateJourneyName(journeyFocusPrompt: string): string {
-  // Convert prompt to concise journey name
-  // Examples:
-  //   "Explore Dashboard and test all widgets" → "Dashboard Widgets Exploration"
-  //   "Navigate to Settings and test user profile" → "Settings User Profile Test"
-  
-  // Simple implementation: truncate and clean
-  let name = journeyFocusPrompt
-    .replace(/^(Explore|Test|Navigate to|Check|Verify)\s+/i, '')  // Remove action verbs
-    .substring(0, 80)  // Limit length
-    .trim();
-  
-  // Capitalize first letter
-  name = name.charAt(0).toUpperCase() + name.slice(1);
-  
-  return name;
-}
-
-/**
- * Create and store a prompt-based journey in database
- * Uses scriptservice's own journey service (ORM/repository layer)
- */
-private async createPromptBasedJourney(
-  journeyId: string,
-  journeyName: string,
-  journeyFocusPrompt: string
-): Promise<void> {
-  try {
-    // Use scriptservice's journey service (ORM/repository)
-    // Import: import { saveJourney } from '../services/journey-service';
-    // (or whatever the actual journey service method is called in scriptservice)
-    
-    await saveJourney({
-      id: journeyId,
-      name: journeyName,
-      description: journeyFocusPrompt,  // Store the full prompt as description
-      projectId: this.task.projectId,
-      type: 'PROMPT_BASED_EXPLORATION',  // Type already exists in schema
-      explorationId: this.explorationId,  // Link to parent exploration
-      createdAt: new Date()
-    });
-    
-    logger.info(`Created prompt-based journey: ${journeyId} - ${journeyName}`);
-  } catch (error) {
-    logger.error(`Failed to create journey ${journeyId}: ${error}`);
-    throw error;
-  }
-}
-```
-
-### Update runSinglePromptBasedJourney
-
-```typescript
-/**
- * Execute a single prompt-based journey with given focus
- */
-private async runSinglePromptBasedJourney(
-  journeyFocusPrompt: string,
-  testDataPrompt?: string
-): Promise<void> {
-  // Generate journey ID and name for prompt-based journey
-  const journeyId = uuidv4();  // NEW: Generate UUID for this journey
-  const journeyName = this.generateJourneyName(journeyFocusPrompt);  // NEW: Derive name from prompt
-  
-  logger.info(`Creating prompt-based journey: ${journeyId} - ${journeyName}`);
-  
-  // Create and store journey in database via journey service
-  await this.createPromptBasedJourney(journeyId, journeyName, journeyFocusPrompt);
-  
-  // Create journey task
-  const task: JourneyTask = {
-    invocationId: uuidv4(),
-    invocationBatchId: this.explorationId,
-    projectId: this.task.projectId,
-    appReleaseId: this.config.webappReleaseVersion ?? "default",
-    autohealEnabled: false,
-    sessionRecordApiKey: this.task.sessionRecordApiKey,
-    ingressEndpoint: await ConfigService.get('test_service_endpoint'),
-    enableTestchimpSdkOnExec: false,
-    urlRegexToCapture: ".*",
-    playwrightConfig: this.config.playwrightConfig,
-    creditBudget: this.getCreditsForNextJourney(),
-    bugCaptureSettings: this.config.bugCaptureSettings,
-    viewportConfig: this.config.viewportConfig,
-    
-    // NEW: Journey identification for prompt-based
-    testId: journeyId,         // Use generated journey ID (as testId for compatibility)
-    testName: {                // Use generated journey name
-      suite: 'Prompt-Based Exploration',
-      name: journeyName
-    },
-    
-    // Enable journey exploration mode with focused prompt
-    journeyExplorationMode: {
-      enabled: true,
-      journeyFocusPrompt,       // Journey-specific focus
-      testDataPrompt
-    }
-  };
-  
-  // Use JourneyRunner (handles both test-based and prompt-based)
-  const runner = new JourneyRunner(
-    task,
-    this.onBugsDiscovered.bind(this),
-    this.onScreenStateVisited.bind(this),
-    // ... all other callbacks
-  );
-  
-  await runner.run();
-}
-```
-
----
-
-## Database Schema (No Changes Needed)
-
-### Journey Entity
-
-The journey table **already supports** both test-based and prompt-based journeys (from agentservice implementation). We're simply reusing the existing structure:
-
-```typescript
-interface Journey {
-  id: string;                      // UUID (test ID for test-based, generated for prompt-based)
-  name: string;                    // Human-readable name
-  description?: string;            // Full prompt for prompt-based, test description for test-based
-  projectId: string;
-  type: 'TEST_BASED' | 'PROMPT_BASED_EXPLORATION';  // Already exists
-  createdAt: Date;
-  explorationId?: string;          // Link to parent exploration
-  // ... other existing fields
-}
-```
-
-**No schema changes required** - we'll use existing journey service (ORM/repository) that scriptservice already has.
-
----
-
-## Flow Comparison
-
-### Test-Based Journey Flow
-
-```
-AppExplorer (test-based)
-  ↓
-  Select test from test tree
-  ↓
-  Create JourneyTask with:
-    - testId: <existing test ID>
-    - testName: <existing test name>
-    - journeyExplorationMode: disabled
-  ↓
-  JourneyRunner executes test steps
-```
-
-### Prompt-Based Journey Flow
-
-```
-AppExplorer (prompt-based)
-  ↓
-  LLM generates journey focus prompt
-  ↓
-  Generate journeyId (UUID)
-  Generate journeyName from prompt
-  ↓
-  Create & store journey via journey service (ORM)
-  ↓
-  Create JourneyTask with:
-    - testId: <generated journey ID>
-    - testName: <generated journey name>
-    - journeyExplorationMode: enabled
-    - journeyFocusPrompt: <specific focus>
-  ↓
-  JourneyRunner executes autonomously via orchestrator
-```
-
----
-
-## Benefits of This Approach
-
-1. **Unified Tracking**: Both test-based and prompt-based journeys use the same journey table and reporting infrastructure
-2. **Traceability**: Each prompt-based journey has a unique ID for bug tracking, screen state linking, execution history
-3. **Naming Consistency**: Generated journey names are descriptive and help understand exploration progress
-4. **No Schema Changes**: Reuses existing journey infrastructure - same table, same ORM service
-5. **No Cross-Service Imports**: Uses scriptservice's own journey service (ORM layer) - no dependencies on agentservice (Python)
-6. **Consolidation**: Moves prompt-based exploration from agentservice to scriptservice using runner-core orchestrator
-7. **Code Deduplication**: Removes browser-use dependency, uses same orchestrator for both test-based and prompt-based modes
-
----
-
-## Example Journey Names Generated
-
-| Journey Focus Prompt | Generated Journey Name |
-|---------------------|----------------------|
-| "Explore Dashboard and test all widgets" | "Dashboard and test all widgets" |
-| "Navigate to Settings and test user profile management" | "Settings and test user profile management" |
-| "Test all menu options in the navigation bar" | "all menu options in the navigation bar" |
-| "Explore admin panel features and permissions" | "admin panel features and permissions" |
-
-(The `generateJourneyName` method can be enhanced with better NLP to create more concise names)

package/releasenotes/RELEASE_0.0.26.md

@@ -1,165 +0,0 @@
-# Release 0.0.26 - Fix Orchestrator Reasoning Logs
-
-## Summary
-Fixed bug where orchestrator reasoning logs (💭 REASONING, 🧠 SELF-REFLECTION, etc.) were not appearing in VS Code output channel due to outputChannel being lost during service recreation.
-
-## Bug Fixed
-
-### The Problem
-Orchestrator reasoning logs were only showing in `console.log()` (debug console), not in:
-- ❌ VS Code "TestChimp Agent" output channel
-- ❌ Log files
-
-Users couldn't see what the agent was thinking!
-
-**Root Cause:**
-When `setAuthConfig()` was called, it recreated `ScenarioService`, which lost the `outputChannel` reference. Workers were then created without access to the output channel, causing all orchestrator logs to fall back to `console.log()` only.
-
-**Call sequence that caused the bug:**
-1. `embeddedService.initialize()` → calls `service.setOutputChannel()` ✅
-2. Then calls `service.setAuthConfig()` which:
-   - Recreates `ScenarioService` ❌ outputChannel lost!
-   - Calls `scenarioService.initialize()` → creates workers without outputChannel
-3. Orchestrator logs → `console.log()` only (not visible in output channel)
-
-## Solution
-
-### Changes in runner-core
-
-**TestChimpService (`index.ts`):**
-
-1. **Store outputChannel** in private field:
-   ```typescript
-   private outputChannel?: any;  // Store outputChannel to preserve it across service recreations
-   ```
-
-2. **Preserve it in setOutputChannel()**:
-   ```typescript
-   setOutputChannel(outputChannel: any): void {
-     this.outputChannel = outputChannel;  // Store for future use
-     if (typeof this.scenarioService?.setOutputChannel === 'function') {
-       this.scenarioService.setOutputChannel(outputChannel);
-     }
-   }
-   ```
-
-3. **Reapply after service recreation** (in both `setAuthConfig()` and `setBackendUrl()`):
-   ```typescript
-   // Reapply outputChannel if we have one (critical for orchestrator logs)
-   if (this.outputChannel) {
-     this.scenarioService.setOutputChannel(this.outputChannel);
-   }
-   ```
-
-### Result
-
-Now orchestrator logs flow correctly:
-```
-Orchestrator (runner-core)
-  ↓ this.logger?.()
-ScenarioWorker.log()
-  ↓ outputChannel.appendLine()
-VS Code Output Channel ✅
-  ↓ dual-write
-Log File ✅
-```
-
-## What Now Appears in Output Channel
-
-**All Environments:**
-- ✅ `💭 REASONING: [agent's reasoning]`
-- ✅ `🧠 SELF-REFLECTION: [guidance for next iteration]`
-- ✅ `💡 Why: [command reasoning]`
-- ✅ `🔧 TOOLS: [tool usage]`
-- ✅ `🚧 BLOCKER: [blocker detection]`
-- ✅ `📝 COMMANDS: [commands to execute]`
-- ✅ `✓ [success messages]`
-- ✅ `✗ [failure messages]`
-
-**Development Only:**
-- ✅ `💰 Reporting token usage: X + Y`
-
-**Staging/Production (Suppressed from channel, but in file):**
-- ❌ Token usage logs (still tracked, just not displayed)
-
-## Testing the Fix
-
-To verify reasoning logs appear:
-1. Run test generation in VS Code
-2. Check "TestChimp Agent" output channel
-3. Look for `💭 REASONING`, `🧠 SELF-REFLECTION`, etc.
-4. Check log file has complete information
-
-## Files Modified
-
-**runner-core:**
-- `/src/index.ts` - Store and reapply outputChannel across service recreations
-
-**vs-extension:**
-- `/package.json` - Updated to use `^0.0.26`
-
-**github-action:**
-- `/package.json` - Updated to use `^0.0.26`
-
-## Published to npm
-
-```
-✅ Published: testchimp-runner-core@0.0.26
-📦 Package Size: 245.9 kB
-📋 Registry: https://registry.npmjs.org/
-🔢 Files: 166
-```
-
-## Verification
-
-**vs-extension:**
-```
-testchimp-vs-extension@0.0.8
-└── testchimp-runner-core@0.0.26 ✅
-```
-
-**github-action:**
-```
-testchimp-runner-action@1.0.18
-└── testchimp-runner-core@0.0.26 ✅
-```
-
-## Build Status
-
-✅ **runner-core** - Published to npm  
-✅ **vs-extension** - Updated and built (492 KiB)  
-✅ **github-action** - Updated and built (1151 KiB)  
-
-## Backward Compatibility
-
-✅ Fully backward compatible
-- No API changes
-- Only fixes a bug in log routing
-- All functionality preserved
-
-## Complete Feature Set in 0.0.26
-
-This version includes all improvements made today:
-1. ✅ Semantic selector preference (getByRole, getByLabel, etc.)
-2. ✅ Playwright expect() assertions instead of manual checks
-3. ✅ Script cleanup feature (removes redundancies)
-4. ✅ Fixed comment placement in generated scripts
-5. ✅ Focused step execution (no extra assertions)
-6. ✅ Environment-aware log filtering (in vs-extension)
-7. ✅ **Orchestrator reasoning logs in output channel** (NEW in 0.0.26)
-
-## Next Steps
-
-For future service recreations, the pattern to follow:
-```typescript
-// When recreating a service, reapply stored configuration:
-if (this.logger) {
-  this.newService.setLogger(this.logger);
-}
-if (this.outputChannel) {
-  this.newService.setOutputChannel(this.outputChannel);
-}
-```
-
-This ensures configuration persists across service lifecycles.
-

package/releasenotes/RELEASE_0.0.27.md

@@ -1,236 +0,0 @@
-# Release 0.0.27 - Clean Logs and Credit Callback Architecture
-
-## Summary
-Major cleanup of logging architecture with timestamps moved to consumers, minimal initialization logs, credit callback support for server-side integration, and version read from package.json.
-
-## Changes in 0.0.27
-
-### 1. Timestamps at Consumer Level
-**Architecture Fix:**
-- ❌ Before: runner-core added timestamps
-- ✅ After: Consumer adds timestamps in their timezone
-
-**runner-core:**
-- Removed all timestamp formatting
-- Reports raw messages via callbacks
-
-**vs-extension:**
-- Added `formatLocalTimestamp()` utility function
-- Wraps outputChannel to add timestamps automatically
-- Format: `HH:MM:SS.mmm` in local timezone
-
-### 2. Minimal Initialization Logs
-**Before:**
-```
-🤖 Initializing Orchestrator Mode
-✓ Orchestrator initialized with 5 tools (DEBUG MODE)
-═══════════════════════════════════════════════════════
-🚀 RUNNER-CORE VERSION: v1.5.0-vision-preserve-values
-═══════════════════════════════════════════════════════
-Initializing Scenario worker...
-Scenario worker initialized with session: scenario_worker_XXX
-Scenario worker initialized (Orchestrator Mode) with session...
-Scenario service initialized
-```
-
-**After:**
-```
-testchimp-runner-core v0.0.27
-📋 Processing scenario: [scenario description]
-```
-
-**Changes:**
-- Single version log (reads from package.json)
-- No internal initialization details
-- Clean, minimal output
-
-### 3. Credit Callback Architecture
-**Purpose:** Allow server-side integration to update DB directly without axios calls
-
-**Implementation:**
-```typescript
-export interface CreditUsage {
-  credits: number;
-  usageReason: CreditUsageReason;
-  jobId?: string;
-  timestamp: number;
-}
-
-export type CreditUsageCallback = (usage: CreditUsage) => void | Promise<void>;
-```
-
-**Behavior:**
-1. **If callback provided** (server-side):
-   - ✅ Call callback → Direct DB update
-   - ❌ NO axios calls made
-   
-2. **If NO callback but auth configured** (client-side):
-   - ❌ No callback
-   - ✅ Makes axios call to backend API
-   
-3. **If neither** (development):
-   - Logs warning
-   - Continues without tracking
-
-**Usage:**
-```typescript
-// Server-side
-const service = new TestChimpService(
-  fileHandler, undefined, backendUrl, maxWorkers,
-  llmProvider, progressReporter, orchestratorOptions,
-  async (creditUsage) => {
-    await db.insertCreditUsage(creditUsage);  // Direct DB
-  }
-);
-
-// Client-side (vs-ext, github-action)
-const service = new TestChimpService(
-  fileHandler, authConfig, backendUrl
-  // No callback - uses axios
-);
-```
-
-### 4. Wrapped OutputChannel
-**Problem:** runner-core wrote directly to outputChannel without timestamps
-
-**Solution:** vs-extension wraps the outputChannel:
-```typescript
-const wrappedChannel = {
-  appendLine: (message) => {
-    const timestamp = formatLocalTimestamp();
-    const shouldLog = isDev || !isVerboseDebugLog;
-    if (shouldLog) {
-      this.outputChannel?.appendLine(`[${timestamp}] ${message}`);
-    }
-  }
-};
-service.setOutputChannel(wrappedChannel);
-```
-
-**Result:**
-- ✅ All logs get timestamps
-- ✅ Filtering applied
-- ✅ Consumer controls presentation
-
-### 5. Dynamic Version in build_local.sh
-**Before:**
-```bash
-cp testchimp-runner-core-0.0.22.tgz .  # Hardcoded!
-```
-
-**After:**
-```bash
-RUNNER_VERSION=$(node -p "require('./package.json').version")
-TARBALL_NAME="testchimp-runner-core-${RUNNER_VERSION}.tgz"
-cp "$PARENT_DIR/runner-core/${TARBALL_NAME}" .
-```
-
-**Benefit:** Always uses correct version, no manual updates needed
-
-## New Exports
-
-```typescript
-export {
-  // Credit usage types
-  CreditUsageCallback,
-  CreditUsage,
-  CreditUsageReason,
-  
-  // Existing exports...
-  TestChimpService,
-  // ...
-}
-```
-
-## Files Modified
-
-### runner-core
-1. `/src/scenario-worker-class.ts` - Removed timestamps, minimal logs, version from package.json
-2. `/src/scenario-service.ts` - Removed initialization logs
-3. `/src/credit-usage-service.ts` - Added callback architecture, callback-first logic
-4. `/src/index.ts` - Credit callback support, preserve across recreations
-
-### vs-extension
-1. `/src/embedded-service.ts` - `formatLocalTimestamp()` utility, wrapped outputChannel, consistent filtering
-2. `/build_local.sh` - Dynamic version detection
-3. `/package.json` - Updated to `^0.0.27`
-
-### github-action
-1. `/package.json` - Updated to `^0.0.27`
-
-## Published to npm
-
-```
-✅ Published: testchimp-runner-core@0.0.27
-📦 Package Size: ~245 kB
-📋 Registry: https://registry.npmjs.org/
-```
-
-## Benefits
-
-### Logging
-1. **Local Timezone** - Timestamps match user's clock
-2. **Clean Output** - Only essential information
-3. **Consumer Control** - Consumer decides format and filtering
-4. **No Verbose Init** - Single version log instead of 10+ lines
-
-### Credit Tracking
-1. **Server-Side** - Direct DB updates, no HTTP overhead
-2. **Client-Side** - Existing axios behavior preserved
-3. **Flexible** - Each consumer decides how to track
-4. **Observable** - Callback provides visibility
-
-### Architecture
-1. **Separation of Concerns** - Library reports, consumer presents
-2. **Environment Agnostic** - Library doesn't assume timezone/environment
-3. **Testable** - Easy to mock callbacks
-4. **Maintainable** - Clear boundaries
-
-## Migration
-
-### For All Consumers
-Update package.json:
-```json
-{
-  "dependencies": {
-    "testchimp-runner-core": "^0.0.27"
-  }
-}
-```
-
-### For Server-Side (Optional)
-Add credit callback:
-```typescript
-const service = new TestChimpService(
-  fileHandler, undefined, backendUrl, maxWorkers,
-  llmProvider, progressReporter, orchestratorOptions,
-  async (creditUsage) => {
-    await creditRepository.insert(creditUsage);
-  }
-);
-```
-
-## Backward Compatibility
-
-✅ Fully backward compatible
-- All parameters optional
-- Existing behavior preserved
-- No breaking changes
-
-## Complete Feature Set
-
-All improvements from today are now in 0.0.27:
-1. ✅ Semantic selector preference
-2. ✅ Playwright expect() assertions
-3. ✅ Script cleanup feature
-4. ✅ Fixed comment placement
-5. ✅ Focused step execution
-6. ✅ Orchestrator reasoning logs in output channel
-7. ✅ Environment-aware log filtering (consumer-side)
-8. ✅ Local timezone timestamps
-9. ✅ Clean initialization logs
-10. ✅ Credit callback architecture
-11. ✅ Version from package.json
-
-Ready for production! 🚀
-