@git.zone/tstest 3.6.7 → 4.1.1

package/readme.plan.md

@@ -1,321 +0,0 @@
-# Improvement Plan for tstest and tapbundle
-
-!! FIRST: Reread /home/philkunz/.claude/CLAUDE.md to ensure following all guidelines !!
-
-## Improved Internal Protocol (NEW - Critical) ✅ COMPLETED
-
-### Current Issues ✅ RESOLVED
-- ✅ TAP protocol uses `#` for metadata which conflicts with test descriptions containing `#`
-- ✅ Fragile regex parsing that breaks with special characters
-- ✅ Limited extensibility for new metadata types
-
-### Proposed Solution: Protocol V2 ✅ IMPLEMENTED
-- ✅ Use Unicode delimiters `⟦TSTEST:META:{}⟧` that won't appear in test names
-- ✅ Structured JSON metadata format
-- ✅ Separate protocol blocks for complex data (errors, snapshots)
-- ✅ Complete replacement of v1 (no backwards compatibility needed)
-
-### Implementation ✅ COMPLETED
-- ✅ Phase 1: Create protocol v2 implementation in ts_tapbundle_protocol
-- ✅ Phase 2: Replace all v1 code in both tstest and tapbundle with v2
-- ✅ Phase 3: Delete all v1 parsing and generation code
-
-#### ts_tapbundle_protocol Directory
-The protocol v2 implementation will be contained in the `ts_tapbundle_protocol` directory as isomorphic TypeScript code:
-- **Isomorphic Design**: All code must work in both browser and Node.js environments
-- **No Node.js Imports**: No Node.js-specific modules allowed (no fs, path, child_process, etc.)
-- **Protocol Classes**: Contains classes implementing all sides of the protocol:
-  - ✅ `ProtocolEmitter`: For generating protocol v2 messages (used by tapbundle)
-  - ✅ `ProtocolParser`: For parsing protocol v2 messages (used by tstest)
-  - ✅ `ProtocolMessage`: Base classes for different message types
-  - ✅ `ProtocolTypes`: TypeScript interfaces and types for protocol structures
-- **Pure TypeScript**: Only browser-compatible APIs and pure TypeScript/JavaScript code
-- **Build Integration**: 
-  - Compiled by `pnpm build` (via tsbuild) to `dist_ts_tapbundle_protocol/`
-  - Build order defined in tspublish.json files
-  - Imported by ts and ts_tapbundle modules from the compiled dist directory
-
-See `readme.protocol.md` for detailed specification.
-
-## Test Configuration System (NEW)
-
-### Global Test Configuration via 00init.ts
-- **Discovery**: Check for `test/00init.ts` before running tests
-- **Execution**: Import and execute before any test files if found
-- **Purpose**: Define project-wide default test settings
-
-### tap.settings() API
-```typescript
-interface TapSettings {
-  // Timing
-  timeout?: number;              // Default timeout for all tests (ms)
-  slowThreshold?: number;        // Mark tests as slow if they exceed this (ms)
-  
-  // Execution Control
-  bail?: boolean;                // Stop on first test failure
-  retries?: number;              // Number of retries for failed tests
-  retryDelay?: number;           // Delay between retries (ms)
-  
-  // Output Control
-  suppressConsole?: boolean;     // Suppress console output in passing tests
-  verboseErrors?: boolean;       // Show full stack traces
-  showTestDuration?: boolean;    // Show duration for each test
-  
-  // Parallel Execution
-  maxConcurrency?: number;       // Max parallel tests (for .para files)
-  isolateTests?: boolean;        // Run each test in fresh context
-  
-  // Lifecycle Hooks
-  beforeAll?: () => Promise<void> | void;
-  afterAll?: () => Promise<void> | void;
-  beforeEach?: (testName: string) => Promise<void> | void;
-  afterEach?: (testName: string, passed: boolean) => Promise<void> | void;
-  
-  // Environment
-  env?: Record<string, string>;  // Additional environment variables
-  
-  // Features
-  enableSnapshots?: boolean;     // Enable snapshot testing
-  snapshotDirectory?: string;    // Custom snapshot directory
-  updateSnapshots?: boolean;     // Update snapshots instead of comparing
-}
-```
-
-### Settings Inheritance
-- Global (00init.ts) → File level → Test level
-- More specific settings override less specific ones
-- Arrays/objects are merged, primitives are replaced
-
-### Implementation Phases
-1. **Core Infrastructure**: Settings storage and merge logic
-2. **Discovery**: 00init.ts loading mechanism
-3. **Application**: Apply settings to test execution
-4. **Advanced**: Parallel execution and snapshot configuration
-
-## 1. Enhanced Communication Between tapbundle and tstest ✅ COMPLETED
-
-### 1.1 Real-time Test Progress API ✅ COMPLETED
-- ✅ Create a bidirectional communication channel between tapbundle and tstest
-- ✅ Emit events for test lifecycle stages (start, progress, completion)
-- ✅ Allow tstest to subscribe to tapbundle events for better progress reporting
-- ✅ Implement a standardized message format for test metadata
-
-### 1.2 Rich Error Reporting ✅ COMPLETED
-- ✅ Pass structured error objects from tapbundle to tstest
-- ✅ Include stack traces, code snippets, and contextual information
-- ✅ Support for error categorization (assertion failures, timeouts, uncaught exceptions)
-- ✅ Visual diff output for failed assertions
-
-## 2. Enhanced toolsArg Functionality
-
-### 2.3 Test Data and Context Sharing (Partial)
-```typescript
-tap.test('data-driven test', async (toolsArg) => {
-  // Parameterized test data (not yet implemented)
-  const testData = toolsArg.data<TestInput>();
-  expect(processData(testData)).toEqual(expected);
-});
-```
-
-## 3. Nested Tests and Test Suites
-
-### 3.2 Hierarchical Test Organization (Not yet implemented)
-- Support for multiple levels of nesting
-- Inherited context and configuration from parent suites
-- Aggregated reporting for test suites
-- Suite-level lifecycle hooks
-
-## 4. Advanced Test Features
-
-### 4.1 Snapshot Testing ✅ (Basic implementation complete)
-
-### 4.2 Performance Benchmarking
-```typescript
-tap.test('performance test', async (toolsArg) => {
-  const benchmark = toolsArg.benchmark();
-  
-  // Run operation
-  await expensiveOperation();
-  
-  // Assert performance constraints
-  benchmark.expect({
-    maxDuration: 1000,
-    maxMemory: '100MB'
-  });
-});
-```
-
-
-## 5. Test Execution Improvements
-
-
-### 5.2 Watch Mode ✅ COMPLETED
-- Automatically re-run tests on file changes
-- Debounced file change detection (300ms)
-- Clear console output between runs
-- Shows which files triggered re-runs
-- Graceful exit with Ctrl+C
-- `--watch-ignore` option for excluding patterns
-
-### 5.3 Advanced Test Filtering (Partial) ⚠️
-```typescript
-// Exclude tests by pattern (not yet implemented)
-tstest --exclude "**/slow/**"
-
-// Run only failed tests from last run (not yet implemented)
-tstest --failed
-
-// Run tests modified in git (not yet implemented)
-tstest --changed
-```
-
-## 6. Reporting and Analytics
-
-### 6.1 Custom Reporters
-- Plugin architecture for custom reporters
-- Built-in reporters: JSON, JUnit, HTML, Markdown
-- Real-time streaming reporters
-- Aggregated test metrics and trends
-
-### 6.2 Coverage Integration
-- Built-in code coverage collection
-- Coverage thresholds and enforcement
-- Coverage trending over time
-- Integration with CI/CD pipelines
-
-### 6.3 Test Analytics Dashboard
-- Web-based dashboard for test results
-- Historical test performance data
-- Flaky test detection
-- Test impact analysis
-
-## 7. Developer Experience
-
-### 7.1 Better Error Messages
-- Clear, actionable error messages
-- Suggestions for common issues
-- Links to documentation
-- Code examples in error output
-
-## Implementation Phases
-
-### Phase 1: Improved Internal Protocol (Priority: Critical) ✅ COMPLETED
-1. ✅ Create ts_tapbundle_protocol directory with isomorphic protocol v2 implementation
-   - ✅ Implement ProtocolEmitter class for message generation
-   - ✅ Implement ProtocolParser class for message parsing
-   - ✅ Define ProtocolMessage types and interfaces
-   - ✅ Ensure all code is browser and Node.js compatible
-   - ✅ Add tspublish.json to configure build order
-2. ✅ Update build configuration to compile ts_tapbundle_protocol first
-3. ✅ Replace TAP parser in tstest with Protocol V2 parser importing from dist_ts_tapbundle_protocol
-4. ✅ Replace TAP generation in tapbundle with Protocol V2 emitter importing from dist_ts_tapbundle_protocol
-5. ✅ Delete all v1 TAP parsing code from tstest
-6. ✅ Delete all v1 TAP generation code from tapbundle
-7. ✅ Test with real-world test suites containing special characters
-
-### Phase 2: Test Configuration System (Priority: High) ✅ COMPLETED
-1. ✅ Implement tap.settings() API with TypeScript interfaces
-2. ✅ Add 00init.ts discovery and loading mechanism
-3. ✅ Implement settings inheritance and merge logic
-4. ✅ Apply settings to test execution (timeouts, retries, etc.)
-
-### Phase 3: Enhanced Communication (Priority: High) ✅ COMPLETED
-1. ✅ Build on Protocol V2 for richer communication
-2. ✅ Implement real-time test progress API
-3. ✅ Add structured error reporting with diffs and traces
-
-### Phase 4: Developer Experience (Priority: Medium) ❌ NOT STARTED
-1. Add watch mode
-2. Implement custom reporters
-3. Complete advanced test filtering options
-4. Add performance benchmarking API
-
-### Phase 5: Analytics and Performance (Priority: Low) ❌ NOT STARTED
-1. Build test analytics dashboard
-2. Implement coverage integration
-3. Create trend analysis tools
-4. Add test impact analysis
-
-## Technical Considerations
-
-### API Design Principles
-- Clean, modern API design without legacy constraints
-- Progressive enhancement approach
-- Well-documented features and APIs
-- Clear, simple interfaces
-
-### Performance Goals
-- Minimal overhead for test execution
-- Efficient parallel execution
-- Fast test discovery
-- Optimized browser test bundling
-
-### Integration Points
-- Clean interfaces between tstest and tapbundle
-- Extensible plugin architecture
-- Standard test result format
-- Compatible with existing CI/CD tools
-
-## Summary of Remaining Work
-
-### ✅ Completed
-- **Protocol V2**: Full implementation with Unicode delimiters, structured metadata, and special character handling
-- **Test Configuration System**: tap.settings() API, 00init.ts discovery, settings inheritance, lifecycle hooks
-- **Enhanced Communication**: Event-based test lifecycle reporting, visual diff output for assertion failures, real-time test progress API
-- **Rich Error Reporting**: Stack traces, error metadata, and visual diffs through protocol
-- **Tags Filtering**: `--tags` option for running specific tagged tests
-
-### ✅ Existing Features (Not in Plan)
-- **Timeout Support**: `--timeout` option and per-test timeouts
-- **Test Retries**: `tap.retry()` for flaky test handling
-- **Parallel Tests**: `.testParallel()` for concurrent execution
-- **Snapshot Testing**: Basic implementation with `toMatchSnapshot()`
-- **Test Lifecycle**: `describe()` blocks with `beforeEach`/`afterEach`
-- **Skip Tests**: `tap.skip.test()` (though it doesn't create test objects)
-- **Log Files**: `--logfile` option saves output to `.nogit/testlogs/`
-- **Test Range**: `--startFrom` and `--stopAt` for partial runs
-
-### ⚠️ Partially Completed
-- **Advanced Test Filtering**: Have `--tags` but missing `--exclude`, `--failed`, `--changed`
-
-### ❌ Not Started
-
-#### High Priority
-
-#### Medium Priority
-2. **Developer Experience**
-   - Watch mode for file changes
-   - Custom reporters (JSON, JUnit, HTML, Markdown)
-   - Performance benchmarking API
-   - Better error messages with suggestions
-
-3. **Enhanced toolsArg**
-   - Test data injection
-   - Context sharing between tests
-   - Parameterized tests
-
-4. **Test Organization**
-   - Hierarchical test suites
-   - Nested describe blocks
-   - Suite-level lifecycle hooks
-
-#### Low Priority
-5. **Analytics and Performance**
-   - Test analytics dashboard
-   - Code coverage integration
-   - Trend analysis
-   - Flaky test detection
-
-### Recently Fixed Issues ✅
-- **tap.todo()**: Now fully implemented with test object creation
-- **tap.skip.test()**: Now creates test objects and maintains accurate test count
-- **tap.only.test()**: Works correctly - when .only tests exist, only those run
-
-### Remaining Minor Issues
-- **Protocol Output**: Some protocol messages still appear in console output
-
-### Next Recommended Steps
-1. Add Watch Mode (Phase 4) - high developer value for fast feedback
-2. Implement Custom Reporters - important for CI/CD integration  
-3. Implement performance benchmarking API
-4. Add better error messages with suggestions

package/readme.protocol.md

@@ -1,287 +0,0 @@
-# Improved Internal Protocol Design
-
-## Current Issues with TAP Protocol
-
-1. **Delimiter Conflict**: Using `#` for metadata conflicts with test descriptions containing `#`
-2. **Ambiguous Parsing**: No clear boundary between test name and metadata
-3. **Limited Extensibility**: Adding new metadata requires regex changes
-4. **Mixed Concerns**: Protocol data mixed with human-readable output
-
-## Proposed Internal Protocol v2
-
-### Design Principles
-
-1. **Clear Separation**: Protocol data must be unambiguously separated from user content
-2. **Extensibility**: Easy to add new metadata without breaking parsers
-3. **Backwards Compatible**: Can coexist with standard TAP for gradual migration
-4. **Machine Readable**: Structured format for reliable parsing
-5. **Human Friendly**: Still readable in raw form
-
-### Protocol Options
-
-#### Option 1: Special Delimiters
-```
-ok 1 - test description ::TSTEST:: {"time":123,"retry":0}
-not ok 2 - another test ::TSTEST:: {"time":45,"error":"timeout"}
-ok 3 - skipped test ::TSTEST:: {"time":0,"skip":"not ready"}
-```
-
-**Pros**: 
-- Simple to implement
-- Backwards compatible with TAP parsers (they ignore the suffix)
-- Easy to parse with split()
-
-**Cons**: 
-- Still could conflict if test name contains `::TSTEST::`
-- Not standard TAP
-
-#### Option 2: Separate Metadata Lines
-```
-ok 1 - test description
-::METADATA:: {"test":1,"time":123,"retry":0}
-not ok 2 - another test  
-::METADATA:: {"test":2,"time":45,"error":"timeout"}
-```
-
-**Pros**:
-- Complete separation of concerns
-- No chance of conflicts
-- Can include arbitrary metadata
-
-**Cons**:
-- Requires correlation between lines
-- More complex parsing
-
-#### Option 3: YAML Blocks (TAP 13 Compatible)
-```
-ok 1 - test description
-  ---
-  time: 123
-  retry: 0
-  ...
-not ok 2 - another test
-  ---
-  time: 45
-  error: timeout
-  stack: |
-    Error: timeout
-      at Test.run (test.js:10:5)
-  ...
-```
-
-**Pros**:
-- Standard TAP 13 feature
-- Structured data format
-- Human readable
-- Extensible
-
-**Cons**:
-- More verbose
-- YAML parsing overhead
-
-#### Option 4: Binary Protocol Markers (Recommended)
-```
-ok 1 - test description
-␛[TSTEST:eyJ0aW1lIjoxMjMsInJldHJ5IjowfQ==]␛
-not ok 2 - another test
-␛[TSTEST:eyJ0aW1lIjo0NSwiZXJyb3IiOiJ0aW1lb3V0In0=]␛
-```
-
-Using ASCII escape character (␛ = \x1B) with base64 encoded JSON.
-
-**Pros**:
-- Zero chance of accidental conflicts
-- Compact
-- Fast to parse
-- Invisible in most terminals
-
-**Cons**:
-- Not human readable in raw form
-- Requires base64 encoding/decoding
-
-### Recommended Implementation: Hybrid Approach
-
-Use multiple strategies based on context:
-
-1. **For timing and basic metadata**: Use structured delimiters
-   ```
-   ok 1 - test name ⟦time:123,retry:0⟧
-   ```
-
-2. **For complex data (errors, snapshots)**: Use separate protocol lines
-   ```
-   ok 1 - test failed
-   ⟦TSTEST:ERROR⟧
-   {"message":"Assertion failed","stack":"...","diff":"..."}
-   ⟦/TSTEST:ERROR⟧
-   ```
-
-3. **For human-readable output**: Keep standard TAP comments
-   ```
-   # Test suite: User Authentication
-   ok 1 - should login
-   ```
-
-### Implementation Plan
-
-#### Phase 1: Parser Enhancement
-1. Add new protocol parser alongside existing TAP parser
-2. Support both old and new formats during transition
-3. Add protocol version negotiation
-
-#### Phase 2: Metadata Structure
-```typescript
-interface TestMetadata {
-  // Timing
-  time: number;           // milliseconds
-  startTime?: number;     // Unix timestamp
-  endTime?: number;       // Unix timestamp
-  
-  // Status
-  skip?: string;          // skip reason
-  todo?: string;          // todo reason
-  retry?: number;         // retry attempt
-  maxRetries?: number;    // max retries allowed
-  
-  // Error details
-  error?: {
-    message: string;
-    stack?: string;
-    diff?: string;
-    actual?: any;
-    expected?: any;
-  };
-  
-  // Test context
-  file?: string;          // source file
-  line?: number;          // line number
-  column?: number;        // column number
-  
-  // Custom data
-  tags?: string[];        // test tags
-  custom?: Record<string, any>;
-}
-```
-
-#### Phase 3: Protocol Messages
-
-##### Success Message
-```
-ok 1 - user authentication works
-⟦TSTEST:META:{"time":123,"tags":["auth","unit"]}⟧
-```
-
-##### Failure Message
-```
-not ok 2 - login fails with invalid password
-⟦TSTEST:META:{"time":45,"retry":1,"maxRetries":3}⟧
-⟦TSTEST:ERROR⟧
-{
-  "message": "Expected 401 but got 500",
-  "stack": "Error: Expected 401 but got 500\n    at Test.run (auth.test.ts:25:10)",
-  "actual": 500,
-  "expected": 401
-}
-⟦/TSTEST:ERROR⟧
-```
-
-##### Skip Message
-```
-ok 3 - database integration test ⟦TSTEST:SKIP:No database connection⟧
-```
-
-##### Snapshot Communication
-```
-⟦TSTEST:SNAPSHOT:user-profile⟧
-{
-  "name": "John Doe",
-  "email": "john@example.com",
-  "roles": ["user", "admin"]
-}
-⟦/TSTEST:SNAPSHOT⟧
-```
-
-### Migration Strategy
-
-1. **Version Detection**: First line indicates protocol version
-   ```
-   ⟦TSTEST:PROTOCOL:2.0⟧
-   TAP version 13
-   ```
-
-2. **Gradual Rollout**:
-   - v1.10: Add protocol v2 parser, keep v1 generator
-   - v1.11: Generate v2 by default, v1 with --legacy flag  
-   - v2.0: Remove v1 support
-
-3. **Feature Flags**:
-   ```typescript
-   tap.settings({
-     protocol: 'v2',        // or 'v1', 'auto'
-     protocolFeatures: {
-       structuredErrors: true,
-       enhancedTiming: true,
-       binaryMarkers: false
-     }
-   });
-   ```
-
-### Benefits of New Protocol
-
-1. **Reliability**: No more regex fragility or description conflicts
-2. **Performance**: Faster parsing with clear boundaries
-3. **Extensibility**: Easy to add new metadata fields
-4. **Debugging**: Rich error information with stack traces and diffs
-5. **Integration**: Better IDE and CI/CD tool integration
-6. **Forward Compatible**: Room for future enhancements
-
-### Example Parser Implementation
-
-```typescript
-class ProtocolV2Parser {
-  private readonly MARKER_START = '⟦TSTEST:';
-  private readonly MARKER_END = '⟧';
-  
-  parseMetadata(line: string): TestMetadata | null {
-    const start = line.lastIndexOf(this.MARKER_START);
-    if (start === -1) return null;
-    
-    const end = line.indexOf(this.MARKER_END, start);
-    if (end === -1) return null;
-    
-    const content = line.substring(start + this.MARKER_START.length, end);
-    const [type, data] = content.split(':', 2);
-    
-    switch (type) {
-      case 'META':
-        return JSON.parse(data);
-      case 'SKIP':
-        return { skip: data };
-      case 'TODO':
-        return { todo: data };
-      default:
-        return null;
-    }
-  }
-  
-  parseTestLine(line: string): ParsedTest {
-    // First extract any metadata
-    const metadata = this.parseMetadata(line);
-    
-    // Then parse the TAP part (without metadata)
-    const cleanLine = this.removeMetadata(line);
-    const tapResult = this.parseTAP(cleanLine);
-    
-    return { ...tapResult, metadata };
-  }
-}
-```
-
-### Next Steps
-
-1. Implement proof of concept with basic metadata support
-2. Test with real-world test suites for edge cases
-3. Benchmark parsing performance
-4. Get feedback from users
-5. Finalize protocol specification
-6. Implement in both tapbundle and tstest