@git.zone/tstest 3.6.7 → 4.1.1

package/readme.hints.md

@@ -1,510 +0,0 @@
-# Architecture Overview
-
-## Project Structure
-
-This project integrates tstest with tapbundle through a modular architecture:
-
-1. **tstest** (`/ts/`) - The test runner that discovers and executes test files
-2. **tapbundle** (`/ts_tapbundle/`) - The TAP testing framework for writing tests
-3. **tapbundle_serverside** (`/ts_tapbundle_serverside/`) - Server-side testing utilities (network port finding, runCommand, env vars, HTTPS certs, MongoDB, S3, test assets)
-
-## Node Runtime CJS JSON Compatibility
-
-Node 24 and newer exposed a `tsx` synchronous-loader regression in versions
-4.22.0 and 4.22.1: JSON reached through CommonJS `require()` was transformed
-into JavaScript in memory, then passed to Node's CommonJS JSON parser. The
-resulting error misleadingly showed generated JavaScript such as `var ...` as
-invalid JSON.
-
-The upstream fix shipped in `tsx` 4.22.2. `tstest` requires
-`@git.zone/tsrun` 2.0.5 or newer, whose `tsx` dependency starts at a fixed
-version. The Node runtime regression test covers the ESM hook to CommonJS
-`require()` to JSON path and verifies that the fixture bytes remain unchanged.
-
-This loader failure is not evidence that `node_modules/**/db.json` was
-overwritten on disk. Any observed file hash or content change is a separate
-filesystem-writer issue and must be traced independently.
-
-## How Components Work Together
-
-### Test Execution Flow
-
-1. **CLI Entry Point** (`cli.js` � `cli.ts.js` � `cli.child.ts`)
-   - The CLI uses tsx to run TypeScript files directly
-   - Accepts glob patterns to find test files
-   - Supports options like `--verbose`, `--quiet`, `--web`
-
-2. **Test Discovery**
-   - tstest scans for test files matching the provided pattern
-   - Defaults to `test/**/*.ts` when no pattern is specified
-   - Supports both file and directory modes
-
-3. **Test Runner**
-   - Each test file imports `tap` and `expect` from tapbundle
-   - Tests are written using `tap.test()` with async functions
-   - Browser tests are compiled with esbuild and run in Chromium via Puppeteer
-
-### Key Integration Points
-
-1. **Import Structure**
-   - Test files import from local tapbundle: `import { tap, expect } from '../../ts_tapbundle/index.js'`
-   - Server-side tests also import from tapbundle_serverside for Node.js-only utilities: `import { tapNodeTools } from '../../ts_tapbundle_serverside/index.js'`
-
-2. **WebHelpers**
-   - Browser tests can use webhelpers for DOM manipulation
-   - `webhelpers.html` - Template literal for creating HTML strings
-   - `webhelpers.fixture` - Creates DOM elements from HTML strings
-   - Automatically detects browser environment and only enables in browser context
-
-3. **Build System**
-   - Uses `tsbuild tsfolders` to compile TypeScript (invoked by `pnpm build`)
-   - Maintains separate output directories: `/dist_ts/`, `/dist_ts_tapbundle/`, `/dist_ts_tapbundle_serverside/`, `/dist_ts_tapbundle_protocol/`
-   - Compilation order is resolved automatically based on dependencies in tspublish.json files
-   - Protocol imports use compiled dist directories:
-     ```typescript
-     // In ts/tstest.classes.tap.parser.ts
-     import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';
-     
-     // In ts_tapbundle/tapbundle.classes.tap.ts  
-     import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';
-     ```
-
-### Test Scripts
-
-The package.json defines several test scripts:
-- `test` - Builds and runs all tests (tapbundle and tstest)
-- `test:tapbundle` - Runs tapbundle framework tests
-- `test:tstest` - Runs tstest's own tests
-- Both support `:verbose` variants for detailed output
-
-### Environment Detection
-
-The framework automatically detects the runtime environment:
-- Node.js tests run directly via tsx
-- Browser tests are compiled and served via a local server
-- WebHelpers are only enabled in browser environment
-
-This architecture allows for seamless testing across both Node.js and browser environments while maintaining a clean separation of concerns.
-
-## Logging System
-
-### Log File Naming (Fixed in v1.9.1)
-
-When using the `--logfile` flag, tstest creates log files in `.nogit/testlogs/`. The log file naming was updated to preserve directory structure and prevent collisions:
-
-- **Old behavior**: `test/tapbundle/test.ts` → `.nogit/testlogs/test.log`
-- **New behavior**: `test/tapbundle/test.ts` → `.nogit/testlogs/test__tapbundle__test.log`
-
-This fix ensures that test files with the same basename in different directories don't overwrite each other's logs. The implementation:
-1. Takes the relative path from the current working directory
-2. Replaces path separators (`/`) with double underscores (`__`)
-3. Removes the `.ts` extension
-4. Creates a flat filename that preserves the directory structure
-
-### Test Timing Display (Fixed in v1.9.2)
-
-Fixed an issue where test timing was displayed incorrectly with duplicate values like:
-- Before: `✅ test name # time=133ms (0ms)`
-- After: `✅ test name (133ms)`
-
-The issue was in the TAP parser regex which was greedily capturing the entire line including the TAP timing comment. Changed the regex from `(.*)` to `(.*?)` to make it non-greedy, properly separating the test name from the timing metadata.
-
-## Protocol Limitations and Improvements
-
-### Current TAP Protocol Issues
-The current implementation uses standard TAP format with metadata in comments:
-```
-ok 1 - test name # time=123ms
-```
-
-This has several limitations:
-1. **Delimiter Conflict**: Test descriptions containing `#` can break parsing
-2. **Regex Fragility**: Complex regex patterns that are hard to maintain
-3. **Limited Metadata**: Difficult to add rich error information or custom data
-
-### Planned Protocol V2
-A new internal protocol is being designed that will:
-- Use Unicode delimiters `⟦TSTEST:⟧` that won't conflict with test content
-- Support structured JSON metadata
-- Allow rich error reporting with stack traces and diffs
-- Completely replace v1 protocol (no backwards compatibility)
-
-### ts_tapbundle_protocol Directory
-The protocol v2 implementation is contained in a separate `ts_tapbundle_protocol` directory:
-- **Isomorphic Code**: All protocol code works in both browser and Node.js environments
-- **No Platform Dependencies**: No Node.js-specific imports, ensuring true cross-platform compatibility
-- **Clean Separation**: Protocol logic is isolated from platform-specific code in tstest and tapbundle
-- **Shared Implementation**: Both tstest (parser) and tapbundle (emitter) use the same protocol classes
-- **Build Process**: 
-  - Compiled by `pnpm build` via tsbuild to `dist_ts_tapbundle_protocol/`
-  - Build order managed through tspublish.json files
-  - Other modules import from the compiled dist directory, not source
-
-This architectural decision ensures the protocol can be used in any JavaScript environment without modification and maintains proper build dependencies.
-
-See `readme.protocol.md` for the full specification and `ts_tapbundle_protocol/` for the implementation.
-
-## Protocol V2 Implementation Status
-
-The Protocol V2 has been implemented to fix issues with TAP protocol parsing when test descriptions contain special characters like `#`, `###SNAPSHOT###`, or protocol markers like `⟦TSTEST:ERROR⟧`.
-
-### Implementation Details:
-
-1. **Protocol Components**:
-   - `ProtocolEmitter` - Generates protocol v2 messages (used by tapbundle)
-   - `ProtocolParser` - Parses protocol v2 messages (used by tstest)
-   - Uses Unicode markers `⟦TSTEST:` and `⟧` to avoid conflicts with test content
-
-2. **Current Status**:
-   - ✅ Basic protocol emission and parsing works
-   - ✅ Handles test descriptions with special characters correctly
-   - ✅ Supports metadata for timing, tags, errors
-   - ⚠️ Protocol messages sometimes appear in console output (parsing not catching all cases)
-
-3. **Key Findings**:
-   - `tap.skip.test()` doesn't create actual test objects, just logs and increments counter
-   - `tap.todo()` method is not implemented (no `addTodo` method in Tap class)
-   - Protocol parser's `isBlockStart` was fixed to only match exact block markers, not partial matches in test descriptions
-
-4. **Import Paths**:
-   - tstest imports from: `import { ProtocolParser } from '../dist_ts_tapbundle_protocol/index.js';`
-   - tapbundle imports from: `import { ProtocolEmitter } from '../dist_ts_tapbundle_protocol/index.js';`
-
-## Test Configuration System (Phase 2)
-
-The Test Configuration System has been implemented to provide global settings and lifecycle hooks for tests.
-
-### Key Features:
-
-1. **00init.ts Discovery**:
-   - Automatically detects `00init.ts` files in the same directory as test files
-   - Creates a temporary loader file that imports both `00init.ts` and the test file
-   - Loader files are cleaned up automatically after test execution
-
-2. **Settings Inheritance**:
-   - Global settings from `00init.ts` → File-level settings → Test-level settings
-   - Settings include: timeout, retries, retryDelay, bail, concurrency
-   - Lifecycle hooks: beforeAll, afterAll, beforeEach, afterEach
-
-3. **Implementation Details**:
-   - `SettingsManager` class handles settings inheritance and merging
-   - `tap.settings()` API allows configuration at any level
-   - Lifecycle hooks are integrated into test execution flow
-
-### Important Development Notes:
-
-1. **Local Development**: When developing tstest itself, use `node cli.js` instead of globally installed `tstest` to test changes
-
-2. **Console Output Buffering**: Console output from tests is buffered and only displayed for failing tests. TAP-compliant comments (lines starting with `#`) are always shown.
-
-3. **TypeScript Warnings**: Fixed async/await warnings in `movePreviousLogFiles()` by using sync versions of file operations
-
-## Enhanced Communication Features (Phase 3)
-
-The Enhanced Communication system has been implemented to provide rich, real-time feedback during test execution.
-
-### Key Features:
-
-1. **Event-Based Test Lifecycle Reporting**:
-   - `test:queued` - Test is ready to run
-   - `test:started` - Test execution begins
-   - `test:completed` - Test finishes (with pass/fail status)
-   - `suite:started` - Test suite/describe block begins
-   - `suite:completed` - Test suite/describe block ends
-   - `hook:started` - Lifecycle hook (beforeEach/afterEach) begins
-   - `hook:completed` - Lifecycle hook finishes
-   - `assertion:failed` - Assertion failure with detailed information
-
-2. **Visual Diff Output for Assertion Failures**:
-   - **String Diffs**: Character-by-character comparison with colored output
-   - **Object/Array Diffs**: Deep property comparison showing added/removed/changed properties
-   - **Primitive Diffs**: Clear display of expected vs actual values
-   - **Colorized Output**: Green for expected, red for actual, yellow for differences
-   - **Smart Formatting**: Multi-line strings and complex objects are formatted for readability
-
-3. **Real-Time Test Progress API**:
-   - Tests emit progress events as they execute
-   - tstest parser processes events and updates display in real-time
-   - Structured event format carries rich metadata (timing, errors, diffs)
-   - Seamless integration with existing TAP protocol via Protocol V2
-
-### Implementation Details:
-- Events are transmitted via Protocol V2's `EVENT` block type
-- Event data is JSON-encoded within protocol markers
-- Parser handles events asynchronously for real-time updates
-- Visual diffs are generated using custom diff algorithms for each data type
-
-## Watch Mode (Phase 4)
-
-tstest now supports watch mode for automatic test re-runs on file changes.
-
-### Usage
-```bash
-tstest test/**/*.ts --watch
-tstest test/specific.ts -w
-```
-
-### Features
-- **Automatic Re-runs**: Tests re-run when any watched file changes
-- **Debouncing**: Multiple rapid changes are batched (300ms delay)
-- **Clear Output**: Console is cleared before each run for clean results
-- **Status Updates**: Shows which files triggered the re-run
-- **Graceful Exit**: Press Ctrl+C to stop watching
-
-### Options
-- `--watch` or `-w`: Enable watch mode
-- `--watch-ignore`: Comma-separated patterns to ignore (e.g., `--watch-ignore node_modules,dist`)
-
-### Implementation Details
-- Uses `@push.rocks/smartchok` for cross-platform file watching
-- Watches the entire project directory from where tests are run
-- Ignores changes matching the ignore patterns
-- Shows "Waiting for file changes..." between runs
-
-## Phase 1 API Improvements (v3.1.0)
-
-### New Features Implemented
-
-#### 1. tap.postTask() - Global Teardown (COMPLETED)
-
-Added symmetric teardown method to complement `tap.preTask()`:
-
-**Implementation:**
-- Created `PostTask` class in `ts_tapbundle/tapbundle.classes.posttask.ts`
-- Mirrors PreTask structure with description and function
-- Integrated into Tap class execution flow
-- Runs after all tests complete but before global `afterAll` hook
-
-**Usage:**
-```typescript
-tap.postTask('cleanup database', async () => {
-  await cleanupDatabase();
-});
-```
-
-**Execution Order:**
-1. preTask hooks
-2. Global beforeAll
-3. Tests (with suite hooks)
-4. **postTask hooks** ← NEW
-5. Global afterAll
-
-#### 2. Suite-Level beforeAll/afterAll (COMPLETED)
-
-Added once-per-suite lifecycle hooks:
-
-**Implementation:**
-- Extended `ITestSuite` interface with `beforeAll` and `afterAll` properties
-- Added `tap.beforeAll()` and `tap.afterAll()` methods
-- Integrated into `_runSuite()` execution flow
-- Properly handles nested suites
-
-**Usage:**
-```typescript
-tap.describe('Database Tests', () => {
-  tap.beforeAll(async () => {
-    await initializeDatabaseConnection(); // Runs once
-  });
-
-  tap.test('test 1', async () => {});
-  tap.test('test 2', async () => {});
-
-  tap.afterAll(async () => {
-    await closeDatabaseConnection(); // Runs once
-  });
-});
-```
-
-**Execution Order per Suite:**
-1. Suite beforeAll ← NEW
-2. Suite beforeEach
-3. Test
-4. Suite afterEach
-5. (Repeat 2-4 for each test)
-6. Child suites (recursive)
-7. Suite afterAll ← NEW
-
-#### 3. tap.parallel() Fluent Entry Point (COMPLETED)
-
-Added fluent API for parallel test creation:
-
-**Implementation:**
-- Updated `TestBuilder` class with `_parallel` flag
-- Builder constructor accepts optional parallel parameter
-- Added `tap.parallel()` method returning configured builder
-- Fixed `testParallel()` to return TapTest<T> (was void)
-
-**Usage:**
-```typescript
-// Simple parallel test
-tap.parallel().test('fetch data', async () => {});
-
-// With full configuration
-tap
-  .parallel()
-  .tags('api', 'integration')
-  .retry(2)
-  .timeout(5000)
-  .test('configured parallel test', async () => {});
-```
-
-**Benefits:**
-- Consistent with other fluent builders (tags, priority, etc.)
-- More discoverable than separate `testParallel()` method
-- Allows chaining parallel with other configurations
-- `testParallel()` kept for backward compatibility
-
-### Documentation Updates
-
-**tapbundle/readme.md:**
-- Added suite-level beforeAll/afterAll documentation
-- Documented postTask with execution order notes
-- Added parallel() fluent API examples
-- Expanded TapTools documentation with all methods
-- Added "Additional Tap Methods" section for fail(), getSettings(), etc.
-- Documented all previously undocumented methods
-
-### Tests
-
-**test/tapbundle/test.new-lifecycle.ts:**
-- Tests postTask execution order
-- Verifies suite-level beforeAll/afterAll
-- Tests nested suite lifecycle
-- Validates parallel() fluent API
-- Confirms all execution order requirements
-
-**Test Results:** All 9 tests passing ✅
-
-### Breaking Changes
-
-None - all changes are additive and backward compatible.
-
-### Migration Guide
-
-No migration needed. New features are opt-in:
-- Continue using existing patterns
-- Adopt new features incrementally
-- `testParallel()` still works (recommended: switch to `parallel().test()`)
-
-## Fixed Issues
-
-### tap.skip.test(), tap.todo(), and tap.only.test() (Fixed)
-
-Previously reported issues with these methods have been resolved:
-
-1. **tap.skip.test()** - Now properly creates test objects that are counted in test results
-   - Tests marked with `skip.test()` appear in the test count
-   - Shows as passed with skip directive in TAP output
-   - `markAsSkipped()` method added to handle pre-test skip marking
-
-2. **tap.todo.test()** - Fully implemented with test object creation
-   - Supports both `tap.todo.test('description')` and `tap.todo.test('description', testFunc)`
-   - Todo tests are counted and marked with todo directive
-   - Both regular and parallel todo tests supported
-
-3. **tap.only.test()** - Works correctly for focused testing
-   - When `.only` tests exist, only those tests run
-   - Other tests are not executed but still counted
-   - Both regular and parallel only tests supported
-
-These fixes ensure accurate test counts and proper TAP-compliant output for all test states.
-
-## Test Timing Implementation
-
-### Timing Architecture
-
-Test timing is captured using `@push.rocks/smarttime`'s `HrtMeasurement` class, which provides high-resolution timing:
-
-1. **Timing Capture**:
-   - Each `TapTest` instance has its own `HrtMeasurement`
-   - Timer starts immediately before test function execution
-   - Timer stops after test completes (or fails/times out)
-   - Millisecond precision is used for reporting
-
-2. **Protocol Integration**:
-   - Timing is embedded in TAP output using Protocol V2 markers
-   - Inline format for simple timing: `ok 1 - test name ⟦TSTEST:time:123⟧`
-   - Block format for complex metadata: `⟦TSTEST:META:{"time":456,"file":"test.ts"}⟧`
-
-3. **Performance Metrics Calculation**:
-   - Average is calculated from sum of individual test times, not total runtime
-   - Slowest test detection prefers tests with >0ms duration
-   - Failed tests still contribute their execution time to metrics
-
-### Edge Cases and Considerations
-
-1. **Sub-millisecond Tests**:
-   - Very fast tests may report 0ms due to millisecond rounding
-   - Performance metrics handle this by showing "All tests completed in <1ms" when appropriate
-
-2. **Special Test States**:
-   - **Skipped tests**: Report 0ms (not executed)
-   - **Todo tests**: Report 0ms (not executed)
-   - **Failed tests**: Report actual execution time before failure
-   - **Timeout tests**: Report time until timeout occurred
-
-3. **Parallel Test Timing**:
-   - Each parallel test tracks its own execution time independently
-   - Parallel tests may have overlapping execution periods
-   - Total suite time reflects wall-clock time, not sum of test times
-
-4. **Hook Timing**:
-   - `beforeEach`/`afterEach` hooks are not included in individual test times
-   - Only the actual test function execution is measured
-
-5. **Retry Timing**:
-   - When tests retry, only the final attempt's duration is reported
-   - Each retry attempt emits separate `test:started` events
-
-### Parser Fix for Timing Metadata
-
-The protocol parser was fixed to correctly handle inline timing metadata:
-- Changed condition from `!simpleMatch[1].includes(':')` to check for simple key:value pairs
-- Excludes prefixed formats (META:, SKIP:, TODO:, EVENT:) while parsing simple formats like `time:250`
-
-This ensures timing metadata is correctly extracted and displayed in test results.
-
-## Streaming Console Output (Fixed)
-
-### Problem
-When tests use `process.stdout.write()` for streaming output (without newlines), each write was appearing on a separate line. This happened because:
-1. Child process stdout data events arrive as separate chunks
-2. `TapParser._processLog()` split on `\n` and processed each segment
-3. `testConsoleOutput()` used `console.log()` which added a newline to each call
-
-### Solution
-The streaming behavior is now preserved by:
-1. **Line buffering for TAP parsing**: Only buffer content that looks like TAP protocol messages
-2. **True streaming for console output**: Use `process.stdout.write()` instead of `console.log()` for partial lines
-3. **Intelligent detection**: `_looksLikeTapStart()` checks if content could be a TAP protocol message
-
-### Implementation Details
-
-**TapParser changes:**
-- Added `lineBuffer` property to buffer incomplete TAP protocol lines
-- Rewrote `_processLog()` to handle streaming correctly:
-  - Complete lines (with newline) are processed through protocol parser
-  - Incomplete lines that look like TAP are buffered
-  - Incomplete lines that don't look like TAP are streamed immediately
-- Added `_looksLikeTapStart()` helper to detect TAP protocol patterns
-- Added `_handleConsoleOutput()` to handle console output with proper streaming
-- Buffer is flushed on process exit
-
-**TsTestLogger changes:**
-- Added `testConsoleOutputStreaming()` method that uses `process.stdout.write()` in verbose mode
-- Added `logToTestFileRaw()` for writing to log files without adding newlines
-- In non-verbose mode, streaming content is appended to the last buffered entry
-
-**TapTestResult changes:**
-- Added `addLogLineRaw()` method that doesn't append newlines
-
-### Usage
-Tests can now use streaming output naturally:
-```typescript
-process.stdout.write("Loading");
-process.stdout.write(".");
-process.stdout.write(".");
-process.stdout.write(".\n");
-```
-
-This will correctly display as `Loading...` on a single line in verbose mode.