@eldrforge/kodrdriv 1.2.23 → 1.2.25

package/PARALLEL-EXECUTION-FIXES.md

@@ -0,0 +1,132 @@
+# Parallel Execution Fixes
+
+## Critical Bugs Fixed
+
+### 1. Recovery System - Checkpoint Deleted When Packages Skipped (Commit b6deb3f)
+
+**Problem**: When using `--mark-completed` during recovery, the checkpoint was deleted even though dependent packages were skipped, making recovery impossible.
+
+**Symptoms**:
+- Misleading "All X packages completed successfully!" message when packages were actually skipped
+- Lost checkpoint, unable to continue execution
+- No way to resume and complete skipped packages
+
+**Fix**: Modified checkpoint cleanup logic to preserve checkpoint when packages are skipped.
+
+**Files Changed**:
+- `src/execution/DynamicTaskPool.ts`: Only cleanup if NO failures AND NO skipped packages
+- `src/execution/TreeExecutionAdapter.ts`: Show accurate message for skipped packages
+- `src/execution/RecoveryManager.ts`: Accept both directory names and package names
+- `src/arguments.ts`: Updated help text for consistency
+- `src/ui/ProgressFormatter.ts`: Updated recovery guidance
+
+### 2. Parallel Execution - Race Conditions in Dependency Updates (Commit 371050c)
+
+**Problem**: When running `kodrdriv tree publish --parallel`, multiple packages updated dependencies simultaneously, causing catastrophic failures.
+
+**Symptoms**:
+```
+1. ENOTEMPTY errors:
+   npm error ENOTEMPTY: directory not empty, rename
+   '/path/node_modules/@eldrforge/git-tools' ->
+   '/path/node_modules/@eldrforge/.git-tools-xxx'
+
+2. "not a git repository" errors:
+   fatal: not a git repository (or any of the parent directories): .git
+
+3. Git state conflicts:
+   error: Working directory has uncommitted changes
+```
+
+**Root Cause**:
+- Dependency update operations (npm install + git commit) ran in parallel
+- Multiple packages tried to:
+  - Update same dependencies → filesystem race conditions
+  - Commit at same time → git state conflicts
+  - Run in different working directories → lost git context
+
+**Solution**: Wrapped dependency update + commit section with per-repository lock (`runGitWithLock`):
+- Operations serialize within each repository
+- Maintains parallelism across different repositories
+- Prevents npm install race conditions
+- Prevents git commit conflicts
+- Preserves working directory context
+
+**Files Changed**:
+- `src/commands/tree.ts`: Import `runGitWithLock` and wrap dependency update section
+
+## Impact
+
+### Before Fixes:
+```bash
+# Parallel publish would fail catastrophically:
+kodrdriv tree publish --parallel
+# → ENOTEMPTY errors
+# → Git repository errors
+# → Unable to recover
+```
+
+### After Fixes:
+```bash
+# Parallel publish works reliably:
+kodrdriv tree publish --parallel
+# → Dependency updates happen serially per repo
+# → Git operations don't conflict
+# → If something fails, recovery actually works
+
+# Recovery now works:
+kodrdriv tree publish --continue --mark-completed "git-tools"
+# → Checkpoint preserved
+# → Remaining packages execute
+# → Can use simple directory names
+```
+
+## Technical Details
+
+### Git Lock Mechanism
+
+The fix uses `runGitWithLock()` from `src/util/gitMutex.ts`:
+
+```typescript
+await runGitWithLock(packageDir, async () => {
+    // Update scoped dependencies (npm install)
+    await updateScopedDependencies(...);
+
+    // Update inter-project dependencies (npm install)
+    await updateInterProjectDependencies(...);
+
+    // Commit changes (git operations)
+    await Commit.execute(...);
+}, `${packageName}: dependency updates`);
+```
+
+This ensures that:
+1. Operations are serialized within each git repository
+2. Parallel execution continues across different repositories
+3. File-based locks coordinate across processes
+4. No race conditions on npm install or git operations
+
+### Checkpoint Preservation
+
+The checkpoint cleanup now checks both conditions:
+
+```typescript
+const allCompleted = this.state.failed.length === 0 &&
+                    this.state.skipped.length === 0;
+if (allCompleted) {
+    await this.checkpointManager.cleanup();
+} else {
+    await this.saveCheckpoint(); // Keep checkpoint for recovery
+}
+```
+
+## Testing
+
+Both fixes have been tested and work correctly:
+- Recovery system preserves checkpoints when needed
+- Parallel execution no longer causes race conditions
+- Directory names work for `--mark-completed`
+
+## Version
+
+These fixes are in version `1.2.24-dev.0` commit `371050c`.

package/PARALLEL_EXECUTION_FIX.md

@@ -0,0 +1,146 @@
+# Parallel Execution Freeze Fix
+
+## Problem
+
+When running `kodrdriv tree publish --parallel`, the execution would freeze because:
+
+1. **Multiple child processes** were spawned (one per package)
+2. All packages were in the **same git repository** (`/Users/tobrien/gitw/calenvarek`)
+3. Each child process had its own **in-memory mutex** (SimpleMutex)
+4. In-memory mutexes **cannot coordinate across separate processes**
+5. Multiple processes tried to run git operations concurrently
+6. Git created `.git/index.lock` files, causing conflicts and hangs
+
+## Root Cause
+
+The `SimpleMutex` class in `src/util/mutex.ts` was designed for **single-process** synchronization. It uses in-memory state that cannot be shared between separate Node.js processes.
+
+When `kodrdriv tree publish --parallel` spawns child processes using `exec()`, each child process:
+- Gets its own memory space
+- Creates its own `RepositoryMutexManager` singleton
+- Has no way to coordinate with other processes
+
+## Solution
+
+Implemented a **file-based locking mechanism** in `src/util/fileLock.ts`:
+
+### Key Features
+
+1. **Cross-Process Safety**: Uses atomic file operations (`wx` flag) that work across processes
+2. **Exponential Backoff**: Retries lock acquisition with increasing delays (100ms → 2000ms max)
+3. **Stale Lock Detection**: Automatically removes locks older than 30 seconds
+4. **Automatic Cleanup**: Releases locks on process exit (normal, SIGINT, SIGTERM, uncaughtException)
+5. **Per-Repository Locking**: Creates `.git/kodrdriv.lock` file in each repository
+
+### Implementation Details
+
+#### FileLock Class
+```typescript
+class FileLock {
+    async lock(): Promise<void> {
+        // Attempts to create lock file atomically with 'wx' flag
+        // Retries with exponential backoff if file exists
+        // Detects and removes stale locks (>30 seconds old)
+    }
+
+    unlock(): void {
+        // Removes lock file
+    }
+}
+```
+
+#### RepositoryFileLockManager Class
+```typescript
+class RepositoryFileLockManager {
+    getRepositoryLock(repoPath: string): FileLock {
+        // Returns file lock for .git/kodrdriv.lock
+    }
+
+    async withGitLock<T>(repoPath, operation, operationName): Promise<T> {
+        // Acquires lock, executes operation, releases lock
+    }
+}
+```
+
+### Changes Made
+
+1. **Created** `src/util/fileLock.ts` - New file-based locking implementation
+2. **Modified** `src/util/gitMutex.ts` - Now uses FileLock instead of SimpleMutex
+3. **No Breaking Changes** - API remains the same, only implementation changed
+
+### How It Works
+
+```
+Parent Process (kodrdriv tree publish --parallel)
+├── Spawns: kodrdriv publish [git-tools]
+│   └── Tries to acquire .git/kodrdriv.lock
+│       ✓ Success! Executes git operations
+│       ✓ Releases lock
+├── Spawns: kodrdriv publish [ai-service]
+│   └── Tries to acquire .git/kodrdriv.lock
+│       ⏳ Waits... lock file exists
+│       ✓ Previous process released lock
+│       ✓ Acquires lock, executes, releases
+├── Spawns: kodrdriv publish [github-tools]
+│   └── (same pattern)
+└── Spawns: kodrdriv publish [kodrdriv]
+    └── (same pattern)
+```
+
+### Testing
+
+- ✅ All 283 existing tests pass
+- ✅ Build succeeds with no linter errors
+- ✅ Lock files are automatically cleaned up on exit
+- ✅ Stale locks (>30s) are automatically removed
+
+## Usage
+
+No changes needed! The fix is transparent:
+
+```bash
+# This now works without freezing
+kodrdriv tree publish --parallel
+
+# Same for other commands that use git operations
+kodrdriv tree commit --parallel
+```
+
+## Lock File Location
+
+Lock files are created at:
+```
+/path/to/repo/.git/kodrdriv.lock
+```
+
+These files:
+- Are automatically created/removed
+- Are gitignored (in .git directory)
+- Are safe to manually delete if stale
+- Contain diagnostic info (PID, timestamp, hostname)
+
+## Performance Impact
+
+- **Minimal overhead**: File operations are fast (microseconds)
+- **Better than deadlock**: Small delay is better than infinite freeze
+- **Automatic backoff**: Reduces contention with exponential delays
+- **Stale lock cleanup**: Prevents indefinite blocking
+
+## Future Improvements
+
+Potential enhancements:
+1. Make lock timeout configurable
+2. Add lock health monitoring/metrics
+3. Implement lock priority/queueing
+4. Add verbose lock acquisition logging
+
+## Verification
+
+To verify the fix works:
+```bash
+cd /Users/tobrien/gitw/calenvarek/kodrdriv
+npm run build
+kodrdriv tree publish --parallel --dry-run
+```
+
+The parallel execution should now proceed without freezing.

package/RECOVERY-FIXES.md

@@ -0,0 +1,72 @@
+# Recovery System Fixes
+
+## Issues Fixed
+
+### 1. Checkpoint Deleted When Packages Are Skipped
+
+**Problem**: When using `--mark-completed` to mark a failed package as done, the system would delete the checkpoint even though dependent packages were skipped. This resulted in:
+- Misleading "All X packages completed successfully! 🎉" message when packages were actually skipped
+- Lost checkpoint making it impossible to continue the execution
+- No way to resume and complete the skipped packages
+
+**Root Cause**: In `DynamicTaskPool.ts`, the cleanup logic only checked for failed packages, not skipped packages:
+
+```typescript
+// Before (WRONG)
+if (this.state.failed.length === 0) {
+    await this.checkpointManager.cleanup();  // Deletes checkpoint
+}
+
+// After (FIXED)
+const allCompleted = this.state.failed.length === 0 && this.state.skipped.length === 0;
+if (allCompleted) {
+    await this.checkpointManager.cleanup();
+}
+```
+
+**Fix**: Modified `src/execution/DynamicTaskPool.ts` to preserve the checkpoint when packages are skipped, and updated `src/execution/TreeExecutionAdapter.ts` to show the correct message when packages are skipped.
+
+### 2. Inconsistent Package Identifier Format
+
+**Problem**: The `--mark-completed` option required NPM package names (e.g., `"@eldrforge/git-tools"`), while `--start-from` accepted directory names (e.g., `"git-tools"`). This was confusing and inconsistent.
+
+**Fix**: Updated `src/execution/RecoveryManager.ts` to accept both directory names and package names for `--mark-completed`, matching the behavior of `--start-from`:
+
+```typescript
+// Now both work:
+kodrdriv tree publish --continue --mark-completed "git-tools"
+kodrdriv tree publish --continue --mark-completed "@eldrforge/git-tools"
+```
+
+The system will:
+1. Try exact package name match first
+2. Fall back to directory name match
+3. Provide helpful error with available packages if not found
+
+**Updated Files**:
+- `src/execution/RecoveryManager.ts` - Added `resolvePackageName()` helper and updated `markCompleted()`
+- `src/arguments.ts` - Updated help text
+- `src/ui/ProgressFormatter.ts` - Updated recovery guidance to use directory names
+
+## Testing
+
+All existing tests pass, including the specific RecoveryManager tests.
+
+## Usage
+
+Now you can use directory names (much simpler):
+
+```bash
+# Start a parallel publish
+kodrdriv tree publish --parallel
+
+# If git-tools fails due to merge conflict, fix it manually, then:
+kodrdriv tree publish --continue --mark-completed "git-tools"
+
+# The remaining packages (ai-service, github-tools, kodrdriv) will now execute
+```
+
+The system will:
+1. Keep the checkpoint if packages are skipped
+2. Show accurate status messages
+3. Allow you to continue execution to complete the skipped packages

package/SUBMODULE-LOCK-FIX.md

@@ -0,0 +1,132 @@
+# Git Submodule Lock File Fix
+
+## Problem
+
+The file-based lock mechanism used to prevent concurrent git operations was failing when used with git submodules. The issue occurred because:
+
+1. In regular git repositories, `.git` is a directory
+2. In git submodules, `.git` is a file containing a `gitdir:` reference pointing to the actual git directory
+3. The code assumed `.git` was always a directory and tried to create lock files in `.git/kodrdriv.lock`
+4. This caused `ENOTDIR` errors when operating on submodules
+
+## Example Submodule Structure
+
+```
+my-submodule/
+├── .git              # FILE (not directory) containing: "gitdir: ../.git/modules/my-submodule"
+├── src/
+└── package.json
+
+parent-repo/
+└── .git/
+    └── modules/
+        └── my-submodule/  # Actual git directory for the submodule
+            ├── HEAD
+            ├── refs/
+            └── objects/
+```
+
+## Solution
+
+Modified `src/util/fileLock.ts` to handle both regular repositories and submodules:
+
+### Key Changes
+
+1. **Added `resolveGitDirectory()` method** that:
+   - Checks if `.git` is a directory (regular repo) or file (submodule)
+   - If it's a file, reads and parses the `gitdir:` reference
+   - Resolves the gitdir path (handles both relative and absolute paths)
+   - Returns the actual git directory path where locks can be created
+
+2. **Updated `getRepositoryLock()` method** to:
+   - Use `resolveGitDirectory()` instead of assuming `.git` is a directory
+   - Create lock files in the resolved git directory
+   - Log the actual lock path for debugging
+
+### Code Implementation
+
+```typescript
+private resolveGitDirectory(repoPath: string): string {
+    const gitPath = path.join(repoPath, '.git');
+
+    try {
+        const stat = fs.statSync(gitPath);
+
+        if (stat.isDirectory()) {
+            // Regular git repository
+            return gitPath;
+        } else if (stat.isFile()) {
+            // Git submodule - .git is a file with format: gitdir: <path>
+            const gitFileContent = fs.readFileSync(gitPath, 'utf-8').trim();
+            const match = gitFileContent.match(/^gitdir:\s*(.+)$/);
+
+            if (match && match[1]) {
+                // Resolve the gitdir path (it's relative to the repo path)
+                const gitDirPath = path.resolve(repoPath, match[1]);
+                this.logger.debug(`Resolved submodule gitdir: ${gitDirPath}`);
+
+                // Ensure the git directory exists
+                if (!fs.existsSync(gitDirPath)) {
+                    throw new Error(`Submodule git directory does not exist: ${gitDirPath}`);
+                }
+
+                return gitDirPath;
+            }
+
+            throw new Error(`Invalid .git file format in ${gitPath}: ${gitFileContent}`);
+        }
+    } catch (error: any) {
+        // Check if error is from statSync (file doesn't exist)
+        if (error.code === 'ENOENT') {
+            throw new Error(`No .git directory or file found in ${repoPath}`);
+        }
+        throw new Error(`Failed to resolve git directory for ${repoPath}: ${error.message}`);
+    }
+
+    throw new Error(`No .git directory or file found in ${repoPath}`);
+}
+```
+
+## Testing
+
+Comprehensive test suite added in `tests/fileLock.test.ts` covering:
+
+1. **Basic lock operations**
+   - Acquire and release locks
+   - Block concurrent lock acquisition
+   - Handle stale locks
+
+2. **Regular repository support**
+   - Create locks in `.git` directory
+
+3. **Submodule support** (NEW)
+   - Handle `.git` file with `gitdir:` reference
+   - Resolve relative gitdir paths
+   - Create locks in the actual git directory
+   - Proper error handling for missing submodule directories
+   - Proper error handling for invalid `.git` file format
+
+4. **Lock manager functionality**
+   - Execute operations under lock
+   - Release locks on operation failure
+   - Serialize multiple operations on same repo
+
+All 12 tests pass successfully with 86%+ coverage on the fileLock module.
+
+## Impact
+
+This fix enables kodrdriv to work correctly in monorepo setups where packages are organized as git submodules, such as:
+
+- Multi-repository projects using git submodules for shared libraries
+- Projects with external dependencies included as submodules
+- Monorepos with complex submodule hierarchies
+
+The fix maintains backward compatibility with regular git repositories while adding robust support for submodules.
+
+## Related Files
+
+- `src/util/fileLock.ts` - Core fix implementation
+- `tests/fileLock.test.ts` - Comprehensive test suite
+- `src/util/gitMutex.ts` - Uses RepositoryFileLockManager
+- `src/commands/publish.ts` - Uses git locks during publish operations
+

package/dist/arguments.js

@@ -1,5 +1,5 @@
 import { Command } from 'commander';
-import path from 'path';
+import path__default from 'path';
 import { z } from 'zod';
 import { PROGRAM_NAME, VERSION, KODRDRIV_DEFAULTS, ALLOWED_COMMANDS, DEFAULT_COMMAND } from './constants.js';
 import { getLogger } from './logging.js';
@@ -263,6 +263,29 @@ const transformCliArgs = (finalCliArgs, commandName)=>{
             if (packageArgument !== undefined) transformedCliArgs.tree.packageArgument = packageArgument;
             if (finalCliArgs.cleanNodeModules !== undefined) transformedCliArgs.tree.cleanNodeModules = finalCliArgs.cleanNodeModules;
             if (finalCliArgs.externals !== undefined) transformedCliArgs.tree.externals = finalCliArgs.externals;
+            // Parallel execution options (using any cast for new properties)
+            const cliArgs = finalCliArgs;
+            if (cliArgs.parallel !== undefined) transformedCliArgs.tree.parallel = cliArgs.parallel;
+            if (cliArgs.maxConcurrency !== undefined) transformedCliArgs.tree.maxConcurrency = cliArgs.maxConcurrency;
+            if (cliArgs.maxRetries !== undefined || cliArgs.retryDelay !== undefined) {
+                transformedCliArgs.tree.retry = {
+                    maxAttempts: cliArgs.maxRetries || 3,
+                    initialDelayMs: cliArgs.retryDelay || 5000,
+                    maxDelayMs: 60000,
+                    backoffMultiplier: 2
+                };
+            }
+            // Recovery options
+            if (cliArgs.statusParallel !== undefined) transformedCliArgs.tree.statusParallel = cliArgs.statusParallel;
+            if (cliArgs.markCompleted !== undefined) {
+                transformedCliArgs.tree.markCompleted = cliArgs.markCompleted.split(',').map((s)=>s.trim());
+            }
+            if (cliArgs.skip !== undefined) {
+                transformedCliArgs.tree.skipPackages = cliArgs.skip.split(',').map((s)=>s.trim());
+            }
+            if (cliArgs.retryFailed !== undefined) transformedCliArgs.tree.retryFailed = cliArgs.retryFailed;
+            if (cliArgs.skipFailed !== undefined) transformedCliArgs.tree.skipFailed = cliArgs.skipFailed;
+            if (cliArgs.validateState !== undefined) transformedCliArgs.tree.validateState = cliArgs.validateState;
         }
     }
     // Nested mappings for 'development' options
@@ -364,7 +387,7 @@ const configure = async (cardigantime)=>{
         const transformedCliArgs = transformCliArgs(cliArgs);
         // Use CardiganTime's built-in generateConfig method
         const configDir = transformedCliArgs.configDirectory || KODRDRIV_DEFAULTS.configDirectory;
-        const absoluteConfigDir = path.isAbsolute(configDir) ? configDir : path.resolve(process.cwd(), configDir);
+        const absoluteConfigDir = path__default.isAbsolute(configDir) ? configDir : path__default.resolve(process.cwd(), configDir);
         await cardigantime.generateConfig(absoluteConfigDir);
         // Return minimal config for consistency, but main processing is done
         const config = await validateAndProcessOptions({});
@@ -598,7 +621,7 @@ async function getCliConfig(program, commands) {
     addSharedOptions(releaseCommand);
     const publishCommand = program.command('publish').option('--merge-method <method>', 'method to merge PR (merge, squash, rebase)', 'squash').option('--from <from>', 'branch/tag to generate release notes from (default: previous release tag)').option('--target-version <targetVersion>', 'target version for release (explicit version like "4.30.0" or semantic bump: "patch", "minor", "major")').option('--interactive', 'present release notes for interactive review and editing').option('--sendit', 'skip all confirmation prompts and proceed automatically').option('--sync-target', 'attempt to automatically sync target branch with remote before publishing').option('--no-milestones', 'disable GitHub milestone integration').option('--from-main', 'force comparison against main branch instead of previous release tag').description('Publish a release');
     addSharedOptions(publishCommand);
-    const treeCommand = program.command('tree [command] [packageArgument]').option('--directory <directory>', 'target directory containing multiple packages (defaults to current directory)').option('--directories [directories...]', 'target directories containing multiple packages (defaults to current directory)').option('--start-from <startFrom>', 'resume execution from this package directory name (useful for restarting failed builds)').option('--stop-at <stopAt>', 'stop execution at this package directory name (the specified package will not be executed)').option('--cmd <cmd>', 'shell command to execute in each package directory (e.g., "npm install", "git status")').option('--parallel', 'execute packages in parallel when dependencies allow (packages with no interdependencies run simultaneously)').option('--excluded-patterns [excludedPatterns...]', 'patterns to exclude packages from processing (e.g., "**/node_modules/**", "dist/*")').option('--continue', 'continue from previous tree publish execution').option('--status', 'check status of running tree publish processes').option('--promote <packageName>', 'mark a package as completed in the execution context (useful for recovery after timeouts)').option('--clean-node-modules', 'for unlink command: remove node_modules and package-lock.json, then reinstall dependencies').description('Analyze package dependencies in workspace and execute commands in dependency order. Supports built-in commands: commit, publish, link, unlink, development, branches, run, checkout');
+    const treeCommand = program.command('tree [command] [packageArgument]').option('--directory <directory>', 'target directory containing multiple packages (defaults to current directory)').option('--directories [directories...]', 'target directories containing multiple packages (defaults to current directory)').option('--start-from <startFrom>', 'resume execution from this package directory name (useful for restarting failed builds)').option('--stop-at <stopAt>', 'stop execution at this package directory name (the specified package will not be executed)').option('--cmd <cmd>', 'shell command to execute in each package directory (e.g., "npm install", "git status")').option('--parallel', 'execute packages in parallel when dependencies allow (packages with no interdependencies run simultaneously)').option('--max-concurrency <number>', 'maximum number of packages to execute concurrently (default: number of CPU cores)', parseInt).option('--max-retries <number>', 'maximum retry attempts for failed packages (default: 3)', parseInt).option('--retry-delay <ms>', 'initial retry delay in milliseconds (default: 5000)', parseInt).option('--excluded-patterns [excludedPatterns...]', 'patterns to exclude packages from processing (e.g., "**/node_modules/**", "dist/*")').option('--continue', 'continue from previous tree publish execution').option('--status', 'check status of running tree publish processes').option('--status-parallel', 'show detailed parallel execution status').option('--promote <packageName>', 'mark a package as completed in the execution context (useful for recovery after timeouts)').option('--mark-completed <packages>', 'mark packages as completed using directory names (comma-separated, for recovery)').option('--skip <packages>', 'skip packages and their dependents (comma-separated)').option('--retry-failed', 'retry all previously failed packages').option('--skip-failed', 'skip failed packages and continue with remaining').option('--validate-state', 'validate checkpoint state integrity').option('--clean-node-modules', 'for unlink command: remove node_modules and package-lock.json, then reinstall dependencies').description('Analyze package dependencies in workspace and execute commands in dependency order. Supports built-in commands: commit, publish, link, unlink, development, branches, run, checkout');
     addSharedOptions(treeCommand);
     const linkCommand = program.command('link [packageArgument]').option('--scope-roots <scopeRoots>', 'JSON mapping of scopes to root directories (e.g., \'{"@company": "../"}\')').description('Create npm file: dependencies for local development');
     addSharedOptions(linkCommand);