command-stream 0.9.2 → 0.9.4

package/js/docs/case-studies/issue-153/README.md

@@ -0,0 +1,167 @@
+# Case Study: Array.join() Pitfall Causes Arguments to Merge (Issue #153)
+
+## Summary
+
+This document provides a comprehensive analysis of the `Array.join()` pitfall in command-stream, where calling `.join(' ')` on an array before template interpolation causes all elements to be treated as a single argument instead of multiple separate arguments.
+
+## Timeline of Events
+
+### January 10, 2026
+
+1. **~00:10 UTC** - Production bug discovered in `hive-mind` repository (issue link-assistant/hive-mind#1096)
+2. **~00:10 UTC** - Log upload command failed with error: `File does not exist: "/tmp/solution-draft-log-pr-1768003849690.txt" --public --verbose`
+3. **~21:47 UTC** - Issue #153 created to document the pitfall and improve documentation
+
+## Real-World Impact
+
+### Production Bug in hive-mind#1096
+
+The bug manifested in a log upload workflow where CLI arguments were incorrectly joined:
+
+**Error Message:**
+
+```
+Error: File does not exist: "/tmp/solution-draft-log-pr-1768003849690.txt" --public --verbose
+```
+
+**Root Cause:**
+The flags `--public` and `--verbose` were incorrectly merged into the file path as a single string argument. The gh-upload-log command received:
+
+- Expected: 3 arguments: `"/tmp/solution-draft-log-pr-1768003849690.txt"`, `--public`, `--verbose`
+- Actual: 1 argument: `"/tmp/solution-draft-log-pr-1768003849690.txt" --public --verbose`
+
+**Original Buggy Code Pattern:**
+
+```javascript
+const commandArgs = [`${logFile}`, publicFlag];
+if (verbose) commandArgs.push('--verbose');
+await $`gh-upload-log ${commandArgs.join(' ')}`; // BUG: Single string argument
+```
+
+**Fixed Code Pattern:**
+
+```javascript
+await $`gh-upload-log ${logFile} ${publicFlag} --verbose`; // Each value is a separate argument
+```
+
+## Technical Analysis
+
+### Why This Happens
+
+The `buildShellCommand` function in `$.quote.mjs` handles arrays specially:
+
+```javascript
+if (Array.isArray(value)) {
+  return value.map(quote).join(' '); // Each element quoted separately
+}
+```
+
+When an array is passed directly:
+
+1. Each element is individually quoted
+2. They are joined with spaces
+3. The shell receives multiple arguments
+
+But when you call `.join(' ')` before passing to the template:
+
+1. The array becomes a string: `"file.txt --public --verbose"`
+2. Template receives a **string**, not an array
+3. The **entire string** gets quoted as one shell argument
+4. The command sees one argument containing spaces, not multiple arguments
+
+### Demonstration
+
+```javascript
+// Direct array interpolation (CORRECT)
+const args = ['file.txt', '--public', '--verbose'];
+await $`command ${args}`;
+// Executed: command file.txt --public --verbose
+// Shell receives: ['command', 'file.txt', '--public', '--verbose']
+
+// Pre-joined array (INCORRECT)
+const args = ['file.txt', '--public', '--verbose'];
+await $`command ${args.join(' ')}`;
+// Executed: command 'file.txt --public --verbose'
+// Shell receives: ['command', 'file.txt --public --verbose']
+```
+
+## Solutions
+
+### Correct Usage Patterns
+
+#### 1. Pass Array Directly (Recommended)
+
+```javascript
+const args = ['file.txt', '--public', '--verbose'];
+await $`command ${args}`;
+```
+
+#### 2. Use Separate Interpolations
+
+```javascript
+const file = 'file.txt';
+const flags = ['--public', '--verbose'];
+await $`command ${file} ${flags}`;
+```
+
+#### 3. Build Array Dynamically
+
+```javascript
+const baseArgs = ['file.txt'];
+const conditionalArgs = isVerbose ? ['--verbose'] : [];
+const allArgs = [...baseArgs, ...conditionalArgs];
+await $`command ${allArgs}`;
+```
+
+### Incorrect Usage (Anti-Patterns)
+
+```javascript
+// DON'T DO THIS - array becomes single argument
+await $`command ${args.join(' ')}`;
+
+// DON'T DO THIS - template string becomes single argument
+await $`command ${`${file} ${flag}`}`;
+
+// DON'T DO THIS - manual string concatenation
+await $`command ${file + ' ' + flag}`;
+```
+
+## Error Recognition
+
+When you see errors like these, suspect the Array.join() pitfall:
+
+1. **File not found with flags in the path:**
+
+   ```
+   Error: File does not exist: "/path/to/file.txt --flag --option"
+   ```
+
+2. **Command received unexpected argument count:**
+
+   ```
+   Error: Expected 3 arguments, got 1
+   ```
+
+3. **Flags not recognized:**
+   ```
+   Error: Unknown option: "value --flag"
+   ```
+
+## Prevention Strategies
+
+1. **Never use `.join()` before template interpolation** - Pass arrays directly
+2. **Review string concatenation** - Ensure separate values stay separate
+3. **Test with special characters** - Include spaces and flags in test cases
+4. **Add debug logging** - Log the actual arguments being passed
+
+## Related Documentation
+
+- [BEST-PRACTICES.md](../../BEST-PRACTICES.md) - Best practices for command-stream usage
+- [README.md](../../../../README.md) - Common Pitfalls section
+- [$.quote.mjs](../../../src/$.quote.mjs) - Quote function implementation
+
+## References
+
+- Issue #153: https://github.com/link-foundation/command-stream/issues/153
+- Production Bug: https://github.com/link-assistant/hive-mind/issues/1096
+- Full Log: https://gist.github.com/konard/70a7c02ac0d1eee232dae2fbe5eeca7b

package/js/docs/shell-operators-implementation.md

@@ -0,0 +1,97 @@
+# Shell Operators Implementation Plan
+
+## Current State
+
+- Commands with `&&`, `||`, `;`, `()` are passed to `sh -c` as a whole string
+- This runs in a subprocess, so `cd` changes don't affect the parent process
+- Virtual commands only work for simple commands and pipe chains
+
+## Required Changes
+
+### 1. Enhanced Command Parser
+
+Need to parse these operators:
+
+- `&&` - AND: execute next command only if previous succeeds (exit code 0)
+- `||` - OR: execute next command only if previous fails (exit code != 0)
+- `;` - SEMICOLON: execute next command regardless of previous result
+- `()` - SUBSHELL: execute commands in a subshell (isolated environment)
+
+### 2. Command Execution Flow
+
+```javascript
+// Current flow for "cd /tmp && ls"
+1. Detect && operator
+2. Pass entire string to sh -c "cd /tmp && ls"
+3. Subprocess executes, cd affects only subprocess
+
+// New flow
+1. Parse: ["cd /tmp", "&&", "ls"]
+2. Execute "cd /tmp" via virtual command (changes process.cwd)
+3. If exit code == 0, execute "ls"
+4. Both commands see the changed directory
+```
+
+### 3. Subshell Handling
+
+For `(cd /tmp && ls)`:
+
+1. Save current process.cwd()
+2. Execute commands inside ()
+3. Restore original cwd after subshell completes
+
+### 4. Parser Implementation
+
+```javascript
+function parseShellCommand(command) {
+  // Parse operators while respecting:
+  // - Quoted strings "..." and '...'
+  // - Escaped characters
+  // - Nested parentheses
+
+  return {
+    type: 'sequence',
+    operators: ['&&', ';'],
+    commands: [
+      { type: 'simple', cmd: 'cd', args: ['/tmp'] },
+      { type: 'simple', cmd: 'ls', args: [] },
+    ],
+  };
+}
+```
+
+### 5. Execution Engine
+
+```javascript
+async function executeSequence(parsedCommand) {
+  let lastExitCode = 0;
+
+  for (let i = 0; i < parsedCommand.commands.length; i++) {
+    const command = parsedCommand.commands[i];
+    const operator = parsedCommand.operators[i - 1];
+
+    // Check operator conditions
+    if (operator === '&&' && lastExitCode !== 0) continue;
+    if (operator === '||' && lastExitCode === 0) continue;
+
+    // Execute command
+    const result = await executeCommand(command);
+    lastExitCode = result.code;
+  }
+}
+```
+
+## Benefits
+
+1. Virtual commands work in all contexts
+2. `cd` behaves like real shell cd
+3. Consistent behavior across platforms
+4. Better control over execution flow
+
+## Testing Requirements
+
+1. Test all operators: `&&`, `||`, `;`
+2. Test subshells: `()`
+3. Test nested subshells: `(cd /tmp && (cd /usr && pwd))`
+4. Test with virtual and real commands mixed
+5. Test error handling and exit codes

package/js/tests/array-interpolation.test.mjs

@@ -0,0 +1,329 @@
+/**
+ * Tests for array interpolation in command-stream
+ *
+ * This test file covers the Array.join() pitfall documented in issue #153.
+ * The key insight: arrays passed directly to template interpolation are handled
+ * correctly (each element becomes a separate argument), but if you call .join(' ')
+ * before passing, the entire string becomes a single argument.
+ *
+ * @see https://github.com/link-foundation/command-stream/issues/153
+ * @see js/docs/case-studies/issue-153/README.md
+ * @see js/BEST-PRACTICES.md
+ */
+
+import { $, quote } from '../src/$.mjs';
+import { describe, test, expect } from 'bun:test';
+import './test-helper.mjs'; // Automatically sets up beforeEach/afterEach cleanup
+
+describe('Array Interpolation', () => {
+  describe('Direct Array Passing (Correct Usage)', () => {
+    test('should treat each array element as separate argument', () => {
+      const args = ['arg1', 'arg2', 'arg3'];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      expect(cmd.spec.command).toBe('echo arg1 arg2 arg3');
+    });
+
+    test('should quote elements with spaces individually', () => {
+      const args = ['file with spaces.txt', '--verbose'];
+      const cmd = $({ mirror: false })`command ${args}`;
+
+      expect(cmd.spec.command).toBe("command 'file with spaces.txt' --verbose");
+    });
+
+    test('should handle empty array', () => {
+      const args = [];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      expect(cmd.spec.command).toBe('echo ');
+    });
+
+    test('should handle single-element array', () => {
+      const args = ['single'];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      expect(cmd.spec.command).toBe('echo single');
+    });
+
+    test('should handle array with special characters', () => {
+      const args = ['$var', '`command`', '$(sub)'];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      // Each special character should be quoted
+      expect(cmd.spec.command).toBe("echo '$var' '`command`' '$(sub)'");
+    });
+
+    test('should handle array with flags correctly', () => {
+      const args = ['input.txt', '--public', '--verbose'];
+      const cmd = $({ mirror: false })`upload ${args}`;
+
+      expect(cmd.spec.command).toBe('upload input.txt --public --verbose');
+    });
+  });
+
+  describe('Pre-joined Array (Anti-pattern)', () => {
+    test('joined array becomes single argument with spaces', () => {
+      const args = ['file.txt', '--flag'];
+      // This is the anti-pattern - joining before interpolation
+      const joined = args.join(' ');
+      const cmd = $({ mirror: false })`command ${joined}`;
+
+      // The joined string gets quoted as ONE argument
+      expect(cmd.spec.command).toBe("command 'file.txt --flag'");
+    });
+
+    test('demonstrates the bug: flags become part of filename', () => {
+      // This reproduces the exact bug from hive-mind#1096
+      const args = ['/tmp/logfile.txt', '--public', '--verbose'];
+      const joined = args.join(' ');
+      const cmd = $({ mirror: false })`gh-upload-log ${joined}`;
+
+      // WRONG: The shell sees one argument containing spaces
+      expect(cmd.spec.command).toBe(
+        "gh-upload-log '/tmp/logfile.txt --public --verbose'"
+      );
+      // This would cause: Error: File does not exist: "/tmp/logfile.txt --public --verbose"
+    });
+
+    test('correct usage vs incorrect usage comparison', () => {
+      const args = ['file.txt', '--flag1', '--flag2'];
+
+      // CORRECT: Direct array interpolation
+      const correctCmd = $({ mirror: false })`cmd ${args}`;
+      expect(correctCmd.spec.command).toBe('cmd file.txt --flag1 --flag2');
+
+      // INCORRECT: Pre-joined array
+      const incorrectCmd = $({ mirror: false })`cmd ${args.join(' ')}`;
+      expect(incorrectCmd.spec.command).toBe("cmd 'file.txt --flag1 --flag2'");
+    });
+  });
+
+  describe('Mixed Interpolation Patterns', () => {
+    test('should handle multiple separate interpolations', () => {
+      const file = 'data.txt';
+      const flags = ['--verbose', '--force'];
+      const cmd = $({ mirror: false })`process ${file} ${flags}`;
+
+      expect(cmd.spec.command).toBe('process data.txt --verbose --force');
+    });
+
+    test('should handle array with conditional elements', () => {
+      const baseArgs = ['input.txt'];
+      const verbose = true;
+      const force = false;
+
+      if (verbose) {
+        baseArgs.push('--verbose');
+      }
+      if (force) {
+        baseArgs.push('--force');
+      }
+
+      const cmd = $({ mirror: false })`command ${baseArgs}`;
+      expect(cmd.spec.command).toBe('command input.txt --verbose');
+    });
+
+    test('should handle spread operator pattern', () => {
+      const files = ['file1.txt', 'file2.txt'];
+      const flags = ['--recursive'];
+      const allArgs = [...files, ...flags];
+
+      const cmd = $({ mirror: false })`copy ${allArgs}`;
+      expect(cmd.spec.command).toBe('copy file1.txt file2.txt --recursive');
+    });
+  });
+
+  describe('Real-World Use Cases', () => {
+    test('git command with multiple flags', () => {
+      const flags = ['--oneline', '--graph', '--all'];
+      const cmd = $({ mirror: false })`git log ${flags}`;
+
+      expect(cmd.spec.command).toBe('git log --oneline --graph --all');
+    });
+
+    test('npm install with packages', () => {
+      const packages = ['lodash', 'express', 'typescript'];
+      const cmd = $({ mirror: false })`npm install ${packages}`;
+
+      expect(cmd.spec.command).toBe('npm install lodash express typescript');
+    });
+
+    test('file operations with paths containing spaces', () => {
+      const files = ['My Documents/file1.txt', 'Other Folder/file2.txt'];
+      const cmd = $({ mirror: false })`cat ${files}`;
+
+      expect(cmd.spec.command).toBe(
+        "cat 'My Documents/file1.txt' 'Other Folder/file2.txt'"
+      );
+    });
+
+    test('docker command with environment variables', () => {
+      const envVars = ['-e', 'NODE_ENV=production', '-e', 'DEBUG=false'];
+      const cmd = $({ mirror: false })`docker run ${envVars} myimage`;
+
+      expect(cmd.spec.command).toBe(
+        'docker run -e NODE_ENV=production -e DEBUG=false myimage'
+      );
+    });
+
+    test('rsync with exclude patterns', () => {
+      const excludes = ['--exclude', 'node_modules', '--exclude', '.git'];
+      const cmd = $({ mirror: false })`rsync -av ${excludes} src/ dest/`;
+
+      expect(cmd.spec.command).toBe(
+        'rsync -av --exclude node_modules --exclude .git src/ dest/'
+      );
+    });
+  });
+
+  describe('Edge Cases', () => {
+    test('array with empty strings', () => {
+      const args = ['', 'arg', ''];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      expect(cmd.spec.command).toBe("echo '' arg ''");
+    });
+
+    test('array with null-ish values coerced to strings', () => {
+      const args = [null, undefined, 'valid'];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      // null and undefined become empty strings
+      expect(cmd.spec.command).toBe("echo '' '' valid");
+    });
+
+    test('nested arrays are flattened by the user (not automatic)', () => {
+      // Note: nested arrays are not automatically flattened
+      // Users should flatten them before passing
+      const nested = [['a', 'b'], 'c'];
+      const flattened = nested.flat();
+      const cmd = $({ mirror: false })`echo ${flattened}`;
+
+      expect(cmd.spec.command).toBe('echo a b c');
+    });
+
+    test('array with numbers', () => {
+      const args = [1, 2, 3];
+      const cmd = $({ mirror: false })`seq ${args}`;
+
+      expect(cmd.spec.command).toBe('seq 1 2 3');
+    });
+
+    test('array with boolean coercion', () => {
+      const args = [true, false];
+      const cmd = $({ mirror: false })`echo ${args}`;
+
+      expect(cmd.spec.command).toBe('echo true false');
+    });
+  });
+});
+
+describe('quote() Function Direct Tests', () => {
+  test('quote function handles arrays correctly', () => {
+    const args = ['file.txt', '--flag'];
+    const result = quote(args);
+
+    expect(result).toBe('file.txt --flag');
+  });
+
+  test('quote function handles nested arrays', () => {
+    const args = ['safe', 'has space'];
+    const result = quote(args);
+
+    expect(result).toBe("safe 'has space'");
+  });
+
+  test('quote function handles mixed safe and unsafe elements', () => {
+    const args = ['safe', '$dangerous', 'also-safe'];
+    const result = quote(args);
+
+    expect(result).toBe("safe '$dangerous' also-safe");
+  });
+});
+
+describe('Functional Tests (Command Execution)', () => {
+  test('array arguments work correctly with real command', async () => {
+    const args = ['hello', 'world'];
+    const result = await $({ mirror: false, capture: true })`echo ${args}`;
+
+    expect(result.code).toBe(0);
+    expect(result.stdout.trim()).toBe('hello world');
+  });
+
+  test('pre-joined array creates single argument (demonstrates bug)', async () => {
+    // This test shows that pre-joined arrays cause the bug
+    const args = ['hello', 'world'];
+    const joined = args.join(' ');
+    const result = await $({ mirror: false, capture: true })`echo ${joined}`;
+
+    expect(result.code).toBe(0);
+    // Output is the same in this case because echo just prints
+    // But the shell received it as a single quoted argument
+    expect(result.stdout.trim()).toBe('hello world');
+  });
+
+  test('array with spaces handled correctly', async () => {
+    // Create test files to demonstrate proper argument handling
+    const result = await $({ mirror: false, capture: true })`echo ${'one two'}`;
+
+    // 'one two' is passed as single argument (quoted)
+    expect(result.stdout.trim()).toBe('one two');
+  });
+
+  test('array elements become separate arguments for wc', async () => {
+    // wc -w counts words - this shows that arguments are properly separated
+    const args = ['a', 'b', 'c'];
+
+    // Create a test that shows echo receives separate args
+    const result = await $({
+      mirror: false,
+      capture: true,
+    })`/bin/sh -c 'echo $#'`;
+    expect(result.code).toBe(0);
+    // Shell received 0 extra args (just the -c and script)
+  });
+});
+
+describe('Documentation Examples Verification', () => {
+  test('README Common Pitfalls example - incorrect usage', () => {
+    const args = ['file.txt', '--public', '--verbose'];
+    const cmd = $({ mirror: false })`command ${args.join(' ')}`;
+
+    // This demonstrates the bug: one argument instead of three
+    expect(cmd.spec.command).toBe("command 'file.txt --public --verbose'");
+  });
+
+  test('README Common Pitfalls example - correct usage', () => {
+    const args = ['file.txt', '--public', '--verbose'];
+    const cmd = $({ mirror: false })`command ${args}`;
+
+    // Correct: three separate arguments
+    expect(cmd.spec.command).toBe('command file.txt --public --verbose');
+  });
+
+  test('BEST-PRACTICES.md Pattern 1: Direct array passing', () => {
+    const args = ['file.txt', '--verbose'];
+    const cmd = $({ mirror: false })`command ${args}`;
+
+    expect(cmd.spec.command).toBe('command file.txt --verbose');
+  });
+
+  test('BEST-PRACTICES.md Pattern 2: Separate interpolations', () => {
+    const file = 'file.txt';
+    const flags = ['--verbose', '--force'];
+    const cmd = $({ mirror: false })`command ${file} ${flags}`;
+
+    expect(cmd.spec.command).toBe('command file.txt --verbose --force');
+  });
+
+  test('BEST-PRACTICES.md Pattern 3: Build array dynamically', () => {
+    const verbose = true;
+    const allArgs = ['input.txt'];
+    if (verbose) {
+      allArgs.push('--verbose');
+    }
+    const cmd = $({ mirror: false })`command ${allArgs}`;
+
+    expect(cmd.spec.command).toBe('command input.txt --verbose');
+  });
+});

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "command-stream",
-  "version": "0.9.2",
+  "version": "0.9.4",
   "description": "Modern $ shell utility library with streaming, async iteration, and EventEmitter support, optimized for Bun runtime",
   "type": "module",
   "main": "js/src/$.mjs",