command-stream 0.9.2 → 0.9.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +63 -0
- package/js/BEST-PRACTICES.md +376 -0
- package/js/docs/IMPLEMENTATION_NOTES.md +129 -0
- package/js/docs/SHELL_OPERATORS_IMPLEMENTATION.md +101 -0
- package/js/docs/case-studies/issue-144/README.md +215 -0
- package/js/docs/case-studies/issue-144/failures-summary.md +161 -0
- package/js/docs/case-studies/issue-146/README.md +244 -0
- package/js/docs/case-studies/issue-153/README.md +167 -0
- package/js/docs/shell-operators-implementation.md +97 -0
- package/js/tests/array-interpolation.test.mjs +329 -0
- package/package.json +1 -1
- package/rust/BEST-PRACTICES.md +382 -0
|
@@ -0,0 +1,215 @@
|
|
|
1
|
+
# Case Study: Windows CI Test Failures (Issue #144)
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
This document provides a comprehensive analysis of the Windows test failures in the command-stream library's CI pipeline, including timeline of events, root causes, and proposed solutions.
|
|
6
|
+
|
|
7
|
+
## Timeline of Events
|
|
8
|
+
|
|
9
|
+
### December 27, 2025
|
|
10
|
+
|
|
11
|
+
1. **14:53:56 UTC** - PR #143 created to transition to new CI/CD template with modern best practices
|
|
12
|
+
2. **15:12:12 UTC** - First commit with Windows tests enabled in CI matrix
|
|
13
|
+
3. **15:12:26 UTC** - First Windows test failures detected (Run ID: 20540726833)
|
|
14
|
+
4. **15:17:47 UTC** - macOS and Windows tests temporarily disabled due to platform issues
|
|
15
|
+
5. **15:20:13 UTC** - macOS tests were re-enabled after symlink resolution fixes
|
|
16
|
+
6. **16:03:39 UTC** - Cross-platform CI re-enabled, but Windows tests fail again
|
|
17
|
+
7. **16:13:42 UTC** - Windows tests disabled again pending path separator fixes
|
|
18
|
+
8. **16:24:10 UTC** - PR #143 merged with Windows tests disabled
|
|
19
|
+
9. **18:17:18 UTC** - Issue #144 created to track Windows CI fixes
|
|
20
|
+
|
|
21
|
+
## Test Execution Results (Windows)
|
|
22
|
+
|
|
23
|
+
From CI Run 20541247679 (2025-12-27T16:03:52Z):
|
|
24
|
+
|
|
25
|
+
| Metric | Count |
|
|
26
|
+
| ----------- | ------- |
|
|
27
|
+
| Total tests | 647 |
|
|
28
|
+
| Passed | 595 |
|
|
29
|
+
| Failed | 47 |
|
|
30
|
+
| Skipped | 5 |
|
|
31
|
+
| Errors | 2 |
|
|
32
|
+
| Duration | 175.88s |
|
|
33
|
+
|
|
34
|
+
### Success Rate: 91.96% (595/647)
|
|
35
|
+
|
|
36
|
+
## Root Cause Analysis
|
|
37
|
+
|
|
38
|
+
### Primary Issue: Shell Detection Failure
|
|
39
|
+
|
|
40
|
+
The core problem is that the `findAvailableShell()` function in `src/$.mjs` only looks for Unix-style shells:
|
|
41
|
+
|
|
42
|
+
```javascript
|
|
43
|
+
const shellsToTry = [
|
|
44
|
+
{ cmd: '/bin/sh', args: ['-l', '-c'], checkPath: true },
|
|
45
|
+
{ cmd: '/usr/bin/sh', args: ['-l', '-c'], checkPath: true },
|
|
46
|
+
{ cmd: '/bin/bash', args: ['-l', '-c'], checkPath: true },
|
|
47
|
+
// ... more Unix paths
|
|
48
|
+
{ cmd: 'sh', args: ['-l', '-c'], checkPath: false },
|
|
49
|
+
{ cmd: 'bash', args: ['-l', '-c'], checkPath: false },
|
|
50
|
+
];
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
On Windows, these paths don't exist, leading to:
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
ENOENT: no such file or directory, uv_spawn 'sh'
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
### Secondary Issues
|
|
60
|
+
|
|
61
|
+
1. **Path Separator Differences**
|
|
62
|
+
- Windows uses backslashes (`\`) vs Unix forward slashes (`/`)
|
|
63
|
+
- Some tests with hardcoded paths fail due to this mismatch
|
|
64
|
+
|
|
65
|
+
2. **Signal Handling (SIGINT)**
|
|
66
|
+
- Windows handles process signals differently than Unix
|
|
67
|
+
- CTRL+C behavior is platform-specific
|
|
68
|
+
- Many signal-related tests timeout or fail
|
|
69
|
+
|
|
70
|
+
3. **Temp Directory Paths**
|
|
71
|
+
- Windows uses `C:\Users\RUNNER~1\AppData\Local\Temp\` format
|
|
72
|
+
- Short path notation (8.3) can cause issues with path matching
|
|
73
|
+
|
|
74
|
+
4. **Timing Differences**
|
|
75
|
+
- Windows process spawning is slower
|
|
76
|
+
- Test expectations like `expect(timeToFirstChunk).toBeLessThan(50)` fail
|
|
77
|
+
- Actual value: 366ms vs expected <50ms
|
|
78
|
+
|
|
79
|
+
## Failed Tests Categories
|
|
80
|
+
|
|
81
|
+
### Category 1: Shell Spawn Failures (ENOENT 'sh')
|
|
82
|
+
|
|
83
|
+
- ProcessRunner Options > should handle cwd option
|
|
84
|
+
- Synchronous Execution (.sync()) > Options in Sync Mode > should handle cwd option
|
|
85
|
+
- Start/Run Options Passing > .start() method with options > should work with real shell commands
|
|
86
|
+
- Options Examples (Feature Demo) > example: real shell command vs virtual command
|
|
87
|
+
- And many more shell-dependent tests
|
|
88
|
+
|
|
89
|
+
### Category 2: Path/CD Command Issues
|
|
90
|
+
|
|
91
|
+
- cd Virtual Command - Command Chains > should persist directory change within command chain
|
|
92
|
+
- cd Virtual Command - Edge Cases > should handle cd with trailing slash
|
|
93
|
+
- cd Virtual Command - Edge Cases > should handle cd with multiple slashes
|
|
94
|
+
- Virtual Commands System > Built-in Commands > should execute virtual cd command
|
|
95
|
+
|
|
96
|
+
### Category 3: Signal Handling Timeouts
|
|
97
|
+
|
|
98
|
+
- CTRL+C Signal Handling > should forward SIGINT to child process
|
|
99
|
+
- CTRL+C Different stdin Modes > should handle CTRL+C with string stdin
|
|
100
|
+
- CTRL+C with Different stdin Modes > should bypass virtual commands with custom stdin
|
|
101
|
+
- streaming interfaces - kill method works
|
|
102
|
+
|
|
103
|
+
### Category 4: Timing-Sensitive Tests
|
|
104
|
+
|
|
105
|
+
- command-stream Feature Validation > Real-time Streaming > should stream data as it arrives
|
|
106
|
+
|
|
107
|
+
### Category 5: Platform-Specific Commands
|
|
108
|
+
|
|
109
|
+
- Built-in Commands > Command Location (which) > which should find existing system commands
|
|
110
|
+
- System Command Piping (Issue #8) > Piping to sort > should pipe to sort for sorting lines
|
|
111
|
+
|
|
112
|
+
## Proposed Solutions
|
|
113
|
+
|
|
114
|
+
### Solution 1: Add Windows Shell Detection (Required)
|
|
115
|
+
|
|
116
|
+
Add Windows shells to `findAvailableShell()`:
|
|
117
|
+
|
|
118
|
+
```javascript
|
|
119
|
+
const shellsToTry = [
|
|
120
|
+
// Windows shells (check first on Windows)
|
|
121
|
+
...(process.platform === 'win32'
|
|
122
|
+
? [
|
|
123
|
+
{ cmd: 'cmd.exe', args: ['/c'], checkPath: false },
|
|
124
|
+
{ cmd: 'powershell.exe', args: ['-Command'], checkPath: false },
|
|
125
|
+
{ cmd: 'pwsh.exe', args: ['-Command'], checkPath: false },
|
|
126
|
+
// Git Bash (most compatible)
|
|
127
|
+
{
|
|
128
|
+
cmd: 'C:\\Program Files\\Git\\bin\\bash.exe',
|
|
129
|
+
args: ['-c'],
|
|
130
|
+
checkPath: true,
|
|
131
|
+
},
|
|
132
|
+
{
|
|
133
|
+
cmd: 'C:\\Program Files\\Git\\usr\\bin\\bash.exe',
|
|
134
|
+
args: ['-c'],
|
|
135
|
+
checkPath: true,
|
|
136
|
+
},
|
|
137
|
+
]
|
|
138
|
+
: []),
|
|
139
|
+
// Unix shells
|
|
140
|
+
{ cmd: '/bin/sh', args: ['-l', '-c'], checkPath: true },
|
|
141
|
+
// ... rest of Unix shells
|
|
142
|
+
];
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
### Solution 2: Path Normalization Helper
|
|
146
|
+
|
|
147
|
+
Create a cross-platform path normalization function:
|
|
148
|
+
|
|
149
|
+
```javascript
|
|
150
|
+
function normalizePath(p) {
|
|
151
|
+
if (process.platform === 'win32') {
|
|
152
|
+
// Convert forward slashes to backslashes for Windows
|
|
153
|
+
// Handle UNC paths and drive letters
|
|
154
|
+
return path.normalize(p);
|
|
155
|
+
}
|
|
156
|
+
return p;
|
|
157
|
+
}
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
### Solution 3: Skip/Adjust Platform-Specific Tests
|
|
161
|
+
|
|
162
|
+
Add platform checks to inherently Unix-specific tests:
|
|
163
|
+
|
|
164
|
+
```javascript
|
|
165
|
+
test.skipIf(process.platform === 'win32')(
|
|
166
|
+
'should forward SIGINT...',
|
|
167
|
+
async () => {
|
|
168
|
+
// Unix-specific signal handling
|
|
169
|
+
}
|
|
170
|
+
);
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
### Solution 4: Increase Timing Tolerances
|
|
174
|
+
|
|
175
|
+
Adjust timing expectations for Windows:
|
|
176
|
+
|
|
177
|
+
```javascript
|
|
178
|
+
const MAX_FIRST_CHUNK_TIME = process.platform === 'win32' ? 500 : 50;
|
|
179
|
+
expect(timeToFirstChunk).toBeLessThan(MAX_FIRST_CHUNK_TIME);
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
## Recommendation
|
|
183
|
+
|
|
184
|
+
Given the complexity of full Windows support, I recommend a phased approach:
|
|
185
|
+
|
|
186
|
+
### Phase 1: Quick Wins (This PR)
|
|
187
|
+
|
|
188
|
+
1. Add basic Windows shell detection with Git Bash fallback
|
|
189
|
+
2. Skip tests that are fundamentally incompatible with Windows
|
|
190
|
+
3. Document Windows limitations in README
|
|
191
|
+
|
|
192
|
+
### Phase 2: Future Work
|
|
193
|
+
|
|
194
|
+
1. Implement full cross-platform path handling
|
|
195
|
+
2. Add Windows-specific virtual command implementations
|
|
196
|
+
3. Create Windows-specific test configurations
|
|
197
|
+
|
|
198
|
+
## Files Changed/To Be Changed
|
|
199
|
+
|
|
200
|
+
| File | Change Type | Description |
|
|
201
|
+
| ------------------------------- | ----------- | ------------------------------- |
|
|
202
|
+
| `src/$.mjs` | Modified | Add Windows shell detection |
|
|
203
|
+
| `tests/*.test.mjs` | Modified | Add platform-specific skips |
|
|
204
|
+
| `.github/workflows/release.yml` | Modified | Re-enable Windows in CI matrix |
|
|
205
|
+
| `README.md` | Modified | Document Windows support status |
|
|
206
|
+
|
|
207
|
+
## References
|
|
208
|
+
|
|
209
|
+
- GitHub Actions Run: https://github.com/link-foundation/command-stream/actions/runs/20541247679
|
|
210
|
+
- PR #143: https://github.com/link-foundation/command-stream/pull/143
|
|
211
|
+
- Issue #144: https://github.com/link-foundation/command-stream/issues/144
|
|
212
|
+
|
|
213
|
+
## Appendix: Full Failure List
|
|
214
|
+
|
|
215
|
+
See `failures-summary.md` for the complete list of 47 failed tests with error details.
|
|
@@ -0,0 +1,161 @@
|
|
|
1
|
+
# Windows CI Test Failures - Detailed Summary
|
|
2
|
+
|
|
3
|
+
Run ID: 20541247679
|
|
4
|
+
Date: 2025-12-27T16:03:52Z
|
|
5
|
+
Platform: Windows Server 2025 (10.0.26100)
|
|
6
|
+
Bun Version: 1.3.5
|
|
7
|
+
|
|
8
|
+
## Complete List of 47 Failed Tests
|
|
9
|
+
|
|
10
|
+
### 1. Shell/Spawn Failures (ENOENT 'sh')
|
|
11
|
+
|
|
12
|
+
These failures occur because Windows doesn't have `sh` in the PATH by default:
|
|
13
|
+
|
|
14
|
+
| Test | Duration | Error |
|
|
15
|
+
| -------------------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------- |
|
|
16
|
+
| command-stream Feature Validation > Real-time Streaming > should stream data as it arrives, not buffered | 406ms | Timing expectation failed (366ms > 50ms expected) |
|
|
17
|
+
| ProcessRunner Options > should handle cwd option | - | ENOENT: no such file or directory, uv_spawn 'sh' |
|
|
18
|
+
| Synchronous Execution (.sync()) > Options in Sync Mode > should handle cwd option | - | ENOENT: no such file or directory, uv_spawn 'sh' |
|
|
19
|
+
|
|
20
|
+
### 2. Command Detection Issues
|
|
21
|
+
|
|
22
|
+
| Test | Duration | Error |
|
|
23
|
+
| ------------------------------------------------------------------------------------------------------------ | -------- | ------------------------ |
|
|
24
|
+
| Built-in Commands (Bun.$ compatible) > Command Location (which) > which should find existing system commands | - | System command not found |
|
|
25
|
+
| String interpolation fix for Bun > Shell operators in interpolated commands should work | - | Shell execution failure |
|
|
26
|
+
| Bun-specific shell path tests > Bun.spawn compatibility is maintained | - | Shell not found |
|
|
27
|
+
|
|
28
|
+
### 3. CD Virtual Command Failures
|
|
29
|
+
|
|
30
|
+
Path handling and quoting issues on Windows:
|
|
31
|
+
|
|
32
|
+
| Test | Duration | Error |
|
|
33
|
+
| ------------------------------------------------------------------------------------------ | -------- | ---------------------------- |
|
|
34
|
+
| cd Virtual Command - Command Chains > should persist directory change within command chain | 32ms | Path resolution failure |
|
|
35
|
+
| cd Virtual Command - Command Chains > should handle multiple cd commands in chain | 31ms | Path resolution failure |
|
|
36
|
+
| cd Virtual Command - Command Chains > should work with git commands in chain | 31ms | Path resolution failure |
|
|
37
|
+
| cd Virtual Command - Edge Cases > should handle cd with trailing slash | 32ms | ENOENT with quoted paths |
|
|
38
|
+
| cd Virtual Command - Edge Cases > should handle cd with multiple slashes | 31ms | ENOENT with multiple slashes |
|
|
39
|
+
| Virtual Commands System > Built-in Commands > should execute virtual cd command | 31ms | Path handling failure |
|
|
40
|
+
|
|
41
|
+
Specific error example:
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
ENOENT: no such file or directory, chdir 'D:\a\command-stream\command-stream\' -> ''C:\Users\RUNNER~1\AppData\Local\Temp\cd-slash-NXM4ex'/'
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
### 4. SIGINT/Signal Handling Failures
|
|
48
|
+
|
|
49
|
+
Windows handles signals differently than Unix:
|
|
50
|
+
|
|
51
|
+
| Test | Duration | Error |
|
|
52
|
+
| --------------------------------------------------------------------------------------------------------------- | -------- | ------------------------- |
|
|
53
|
+
| CTRL+C Baseline Tests (Native Spawn) > should handle Node.js inline script with SIGINT | 531ms | Signal handling failure |
|
|
54
|
+
| CTRL+C Baseline Tests (Native Spawn) > should handle Node.js script file | 1015ms | Signal handling failure |
|
|
55
|
+
| CTRL+C Different stdin Modes > should handle CTRL+C with string stdin | 5015ms | **TIMEOUT** |
|
|
56
|
+
| CTRL+C Different stdin Modes > should handle CTRL+C with Buffer stdin | 5016ms | **TIMEOUT** |
|
|
57
|
+
| CTRL+C Signal Handling > should forward SIGINT to child process when external CTRL+C is sent | 3000ms | Signal not forwarded |
|
|
58
|
+
| CTRL+C Signal Handling > should not interfere with user SIGINT handling when no children active | 547ms | Handler conflict |
|
|
59
|
+
| CTRL+C Signal Handling > should not interfere with child process signal handlers | 1031ms | Handler conflict |
|
|
60
|
+
| CTRL+C with Different stdin Modes > should bypass virtual commands with custom stdin for proper signal handling | 547ms | Signal handling failure |
|
|
61
|
+
| CTRL+C with Different stdin Modes > should handle Bun vs Node.js signal differences | 1047ms | Platform difference |
|
|
62
|
+
| CTRL+C with Different stdin Modes > should properly cancel virtual commands and respect user SIGINT handlers | 547ms | Handler cleanup issue |
|
|
63
|
+
| SIGINT Cleanup Tests (Isolated) > should forward SIGINT to child processes | 547ms | Signal forwarding failure |
|
|
64
|
+
|
|
65
|
+
### 5. Git/GH Command Integration Failures
|
|
66
|
+
|
|
67
|
+
| Test | Duration | Error |
|
|
68
|
+
| --------------------------------------------------------------------------------------------------------------------------------------------- | -------- | --------------------- |
|
|
69
|
+
| Git and GH commands with cd virtual command > Git operations in temp directories > should handle git commands in temp directory with cd chain | 31ms | Path resolution |
|
|
70
|
+
| Git and GH commands with cd virtual command > Git operations in temp directories > should handle git branch operations with cd | 31ms | Path resolution |
|
|
71
|
+
| Git and GH commands with cd virtual command > Git operations in temp directories > should handle multiple temp directories with cd | 47ms | Path resolution |
|
|
72
|
+
| Git and GH commands with cd virtual command > Git operations in temp directories > should handle git diff operations after cd | 47ms | Path resolution |
|
|
73
|
+
| Git and GH commands with cd virtual command > Combined git and gh workflows > should simulate solve.mjs workflow pattern | 5047ms | **TIMEOUT** |
|
|
74
|
+
| Git and GH commands with cd virtual command > Combined git and gh workflows > should preserve cwd after command chains | 1015ms | CWD not preserved |
|
|
75
|
+
| Git and GH commands with cd virtual command > Combined git and gh workflows > should work with complex git workflows using operators | 31ms | Operator failure |
|
|
76
|
+
| Git and GH commands with cd virtual command > Path resolution and quoting with cd > should handle paths with spaces in git operations | 32ms | Quote handling |
|
|
77
|
+
| Git and GH commands with cd virtual command > Path resolution and quoting with cd > should handle special characters in paths | 31ms | Special char escaping |
|
|
78
|
+
|
|
79
|
+
### 6. GitHub CLI (gh) Failures
|
|
80
|
+
|
|
81
|
+
| Test | Duration | Error |
|
|
82
|
+
| ------------------------------------------------------------------------------------------------- | -------- | --------------- |
|
|
83
|
+
| GitHub CLI (gh) commands > gh auth status returns correct exit code and output structure | 5047ms | **TIMEOUT** |
|
|
84
|
+
| Examples Execution Tests > should not interfere with user SIGINT handling when no children active | 500ms | Signal handling |
|
|
85
|
+
|
|
86
|
+
### 7. jq Streaming Failures
|
|
87
|
+
|
|
88
|
+
| Test | Duration | Error |
|
|
89
|
+
| ----------------------------------------------------------------------------------------- | -------- | ------------- |
|
|
90
|
+
| jq streaming tests > stream of JSON objects through jq -c | 1015ms | Pipe handling |
|
|
91
|
+
| jq streaming tests > generate and process array elements as stream | 782ms | Pipe handling |
|
|
92
|
+
| jq streaming with pipe \| syntax > stream of JSON objects through jq -c using pipe syntax | 140ms | Pipe syntax |
|
|
93
|
+
| jq streaming with pipe \| syntax > process array elements as stream using pipe syntax | 110ms | Pipe syntax |
|
|
94
|
+
|
|
95
|
+
### 8. Shell Feature Failures
|
|
96
|
+
|
|
97
|
+
| Test | Duration | Error |
|
|
98
|
+
| ------------------------------------------------------------------------------------------------------------------------ | -------- | ----------------- |
|
|
99
|
+
| Shell Settings (set -e / set +e equivalent) > Shell Replacement Benefits > should provide better error objects than bash | 125ms | Shell differences |
|
|
100
|
+
| Cleanup Verification > should not affect cwd when cd is in subshell | 406ms | Subshell handling |
|
|
101
|
+
| Options Examples (Feature Demo) > example: real shell command vs virtual command | 16ms | Shell not found |
|
|
102
|
+
|
|
103
|
+
### 9. Output/Streaming Failures
|
|
104
|
+
|
|
105
|
+
| Test | Duration | Error |
|
|
106
|
+
| -------------------------------------------------------------------------------------------------------- | -------- | ------------------- |
|
|
107
|
+
| Start/Run Edge Cases and Advanced Usage > should work with real shell commands that produce large output | - | Shell spawn failure |
|
|
108
|
+
| Start/Run Options Passing > .start() method with options > should work with real shell commands | 16ms | Shell spawn failure |
|
|
109
|
+
| Stderr output handling in $.mjs > long-running commands with stderr output should not hang | 812ms | Stream handling |
|
|
110
|
+
| streaming interfaces - kill method works | 5015ms | **TIMEOUT** |
|
|
111
|
+
|
|
112
|
+
### 10. System Command Piping Failures
|
|
113
|
+
|
|
114
|
+
| Test | Duration | Error |
|
|
115
|
+
| ----------------------------------------------------------------------------------------- | -------- | ------------------------ |
|
|
116
|
+
| System Command Piping (Issue #8) > Piping to sort > should pipe to sort for sorting lines | 16ms | sort command differences |
|
|
117
|
+
| System Command Piping (Issue #8) > Piping to sort > should handle sort with reverse flag | 16ms | sort -r differences |
|
|
118
|
+
|
|
119
|
+
## Key Error Messages
|
|
120
|
+
|
|
121
|
+
### ENOENT Shell Spawn
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
ENOENT: no such file or directory, uv_spawn 'sh'
|
|
125
|
+
path: "sh",
|
|
126
|
+
syscall: "uv_spawn",
|
|
127
|
+
errno: -4058,
|
|
128
|
+
code: "ENOENT"
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
### Path Resolution with Quotes
|
|
132
|
+
|
|
133
|
+
```
|
|
134
|
+
ENOENT: no such file or directory, chdir 'D:\a\command-stream\command-stream\' -> ''C:\Users\RUNNER~1\AppData\Local\Temp\cd-slash-NXM4ex'/'
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### Signal Handling
|
|
138
|
+
|
|
139
|
+
```
|
|
140
|
+
kill() failed: ESRCH: no such process
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
## Statistics
|
|
144
|
+
|
|
145
|
+
- **Total failures: 47**
|
|
146
|
+
- **Timeout failures: 6** (tests that exceeded 5000ms limit)
|
|
147
|
+
- **ENOENT failures: ~20** (shell not found)
|
|
148
|
+
- **Signal handling failures: ~11**
|
|
149
|
+
- **Path/CD failures: ~10**
|
|
150
|
+
|
|
151
|
+
## Environment Details
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
Platform: win32
|
|
155
|
+
OS: Microsoft Windows Server 2025 10.0.26100
|
|
156
|
+
Runner: GitHub Actions windows-latest (windows-2025)
|
|
157
|
+
Bun: 1.3.5+1e86cebd7
|
|
158
|
+
Git: 2.52.0.windows.1
|
|
159
|
+
PowerShell: 7.x available
|
|
160
|
+
Git Bash: Available at C:\Program Files\Git\bin\bash.exe
|
|
161
|
+
```
|
|
@@ -0,0 +1,244 @@
|
|
|
1
|
+
# Case Study: JavaScript to Rust Translation (Issue #146)
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
This document provides a comprehensive analysis of the process, challenges, and lessons learned from translating the command-stream JavaScript library to Rust.
|
|
6
|
+
|
|
7
|
+
## Project Overview
|
|
8
|
+
|
|
9
|
+
### Original JavaScript Codebase
|
|
10
|
+
|
|
11
|
+
- **Main file**: `src/$.mjs` (~6,765 lines)
|
|
12
|
+
- **Shell parser**: `src/shell-parser.mjs` (~403 lines)
|
|
13
|
+
- **Utilities**: `src/$.utils.mjs` (~101 lines)
|
|
14
|
+
- **Virtual commands**: 21 command files in `src/commands/`
|
|
15
|
+
- **Total**: ~8,400 lines of JavaScript
|
|
16
|
+
|
|
17
|
+
### Rust Translation
|
|
18
|
+
|
|
19
|
+
- **Main library**: `rust/src/lib.rs`
|
|
20
|
+
- **Shell parser**: `rust/src/shell_parser.rs`
|
|
21
|
+
- **Utilities**: `rust/src/utils.rs`
|
|
22
|
+
- **Virtual commands**: 21 command modules in `rust/src/commands/`
|
|
23
|
+
|
|
24
|
+
## Timeline of Development
|
|
25
|
+
|
|
26
|
+
### Phase 1: Code Organization
|
|
27
|
+
|
|
28
|
+
1. Created `js/` folder structure to house JavaScript code
|
|
29
|
+
2. Updated `package.json` to point to new `js/src/` location
|
|
30
|
+
3. Updated all import statements in tests and examples
|
|
31
|
+
|
|
32
|
+
### Phase 2: Rust Project Setup
|
|
33
|
+
|
|
34
|
+
1. Created `rust/` folder with Cargo.toml
|
|
35
|
+
2. Defined dependencies:
|
|
36
|
+
- `tokio` for async runtime
|
|
37
|
+
- `which` for command lookup
|
|
38
|
+
- `nix` for Unix signal handling
|
|
39
|
+
- `regex` for pattern matching
|
|
40
|
+
- `chrono` for timestamps
|
|
41
|
+
- `filetime` for file timestamp operations
|
|
42
|
+
|
|
43
|
+
### Phase 3: Core Translation
|
|
44
|
+
|
|
45
|
+
1. Translated shell parser (tokenizer, parser, AST types)
|
|
46
|
+
2. Translated utilities (tracing, command results, ANSI handling)
|
|
47
|
+
3. Translated main library (ProcessRunner, shell detection)
|
|
48
|
+
4. Translated all 21 virtual commands
|
|
49
|
+
|
|
50
|
+
## Key Translation Patterns
|
|
51
|
+
|
|
52
|
+
### 1. JavaScript Async to Rust Async
|
|
53
|
+
|
|
54
|
+
**JavaScript:**
|
|
55
|
+
|
|
56
|
+
```javascript
|
|
57
|
+
async function sleep({ args, abortSignal }) {
|
|
58
|
+
const seconds = parseFloat(args[0] || 0);
|
|
59
|
+
await new Promise((resolve) => setTimeout(resolve, seconds * 1000));
|
|
60
|
+
return { stdout: '', code: 0 };
|
|
61
|
+
}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
**Rust:**
|
|
65
|
+
|
|
66
|
+
```rust
|
|
67
|
+
pub async fn sleep(ctx: CommandContext) -> CommandResult {
|
|
68
|
+
let seconds: f64 = ctx.args.first()
|
|
69
|
+
.and_then(|s| s.parse().ok())
|
|
70
|
+
.unwrap_or(0.0);
|
|
71
|
+
|
|
72
|
+
tokio::time::sleep(Duration::from_secs_f64(seconds)).await;
|
|
73
|
+
CommandResult::success_empty()
|
|
74
|
+
}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### 2. JavaScript Object Literals to Rust Structs
|
|
78
|
+
|
|
79
|
+
**JavaScript:**
|
|
80
|
+
|
|
81
|
+
```javascript
|
|
82
|
+
const result = {
|
|
83
|
+
stdout: output,
|
|
84
|
+
stderr: '',
|
|
85
|
+
code: 0,
|
|
86
|
+
async text() {
|
|
87
|
+
return this.stdout;
|
|
88
|
+
},
|
|
89
|
+
};
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
**Rust:**
|
|
93
|
+
|
|
94
|
+
```rust
|
|
95
|
+
#[derive(Debug, Clone)]
|
|
96
|
+
pub struct CommandResult {
|
|
97
|
+
pub stdout: String,
|
|
98
|
+
pub stderr: String,
|
|
99
|
+
pub code: i32,
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
impl CommandResult {
|
|
103
|
+
pub fn success(stdout: impl Into<String>) -> Self {
|
|
104
|
+
CommandResult {
|
|
105
|
+
stdout: stdout.into(),
|
|
106
|
+
stderr: String::new(),
|
|
107
|
+
code: 0,
|
|
108
|
+
}
|
|
109
|
+
}
|
|
110
|
+
}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### 3. JavaScript Closures to Rust Trait Objects
|
|
114
|
+
|
|
115
|
+
**JavaScript:**
|
|
116
|
+
|
|
117
|
+
```javascript
|
|
118
|
+
function trace(category, messageOrFunc) {
|
|
119
|
+
const message =
|
|
120
|
+
typeof messageOrFunc === 'function' ? messageOrFunc() : messageOrFunc;
|
|
121
|
+
console.error(`[TRACE] [${category}] ${message}`);
|
|
122
|
+
}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
**Rust:**
|
|
126
|
+
|
|
127
|
+
```rust
|
|
128
|
+
pub fn trace_lazy<F>(category: &str, message_fn: F)
|
|
129
|
+
where
|
|
130
|
+
F: FnOnce() -> String,
|
|
131
|
+
{
|
|
132
|
+
if !is_trace_enabled() {
|
|
133
|
+
return;
|
|
134
|
+
}
|
|
135
|
+
trace(category, &message_fn());
|
|
136
|
+
}
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
### 4. JavaScript Error Handling to Rust Result Types
|
|
140
|
+
|
|
141
|
+
**JavaScript:**
|
|
142
|
+
|
|
143
|
+
```javascript
|
|
144
|
+
try {
|
|
145
|
+
const content = fs.readFileSync(path, 'utf8');
|
|
146
|
+
return { stdout: content, code: 0 };
|
|
147
|
+
} catch (error) {
|
|
148
|
+
if (error.code === 'ENOENT') {
|
|
149
|
+
return { stderr: `cat: ${file}: No such file or directory`, code: 1 };
|
|
150
|
+
}
|
|
151
|
+
throw error;
|
|
152
|
+
}
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
**Rust:**
|
|
156
|
+
|
|
157
|
+
```rust
|
|
158
|
+
match fs::read_to_string(&path) {
|
|
159
|
+
Ok(content) => CommandResult::success(content),
|
|
160
|
+
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
|
|
161
|
+
CommandResult::error(format!("cat: {}: No such file or directory\n", file))
|
|
162
|
+
}
|
|
163
|
+
Err(e) => CommandResult::error(format!("cat: {}: {}\n", file, e)),
|
|
164
|
+
}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
## Challenges Encountered
|
|
168
|
+
|
|
169
|
+
### 1. Tagged Template Literals
|
|
170
|
+
|
|
171
|
+
JavaScript's tagged template literal syntax `$\`echo hello\``has no direct Rust equivalent. We implemented the`$()` function as a regular function call instead.
|
|
172
|
+
|
|
173
|
+
### 2. Event Emitter Pattern
|
|
174
|
+
|
|
175
|
+
JavaScript's EventEmitter pattern required translation to Rust's channel-based communication using `tokio::sync::mpsc`.
|
|
176
|
+
|
|
177
|
+
### 3. Process Group Handling
|
|
178
|
+
|
|
179
|
+
Unix process group management differs between Node.js and Rust. We used the `nix` crate for proper signal handling.
|
|
180
|
+
|
|
181
|
+
### 4. Async Iterator Pattern
|
|
182
|
+
|
|
183
|
+
JavaScript's `for await (const chunk of stream)` was translated to Rust's async stream patterns using channels.
|
|
184
|
+
|
|
185
|
+
## Lessons Learned
|
|
186
|
+
|
|
187
|
+
### 1. Type Safety Benefits
|
|
188
|
+
|
|
189
|
+
Rust's type system caught several edge cases that existed in the JavaScript code:
|
|
190
|
+
|
|
191
|
+
- Null/undefined handling became explicit with `Option<T>`
|
|
192
|
+
- Error handling became explicit with `Result<T, E>`
|
|
193
|
+
- String encoding issues were caught at compile time
|
|
194
|
+
|
|
195
|
+
### 2. Memory Management
|
|
196
|
+
|
|
197
|
+
Rust's ownership model required explicit decisions about:
|
|
198
|
+
|
|
199
|
+
- When to clone vs borrow data
|
|
200
|
+
- Lifetime of process handles
|
|
201
|
+
- Cleanup of resources on cancellation
|
|
202
|
+
|
|
203
|
+
### 3. Cross-Platform Considerations
|
|
204
|
+
|
|
205
|
+
Both JavaScript and Rust require platform-specific code for:
|
|
206
|
+
|
|
207
|
+
- Shell detection (Windows vs Unix)
|
|
208
|
+
- Signal handling (SIGINT, SIGTERM)
|
|
209
|
+
- File permissions
|
|
210
|
+
|
|
211
|
+
### 4. Testing Strategy
|
|
212
|
+
|
|
213
|
+
Unit tests were essential for:
|
|
214
|
+
|
|
215
|
+
- Verifying parity with JavaScript behavior
|
|
216
|
+
- Catching edge cases early
|
|
217
|
+
- Documenting expected behavior
|
|
218
|
+
|
|
219
|
+
## Architecture Comparison
|
|
220
|
+
|
|
221
|
+
| Component | JavaScript | Rust |
|
|
222
|
+
| --------------- | ---------------------- | ----------------------- |
|
|
223
|
+
| Async Runtime | Node.js/Bun event loop | Tokio |
|
|
224
|
+
| Process Spawn | child_process.spawn | tokio::process::Command |
|
|
225
|
+
| Channels | EventEmitter | mpsc channels |
|
|
226
|
+
| Error Handling | try/catch | Result<T, E> |
|
|
227
|
+
| String Handling | UTF-16 strings | UTF-8 String |
|
|
228
|
+
| File I/O | fs module | std::fs |
|
|
229
|
+
| Signal Handling | process.on('SIGINT') | tokio::signal |
|
|
230
|
+
|
|
231
|
+
## Future Improvements
|
|
232
|
+
|
|
233
|
+
1. **Streaming Improvements**: Implement async iterator traits for better streaming support
|
|
234
|
+
2. **Error Types**: Create more specific error types for different failure modes
|
|
235
|
+
3. **Performance**: Benchmark and optimize critical paths
|
|
236
|
+
4. **Platform Support**: Add more Windows-specific implementations
|
|
237
|
+
5. **CI/CD**: Add Rust builds to existing CI pipeline
|
|
238
|
+
|
|
239
|
+
## References
|
|
240
|
+
|
|
241
|
+
- Original Issue: https://github.com/link-foundation/command-stream/issues/146
|
|
242
|
+
- Pull Request: https://github.com/link-foundation/command-stream/pull/147
|
|
243
|
+
- Rust Book: https://doc.rust-lang.org/book/
|
|
244
|
+
- Tokio Documentation: https://tokio.rs/
|