rspec-agents 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/bin/rspec-agents +24 -0
- data/lib/async_workers/channel_config.rb +34 -0
- data/lib/async_workers/doc/process_manager_design.md +512 -0
- data/lib/async_workers/errors.rb +21 -0
- data/lib/async_workers/managed_process.rb +284 -0
- data/lib/async_workers/output_stream.rb +86 -0
- data/lib/async_workers/rpc_channel.rb +159 -0
- data/lib/async_workers/transport/base.rb +57 -0
- data/lib/async_workers/transport/stdio_transport.rb +91 -0
- data/lib/async_workers/transport/unix_socket_transport.rb +112 -0
- data/lib/async_workers/worker_group.rb +175 -0
- data/lib/async_workers.rb +17 -0
- data/lib/rspec/agents/agent_response.rb +61 -0
- data/lib/rspec/agents/agents/base.rb +123 -0
- data/lib/rspec/agents/cli.rb +342 -0
- data/lib/rspec/agents/conversation.rb +308 -0
- data/lib/rspec/agents/criterion.rb +237 -0
- data/lib/rspec/agents/doc/2026_01_22_observer-system-design.md +757 -0
- data/lib/rspec/agents/doc/2026_01_23_parallel_spec_runner-design.md +1060 -0
- data/lib/rspec/agents/doc/2026_01_27_event_serialization-design.md +294 -0
- data/lib/rspec/agents/doc/2026_01_27_experiment_aggregation_design.md +831 -0
- data/lib/rspec/agents/doc/2026_01_29_rspec-agents-studio-design.md +1332 -0
- data/lib/rspec/agents/doc/2026_01_29_testing-framework-design.md +1037 -0
- data/lib/rspec/agents/doc/2026_02_04-parallel-runner-ui.md +537 -0
- data/lib/rspec/agents/doc/2026_02_05_html_renderer_extensions.md +708 -0
- data/lib/rspec/agents/doc/scenario_guide.md +289 -0
- data/lib/rspec/agents/dsl/agent_proxy.rb +141 -0
- data/lib/rspec/agents/dsl/criterion_definition.rb +78 -0
- data/lib/rspec/agents/dsl/graph_builder.rb +38 -0
- data/lib/rspec/agents/dsl/runner_factory.rb +52 -0
- data/lib/rspec/agents/dsl/scenario_set_dsl.rb +166 -0
- data/lib/rspec/agents/dsl/test_context.rb +223 -0
- data/lib/rspec/agents/dsl/user_proxy.rb +71 -0
- data/lib/rspec/agents/dsl.rb +398 -0
- data/lib/rspec/agents/evaluation_result.rb +44 -0
- data/lib/rspec/agents/event_bus.rb +78 -0
- data/lib/rspec/agents/events.rb +141 -0
- data/lib/rspec/agents/isolated_event_bus.rb +86 -0
- data/lib/rspec/agents/judge.rb +244 -0
- data/lib/rspec/agents/llm/anthropic.rb +143 -0
- data/lib/rspec/agents/llm/base.rb +64 -0
- data/lib/rspec/agents/llm/mock.rb +181 -0
- data/lib/rspec/agents/llm/response.rb +52 -0
- data/lib/rspec/agents/matchers.rb +554 -0
- data/lib/rspec/agents/message.rb +81 -0
- data/lib/rspec/agents/metadata.rb +120 -0
- data/lib/rspec/agents/observers/base.rb +70 -0
- data/lib/rspec/agents/observers/parallel_terminal_observer.rb +151 -0
- data/lib/rspec/agents/observers/rpc_notify_observer.rb +43 -0
- data/lib/rspec/agents/observers/terminal_observer.rb +103 -0
- data/lib/rspec/agents/parallel/controller.rb +284 -0
- data/lib/rspec/agents/parallel/example_discovery.rb +153 -0
- data/lib/rspec/agents/parallel/partitioner.rb +31 -0
- data/lib/rspec/agents/parallel/run_result.rb +22 -0
- data/lib/rspec/agents/parallel/ui/interactive_ui.rb +605 -0
- data/lib/rspec/agents/parallel/ui/interleaved_ui.rb +139 -0
- data/lib/rspec/agents/parallel/ui/output_adapter.rb +127 -0
- data/lib/rspec/agents/parallel/ui/quiet_ui.rb +100 -0
- data/lib/rspec/agents/parallel/ui/ui_factory.rb +53 -0
- data/lib/rspec/agents/parallel/ui/ui_mode.rb +101 -0
- data/lib/rspec/agents/prompt_builders/base.rb +113 -0
- data/lib/rspec/agents/prompt_builders/criterion_evaluation.rb +136 -0
- data/lib/rspec/agents/prompt_builders/goal_achievement_evaluation.rb +142 -0
- data/lib/rspec/agents/prompt_builders/grounding_evaluation.rb +172 -0
- data/lib/rspec/agents/prompt_builders/intent_evaluation.rb +111 -0
- data/lib/rspec/agents/prompt_builders/topic_classification.rb +105 -0
- data/lib/rspec/agents/prompt_builders/user_simulation.rb +131 -0
- data/lib/rspec/agents/runners/headless_runner.rb +272 -0
- data/lib/rspec/agents/runners/parallel_terminal_runner.rb +220 -0
- data/lib/rspec/agents/runners/terminal_runner.rb +186 -0
- data/lib/rspec/agents/runners/user_simulator.rb +261 -0
- data/lib/rspec/agents/scenario.rb +133 -0
- data/lib/rspec/agents/scenario_loader.rb +145 -0
- data/lib/rspec/agents/serialization/conversation_renderer.rb +161 -0
- data/lib/rspec/agents/serialization/extension.rb +199 -0
- data/lib/rspec/agents/serialization/extensions/core_extension.rb +66 -0
- data/lib/rspec/agents/serialization/presenters.rb +281 -0
- data/lib/rspec/agents/serialization/run_data_aggregator.rb +197 -0
- data/lib/rspec/agents/serialization/run_data_builder.rb +189 -0
- data/lib/rspec/agents/serialization/templates/_alpine.min.js +5 -0
- data/lib/rspec/agents/serialization/templates/_base_components.css +196 -0
- data/lib/rspec/agents/serialization/templates/_base_components.js +46 -0
- data/lib/rspec/agents/serialization/templates/_conversation_fragment.html.haml +34 -0
- data/lib/rspec/agents/serialization/templates/_metadata_default.html.haml +17 -0
- data/lib/rspec/agents/serialization/templates/_scripts.js +89 -0
- data/lib/rspec/agents/serialization/templates/_styles.css +1211 -0
- data/lib/rspec/agents/serialization/templates/conversation_document.html.haml +29 -0
- data/lib/rspec/agents/serialization/templates/test_suite.html.haml +238 -0
- data/lib/rspec/agents/serialization/test_suite_renderer.rb +207 -0
- data/lib/rspec/agents/serialization.rb +374 -0
- data/lib/rspec/agents/simulator_config.rb +336 -0
- data/lib/rspec/agents/spec_executor.rb +494 -0
- data/lib/rspec/agents/stable_example_id.rb +147 -0
- data/lib/rspec/agents/templates/user_simulation.erb +9 -0
- data/lib/rspec/agents/tool_call.rb +53 -0
- data/lib/rspec/agents/topic.rb +307 -0
- data/lib/rspec/agents/topic_graph.rb +236 -0
- data/lib/rspec/agents/triggers.rb +122 -0
- data/lib/rspec/agents/turn.rb +63 -0
- data/lib/rspec/agents/turn_executor.rb +91 -0
- data/lib/rspec/agents/version.rb +7 -0
- data/lib/rspec/agents.rb +145 -0
- metadata +242 -0
|
@@ -0,0 +1,537 @@
|
|
|
1
|
+
# Parallel Runner UI Design
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
The parallel runner executes multiple spec conversations concurrently. This document describes three output modes to address different use cases and environments.
|
|
6
|
+
|
|
7
|
+
## Output Modes
|
|
8
|
+
|
|
9
|
+
| Mode | Flag | Use Case | Default When |
|
|
10
|
+
|------|------|----------|--------------|
|
|
11
|
+
| Interactive | `--ui=interactive` | Local development, watching tests run | TTY detected, ≥80x24 terminal |
|
|
12
|
+
| Interleaved | `--ui=interleaved` | CI environments, piped output | Non-TTY or `CI` env var set |
|
|
13
|
+
| Quiet | `--ui=quiet` | Large test suites, log analysis | Never (explicit opt-in) |
|
|
14
|
+
|
|
15
|
+
### Mode Selection Logic
|
|
16
|
+
|
|
17
|
+
```ruby
|
|
18
|
+
def select_ui_mode(explicit_mode:, output:)
|
|
19
|
+
return explicit_mode if explicit_mode
|
|
20
|
+
|
|
21
|
+
if !output.tty? || ENV["CI"]
|
|
22
|
+
:interleaved
|
|
23
|
+
elsif terminal_size_sufficient?(80, 24)
|
|
24
|
+
:interactive
|
|
25
|
+
else
|
|
26
|
+
:interleaved
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## Mode 1: Interactive (Tabbed Interface)
|
|
34
|
+
|
|
35
|
+
Full-screen terminal UI with progress tracking and per-worker conversation views.
|
|
36
|
+
|
|
37
|
+
### Layout
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
╭─────────────────────────────────────────────────────────────────────────────╮
|
|
41
|
+
│ ⚡ Running specs [████████░░░░░░░░░░░░] 12/25 2 ✗ 10 ✓ 4 workers │
|
|
42
|
+
╰─────────────────────────────────────────────────────────────────────────────╯
|
|
43
|
+
|
|
44
|
+
[1 ◐] 2 ○ 3 ✓ 4 ✗ [f]ollow on
|
|
45
|
+
|
|
46
|
+
┌─ Worker 1: creates user account ────────────────────────────────────────────┐
|
|
47
|
+
│ │
|
|
48
|
+
│ User: Create an account for john@example.com │
|
|
49
|
+
│ Agent: I'll create that account for you... │
|
|
50
|
+
│ → create_user(email: "john@example.com") │
|
|
51
|
+
│ Agent: The account has been created successfully. │
|
|
52
|
+
│ User: Now verify the email was sent │
|
|
53
|
+
│ Agent: Checking the email queue... │
|
|
54
|
+
│ → check_email_queue(recipient: "john@example.com") │
|
|
55
|
+
│ Agent: ▌ │
|
|
56
|
+
│ │
|
|
57
|
+
│ │
|
|
58
|
+
│ │
|
|
59
|
+
└─────────────────────────────────────────────────────────────────────────────┘
|
|
60
|
+
1-4 switch · ←→ prev/next · ↑↓ scroll · f follow · a auto-rotate · q quit
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### Components
|
|
64
|
+
|
|
65
|
+
#### Progress Header
|
|
66
|
+
|
|
67
|
+
Always visible at the top. Shows overall test run status.
|
|
68
|
+
|
|
69
|
+
```
|
|
70
|
+
⚡ Running specs [████████░░░░░░░░░░░░] 12/25 2 ✗ 10 ✓ 4 workers
|
|
71
|
+
▲ ▲ ▲ ▲ ▲
|
|
72
|
+
│ │ │ │ └─ worker count
|
|
73
|
+
│ │ │ └─ passed (green)
|
|
74
|
+
│ │ └─ failed (red)
|
|
75
|
+
│ └─ completed/total
|
|
76
|
+
└─ visual progress bar
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Progress bar characters: `█` filled, `░` empty.
|
|
80
|
+
|
|
81
|
+
When complete, header updates to show final status:
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
⚡ Completed [████████████████████] 25/25 2 ✗ 23 ✓ 1.2s
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
#### Tab Bar
|
|
88
|
+
|
|
89
|
+
Shows all workers with status indicators. Selected tab is highlighted.
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
[1 ◐] 2 ○ 3 ✓ 4 ✗
|
|
93
|
+
▲ ▲ ▲ ▲
|
|
94
|
+
│ │ │ └─ status indicator
|
|
95
|
+
│ │ └─ worker number
|
|
96
|
+
│ └─ status indicator (spinner when active)
|
|
97
|
+
└─ selected (brackets)
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Status indicators:
|
|
101
|
+
|
|
102
|
+
| Symbol | Meaning | Color |
|
|
103
|
+
|--------|---------|-------|
|
|
104
|
+
| `◐` `◓` `◑` `◒` | Running (animated) | cyan |
|
|
105
|
+
| `○` | Idle/waiting | dim white |
|
|
106
|
+
| `✓` | Last example passed | green |
|
|
107
|
+
| `✗` | Last example failed | red |
|
|
108
|
+
| `⏸` | Pending | yellow |
|
|
109
|
+
|
|
110
|
+
The spinner animates through `◐ → ◓ → ◑ → ◒` at ~200ms intervals.
|
|
111
|
+
|
|
112
|
+
Right side of tab bar shows current mode indicator: `[f]ollow on` or `[a]uto-rotate`.
|
|
113
|
+
|
|
114
|
+
#### Content Pane
|
|
115
|
+
|
|
116
|
+
Scrollable view of the selected worker's conversation history.
|
|
117
|
+
|
|
118
|
+
Header shows worker number and current example:
|
|
119
|
+
|
|
120
|
+
```
|
|
121
|
+
┌─ Worker 1: creates user account ─────────────────────────────────────────────┐
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
When example completes, header updates:
|
|
125
|
+
|
|
126
|
+
```
|
|
127
|
+
┌─ Worker 1: creates user account ✓ (1.2s) ────────────────────────────────────┐
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Content area displays conversation events:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
User: Create an account for john@example.com
|
|
134
|
+
Agent: I'll create that account for you...
|
|
135
|
+
→ create_user(email: "john@example.com")
|
|
136
|
+
Agent: The account has been created successfully.
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Event formatting:
|
|
140
|
+
|
|
141
|
+
| Event | Format |
|
|
142
|
+
|-------|--------|
|
|
143
|
+
| User message | `User: {text}` (dim label) |
|
|
144
|
+
| Agent response | `Agent: {text}` (dim label) |
|
|
145
|
+
| Tool call | ` → {tool_name}({args})` (magenta, indented) |
|
|
146
|
+
| Example started | `○ {description}...` |
|
|
147
|
+
| Example passed | `✓ {description} ({duration})` (green) |
|
|
148
|
+
| Example failed | `✗ {description}` (red) + error message |
|
|
149
|
+
|
|
150
|
+
When worker is idle:
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
│ │
|
|
154
|
+
│ Waiting for next example... │
|
|
155
|
+
│ │
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
When worker completes all work:
|
|
159
|
+
|
|
160
|
+
```
|
|
161
|
+
│ │
|
|
162
|
+
│ ✓ Worker finished (3 examples) │
|
|
163
|
+
│ │
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
#### Help Bar
|
|
167
|
+
|
|
168
|
+
Always visible at the bottom. Shows available keyboard shortcuts.
|
|
169
|
+
|
|
170
|
+
```
|
|
171
|
+
1-4 switch · ←→ prev/next · ↑↓ scroll · f follow · a auto-rotate · q quit
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
### Keyboard Controls
|
|
175
|
+
|
|
176
|
+
| Key | Action |
|
|
177
|
+
|-----|--------|
|
|
178
|
+
| `1`-`9` | Switch to worker N |
|
|
179
|
+
| `←` / `h` | Previous worker |
|
|
180
|
+
| `→` / `l` | Next worker |
|
|
181
|
+
| `↑` / `k` | Scroll up |
|
|
182
|
+
| `↓` / `j` | Scroll down |
|
|
183
|
+
| `Page Up` | Scroll up one page |
|
|
184
|
+
| `Page Down` | Scroll down one page |
|
|
185
|
+
| `Home` / `g` | Scroll to top |
|
|
186
|
+
| `End` / `G` | Scroll to bottom |
|
|
187
|
+
| `f` | Toggle follow mode |
|
|
188
|
+
| `a` | Toggle auto-rotate mode |
|
|
189
|
+
| `q` | Quit (triggers fail-fast) |
|
|
190
|
+
| `Ctrl+C` | Force quit |
|
|
191
|
+
|
|
192
|
+
### Behavior Modes
|
|
193
|
+
|
|
194
|
+
#### Follow Mode (default: on)
|
|
195
|
+
|
|
196
|
+
Automatically switches to whichever worker produces new output. The view stays on the current worker if:
|
|
197
|
+
- User has manually switched workers in the last 3 seconds
|
|
198
|
+
- User is scrolled up (not at bottom of content)
|
|
199
|
+
|
|
200
|
+
Indicator: `[f]ollow on` or `[f]ollow off` in tab bar.
|
|
201
|
+
|
|
202
|
+
#### Auto-Rotate Mode (default: off)
|
|
203
|
+
|
|
204
|
+
Cycles through workers with activity every 2 seconds. Useful for passively monitoring all workers.
|
|
205
|
+
|
|
206
|
+
Indicator: `[a]uto-rotate` in tab bar when active.
|
|
207
|
+
|
|
208
|
+
#### Manual Mode
|
|
209
|
+
|
|
210
|
+
When both follow and auto-rotate are off, the view stays on the selected worker until explicitly changed.
|
|
211
|
+
|
|
212
|
+
### Buffer Management
|
|
213
|
+
|
|
214
|
+
Each worker maintains a circular buffer of the last 500 lines. Older content is discarded but remains in log files (if `--log-dir` specified).
|
|
215
|
+
|
|
216
|
+
### Terminal Size Handling
|
|
217
|
+
|
|
218
|
+
Minimum supported size: 80 columns × 24 rows.
|
|
219
|
+
|
|
220
|
+
On resize:
|
|
221
|
+
- Re-render entire UI
|
|
222
|
+
- Truncate/wrap content as needed
|
|
223
|
+
- If terminal becomes too small, show warning and degrade to interleaved mode
|
|
224
|
+
|
|
225
|
+
### Color Scheme
|
|
226
|
+
|
|
227
|
+
```ruby
|
|
228
|
+
COLORS = {
|
|
229
|
+
red: "\e[31m", # failures, errors
|
|
230
|
+
green: "\e[32m", # passes, success
|
|
231
|
+
yellow: "\e[33m", # pending, warnings
|
|
232
|
+
blue: "\e[34m", # group headers
|
|
233
|
+
magenta: "\e[35m", # tool calls
|
|
234
|
+
cyan: "\e[36m", # active spinner, progress
|
|
235
|
+
white: "\e[37m", # normal text
|
|
236
|
+
dim: "\e[2m", # labels, secondary text
|
|
237
|
+
bold: "\e[1m", # selected tab, emphasis
|
|
238
|
+
inverse: "\e[7m", # alternative highlight
|
|
239
|
+
reset: "\e[0m"
|
|
240
|
+
}
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Box drawing characters: `╭ ╮ ╰ ╯ │ ─ ┌ ┐ └ ┘`
|
|
244
|
+
|
|
245
|
+
---
|
|
246
|
+
|
|
247
|
+
## Mode 2: Interleaved (Current Behavior)
|
|
248
|
+
|
|
249
|
+
Simple streaming output with worker prefixes. Default for CI environments.
|
|
250
|
+
|
|
251
|
+
### Output Format
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
⚡ Parallel spec runner (4 workers)
|
|
255
|
+
|
|
256
|
+
[1] ○ creates user account...
|
|
257
|
+
[2] ○ validates email format...
|
|
258
|
+
[1] User: Create an account for john@example.com
|
|
259
|
+
[1] Agent: I'll create that account...
|
|
260
|
+
[2] Agent: Checking the email format...
|
|
261
|
+
[1] → create_user(email: "john@example.com")
|
|
262
|
+
[2] → validate_email(email: "test")
|
|
263
|
+
[2] ✗ FAILED
|
|
264
|
+
[2] Expected valid email format
|
|
265
|
+
[1] ✓ (1.2s)
|
|
266
|
+
[3] ○ handles authentication error...
|
|
267
|
+
[3] User: Try to login with bad credentials
|
|
268
|
+
...
|
|
269
|
+
|
|
270
|
+
25 examples, 2 failures
|
|
271
|
+
|
|
272
|
+
Failures:
|
|
273
|
+
|
|
274
|
+
1) validates email format
|
|
275
|
+
Expected valid email format
|
|
276
|
+
# ./spec/email_spec.rb:42
|
|
277
|
+
|
|
278
|
+
Failed examples:
|
|
279
|
+
|
|
280
|
+
rspec ./spec/email_spec.rb:42 # validates email format
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
### Worker Prefix
|
|
284
|
+
|
|
285
|
+
Each line is prefixed with `[N]` where N is the worker number (1-indexed).
|
|
286
|
+
|
|
287
|
+
Colors per worker (cycles if more than 6 workers):
|
|
288
|
+
|
|
289
|
+
```ruby
|
|
290
|
+
WORKER_COLORS = [:cyan, :yellow, :magenta, :blue, :green, :white]
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
### Indentation
|
|
294
|
+
|
|
295
|
+
```
|
|
296
|
+
[1] ○ example description... # example start
|
|
297
|
+
[1] User: message # conversation (4 spaces)
|
|
298
|
+
[1] Agent: response # conversation (4 spaces)
|
|
299
|
+
[1] → tool_call() # tool call (4 spaces)
|
|
300
|
+
[1] ✓ (1.2s) # example end
|
|
301
|
+
```
|
|
302
|
+
|
|
303
|
+
### Thread Safety
|
|
304
|
+
|
|
305
|
+
All output is mutex-protected to prevent corrupted lines:
|
|
306
|
+
|
|
307
|
+
```ruby
|
|
308
|
+
@mutex.synchronize do
|
|
309
|
+
@output.puts "[#{worker}] #{message}"
|
|
310
|
+
end
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
---
|
|
314
|
+
|
|
315
|
+
## Mode 3: Quiet
|
|
316
|
+
|
|
317
|
+
Minimal terminal output with comprehensive log files. For large test suites where real-time output is impractical.
|
|
318
|
+
|
|
319
|
+
### Terminal Output
|
|
320
|
+
|
|
321
|
+
```
|
|
322
|
+
⚡ Parallel spec runner (4 workers)
|
|
323
|
+
|
|
324
|
+
.....F...F..........F....
|
|
325
|
+
|
|
326
|
+
25 examples, 3 failures (logs: tmp/parallel-run-20240115-143022/)
|
|
327
|
+
|
|
328
|
+
Failures:
|
|
329
|
+
|
|
330
|
+
1) validates email format
|
|
331
|
+
Expected valid email format
|
|
332
|
+
# ./spec/email_spec.rb:42
|
|
333
|
+
|
|
334
|
+
2) handles timeout
|
|
335
|
+
Connection timed out
|
|
336
|
+
# ./spec/network_spec.rb:87
|
|
337
|
+
|
|
338
|
+
3) processes large file
|
|
339
|
+
Memory limit exceeded
|
|
340
|
+
# ./spec/file_spec.rb:123
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
Progress characters:
|
|
344
|
+
|
|
345
|
+
| Character | Meaning |
|
|
346
|
+
|-----------|---------|
|
|
347
|
+
| `.` | Pass |
|
|
348
|
+
| `F` | Failure |
|
|
349
|
+
| `*` | Pending |
|
|
350
|
+
| `E` | Error |
|
|
351
|
+
|
|
352
|
+
### Log Files
|
|
353
|
+
|
|
354
|
+
When `--log-dir` is specified (or defaults to `tmp/parallel-run-{timestamp}/`):
|
|
355
|
+
|
|
356
|
+
```
|
|
357
|
+
tmp/parallel-run-20240115-143022/
|
|
358
|
+
├── summary.log # Overall run summary
|
|
359
|
+
├── worker-1.log # Full output for worker 1
|
|
360
|
+
├── worker-2.log # Full output for worker 2
|
|
361
|
+
├── worker-3.log # Full output for worker 3
|
|
362
|
+
├── worker-4.log # Full output for worker 4
|
|
363
|
+
└── failures.log # All failures with full context
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
#### summary.log
|
|
367
|
+
|
|
368
|
+
```
|
|
369
|
+
Parallel Spec Run - 2024-01-15 14:30:22
|
|
370
|
+
Workers: 4
|
|
371
|
+
Total examples: 25
|
|
372
|
+
Passed: 22
|
|
373
|
+
Failed: 3
|
|
374
|
+
Pending: 0
|
|
375
|
+
Duration: 45.2s
|
|
376
|
+
|
|
377
|
+
Failed examples:
|
|
378
|
+
./spec/email_spec.rb:42
|
|
379
|
+
./spec/network_spec.rb:87
|
|
380
|
+
./spec/file_spec.rb:123
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
#### worker-N.log
|
|
384
|
+
|
|
385
|
+
Full conversation history for each worker, same format as interleaved mode without the `[N]` prefix.
|
|
386
|
+
|
|
387
|
+
#### failures.log
|
|
388
|
+
|
|
389
|
+
Complete context for each failure:
|
|
390
|
+
|
|
391
|
+
```
|
|
392
|
+
================================================================================
|
|
393
|
+
FAILURE 1: validates email format
|
|
394
|
+
================================================================================
|
|
395
|
+
Location: ./spec/email_spec.rb:42
|
|
396
|
+
Worker: 2
|
|
397
|
+
Duration: 0.8s
|
|
398
|
+
|
|
399
|
+
Conversation:
|
|
400
|
+
User: Validate the email "not-an-email"
|
|
401
|
+
Agent: I'll check if this is a valid email format...
|
|
402
|
+
→ validate_email(email: "not-an-email")
|
|
403
|
+
Agent: The validation returned false as expected.
|
|
404
|
+
|
|
405
|
+
Error:
|
|
406
|
+
Expected valid email format
|
|
407
|
+
|
|
408
|
+
Diff:
|
|
409
|
+
expected: true
|
|
410
|
+
got: false
|
|
411
|
+
|
|
412
|
+
Backtrace:
|
|
413
|
+
./spec/email_spec.rb:42:in `block (2 levels) in <top>'
|
|
414
|
+
./lib/rspec/agents/runner.rb:156:in `execute'
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
---
|
|
418
|
+
|
|
419
|
+
## CLI Options
|
|
420
|
+
|
|
421
|
+
```
|
|
422
|
+
--ui=MODE Output mode: interactive, interleaved, quiet (default: auto)
|
|
423
|
+
--[no-]color Force color on/off (default: auto-detect)
|
|
424
|
+
--log-dir=PATH Directory for log files (quiet mode, or --save-logs)
|
|
425
|
+
--save-logs Save log files even in interactive/interleaved modes
|
|
426
|
+
```
|
|
427
|
+
|
|
428
|
+
### Examples
|
|
429
|
+
|
|
430
|
+
```bash
|
|
431
|
+
# Auto-detect (interactive if TTY, interleaved if CI)
|
|
432
|
+
bin/rspec-agents spec/
|
|
433
|
+
|
|
434
|
+
# Force interactive mode
|
|
435
|
+
bin/rspec-agents --ui=interactive spec/
|
|
436
|
+
|
|
437
|
+
# CI-friendly output
|
|
438
|
+
bin/rspec-agents --ui=interleaved spec/
|
|
439
|
+
|
|
440
|
+
# Quiet mode with logs
|
|
441
|
+
bin/rspec-agents --ui=quiet --log-dir=tmp/test-logs spec/
|
|
442
|
+
|
|
443
|
+
# Interactive with log backup
|
|
444
|
+
bin/rspec-agents --ui=interactive --save-logs spec/
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
---
|
|
448
|
+
|
|
449
|
+
## Implementation Architecture
|
|
450
|
+
|
|
451
|
+
```
|
|
452
|
+
┌─────────────────────────────────────────────────────────────────────────────┐
|
|
453
|
+
│ ParallelTerminalRunner │
|
|
454
|
+
│ │
|
|
455
|
+
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
|
456
|
+
│ │ UIFactory │───▶│ OutputAdapter │◀───│ EventBus │ │
|
|
457
|
+
│ │ │ │ (interface) │ │ │ │
|
|
458
|
+
│ │ select_mode() │ └────────┬────────┘ │ publish(event) │ │
|
|
459
|
+
│ └─────────────────┘ │ └─────────────────┘ │
|
|
460
|
+
│ │ │
|
|
461
|
+
│ ┌──────────────────────┼──────────────────────┐ │
|
|
462
|
+
│ │ │ │ │
|
|
463
|
+
│ ▼ ▼ ▼ │
|
|
464
|
+
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
|
465
|
+
│ │ InteractiveUI │ │ InterleavedUI │ │ QuietUI │ │
|
|
466
|
+
│ │ │ │ │ │ │ │
|
|
467
|
+
│ │ - ProgressBar │ │ - PrefixedOut │ │ - DotProgress │ │
|
|
468
|
+
│ │ - TabBar │ │ - Mutex │ │ - LogWriter │ │
|
|
469
|
+
│ │ - ContentPane │ │ │ │ │ │
|
|
470
|
+
│ │ - KeyboardInput │ │ │ │ │ │
|
|
471
|
+
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
|
472
|
+
│ │
|
|
473
|
+
└─────────────────────────────────────────────────────────────────────────────┘
|
|
474
|
+
```
|
|
475
|
+
|
|
476
|
+
### Key Classes
|
|
477
|
+
|
|
478
|
+
#### OutputAdapter (Interface)
|
|
479
|
+
|
|
480
|
+
```ruby
|
|
481
|
+
module OutputAdapter
|
|
482
|
+
def on_run_started(worker_count:, example_count:); end
|
|
483
|
+
def on_run_finished(results:); end
|
|
484
|
+
def on_example_started(worker:, event:); end
|
|
485
|
+
def on_example_finished(worker:, event:); end
|
|
486
|
+
def on_conversation_event(worker:, event:); end
|
|
487
|
+
end
|
|
488
|
+
```
|
|
489
|
+
|
|
490
|
+
#### InteractiveUI
|
|
491
|
+
|
|
492
|
+
Manages full-screen TUI with:
|
|
493
|
+
- `Screen` - ANSI rendering, cursor control
|
|
494
|
+
- `ProgressBar` - progress header component
|
|
495
|
+
- `TabBar` - worker tab component
|
|
496
|
+
- `ContentPane` - scrollable conversation view
|
|
497
|
+
- `KeyboardReader` - async input handling (using `IO.select` or `tty-reader`)
|
|
498
|
+
|
|
499
|
+
#### InterleavedUI
|
|
500
|
+
|
|
501
|
+
Simple mutex-protected line output with worker prefixes.
|
|
502
|
+
|
|
503
|
+
#### QuietUI
|
|
504
|
+
|
|
505
|
+
Dot-progress output with `LogWriter` for file output.
|
|
506
|
+
|
|
507
|
+
### Dependencies
|
|
508
|
+
|
|
509
|
+
Required gems (already available or minimal additions):
|
|
510
|
+
|
|
511
|
+
```ruby
|
|
512
|
+
# For interactive mode
|
|
513
|
+
gem "io-console" # Part of Ruby stdlib, for raw keyboard input
|
|
514
|
+
|
|
515
|
+
# Optional, for richer TUI
|
|
516
|
+
gem "tty-cursor" # ANSI cursor control
|
|
517
|
+
gem "tty-screen" # Terminal size detection
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
---
|
|
521
|
+
|
|
522
|
+
## Accessibility Considerations
|
|
523
|
+
|
|
524
|
+
- All status information is conveyed through text, not just color
|
|
525
|
+
- Symbols (✓ ✗ ○) provide status even without color
|
|
526
|
+
- Keyboard-only navigation (no mouse required)
|
|
527
|
+
- Screen reader compatible in interleaved/quiet modes
|
|
528
|
+
|
|
529
|
+
---
|
|
530
|
+
|
|
531
|
+
## Future Enhancements
|
|
532
|
+
|
|
533
|
+
1. **Split view** - Show 2-4 workers simultaneously in panels
|
|
534
|
+
2. **Search** - `/` to search within current worker's output
|
|
535
|
+
3. **Filter** - Show only failures, only specific workers
|
|
536
|
+
4. **Replay** - Load logs and replay in interactive mode
|
|
537
|
+
5. **Web UI** - Browser-based viewer for remote/CI runs
|