ruby_raider 3.9.0 → 3.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/release.yml +18 -29
- data/.github/workflows/security.yml +68 -0
- data/.rubocop.yml +28 -0
- data/README.md +23 -4
- data/docs/superpowers/plans/2026-10-25-failure-video-replay.md +400 -0
- data/docs/superpowers/plans/2026-10-26-parity-cleanup.md +233 -0
- data/docs/superpowers/plans/2026-10-28-failure-card.md +295 -0
- data/docs/superpowers/specs/2026-10-25-failure-video-replay-design.md +214 -0
- data/docs/superpowers/specs/2026-10-28-failure-card-design.md +210 -0
- data/lib/commands/scaffolding_commands.rb +3 -3
- data/lib/commands/utility_commands.rb +36 -6
- data/lib/generators/cucumber/templates/cucumber.tt +1 -1
- data/lib/generators/cucumber/templates/feature.tt +5 -0
- data/lib/generators/cucumber/templates/partials/appium_env.tt +6 -0
- data/lib/generators/cucumber/templates/partials/selenium_env.tt +22 -0
- data/lib/generators/cucumber/templates/partials/watir_env.tt +22 -0
- data/lib/generators/cucumber/templates/partials/web_steps.tt +4 -0
- data/lib/generators/helper_generator.rb +12 -1
- data/lib/generators/infrastructure/azure_generator.rb +1 -0
- data/lib/generators/infrastructure/github_generator.rb +1 -0
- data/lib/generators/infrastructure/templates/azure_pipelines.tt +12 -1
- data/lib/generators/infrastructure/templates/github.tt +12 -0
- data/lib/generators/infrastructure/templates/github_appium.tt +23 -0
- data/lib/generators/invoke_generators.rb +20 -2
- data/lib/generators/rspec/rspec_generator.rb +0 -1
- data/lib/generators/rspec/templates/spec.tt +12 -0
- data/lib/generators/templates/common/gemfile.tt +1 -1
- data/lib/generators/templates/common/git_ignore.tt +7 -1
- data/lib/generators/templates/common/partials/mobile_config.tt +5 -0
- data/lib/generators/templates/common/partials/web_config.tt +9 -1
- data/lib/generators/templates/common/read_me.tt +84 -2
- data/lib/generators/templates/helpers/browser_helper.tt +5 -1
- data/lib/generators/templates/helpers/driver_helper.tt +1 -0
- data/lib/generators/templates/helpers/partials/selenium_driver.tt +4 -1
- data/lib/generators/templates/helpers/session_state.tt +206 -0
- data/lib/generators/templates/helpers/spec_helper.tt +28 -0
- data/lib/generators/templates/helpers/video_helper.tt +338 -171
- data/lib/ruby_raider/ffmpeg.rb +280 -0
- data/lib/ruby_raider.rb +2 -3
- data/lib/scaffolding/project_detector.rb +1 -3
- data/lib/scaffolding/scaffolding.rb +0 -1
- data/lib/utilities/utilities.rb +15 -2
- data/lib/version +1 -1
- data/ruby_raider.gemspec +4 -1
- data/sig/commands/utility_commands.rbs +4 -1
- data/sig/generators/invoke_generators.rbs +2 -3
- data/sig/ruby_raider/ffmpeg.rbs +23 -0
- data/sig/utilities/utilities.rbs +1 -0
- data/spec/commands/raider_commands_spec.rb +56 -1
- data/spec/generators/fixtures/video_helper_stubs/selenium-webdriver.rb +28 -0
- data/spec/generators/video_helper_spec.rb +608 -0
- data/spec/integration/commands/browser_update_after_creation_spec.rb +7 -6
- data/spec/integration/commands/utility_commands_spec.rb +2 -2
- data/spec/integration/content/azure_content_spec.rb +33 -0
- data/spec/integration/content/ci_content_spec.rb +54 -0
- data/spec/integration/content/common_content_spec.rb +35 -0
- data/spec/integration/content/config_content_spec.rb +32 -0
- data/spec/integration/content/cucumber_reporter_content_spec.rb +47 -1
- data/spec/integration/content/gemfile_content_spec.rb +2 -3
- data/spec/integration/content/helper_content_spec.rb +178 -0
- data/spec/integration/content/page_content_spec.rb +0 -1
- data/spec/integration/content/readme_content_spec.rb +79 -0
- data/spec/integration/content/rspec_reporter_content_spec.rb +2 -2
- data/spec/integration/content/syntax_validation_spec.rb +15 -0
- data/spec/integration/content/test_content_spec.rb +27 -3
- data/spec/integration/end_to_end_features_spec.rb +0 -2
- data/spec/integration/end_to_end_spec.rb +275 -26
- data/spec/integration/generators/azure_generator_spec.rb +7 -6
- data/spec/integration/generators/common_generator_spec.rb +1 -1
- data/spec/integration/generators/helpers_generator_spec.rb +22 -0
- data/spec/integration/generators/invoke_generators_spec.rb +52 -0
- data/spec/integration/helpers/session_state_spec.rb +428 -0
- data/spec/integration/scaffolding_e2e_spec.rb +13 -24
- data/spec/scaffolding/scaffold_project_detector_spec.rb +0 -1
- data/spec/scaffolding/scaffolding_features_spec.rb +0 -1
- data/spec/system/selenium_spec.rb +0 -1
- data/spec/system/watir_spec.rb +0 -1
- data/spec/utilities/ffmpeg_spec.rb +424 -0
- data/spec/utilities/utilities_spec.rb +19 -2
- data/spec/utilities/video_command_spec.rb +120 -0
- metadata +65 -11
- data/lib/commands/adopt_commands.rb +0 -42
- data/lib/commands/loaded_commands.rb +0 -9
- data/lib/commands/plugin_commands.rb +0 -32
- data/sig/commands/adopt_commands.rbs +0 -8
- data/sig/commands/loaded_commands.rbs +0 -5
- data/sig/commands/plugin_commands.rbs +0 -9
|
@@ -0,0 +1,400 @@
|
|
|
1
|
+
# Failure Video Replay Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
4
|
+
|
|
5
|
+
**Goal:** Every failing test in a newly generated Raider project gets a WebM (web) or MP4 (Appium) replay that plays inline in the raider_reporter HTML report and in raider_desktop. Nothing has to be installed by hand.
|
|
6
|
+
|
|
7
|
+
**Architecture:** The ruby_raider gem gets a pinned, checksum-verified ffmpeg downloader (`RubyRaider::Ffmpeg`). Generated projects get a revived `VideoHelper`: it captures page screenshots on an interval and encodes them to VP8 WebM only when the test fails. Appium projects use Appium's own recorder. raider_reporter gets a separate video size cap, upload ordering that puts screenshots first, and a `<video>` player. raider_desktop gets a scoped `raider-media://` protocol and a player in the result card.
|
|
8
|
+
|
|
9
|
+
**Tech Stack:** Ruby 3.2+ (ruby_raider, raider_reporter, RSpec), Selenium WebDriver / Watir / appium_lib in generated projects, LGPL ffmpeg with libvpx, Electron + React + Vitest (raider_desktop).
|
|
10
|
+
|
|
11
|
+
**Spec:** `docs/superpowers/specs/2026-10-25-failure-video-replay-design.md` (ruby_raider repo). Read it before starting any task.
|
|
12
|
+
|
|
13
|
+
**Repos and branches:**
|
|
14
|
+
|
|
15
|
+
| Repo | Base | Branch | Release |
|
|
16
|
+
|---|---|---|---|
|
|
17
|
+
| `~/ruby_raider` | `origin/master` (3.9.0) | `feat/failure-video-replay` (exists, spec committed) | 3.10.0 |
|
|
18
|
+
| `~/raider_reporter` | `main` (0.9.0) | `feature/video-attachments` | 0.10.0 |
|
|
19
|
+
| `~/raider_desktop` | `origin/master` (3.6.0) | `feature/video-player` | 3.7.0 |
|
|
20
|
+
|
|
21
|
+
Order: Task 1 (spike) gates everything. Then raider_reporter (Tasks 2–4), ruby_raider (5–9), raider_desktop (10–11). Desktop does not depend on ruby_raider and can run in parallel with Tasks 5–9.
|
|
22
|
+
|
|
23
|
+
## Global Constraints
|
|
24
|
+
|
|
25
|
+
- Video is on by default in new projects: `video.enabled: true` in generated `config/config.yml`. `VIDEO=off` disables it for one run.
|
|
26
|
+
- Keep the replay only when the test fails. Passing tests pay the capture cost but not the encode cost.
|
|
27
|
+
- ffmpeg resolution order is `RAIDER_FFMPEG_PATH`, then the cached pinned binary, then a download. A system `ffmpeg` on PATH is never used unless `RAIDER_FFMPEG_PATH` points to it.
|
|
28
|
+
- Cache path: `~/.cache/raider/ffmpeg/<version>/ffmpeg[.exe]`, or `%LOCALAPPDATA%\raider\ffmpeg\<version>\` on Windows. `RAIDER_CACHE_DIR` overrides the root.
|
|
29
|
+
- Download: HTTPS only. Verify SHA-256 before an atomic rename and `chmod +x`. On a mismatch, delete the file, warn, and record no video. Never query "latest".
|
|
30
|
+
- Platforms: `darwin-arm64`, `darwin-x64`, `linux-x64`, `windows-x64`.
|
|
31
|
+
- Web output is WebM/VP8. Appium output is the MP4 that Appium returns, not re-encoded.
|
|
32
|
+
- Config block (web and mobile): `video: { enabled: true, fps: 5, max_width: 1280 }`.
|
|
33
|
+
- raider_reporter: `max_video_bytes` defaults to 50 MB (`50 * 1024 * 1024`) for `video/*`. `max_attachment_bytes` stays at 5 MB for everything else.
|
|
34
|
+
- Upload budget is unchanged: 45 MB and 1000 files. Non-video attachments go first, then videos fill whatever room is left.
|
|
35
|
+
- Desktop serves media only through `raider-media://` and only from `<abs>/raider-results/attachments(-N)?/<file>`. No general `file://` access.
|
|
36
|
+
- Recording must never raise into a test or change its result or exit code. After the first error: one `[video]` warning, then recording is disabled for the rest of the process.
|
|
37
|
+
- Slowdown limit: under 15% added wall time on the generated sample suite compared with `VIDEO=off`. If it's missed, **stop and ask the user**; do not choose a trade-off.
|
|
38
|
+
- The `ffmpeg-<version>` release is not published yet. Build against a placeholder manifest (`sha256: nil`). With the placeholder, `ensure!` warns `ffmpeg build not published yet` and returns nil.
|
|
39
|
+
|
|
40
|
+
## Review Focus
|
|
41
|
+
|
|
42
|
+
1. **Odd or long test names, and two processes running at once:** names with `/`, quotes or emoji, or two parallel_tests processes running the same test name, must not collide or break paths. Use a per-test `Dir.mktmpdir`; the name only goes into the attachment label. *(Task 7 test.)*
|
|
43
|
+
2. **Driver dies before the after hook:** the session crashed or `quit` already ran, so `stop` must return nil, warn once, and not raise. *(Task 7 test.)*
|
|
44
|
+
3. **First run with parallel_tests:** N processes start the ffmpeg download at the same moment. Exactly one binary ends up in the cache, it's valid, and no process sees a partial file. *(Task 5 test.)*
|
|
45
|
+
4. **Encoding fails:** ffmpeg exits non-zero or produces a 0-byte file. No attachment, one warning, the test result is unchanged. *(Task 7 test.)*
|
|
46
|
+
5. **Video over the cap:** a 60 MB video is rejected with a warning, and the screenshot on the same test is still attached. *(Task 2 test.)*
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
### Task 1: Spike — background capture safety and cost (gate)
|
|
51
|
+
|
|
52
|
+
Throwaway. The goal is a decision recorded in the spec, not code to keep.
|
|
53
|
+
|
|
54
|
+
**Files:**
|
|
55
|
+
- Create (scratch, not committed): a generated project via `bin/raider new spike -p framework:rspec automation:selenium` and a `spike_recorder.rb` inside it
|
|
56
|
+
- Modify: `docs/superpowers/specs/2026-10-25-failure-video-replay-design.md` (append a "Spike result" section)
|
|
57
|
+
|
|
58
|
+
**Interfaces:**
|
|
59
|
+
- Produces: `CAPTURE_MODE`, either `:thread` (background thread calling `save_screenshot` every `1.0/fps` s) or `:listener` (capture after each driver command through a `Selenium::WebDriver::Support::AbstractEventListener`). Task 7 implements the chosen one.
|
|
60
|
+
|
|
61
|
+
- [ ] **Step 1:** In the spike project, run a background thread that calls `driver.save_screenshot` every 200 ms while the sample suite runs, for headless Chrome, headed Chrome and Firefox. Run each 20 times.
|
|
62
|
+
Pass: zero `Net::ReadTimeout`, `Errno::*`, `Selenium::WebDriver::Error::*` or garbled-response errors that are absent in a `VIDEO=off` run.
|
|
63
|
+
- [ ] **Step 2:** Measure wall time for each browser (median of 5 runs) with capture on and off.
|
|
64
|
+
- [ ] **Step 3:** If the thread approach fails Step 1 or goes over 15% in Step 2, repeat both steps with listener capture (frames after `navigate_to`, `click`, `change_value` and `execute_script`, plus one final frame).
|
|
65
|
+
- [ ] **Step 4:** Append to the spec: chosen `CAPTURE_MODE`, error counts, and median slowdown per browser. **If neither approach is under 15%, stop and ask the user which gives way (the limit, the fps, or default-on for that browser). Do not continue.**
|
|
66
|
+
- [ ] **Step 5: Commit**
|
|
67
|
+
```bash
|
|
68
|
+
git add docs/superpowers/specs/2026-10-25-failure-video-replay-design.md
|
|
69
|
+
git commit -m "Record video capture spike result"
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
### Task 2: raider_reporter — separate size cap for video
|
|
75
|
+
|
|
76
|
+
**Files:**
|
|
77
|
+
- Modify: `lib/raider_reporter/configuration.rb:9-20`
|
|
78
|
+
- Modify: `lib/raider_reporter.rb:48-56` (`start_collector!`) and `:63-84` (`attach`)
|
|
79
|
+
- Modify: `lib/raider_reporter/collector.rb:11-13,115-125` (`keep?`)
|
|
80
|
+
- Test: `spec/raider_reporter/attach_spec.rb`, `spec/raider_reporter/configuration_spec.rb`
|
|
81
|
+
|
|
82
|
+
**Interfaces:**
|
|
83
|
+
- Produces: `Configuration#max_video_bytes` (Integer, default `50 * 1024 * 1024`); `Collector.new(max_attachment_bytes:, max_video_bytes:, ...)`; `RaiderReporter.attachment_limit(content_type) -> Integer` (private helper, `video/*` → video cap, else attachment cap).
|
|
84
|
+
|
|
85
|
+
- [ ] **Step 1: Write the failing tests**
|
|
86
|
+
```ruby
|
|
87
|
+
# configuration_spec.rb
|
|
88
|
+
it "defaults max_video_bytes to 50 MB" do
|
|
89
|
+
expect(described_class.new({}).max_video_bytes).to eq(50 * 1024 * 1024)
|
|
90
|
+
end
|
|
91
|
+
|
|
92
|
+
# attach_spec.rb
|
|
93
|
+
it "accepts a video larger than max_attachment_bytes but within max_video_bytes" do
|
|
94
|
+
RaiderReporter.configuration.max_attachment_bytes = 3
|
|
95
|
+
RaiderReporter.configuration.max_video_bytes = 10
|
|
96
|
+
# start collector/run/test as in the existing specs; write "fivebytes" to clip.webm
|
|
97
|
+
expect(RaiderReporter.attach("video", path, type: :webm)).to be(true)
|
|
98
|
+
expect(finish(collector).attachments.map(&:content_type)).to eq(["video/webm"])
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
it "rejects a video over max_video_bytes and keeps the screenshot on the same test" do
|
|
102
|
+
RaiderReporter.configuration.max_video_bytes = 3
|
|
103
|
+
# attach shot.png (2 bytes, type: :png) then clip.webm (4 bytes, type: :webm)
|
|
104
|
+
expect(RaiderReporter.attach("video", video_path, type: :webm)).to be(false)
|
|
105
|
+
expect(err.string).to include("clip.webm").and include("larger than 3 bytes")
|
|
106
|
+
expect(finish(collector).attachments.map(&:name)).to eq(["screenshot"])
|
|
107
|
+
end
|
|
108
|
+
```
|
|
109
|
+
- [ ] **Step 2:** Run `bundle exec rspec spec/raider_reporter/attach_spec.rb spec/raider_reporter/configuration_spec.rb`. Expected: FAIL (`undefined method max_video_bytes`).
|
|
110
|
+
- [ ] **Step 3:** Add `max_video_bytes`. Resolve the content type once in `attach` (before the size check) and compare against the limit for that type. Pass `max_video_bytes:` into `Collector`, and make `keep?` choose its limit from `attachment.content_type`.
|
|
111
|
+
- [ ] **Step 4:** Run the same command, then `bundle exec rspec`. Expected: all PASS.
|
|
112
|
+
- [ ] **Step 5: Commit** `git commit -am "Give video attachments their own 50 MB size cap"`
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
### Task 3: raider_reporter — screenshots first in the upload budget
|
|
117
|
+
|
|
118
|
+
**Files:**
|
|
119
|
+
- Modify: `lib/raider_reporter/writers/uploader.rb:54-73` (`within_budget`)
|
|
120
|
+
- Test: `spec/raider_reporter/writers/uploader_spec.rb` (next to "drops attachments beyond the byte budget…", line 48)
|
|
121
|
+
|
|
122
|
+
**Interfaces:**
|
|
123
|
+
- Consumes: `Payload#attachment_entries` (each `{attachment:, ...}`), `MAX_UPLOAD_ATTACHMENT_BYTES`, `MAX_UPLOAD_ATTACHMENTS`.
|
|
124
|
+
- Produces: same method signature. The selection becomes "all non-video entries in order, then video entries in order, until the byte or count limit". Survivors keep their original per-result order.
|
|
125
|
+
|
|
126
|
+
- [ ] **Step 1: Write the failing test**
|
|
127
|
+
```ruby
|
|
128
|
+
it "keeps screenshots ahead of an earlier video when the byte budget is exceeded" do
|
|
129
|
+
# result A: clip.webm (40 MB); result B: shot.png (10 MB) — fixture files via File.truncate
|
|
130
|
+
# stub MAX_UPLOAD_ATTACHMENT_BYTES at 45 MB (default)
|
|
131
|
+
uploaded = parts_of(upload(payload)) # use the spec's existing multipart helpers
|
|
132
|
+
expect(uploaded).to include("shot.png")
|
|
133
|
+
expect(uploaded).not_to include("clip.webm")
|
|
134
|
+
expect(warnings.join).to include("not uploading 1 attachment")
|
|
135
|
+
end
|
|
136
|
+
```
|
|
137
|
+
- [ ] **Step 2:** Run `bundle exec rspec spec/raider_reporter/writers/uploader_spec.rb`. Expected: the new test FAILS (the video is kept and the screenshot dropped).
|
|
138
|
+
- [ ] **Step 3:** Rewrite `within_budget` to build a set of entries to keep using that priority, then filter each result's attachments by membership in that set (replacing the `seen` counter). Keep the warning text exactly the same.
|
|
139
|
+
- [ ] **Step 4:** Run the uploader spec, then the full suite. Expected: PASS, including the existing budget and count-limit specs unchanged.
|
|
140
|
+
- [ ] **Step 5: Commit** `git commit -am "Upload screenshots before videos when over the attachment budget"`
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
### Task 4: raider_reporter — inline video player, release 0.10.0
|
|
145
|
+
|
|
146
|
+
**Files:**
|
|
147
|
+
- Modify: `lib/raider_reporter/writers/html.rb:220-234` (`attachments`) and the CSS block near `:265`
|
|
148
|
+
- Modify: `lib/raider_reporter/version.rb:4` → `"0.10.0"`, `CHANGELOG.md`, `README.md` ("Attaching screenshots and videos": document `max_video_bytes` and the upload ordering)
|
|
149
|
+
- Test: `spec/raider_reporter/writers/html_spec.rb` (next to "inlines images…", line 119)
|
|
150
|
+
|
|
151
|
+
**Interfaces:**
|
|
152
|
+
- Consumes: `safe_href(path)`.
|
|
153
|
+
- Produces: for `video/*`, `<video controls preload="metadata" src="HREF"></video><a href="HREF" download>NAME</a>`.
|
|
154
|
+
|
|
155
|
+
- [ ] **Step 1: Write the failing test**
|
|
156
|
+
```ruby
|
|
157
|
+
it "renders videos as an inline player with a download link and drops unsafe video paths" do
|
|
158
|
+
html = render(attachments: [
|
|
159
|
+
{ "name" => "video", "content_type" => "video/webm", "path" => "attachments/a b/clip.webm" },
|
|
160
|
+
{ "name" => "evil", "content_type" => "video/mp4", "path" => "../x.mp4" }
|
|
161
|
+
])
|
|
162
|
+
expect(html).to include(%(<video controls preload="metadata" src="attachments/a%20b/clip.webm"></video>))
|
|
163
|
+
expect(html).to include(%(<a href="attachments/a%20b/clip.webm" download>video</a>))
|
|
164
|
+
expect(html).not_to include("x.mp4")
|
|
165
|
+
end
|
|
166
|
+
```
|
|
167
|
+
- [ ] **Step 2:** Run `bundle exec rspec spec/raider_reporter/writers/html_spec.rb`. Expected: FAIL.
|
|
168
|
+
- [ ] **Step 3:** Add a `video/` branch in `attachments`. Add CSS: `.a video { max-width: 100%; max-height: 360px; display: block }`.
|
|
169
|
+
- [ ] **Step 4:** Run the full suite and `bundle exec rubocop`. Expected: PASS, no offenses. Then manually: `cd examples/rspec_basic`, attach any small `.webm` in the failing example, run `bundle exec rspec`, and open `raider-results/index.html`. The video plays.
|
|
170
|
+
- [ ] **Step 5: Commit and build**
|
|
171
|
+
```bash
|
|
172
|
+
git commit -am "Play video attachments inline in the HTML report; release 0.10.0"
|
|
173
|
+
gem build raider_reporter.gemspec # publishing to rubygems is the user's call
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
### Task 5: ruby_raider — `RubyRaider::Ffmpeg` provisioning
|
|
179
|
+
|
|
180
|
+
**Files:**
|
|
181
|
+
- Create: `lib/ruby_raider/ffmpeg.rb` (must not require `thor` or anything under `lib/commands`)
|
|
182
|
+
- Create: `sig/ruby_raider/ffmpeg.rbs`
|
|
183
|
+
- Test: `spec/utilities/ffmpeg_spec.rb`
|
|
184
|
+
|
|
185
|
+
**Interfaces:**
|
|
186
|
+
- Produces (used by Tasks 6 and 7):
|
|
187
|
+
- `RubyRaider::Ffmpeg::MANIFEST`: `{ version: String, assets: { 'darwin-arm64' => { url: String, sha256: String | nil }, ... } }` (placeholder `sha256: nil` for every platform until the release exists)
|
|
188
|
+
- `RubyRaider::Ffmpeg.platform(host_os: RbConfig::CONFIG['host_os'], host_cpu: RbConfig::CONFIG['host_cpu']) -> String | nil`
|
|
189
|
+
- `RubyRaider::Ffmpeg.cache_dir(env: ENV) -> String`
|
|
190
|
+
- `RubyRaider::Ffmpeg.resolve(env: ENV, manifest: MANIFEST) -> String | nil` (no network: the env override if executable, else the cached binary)
|
|
191
|
+
- `RubyRaider::Ffmpeg.ensure!(env: ENV, manifest: MANIFEST, fetch: method(:http_get), warn: ->(m) { $stderr.puts("[video] #{m}") }) -> String | nil`
|
|
192
|
+
- `fetch` contract: `fetch.call(url, destination_path)` writes bytes and raises on failure.
|
|
193
|
+
|
|
194
|
+
- [ ] **Step 1: Write the failing tests** (use `Dir.mktmpdir` as `RAIDER_CACHE_DIR`, a test manifest whose `sha256` is the digest of `"#!/bin/sh\n"`, and a fake `fetch` that writes those bytes)
|
|
195
|
+
```ruby
|
|
196
|
+
describe RubyRaider::Ffmpeg do
|
|
197
|
+
it 'maps host os/cpu to the four platforms and nil otherwise' do
|
|
198
|
+
expect(described_class.platform(host_os: 'darwin23', host_cpu: 'arm64')).to eq('darwin-arm64')
|
|
199
|
+
expect(described_class.platform(host_os: 'darwin23', host_cpu: 'x86_64')).to eq('darwin-x64')
|
|
200
|
+
expect(described_class.platform(host_os: 'linux-gnu', host_cpu: 'x86_64')).to eq('linux-x64')
|
|
201
|
+
expect(described_class.platform(host_os: 'mingw32', host_cpu: 'x64')).to eq('windows-x64')
|
|
202
|
+
expect(described_class.platform(host_os: 'linux-gnu', host_cpu: 'aarch64')).to be_nil
|
|
203
|
+
end
|
|
204
|
+
it 'prefers RAIDER_FFMPEG_PATH and never downloads then'
|
|
205
|
+
it 'ignores ffmpeg on PATH when no override is set' # stub PATH to a dir containing ffmpeg; resolve → nil
|
|
206
|
+
it 'downloads once, verifies, caches at <cache>/ffmpeg/<version>/ffmpeg, chmods +x'
|
|
207
|
+
it 'makes no fetch call on a second ensure!'
|
|
208
|
+
it 'deletes the file and returns nil with one warning on a checksum mismatch'
|
|
209
|
+
it 'returns nil with "ffmpeg build not published yet" when the manifest sha256 is nil'
|
|
210
|
+
it 'returns nil with one warning when fetch raises'
|
|
211
|
+
it 'leaves exactly one valid binary when 4 threads call ensure! concurrently' # Review Focus 3
|
|
212
|
+
end
|
|
213
|
+
```
|
|
214
|
+
- [ ] **Step 2:** Run `bundle exec rspec spec/utilities/ffmpeg_spec.rb`. Expected: FAIL (`uninitialized constant RubyRaider::Ffmpeg`).
|
|
215
|
+
- [ ] **Step 3:** Implement. Download to `<dest>.<pid>.<rand>.part`, verify with `Digest::SHA256.file`, then `File.rename` into place. Rename-wins handles concurrency: when the destination already exists and verifies, discard the `.part` file. `http_get` uses `Net::HTTP` with `use_ssl` required, follows up to 5 redirects to `https` only, and uses `open_timeout 10`, `read_timeout 60`.
|
|
216
|
+
- [ ] **Step 4:** Run the spec, then `bundle exec rspec spec/utilities`, `bundle exec rubocop lib/ruby_raider spec/utilities` and `bundle exec steep check` (the repo has a Steepfile). Expected: PASS.
|
|
217
|
+
- [ ] **Step 5: Commit** `git commit -m "Add pinned, checksum-verified ffmpeg provisioning"`
|
|
218
|
+
|
|
219
|
+
---
|
|
220
|
+
|
|
221
|
+
### Task 6: ruby_raider — `raider u video install|status|on|off`
|
|
222
|
+
|
|
223
|
+
**Files:**
|
|
224
|
+
- Modify: `lib/commands/utility_commands.rb` (add after `desktop`, ~line 100; also fix the stray `desc 'config'` at line 71 by deleting it, since its options leak onto `timeout`)
|
|
225
|
+
- Modify: `lib/utilities/utilities.rb` (add `video_enabled=(Boolean)`, written to `config/config.yml` under `video.enabled`, following the existing setters)
|
|
226
|
+
- Test: `spec/commands/raider_commands_spec.rb` or a new `spec/utilities/video_command_spec.rb`, following `headless_config_spec.rb`
|
|
227
|
+
|
|
228
|
+
**Interfaces:**
|
|
229
|
+
- Consumes: `RubyRaider::Ffmpeg.ensure!`, `.resolve`, `MANIFEST[:version]`.
|
|
230
|
+
- Produces: CLI `raider u video SUBCOMMAND`.
|
|
231
|
+
- `install`: exit 0 and print the path; exit 1 when `ensure!` returns nil.
|
|
232
|
+
- `status`: print `ffmpeg <version>: <path>`, or `ffmpeg <version>: not installed`.
|
|
233
|
+
- `on` / `off`: set `video.enabled`.
|
|
234
|
+
|
|
235
|
+
- [ ] **Step 1: Write the failing tests:** `install` exits 1 with the placeholder manifest; `off` then `on` round-trips `video.enabled` in a temp `config/config.yml`; `status` prints `not installed` with an empty cache; `timeout` no longer lists `config`'s options in `raider u help timeout`.
|
|
236
|
+
- [ ] **Step 2:** Run them. Expected: FAIL.
|
|
237
|
+
- [ ] **Step 3:** Implement. `require 'ruby_raider/ffmpeg'` lazily inside the method, like `desktop` does.
|
|
238
|
+
- [ ] **Step 4:** Run the specs, then `bundle exec rspec spec/commands spec/utilities`. Expected: PASS.
|
|
239
|
+
- [ ] **Step 5: Commit** `git commit -am "Add raider u video install/status/on/off"`
|
|
240
|
+
|
|
241
|
+
---
|
|
242
|
+
|
|
243
|
+
### Task 7: ruby_raider — revive `VideoHelper` in generated projects
|
|
244
|
+
|
|
245
|
+
**Files:**
|
|
246
|
+
- Rewrite: `lib/generators/templates/helpers/video_helper.tt`. Drop `CdpRecorder` and `ScreenCaptureRecorder`, add `FrameRecorder`, keep `AppiumRecorder` and `NullRecorder`.
|
|
247
|
+
- Modify: `lib/generators/helper_generator.rb` (add `generate_video_helper` → `helpers/video_helper.rb`, for every project)
|
|
248
|
+
- Modify: `lib/generators/templates/common/partials/web_config.tt`, `mobile_config.tt` (append the `video:` block from Global Constraints)
|
|
249
|
+
- Modify: `lib/generators/templates/common/gemfile.tt:16` → `gem 'raider_reporter', '~> 0.10'`
|
|
250
|
+
- Test: `spec/integration/content/helper_content_spec.rb` (content) and `spec/generators/video_helper_spec.rb` (behaviour: render the template for selenium and appium into a tmp dir, `load` it, and drive it with test doubles)
|
|
251
|
+
|
|
252
|
+
**Interfaces:**
|
|
253
|
+
- Consumes: `RubyRaider::Ffmpeg.ensure!` (via `require 'ruby_raider/ffmpeg'`), `CAPTURE_MODE` from Task 1.
|
|
254
|
+
- Produces (used by Task 8):
|
|
255
|
+
- `VideoHelper.enabled? -> Boolean`: `config['video']['enabled'] != false && ENV['VIDEO'] != 'off' && !@disabled`
|
|
256
|
+
- `VideoHelper.recorder_for(driver_or_browser) -> recorder`: Appium driver → `AppiumRecorder`, Selenium or Watir → `FrameRecorder`, disabled or error → `NullRecorder`
|
|
257
|
+
- `recorder.start -> nil` (no name argument; uses its own `Dir.mktmpdir`)
|
|
258
|
+
- `recorder.stop(keep:) -> String | nil`. When `keep: false`, it discards frames without encoding and returns nil. When `keep: true`, it returns the path to `video.webm` (web) or `video.mp4` (Appium) inside the recorder's tmpdir, or nil.
|
|
259
|
+
- `recorder.cleanup -> nil` (removes the tmpdir; Task 8 calls it after `attach`)
|
|
260
|
+
- `VideoHelper.disable!(reason) -> nil`: warns `[video] <reason>; recording disabled for this run` once
|
|
261
|
+
- If `CAPTURE_MODE == :listener`: `VideoHelper::FrameListener < Selenium::WebDriver::Support::AbstractEventListener` and `VideoHelper.listener -> FrameListener | nil`, which `driver_helper`/`browser_helper` pass as `listener:` when creating the driver.
|
|
262
|
+
|
|
263
|
+
- [ ] **Step 1: Write the failing tests**
|
|
264
|
+
```ruby
|
|
265
|
+
# content
|
|
266
|
+
it 'generates helpers/video_helper.rb with valid syntax for rspec_selenium, cucumber_watir, rspec_ios'
|
|
267
|
+
it 'config.yml has video: enabled: true, fps: 5, max_width: 1280'
|
|
268
|
+
it 'video_helper has no ScreenCaptureRecorder and no CdpRecorder'
|
|
269
|
+
|
|
270
|
+
# behaviour (fake driver double responds to save_screenshot(path) by writing a 1x1 PNG; fake ffmpeg is a shell script given via RAIDER_FFMPEG_PATH)
|
|
271
|
+
it 'stop(keep: false) never invokes ffmpeg and returns nil'
|
|
272
|
+
it 'stop(keep: true) invokes ffmpeg with libvpx and returns <tmpdir>/video.webm'
|
|
273
|
+
it 'returns nil and warns once when ffmpeg exits non-zero or writes 0 bytes' # Review Focus 4
|
|
274
|
+
it 'returns nil, warns once, and does not raise when save_screenshot raises (driver quit)' # Review Focus 2
|
|
275
|
+
it 'two recorders started at once use different tmpdirs' # Review Focus 1
|
|
276
|
+
it 'VIDEO=off makes recorder_for return a NullRecorder'
|
|
277
|
+
it 'scales frames to max_width via ffmpeg -vf scale=min(1280\,iw):-2'
|
|
278
|
+
```
|
|
279
|
+
- [ ] **Step 2:** Run `bundle exec rspec spec/integration/content/helper_content_spec.rb spec/generators/video_helper_spec.rb`. Expected: FAIL.
|
|
280
|
+
- [ ] **Step 3:** Implement the `CAPTURE_MODE` from Task 1. The capture interval is `1.0 / fps`. Encoder flags are the builder's choice (libvpx, `-deadline realtime`), as long as a 30 s failing test stays under 50 MB. Wrap every recorder method in `rescue StandardError => e; VideoHelper.disable!(e.message); nil`.
|
|
281
|
+
- [ ] **Step 4:** Run the specs, then `bundle exec rspec spec/integration/content` and `bundle exec rubocop`. Expected: PASS.
|
|
282
|
+
- [ ] **Step 5: Commit** `git commit -am "Generate VideoHelper with frame recorder and Appium recorder"`
|
|
283
|
+
|
|
284
|
+
---
|
|
285
|
+
|
|
286
|
+
### Task 8: ruby_raider — wire recording into RSpec and Cucumber hooks
|
|
287
|
+
|
|
288
|
+
**Files:**
|
|
289
|
+
- Modify: `lib/generators/templates/helpers/spec_helper.tt:3-61`
|
|
290
|
+
- Modify: `lib/generators/cucumber/templates/partials/selenium_env.tt`, `watir_env.tt`, `appium_env.tt`
|
|
291
|
+
- Test: `spec/integration/content/helper_content_spec.rb`, `spec/integration/content/cucumber_reporter_content_spec.rb`, `spec/integration/content/rspec_reporter_content_spec.rb`
|
|
292
|
+
|
|
293
|
+
**Interfaces:**
|
|
294
|
+
- Consumes: Task 7's `VideoHelper.recorder_for`, `start`, `stop(keep:)` and `cleanup`.
|
|
295
|
+
- Produces: `VideoHelper.final_attempt?(example) -> Boolean`, which lives in `video_helper.tt`. It's true when the example passed, or when `example.metadata[:retry_attempts].to_i + 1 >= (example.metadata[:retry] || RSpec.configuration.default_retry_count).to_i`. Confirm the metadata keys against the installed rspec-retry source before relying on them.
|
|
296
|
+
|
|
297
|
+
- [ ] **Step 1: Write the failing tests**
|
|
298
|
+
```ruby
|
|
299
|
+
it 'spec_helper requires video_helper, starts a recorder in before(:each) after the viewport resize'
|
|
300
|
+
it 'spec_helper attaches "video" only when example.exception && VideoHelper.final_attempt?(example)'
|
|
301
|
+
it 'spec_helper stops the recorder before quit_driver'
|
|
302
|
+
it 'cucumber envs (selenium, watir, appium) start in Before and attach "video" only when scenario.failed?, before quit'
|
|
303
|
+
it 'final_attempt? is false on attempt 1 of 3 and true on attempt 3 of 3 and when retry count is 0' # in video_helper_spec.rb
|
|
304
|
+
```
|
|
305
|
+
- [ ] **Step 2:** Run the three content specs. Expected: FAIL.
|
|
306
|
+
- [ ] **Step 3:** Implement. The type is `type: :webm` for web and `type: :mp4` for Appium. Call `cleanup` in an `ensure`.
|
|
307
|
+
- [ ] **Step 4:** Run `bundle exec rspec spec/integration/content spec/generators`. Expected: PASS.
|
|
308
|
+
- [ ] **Step 5: Commit** `git commit -am "Attach failure videos from RSpec and Cucumber hooks"`
|
|
309
|
+
|
|
310
|
+
---
|
|
311
|
+
|
|
312
|
+
### Task 9: ruby_raider — CI cache, docs, release 3.10.0, end-to-end acceptance
|
|
313
|
+
|
|
314
|
+
**Files:**
|
|
315
|
+
- Modify: `lib/generators/infrastructure/templates/github.tt` (before `Run tests`: an `actions/cache@v4` step for `~/.cache/raider`, keyed `raider-ffmpeg-<%= RubyRaider::Ffmpeg::MANIFEST[:version] %>-${{ runner.os }}`, then `bundle exec raider u video install`)
|
|
316
|
+
- Modify: `lib/generators/infrastructure/templates/github_appium.tt` (same, both jobs)
|
|
317
|
+
- Modify: `lib/generators/infrastructure/templates/azure_pipelines.tt` (`Cache@2` task with the same key and path, plus the install step)
|
|
318
|
+
- Modify: `lib/generators/templates/common/read_me.tt` (a "Video replay" section covering defaults, `VIDEO=off`, `raider u video ...` and the cache path), `README.md` ("What's Included" bullet plus an upgrade note for existing projects), `lib/version` → `3.10.0`
|
|
319
|
+
- Test: `spec/integration/content/ci_content_spec.rb`, `spec/integration/content/azure_content_spec.rb`, `spec/integration/content/readme_content_spec.rb`, `spec/integration/end_to_end_spec.rb`
|
|
320
|
+
|
|
321
|
+
**Interfaces:**
|
|
322
|
+
- Consumes: everything above.
|
|
323
|
+
|
|
324
|
+
- [ ] **Step 1: Write the failing content tests:** the cache step and key appear in all three CI templates, before the tests step; `raider u video install` appears; the Readme mentions `VIDEO=off`.
|
|
325
|
+
- [ ] **Step 2:** Run them. Expected: FAIL. Then implement and re-run. Expected: PASS.
|
|
326
|
+
- [ ] **Step 3: End-to-end acceptance** (needs a temporary real manifest: point `RAIDER_FFMPEG_PATH` at a locally installed LGPL ffmpeg until the release exists). Add to `end_to_end_spec.rb`, tagged so it is skipped without a browser:
|
|
327
|
+
- `rspec_selenium` with one forced failure (headless Chrome): `raider-results/attachments/**/video.webm` exists for the failing example only, and `index.html` contains `<video`
|
|
328
|
+
- the same for `cucumber_watir` (Firefox)
|
|
329
|
+
- `VIDEO=off`: no video files
|
|
330
|
+
- a tampered cached binary: the suite's exit code equals the `VIDEO=off` exit code, and stderr has exactly one `[video]` line
|
|
331
|
+
- manual (needs a simulator): `rspec_ios` against the sample app with one forced failure. `video.mp4` is attached and plays in the report and in desktop.
|
|
332
|
+
- [ ] **Step 4: Slowdown check:** the median of 5 runs of the generated sample suite, video on vs `VIDEO=off`, must be under 15%. Record the numbers in the PR description. **Over 15%: stop and ask the user.**
|
|
333
|
+
- [ ] **Step 5:** Run `bundle exec rspec` (full suite), `bundle exec rubocop` and `bundle exec reek`. Expected: PASS.
|
|
334
|
+
- [ ] **Step 6: Commit** `git commit -am "Cache ffmpeg in generated CI, document video replay; release 3.10.0"`
|
|
335
|
+
|
|
336
|
+
---
|
|
337
|
+
|
|
338
|
+
### Task 10: raider_desktop — scoped `raider-media://` protocol
|
|
339
|
+
|
|
340
|
+
**Files:**
|
|
341
|
+
- Create: `src/main/handlers/media/mediaProtocol.ts`
|
|
342
|
+
- Modify: `src/main/index.ts` (call `registerMediaScheme()` at module top level, before `app.whenReady` at line 120; call `registerMediaHandler()` inside `whenReady`)
|
|
343
|
+
- Test: `test/integration/mediaProtocol.test.ts`
|
|
344
|
+
|
|
345
|
+
**Interfaces:**
|
|
346
|
+
- Produces (used by Task 11):
|
|
347
|
+
- `toMediaUrl(absPath: string): string` → `raider-media://file/${encodeURIComponent(absPath)}`. Export it from `src/renderer/src/foundation/mediaUrl.ts` (renderer side) and re-implement the decode in main.
|
|
348
|
+
- `resolveMediaPath(requestUrl: string): Promise<string | null>`. Returns the absolute real path only if all of these hold: the host is `file`; the decoded path is absolute; `fs.promises.realpath(p) === path.normalize(p)` (rejects symlinks anywhere in the path); the final segments match `raider-results/attachments(-\d+)?/<name>` with no `..`; the extension is `.webm`, `.mp4` or `.mov`; `lstat` reports a regular file.
|
|
349
|
+
- `parseRange(header: string | null, size: number): { start: number; end: number } | null`
|
|
350
|
+
- `registerMediaScheme(): void` (`protocol.registerSchemesAsPrivileged([{ scheme: 'raider-media', privileges: { stream: true, supportFetchAPI: true } }])`), `registerMediaHandler(): void` (`protocol.handle('raider-media', ...)`). A 206 response is a `fs.createReadStream` slice with `Content-Range` and `Accept-Ranges: bytes`. A 200 response is the whole file. A path that fails the checks gets 404, and an unsatisfiable range gets 416.
|
|
351
|
+
|
|
352
|
+
- [ ] **Step 1: Write the failing tests** (tmp project dir with `raider-results/attachments/x/clip.webm`)
|
|
353
|
+
```ts
|
|
354
|
+
it('resolves a video under raider-results/attachments', ...) // → real path
|
|
355
|
+
it('resolves under attachments-2 (shards)', ...)
|
|
356
|
+
it('rejects ../ escapes, relative paths, other hosts, .html/.png, missing files', ...) // → null
|
|
357
|
+
it('rejects a symlinked attachments dir and a symlinked file', ...)
|
|
358
|
+
it('rejects a file outside raider-results even if named attachments', ...)
|
|
359
|
+
it('parseRange: "bytes=0-" → {0,size-1}; "bytes=10-19" → {10,19}; "bytes=-5" → {size-5,size-1}; beyond size → null; garbage → null', ...)
|
|
360
|
+
```
|
|
361
|
+
- [ ] **Step 2:** Run `npx vitest run test/integration/mediaProtocol.test.ts`. Expected: FAIL.
|
|
362
|
+
- [ ] **Step 3:** Implement as specified above.
|
|
363
|
+
- [ ] **Step 4:** Run the test, then `npx vitest run` and `npm run lint` (or the repo's lint script). Expected: PASS.
|
|
364
|
+
- [ ] **Step 5: Commit** `git commit -m "Serve result videos through a scoped raider-media protocol"`
|
|
365
|
+
|
|
366
|
+
---
|
|
367
|
+
|
|
368
|
+
### Task 11: raider_desktop — video in results and result card, release 3.7.0
|
|
369
|
+
|
|
370
|
+
**Files:**
|
|
371
|
+
- Modify: `src/renderer/src/foundation/Types/testResult.ts` (add `video?: string`)
|
|
372
|
+
- Modify: `src/renderer/src/foundation/raiderResults.ts:68-71` (`toRow`: also pick the first `video/*` attachment through `safeAttachmentPath`, and set `row.video = \`${resultsDir}/${safe}\``)
|
|
373
|
+
- Create: `src/renderer/src/foundation/mediaUrl.ts` (`toMediaUrl`)
|
|
374
|
+
- Modify: `src/renderer/src/components/TestResultCard/index.tsx` (thread `video` through as `screenshot` is at lines 69 and 167; render it above the screenshot button near line 275)
|
|
375
|
+
- Modify: `package.json` version → `3.7.0`
|
|
376
|
+
- Test: `test/raiderResults.test.ts`, `test/components/TestResultCard.test.tsx`
|
|
377
|
+
|
|
378
|
+
**Interfaces:**
|
|
379
|
+
- Consumes: `toMediaUrl` (Task 10).
|
|
380
|
+
- Produces: `<video controls preload="metadata" src={toMediaUrl(video)} data-testid="result-video" />`, inside a collapsed `<details>` labelled with a new i18n key `testResult.video` in `src/renderer/src/assets/translations/en.json` (and every other locale file present).
|
|
381
|
+
|
|
382
|
+
- [ ] **Step 1: Write the failing tests**
|
|
383
|
+
```ts
|
|
384
|
+
it('maps the first video/* attachment to row.video under resultsDir', ...)
|
|
385
|
+
it('ignores a video with an unsafe path', ...)
|
|
386
|
+
it('renders a collapsed video player with a raider-media src when video is set', ...)
|
|
387
|
+
it('renders no video element when video is absent', ...)
|
|
388
|
+
```
|
|
389
|
+
- [ ] **Step 2:** Run `npx vitest run test/raiderResults.test.ts test/components/TestResultCard.test.tsx`. Expected: FAIL.
|
|
390
|
+
- [ ] **Step 3:** Implement.
|
|
391
|
+
- [ ] **Step 4:** Run `npx vitest run`. Then manually: `npm run dev`, open a project whose `raider-results` has a failing test with `video.webm`, and check that the card plays the video and seeking works (this exercises the range path).
|
|
392
|
+
- [ ] **Step 5: Commit** `git commit -am "Play failure videos in the result card; release 3.7.0"`
|
|
393
|
+
|
|
394
|
+
---
|
|
395
|
+
|
|
396
|
+
## Not in this plan (tracked elsewhere)
|
|
397
|
+
|
|
398
|
+
- Publishing the LGPL ffmpeg builds and filling in the real checksums. This is blocked on choosing who cuts the `ffmpeg-<version>` release. Once it's published, change only `MANIFEST` in `lib/ruby_raider/ffmpeg.rb` and re-run Task 9 Step 3 without `RAIDER_FFMPEG_PATH`.
|
|
399
|
+
- The RSpec retry attachment pile-up in raider_reporter (spec §3), as a separate ticket.
|
|
400
|
+
- Publishing the gems and the desktop release.
|