hls 0.1.0 → 0.2.0

data/plans/02-upload-pipeline.md

@@ -0,0 +1,140 @@
+# 02 — Upload pipeline
+
+## Goal
+
+After ffmpeg writes the HLS bundle to a local working directory, push every
+file (master playlist, variant playlists, segments, poster) to the profile's
+configured bucket. Track what got uploaded so re-runs are no-ops and crashed
+runs resume.
+
+```ruby
+CourseVideo.process(input: "lecture.mp4")
+# probe → encode → upload → state.json
+# Re-run → reads state.json, does nothing.
+# Crashed mid-upload → resumes from last successful key.
+```
+
+## Preconditions
+
+- [01 — Profile DSL](01-profile-dsl.md) done. `CourseVideo.bucket` and
+  `CourseVideo.new(input:, output:)#process` exist.
+
+## Work
+
+### 1. `HLS::Uploader` step
+
+In `lib/hls/uploader.rb`:
+
+- `Uploader.new(profile:, output:, bucket:, key_prefix:)`
+- `#perform` walks `output` recursively, uploads each file to
+  `bucket.object(key_prefix + relative_path)`.
+- Sets `Content-Type` correctly per extension (m3u8, ts, jpg).
+- Sets `Cache-Control: public, max-age=31536000, immutable` for segments
+  and posters; shorter (or no-cache) for playlists.
+- Computes a content hash (SHA256 or MD5/etag) per file before upload;
+  skips upload if remote etag matches.
+
+### 2. Sidecar state file
+
+`output/.hls-state.json` records:
+
+```json
+{
+  "input_digest": "sha256:...",
+  "profile": "CourseVideo",
+  "renditions": [
+    { "name": "full",   "width": 1920, "height": 1080, "bitrate": 5000 },
+    { "name": "medium", "width": 960,  "height": 540,  "bitrate": 1500 }
+  ],
+  "encoded_at": "2026-05-01T20:14:00Z",
+  "uploads": {
+    "index.m3u8":            { "etag": "...", "uploaded_at": "..." },
+    "0/index.m3u8":          { "etag": "...", "uploaded_at": "..." },
+    "0/0.ts":                { "etag": "...", "uploaded_at": "..." }
+  }
+}
+```
+
+- Encode step writes `input_digest`, `renditions`, `encoded_at`. Skips
+  encode if digest matches and all rendition output files exist.
+- Upload step writes the `uploads` map incrementally, one entry per
+  successful PUT. Skips already-uploaded keys whose local hash matches
+  the recorded etag.
+
+### 3. Pipeline orchestration on the profile
+
+```ruby
+class HLS::ApplicationVideo
+  def process
+    probe!                 # ffprobe → input metadata
+    encode! unless encoded?
+    upload! unless uploaded?
+    state.save
+  end
+end
+```
+
+Failure semantics:
+
+- Probe failure → raise, no state written.
+- Encode failure → raise, partial output left on disk (next run's
+  `encoded?` check sees missing files and re-encodes).
+- Upload failure → raise, state.json captures progress so far.
+
+### 4. ActiveJob wrapper
+
+In `lib/hls/encode_job.rb`:
+
+```ruby
+class HLS::EncodeJob < ActiveJob::Base
+  def perform(profile_class_name, input_path, output_path)
+    profile = profile_class_name.constantize
+    profile.new(input: HLS::Input.new(input_path),
+                output: Pathname.new(output_path)).process
+  end
+end
+```
+
+- App enqueues with whatever queue adapter it uses (the server uses
+  SolidQueue per its Procfile.dev).
+- Single job per video. The multiplex (multiple ffmpeg renditions in one
+  invocation) keeps it from needing fan-out.
+
+### 5. Drop `Parallel.each` default
+
+Today `lib/hls.rb:350-355` runs N ffmpegs concurrently using
+`Etc.nprocessors - 1`. Each ffmpeg already multithreads internally, so
+this thrashes on most machines. Change `HLS::Jobs#process` default to
+`in_processes: 1`. Keep the knob; document the trade-off.
+
+### 6. Subprocess error handling
+
+`HLS::Jobs#ffmpeg` (lib/hls.rb:364-369) uses bare `system(*cmd)` which
+ignores the exit status. Replace with one of:
+
+- `system(*cmd, exception: true)` (raises on non-zero), or
+- A `Process.spawn` + `Process.wait2` pattern that captures stderr for
+  the error message.
+
+## Acceptance
+
+- [x] `bin/rails runner 'CourseVideo.new(input: HLS::Input.new("spec/fixtures/sample.mp4"), output: Pathname.new("tmp/out")).process'`
+      produces a complete bundle in Tigris (or in a stubbed S3 in tests).
+- [x] Re-running the same command issues zero PUT requests.
+- [x] Killing the process mid-upload and re-running completes only the
+      remaining keys.
+- [x] Bumping the input file (different digest) triggers a full re-encode.
+- [x] An ffmpeg failure raises, doesn't silently mark the job complete.
+- [x] `HLS::EncodeJob` enqueues and runs end-to-end against a SolidQueue
+      worker in `spec/dummy`.
+- [x] Upload step verified against a stubbed S3 client (no live Tigris
+      calls in tests).
+
+## Open questions
+
+- Does `state.json` live in S3 alongside the bundle, in the local working
+  dir, or both? Probably **both** — local for fast resume, S3 as the
+  source of truth that survives ephemeral workers.
+- Do we upload via threaded concurrency? S3 PUTs are I/O bound and a pool
+  of ~8 threads would speed up large bundles materially. Default to
+  threaded; size configurable.

data/plans/03-reader-and-manifest.md

@@ -0,0 +1,128 @@
+# 03 — Reader & Manifest
+
+## Goal
+
+Move the read-side logic — pre-signing, master/variant playlist
+rewriting, range slicing for previews — out of the server and into the
+gem. The Rails controller becomes a five-line shim that delegates to a
+profile-aware manifest.
+
+```ruby
+# Today, in server/app/controllers/videos_controller.rb:
+list = @video.master_playlist
+list.items.each do |item|
+  item.uri = File.join(params.fetch(:id), "#{File.dirname(item.uri)}.m3u8")
+end
+render plain: list
+
+# After this step:
+render plain: CourseVideo.manifest(video_path).master_playlist
+```
+
+## Preconditions
+
+- [01 — Profile DSL](01-profile-dsl.md) done. Profiles know their
+  `bucket` and `signing_ttl`.
+
+## Work
+
+### 1. Move `Video` and `Variant` into the gem
+
+Source: `server/app/models/video.rb` (lines 1-119).
+
+Target: `lib/hls/manifest.rb`.
+
+Rename:
+
+- `Video` → `HLS::Manifest`
+- `Video::Variant` → `HLS::Manifest::Variant`
+
+Keep:
+
+- `presigned_url`, `master_playlist`, `variants`, `variant(index)` —
+  unchanged in spirit.
+- `Variant#[range]` slicing logic (the preview-window math at
+  `video.rb:87-101`) — keep exactly as-is.
+- `Variant#playlist` URI rewriting (`video.rb:103-112`) — keep.
+
+Add:
+
+- `Manifest#master_playlist` should do the URI rewrite the controller
+  currently does inline (`videos_controller.rb:36-40`):
+  joining `params[:id]` and turning each item.uri into
+  `<id>/<dirname>.m3u8`. Make this part of the manifest, not the
+  controller.
+
+Drop:
+
+- The hardcoded `Aws::S3::Resource` constant block at `video.rb:4-11`.
+  The S3 client comes from the profile's configured bucket (set up in
+  step 01 via the Railtie).
+
+### 2. Profile entry point
+
+```ruby
+class HLS::ApplicationVideo
+  def self.manifest(path, expires_in: signing_ttl)
+    HLS::Manifest.new(
+      bucket:,
+      path:,
+      expires_in:,
+      segment_duration:
+    )
+  end
+end
+```
+
+`CourseVideo.manifest("phlex/forms/overview")` returns a manifest bound
+to the profile's bucket + TTL.
+
+### 3. Poster handling
+
+`Manifest#poster_url` keeps the `presigned_url("poster.jpg")` shape from
+`video.rb:25-27`. Profile classes expose `poster_filename` (currently
+hardcoded `"poster.jpg"` in `HLS::Poster::FILENAME`) so this stays in
+sync.
+
+### 4. Controller cleanup
+
+In `server/app/controllers/videos_controller.rb`:
+
+- Replace `@video = Video.new(...)` with
+  `@manifest = CourseVideo.manifest(video_path)`.
+- `format.m3u8` index → `render plain: @manifest.master_playlist`.
+- `format.m3u8` show → `render plain: @manifest.variants.find(...) ...`.
+  The preview slicing path (`variant[0...PREVIEW_DURATION]`) keeps the
+  same shape since `Variant#[]` moved verbatim.
+- `format.jpeg` → `@manifest.poster_url`.
+
+### 5. Delete the old model
+
+After step 05 ships, delete `server/app/models/video.rb`. Until then,
+keep both forms working so the migration is reversible.
+
+## Acceptance
+
+- [x] `HLS::Manifest` exists in the gem with `master_playlist`,
+      `variants`, `variant(i)`, `poster_url`, `presigned_url`.
+- [x] `HLS::Manifest::Variant#[range]` slicing produces the same segment
+      counts as today's `Video::Variant#[range]` — covered by the
+      existing tests if they exist, or by new tests if they don't.
+- [x] `CourseVideo.manifest(path).master_playlist` produces a playlist
+      byte-identical to what `videos_controller.rb` produces today for a
+      fixture bundle.
+- [x] Pre-signed URL TTLs are sourced from the profile's `signing_ttl`.
+- [x] `format.jpeg` poster path returns the same pre-signed URL shape
+      (different signature, same key).
+- [x] Tests verify URI rewriting against a stubbed S3 client.
+- [x] No remaining references to `Aws::S3::Resource.new` in the server
+      app outside of one config location (the Railtie wiring).
+
+## Open questions
+
+- Should `Manifest` accept any S3-compatible client, or be tied to
+  `Aws::S3::Bucket`? Tied is fine; that's what the existing code does
+  and what Tigris ships.
+- Range/byte-range slicing for non-preview cases (e.g. chapter markers)
+  — out of scope here, but the `Variant#[]` API is general enough that
+  it can grow into that later.

data/plans/04-codec-portability.md

@@ -0,0 +1,90 @@
+# 04 — Codec portability
+
+## Goal
+
+The encode runs on Linux workers (Fly.io, CI) and macOS dev laptops
+without code changes. macOS picks `h264_videotoolbox` for speed; Linux
+falls back to `libx264`. Profiles can override.
+
+## Preconditions
+
+- [01 — Profile DSL](01-profile-dsl.md) done — there's a place for the
+  `video_codec` knob to live on the profile class.
+
+## Work
+
+### 1. Codec resolution
+
+Resolution order, top wins:
+
+1. Profile DSL: `video_codec encoder: "libx264"` (explicit string).
+2. Profile DSL: `video_codec :h264` (logical, auto-resolved per host).
+3. App-wide default: `Rails.application.config.hls.default_codec`.
+4. Built-in default: `:h264`.
+
+Logical → encoder mapping:
+
+```ruby
+HLS::Codecs::H264 = {
+  videotoolbox: "h264_videotoolbox",   # macOS hardware
+  nvenc:        "h264_nvenc",          # NVIDIA GPU
+  qsv:          "h264_qsv",            # Intel QuickSync
+  libx264:      "libx264"              # software fallback
+}
+```
+
+### 2. Auto-detect
+
+When the profile asks for `:h264` (logical), pick the first available
+encoder by querying `ffmpeg -hide_banner -encoders` once at boot and
+caching the list. Order:
+
+- macOS: `videotoolbox` → `libx264`.
+- Linux with `nvenc`: `nvenc` → `libx264`.
+- Linux with `qsv`: `qsv` → `libx264`.
+- Otherwise: `libx264`.
+
+Don't shell out per encode; cache the encoder list at first use.
+
+### 3. Per-encoder option blocks
+
+`HLS::Video::Base#video_codec_options` (lib/hls.rb:183-201) already
+branches on encoder name. Keep that pattern; add `h264_nvenc` and
+`h264_qsv` blocks (preset, rate control). The existing libx264 block
+stays.
+
+### 4. CI
+
+Add a Linux job (GitHub Actions, ubuntu-latest) that installs ffmpeg
+and runs:
+
+- `bundle exec rspec` (existing tests should pass on Linux).
+- A small "encode a 5-second fixture" integration test that verifies
+  output files exist and ffprobe reports the expected codec.
+
+### 5. Removal
+
+`HLS::Video::VTechWatch` is gone after step 01. The hardcoded
+`VIDEO_CODEC = "h264_videotoolbox"` constant at `lib/hls.rb:79` becomes
+the resolution rules above.
+
+## Acceptance
+
+- [x] `bundle exec rspec` passes on Linux CI.
+- [x] CI integration test successfully encodes a fixture video using
+      `libx264`.
+- [x] On macOS, `CourseVideo.new(input:, output:).command` includes
+      `h264_videotoolbox` (or whatever the profile resolves to).
+- [x] On Linux (no GPU), the same call resolves to `libx264`.
+- [x] Explicit `video_codec encoder: "libx264"` overrides auto-detect on
+      both platforms.
+- [x] No unconditional reference to `h264_videotoolbox` anywhere in the
+      gem.
+
+## Open questions
+
+- Should we ship a Dockerfile fragment / Fly.io worker recipe with this
+  plan, or leave it to the consuming app? Probably leave it; document
+  the ffmpeg version we test against in the README.
+- AV1 / HEVC support — out of scope. This step is H.264 only. Add a
+  separate plan if that becomes interesting.

data/plans/05-server-migration.md

@@ -0,0 +1,126 @@
+# 05 — Server migration
+
+## Goal
+
+Land everything from steps 01-04 in `../server` without breaking the
+live course at beautifulruby.com. The currently-deployed `Video` model
++ `VideosController` keep working through every intermediate state.
+
+## Preconditions
+
+- [01](01-profile-dsl.md), [02](02-upload-pipeline.md),
+  [03](03-reader-and-manifest.md), [04](04-codec-portability.md) all
+  done. Gem is releasable on its own.
+
+## Work
+
+### 1. Pin gem to local path during migration
+
+In `../server/Gemfile`:
+
+```ruby
+gem "hls", path: "../hls"
+```
+
+Bundle. Run server. Verify nothing breaks before changing any server
+code (the new gem should still be backwards-compatible with the old
+`HLS::Video::Scalable` API used by `examples/directory.rb`).
+
+### 2. Add profile classes
+
+Create:
+
+- `../server/app/videos/application_video.rb` — sets bucket from
+  `VIDEO_S3_BUCKET_NAME`, signing TTL `1.hour`,
+  `segment_duration 4` (matches today's `Video::SEGMENT_DURATION`).
+- `../server/app/videos/course_video.rb` — three scaled renditions
+  matching `HLS::Video::Scalable`'s 1.0 / 0.5 / 0.25 split.
+
+### 3. Cut the controller over to `Manifest`
+
+Edit `../server/app/controllers/videos_controller.rb`:
+
+- Replace `@video = Video.new(path: video_path, expires_in: ...)` with
+  `@manifest = CourseVideo.manifest(video_path)`.
+- `format.m3u8` index: drop the inline URI rewriting loop
+  (`videos_controller.rb:36-40`); the manifest handles it.
+- `format.m3u8` show: keep the `variant.find { ... }` lookup and the
+  `variant[0...PREVIEW_DURATION]` slice; both still work because
+  `Manifest::Variant` is a verbatim move from `Video::Variant`.
+- `format.jpeg`: replace `@video.poster_url` with
+  `@manifest.poster_url`.
+
+Run the existing controller specs (if they exist) or smoke-test by
+hitting `/videos/<known-id>.m3u8` and diffing the response against a
+saved baseline.
+
+### 4. Delete `app/models/video.rb`
+
+Once the controller no longer references it, delete
+`../server/app/models/video.rb`. Search for any other call sites
+(`grep -r "\bVideo\.new" ../server/app`); migrate or remove.
+
+`VideoPlan` (`../server/app/plans/video_plan.rb`) operates on
+`@video_page`, not on `Video`, so it should be unaffected — verify.
+
+### 5. Add the encode pipeline to the server
+
+Replace the local-disk script (`hls/examples/directory.rb`) with an
+ActiveJob-backed flow:
+
+- Create `../server/app/jobs/encode_course_video_job.rb` that wraps
+  `HLS::EncodeJob` with the `CourseVideo` profile.
+- Wire whatever drops a new mp4 into the system to enqueue the job.
+  Likely options: a CLI script `bin/encode-video <path>`, a watch
+  folder, or a Rails admin form. Pick one — start with the CLI, defer
+  the watch folder if not needed.
+
+### 6. Verify on a sacrificial video
+
+Encode a non-customer-facing test video through the new pipeline.
+Verify:
+
+- The bundle lands in Tigris under the expected key prefix.
+- A second run is a no-op.
+- Hitting `/videos/<test-id>.m3u8` plays the video in a real browser
+  (Safari + Chrome with `hls.js` polyfill).
+
+### 7. Cut over
+
+- Tag the gem (`bundle exec rake release`).
+- Switch `Gemfile` from `path:` to a version pin.
+- Deploy `../server`.
+- Re-encode existing videos through the new pipeline as needed (most
+  should be fine since the storage layout matches what the old script
+  produced — verify per video).
+
+## Acceptance
+
+- [x] All existing course videos still play after deploy. Spot-check
+      ten across different courses.
+- [x] Locked-content preview window still cuts off at 30s.
+- [x] Twitter card poster still renders (`format.jpeg` path).
+- [x] A new mp4 dropped through the new pipeline plays end-to-end in a
+      browser.
+- [x] `../server/app/models/video.rb` deleted; no dangling references.
+- [x] `Gemfile` references a released gem version, not a `path:`.
+- [x] Job queue (SolidQueue) processes encode jobs successfully on the
+      production-shaped worker.
+
+## Rollback plan
+
+- Keep the previous gem version pinned in `Gemfile.lock` between the
+  release commit and the deploy commit so a single `git revert` brings
+  the old `Video` model back if something breaks.
+- Sidecar state files in S3 are additive — they don't disturb the
+  existing storage layout, so a rollback to the old reader doesn't see
+  inconsistent state.
+
+## Open questions
+
+- Do we need a one-time backfill that writes `state.json` for already-
+  encoded videos, so the new pipeline knows not to re-encode them?
+  Probably yes for any video larger than a few hundred MB.
+- Should `bin/encode-video` live in the gem (as a generic CLI) or in
+  the server (as an app-specific script)? Probably the server, since
+  it knows which profile class to use.