task_progress 0.0.1

data/README.md

@@ -0,0 +1,158 @@
+# TaskProgress
+
+Progress for task trees: in one process, across threads, Fibers, Ractors and subprocesses, and out to a terminal or a browser.
+
+Think progress bars, but properly, for complex task trees that the user needs to be updated on. Fit for both the web and the CLI.
+
+The progress is not a mutable structure but an _event log_ which allows task progress collection to be distributed. Every change goes through one lock, which numbers it and commits it as a batch. Outputs receive the batches in order: a terminal renderer, a pipe, an SSE stream, a Sossego stream. A batch is one transaction. On a pipe it is one line, in a stream one event or row, so a reader gets all of it or none of it.
+
+Any run of batches can be squashed into one, and a reader that joins late starts from that squash. This also lends itself perfectly to redraw-the-world UIs like React - but using it with Stimulus is a breeze as well.
+
+<div>
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="docs/bench-dark.png">
+  <img alt="The bench (bin/dev) mid-job: an import with worker threads and an engine subprocess mounted under its own node, drawn by the Stimulus and React renderers side by side above the stream inspector" src="docs/bench-light.png" width="100%">
+</picture>
+</div>
+
+The [MANUAL](MANUAL.md) has everything, and [RECIPES](RECIPES.md) whole setups to copy from - a Rails job polled from a page, subprocesses drawn in one terminal, a Rails action streaming to React. This page has what you need first.
+
+## Install
+
+```ruby
+gem "task_progress", github: "julik/task_progress"
+```
+
+Ruby 3.4 or newer. No runtime dependencies. Typed: RBS signatures ship in `sig/generated`, and the code is checked against them with Steep.
+
+## A bar
+
+```ruby
+bar = TaskProgress.bar("Importing", total: rows.size)
+rows.each { import(it) && bar.tick }
+
+TaskProgress.bar("Importing", total: rows.size).iterate(rows) { |row| import(row) }  # or let it do the counting
+```
+
+```
+⠹ Importing  ██████████░░░░░░░░░░ 21/40  19/s  ETA 1.0s  1.1s
+```
+
+It finishes at its total, and is finished for you when the program exits. `bar.puts("Skipped row 12")` prints a line above it that stays.
+
+## A tree
+
+```ruby
+TaskProgress.cli("Import") do |root|
+  download = root.subtask("Download", total: io.size, unit: "bytes")
+  rows = root.subtask("Rows", total: rows.size, weight: 3, cancelable: true)
+
+  download.run { copy(download.track_io(io)) }
+  rows.run { rows.each { import(it) && rows.tick } }
+end
+```
+
+A block is a task's work: done when it returns, failed with the error when it raises. A parent reads 100% only once it and everything under it are done. On a terminal the tree redraws in place; piped or on CI it writes an execution log instead - one finished line per thing worth saying, never a redraw.
+
+## With Turbo
+
+In a Rails app with turbo-rails, a job draws its tree in every page that shows it - from whichever process it runs in, with no JavaScript of your own. The tree is drawn for a record, here a `CsvImport` - any Active Record model will do, the one whose page shows the progress:
+
+```ruby
+class ImportJob < ApplicationJob
+  def perform(csv_import)
+    TaskProgress.broadcast(csv_import, "Importing") do |root|
+      root.subtask("Rows", total: csv_import.rows.size, cancelable: true) do |rows|
+        csv_import.rows.each { import_row(it) && rows.tick }
+      end
+    end
+  end
+end
+```
+
+```erb
+<%# app/views/csv_imports/show.html.erb %>
+<%= task_progress_tree @csv_import %>
+```
+
+The job draws the whole tree as HTML a few times a second, and Action Cable takes it to every page, where Turbo morphs it in. Cancel buttons are plain forms. Nothing of Rails is needed for the rest of the gem: outside Rails, `to:` says where frames go, and `TurboSSE` streams them to a `<turbo-stream-source>`.
+
+## In a web app
+
+Stream any task - the whole tree, or one branch of it - as Server-Sent Events:
+
+```ruby
+# Rails
+headers.merge!(TaskProgress::SSE::HEADERS)
+self.response_body = TaskProgress::SSE.new(task)
+```
+
+and draw it with the browser client that ships in `js/` (`TaskProgress.js_path`): plain ES modules, no build step, with a Stimulus controller and a React component on top.
+
+```html
+<div data-controller="task-tree" data-task-tree-url-value="/jobs/42/progress"></div>
+```
+
+Where SSE does not get through, `TaskProgress::Poll` answers plain GETs - with a 304 while nothing changed - and `followPollingAt` polls it. Work that runs in the browser itself can be grafted under a server task with `LocalTasks`.
+
+## Getting the browser client
+
+The JS ships inside the gem, at the gem's version, and is not on npm. Take it from the gem in one of three ways:
+
+- **Rails with importmap-rails:** nothing to do. The Railtie serves `js/` at `/task_progress/js` and pins it in your import map as `task_progress`, `task_progress/react`, `task_progress/stimulus`, `task_progress/local_tasks`, `task_progress/row_view` and `task_progress/clock`. Add `<link rel="stylesheet" href="/task_progress/js/task_tree.css">` for the styles.
+- **A bundler - Vite, esbuild, Inertia:** copy it into your app, and install the copy as a package:
+
+  ```
+  $ bin/rails task_progress:vendor                    # or `task_progress vendor` outside Rails
+  $ npm install ./vendor/javascript/task_progress
+  ```
+
+  Then `import { TaskTreeView } from "task_progress/react"` and `import "task_progress/task_tree.css"`, with TypeScript declarations included. Commit the copy, and vendor again after updating the gem - Rails warns at boot while the copy is from another version.
+- **Anything else on Rack:** serve `TaskProgress.js_path` under any prefix. The modules import each other by relative path, and need no build step.
+
+React and Stimulus are not bundled: the components import `react`, `react-dom/client` and `@hotwired/stimulus` by name, from your app.
+
+## Handing work off
+
+Create the tree here, and let a job write to it there - through Redis:
+
+```ruby
+TaskProgress.stream_store = TaskProgress::RedisStreamStore.new(RedisClient.config(url: ENV["REDIS_URL"]).new_pool)
+
+progress = TaskProgress.handoff("Importing", cancelable: true)
+ImportJob.perform_later(csv_import, progress)              # the task is an argument like any other
+
+def perform(csv_import, progress)                          # ...and arrives adopted: perform is its work
+  progress.subtask("Rows", total: csv_import.rows.size) { |rows| ... }
+end
+
+TaskProgress::SSE.response(progress.key)                   # a page's stream, anywhere that reaches Redis
+```
+
+`csv_import` is the same `CsvImport` record as above; `progress` is the tree. `TaskProgress.for_page(progress.key)` is everything a page needs - links with a signed token in them, never the key, and the tree as it stands - for a Stimulus value or an Inertia prop: `<TaskTreeView progress={progress} onComplete={() => router.reload()} />`, with the client from `rails task_progress:vendor` - or pinned by itself, with importmap-rails. The page shows the job queued before a worker has it. A worker that dies goes stale, then fails; a retry starts the tree over as attempt 2. `task_progress watch KEY` draws it in a terminal.
+
+## Across processes
+
+```ruby
+engine = root.mount("Engine", weight: 3)
+engine.spawn(RbConfig.ruby, "engine.rb")  # engine.rb uses TaskProgress.cli, and reports up a pipe
+```
+
+The child's tree lands under the mount, and a cancel goes down to it the same way. Ractors work too, on Ruby 4.0.
+
+## Canceling
+
+```ruby
+task.cancel!("Not needed")                     # from any thread
+root.request_cancel(path, "From the browser")  # as a request: :canceled, :forwarded, :moot or :refused
+```
+
+Whoever works on a canceled task next gets a `TaskProgress::Canceled`, which `run` takes as the end of that work. Blocking work can use `on_cancel { }` to unblock. The renderers show a cancel button on tasks marked `cancelable: true`, and `TaskProgress::Token` signs the right to press it.
+
+## The bench
+
+`bin/dev` starts a bench on http://localhost:9292: pick a scenario and watch it in both renderers side by side, with a stream inspector. Cancel tasks, cut the stream to watch it resume, or join a job late. The terminal demo is `ruby -Ilib examples/cli_demo.rb`.
+
+## License
+
+MIT

data/RECIPES.md

@@ -0,0 +1,413 @@
+# TaskProgress recipes
+
+Whole setups to copy from, start to end. The [README](README.md) has the short version, and the [MANUAL](MANUAL.md) everything else.
+
+- [Relaying a command's progress: ffmpeg](#relaying-a-commands-progress-ffmpeg)
+- [A video transcoded in a background job, polled from a Rails page](#a-video-transcoded-in-a-background-job-polled-from-a-rails-page)
+- [The same video on an Inertia page, with React](#the-same-video-on-an-inertia-page-with-react)
+- [Videos transcoded in subprocesses, drawn in one terminal](#videos-transcoded-in-subprocesses-drawn-in-one-terminal)
+- [A Rails action that streams its own progress to React](#a-rails-action-that-streams-its-own-progress-to-react)
+- [Upload progress, all in the browser](#upload-progress-all-in-the-browser)
+
+Every Rails recipe leans on what the Railtie sets up by itself: the browser client served at `/task_progress/js`, trees as job arguments, and the `task_progress_tree` helper.
+
+## Relaying a command's progress: ffmpeg
+
+Most long work is somebody else's program, and most such programs say how far along they are - ffmpeg among the most talkative. Read what it says, and move a task along with it. The two recipes after this one use it as it is.
+
+```ruby
+# lib/transcoder.rb
+require "open3"
+
+# Runs ffmpeg, and moves `task` along through the seconds of the input as ffmpeg gets through
+# them. ffmpeg writes its progress as key=value lines to wherever `-progress` says - here, its
+# stdout. Canceling the task asks ffmpeg to stop, the way a person at its terminal would: with a "q"
+module Transcoder
+  Failed = Class.new(StandardError)
+
+  def self.run(input, output, task, *options)
+    duration = Float(IO.popen(["ffprobe", "-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", input], &:read))
+    task.total = duration.ceil
+    command = ["ffmpeg", "-hide_banner", "-y", "-i", input, *options, "-progress", "pipe:1", "-nostats", output]
+    Open3.popen3(*command) do |stdin, stdout, stderr, wait|
+      said = Thread.new { stderr.read rescue "" } # where ffmpeg says what went wrong
+      task.on_cancel { stdin.write("q") rescue nil }
+      speed = nil
+      stdout.each_line do |line|
+        key, value = line.chomp.split("=", 2)
+        case key
+        when "speed" then speed = value.strip
+        when "out_time_us"
+          seconds = value.to_i / 1_000_000
+          task.advance(seconds - task.count, message: ("#{speed} realtime" if speed)) if seconds > task.count
+        end
+      end
+      raise Failed, said.value.lines.last(3).join.strip unless wait.value.success?
+    end
+  end
+end
+```
+
+```ruby
+TaskProgress.cli("clip.mov") do |root|
+  root.subtask("720p", cancelable: true) do |task|
+    Transcoder.run("clip.mov", "clip-720p.mp4", task, "-vf", "scale=-2:720", "-c:v", "libx264")
+  end
+end
+```
+
+```
+⠋ clip.mov  ██████████████░░░░░░  70%  0.5s
+  ⠋ 720p  ██████████████░░░░░░ 14/20  0.5s  15.1x realtime
+```
+
+The task counts seconds of video, so the bar comes for free - and after the first second, the rate and the time left too. It finishes by itself at its total. Canceled, ffmpeg is told to stop, and the next `advance` raises `TaskProgress::Canceled`, which ends the work quietly. If ffmpeg fails, the task fails with what ffmpeg last said:
+
+```
+✗ clip.mov › 720p: Error opening output /nonexistent/clip-720p.mp4: No such file or directory
+```
+
+Anything else that prints its progress goes the same way: `rsync --info=progress2`, `curl --progress-bar`, `pg_restore --verbose`, a `docker build`. Read its lines, find the number, and `advance` or `update(percent)`.
+
+## A video transcoded in a background job, polled from a Rails page
+
+A video is uploaded, a job transcodes it, and the page asks every second how it is doing, with a plain GET - no WebSockets, no SSE, nothing a proxy can get in the way of. Nothing is sent while nothing changes: an unchanged tree is a 304.
+
+**1. Where trees live.** Jobs run in other processes, so the tree goes through Redis:
+
+```ruby
+# Gemfile
+gem "task_progress"
+gem "redis-client"
+
+# config/initializers/task_progress.rb
+TaskProgress.stream_store = TaskProgress::RedisStreamStore.new(RedisClient.config(url: ENV.fetch("REDIS_URL", "redis://localhost:6379")).new_pool(size: 10))
+```
+
+In development with the `:async` queue adapter, jobs run in the same process as the pages, and you can leave this out - the stream is then kept in memory.
+
+**2. Remember which tree is whose.** A tree is named by its key:
+
+```ruby
+add_column :videos, :progress_key, :string
+```
+
+**3. Hand the tree to the job.**
+
+```ruby
+# app/controllers/videos_controller.rb
+class VideosController < ApplicationController
+  def create
+    video = Video.create!(video_params) # has_one_attached :original, has_many_attached :renditions
+    progress = TaskProgress.handoff("Transcoding #{video.original.filename}", cancelable: true)
+    video.update!(progress_key: progress.key)
+    TranscodeJob.perform_later(video, progress)
+    redirect_to video
+  end
+
+  def show
+    @video = current_user.videos.find(params[:id])
+  end
+end
+```
+
+The tree is pending from the moment it is handed off, so the page shows the video queued before any worker has it. No routes or actions for the progress itself: the Railtie mounts them, and the page gets links to them from `TaskProgress.for_page` - links with a signed token in them, never the tree's key. Handing them out in `show`, which only finds the user's own videos, is the authorization.
+
+**4. Do the work.** The job gets the tree as an argument, already taken up. Its `perform` is the tree's work: done when it returns, failed with the error when it raises. Each size is a task of its own, moved along by [the ffmpeg relay](#relaying-a-commands-progress-ffmpeg):
+
+```ruby
+# app/jobs/transcode_job.rb
+class TranscodeJob < ApplicationJob
+  SIZES = {"1080p" => [], "720p" => ["-vf", "scale=-2:720"], "480p" => ["-vf", "scale=-2:480"]}
+  retry_on Transcoder::Failed, attempts: 2
+
+  def perform(video, progress)
+    video.original.open do |original|
+      SIZES.each do |size, options|
+        output = "#{Dir.tmpdir}/#{video.id}-#{size}.mp4"
+        progress.subtask(size, weight: 2, cancelable: true) do |task|
+          Transcoder.run(original.path, output, task, *options, "-c:v", "libx264", "-preset", "fast")
+        end
+        progress.subtask("Storing #{size}") { video.renditions.attach(io: File.open(output), filename: "#{size}.mp4") }
+      end
+    end
+  end
+end
+```
+
+Storing is a task too, without a measure: it counts for little (weight 1 against the transcode's 2) and shows a spinner while it runs. A retry starts the tree over, and the page shows "attempt 2". A worker that dies without a word goes stale on the page after a few seconds, and fails after a minute. Canceling "720p" from the page stops ffmpeg and ends that task - the job goes on with 480p; cancel the root to stop the job, without failing it.
+
+**5. Draw it.** With importmap-rails, the Railtie has pinned `task_progress/stimulus` already.
+
+```js
+// app/javascript/controllers/index.js
+import TaskTreeController from "task_progress/stimulus"
+application.register("task-tree", TaskTreeController)
+```
+
+```erb
+<%# app/views/videos/show.html.erb %>
+<link rel="stylesheet" href="/task_progress/js/task_tree.css">
+<div class="tree" data-controller="task-tree"
+     data-task-tree-progress-value="<%= TaskProgress.for_page(@video.progress_key).to_json %>"
+     data-task-tree-poll-value="1"></div>
+```
+
+The tree is drawn at once, from the snapshot `for_page` puts in. Then it polls every second (`poll-value`), draws once a frame at most, and stops once the video is through. Its cancel buttons ask where `for_page` says.
+
+To have each poll wait for news instead - long polling - set `data-task-tree-wait-value="25"` in place of the poll value; every waiting poll holds a server thread, so size Puma for the pages you expect open at once. Leave both out, and it follows the tree over Server-Sent Events, falling back to polling by itself where those do not get through.
+
+## The same video on an Inertia page, with React
+
+The job and the controller's `create` are [the ones above](#a-video-transcoded-in-a-background-job-polled-from-a-rails-page). What changes is the page: a React component that Inertia renders, and that reloads the video once its renditions are there.
+
+**1. The browser client, vendored from the gem.**
+
+```
+$ bin/rails task_progress:vendor
+$ npm install ./vendor/javascript/task_progress
+```
+
+The first copies the gem's `js/` into `vendor/javascript/task_progress`, as a package named `task_progress` with TypeScript declarations; the second links it into `node_modules`, for Vite to resolve like any other. React is a peer dependency - the one your app has. Commit the vendored copy, and run the task again after updating the gem - Rails warns at boot when the copy is from another version of the gem.
+
+**2. One prop.**
+
+```ruby
+# app/controllers/videos_controller.rb
+def show
+  video = current_user.videos.find(params[:id])
+  render inertia: "Videos/Show", props: {
+    video: video.as_json(methods: :rendition_urls),
+    progress: -> { TaskProgress.for_page(video.progress_key) if video.progress_key }
+  }
+end
+```
+
+`for_page` hands out the links to the tree's events, polling and cancellations - with a signed token in them, good for an hour - and the tree as it stands, for the first render. As a lambda, Inertia leaves it out of a partial reload that does not ask for it.
+
+**3. The component.**
+
+```tsx
+// app/frontend/pages/Videos/Show.tsx
+import { router } from "@inertiajs/react"
+import { TaskTreeView, type Progress } from "task_progress/react"
+import "task_progress/task_tree.css"
+
+export default function Show({ video, progress }: { video: Video; progress: Progress | null }) {
+  return (
+    <>
+      <VideoPlayer video={video} />
+      {progress && (
+        <div className="tree">
+          <TaskTreeView progress={progress} onComplete={() => router.reload({ only: ["video"] })} />
+        </div>
+      )}
+    </>
+  )
+}
+```
+
+It draws the snapshot straight away - server-side rendering included - follows the tree over Server-Sent Events, and falls back to polling by itself where those do not get through. When the root is done, `onComplete` reloads `video`, and the player has its renditions; `onFail` is there for the other outcome. Navigating away closes the stream.
+
+To draw it your own way - with your design system's components, say - take the rows instead:
+
+```tsx
+import { useTaskTree } from "task_progress/react"
+
+const { rows, cancel } = useTaskTree(progress, { onComplete: () => router.reload({ only: ["video"] }) })
+return rows.map((row) => (
+  <ProgressRow key={row.key} depth={row.depth} name={row.name} percent={row.percent} striped={row.indeterminate}
+    status={row.status} message={row.message} onCancel={row.cancelable ? () => cancel(row) : undefined} />
+))
+```
+
+## Videos transcoded in subprocesses, drawn in one terminal
+
+A script that transcodes a pile of videos into two sizes each, a few videos at a time, each in a Ruby process of its own - and draws the lot as one tree. Every process reports up a pipe; Ctrl-C cancels everything and stops every ffmpeg, and a process that crashes fails its own branch and the run with it.
+
+**The worker** is an ordinary command-line program, using [the ffmpeg relay](#relaying-a-commands-progress-ffmpeg). Run on its own, it draws its own tree:
+
+```ruby
+# transcode.rb
+require "task_progress"
+require_relative "lib/transcoder"
+
+input = ARGV.fetch(0)
+base = File.basename(input, ".*")
+TaskProgress.cli(File.basename(input)) do |root|
+  root.subtask("1080p", cancelable: true) { |t| Transcoder.run(input, "#{base}-1080p.mp4", t, "-c:v", "libx264", "-preset", "fast") }
+  root.subtask("720p", cancelable: true) { |t| Transcoder.run(input, "#{base}-720p.mp4", t, "-vf", "scale=-2:720", "-c:v", "libx264", "-preset", "fast") }
+end
+```
+
+**The parent** mounts one worker per video. ffmpeg uses several cores of its own, so it runs half as many at once as there are cores:
+
+```ruby
+# transcode_all.rb
+require "task_progress"
+require "etc"
+
+files = ARGV
+TaskProgress.cli("Transcode #{files.length} files", cancelable: true) do |root|
+  queue = Queue.new
+  files.each { queue << [it, root.mount(File.basename(it), cancelable: true)] }
+  queue.close
+
+  workers = [Etc.nprocessors / 2, files.length].min.clamp(1, nil).times.map do
+    Thread.new do
+      while (file, mount = queue.pop)
+        mount.spawn(RbConfig.ruby, File.expand_path("transcode.rb", __dir__), file)
+      end
+    end
+  end
+  workers.each(&:join)
+end
+```
+
+```
+$ ruby transcode_all.rb *.mov
+```
+
+Every video shows from the start, waiting its turn. `TaskProgress.cli` in the worker sees it was started by `spawn` and writes its batches up the pipe instead of drawing; `spawn` returns once the worker exits. On a terminal the tree redraws in place. Piped, or on CI, it writes an execution log instead:
+
+```
+[ 10%   0.6s] Transcode 2 files › a.mov › 1080p 4/20
+[ 99%   2.4s] ✓ Transcode 2 files › b.mov › 1080p  1.7s
+[ 99%   4.7s] ✓ Transcode 2 files › a.mov  4.1s
+[ 99%   4.7s] ✓ Transcode 2 files › b.mov  4.1s
+[100%   4.7s] ✓ Transcode 2 files  4.1s
+```
+
+ffmpeg is a process of its own already, so the relay works from threads in one Ruby process just as well. A Ruby process per video is what to reach for when the Ruby side does real work of its own, or might fall over. Workers on other machines report the same way through Redis: hand each one a tree with `TaskProgress.handoff`, `adopt` it in the worker, and mount it in the parent with `root.mount(name).follow_stream(TaskProgress.stream_store, key)`. `task_progress watch KEY` draws any one of them from yet another terminal.
+
+## A Rails action that streams its own progress to React
+
+A report that takes a minute to build, built while the page watches: the request does the work, and its response is the progress, as Server-Sent Events. Nothing to queue, nothing to store - when the report is ready, the stream ends.
+
+**1. The action.** `ActionController::Live` runs it on a thread of its own and sends what it writes as it writes it. The work goes on another thread, and the stream follows it:
+
+```ruby
+# config/routes.rb
+get "/reports/:id/build" => "reports#build", as: :build_report
+
+# app/controllers/reports_controller.rb
+class ReportsController < ApplicationController
+  include ActionController::Live
+
+  def build
+    report = Report.find(params[:id])
+    response.headers.merge!(TaskProgress::SSE::HEADERS)
+    root = TaskProgress.new("Building #{report.name}", cancelable: true)
+    work = Thread.new do
+      Rails.application.executor.wrap do
+        root.run { |r| report.build!(progress: r) }
+      end
+    end
+    TaskProgress::SSE.new(root).each { response.stream.write(it) }
+  rescue ActionController::Live::ClientDisconnected
+    root.cancel!("Nobody is watching any more")
+  ensure
+    work&.join
+    response.stream.close
+  end
+end
+```
+
+```ruby
+# app/models/report.rb
+def build!(progress:)
+  queries = progress.subtask("Queries", total: sections.size)
+  sections.each { |section| section.query! && queries.tick }
+  progress.subtask("PDF", indeterminate: true) { render_pdf! }
+end
+```
+
+The tree is sent whole first, then every change as it happens, and the stream ends once the root has finished. A page that goes away is noticed at the next write - a change, or the ping every 15 seconds - and cancels the work. The work is the request's own: each connection builds the report anew, so link to the stream only where you mean to start it. Every open stream holds a Puma thread.
+
+**2. Load the client.**
+
+```
+$ bin/importmap pin react react-dom/client
+```
+
+The Railtie pins `task_progress/react` itself. In an app with a bundler, vendor it instead - see [the Inertia recipe](#the-same-video-on-an-inertia-page-with-react).
+
+**3. Draw it.** In a React app:
+
+```jsx
+import { TaskTreeView } from "task_progress/react"
+
+export function ReportBuilder({ report }) {
+  const [building, setBuilding] = useState(false)
+  if (!building) return <button onClick={() => setBuilding(true)}>Build the report</button>
+  return <div className="tree"><TaskTreeView url={`/reports/${report.id}/build`} /></div>
+}
+```
+
+Or on a page that is not a React app, with no JSX and no build step:
+
+```erb
+<link rel="stylesheet" href="/task_progress/js/task_tree.css">
+<div class="tree" id="report-progress"></div>
+<script type="module">
+  import { createTaskTreeRoot } from "task_progress/react"
+  createTaskTreeRoot(document.getElementById("report-progress")).render({ url: "<%= build_report_path(@report) %>" })
+</script>
+```
+
+`useTaskTree(url)` hands you the tree to draw your own way: `tree.rows()` is what to draw, and `rowView(tree, row)` from `row_view.js` what each row says. A tree of thousands of tasks folds what is done and what waits into a row each, so a big report draws as a handful of rows.
+
+## Upload progress, all in the browser
+
+Files going up from a form, a bar for each, with no tree on the server and nothing to set up there. The tree lives in the page: `LocalTasks` keeps it, `trackUpload` follows an `XMLHttpRequest` - `fetch` reports nothing about a body going up - and the renderers draw it as it changes.
+
+**React:**
+
+```jsx
+import { useState } from "react"
+import { LocalTasks, trackUpload } from "task_progress/local_tasks"
+import { TaskTreeView } from "task_progress/react"
+
+function upload(files) {
+  const uploads = new LocalTasks("Uploading", { cancelable: true })
+  for (const file of files) {
+    const xhr = new XMLHttpRequest()
+    xhr.open("PUT", `/attachments/${encodeURIComponent(file.name)}`)
+    trackUpload(xhr, uploads.root.subtask(file.name, { unit: "bytes", cancelable: true }), { abortOnCancel: true })
+    xhr.send(file)
+  }
+  return uploads
+}
+
+export function Uploader() {
+  const [uploads, setUploads] = useState(null)
+  return <>
+    <input type="file" multiple onChange={(e) => setUploads(upload(e.target.files))} />
+    {uploads && <div className="tree"><TaskTreeView local={uploads} /></div>}
+  </>
+}
+```
+
+**Stimulus** - the same `upload(files)`, drawn by the `task-tree` controller through an outlet:
+
+```html
+<input type="file" multiple data-controller="uploader" data-action="uploader#upload" data-uploader-task-tree-outlet="#uploads">
+<div class="tree" id="uploads" data-controller="task-tree"></div>
+```
+
+```js
+// app/javascript/controllers/uploader_controller.js
+export default class extends Controller {
+  static outlets = ["task-tree"]
+  upload(event) { this.taskTreeOutlet.show(upload(event.target.files)) }
+}
+```
+
+That is all of it. What you get:
+
+- A row per file, with bytes sent, the rate and the time left, under a row for the lot. Put `task_tree.css` on the page for them.
+- A file is done when the server answers 2xx, not when the last byte has gone out. Any other answer fails it with the status, and so do a network error and a timeout.
+- Every cancel button works here, with nothing on the server to ask: a file's button aborts its request, and the top one aborts them all.
+- The top row finishes when you say so. To have it end with the files, collect `task.whenFinished()` from each `trackUpload` and close it once all have ended: `Promise.all(endings).then(() => uploads.root.done())`. It resolves with how the task ended - `"done"`, `"failed"` or `"canceled"` - if you want a failure to fail the lot.
+
+Uploading to S3, or to Active Storage's direct uploads, is the same `XMLHttpRequest`, with another URL and its headers. When the files are a step in a job the server tracks, graft the uploads under that job's task instead - see [Tasks that run in the browser](MANUAL.md#uploads) in the MANUAL.

data/config/importmap.rb

@@ -0,0 +1,17 @@
+# Drawn into the app's import map by the Railtie when importmap-rails is in use, before the app's
+# own config/importmap.rb - so a pin of the same name there wins. The files are pinned where the
+# Railtie serves them as they are: through the asset pipeline they would be fingerprinted, and
+# the relative imports between them would point at names it does not serve
+js = Rails.application.config.task_progress.js_path
+
+{
+  "task_progress" => "task_tree.js",
+  "task_progress/react" => "react_task_tree.js",
+  "task_progress/stimulus" => "stimulus_controller.js",
+  "task_progress/local_tasks" => "local_tasks.js",
+  "task_progress/row_view" => "row_view.js",
+  "task_progress/clock" => "task_progress_clock.js"
+}.each do |name, file|
+  # Not preloaded: the React and Stimulus parts import what only some pages have
+  pin name, to: "#{js}/#{file}", preload: false, integrity: false
+end

data/exe/task_progress

@@ -0,0 +1,51 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "optparse"
+require_relative "../lib/task_progress"
+
+url = ENV.fetch("REDIS_URL", "redis://localhost:6379")
+# In a Rails app, where the Railtie looks for it too
+default_dir = File.exist?("config/application.rb") ? "vendor/javascript/task_progress" : "task_progress"
+parser = OptionParser.new do |o|
+  o.banner = <<~TEXT
+    Usage: task_progress watch KEY [--redis URL]
+           task_progress vendor [DIR]
+
+    watch    Draws a tree handed off with TaskProgress.handoff - running anywhere that reaches
+             the same Redis - until it finishes. Exits 0 if it was complete, 1 if not. Ctrl-C
+             stops watching, and leaves the work alone.
+    vendor   Copies the browser client into DIR (#{default_dir}), replacing what an earlier
+             vendoring left there, for a bundler to take from - `npm install ./DIR` makes it
+             the `task_progress` package.
+
+  TEXT
+  o.on("--redis URL", "Where the tree is kept (REDIS_URL, or #{url})") { url = it }
+end
+command, arg = parser.parse(ARGV)
+
+case command
+when "vendor"
+  dir = arg || default_dir
+  begin
+    files = TaskProgress.vendor_js(dir)
+  rescue ArgumentError => e
+    abort e.message
+  end
+  puts "The task_progress client #{TaskProgress::VERSION} is in #{dir}: #{files.join(", ")}"
+when "watch"
+  abort(parser.help) unless arg
+  begin
+    require "redis-client"
+  rescue LoadError
+    abort "task_progress watch needs the redis-client gem"
+  end
+  stream_store = TaskProgress::RedisStreamStore.new(RedisClient.config(url:).new_pool(size: 2))
+  begin
+    exit(TaskProgress.watch(arg, stream_store:).complete? ? 0 : 1)
+  rescue KeyError => e
+    abort e.message
+  end
+else
+  abort(parser.help)
+end