fun_ci 2.1.1-aarch64-linux → 2.2.0-aarch64-linux

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9749650269dc0a0cbdfaaf5d8c5cf699506bf2f74616cd7559e9b2053ce2afb3
-  data.tar.gz: 2157ecd89e3e75ad0aa9ee0a85b69a74560616446d088605433913a516049caa
+  metadata.gz: 3ffca67ae0d98ad5e4ca3f79a1780e476694e02faddca85821bbbfb05b0e5883
+  data.tar.gz: 4ab9b5ebd8d926121840f4831c8c0fd42d43db31c633375f28187fe1e253460b
 SHA512:
-  metadata.gz: d18c609aed8b49e1e5d9275d49f5e7ddf3fe773aa96b52b396b7cdc2c912c28c4fe8cdca0dc269d949e2ca6f9e9022ff9ede7b551f765e19b8f31578f5304376
-  data.tar.gz: f092c4242f2bd1ab1df43c5e20a7a079ba4b238779cb3cdd3e85e7a87bf2ec3ea80073a87d6f317fc36363e84ae763073bea4c49dfae430ed8ad925cce0a1168
+  metadata.gz: ac4468eaca795bbe2e91ebe364b25bf5df9c90a921adc343451566a281635169e210954bfc129e9cad7f3ba00131e42b556f661da50bc850fe69d8c8b1ca18bb
+  data.tar.gz: a33fcce540b66de69cd392b12aecc117347ce6beefe6267c7b1fe1e65453d97615db9108b6f63dbb2c490671bdbf2d42d6f3980cf5a274e79f5fcdae3714e1c8

data/CHANGELOG.md

@@ -7,6 +7,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.2.0] - 2026-10-01
+
+### Added
+- Daily and weekly jobs, for checks too long for a stage's budget. A script
+  in `.fun-ci/daily/` or `.fun-ci/weekly/` runs at most once a day or once a
+  week, started by your commits, in a worktree of its own, with 24 hours to
+  finish. The console shows them in a section of their own under the table,
+  and `c` cancels a running one; on a screen too short for both, the jobs
+  give way to the branches. `fun-ci check` lists them.
+- The console's job section needs renderer 2.2.0, which the platform gems
+  bring. With an older renderer, as `cargo install` left it, the console
+  works but shows no jobs.
+- `fun-ci prune` removes the jobs' worktrees too.
+- Agents ask about jobs: `fun-ci jobs` lists them, `fun-ci why --job NAME`
+  prints what was kept about one's last run, `fun-ci status` names the jobs
+  that tested the commit and any failing on another, and `fun-ci events`
+  says when a job starts and finishes. The instructions `fun-ci init` writes
+  for agents mention them.
+
+### Fixed
+- On a short screen, a project whose branches had all passed and were left
+  out (counted beside the keys) kept its label, with nothing under it. The
+  label now goes with them.
+
 ## [2.1.1] - 2026-09-30
 
 ### Fixed

data/README.md

@@ -53,6 +53,23 @@ Each run happens in a git worktree of its own under `.git/fun-ci/worktrees/`, ch
 
 Results are kept in SQLite under `$XDG_STATE_HOME/fun-ci/` (`~/.local/state/fun-ci/` by default), shared by every project on the machine.
 
+## Daily and weekly jobs
+
+Some checks take longer than the time between two commits: a mutation run, a soak test, a full audit of your dependencies. Put each in a script of its own under `.fun-ci/daily/` or `.fun-ci/weekly/`:
+
+```
+.fun-ci/daily/mutation.sh
+.fun-ci/weekly/soak.sh
+```
+
+The script's name is the job's name. Like a stage, it gets the commit's hash as its first argument and passes when it exits 0, and `FUN_CI_JOB` tells it which job it is. There is nothing else to configure.
+
+Commits start jobs. When the `post-commit` hook runs a pipeline, it also starts each of the project's jobs that is due, beside the pipeline, on the commit just made. A job is due when it has never run, when its last run was cancelled, or a day (daily) or a week (weekly) after its last run started. A project nobody commits to runs no jobs. Each job runs once at a time, in a worktree of its own under `.git/fun-ci/jobs/`, so it never holds up a commit's run, and has 24 hours before it is stopped. A newer commit never cancels a job. A job's result changes no run, no streak and no exit code.
+
+A failed job keeps its evidence as a failed stage does; add entries under `evidence: jobs: <name>:` in `.fun-ci/config` to keep more. `fun-ci check` lists the jobs it found, and says which scripts it can't run.
+
+`fun-ci jobs` lists the jobs, each with how its last run went, on which commit, and when it is due again. `fun-ci why --job soak` prints everything kept about the job's last run (`--raw` for its whole output, `--json` for a document), and exits as `why` does for a stage. `fun-ci status` names the jobs whose last run tested the commit you ask about, and any job failing on another commit, and `fun-ci events` says when a job starts and finishes. None of them changes a commit's verdict. A job's name is letters, digits, `.`, `_` and `-`, so those commands can be pasted as they are printed.
+
 ## Watching: the console
 
 ```bash
@@ -88,12 +105,16 @@ Five minutes after the latest run finished, the header goes quiet: a starry nigh
 |---|---|
 | ![A fire in a stone hearth, with a green lamp in the corner](docs/screenshots/quiet-fireplace.png) | ![A moonlit island with a palm, with a red lamp in the corner](docs/screenshots/quiet-island.png) |
 
+The daily and weekly jobs have a section of their own under the last project: each job's name, how often it runs, its mark, what its last run did on which branch and commit, and when it is due again (`passed on main 3a1f9c2 · due in 14h`), or that it runs on your next commit. When none of them failed or runs, the section folds into one pale line.
+
+![The daily and weekly jobs under the table: soak failed a day ago on wip/foo, with a red stripe, mutation running for over an hour, with a blue one, deps-audit passed and due again in 14 hours, and lint-deep due on the next commit](docs/screenshots/jobs.png)
+
 A branch that conflicts with the trunk says so on the line under its name, `conflicts with main`, and the header plays two strands braiding into a knot; when a later commit on the branch merges cleanly, the knot unties. A project whose trunk fun-ci last fetched over an hour ago, or couldn't fetch, says so beside its name.
 
 Keys:
 
 - `j` and `k`, or the arrow keys, move the cursor down and up
-- `c` cancels the run under the cursor: one waiting to start at once, a running one once you answer `y` (`n` or `Esc` keeps it running); the footer offers it only while a run is running or waits to start
+- `c` cancels the run under the cursor: one waiting to start at once, a running one once you answer `y` (`n` or `Esc` keeps it running); the footer offers it only while a run is running or waits to start. The cursor moves on past the last branch into the jobs, and `c` cancels a running job the same way; a cancelled job runs again on your next commit
 - `q` quits
 
 The console is drawn in 24-bit colour when `COLORTERM` says the terminal has it (`truecolor` or `24bit`), and in 256 colours otherwise. The pictures above are the renderer's output replayed in a terminal emulator, set in Menlo; in your terminal the text is in your terminal's font.
@@ -227,6 +248,8 @@ fun-ci status [commit] [--trunk]                Where a commit's run stands; the
 fun-ci wait [commit]                            Wait until that verdict is decided, then exit with it
 fun-ci why [commit] [stage|trunk]               Everything kept about why a stage failed, or the conflict with the trunk
 fun-ci runs                                     This project's recent runs, newest first
+fun-ci jobs                                     This project's daily and weekly jobs and how each stands
+fun-ci why --job <name>                         Everything kept about a job's latest run
 fun-ci events [--follow]                        The runs' events as JSON lines
 fun-ci extract <stage> --output <file>          Try a stage's extractors on a saved output
 fun-ci trigger <commit> <branch>                Run the whole pipeline in the foreground

data/lib/fun_ci/agent/command_support.rb

@@ -4,6 +4,7 @@ require_relative "options"
 require_relative "reports"
 require_relative "exit_code"
 require_relative "trunk_verdict"
+require_relative "job_report"
 
 module FunCi
   module Agent
@@ -31,6 +32,12 @@ module FunCi
 
       def reports = @reports ||= Reports.new(@context.db, @context.git, @context.clock, @context.trunk)
 
+      # After recording failed the jobs whose process died, which would otherwise read as running.
+      def job_reports
+        @context.pipeline.watch(@context.db)
+        JobReports.new(@context.db, @context.git.toplevel, @context.clock)
+      end
+
       def usage(message)
         @context.io.stderr.puts "fun-ci #{self.class::NAME}: #{message}"
         ExitCode::USAGE

data/lib/fun_ci/agent/commands.rb

@@ -10,6 +10,7 @@ require_relative "runs_command"
 require_relative "wait_command"
 require_relative "events_command"
 require_relative "why_command"
+require_relative "jobs_command"
 require_relative "../trunk/local"
 
 module FunCi
@@ -18,7 +19,7 @@ module FunCi
     # database and the project's git.
     module Commands
       ALL = { "status" => StatusCommand, "runs" => RunsCommand, "wait" => WaitCommand,
-              "events" => EventsCommand, "why" => WhyCommand }.freeze
+              "events" => EventsCommand, "why" => WhyCommand, "jobs" => JobsCommand }.freeze
 
       def self.run(name, args, context)
         ALL.fetch(name).new(context).run(args)

data/lib/fun_ci/agent/due_in.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module FunCi
+  module Agent
+    # How long until something, rounded up, since it is still to come, in the
+    # largest unit it reaches once rounded: `45m`, `22h`, `7d`; as the console says it.
+    module DueIn
+      def self.words(seconds)
+        minutes = up(seconds, 60).clamp(1..)
+        hours = up(seconds, 3600)
+        return "#{minutes}m" if minutes < 60
+        return "#{hours}h" if hours < 24
+
+        "#{up(seconds, 86_400)}d"
+      end
+
+      def self.up(seconds, unit) = (seconds.to_r / unit).ceil
+      private_class_method :up
+    end
+  end
+end

data/lib/fun_ci/agent/events_command.rb

@@ -4,19 +4,23 @@ require "json"
 require_relative "command_support"
 require_relative "snapshots"
 require_relative "events"
+require_relative "job_events"
 require_relative "waiting"
 
 module FunCi
   module Agent
     # `fun-ci events [--follow] [--only failures]` (acceptance-tests.md,
-    # AT-9.16): one JSON line per event of the project's recent runs, then,
-    # with --follow, each new one as it happens, until interrupted.
+    # AT-9.16, AT-13.23): one JSON line per event of the project's recent runs
+    # and job runs, then, with --follow, each new one as it happens, until
+    # interrupted.
     class EventsCommand
       include CommandSupport
 
       NAME = "events"
       SCHEMA = 1
       RECENT_RUNS = 10
+      # The first look: no runs, no job runs.
+      NOTHING_SEEN = [{}, {}].freeze
 
       def initialize(context)
         @context = context
@@ -24,7 +28,7 @@ module FunCi
 
       def run(args)
         options = Options.parse(args, takes: %i[follow only json])
-        seen = emit({}, options.only)
+        seen = emit(NOTHING_SEEN, options.only)
         follow(seen, options.only) if options.follow
         0
       rescue Options::Invalid => e
@@ -43,13 +47,16 @@ module FunCi
         # Following ends when whoever follows stops it.
       end
 
-      # Prints what happened since `before`, and answers the look it took.
-      def emit(before, only)
-        after = snapshots.take(limit: RECENT_RUNS)
-        Events.between(before, after).select { |event| !only || Events.failure?(event) }.each { |event| print(event) }
+      # Prints what happened since `before` ([runs, job runs]), and answers the look it took.
+      def emit((runs, jobs), only)
+        after = [snapshots.take(limit: RECENT_RUNS), snapshots.jobs(limit: RECENT_RUNS)]
+        events = Events.between(runs, after.first) + JobEvents.between(jobs, after.last)
+        events.select { |event| !only || failure?(event) }.each { |event| print(event) }
         after
       end
 
+      def failure?(event) = Events.failure?(event) || JobEvents.failure?(event)
+
       def print(event)
         @context.io.stdout.puts JSON.generate({ schema: SCHEMA, **event })
         @context.io.stdout.flush

data/lib/fun_ci/agent/job_events.rb

@@ -0,0 +1,37 @@
+# frozen_string_literal: true
+
+require_relative "../jobs/state"
+
+module FunCi
+  module Agent
+    # What happened to a project's daily and weekly job runs between two
+    # looks at them (acceptance-tests.md, AT-13.23), oldest first: a job run
+    # started, and ended with its state and seconds. The first look is at nothing.
+    module JobEvents
+      # state: as Jobs::State says it; seconds: how long it ran, once it ended.
+      JobState = Data.define(:id, :job, :cadence, :sha, :branch, :state, :seconds)
+
+      # The states of a run that has not ended.
+      GOING = %w[running due].freeze
+
+      # before, after: { job run id => JobState }.
+      def self.between(before, after)
+        after.values.sort_by(&:id).flat_map { |run| of(run, before[run.id]) }
+      end
+
+      def self.failure?(event) = event[:event] == "job_finished" && Jobs::State.needs_you?(event[:state])
+
+      def self.of(run, was)
+        about = { job: run.job, cadence: run.cadence, commit: run.sha, branch: run.branch }
+        [*(was ? [] : [{ event: "job_started", **about }]), *ended(run, was, about)]
+      end
+
+      def self.ended(run, was, about)
+        return [] if was&.state == run.state || GOING.include?(run.state)
+
+        [{ event: "job_finished", **about, state: run.state, seconds: run.seconds }]
+      end
+      private_class_method :of, :ended
+    end
+  end
+end

data/lib/fun_ci/agent/job_json.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module FunCi
+  module Agent
+    # One job as `jobs --json` and `status --json` give it. Fields once
+    # published keep their names.
+    module JobJson
+      def self.document(report)
+        stage = report.stage
+        { name: report.name, cadence: report.cadence, state: report.state,
+          commit: stage && { sha: report.sha, branch: report.branch }, seconds: stage&.seconds,
+          started_at: report.started_at&.utc&.iso8601, due_at: report.due_at&.utc&.iso8601, due: report.due? }
+      end
+    end
+  end
+end

data/lib/fun_ci/agent/job_report.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+require "time"
+require_relative "report_stage"
+require_relative "verdict"
+require_relative "../jobs/standings"
+require_relative "../persistence/raw_outputs"
+
+module FunCi
+  module Agent
+    # A daily or weekly job as an agent is told it: where it stands
+    # (Jobs::Standing), and its latest run as a RunReport::Stage named after
+    # the job, nil while it never ran.
+    JobReport = Data.define(:standing, :stage)
+
+    # The jobs `status` names for a commit: those whose latest run tested it,
+    # and those failing on another commit, which an agent would not hear of otherwise.
+    CommitJobs = Data.define(:on_commit, :failing)
+
+    class CommitJobs
+      NONE = new(on_commit: [], failing: [])
+    end
+
+    class JobReport
+      VERDICTS = { "passed" => :passed, "failed" => :failed, "lost" => :failed, "over_budget" => :over_budget,
+                   "running" => :undecided }.freeze
+
+      def name = standing.name
+      def cadence = standing.cadence
+      def state = standing.state
+      def needs_you? = standing.needs_you?
+      def due_at = standing.due_at
+
+      # Whether the next commit starts it.
+      def due? = standing.due_now?
+
+      # The commit and branch its latest run tested, and when that started.
+      def sha = standing.run&.dig(:commit_hash)
+      def branch = standing.run&.dig(:branch)
+      def started_at = standing.run && Time.parse(standing.run[:started_at])
+
+      # As `status` would exit for a stage in this state; a job that never ran, or was cancelled, has none.
+      def verdict = VERDICTS.fetch(state, :unknown)
+    end
+
+    # The daily and weekly jobs of a project, as JobReports.
+    class JobReports
+      def initialize(db, project, clock)
+        @db = db
+        @project = project
+        @clock = clock
+      end
+
+      # By name.
+      def all = Jobs::Standings.new(@db, @project, now: @clock.now).all.map { |standing| report(standing) }
+
+      # The job named, or nil when the project has no such job.
+      def named(name) = all.find { |report| report.name == name }
+
+      # The jobs whose latest run tested `sha`, and those failing on another commit.
+      def of_commit(sha)
+        reports = all
+        CommitJobs.new(on_commit: reports.select { |report| report.sha == sha },
+                       failing: reports.select { |report| report.needs_you? && report.sha != sha })
+      end
+
+      # What the job's latest run kept of its raw output, or nil.
+      def raw_output(report) = report.stage && raw.read(report.stage.id)
+
+      private
+
+      def raw = Persistence::RawOutputs.for_jobs(@db.filename("main"))
+
+      def report(standing)
+        run = standing.run
+        JobReport.new(standing: standing, stage: run && stage(standing.name, run))
+      end
+
+      def stage(name, run)
+        RunReport::Stage.from_row(name,
+                                  run.merge(budget: run[:budget] || Jobs::Job::BUDGET, raw_bytes: raw.bytes(run[:id])))
+      end
+    end
+  end
+end

data/lib/fun_ci/agent/job_why.rb

@@ -0,0 +1,59 @@
+# frozen_string_literal: true
+
+require "json"
+require_relative "job_report"
+require_relative "job_why_text"
+require_relative "job_why_json"
+require_relative "exit_code"
+require_relative "options"
+
+module FunCi
+  module Agent
+    # `fun-ci why --job NAME [--json] [--raw]` (acceptance-tests.md, AT-13.21):
+    # everything kept about the job's latest run, exiting as `why` does for a
+    # stage in the same state; a job that never ran has no verdict.
+    class JobWhy
+      NO_JOBS = "this project has no daily or weekly jobs; put their scripts in .fun-ci/daily/ or .fun-ci/weekly/"
+
+      def initialize(context, jobs)
+        @context = context
+        @jobs = jobs
+      end
+
+      # Raises Options::Invalid for a name that is no job of the project.
+      def answer(options)
+        report = @jobs.named(options.job) || raise(Options::Invalid, no_job(options.job))
+        return never_ran(report, options.json) unless report.stage
+
+        options.raw ? raw(report) : say(report, options.json)
+        ExitCode::FOR.fetch(report.verdict)
+      end
+
+      private
+
+      def say(report, json) = json ? print_json(report) : print_lines(JobWhyText.lines(report))
+
+      def never_ran(report, json)
+        json ? print_json(report) : print_lines([JobWhyText.never_ran(report)])
+        ExitCode::FOR.fetch(:unknown)
+      end
+
+      def raw(report)
+        text = @jobs.raw_output(report)
+        return @context.io.stdout.write(text) if text
+
+        @context.io.stderr.puts "fun-ci why: no raw output is kept for job #{report.name}"
+      end
+
+      def no_job(name)
+        names = @jobs.all.map(&:name)
+        return NO_JOBS if names.empty?
+
+        "no job '#{name}' in this project: its jobs are #{names.join(", ")}"
+      end
+
+      def print_json(report) = @context.io.stdout.puts(JSON.generate(JobWhyJson.document(report)))
+      def print_lines(lines) = lines.each { |line| @context.io.stdout.puts line }
+    end
+  end
+end

data/lib/fun_ci/agent/job_why_json.rb

@@ -0,0 +1,31 @@
+# frozen_string_literal: true
+
+module FunCi
+  module Agent
+    # `fun-ci why --job NAME --json` (acceptance-tests.md, AT-13.21): the job,
+    # how its latest run ended, its evidence, or else why there is none, as
+    # `why --json` gives them for a stage. Fields once published keep their
+    # names; a change bumps SCHEMA.
+    module JobWhyJson
+      SCHEMA = 1
+
+      def self.document(report)
+        stage = report.stage
+        { schema: SCHEMA, name: report.name, cadence: report.cadence, state: report.state,
+          commit: stage && { sha: report.sha, branch: report.branch }, **ending(stage), **evidence(report),
+          raw_output: { kept: !stage&.raw_bytes.nil?, bytes: stage&.raw_bytes || 0 } }
+      end
+
+      # The evidence of a run that failed or ran over budget, or else why there is none.
+      def self.evidence(report)
+        failed = report.needs_you?
+        { evidence: failed ? report.stage.evidence.to_h : nil, no_evidence: failed ? nil : report.state }
+      end
+
+      def self.ending(stage)
+        { exit_status: stage&.exit_status, signal: stage&.signal, seconds: stage&.seconds, budget: stage&.budget }
+      end
+      private_class_method :ending, :evidence
+    end
+  end
+end

data/lib/fun_ci/agent/job_why_text.rb

@@ -0,0 +1,51 @@
+# frozen_string_literal: true
+
+require_relative "stage_summary"
+require_relative "evidence_text"
+require_relative "span"
+
+module FunCi
+  module Agent
+    # What `fun-ci why --job NAME` prints (acceptance-tests.md, AT-13.21): the
+    # job and the commit its latest run tested, how that run ended, then
+    # everything kept about it.
+    module JobWhyText
+      PASSED = "Nothing is kept about a job that passed."
+      RUNNING = "It is still running; fun-ci wait doesn't wait for jobs, so ask again later."
+      CANCELLED = "It was cancelled; it runs again on the next commit."
+
+      def self.lines(report)
+        summary = StageSummary.line(report.stage, seconds: Span.method(:words))
+        [header(report), summary, *body(report.stage), *raw(report), *only_tail(report)]
+      end
+
+      def self.header(report) = "fun-ci: job #{report.name} (#{report.cadence}) on #{report.branch} #{report.sha[0, 7]}"
+
+      def self.never_ran(report)
+        "fun-ci: job #{report.name} (#{report.cadence}) has not run yet; it runs on the next commit."
+      end
+
+      def self.body(stage)
+        return [PASSED] if stage.state == "passed"
+        return [RUNNING] if stage.state == "running"
+        return [CANCELLED] if stage.state == "cancelled"
+
+        EvidenceText.lines(stage.evidence)
+      end
+
+      def self.raw(report)
+        report.stage.raw_bytes ? ["", "The whole output: fun-ci why --job #{report.name} --raw"] : []
+      end
+
+      # So an agent that reads it knows it can write the extractor itself.
+      def self.only_tail(report)
+        excerpts = report.stage.evidence.excerpts
+        return [] if excerpts.empty? || excerpts.any? { |excerpt| excerpt[:extractor] != "output-tail" }
+
+        ["", "Only the output's last lines were kept; extractors under `evidence: jobs: #{report.name}:` " \
+             "in .fun-ci/config keep more."]
+      end
+      private_class_method :body, :raw, :only_tail
+    end
+  end
+end

data/lib/fun_ci/agent/jobs_command.rb

@@ -0,0 +1,43 @@
+# frozen_string_literal: true
+
+require "json"
+require_relative "command_support"
+require_relative "jobs_text"
+require_relative "job_why"
+require_relative "job_json"
+
+module FunCi
+  module Agent
+    # `fun-ci jobs [--json]` (acceptance-tests.md, AT-13.22): the project's
+    # daily and weekly jobs and how each stands.
+    class JobsCommand
+      include CommandSupport
+
+      NAME = "jobs"
+      SCHEMA = 1
+
+      def initialize(context)
+        @context = context
+      end
+
+      def run(args)
+        options = Options.parse(args, takes: %i[json])
+        say(job_reports.all, options.json)
+        0
+      rescue Options::Invalid => e
+        usage(e.message)
+      end
+
+      private
+
+      def say(reports, json)
+        return out.puts(JSON.generate(schema: SCHEMA, jobs: reports.map { |report| JobJson.document(report) })) if json
+        return out.puts("fun-ci: #{JobWhy::NO_JOBS}") if reports.empty?
+
+        JobsText.lines(reports, @context.clock.now).each { |line| out.puts line }
+      end
+
+      def out = @context.io.stdout
+    end
+  end
+end

data/lib/fun_ci/agent/jobs_text.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+require_relative "age"
+require_relative "due_in"
+require_relative "span"
+
+module FunCi
+  module Agent
+    # One line per daily or weekly job (acceptance-tests.md, AT-13.22), the
+    # names lined up: its cadence, its latest run's state in a word of four
+    # letters at most and how long it took, the branch and commit it tested
+    # and how long ago it started, as `runs` counts ages, and when the job is
+    # due again; then the `why --job`
+    # command of each job that failed or ran over budget.
+    module JobsText
+      WORDS = { "passed" => "ok", "failed" => "FAIL", "lost" => "LOST", "over_budget" => "OVER", "running" => "...",
+                "due" => "due", "cancelled" => "x" }.freeze
+
+      # now: the time the ages and due times count from.
+      def self.lines(reports, now)
+        width = reports.map { |report| report.name.size }.max
+        reports.map { |report| line(report, width, now) } + whys(reports)
+      end
+
+      def self.whys(reports)
+        reports.select(&:needs_you?).map { |report| "fun-ci why --job #{report.name}" }
+      end
+
+      def self.line(report, width, now)
+        head = "#{report.name.ljust(width)}  #{report.cadence.ljust(6)}  #{WORDS.fetch(report.state,
+                                                                                       report.state).ljust(4)}"
+        return "#{head}  never ran  runs on the next commit" unless report.stage
+
+        "#{head}  #{seconds(report.stage)}  #{tested(report, now)}  #{due(report, now)}"
+      end
+
+      # `wip/foo 9e0b1d4  1h ago`
+      def self.tested(report, now) = "#{report.branch} #{report.sha[0, 7]}  #{Age.words(now - report.started_at)}"
+
+      def self.seconds(stage) = (stage.seconds ? Span.words(stage.seconds) : "").rjust(6)
+
+      def self.due(report, now)
+        return "running" if report.state == "running"
+
+        report.due_at ? "due in #{DueIn.words(report.due_at - now)}" : "runs on the next commit"
+      end
+      private_class_method :whys, :line, :tested, :seconds, :due
+    end
+  end
+end

data/lib/fun_ci/agent/live_pipeline.rb

@@ -28,8 +28,13 @@ module FunCi
         Process.detach(pid).join
       end
 
-      # Records failed each slow suite whose process died (acceptance-tests.md, AT-8.3).
-      def watch(db) = Pipeline::RunCanceller.new.record_dead(db)
+      # Records failed each slow suite and each daily or weekly job whose
+      # process died (acceptance-tests.md, AT-8.3, AT-13.6).
+      def watch(db)
+        canceller = Pipeline::RunCanceller.new
+        canceller.record_dead(db)
+        canceller.record_dead_jobs(db)
+      end
     end
   end
 end

data/lib/fun_ci/agent/options.rb

@@ -12,12 +12,12 @@ module FunCi
       class Invalid < StandardError; end
 
       Parsed = Data.define(:rev, :stage, :need, :json, :within, :follow_branch, :limit, :branch, :follow, :only, :raw,
-                           :trunk)
+                           :trunk, :job)
       DEFAULTS = { need: "fast", json: false, within: nil, follow_branch: false, limit: 10, branch: nil,
-                   follow: false, only: nil, raw: false, trunk: false }.freeze
+                   follow: false, only: nil, raw: false, trunk: false, job: nil }.freeze
       SWITCHES = { need: ["--need LEVEL"], json: ["--json"], within: ["--within DURATION"], raw: ["--raw"],
                    follow_branch: ["--follow-branch"], limit: ["-n N", Integer], branch: ["--branch NAME"],
-                   follow: ["--follow"], only: ["--only KIND"], trunk: ["--trunk"] }.freeze
+                   follow: ["--follow"], only: ["--only KIND"], trunk: ["--trunk"], job: ["--job NAME"] }.freeze
       FILTERS = %w[failures].freeze
       MULTIPLIERS = { "" => 1, "s" => 1, "m" => 60 }.freeze
 

data/lib/fun_ci/agent/output.rb

@@ -9,6 +9,8 @@ require_relative "runs_text"
 require_relative "exit_code"
 require_relative "evidence_text"
 require_relative "trunk_why_json"
+require_relative "job_json"
+require_relative "job_report"
 
 module FunCi
   module Agent
@@ -24,8 +26,11 @@ module FunCi
         @trunk = trunk
       end
 
-      def report(report)
-        @json ? print_json(StatusJson.document(report)) : print_lines(StatusText.lines(report, trunk: @trunk))
+      # jobs: the commit's CommitJobs, when asked about.
+      def report(report, jobs: nil)
+        return print_json(with_jobs(StatusJson.document(report), jobs)) if @json
+
+        print_lines(StatusText.lines(report, trunk: @trunk, jobs: jobs || CommitJobs::NONE))
       end
 
       # Everything kept about the stage named, or says no stage decided the verdict.
@@ -87,6 +92,13 @@ module FunCi
 
       private
 
+      def with_jobs(document, jobs)
+        return document unless jobs
+
+        document.merge(jobs: jobs.on_commit.map { |job| JobJson.document(job) },
+                       failing_jobs: jobs.failing.map { |job| JobJson.document(job) })
+      end
+
       def print_json(document) = @stdout.puts(JSON.generate(document))
       def print_lines(lines) = lines.each { |line| @stdout.puts line }
     end