wallflower 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ba9a1d426696f5a091d18e4266c1ef2e5dfcb8166e110b3e143e3417fa055af7
-  data.tar.gz: '09abd80d4d1f08ec0a38bbe7ef19b5ae75601631da08edc3125cb6027a987097'
+  metadata.gz: 777814642256dec3caef9316ece5411d3ae340dd7073f3b56258b2a79804603c
+  data.tar.gz: 19545678d642b13d461f26408fc8f8844d3dd79fccaddf10c2b394e892561cef
 SHA512:
-  metadata.gz: dc04356f627dcee3e1181778f479a49762ef5b72444238df923dbb37e8a71f710d1327eeffc0020fbe84e447c08088baac6e0c3b2e8f8b995ee147b1ab26035b
-  data.tar.gz: 8e98021b01854edc0548b4f8f73eaff8759c97bd572bfadf951d685cc169fe3d7cfbfe71bb441619d68da7a7e03e55ac3a3311c39386a632f43b6a48d180875d
+  metadata.gz: 1fbef5b2d7cf042c5c4c6063726696520396fc4fba134ae5dfda7ce4456b5d25c904e26fe1efd1a718f7a555c58e36a163f77cc41008acaa21ec78108ee63778
+  data.tar.gz: 345abf578eb99a1043797466ad1c5c81d29f91086bd4ec1497c24d20bdb95b2551cfff6925409ea7ebe7afb1b09360b23fd4e57bb4d1a0691b4ebfbe1888b60b

data/README.md

@@ -4,7 +4,7 @@ Background task tracking for Rails apps: a person starts long-running work, foll
 
 ## Installation
 
-Add the gem and run its install generator, which copies a migration for its tasks table:
+Add the gem and run its install generator, which copies the migrations for its tables and a Stimulus controller to `app/javascript/controllers/wallflower/catch_up_controller.js`:
 
 ```ruby
 gem "wallflower"
@@ -16,6 +16,13 @@ bin/rails generate wallflower:install
 bin/rails db:migrate
 ```
 
+After updating the gem, run its update generator to copy any migrations added since it was installed and the latest copy of its Stimulus controller, then migrate:
+
+```bash
+bin/rails generate wallflower:update
+bin/rails db:migrate
+```
+
 Mount the engine at the path its pages should live under:
 
 ```ruby
@@ -63,6 +70,22 @@ A runner that produces a file attaches it as the task's result:
 task.attach_result(io: StringIO.new(csv), filename: "transactions.csv")
 ```
 
+A runner whose result is a page in the app, such as a report, links the task to it:
+
+```ruby
+task.link_result(console_report_path(report))
+```
+
+A finished task's page then offers an Open report button. The page itself stays the app's to draw.
+
+A runner that works through rows can refuse one it cannot take, with a label for the row and a reason:
+
+```ruby
+task.refuse(label: "Row 4", reason: "Amount is missing")
+```
+
+Call `task.advance` for every row, refused or not. `advance` saves at most once a second, so calling it for every row of a large task does not slow it down, and the final count is saved when the task finishes. A finished task's page shows how many rows were changed (rows done less rows refused) and how many were refused, and lists each refused row with its reason.
+
 A finished task's page then offers a Download button. The file is sent through Wallflower's own route, which gives it only to the person who started the task and answers anyone else with not found.
 
 ## Starting a task
@@ -73,8 +96,48 @@ task = Wallflower.start(kind: :export_transactions, person: current_user, accoun
 
 The task is returned queued and its job is enqueued. When the job runs, the task is marked running, its runner is called, and when the runner returns the task is marked finished with the time it finished.
 
+## When a task finishes
+
+Set `on_finish` to run code once with the task when it finishes, such as sending the person a notification through the app's own notification system:
+
+```ruby
+Wallflower.configure do |config|
+  config.on_finish = ->(task) { TaskFinishedNotifier.with(task: task).deliver(task.person) }
+end
+```
+
+With no `on_finish` set, tasks still run to the end.
+
+## When a task fails
+
+A runner that raises marks its task failed. The task keeps the error's message, its page shows it beside how far the task got, the error is reported through `Rails.error`, and `on_finish` runs once with the failed task.
+
+A task still running with no update for longer than `stall_after`, one hour by default, is marked failed by `Wallflower::SweepJob`, with a reason saying it stalled. Schedule the sweep the way the app schedules any recurring job, for example in Solid Queue's `config/recurring.yml`:
+
+```yaml
+production:
+  wallflower_sweep:
+    class: Wallflower::SweepJob
+    schedule: every 10 minutes
+```
+
+```ruby
+Wallflower.configure do |config|
+  config.stall_after = 30.minutes
+  config.keep_for = 90.days # unset by default, which keeps every task forever
+end
+```
+
+With `keep_for` set, the same sweep deletes each finished or failed task that ended longer ago than that, with its file and its refused rows. Queued and running tasks stay, however old. With it unset, nothing is ever deleted.
+
+## The task list
+
+The engine's root lists the tasks the signed-in person started in the current account, newest first. Each row shows the kind's title, the status as a coloured badge, the progress and when it was started, links to the task's page, and updates while the task runs.
+
 ## The task page
 
 A task's page lives at `tasks/:id` under the engine's mount path. The person who started the task sees its kind's title, its status and, once the runner has set a total, a progress bar of done against total. The page listens for changes to its task over Turbo Streams, so its status and progress update without a reload, in a browser and in the Hotwire Native iOS app alike. Anyone else, and the same person viewing from another account, gets a not-found response.
 
+The task page and the task list reload in place when they are shown again after being hidden, such as when the iOS app returns from the background, so they show changes broadcast while they could not receive them. A page that stays visible makes no extra request. The copied controller is registered by the host's own Stimulus setup, as `wallflower--catch-up`.
+
 Wallflower's pages are drawn with keystone_ui, and live updates need Action Cable and turbo-rails, which the host's layout already loads for Turbo Streams.

data/app/controllers/wallflower/tasks_controller.rb

@@ -3,6 +3,7 @@
 module Wallflower
   class TasksController < ApplicationController
     def index
+      @tasks = visible_tasks.order(created_at: :desc)
     end
 
     def show

data/app/javascript/wallflower/catch_up_controller.js

@@ -0,0 +1,20 @@
+import { Controller } from "@hotwired/stimulus"
+
+export default class extends Controller {
+  connect() {
+    this.page = this.element.ownerDocument
+    this.catchUp = this.catchUp.bind(this)
+    this.page.addEventListener("visibilitychange", this.catchUp)
+  }
+
+  disconnect() {
+    this.page.removeEventListener("visibilitychange", this.catchUp)
+  }
+
+  catchUp() {
+    if (this.page.visibilityState !== "visible") return
+
+    const view = this.page.defaultView
+    view.Turbo.visit(view.location.href, { action: "replace" })
+  }
+}

data/app/jobs/wallflower/run_job.rb

@@ -3,9 +3,19 @@
 module Wallflower
   class RunJob < ActiveJob::Base
     def perform(task)
+      run(task)
+      Wallflower.configuration.on_finish&.call(task)
+    end
+
+    private
+
+    def run(task)
       task.update!(status: "running")
       Wallflower.kind(task.kind).runner.constantize.new.call(task)
       task.update!(status: "finished", finished_at: Time.current)
+    rescue StandardError => error
+      task.update!(status: "failed", error_message: error.message, finished_at: Time.current)
+      Rails.error.report(error, handled: true, context: { wallflower_task_id: task.id })
     end
   end
 end

data/app/jobs/wallflower/sweep_job.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Wallflower
+  class SweepJob < ActiveJob::Base
+    def perform
+      fail_stalled
+      delete_old
+    end
+
+    private
+
+    def delete_old
+      keep_for = Wallflower.configuration.keep_for
+      return unless keep_for
+
+      Task.where(status: %w[finished failed]).where(finished_at: ...keep_for.ago).find_each(&:destroy!)
+    end
+
+    def fail_stalled
+      Task.where(status: "running").where(updated_at: ...Wallflower.configuration.stall_after.ago).find_each do |task|
+        task.update!(status: "failed", error_message: "Stalled: no progress for #{Wallflower.configuration.stall_after.inspect}", finished_at: Time.current)
+        Wallflower.configuration.on_finish&.call(task)
+      end
+    end
+  end
+end

data/app/models/wallflower/refusal.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+module Wallflower
+  class Refusal < ApplicationRecord
+    self.table_name = "wallflower_refusals"
+
+    belongs_to :task
+  end
+end

data/app/models/wallflower/task.rb

@@ -8,11 +8,18 @@ module Wallflower
     belongs_to :account, polymorphic: true, optional: true
 
     has_one_attached :result_file
+    has_many :refusals, dependent: :destroy
 
     after_update_commit do
       broadcast_replace_to self, target: ActionView::RecordIdentifier.dom_id(self, :live), partial: "wallflower/tasks/live", locals: { task: self }
     end
 
+    STATUS_VARIANTS = { "finished" => :success, "failed" => :danger, "running" => :info }.freeze
+
+    def status_variant
+      STATUS_VARIANTS.fetch(status, :neutral)
+    end
+
     def kind_title
       Wallflower.kind(kind).title
     end
@@ -21,12 +28,30 @@ module Wallflower
       result_file.attach(io: io, filename: filename)
     end
 
+    def changed_count
+      done - refusals.size
+    end
+
+    def link_result(url)
+      update!(result_url: url)
+    end
+
+    def refuse(label:, reason:)
+      refusals.create!(label: label, reason: reason)
+    end
+
     def set_total(total)
       update!(total: total)
     end
 
+    PROGRESS_SAVE_INTERVAL = 1.second
+
     def advance(by = 1)
-      increment(:done, by).save!
+      increment(:done, by)
+      return if @progress_saved_at && Time.current - @progress_saved_at < PROGRESS_SAVE_INTERVAL
+
+      save!
+      @progress_saved_at = Time.current
     end
   end
 end

data/app/views/wallflower/tasks/_live.html.erb

@@ -1,5 +1,8 @@
 <div id="<%= dom_id(task, :live) %>">
-  <p id="<%= dom_id(task, :status) %>"><%= task.status.humanize %></p>
+  <p id="<%= dom_id(task, :status) %>"><%= render Keystone::Ui::BadgeComponent.new(label: task.status.humanize, variant: task.status_variant) %></p>
+  <% if task.status == "failed" %>
+    <p id="<%= dom_id(task, :error) %>"><%= task.error_message %></p>
+  <% end %>
   <div id="<%= dom_id(task, :progress) %>">
     <% if task.total %>
       <%= render Keystone::Ui::ProgressComponent.new(value: task.done, max: task.total, label: "#{task.done} of #{task.total}") %>

data/app/views/wallflower/tasks/index.html.erb

@@ -1 +1,12 @@
-<h1>Tasks</h1>
+<%= tag.div(data: { controller: "wallflower--catch-up" }) %>
+<%= ui_page(max_width: :md) do %>
+  <h1>Tasks</h1>
+  <% @tasks.each do |task| %>
+    <div data-task-row>
+      <%= turbo_stream_from task %>
+      <%= link_to task.kind_title, task_path(task) %>
+      <%= render "wallflower/tasks/live", task: task %>
+      <p>Started <%= task.created_at.strftime("%b %-d, %Y") %></p>
+    </div>
+  <% end %>
+<% end %>

data/app/views/wallflower/tasks/show.html.erb

@@ -1,7 +1,19 @@
 <%= turbo_stream_from @task %>
+<%= tag.div(data: { controller: "wallflower--catch-up" }) %>
 <%= ui_page(max_width: :md) do %>
   <h1><%= @task.kind_title %></h1>
   <%= render "wallflower/tasks/live", task: @task %>
+  <% if @task.status == "finished" %>
+    <p id="<%= dom_id(@task, :outcome) %>"><%= @task.changed_count %> changed, <%= @task.refusals.size %> refused</p>
+    <% if @task.refusals.any? %>
+      <div id="<%= dom_id(@task, :refusals) %>">
+        <%= ui_data_table(items: @task.refusals.order(:id), columns: [ { label: "Row" }, { reason: "Reason" } ]) %>
+      </div>
+    <% end %>
+  <% end %>
+  <% if @task.status == "finished" && @task.result_url.present? %>
+    <%= ui_button(label: "Open report", href: @task.result_url, variant: :secondary) %>
+  <% end %>
   <% if @task.status == "finished" && @task.result_file.attached? %>
     <%= ui_button(label: "Download", href: download_task_path(@task), variant: :secondary) %>
   <% end %>

data/lib/generators/wallflower/install/install_generator.rb

@@ -10,10 +10,18 @@ module Wallflower
 
       source_root File.expand_path("templates", __dir__)
 
-      desc "Installs Wallflower: copies its migration."
+      desc "Installs Wallflower: copies its migrations and its Stimulus controller."
 
       def copy_migration
         migration_template "create_wallflower_tasks.rb.erb", "db/migrate/create_wallflower_tasks.rb"
+        migration_template "create_wallflower_refusals.rb.erb", "db/migrate/create_wallflower_refusals.rb"
+        migration_template "add_result_url_to_wallflower_tasks.rb.erb", "db/migrate/add_result_url_to_wallflower_tasks.rb"
+        migration_template "add_error_message_to_wallflower_tasks.rb.erb", "db/migrate/add_error_message_to_wallflower_tasks.rb"
+      end
+
+      def copy_catch_up_controller
+        source = File.expand_path("../../../../app/javascript/wallflower/catch_up_controller.js", __dir__)
+        create_file "app/javascript/controllers/wallflower/catch_up_controller.js", File.read(source)
       end
     end
   end

data/lib/generators/wallflower/install/templates/add_error_message_to_wallflower_tasks.rb.erb

@@ -0,0 +1,5 @@
+class AddErrorMessageToWallflowerTasks < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    add_column :wallflower_tasks, :error_message, :text
+  end
+end

data/lib/generators/wallflower/install/templates/add_result_url_to_wallflower_tasks.rb.erb

@@ -0,0 +1,5 @@
+class AddResultUrlToWallflowerTasks < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    add_column :wallflower_tasks, :result_url, :string
+  end
+end

data/lib/generators/wallflower/install/templates/create_wallflower_refusals.rb.erb

@@ -0,0 +1,10 @@
+class CreateWallflowerRefusals < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    create_table :wallflower_refusals do |t|
+      t.references :task, null: false, foreign_key: { to_table: :wallflower_tasks }
+      t.string :label, null: false
+      t.string :reason, null: false
+      t.timestamps
+    end
+  end
+end

data/lib/generators/wallflower/update/update_generator.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module Wallflower
+  module Generators
+    class UpdateGenerator < Rails::Generators::Base
+      include ActiveRecord::Generators::Migration
+
+      source_root File.expand_path("../install/templates", __dir__)
+
+      desc "Updates Wallflower: copies the migrations and the Stimulus controller added since it was installed."
+
+      def copy_refusals_migration
+        migration_template "create_wallflower_refusals.rb.erb", "db/migrate/create_wallflower_refusals.rb"
+      end
+
+      def copy_result_url_migration
+        migration_template "add_result_url_to_wallflower_tasks.rb.erb", "db/migrate/add_result_url_to_wallflower_tasks.rb"
+      end
+
+      def copy_error_message_migration
+        migration_template "add_error_message_to_wallflower_tasks.rb.erb", "db/migrate/add_error_message_to_wallflower_tasks.rb"
+      end
+
+      def copy_catch_up_controller
+        source = File.expand_path("../../../../app/javascript/wallflower/catch_up_controller.js", __dir__)
+        create_file "app/javascript/controllers/wallflower/catch_up_controller.js", File.read(source)
+      end
+    end
+  end
+end

data/lib/wallflower/configuration.rb

@@ -2,12 +2,13 @@
 
 module Wallflower
   class Configuration
-    attr_accessor :authentication_method, :layout, :current_person_method, :current_account_method
+    attr_accessor :authentication_method, :layout, :current_person_method, :current_account_method, :on_finish, :stall_after, :keep_for
 
     def initialize
       @authentication_method = :authenticate_user!
       @layout = "application"
       @current_person_method = :current_user
+      @stall_after = 1.hour
     end
   end
 

data/lib/wallflower/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Wallflower
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: wallflower
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Tyler Schneider
@@ -92,15 +92,22 @@ files:
 - Rakefile
 - app/controllers/wallflower/application_controller.rb
 - app/controllers/wallflower/tasks_controller.rb
+- app/javascript/wallflower/catch_up_controller.js
 - app/jobs/wallflower/run_job.rb
+- app/jobs/wallflower/sweep_job.rb
 - app/models/wallflower/application_record.rb
+- app/models/wallflower/refusal.rb
 - app/models/wallflower/task.rb
 - app/views/wallflower/tasks/_live.html.erb
 - app/views/wallflower/tasks/index.html.erb
 - app/views/wallflower/tasks/show.html.erb
 - config/routes.rb
 - lib/generators/wallflower/install/install_generator.rb
+- lib/generators/wallflower/install/templates/add_error_message_to_wallflower_tasks.rb.erb
+- lib/generators/wallflower/install/templates/add_result_url_to_wallflower_tasks.rb.erb
+- lib/generators/wallflower/install/templates/create_wallflower_refusals.rb.erb
 - lib/generators/wallflower/install/templates/create_wallflower_tasks.rb.erb
+- lib/generators/wallflower/update/update_generator.rb
 - lib/wallflower.rb
 - lib/wallflower/configuration.rb
 - lib/wallflower/engine.rb